Delivering sustained and scheduled testing

Registered by Nicholas Skaggs on 2012-04-27

Discussing testing needs, surrounding the different forms of testing, iso, sru, and application testing. How to schedule and build a better process, incorporating new tools and proposed repository.

Blueprint information

Status:
Complete
Approver:
Jono Bacon
Priority:
Undefined
Drafter:
None
Direction:
Needs approval
Assignee:
Nicholas Skaggs
Definition:
Obsolete
Series goal:
None
Implementation:
Unknown
Milestone target:
None
Completed by
Nicholas Skaggs on 2012-05-11

Related branches

Sprints

Whiteboard

Ubuntu does have testers capable of understanding packages and applications, and aware of the problems that affected those in previous releases and their versions. We should focus on testing packages as their updated versions are released.

This is what testers that run the Development Release everyday as their main OS already do, but in a non-focused way. They report the bugs they found, no matter in which package. This is good, not covered by ISO-Testing, and should continue and be stimulated. But QA could help with a simple message letting people know what packages were updated / should be (stress) tested everyday.

Going a little further, it would be amazing if developers had some way f indicating what should be tested / stress tested in a package. Maybe that could be added to Launchpad somehow when the developer uploads the package. If everyday DR testers knew which packages to focus today and, ideally, what to test in each updated package, it could enhance our productivity a lot.

To make all of this work, we need a repo that contains all packages/releases for "Today -1". Testers need to use this repo, so they will all be on the same page, same packages, same versions. We need a webpage telling what are the latest changes (changelogs) to packages *in this repo*. If we use normal repos, the constant flow of updates makes testing and keeping testers platform homogeneous impossible.

We should also invest on making LiveTesting (our versions of "Testing Days" - let's not use the same name as other distro's on that too, please) work, focus it on critical ones, like Ubiquity, Jockey, etc. See https://wiki.ubuntu.com/LiveTesting/Specs for our current draft on this project. Meetings are ongoing and it will start to happen soon. We'll need support from QA to make things easier.

We must offer some interface through which developers can request testing for their packages. In doing so, it's reasonable to imagine the developer will commit to following up with the testing, provide info on what should be stress tested, and use the testing-results to improve the target package. A special tag, identifying scheduled-test-results, should be used on LP. [Effenberg0x0 at Ubuntu.com]

(?)

Work Items