Scalable installation testing
Currently, installation testing is coordinated by means of a wiki page containing several tables. This approach has a number of scalability problems, including contention on the wiki page, overhead of recording test results, and lack of awareness in the community.
In order to broaden testing participation, anyone who installs Ubuntu should be able to quickly and easily inform us about whether their installation succeeded or failed and which installation parameters they used. The QA team should be able to view reports of test coverage based on this data.
Blueprint information
- Status:
- Complete
- Approver:
- Matt Zimmerman
- Priority:
- Medium
- Drafter:
- Henrik Nilsen Omma
- Direction:
- Needs approval
- Assignee:
- Henrik Nilsen Omma
- Definition:
- Approved
- Series goal:
- Accepted for feisty
- Implementation:
-
Implemented
- Milestone target:
- None
- Started by
- Matt Zimmerman
- Completed by
- Henrik Nilsen Omma
Whiteboard
review: Please incorporate the whiteboard comments. Did you identify any risks WRT doing this? WHat if there's no Internet connectivity? -- smurf
In order to maximize participation, we should not tie the tests to specific test cases. Any successful installation with network connectivity should be able to give us a report. -mdz, 2006-11-08
pitti: reviewed, whiteboard comments still need to be addressed, back to drafting
sfllaw: Addressed whiteboard comments, back to Review.
pitti, 2006-11-30: talked with mdz about his idea; made spec comments inline, set back to discussion
2007-01-26 kamion: Deferring for now after discussion with Simon since the spec isn't complete and this has interesting dependencies on a remote database somewhere; but if this makes great strides forward soon I'd be willing to un-defer it
2007-01-31 mdz: during the sprint, Henrik began work on an alternative approach which is simpler to implement and achieves many of the same goals. Hijacking this spec to track the status of that instead.
2007-01-31 heno: I've pointed the LP spec URL at the spec I wrote. The howto is here: https:/
2007-02-20 heno: setting status to 'Pending Approval' though the implementation is ahead of the spec(s). Both have evolved over the Herd releases. Anyway, the 'Good progress' status is correct. I hope to trial a new system for Herd 5 and use it more widely for the Beta testing.
2007-02-22 mdz: Setting to Approved, this matches what we discussed in Oslo. We'll fine-tune based on experience with this trial.
2007-04-30: This worked quite well for the Feisty release in the end. We will extend and harden it for Gutsy: iso-test-tracker
