Identify automateable test cases and reorg appropriately

Registered by Victor Tuson Palau

Currently there are 200+ test cases between the 3 checkbox flavors and some are automated, some are manual. Some of the manual ones can be automated and should be. This is to go through each test case, item by item and determine which of the manual cases can or should be automated. Some of them need to be left manual due to them being user experience or requiring physical manipulation of the system (key tests, plugging/unplugging USB devices, monitors, etc) but some can definitely be automated and we need to find those.

I've added a second part and that's a reorg. I've noticed that there is an overlap in test cases to some degree between the three checkboxes. It would behoove us to probably do away with -compatibility and merge those cases directly into -certification, and remove any duplicated test cases. Then we could even consider paring down the tests inside Checkbox proper to only ones that we think an end user would be interested in, and put everything else in the new Uber-Checkbox-Cert to be used for Ubuntu Ready and Certified testing, and co-opting as necessary by other teams.

Finally we need to organize a list of all test cases and info on them (automated or manual, which checkbox they're found in, etc) and identify those cases that are deemed a Must Pass for certification purposes. This list could be subsets of the larger list based on system type (e.g. server specific tests, netbook specific tests).

Blueprint information

Kevin Krafthefer
Jeff Lane 
Needs approval
Series goal:
Accepted for archives
Milestone target:
milestone icon may2011
Started by
Victor Tuson Palau
Completed by
Victor Tuson Palau

Related branches



[cert-team] list all test cases that are currently run on cert runs (~250): done
[cert-team] identify all test cases from list that are valid for hw certifiation: done
[cert-team] separate cert test cases so they can run on their own: done
[cert-team] make existing automated wifi test run as part of the cert runs: TODO
[cert-team] make existing automated sound test run as part of the cert runs: TODO
[cert-team] inventory of other existing cert automated test that have not been integrated : TODO
[cert-team] review list of test in cert suite and score "value of automation": done
[cert-team] Identify top 25% test to automate : TODO
[cert-team] meet with right teams about how would be the best way to automate them: TODO
[cert-team] automate top 25% tests: TODO


Work Items

This blueprint contains Public information 
Everyone can see this information.