Identify automateable test cases and reorg appropriately
Currently there are 200+ test cases between the 3 checkbox flavors and some are automated, some are manual. Some of the manual ones can be automated and should be. This is to go through each test case, item by item and determine which of the manual cases can or should be automated. Some of them need to be left manual due to them being user experience or requiring physical manipulation of the system (key tests, plugging/unplugging USB devices, monitors, etc) but some can definitely be automated and we need to find those.
I've added a second part and that's a reorg. I've noticed that there is an overlap in test cases to some degree between the three checkboxes. It would behoove us to probably do away with -compatibility and merge those cases directly into -certification, and remove any duplicated test cases. Then we could even consider paring down the tests inside Checkbox proper to only ones that we think an end user would be interested in, and put everything else in the new Uber-Checkbox-Cert to be used for Ubuntu Ready and Certified testing, and co-opting as necessary by other teams.
Finally we need to organize a list of all test cases and info on them (automated or manual, which checkbox they're found in, etc) and identify those cases that are deemed a Must Pass for certification purposes. This list could be subsets of the larger list based on system type (e.g. server specific tests, netbook specific tests).
Blueprint information
- Status:
- Complete
- Approver:
- Kevin Krafthefer
- Priority:
- High
- Drafter:
- Jeff Lane
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Superseded
- Series goal:
- Accepted for archives
- Implementation:
- Blocked
- Milestone target:
- may2011
- Started by
- Victor Tuson Palau
- Completed by
- Victor Tuson Palau
Related branches
Related bugs
Sprints
Whiteboard
[cert-team] list all test cases that are currently run on cert runs (~250): done
[cert-team] identify all test cases from list that are valid for hw certifiation: done
[cert-team] separate cert test cases so they can run on their own: done
[cert-team] make existing automated wifi test run as part of the cert runs: TODO
[cert-team] make existing automated sound test run as part of the cert runs: TODO
[cert-team] inventory of other existing cert automated test that have not been integrated : TODO
[cert-team] review list of test in cert suite and score "value of automation": done
[cert-team] Identify top 25% test to automate : TODO
[cert-team] meet with right teams about how would be the best way to automate them: TODO
[cert-team] automate top 25% tests: TODO