Adding test coverage to Ubuntu
Start building a comprehensive regression testing suite, this blueprint is a discussion to decide where to start adding test cases, so basically choose the most important missing tests and have a clear list of priorities.
Blueprint information
- Status:
- Complete
- Approver:
- Pete Graner
- Priority:
- High
- Drafter:
- Gema Gomez
- Direction:
- Approved
- Assignee:
- Canonical Platform QA Team
- Definition:
- Approved
- Series goal:
- Accepted for quantal
- Implementation:
-
Implemented
- Milestone target:
-
ubuntu-12.10
- Started by
- Gema Gomez
- Completed by
- Gema Gomez
Whiteboard
Next steps to improving coverage during Q:
- Migrate existing tests that are run by the QA team to UATH
[hggdh2, psivaa] * Kernel SRU tests (QRT, xfstests)
[hggdh2, psivaa] * ISO Server Testing (smoke testing)
[jibel] * Upgrade testing
[jibel] * Auto package testing
[nuclearbob] * QRT Testing
[mrazik, retoaded] * Autopilot
[nuclearbob] * Bootspeed
[gema] * Bootspeed testing graphical representation design
[nuclearbob, gema] * Implementing the new graphs
- Adding new tests to UATH:
[gema,mrazik, babyface] * Networking testing
[slangasek] * IPV6 testing
07/20/2012, gema: These test cases are more development or package level test cases than testing we can run at system level on the images. I think they are good to be run at development level but do not grant UTAH migration, since they are very specific and low level.
[hggdh2] * 12.10 kernel testing integration with 12.04 (https:/
[jibel] * LibreOffice tests
[jibel] * jdk tests
[hggdh2, psivaa] * Adding Kernel SRU testing on XEN
[hggdh2, psivaa] * Adding ecryptfs tests to Kernel SRU testing
[nuclearbob] * HDAudio
[josepht] * Power management (cking)
[pitti] * Automatic suspend/resume UI integration tests, VM based (covering indicator menus, screen savers, inhibition by video player, automatic timeout, and low-on-battery)
* hardware specific suspend/resume: cking has scripts which do this; apw also has a script for this available by default with checkbox: /usr/share/
[babyface] * Adding Chinese images to smoke testing
[babyface,jibel] * Add automated UI driven ubiquity test cases (migrate the tests from http://
[psivaa] * AMD64 and Mac iso testing
- Adding new HW/infrastructure:
[hggdh2] * Fully automating ARM testing (panda boards)
[retoaded, nuclearbob] * Deploying MAAS in the lab and start using it for provisioning machines
- Keeping track of results
[babyface] * Assign responsibilities to keep an eye on regressions
[gema] * Test case guidelines
[gema] * Analyse coverage as a whole and list holes in our coverage
How to add test jobs to jenkins
- Do not use matrix jobs because there is no plugin to count results in that case
Test Cases to be added:
Work Items
Work items:
[hggdh2] Move Kernel SRU tests QRT, xfstests to UTAH: POSTPONED
[psivaa] Help hggdh2 move kernel SRU tests to UTAH: POSTPONED
[psivaa] Move current smoke testing server to UTAH: DONE
[javier.collado] Move current smoke testing desktop / alternate to UTAH: POSTPONED
[jibel] Move Upgrade testing to UTAH: POSTPONED
[jibel] Move Auto package testing to UTAH: POSTPONED
[nuclearbob] Move QRT Testing to UTAH: POSTPONED
[mrazik] Adapt Autopilot tests to work with UTAH: DONE
[nuclearbob] Move bootspeed testing to UTAH: DONE
[joetalbott] Improve bootspeed testing graphics: POSTPONED
[mrazik] Networking testing, add new tests: POSTPONED
[gema] Assist mrazik in the networking tests addition (no bandwidth): POSTPONED
[stgraber] Send IPV6 testing to the QA team (implemented test, sent to QA): DONE
[gema] Add IPV6 to UTAH: POSTPONED
[hggdh2] 12.10 kernel testing integration with 12.04 (https:/
[psivaa] Add Kernel SRU testing on XEN: POSTPONED
[hggdh2] Adding ecryptfs tests to Kernel SRU testing: DONE
[nuclearbob] Add HDAudio tests: POSTPONED
[joetalbott] Add Power management tests from cking: POSTPONED
[pitti] Write screen saver integration tests (starts after configured idle timoeut, wake up by activity, inhibition API, totem inhibition) (tests written, but need rollout to UTAH in jenkins): POSTPONED
[pitti] Write suspend UI integration tests (indicator menus, suspend after configured idle time, suspend after low-on-battery) : POSTPONED
[victor.zhou] Add Chinese images to smoke testing, once tests have been moved to UTAH: POSTPONED
[victor.zhou] Add automated UI driven ubiquity test cases (migrate the tests from http://
[psivaa] Add AMD64 and Mac iso testing to the lab: POSTPONED
[hggdh2] Fully automating ARM Kernel testing (panda boards): POSTPONED
[larry-e-works] Deploying MAAS in the lab and make it stable for production: POSTPONED
[nuclearbob] Start using MAAS from UTAH to provision machines: POSTPONED
[victor.zhou] Keep an eye on regressions: DONE
[gema] Write Test case guidelines for UTAH users: DONE
[gema] Analyse coverage as a whole (our coverage is minimal, needs to be improved in all fronts): DONE
Dependency tree
* Blueprints in grey have been implemented.
