Test development in the Saucy cycle

Registered by Martin Pitt

Various tests that the upstream QA team will develop in the Saucy cycle

Note that on UDS 1305 we will also discuss the underlying test technologies development in https://blueprints.launchpad.net/ubuntu/+spec/community-s-testing-technologies

Blueprint information

Status:
Complete
Approver:
Gema Gomez
Priority:
Medium
Drafter:
Martin Pitt
Direction:
Approved
Assignee:
Canonical Platform QA Team
Definition:
Approved
Series goal:
Accepted for saucy
Implementation:
Implemented
Milestone target:
milestone icon ubuntu-13.06
Started by
Gema Gomez
Completed by
Gema Gomez

Whiteboard

== Network ==

Might be able to take inspiration/ideas from Chromium tests:
http://git.chromium.org/gitweb/?p=chromiumos/third_party/autotest.git;a=blob;f=server/site_wifitest.py;h=caaf9ac02c6a5b5d3aed3355bc0efabba0245060;hb=HEAD

== (GNOME) installed tests: ==
- in the first iteration they should just go into a -dev package (if existing), or a separate -tests package
- integrate very well into autopkgtest framework
- should be tagged similarly to autopkgtest'ified sources: "Testsuite: installed", so that a test runner can easily find and install them all
- the total amount and size of those will not be very big, so it's reasonable to keep them in the archive instead of building something like ddebs.ubuntu.com
- not covered by WIs, packaging those as they come in from upstream is a rather simple task

== Ubiquity UI testing ==
Script to set up a loop device backed MD device and do some adjustments to run ubiquity for UI testing without an actual hard disk: http://people.canonical.com/~pitti/scripts/test-ubiquity.sh.txt . This should be suitable for an autopkgtest.
Note, if you have lots and lots of RAM, you can also try "sudo modprobe scsi_debug dev_size_mb=6000".

(?)

Work Items

Work items for ubuntu-13.05:
[pitti] add network-manager tests for ethernet: DONE
[pitti] add network-manager tests for IPv6 PE for ethernet: DONE

Work items for ubuntu-13.06:
[pitti] write gvfs test for PtP camera: DONE
[jibel] Make a prototype for DKMS testing: DONE

Work items for ubuntu-13.07:
[pitti] write gvfs test for MTP music player (needs umockdev issue #16 fixed): DONE
[jibel] development of DKMS testing: DONE
[jibel] Start the run of DKMS tests automatically on new uploads: DONE
[jibel] DKMS - Mockup of dashboard pages : DONE
[jibel] DKMS - Weekly report: DONE
[pitti] figure out the best way to simulate a hard disk for ubiquity autopilot tests: DONE
[jibel] Automate run of Ubuntu Desktop Apps tests in the lab (https://jenkins.qa.ubuntu.com/job/autopilot-ubuntu-applications/): DONE

Work items for ubuntu-13.08:
[pitti] write shotwell test for PtP camera: DONE
[jibel] jhbuild notifications (RT #60258): POSTPONED
[jibel] (and pitti): jhbuild: review currently failing tests, identify limitations of our test bed: POSTPONED
[jibel] jhbuild: Queue new builds while builds are already running: POSTPONED
[jibel] jhbuild: Fix concurrent fetch of the moduleset that leads to a 0-byte file: POSTPONED

Work items for ubuntu-13.09:
[pitti] investigate how {dialer,messaging}-app can be tested robustly: DONE
[pitti] write test for Ubuntu phone data exchange (does not currently support anything, should learn MTP at some point): POSTPONED
[pitti] add network-manager test for multiple APs: POSTPONED
[pitti] add network-manager test for preferring 5 GHz if 2.4 and 5 are available: POSTPONED

Work items for ubuntu-13.10:
[jibel] notifications on DKMS test failures and recovery: POSTPONED
[pitti] write initial smoketests for phone app: DONE
[pitti] extend with-ofono-phonesim to make it possible to simulate SMS by sending GUI scripts: DONE
[pitti] write initial smoketests for messaging app: DONE
[pitti] write initial AP tests for power indicator functionality/bugs: POSTPONED
[rhuddie] performance monitoring tests for mir, try turn into tests with pass/fail: POSTPONED
[chris.gagnon] convert old performance from GTK to QML test runner: POSTPONED
[chris.gagnon] work with thomi to convince CI to turn on Mir for existing tests: DONE
[iahmad] create UI toolkit automed tests: POSTPONED
[vrruiz] create system settings tests: POSTPONED
[thomir-deactivatedaccount] identify and write AP tests for the 20% of our reported bugs that are eligible for automated testing: POSTPONED
[veebers] write AP tests for gaps in the dash: POSTPONED

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.