Test development in the Saucy cycle

Registered by Martin Pitt on 2013-05-03

Various tests that the upstream QA team will develop in the Saucy cycle

Note that on UDS 1305 we will also discuss the underlying test technologies development in https://blueprints.launchpad.net/ubuntu/+spec/community-s-testing-technologies

Blueprint information

Status:
Complete
Approver:
Gema Gomez
Priority:
Medium
Drafter:
Martin Pitt
Direction:
Approved
Assignee:
Canonical Platform QA Team
Definition:
Approved
Series goal:
Accepted for saucy
Implementation:
Implemented
Milestone target:
milestone icon ubuntu-13.06
Started by
Gema Gomez on 2013-11-11
Completed by
Gema Gomez on 2013-11-11

Whiteboard

== Network ==

Might be able to take inspiration/ideas from Chromium tests:
http://git.chromium.org/gitweb/?p=chromiumos/third_party/autotest.git;a=blob;f=server/site_wifitest.py;h=caaf9ac02c6a5b5d3aed3355bc0efabba0245060;hb=HEAD

== (GNOME) installed tests: ==
- in the first iteration they should just go into a -dev package (if existing), or a separate -tests package
- integrate very well into autopkgtest framework
- should be tagged similarly to autopkgtest'ified sources: "Testsuite: installed", so that a test runner can easily find and install them all
- the total amount and size of those will not be very big, so it's reasonable to keep them in the archive instead of building something like ddebs.ubuntu.com
- not covered by WIs, packaging those as they come in from upstream is a rather simple task

== Ubiquity UI testing ==
Script to set up a loop device backed MD device and do some adjustments to run ubiquity for UI testing without an actual hard disk: http://people.canonical.com/~pitti/scripts/test-ubiquity.sh.txt . This should be suitable for an autopkgtest.
Note, if you have lots and lots of RAM, you can also try "sudo modprobe scsi_debug dev_size_mb=6000".

(?)

Work Items

Work items for ubuntu-13.05:
[pitti] add network-manager tests for ethernet: DONE
[pitti] add network-manager tests for IPv6 PE for ethernet: DONE

Work items for ubuntu-13.06:
[pitti] write gvfs test for PtP camera: DONE
[jibel] Make a prototype for DKMS testing: DONE

Work items for ubuntu-13.07:
[pitti] write gvfs test for MTP music player (needs umockdev issue #16 fixed): DONE
[jibel] development of DKMS testing: DONE
[jibel] Start the run of DKMS tests automatically on new uploads: DONE
[jibel] DKMS - Mockup of dashboard pages : DONE
[jibel] DKMS - Weekly report: DONE
[pitti] figure out the best way to simulate a hard disk for ubiquity autopilot tests: DONE
[jibel] Automate run of Ubuntu Desktop Apps tests in the lab (https://jenkins.qa.ubuntu.com/job/autopilot-ubuntu-applications/): DONE

Work items for ubuntu-13.08:
[pitti] write shotwell test for PtP camera: DONE
[jibel] jhbuild notifications (RT #60258): POSTPONED
[jibel] (and pitti): jhbuild: review currently failing tests, identify limitations of our test bed: POSTPONED
[jibel] jhbuild: Queue new builds while builds are already running: POSTPONED
[jibel] jhbuild: Fix concurrent fetch of the moduleset that leads to a 0-byte file: POSTPONED

Work items for ubuntu-13.09:
[pitti] investigate how {dialer,messaging}-app can be tested robustly: DONE
[pitti] write test for Ubuntu phone data exchange (does not currently support anything, should learn MTP at some point): POSTPONED
[pitti] add network-manager test for multiple APs: POSTPONED
[pitti] add network-manager test for preferring 5 GHz if 2.4 and 5 are available: POSTPONED

Work items for ubuntu-13.10:
[jibel] notifications on DKMS test failures and recovery: POSTPONED
[pitti] write initial smoketests for phone app: DONE
[pitti] extend with-ofono-phonesim to make it possible to simulate SMS by sending GUI scripts: DONE
[pitti] write initial smoketests for messaging app: DONE
[pitti] write initial AP tests for power indicator functionality/bugs: POSTPONED
[rhuddie] performance monitoring tests for mir, try turn into tests with pass/fail: POSTPONED
[chris.gagnon] convert old performance from GTK to QML test runner: POSTPONED
[chris.gagnon] work with thomi to convince CI to turn on Mir for existing tests: DONE
[iahmad] create UI toolkit automed tests: POSTPONED
[vrruiz] create system settings tests: POSTPONED
[thomir] identify and write AP tests for the 20% of our reported bugs that are eligible for automated testing: POSTPONED
[veebers] write AP tests for gaps in the dash: POSTPONED

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.