System Image Upgrade Testing

Registered by Jean-Baptiste Lallement

This blueprint captures tasks for system image upgrade testing for Trusty

Blueprint information

Status:
Started
Approver:
Julien Funk
Priority:
Essential
Drafter:
Jean-Baptiste Lallement
Direction:
Needs approval
Assignee:
None
Definition:
Approved
Series goal:
Accepted for trusty
Implementation:
Started
Milestone target:
milestone icon ubuntu-14.04
Started by
Jean-Baptiste Lallement

Related branches

Sprints

Whiteboard

System-image and ubuntu-download-manager have fairly extensive unit testsuite with pretty good code coverage.
When package builds it runs the full test suite and a subset of it during autopkgtest

But it never goes outside of localhost and there is no extensive integration tests. More particularly, there is no end to end test against the real server and the real UI that the mocks are not able to catch.

https://bugs.launchpad.net/bugs/1277589 is an example of bug critical bug that escaped current testing.

The purpose of this project is to implement the tests described in [1] and automate them in CI.

= References =
[1] https://wiki.ubuntu.com/Process/Merges/TestPlan/ubuntu-system-image
[2] https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0AhmCwOxnFq5LdGt0bDJlNmoyZ2ZCU3RIRDR0RzhxV3c&usp=drive_web#gid=0
[3] https://wiki.ubuntu.com/SoftwareUpdates#Phone
[4] https://wiki.ubuntu.com/DownloadService/
[5] http://bazaar.launchpad.net/~ubuntuone-hackers/ubuntu-download-manager/trunk/view/head:/docs/DownloadStateMachine.dia
[6] https://code.launchpad.net/~diegosarmentero/ubuntu-system-settings/click-updates/+merge/208567

(?)

Work Items

Work items:
[chris.gagnon] Implement proxy for system-settings to make it testable by Autopilot: DONE
[chris.gagnon] Implement upgrade test case 1 - "n-1 to n": INPROGRESS
[chris.gagnon] Implement upgrade test case 2 - "n-2 to n": INPROGRESS
[chris.gagnon] (with CI) deploy test cases 1 and 2 in CI: TODO

This blueprint contains Public information 
Everyone can see this information.