Discussing PS-related product processes

Registered by Didier Roche on 2012-10-23

New PS-to-distro process to land their code on a daily basis on ubuntu.

Blueprint information

Status:
Started
Approver:
Sebastien Bacher
Priority:
High
Drafter:
Didier Roche
Direction:
Needs approval
Assignee:
Didier Roche
Definition:
Approved
Series goal:
Accepted for raring
Implementation:
Started
Milestone target:
None
Started by
Sebastien Bacher on 2012-11-07

Related branches

Sprints

Whiteboard

as you probably know already, PS is our upstream for a lot of desktop
components nowadays (Unity, compiz, webapps, indicators, multi touch
stack...).
The past cycle has been a real ride in term of features, which spawn the
release team, translation team and documentation team with FFe/UIFe. We
need to discuss a way to ease the process in both ways with all involved
parts.

Seeing the importance of those components on our stack today, I think
for instance that having a standing FF/UIF exception as we have for
GNOME components in ubuntu will make sense. However, the counter-part
will be that PS will work on getting things landing only when they are
ready, to avoid further and further refinements (and additional
documentation changes) as we had in the past just to "match the date
gate". So this one can clearly be a win-win situation.

Also, I want to discuss about what can land in a SRU. Little (few pixel
move) change, not really impacting the documentation may want to be
considered. We currently have 2 examples we can discuss in the session:

- https://bugs.launchpad.net/unity/+bug/1043808 (Dash: Preview
activation doesn't have instant feedback). Design worked on a spinner to
partially address this one. This is a behavior change in some way, for a
transient state, however it can be completely acceptable in a SRU as it
will make the quantal experience better and don't change doc/add new
strings, and so on.

- Another one is the ribbon on the application lens for software-center
content. This one is giving (due to pixelized images with the magazines)
a lot of headaches to design and they would want to remove it. This
specific case is an UI change, but doesn't seem it would impact the
understanding of the lens.

We can base the UDS discussion on those examples to see how we can get
the process smoother and more reliable for everyone in the next cycle
and going on. :)

Also, we can present the new release process for those products.
-----

* Issues with the current process
* New PS release process
* FFe/UIFe
* What can be SRUed?

Current situation:
 * Big improvement already in quality

Problems:
 * lots of FFe and UIFe burdening various teams (doc/release/platform/design)
 * Disruptive changes come in large code changes across the stack, reducing the volume of changes could help a lot
*

Proposed process:

 Public read-only copy for this session
 - http://people.canonical.com/~alan/Branching_landing_process_for_R.png (png)
 - http://ubuntuone.com/5KYdl6hs0KJywTOER7ak2U (pdf)

 https://docs.google.com/a/canonical.com/drawings/d/19P8RHcxGTsZk704Euoo1g0AhaMFuWwDcTZGopOErJfo/edit (canonical only)

Clarification
In this context, autopkgtest refers to autopilot tests.

Suggestions:
 * ensure the fix includes a test during the review process, verify the previous failure against the build from the prior day and run the test against the fixed code and make sure the problem is gone this ensures both that the code is good and that the test is targetting the right code

What tests to run? We should work with QA to figure out how we can have confidence in the tests run in autopkgtest. Balancing reward for autopilot tests vs. what we can run in autopkgtest. We will start this discussion on the Improving Unity's Testability session tomorrow

Currently the autopilot tests are triggered by packages landing in the staging PPA, ie. after each commit merge to the trunk.

Remove the staging PPA from the upstream testing process and insert it into the daily builds process in the automated preparation step and then we'll use manual copies of the binaries to proposed.

On the changelog, put the upstream commiter name by bug fix.

(?)

Work Items

Work items:
[cjwatson] Check on disabling PPC builds in the unity-team staging PPA (already done in late September after bug 897999 was fixed): DONE
[mrazik] Work with didrocks to ensure autopkgtests (autopilot?) coverage: DONE
[didrocks] Work with cjwatson to decide which bot should be doing the copies: DONE
[didrocks] Build the automation stack for daily upload to ubuntu, having some per stack capability for autopackaging, listing bugs, blocking in case of packaging change: DONE
[didrocks] Write tests for this automated stack: DONE
[jibel] Deploy the scripts in the QA Lab (57364): DONE
[jibel] Implement the workflow in Jenkins: DONE
[jibel] Create scripts to automate deployement of a new stack in jenkins: DONE
[jibel] Write script to manually force the publication to the archive: DONE
[didrocks] Have autopilot plugged in with this process and reporting the correct information: DONE
[mrazik] Help didrocks with the autopilot bits and getting the right outcome: DONE
[sil2100] Fix the flackyness of the autopilot unity tests: DONE
[didrocks] Figure out the wrong compiz session settings making a lot of unity tests failing: DONE
[didrocks] Create some dashboard to report the status of the last daily run: DONE
[mrazik] Create a measurement system to report progress on tests, numbers of uploads, what was making the test failings, per period of time: POSTPONED
[mrazik] Kill the staging unity-team ppa: DONE
[didrocks] Create the new stating ppa under ~ubuntu-unity (non virtual ppa): DONE
[didrocks] Update, document and announce the new release process diagram for everyone to have a clear view on it: DONE
[ken-vandine] Merge inline webapps and webcreds packages, run autoreconf and switch to dh9: DONE
[robru] Help Ken on inline webapps and webcreds packages, run autoreconf and switch to dh9: DONE
[mterry] Merge inline the unity stack packages, run autoreconf and switch to dh9: DONE
[mathieu-tl] Merge inline the indicators stack packages, run autoreconf and switch to dh9: DONE
[didrocks] Help the team with the coherence of the packaging: DONE
[mrazik] Switch all jenkins bits to inline packaging: DONE
[mrazik] Ensure we build on arm* in jenkins for every upstream commit: POSTPONED