Further improvements to the Checkbox release process

Registered by Brendan Donegan on 2013-04-04

We have made good progress in the last six months on delivering well tested and stable releases of Checkbox to our PPAs. The aim of this blueprint is to continue down that road and reduce the amount of time this process takes so that we can do even more frequent releases while remaining rock solid.

Right now the release process for Checkbox involves quite a lot of manual tasks:

1.) Preparing the release for testing by creating a release branch for both checkbox and checkbox-certification and building it in a testing PPA
2.) Performing a full run of the client-cert.whitelist (using checkbox-certification-client), sending results to the certification site and analysing the results for any unusual failures.
3.) Checking results of automated checkbox-certification-server testing on the certification site
4.) Performing functional tests described in https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0ApQ2JshzVOLydHhDZU8zQ3JZNjgtZ0FyTGVnYS1rS0E&usp=sharing
5.) Conducting release meeting with representatives from different parties using Checkbox
6.) Preparing release notes consisting of changelog and whitelist diffs
7.) Copying binaries from testing PPA to the public PPA
8.) Preparing the Ubuntu Checkbox release

All in all these tasks account for at least one full person day. In this cycle we want to attempt to automate and otherwise improve those tasks that satisfy the equation of bringing maximum benefit for minimum effort.

These have been identified as:

1.) Creating files containing release notes (mostly) automatically
2.) Automatically preparing the upload of Checkbox to the development version of Ubuntu. Note that for this there may be some precedent in other teams so we should try and reuse their work. See http://blog.didrocks.fr/post/Unity%3A-release-early%2C-release-often%E2%80%A6-release-daily
3.) Improving the already existing automated testing of checkbox-certification-server to send a notification on failure, where 'failure' is defined as any test failing or the submission to the certification site failing.

We may also like to take time to discuss a strategy for versioning Checkbox 0.16 such that we do not run into the problems encountered in the last development cycle regarding the frozen Ubuntu package superseding the 'stable' version.

We should also look into a way to manage packaging in such a way that we don't end up with frequent conflicts in debian/changelog caused when there is a delay between proposal of a merge and the merge itself.

Blueprint information

Status:
Not started
Approver:
Ara Pulido
Priority:
Undefined
Drafter:
Brendan Donegan
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

ramble ramble ramble:

1) decouple project version from debian version
2) stop being a native package
3) release notes are by definition manual, changelog is by definition automatic
4) release all the time, micro changes, back out broken stuff, improve ci loop to have confidence about jobs and scripts, release daily (or many times a day), eventually

(?)

Work Items

Work items:
As the Checkbox release manager, I want to establish an improved versioning scheme that doesn't fall down during Feature Freeze, so that I don't have to do unusual tricks in changelogs to make things work how they should - M: TODO
As a developer of Checkbox, I want to come up with a way of managing debian/changelog to reduce conflicts in Tarmac, so that I don't waste unnecessary amounts of time rebasing - M: TODO
As the Checkbox release manager, I want to create files containing release notes automatically, so that I don't need to spend time on this mundane task every release - M: TODO
As the Checkbox release manager, I want to prepare uploads of Checkbox to Ubuntu automatically, so that I don't need to spend a lot of time doing this every release - M: TODO
A user of checkbox-certification-server, I want to have enhanced automated testing of checkbox-certification-server to include checking that all tests passed, so that I can be even more sure there will be no regressions affecting my work - L: TODO

This blueprint contains Public information 
Everyone can see this information.