CI with the Linux Linaro packages

Registered by Ricardo Salveti

Have Continuous Integration in place with the Linux Linaro packages. Make sure the packages for 11.09 release gets created by using the CI loop.

Blueprint information

Ricardo Salveti
John Rigby
John Rigby
Series goal:
Milestone target:
Completed by
John Rigby

Related branches



[asac, 28 Sep, 2011]: needs progress by end of day, otherwise this is not getting done by 11.09 and goes through post-mortem.
[rsalveti, 29 Sep, 2011]: Not fully integrated, but the build scripts were all done already, now we just need to integrate it with jenkins and lava.
[dzin, 3 Oct, 2011]: For 11.09 the build scripts have been done, but the integration with jenkins and LAVA will be broken out into a new blueprint targeted for 11.10.
[dzin 20 Sep, 2012] remove milestone info.

Headline: The Linux Linaro packages are now generated and validated at a continuous integration loop.
Acceptance: Build bot in place, build at every new commit or at least daily. Results from the merge published at a staging git tree. Packages generated, with hwpack in place and tested at Lava.

Draft and discussions:
 - Work implemented at a build bot (jenkins? need to check)
 - Separate sauce + packaging + config into a different tree
   - Packaing is easily separated because it all lives in debian and debian.linaro
   - Sauce is a different issue since it touches files all over the kernel tree so:
     - What does separate sauce mean?
       - A separate head that has sauce applied? This could just be a clone (or modified clone) of a Ubuntu tree.
       - A directory containing sauce extracted from Ubuntu with git format patch?
    - Extracting Ubuntu sauce can be done by a human periodically and put in a separate head. Then the CI script will just merge or replay (via rebase with appropriate arguments)
 - Scripts that merge both trees and produce a tree to be built
 - Should this tree be pushed somewhere?
    - The staging tree should have the state to remember what inputs went into it (packaging commit id and upstream commit id).
    - This state could be in tag, in topmost commit message or in debian.linaro/changelog.
 - Check:
   - ABI
     - 3 steps: no check, manual check and bump, automated bump
   - Config changes
 - Sauce:
   - Steps:
     - No sauce for the beginning
     - Entire sauce
     - Drop the sauce we don't need (x86 only)
 - Deepti is helping with the Jenkins part
 - Cross build for every Kernel in CI
 - Per <period> an upload of the resulted branch
   - Source package
   - Build natively
 - How to report failures?
   - Config conflicts
   - ABI bump
   - Rebase conflict
   - Build failure
   - Deploy failure?
 - HWpack is produced
 - Tested at LAVA


Work Items

Work items:
Sync with Deepti and check what is needed from our side to make the script available at Jenkins, and understand the Jenkins requirements: TODO
[rsalveti] Sync with Deepti to understand how the hwpack is created from the build, and what is needed to fire up new tests at Lava to validate the kernel: DONE
Create script to automatically rebase and build existing packaged kernel onto updated upstream kernel: INPROGRESS
Create per flavor packaging/config trees from the existing omap-only, mx51-only... trees/tags without sauce: TODO
Create script to merge current head or specific commit or tag of linux-linaro with a particular flavour packaging and build: TODO
Create a git tree at that should be updated with the merge results: TODO
Change script to tag and push the resulted changes to the staging tree (independent of success or failure of build): TODO
Add config seed files for configs for satisfying ubuntu enforce: DONE
Add config seed files for configs for standard Linaro features: DONE
Add config seed files for configs for per flavour/soc: DONE
Create kernel build scripts to create configs from seed files: DONE
Make the build script to compare the config result to previous build and report changes but do not fail the build: TODO
Add ability to apply sauce (from directory of patches or git tree and commit id's of beginning and ending of sauce): TODO
Improve the build script to detect abi change, bump abi and restart build: TODO
Create script for assisting in creation of sauce tree or dir: TODO
Add filtering to sauce creation script: TODO
Each friday, create source pkg from latest successful build and upload to ppa: TODO

This blueprint contains Public information 
Everyone can see this information.