CI with the Linux Linaro packages - Jenkins and LAVA Integration

Registered by David Zinman on 2011-10-03

As part of the Continuous Integration effort in place with the Linux Linaro packages, integrate with jenkins scripts and LAVA

Blueprint information

Status:
Complete
Approver:
David Zinman
Priority:
Essential
Drafter:
John Rigby
Direction:
Approved
Assignee:
John Rigby
Definition:
Approved
Series goal:
Accepted for 11.11
Implementation:
Implemented
Milestone target:
milestone icon 11.10
Started by
Ricardo Salveti
Completed by
David Zinman

Related branches

Sprints

Whiteboard

[dzin, 3 Oct, 2011]: The work items here were unfinished for release 11.09 and have been broken out of https://blueprints.launchpad.net/linaro-ubuntu/+spec/packaged-linux-linaro-ci with the headline, acceptance criteria and discussions copied over.
[jcrigby, 3 Oct, 2011]: Removing discussion cruft. It is still in original bp if needed. Redoing WI's based on better knowledge of implementation details:
No need for separate sauce trees as the upstream Ubuntu tree is the source of the sauce that the latest Linaro kernel is the filtered sauce. No need for sauce filtering as it rarely if ever causes problems. We always bump abi so no need for auto abi bumping.
Remaining issues to determine are publishing merge results via git and also publishing of resulting source package.
[dzin, 5 Oct, 2011] Added work items from duplicated BP
[jcrigby, 5 Oct, 2011] Merge/refactor duplicate/redundant WI's
[rsalveti, 5 Oct, 2011] When publishing the merged tree, push it initially at github or gitorious, by creating an account and a ssh key just for the kernel CI usage (we can just create a mirror at git.linaro.org and a cron to sync both trees later).
[jcrigby, 10 Oct, 2011] No need for source package generation so tweeking WI's to reflect that.
[jcrigby, 11 Oct, 2011] With the introduction of linux-linaro-3.1 we need 3.1 build jobs. We will need both 3.0 and 3.1 for a while yet.
[jcrigby, 12 Oct 2011] Build's work but are currently turned off until deepti tells me where to put the SSH keys for publishing, resolved now.
[jcrigby, 13 Oct 2011] Assuming 3.1 will not be the release this cycle I am marking the 3.1 WI postponed.
[fboudra, 14 Oct 2011] I've been informed that the Kernel WG has planned to deliver 3.1.
[jcrigby, 14 Oct 2011] Changing 3.1 jobs back todo. Also LP import of github trees is taking a long time so the PPA builds are blocked for the moment.
[jcrigby, 17 Oct 2011] The LAVA submission script is still having trouble, marking this blocked until we figure out what the problem is. The LP imports of the github results are still running each day but are not complete. The 3.1 jobs have been created and appear to be running well.
[fboudra, 18 Oct 2011] Could we report a bug and link it to the blueprint so we can track the issue?
[jcrigby, 19 Oct 2011] plars pointed out the problem., the job I was testing with was using "beagle" as board type which needed to be "beaglexm". Lava submissions are working now. Only remaining issue is the git-bzr tree syncing on LP.
[dzin, Nov 9, 2011]: broke out blocked item into bug: 888109 and marked implemented.

Headline: The Linux Linaro packages are now generated and validated in a continuous integration loop.
Acceptance: Build bot in place, build at every new commit or at least daily. Results from the merge published at a staging git tree. Packages generated, with hwpack in place and tested at Lava.

(?)

Work Items

Work items:
Branch lci-build-tools as basis of new packaged kernel scripts: DONE
Create script for building linux-image package deb's from packaged kernel git repo: DONE
Create jobs for building linux-image deb binaries and hwpacks for three supported flavours and shared flavour: DONE
Add triggers to linux-image jobs to start hwpack lava testing jobs as existing Jenkins jobs do: DONE
Consult with Deepti on place to archive hwpacks: DONE
Add auto merge of upstream linux-linaro to build script: DONE
Implement git publishing of tagged merged ci git trees to github: DONE
Implement weekly armel native PPA building of kernel linux-image .debs from published ci git tree: BLOCKED
Submit scripts to Deepti for review and merging: DONE
Create jobs for CI with new 3.1 linux-linaro kernel git: DONE
Use ssh key stored in private place on jenkins instead of key embedded in git publish script: DONE

This blueprint contains Public information 
Everyone can see this information.