Support Kernel Maintainers through One Time Git Tree Submission
To make the daily job of kernel upstream maintainers kernel CI easier and to give them a tool to catch regressions before the landing/damage is done, we want to support one time submissions for testing custom git trees and defconfigs as part of the kernel CI service. This blueprint covers the first step including one pilot user that is required to use the jenkins "build now" feature to submit his jobs.
Big Vision:
The end goal is to allow kernel maintainers with a kernel CI account to use a command line tool to submit a job that validates a git tree and a defconfig for a target board through the lianro CI kernel service. The service will then go off and produce a fresh build and send the result to the LAVA lab for validation (if a matching board is available in the lab). The kernel maintainer can then use the command line tool to query and monitor status of his job and gets the build and test results in one shot.
First Step (this blueprint):
As a first step this blueprint creates a pilot jenkins job that accepts four parameters manually in the jenkins UI:
1. a kernel git url
2. a defconfig
3. optionally a (board type, hwpack type) tuple
4. commit id/ref
This blueprints implements the one time tree submission through a _single_ jenkins job. Every kernel maintainer that wants to do one time submission would have his own build job that he can use to do all his one-off-builds.
This blueprint proposes to minimize clone time and bandwidth by setting the jenkins up with a static git URL from upstream master branch and then fetching just the additional refs from the remote the user wants to test. This will ensure that the smart workspace caching of jenkins is reused and clone time significantly speed-up.
NOTE: This blueprint has risks that the approach above hits technical problems due to the way jenkins operates; hence a quick feasibility check/discussion should be done well before the blueprint is started. If not feasible, set this blueprint to blocked and discuss with PM/TL.
Blueprint information
- Status:
- Complete
- Approver:
- David Zinman
- Priority:
- High
- Drafter:
- Deepti B. Kalakeri
- Direction:
- Approved
- Assignee:
- Deepti B. Kalakeri
- Definition:
- Approved
- Series goal:
- Accepted for engineering
- Implementation:
-
Implemented
- Milestone target:
-
2011.10
- Started by
- Deepti B. Kalakeri
- Completed by
- Deepti B. Kalakeri
Related branches
Related bugs
Sprints
Whiteboard
[asac, Sep 1, 2011]: add comments and status updates like this; update early, update often :)
[asac, Oct 5, 2011]: please check if this is ready to code. and drop a comment if thats the case (or not); this is high priority to get pilot done before connect.
[asac, Oct 10, 2011]: ping. I would very much approve this for 11.10 - let me know if there are any concerns doing that.
[dzin, Oct 11, 2011]: Re-target to 11.10
[deepti, Oct 24 2011]: Added a new script to take care of one time job submission instead. Sent the script for review
Headline: Pull requests can now be validated once across many defconfigs and boards.
Acceptance: A Pilot user can now submit one-off builds through the jenkins UI by setting git url, defconfig, board and hwpack type as variables when submitting a "new build"; the git tree/head can be varied from one build to another and the LAVA scheduler shows that the expected kernel is booted and the results are submitted to the a dedicated stream bundle for all one-time-jobs of the user
Notes:
The Infrastructure CI Kernel Service started a pilot with the kernel WG to try out latest a prototype for supporting one-off kernel builds that get validated in the lab. This tool will make life for ARM kernel maintainers easier by giving them a to quickly validate pull request in one shot across many defconfigs and boards. The Infrastructure Team anticipates that stabilzation of this feature will continue for another couple of month before this one-off service can be offered reliably to a wider audience.
Work Items
Work items:
Verify that feasibility check was done; if not do it, comment or get somebody else to do and comment OR don't start: DONE
Setup Jenkins Job for pilot user "deepak": DONE
Create a pilot branch off our production jenkins scripts in bzr and make jenkins job use that: DONE
Send the script written for one time job submission for review: DONE
Configure GIT_REMOTE_URL and GIT_REMOTE_REF parameters in jenkins job script with empty string default: DONE
Add code that will fetch the remote and reset --hard the working tree to the right head commit _iff_ GIT_REMOTE_URL and GIT_REMOTE_REF are _both_ set: DONE
Submit multiple one off tests with different trees for panda through the same jenkins job; validate that kernel version in the lab job matches the version expected from the GIT_REMOTE one off tree: DONE
Create a local http git repository on the jenkins master on the ec2 instance to improve kernel download time: DONE
