Linaro Android Infrastructure

Source Overlays for Restricted (EULA) binaries

Registered by Alexander Sack

Add support to android build to pull tarballs with proprietary components from a remote host using a statically configured
ssh URL prefix (e.g. ssh://snapshots.linaro.org/binaries/) and a path variable given in the build config (e.g. android/origen/20111210/source-overlay.tar.bz2)

The downloaded tarball is then unpacked inside the repo sync area which ensures that the binaries get picked up and installed to the proper location in the android images during the "normal" build.

To avoid accidentially leaking binaries in unprotected builds, this operation should only pull from a specific location on snapshots.linaro.org and match the board name in the source URL with the board name of the build...

See the EULA spec to see that this makes sense: https://blueprints.launchpad.net/linaro-infrastructure-misc/+spec/click-through-licensing-for-images

This ensures that for the first iteration its _hard_ to get the EULA protected binaries shipped in a build that wouldn't be protected on snapshots.linaro.org

Blueprint information

Status:
Complete
Approver:
Данило Шеган
Priority:
Essential
Drafter:
None
Direction:
Approved
Assignee:
James Tunnicliffe
Definition:
Approved
Series goal:
Accepted for trunk
Implementation:
Implemented
Milestone target:
milestone icon 11.12
Started by
Paul Sokolovsky
Completed by
Paul Sokolovsky

Related branches

Sprints

Whiteboard

Meta:
Roadmap id: ANDROID2011-ACCEL-OVERLAYS
Headline: Android Build system can now use source overlays from snapshots.linaro.org/binaries/ to supplement the build with proprietary binaries that need user click through.
Acceptance: Android Build downloads android/origen/20111210/source-overlay.tar.bz2 from snapshots.linaro.org/binaries (or as fallback from a cloned directory structure hosted manually on the jenkins control node directly), unpacks the tarball after the repo sync so that the bits in there are included during the build. If all falls apart the config option is not honoured and a hard coded tarball gets copied for all origen/snowball jobs to the slave and is unpacked there.

Notes:
[dooferlad 2011-12-13] Proof of concept downloading through a license click through done. Can easily convert into a script to download files like wget or incorporate into another python script: lp:~dooferlad/+junk/download_through_license
[pfalcon 2011-12-14] Discussing approach with Danilo and James, we decided to go with "push from master to slave" approach.
[dooferlad 2011-12-15] Mirroring is set up for hourly. Currently set up using wget with no clobber because we don't get a modification time header from the web server. If a file changes, we won't pick up the changes. If a new file is created, we will fetch it. If a file exists, we assume it hasn't changed. Script to do this is currently stored at lp:~dooferlad/+junk/mirror_snapshots_android. Will do something more permanent, and store it somewhere more permanent, once this is up and running.

(?)

Work Items

Work items:
[gesha] Set-up snapshots.linaro.org to always let android-build.linaro.org through: DONE
[dooferlad] Fetch the tarball from snapshots.linaro.org by avoiding the click through: DONE
[dooferlad] Set up snapshots.linaro.org => android-build.l.o build prerequisites mirroring: DONE
[pfalcon] Install Copy-to-Slave plugin on android-build: DONE
[pfalcon] Study/Test Copy-to-Slave plugin: DONE
[pfalcon] Deal with Copy-to-Slave security issue: DONE
[pfalcon] Add SOURCE_OVERLAY support: DONE
Make sure that we disable artifact archiving for jobs using vendor overlays: DONE

Dependency tree

* Blueprints in grey have been implemented.

Nearby

This blueprint contains Public information 
Everyone can see this information.