Hosting and Distribution of Linaro Builds

Registered by Zach Pfeffer on 2011-04-28

This topic will focus on building and distributing Linaro Android builds.

Topics:

1. Binary and source distribution
2. Source and binary overlays (proprietary components and blobs from TI, Samsung, etc)
3. License click-through
4. Build and distribution location or method (host images and/or support on-demand image builds)
5. Release tagging and naming
6. Benchmark distribution and hosting
7. Release notes
8. Easy communication of build differences
9. Cloud based incremental builds

The Android platform team is aiming for releases every month.

See https://android-build.linaro.org for our daily builds.

Possible Action Items from the discussion:

1. Mirror the binaries/source to snapshot and releases.linaro.
1.1 Create a "make this build a release" feature.
1.2 Create stable links for each build.
1.3 Modify the manifest to list the specific commit id of a Linaro hosted git that can be used to recreate the build.
1.4 Create tarball snapshots.

2. Check each build for legally acceptable code as part of the continuous integration loop.
2.1 Investigate the use of www.spdx.org from <email address hidden> to track licenses.
2.2 Add all legal meta info to the build.

3. Review the appropriateness of source overlays.
3.1 Create a way to facilitate source overlays pending review.

4. Create a way to facilitate binary overlays (copying binary blobs to media) from vendor's websites.

5. Implement click-through licenses for all proprietary components through license display or through a key based mechanism.

6. Tag and name each release.

7. Distribute benchmarks and benchmark results.

8. Generate release notes for each release.

9. Create a way to easily communicate source differences between builds.

10. Create a cloud-based incremental build mechanism that allows cloning and hacking on released builds - in the cloud.

11. Promote the build site.

-Zach

Session notes: http://summit.ubuntu.com/uds-o/meeting/linaro-android-o-build-distribution/

Blueprint information

Status:
Not started
Approver:
Zach Pfeffer
Priority:
Essential
Drafter:
Zach Pfeffer
Direction:
Needs approval
Assignee:
Zach Pfeffer
Definition:
New
Series goal:
Accepted for 2011q2
Implementation:
Unknown
Milestone target:
milestone icon connect-q2.11

Related branches

Sprints

Whiteboard

[zpfeffer]
Talk about hosting Android builds on snapshots.linaro.org.

[zpfeffer]
Some user stories (As a <role> I want <to complete some task> so that <the value it brings>)?

[jeremychang May 20, 2011]: As the user and validation tool, I want the android dailybuild image can be host on snapshots.linaro.org so that the url should be more consistent and no need to care about ssl issue.

Overview:

The overall goal of this blueprint is to improve hosting and distribution of Linaro Android builds. The general themes are:

1. Make the build system more flexible
2. Make old build recreation possible (for bug triage)
3. Tighten up software licensing
4. Increase documentation
5. Enable better and easier local building
6. Communicate benchmarks and test results.

Linaro has an opportunity to really shine here. Most community distributions simply throw a Wiki page together with build instructions. Linaro can provide first-class build support and hosting to enable people to use our builds effectively.

User Stories
-------------------

Supporting: https://wiki.linaro.org/Cycles/1111/TechnicalTopics/Platforms

TR - P1.1, Android LEB, Essential, Provide a Linaro optimised build of Android, Android
--------------------------------------------------------------------------------------------------------------------------------------

An an Android developer I would like an LEB

Implement click-through licenses for all proprietary components through license display or through a key based mechanism (long task - split out): TODO

As an Android developer I would like a fully enabled LEB on Panda

linaro-android-leb-panda

As an Android developer I would like a fully enabled LEB on Snowball

linaro-android-leb-snowball

As an Android release engineer I would like to easily mark a build for release so that users can easily find it and be able to refer to the release location in the future.

linaro-android-build-release

Create a 'make this build a release' feature for https://android-build.linaro.org: TODO
Create stable links for each build: TODO
Mirror releases to http://releases.linaro.org: TODO
Create a snapshot manifest that explicitly lists the commit ids of each head for local build duplication: TODO
Tag and record the tag for each build release: TODO
Distribute benchmark/test results with each build: TODO

As an Android developer I would like a place to download the latest archive and all previous archives.

linaro-android-build-mirror

Mirror daily builds to http://snapshots.linaro.org: TODO

As an Android developer I would like to be able to easily work with an LEB locally so I can easily contribute.

linaro-android-build-local

Create monthly releases that contain all the builds we create, all the tools used to create those builds, release notes about them and instructions for building locally: TODO
Create a script for people to sync and build Linaro Android builds locally: TODO
Automate local build test validation (long task - split out): TODO

linaro-android-build-local-generic

Write, test and publish steps to recreate the generic Android build: TODO
Write, and publish with each build, release notes for the generic Android build: TODO

linaro-android-build-local-pandroid

Write, test and publish steps to recreate the Linaro hosted Pandroid build: TODO
Write, and publish with each build, release notes for the Linaro hosted Pandroid build: TODO

linaro-android-build-local-panda-leb

Write, test and publish steps to recreate the Linaro hosted Panda-LEB build: TODO
Write, and publish with each build, release notes for the Linaro hosted Panda-LEB build: TODO

linaro-android-build-local-snowball-leb

Write, test and publish steps to recreate the Linaro hosted Snowball-LEB build: TODO
Write, and publish with each build, release notes for the Linaro hosted Snowball-LEB build: TODO

As a lawyer I would like to know what licenses all the components of a distribution are under so that I can ensure that Linaro's build are legal to distrubute.

Publish SPDX licensing info with builds: TODO

linaro-android-leb-legal-panda

Write or generate www.spdx.org compliant license spread sheets for Panda-LEB build (long task - split out): TODO

linaro-android-leg-legal-generic

Write or generate www.spdx.org compliant license spread sheets for Android generic build (long task - will split out): TODO

As a integration engineer I would like to be able to easily find the difference between two Android trees so that I can find implementations that can be consolidated.

linaro-android-build-tool-diff

Write a tool to generate and communicate differences between two Android trees (long task - split out): TODO

TR - P1.2, Android Toolchain packages, High, Provide Linaro Toolchain and Platform code ready for consumption by android product builders, Android
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

As an Android developer I would like to be able to integrate proprietary components into a Linaro build to enable features that I may have a license for but Linaro does not.

Document and/or implement tools to enable binary overlays (long task - split out): TODO
Document and/or implement tools to enable source overlays (long task - split out): TODO

TR - Need one, requirement fulfills: P1, Linaro for product builders and innovators, focus on making Linaro Evaluation Builds and developer images a quick starting point for product builders that want full member board enablement as well as Linaro improvements in one shot.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

As an Android engineer I would like to rebuild components in the cloud to work with and test builds remotely.

linaro-android-rebuild

Modify the build system to retain build products: TODO
Implement build system rebuild: TODO
Open cloud builds up for browsing and modification: TODO

TR - Need one, marketing related
---------------------------------------------------

linaro-android-promote

Start working our build system into IRC conversations on #android, #android-dev and the android-platfrom mailing lists: TODO
Post out benchmark results to various websites (slashdot, ars technica, toms hardware): TODO

TR - Need one about benchmarking LEBs and promoting Linaro. We need to define a TR here that says, support external build hosting to make benchmarking external builds easiy and promotes linaro's build infrastructure.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

As an Android developer I would like to use the Linaro build system to create non-linaro builds so that I can easily work with all types of builds.

linaro-android-external-build-support

Extend build system to skip tarball creation: TODO
Enable users to download the out directory directly: TODO
Mirror all components of a released build on a Linaro server: TODO

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.