Export additional build info for use when serving builds

Registered by Georgy Redkozubov

Over the past few month, linaro infrastructure added options for restricted hosting of build artifacts coming out of linaro infrastructure build services.

Those features namely were: EULA-restriction as well as OPENID protection of builds.

The EULA restriction was implemented using a EULA.txt trigger that allows the builds to protect its build by asking for a EULA.
It was implemented by following BPs:
 + https://blueprints.launchpad.net/linaro-license-protection/+spec/eula-handling-improvements
 + https://blueprints.launchpad.net/linaro-license-protection/+spec/multiple-eula-handling-improvements

Follow up thoughts revealed that protected publishing comes with a wider set of requirements:

 1. the eula text should be defined in the file itself
 2. the theme for displaying the EULA should be selectable
 3. the mechanism should be extensible to later add more EULA options, such as collecting user email and name information before admitting a download
 4. openid protection shoud be configurable without IS interaction

To address these, this blueprint proposes introduction of a general BUILD-INFO.txt format. The format should be extensible and should come with clear format spec version that allows to change the format in case new requirements arrive.

As such, this blueprint will close out by delivering a BUILD-INFO.txt file format that allow to select our existing EULA protection, include the EULA text and allows to configure a theme. BUILD-INFO will also be used to "configure artifacts" as "openly downloadable"

Follow up blueprints will extend the format by adding support for including hosting theme for delivering member branding, as well as include the eula text and version in the BUILD-INFO.txt. Also, there are planned follow up efforts for "openid" and "contact data collection upon EULA acceptance". Those blueprints can be found here:
  + EULA text, version and branding: https://blueprints.launchpad.net/linaro-license-protection/+spec/eula-text-version-and-branding-handling
  + OpenID support: https://blueprints.launchpad.net/linaro-license-protection/+spec/license-protection-openid-support
  + Contact data collection: https://blueprints.launchpad.net/linaro-license-protection/+spec/contact-data-collection-for-eula-publishing

Further, this blueprints will track and ensure that the existing build and platform artifacts taht currently shiip a EULA.txt and OPEN-EULA.txt have been transitioned to the new format.

BUILD-INFO.txt format:

Format-Version: 0.1
  Version of the BUILD-INFO format.
Files-Pattern: *.img, *.tar.bz2
  Filename patterns are specified using a simplified shell glob syntax. Will be used to identify protected files.
Build-Name: landing-snowball
  Would be a unique over all the builds of the same type. For now used as a placeholder and will be ignored. To be added later by build services?
Theme: stericsson
  Theme name for selecting proper theming on download and license pages, vendor name.
License-Type: open/protected
 open - Open builds. No license page is displayed.
 protected - EULA protected builds. If 'OpenID-Launchpad-Teams' is defined then OpenID protection is used, otherwise simple Accept/Decline license page is displayed before accessing protected files.
OpenID-Launchpad-Teams: linaro
  LP team names members of are allowed to access protected files. No OpenID protection if absent.
Collect-User-Data: yes/no
  Default to 'no' if not present. If the field is set to 'yes' then Name and E-Mail (some other fields?) fields are asked to be filled. If it will be needed in the future it could be expanded with 'Collect-User-Data-Fields' field later or any other approach.
License-Text: EULA full text included in BUILD-INFO.txt

- If line begins with space character ' ' it is treated as a part of last found variable which turns as multiline variable and is appended to it.
- Variable names are case insensitive.

Blueprint information

Status:
Complete
Approver:
David Zinman
Priority:
High
Drafter:
Georgy Redkozubov
Direction:
Approved
Assignee:
Georgy Redkozubov
Definition:
Approved
Series goal:
Accepted for trunk
Implementation:
Implemented
Milestone target:
milestone icon 2012.07
Started by
Georgy Redkozubov
Completed by
Данило Шеган

Whiteboard

[gesha, May 21, 2012] Agreed on BUILD-INFO.txt format. Added to description.
[gesha, May 23, 2012] Added Files-Pattern field to BUILD-INFO.txt format.
[dzin, June 6, 2012] Move to 12.06, partially done.
[dzin, June 8, 2012] Where is this blocked?
[gesha, June 13, 2012] RT #489 is filed for deploying build-info support on snapshots.l.o
[gesha, June 13, 2012] RT #490 is filed for deploying build-info support on releases.l.o
[gesha, June 20, 2012] Waiting for android team adding support for BUILD-INFO.txt.
[gesha, June 20, 2012] README file in repository contains description of BUILD-INFO.txt format.
[dzin, June 21, 2012] Unfinished items should be split out for 12.07 cycle.
[danilo 2012-06-29] Move to the next cycle because code, even if completed, hasn't been deployed.
[danilo 2012-07-11] Django-refactoring code moved out with postponed items to https://blueprints.launchpad.net/linaro-license-protection/+spec/django-rewrite

Meta:
Headline: Build information is defined and exported for external use.
Acceptance: We have an extensible BUILD-INFO.txt format defined and the existing restricted publishing feature (EULA/OPENEULA) has been ported to it. Platform/Build services that publish restricted artifacts make use of the new mechanism and the old mechanism was superseded.
Roadmap id: CARD-150

(?)

Work Items

Work items for 2012.05:
Define format/content (structure, fields) of the BUILD-INFO.txt: DONE
Add support to for the new BUILD-INFO.txt format to the protection code running on the hosting server: DONE
Update Description to include the agreed spec format: DONE

Work items for 2012.06:
Add support for publisher to support BUILD-INFO.txt files as part of release artifacts: DONE
Roll out new protection code to production: DONE
Validate that old EULA and openid protection still works: DONE
[fboudra] Migrate existing ubuntu platform builds to use the BUILD-INFO.txt feature: DONE
Validate that ubuntu publishing directories are properly protected and make use of this feature, drop a check log in the whiteboard: DONE
[pfefferz] Migrate existing android platform builds to use the BUILD-INFO.txt feature: POSTPONED
Migrate other builds and releases to use the BUILD-INFO.txt feature: POSTPONED
Validate that android publishing directories are properly protected and make use of this feature, drop a check log in the whiteboard: POSTPONED
Validate that license protection works well on releases.linaro.org: POSTPONED
Work with release team to update releases.linaro.org artifacts to use this new feature: POSTPONED
Document the final specs and how publishing services should use this BUILD-INFO.txt on a wiki page; document the wiki page used in the whiteboard: POSTPONED
Announce the availability of this new format to mailing list: POSTPONED

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.