Initial approach to doing native toolchain builds in LAVA

Meta:
Headline: Initial research and prototyping for using lava-dispatcher for doing native toolchain build in LAVA were done.
Acceptance: Toolchain builds in local lava-dispatcher installation using qemu are possible, or blockers and missing features towards supporting this are identified.

[pfalcon 2012-10-23] Filed bug #1070309.
[pfalcon 2012-10-23] Filed https://code.launchpad.net/~pfalcon/lava-dispatcher/run-as-non-root/+merge/131051
[pfalcon 2012-10-24] Filed bug #1070838, bug #1070855, bug #1070881
[pfalcon 2012-10-30] Connect session "Linaro CI/LAVA for GCC: cbuild migration episode 1" : http://summit.linaro.org/lce12/meeting/21232/linaro-cilava-for-gcc-cbuild-migration-episode-1/
[dooferlad 2012-11-01] Post-session discussion with Toolchain WG: https://docs.google.com/a/linaro.org/document/d/1qUsoA2FJeUuT-Elt6xaIkrKdw0JTfXkAq6TQ5oAg3J4/edit (also mirror of session notes from above).
[pfalcon 2012-11-01] File bug https://bugs.launchpad.net/lava-dispatcher/+bug/1073899
[pfalcon 2012-11-08] File bug https://bugs.launchpad.net/lava-dispatcher/+bug/1076381
[pfalcon 2012-11-08] Ubuntu Natty+ can't build pristine gcc, not until http://gcc.gnu.org/ml/gcc-patches/2011-08/msg01649.html . Workarounds and background are described here: https://wiki.linaro.org/MichaelHope/Sandbox/MultiarchWorkarounds , https://wiki.linaro.org/MichaelHope/Sandbox/MultiarchNotes
[pfalcon 2012-11-08] Filed lp:1076450, lp:1076480 (lp:cbuild)
[pfalcon 2012-11-09] Filed lp:1076950, lp:1076957, lp:1076962
[pfalcon 2012-11-09] Michael started related blueprint: https://blueprints.launchpad.net/cbuild/+spec/reproduce-cbuild-from-scratch
[pfalcon 2012-11-14] Got Panda board configured and doing simple tests with lava-dispatcher
[pfalcon 2012-11-19] Got (plucked) gcc build on Panda, after facing few reliability and other issues like LAVA test partition size.
[pfalcon 2012-11-19] Got suggestion from LAVA folks to use USB flashdrives (attached to some boards in lab) as means to both speed up build and resolve space issues.
[pfalcon 2012-11-19] Michael Hope provided production build image:
=======
The
rootfs image is at:
 http://people.linaro.org/~michaelh/incoming/auto-build-precise-r1.tar.xz

The kernel etc that go in the FAT partition are checked in:
 http://bazaar.launchpad.net/~linaro-toolchain-dev/cbuild/hardware/files/head:/distro/precise/r1/files/
=======
[pfalcon 2012-11-20] CBuild/lava-dispatcher setup for local development: https://code.launchpad.net/~linaro-infrastructure/cbuild/lava-build-slave , README: http://bazaar.launchpad.net/~linaro-infrastructure/cbuild/lava-build-slave/view/head:/slaves/lava-build/README
[pfalcon 2012-11-22] Build against prod LAVA: http://validation.linaro.org/lava-server/scheduler/job/39575 . This doesn't have Michael's image intergration, nor uses USB drive, so had to be plucked to fit in 3G testrootfs partition on LAVA boards.
[pfalcon 2012-11-23] Having trouble with build image as supplied by Michael - it causes my Panda to hard lockup after some time with build started.
[pfalcon 2012-11-27] The job above was done against private LAVA stream, here's the one which is accessible: http://validation.linaro.org/lava-server/scheduler/job/40088 (again, done with stable Oneiric image).
[pfalcon 2012-11-27] Tried Michael's image in LAVA production, same issue, builds hang: https://validation.linaro.org/lava-server/scheduler/job/40107/log_file https://validation.linaro.org/lava-server/scheduler/job/40184/log_file https://validation.linaro.org/lava-server/scheduler/job/40187/log_file

Work items:
Decide whether to start with entire LAVA system or with lowest level component (lava-dispatcher): DONE
Install lava-dispatcher locally: DONE
Install LAVA in vagrant sandbox: DONE
Identify issues, file bugs for them: DONE
Run a gcc build with lava-dispatcher in QEMU: DONE
Run a gcc build with lava-dispatcher in pandaboard: DONE
Run a test gcc builds against production LAVA: DONE