Ubuntu LEB and LAVA: Current status and future planning for proper image testing and validation

Registered by Ricardo Salveti

This session will present how the Ubuntu LEB images are currently tested with LAVA, and discuss about future planning/development to reach the goal of having the image testing mostly automated (and covering how manual test report should also be handled).

 * Daily images
 * Snapshots -> LAVA
 * State of the test cases
 * How to handle manual test reports

Blueprint information

Not started
Ricardo Salveti
Ricardo Salveti
Needs approval
Ricardo Salveti
Series goal:
Milestone target:

Related branches



Session Notes:

Ubuntu LEB and LAVA

 * Daily images
  - Currently they are "cron jobs" that always gets the latest rootfs and hwpacks and pushes the test runs
  - Problems:
    - Can easily retest the same thing over the time
    - Hard to track the test results per images
    - Missing builds, like respins per day
    - Test list, how it's properly maintained
    - Not necessarily all hwpacks are tested, not even sure if we're testing what we care about
  - Solution:
    - Remove the responsability of the test runs from the validation team and move to the dev platform
    - Jobs at jenkins to create the hwpack and the rootfs images with live-build (cross bootstrap)
    - Hook the images with LAVA, to submit the test requests and present a way for people to check the test results later on
    - The Dev Plat team should now maintain the list of test cases to be pushed at each image we have

 * Pre-Installed image:
   - Available as well for all flavors we care
   - Could be the target for LAVA when testing Ubuntu LEB images
   - A jenkins job would define what an image is, and how it should be created (from a hwpack and rootfs)
   - Needs development to support these kind of images
   - Pre-seed ubuntu images might also be an option
   - Would need to force to accept the license with the pre-hook script at lmc (accepting the license as debkeys)

 * State of the test cases
  - Lava-test is the way to go when integrating and creating new test cases
  - out of tree tests are possible (for private tests)
  - any important test for linaro should all go inside lava-test

 * Test result probe with rpc
  - View that returns a json with the test results
  - What kind of output: test results and link with log
  - Do basically what android is already doing

 * How to handle manual test reports
  - Prototype already available
  - Still useful and something we want to support

 * Easy way to submit test jobs per images
  - would be nice to have a way to request the latest version of an image/hwpack rather than having to specify
  - web UI - View with image lists and test cases

 * Boot Loader
  - Hard to support, no real solution atm
  - Hardware solution (dual sd card, usb sd writer, jtag)

 * Master images
  - Update as needed
  - Maybe new features at u-boot could request it
  - Or critical bugs at the kernel level


Work Items