Automated upgrade testing to 14.04

Registered by Martin Pitt on 2013-10-18

We will have a lot of different upgrade paths towards 14.04, e. g. 12.04 pure, then the various 12.04.x installations with the Q/R/S enablement stacks, and so on. Right now, an apt-get dist-upgrade from 12.04 LTS to saucy is broken in many ways (backported stack, lots of file conflicts, etc.).

Blueprint information

Steve Langasek
Martin Pitt
Needs approval
Canonical Platform QA Team
Series goal:
Accepted for trusty
Beta Available
Milestone target:
milestone icon ubuntu-14.01
Started by
Martin Pitt on 2014-03-07

Related branches



= Upgrade scenarios =
 * P -> T
 * P with Q/R/S HWE enablement stack enabled→ T
 * S -> T
 * Q -> S (as raring goes EOL earlier, we need to support a direct upgrade path)
all of them with -updates installed

 * ubuntu-desktop
 * ubuntu-standard
 * ubuntu-standard + dns-server, lamp-server, mail-server, openssh-server, postgresql-server, print-server, samba-server
 * ubuntu-standard + all of main
 * ubuntu-standard + all packages with a .desktop file from universe and main (base VM: about 80 GB)
 * ubuntu-standard + dns-server, lamp-server, mail-server, openssh-server, postgresql-server, print-server, samba-server on armhf (or possibly a subset)

= Backend =
 * kvm for ubuntu-desktop (required to verify systems boots after upgrade and right kernel is loaded, X starts with default config)
 * LXC for the rest

auto-upgrade-testing needs some updates:
  * currently uses vm-builder (obsolete)
  * servers in the lab use precise; might have some bugs; but we have a backport of 1.0, plus enablement kernel
  * Stephane added an LXC backend to a-u-t, needs some cleanup

 = post-upgrade tests =
  * check installed kernel/grub
  * check installed metapackages, and absence of leftovers from backported enablement stacks
  * dpkg conffile prompts
  * obsolete conffiles
  * state of all installed upstart jobs
  * desktop: expected processes: unity, indicators, panel

= what information to include in reports =
  * verbose apt problem resolver
  * full console log
  * list of installed packages
  * difference of packages compared to a pristine 14.04 installation
  * status of upstart jobs
  * tar of /etc, /var/log/ (includes apt-clone), /home/ubuntu/

= Results =


Work Items

Work items for ubuntu-14.01:
[bdmurray] see if it is possible to upgrade from Q to S (possibly requires an ubuntu-release-upgrader fix,but should be in the meta-release files only): DONE
[jibel] refresh auto-upgrade-testing to be able to run upgrades with LXC: DONE
[stgraber] provide a precise LXC which survives upgrade testing (current version in precise-backports): DONE

Work items for ubuntu-14.02:
[jibel] refresh auto-upgrade-testing to be able to build and run KVM upgrades: DONE
[jibel] Create test profiles for the scenarios mentioned above: DONE
[jibel] Create test profiles for the Precise + ubuntu-desktop + HWE stacks: DONE
[jibel] Publish tests in VMs and main profile to public instance: DONE
[jibel] coordinate with CI to deploy this into the lab: DONE
[pitti] add tests for upstart and process state: DONE
[pitti] fix conffile prompt detection: DONE
[mlankhorst] add HWE enablement transitional packages to trusty which move back to the standard kernel/ DONE

Work items for ubuntu-14.03:
[jibel] publish results to CI dashboard (bug 1289364): INPROGRESS