Testing in a Cadence

Registered by Nicholas Skaggs on 2012-10-08

Define and identify requirements for participation and reporting of test results on a regular basis during the cycle.

Blueprint information

Not started
Jono Bacon
Nicholas Skaggs
Series goal:
Accepted for raring
Milestone target:
milestone icon ubuntu-13.04

Related branches



Quick history of 'cadence testing'

    started in quantal cycle, intended as experiment to help 'assure quality;


Recap of quantal cycle results


Blueprints and sessions informing this one:
Issues faced last cycle:
visibility -- tools built around milestones, don't show much visibility on a cadence basis
schedule -- confusing, shared calendar
stability -- non-developer centric scheduling and testing; code was caught in an untestable state during planned testing
Review decisions and new schedule for Raring
-- https://wiki.ubuntu.com/RaringRingtail/ReleaseSchedule
Tool needs and discussion

    should be able to use qatracker as-is

    other tools / report needs?

Defining the cadence process

    testing every other week. Once every 2 weeks, for a week long period, manually run testcases and report results. Development will not stop, no freezes of any kind will occur. Images will continue to roll out once a day during the week, and testing will occur independantly throughout the course of the week.

    Should define as Sunday - Saturday?

    Request Friday - Thursday as that gives weekend at the start for in order that most people available are available at the start.

    Should publish calendar seperate to release schedule?

    testing schedule with intending testing?

    Summary of these results should feed into the Quality Report (http://summit.ubuntu.com/uds-r/meeting/21082/qa-r-weather-report/)

    What is needed during the testing week to communicate status or progress for everyone?


    Create a milestone cadence name, with all of the dates filled in for the week. Default to the history view so all the testing and results can be seen. Additionally, a report view will be generated for each cadence week.




single version during cadence week

    Should we target specific packages for testing inclusion as part of cadence weeks?

    Try to align with known landing points?

    Work with team leads as releases happen, testing after landing in archive is 'ok'

    Workflow can mirror iso workflow as above

How does the milestone affect cadence?

    Run milestone the same as before

    Make sure cadence


Work Items

Work items:
[nskaggs] Create seperate calendar, listing testing details as we are informed. Each cadence week points to wiki page, with specifics.: DONE
[nskaggs] Communicate changes to release schedule: DONE
[nskaggs] Confirm a weeks worth of images remain availible from cadence weeks for debugging: DONE
[nskaggs] ensure installer has testing focus before beta, to ensure it's in good shape for beta milestone (can't fix bugs past freeze): DONE

This blueprint contains Public information 
Everyone can see this information.