Testing in a Cadence
Define and identify requirements for participation and reporting of test results on a regular basis during the cycle.
Blueprint information
- Status:
- Not started
- Approver:
- Jono Bacon
- Priority:
- Essential
- Drafter:
- None
- Direction:
- Approved
- Assignee:
- Nicholas Skaggs
- Definition:
- Approved
- Series goal:
- Accepted for raring
- Implementation:
-
Unknown
- Milestone target:
-
ubuntu-13.04
- Started by
- Completed by
Whiteboard
Quick history of 'cadence testing'
started in quantal cycle, intended as experiment to help 'assure quality;
https:/
Recap of quantal cycle results
http://
Blueprints and sessions informing this one:
http://
http://
Issues faced last cycle:
visibility -- tools built around milestones, don't show much visibility on a cadence basis
schedule -- confusing, shared calendar
stability -- non-developer centric scheduling and testing; code was caught in an untestable state during planned testing
Discussion:
Review decisions and new schedule for Raring
-- https:/
Tool needs and discussion
should be able to use qatracker as-is
other tools / report needs?
Defining the cadence process
testing every other week. Once every 2 weeks, for a week long period, manually run testcases and report results. Development will not stop, no freezes of any kind will occur. Images will continue to roll out once a day during the week, and testing will occur independantly throughout the course of the week.
Should define as Sunday - Saturday?
Request Friday - Thursday as that gives weekend at the start for in order that most people available are available at the start.
Should publish calendar seperate to release schedule?
testing schedule with intending testing?
Summary of these results should feed into the Quality Report (http://
What is needed during the testing week to communicate status or progress for everyone?
ISOs:
Create a milestone cadence name, with all of the dates filled in for the week. Default to the history view so all the testing and results can be seen. Additionally, a report view will be generated for each cadence week.
Example:
http://
http://
single version during cadence week
Packages:
Should we target specific packages for testing inclusion as part of cadence weeks?
Try to align with known landing points?
Work with team leads as releases happen, testing after landing in archive is 'ok'
Workflow can mirror iso workflow as above
How does the milestone affect cadence?
Run milestone the same as before
Make sure cadence
Work Items
Work items:
[nskaggs] Create seperate calendar, listing testing details as we are informed. Each cadence week points to wiki page, with specifics.: DONE
[nskaggs] Communicate changes to release schedule: DONE
[nskaggs] Confirm a weeks worth of images remain availible from cadence weeks for debugging: DONE
[nskaggs] ensure installer has testing focus before beta, to ensure it's in good shape for beta milestone (can't fix bugs past freeze): DONE
