11.10 Certification Testing

Registered by Marc Legris on 2011-04-14

Continuing with the weekly build testing will naturally transition from 11.04 to 11.10. This will have to take into account the systems that have been added since 11.04, removal of outdated systems, and the continual improvement of the entire test suite. The main goal going forward will be identifying bugs as early as possible, and comprehensive testing to ensure the final image of 11.10 will meet the high standards of our users.

This process should help eliminate most if not all of the false positive tests we have seen in the past, the reducing the total time of a certification cycle, and SOP guidelines for each step of a cycle. In addition, the passover from enablement to certification will have to be smoothed and SOP agreed upon with both parties. Different components (GPU/WIFI/etc) inside of the same model hardware will have to be addressed and resolved to remove customer confusion over certified systems. Developer access to certification centers to resolve certification failures also needs to be established.

Another major component of certification is pre-testing with the Alpha, Beta and RC images. In order to make sure the final release candidate passes on the most machines, this testing will have to be co-ordinated and rigid timelines enforced. Once the gold candidate image has been selected, each week the daily build will be tested, along with SRUs, to make sure regressions are not introduced into any certified system. The selection of systems will have to be managed, as well as a subset when full system testing is not practicable.

Blueprint information

Status:
Complete
Approver:
Ara Pulido
Priority:
Essential
Drafter:
Marc Legris
Direction:
Approved
Assignee:
None
Definition:
Approved
Series goal:
None
Implementation:
Implemented
Milestone target:
milestone icon november2011
Started by
Victor Tuson Palau
Completed by
Ara Pulido

Related branches

Sprints

Whiteboard

== Definition of Done ==
* Full certification run has been completed one month before Kernel Freeze (starting when Alpha3 image is published) on all 11.10 applicable systems
* Full certification run has been completed at KernelFreeze for systems failing on Alpha3
* Full certification run has been completed with the final image on all 11.10 applicable systems
* False negative results are being investigated and solved on a weekly basis
* Component tracking for systems with variable internal components has been implemented
* Developer access protocol to certification systems has been established
* Firm testing timelines for each part of certification has been set
* Bootchart metrics results are delivered every week at least once
* Weekly testing results are delivered every week, starting on Alpha 1
* Bugs blocking certification are triaged and escalated properly within 2 days from reporting

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.