certify-planning

Align OEM Services and Certification testing

Registered by Ara Pulido

The QA team from OEM Services (OEMS QA) uses their own fork of Checkbox (checkbox-oem-qa) to test the systems that we will then certify as pre-installed only. This fork of checkbox is based on checkbox-compatibility, mainly because this is the flavour that contains the needed interfaces for the OEMS QA team (mainly with C3), but using mainly their own tests (they maintain over 1500 manual cases for testing applications and proprietary tests they build for customers).

Our certification whitelist, and many of the needed tests for certification are in checkbox-certification, and not checkbox-compatibility, so these never get run by the OEMS QA, making it difficult for us to issue pre-installed only certificates.

Also, the OEMS QA, as they didn't know the existance of checkbox-certification, created some tests which goal is very similar to some of the ones we have, creating some redundancy that could have much better handeled.

The goal of this blueprint is to discuss the various reasons of this discrepancies and trying to find a way and processes to share each other team's knowledge and make the hardware testing among Canonical consistent.

Blueprint information

Status:
Started
Approver:
Victor Tuson Palau
Priority:
Medium
Drafter:
Ara Pulido
Direction:
Needs approval
Assignee:
None
Definition:
Approved
Series goal:
None
Implementation:
Started
Milestone target:
milestone icon november2011
Started by
Victor Tuson Palau

Related branches

Sprints

Whiteboard

== Definition of Done ==

* checkbox-compatibility is removed from the list of "checkboxes" - done
* There are no duplicated tests in both checkbox/checkbox-compatibility and checkbox-oem-qa
* OEMS QA, as part of their testing, always run the tests included in the certification.whitelist in a way that does not affect their workflow
* The Certification team can review the certification tests that were run by OEMS QA in a way that does not affect their workflow
* The delta between checkbox-oem-qa and checkbox-compatibility is measured on a monthly basis to avoid it from growing again
* OEMS QA is aware of the work in checkbox/checkbox-compatibility and participate in it; The Cert team is aware of the work in checkbox-oem-qa and participate in it (with suggestions of things that could go upstream). There is a biweekly call to make this happen.
* There is documentation about C3 OOPS and how to integrate 3rd party suites into checkbox.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.