Desktop Certification scope
The current test suite used to award certification has been put together in something of an ad-hoc fashion and has attempted to cater for both servers and clients (desktops/laptops). The level of coverage in each functional area therefore does not entirely match with the importance of that functional area. A good example is that there are only basic tests for WLAN, whereas a consumer would consider this a critical functional area.
The goal of this blueprint is to identify the hardware areas which must be working in order to award certification and analyze gaps in coverage. Some areas are already tested in some fashion (e.g. wifi) but the level of testing does not match the importance of the functionality. Some areas are not tested at all.
Potential candidates for areas which have a high importance and should have solid coverage are:
- WiFi
- Graphics
- Suspend/Resume
- Audio (Input & Output)
Achieving better coverage will benefit everyone - consumers by giving them confidence in systems that are Ubuntu Certified, OEMs through increased sales, Canonical itself with potentially more certification business and the Hardware Certification team, through a decrease in the amount of time spent handling complaints from consumers who have ended up with certified systems that didn’t quite work perfectly.
Blueprint information
- Status:
- Complete
- Approver:
- Kevin Krafthefer
- Priority:
- Essential
- Drafter:
- Brendan Donegan
- Direction:
- Approved
- Assignee:
- Kevin Krafthefer
- Definition:
- Approved
- Series goal:
- None
- Implementation:
-
Implemented
- Milestone target:
-
november2011
- Started by
- Brendan Donegan
- Completed by
- Ara Pulido
Whiteboard
Preparation tasks:
- Choose the 4 most important functional areas covered by certification (chosen as Audio, Graphics, Wifi and Suspend/Resume)
- Identify other functional areas that may require improvement or are not tested and write new test specifications for them. Areas identified are:
* Laptops using the battery (Cking: can you boot on battery?, can you suspend/resume on battery?)
* Wake on LAN: very important for enterprise
* Thermal management: Fan control, throttling (Phoronix test suite)
* Wireless performance (connect speed)
- Maybe having a limit to pass certification
* Boot time
* TPM
* Check for kernel panics on boot and resume from suspend/hibernate
* accelerometer drive locking
* Fingerprint readers
* 3G connections
* HDMI audio
* External monitors
- Audit this list and potentially remove tests, in consultation with stakeholders.
- Determine whether each test is whitelisted or greylisted (run, but failure of the test is only noted on the system and doesn't fail certification)
Definition of done
- All functional areas have been identified. [DONE]
- Top 4 areas have been identified from the shortlist defined in this blueprint. [DONE]
- Coverage has been measured in these 4 areas. [NOT ATTEMPTED]
- A list of new tests has been drawn up which increases coverage in the top 4 areas to a very high level and provides good coverage in other areas. [1/4 DONE]
- The tests have been implemented and a whitelist created. [1/4 DONE]
- Tests which are 'greylisted' have been clearly identified.
- Certificates issued for 11.10 are based on the results of runs using these tests and the new whitelist (and also take into account the greylist)
