Community Hardware Testing Infrastructure

Registered by Marc Tardif on 2011-04-12

This blueprint is described as a conversation to provide an infrastructure enabling the community and platform to collaborate on the results from testing hardware:

Community: Wouldn't it be nice if we could all share the results from testing our hardware?

Platform: Yes, it would be very nice!

Community: Why isn't this done already?

Platform: Part of the work was already done in Launchpad to store submissions from Checkbox but it only provides an interface to the files uploaded.

Community: How about providing an API?

Platform: That's the idea, detailed in the whiteboard below, so that we can all share the results by generating reports for example.

Community: Are there enough result submissions for reports?

Platform: There are already hundreds of thousands, if not millions, of submissions that could be used from Launchpad.

Community: Are there enough tests per submission?

Platform: Once we can all reap the rewards from our testing, then it makes more sense to add more tests per submission.

Community: This is going to rock!

Blueprint information

Status:
Complete
Approver:
Ara Pulido
Priority:
High
Drafter:
Marc Tardif
Direction:
Needs approval
Assignee:
None
Definition:
Approved
Series goal:
None
Implementation:
Implemented
Milestone target:
milestone icon november2011
Started by
Victor Tuson Palau on 2011-09-07
Completed by
Ara Pulido on 2012-05-04

Related branches

Sprints

Whiteboard

== Definition of Done ==

* API to submit new test results and append to existing submissions.
* API to retrieve sets of results across multiple submissions.
* API to query results based on hardware devices.
* API to get reporting information.
* API to specify and query on unstructured tags.
* API should support both qualitative and quantitative results.
* API and interface should support the concept of distro series releases.
* Interface to input and output common formats like JUnit.
* User experience should support OpenID integration nicely.
* Mechanism to support uploading results from submissions created offline.
* Preserve backward compatibility with Checkbox on existing stable releases of Ubuntu.
* Preserve backward compatibility with existing submissions stored in Launchpad.
* Produce a sample report using existing submissions to show most popular hardware across Ubuntu releases.
* Support hundreds of systems each uploading a hundred test results per day each containing a total of 1 megabyte of data on average.
* Support detecting whether a system only contains packages from the archives or packages from arbitrary PPAs.
* Have confirmation from the security team that the service does not send sensitive information.
* Have a presence in the live environment to determine whether a system is friendly.
* See at least 250 submissions from the community between beta and release.
* See at least 1 contribution from the community to Checkbox before release.

(?)

Work Items