Community Hardware Testing Infrastructure
This blueprint is described as a conversation to provide an infrastructure enabling the community and platform to collaborate on the results from testing hardware:
Community: Wouldn't it be nice if we could all share the results from testing our hardware?
Platform: Yes, it would be very nice!
Community: Why isn't this done already?
Platform: Part of the work was already done in Launchpad to store submissions from Checkbox but it only provides an interface to the files uploaded.
Community: How about providing an API?
Platform: That's the idea, detailed in the whiteboard below, so that we can all share the results by generating reports for example.
Community: Are there enough result submissions for reports?
Platform: There are already hundreds of thousands, if not millions, of submissions that could be used from Launchpad.
Community: Are there enough tests per submission?
Platform: Once we can all reap the rewards from our testing, then it makes more sense to add more tests per submission.
Community: This is going to rock!
Blueprint information
- Status:
- Complete
- Approver:
- Ara Pulido
- Priority:
- High
- Drafter:
- Marc Tardif
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Approved
- Series goal:
- None
- Implementation:
-
Implemented
- Milestone target:
-
november2011
- Started by
- Victor Tuson Palau
- Completed by
- Ara Pulido
Whiteboard
== Definition of Done ==
* API to submit new test results and append to existing submissions.
* API to retrieve sets of results across multiple submissions.
* API to query results based on hardware devices.
* API to get reporting information.
* API to specify and query on unstructured tags.
* API should support both qualitative and quantitative results.
* API and interface should support the concept of distro series releases.
* Interface to input and output common formats like JUnit.
* User experience should support OpenID integration nicely.
* Mechanism to support uploading results from submissions created offline.
* Preserve backward compatibility with Checkbox on existing stable releases of Ubuntu.
* Preserve backward compatibility with existing submissions stored in Launchpad.
* Produce a sample report using existing submissions to show most popular hardware across Ubuntu releases.
* Support hundreds of systems each uploading a hundred test results per day each containing a total of 1 megabyte of data on average.
* Support detecting whether a system only contains packages from the archives or packages from arbitrary PPAs.
* Have confirmation from the security team that the service does not send sensitive information.
* Have a presence in the live environment to determine whether a system is friendly.
* See at least 250 submissions from the community between beta and release.
* See at least 1 contribution from the community to Checkbox before release.
