Exploring a hardware database for testing

Registered by Nicholas Skaggs

Discussing the usage of HEXR for testing purposes

Example use cases;
Using your hardware profile inside the qatracker during reporting

Ultimate goal;
Targeted testcases for specific hardware configurations. Testing campaigns to push testcases and results back to developers

Blueprint information

Status:
Not started
Approver:
Jono Bacon
Priority:
Medium
Drafter:
Nicholas Skaggs
Direction:
Needs approval
Assignee:
None
Definition:
Pending Approval
Series goal:
Accepted for raring
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Demo of HEXR

    Will use checkbox to upload submissions

    Can see machines and working status of machines, components, etc

    Can create testing pools of machines based on any criteria in database

    Lacks a 'my hexr' feel -- no way to link specific person to a machine

    Lacking true API

Is a tool like this useful? Yes
Do we want this tool? Yes
Not yet open source, but planned to be
Do we want to wait for the open source release or not before adopting? No
Do we want to wait for features we 'need' before releasing? No
Make availible asap, as-is
Is it possible to provide a feature that provide suggestions of drivers and packages might be needed when we do a HEXR upload for a new platform to be enabled? This will be extremely valuable for OEM enablement. This may require integration of database on launchpad and release notes.
yes, driver information is availible
Current HW DB:
Results tracker
ubuntu friendly
hexr
launchpad
Needs/Enhancements:
Can we contact user machines?
API for linking machines, ie the qatracker
Pushing a testing payload to a machine (even turning it on, etc) based on data found in HEXR
Need to create seperate machine instances depending on drivers (so we can be specific when testing as to what is working and what isn't)
Access controlled by single-sign on
Sign in to read? Sign in to edit?
Everybody can read
You can edit your own stuff only
Launchpad team for admin access (can read and write everything)
Agree to make your data availible in HEXR and agree to make your systems available for testing

Old List of Ideas / Needs / Requirements
Needs

    Ability to diagnose a user's system and create a hardware profile

    How can we enter a custom-built machine (I care more about components than product names)?

    seems to use motherboard vendor and name

    Can we package up upload-hw.py?

    make it depend on dependencies, and simply have users install it and run

    Is there a userguide to submitting a system?

    can help create if needed

    internal website help buttons or links to docs would help

    Ability to associate a hardware profile with a specific user's contact information (aka, email address)

    I’m thinking linking and integrating with launchpad is probably the way to go here

    Would like to NOT have to go to lp to get contact email address; even if machine is linked to user via lp profile

    Ability for specific user to opt-in and opt-out (they need to be able to change their mind ) to being contacted for testing

    Don’t want to overload with lots of “account” features or place maintenance burden on users

    Perhaps make it an attribute that can be changed by the user for their system only

    Ability for a specific user to update their hardware profile once submitted

    Ability for a specific user to view their hardware profile once submitted

    I seemingly have to search for my system by driver or vendor name and drill down and find it. Source isn’t a filterable option; and I can’t select a specific person

    Ability for a specific user to link to their hardware profile

    for things like bug reports, testing tracker, etc

    any API for automagically pulling based on ubuntu sso?

    Ability to search the database for users who have opted in for testing and meet my hardware criteria (eg, Atheros AR5007 Wireless LAN adaptor)

    playing around with testing pools, I think you can solve this by making a “contactable for testing” attribute, along with including additional information in the export related to the person and links to the system

Nice to haves

    Testing pools don’t allow for multiple selection of devices very easily

    can I make a search (with all it’s filters, etc) a pool automagically?

Notes

    Drivers don’t seem to be useful

    data issue only?

    can we ensure drivers are a part of the community version?

    are they reported now by the tool -- I didn’t see that my nvidia driver version as being reported on my submission

    How do we handle updates to a system?

    can I see that it didn’t use to work, but now does?

    can I tell if something is needed to make it work?

User Stories
The kernel team imports a patch from upstream to enable support for the Atheros AR5007 wireless chipsets. Nicholas searches HEXR for machines containing the chipset and gets a results page showing the current machines containing the device, along with a listing of the machine, person, launchpad profile, and contact email address. Nicholas copies the listing of email addresses from the page and sends an email off directing the users to the testing tracker to test out the new kernel.
A QA community member wants to make his system available for testing. He installs a package and is walked through adding his system to the community HEXR database. He is prompted to allow his system to be a part of the testing pool, and is taken to his submission on HEXR to report compatibility.
A QA community member is contacted about a new package to enable hardware identified in his system. Upon completing the testing, the hardware is now working. The community member updates his hw profile to specify the hardware is now working.
Future Thoughts/Ideas:

    How can a user maintain their system inside HEXR?

    user account?

    package automatically monitoring for hardware changes (what about hardware support changes?)

    Is there a need to be able to generically search for hardware that meets certain technologies?

    PAE/non-PAE; 64-bit or 32-bit only, sse extensions, vt extensions, etc

    Where does ubuntu friendly fit?

(?)

Work Items

Work items:
[nskaggs] Talk to mhall119 about creating skunkworks for hexr: DONE
[schwuk] Stand-up new public hexr system for community: TODO
[schwuk] Move over to using checkbox for submissions: TODO
[schwuk] Open source hexr: TODO
[schwuk] Speak with checkbox team to add options to submit to HEXR: TODO
[schwuk] Provide linking and contact information for machine owner: TODO
[schwuk] Provide API call to provide machine information: TODO

This blueprint contains Public information 
Everyone can see this information.