Improvements to the validation lab
Canonical now has a validation lab with several systems, usable by teams within Canonical, to conduct testing on real, representative hardware. Currently, only a few teams within the company know of this lab's existence. Access to the same is arbitrated manually via a calendar, which lends itself to conflicts, and as a consequence, there isn't a clear idea of how much the lab is being used, and whether the company is extracting as much benefit as possible from this infrastructure.
Improvements are needed in three areas:
- Coming up with a sound and ideally automated process for booking, arbitrating and accessing validation lab resources. This should supersede the current "I book my dates manually in the calendar and then ask around to see if nobody else is using it" approach. Also, the actual access/installation procedures should be streamlined as much as possible, to encourage use.
- Promoting the lab so that all areas are aware of, and benefit from, its existence.
- Generating usage metrics so the lab's usefulness can be assessed. Ideally this could take data from the booking and control system/mechanism.
Promotion, booking system and access/installation improvements will benefit teams within Canonical by providing them with equipment needed to run tests requiring actual hardware, as well as easy ways to book use of that equipment in a non-conflicting manner. Booking and access/installation will benefit Hardware Certification by reducing the burden of arbitrating the infrastructure. Finally, reporting enables management to make decisions regarding the validation lab infrastructure, by showing its value to the company and the degree to which the resources are used.
Blueprint information
- Status:
- Complete
- Approver:
- Ara Pulido
- Priority:
- Low
- Drafter:
- Daniel Manrique
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Obsolete
- Series goal:
- None
- Implementation:
-
Not started
- Milestone target:
- None
- Started by
- Completed by
- Victor Tuson Palau
Whiteboard
==Definition of Done==
1 - The entire company knows about the validation lab, who to contact, and where to look for information (a wiki page and make sure everybody knows about it).
2 - People who come to use the validation lab can see the systems that are in it, when they are available.
3 - People who want to use a validation lab system can indicate so, so the system is reserved and available for them during that timeframe.
4 - People who have reserved a system can install a specific Ubuntu release, so the system is freshly installed and ready for them to log in and work. The installation process needs to be improved, so it is 100% automated.
5 - People can do points 2-4 in a self-service fashion, through a scheduling/
6 - Reports can be generated showing the percentage of time the validation lab systems are booked; ideally, the report should also indicate how many booking requests were denied due to unavailability.
== Requirements that arose during discussions ==
The scheduling/
- Enable people to schedule: Tests that need to be run in a fixed time and Tests that need to be run whenever availalable.
- Handle queueing
- Automated jobs (see scheduling above).
- Locking of systems that are in use, so they don't get used by someone else. Come up with a way to stop someone from NOT checking the locks/schedule and blindly SSHing into a locked system.
- Notification by email that a requested system is ready, emailing automated test results (possibly), how else can we leverage automated e-mail notifications? This is good because we avoid tying people up and results get notified asynchronously.
Contact should be made with other teams in Canonical that are implementing/have implemented something like this in the past (zyga, Dustin Kirkland).
== Additional topics ==
Additional Topics: (1) (if appropriate) how does hardware get added? Can it be requested?
This topic wasn't discussed in UDS but it feels to me like the validation lab should be "generic", the use case is to "provide folks access to hardware", and the different use case where folks need access to *specific* hardware is better taken care of by providing access to systems that are used for hardware certification. As in this case they want to test compatibility and/or specific bugs, and it's better/easier to schedule a session with a HW cert engineer so that things can be interactive/faster.
To further validate the validation lab's use case, access should be as automated/
[Update] - this bp is no longer part of the Oneiric scope for the certification team
