Creation and implementation of an Ubuntu Cloud certification programme

Registered by Ara Pulido on 2012-10-08

The current Ubuntu Server Certified Hardware programme is a programme intended to ensure that a particular release of Ubuntu works, up to a certain level, with a specific hardware configuration. All tests in the Ubuntu Server Certified Hardware programme are hardware related, with no specific software stack tests.

Some customers have been asking to create an extension of that programme that will specifically test a version of OpenStack, with a specific version of Ubuntu, running in a hardware configuration.

This new programme will include new Open Stack tests and some hardware related ones for functionality that helps creating and managing a private cloud. These tests would be run on top of the Ubuntu Server Certified Hardware test suite and therefore, every server that is Ubuntu Cloud Infrastructure Ready will automatically appear in the list of Ubuntu Server Certified Hardware, specifying that the system is ready to be part of an Ubuntu Cloud Infrastructure deployment.

Blueprint information

Status:
Not started
Approver:
Ara Pulido
Priority:
Essential
Drafter:
Jeff Lane
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Definition of Done:
UDS: Decisions/Agreements reached on the scope and process for cloud certification
Post-UDS: Program is fleshed out, codified and ready to roll
  * Programme is defined and documented
  * Changes have been made to checkbox as needed
  * Whitelists have been created defining the Cloud Certification Test Suite
  * Code has been merged into production
  * Documentation/Guides have been updated or created for Testers

Questions:
1: What does "Cloud" mean? It's a vague term
For the purposes of this blueprint, and user stories derived from it, "Cloud" shall mean a minimal set of hardware necessary to implement the Ubuntu Cloud infrastructure. The goal is to create a certification programme that demonstrates that Ubuntu functions well on a certified system as a Host OS. According to the technical resources for Ubuntu Cloud, this can be accomplished on a single system.

2: What are we going to consider a "Cloud" for cert purposes?
A single server, or perhaps a small number of servers to create a small instance of Ubuntu Cloud. The focus needs to be on Ubuntu Cloud from a hardware point of view. It may be necessary to include MaaS and JuJu in testing as those technologies are suggested for deploying Ubuntu Cloud infrastructure. However, there is also a 12.04 LTS Live USB Cloud image.

3: What software bits make up a cloud?
The scope of this test will be limited to Ubuntu as a Host, Ubuntu as a guest, MaaS and JuJu. We may also include using XenServer/XCP as a hypervisor with Ubuntu as a guest. This will require more investigation into Xen infrastructure

4: What software bits will we require for Cloud certification? Suggestions:
  * Ubuntu Server
  * Ubuntu Cloud images
  * OpenStack
  * MAAS
  * Juju

5: What teams/people should be involved?
    Ara
    Nick Barcet
    Dave Walker
    Ameet
    Massimo
    ??

6: How do we test it?
Carefully. Testing should be hardware focused, demonstrating that the hardware can perform in a cloud environment. This should include running a VM on an Ubuntu host, potentially testing NIC bonding/aggregation, storage testing of SAN and external storage solutions.

7: What are the fail criteria for testing?
This is still TBD, however, to start the following whitelist test items MUST pass in order for a server to pass the certification testing: (NOTE the list is still in flux and tests may be moved, added or removed from both whitelist and greylist prior to when the program begins).

  * Creating a virtual machine using OpenStack with KVM (use LXC if KVM not supported - will need a note added) as hypervisor
  * Network bonding
  * Network Failover
  * VLAN tagging capabilities
  * Object storage tests - Swift
  * Storage backend solution testing (solidfire, netapp...)
  * Passing this test would list the backend storage as cloud ready too
  * iSCSI root
  * Suspend

The following items are considered greylist items. They will be tested, but will not cause the cert to fail. Failures will result in a note attached to the certificate explaining any issues or workarounds necessary for full functionality.

  * Creating a virtual machine using OpenStack with Xen/XCP as hypervisor
  * Availability of Ubuntu tools to do firmware updates
  * Provisioning of Server using MaaS - to be move to whitelist in the future
  * Wake on Lan
  * Power cycling
  * image deployment
  * Deployment of services using Juju (Wordpress test)
  * Object storage tests - Ceph

Links:
http://www.ubuntu.com/cloud
https://help.ubuntu.com/community/UbuntuCloudInfrastructure

Goals for UDS Session:
Introduce the Cloud Certification Programme and what it's about.
Review the proposed tests.
Garner suggestions on what hardware would could be included in a "wishlist" to create a basic development lab.

(?)

Work Items

Work items:
[bladernr] Draft the blueprint: INPROGRESS
[bladernr] Invite people who should be involved in the discussion: DONE
[bladernr] Add user stories to blueprint: DONE
[mprasodjo] Schedule private session(s) with invitees: TODO
Define the scope of the program: TODO
Define the basic hardware requirements: TODO
Define the basic software stack requirements: TODO
Document/Define the Cloud Certification Programme: TODO
Decide on Pricing: TODO
Draft marketing/sales documentation: TODO
Define the overall testing process for cloud: TODO
As Canonical, I want to have a robust Cloud Certification Programme. So I can Show the world Ubuntu is the OS to use in Cloud Deployments.: TODO
As Ara, I want to know what needs to be added to checkbox for Cloud Certification Testing. So I can have a well defined Cloud Certification Suite.: TODO
As Jeff, I want to know what infrastructure I'll need to set up a development testbed for Cloud certificaiton. So I can work on creating a cloud certification test suite.: TODO
As Victor, I want to have the cloud ready bits of public certification entries converted to promote cloud certified hardware. So I can further assure OEMs and customers that their hardware is truly Certified for a cloud environment.: TODO
As Hardware Certification, I want to be sure the Cloud Certification covers the hypervisors we care about, So I can inform customers that a server will function as a node in their cloud deployment.: TODO
As Hardware Certification, I want to be sure Networking is adequately tested in the Cloud Scope. So I can show customers that Cloud Certified servers have reliable network devices.: TODO
As a Storage Fan, I want to test storage in a Cloud scope. So I can be sure that my storage solution will function well in my cloud deployment: TODO
As an OEM, I want to be sure my Firmware Flashing utilities work with Cloud Certified servers. So I can reassure my customers that they can update their systems.: TODO
As the DC Engineer, I want a robust way to provision my hardware. So I can quickly deploy new nodes in my Cloud.: TODO
As Jeff, I want to create a Cloud Certification Suite above and beyond the server Certification suite. So I can show that there are servers, and there are Cloud Certified servers.: TODO
As the OEM customer, I want to have full documentation on the Cloud Certification programme, User Guides and so forth. So I can have my engineers up to speed and ready to go with Cloud Certification Testing.: TODO