Cache required Fedora Image for devstack testing

Registered by Ronald Bradford on 2015-05-27

Reduce the check-functional-dsvm-magnum gate run time by caching the required fedora image. A typical run of 43 minutes includes 7 minutes (15% of time) to download the fedora image from fedorapeople.org.

Blueprint information

Status:
Complete
Approver:
Adrian Otto
Priority:
Medium
Drafter:
Ronald Bradford
Direction:
Approved
Assignee:
None
Definition:
Superseded
Series goal:
Accepted for mitaka
Implementation:
Unknown
Milestone target:
None
Completed by
hongbin on 2016-05-01

Related branches

Sprints

Whiteboard

See also: https://blueprints.launchpad.net/magnum/+spec/ubuntu-image-build

The devstack test obtains a Fedora 21 atomic image at https://fedorapeople.org by the code https://git.openstack.org/cgit/openstack/magnum/tree/devstack/plugin.sh#n25

There are several possible steps to improving this operation. I am not the best person to judge the relative metrics. This is the information collected to date.

1. Upload the image to tarballs.openstack.org for NodePool usage. This will reduce the network overhead for Nodes launched in RackSpace, but not as much for HP Cloud. This can also remove a potential SPOF for fedorapeople.org.

sdake has informed me this image is actually a combination of the fedora atomic 21 + rpm-ostree upgrade + kube 0.15 and is then uploaded by OS.

Heat currently uploads a fedora test image to http://tarballs.openstack.org/heat-test-image which is published by https://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/heat.yaml#n34 (thanks clarkb)

I am not sure we want general population to use Nodepool specific locations. How do we have the devstack/plugin.sh be one version for general population usage and another for Zuul?

2. Look for consolidation with Heat project

Infra has suggested that ideally using the same image will be beneficial in subsequent caching.

sdake has stated that using same image as Heat is a non starter as Magnum requires atomic images.

3. Use a cached image with NodePool instances.

The goal is to have the required image on the default disk builder image that is used to launch all new instances. This image is built daily by the Infra team.
This would eliminate the need download the file completely. However in order for this be achieved there needs to be a consolidation with Heat (See point 2) in order to minimize the size of the launch image.

This will also eliminate the problem of mirrors and future NodePool usage of multiple clouds (currently 2).

Also for reference Fedora 22 was released on May 26 so this could be considered if tested appropriately.

----------------------------
For reference the stable/juno devstack version has a Fedora image baked in which is not appropriate. It may be of benefit to determine this images use as well.

git clone git://git.openstack.org/openstack-dev/devstack
cd devstack/
git checkout origin/stable/juno
tools/image_list.sh

http://download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img
http://download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-uec.tar.gz
http://download.openvz.org/template/precreated/ubuntu-12.04-x86_64.tar.gz
http://partnerweb.vmware.com/programs/vmdkimage/cirros-0.3.2-i386-disk.vmdk
https://download.fedoraproject.org/pub/alt/openstack/20/x86_64/Fedora-x86_64-20-20140618-sda.qcow2
https://github.com/downloads/citrix-openstack/warehouse/cirros-0.3.0-x86_64-disk.vhd.tgz
http://tarballs.openstack.org/trove/images/ubuntu_mysql.qcow2/ubuntu_mysql.qcow2

./stack.sh

glance image-list
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
| 4efc42ed-33a8-4341-be65-7ae669cff590 | cirros-0.3.2-x86_64-uec | ami | ami | 25165824 | active |
| 50c97365-a940-4045-b851-531fd3480b74 | cirros-0.3.2-x86_64-uec-kernel | aki | aki | 4969360 | active |
| 0cfb4f80-a56e-4d3b-bd9c-a0a83a3997f0 | cirros-0.3.2-x86_64-uec-ramdisk | ari | ari | 3723817 | active |
| 18ab6877-ed57-4c08-a85f-c9924a5d901e | Fedora-x86_64-20-20140618-sda | qcow2 | bare | 209649664 | active |
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+

(hongbin) This BP is superseded by https://blueprints.launchpad.net/magnum/+spec/fedora-atomic-image-build .

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.