Install from source rather than centos+rdo

Registered by Steven Dake

Install from source

Blueprint information

Status:
Complete
Approver:
Steven Dake
Priority:
High
Drafter:
Steven Dake
Direction:
Approved
Assignee:
Alicja Kwasniewska
Definition:
Approved
Series goal:
Accepted for liberty
Implementation:
Implemented
Milestone target:
milestone icon liberty-2
Started by
Steven Dake
Completed by
Paul Bourke

Related branches

Sprints

Whiteboard

https://etherpad.openstack.org/p/install-from-source-remaining-work
====================================================
pbourke:
As part of this bp we should likely also update requirements.txt from global to reduce chance of incompatibilities in the master tree:

SamYaple pbourke: where did we lang on pulling in upstream requirements for the projects? 13:46
SamYaple land* 13:46
pbourke umm 13:47
pbourke remind me of that discussion? 13:47
SamYaple https://github.com/openstack/requirements 13:47
SamYaple we can use that to update teh requirements.txt 13:47
SamYaple the reason being those only get more restrictive rather than less, so it is safe and we catch a few issues earlier 13:48
*** athomas has joined #kolla 13:48
SamYaple downstream projects always update based on that repo, but it takes a few days 13:48
pbourke so you mean you would like to grab the global-requirements from there before building each component? 13:50
SamYaple uh not quite 13:50
SamYaple hold on 13:50
SamYaple https://github.com/SamYaple/yaodu/blob/master/ansible/roles/docker_build/templates/ubuntu/keystone/Dockerfile.j2#L7 13:51
SamYaple cd /opt/kolla/global-req/ && python update.py /keystone 13:51
SamYaple that will update all the items in the requirements.txt file for keystone 13:52
SamYaple (not add new ones, just update existing since issue were encountered upstream) 13:52
pbourke ok. is that standard practice for consumers when installing components from source? 13:53
pbourke (openstack components) 13:53
SamYaple how many from source installs do you think people do :) 13:53
SamYaple i dont think there is a standard practice here 13:53
* pbourke shrugs 13:54
SamYaple but in any case it is a safe operation, it will only catch issues that didn't exist when the branch/tag was created due to new package versions
================================================================
pbourke:
I had started this as part of https://blueprints.launchpad.net/kolla/+spec/add-optional-oraclelinux-images. There are some key decisions that need to be made around how source installs are done however, so have agreed to work with Alicja & MichaƂ to make this happen for centos. The outcome should easily apply into OL and other images.

Wrt to the decisions outstanding mentioned above, some were discussed on IRC (logs, http://eavesdrop.openstack.org/irclogs/%23kolla/%23kolla.2015-06-16.log.html). I can summarise / pull out some of the main ones:

1. What source format should we support i.e. tarball/egg/git repo/wheel/etc.
2. Should these be downloaded as part of the Dockerfile or using an external process and ADD'ed in.

Sam has quite a bit of experience in the various approaches and has a good argument for using tarballs, which are fetched *outside* of the image build process. However they are fetched can be flexible, the end result should be a tarball that is provided to the Dockerfile build process.

A key use case this provides is to apply a patch or ref change to a git repo, and build a tarball from that which can be plugged into the image build without having to change the Dockerfile. Another benefit includes not requiring external internet access for this part of the image build.

There are examples in the yaodu repo of this kind of thing that we should review and apply here (https://github.com/SamYaple/yaodu/blob/master/ansible/roles/docker_build/templates/ubuntu/keystone/Dockerfile.j2#L4-L8)

Gerrit topic: https://review.openstack.org/#q,topic:bp/install-from-source,n,z

Addressed by: https://review.openstack.org/193166
    Keystone installation from source

Addressed by: https://review.openstack.org/197025
    Add source base container link for centos

Addressed by: https://review.openstack.org/197027
    Move declaration of TMPDIR to top of build-docker-image

Addressed by: https://review.openstack.org/197387
    Glance installation from source

Addressed by: https://review.openstack.org/197897
    Update centos-source-base from symlink to a real layer

Addressed by: https://review.openstack.org/197919
    Add new script to fetch tarballs for source installation

Addressed by: https://review.openstack.org/201665
    Glance installation from source

Addressed by: https://review.openstack.org/201886
    Add build from source for Heat

Addressed by: https://review.openstack.org/201895
    Add build from source for Magnum

Addressed by: https://review.openstack.org/202421
    Add build from source for Ceilometer

Addressed by: https://review.openstack.org/202424
    Add build from source for Barbican

Addressed by: https://review.openstack.org/202441
    Add build from source for Zaqar

Addressed by: https://review.openstack.org/202446
    Add build from source for Horizon

Addressed by: https://review.openstack.org/202568
    Add build from source for Nova

Addressed by: https://review.openstack.org/202662
    Add build from source for Neutron

Addressed by: https://review.openstack.org/202947
    Add build from source for Designate

Addressed by: https://review.openstack.org/205086
    Adding cinder source image

Addressed by: https://review.openstack.org/205952
    Add build from source for gnocchi

Addressed by: https://review.openstack.org/206309
    Add fedora base image for build from source

Addressed by: https://review.openstack.org/206314
    Add Ubuntu base image for build from source

Addressed by: https://review.openstack.org/207327
    Fedora - Install from source for Keystone

Addressed by: https://review.openstack.org/207344
    Fedora - Install from source for Cinder

Addressed by: https://review.openstack.org/207346
    Fedora - Install from source for Glance

Addressed by: https://review.openstack.org/207356
    Fedora - Install from source for Heat

Addressed by: https://review.openstack.org/207709
    Fedora - Install from source for Neutron

Addressed by: https://review.openstack.org/207710
    Fedora - Install from source for Magnum

Addressed by: https://review.openstack.org/207723
    Fedora - Install from source for Ceilometer

Addressed by: https://review.openstack.org/207827
    Fedora - Install from source for Nova

Addressed by: https://review.openstack.org/207838
    Fedora - Install from source for Horizon

Addressed by: https://review.openstack.org/208110
    Add build from source for Swift

Addressed by: https://review.openstack.org/208789
    Fedora - Install from source for Swift

Addressed by: https://review.openstack.org/208795
    Fedora - Install from source for Gnocchi

Addressed by: https://review.openstack.org/209336
    Fedora - Install from source for Designate

Addressed by: https://review.openstack.org/209430
    Add horizon source for oraclelinux

Addressed by: https://review.openstack.org/211221
    Remove tox -egenconfig since its not needed for cinder

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.