Containerize components on Fuel node to enable upgrade
This blueprint covers the implementation of Docker containers to contain each of the Fuel components, including nailgun, astute/naily, cobbler, nginx, Fuel manifests, OSTF, postgres, and other components. Each container defines its requirements, determining container launch order. Tasks include isolation of Puppet modules, implementation of a base Docker profile, creation of a collection of shared volumes, and a clever launcher capable of starting containers with the necessary volumes and network relationships attached.
Blueprint information
- Status:
- Complete
- Approver:
- Mike Scherbakov
- Priority:
- Essential
- Drafter:
- Matthew Mosesohn
- Direction:
- Approved
- Assignee:
- Matthew Mosesohn
- Definition:
- Approved
- Series goal:
- Accepted for 5.0.x
- Implementation:
-
Implemented
- Milestone target:
-
5.0
- Started by
- Vladimir Kuklin
- Completed by
- Mike Scherbakov
Related branches
Related bugs
Sprints
Whiteboard
Done:
* Docker containers for nailgun services
- Puppetized:
-Merged: nginx, cobbler, rsync, postgresql, ostf
-Ready to merge: host only manifests
-In progress, but images exist: rabbitmq, nailgun, rsyslog, mcollectived
* Refactored every nailgun component to run independently (see item above)
* Extended version.yaml (from Ivan K) to specify production value
* Package docker-io, lxc RPMs (merged)
* Storage container framework for puppet manifests, software repositories, container service logs, and diagnostic snapshot
* Fuel base OS requirements (MERGED)
- Light manifests for non-contained features (NTP, iptables, ssh key)
- 6.5 CentOS kernel
* Extend firewall puppet module to handle docker iptables rules
* Find best solution for compressing Docker images (lrzip)
* Cobbler workaround to forward DHCP requests from nodes to Cobbler container via iptables modules and dhcrelay
* Tweak Cobbler profile config to override SERVER variable to admin IP when astute creates cobbler profiles.
* Refactor yaml path lookup to search /etc/fuel for astute.yaml and version.yaml (necessary to overcome LXC linking limitations)
* Refactor astute for RPM based install
* Rebase docker to 0.10.0 to fix some critical runtime bugs
* refactor bootstrap_
* dockerctl - docker container launcher daemon to handle names, source files, volumes (on review)
* Refactor docker image build and compression stage of Fuel ISO (testing now)
Todo:
* Fix docker init script to kill containers better
* Disk space requirements assessment (~15gb extra before cleanup of container artifacts) - documentation task
* Containers log rotation - review #91406
Nice to have:
* [Related to updating Fuel deployments] Patch nailgun to add yum repo and path to puppet modules (not started)
- Add variables for fuel_yum_repo, fuel_apt_repo in Cobbler metadata and into kickstart/pressed (not started)
- Add fuel puppet manifest version to astute deployment_
* Upgrade manager (PoC needs a DB migration example) (WIP Evgeniy L)
- May be delayed out of this release
* Base container items:
- SSH daemon for each container (NOT NEEDED - lxc-attach works)
- Tests support for each component (split run_tests.sh for nailgun into individual components)
* Tests for upgrade and rollback
* public Fuel docker container repository
* Extend default astute.yaml with credentials so they can be synced across all containers easily (instead of hardcoded in puppet) - not crucial for 5.0
Gerrit topic: https:/
Addressed by: https:/
Allow nginx to deploy separate from nailgun
Addressed by: https:/
Allow separate RabbitMQ server for nailgun
Addressed by: https:/
Separate OSTF and Postgres from nailgun
Addressed by: https:/
Separate nailgun::cobbler from nailgun
Addressed by: https:/
WIP. Optional install OSTF from packages
Addressed by: https:/
Add capability for installing astute from pkg
Addressed by: https:/
Add tftp conntrack modules on master node boot
Addressed by: https:/
Add dhcp package to the CentOS repository
Addressed by: https:/
Facts update: check several files for needed info
Addressed by: https:/
Added docker container refs
Addressed by: https:/
Add docker-io to RPM requirements
Addressed by: https:/
Add iptables rule parser for docker
Addressed by: https:/
Added base host only manifest with docker
Gerrit topic: https:/
Addressed by: https:/
Add docker support for nailgun::venv class
Addressed by: https:/
Allow separate Astute server from nailgun
Addressed by: https:/
Add server override feature for cobbler
Addressed by: https:/
Add standalone nailgun role
Addressed by: https:/
Added mcollective stand-alone manifest examples
Addressed by: https:/
Docker container support for rsyslog
Addressed by: https:/
Standalone nginx site.pp and dir fixes
Addressed by: https:/
Deploy fuel node with docker containers
Addressed by: https:/
Add missing cobbler dependencies
Addressed by: https:/
Use correct path for send2syslog.py
Addressed by: https:/
Fix ulimit for /etc/sysconfig/
Addressed by: https:/
anna-install network-console to fix sshd in ubuntu
Addressed by: https:/
Fix paths for nginx for docker containers
Addressed by: https:/
makefile for docker container build process
Addressed by: https:/
disable chroot for rsyncd in docker
Addressed by: https:/
update produciton check for venv paths
Addressed by: https:/
Update OSTF to use ostf.conf instead of params
Addressed by: https:/
Add base nailgun packages for host only role
Addressed by: https:/
Add nailgun host and port to ostf in docker env
Addressed by: https:/
fix OSTF init workflow for docker deployment
Addressed by: https:/
Deploy fuel-dockerctl tools via puppet
Addressed by: https:/
Add tar dependency to shotgun
Addressed by: https:/
Add helper aliases for cli tools in docker containers
Addressed by: https:/
Add docker packages to dev ISO doc
Addressed by: https:/
Fix conditional for docker prebuilt option
Gerrit topic: https:/
Addressed by: https:/
Update development documentation about Astute installation
Addressed by: https:/
Split pre_hooks into setup and start sections
Addressed by: https:/
Add openssh-clients requirement to shotgun
Addressed by: https:/
Add cobbler and astute volumes for diagnostic snapshot
Addressed by: https:/
Export astute and cobbler etc dirs as volumes
Addressed by: https:/
Download base images from Fuel mirror
Addressed by: https:/
Add retries to nailgun syncdb for docker deployment
Addressed by: https:/
Moved nailgun syncdb and fixtures to puppet
Gerrit topic: https:/
Addressed by: https:/
Add log rotation for docker containers
Addressed by: https:/
Clean up dockerctl usage and remove unimplemented funcs
Addressed by: https:/
Added Docker deployment information
Addressed by: https:/
Add several ways to determine fuel version for dockerctl
Addressed by: https:/
Update development documentation about ISO building
Addressed by: https:/
Put version.yaml into /etc/fuel/$RELEASE/ dir
Addressed by: https:/
Get fuel version from version.yaml
Work Items
Dependency tree
* Blueprints in grey have been implemented.
