Fuel for OpenStack

Merge OpenStack Puppet Modules

Registered by Matthew Mosesohn

In order to achieve better stability in Fuel and benefit the community, we need to merge our Puppet modules upstream. This process involves merging core OpenStack components first, then supporting components/HA architecture second.

Feature Lead: Aleksandr Didenko
Mandatory Design Reviewers: Vladimir Kuklin
Developers: Aleksandr DIdenko, Bogdan Dobrelya
QA: Anastasia Urlapova
Acceptance criteria: system tests passed

Blueprint information

Status:
Complete
Approver:
Mike Scherbakov
Priority:
High
Drafter:
Matthew Mosesohn
Direction:
Approved
Assignee:
Aleksandr Didenko
Definition:
Approved
Series goal:
Accepted for 6.0.x
Implementation:
Implemented
Milestone target:
milestone icon 6.1
Started by
Aleksandr Didenko
Completed by
Dmitry Borodaenko

Related branches

Whiteboard

[Dmitry Borodaenko 2014-12-16]
Closed as Implemented in 6.0: the only way to *complete* a blueprint defined like this one is to implement a CI process that would keep Fuel in sync with all upstream modules *automatically*. Until we can do that, best we can do is sync a specific set of modules up to specific versions in every release of Fuel, and the only way to *track* that kind of work is to create a separate bug for every such update, per module. Since there's no need for design, it would be easier to track that as bugs instead of blueprints.

Early components:
- Nova
- Glance
- Cinder
- Swift
Later (harder) components:
- Keystone
- Horizon
- Neutron
- Savanna
- Murano
- Heat
- Ceph (custom, no direct upstream. Needs to be moved (Xarses)
Supporting components:
- MySQL
- Galera
- Cobbler
- RabbitMQ
- QPID
- Corosync/Pacemaker
- HAProxy
- PostgreSQL

---------------------
Testing methodology:
https://etherpad.openstack.org/p/fuel_puppet_modules_upgrade

---------------------
Gerrit topic: https://review.openstack.org/#q,topic:bp/merge-openstack-puppet-modules,n,z

---------------------
MERGED:

* Essential patches needed for proper CI and testing of OpenStack modules synced from upstream:

Addressed by: https://review.openstack.org/99902
    Sync puppetlabs-stdlib module to 4.2.1

Addressed by: https://review.openstack.org/99904
    Remove stdlib::safe_package dependencies

Addressed by: https://review.openstack.org/99946
    Adapt keystone_endpoint for new upstream modules

Addressed by: https://review.openstack.org/100821
    Refactor mysql for upstream modules

* Openstack modules:

Addressed by: https://review.openstack.org/101166
    Sync cinder module from puppetlabs upstream

Addressed by: https://review.openstack.org/101471
    Adapt synced puppetlab-cinder for Fuel usage

Addressed by: https://review.openstack.org/86350
    Sync puppet-nova

Addressed by: https://review.openstack.org/86351
    Adapt synced puppet-nova for Fuel usage

Addressed by: https://review.openstack.org/86574
    Sync puppet-heat

Addressed by: https://review.openstack.org/86598
    Adapt synced puppet-heat for Fuel usage

Addressed by: https://review.openstack.org/99110
    Sync glance module from puppetlabs upstream

Addressed by: https://review.openstack.org/99150
    Adapt synced puppetlab-glance for Fuel usage

Addressed by: https://review.openstack.org/103078
    Sync puppet-ceilometer

Addressed by: https://review.openstack.org/103092
    Adapt synced puppet-ceilometer for Fuel usage (WIP)

Addressed by: https://review.openstack.org/127268
    Syn swift module with upstream

Addressed by: https://review.openstack.org/127589
    Adapt synced swift module

Addressed by: https://review.openstack.org/86007
    Sync puppet-keystone

Addressed by: https://review.openstack.org/86008
    Adapt synced puppet-keystone for Fuel usage

Addressed by: https://review.openstack.org/103277
    *WIP* Purge neutron folder before sync

Addressed by: https://review.openstack.org/103278
    Sync with upstream puppet-neutron

* Non-openstack modules:

Addressed by: https://review.openstack.org/92351
    Sync firewall module from puppetlabs upstream

Addressed by: https://review.openstack.org/92361
    Adapt synced puppet-firewall for Fuel usage

Addressed by: https://review.openstack.org/99966
    Sync puppetlabs-apt to 1.4.2

Addressed by: https://review.openstack.org/100850
    Sync puppet ssh module to v2.3.6 from upstream

Addressed by: https://review.openstack.org/100878
    Remove unused puppet module mmm

Addressed by: https://review.openstack.org/100937
    Remove unused puppet module nagios

Addressed by: https://review.openstack.org/100947
    Remove unnecessary files from puppet/astute directory

Addressed by: https://review.openstack.org/101167
    Remove unused puppet module squid

Addressed by: https://review.openstack.org/101215
    Sync puppet concat module to v1.1.0 from upstream

Addressed by: https://review.openstack.org/101229
    Remove unused puppet module git

Addressed by: https://review.openstack.org/101550
    Remove unused module keepalived

Addressed by: https://review.openstack.org/102138
    Remove unused puppet module selinux

Addressed by: https://review.openstack.org/102516
    Remove unused puppet module puppetdb

Addressed by: https://review.openstack.org/102184
    Sync puppet module lvm to v0.3.1 from upstream

Addressed by: https://review.openstack.org/102231
    Sync puppet module memcached to v2.5.0 from upstream

Addressed by: https://review.openstack.org/101198
    Sync sysctl module to v0.0.4 from upstream

Addressed by: https://review.openstack.org/101202
    Adapt synced sysctl module for Fuel

---------------------
NOT MERGED:

* Openstack modules:

Addressed by: https://review.openstack.org/132687
    Add new module 'apache'

Addressed by: https://review.openstack.org/132564
    Sync horizon module with upstream

Addressed by: https://review.openstack.org/132688
    Adapt synced horizon module

* Non-openstack modules:

Addressed by: https://review.openstack.org/102483
    Sync puppet module ntp to v3.1.1 from upstream

Addressed by: https://review.openstack.org/102830
    Adapt puppet module ntp for Fuel

Addressed by: https://review.openstack.org/102871
    Sync puppet module rsync to v0.3.0 from upstream

Addressed by: https://review.openstack.org/103075
    Sync puppet module xinetd to v1.3.0 from upstream

Addressed by: https://review.openstack.org/103655
    Clean up haproxy_service['mysqld']->Anchor['galera'] dep.

Addressed by: https://review.openstack.org/104513
    Add libguestfs-tools for Ubuntu

Addressed by: https://review.openstack.org/104607
    Revert "Sync sysctl module to v0.0.4 from upstream"

Addressed by: https://review.openstack.org/104613
    Revert "Adapt synced sysctl module for Fuel"

Addressed by: https://review.openstack.org/107638
    Sync haproxy 0.5.0 module from puppetslab.

Addressed by: https://review.openstack.org/107639
    Adapt synced haproxy 0.5.0 for Fuel usage.

Addressed by: https://review.openstack.org/119076
    Sync inifile with upstream

Gerrit topic: https://review.openstack.org/#q,topic:bp/ml2-neutron,n,z

Addressed by: https://review.openstack.org/126006
    Sync puppet-corosync

Addressed by: https://review.openstack.org/126010
    fix incompatibles between new Neutron and old Keystone manifests

Addressed by: https://review.openstack.org/126027
    Adapt synced Corosync module

Addressed by: https://review.openstack.org/126414
    Adaptation cluster::corosync::cs_service to asymetric cluster

Addressed by: https://review.openstack.org/127165
    Sync puppetlabs rabbitmq module

Addressed by: https://review.openstack.org/127166
    Adapt rabbitmq module for Fuel

Addressed by: https://review.openstack.org/128214
    Fix NSX plugin manifests for upstream neutron modules

Addressed by: https://review.openstack.org/128219
    Adding l3_ext network type for NSX support

Addressed by: https://review.openstack.org/130476
    Sync with upstream puppet-neutron

Addressed by: https://review.openstack.org/130477
    Adapt upstream puppet-neutron into fuel

Addressed by: https://review.openstack.org/130478
    Neutron HA composition model

Addressed by: https://review.openstack.org/130479
    TEMPORARY neutron-keystone fix

Addressed by: https://review.openstack.org/130484
    Adding l3_ext network type for NSX support

Addressed by: https://review.openstack.org/130723
    Adding l3_ext network type for NSX support

Addressed by: https://review.openstack.org/130739
    Fix NSX plugin for upstream neutron modules

Gerrit topic: https://review.openstack.org/#q,topic:adapt-postgres,n,z

Addressed by: https://review.openstack.org/131073
    Rebase postgresql to puppetlabs-postgresql-4.0.0

Gerrit topic: https://review.openstack.org/#q,topic:bug/1384149,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bug/1388049,n,z

Addressed by: https://review.openstack.org/132217
    Fix ntp conf template name in host-upgrade.pp

Addressed by: https://review.openstack.org/129925
    Use separate pid files for Pacemaker OCF

Addressed by: https://review.openstack.org/130823
    Remove old basic service_disables

Addressed by: https://review.openstack.org/132318
    Refactor Heat HA to wrappers

Addressed by: https://review.openstack.org/129351
    New service provider

Addressed by: https://review.openstack.org/141325
    Rebase postgresql to puppetlabs-postgresql-4.0.0

Addressed by: https://review.openstack.org/145788
    Sync apt module from puppetlabs

Gerrit topic: https://review.openstack.org/#q,topic:bp/iotrowski,n,z

Addressed by: https://review.openstack.org/146394
    Merge camptocamp/openssl upstream module

Addressed by: https://review.openstack.org/146408
    Merge puppetlabs-xinetd v1.3.1 module

Gerrit topic: https://review.openstack.org/#q,topic:bp/merge-openstack-puppet-module,n,z

Addressed by: https://review.openstack.org/148894
    Merge puppetlabs/cinder upstream module

Addressed by: https://review.openstack.org/148895
    Adapt puppetlabs/cinder module to Fuel

Addressed by: https://review.openstack.org/148916
    Initial import of puppet-openstacklib module

Addressed by: https://review.openstack.org/189281
    Merge puppetlabs/stdlib 4.6.0

Addressed by: https://review.openstack.org/189567
    Merge puppetlabs/concat 2.0.1

Gerrit topic: https://review.openstack.org/#q,topic:merge-openstack-puppet-module,n,z

Addressed by: https://review.openstack.org/189598
    Adapt tests and modules to concat 2.0.1

Addressed by: https://review.openstack.org/189689
    Merge saz/ssh 2.8.1

Addressed by: https://review.openstack.org/190055
    Merge puppetlabs/firewall 1.6.0

Addressed by: https://review.openstack.org/190083
    Merge puppetlabs/vcsrepo 1.6.0

Addressed by: https://review.openstack.org/190105
    Merge puppetlabs/inifile 1.3.0

Addressed by: https://review.openstack.org/190165
    Merge puppetlabs/apt 2.0.1

Addressed by: https://review.openstack.org/190168
    Remove unused epel and openstack_extras modules

Addressed by: https://review.openstack.org/190170
    Merge puppetlabs/lvm 0.5.0

Addressed by: https://review.openstack.org/190192
    Merge saz/memcached 2.8.1

Addressed by: https://review.openstack.org/190203
    Upgrade OpenStack puppet modules in fuel-library

Gerrit topic: https://review.openstack.org/#q,topic:bp/pacemaker-improvements,n,z

Addressed by: https://review.openstack.org/191769
    Remove openstack_extras module

Gerrit topic: https://review.openstack.org/#q,topic:bug/1383639,n,z

Addressed by: https://review.openstack.org/192597
    Adapt synced puppetlabs-mongodb module for Fuel usage

Addressed by: https://review.openstack.org/192714
    Adapt synced Heat module for Fuel usage

Addressed by: https://review.openstack.org/194091
    Adapt synced Cinder module for Fuel usage

Addressed by: https://review.openstack.org/194647
    Adapt synced Nova module for Fuel usage

Addressed by: https://review.openstack.org/195137
    Adapt synced Cinder module for Fuel usage

Addressed by: https://review.openstack.org/194683
    [DONOTMERGE] Adapt synced keystone module

Addressed by: https://review.openstack.org/192602
    Upgrade Heat puppet module

Addressed by: https://review.openstack.org/194202
    sync keystone with upstream

Gerrit topic: https://review.openstack.org/#q,topic:bug/1468230,n,z

Addressed by: https://review.openstack.org/195112
    Sync cinder with upstream

Addressed by: https://review.openstack.org/196933
    Remove puppet-network

Gerrit topic: https://review.openstack.org/#q,topic:bp/upgrade-openstack-puppet-modules,n,z

(?)

Work Items

Work items:
Design workflow doc: DONE
Essential compatibility patches: DONE
Dependency components: DONE
Removing unneeded components: DONE
Early components: DONE
Later components (only horizon left): INPROGRESS
Supporting components: TODO

Dependency tree

* Blueprints in grey have been implemented.

Nearby

This blueprint contains Public information 
Everyone can see this information.