need to be more flexible in ceph role assignments

Registered by Gregory Elkinbard

We currently default RadosGW and monitors to the controllers. As we have learned at customer's deployments this is not a brilliant idea. Both openstack and ceph are full of bugs and we do nothing to prevent bad neighbor on the controllers. So a run away monitor process at Hawaii took out a controllers and through Fuel HA recipe bug took out both OpenStack and Ceph clusters. We need to allow for monitors and radosgw to be separately definable roles which can be flexibly assigned to controllers, computes or standalone nodes

Blueprint information

Andrew Woodward
Gregory Elkinbard
Dmitry Borodaenko
Series goal:
Accepted for future
Slow progress
Milestone target:
milestone icon next
Started by
Vladimir Kuklin

Related branches



Two new roles have to be created: ceph-mon and ceph-rgw. The roles should be defined in openstack.yaml (with no role-specific volumes), the ceph Puppet module should be updated to further decouple ceph::mon and ceph::radosgw classes.

N.B.: ceph::radosgw currently assumes that apache is already installed, need to verify if that assumption has to be amended on a standalone ceph-rgw node.

angdraug: also see superceded blueprint


Work Items

Work items:
Create ceph-mon role in openstack.yaml: INPROGRESS
-- needs better description: TODO
Create ceph-rgw role in openstack.yaml: INPROGRESS
-- also needs better description: TODO
Extract Fuel specific parts of ceph Puppet module into ceph::fuel: INPROGRESS
Make ceph::mon class self-sufficient: INPROGRESS
Make ceph::radosgw class self-sufficient: INPROGRESS
Add post-deployment checks for ceph-mon and ceph-rgw: TODO
Serializer changes: INPROGRESS
-- update tests: TODO
-- ceph-primary-mon FIRST: DONE
-- ceph-mons second : DONE
-- ceph-osds third: POSTPONED
-- ceph-rgw LAST - Not needed : DONE
-- generate rbd_secret_uuid: POSTPONED
-- generate fsid: POSTPONED
-- generate mon_hosts & mon_ip_addresses : POSTPONED
Make sure the primary ceph-mon is deployed before other nodes: TODO

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.