Multiple External Ceph Support

Registered by John Fulton on 2019-09-11

This blueprint proposes that TripleO get the ability to configure a single overcloud to use an arbitrary number of external ceph clusters.

TripleO can currently configure the overcloud to use only one external ceph cluster via the environments/ceph-ansible/ceph-ansible-external.yaml [1] environment file. It is incumbent on the external Ceph cluster administrator to create a keyring [2] which can access a pool and to then provide the key for keyring, the list of Ceph monitor IP addresses, and the Ceph cluster FSID. The TripleO user may then use these parameters as input to the ceph-ansible-external.yaml [1] environment file and then start the overcloud deployment. This results parameters getting passed to the ceph-ansible client role and a ceph configuration file being created in /etc/ceph/_cluster_name_.conf containing the CephClusterFSID and CephExternalMonHost and a key file [2] being created for that Ceph cluster.

If TripleO could support multiple Ceph clusters, then a new paramter like the following would be added:

CephMultiBackendsHash:
    ceph1:
      CephClusterFSID: '4b5c8c0a-ff60-454b-a1b4-9747aa737d19'
      CephClientKey: 'AQDLOh1VgEp6FRAAFzT7Zw+Y9V6JJExQAsRnRQ=='
      CephExternalMonHost: '172.16.1.7, 172.16.1.8'
    ceph2:
      CephClusterFSID: 'a7f64266-0894-4f1e-a635-d0aeaca0e993'
      CephClientKey: 'secretsecretsecretsecretsecretsecret33=='
      CephExternalMonHost: '10.1.1.66, 10.1.1.77'
      CinderRbdPoolName: cinder
    ...
    cephN:

This would then result in TripleO running ceph-ansible for the ceph-client role N times with the same parameters for each run except different parameters would be passed per run if it's included in the cluster name's hash. E.g. the FSID would be different for the cluster named ceph1. These runs would result in N keyring files getting created in the appropriate directory, N Ceph conf files like /etc/ceph/{ceph1,ceph2,...,cephN}.conf, and so on. It is then incumbent on other tools to configure different OpenStack services which use Ceph to use that backend; e.g. the cinder.conf could have entries for more than one backend and those backends could rely on the ceph client configuration implemented by this blueprint.

[1] https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/environments/ceph-ansible/ceph-ansible-external.yaml
[2] https://docs.ceph.com/docs/master/man/8/ceph-authtool/

Blueprint information

Status:
Complete
Approver:
wes hayutin
Priority:
Undefined
Drafter:
John Fulton
Direction:
Approved
Assignee:
John Fulton
Definition:
Approved
Series goal:
None
Implementation:
Implemented
Milestone target:
None
Started by
John Fulton on 2020-01-02
Completed by
John Fulton on 2020-02-07

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.opendev.org/#/q/topic:ceph_extra_keys

Addressed by: https://review.opendev.org/700947
    WIP/DNM: Introduce CephExtraKeys

Addressed by: https://review.opendev.org/701261
    Introduce CephExtraKeys

Gerrit topic: https://review.opendev.org/#/q/topic:bp/multiple-external-ceph

Addressed by: https://review.opendev.org/702141
    WIP/DNM: Introduce CephMultiBackendsHash

Addressed by: https://review.opendev.org/702143
    WIP/DNM: Add support for ceph_multi_backends_hash

Gerrit topic: https://review.opendev.org/#/q/topic:bug/1861488

Addressed by: https://review.opendev.org/707847
    Document CephExternalMultiConfig and CephExtraKeys

Addressed by: https://review.opendev.org/708512
    Add support for ceph_external_multi_config

Addressed by: https://review.opendev.org/708513
    Introduce CephExternalMultiConfig

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.