Redesign RabbitMQ Pacemaker resource as multi-master with disc/ram nodes

Registered by Bogdan Dobrelya

The RabbitMQ cluster is configured as active-active cluster with mirrored queues with all
rabbit nodes running as a disc ones. But it would be nice to make some nodes be able to
run as RAM nodes.

But the current implementation of the pacemaker resource for RabbitMQ cluster has a single master node limitation (note, we refer to the master node as the one to whom the other rabbit nodes join dynamically), hence we cannot assign, for example, the single master to be a disc node and slaves - to be ram nodes as that would put the data integrity at the risk. There are should be more than 1 master to make this work.

The proposed changes:
* Introduce multi-master logic for existing Rabbitmq resource agent OCF script. The criteria of master promoting should be 1) rabbit node joined to the cluster, 2) rabbit node synced all of its data to the slave-pids, 3) there is at least 1 disc node exist in the cluster and is running as a Master resource.
* Introduce the random join delay for rabbit nodes to prevent simultanous replicas and master elections race conditions
* Introduce rabbit disc node status for elected mulstistate resource masters and rabbit ram node status for joining multistte resource slaves

Alternative is to deprecate mirrored clustering and replace it by shovel, see
http://lists.openstack.org/pipermail/openstack-dev/2015-June/066136.html

Blueprint information

Status:
Complete
Approver:
Vladimir Kuklin
Priority:
Undefined
Drafter:
Bogdan Dobrelya
Direction:
Needs approval
Assignee:
Bogdan Dobrelya
Definition:
Obsolete
Series goal:
Accepted for future
Implementation:
Deferred
Milestone target:
milestone icon 7.0
Completed by
Bogdan Dobrelya

Related branches

Sprints

Whiteboard

Feature Lead: Bogdan Dobrelya
Developers: Bogdan Dobrelya, Vladimir Kuklin, Alex Khivin
Mandatory Design Reviewers: Vladimir Kuklin, Alex Khivin
QA: Andrey Sledzinskiy

Gerrit topic: https://review.openstack.org/#q,topic:108792,n,z

Addressed by: https://review.openstack.org/108792
    RabbitMQ node resource level fencing

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.