Check that RabbitMQ has no split brain into cluster

Due to some reasons RabbitMQ can reach split brain into cluster

Additional check into LMA toolchain Grafana and zabbix monitoring for RabbitMQ split brain can help to identify this issue in short time. Inoperative RabbitMQ affects whole cluster.

To check RabbitMQ split brain we can add a simple compare of two outputs:
from any controller:
# pcs status
and from each controller:
# rabbitmqctl cluster_status

root@node-1:~# pcs status
 Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server]
     Masters: [ node-1.domain.local ]
     Slaves: [ node-2.domain.local node-4.domain.local ]

# rabbitmqctl cluster_status
Cluster status of node 'rabbit@node-1' ...
[{nodes,[{disc,['rabbit@node-1','rabbit@node-2','rabbit@node-4']}]},
 {running_nodes,['rabbit@node-2','rabbit@node-4','rabbit@node-1']},
 {cluster_name,<<"<email address hidden>">>},
 {partitions,[]}]
...done.

cluster_name for "rabbitmqctl cluster_status" on any controller should be the same, and it also should be equal to
"pcs status, master_p_rabbitmq-server Masters: NODE-XXX"

Example:
(pcs status, master_p_rabbitmq-server Masters: [ node-1.domain.local ])=={cluster_name,<<"<email address hidden>">>}

node-1 into both outputs is a Master.