ostf-rabbit-replication-tests

Registered by Tatyanka

Finally , we have enabled rabbit management plagins on slaves, so we can easy to check rabbit health(replication / exanchges/ etc) in OSTF
Cases fro bdodrelya
RabbitMQ OSTF test performs only "rabbitmqctl cluster_status" check . No functionality is checked by OSTF, so it never covers pacemaker logic concerning assembly rabbitmq cluster.

There are some situations when OSTF doesn't reflect actual rabbitmq status:
- "rabbitmqctl cluster_status" shows that all nodes are running but pacemaker ocf script hasn't opened 5673 port in iptables yet (or there is remained an extra iptables rule that blocks 5673 port);
- "rabbitmqctl cluster_status" shows that all nodes are running but pacemaker is just checking if the rabbitmq starts (start phase) and it is going to shutdown rabbitmq before performing further steps;
- "rabbitmqctl cluster_status" shows that all nodes are running but it is still inaccessible thru haproxy because of haproxy, vip__management, network or any other issue. In this case rabbitmq looks like a nonworking for other services.

We want to make sure that rabbitmq is successfully assembled by pacemaker and ready to serve requests from other services.
The better way would be perform creating some queue and sending some test messages, but it is not realized in OSTF yet (please see https://gist.github.com/Mihalych/04bccd950361ac3c760f script for it).

Also, sometimes rabbitmq cluster looks healthy, but "rabbitmqctl list_channels" hangs and Openstack services cannot use channels as well. I believe this case should be also addressed.

Blueprint information

Status:
Complete
Approver:
Tatyanka
Priority:
High
Drafter:
Tatyanka
Direction:
Approved
Assignee:
Tatyanka
Definition:
Approved
Series goal:
Accepted for 6.1.x
Implementation:
Implemented
Milestone target:
milestone icon 6.1
Started by
Tatyanka
Completed by
Tatyanka

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/ostf-rabbit-replication-tests,n,z

Addressed by: https://review.openstack.org/148520
    Add additional test for verification of rabbitmq

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.