Datasources on the new bus

Registered by Alexander Yip

Splitting up datasources and policy engines to communicate over a bus requires that each policy engine get the entire snapshot of the datasource after the policy engine subscribes to the datasource, and then apply updates to the snapshot to stay in sync with the datasource.

The pub-sub channel does not provide the sender a way to control who receives each update, so the sender cannot coordinate sending the snapshot first and then following with the updates. Instead the sender can only send updates continuously and send a snapshot out of band. To give the recipient a way to know when to start applying updates to the snapshot, the sender must assign sequence numbers to each update, and also assign a sequence number to the snapshot.

This communication pattern called the "Clone" pattern in the zeromq manual.
See http://zguide.zeromq.org/php:chapter5#Reliable-Pub-Sub-Clone-Pattern

Blueprint information

Status:
Complete
Approver:
None
Priority:
High
Drafter:
Alexander Yip
Direction:
Approved
Assignee:
Eric K
Definition:
Approved
Series goal:
Accepted for newton
Implementation:
Implemented
Milestone target:
milestone icon newton-2
Started by
Eric K
Completed by
Tim Hinrichs

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/dist-datasources-on-bus,n,z

Addressed by: https://review.openstack.org/268360
    [WIP] datasource pub-sub on new bus

Gerrit topic: https://review.openstack.org/#q,topic:bug/1544139,n,z

Addressed by: https://review.openstack.org/304991
    [WIP] dse2 sequenced, differential pub-sub

Addressed by: https://review.openstack.org/356806
    DSE2 - resubscribe if update missing for too long

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.