Peer Discovery through Metadata

Registered by justinsb on 2014-01-23

(Updated based on ML feedback, and to include proposed API)

When running a clustered service on Nova, typically each node needs to find its peers. In the physical world, this is typically done using multicast. On the cloud, we either can't or don't want to use multicast. The proposed feature would add new functionality to the metadata service to allow bootstrapping cross-instance discovery: instances could POST data to the metadata service that should be shared, and they can GET this data for any other instances in the same project as the current instance. I expect this data will likely be an IP address and port. The clustered service would then use this information to try to connect to suitable peers and start a full gossip protocol.

This can likely be done as an extension to: We could add either an endpoint for shared data, or just support a 'shared' query parameter or JSON key.


Example API


request / response as per

A “shared” query parameter can also be passed. If the value is ‘project’ (?), the specified metadata keys passed will be made available to all other instances in the same (keystone) project. Other values are not currently supported and will therefore return a 4xx (?) error.

Note that because entries cannot be changed once set, the sharing on the entries also cannot be changed once set.

Instances can retrieve shared keys by requesting the shared.json (?) endpoint. The shared entries visible to the current instances (including any shared by the instance itself) are returned. For symmetry with the unshared case, the from_server_ prefix is included e.g.

JSON Response
    "meta": {
        "from_server_key_1": “value1_server1”,
        "from_server_key_2": "value2_server1”,
    "uuid": "d8e02d56-2648-49a3-bf97-6be8f1204f38"
    "meta": {
        "from_server_key_1": “value1_server2”,
        "from_server_key_2": “value2_server2”,
    "uuid": “12345678-1234-5678-1234-123456780123”

If no shared values are visible, an empty response is returned:



* Expose a per-instance keystone token, and thus make it easy for the instance to query the normal nova api endpoint. That's a good idea (and I'd love to see it implemented), but is a lot more complicated. It would also put the burden on the API user. This blueprint would solves the use case a lot more directly & quickly

* Automatically expose IP addresses of other instances in the project / security group etc. I like this (it was my first inclination), but there are security concerns about exposing information about other instances, even within a project. This BP is only useful if deployed, so we can't go that route.

Blueprint information

John Garbutt
Needs approval
Series goal:
Milestone target:
Started by
John Garbutt on 2014-01-28

Related branches



Might be worth redrafting to take account of the ML discussion. Can we just have a quick API example included, just so the documentation team can use this blueprint to do what they need to do. Also, just to copy the other endpoints of /user_data, /vendor_data, why not go for /peer_data? --johnthetubaguy

Gerrit topic:,topic:bp/first-discover-your-peers,n,z

Addressed by:
    Support peer discovery through the metadata service

I'd like to sign off on this one, as well, before approved. I posted some initial feedback on the dev ML. --russellb

deferred from icehouse-3 to "next":

Removed from next, as next is now reserved for near misses from the last milestone --johnthetubaguy


Work Items

This blueprint contains Public information 
Everyone can see this information.