Agent Group and Tooz Driver

Registered by Robin Wang on 2015-04-16

Neutron needs to maintain up-to-date information of its variant agents,
especially the health status of each agent, to apply proper operations on
them and provide stable services to end users. For example, schedulers
always make sure the agent is alive before assigning tasks. HA periodically
checks agent failure to decide whether trigger a recovery operation.

Currently, this is done by agent heartbeat message, and DB persistence
of heartbeat information. Agent periodically sends heartbeat message to
neutron-server(by default every 30 seconds). Heartbeat timestamp and agent
configuration information is parsed on server side and written into DB.
When scheduler or HA needs to get agent status, those records should be
retrieved from DB and compared with agent_down_time. Neutron server cost
is not trivial and the cost increases when scale gets bigger. This mechanism
is extremely inefficient and not scalable.

A much more lightweight, scalable and reliable agent status monitoring
and querying mechanism is desired for neutron. We would like to introduce
AgentGroup to neutron, which provides internal API for neutron agent
grouping, membership registration, health status check, group member
querying, etc. Modules such as failure detective, scheduler and HA will
use AgentGroup and its API.

There are many different backend which can be used to implement those
functionalities in AgentGroup. Instead of having neutron to maintain,
test, upgrade those code, we intend to introduce a tooz driver, and use
tooz library to manage/maintain 3rd party backend such as zookeeper,
memcached, redis, etc. Also, we should guarantee backward compatibility
for original DB-based mechanism. Since it uses tables and data model that is
intrinsically tied to the internal of neutron, tooz cannot take it over.
So a DB-based driver is needed for AgentGroup.

Please see spec for more details.

Blueprint information

Robin Wang
Needs approval
Robin Wang
Series goal:
Milestone target:
Completed by
Armando Migliaccio on 2015-11-09

Related branches



Nov-09-2015(armax): If someone is interested in pursuing it, this must be re-submitted according to guidelines defined in [1]



Gerrit topic:,topic:bp/agent-group-and-tooz-driver,n,z

Addressed by:
    Spec for: Agent Group and Tooz Driver


Work Items

This blueprint contains Public information 
Everyone can see this information.


No subscribers.