add conf for number of condutor workers

Registered by Peter Feiner

Nova-conductor processes access the database directly. Since the database
connection driver is typically implemented in a library beyond the purview of
eventlet’s monkeypatching (i.e., a native python extension like _mysql.so),
blocking database calls will block all eventlet coroutines. Since much of what
nova-conductor does is access the database, a nova-conductor process’s handling
of requests is effectively serial.

To mitigate this problem, you can simply run more nova-conductor processes.
Deploying multiple conductor workers per host avoids serialization on database
accesses caused by libmysqlclient.so blocking eventlet's single thread. In an
experiment on a 24-core machine, when creating 20 VMs in parallel, maximum
creation time was reduced by approx. 10s when using 20 conductor processes
vis-a-vis a single conductor process. Profiling showed that all of the savings
came from faster calls into nova.db.sqlalchemy.api.

To make running multiple nova-conductor processes straightforward, there should
be a workers=N option in the [conductor] section of nova.conf -- just like the
osapi_compute_workers=N flag in the [DEFAULT] section.

Blueprint information

Status:
Complete
Approver:
Russell Bryant
Priority:
Medium
Drafter:
None
Direction:
Approved
Assignee:
Peter Feiner
Definition:
Approved
Series goal:
Accepted for havana
Implementation:
Implemented
Milestone target:
milestone icon 2013.2
Started by
Russell Bryant
Completed by
Russell Bryant

Related branches

Sprints

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.