NVP Plugin: decouple API calls from backend operations

Registered by Salvatore Orlando

Currently the NVP plugin receives requests from the Quantum API layer, performs all the relevant operations on the NVP plugin, and then updates the database when they're successfully completed.

This has a few disadvantages:
- GET operations always hit the NVP platform, and the only info retrieved from there which is not already in the Quantum DB is the status of the resource; this makes GET operations considerably slow.
- POST/PUT/DELETE operations do not return if the operations on NVP are not completed. Some are fairly slow
- There is a lot of orchestration code in the plugin logic
- The plugin has to explicitly deal with rolling back NVP resources in case of failures

The above listed disadvantages cause several downsides in the NVP plugin:
- excessive chattiness between Quantum and NVP
- slow API execution (especially when VMs are booted)
- hard to maintain (and hence error prone) code

For this reason we are proposing to migrate to a different model, which shall have the following benefits:

- All operations on Quantum terminate on the Quantum DB and responses are immediately returned to the user
- Resources' statuses are retrieved asynchrously (periodic tasks) - the backend might also provide notifications; appropriate mechanisms might be put in place to ensure 'stale' state information are never returned.
- Extremely reduced chattines between Quantum and NVP
- Simplified code base, with the complexity moved away from the plugin and split across several modules
- possibility of scaling out the Quantum plugin backend.

To this aim POST/PUT/DELETE operations will be performed asynchronously on the NVP backend. It will be up to the plugin ensuring that there's no config mismatch between Quantum and NVP, and if there is, the mismatch should be reported to the user when querying the objects/

More details about the design and the implementation plan will follow.
This blueprint is likely to be split in multiple blueprints after the design will be committed.

Blueprint information

Status:
Complete
Approver:
Aaron Rosen
Priority:
Undefined
Drafter:
Salvatore Orlando
Direction:
Approved
Assignee:
Salvatore Orlando
Definition:
Superseded
Series goal:
None
Implementation:
Deferred
Milestone target:
milestone icon next
Completed by
Armando Migliaccio

Related branches

Sprints

Whiteboard

I know it's a bit of a divergence, but we've been working on https://github.com/jkoelker/quark which ends up addressing a lot of the points above. We could compare notes from our implementation or something more in depth, but so far, Quark works well for us in testing. - Matt Dietz

----
This will be deferred to icehouse. If a big, scary change, and it's not a good idea to risk it in the 3rd milestone.
----
Update 2014-03-02
The blueprint will be completed in J, so there's no need to keep in the I-3 list

Gerrit topic: https://review.openstack.org/#q,topic:bp/nvp-async-backend-communication,n,z

Addressed by: https://review.openstack.org/54578
    Include lswitch id in NSX plugin port mappings

Addressed by: https://review.openstack.org/54579
    Add DB mappings with NSX logical switches

Addressed by: https://review.openstack.org/54584
    Add DB mappings with NSX logical routers

Gerrit topic: https://review.openstack.org/#q,topic:bp/nsx-async-backend-communication,n,z

Addressed by: https://review.openstack.org/71692
    DB Mappings for NSX security groups

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.