API service/worker architecture for async operation

Registered by Murali Allada on 2013-11-26

This blueprint discusses a few possible options for the API service workflow and architecture.

The objective is to design the API worker architecture so that it is extensible,
can scale independently and can be deployed independently. The idea is to quickly have an implementation which can support a wide variety of use cases.

Proposed architecture:
-----------------------------

The API service should be capable of abstracting the interface with various clients. In essence provide a view to the client. The control layer (controller class in solum) can be used for that. Authentication would happen here. Request validations, transformations, etc would happen here too. The controller passes on requests to the dispatcher.

The Dispatch layer will encapsulate the routing mechanism required to get the request to the right handler for fulfillment. This could be with a direct call to a known method in the handler class (first version), or by placing a message on a queue which the handler can read (for scaling, and handling requests asynchronously). RabbitMQ or Oslo Messaging [1], can be used for queuing messages.

Handlers are responsible for fulfilling the request.

1) | Controller |------->| Dispatcher |-------->| Handler |-------> Database

2) | Controller |------->| Dispatcher |-------->| queue |--------->| Handler |------> Database

Comments:
---------------

- To start things off, we can implement workflow #1 shown above and make all requests synchronous.

[1] https://wiki.openstack.org/wiki/Oslo/Messaging. I think Oslo messaging is used in other openStack projects.

Blueprint information

Status:
Complete
Approver:
Adrian Otto
Priority:
Essential
Drafter:
Murali Allada
Direction:
Approved
Assignee:
Murali Allada
Definition:
Discussion
Series goal:
Accepted for icehouse
Implementation:
Implemented
Milestone target:
milestone icon 2014.1.1
Started by
Adrian Otto on 2013-12-17
Completed by
Murali Allada on 2014-05-28

Related branches

Sprints

Whiteboard

Follow the discussion about this blueprint here:

https://www.mail-archive.com/search?<email address hidden>&q=subject:%22Re%3A+%5Bopenstack-dev%5D+%5BSolum%5D+API+worker+architecture%22

Decisions made so far:

Start with support for synchronous request processing. Async support can be added later.

Gerrit topic: https://review.openstack.org/#q,topic:bp/below,n,z

Addressed by: https://review.openstack.org/62466
    API worker framework

For the Ice house release, we've implemented a simpler architecture.

| Controller |-------->| Handler |-------> |Database|

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.