Asynchronous Processing in Glance (partial)

Registered by Mark Washenberger

Provide and use an internal library api for asynchronous processing in glance, to do things like image imports (asynchronously processing a copy-from). This can later be used to support image conversion and metadata extraction, though those should fall under other blueprints. A simple eventlet thread-based implementation may be provided, alongside something more like a remote rpc queue worker. There should be room for the implementation of the library api to be backed by other cool drivers, like ZeroVM (

Blueprint information

Nikhil Komawar
Series goal:
Accepted for juno
Milestone target:
milestone icon 2014.2
Started by
Mark Washenberger
Completed by
Thierry Carrez

Related branches



Targeting to h2 based on how centrally it figures in other blueprints

discussion items:-

requirements at :-

Gerrit topic:,topic:bp/async-glance-workers,n,z

Addressed by:
    Enable tasks data model and table for async worker

Addressed by:
    Adds tasks to db api

Addressed by:
    Adds domain level support for tasks

Addressed by:
    Introduce Task Info Table

Addressed by:
Add support for PartialTask list

Addressed by:
    Enable tasks REST API for async worker

Addressed by:
    Adds executor interface and example import script

Addressed by:
    Documentation for Glance tasks

How much work is left here? Should this be moved out from i-3?
-- flaper87 2014-02-18
(Response provided on IRC chat - attaching the same herewith temporarily for tracking purposes)
11:13:04 [ nikhil__] flaper87: hi there
11:14:25 [ flaper87] nikhil__: hey
11:16:28 [ nikhil__] flaper87: I'm about to send out a plan to markwash for the async bp
11:16:50 [ nikhil__] we've a consistently working exeutor here at rackspace with import and export use cases
11:17:07 [ nikhil__] which can be used to build a nice one in short period of time
11:17:27 [ nikhil__] also, can share the details about the plan with you if interested
11:17:37 [ flaper87] nikhil__: awesome, yeah please. I am
11:17:39 [ nikhil__] (all this is in reference to you comment on the BP)
11:17:46 [ nikhil__] ohk sure
11:18:00 [ nikhil__] think we can definitely get this in i-3
11:18:17 [ nikhil__] on regular deadlines
11:18:32 [ nikhil__] some work is started at
11:19:18 * flaper87 clicks
11:19:52 [ flaper87] nikhil__: starred, I'll make sure I review that
11:20:09 [ flaper87] so, you think it is doable for i-3
11:20:10 [ flaper87] that's awesome
11:20:24 [ flaper87] I saw many other bps linked to the async-workers one
11:21:54 [ nikhil__] flaper87: yeah, based on discussion with markwash last week we planned async bp and import for i-3
11:22:07 [ flaper87] nikhil__: awesome!
11:22:19 [ nikhil__] and export to a little possibility
11:22:22 [ nikhil__] cool

This work would take a lot of input from this paste (which is Rackspace's internal proof of concept implementation as a placeholder for some work done on this BP) as well as from review feedback and suggestions provided by the community.

markwash untagged 2014-03-03

Addressed by:
    Ensures that task.message is of type unicode

Addressed by:
    Return 405 when attempting DELETE on /tasks

Addressed by:
    Adds TaskStub class

Addressed by:
    Get rid of TaskDetails in favor of TaskStub

Addressed by:
    Basic Eventlet Task Executor with taskflow

Latest discussion and updates from the Atlanta summit at


Work Items

Dependency tree

* Blueprints in grey have been implemented.