Multi-phase tasks

Registered by Joshua Harlow

Likely a common use case is that tasks will not all be able to be cleanly separated into small atomic isolated units/tasks. In order to accommodate this it would be nice to be able to have the default task be a multi-phase task. This will help for those use-cases where its not easily possible to rework all the code into small atomic isolated units/tasks and combine them together with flows.


A multi-phase task would be a task that can be split up into logical phases.

*For example*

Lets say a task is to make a directory (the mkdirtask) '/tmp/b/c':

The phases for this could be:
1. Make directory /tmp
2. Make directory /tmp/b
3. Make directory /tmp/b/c

Instead of having to compose 3 different tasks to accomplish this it would be nice to support having that task be one task with multiple phases, with a single revert() method (aka delete the directories that were created previously).

---- Interesting part ---

Its pretty simple to have a single task yield back multiple things, and track these multiple things as segements of the overall task and then on failure of the whole task it isn't to hard to give the task back information to let it resume from. It would be interesting if we could automatically put the task back into the last place, possibly by having the task register yielding functions and the resumption can be done by jumping to the last function that was occuring before the failure occured? This would be one way to do this.

The other problem is that if an exception occurs after the task yielded but before a call to the next() function (to yield again) we likely need to be able to use the yielding syntax to close() the task (allowing it to cleanup). This seems pretty easy.

--- The iffy part ---

You could imagine a task getting its inputs from other yielding tasks, similar to a task blocking on other coroutine based tasks. This could then allow for a task to stop halfway through and wait on something else. This would be neat, but would be heck of hard to resume or analyze (sphagetti anyone?). Basically openstack already has this type of thing via eventlet, so I am not sure we want to add on another level of this. It seems like heat did though, more info soon.

Blueprint information

Joshua Harlow
Joshua Harlow
Needs approval
Series goal:
Milestone target:
Completed by
Joshua Harlow

Related branches




Work Items

This blueprint contains Public information 
Everyone can see this information.