Locally run independent parts of workflows at the same time

Registered by Joshua Harlow

When a workflow has components that can be ran simulatentously locally we should attempt to do so, for example lets take the following workflow.

A ---- B ---- C
                 | ---- Z
D ---- E ------

Notice Z can not be ran before "D-E" have finished and "A-B-C" have finished but both of those 2 (D-E and A-B-C) could be ran at the same time since they have no dependence on each other. It would be interesting to see how we can accomodate this type of pattern without having to bring in the more heavy weight distributed pattern (which should also solve this).

Perhaps by determing the independent tasks (maybe by finding topological sort of the graph and then examing the topological sort to determine what can be ran in parallel? Maybe the subgraphs in the workflows - https://en.wikipedia.org/wiki/Glossary_of_graph_theory#Subgraphs might be useful also?)

After finding these 'groups' then run each group using greenthreads/threads and stop the activation of other 'groups' until those initial groups have finished (and repeat until all groups have ran).

Blueprint information

Status:
Complete
Approver:
Joshua Harlow
Priority:
Medium
Drafter:
Joshua Harlow
Direction:
Needs approval
Assignee:
Joshua Harlow
Definition:
Obsolete
Series goal:
None
Implementation:
Needs Code Review
Milestone target:
None
Started by
Joshua Harlow
Completed by
Joshua Harlow

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/locally-run-many-at-once,n,z

Addressed by: https://review.openstack.org/34488
    Add a locally running threaded flow.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.