Asynchronously process tasks using celery

Registered by Adrian Turjak

Some tasks contain enough actions and work that they take a long time to respond and may time out before returning. This won't affect the task itself, since it should still complete before attempting to return, but the user may be confused by this and while this task is processing an API worker is blocked.

The majority of tasks could be processed asynchronously, but this would mean that tasks don't respond at all beyond basic validation and you must query for their status. This could be considered an API breaking change and require an introduction of a v2 API.

The work here would be to move task processing to celery workers. This logic is currently handled in the TaskView class and could easily enough be moved to a celery worker construct. We can then tag certain tasks as async or not, and for existing tasks we can effectively make it so that the Views wait until a worker completes before they respond.

Ideally we should split the API and task processing, so all tasks are handled asynchronously. A task is started, and will respond with a 202. User can then query for status, although Adjutant will send emails and such as the task gets processed. We could also introduce notification support of some sort for task status.

Work items vaguely include:
- move task processing to celery workers
- open task list and task query APIs to users with sensible permissions and filtering for non-admins
- create notifications for tasks as they are processed

Related to this work is may be the redefinition of taskviews and the current way we use them (for more than outright task processing).

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
Adrian Turjak
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.