Redesign Mistral Scheduler

Registered by Renat Akhmerov on 2018-07-12

Based on the etherpad https://etherpad.openstack.org/p/mistral-ptg-rocky

...
It uses too agressive polling, it creates significant load on DB
Due to agressive DB polling in Scheduler Mistral can't be scaled well. Every new engine node increases load on DB
There are situations when a delayed call (the entity that the Scheduler works with) is deleted from DB but, in fact, is not processed yet. It leaves the system in an inconsistent state (a WF may get stuck in RUNNING state)
...

Blueprint information

Status:
Complete
Approver:
Dougal Matthews
Priority:
High
Drafter:
Renat Akhmerov
Direction:
Approved
Assignee:
Renat Akhmerov
Definition:
New
Series goal:
Accepted for stein
Implementation:
Implemented
Milestone target:
milestone icon train-1
Started by
Renat Akhmerov on 2018-07-16
Completed by
Renat Akhmerov on 2019-12-23

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/mistral-redesign-scheduler,n,z

Addressed by: https://review.openstack.org/589015
    WIP: designing a new scheduler

Gerrit topic: https://review.opendev.org/#/q/topic:bp/mistral-redesign-scheduler

Addressed by: https://review.opendev.org/666954
    WIP: Integrating the new scheduler into the system

Addressed by: https://review.opendev.org/669558
    Add a migration to create the scheduled_jobs table

Addressed by: https://review.opendev.org/669559
    Add db api tests for scheduled jobs

Addressed by: https://review.opendev.org/669560
    Allow to delete multiple objects with advanced filters

Addressed by: https://review.opendev.org/669561
    WIP: Bulk update and delete of stored scheduled jobs

Addressed by: https://review.opendev.org/676628
    WIP: Improve new scheduler

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.