OpenStack Compute (Nova)

Scheduler: SolverScheduler - Complex constraint based resource placement

Registered by Debo~ Dutta on 2013-09-13

OpenStack scheduler has nice options like filters which allow resource selection based on simple constraints e.g. Don't put instances in a set on the same host. However for complex constraints, building a filter would be as complex as building a real constraint solver e.g. place VMs while minimizing average (VM-storage-bandwidth) with complex constraints. On the other hand, complex solvers are available in open source (PULP, CVXOPT, COIN_OR). Hence a natural solution to the above problem is to design a pluggable scheduler that leverages existing solvers. We believe that this will open new avenues for complex constraint (and objectives) based resource placement in large OpenStack deployments.

Collaborators: Yathi Udupi, Debo Dutta

Blueprint information

Status:
Started
Approver:
Joe Gordon
Priority:
Medium
Drafter:
Debo~ Dutta
Direction:
Needs approval
Assignee:
Yathiraj Udupi
Definition:
Pending Approval
Series goal:
None
Implementation:
Needs Code Review
Milestone target:
milestone icon next
Started by
Yathiraj Udupi on 2013-11-19

Related branches

Sprints

Whiteboard

Deferred to icehouse-3 as the blueprint was not approved by the icehouse-2 blueprint approval deadline. --russellb

Gerrit topic: https://review.openstack.org/#q,topic:bp/solver-scheduler,n,z

Addressed by: https://review.openstack.org/#/c/46588/
     Add a pluggable solver scheduler
This provides a solver scheduler, where complex constraints can be specified using pluggable solvers. An illustrative implementation of a solver is included and is based on PULP. Here the constraints are based on the disk space and memory requirements, and optimization is done using the host's free ram as a cost metric.

Addressed by: https://review.openstack.org/46588
     Add a pluggable solver scheduler

I'm not sure we want to add this right now since we're in the middle of making plans to split the scheduler out into a new repo. --russellb

russellb, why should the splitting out the scheduler effort block this? -- jogo

jogo, Can you please review this blueprint and get it approved for the next icehouse milestone target? The latest code patch now passes the JENKINS build and provides the base solver_scheduler driver and a reference implementation of a solver that is based on PULP. This patch is now ready for review. --Thanks, Yathi.

Addressed by: https://review.openstack.org/#/c/70654/
     Add a pluggable LP solver to the solver scheduler

Addressed by: https://review.openstack.org/70654
    Add a pluggable LP solver to the solver scheduler

Addressed by: https://review.openstack.org/74201
    Adding ram, disk, vcpu resource linear constraints

Addressed by: https://review.openstack.org/74216
    Adding the Volume Affinity Cost function

Addressed by: https://review.openstack.org/74261
    Add Active-Host Constraint to the Solver Scheduler

Addressed by: https://review.openstack.org/74264
    Add Max-Instances-Per-Host Constraint to the Solver Scheduler

Addressed by: https://review.openstack.org/74268
    Implements: blueprint solver-scheduler

Addressed by: https://review.openstack.org/74270
    Add One-Host-Per-Instance Constraint to the Solver Scheduler

Addressed by: https://review.openstack.org/74277
    Adding the Availability Zone constraint

Addressed by: https://review.openstack.org/74288
    Adding the IO Ops constraint

Apologies, this missed the deadline for Feature Freeze. Please rebase patches as soon as Juno opens, and we will try to get this in during that period. --johnthetubaguy (5th March 2014)

However seems like some of the approaches need to be discussed more, and we need a much clearer blueprint, with steps on what is being done, so we can try and stop blocking these changes after so much work has been done. Un-approving this one. Need more details on what configuration values will be added or altered, and how exisiting filters and constraints will be consolidated. --johnthetubaguy

-------
[Yathi - March 5 2014]
We would like to make a request for FFE for the Solver Scheduler work. A lot of work has gone into it since Sep’13, and the first patch has gone through several iteration after some reviews. The first patch - https://review.openstack.org/#/c/46588/ introduces the main solver scheduler driver, and a reference solver implementation, and the subsequent patches that are already added provide the pluggable solver, and individual support for adding constraints, costs, etc.

First Patch: https://review.openstack.org/#/c/46588/
Second patch with enhanced support for pluggable constraints and costs: - https://review.openstack.org/#/c/70654/
Subsequent patches add the constraints and the costs.

John,
Regarding the concerns you raised above to make this BP "pending Approval", the plan is not many new configuration values will be added, just the ones to specify the solver to use, and the pluggable constraints, and costs to use, with the weights for the costs. (these are mainly part of the second patch - https://review.openstack.org/#/c/70654/ Something similar to FilterScheduler, where the filters to be used are specified)

The plan is to gradually support the concepts for the existing filters as the constraints that are accepted by our Solver Scheduler. Depending on the constraints and the costs chosen, the final scheduling will be done by solving the problem as an optimization problem.

Please reconsider this blueprint, and allow a FFE.
---------

(?)

Work Items