Scheduler: Utilization aware scheduling infrastructure.

Registered by Don Dugger

There are situations where it is desirable to be able to schedule VMs based upon transient resource usage beyond the current reliance on specific metrics like memory usage and CPU utilization. Advanced scheduling decisions can be made based upon enhanced usage statistics encompasing things like memory cache utilization, memory bandwidth utilization, network bandwith utilization or other, currently undefined metrics, that might be available in future platforms. This bleprint will provide an extensible framework that can be used to take advantage of current and future platform utilization metrics.

Blueprint information

Status:
Complete
Approver:
Russell Bryant
Priority:
Low
Drafter:
Don Dugger
Direction:
Approved
Assignee:
Don Dugger
Definition:
Approved
Series goal:
Accepted for icehouse
Implementation:
Implemented
Milestone target:
milestone icon 2014.1
Started by
Russell Bryant
Completed by
John Garbutt

Related branches

Sprints

Whiteboard

this bp seems like this one: https://blueprints.launchpad.net/nova/+spec/utilization-based-scheduling ---wangpan

Gerrit topic: https://review.openstack.org/#q,topic:bp/utilization-aware-scheduling,n,z

Addressed by: https://review.openstack.org/35759
    Stored metrics into compute_nodes as a json dictionary

Addressed by: https://review.openstack.org/35760
    Added plugin (e.g. CPU) to monitor and collect resource data

Addressed by: https://review.openstack.org/35764
    Extend the scheduler HostState for metrics from compute_node

Addressed by: https://review.openstack.org/35765
    Added a new scheduler resource weight plugin.

Addressed by: https://review.openstack.org/35767
    Make compute support plugins and store metrics in DB.

Addressed by: https://review.openstack.org/44007
    Added plugin (e.g. network) to monitor and collect resource data

Addressed by: https://review.openstack.org/58435
    Corrected typo in host_manager

Addressed by: https://review.openstack.org/61926
    Added a new scheduler filter for metrics

Addressed by: https://review.openstack.org/62813
    MetricsWeigher: Added support of unavailable metrics

Addressed by: https://review.openstack.org/64403
    Added temperature monitor for compute node

Addressed by: https://review.openstack.org/64404
    Added power monitor for compute node

Addressed by: https://review.openstack.org/63689
    Abstract add_timestamp out of ComputeDriverCPUMonitor class

Addressed by: https://review.openstack.org/65631
    Add node manager to read power/temperature of compute node

Addressed by: https://review.openstack.org/73139
    Change parameters of add_timestamp in ComputeDriverCPUMonitor class

Merged (Havanna):
===============

https://review.openstack.org/#/c/35759/
https://review.openstack.org/#/c/35764/
https://review.openstack.org/#/c/35765/
https://review.openstack.org/#/c/35767/
https://review.openstack.org/#/c/58435/

Merged (Icehouse):
===============

https://review.openstack.org/#/c/35760/
https://review.openstack.org/#/c/61926/
https://review.openstack.org/#/c/62813/
https://review.openstack.org/#/c/63689/

Pending:
=======

https://review.openstack.org/#/c/44007/
https://review.openstack.org/#/c/65631/
https://review.openstack.org/#/c/64403/
https://review.openstack.org/#/c/64404/
73139

Seems like all the above could be part of an "add-useful-metrics" blueprint(s). Looks like the core of this is now merged --johnthetubaguy (March 4th 2014)

Apologies, this missed the deadline for Feature Freeze. Marking this one as Implemented, so please open a new blueprint for the remaining patches. Please rebase patches as soon as Juno opens, and we will try to get this in during that period. --johnthetubaguy (5th March 2014)

I created a new blueprint "add-useful-metrics" for Juno, and all pending patches can fit into that blueprint. -- shane-wang (April 23, 2014)

Gerrit topic: https://review.openstack.org/#q,topic:bp/add-useful-metrics,n,z

Addressed by: https://review.openstack.org/89766
    Add useful metrics into utilization based scheduling

Gerrit topic: https://review.openstack.org/#q,topic:bp/add-utilization-based-weighers,n,z

Addressed by: https://review.openstack.org/90647
    Add utilization based weighers on top of MetricsWeigher

(?)

Work Items