Enable nova use filters/weight from oslo

Registered by Guangya Liu (Jay Lau) on 2013-11-10

Now in oslo, we already put some scheduler filters/weights logic there and cinder is using oslo scheduler filters/weights logic, seems we want both nova&cinder use this logic in future.

It is better to enable nova use filter/weight logic from oslo and also use entry point to handle all filters/weight logic.

We can first sync from oslo with following filteres/weights:
1) availablity_zone_filter.py
2) capabilities_filter.py
3) extra_specs_ops.py
4) json_filter.py
5) retry_filter.py
6) base_filter.py
7) base_handle.py
8) base_weight.py

The last work item is enable nova use entry point to handle scheduler filters/weights.

Blueprint information

Status:
Started
Approver:
Russell Bryant
Priority:
Undefined
Drafter:
None
Direction:
Needs approval
Assignee:
Guangya Liu (Jay Lau)
Definition:
Drafting
Series goal:
None
Implementation:
Started
Milestone target:
None
Started by
Guangya Liu (Jay Lau) on 2014-01-07

Related branches

Sprints

Whiteboard

@jay-lau-513 The question in my mind is: if there is only one scheduler in OpenStack, not a separate one in Nova and Cinder, then what's the value of pulling common code into Oslo? The code in Oslo will be used by only one project. --alaski

I'm not sure I see the benefit of filters/weights in oslo with the move to Gantt. Ultimately Nova and Cinder can share a scheduler rather than sharing common code in two separate schedulers. Is there something I'm missing? --alaski

@alaski: I do agree with you. I do not see the benefit of this blueprint when Gantt is aready going on. --aloga

@aloga,do you mean we need do this in Gantt?

I cannot see the benefit of first syncing the code from nova to oslo (since the scheduler in oslo is obsolete), then to gantt, when gantt has already begun and it has forklifted the code from nova (and it is being kept in sync with nova). -- aloga

@aloga, scheduler in oslo is obsolate? Where can I get some info for this? We do not need to maintain scheduler in Oslo even for some common util? The final goal of this bp is enable nova scheduler or Gantt using entry point to manage filters and weights as it is more flexible.

@Andrew, my original thinking is that we can directly enable Gantt scheduler use oslo utils, but as now Gantt is not ready for such code change so what I want to do is first do this in nova and then sync up to Gantt as most of the code in Gantt are get from nova. What do you think. --jay-lau-513

Deferred to icehouse-3 as the blueprint was not approved by the icehouse-2 blueprint approval deadline. --russellb

You say "Move all filter logic to nova/openstack/common/scheduler". Can you clarify exactly what would be moved, and what from oslo would be used by nova? I would to make sure the scope of this blueprint is clear before approving it. --russellb
@Russell, I have updated the bp, please help check. Thanks. --jay-lau-513

I think you should take into account this BP https://blueprints.launchpad.net/nova/+spec/normalize-scheduler-weights and make it dependant. When it will be merged it should be taken into account also by the weighers in cinder. -- aloga

In principle I like the idea, but it would be good to see what the impact will be, maybe some code showing the direction will help review this? --johnthetubaguy

With Icehouse-1 just over two weeks away (December 5th), moving this to Icehouse-2 since it hasn't been started yet. If it does get into Icehouse-1, we can re-target the BP. --jogo

Thanks John and Joe. i'm OK to set the milestone to i-2. Will provide patch later for your review. --jay-lau-513

Will this BP collide with https://blueprints.launchpad.net/nova/+spec/forklift-scheduler-breakout ?

Seems not much relationship, https://blueprints.launchpad.net/nova/+spec/forklift-scheduler-breakout want to do a "unified scheduler" for many projects such as nova/cinder/neutron/ironic etc, but this bp is just want to leverage oslo in nova scheduler. forklift-scheduler-breakout should be a long term roadmap which may need more discussion. Thanks. --jay-lau-513

@Russell, do you think that we need to proceed on nova or move to gantt? For long term, I think that I can proceed this to gantt and leave nova as it is. Thanks. --jay-lau-513

Gerrit topic: https://review.openstack.org/#q,topic:bp/nova-oslo-scheduler,n,z

Addressed by: https://review.openstack.org/65198
    Split AvailabilityZoneFilter to two filters

This bp depends on some oslo change: https://review.openstack.org/#/c/65418/ https://review.openstack.org/#/c/65424/ --jay-lau-513

Addressed by: https://review.openstack.org/66105
    Enable nova scheduler use entry point for weight

Addressed by: https://review.openstack.org/66285
    Change weight_multiplier() to _weight_multiplier() for weight

deferred from icehouse-3 to "next": http://lists.openstack.org/pipermail/openstack-dev/2014-February/026335.html

Removed from next, as next is now reserved for near misses from the last milestone --johnthetubaguy@jay-lau-513 The question in my mind is: if there is only one scheduler in OpenStack, not a separate one in Nova and Cinder, then what's the value of pulling common code into Oslo? The code in Oslo will be used by only one project. --alaski

I'm not sure I see the benefit of filters/weights in oslo with the move to Gantt. Ultimately Nova and Cinder can share a scheduler rather than sharing common code in two separate schedulers. Is there something I'm missing? --alaski

@alaski: I do agree with you. I do not see the benefit of this blueprint when Gantt is aready going on. --aloga

@aloga,do you mean we need do this in Gantt?

I cannot see the benefit of first syncing the code from nova to oslo (since the scheduler in oslo is obsolete), then to gantt, when gantt has already begun and it has forklifted the code from nova (and it is being kept in sync with nova). -- aloga

@aloga, scheduler in oslo is obsolate? Where can I get some info for this? We do not need to maintain scheduler in Oslo even for some common util? The final goal of this bp is enable nova scheduler or Gantt using entry point to manage filters and weights as it is more flexible.

@Andrew, my original thinking is that we can directly enable Gantt scheduler use oslo utils, but as now Gantt is not ready for such code change so what I want to do is first do this in nova and then sync up to Gantt as most of the code in Gantt are get from nova. What do you think. --jay-lau-513

Deferred to icehouse-3 as the blueprint was not approved by the icehouse-2 blueprint approval deadline. --russellb

You say "Move all filter logic to nova/openstack/common/scheduler". Can you clarify exactly what would be moved, and what from oslo would be used by nova? I would to make sure the scope of this blueprint is clear before approving it. --russellb
@Russell, I have updated the bp, please help check. Thanks. --jay-lau-513

I think you should take into account this BP https://blueprints.launchpad.net/nova/+spec/normalize-scheduler-weights and make it dependant. When it will be merged it should be taken into account also by the weighers in cinder. -- aloga

In principle I like the idea, but it would be good to see what the impact will be, maybe some code showing the direction will help review this? --johnthetubaguy

With Icehouse-1 just over two weeks away (December 5th), moving this to Icehouse-2 since it hasn't been started yet. If it does get into Icehouse-1, we can re-target the BP. --jogo

Thanks John and Joe. i'm OK to set the milestone to i-2. Will provide patch later for your review. --jay-lau-513

Will this BP collide with https://blueprints.launchpad.net/nova/+spec/forklift-scheduler-breakout ?

Seems not much relationship, https://blueprints.launchpad.net/nova/+spec/forklift-scheduler-breakout want to do a "unified scheduler" for many projects such as nova/cinder/neutron/ironic etc, but this bp is just want to leverage oslo in nova scheduler. forklift-scheduler-breakout should be a long term roadmap which may need more discussion. Thanks. --jay-lau-513

@Russell, do you think that we need to proceed on nova or move to gantt? For long term, I think that I can proceed this to gantt and leave nova as it is. Thanks. --jay-lau-513

Gerrit topic: https://review.openstack.org/#q,topic:bp/nova-oslo-scheduler,n,z

Addressed by: https://review.openstack.org/65198
    Split AvailabilityZoneFilter to two filters

This bp depends on some oslo change: https://review.openstack.org/#/c/65418/ https://review.openstack.org/#/c/65424/ --jay-lau-513

Addressed by: https://review.openstack.org/66105
    Enable nova scheduler use entry point for weight

Addressed by: https://review.openstack.org/66285
    Change weight_multiplier() to _weight_multiplier() for weight

deferred from icehouse-3 to "next": http://lists.openstack.org/pipermail/openstack-dev/2014-February/026335.html

Removed from next, as next is now reserved for near misses from the last milestone --johnthetubaguy

Marking this blueprint as definition: Drafting. If you are still working on this, please re-submit via nova-specs. If not, please mark as obsolete, and add a quick comment to describe why. --johnthetubaguy (20th April 2014)

(?)

Work Items

Work items:
1) Split AvailabilityZoneFilter to two filters: INPROGRESS
2) Sync base filter logic from oslo to nova: INPROGRESS
3) Sync base weight logic from oslo to nova: TODO
4) Sync all filters from oslo to nova: TODO

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.