API: Add soft-affinity policy for server-group

Registered by Balazs Gibizer

As a tenant I would like to schedule instances on the same host if possible,
so that I can achieve colocation. However if it is not possible to schedule
some instance to the same host then I still want that the subsequent
instances are scheduled together on another host. In this way I can express
a good-to-have relationship between a group of instances.

As a tenant I would like to schedule instances on different hosts if possible.
However if it is not possible I still want my instances to be scheduled even
if it means that some of them are placed on a host where another instances
are runnig from the same group.

End User might want to have a less strict affinity and anti-affinity
rule than what is today available in server-group API extension.
With the proposed good-to-have affinity rule the End User can request nova
to schedule the instance to the same host if possible. However if it is not
possible (e.g. due to resource limitations) then End User still wants to keep
the instances on a small amount of different host.
With the proposed good-to-have anti-affinity rule the End User can request
nova to spread the instances in the same group as much as possible.

This change would extend the existing server-group API extension with two new
policies soft-affinity and soft-anti-affinity.
When a instance is booted into a group with soft-affinity policy the scheduler
will use a new weight AffinityWeight to sort the available hosts according to
the number of instances running on them from the same server-group in a
descending order.
When an instance is booted into a group with soft-anti-affinity policy the
scheduler will use a new weight AntiAffinityWeight to sort the available hosts
according to the number of instances running on them from the same
server-group in a ascending order.

The two new weigh will get the necessary information about the number of
instances per host through the weight_properties (filter_properties) in
a similar way as the GroupAntiAffinityFilter gets the list of hosts used by
a group via the filter_properties.

These new soft-affinity and soft-anti-affinity policies are mutually exclusive
with each other and with the other existing server-group policies.

Blueprint information

Status:
Complete
Approver:
John Garbutt
Priority:
Low
Drafter:
Balazs Gibizer
Direction:
Approved
Assignee:
Balazs Gibizer
Definition:
Approved
Series goal:
Accepted for mitaka
Implementation:
Implemented
Milestone target:
milestone icon mitaka-2
Started by
Balazs Gibizer
Completed by
John Garbutt

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/soft-affinity-for-server-group,n,z

Addressed by: https://review.openstack.org/140017
    Add soft policies for server-group feature

Addressed by: https://review.openstack.org/147514
    Add functional test for server group

Addressed by: https://review.openstack.org/147516
    scheduler: add soft-(anti-)affinity weighers

Addressed by: https://review.openstack.org/147517
    scheduler: add instance count to filter_properties

Addressed by: https://review.openstack.org/147520
    api: add soft-affinity policies for server groups

Addressed by: https://review.openstack.org/147521
    Add functional tests to soft affinity capabilities

Addressed by: https://review.openstack.org/148275
    scheduler:refactor group info in filter_properties

Addressed by: https://review.openstack.org/148277
    scheduler: add num of instance per host in a group

Addressed by: https://review.openstack.org/145761
    Fix leaking exceptions from scheduler utils

Addressed by: https://review.openstack.org/150424
    object: serialize set to list

This isn't going to make it for kilo, lets delay this to L --johnthetubaguy 4th Feb 2015

Addressed by: https://review.openstack.org/168372
    Add soft policies for server-group feature

Sorry, we have now hit the non-priority feature freeze for Liberty. You will need to resubmit this blueprint for Mitaka or apply for an exception. For more details on why this is happening, and the rest of the process details, please see: https://wiki.openstack.org/wiki/Nova/Liberty_Release_Schedule
--johnthetubaugy 3rd July 2015

Addressed by: https://review.openstack.org/210093
    api: add soft-affinity policies for server groups

Gerrit topic: https://review.openstack.org/#q,topic:soft-affinity-for-server-group,n,z

Addressed by: https://review.openstack.org/212504
    Add soft policies for server-group feature

Addressed by: https://review.openstack.org/199205
    Modify Scheduler RPC API to use RequestSpec obj

Addressed by: https://review.openstack.org/203078
    Modify conductor to use RequestSpec object

While there is python-novaclient work, technically the nova part of this is now complete, so marking this as complete. --johnthetubaguy 2015.01.15

Addressed by: https://review.openstack.org/268069
    Adds relase notes for soft affinity feature

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.