Scheduling interaction for cells

Registered by John Garbutt on 2015-06-15

In order to schedule instance builds to compute hosts Nova and the scheduler will need to take into account that hosts are grouped into cells. It is not necessary that this is apparent when Nova is requesting a placement decision.

Blueprint information

Status:
Complete
Approver:
John Garbutt
Priority:
High
Drafter:
Andrew Laski
Direction:
Approved
Assignee:
Dan Smith
Definition:
Approved
Series goal:
Accepted for ocata
Implementation:
Implemented
Milestone target:
milestone icon ocata-3
Started by
Andrew Laski on 2016-06-03
Completed by
Matt Riedemann on 2017-01-27

Related branches

Sprints

Whiteboard

lets bump this one, doesn't seem likely for liberty now. --johnthetubaguy 27th August 2015

Gerrit topic: https://review.openstack.org/#q,topic:bp/cells-scheduling-interaction,n,z

Addressed by: https://review.openstack.org/239995
    Proposal for cells v2 scheduling interaction

Addressed by: https://review.openstack.org/254434
    Clean up filter_properties/availability_zone in boot

Addressed by: https://review.openstack.org/258628
    WIP Persist the request spec during an instance boot

Addressed by: https://review.openstack.org/263925
    WIP Populate instance_mappings during boot.

Addressed by: https://review.openstack.org/263926
    WIP Add buildrequest db/object

Addressed by: https://review.openstack.org/263927
    WIP Create Instance from BuildRequest if not in a cell

Gerrit topic: https://review.openstack.org/#q,topic:cells_schedule_instance_mapping,n,z

Addressed by: https://review.openstack.org/278124
    Add BuildRequest object

Addressed by: https://review.openstack.org/278125
    Create BuildRequest object during boot process

Addressed by: https://review.openstack.org/273641
    Change populate_security_groups to return a SecurityGroupList

Addressed by: https://review.openstack.org/276882
    Improve unit tests for instance multiple create

Gerrit topic: https://review.openstack.org/#q,topic:cells_pop_instance_mappings,n,z

Addressed by: https://review.openstack.org/278598
    Make InstanceMappings.cell_id nullable

Addressed by: https://review.openstack.org/286309
    Include CellMapping in InstanceMapping object

So lets stop here:
https://review.openstack.org/#/c/263925
Marking as needs code review and partial.

So the remaining patches that have a +W already get a FFE, so we can merge what we have reviewed. --johnthetubaguy 3rd March 2016

Re-opening this for Newton since it was partially implemented in Mitaka. We could create a new blueprint, but then we have to update the links in all of the patches, and the other bp dependencies, and the spec links, it just gets to be a mess, so I'm just going to re-open this and move forward. -- mriedem 20160317

Addressed by: https://review.openstack.org/294208
    Re-propose scheduling interaction for cells

Addressed by: https://review.openstack.org/294718
    Add BuildRequest object

Addressed by: https://review.openstack.org/319379
    WIP Move instance creation to conductor

Gerrit topic: https://review.openstack.org/#q,topic:bp/add-buildrequest-obj,n,z

Addressed by: https://review.openstack.org/325985
    Cells: Handle delete with BuildRequest

Addressed by: https://review.openstack.org/332479
    Stop instance build if BuildRequest deleted

Gerrit topic: https://review.openstack.org/#q,topic:cells_instance_list,n,z

Gerrit topic: https://review.openstack.org/#q,topic:cells_instance_name_hack,n,z

Addressed by: https://review.openstack.org/356137
    Add BuildRequestList object

Addressed by: https://review.openstack.org/356138
    WIP Pull from cell0 and build_requests for instance list

Addressed by: https://review.openstack.org/328491
    New discover command to add new hosts to a cell

Gerrit topic: https://review.openstack.org/#q,topic:cells_instance_from_build_request,n,z

We still have work to do here so this is getting deferred to Ocata. -- mriedem 20160901

Addressed by: https://review.openstack.org/367557
    Add schedule_and_build_instances conductor method

Addressed by: https://review.openstack.org/372644
    Tests: improve assertJsonEqual diagnostic message

Addressed by: https://review.openstack.org/372645
    WIP Make schedule_and_build_instances cell aware

Addressed by: https://review.openstack.org/374267
    WIP Return build_requests instead of instances

Addressed by: https://review.openstack.org/379320
    Add schedule_and_build_instances RPC method

Addressed by: https://review.openstack.org/381275
    Proposal for cells v2 scheduling interaction

Gerrit topic: https://review.openstack.org/#q,topic:ocata_cells_conductor_deployment,n,z

Addressed by: https://review.openstack.org/392925
    Add CellDatabase fixture

Addressed by: https://review.openstack.org/393438
    Refactor two nearly useless secgroup tests

Addressed by: https://review.openstack.org/394656
    Store security groups in RequestSpec

Addressed by: https://review.openstack.org/395746
    Rename security_group parameter in compute.API:create

Addressed by: https://review.openstack.org/395747
    WIP: Transform requested secgroup names to uuids

Addressed by: https://review.openstack.org/396417
    Make RPCFixture support multiple connections

Addressed by: https://review.openstack.org/396436
    Add NeutronSecurityGroup object

Addressed by: https://review.openstack.org/396469
    Return list of NeutronSecurityGroup objects from populate_security_groups

Addressed by: https://review.openstack.org/396775
    WIP Multicell support for instance listing

Gerrit topic: https://review.openstack.org/#q,topic:bp/cells-sched-staging,n,z

Addressed by: https://review.openstack.org/398645
    WIP Experimenting with CellDatabases test fixture

Addressed by: https://review.openstack.org/399710
    Add a CellDatabases test fixture

Addressed by: https://review.openstack.org/404950
    Reset indirection_api for each test

Addressed by: https://review.openstack.org/405684
    Add SingleCellSimple fixture

Addressed by: https://review.openstack.org/406260
    Add SecurityGroup.identifier to prefer uuid over name

Addressed by: https://review.openstack.org/406379
    Fix up non-cells-aware context managers in test_db_api

Addressed by: https://review.openstack.org/406380
    Setup CellsV2 environment in base test

Addressed by: https://review.openstack.org/407238
    Make servers api view load instance fault from proper cell

Addressed by: https://review.openstack.org/410471
    Fix non-parameterized service id in hypervisors sample tests

Addressed by: https://review.openstack.org/410472
    Make the SingleCellSimple fixture a little more comprehensive

Addressed by: https://review.openstack.org/410473
    WIP: Make api_samples tests use simple cell environment

Addressed by: https://review.openstack.org/411525
    WIP: nova-status: implement _check_cellsv2

Addressed by: https://review.openstack.org/411517
    Add nova-status upgrade check command framework

Addressed by: https://review.openstack.org/411883
    Add basic placement status checking

Gerrit topic: https://review.openstack.org/#q,topic:bp/resource-providers-scheduler-db-filters,n,z

Addressed by: https://review.openstack.org/417962
    WIP: Make metadata server know about cell mappings

Addressed by: https://review.openstack.org/418141
    Don't bypass cellsv1 replication if cellsv2 maps are in place

Addressed by: https://review.openstack.org/424719
    WIP: Remove duplicate BDM validation

Addressed by: https://review.openstack.org/424737
    WIP: Collected release notes for Ocata CellsV2

Addressed by: https://review.openstack.org/426532
    Revert "Move instance creation to conductor"

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.