Standardize CPU resource tracking

Registered by Jay Pipes on 2018-03-21

Standardize the tracking of both dedicated and shared CPU resources

Blueprint information

Status:
Complete
Approver:
None
Priority:
Undefined
Drafter:
Jay Pipes
Direction:
Approved
Assignee:
Stephen Finucane
Definition:
Approved
Series goal:
Accepted for train
Implementation:
Implemented
Milestone target:
None
Started by
Matt Riedemann on 2019-08-29
Completed by
Eric Fried on 2019-09-23

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/cpu-resources,n,z

Addressed by: https://review.openstack.org/555081
    Standardize CPU resource tracking

Addressed by: https://review.openstack.org/561770
    WIP: PCPU: Define numa dedicated CPU resource class

Addressed by: https://review.openstack.org/561771
    WIP: PCPU, VCPU: Add respective conf options

Gerrit topic: https://review.openstack.org/#q,topic:bp/the,n,z

Addressed by: https://review.openstack.org/576021
    NUMACell, InstanceNUMACell: Adopt 'PCPU' changes

Addressed by: https://review.openstack.org/577038
    Report PCPU to placement

Addressed by: https://review.openstack.org/577372
    Virt driver: Build guest xml

Gerrit topic: https://review.openstack.org/#/q/topic:bp/cpu-resources

Gerrit topic: https://review.opendev.org/#/q/topic:bp/cpu-resources

Addressed by: https://review.opendev.org/555081
    Standardize CPU resource tracking

Addressed by: https://review.opendev.org/659300
    Follow up for I9200c2b18369da8b9194a8ccbbf6f1baf6a9613c

Addressed by: https://review.opendev.org/666032
    Additional upgrade clarifications for cpu-resources

Addressed by: https://review.opendev.org/537414
    objects: Remove legacy '_from_dict' functions

Addressed by: https://review.opendev.org/671789
    objects: Rename 'nova.objects.instance_numa_topology'

Addressed by: https://review.opendev.org/671790
    libvirt: Remove unnecessary try-catch around 'getCPUMap'

Addressed by: https://review.opendev.org/671791
    claims: Remove useless caching

Addressed by: https://review.opendev.org/671792
    Add '[compute] cpu_dedicated_set' option

Addressed by: https://review.opendev.org/671793
    libvirt: Start reporting PCPU inventory to placement

Addressed by: https://review.opendev.org/671794
    libvirt: Start reporting 'pcpus' to the resource tracker

Addressed by: https://review.opendev.org/671795
    trivial: Rename exception argument

Addressed by: https://review.opendev.org/671796
    trivial: Remove unused function parameter

Addressed by: https://review.opendev.org/671797
    Remove 'hardware.get_host_numa_usage_from_instance'

Addressed by: https://review.opendev.org/671798
    Remove 'hardware.host_topology_and_format_from_host'

Addressed by: https://review.opendev.org/671799
    Remove 'hardware.instance_topology_from_instance'

Addressed by: https://review.opendev.org/671800
    WIP: hardware: Differentiate between shared and dedicated CPUs

Addressed by: https://review.opendev.org/671801
    Add support translating CPU policy extra specs, image meta

Addressed by: https://review.opendev.org/551026
    Remove deprecated CPU, RAM, disk claiming in resource tracker

Addressed by: https://review.opendev.org/672106
    Remove 'nova.virt.driver.ComputeDriver.estimate_instance_overhead'

Addressed by: https://review.opendev.org/672221
    WIP: Report PCPU to placement

Addressed by: https://review.opendev.org/672222
    WIP: Virt driver: Build guest xml

Addressed by: https://review.opendev.org/672223
    WIP: Add scheduler request filter to alias cpu_policy

Addressed by: https://review.opendev.org/672224
    WIP Upgrade patch to handle PCPU resource

Addressed by: https://review.opendev.org/672336
    tests: Split NUMA object tests

Addressed by: https://review.opendev.org/672565
    Rework 'hardware.numa_usage_from_instances'

Addressed by: https://review.opendev.org/672669
    Follow-up for I2936ce8cb293dc80e1a426094fdae6e675461470

Addressed by: https://review.opendev.org/672693
    libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available'

Addressed by: https://review.opendev.org/674103
    objects: Rename 'fields' import to 'obj_fields'

Addressed by: https://review.opendev.org/674104
    WIP: Add proxy fields to NUMATopology object

Addressed by: https://review.opendev.org/674395
    trivial: Rewrap definitions of 'NUMACell'

Addressed by: https://review.opendev.org/674396
    libvirt: Remove unnecessary argument

Addressed by: https://review.opendev.org/674893
    trivial: Remove unused '_instance_to_allocations_dict' function

Addressed by: https://review.opendev.org/674894
    scheduler: Flatten 'ResourceRequest.from_extra_specs', 'from_image_props'

Addressed by: https://review.opendev.org/674895
    Add reshaper for PCPU

Addressed by: https://review.opendev.org/675276
    Correct nit with cpu-resources

Addressed by: https://review.opendev.org/675338
    Remove 'hw:cpu_policy', 'hw:mem_page_size' extra specs from API samples

Addressed by: https://review.opendev.org/675570
    fakelibvirt: Stop distinguishing between NUMA, non-NUMA

Addressed by: https://review.opendev.org/675571
    libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait

Addressed by: https://review.opendev.org/675589
    Follow up for I491eb7abe1405b27c552cdef142c688a46e914f2

Addressed by: https://review.opendev.org/676729
    libvirt: Fold in argument to '_update_provider_tree_for_vgpu'

Addressed by: https://review.opendev.org/678861
    libvirt: Simplify 'fakelibvirt.HostInfo' object

Addressed by: https://review.opendev.org/678902
    libvirt: Start checking compute usage in functional tests

Addressed by: https://review.opendev.org/679273
    Follow-up for I491eb7abe1405b27c552cdef142c688a46e914f2

Addressed by: https://review.opendev.org/679745
    libvirt: Always enable the NUMATopologyFilter

Addressed by: https://review.opendev.org/680106
    objects: Add 'InstanceNUMATopology.cpu_pinning' property

Addressed by: https://review.opendev.org/680107
    Validate CPU config options against running instances

Addressed by: https://review.opendev.org/680108
    objects: Add 'NUMACell.pcpuset' field

Addressed by: https://review.opendev.org/680229
    trivial: Use sane indent

Addressed by: https://review.opendev.org/680983
    Interim follow ups for cpu-resources series

Addressed by: https://review.opendev.org/681060
    fakelibvirt: Make 'Connection.getHostname' unique

Addressed by: https://review.opendev.org/681061
    libvirt: Mock 'libvirt_utils.file_open' properly

Addressed by: https://review.opendev.org/644565
    Apply SEV-specific guest config when SEV is required

Addressed by: https://review.opendev.org/680158
    Reject live migration and suspend on SEV guests

Addressed by: https://review.opendev.org/666616
    Enable booting of libvirt guests with AMD SEV memory encryption

Addressed by: https://review.opendev.org/681374
    Counting both of VCPU and PCPU as core quota

Addressed by: https://review.opendev.org/681383
    DNM: Try to fallback to PCPU request when VCPU failed

Addressed by: https://review.opendev.org/681626
    trivial: Re-add some useful checks

Addressed by: https://review.opendev.org/681719
    fixup! Add support for translating CPU policy extra specs, image meta

Addressed by: https://review.opendev.org/681750
    tests: Additional functional tests for pinned instances

Addressed by: https://review.opendev.org/681751
    trivial: Remove single-use classmethod

Addressed by: https://review.opendev.org/678447
    db: Add resources column in instance_extra table

Addressed by: https://review.opendev.org/678448
    object: Introduce Resource and ResourceList objs

Addressed by: https://review.opendev.org/678449
    Add resources dict into _Provider

Addressed by: https://review.opendev.org/678450
    Retrieve the allocations early

Addressed by: https://review.opendev.org/678452
    Claim resources in resource tracker

Addressed by: https://review.opendev.org/678453
    libvirt: Enable driver discovering PMEM namespaces

Addressed by: https://review.opendev.org/678454
    libvirt: report VPMEM resources by provider tree

Addressed by: https://review.opendev.org/678455
    libvirt: Support VM creation with vpmems and vpmems cleanup

Addressed by: https://review.opendev.org/678456
    Parse vpmem related flavor extra spec

Addressed by: https://review.opendev.org/679640
    libvirt: Enable driver configuring PMEM namespaces

Addressed by: https://review.opendev.org/683437
    docs: Update CPU topologies guide to reflect the new PCPU world

Addressed by: https://review.opendev.org/683485
    docs: Update CPU topologies guide to reflect the new PCPU world

Addressed by: https://review.opendev.org/681807
    [DNM] cpu pinning testing

Addressed by: https://review.opendev.org/705666
    hardware: Add TODO to remove '(un)pin_cpu_with_siblings'

Addressed by: https://review.opendev.org/705667
    Address release note nits for cpu-resources series

Addressed by: https://review.opendev.org/705668
    doc: Address some trivial nits with port QoS doc

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.