Expose host capabilities

Registered by hongbin

Nova has API and implementation to expose capabilities of each host (i.e. cpu, memory, etc.). First, introduce the compute_host table. Then introduce a periodically task to populate the table. Then, expose host info via REST API.

Blueprint information

Series goal:
Milestone target:
Started by
Completed by

Related branches



In this spec, we are referring to creation of a table very similar to the compute_nodes table in Nova[1].

Nova currently populates this table via it's compute drivers. For linux, it uses the libvirt APIs to find out about the current host capabilities. This is also done periodically to ensure that the right data is presented to the user.

In between this, there's a resource tracker, that tracks the resources from an OpenStack view per say. We won't implement a resource tracker in zun to start with.

We could start with the following work items:

1. Create a os-capability package in zun. This package can have drivers for various operating systems to obtain the data for cpu/mem/numa nodes etc.

I am also in favor of seeing this package released as a library in OpenStack, as opposed to residing completely in zun's code. But since we are in rapid development mode, we will just keep the design as such that the packages can be split out anytime, we want to create a library out of it.

2. For Linux Operating systems, use the following commands for cpu and mem stats:

For cpu - use lscpu and parse the o/p. This will give us the NUMA nodes value as well. I have the code ready for this.
For memory - use /proc/meminfo and parse the o/p for the details on memory.

3. Create a host_capabilities (or compute_nodes) table that would have the following entries to begin with:

- ID
- hostname
- total cpu
- used_cpu
- no_of_containers
- total memory (can be further divided into):
MemTotal: 263636352 kB
MemFree: 180490624 kB
MemAvailable: 214005120 kB
Buffers: 3328 kB
Cached: 35926272 kB
- NumaTopology Object, consisting of cpuset details.

4. Implement a periodic task in the driver that would refresh this host capabilities data periodically.

[1] : https://github.com/openstack/nova/blob/fdc150897e350d3e681b6b86cd863342ab6ca425/nova/objects/compute_node.py

Gerrit topic: https://review.openstack.org/#q,topic:bp/expose-host-capabilities,n,z

Addressed by: https://review.openstack.org/432687
    [WIP] Add resource provider table in DB

Addressed by: https://review.openstack.org/432722
    DB: Add inventory table

Addressed by: https://review.openstack.org/432782
    DB: Add resource class table

Addressed by: https://review.openstack.org/432841
    DB: Add allocation table

Addressed by: https://review.openstack.org/433296
    Add resource provider objects

Addressed by: https://review.openstack.org/433324
    Add resource class object

Addressed by: https://review.openstack.org/433915
    Add tests for custom object fields

Addressed by: https://review.openstack.org/433920
    Define a ResourceClassField in object

Addressed by: https://review.openstack.org/433994
    [WIP] Add inventory object

Addressed by: https://review.openstack.org/434119
    Implement etcd db API for resource_provider

Addressed by: https://review.openstack.org/434126
    [WIP] Add resource_providers API endpoint

Addressed by: https://review.openstack.org/434616
    Eagerly load resource_provider from foreignkey

Addressed by: https://review.openstack.org/434909
    Implement etcd API for ResourceClass

Addressed by: https://review.openstack.org/456567
    Add memory info to compute node.

Addressed by: https://review.openstack.org/457126
    Add container number info to compute node.

Addressed by: https://review.openstack.org/458751
    Add cpu, cpu_used for compute node

Addressed by: https://review.openstack.org/460361
    Add host operating system info to compute node

Addressed by: https://review.openstack.org/462016
    Add labels info to compute node

@hongbin, I guess the host capability is almost there, could this BP marked as complete? Any other capability that you prefer to be added?

Sorry, seems this BP has been implemented long time ago.


Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.