PCI Passthrough base

Registered by Boris Pavlovic on 2013-05-20

PCI Passthrough is used to use pci devices in VMs without any virtualization.

This blueprint will provide base integration of PCI passthrough with OpenStack based on PCI labels.

It will include:
1) Mechanism to specify available PCI devices on host through nova.conf
2) Mechanism to store PCI devices state in DB
3) Specific InstanceTypeExtraSpec that contains list of PCI labels that are required by InstanceType.
4) Scheduling based on PCI labels

PCI label is high level abstract name of device, all operations such as:
a) scheduling,
b) specifying required by instance_types devices
c) attaching pci device to instance # I mean work in DB, not virt layer
are make with PCI lables, not PCI addresses.
Using PCI lables makes this approach very flexible.

Why labels and not device address?
For example if you have 10 nodes, on each SR-IOV then each node will have 7 pci addresses.
So if all these addresses on all nodes have one logical name, for example "with_eth".
Then it is pretty simple to organize:
1) scheduling // get all nodes that have available devices with this label
2) specifying what devices you need
3) attaching devices

How to use it (base use case):
1) Specify on all compute nodes available for passthrough PCI devices:
add to nova.conf:
pci_passthrough_devices=[{"label": "some_name", "address": "xxxx:xx:xx.x"}]
# This will create/update db records for PCI devices on host (when we run nova compue)
2) Add to any InstanceType extra specs {'pci_passthrough:labels': '["some_name"]'}
3) Create Instance with this InstanceType, it will create VM with device with address "xxxx:xx:xx.x"

Blueprint information

Russell Bryant
Boris Pavlovic
jiang, yunhong
Series goal:
Accepted for havana
Milestone target:
milestone icon 2013.2
Started by
Boris Pavlovic on 2013-05-20
Completed by
Russell Bryant on 2013-09-03

Related branches



UML diagrams of changes + Code will be soon. -- boris-42


Does this blueprint answers the case where PCI device is intended to be used as Network interface (connect to certain virtual network)? How it guarantees that chosen device has access to provider network that Virtual Network is defined on?


Nope, there is no fine grade work here. This is base for PCI passthrough, that allows only:
1) specifying devices in nova.conf that are able for pci passthrough
2) abstract from the address to labels (to be able to make "the same" devices on different hostes)
3) guarantee that two VMs won't be able to get on device in the same time
4) scheduling by labels
5) specifying required labels (device with this labels should be passed through) instance_type_extra_specs ---- boris-42


Should the admin configure an entry {"label": "some_name", "address": "xxxx:xx:xx.x"} for each address? It can be tedious if one has more than a dozen. -- itzikb1


There will be two different ways to specify devices:
a) manually
b) auto-discovering

So you will be able to chose the method. -- boris-42

How will auto-discovering work? -- itzikb1

please check https://review.openstack.org/#/c/34819/ for the auto-discovery patch. - yjiang5

Gerrit topic: https://review.openstack.org/#q,topic:bp/pci-passthrough-libvirt,n,z

Addressed by: https://review.openstack.org/34644
    PCI passthrough DB Layer

Addressed by: https://review.openstack.org/34645
    PCI passthrough Conductor layer

Addressed by: https://review.openstack.org/34646
    PCI passthrough Utils layer

Addressed by: https://review.openstack.org/34647
    PCI passthrough Scheduler layer

Addressed by: https://review.openstack.org/34648
    PCI passthrough Compute layer

Gerrit topic: https://review.openstack.org/#q,topic:merge-with-boris,n,z

Addressed by: https://review.openstack.org/35099
    PCI passthrough DB Layer

Addressed by: https://review.openstack.org/35101
    PCI passthrough Utils layer

Gerrit topic: https://review.openstack.org/#q,topic:pci-passthrough-enhancement,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bp/pci-passthrough-base,n,z

Addressed by: https://review.openstack.org/39606
    Add PCI device object support

Addressed by: https://review.openstack.org/35749
    Add scheduler support for PCI passthrough

Addressed by: https://review.openstack.org/38450
    Add PCI device filters support

Addressed by: https://review.openstack.org/35074
    PCI devices resource tracker

Addressed by: https://review.openstack.org/35075
    Add PCI device tracker to compute resource tracker

Addressed by: https://review.openstack.org/34819
    PCI device auto discover

Addressed by: https://review.openstack.org/35755
    PCI alias support

Addressed by: https://review.openstack.org/39892
    PCI utils and exceptions

Addressed by: https://review.openstack.org/39893
    Pci Device DB support

Addressed by: https://review.openstack.org/39890
    Add PCI stats


Work Items

Dependency tree

* Blueprints in grey have been implemented.