Support QEMU Native TLS for migration and disks over NBD

Registered by Kashyap Chamarthy on 2018-12-14


The encryption offerred by Nova (via `live_migration_tunnelled`) today does not secure all the different migration streams of a Nova instance, namely: guest RAM, device state, and disks (via NBD) when using non-shared storage. Further, the "tunnelling via libvirtd" has inherent limitations: (a) it cannot handle live migration of disks in a non-shared storage setup (a.k.a. "block migration"); and (b) has a huge performance overhead and latency, because it burns more CPU and memory bandwidth due to increased number of data copies on both source and destination hosts.

This change

To solve the existing limitation, this patch introduces a new config option `live_migration_with_native_tls`, which will use the "native TLS" (i.e. TLS built into QEMU, and relevant support in libvirt). This will secure all migration streams, including disks that are not on shared storage—all of this without incurring the limitations of the "tunnelled via libvirtd" transport.


(1) This needs at least: libvirt 4.4.0 and QEMU 2.11.

(2) A TLS environment—i.e. CA, server, and client certificates, their file permissions, et al—must be "correctly" configured (typically by an installer tool) on all relevant Compute nodes.

(3) Ensure the following TLS-related config attributes in /etc/libvirt/qemu.conf/ are in place on all relevant Compute nodes, e.g.:

      default_tls_x509_cert_dir = "/etc/pki/qemu/"
      default_tls_x509_verify = 1

Note that there are other TLS-related config attributes in `/etc/libvirt/qemu.conf`. But if you set the both `default_*` parameters for all certificates, then there is no need to specify any of the others. In OpenStack's case, we just stick to setting up only the `default_*` case.

    * * *

NB: In the long-term, we will depreprecate the existing `live_migration_tunnelled` config option, as tunnelling via 'libvirtd' has little compelling reasons, if any, due to problems discussed in the "Why?" section earlier.

Blueprint information

Dan Smith
Kashyap Chamarthy
Kashyap Chamarthy
Series goal:
Accepted for stein
Milestone target:
milestone icon stein-3
Started by
melanie witt
Completed by
Matt Riedemann


Gerrit topic:,topic:bug/1798796,n,z

Addressed by:
    libvirt: Support native TLS for migration and disks over NBD

Gerrit topic:,topic:bp/support-qemu-native-tls-for-live-migration,n,z

Addressed by:
    docs: Secure live migration with QEMU-native TLS

We were holding approval of this blueprint pending the preliminary review of a docs patch for the series. That has been done at this point, and so now we're approving the blueprint. -- melwitt 20190110

Gerrit topic:,topic:Native_TLS,n,z

Addressed by:
    libvirt: A few miscellaneous items related to "native TLS"

Addressed by:
    docs: Update references to "QEMU-native TLS" document


Work Items

This blueprint contains Public information 
Everyone can see this information.


No subscribers.