Nested DPDK Support

Registered by Gary Loughnane on 2017-07-10

Motivation:
Add support for a Kuryr-DPDK driver for nested Kubernetes pods.

Problem Statement:
A Major breakthrough in cloud-native technology is driving the giant telecom and cable industries to transition from the existing VM based infrastructure to a hybrid (unified) infrastructure supporting nested containers through solutions such as Kuryr-Kubernetes. This has given rise to the need to support multi-homed pods.

High performance data plane networking with ultra-low latency requires networks to be isolated into management/control plane and data plane which is an indispensable feature of telecom infrastructure. Dataplane and multi-network feature are not natively supported in Kubernetes.

Solution:
Intel’s DPDK provides ultra low latency and fast packet processing. The performance benefits of DPDK coupled with the lightweight portability of containers, make this combination a perfect fit for NFV use cases.

Intel's proposed solution is to develop a Kuryr nested DPDK plugin that uses the DPDK library and driver to alleviate kernel bound interface to DPDK driver and assign them to a container network namespace.

The solution has two main areas:
- Kuryr Controller sends a request to Neutron for a new VirtIO port to be attached to the VM.
- Kuryr CNI binds the interface to the DPDK driver and makes it available to a DPDK VNF running within the pod.

The plugin assumes that the environment has already been configured for DPDK e.g. OVS-DPDK, Hugepages, CPU Pinning, etc.

The benefit of this solution is the optimised running of a DPDK VNF in a nested pod which provides high performance networking.

Approach:
Phase 1
- Develop Kuryr Nested DPDK plugin (now complete, upstream in progress).
Phase 2
- Work with Kuryr community on multi-VIF support that will allow for the connection of both a management/control plane (e.g. MACVLAN/VLAN trunk) and data plane (DPDK).

Challenges:
- Native CNI or Kubernetes networking model is a IP-per-Pod model.
- The DPDK-based data path is just a channel to rx/tx packets.
- Kubernetes scheduling criteria only takes compute resources (CPUand memory) into consideration and is agnostic to other hardware features of the nodes like hugepages. Kubernetes also doesn't address noisy neighbor scenarios.
- No device isolation in Kubernetes.

References:
DPDK SR-IOV CNI plugin https://github.com/Intel-Corp/sriov-cni
CPU Mangaer for K8s (CMK) https://github.com/intelsdi-x/CPU-Manager-for-Kubernetes
Node Feature Discovery https://github.com/kubernetes-incubator/node-feature-discovery
Hugepage volume plugin PR https://github.com/kubernetes/kubernetes/pull/47658
SR-IOV support in NFD PR https://github.com/kubernetes-incubator/node-feature-discovery/pull/49
Implemented opaque resource discovery PR https://github.com/kubernetes-incubator/node-feature-discovery/pull/61
Kuryr multi VIF patch https://review.openstack.org/#/c/471012/

Blueprint information

Status:
Started
Approver:
Antoni Segura Puimedon
Priority:
Undefined
Drafter:
Gary Loughnane
Direction:
Needs approval
Assignee:
Gary Loughnane
Definition:
Approved
Series goal:
None
Implementation:
Started
Milestone target:
None
Started by
Daniel Mellado on 2018-02-27

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/nested-dpdk-support,n,z

Addressed by: https://review.openstack.org/559363
    Add DPDK support for nested pods

Gerrit topic: https://review.opendev.org/#/q/topic:bp/nested-dpdk-support

Addressed by: https://review.opendev.org/559363
    Add DPDK support for nested pods

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.