extend kolla to support deploying openvswitch with dpdk

Registered by sean mooney

------------- problem description-----------
DPDK, the Data Plane development kit is an opensouce set of libraries and drivers for fast packet processing in userspace.
see: http://dpdk.org/

the openvswitch project supports integration with dpdk as a optional build time dependency for for accelerated switching as an
alternative to the kernel data plane. one of the acclerated interfaces provide by dpdk is called vhost-user.

as of the kilo release of openstack nova has had support for the vhost-user interface type supported by dpdk.
in kilo support for vhost-user with neuton was provided by the networking-ovs-dpdk agent and ml2 driver.
as of mitaka vhost-user support in now fully opstream and intergrated in the standard openvswitch agnet/ml2 driver
as well as other neutron backends such as networking-odl and networking-ovn.

this change proposes extending kolla to optionally deploy ovs with dpdk in place of kernel ovs to support high performance workloads such as small packet processing typical of telco/nfv usecase.

----------------proposed change-----------------

to support ovs with dpdk a new ovs_type parameter would be introduced.
ovs_type would default to kernel ovs and could be set globally via the global.yml or overridden on a per node
basis via the inventory file.

the deploy playbook will be extended to consult this new parameter when selecting the ovs container image to deploy.

to support co existence of kernel ovs and ovs with dpdk in the same deployment the following:
woudl be added to the https://github.com/openstack/kolla/blob/master/ansible/roles/neutron/defaults/main.yml to specify the new container image.

openvswitch_dpdk_db_image: "{{ docker_registry ~ '/' if docker_registry else '' }}{{ docker_namespace }}/{{ kolla_base_distro }}-{{ kolla_install_type }}-openvswitch-dpdk-db-server"
openvswitch_dpdk_db_tag: "{{ openstack_dpdk_release }}"
openvswitch_dpdk_db_image_full: "{{ openvswitch_dpdk_db_image }}:{{ openvswitch_dpdk_db_tag }}"

openvswitch_dpdk_vswitchd_image: "{{ docker_registry ~ '/' if docker_registry else '' }}{{ docker_namespace }}/{{ kolla_base_distro }}-{{ kolla_install_type }}-openvswitch-dpdk-vswitchd"
openvswitch_dpdk_vswitchd_tag: "{{ openstack_dpdk_release }}"
openvswitch_dpdk_vswitchd_image_full: "{{ openvswitch_dpdk_vswitchd_image }}:{{ openvswitch_dpdk_vswitchd_tag }}"

kolla would be extended with a new container which would install ovs-dpdk from binary as it is now packaged on
centos and rhel as tech preview and on ubuntu 16.04 in the main repo. it is also available on ubuntu 14.04 via the mitaka cloud archive.

if source install of ovs is allowed in the future via
https://review.openstack.org/#/c/309726/ or https://review.openstack.org/#/c/275932/
installation from source code be added but is not required at this time as the package is now available in distros.

While source install is not required it does allow the use of know feature as they emerge. one feature of particular
interest that is expected to land before the newton release is userspace connection tracker support.
having userspace conntrack would enable to use of the new neutron conntrack security group driver with ovs with dpdk.

if souce build is not supported in tree deployers will be able to "bring there own" container image by overriding the openvswitch_dpdk_* paramater and reuse the deployment playbook to provision it.

the kolla docks will be extended to include a guide for deploying with ovs with dpdk as we as best practices for its use.

this blueprint depends on https://blueprints.launchpad.net/kolla/+spec/kernel-hugepage-config
as hugepages are a prerequisite for deploying ovs with dpdk.

Blueprint information

Status:
Complete
Approver:
Steven Dake
Priority:
Medium
Drafter:
sean mooney
Direction:
Approved
Assignee:
sean mooney
Definition:
Approved
Series goal:
Proposed for pike
Implementation:
Implemented
Milestone target:
milestone icon pike-3
Started by
sean mooney
Completed by
Jeffrey Zhang

Related branches

Sprints

Whiteboard

Sean, what is the strategy for DPDK kernel module deployment (like igb_uio.ko) in kolla? As far as I know, they are not part of OS binary DPDK distributions (except uio_pci_generic.ko) and need to be built.
-- MarMat

Gerrit topic: https://review.openstack.org/#q,topic:bp/ovs-dpdk,n,z

Addressed by: https://review.openstack.org/342354
    [WIP] adds support for ovs with dpdk netdev datapath

Addressed by: https://review.openstack.org/408866
    adds docs for ovs with dpdk

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.