Adding ports pools to speed up containers booting and deletion

Registered by Luis Tomas Bolivar

Every time a container is created or deleted, there is a call from Kuryr to
Neutron to create/remove the port used by the container (and to attach the
port to the trunk for the nested vlan case).

In order to speed up both container creation and deletion we plan to have a
Resource Manager in charge of the port creation/deletion Neutron interactions,
that will maintain pools of Neutron resources. This Resource Manager speeds
up containers creation/deletion by:
- Pre-creating ports (and subports for the nested case) to be later used
by containers when booting up -- instead of having to create them during
container boots up process
- Avoiding the pooling for port to become ACTIVE, as there will be already
available to be used
- Ensure ports reuse upon container deletion, to minimize the need for ports
creation/deletion. This removes the port-deletion call from the container
deletion process

The resource manager will handle a pool of ports and will be in charge of:
- Ensuring at least X ports are on the pool for each group, i.e., for each
security group and tenant
- Clean up released ports to be reused by future pods
- Ensure either a max size of the pool, or that ports not being used during
a certain period of time are removed.

Thanks to having the ports pool, during the container creation process,
instead of calling neutron port create and then waiting for the port to become
active, a port will be selected from the right pool queue (no need to call
neutron for this), and then the port info will be updated with the container
name (call to neutron port_update). Similarly, for the port deletion we skip the
call to remove the port as it is just included in a cleanup pool, and the
Resource Manager will handle it asynchronously.

Blueprint information

Status:
Complete
Approver:
Antoni Segura Puimedon
Priority:
High
Drafter:
Luis Tomas Bolivar
Direction:
Approved
Assignee:
Luis Tomas Bolivar
Definition:
Approved
Series goal:
Accepted for pike
Implementation:
Implemented
Milestone target:
milestone icon pike-2
Started by
Antoni Segura Puimedon
Completed by
Luis Tomas Bolivar

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/ports-pool,n,z

Addressed by: https://review.openstack.org/427681
    Kuryr Kubernetes Resource Manager design reference document

Addressed by: https://review.openstack.org/436875
    Adding support for ports pool handler

Addressed by: https://review.openstack.org/436876
    Generic vif driver extension to enable ports reuse

Addressed by: https://review.openstack.org/436877
    Generic vif pool driver extension to precreate reusable ports

Addressed by: https://review.openstack.org/436893
    Nested vif driver extension to enable ports reuse

Addressed by: https://review.openstack.org/436894
    Nested vlan vif pool driver extension to precreate reusable subports

Addressed by: https://review.openstack.org/472734
    Change from GenericVIFPool to NeutronVIFPool

Addressed by: https://review.openstack.org/474576
    Delete or recover precreated ports upon controller restart

Addressed by: https://review.openstack.org/474589
    Add ports pool configuration information at README

Addressed by: https://review.openstack.org/475389
    Update ports manager spec with reboot information

Gerrit topic: https://review.openstack.org/#q,topic:nested-precreated-ports,n,z

Addressed by: https://review.openstack.org/510157
    Avoid neutron calls at recovering precreated ports

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.