Split neutron into base library and servers/agents

Registered by Doug Wiegley

Part of the goal of splitting functionality and vendor code out of neutron was to increase focus on core neutron, getting it stable, and increasing the velocity of making changes. Since the services repos and decomposed plugins/drivers are both importing and using internal neutron interfaces, without running existing tests against the universe of all of this code, it becomes difficult to re-factor or make simple changes without breaking these newly de-composed repos.

Blueprint information

Status:
Complete
Approver:
Armando Migliaccio
Priority:
High
Drafter:
Doug Wiegley
Direction:
Approved
Assignee:
Boden R
Definition:
Approved
Series goal:
Accepted for ocata
Implementation:
Implemented
Milestone target:
milestone icon ocata-rc1
Started by
Kyle Mestery
Completed by
Armando Migliaccio

Related branches

Sprints

Whiteboard

Feb-07-2017 (boden): I'm fine with armax's suggestion below of closing this an opening more measurable BPs as needed. For reference I'm working on moving networking-ovn off neutron and as a result a number of patches are out for review (and more underway): https://review.openstack.org/#/q/status:open+AND+project:openstack/neutron-lib+AND+(topic:%255Erehome+OR+message:rehome)

Feb-06-2017(armax): the split is well under way, with speed bumps along the way. Personally I'd vote to close this blueprint and create new ones that are more measurable and smaller in scope (e.g. make project foo stop importing neutron) and so on.

Dec-01-2016(armax): we're chewing away at what we can decouple. We are optmistically trying to get at least one or two projects free of neutron imports by the end of ocata, but it's a tall order.

Aug-08-2016(armax): ongoing effort

Mar-08-2016(armax): Moving to Newton-1; we need to better define completion criteria, but we're getting close to have something functional end-to-end

Feb-29-2016(dougwig): this is tough to track as a single item, since work will be ongoing throughout Newton as well. The limited goal in Mitaka was completely severing lbaas. At present, the library exists and is plumbed throughout the infra. The first rev is being used by neutron and neutron-lbaas. Patches exist for bumping both to the second version of the lib. More patches exist to delete a lot of cruft from lbaas that will mean less to migrate, and plans are in place to stop the dependency on test code. The remaining items that were aimed at Mitaka but will miss are base db model/migration foo, and data model foo, both of which are ongoing, but neither of which needs to land in the critical end of mitaka timeframe (they can iterate in gerrit for now.) As soon as the Mitaka branch is baked, we can: 1) turn on deprecation warnings, 2) nuke lbaas v1 and v2 agent, 3) start mass import renames in neutron (yuck yuck), and 4) merge the above db/model items. The goal of completely severing lbaas is realistically about 3-4 weeks away. The separation of the remaining *aas are next after that, and stadium projects after that.

Jan-24-2016(armax): this is well undergoing, with more to be ironed out during mid-cycle.

Dec-07-2015(armax): need to write the current plan down, and next big hurdle is coming up with a plan of attack for parts of neutron.db

August-26 (mestery): Moving to Mitaka!

June-15 (mestery): Lets try to land as much as we can in Liberty.

Gerrit topic: https://review.openstack.org/#q,topic:bp/neutron-lib,n,z

Addressed by: https://review.openstack.org/154736
    Split neutron-lib out of neutron

Addressed by: https://review.openstack.org/247855
    Initial cookiecutter commit

Gerrit topic: https://review.openstack.org/#q,topic:initial-exceptions,n,z

Addressed by: https://review.openstack.org/247856
    Initial shared exceptions and i18n macros

Gerrit topic: https://review.openstack.org/#q,topic:review-guidelines,n,z

Addressed by: https://review.openstack.org/247860
    Beginning of lib review guidelines

Addressed by: https://review.openstack.org/248843
    Follow-up to initial cookiecutter commit

Addressed by: https://review.openstack.org/248847
    Insert references to debtcollector in review guidelines

Addressed by: https://review.openstack.org/251521
    Remove dependency on neutron for topics

Addressed by: https://review.openstack.org/252028
    Remove dependency on neutron for topics

Addressed by: https://review.openstack.org/253254
    Initial list of constants

Addressed by: https://review.openstack.org/253660
    Setup for devref documentation

Addressed by: https://review.openstack.org/253661
    Adding callback mechanism

Addressed by: https://review.openstack.org/254368
    Removing adv svcs dependencies on neutron

Addressed by: https://review.openstack.org/255116
    Add neutron/plugin/common/constants.py module to neutron-lib

Addressed by: https://review.openstack.org/259053
    API validators and conversions

Addressed by: https://review.openstack.org/265997
    Callbacks: replace kwargs with dict positional arg

Addressed by: https://review.openstack.org/268232
    Start using neutron-lib for shared constants and exceptions

Gerrit topic: https://review.openstack.org/#q,topic:migrate-rpc,n,z

Addressed by: https://review.openstack.org/268343
    Lib specific hacking rules

Addressed by: https://review.openstack.org/267807
    Move neutron.common.rpc into neutron-lib

Addressed by: https://review.openstack.org/253263
    Migrate some stuff to neutron-lib

Gerrit topic: https://review.openstack.org/#q,topic:no-neutron-rule,n,z

Addressed by: https://review.openstack.org/270630
    Add a hacking check against importing neutron

Addressed by: https://review.openstack.org/273909
    Move neutron.common.rpc into neutron-lib

Addressed by: https://review.openstack.org/279734
    Callback mechanism using Oslo Versioned Objects (PoC)

Gerrit topic: https://review.openstack.org/#q,topic:paginate-from-oslo.db,n,z

Addressed by: https://review.openstack.org/285863
    WIP: Use paginate_query from oslo_db

Addressed by: https://review.openstack.org/286875
    VPNaaS: make use of neutron_lib exceptions

Addressed by: https://review.openstack.org/286878
    LBaaS: make use of neutron_lib exceptions

Addressed by: https://review.openstack.org/286929
    FWaaS: make use of neutron_lib exceptions

Addressed by: https://review.openstack.org/297933
    Move neutron.common.exceptions to neutron-lib

Addressed by: https://review.openstack.org/299017
    [WIP] Switch to neutron lib constants and exceptions

Gerrit topic: https://review.openstack.org/#q,topic:neutron-lib,n,z

Addressed by: https://review.openstack.org/302000
    [WIP][DNM] Does this break anything?

Addressed by: https://review.openstack.org/302497
    Remove unused PolicyFileNotFound exception

Addressed by: https://review.openstack.org/303891
    [WIP] Switch to DB api and model_base from neutron-lib

Addressed by: https://review.openstack.org/303892
    [WIP][DNM] For Doug

Addressed by: https://review.openstack.org/307893
    [WIP] Use converters and validators from neutron-lib

Addressed by: https://review.openstack.org/308461
    Use exceptions from neutron-lib

Addressed by: https://review.openstack.org/285439
    Switch to using hacking checks from neutron-lib

Addressed by: https://review.openstack.org/308710
    Switch to inheriting hacking checks from neutron-lib

Addressed by: https://review.openstack.org/308712
    Switch to using attributes from neutron-lib

Addressed by: https://review.openstack.org/309776
    Revert "Switch to inheriting hacking checks from neutron-lib"

Addressed by: https://review.openstack.org/309854
    Switch to using hacking checks from neutron-lib

Addressed by: https://review.openstack.org/309855
    Switch to using hacking checks from neutron-lib

Addressed by: https://review.openstack.org/309856
    Switch to using hacking checks from neutron-lib

Addressed by: https://review.openstack.org/309858
    Revert "Revert "Switch to inheriting hacking checks from neutron-lib""

Addressed by: https://review.openstack.org/310909
    Remove ICMPV6_ALLOWED_TYPES

Addressed by: https://review.openstack.org/313794
    Use IPv[46]_BITS instead of IPV[46]_MAX_PREFIXLEN

Addressed by: https://review.openstack.org/314029
    Using constants from neutron-lib

Addressed by: https://review.openstack.org/330357
    Use neutron-lib converters

Gerrit topic: https://review.openstack.org/#q,topic:dashboard/neutron-lib,n,z

Addressed by: https://review.openstack.org/337694
    Add neutron-lib to milestone review dashboard

Addressed by: https://review.openstack.org/337731
    Refactor/prepare db common utils for neutron-lib

Addressed by: https://review.openstack.org/303867
    Add Neutron context module and some policy methods

Addressed by: https://review.openstack.org/339875
    Base DB stuff, step 1: rehome

Addressed by: https://review.openstack.org/354833
    Add tool to list moved globals

Addressed by: https://review.openstack.org/319328
    Rehome neutron common rpc

Addressed by: https://review.openstack.org/357943
    Move some neutron.plugins.common stuff

Addressed by: https://review.openstack.org/357959
    WIP - Convert common.utils to use the neutron_lib versions

Addressed by: https://review.openstack.org/358178
    Add __init__ methods in model_base for debtcollector

Addressed by: https://review.openstack.org/358173
    Move get_unique_keys() out of model_base.py

Addressed by: https://review.openstack.org/363845
    Use model_base from neutron-lib

Addressed by: https://review.openstack.org/363847
    Use model_base from neutron-lib

Addressed by: https://review.openstack.org/386719
    Drop use of neutron's eventlet utility function

Addressed by: https://review.openstack.org/386751
    Introduce Plugin Registry for Neutron

Addressed by: https://review.openstack.org/386845
    Spin off plugin directory out of Neutron Manager

Addressed by: https://review.openstack.org/386773
    Migrate references from neutron to neutron-lib

Addressed by: https://review.openstack.org/387747
    Add neutron context and policy modules to neutron-lib

Addressed by: https://review.openstack.org/388157
    Spin off context/policy engine

Addressed by: https://review.openstack.org/386846
    DNM: see the effects of directory/context/policy on neutron codebase

Gerrit topic: https://review.openstack.org/#q,topic:fira1,n,z

Addressed by: https://review.openstack.org/396349
    Switch to using plugins directory in lieu of neutron manager

Addressed by: https://review.openstack.org/396390
    Add missing is_loaded() method on directory's interface

Addressed by: https://review.openstack.org/396813
    Switch to using plugins directory in lieu of neutron manager

Addressed by: https://review.openstack.org/397535
    Switch to using plugins directory in lieu of neutron manager

Addressed by: https://review.openstack.org/397924
    Switch to using plugins directory in lieu of neutron manager

Gerrit topic: https://review.openstack.org/#q,topic:plugin-directory,n,z

Addressed by: https://review.openstack.org/394164
    Use neutron-lib for well known plugin types

Addressed by: https://review.openstack.org/405056
    Sync latest neutron callbacks into lib

Addressed by: https://review.openstack.org/405101
    Sync up bare minimum from neutron.db.api into lib

Addressed by: https://review.openstack.org/414902
    neutron-lib: use CORE from neutron lib constants

Addressed by: https://review.openstack.org/419245
    Revisit exported policy module

Addressed by: https://review.openstack.org/421472
    Use neutron-lib definition of neutron-fwaas API

Addressed by: https://review.openstack.org/433034
    Remove deprecated methods supported by neutron_lib

Addressed by: https://review.openstack.org/441728
    Use neutron-lib's context module instead of neutron

Addressed by: https://review.openstack.org/442167
    Use neutron-lib's context module

Addressed by: https://review.openstack.org/442555
    Migrate Events and Resources to neutron-lib

Addressed by: https://review.openstack.org/409557
    [WIP] Operation Pig Bristle, part 7

Addressed by: https://review.openstack.org/409556
    [WIP] Operation Pig Bristle, part 5

Addressed by: https://review.openstack.org/409555
    Eliminate lookup of "resource extend" funcs by name

Addressed by: https://review.openstack.org/409554
    Eliminate lookup of model query hooks by name

Addressed by: https://review.openstack.org/407837
    [WIP] Operation Pig Bristle, part 6

Addressed by: https://review.openstack.org/458982
    Stop using CommonDbMixin

Addressed by: https://review.openstack.org/471229
    Stop using CommonDbMixin

Addressed by: https://review.openstack.org/471294
    Stop using CommonDbMixin

Addressed by: https://review.openstack.org/478363
    Move Firewall Exceptions to a common location

Addressed by: https://review.openstack.org/478883
    Use API Definitions from Neutron-Lib

Addressed by: https://review.openstack.org/575963
    Bump neutron-lib version

(?)

Work Items

Work items:
Constants: DONE
Conf items: TODO
Base DB: INPROGRESS
RPC: TODO

This blueprint contains Public information 
Everyone can see this information.