Online Schema Migrations

Registered by Mike Bayer on 2015-06-17

This specification discusses the issue of database schema migrations which
may proceed while allowing both the previous and the updated version of the
Neutron database API to run against that schema at the same time. By using
separate "expand" and "contract" workflows, new constructs may be added
to the schema while the old version of the application continues to run.

This is (the simplest) part of a larger approach which is to allow a Neutron application to
be upgraded to a new version without incurring downtime while the database
schema is migrated.

Blueprint information

Status:
Complete
Approver:
Kyle Mestery
Priority:
High
Drafter:
Mike Bayer
Direction:
Approved
Assignee:
Ihar Hrachyshka
Definition:
Approved
Series goal:
Accepted for liberty
Implementation:
Implemented
Milestone target:
milestone icon 7.0.0
Started by
Kyle Mestery on 2015-07-24
Completed by
Armando Migliaccio on 2015-09-01

Related branches

Sprints

Whiteboard

August-26 (mestery): Ihar, what is the status of this BP? Is it implemented yet?
August-26 (ihrachyshka): it's almost there. the only piece that should be a blocker for the spec is: https://review.openstack.org/216313 Autogenerate patches are welcome to go but are not blockers.
Sept 1 (armax): change 216313 is merged. Unless there's some bug lurking like (bug 1490767), this looks like it's complete.
UPD: I forgot that there is also this cleanup that we should get in L: https://review.openstack.org/#/q/Iab29a785fca1ce86148f77f5973b9d7debce44a8,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bp/online-schema-migrations,n,z

Addressed by: https://review.openstack.org/192937
    Propose scripted online schema migrations

Addressed by: https://review.openstack.org/194197
    Ease debugging alembic by passing proper scripts path in alembic.ini

Addressed by: https://review.openstack.org/194198
    WIP: [neutron-db-manage] generate new revisions in split trees

Gerrit topic: https://review.openstack.org/#q,topic:bp/reference-implementation-split,n,z

Addressed by: https://review.openstack.org/202627
    migrations: rearrange the tree to support split migration phases

Addressed by: https://review.openstack.org/203337
    Revert "VPNaaS: Temporarily disable check_migration"

Addressed by: https://review.openstack.org/203340
    [neutron-db-manage] revision: properly bootstrap a new branch

Addressed by: https://review.openstack.org/203341
    Migrate to multi branch directory structure

Addressed by: https://review.openstack.org/203348
    ensure_dir: move under neutron.common.utils

Addressed by: https://review.openstack.org/204122
    script.py.mako: added license text, removed downgrade section

Gerrit topic: https://review.openstack.org/#q,topic:bug/1470625,n,z

Addressed by: https://review.openstack.org/206690
    script.py.mako: update to latest standards

Addressed by: https://review.openstack.org/206691
    switch to multi branch migration tree for liberty

Addressed by: https://review.openstack.org/206746
    [neutron-db-manage] check_migration: validate labels

Addressed by: https://review.openstack.org/206924
    [neutron-db-manage] remove old HEAD file when updating for branches

Addressed by: https://review.openstack.org/206984
    Killed existing downgrade rules in migration scripts

Addressed by: https://review.openstack.org/206986
    Killed existing downgrade rules in migration scripts

Gerrit topic: https://review.openstack.org/#q,topic:bug/1434103,n,z

Addressed by: https://review.openstack.org/206989
    script.py.mako: added license header and missing branch_labels

Addressed by: https://review.openstack.org/206990
    Killed existing downgrade rules in migration scripts

Gerrit topic: https://review.openstack.org/#q,topic:bug/1478512,n,z

Addressed by: https://review.openstack.org/207903
    Merge remote-tracking branch 'origin/master' into merge-branch

Gerrit topic: https://review.openstack.org/#q,topic:online-schema-migrations,n,z

Addressed by: https://review.openstack.org/203842
    Implement expand/contract autogenerate extension

Addressed by: https://review.openstack.org/213303
    Add non-model index names to autogen exclude filters

Addressed by: https://review.openstack.org/213870
    Implement expand/contract autogenerate extension

Addressed by: https://review.openstack.org/213871
    Add non-model index names to autogen exclude filters

Addressed by: https://review.openstack.org/215048
    neutron-db-manage: sync HEADS file with 'current' output

Addressed by: https://review.openstack.org/215076
    Dropped release name from migration branch labels

Addressed by: https://review.openstack.org/216313
    [neutron-db-manage] Introduce contract and expand commands

Addressed by: https://review.openstack.org/217712
    Adopt the migration chaining and branch names from neutron

Addressed by: https://review.openstack.org/217716
    Adopt the migration chaining and branch names from neutron

Addressed by: https://review.openstack.org/217719
    Adopt the migration chaining and branch names from neutron

Gerrit topic: https://review.openstack.org/#q,topic:bug/1487598,n,z

Addressed by: https://review.openstack.org/218710
    Merge remote-tracking branch 'origin/master' into merge-branch

Gerrit topic: https://review.openstack.org/#q,topic:bug/1334634,n,z

Addressed by: https://review.openstack.org/219156
    Merge remote-tracking branch 'origin/master' into pecan

Gerrit topic: https://review.openstack.org/#q,topic:sanity_test,n,z

Addressed by: https://review.openstack.org/220091
    [neutron-db-manage] Add check for branches

Gerrit topic: https://review.openstack.org/#q,topic:bug/1490767,n,z

Addressed by: https://review.openstack.org/221712
    Add test to check that correct functions is used in expand/contract

Gerrit topic: https://review.openstack.org/#q,topic:bug/1478847,n,z

Addressed by: https://review.openstack.org/224334
    Merge remote-tracking branch 'origin/master' into merge-pecan-resolve-conflict

Gerrit topic: https://review.openstack.org/#q,topic:bug/1486039,n,z

Addressed by: https://review.openstack.org/224357
    Merge remote-tracking branch 'origin/master' into walnut

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.