Models should be the same as the state in DB after migration

Registered by Sergey Kraynev

We are using declarative_base in cinder.db.sqlalchemy.models just for reflection, not for db creation.
For creation we are using migrations.
We don't have any tests that checks, that our models are up-to-date. Also we are testing it only on sqlite, which couldn't test such things as nullable constraints.

So We should:

1) Use always explicit nullable parameter for columns. There is a lot of mistakes in current cinder models implementation. (sometimes in schema is nullable=True and in model nullable=False and vise versa). Also at this moment when you see Column description without nullable, you couldn't be sure that is author forget to set nullable='False' or it is really nullable='True'. So the easiest way to track all this and to fix and to avoid such things is to have one rule "always use explicit nullable".
2) Add in __table_args__ indexes and unique constraints
3) Fix all mistakes in models
4) Fix all mistakes in migrations
5) Sync effects of migrations in different backends.
6) Add tests that ensures that models are up-to-date.

This will allow us to find probably some mistake, or missing indexes and make work with db more clean.

Blueprint information

Status:
Complete
Approver:
John Griffith
Priority:
Medium
Drafter:
Sergey Kraynev
Direction:
Approved
Assignee:
Ivan Kolodyazhny
Definition:
Obsolete
Series goal:
Accepted for alpha-trunk
Implementation:
Unknown
Milestone target:
milestone icon ongoing
Completed by
Sean McGinnis

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/db-sync-models-with-migrations,n,z

Addressed by: https://review.openstack.org/48935
    Update Lockutils from oslo

Addressed by: https://review.openstack.org/48936
    Add testing modules of migrations from oslo

Addressed by: https://review.openstack.org/48937
    Fix sqlalchemy models

Addressed by: https://review.openstack.org/48938
    Fix sqlalchemy migrations

Addressed by: https://review.openstack.org/66577
    Sync database table iscsi_targets with model IscsiTarget

Addressed by: https://review.openstack.org/66578
    Sync database table quotas with model state Quota

Addressed by: https://review.openstack.org/66579
    Sync database with state of models Encryption and QuotaUsages

<jdg>
I have no idea how to track this any longer. Overlapped projects, multiple patch sets with no explanation or updates to the whiteboard leaves things in a state where I have no idea what's completed, if you're complete if this is ongoing etc etc.

We really need to work on blueprints and updating info going forward. I'm removing this from target for now simply because I have no idea what's going on here any more and due to the confusion created by having the same BP name in multiple projects (things fall apart when you do this in terms of the auto-tracking and updating). I'd ask that in the future when creating the same bp in multiple projects we use the project name as a prefix or a suffix to denote the different contexts.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.