Split openstack.common.db code into a separate oslo.db library

Registered by Viktor Serhieiev on 2013-11-20

The DB API is pretty stable now, so it should be released as a separate package.

To get oslo.db released as separate library, we should follow the next steps:
  - solve the possible conflicts with the OpenStack projects due to defining the same oslo.config options in openstack common modules (at least lockutils and log)
  - clean up oslo.db code remove dependency on eventlet
  - sync openstack.common.db code version into all projects
  - make oslo.db library

Proof of concept of oslo.db module is here: https://github.com/malor/oslo.db
POC oslo.db module in action: https://review.openstack.org/#/c/42159/

Blueprint information

Doug Hellmann
Viktor Serhieiev
Viktor Serhieiev
Series goal:
Accepted for juno
Milestone target:
milestone icon juno-2
Started by
Viktor Serhieiev on 2013-12-04
Completed by
Viktor Serhieiev on 2014-07-01


Do you have a plan for handling the config option issue? - dhellmann

There are several ways to solve the config option issue:

    I - Move lockutils and log out of incubation first.
In such case we must release oslo.lockutils and oslo.log first, and use these modules in all projects. (it’s going to take a long time...)
    II - Remove oslo.config options from openstack.common modules (and update all project as well). (seems to be very hard to do)
    III - Drop dependency on common log and lockutils modules.
Seems to be the simplest way, so we started to implement it.
There are patches, that remove common log usage from oslo.db

... and patches, that drop dependency on common lockutils
https://review.openstack.org/#/c/56354/ - (already merged)

- viktors

To clean up oslo.db code from eventlet - see mail list thread http://lists.openstack.org/pipermail/openstack-dev/2013-December/020805.html

In a few fords - we use eventlet to wrap DB API method calls to be executed by tpool thread.
We can't just drop tpool support at all, so we should put the tpool wrapper class in a separate module which would be completely optional for users of the library.

- viktors

Gerrit topic: https://review.openstack.org/#q,topic:bp/oslo-db-lib,n,z

Addressed by: https://review.openstack.org/58475
    Drop dependency on processutils from oslo db code (merged)

Addressed by: https://review.openstack.org/59215
    Drop dependency on fileutils from oslo db tests (merged)

Gerrit topic: https://review.openstack.org/#q,topic:I6fa140f2b1f03c43ae7cab98ac8a12d71188af76,n,z

Addressed by: https://review.openstack.org/60031
    Remove eventlet tpool from common db.api (merged)

Addressed by: https://review.openstack.org/57689
    Drop dependency on log from oslo db code (merged)

Gerrit topic: https://review.openstack.org/#/q/status:open+project:openstack/oslo-incubator+branch:master+topic:remove_global_engine,n,z
(patches, related to bug 1263908)

Addressed by: https://review.openstack.org/63782
    Use ContextFormatter for imparting context info (merged)

... and one more review chain from Ben
2014-01-31T14:42:27 <bnemec> viktors: Tail end of the review chain: https://review.openstack.org/#/c/69032/
See this chain at gerrit: https://review.openstack.org/#/q/status:open+project:openstack/oslo-incubator+branch:master+topic:bug-1271706,n,z

Addressed by: https://review.openstack.org/72984
    Restore the ability to load the DB backend lazily (merged)

Addressed by: https://review.openstack.org/71874
    Refactor database migration manager to use given engine

Addressed by: https://review.openstack.org/74737
    Remove CONF.database.connection default value (merged)

Addressed by: https://review.openstack.org/76916
    Add from_config() method to EngineFacade (merged)

Addressed by: https://review.openstack.org/78663
    Drop dependency oslo.db from common.log (merged)

Addressed by: https://review.openstack.org/87536
    Use oslotest instead of common test module (merged)

oslo-db-core is set up in gerrit with viktors and oslo-core as members. If we have any other reviewers, they still need to be added. - dhellmann (2014-04-30)

I suppose, that we should also add Poman Podoliaka to oslo-db-core. -- viktors (2014-05-12)

Addressed by: https://review.openstack.org/100175
    Mark db code as obsolete


Work Items

Work items:
Create Initial Repository - See https://github.com/malor/oslo.db: DONE
Update MAINTAINERS in incubator with status and name - See https://review.openstack.org/#/c/88551 (Mark db code as graduating) : DONE
Run graduate.sh: DONE
Fix the output of graduate.sh: DONE
Publish git repo: See https://github.com/malor/oslo.db: DONE
Oslo team review new repository: DONE
openstack-infra/config - See patch https://review.openstack.org/#/c/78955/ : DONE
openstack-infra/config - modules/openstack_project/files/review.projects.yaml: DONE
openstack-infra/config - modules/openstack_project/files/gerrit/acls/stackforge/project-name.config: DONE
openstack-infra/config - modules/openstack_project/files/jenkins_job_builder/config/projects.yaml: DONE
openstack-infra/config - modules/openstack_project/files/zuul/layout.yaml: DONE
Update Gerrit Groups and ACLs: DONE
openstack/requirements projects.txt - See https://review.openstack.org/#/c/92890 : DONE
openstack/governance reference/programs.yaml. See https://review.openstack.org/90127 : DONE
Update list of libraries on Oslo wiki page - See https://wiki.openstack.org/wiki/Oslo#oslo.db : DONE
Update .gitreview in new repository https://review.openstack.org/#/c/91384 : DONE
Make the library do something: DONE
Give openstackci Owner permissions on PyPI: DONE
Tag a release: DONE
Mark graduated code obsolete in oslo-incubator: https://review.openstack.org/100175 (Mark db code as obsolete) : DONE
openstack/requirements - global-requirements.txt - https://review.openstack.org/#/c/91407/ : DONE
openstack-infra/devstack-gate - devstack-vm-gate-wrap.sh See https://review.openstack.org/#/c/94173/ (Add oslo.db to the devstack gate) : DONE
openstack-dev/devstack - lib/oslo https://review.openstack.org/91405 : DONE
openstack-dev/devstack - *install* the code https://review.openstack.org/#/c/100991/: DONE
openstack-dev/devstack - stackrc https://review.openstack.org/91405 : DONE
Update project list on docs.openstack.org See https://review.openstack.org/#/c/96707/ (Add oslo.db link to the documentation) : DONE

This blueprint contains Public information 
Everyone can see this information.