Get Keystone Running in Python 3

Registered by David Stanek on 2014-03-27

Keystone should run just as well under Python 3 as it does under Python 2. There is currently no way to run under Python 3 due to issues in Keystone itself as well as many third-party libraries. My goal is to get to the point where I can work on Keystone using Python3 as my primary Python version.

The keystone-py3kcompat blueprint (https://blueprints.launchpad.net/keystone/+spec/keystone-py3kcompat) aims at fixing many of the code issues using six.

This blueprint will target running Keystone on Python 3 by:
  - making the Jenkins job gate-keystone-python33 useful
  - dealing with issues in third-party libraries
  - fixing any code issues not addressed by keystone-py3kcompat

Blueprint information

Status:
Complete
Approver:
Steve Martinelli
Priority:
Low
Drafter:
David Stanek
Direction:
Approved
Assignee:
David Stanek
Definition:
Approved
Series goal:
Accepted for newton
Implementation:
Implemented
Milestone target:
milestone icon newton-1
Started by
Dolph Mathews on 2014-07-22
Completed by
Steve Martinelli on 2016-05-21

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/python3,n,z

Addressed by: https://review.openstack.org/83565
    Make the py33 Jenkins job happy

Gerrit topic: https://review.openstack.org/#q,topic:bp/keystone-py3kcompat,n,z

Addressed by: https://review.openstack.org/83832
    Moves database setup/teardown closer to its usage

Addressed by: https://review.openstack.org/83834
    First real Python 3 tests

Addressed by: https://review.openstack.org/84050
    Replace all use of mox with mock

Addressed by: https://review.openstack.org/84051
    setUp must be called on a fixture's parent first

Addressed by: https://review.openstack.org/84052
    Fix cache configuration checks

Addressed by: https://review.openstack.org/84053
    Fixed the size limit tests in Python 3

Addressed by: https://review.openstack.org/84054
    Fixed the policy tests in Python 3

Addressed by: https://review.openstack.org/84055
    Adds several more tests to the Python 3 test run

Addressed by: https://review.openstack.org/83833
    wip: this needs to be made in oslo

Addressed by: https://review.openstack.org/87980
    Sync with oslo-incubator 74ae271

Addressed by: https://review.openstack.org/99669
    Middleware tests now run under Python3

Addressed by: https://review.openstack.org/95827
    Adds a fork of python-ldap for Py3 testing

Addressed by: https://review.openstack.org/95826
    Updates Python3 requirements to match Python2

Addressed by: https://review.openstack.org/102624
    Adds oslo.db support for Python 3 tests

Addressed by: https://review.openstack.org/102734
    Fixes a Python3 syntax error

Addressed by: https://review.openstack.org/102735
    Adds several more test modules that pass on Py3

Addressed by: https://review.openstack.org/102736
    Fixes test_wsgi for Python3

Addressed by: https://review.openstack.org/102737
    Fixes test_exceptions.py for Python3

Gerrit topic: https://review.openstack.org/#q,topic:tmp,n,z

Addressed by: https://review.openstack.org/108808
    Disables LDAP unit tests

Addressed by: https://review.openstack.org/108809
    Reduces the amount of mocked imports for Python 3

Addressed by: https://review.openstack.org/108810
    Add the new oslo.i18n as a dependency for Python 3

Addressed by: https://review.openstack.org/115678
    Sync Py2 and Py3 requirements files

Addressed by: https://review.openstack.org/125409
    Mocks out the memcache library for tests

Addressed by: https://review.openstack.org/125410
    Fixes a type check to make it work in Python 3

Addressed by: https://review.openstack.org/130579
    Updates Python3 requirements

Gerrit topic: https://review.openstack.org/#q,topic:bp/removed-as-of-kilo,n,z

Addressed by: https://review.openstack.org/177402
    pep8 whitespace changes

Addressed by: https://review.openstack.org/177403
    Fixes order of imports for pep8

Addressed by: https://review.openstack.org/177404
    Ignore multiple imports per line for six.moves

Addressed by: https://review.openstack.org/177405
    Replaced filter with a list comprehension

Addressed by: https://review.openstack.org/177406
    eventlet now supports Python3

Addressed by: https://review.openstack.org/177407
    pycadf now supports Python3

Addressed by: https://review.openstack.org/177408
    Fixes mocking of oslo messaging for Python3

Addressed by: https://review.openstack.org/177409
    Updates the *py3 requirements files

Addressed by: https://review.openstack.org/177410
    Fixes use of dict methods for Python3

Addressed by: https://review.openstack.org/177411
    Handles Python3 builtin changes

Addressed by: https://review.openstack.org/177412
    Handles modules that moved in Python3

Addressed by: https://review.openstack.org/177413
    Fixes a whitespace issue

Addressed by: https://review.openstack.org/177414
    Add mocking for ldappool for Python3 tests

Addressed by: https://review.openstack.org/177415
    Fixes deprecations test for Python3

Addressed by: https://review.openstack.org/177416
    Refactor deprecations tests

Addressed by: https://review.openstack.org/177417
    Add mocking for memcache for Python3 tests

Addressed by: https://review.openstack.org/177418
    basestring no longer exists in Python3

Addressed by: https://review.openstack.org/181360
    Fixes broken federation test

Addressed by: https://review.openstack.org/198165
    Fix tox -e py34

Gerrit topic: https://review.openstack.org/#q,topic:py3,n,z

Addressed by: https://review.openstack.org/193866
    Python 3: Replace assertRaisesRegexp to its six implementation

Addressed by: https://review.openstack.org/203853
    test_base64utils works with py34

Addressed by: https://review.openstack.org/203892
    Fix pemutils for py34

Gerrit topic: https://review.openstack.org/#q,topic:py34,n,z

Addressed by: https://review.openstack.org/203896
    Fix test_utils for py34

Addressed by: https://review.openstack.org/203900
    Ensure database options registered for tests

Addressed by: https://review.openstack.org/204804
    Fix s3.core for py34

Addressed by: https://review.openstack.org/204807
    Fix test_exception.py for py34

Addressed by: https://review.openstack.org/205352
    test_backend_sql work with python34

Gerrit topic: https://review.openstack.org/#q,topic:iteritems,n,z

Addressed by: https://review.openstack.org/206785
    Fix SmarterEncoder for python3

Addressed by: https://review.openstack.org/231635
    Update test modules passing on py34

Addressed by: https://review.openstack.org/232711
    Handle fernet payload timestamp differences

Addressed by: https://review.openstack.org/231710
    Fix fernet key writing for python 3

Addressed by: https://review.openstack.org/231711
    Fix fernet padding for python 3

Addressed by: https://review.openstack.org/236096
    Fix key_repository_signature method for python3

Addressed by: https://review.openstack.org/237210
    Adds already passing tests to py34 run

Addressed by: https://review.openstack.org/237211
    Fixes kvs cache key mangling issue for Py3

Addressed by: https://review.openstack.org/260721
    Use ldap3 for DN comparison

Addressed by: https://review.openstack.org/262474
    Use six.moves.reload_module instead of builtin reload

Addressed by: https://review.openstack.org/261253
    Replace unicode with six.text_type

Gerrit topic: https://review.openstack.org/#q,topic:test_python3,n,z

Addressed by: https://review.openstack.org/271965
    test_credential.py work with python34

Addressed by: https://review.openstack.org/271886
    Consolidate `test_contrib_ec2.py` into `test_credential.py`

Addressed by: https://review.openstack.org/275541
    Initialize the policy engine where it is needed

Gerrit topic: https://review.openstack.org/#q,topic:python3,n,z

Addressed by: https://review.openstack.org/278053
    Uses open context manager for templated catalogs

Addressed by: https://review.openstack.org/278054
    Stop using nose as a Python3 test runner

Addressed by: https://review.openstack.org/278055
    Enables token_data_helper tests for Python3

Addressed by: https://review.openstack.org/278147
    Disable the ipv6 tests in py34

Addressed by: https://review.openstack.org/280671
    Enables the notification tests in py3

Addressed by: https://review.openstack.org/280672
    Fix keystone.common.wsgi to explicitly use bytes

Addressed by: https://review.openstack.org/280673
    Fixes the templated backend tests for Python3

Addressed by: https://review.openstack.org/280674
    Fixes to get cert tests running in Py3

Addressed by: https://review.openstack.org/280675
    Fixes hacking for Py3 tests

Addressed by: https://review.openstack.org/312060
    Port test_v2 unit test to Python 3

Addressed by: https://review.openstack.org/312061
    Port test_v3_auth unit test to Python 3

Addressed by: https://review.openstack.org/319469
    Add python 3 release note.

(?)

Work Items

Work items:
Temporarily remove broken dependencies from the Python 3 virtualenv: INPROGRESS
Temporarily disable the full test suite under Python 3: INPROGRESS
testtools (or fixtures) appears to fail under Python 3: INPROGRESS
replace mox with mock: TODO
See if there is a more automated way to manage the py3 requirements files: TODO
Python Paste contains syntax that is not Python 3 compatible: TODO
Remove eventlet dependency: TODO
oslo.messaging tries to pull in eventlet and should be fixed: TODO
pycadf seems to want eventlet too: TODO
pysqlite does not install on Python 3 (do we need it?): TODO
python-memcached does not install on Python 3: TODO
python-ldap does not install on Python 3: TODO

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.