Python 3 Support

Registered by Michael Terry on 2012-03-31

The aim is that we get to full Python 2.7+/3.4+ support in duplicity 0.8 and then cut off Python 2 support (move to Python 3 only) for 0.9.

We will use the http://python-future.org/futurize.html approach.

Blueprint information

Status:
Started
Approver:
Kenneth Loafman
Priority:
High
Drafter:
Aaron Whitehouse
Direction:
Approved
Assignee:
Aaron Whitehouse
Definition:
Approved
Series goal:
Proposed for 0.8-series
Implementation:
Slow progress
Milestone target:
milestone icon 0.8.00
Started by
Aaron Whitehouse on 2018-06-10

Whiteboard

Original draft Blueprint by Michael Terry (https://launchpad.net/~mterry).

=== Approach ===
Please see the various mailing list discussions about this:
https://<email address hidden>/msg04613.html
https://<email address hidden>/msg04614.html
https://lists.nongnu.org/archive/html/duplicity-talk/2015-07/msg00018.html
(and threads linked within).

The aim is that we get to full Python 2.7+/3.4+ support in duplicity 0.8 and then cut off Python 2 support (move to Python 3 only) for 0.9.

One of the big risks is that we are working on a Python 3 branch separate to a working Python 2 version and these get out of sync (as has happened in previous attempts). We therefore want to push all compatible changes back into trunk ASAP.

We will use the http://python-future.org/futurize.html approach.

The futurize tool spits out code looks and feels like Python 3, but runs on Python 2.7 with imports that make those Python 2 functions act like their Python 3 equivalents. We can then maintain just the one branch.

Then in 0.9 we can drop the compatibility imports.

One of the most time consuming, and least easy to automate, parts of supporting both Python 2 and 3 is string literals. This is because simple strings (e.g. a = "Hello") will be treated as bytes (e.g. encoded ASCII) in Python 2 and Unicode in Python 3. As we are trying to support both Python 2 and Python 3 for at least a transition period, we may end up with odd behaviour wherever we have an unadorned string. The versions of Python 2 and 3 we are targeting means that we can "adorn" strings with letters to indicate what type of string (u for Unicode or b for Bytes). An important preliminary step to Python 2/3 support is therefore for us to add these adornments to each and every string literal in the code base. See
https://blueprints.launchpad.net/duplicity/+spec/adorn-string-literals
for details on this preliminary task.

(?)

Work Items

Work items:
Investigate list of dependencies that still need Python 3 support. See: https://<email address hidden>/msg04613.html : TODO
mark string literals with u/b https://blueprints.launchpad.net/duplicity/+spec/adorn-string-literals : INPROGRESS
convert str() to unicode()/bytes() -- we can potentially futurize all str()s to bytes() and then chip away at converting bytes to (unicode) strings over time: INPROGRESS
http://python-future.org/futurize.html stage 1, which should still be valid Python 2 code: TODO
http://python-future.org/futurize.html stage 2. Set up tox etc for both Python 3 and Python 2 testing. fix remaining 2to3 exceptions (https://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/testing/test_code.py test_2to3) that are not done by futurize. All needs to happen in one go, with risk of falling out of sync with trunk: TODO
Post version 0.8, run proper 2to3 and remove all the 2/3 compatibility to be straight Python 3: TODO

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.