Python Build/Install Process Simplification
The current python wheel/venv build process is not easily understood, and the install process has become complicated. This blueprint aims to work towards making it simpler to deploy, simpler to understand and to make many of the current features which are forced on deployers to be opt-in.
Blueprint information
- Status:
- Started
- Approver:
- Jean-Philippe Evrard
- Priority:
- Medium
- Drafter:
- Jesse Pretorius
- Direction:
- Approved
- Assignee:
- Jesse Pretorius
- Definition:
- Approved
- Series goal:
- None
- Implementation:
-
Started
- Milestone target:
- None
- Started by
- Jesse Pretorius
- Completed by
Related branches
Sprints
Whiteboard
Gerrit topic: https:/
Addressed by: https:/
Spec: Python Build/Install Process Simplification
Addressed by: https:/
Implement pypiserver and pypi proxy cache
Addressed by: https:/
Implement changes for pypi cache
Addressed by: https:/
Add ability to set the default index
Addressed by: https:/
[TEST] Check the existing global pip.conf contents
Addressed by: https:/
Set the default pip index to use the pypi cache when possible
Addressed by: https:/
Remove selfcheck.json
Addressed by: https:/
Add ability to disable implementing pip.conf
Addressed by: https:/
Remove selfcheck.json
Addressed by: https:/
Add find-links back to pip.conf
Addressed by: https:/
Allow the upstream pypi mirror to be changed
Addressed by: https:/
Add find-links, even when not being locked down
Addressed by: https:/
Correct location of venv build logs
Addressed by: https:/
[WIP] Make the git clone and wheel build optional
Addressed by: https:/
Correct location of venv build logs
Addressed by: https:/
Correct location of venv build logs
Addressed by: https:/
Correct location of venv build logs
Addressed by: https:/
Correct manifest index of openstack git folders
Addressed by: https:/
Use requirements_
Addressed by: https:/
Correct manifest index of openstack git folders
Addressed by: https:/
Correct manifest index of openstack git folders
Addressed by: https:/
Correct manifest index of openstack git folders
Addressed by: https:/
Replace host_need_pip with standard override mechanism
Addressed by: https:/
Remove host_need_pip
Addressed by: https:/
Remove pip_lock_
Addressed by: https:/
Remove pip_lock_
Addressed by: https:/
Change logging for source downloads and wheel build
Addressed by: https:/
Change logging for source downloads and wheel build
Gerrit topic: https:/
Addressed by: https:/
Remove the implementation of constraints in pip.conf
Addressed by: https:/
Remove the implementation of constraints in pip.conf
Addressed by: https:/
Remove pip_lock_
Addressed by: https:/
Remove pip_lock_
Addressed by: https:/
Replace host_need_pip with standard override mechanism
Addressed by: https:/
Remove host_need_pip
Addressed by: https:/
Remove default pip_links value
Addressed by: https:/
Use correct protocol for pypi reverse proxy
Addressed by: https:/
Use correct protocol for pypi reverse proxy
Addressed by: https:/
[WIP] Use a common python build role for source builds
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
tox.ini: Expose USER environment variable to execution environment
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Gerrit topic: https:/
Addressed by: https:/
ansible-
Addressed by: https:/
ansible-
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Use a common python build/install role for source-base deployment
Addressed by: https:/
Remove 'executable' arg from command task
Addressed by: https:/
Revert "Use a common python build/install role for source-base deployment"
Addressed by: https:/
Disable the pip version check
Addressed by: https:/
Ensure that the repo servers also use the pypi cache
Addressed by: https:/
Ensure that pip.conf templates variables properly
Addressed by: https:/
Override the pip index used for the wheel build
Addressed by: https:/
Allow pypiserver to be upgraded before repo build
Addressed by: https:/
Disable the pip version check
Addressed by: https:/
Ensure that repo checks use the right values
Addressed by: https:/
Allow repo host packages to be upgraded before repo build
Addressed by: https:/
Add pip_install_options to pip install tasks
Addressed by: https:/
Disable the pip version check
Addressed by: https:/
Disable the pip version check
Addressed by: https:/
Ensure that pip.conf templates variables properly
Addressed by: https:/
Ensure that pip.conf templates variables properly
Addressed by: https:/
Ensure that pip.conf templates variables properly
Addressed by: https:/
Allow repo host packages to be upgraded before repo build
Addressed by: https:/
Ensure that repo checks use the right values
Addressed by: https:/
Revise the role to properly cater to use-cases
Addressed by: https:/
Remove distro package indexing from py_pkgs lookup
Addressed by: https:/
Remove test-requirements files from py_pkgs lookup
Addressed by: https:/
Remove distro package indexing from py_pkgs lookup
Addressed by: https:/
Remove distro package indexing from py_pkgs lookup
Addressed by: https:/
Remove distro package indexing from py_pkgs lookup
Addressed by: https:/
Remove test-requirements files from py_pkgs lookup
Addressed by: https:/
Remove test-requirements files from py_pkgs lookup
Addressed by: https:/
Remove test-requirements files from py_pkgs lookup
Addressed by: https:/
Remove docs/requiremen
Addressed by: https:/
Remove docs/requiremen
Addressed by: https:/
Remove docs/requiremen
Addressed by: https:/
Remove docs/requiremen
Addressed by: https:/
Ensure that the repo servers also use the pypi cache
Addressed by: https:/
Remove requirements file names from output
Addressed by: https:/
Remove requirements file names from output
Addressed by: https:/
Remove requirements file names from output
Addressed by: https:/
Remove requirements file names from output
Addressed by: https:/
Remove default pip_links value
Addressed by: https:/
Remove *.txt file content from py_pkgs lookup output
Addressed by: https:/
Remove *.txt file content from py_pkgs lookup output
Addressed by: https:/
Remove *.txt file content from py_pkgs lookup output
Addressed by: https:/
Remove *.txt file content from py_pkgs lookup output
Addressed by: https:/
Cleanup directory creation tasks
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
[WIP] Use a common python build/install role for source-base deployment
Addressed by: https:/
[WIP] Use a common python build/install role (glance)
Addressed by: https:/
Remove zuul v2 shim
Addressed by: https:/
Use tests repo for common role test requirements
Addressed by: https:/
Use tests repo for common role test requirements
Addressed by: https:/
Document the purpose of the test-requiremen
Addressed by: https:/
Add shade & python-
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Install hacking first and let it pull in right linter versions
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Ensure clouds.yaml is placed correctly
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Ensure remote_user and become are consistently implemented
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Work around permissions issue downloading hatop
Addressed by: https:/
Use 'become' for haproxy service state setting
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Use local connection and become for host preparation
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Add cinder and openstack clients to Ansible venv
Addressed by: https:/
Add cinder and openstack clients to Ansible venv
Addressed by: https:/
Remove redundant apt cache update task
Addressed by: https:/
Switch connection/become settings for cinder
Addressed by: https:/
Use tests repo for common role test requirements
Addressed by: https:/
Use tests repo for common role test requirements
Addressed by: https:/
Use tests repo for common role test requirements
Addressed by: https:/
Reduce testing tech debt
Addressed by: https:/
[WIP] Add become to load kernel module
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Switch connection/become settings for nova
Addressed by: https:/
Remove localhost from neutron agents group
Addressed by: https:/
Switch compute to use different inventory_hostname
Addressed by: https:/
Switch compute to use different inventory_hostname
Addressed by: https:/
Switch compute to use different inventory_hostname
Addressed by: https:/
Switch compute/storage to use different inventory_hostname
Addressed by: https:/
Fixes compute error
Addressed by: https:/
Temporarily set functional test to non-voting
Addressed by: https:/
Make certificate generation host configurable
Addressed by: https:/
Revert "Temporarily set functional test to non-voting"
Addressed by: https:/
Remove the upstream pypi reverse proxy
Addressed by: https:/
Remove the upstream pypi reverse proxy
Addressed by: https:/
Use upstream pypi before the repo is built
Addressed by: https:/
Use upstream pypi before the repo is built
Addressed by: https:/
Use root remote_user for swift setup
Addressed by: https:/
Switch swift to use different inventory_hostname
Addressed by: https:/
Add rabbitmq/oslomsg groups to inventory
Addressed by: https:/
Add missing domain_id argument to os_project task
Addressed by: https:/
Implement clouds.yaml for swift installs
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Remove repo_requires_
Addressed by: https:/
Remove pip_install meta-dep
Addressed by: https:/
Move playbooks/config files into role examples
Addressed by: https:/
Remove rally_requires_
Addressed by: https:/
Use distribution packages for pip
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove pip_install role execution
Addressed by: https:/
Revert "defaults: Add virtualenv to default pip packages"
Addressed by: https:/
Allow the role to only configure pip, not install it
Addressed by: https:/
Revert "Revert "Install virtualenv for CentOS via distro packages""
Addressed by: https:/
Remove pip from packages installed
Addressed by: https:/
Eliminate installing pip on host/containers
Addressed by: https:/
Replace default pip index check with upper constraints check
Addressed by: https:/
Replace default pip index check with upper constraints check
Addressed by: https:/
Use repo_build_
Addressed by: https:/
Use repo_build_
Addressed by: https:/
Execute image setup against octavia_
Addressed by: https:/
Execute flavor setup against octavia_
Addressed by: https:/
Execute network setup against octavia_
Addressed by: https:/
Move role setup into service setup tasks
Addressed by: https:/
Execute security group setup against octavia_
Addressed by: https:/
Remove octavia_
Addressed by: https:/
Ensure the octavia flavor is private
Addressed by: https:/
Ensure the octavia network setup is in the right project
Addressed by: https:/
Add retries to service role creation
Addressed by: https:/
Use interface instead of endpoint_type
Addressed by: https:/
Clean up tests a bit
Addressed by: https:/
Add retries to all os_* module tasks
Addressed by: https:/
Use a venv to build the python wheels
Addressed by: https:/
Simplify pip/setuptools/
Addressed by: https:/
Only test for repo availability once
Addressed by: https:/
Execute service setup against a delegated host using Ansible built-in modules
Addressed by: https:/
Only test for repo availability once
Addressed by: https:/
Create virtualenv with '--never-download'
Addressed by: https:/
Remove unnecessary distro package installs
Addressed by: https:/
Execute service setup against a delegated host
Addressed by: https:/
Create virtualenv with '--never-download'
Addressed by: https:/
Remove pip_install role execution
Addressed by: https:/
Simplify pip/setuptools/
Addressed by: https:/
Remove pip from packages installed
Addressed by: https:/
Eliminate installing pip on host/containers
Addressed by: https:/
Remove the last remnants of get-pip.py
Addressed by: https:/
Add python-pip to distro packages temporarily
Addressed by: https:/
Revert "Add python-pip to distro packages temporarily"
Addressed by: https:/
Use the virtualenv's pip to build the wheels
Addressed by: https:/
Gather facts for openstack_openrc role
Addressed by: https:/
Clean up tests a bit
Addressed by: https:/
Gather facts for openstack_openrc role
Addressed by: https:/
Remove octavia_
Addressed by: https:/
Remove the last remnants of get-pip.py
Addressed by: https:/
Use the virtualenv's pip to build the wheels
Addressed by: https:/
Upgrade pip/setuptools/
Addressed by: https:/
Upgrade pip/setuptools/
Addressed by: https:/
Implement a simple task router to match tags
Addressed by: https:/
Correct cache_valid_time variable name
Addressed by: https:/
Upgrade pip/setuptools/
Addressed by: https:/
Correct cache_valid_time variable name
Addressed by: https:/
Create the venv with the right python version
Addressed by: https:/
Add --always-copy when building the venv
Addressed by: https:/
Only add --no-site-packages when necessary
Addressed by: https:/
Upgrade pip/setuptools/
Addressed by: https:/
Create the venv with the right python version
Addressed by: https:/
Add --always-copy when building the venv
Addressed by: https:/
Only add --no-site-packages when necessary
Addressed by: https:/
Add missing endpoint_type
Addressed by: https:/
Simplify builds/installs by eliminating venv packaging
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Consider pre-release wheels for installation
Addressed by: https:/
Add retries to wheel build task
Addressed by: https:/
Use updated syntax for task changed result
Addressed by: https:/
Use the same wheel path as the repo_build
Addressed by: https:/
Remove molteniron playbook mistakenly left behind
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Simplify builds/installs by eliminating venv packaging
Addressed by: https:/
Consider pre-release wheels for installation
Addressed by: https:/
Add retries to wheel build task
Addressed by: https:/
Use updated syntax for task changed result
Addressed by: https:/
Use the same wheel path as the repo_build
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Remove molteniron playbook mistakenly left behind
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Use a common python build/install role
Addressed by: https:/
Ensure dashboard plugins are found by py_pkgs
Addressed by: https:/
Move novnc python package install into common venv_build task
Addressed by: https:/
Upgrade pip/setuptools/
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Remove python-
Addressed by: https:/
Move heat domain setup into service setup tasks
Addressed by: https:/
[WIP] Remove venv build process
Addressed by: https:/
Remove unnecessary distro package installs
Addressed by: https:/
Move heat domain setup into service setup tasks
Addressed by: https:/
Use a common python build/install role
