Composable overcloud upgrades

Registered by Jiří Stránský

The core idea is to switch upgrades from node-focused approach to service-focused approach. Instead of constructing the upgrades workflow and code around which nodes to upgrade, we'd base them around which services to upgrade, and a service upgrade would be cloud-wide, touching all nodes where such service is deployed. This might mean a complete rewrite of our upgrades approach.

Blueprint information

Status:
Complete
Approver:
Steven Hardy
Priority:
Essential
Drafter:
Jiří Stránský
Direction:
Approved
Assignee:
Steven Hardy
Definition:
Approved
Series goal:
Accepted for ocata
Implementation:
Implemented
Milestone target:
milestone icon ocata-rc1
Started by
Steven Hardy
Completed by
Emilien Macchi

Related branches

Sprints

Whiteboard

https://etherpad.openstack.org/p/tripleo-composable-upgrades

https://review.openstack.org/319264

Gerrit topic: https://review.openstack.org/#q,topic:bp/overcloud-upgrades-per-service,n,z

Addressed by: https://review.openstack.org/324330
    Upgrades: structure and keystone upgrade

Addressed by: https://review.openstack.org/325835
    [WIP] Composable upgrades for heat-api and heat-engine

Addressed by: https://review.openstack.org/324331
    Composable upgrades: glance

Addressed by: https://review.openstack.org/325860
    Composable upgrades: cinder

Addressed by: https://review.openstack.org/393448
    WIP prototyping composable upgrades with Heat+Ansible

Addressed by: https://review.openstack.org/397859
    Add heat ansible agent to overcloud-full image

Addressed by: https://review.openstack.org/400728
    Default UpgradeLevelNovaCompute to "auto"

Addressed by: https://review.openstack.org/403441
    Saving some updates "DO NOT MERGE"

Addressed by: https://review.openstack.org/405207
    Add heat-agent-ansible to overcloud-images.yaml

Addressed by: https://review.openstack.org/405240
    Add glance service support for composable upgrades

Addressed by: https://review.openstack.org/405241
    Add nova service support for composable upgrades

Addressed by: https://review.openstack.org/405242
    Add neutron service support for composable upgrades

Addressed by: https://review.openstack.org/405243
    Add cinder service support for composable upgrades

Addressed by: https://review.openstack.org/405244
    Add heat service support for composable upgrades

Addressed by: https://review.openstack.org/405245
    Add swift service support for composable upgrades

Addressed by: https://review.openstack.org/405246
    Add mistral service support for composable upgrades

Addressed by: https://review.openstack.org/405247
    Add sahara service support for composable upgrades

Addressed by: https://review.openstack.org/405248
    Add ironic service support for composable upgrades

Addressed by: https://review.openstack.org/405249
    Add telemetry service support for composable upgrades

Addressed by: https://review.openstack.org/405250
    Add snmp service support for composable upgrades

Addressed by: https://review.openstack.org/409857
    Run upgrade steps before post-deploy config

Addressed by: https://review.openstack.org/411310
    Move UpgradeInitCommand to role templates

Addressed by: https://review.openstack.org/403397
    WIP -- Add pacemaker ansible module of composable upgrade

Addressed by: https://review.openstack.org/408631
    Adds a step0 for pre upgrade-init checks

Addressed by: https://review.openstack.org/412494
    WIP - Add the pacemaker ansible modules during upgrade init

Addressed by: https://review.openstack.org/414598
    Adds a pre-upgrade check that service is running (step0)

Gerrit topic: https://review.openstack.org/#q,topic:bug/1655651,n,z

Addressed by: https://review.openstack.org/419886
    Adds a AllNodesPreUpgradeConfig before the upgrade steps

Addressed by: https://review.openstack.org/423160
    Add support for batched upgrades to composable upgrades

Addressed by: https://review.openstack.org/423182
    Add upgrade support for ceph-mon service

Addressed by: https://review.openstack.org/418344
    Add disable_upgrade_deployment flag to roles_data.yaml

Addressed by: https://review.openstack.org/423350
    Add upgrade support for ceph OSD service

Addressed by: https://review.openstack.org/405625
    Add support for not using admin_token in Ceph/RGW

Addressed by: https://review.openstack.org/423360
    Add upgrade support for CephRGW service

Addressed by: https://review.openstack.org/423384
    Skip upgrade steps where no tasks are defined

Addressed by: https://review.openstack.org/424600
    Skip the PostDeploySteps on roles that disable_upgrade_deployment

Addressed by: https://review.openstack.org/424661
    Dump and run puppet after each compute upgrade

Addressed by: https://review.openstack.org/424715
    Adds newton_ocata_upgrade_init_common.sh to the upgrade init

Addressed by: https://review.openstack.org/424905
    Disable puppet on upgrade for roles not upgrading

Addressed by: https://review.openstack.org/425086
    Add support for rendering j2.fragment files

Addressed by: https://review.openstack.org/425154
    Use the new hiera hook in all remaining templates

Addressed by: https://review.openstack.org/425220
    Add support for jinja2 includes

Addressed by: https://review.openstack.org/424072
    Add SkipUpgradeConfigTags for upgrade config

Addressed by: https://review.openstack.org/425638
    Add support for Jinja2 includes

Addressed by: https://review.openstack.org/425659
    Do not try to update the 'ceph' metapackage from CephMon role

Addressed by: https://review.openstack.org/426919
    Deprecate legacy Ceph upgrade scripts

Addressed by: https://review.openstack.org/426901
    Remove 'step' parameter to use the same mechanisme for skip-tags

Addressed by: https://review.openstack.org/427629
    Don't run ceilometer-upgrade via upgrade_tasks

Addressed by: https://review.openstack.org/428288
    Remove special handling for OVS upgrade

Addressed by: https://review.openstack.org/428309
    Simplify/fix config enabled conditions for upgrades

Addressed by: https://review.openstack.org/428310
    Reduce number of steps for upgrades

Gerrit topic: https://review.openstack.org/#q,topic:bp/overcloud-updates-per-service,n,z

Addressed by: https://review.openstack.org/428348
    Disable batch upgrade deployments for disabled roles

Gerrit topic: https://review.openstack.org/#q,topic:bp/overcloud-updates-per-service_fixup,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bug/1662344,n,z

Addressed by: https://review.openstack.org/430357
    Make UpgradeBatch resource depends on BatchConfig step0

Addressed by: https://review.openstack.org/431513
    Add upgrade tasks for opstools services

Addressed by: https://review.openstack.org/431349
    Remove aodh db sync from ansible upgrade

Addressed by: https://review.openstack.org/414562
    Adds cluster status check_and_fail before running upgrade (step0)

Gerrit topic: https://review.openstack.org/#q,topic:placement/upgrade,n,z

Addressed by: https://review.openstack.org/433588
    Add sensu and fluentd client to major upgrade environment

Addressed by: https://review.openstack.org/434468
    Add major-upgrade-converge environment.

Addressed by: https://review.openstack.org/434795
    Add Newton to Ocata UpgradeInitCommonCommand

Addressed by: https://review.openstack.org/435190
    Add Newton to Ocata UpgradeInitCommonCommand

Addressed by: https://review.openstack.org/425285
    Add support for upgrading ec2-api

Addressed by: https://review.openstack.org/438289
    Add auditd upgrate steps

Addressed by: https://review.openstack.org/438288
    Add etcd composable upgrade steps

Addressed by: https://review.openstack.org/438298
    Add upgrade task for panko api

Addressed by: https://review.openstack.org/438462
    Install openstack-panko-api on upgrade

Addressed by: https://review.openstack.org/438466
    Install openstack-heat-agents on upgrade

Addressed by: https://review.openstack.org/438091
    Make sure we don't stop service when pacemaker is handling them.

Addressed by: https://review.openstack.org/437545
    Add upgrade support for Horizon

Addressed by: https://review.openstack.org/438689
    Adds upgrade tasks for OpenDaylight services

Addressed by: https://review.openstack.org/435906
    Add zaqar service for composable upgrade

Addressed by: https://review.openstack.org/438090
    Put service stop at step1 and quiesce at step2.

Addressed by: https://review.openstack.org/438959
    Add validation for VPP upgrade tasks

Addressed by: https://review.openstack.org/438657
    upgrades/validation: only run validation when services exist

Addressed by: https://review.openstack.org/439042
    Add support for upgrading ec2-api

Addressed by: https://review.openstack.org/439046
    Add zaqar service for composable upgrade

Addressed by: https://review.openstack.org/439047
    Add auditd upgrate steps

Addressed by: https://review.openstack.org/439037
    upgrades: fix ec2api conditional

Addressed by: https://review.openstack.org/439370
    upgrades: fix ec2api conditional

Addressed by: https://review.openstack.org/439550
    Add etcd composable upgrade steps

Addressed by: https://review.openstack.org/439553
    Add upgrade support for Horizon

Addressed by: https://review.openstack.org/439025
    WIP Upgrades: fix up the rabbitmq HA mode like in new ocata deployments

Addressed by: https://review.openstack.org/439095
    Add missing upgrade step for pacemaker controlled resources.

Addressed by: https://review.openstack.org/439807
    Put service stop at step1 and quiesce at step2.

Addressed by: https://review.openstack.org/436990
    Remove the openvswitch special case in tripleo_upgrade_node.sh

Addressed by: https://review.openstack.org/440488
    upgrades/validation: only run validation when services exist

Addressed by: https://review.openstack.org/440489
    Add mistral service support for composable upgrades

Addressed by: https://review.openstack.org/440516
    Upgrades: fix up the rabbitmq HA mode like in new ocata deployments

Addressed by: https://review.openstack.org/441184
    Remove the openvswitch special case upgrade code

Addressed by: https://review.openstack.org/442303
    Use the new hiera hook in all remaining templates

Addressed by: https://review.openstack.org/443329
    Adds upgrade tasks for OpenDaylight services

Addressed by: https://review.openstack.org/441967
    Remove double quotes in the "when" Ansible conditional.

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.