Enable custom roles

Registered by Steven Hardy

Beyond the composable services within roles requirement, there is a need to enable flexible definition of custom roles, e.g groups of nodes in addition to the fixed Controller, Compute and *Storage roles.

This will enable independently scaling certain services beyond the 3-controller topology, and also simplify integration of optional or 3rd party tools which require dedicated nodes (such as some SDN controller nodes).

Blueprint information

Status:
Complete
Approver:
Steven Hardy
Priority:
High
Drafter:
Steven Hardy
Direction:
Approved
Assignee:
Steven Hardy
Definition:
Approved
Series goal:
Accepted for newton
Implementation:
Implemented
Milestone target:
milestone icon newton-rc1
Started by
Steven Hardy
Completed by
Emilien Macchi

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles,n,z

Addressed by: https://review.openstack.org/315679
    WIP prototyping fully composable/custom roles

Addressed by: https://review.openstack.org/336625
    WIP wire in jinja templating for custom roles

Addressed by: https://review.openstack.org/336150
    Remove depends_on: Networks from ControllerServiceChain

Addressed by: https://review.openstack.org/315616
    Replace NodeConfigIdentifiers with DeployIdentifier

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles_3,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles_5,n,z

Addressed by: https://review.openstack.org/342900
    Convert EndpointMap to not require per-service VIP parameters

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles_9,n,z

Addressed by: https://review.openstack.org/337587
    Move AllNodesValidationDeployments into jinja template loop

Addressed by: https://review.openstack.org/337267
    Move AllNodesDeployments into jinja template loop

Addressed by: https://review.openstack.org/347050
    Remove some properties from overcloud.yaml Controller group

Addressed by: https://review.openstack.org/347262
    Move per-role *SchedulerHints parameters into role templates

Addressed by: https://review.openstack.org/347263
    Remove KeyName parameter from overcloud template

Addressed by: https://review.openstack.org/347264
    Move *Image parameters into role templates

Addressed by: https://review.openstack.org/347265
    Move *Flavor parameters into per-role templates

Addressed by: https://review.openstack.org/347925
    Filter null/None service names

Addressed by: https://review.openstack.org/347926
    Remove Compute role parameters from overcloud.yaml

Addressed by: https://review.openstack.org/348441
    Add EnabledServices output to overcloud.yaml

Addressed by: https://review.openstack.org/348972
    Convert AllNodesConfig hosts config to a map

Addressed by: https://review.openstack.org/348973
    Create per-role node_names in hieradata for all nodes

Addressed by: https://review.openstack.org/348974
    WIP Create composable mapping between enabled services and role ips

Addressed by: https://review.openstack.org/345510
    Allow map_replace substitution of network names

Addressed by: https://review.openstack.org/353032
    Convert ServiceNetMap to a nested template

Addressed by: https://review.openstack.org/353637
    Align node_ips hiera keys with the service name.

Addressed by: https://review.openstack.org/353680
    Switch rabbit_hosts list to use hiera interpolation

Addressed by: https://review.openstack.org/346055
    Remove deprecated net_ip_subnet_map output

Addressed by: https://review.openstack.org/354381
    Remove deprecated net_ip_uri_map output

Addressed by: https://review.openstack.org/354391
    Remove unneeded net_ip_map dump into hieradata

Addressed by: https://review.openstack.org/354414
    Pass ServiceNetMap to services

Addressed by: https://review.openstack.org/318344
    Now fly and set cinder::api::bind_host within service template

Addressed by: https://review.openstack.org/355064
    Remove deprecated node_ips hiera keys

Addressed by: https://review.openstack.org/355065
    Rename NeutronServer to NeutronApi to align with ServiceNetMap

Addressed by: https://review.openstack.org/355066
    Add service_ips in csv format to allNodes hieradata

Addressed by: https://review.openstack.org/355067
    Move ceph cluster config out of allNodesConfig

Addressed by: https://review.openstack.org/355068
    Generate composable service node_names lists

Addressed by: https://review.openstack.org/356657
    Move out of roles any Ceph setting

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles2,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles3,n,z

Addressed by: https://review.openstack.org/361029
    Create hiera service_enabled keys for enabled services

Addressed by: https://review.openstack.org/359645
    Conditionally expose haproxy endpoints

Addressed by: https://review.openstack.org/361578
    Rename opendaylight service to opendaylight_api

Addressed by: https://review.openstack.org/361731
    Remove hard-coded per-service rabbitmq hieradata

Addressed by: https://review.openstack.org/361777
    Create NetIpListMap for all roles

Addressed by: https://review.openstack.org/361778
    Composable per-service VIP hieradata

Addressed by: https://review.openstack.org/362800
    Cleanup haproxy service from explicit services enablement

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles_16,n,z

Addressed by: https://review.openstack.org/364747
    Remove mangling swift memcached list (now in puppet-tripleo)

Addressed by: https://review.openstack.org/364748
    Convert SwiftDevicesAndProxyConfig to composable format

Addressed by: https://review.openstack.org/365735
    WIP allow overcloud.j2.yaml

Addressed by: https://review.openstack.org/365763
    Move role deployment steps into overcloud.yaml

Addressed by: https://review.openstack.org/364749
    Move per-role NetIpListMap's into jinja template loop

Addressed by: https://review.openstack.org/365791
    Move RedisVirtualIP hiera into VipMap

Addressed by: https://review.openstack.org/365792
    Remove *ExtraConfig parameters from overcloud.yaml

Addressed by: https://review.openstack.org/365793
    Move role ResourceGroups inside the jinja2 loop

Addressed by: https://review.openstack.org/365794
    Convert allNodesConfig properties to composable jinja2

Addressed by: https://review.openstack.org/365795
    Convert VipConfig list to jinja2

Addressed by: https://review.openstack.org/365796
    Convert deploy steps to jinja2 loop

Gerrit topic: https://review.openstack.org/#q,topic:bug/1618510,n,z

Addressed by: https://review.openstack.org/366128
    Set Redis VIP on all nodes

Addressed by: https://review.openstack.org/367282
    Convert UpdateWorkflow to support composable roles

Addressed by: https://review.openstack.org/367295
    Convert AllNodesExtraConfig to support composable roles

Addressed by: https://review.openstack.org/368801
    Get template contents from plan, not local path

Gerrit topic: https://review.openstack.org/#q,topic:bug/1577928,n,z

Addressed by: https://review.openstack.org/368821
    [squash] Required backports for stable/newton

Addressed by: https://review.openstack.org/370486
    Revert "Convert allNodesConfig properties to composable jinja2"

Addressed by: https://review.openstack.org/370855
    TESTING

Addressed by: https://review.openstack.org/392994
    Make service profiles default to rabbitmq_node_ips

Addressed by: https://review.openstack.org/452734
    Merge pre|post puppet resources into pre|post config.

Addressed by: https://review.openstack.org/459229
    Merge pre|post puppet resources into pre|post config.

Gerrit topic: https://review.opendev.org/#/q/topic:custom-roles

Addressed by: https://review.opendev.org/745786
    Allows to use custom role name for validations

Addressed by: https://review.opendev.org/746635
    Allows to use custom role name for validations

Addressed by: https://review.opendev.org/747336
    Allows to use custom role name for validations

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.