Enable custom roles

Registered by Steven Hardy on 2016-05-09

Beyond the composable services within roles requirement, there is a need to enable flexible definition of custom roles, e.g groups of nodes in addition to the fixed Controller, Compute and *Storage roles.

This will enable independently scaling certain services beyond the 3-controller topology, and also simplify integration of optional or 3rd party tools which require dedicated nodes (such as some SDN controller nodes).

Blueprint information

Steven Hardy
Steven Hardy
Steven Hardy
Series goal:
Accepted for newton
Milestone target:
milestone icon newton-rc1
Started by
Steven Hardy on 2016-05-09
Completed by
Emilien Macchi on 2016-09-17

Related branches



Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles,n,z

Addressed by: https://review.openstack.org/315679
    WIP prototyping fully composable/custom roles

Addressed by: https://review.openstack.org/336625
    WIP wire in jinja templating for custom roles

Addressed by: https://review.openstack.org/336150
    Remove depends_on: Networks from ControllerServiceChain

Addressed by: https://review.openstack.org/315616
    Replace NodeConfigIdentifiers with DeployIdentifier

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles_3,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles_5,n,z

Addressed by: https://review.openstack.org/342900
    Convert EndpointMap to not require per-service VIP parameters

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles_9,n,z

Addressed by: https://review.openstack.org/337587
    Move AllNodesValidationDeployments into jinja template loop

Addressed by: https://review.openstack.org/337267
    Move AllNodesDeployments into jinja template loop

Addressed by: https://review.openstack.org/347050
    Remove some properties from overcloud.yaml Controller group

Addressed by: https://review.openstack.org/347262
    Move per-role *SchedulerHints parameters into role templates

Addressed by: https://review.openstack.org/347263
    Remove KeyName parameter from overcloud template

Addressed by: https://review.openstack.org/347264
    Move *Image parameters into role templates

Addressed by: https://review.openstack.org/347265
    Move *Flavor parameters into per-role templates

Addressed by: https://review.openstack.org/347925
    Filter null/None service names

Addressed by: https://review.openstack.org/347926
    Remove Compute role parameters from overcloud.yaml

Addressed by: https://review.openstack.org/348441
    Add EnabledServices output to overcloud.yaml

Addressed by: https://review.openstack.org/348972
    Convert AllNodesConfig hosts config to a map

Addressed by: https://review.openstack.org/348973
    Create per-role node_names in hieradata for all nodes

Addressed by: https://review.openstack.org/348974
    WIP Create composable mapping between enabled services and role ips

Addressed by: https://review.openstack.org/345510
    Allow map_replace substitution of network names

Addressed by: https://review.openstack.org/353032
    Convert ServiceNetMap to a nested template

Addressed by: https://review.openstack.org/353637
    Align node_ips hiera keys with the service name.

Addressed by: https://review.openstack.org/353680
    Switch rabbit_hosts list to use hiera interpolation

Addressed by: https://review.openstack.org/346055
    Remove deprecated net_ip_subnet_map output

Addressed by: https://review.openstack.org/354381
    Remove deprecated net_ip_uri_map output

Addressed by: https://review.openstack.org/354391
    Remove unneeded net_ip_map dump into hieradata

Addressed by: https://review.openstack.org/354414
    Pass ServiceNetMap to services

Addressed by: https://review.openstack.org/318344
    Now fly and set cinder::api::bind_host within service template

Addressed by: https://review.openstack.org/355064
    Remove deprecated node_ips hiera keys

Addressed by: https://review.openstack.org/355065
    Rename NeutronServer to NeutronApi to align with ServiceNetMap

Addressed by: https://review.openstack.org/355066
    Add service_ips in csv format to allNodes hieradata

Addressed by: https://review.openstack.org/355067
    Move ceph cluster config out of allNodesConfig

Addressed by: https://review.openstack.org/355068
    Generate composable service node_names lists

Addressed by: https://review.openstack.org/356657
    Move out of roles any Ceph setting

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles2,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles3,n,z

Addressed by: https://review.openstack.org/361029
    Create hiera service_enabled keys for enabled services

Addressed by: https://review.openstack.org/359645
    Conditionally expose haproxy endpoints

Addressed by: https://review.openstack.org/361578
    Rename opendaylight service to opendaylight_api

Addressed by: https://review.openstack.org/361731
    Remove hard-coded per-service rabbitmq hieradata

Addressed by: https://review.openstack.org/361777
    Create NetIpListMap for all roles

Addressed by: https://review.openstack.org/361778
    Composable per-service VIP hieradata

Addressed by: https://review.openstack.org/362800
    Cleanup haproxy service from explicit services enablement

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-roles_16,n,z

Addressed by: https://review.openstack.org/364747
    Remove mangling swift memcached list (now in puppet-tripleo)

Addressed by: https://review.openstack.org/364748
    Convert SwiftDevicesAndProxyConfig to composable format

Addressed by: https://review.openstack.org/365735
    WIP allow overcloud.j2.yaml

Addressed by: https://review.openstack.org/365763
    Move role deployment steps into overcloud.yaml

Addressed by: https://review.openstack.org/364749
    Move per-role NetIpListMap's into jinja template loop

Addressed by: https://review.openstack.org/365791
    Move RedisVirtualIP hiera into VipMap

Addressed by: https://review.openstack.org/365792
    Remove *ExtraConfig parameters from overcloud.yaml

Addressed by: https://review.openstack.org/365793
    Move role ResourceGroups inside the jinja2 loop

Addressed by: https://review.openstack.org/365794
    Convert allNodesConfig properties to composable jinja2

Addressed by: https://review.openstack.org/365795
    Convert VipConfig list to jinja2

Addressed by: https://review.openstack.org/365796
    Convert deploy steps to jinja2 loop

Gerrit topic: https://review.openstack.org/#q,topic:bug/1618510,n,z

Addressed by: https://review.openstack.org/366128
    Set Redis VIP on all nodes

Addressed by: https://review.openstack.org/367282
    Convert UpdateWorkflow to support composable roles

Addressed by: https://review.openstack.org/367295
    Convert AllNodesExtraConfig to support composable roles

Addressed by: https://review.openstack.org/368801
    Get template contents from plan, not local path

Gerrit topic: https://review.openstack.org/#q,topic:bug/1577928,n,z

Addressed by: https://review.openstack.org/368821
    [squash] Required backports for stable/newton

Addressed by: https://review.openstack.org/370486
    Revert "Convert allNodesConfig properties to composable jinja2"

Addressed by: https://review.openstack.org/370855

Addressed by: https://review.openstack.org/392994
    Make service profiles default to rabbitmq_node_ips

Addressed by: https://review.openstack.org/452734
    Merge pre|post puppet resources into pre|post config.

Addressed by: https://review.openstack.org/459229
    Merge pre|post puppet resources into pre|post config.


Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.