User defined passwords should be easier to define

Also, we're lacking docs on how to do this.

Actually, defining custom passwords doesn't work at all, e.g if you create a password file like:

[stack@instack ~]$ cat tripleo-overcloud-passwords.yaml
parameter_defaults:
  NeutronMetadataProxySharedSecret: apassword
  GlancePassword: apassword
  NovaPassword: apassword
  GnocchiPassword: apassword
  IronicPassword: apassword
  HeatPassword: apassword
  RabbitmqPassword: apassword
  RedisPassword: apassword
  TrovePassword: apassword
  CinderPassword: apassword
  SwiftPassword: apassword
  AdminToken: apassword
  SwiftHash: apassword
  HaproxyStatsPassword: apassword
  SaharaPassword: apassword
  ManilaPassword: apassword
  CeilometerSecret: apassword
  MistralPassword: apassword
  AodhPassword: apassword
  ZaqarPassword: apassword
  NeutronPassword: apassword
  DemoPassword: apassword
  CeilometerPassword: apassword
  AdminPassword: apassword
  MysqlClustercheckPassword: apassword
  HeatStackDomainPassword: apassword

Then deploy like:

openstack overcloud deploy --templates -e tripleo-overcloud-passwords.yaml

The stack create works, but we write out the random passwords to tripleo-overcloud-passwords, which means the call to os-cloud-config fails:

2016-08-10 10:25:02 [overcloud]: CREATE_COMPLETE Stack CREATE completed successfully
Stack overcloud CREATE_COMPLETE
/home/stack/.ssh/known_hosts updated.
Original contents retained as /home/stack/.ssh/known_hosts.old
The request you have made requires authentication. (HTTP 401) (Request-ID: req-67d6570d-147d-468d-91

[root@overcloud-controller-0 ~]# cat /etc/heat/heat.conf | grep apassword
connection = mysql+pymysql://heat:apassword@192.0.2.15/heat
password = apassword
password=apassword

So we need to fix this as a first step.

Fix proposed to branch: master
Review: https://review.openstack.org/376438

Review - https://review.openstack.org/#/c/376438/

Change abandoned by Saravanan KR (<email address hidden>) on branch: master
Review: https://review.openstack.org/376438
Reason: https://review.openstack.org/#/c/375544/ should solve the issue.

https://review.openstack.org/#/c/375544/ landed so we need to confirm if this is now fixed and thus can be closed

Ok so I can confirm the problems described in comment #2 are now fixed, I did the following:

1. Deploy an overcloud with a custom environment file specifying passwords (I got these by grepping in the tht tree so probably incomplete):

[stack@instack ~]$ cat password_env.yaml
parameter_defaults:
  AdminPassword: fooxyz
  AodhPassword: fooxyz
  BarbicanPassword: fooxyz
  CeilometerPassword: fooxyz
  CinderPassword: fooxyz
  ContrailConfigIfmapUserPassword: fooxyz
  ContrailControlIfmapUserPassword: fooxyz
  GlancePassword: fooxyz
  GnocchiPassword: fooxyz
  HAProxyStatsPassword: fooxyz
  HeatPassword: fooxyz
  HeatStackDomainAdminPassword: fooxyz
  IronicPassword: fooxyz
  ManilaNetappPassword: fooxyz
  ManilaPassword: fooxyz
  ManilaServiceInstancePassword: fooxyz
  MistralPassword: fooxyz
  MonitoringRabbitPassword: fooxyz
  MysqlClustercheckPassword: fooxyz
  MysqlRootPassword: fooxyz
  NeutronNuageVSDPassword: fooxyz
  NeutronPassword: fooxyz
  NovaPassword: fooxyz
  OpenDaylightPassword: fooxyz
  PcsdPassword: fooxyz
  PLUMgridL2GatewayPassword: fooxyz
  PLUMgridPassword: fooxyz
  RabbitPassword: fooxyz
  RedisPassword: fooxyz
  SaharaPassword: fooxyz
  SnmpdReadonlyUserPassword: fooxyz
  SwiftPassword: fooxyz

openstack overcloud deploy --templates /tmp/tripleo-heat-templates/ -e password_env.yaml

We can see that the password is now set as expected on the nodes:

[root@overcloud-controller-0 etc]# grep fooxyz heat/heat.conf
stack_domain_admin_password = fooxyz
connection = mysql+pymysql://heat:fooxyz@192.0.2.8/heat
rabbit_password = fooxyz
password = fooxyz
password=fooxyz
[root@overcloud-controller-0 etc]# grep fooxyz nova/nova.conf
password=fooxyz
password=fooxyz
connection=mysql+pymysql://nova_api:fooxyz@192.0.2.8/nova_api
rabbit_password=fooxyz
connection=mysql+pymysql://nova:fooxyz@192.0.2.8/nova

However, if I s/xyz/abc in the password environment, and deploy again to update the existing stack:

openstack overcloud deploy --templates /tmp/tripleo-heat-templates/ -e password_env.yaml

We then see:

    Error: /Stage[main]/Nova::Compute/Nova::Generic_service[compute]/Service[nova-compute]: Failed to call refresh: Could not restart Service[nova-compute]: Execution of '/usr/bin/systemctl restart openstack-nova-compute' returned 1: Job for openstack-nova-compute.service failed because the control process exited with error code. See "systemctl status openstack-nova-compute.service" and "journalctl -xe" for details.
    Error: /Stage[main]/Nova::Compute/Nova::Generic_service[compute]/Service[nova-compute]: Could not restart Service[nova-compute]: Execution of '/usr/bin/systemctl restart openstack-nova-compute' returned 1: Job for openstack-nova-compute.service failed because the control process exited with error code. See "systemctl status openstack-nova-compute.service" and "journalctl -xe" for details.

I assume this is an ordering issue, we must apply the rabbit password change and then restart nova before the config has been applied with the new password?

Ok to enumerate the tasks required to fix this:

1. Figure out ordering issues so the rabbit password can be changed on update

2. Output a password environment file in addition to the generated tripleo-overcloud-passwords file - the data contained will be the same, but the format will be directly consumable via heat similar to my example above.

3. Have the option to disable internal generation of random passwords, so the deployment will fail if an operator provided password file doesn't contain a password for a new service.

Thanks Steven for the direction.

I have tried a below commands on the controller after the failure,
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl start_app

After this, services are successfully started, I am looking in how to incorporate this change. I am also checking if there is an option in puppetlabs-rabbitmq to reset.

Fix proposed to branch: master
Review: https://review.openstack.org/402061

Fix proposed to branch: master
Review: https://review.openstack.org/414424

Fix proposed to branch: master
Review: https://review.openstack.org/414447

Reviewed: https://review.openstack.org/414424
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=8afc840f4dbd3b2cd811e7c9f6ad3db917caa264
Submitter: Jenkins
Branch: master

commit 8afc840f4dbd3b2cd811e7c9f6ad3db917caa264
Author: Saravanan KR <email address hidden>
Date: Fri Dec 23 10:19:38 2016 +0530

    Provided an option to disable the password generation

    Operators may choose to use the passwords provided by them and
    does not intend to generate during the deployment. Provided an
    option to disable password generation if requested by user explicity.
    It will be helpful to identify if there are any new passwords added
    which has to be added during the deployment update by the operator.
    By default, the password generation will be enabled.

    Partial-Bug: #1611704

    Change-Id: I141a0727db9f19f80712e2c40069622ecc22d969

Reviewed: https://review.openstack.org/414447
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=26b3ac7af1a2855266e5f65487b539cc2d284f9a
Submitter: Jenkins
Branch: master

commit 26b3ac7af1a2855266e5f65487b539cc2d284f9a
Author: Saravanan KR <email address hidden>
Date: Fri Dec 23 12:33:03 2016 +0530

    Provide an option to disable password generation on deployment

    Operators may choose to use the passwords provided by them and
    does not intend to generate during the deployment. Provided an
    option to disable password generation if requested by user explicity.
    It will be helpful to identify if there are any new passwords added
    which has to be added during the deployment update by the operator.
    By default, the password generation will be enabled.

    Partial-Bug: #1611704
    Depends-On: I141a0727db9f19f80712e2c40069622ecc22d969
    Change-Id: Iedfd7fd1456ed61cb9208532d15bdb63fbc01de9

Fix proposed to branch: master
Review: https://review.openstack.org/427044

Update on the progress:
> 1. Figure out ordering issues so the rabbit password can be changed on update
https://review.openstack.org/#/c/402061/ - In discussion

> 2. Output a password environment file in addition to the generated tripleo-overcloud-passwords file - the data contained will be the same, but the format will be directly consumable via heat similar to my example above.
https://review.openstack.org/#/c/427044/ - In progress

> 3. Have the option to disable internal generation of random passwords, so the deployment will fail if an operator provided password file doesn't contain a password for a new service.
Completed - https://review.openstack.org/#/c/414424/ && https://review.openstack.org/#/c/414447/

Reviewed: https://review.openstack.org/402061
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=76931e535c69ab437c4710dc749bc363f4533ea5
Submitter: Jenkins
Branch: master

commit 76931e535c69ab437c4710dc749bc363f4533ea5
Author: Saravanan KR <email address hidden>
Date: Thu Nov 24 19:35:59 2016 +0530

    Add support to changing the Rabbitmq password on update

    Rabbitmq Password is set on the fresh deployment, but during
    update, if the password is changed, it is modified in all config
    files including rabbitmq config. But the rabbitmq connection fails
    because the new password is not successful applied to rabbitmq.
    Setting the rabbitmq_user will invoke 'rabbitmqctl change_password'.

    Scenario: The password change is applied on Step1 when configuring
    Rabbitmq. Other services may be updated on different Steps. Till
    other services config is updated with new rabbitmq password, and
    restarted, the connections will get Access Denied response. It has
    cyclic dependency. So the passwords will be changes at Step1 and
    once all services are updated, the connections will work as is.

    Partial-Bug: #1611704
    Change-Id: I44865af3d5eb2d37eb648ac7227277e86c8fbc54

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/451742

Reviewed: https://review.openstack.org/451742
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=b5f07ccb1d1cacb122d8d7d78be1bf4661813504
Submitter: Jenkins
Branch: stable/newton

commit b5f07ccb1d1cacb122d8d7d78be1bf4661813504
Author: Saravanan KR <email address hidden>
Date: Thu Nov 24 19:35:59 2016 +0530

    Add support to changing the Rabbitmq password on update

    Rabbitmq Password is set on the fresh deployment, but during
    update, if the password is changed, it is modified in all config
    files including rabbitmq config. But the rabbitmq connection fails
    because the new password is not successful applied to rabbitmq.
    Setting the rabbitmq_user will invoke 'rabbitmqctl change_password'.

    Scenario: The password change is applied on Step1 when configuring
    Rabbitmq. Other services may be updated on different Steps. Till
    other services config is updated with new rabbitmq password, and
    restarted, the connections will get Access Denied response. It has
    cyclic dependency. So the passwords will be changes at Step1 and
    once all services are updated, the connections will work as is.

    Partial-Bug: #1611704
    Change-Id: I44865af3d5eb2d37eb648ac7227277e86c8fbc54
    (cherry picked from commit 76931e535c69ab437c4710dc749bc363f4533ea5)