New configuration strategy needed to adapt to oslo-messaging

Registered by Le Tian Ren

oslo messaging use oslo.config as configuration utility to setup Transport object. Here are 2 problems need to consider:
1)
oslo.messaging: parse options in the default section as amqp paramerters input
our current design: 2 message brokers to setup, options are parsed from [openstack] and [powervc] sections respectively.
2)
oslo.config.cfg.CONF is a single global configuration options container shared in a process.

A new configuration strategy is needed to solve the 2 problems.

Here is one proposal:

1. devide powervc.conf into powervc.conf, amqp-powervc.conf, amqp-openstack.conf.
2. all amqp related options in both [powervc] and [openstack] sections of powervc.conf will be removed.
3. all amqp related options for powervc are configured in amqp-powervc.conf.
4. all amqp related options for openstack are configured in amqp-openstack.conf.
5. at the beginning of a driver service launching,
  5a. initialize and parse for an oslo.config.cfg.ConfigOpts object with amqp-powervc.conf and save as amqp_powervc_conf.
  5b. initialize and parse for an oslo.config.cfg.ConfigOpts object with amqp-openstack.conf and save as amqp_openstack_conf. for neutron-powervc-agent there is a little difference due to control_exchange setup, currenlty we use another conf amqp-openstack-neutron.conf specifically instead of amqp-openstack.conf.
6. remove the two amqp conf file from the sys.argv list
7. then initialize and parse for the global oslo.config.cfg.CONF with powervc.conf and other confs like nova.conf, cinder.conf..
8. when create transport oslo.messaging.Transport, use corresponder amqp conf.

The new service launching commands are:

Glance:
python glance-powervc/bin/glance-powervc --config-file /etc/powervc/amqp-openstack.conf --config-file /etc/powervc/amqp-powervc.conf --config-file /etc/powervc/powervc.conf --logfile /var/log/powervc/glance-powervc.log

Neutron:
python neutron-powervc/bin/neutron-powervc-agent --config-file /etc/powervc/amqp-openstack-neutron.conf --config-file /etc/powervc/amqp-powervc.conf --config-file /etc/powervc/powervc-neutron.conf --config-file /etc/powervc/powervc.conf --logfile /var/log/powervc/neutron-powervc.log

Cinder:
python cinder-powervc/bin/cinder-powervc --config-file /etc/powervc/amqp-openstack.conf --config-file /etc/powervc/amqp-powervc.conf --config-file /etc/powervc/powervc.conf --config-file /etc/cinder/cinder.conf --logfile /var/log/powervc/cinder-powervc.log

Nova:
python nova-powervc/bin/nova-powervc --config-file /etc/powervc/amqp-openstack.conf --config-file /etc/powervc/amqp-powervc.conf --config-file /etc/powervc/powervc.conf --config-file /etc/nova/nova.conf --logfile /var/log/powervc/nova-powervc.log

Blueprint information

Status:
Not started
Approver:
Hydrogen Driver Team
Priority:
Undefined
Drafter:
Le Tian Ren
Direction:
Needs approval
Assignee:
Le Tian Ren
Definition:
Approved
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Aug 13. I'm gonna create a new branch in github for this experiment, so don't bother other fixes until it works well and become stable.

Aug 14. new branch created, https://github.com/stackforge/powervc-driver/tree/experimental

Sep 1. implementation done, https://review.openstack.org/#/c/116919/10

Gerrit topic: https://review.openstack.org/#q,topic:bug/1356225,n,z

Addressed by: https://review.openstack.org/116919
    New config strategy to adapt to oslo-messaging

Gerrit topic: https://review.openstack.org/#q,topic:bug/1370337,n,z

Addressed by: https://review.openstack.org/122067
    Upstart process daemon scripts need to change.

Addressed by: https://review.openstack.org/122082
    Upstart process daemon scripts need to change.

(?)

Work Items

Work items:
create new branch for long-lived features, https://github.com/stackforge/powervc-driver/tree/experimental : DONE
implementation, https://bugs.launchpad.net/powervc-driver/+bug/1356225 : DONE

This blueprint contains Public information 
Everyone can see this information.