Cookbook refactoring (templates, attributes and libraries)

Registered by Jan Klare

During the mitaka cycle we should try to improve the usability and "patchability" of our cookbooks by removing some of the sticks and stones we added during the last years while implementing more and more features. One of the biggest issues in our cookbooks is the high amount of usecase specific attribtutes, which are scattered over multiple cookbooks. In addition to the simply too high amount of these attributes, we have two layers of attributes by default, which originate from the common cookbook and the underlying service cookbooks, which use some attributes set in common. These two layers of attributes complicate the wrapping of our cookbooks, since a wrapper create a third layer (and maybe a fourth if you also consider the environment or roles). I think we should follow the described steps below to make our cookbooks usable and more attractive again to a broader number of users (and possibly contributors)

1) To remove a lot of the attributes mentioned above, we should refactor our currently very static service configuration templates to use a more logic based approach like described here:
https://github.com/openstack/openstack-chef-specs/blob/master/specs/liberty/all/refactor_config_templates.rst
This approach also allows users to easily wrap the cookbook and add configuration options to the service configuration file, without touching the template at all.

2) To remove the complex interdependency between our cookbooks, we should try to move most of the attributes predefined in common and then inherited down to the service cookbooks, from common and just define them in the service ones. This completely removes the second layer of attributes and allows a lot easier wrapping (since you only have one layer of attributes per default)

3) To remove another big bunch of possibly unused and very usecase specific code and attributes, we should remove most of the deployment specific switchcases. A good example here is the current implementation of configurable neutron plugins. If we agree on a convention over configuration model here and use the same logic described in 1) to generate the plugins.ini, we can remove most of the plugin specific code and allow the user to set the needed configuration options in his very own wrapper cookbook (the same goes for most of the other switchcases, since most of them point to configuration files).

Besides the three steps described above, we should think about dropping the support for platforms with no active contributors and no contribution for over a year (currently suse). The refactoring should provide more generic cookbooks, which would make a reimplementation at a later point in time (or even in a wrapper) very easy in the case that someone wants to pick up the support and code maintenance.

The best point to start the refactoring is the common cookbook and its libraries. After the refactoring is in a final stage here, we should start the refactoring and implementation of the steps above into the "core" service cookbooks:

- openstack-identity
- openstack-compute
- opernstack-network
- openstack-block-storage
- openstack-image

Blueprint information

Status:
Complete
Approver:
Jan Klare
Priority:
Essential
Drafter:
Jan Klare
Direction:
Needs approval
Assignee:
Jan Klare
Definition:
Approved
Series goal:
Accepted for mitaka
Implementation:
Implemented
Milestone target:
None
Started by
Jan Klare
Completed by
Jan Klare

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:refactoring,n,z

Addressed by: https://review.openstack.org/250854
    refactoring final step

Gerrit topic: https://review.openstack.org/#q,topic:cleanup_libs,n,z

Addressed by: https://review.openstack.org/249133
    library cleanup and refactoring

Gerrit topic: https://review.openstack.org/#q,topic:bp/cookbook-refactoring,n,z

Addressed by: https://review.openstack.org/248767
    refactoring final step

Addressed by: https://review.openstack.org/254220
    WIP refactoring final step

Addressed by: https://review.openstack.org/253039
    refactoring final step

Addressed by: https://review.openstack.org/272651
    WIP adaptions to our repo to allow testing the refactored cookbooks

Addressed by: https://review.openstack.org/276147
    minor changes to adapt to refactored cookbooks

Addressed by: https://review.openstack.org/246322
    refactoring final step

Addressed by: https://review.openstack.org/272656
    adaptions to work with refactored cookbooks

Addressed by: https://review.openstack.org/272655
    adaptions to work with refactored cookbooks

Addressed by: https://review.openstack.org/278864
    database and message queue refactoring

Addressed by: https://review.openstack.org/278875
    use bind_service attribute instead of endpoints

Addressed by: https://review.openstack.org/278877
    use bind_service instead of endpoints and cluster properly

Addressed by: https://review.openstack.org/279705
    invert the order of endpoint and bind_service attributes

Addressed by: https://review.openstack.org/279708
    invert the order of endpoint and bind_service attributes

Addressed by: https://review.openstack.org/280346
    invert the order of endpoint and bind_service attributes

Addressed by: https://review.openstack.org/280548
    invert the order of endpoint and bind_service attributes

Addressed by: https://review.openstack.org/280549
    invert the order of endpoint and bind_service attributes

Addressed by: https://review.openstack.org/281845
    Use new bind_address method from Common to get address

Addressed by: https://review.openstack.org/281848
    Now using the new bind_address method from common

Gerrit topic: https://review.openstack.org/#q,topic:bind_address,n,z

Addressed by: https://review.openstack.org/288387
    Refactor using new style

Addressed by: https://review.openstack.org/289809
    add heat back to allinone role after refactoring

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.