Provide mechanism for extend_start to be customized by the underlay deployment tool

Registered by Steven Dake

Mesos and Ansible have slightly different underlay deployment mechanisms. Mesos downloads a bunch of files from zookeeper and then runs:
https://github.com/openstack/kolla-mesos/blob/master/services/mariadb/mariadb.yml.j2#L17

We agreed that where we can have a common architecture, we should do so, so this commands location is really in the wrong spot. It belongs in line 25 of this file (under a separate section). I had always envisioned the json files that control the configuration of the bootstrapping of the VM to be stored in one place or at least be of equivalent content as to avoid unnecessary deviation and to act as a hard ABI with version control. I think most people are on the same page on this idea.

Once that command is moved around, we can then may kolla_set_configs provide some type of output as to which kolla_extend_start.sh script should be run. This has the advantage of allowing zookeeper to fetch it in set_configs.py in the mesos case (or just copy from the host bindmount in the Ansible case).

Blueprint information

Status:
Complete
Approver:
None
Priority:
Essential
Drafter:
Steven Dake
Direction:
Needs approval
Assignee:
None
Definition:
Obsolete
Series goal:
None
Implementation:
Not started
Milestone target:
None
Completed by
Steven Dake

Related branches

Sprints

Whiteboard

Note I marked this essential because it affects the ABI and would be nice to finish before mitaka-3. I don't want the ansible code to require custom extend starts all throughout the containers - I want the default containers extend_start's kept in place as much as possible, but recognize Mesos needs its own extend_start.sh for whatever reason. Defining this reason would be extremely helpful to me so I can speak intelligently about it going forward. --sdake

I did a quick look at our(mesos) extend_start scripts right now, it’s only 2 services and both of them should be replaced with kolla ones. But I still think we should have an ability to run completely different extend_start scripts, or make sure kolla will care about compatibility with kolla-mesos, but I not sure we have enough people(in kolla-mesos) to catch all compatibility problemes between repos durin this rarid development phase. --kproskurin

Kolla completely cares about compatibility with kolla-mesos, or the core reviewer team wouldn't have voted unanimously to add the kolla-mesos repository to the TC governance repository as a deliverable. That said, kolla-mesos and kolla-ansible and kollla repos (once all split out whenever that happens, which we think is Newton) should be operating as *one* community with obviously some people more expert ins the mesos parts and some people more expert in the ansible parts. I am totally not satisfied with ABI incompatibility once rough prototyping is finished.

Lets make the ABI work for both repositories and version it properly. What we have now is two ABIs. I just propose fixing the one ABI problem that is clearly obvious which is the custom extend_start command operation. The other services in kolla-mesos YML files have some other options which may make sense to place into the set_configs.py ABI (the JSON or YML parsing), I'm not certain. If there are, please add them to the work items list as TODOs, as we need one ABI and they need to be merged. Note I am not anti-add-to-the-ABI but a common project absolutely definitely should have only one ABI not two. It is highly dangerous for a project to end up with two ABIs to the same functionality. Angus knows this I'm sure ;) --sdake

(asalkeld) Steve this is really not about ABI, it's about the ordering of operations (aka dependencies).
Ansible has a defined ordering by the start order of the containers, kolla-mesos starts everything at
the same time. So we have a system in place that allows use zookeeper to keep track of what
has been done. We want to use as much as possible all the extend starts, how ever we need
to orchestrate them within our "framework" so that the dependencies can be tracked.
I think the simplest solution here is to have something like:

https://github.com/openstack/kolla/blob/master/docker/base/start.sh#L29
if [ -n "$KOLLA_EARLY_EXTEND_START_DISABLE" ] ; then
source kolla_extend_start
fi

so that we can action the bootstrap within the context of our start script.

Gerrit topic: https://review.openstack.org/#q,topic:bp/custom-extend-start,n,z

Addressed by: https://review.openstack.org/283339
    Skip extend_start.sh if KOLLA_SKIP_EXTEND_START is defined

(?)

Work Items

Work items:
move command directive from line 17 to line 25 section across bootstrapping code in kolla-mesos repo: TODO
make set_configs.py fetch kolla_extend_start.sh from zookeeper and install it: TODO

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.