Glare resources

Registered by Renat Akhmerov on 2017-03-09

Now:

I as a user want to create a workflow that uses some additional resources like shell scripts (e.g. through ssh), binaries, images etc. I have the following options:

* Do all processing involving these resources out of mistral via asynchronous actions. We just delegate to an external system.
  - Pros:
    * Very generic and flexible, we can implement any processing logic in an external system (e.g. services with REST API)
  - Cons:
    * We have to create some external system that takes care of processing workflow tasks (more precisely, actions)
* If we need only shell scripts we can write them right in the workflow text
  - Pros:
    * We see the full text of the script right in the workflow
  - Cons:
    * If the script is big then workflow text becomes unreadable

The goal:

* Be able to use Glare as an external storage for workflow resources
* Define the syntax how references to Glare resources should look like in Mistral workflows
* Create a new Glare artifact type to be able to store workflows with resources in Glare
* Mistral CLI commands that allow to upload resources in Glare in a convenient way (e.g. one command to create a wf in Mistral and upload resources to Glare)

Solution ideas/steps:

* Keep all resources needed by a workflow in Glare so that the workflow operates only with references (URIs) pointing to needed resources. At a particular place where the references is used the content of the resource gets requested from Glare.
* Create a spec
* Config property pointing to Glare base url (glare_url=https://my.glare.com:9494/artefacts/workflows/)

Workflow example:

heal_vms_wf:
  input:
    - hosts
    - heal_vm_script: "1230123-123-123-123-123-123/content/heal_vm_script.sh"

  tasks:
    heal_vms:
      with-items: host in <% $.hosts %>
      action: std.ssh cmd=<% $. heal_vm_script %> host=<% $.host %>

Questions:

* How do we specify that a link points to a Glare artefact? (Imagine if we want to support multiple link types). It would be cool to have something like: glare://heal_vm_script.sh (base url is already in config)
* How do we address a resource in a convenient way w/o having to know artefact id? Idea: glare://{my_wf_artefact, 1.0}/heal_vm_script.sh
* How do we add Glare resources? Idea: mistral workflow-create my_wf.yaml --resource my_sh_script=my_sh_script.sh --resource my_image=my_image.iso

mistral workflow-create my_wf.yaml --resource my_sh_script=my_sh_script.sh --resource my_image=my_image.iso

workflowartifact:
        name:my_wf
        version:1.0
        workflow:my_wf.yaml (/artifacts/workflows/<id>/workflow) type:Blob
        resources:
                my_sh_script.sh (/artifacts/workflows/<id>/resources/my_sh_script.sh)
                my_image.iso /artifacts/workflows/<id>/resources/my_image.iso)

glare://artifact_name/resources_name (glare://my_wf/ my_sh_script.sh)

Links:

* https://github.com/openstack/python-glareclient/blob/master/glareclient/v1/artifacts.py#L251-L264
* http://paste.openstack.org/show/596680/

Blueprint information

Status:
Not started
Approver:
Renat Akhmerov
Priority:
High
Drafter:
Renat Akhmerov
Direction:
Approved
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Not started
Milestone target:
None

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.