Support private github repositories

Registered by Ravi Sankar Penta

The OpenStack user should be able to optionally specify private github/git repositories in the plan file.

Each plan file may have many artifacts and each artifact can specify source link to public or private git repository.
When private git repository is specified, a new field 'private' need to be set to true.
Example plan file with private git repo:

artifacts:
- name: <xyz>
  content:
    href: <email address hidden>/<some-private-repo>
    private: true
...

REST Api changes:
----------------
- Plan create (POST /plans api)
  * Support optional param 'private' for each artifact
  * If 'private' field is set, plan handler should create RSA ssh key pair (use python library)
  * Store generated public key along with other artifact information in existing 'raw_content' field in plan object.
  * Store private key in barbican: post credentials to /v1/secrets barbican REST api (use barbican python client)
    Persist barbican private key reference as 'deploy_keys_uri' field in plan object.
  * If plan has private repo, REST response should suggest the user to manually add solum generated public ssh key as deploy key for the corresponding repo in github.

  TODO: Once solum UI matures, we can suggest user to provide oauth privileges to solum by registering in github so that solum can automatically add generated public ssh key as deploy key in github.

- Plan(s) show (GET /plans or GET /plans/<plan-id> api)
  * Return solum generated public key for the artifact if present in the plan object.

- Plan delete (DELETE /plans/<plan-id> api)
  * Should clear solum generated private keys from barbican if present before destroying itself.

- Assembly create (POST /assemblies api)
  * Fetch plan object and if deploy_keys_uri is present, get private key from barbican and
    propagate these creds to the build step.

CLI changes:
------------
- CLI command 'solum app show/list' should include solum generated public key in case of private repo.

Buid-App changes:
-----------------
- build-app script will have a utility functions add_ssh_creds and remove_ssh_creds that wraps any git operations.
- add_ssh_creds will do these steps:
   a. Run ssh-agent in a shell
   b. Register private key with ssh-agent
- remove_ssh_creds will do these steps:
   d. Kill ssh-agent in (a)

Enable barbican/python-barbicanclient on devstack for testing.
Add python-barbicanclient to solum requirements.
Need alembic migration for new plan field 'deploy_keys_uri'.
In the workflow CLI-->REST-->Build-App, care should be taken not to log user credentials to terminal/logfile.

Blueprint information

Status:
Complete
Approver:
None
Priority:
Undefined
Drafter:
Ravi Sankar Penta
Direction:
Needs approval
Assignee:
Ravi Sankar Penta
Definition:
New
Series goal:
None
Implementation:
Implemented
Milestone target:
None
Started by
Adrian Otto
Completed by
Adrian Otto

Related branches

Sprints

Whiteboard

Great description.

Few points

- We should implement the run_git_command as a Python script and use Fabric for sending the passphrase instead of bash and expect.

- Good point about not logging user credentials.

Btw, did not understand the note that you have added.

Finally, all the above contents can be moved to the corresponding spec.

Gerrit topic: https://review.openstack.org/#q,topic:bug/1319604,n,z

Addressed by: https://review.openstack.org/103972
    WIP: Added support for private GitHub repos

Addressed by: https://review.openstack.org/105605
    WIP: Added support for private GitHub repos

Addressed by: https://review.openstack.org/109467
    WIP: Added support for private GitHub repos

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.