Pull integration of Solum from an external Git repo

Registered by Devdatta Kulkarni on 2013-11-22

This blueprint targets the functionality to pull source code from an external Git repository into Solum.

Proposed workflow:

Solum will provide a API endpoint against which users would be able to do a POST with url of an existing Github repository in the request body. Solum will perform appropriate checks on the specified url, and if they pass, will clone the repository and store it for further processing.

Work Items (Adding them here as the Work Items section seems not to be working for me):

Following work items are envisioned:

1) Define and implement the appropriate API resource

2) Define and implement a mechanism to allow specification and enforcement of security rules for the Github repository urls. Some of the example rules could be:
- allowed structure of url (e.g.: ^http://github.com/[a-z/-]+$)
- whitelisted urls
- blacklisted urls

3) Investigate and implement the functionality of cloning the specified git repository
     3.1) Investigation piece 1: Whether Github's API can be used for this purpose or do we need to use git cli wrapped in Fabric for this.
     3.2) Investigation piece 2: Where do we store the cloned git repo? We need this so that steps further in the workflow can use it for actions such as detecting the language pack required, compiling, packaging, etc. Two options that jump out are:
    - storing the repo in some temporary known location on the API server
    - storing the repo in a Swift container

Blueprint information

Adrian Otto
Devdatta Kulkarni
Krishna Raman
Series goal:
Accepted for icehouse
Milestone target:
milestone icon 2014.1.1
Started by
Adrian Otto on 2014-04-01
Completed by
Adrian Otto on 2014-04-08

Related branches



KR: (2) If we treat user repo as upstream then we probably don't need a full clone of the repository. We probably only need to record the URL of repo and a commit hash.
KR: (2, 3.2) We also dont need a full clone. Most likely a a shallow clone or git-archive is enough.
KR: (1) If user registers the remote git URL in the plan document then we just need to expose a simple REST API which accepts github json. @see https://help.github.com/articles/post-receive-hooks#the-payload
KR: (3.1) Github provides the ability to call any web accessible URL



Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.