Set up a git code hosting staging server with RhodeCode
We need to set up a staging server that will mimick the future production staging server and be able to import git repositories periodically from the production server. We want to use RhodeCode for the interface, but since we might require some changes to the tree, we'll want to be able to easily deploy our own tree, and we want to keep the configuration in a separate branch where changes go through review and QA before they are automatically deployed.
Blueprint information
- Status:
- Complete
- Approver:
- Alan Bennett
- Priority:
- High
- Drafter:
- Milo Casagrande
- Direction:
- Approved
- Assignee:
- Stevan Radaković
- Definition:
- Approved
- Series goal:
- Accepted for trunk
- Implementation:
-
Implemented
- Milestone target:
-
2013.02
- Started by
- Milo Casagrande
- Completed by
- Alan Bennett
Related branches
Related bugs
Sprints
Whiteboard
Meta:
Headline: Set up a staging server for git code hosting.
Acceptance: There is a staging server that allows setting up new repositories (including private and restricted repositories) and users (syncs them with login.linaro.org LDAP). It automatically updates from our production git code hosting server for easier testing. We've got clear deployment steps that will help us deploy and keep up to date a production server as well.
Roadmap id: CARD-148
[danilo, 2013-01-16] See also deployment plan that Gesha has prepared (we've already got new server set up for staging): https:/
[stevanr 2013-01-16] Added work items.
[stevanr 2013-01-16] Milo has asked IS to grant us the access to git.linaro.org. RT ticket: https:/
[milo, 2013-01-17] SSH access to git.linaro.org has been provided.
[milo, 2013-01-17] Created git.linaro.org repository: http://
[milo, 2013-01-17] Danilo sent us a simple script to copy the repositories in git.l.o: https:/
[milo, 2013-01-17] Configuration repository created at: http://
[stevanr 2013-01-18] Added first versions of INSTALL and production.ini file to rhodecode-
[milo, 2013-01-18] Added initial script to automate installation process, code available in rhodeoce-config branch.
[milo, 2012-01-23] Script to automate installation and INSTALL file instructions are available here:
http://
[milo, 2012-01-24] Script to clone git repositories from git.l.o is available here: http://
[milo, 2012-01-24] Added txt file with directories listing of git repositories to clone: listing does not include people's repositories.
[stevanr, 2012-01-28] Upgrade modifications for setup scripts ready at http://
[milo, 2013-01-29] RhodeCode LDAP integration does not support groups syncing: it is necessary to write a script to automate that process. All the other configurations are straightforward and performed via the web interface.
[milo, 2013-01-29] RhodeCode is up and running at: http://
[milo, 2013-01-30] LDAP has been configured on the staging instance, login should be possible with Linaro credentials.
[milo, 2013-01-30] 151 git repositories are now mirrored on staging instance: issue with RhodeCode showing only a subset in the home page, but they are accessible with full URL.
[milo, 2013-01-30] The problem that only a subset of groups are shown is a bug: http://
[stevanr, 2013-01-30] Setup script updated with 'Update Rhodecode instance' section.
[stevanr, 2013-01-31] Document describing potential problems and implementation details for git access over SSH keys for public repositories can be found at http://
[milo, 2013-01-31] File ticket #802 for Canonical IS asking about private git repositories hosted on git.linaro.org and how to easily mirror them with the staging server.
[milo, 2013-01-31] Repositories already in the staging instance are being mirrored correctly, cron job runs each weekday at 1am. Missing repositories: private ones, and peoples.
[milo, 2013-02-01] staging.
[stevanr, 2013-02-01] Nice LDAP setup tutorial for local environments and testing: http://
[milo, 2013-02-04] Added info about RhodeCode private repositories in the SETUP file from rhodecode-config branch.
[milo, 2013-02-05] Added new work item to not forget about repositories that are not real directories on git.l.o but symlinks, like people but also others too.
[stevanr, 2013-02-12] Update script already takes care of backing up the previous machine state, so marking the auto-check script WI as POSTPONED.
[stevanr, 2013-03-03] Opened two bugs for last two WI's, marking them as POSTPONED and blueprint as implemented.
[stevanr, 2013-03-03] Bugs opened: https:/
Work Items
Work items:
[milo] Get access to git.linaro.org: DONE
[stevanr] Create virtual environment on local machine and install Rhodecode on LTS: DONE
[milo] Clone Rhodecode into git.linaro.org: DONE
[stevanr] Find out how to install Rhodecode from source: DONE
[milo] Create config repository with all the necessary setup files: DONE
[milo] Write a script to automate the initial setup process: DONE
[stevanr] Write a script to automate future updates: DONE
[milo] Get the ec2 box set up with new configuration: DONE
[milo] Clone all the repositories to staging server: DONE
[milo] Automate repositories update with a script: DONE
[milo] Investigate LDAP support and potential issues with ownership: DONE
[stevanr] Eplore potential issues with setting up the SSH keys for public repositories access to git server: DONE
[milo] Investigate rhodecode private git repositories: DONE
[stevanr] Write a script for quick testing the RhodeCode instance after the upgrade is performed: POSTPONED
[stevanr] Write script to sync LDAP groups with RhodeCode database using its API: DONE
[stevanr] Work with Marcin from Rhodecode to get the script upstream and deploy afterwords: POSTPONED
[stevanr] Check repositories on git.l.o that are symlinked in the filesystem (people, and others too): POSTPONED
Dependency tree
* Blueprints in grey have been implemented.
