Set-up "dumb" HTTP git hosting server on git.linaro.org

Registered by Данило Шеган

Provide a "dumb" HTTP git hosting server on git.linaro.org to allow repeated parallel cloning without bringing the server to its knees due to huge memory usage.

Blueprint information

Status:
Complete
Approver:
Данило Шеган
Priority:
High
Drafter:
Данило Шеган
Direction:
Approved
Assignee:
Georgy Redkozubov
Definition:
Approved
Series goal:
Accepted for trunk
Implementation:
Implemented
Milestone target:
milestone icon 2012.02
Started by
Данило Шеган
Completed by
Данило Шеган

Related branches

Sprints

Whiteboard

Notes:
Gesha: "git repack -a -d" is sufficient to make a repository accessible: it may increase repository size on disk (for arm-soc.git it increased from about 150 to 550M, no growth for linux-linaro-3.1.git)
Danilo: arm-soc.git had no git remote configuration (i.e. look at "config" file inside the repo or "git remote show"), for linux-linaro-3.1.git there was the config of 'origin' remote repository. In my testing, linux-linaro-3.1.git tree also increased in size with "git repack -a -d".
Danilo: git memory usage after repack is still large, we are moving forward.
Gesha: Smaller pack sizes result in large disk usage.
Danilo: Linux tree took 4.4G of disk space with small pack sizes and it didn't help with memory usage either.
Danilo: Repositories with alternates set-up: https://pastebin.linaro.org/394/
Danilo on 2012-02-26: split out remaining work into https://blueprints.launchpad.net/linaro-infrastructure-misc/+spec/dumb-http-git-server-part2

Meta:
Headline: Linaro Infrastructure now has uses a scalable HTTP git hosting solution for their default read-only access.

Acceptance: We have read-only git serving for /kernel/linux-linaro-3.1.git and /people/arnd/arm-soc.git

(?)

Work Items

Work items:
[danilo] Get /git-ro/ set up to serve git repositories via "dumb" HTTP: DONE
Get access to git.linaro.org: DONE
Investigate why eg. "git clone http://git.linaro.org/git-ro/kernel/linux-linaro-3.1.git/" doesn't work: DONE
Find minimal steps required to set it up properly so it's accessible from /git-ro/: DONE
Check that pushing over git protocol doesn't "break" the repository metadata: DONE
Check that potential local pulls/merges (eg. through cronjobs) do not "break" the repository metadata: DONE
Compare regular git protocol memory usage before and after repack: DONE
Need to investigate how using smaller packs (--max-pack-size) affects things: DONE
[danilo] Find repositories using objects/info/alternates: DONE
File an RT to update repositories using objects/info/alternates with objects/info/http-alternates: DONE
Announce to linaro-dev the process to use for hosting your own trees to make stuff available over dumb HTTP: POSTPONED
Update https://wiki.linaro.org/Process/GitHostingAccounts to describe desired set-up for dumb HTTP: POSTPONED

This blueprint contains Public information 
Everyone can see this information.