Comment 9 for bug 1167379

Revision history for this message
Milo Casagrande (milo) wrote : Re: [Bug 1167379] Re: Clone from staging.git.l.o is very slow when using dumb protocol

On Tue, Apr 16, 2013 at 4:32 PM, Stevan Radaković
<email address hidden> wrote:
> Dumping some thoughts for us to test already mentioned on IRC...
> I know apache isn't the bottleneck but if we lift some weight from it, we might get some improvements, so I think it's worth giving the options below a shot:
>
> 1. git-http-backend: it can be configured with apache without interfering with Rhodecode. Found this article http://p0l0.binware.org/index.php/2011/08/26/git-over-http-git-http-backend/
> Of course it should be configured with accelerated static for apache 2.x options, which can be found in official git-http-backend documentation here https://www.kernel.org/pub/software/scm/git/docs/git-http-backend.html

That is exactly what we are doing on staging.android.git.l.o.

The problem I see with that approach, at least with RhodeCode in mind,
is that RhodeCode becomes sort of useless.
We would need to bypass it if we want to do like on staging android,
or we sort of end up doing the same thing that RhodeCode was doing
before with the smart protocol: calling and external process, running
git and passing the output to the original request. That might become
again the bottleneck.

Bypassing RhodeCode means also we will lose all the access control features.

> 2. Play around with mpm_worker setup and try mpm_prefork as well. Like
> reducing the start server and spare threads might free up some memory
> and cpu power for us to make things go faster. And prefork has proven
> better in some concurrent testing cases for me a while ago. Again, we
> should think about the options and test various settings.

I'm trying locally with that, cloning 8 repositories at the same time.
At the moment I do not see much difference on Apache side.
The big CPU usage always happens from RhodeCode.

> Also, I'm bumping the priority on this one because we mustn't go to
> production if we don't at least something out.

We will definitely not go in production at this stage. We will
probably lift git.l.o as is now, and move it to another server.
We'll still work on this during this week, sync up with Alan tomorrow,
and then probably with Philip in the next days.

--
Milo Casagrande | Automation Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs