GNU bug report logs - #26201
Downloading substitutes is too slow upon nginx cache misses

Previous Next

Full log

View this message in rfc822 format

From: ludo <at> gnu.org (Ludovic Courtès)
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: 26201 <at> debbugs.gnu.org, dian_cecht <at> zoho.com
Subject: bug#26201: No notification of cache misses when downloading substitutes
Date: Tue, 21 Mar 2017 17:43:29 +0100

Hello!

Tobias Geerinckx-Rice <me <at> tobias.gr> skribis:

> Oh, OK. I'm not an expert on how Hydra's set up these days, but will
> assume it's not too different from my own (a fast nginx proxy_cache,
> mirror.hydra.gnu.org, in front of a slower build farm, hydra.gnu.org).

I think there’s room for improvement in our nginx config at
<https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/nginx/mirror.conf>.

For instance, I just discovered ‘proxy_cache_lock’ while looking at
<http://nginx.org/en/docs/http/ngx_http_proxy_module.html>; looks useful
in reducing load on hydra.gnu.org.  Surely there are other ways to tweak
caching.

Besides, I’d like to use ‘guix publish’ on hydra.gnu.org.  I suspect
it’s going to be faster than Starman (the HTTP server behind Hydra), and
also it uses an in-process gzip by default, as opposed to bzip2 which is
what Hydra uses (better compression ratio, but super CPU-intensive).

At any rate, clients should not paper over server-side performance
issues IMO.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.