GNU bug report logs - #26201
Downloading substitutes is too slow upon nginx cache misses

Previous Next

Full log

View this message in rfc822 format

From: ludo <at> gnu.org (Ludovic Courtès)
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: 26201 <at> debbugs.gnu.org
Subject: bug#26201: hydra.gnu.org uses ‘guix publish’ for nars and narinfos
Date: Tue, 28 Mar 2017 16:47:14 +0200

Hey!

Tobias Geerinckx-Rice <me <at> tobias.gr> skribis:

> On 26/03/17 19:35, Tobias Geerinckx-Rice wrote:
>> I can try to do some simple tests tomorrow.
>
> Two observations:
>
> - ‘proxy_cache_lock_timeout’ alone won't suffice to serialise requests;
>   ‘proxy_cache_lock_age’ must also be set to an equally ridiculously
>   long span. Otherwise, multiple requests will still be sent to ‘guix
>   publish’ if they are more than 5s apart. Bleh.
>
>   (The problem then becomes that clients will stall while the file is
>    being cached, as explained by Mark. curl patiently waited.)

Setting ‘proxy_cache_lock_timeout’ to 5s is reasonable I think: if
you’re unlucky, you wait for 5 seconds, and then we get ‘guix publish’
threads serving the same request in parallel; in the most common case,
there’s only ever one instance of a given request being served at a
given time.

> - Say client A requests a nar from ‘guix publish’ (no nginx involved).
>   If another client requests the same nar while A's still downloading,
>   ‘guix publish’ will... silently drop A's connection?
>   I was not expecting this.

That would be a bug.  Do you have an easy way to reproduce?

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.