GNU bug report logs - #26201
Downloading substitutes is too slow upon nginx cache misses

Previous Next

Package: guix;

Reported by: <dian_cecht <at> zoho.com>

Date: Tue, 21 Mar 2017 01:46:02 UTC

Severity: important

Tags: fixed

Done: ludo <at> gnu.org (Ludovic Courtès)

Bug is archived. No further changes may be made.

Full log


Message #90 received at 26201 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mark H Weaver <mhw <at> netris.org>
Cc: Tobias Geerinckx-Rice <me <at> tobias.gr>, 26201 <at> debbugs.gnu.org,
 guix-sysadmin <at> gnu.org
Subject: Re: bug#26201: hydra.gnu.org uses ‘guix publish’ for nars and narinfos
Date: Mon, 17 Apr 2017 23:36:06 +0200
Hello,

ludo <at> gnu.org (Ludovic Courtès) skribis:

> Other solutions I’ve thought about:
>
>   1. Produce narinfos and nars periodically rather than on-demand and
>      serve them as static files.
>
>      pros: better HTTP latency and bandwidth
>      pros: allows us to add a Content-Length for nars
>      cons: doesn’t reduce load on hydra.gnu.org
>      cons: introduces arbitrary delays in delivering nars
>      cons: difficult/expensive to know what new store items are available
>
>   2. Produce a narinfo and corresponding nar the first time they are
>      requested.  So, the first time we receive “GET foo.narinfo”, return
>      404 and spawn a thread to compute foo.narinfo and foo.nar.  Return
>      200 only when both are ready.
>
>      The precomputed nar{,info}s would be kept in a cache and we could
>      make sure a narinfo and its nar have the same lifetime, which
>      addresses one of the problems we have.
>
>      pros: better HTTP latency and bandwidth
>      pros: allows us to add a Content-Length for nars
>      pros: helps keep narinfo/nar lifetime in sync
>      cons: doesn’t reduce load on hydra.gnu.org
>      cons: exposes inconsistency between the store contents and the HTTP
>            response (you may get 404 even if the thing is actually in
>            store), but maybe that’s not a problem

The ‘wip-publish-baking’ implements #2 as a new option to ‘guix
publish’.  It gives some control on the upper bound on CPU usage since
we can specify how many worker threads are used.

I’ll finish it soon so we can experiment with it.

Thanks,
Ludo’.




This bug report was last modified 8 years and 55 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.