GNU bug report logs -
#26201
Downloading substitutes is too slow upon nginx cache misses
Previous Next
Reported by: <dian_cecht <at> zoho.com>
Date: Tue, 21 Mar 2017 01:46:02 UTC
Severity: important
Tags: fixed
Done: ludo <at> gnu.org (Ludovic Courtès)
Bug is archived. No further changes may be made.
Full log
Message #90 received at 26201 <at> debbugs.gnu.org (full text, mbox):
Hello,
ludo <at> gnu.org (Ludovic Courtès) skribis:
> Other solutions I’ve thought about:
>
> 1. Produce narinfos and nars periodically rather than on-demand and
> serve them as static files.
>
> pros: better HTTP latency and bandwidth
> pros: allows us to add a Content-Length for nars
> cons: doesn’t reduce load on hydra.gnu.org
> cons: introduces arbitrary delays in delivering nars
> cons: difficult/expensive to know what new store items are available
>
> 2. Produce a narinfo and corresponding nar the first time they are
> requested. So, the first time we receive “GET foo.narinfo”, return
> 404 and spawn a thread to compute foo.narinfo and foo.nar. Return
> 200 only when both are ready.
>
> The precomputed nar{,info}s would be kept in a cache and we could
> make sure a narinfo and its nar have the same lifetime, which
> addresses one of the problems we have.
>
> pros: better HTTP latency and bandwidth
> pros: allows us to add a Content-Length for nars
> pros: helps keep narinfo/nar lifetime in sync
> cons: doesn’t reduce load on hydra.gnu.org
> cons: exposes inconsistency between the store contents and the HTTP
> response (you may get 404 even if the thing is actually in
> store), but maybe that’s not a problem
The ‘wip-publish-baking’ implements #2 as a new option to ‘guix
publish’. It gives some control on the upper bound on CPU usage since
we can specify how many worker threads are used.
I’ll finish it soon so we can experiment with it.
Thanks,
Ludo’.
This bug report was last modified 8 years and 55 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.