GNU bug report logs -
#44193
[PATCH 0/1] 'guix publish --cache' can publish items not yet cached
Previous Next
Reported by: Ludovic Courtès <ludo <at> gnu.org>
Date: Sat, 24 Oct 2020 14:50:02 UTC
Severity: normal
Tags: patch
Done: Ludovic Courtès <ludo <at> gnu.org>
Bug is archived. No further changes may be made.
Full log
View this message in rfc822 format
[Message part 1 (text/plain, inline)]
Your bug report
#44193: [PATCH 0/1] 'guix publish --cache' can publish items not yet cached
which was filed against the guix-patches package, has been closed.
The explanation is attached below, along with your original report.
If you require more details, please reply to 44193 <at> debbugs.gnu.org.
--
44193: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=44193
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems
[Message part 2 (message/rfc822, inline)]
Ludovic Courtès <ludo <at> gnu.org> skribis:
> * guix/scripts/publish.scm (show-help, %options): Add
> '--cache-bypass-threshold'.
> (low-compression): New procedure.
> (cache-bypass-threshold): New parameter.
> (bypass-cache?): New procedure.
> (render-narinfo/cached): Call 'render-narinfo' when 'bypass-cache?'
> returns true.
> (render-nar/cached): Call 'render-nar' when 'bypass-cache?' returns
> true.
> (guix-publish): Parameterize 'cache-bypass-threshold'.
> * tests/publish.scm ("with cache", "with cache, lzip + gzip")
> ("with cache, uncompressed"): Pass '--cache-bypass-threshold=0'.
> ("with cache, vanishing item"): Expect 200 for RESPONSE.
> ("with cache, cache bypass"): New test.
> ---
> doc/guix.texi | 24 +++++++++++-
> guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++--------
> tests/publish.scm | 43 ++++++++++++++++++--
> 3 files changed, 130 insertions(+), 22 deletions(-)
Pushed as ecaa102a58ad3ab0b42e04a3d10d7c761c05ec98.
Ludo’.
[Message part 3 (message/rfc822, inline)]
Hello!
The ‘--cache’ mode of ‘guix publish’ is nice for many reasons. Until
now though, it would only return 200 to narinfo and nar requests
corresponding to items already in cache—already “baked”. Thus, the
first narinfo request for an item would always return 404; one would
have to wait until the item is baked to get 200 and download the
substitute.
If you’re fetching substitutes for popular packages on a popular
instance, the extra delay is most likely invisible. However, if you’re
using an instance with few users, or if you’re interesting in
substitutes for an package that’s not popular, the behavior described
above is a showstopper. (Many people here have criticized it in the
past. :-))
This patch changes the behavior of ‘--cache’: if a store item is not
yet in cache, and if it’s “small enough”, then narinfo/nar requests for
it immediately return 200, as in the no-cache mode. You’re trading
possibly increased server resource usage for reduced publication delay.
To put an upper bound on the extra resource usage, the
‘--cache-bypass-threshold’ allows users to control the maximum size
of a store item that can receive this treatment.
An interesting use case that can benefit from this change is .drv
substitutes: .drv themselves can be substitutes, so you can run:
guix build /gnu/store/….drv
and the daemon will fetch the .drv and its closure and start
building it (since commit 9c9982dc0c8c38ce3821b154b7e92509c1564317).
It’s not something we use much so far, but maybe we could put it
to good use in the future (I know Chris has been playing with it
in the context of the Data Service).
Thoughts?
Ludo’.
Ludovic Courtès (1):
publish: Add '--cache-bypass-threshold'.
doc/guix.texi | 24 +++++++++++-
guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++--------
tests/publish.scm | 43 ++++++++++++++++++--
3 files changed, 130 insertions(+), 22 deletions(-)
--
2.28.0
This bug report was last modified 4 years and 267 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.