GNU bug report logs - #52805
[PATCH] substitute: Add a ‘--clear-cache’ option.

Previous Next

To reply to this bug, email your comments to 52805 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: guix-patches <at> gnu.org
Subject: [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Sun, 26 Dec 2021 17:21:26 +0100

Unlike ‘rm -rf ~/.cache/guix/substitute’ this respects XDG_CACHE_HOME.

Use the --OPTION form for consistency with the existing ‘guix substitute’
interface, where other guix subcommands would prefer an ACTION.  This
subcommand really supports only one OPTION at a time, anyway.

* guix/scripts/substitute.scm (guix-substitute):
Add a new ‘--clear-cache’ option argument.
(show-help): Document it.
---
 guix/scripts/substitute.scm | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
index c044e1d47a..1e71b4848b 100755
--- a/guix/scripts/substitute.scm
+++ b/guix/scripts/substitute.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2014 Nikita Karetnikov <nikita <at> karetnikov.org>
 ;;; Copyright © 2018 Kyle Meyer <kyle <at> kyleam.com>
 ;;; Copyright © 2020 Christopher Baines <mail <at> cbaines.net>
+;;; Copyright © 2021 Tobias Geerinckx-Rice <me <at> tobias.gr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -40,7 +41,7 @@ (define-module (guix scripts substitute)
   #:use-module (guix cache)
   #:use-module (gcrypt pk-crypto)
   #:use-module (guix pki)
-  #:use-module ((guix build utils) #:select (mkdir-p))
+  #:use-module ((guix build utils) #:select (delete-file-recursively mkdir-p))
   #:use-module ((guix build download)
                 #:select (uri-abbreviation nar-uri-abbreviation
                           (open-connection-for-uri
@@ -239,6 +240,8 @@ (define-syntax with-networking
 (define (show-help)
   (display (G_ "Usage: guix substitute [OPTION]...
 Internal tool to substitute a pre-built binary to a local build.\n"))
+  (display (G_ "
+      --clear-cache      forget previously cached availability information"))
   (display (G_ "
       --query            report on the availability of substitutes for the
                          store file names passed on the standard input"))
@@ -749,6 +752,10 @@ (define reply-port
   (with-networking
    (with-error-handling                           ; for signature errors
      (match args
+       (("--clear-cache")
+        ;; This is meant to be a big hammer, so don't pussyfoot around with
+        ;; maybe-remove-expired-cache-entries.
+        (delete-file-recursively %narinfo-cache-directory))
        (("--query")
         (let ((acl (current-acl)))
           (let loop ((command (read-line)))
-- 
2.34.0

Message #8 received at 52805 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: 52805 <at> debbugs.gnu.org
Subject: Re: bug#52805: [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Mon, 27 Dec 2021 22:12:28 +0100

Hello Tobias,

> Unlike ‘rm -rf ~/.cache/guix/substitute’ this respects XDG_CACHE_HOME.

This script is invoked by the daemon as root. In that context,
%narinfo-cache-directory is "/var/guix/substitute/cache", whereas when
run as an user it is likely: "~/.cache/guix/substitute".

That would mean that users need to run "sudo guix substitute
--clear-cache". Moreover this script is an internal undocumented tool,
so I'm not in favor of adding user commands to it.

Maybe "guix gc" would be a better place?

Thanks,

Mathieu

Message #11 received at 52805 <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 52805 <at> debbugs.gnu.org
Subject: Re: bug#52805: [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Mon, 27 Dec 2021 22:20:16 +0100

[Message part 1 (text/plain, inline)]

Hi Mathieu,

Thanks for the review!

Mathieu Othacehe 写道：
> This script is invoked by the daemon as root. In that context,
> %narinfo-cache-directory is "/var/guix/substitute/cache", 
> whereas when
> run as an user it is likely: "~/.cache/guix/substitute".

Correct.

> That would mean that users need to run "sudo guix substitute
> --clear-cache".

If they want to clear /var instead of ~/.cache, sure.

That sounds like a sane interface, if clearing ‘root’'s cache were 
the intention?  How would you even do that without sudo, 
privilege-wise?

> Moreover this script is an internal undocumented tool,
> so I'm not in favor of adding user commands to it.

OK.

The point was to have a little command I could ask other users to 
run, that's guaranteed to clear the correct directory, and doesn't 
involve a spooky rm (which saves some back-and-forth :-).

> Maybe "guix gc" would be a better place?

No, I'd rather not add yet another option to ‘guix gc’ that has 
nothing to do with GC.

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 52805 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>, Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 52805 <at> debbugs.gnu.org
Subject: Re: [bug#52805] [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Wed, 29 Dec 2021 17:49:28 +0100

Hi Tobias,

On Mon, 27 Dec 2021 at 22:20, Tobias Geerinckx-Rice via Guix-patches via <guix-patches <at> gnu.org> wrote:

>> Maybe "guix gc" would be a better place?
>
> No, I'd rather not add yet another option to ‘guix gc’ that has
> nothing to do with GC.

Why?  Well, garbage collection is about automatic memory management and
cache is about efficiency of memory usage.  Therefore, it is appears to
me relevant to have an option “guix gc --clear-cache” to force the clean
of various caches.

Cheers,
simon

Message #17 received at 52805 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 52805 <at> debbugs.gnu.org
Subject: Re: bug#52805: [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Wed, 05 Jan 2022 21:59:44 +0100

Hello,

Tobias Geerinckx-Rice <me <at> tobias.gr> skribis:

> Mathieu Othacehe 写道：

[...]

>> Moreover this script is an internal undocumented tool,
>> so I'm not in favor of adding user commands to it.

+1

> The point was to have a little command I could ask other users to run,
> that's guaranteed to clear the correct directory, and doesn't involve
> a spooky rm (which saves some back-and-forth :-).

In which case do users have to ‘rm -rf ~/.cache/guix/substitute’?

That directory is only used by ‘guix weather’.  In that case,
‘--clear-cache’ could be added to ‘guix weather’ maybe?

Now, ideally there would not be any cache-related option because the
cache would always be valid.  If it’s not, it would be nice to see how
that can be improved or mitigated.

Thoughts?

Ludo’.

Message #20 received at 52805 <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 52805 <at> debbugs.gnu.org
Subject: Re: bug#52805: [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Wed, 05 Jan 2022 23:41:58 +0100

[Message part 1 (text/plain, inline)]

Hi Ludo'!

Ludovic Courtès 写道：
> That directory is only used by ‘guix weather’.

Let me back up then: why does ‘guix weather’ cache responses at 
all?

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 52805 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 52805 <at> debbugs.gnu.org
Subject: Re: bug#52805: [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Tue, 11 Jan 2022 13:47:33 +0100

Hi,

Tobias Geerinckx-Rice <me <at> tobias.gr> skribis:

> Ludovic Courtès 写道：
>> That directory is only used by ‘guix weather’.
>
> Let me back up then: why does ‘guix weather’ cache responses at all?

For the same reason as ‘guix substitute’: caching allows it to send only
requests for which it doesn’t already know the answer.

Ludo’.

Message #26 received at 52805 <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 52805 <at> debbugs.gnu.org
Subject: Re: bug#52805: [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Tue, 11 Jan 2022 19:00:19 +0100

[Message part 1 (text/plain, inline)]

Hiya Ludo',

Ludovic Courtès 写道：
> For the same reason as ‘guix substitute’: caching allows it to 
> send only
> requests for which it doesn’t already know the answer.

Thanks!  The problem is this reason doesn't make sense.

N.B. I agree it's an acceptable trade-off for ‘guix substitute’, 
which has different operating parameters.

When I run ‘guix weather’ twice in a row, I am *never* interested 
in reviewing the stale numbers I got the first time (or worse: a 
mix of stale and new numbers).  I don't care if the data is very 
fresh: I am asking for fresher.

Caching is never to my advantage: it is ‘saving’ only the work I 
asked it to perform, against my wishes.  ‘Look how fast I fed you 
possibly bogus data against your will!’ is not an impressive 
boast.

Is my concern clear?  Reasonable?

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #29 received at 52805 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 52805 <at> debbugs.gnu.org
Subject: Re: bug#52805: [PATCH] substitute: Add a ‘--clear-cache’ option.
Date: Wed, 12 Jan 2022 22:04:37 +0100

Hi,

Tobias Geerinckx-Rice <me <at> tobias.gr> skribis:

> Caching is never to my advantage: it is ‘saving’ only the work I asked
> it to perform, against my wishes.  ‘Look how fast I fed you possibly
> bogus data against your will!’ is not an impressive boast.
>
> Is my concern clear?  Reasonable?

It’s clear, but I’m not sure it’s reasonable.  :-)

When do you have to worry about caches?

‘guix weather’ can cache negative replies (404s) for
%narinfo-negative-ttl = 10mn, unless the server provides a different
‘Cache-Control’ TTL.  So there’s a possibility that during that time
window it’ll tell you “nope” when in fact the thing has just arrived.

I think that’s acceptable (there’s a slight delay but nothing’s
“bogus”), and it’s kind to server admins.

(It’s possible for admins to run ‘guix publish --negative-ttl=0’ if they
don’t mind extra traffic from users.)

That said, if you find it sufficiently annoying, what about adding
‘--clear-cache’ to ‘guix weather’?

HTH!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.