GNU bug report logs - #63817
[PATCH] substitute: Delete cached narinfos more than two-month old.

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#63817: closed ([PATCH] substitute: Delete cached narinfos
 more than two-month old.)
Date: Thu, 08 Jun 2023 21:51:01 +0000

[Message part 1 (text/plain, inline)]

Your message dated Thu, 08 Jun 2023 23:49:50 +0200
with message-id <87h6rhy681.fsf <at> gnu.org>
and subject line Re: bug#63817: [PATCH] substitute: Delete cached narinfos more than two-month old.
has caused the debbugs.gnu.org bug report #63817,
regarding [PATCH] substitute: Delete cached narinfos more than two-month old.
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
63817: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=63817
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH] substitute: Delete cached narinfos more than two-month old.
Date: Wed, 31 May 2023 23:22:01 +0200

This allows 'guix substitute' to shrink the cache a bit more, which
saves space and improves performance of cache-cleanup phases since fewer
entries need to be traversed.

* guix/scripts/substitute.scm (cached-narinfo-expiration-time): Define
'max-ttl' and use it as an upper bound.
---
 guix/scripts/substitute.scm | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

Hi!

You might have noticed that /var/guix/substitute/cache tends to grow.
On my laptop I have:

--8<---------------cut here---------------start------------->8---
$ du -hs /var/guix/substitute/cache/
152M    /var/guix/substitute/cache/
$ du /var/guix/substitute/cache/ |sort -k1 -n

[...]

16196   /var/guix/substitute/cache/kzwjeblndsbkjzmjailrt4bnhguil7tqjmewzcyw22hgajbhfy3q
136216  /var/guix/substitute/cache/4refhwxbjmeua2kwg2nmzhv4dg4d3dorpjefq7kiciw2pfhaf26a
155364  /var/guix/substitute/cache/
--8<---------------cut here---------------end--------------->8---

This full of old entries that I’m unlikely to need:

--8<---------------cut here---------------start------------->8---
$ ls -lt /var/guix/substitute/cache/4refhwxbjmeua2kwg2nmzhv4dg4d3dorpjefq7kiciw2pfhaf26a |tail -10
-rw------- 1 root root  1187 Nov 30 18:49 xlxgm0lzg25fsz99qql8wp8y4l0ijvy6
-rw------- 1 root root  3182 Nov 30 18:49 yqdklfzzxx2zm2vvx11pm5my2jzlzwmy
-rw------- 1 root root  1180 Nov 30 18:49 18pzv1gjw9mi3bzkvb2ms9am37kd960i
-rw------- 1 root root  1484 Nov 30 18:49 qqpc6n1q5jz29zzwf1jn8srq3a5ysy86
-rw------- 1 root root  1060 Nov 30 18:27 9r6w8hrw2bpm2sjp03ynl1aifjkcn9wi
-rw------- 1 root root  1056 Nov 30 18:27 ghdwy0xa2h9xi5qh2picb19fhhk0fzhp
-rw------- 1 root root  2108 Nov 30 18:27 ixp9xd4cbs774yh5qkvywmjk1606s8i8
-rw------- 1 root root  1052 Nov 30 18:27 laawrywlbx1y76sdribiw8aygcvadili
-rw------- 1 root root  1092 Nov 30 18:27 m72pfbhqhbhi7gj8kvy66lr99v7hvk7z
-rw------- 1 root root  1070 Nov 30 18:27 ym69k8mzmxbw1ar76nzghl38h4g7g8v4
--8<---------------cut here---------------end--------------->8---

That six-month old because that’s the TTL and ci.guix.gnu.org has been
advertising for some time now.

Long story short: we’d rather trim the cache more aggressively.

Thoughts?

Ludo’.

diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
index 3626832dda..92935268b3 100755
--- a/guix/scripts/substitute.scm
+++ b/guix/scripts/substitute.scm
@@ -167,6 +167,12 @@ (define (lookup-narinfo caches path authorized?)
 
 (define (cached-narinfo-expiration-time file)
   "Return the expiration time for FILE, which is a cached narinfo."
+  (define max-ttl
+    ;; Higher bound on the TTL used to avoid keeping around cached narinfos
+    ;; for too long, which makes the cache bigger and more expensive to
+    ;; traverse.
+    (* 2 30 24 60 60))                            ;2 months
+
   (catch 'system-error
     (lambda ()
       (call-with-input-file file
@@ -174,10 +180,10 @@ (define (cached-narinfo-expiration-time file)
           (match (read port)
             (('narinfo ('version 2) ('cache-uri uri)
                        ('date date) ('ttl ttl) ('value #f))
-             (+ date ttl))
+             (+ date (min ttl max-ttl)))
             (('narinfo ('version 2) ('cache-uri uri)
                        ('date date) ('ttl ttl) ('value value))
-             (+ date ttl))
+             (+ date (min ttl max-ttl)))
             (x
              0)))))
     (lambda args

base-commit: 77f52db416a13e195d090cad4e9e7658feb2e86b
-- 
2.40.1

[Message part 3 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: 63817-done <at> debbugs.gnu.org
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>,
 Christopher Baines <mail <at> cbaines.net>, Ricardo Wurmus <rekado <at> elephly.net>
Subject: Re: bug#63817: [PATCH] substitute: Delete cached narinfos more than
 two-month old.
Date: Thu, 08 Jun 2023 23:49:50 +0200

Ludovic Courtès <ludo <at> gnu.org> skribis:

> This allows 'guix substitute' to shrink the cache a bit more, which
> saves space and improves performance of cache-cleanup phases since fewer
> entries need to be traversed.
>
> * guix/scripts/substitute.scm (cached-narinfo-expiration-time): Define
> 'max-ttl' and use it as an upper bound.

Pushed as 3f5e14182931f123c10513a3e1e2abaebfb52279.

Ludo'.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.