GNU bug report logs - #73674
[PATCH] shell: Enable caching when using deterministic package transformations.

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#73674: closed ([PATCH] shell: Enable caching when using
 deterministic package transformations.)
Date: Mon, 21 Oct 2024 22:32:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Tue, 22 Oct 2024 00:30:41 +0200
with message-id <87bjzdccmm.fsf <at> gnu.org>
and subject line Re: [bug#73674] [PATCH] shell: Enable caching when using deterministic package transformations.
has caused the debbugs.gnu.org bug report #73674,
regarding [PATCH] shell: Enable caching when using deterministic package transformations.
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
73674: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=73674
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludovic.courtes <at> inria.fr>
Subject: [PATCH] shell: Enable caching when using deterministic package
 transformations.
Date: Mon,  7 Oct 2024 10:35:55 +0200

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>

Until now, using any package transformation would disable the automatic
GC root creation and caching in ‘guix shell’.  This change introduces a
finer-grain distinction: a command like:

  guix shell --tune inkscape

is now subject to caching, whereas:

  guix shell --with-latest=inkscape inkscape

remains non-cacheable.

* guix/transformations.scm (%transformations-with-external-dependencies):
New variable.
(cacheable-transformation-option-key?): New procedure.
* guix/scripts/shell.scm (profile-cached-gc-root): In the
‘transformation-option-key?’ clause, call ‘loop’ when
‘cacheable-transformation-option-key?’ returns true.

Change-Id: I847b661dfea20ecf851db2023a5c7ea8c5b5ca7c
---
 guix/scripts/shell.scm   | 13 ++++++++-----
 guix/transformations.scm | 20 +++++++++++++++++++-
 2 files changed, 27 insertions(+), 6 deletions(-)

diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm
index 0584a7e018..d23362a15d 100644
--- a/guix/scripts/shell.scm
+++ b/guix/scripts/shell.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2021-2023 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2021-2024 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2023 Janneke Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -25,6 +25,7 @@ (define-module (guix scripts shell)
                                      show-native-build-options-help)
   #:autoload   (guix transformations) (options->transformation
                                        transformation-option-key?
+                                       cacheable-transformation-option-key?
                                        show-transformation-options-help)
   #:autoload   (guix grafts) (%graft?)
   #:use-module (guix scripts)
@@ -417,11 +418,13 @@ (define (profile-cached-gc-root opts)
        ;; Arbitrary expressions might be non-deterministic or otherwise depend
        ;; on external state so do not cache when they're used.
        (values #f #f))
-      ((((? transformation-option-key?) . _) . _)
+      ((((? transformation-option-key? key) . _) . rest)
        ;; Transformation options are potentially "non-deterministic", or at
-       ;; least depending on external state (with-source, with-commit, etc.),
-       ;; so do not cache anything when they're used.
-       (values #f #f))
+       ;; least depending on external state (with-source, with-commit, etc.).
+       ;; Cache only those that are known to be "cacheable".
+       (if (cacheable-transformation-option-key? key)
+           (loop rest system file (cons (first opts) specs))
+           (values #f #f)))
       ((('profile . _) . _)
        ;; If the user already specified a profile, there's nothing more to
        ;; cache.
diff --git a/guix/transformations.scm b/guix/transformations.scm
index 582f8a2729..ea8b7a0844 100644
--- a/guix/transformations.scm
+++ b/guix/transformations.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2016-2023 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2016-2024 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2021 Marius Bakke <marius <at> gnu.org>
 ;;; Copyright © 2023 Sarthak Shah <shahsarthakw <at> gmail.com>
 ;;; Copyright © 2023, 2024 Efraim Flashner <efraim <at> flashner.co.il>
@@ -63,6 +63,7 @@ (define-module (guix transformations)
 
             show-transformation-options-help
             transformation-option-key?
+            cacheable-transformation-option-key?
             %transformation-options))
 
 ;;; Commentary:
@@ -938,6 +939,16 @@ (define %transformations
     (with-latest . ,transform-package-latest)
     (with-version . ,transform-package-version)))
 
+(define %transformations-with-external-dependencies
+  ;; Subset of options that depend on external resources and that can thus be
+  ;; considered "non-deterministic" and non-cacheable.
+  '(with-source
+    with-branch
+    with-git-url
+    with-patch
+    with-latest
+    with-version))
+
 (define (transformation-procedure key)
   "Return the transformation procedure associated with KEY, a symbol such as
 'with-source', or #f if there is none."
@@ -952,6 +963,13 @@ (define (transformation-option-key? key)
 For example, (transformation-option-key? 'with-input) => #t."
   (->bool (transformation-procedure key)))
 
+(define (cacheable-transformation-option-key? key)
+  "Return true if KEY corresponds to a transformation option whose result can
+be cached--i.e., the transformation is deterministic and does not depend on
+external resources."
+  (and (transformation-option-key? key)
+       (not (memq key %transformations-with-external-dependencies))))
+
 
 ;;;
 ;;; Command-line handling.

base-commit: 73ec844389e91cb0f5a2647070516fc8d19d8730
-- 
2.46.0

[Message part 3 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: 73674-done <at> debbugs.gnu.org
Cc: Tobias Geerinckx-Rice <me <at> tobias.gr>, Christopher Baines <guix <at> cbaines.net>,
 Josselin Poiret <dev <at> jpoiret.xyz>, Simon Tournier <zimon.toutoune <at> gmail.com>,
 Mathieu Othacehe <othacehe <at> gnu.org>
Subject: Re: [bug#73674] [PATCH] shell: Enable caching when using
 deterministic package transformations.
Date: Tue, 22 Oct 2024 00:30:41 +0200

Ludovic Courtès <ludo <at> gnu.org> skribis:

> From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
>
> Until now, using any package transformation would disable the automatic
> GC root creation and caching in ‘guix shell’.  This change introduces a
> finer-grain distinction: a command like:
>
>   guix shell --tune inkscape
>
> is now subject to caching, whereas:
>
>   guix shell --with-latest=inkscape inkscape
>
> remains non-cacheable.
>
> * guix/transformations.scm (%transformations-with-external-dependencies):
> New variable.
> (cacheable-transformation-option-key?): New procedure.
> * guix/scripts/shell.scm (profile-cached-gc-root): In the
> ‘transformation-option-key?’ clause, call ‘loop’ when
> ‘cacheable-transformation-option-key?’ returns true.
>
> Change-Id: I847b661dfea20ecf851db2023a5c7ea8c5b5ca7c

I went ahead and pushed it as 6d8107c967a554e23b0c58c8933894d4468ad596.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.