GNU bug report logs - #73674
[PATCH] shell: Enable caching when using deterministic package transformations.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 73674 in the body.
You can then email your comments to 73674 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludovic.courtes <at> inria.fr>
Subject: [PATCH] shell: Enable caching when using deterministic package
 transformations.
Date: Mon,  7 Oct 2024 10:35:55 +0200

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>

Until now, using any package transformation would disable the automatic
GC root creation and caching in ‘guix shell’.  This change introduces a
finer-grain distinction: a command like:

  guix shell --tune inkscape

is now subject to caching, whereas:

  guix shell --with-latest=inkscape inkscape

remains non-cacheable.

* guix/transformations.scm (%transformations-with-external-dependencies):
New variable.
(cacheable-transformation-option-key?): New procedure.
* guix/scripts/shell.scm (profile-cached-gc-root): In the
‘transformation-option-key?’ clause, call ‘loop’ when
‘cacheable-transformation-option-key?’ returns true.

Change-Id: I847b661dfea20ecf851db2023a5c7ea8c5b5ca7c
---
 guix/scripts/shell.scm   | 13 ++++++++-----
 guix/transformations.scm | 20 +++++++++++++++++++-
 2 files changed, 27 insertions(+), 6 deletions(-)

diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm
index 0584a7e018..d23362a15d 100644
--- a/guix/scripts/shell.scm
+++ b/guix/scripts/shell.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2021-2023 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2021-2024 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2023 Janneke Nieuwenhuizen <janneke <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -25,6 +25,7 @@ (define-module (guix scripts shell)
                                      show-native-build-options-help)
   #:autoload   (guix transformations) (options->transformation
                                        transformation-option-key?
+                                       cacheable-transformation-option-key?
                                        show-transformation-options-help)
   #:autoload   (guix grafts) (%graft?)
   #:use-module (guix scripts)
@@ -417,11 +418,13 @@ (define (profile-cached-gc-root opts)
        ;; Arbitrary expressions might be non-deterministic or otherwise depend
        ;; on external state so do not cache when they're used.
        (values #f #f))
-      ((((? transformation-option-key?) . _) . _)
+      ((((? transformation-option-key? key) . _) . rest)
        ;; Transformation options are potentially "non-deterministic", or at
-       ;; least depending on external state (with-source, with-commit, etc.),
-       ;; so do not cache anything when they're used.
-       (values #f #f))
+       ;; least depending on external state (with-source, with-commit, etc.).
+       ;; Cache only those that are known to be "cacheable".
+       (if (cacheable-transformation-option-key? key)
+           (loop rest system file (cons (first opts) specs))
+           (values #f #f)))
       ((('profile . _) . _)
        ;; If the user already specified a profile, there's nothing more to
        ;; cache.
diff --git a/guix/transformations.scm b/guix/transformations.scm
index 582f8a2729..ea8b7a0844 100644
--- a/guix/transformations.scm
+++ b/guix/transformations.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2016-2023 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2016-2024 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2021 Marius Bakke <marius <at> gnu.org>
 ;;; Copyright © 2023 Sarthak Shah <shahsarthakw <at> gmail.com>
 ;;; Copyright © 2023, 2024 Efraim Flashner <efraim <at> flashner.co.il>
@@ -63,6 +63,7 @@ (define-module (guix transformations)
 
             show-transformation-options-help
             transformation-option-key?
+            cacheable-transformation-option-key?
             %transformation-options))
 
 ;;; Commentary:
@@ -938,6 +939,16 @@ (define %transformations
     (with-latest . ,transform-package-latest)
     (with-version . ,transform-package-version)))
 
+(define %transformations-with-external-dependencies
+  ;; Subset of options that depend on external resources and that can thus be
+  ;; considered "non-deterministic" and non-cacheable.
+  '(with-source
+    with-branch
+    with-git-url
+    with-patch
+    with-latest
+    with-version))
+
 (define (transformation-procedure key)
   "Return the transformation procedure associated with KEY, a symbol such as
 'with-source', or #f if there is none."
@@ -952,6 +963,13 @@ (define (transformation-option-key? key)
 For example, (transformation-option-key? 'with-input) => #t."
   (->bool (transformation-procedure key)))
 
+(define (cacheable-transformation-option-key? key)
+  "Return true if KEY corresponds to a transformation option whose result can
+be cached--i.e., the transformation is deterministic and does not depend on
+external resources."
+  (and (transformation-option-key? key)
+       (not (memq key %transformations-with-external-dependencies))))
+
 
 ;;;
 ;;; Command-line handling.

base-commit: 73ec844389e91cb0f5a2647070516fc8d19d8730
-- 
2.46.0

Message #10 received at 73674-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 73674-done <at> debbugs.gnu.org
Cc: Tobias Geerinckx-Rice <me <at> tobias.gr>, Christopher Baines <guix <at> cbaines.net>,
 Josselin Poiret <dev <at> jpoiret.xyz>, Simon Tournier <zimon.toutoune <at> gmail.com>,
 Mathieu Othacehe <othacehe <at> gnu.org>
Subject: Re: [bug#73674] [PATCH] shell: Enable caching when using
 deterministic package transformations.
Date: Tue, 22 Oct 2024 00:30:41 +0200

Ludovic Courtès <ludo <at> gnu.org> skribis:

> From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
>
> Until now, using any package transformation would disable the automatic
> GC root creation and caching in ‘guix shell’.  This change introduces a
> finer-grain distinction: a command like:
>
>   guix shell --tune inkscape
>
> is now subject to caching, whereas:
>
>   guix shell --with-latest=inkscape inkscape
>
> remains non-cacheable.
>
> * guix/transformations.scm (%transformations-with-external-dependencies):
> New variable.
> (cacheable-transformation-option-key?): New procedure.
> * guix/scripts/shell.scm (profile-cached-gc-root): In the
> ‘transformation-option-key?’ clause, call ‘loop’ when
> ‘cacheable-transformation-option-key?’ returns true.
>
> Change-Id: I847b661dfea20ecf851db2023a5c7ea8c5b5ca7c

I went ahead and pushed it as 6d8107c967a554e23b0c58c8933894d4468ad596.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.