GNU bug report logs - #45104
pull: Add a "with-substitutes" option.

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#45104: closed (pull: Add a "with-substitutes" option.)
Date: Sun, 31 Jan 2021 17:39:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Sun, 31 Jan 2021 18:37:57 +0100
with message-id <87sg6hypl6.fsf <at> gnu.org>
and subject line Re: [bug#45104] pull: Add a "with-substitutes" option.
has caused the debbugs.gnu.org bug report #45104,
regarding pull: Add a "with-substitutes" option.
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
45104: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=45104
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: guix-patches <at> gnu.org
Subject: pull: Add a "with-substitutes" option.
Date: Mon, 07 Dec 2020 16:39:59 +0100

[Message part 3 (text/plain, inline)]

Hello,

When "guix pull" is run before the CI server is done baking substitutes,
the user may need to build some derivations, which can be long on
low-end hardware.

This patch adds an option to "guix pull" so that it updates to the
latest commit with available substitutes.

This work is still in progress, but I'd like to gather some impressions
on that before going further.

Thanks,

Mathieu

[0001-scripts-pull-Add-with-substitutes-option.patch (text/x-diff, inline)]

From d399f8dbb9e38a82241b9048b8b04758fae10005 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe <at> gnu.org>
Date: Mon, 7 Dec 2020 16:12:22 +0100
Subject: [PATCH] scripts: pull: Add "with-substitutes" option.

---
 guix/scripts/pull.scm | 38 +++++++++++++++++++++++++++++++++++---
 1 file changed, 35 insertions(+), 3 deletions(-)

diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
index 83cdc1d1eb..4609f8614e 100644
--- a/guix/scripts/pull.scm
+++ b/guix/scripts/pull.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2013, 2014, 2015, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2017 Marius Bakke <mbakke <at> fastmail.com>
 ;;; Copyright © 2020 Tobias Geerinckx-Rice <me <at> tobias.gr>
+;;; Copyright © 2020 Mathieu Othacehe <othacehe <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -20,6 +21,7 @@
 
 (define-module (guix scripts pull)
   #:use-module ((guix ui) #:hide (display-profile-content))
+  #:use-module (guix ci)
   #:use-module (guix colors)
   #:use-module (guix utils)
   #:use-module ((guix status) #:select (with-status-verbosity))
@@ -64,6 +66,7 @@
   #:re-export (display-profile-content
                channel-commit-hyperlink)
   #:export (channel-list
+            find-lastest-commit-with-substitutes
             guix-pull))
 
 
@@ -169,6 +172,9 @@ Download and deploy the latest version of Guix.\n"))
                  (lambda (opt name arg result)
                    (alist-cons 'validate-pull warn-about-backward-updates
                                result)))
+         (option '("with-substitutes") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'with-substitutes? #t result)))
          (option '("disable-authentication") #f #f
                  (lambda (opt name arg result)
                    (alist-cons 'authenticate-channels? #f result)))
@@ -526,6 +532,26 @@ true, display what would be built without actually building it."
           (leave (G_ "while creating symlink '~a': ~a~%")
                  link (strerror (system-error-errno args))))))))
 
+
+;;;
+;;; Substitutes.
+;;;
+
+(define (find-lastest-commit-with-substitutes)
+  (let ((urls %default-substitute-urls))
+    (any (lambda (url)
+           (let* ((build (match (latest-builds url 1
+                                               #:job "guix.x86_64-linux"
+                                               #:status 0) ;success
+                           ((build) build)))
+                  (evaluation (evaluation url
+                                          (build-evaluation build)))
+                  (commit (match (evaluation-checkouts evaluation)
+                            ((checkout)
+                             (checkout-commit checkout)))))
+             commit))
+         urls)))
+
 
 ;;;
 ;;; Queries.
@@ -731,8 +757,9 @@ Use '~/.config/guix/channels.scm' instead."))
 
   (let ((ref (assoc-ref opts 'ref))
         (url (or (assoc-ref opts 'repository-url)
-                 (environment-variable))))
-    (if (or ref url)
+                 (environment-variable)))
+        (with-substitutes? (assoc-ref opts 'with-substitutes?)))
+    (if (or ref url with-substitutes?)
         (match (find guix-channel? channels)
           ((? channel? guix)
            ;; Apply '--url', '--commit', and '--branch' to the 'guix' channel.
@@ -745,7 +772,12 @@ Use '~/.config/guix/channels.scm' instead."))
                       (channel (inherit guix)
                                (url url) (commit #f) (branch branch)))
                      (#f
-                      (channel (inherit guix) (url url))))
+                      (let ((commit
+                             (and with-substitutes?
+                                  (find-lastest-commit-with-substitutes))))
+                        (channel (inherit guix)
+                                 (url url)
+                                 (commit commit)))))
                    (remove guix-channel? channels))))
           (#f                           ;no 'guix' channel, failure will ensue
            channels))
-- 
2.29.2

[Message part 5 (message/rfc822, inline)]

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 45104-done <at> debbugs.gnu.org, Christopher Baines <mail <at> cbaines.net>
Subject: Re: [bug#45104] pull: Add a "with-substitutes" option.
Date: Sun, 31 Jan 2021 18:37:57 +0100

Hey Ludo,

> It does mean that in the example above we need to add an explicit
> (use-modules (guix ci)).
>
> Otherwise LGTM, thanks!

I took your remarks into account and pushed as
041a9466ea23d6ae811491bcf529bf9487317b48.

Thanks for reviewing!

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.