From unknown Tue Jun 17 01:48:45 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#70895 <70895@debbugs.gnu.org> To: bug#70895 <70895@debbugs.gnu.org> Subject: Status: [PATCH] grafts: Only compute necessary graft derivations. Reply-To: bug#70895 <70895@debbugs.gnu.org> Date: Tue, 17 Jun 2025 08:48:45 +0000 retitle 70895 [PATCH] grafts: Only compute necessary graft derivations. reassign 70895 guix-patches submitter 70895 David Elsing severity 70895 normal tag 70895 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Sun May 12 09:43:34 2024 Received: (at submit) by debbugs.gnu.org; 12 May 2024 13:43:34 +0000 Received: from localhost ([127.0.0.1]:54527 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s69UE-00013Q-40 for submit@debbugs.gnu.org; Sun, 12 May 2024 09:43:34 -0400 Received: from lists.gnu.org ([209.51.188.17]:45462) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s69U9-00013K-NB for submit@debbugs.gnu.org; Sun, 12 May 2024 09:43:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s69U9-0003n7-2d for guix-patches@gnu.org; Sun, 12 May 2024 09:43:29 -0400 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s69U6-0004ZM-RI for guix-patches@gnu.org; Sun, 12 May 2024 09:43:28 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 717CD240101 for ; Sun, 12 May 2024 15:43:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1715521403; bh=ApplQY3r0jE0agJLQ9yqBdE9+quD3I5/CbS2BOGuCM0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version: Content-Transfer-Encoding:From; b=ruO9HL+1R40vqZeY1hHu2b26q+hM9g7wkbLfKVA2eJHKA5nVPqMZD/12DzBFzsB62 wAjXA7lEHy5Ljls2X5TIoKYn1TOb63z7+s/bMl+8UdyTF8m7jcKQIDWqBIkaP0ZB6p qw2mu0KddS/pq8umMl8UozR2RArFr5oaFli/Uxj5cM/h4oVRGPYfmPHSPxRrTdiB4q vVlxvHLwuzLsxPmFyDCk4wi98VAIFdMAVKzfiRxCFMQT+exNPy006wcFa10t/bIJrB wnafe+q7FBvhgrR1wRunHRsf01rp1J/aypPPPVhxcPeHsNjLX26xCUzrEC546D49MT 1Nh1XnlXbOfsA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4VckNB6Rywz6tvm; Sun, 12 May 2024 15:43:22 +0200 (CEST) From: David Elsing To: guix-patches@gnu.org Subject: [PATCH] grafts: Only compute necessary graft derivations. Date: Sun, 12 May 2024 13:42:05 +0000 Message-ID: <20240512134301.2234-1-david.elsing@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=185.67.36.66; envelope-from=david.elsing@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: David Elsing X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Previously, derivations for grafted packages were computed for all packages with replacements, regardless of whether they are actually referenced by the package output in question. This can cause ungrafted packages to be built even if they are not required. This commit delays calculating these derivations until they are found to actually be applicable. * guix/packages.scm (input-graft): Put records into the 'replacement' field of records instead of the corresponding grafted package derivations. (graft-derivation*): Move to... (package->derivation, package->cross-derivation) ... here. * guix/grafts.scm (): New record type. (cumulative-grafts): Turn the records in the 'replacement' field of applicable grafts into derivations. --- guix/grafts.scm | 41 ++++++++++++++++++++++++++++++++++++----- guix/packages.scm | 21 +++++++++++++-------- 2 files changed, 49 insertions(+), 13 deletions(-) diff --git a/guix/grafts.scm b/guix/grafts.scm index f4df513daf..5939192864 100644 --- a/guix/grafts.scm +++ b/guix/grafts.scm @@ -42,18 +42,25 @@ (define-module (guix grafts) graft-derivation graft-derivation/shallow + graft-package + %graft-with-utf8-locale?) #:re-export (%graft? ;for backward compatibility without-grafting set-grafting grafting?)) +(define-record-type* graft-package make-graft-package + graft-package? + (package graft-package-package) + (target graft-package-target)) + (define-record-type* graft make-graft graft? (origin graft-origin) ;derivation | store item (origin-output graft-origin-output ;string | #f (default "out")) - (replacement graft-replacement) ;derivation | store item + (replacement graft-replacement) ;derivation | store item | graft-package (replacement-output graft-replacement-output ;string | #f (default "out"))) @@ -283,6 +290,28 @@ (define (dependency-grafts items) #:system system))))) (reference-origins drv items))) + (define package-derivation + (@ (guix packages) package-derivation)) + (define package-cross-derivation + (@ (guix packages) package-cross-derivation)) + + ;; Turn all 'replacement' fields which are records into + ;; grafted package derivations with #:grafts? #t. + (define (calc-remaining-grafts grafts) + (map + (lambda (item) + (graft + (inherit item) + (replacement + (match (graft-replacement item) + (($ package target) + (if target + (package-cross-derivation + store package target system #:graft? #t) + (package-derivation store package system #:graft? #t))) + (new new))))) + grafts)) + (with-cache (list (derivation-file-name drv) outputs grafts) (match (non-self-references store drv outputs) (() ;no dependencies @@ -299,10 +328,12 @@ (define (dependency-grafts items) ;; Use APPLICABLE, the subset of GRAFTS that is really ;; applicable to DRV, to avoid creating several identical ;; grafted variants of DRV. - (let* ((new (graft-derivation/shallow* store drv applicable - #:outputs outputs - #:guile guile - #:system system)) + (let* ((new (graft-derivation/shallow* + store drv + (calc-remaining-grafts applicable) + #:outputs outputs + #:guile guile + #:system system)) (grafts (append (map (lambda (output) (graft (origin drv) diff --git a/guix/packages.scm b/guix/packages.scm index abe89cdb07..1b816d0e24 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -1778,8 +1778,9 @@ (define (input-graft system) (mcached eq? (=> %package-graft-cache) (mlet %store-monad ((orig (package->derivation package system #:graft? #f)) - (new (package->derivation replacement system - #:graft? #t))) + (new -> (graft-package + (package package) + (target #f)))) (return (graft (origin orig) (origin-output output) @@ -1800,9 +1801,9 @@ (define (input-cross-graft target system) (mlet %store-monad ((orig (package->cross-derivation package target system #:graft? #f)) - (new (package->cross-derivation replacement - target system - #:graft? #t))) + (new -> (graft-package + (package package) + (target target)))) (return (graft (origin orig) (origin-output output) @@ -1996,14 +1997,14 @@ (define* (bag->cross-derivation bag #:optional context) (define bag->derivation* (store-lower bag->derivation)) -(define graft-derivation* - (store-lift graft-derivation)) - (define* (package->derivation package #:optional (system (%current-system)) #:key (graft? (%graft?))) "Return the object of PACKAGE for SYSTEM." + (define graft-derivation* + (store-lift graft-derivation)) + ;; Compute the derivation and cache the result. Caching is important ;; because some derivations, such as the implicit inputs of the GNU build ;; system, will be queried many, many times in a row. @@ -2030,6 +2031,10 @@ (define* (package->cross-derivation package target #:key (graft? (%graft?))) "Cross-build PACKAGE for TARGET (a GNU triplet) from host SYSTEM (a Guix system identifying string)." + + (define graft-derivation* + (store-lift graft-derivation)) + (mcached (mlet* %store-monad ((bag -> (package->bag package system target #:graft? graft?)) (drv (bag->derivation bag package))) -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 03 16:58:35 2024 Received: (at 70895) by debbugs.gnu.org; 3 Jun 2024 20:58:35 +0000 Received: from localhost ([127.0.0.1]:55838 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sEElH-0002XD-4h for submit@debbugs.gnu.org; Mon, 03 Jun 2024 16:58:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38762) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sEElF-0002Wz-F5 for 70895@debbugs.gnu.org; Mon, 03 Jun 2024 16:58:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sEEkx-0004Vw-0s; Mon, 03 Jun 2024 16:58:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=Ikpl9wChKgligyYHb+0C3h/OvBM+U3vGjpbhzhtjw0A=; b=aez5cu3+YuUolTU1pjlX KiPduB7h9wB6aajvg7/kDyvBdDJFuYYl9b53paXg5seiujC4ZeV9CpjOqWea22gpWiHoTJDSxVcYe MdNYKHiz2uzHrCGQPxOWiPec0K2Q/2OmXOzNy34+dk9ZJkqW2VV+WsKx4F/EqyFxMnAaUyOz4zIbQ MukXdb8vkdL/ACBFdRE2JGN0eE6tk+fqgzXVdWxJN0cTeGCmKvKNlUH6NUKQ+fM4GW9Seps4KQ3fo 4yQ8S4pM41396hHHdHm9DFQ+KkUF9/6NePqkhuLyzqyjxAKAYZppvMt68/QYyRyD7CZMD1jsiO4Zo /vhuQ9DIxMrzLw==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: David Elsing Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <20240512134301.2234-1-david.elsing@posteo.net> (David Elsing's message of "Sun, 12 May 2024 13:42:05 +0000") References: <20240512134301.2234-1-david.elsing@posteo.net> Date: Mon, 03 Jun 2024 22:58:10 +0200 Message-ID: <87zfs1921p.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi David, David Elsing skribis: > Previously, derivations for grafted packages were computed for all > packages with replacements, regardless of whether they are actually > referenced by the package output in question. This can cause ungrafted > packages to be built even if they are not required. > > This commit delays calculating these derivations until they are found to > actually be applicable. Neat, good idea! [...] > (define-record-type* graft make-graft > graft? > (origin graft-origin) ;derivation | store it= em > (origin-output graft-origin-output ;string | #f > (default "out")) > - (replacement graft-replacement) ;derivation | store it= em > + (replacement graft-replacement) ;derivation | store it= em | graft-package > (replacement-output graft-replacement-output ;string | #f > (default "out"))) >=20=20 > @@ -283,6 +290,28 @@ (define (dependency-grafts items) > #:system system))))) > (reference-origins drv items))) >=20=20 > + (define package-derivation > + (@ (guix packages) package-derivation)) > + (define package-cross-derivation > + (@ (guix packages) package-cross-derivation)) > + > + ;; Turn all 'replacement' fields which are records into > + ;; grafted package derivations with #:grafts? #t. > + (define (calc-remaining-grafts grafts) > + (map > + (lambda (item) > + (graft > + (inherit item) > + (replacement > + (match (graft-replacement item) > + (($ package target) > + (if target > + (package-cross-derivation > + store package target system #:graft? #t) > + (package-derivation store package system #:graft? #t))) > + (new new))))) > + grafts)) While this does the job, it breaks an abstraction (grafts are lower-level than packages) and creates a circular dependency between (guix grafts) and (guix packages) as a result (not technically a problem at this point, but it shows that something=E2=80=99s deserves to be clarifi= ed). Maybe there=E2=80=99s a simpler way to achieve this though. What about all= owing monadic values in the =E2=80=98origin=E2=80=99 and =E2=80=98replacement=E2= =80=99 fields of ? Their values would be bound lazily, only when needed by =E2=80=98graft-derivation=E2=80=99. WDYT? Thanks a lot for diving into this! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 05 17:49:34 2024 Received: (at 70895) by debbugs.gnu.org; 5 Jun 2024 21:49:34 +0000 Received: from localhost ([127.0.0.1]:40326 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sEyVc-00040K-H9 for submit@debbugs.gnu.org; Wed, 05 Jun 2024 17:49:34 -0400 Received: from mout01.posteo.de ([185.67.36.65]:58357) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sEyVW-0003zq-QL for 70895@debbugs.gnu.org; Wed, 05 Jun 2024 17:49:26 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 58D5B240027 for <70895@debbugs.gnu.org>; Wed, 5 Jun 2024 23:43:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1717623798; bh=NAZzLucnHw4Ia3oQxZsLY/PeGddenwKrFq/gfcSnzN4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding:From; b=VjAj97OVZKr1/+0jhVzKXZT7a7/FmAqj84s9sU+o0xmaWM3P7ZldZ0eeay2U8gU2z Us9dx8AIWpK0YrzfB77yWY3yp2pnblvGHvbgqucMchlkSb4+vZmmwwGyh1PHCuNuOT PpiBlagyhTmiKe5t/d/GEcbmdowU/NDlZIgnxAATSc5IFavxp9jfvWQMx0OOsjiEO7 SXNinzVjEQMudOrIBsGh+6wbv91C4rzxVXoZ5dTd4XJJAPPuwY+N7DOJPNjHDjp7GB u+HZhUQBa0gM2GW5l99aEdvaSNe54TI+mRf72BySHSzHXYe3JPHkhLUt5qElp/8Vz0 mBvJs7E7xoY9w== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Vvgts539nz6txh; Wed, 5 Jun 2024 23:43:17 +0200 (CEST) From: David Elsing To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <87zfs1921p.fsf@gnu.org> References: <20240512134301.2234-1-david.elsing@posteo.net> <87zfs1921p.fsf@gnu.org> Date: Wed, 05 Jun 2024 21:43:16 +0000 Message-ID: <7ycyov5amj.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ludovic Court=C3=A8s writes: Hi Ludo', > While this does the job, it breaks an abstraction (grafts are > lower-level than packages) and creates a circular dependency between > (guix grafts) and (guix packages) as a result (not technically a problem > at this point, but it shows that something=E2=80=99s deserves to be clari= fied). > > Maybe there=E2=80=99s a simpler way to achieve this though. What about a= llowing > monadic values in the =E2=80=98origin=E2=80=99 and =E2=80=98replacement= =E2=80=99 fields of ? > Their values would be bound lazily, only when needed by > =E2=80=98graft-derivation=E2=80=99. Yes, that's a good idea, this makes the patch a lot shorter as well. The 'origin' field need to computed anyway to check whether the graft is applicable, so only the 'replacement' field needs to be bound lazily. In cumulative-grafts, I check whether the replacement field is a monadic value using 'procedure?' and then call 'run-with-store'. Is there a better way to do this? I see that values in the %state-monad are repesented as plain lambdas. Also, is it correct to set the #:guile-for-build and #:system keyword arguments here, as they are already specified in package->derivation and package->cross-derivation? As an alternative, is it possible to 'ungexp' a value in the store monad which returns a derivation? Then the derivation for the 'replacement' field could be calculated in 'mapping' in 'graft-derivation/shallow'. I wouldn't know how to define a gexp-compiler though, except by defining one for any procedure and assuming it is a value in the store monad. Cheers, David From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 05 17:59:25 2024 Received: (at 70895) by debbugs.gnu.org; 5 Jun 2024 21:59:25 +0000 Received: from localhost ([127.0.0.1]:41044 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sEyfF-0004RV-5S for submit@debbugs.gnu.org; Wed, 05 Jun 2024 17:59:25 -0400 Received: from mout01.posteo.de ([185.67.36.65]:38841) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sEyfC-0004R6-JJ for 70895@debbugs.gnu.org; Wed, 05 Jun 2024 17:59:24 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 296EC240027 for <70895@debbugs.gnu.org>; Wed, 5 Jun 2024 23:51:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1717624319; bh=j9xZCsJrd17MFI48R9RdtW3DTzHDgA8Ueyt6F+SUgbQ=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding:From; b=rtvJ3exbh4Y6rQVi2fCxXrOhNYOS4dsP2VInHEmHkNAXb45Upl2zgrtPRbwcX8tk4 qFxRx9lQykJw48J1J8PsKA3pPsTguI4nEILX5r0vMNOMiWUlOTxuiHNAPSLNlOEKCE M9AO6/1PcKxK3poxC0g2xM0uQ1dRf0CMkbQ6SmiqnNhDi1xcm/Tt97xueLILFc0TB3 P+kQ9RXt5DyWkfsH2aArKYWhhSHggsgiMkAVXbG5SpfpZ86lJ+ToMy5z4s+DsDSEx+ wu+sqSmxAjcw4W273DhZ2rtaKCt+oGkuNei49YhIKMgtBGjhddtIe9imooyoYcSb/U vrxgetP5RJ5JA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Vvh4t4qWrz9rxD; Wed, 5 Jun 2024 23:51:58 +0200 (CEST) From: David Elsing To: 70895@debbugs.gnu.org Subject: [PATCH v2] grafts: Only compute necessary graft derivations. Date: Wed, 5 Jun 2024 21:51:42 +0000 Message-ID: <86be6fa895a3a5d9625a49e11181f483fca3c462.1717623537.git.david.elsing@posteo.net> In-Reply-To: <7ycyov5amj.fsf@posteo.net> References: <7ycyov5amj.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: David Elsing , =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/packages.scm (input-graft, input-cross-graft): Store the monadic value of the replacement in the 'replacement' field of instead of unwrapping it. (cumulative-grafts): Turn monadic values in the 'replacement' field of applicable grafts into derivations. --- guix/grafts.scm | 18 +++++++++++++++++- guix/packages.scm | 11 ++++++----- 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/guix/grafts.scm b/guix/grafts.scm index f4df513daf..2f2ddbc83a 100644 --- a/guix/grafts.scm +++ b/guix/grafts.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2014-2023 Ludovic Courtès +;;; Copyright © 2024 David Elsing ;;; ;;; This file is part of GNU Guix. ;;; @@ -283,6 +284,20 @@ (define (dependency-grafts items) #:system system))))) (reference-origins drv items))) + ;; If the 'replacement' field of the record is a procedure, + ;; this means that it is a value in the store monad and the actual + ;; derivation needs to be computed here. + (define (finalize-graft item) + (let ((replacement (graft-replacement item))) + (if (procedure? replacement) + (graft + (inherit item) + (replacement + (run-with-store store replacement + #:guile-for-build guile + #:system system))) + item))) + (with-cache (list (derivation-file-name drv) outputs grafts) (match (non-self-references store drv outputs) (() ;no dependencies @@ -299,7 +314,8 @@ (define (dependency-grafts items) ;; Use APPLICABLE, the subset of GRAFTS that is really ;; applicable to DRV, to avoid creating several identical ;; grafted variants of DRV. - (let* ((new (graft-derivation/shallow* store drv applicable + (let* ((new (graft-derivation/shallow* store drv + (map finalize-graft applicable) #:outputs outputs #:guile guile #:system system)) diff --git a/guix/packages.scm b/guix/packages.scm index abe89cdb07..946ccc693a 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -10,6 +10,7 @@ ;;; Copyright © 2022 Maxime Devos ;;; Copyright © 2022 jgart ;;; Copyright © 2023 Simon Tournier +;;; Copyright © 2024 David Elsing ;;; ;;; This file is part of GNU Guix. ;;; @@ -1778,8 +1779,8 @@ (define (input-graft system) (mcached eq? (=> %package-graft-cache) (mlet %store-monad ((orig (package->derivation package system #:graft? #f)) - (new (package->derivation replacement system - #:graft? #t))) + (new -> (package->derivation replacement system + #:graft? #t))) (return (graft (origin orig) (origin-output output) @@ -1800,9 +1801,9 @@ (define (input-cross-graft target system) (mlet %store-monad ((orig (package->cross-derivation package target system #:graft? #f)) - (new (package->cross-derivation replacement - target system - #:graft? #t))) + (new -> (package->cross-derivation replacement + target system + #:graft? #t))) (return (graft (origin orig) (origin-output output) -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 18 18:01:22 2025 Received: (at 70895) by debbugs.gnu.org; 18 Jan 2025 23:01:22 +0000 Received: from localhost ([127.0.0.1]:44069 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tZHog-0005MQ-5O for submit@debbugs.gnu.org; Sat, 18 Jan 2025 18:01:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49846) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tZHod-0005MC-H7 for 70895@debbugs.gnu.org; Sat, 18 Jan 2025 18:01:20 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tZHoX-0004vE-Ts; Sat, 18 Jan 2025 18:01:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=RX8h0LRoVh3frYqgj/qVOga1WwFYMelJlUDCzP37gf4=; b=nysLLpraZgY4HsTrv9ca dCQsbPm03YAGX5PL+u9ICJSKVmGCI0hj2DsEVHXUqPkum3c/CVlRWhe3W4hfPhmU16dth+GW8/3SW +khZV2YlvEAbx5/2yUlotwftA3NjhGaTBxd45a86n4eyU250EGLE+JkIlVlGYG/h4MK52SYEXv8hr D47iji1rhbNonLKeeNKzm8hQaRBQp9DnQ6X863CJ+ohUaKkSj8bh8kKLkaadrMnHgFveZr4+wqAQh Cb3J37Nf4+cOptFXlnGsoMrxiln5+R5/1q5aSfK+V0RW7WQcofMrIQLcTqccYz7/lS7GDeR8m+hTx DekLZ2jPnjxJ6g==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: David Elsing Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <7ycyov5amj.fsf@posteo.net> (David Elsing's message of "Wed, 05 Jun 2024 21:43:16 +0000") References: <20240512134301.2234-1-david.elsing@posteo.net> <87zfs1921p.fsf@gnu.org> <7ycyov5amj.fsf@posteo.net> Date: Sun, 19 Jan 2025 00:01:07 +0100 Message-ID: <87ikqbsp58.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi David, Sorry for not replying earlier. David Elsing skribis: >> While this does the job, it breaks an abstraction (grafts are >> lower-level than packages) and creates a circular dependency between >> (guix grafts) and (guix packages) as a result (not technically a problem >> at this point, but it shows that something=E2=80=99s deserves to be clar= ified). >> >> Maybe there=E2=80=99s a simpler way to achieve this though. What about = allowing >> monadic values in the =E2=80=98origin=E2=80=99 and =E2=80=98replacement= =E2=80=99 fields of ? >> Their values would be bound lazily, only when needed by >> =E2=80=98graft-derivation=E2=80=99. > > Yes, that's a good idea, this makes the patch a lot shorter as well. > The 'origin' field need to computed anyway to check whether the graft is > applicable, so only the 'replacement' field needs to be bound lazily. > > In cumulative-grafts, I check whether the replacement field is a monadic > value using 'procedure?' and then call 'run-with-store'. Is there a > better way to do this? I see that values in the %state-monad are > repesented as plain lambdas. Also, is it correct to set the > #:guile-for-build and #:system keyword arguments here, as they are > already specified in package->derivation and package->cross-derivation? What you did is correct. #:guile-for-build and #:system are use to specify the default value in case it=E2=80=99s not already specified; passi= ng them here is unnecessary but it cannot hurt. > As an alternative, is it possible to 'ungexp' a value in the store monad > which returns a derivation? Then the derivation for the 'replacement' > field could be calculated in 'mapping' in 'graft-derivation/shallow'. > I wouldn't know how to define a gexp-compiler though, except by defining > one for any procedure and assuming it is a value in the store monad. I=E2=80=99m not sure what you mean since there=E2=80=99s no gexp here. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 18 18:02:31 2025 Received: (at 70895-done) by debbugs.gnu.org; 18 Jan 2025 23:02:31 +0000 Received: from localhost ([127.0.0.1]:44074 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tZHpm-0005P7-Uv for submit@debbugs.gnu.org; Sat, 18 Jan 2025 18:02:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:60254) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tZHpk-0005Oh-99 for 70895-done@debbugs.gnu.org; Sat, 18 Jan 2025 18:02:28 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tZHpf-0004yv-0K; Sat, 18 Jan 2025 18:02:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=+9FMYKiMygTY/XZztzfNV+T1wkfK+UegxPhY+BXGzCI=; b=N7nHRQOC0W/6s2PB/WpD j8UfB3UXhxCNpd/9v62A20N1SwYmIHM6VAf37ncIX8uAajPJ1RWRFmsdcCDn83LHPXQBh5G63IyZr 5obYJFlJhLmUU/E0pNC75a7tZ/ez7u91wOGJfCvH22y+v1UM9vDsT6VBantntqcHK0CAY4edSLQKz vWZQShmaB1b0fcx6rSuhSKUw0/6fJQMq9OcfTFTI/l9URniwpG26qg0spT84ClC2zhaoUBrXy35fr VGvo4KLkOe/VPMj1hq3diK3RdBYKAwA0mpqD0q36UMsp2fE+Ov7lPCe6vB5dBzJTKPhuEFjK1fbsg ToHlTj0SX/ynqA==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: David Elsing Subject: Re: [bug#70895] [PATCH v2] grafts: Only compute necessary graft derivations. In-Reply-To: <86be6fa895a3a5d9625a49e11181f483fca3c462.1717623537.git.david.elsing@posteo.net> (David Elsing's message of "Wed, 5 Jun 2024 21:51:42 +0000") References: <7ycyov5amj.fsf@posteo.net> <86be6fa895a3a5d9625a49e11181f483fca3c462.1717623537.git.david.elsing@posteo.net> Date: Sun, 19 Jan 2025 00:02:21 +0100 Message-ID: <87ed0zsp36.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895-done Cc: 70895-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) David Elsing skribis: > * guix/packages.scm (input-graft, input-cross-graft): Store the monadic v= alue > of the replacement in the 'replacement' field of instead of unwra= pping > it. > (cumulative-grafts): Turn monadic values in the 'replacement' field of > applicable grafts into derivations. It took many months but I finally applied it. I had to update graft-related tests in =E2=80=98tests/packages.scm=E2=80=99; I also added a= couple of comments in the code. Thanks for your work, and apologies again for the delay! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 19 16:30:21 2025 Received: (at 70895) by debbugs.gnu.org; 19 Jan 2025 21:30:21 +0000 Received: from localhost ([127.0.0.1]:47645 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tZcs9-0005HB-6F for submit@debbugs.gnu.org; Sun, 19 Jan 2025 16:30:21 -0500 Received: from mout01.posteo.de ([185.67.36.65]:41207) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tZcs7-0005Dd-Ha for 70895@debbugs.gnu.org; Sun, 19 Jan 2025 16:30:20 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id B15F8240029 for <70895@debbugs.gnu.org>; Sun, 19 Jan 2025 22:30:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1737322213; bh=Jmt7LYXEjAZSW7S3+2yeAPzm+AxepyCflo/EZ6tYzmM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding:From; b=JDigFa9QGobSF7AGBKinjMd+tIVf0TbjOi2AaYtEVqZHyRfSD+FAaMYkTzsXfpwi/ DWYj1VJLAyjYVuan2Mi6niJAF98Bhtx5k0ci8JQRfv0W+99X3DkF9wEmIT1slj3TmO Uqi4LZORAlT4hyfp9ppNlfjFthwrffKx+6G6rgazPcHDHkomNrJddc2BnWh7n/B1v3 7nnoqz1oD+I0MMIgU7izUjqjxLNswab2nAED9V3sm+jw11FXcuulKb519dhVDJLXS2 lBBVrcOA6MogahECUn+j0fgz7Z+sAB3mUht4IUgVKKkFOYWRwTZe/HmpSalkZQ1LuC tdHgLRURN3R7Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4YbmpY1jNqz9rxL; Sun, 19 Jan 2025 22:30:13 +0100 (CET) From: David Elsing To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <87ikqbsp58.fsf@gnu.org> References: <20240512134301.2234-1-david.elsing@posteo.net> <87zfs1921p.fsf@gnu.org> <7ycyov5amj.fsf@posteo.net> <87ikqbsp58.fsf@gnu.org> Date: Sun, 19 Jan 2025 21:30:12 +0000 Message-ID: <86sepetrtn.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.3 (---) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.3 (----) Hi, Ludovic Court=C3=A8s writes: > What you did is correct. #:guile-for-build and #:system are use to > specify the default value in case it=E2=80=99s not already specified; pas= sing > them here is unnecessary but it cannot hurt. Ok, thanks! >> As an alternative, is it possible to 'ungexp' a value in the store monad >> which returns a derivation? Then the derivation for the 'replacement' >> field could be calculated in 'mapping' in 'graft-derivation/shallow'. >> I wouldn't know how to define a gexp-compiler though, except by defining >> one for any procedure and assuming it is a value in the store monad. > > I=E2=80=99m not sure what you mean since there=E2=80=99s no gexp here. I meant that in 'graft-derivation/shallow', the 'mapping' variable is a list of gexps, where 'ungexp' is called on the graft-replacement of the grafts. So instead of turning the monadic value into a derivation beforehand, this could be done here instead, right? Considering a gexp-compiler returns a store monad value of a derivation (IIUC), my question was whether it is possible to use such a value in a gexp directly (i.e. without the 'return' in the gexp-compiler for a derivation, such that the evaluation of the derivation is delayed until the gexp is lowered to a derivation). These values would still need to be identified by 'procedure?' I guess, so it would not be better than currently. Does this make sense or did I misunderstand something? Cheers, David From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 19 16:30:13 2025 Received: (at 70895-done) by debbugs.gnu.org; 19 Jan 2025 21:30:13 +0000 Received: from localhost ([127.0.0.1]:47639 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tZcs0-0005Cf-I3 for submit@debbugs.gnu.org; Sun, 19 Jan 2025 16:30:13 -0500 Received: from mout01.posteo.de ([185.67.36.65]:52787) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tZcry-00059i-6m for 70895-done@debbugs.gnu.org; Sun, 19 Jan 2025 16:30:10 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 554B0240027 for <70895-done@debbugs.gnu.org>; Sun, 19 Jan 2025 22:30:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1737322201; bh=X6DWAh5P0K3junLC1fmV9MZVafCoG0BfpWQ9ua8wg4c=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding:From; b=p5JE7JjxsTd7t+Z4D2vlIcgymxxtluZ1+uI46wGc4JJ9hf1ob6Axz+w6VLyshrsNf IyTcVd43e0M7ppPSaFEfqFO+lgSm6I/WeduK+TCmE0KaaE608xEqK0oyF7NM2MVonQ 6FNqZCjyQ9UB7v3qJAI4TXY6JfMFyFVAQEmDvNw04D5+v7C1+ZedEhbj3eHvpHXK2Q iLU1Hs93qu9ohKwp0hAf80LjoojVrdKdzuTNQDy8rw7+VHiZGucdeYNNqi2NMk0KcA nvKyKdz+ckzmzjPf94BdD/hEGHTXg1/bZGQMF5k8W3MfUAbIsLQiTDmf5Cchh6cCiX WxkAgHUbXo5LA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4YbmpJ42HYz9rxG; Sun, 19 Jan 2025 22:30:00 +0100 (CET) From: David Elsing To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#70895] [PATCH v2] grafts: Only compute necessary graft derivations. In-Reply-To: <87ed0zsp36.fsf@gnu.org> References: <7ycyov5amj.fsf@posteo.net> <86be6fa895a3a5d9625a49e11181f483fca3c462.1717623537.git.david.elsing@posteo.net> <87ed0zsp36.fsf@gnu.org> Date: Sun, 19 Jan 2025 21:30:00 +0000 Message-ID: <86tt9utrtz.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.3 (---) X-Debbugs-Envelope-To: 70895-done Cc: 70895-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.3 (----) Hi Ludo, Ludovic Court=C3=A8s writes: > It took many months but I finally applied it. I had to update > graft-related tests in =E2=80=98tests/packages.scm=E2=80=99; I also added= a couple of > comments in the code. Thank you! > Thanks for your work, and apologies again for the delay! No problem, I think it's quite important that the change is working correctly, because grafts are used all the time. Best, David From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 20 18:27:45 2025 Received: (at 70895) by debbugs.gnu.org; 20 Jan 2025 23:27:45 +0000 Received: from localhost ([127.0.0.1]:52036 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ta1BI-0000hr-KM for submit@debbugs.gnu.org; Mon, 20 Jan 2025 18:27:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53420) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1ta1BE-0000hX-PY for 70895@debbugs.gnu.org; Mon, 20 Jan 2025 18:27:42 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ta1B9-0000UO-Fp; Mon, 20 Jan 2025 18:27:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=Gf3LYOUt7WcVb6kf1eUmOFqW+CqDzAXwDvbdPyrnJ2s=; b=aVhbbZ5TfvSf7Cg5F2Q/ zEzB6ALVRCC75pF42YHicxGOXJgg77F1muGckZJgSZffp0fykkUSs7GeobcERkgSUweA+gqY08Q70 wjJqLEGWZcgHoH/LO6Aeo9hLWRkO2K2PXC/YB2Jzxieuih7Nu/mR02AW+ajJ3lZ9x6TnnQM8vIUdY SMvc2C9xTzaojZ7v0eEyM+zAnFq/F3J0JafRN5Ab/RpN91IvAzOYap59RriLLX2eyK+YnoabCm9Ma 4XbnJH2DjpYWfdQ+LvRRDoCjLirHMXCcHjlVKdbgxVYZ0pmREsCtnaRe5LHZ8/UCjbRzWMhFydkXx C71gagpakFzyFg==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: David Elsing Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <86sepetrtn.fsf@posteo.net> (David Elsing's message of "Sun, 19 Jan 2025 21:30:12 +0000") References: <20240512134301.2234-1-david.elsing@posteo.net> <87zfs1921p.fsf@gnu.org> <7ycyov5amj.fsf@posteo.net> <87ikqbsp58.fsf@gnu.org> <86sepetrtn.fsf@posteo.net> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Primidi 1 =?utf-8?Q?Pluvi=C3=B4se?= an 233 de la =?utf-8?Q?R=C3=A9volution=2C?= jour de la =?utf-8?Q?Laur=C3=A9ole?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 21 Jan 2025 00:27:30 +0100 Message-ID: <87h65tkqvx.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, David Elsing skribis: > I meant that in 'graft-derivation/shallow', the 'mapping' variable is a > list of gexps, where 'ungexp' is called on the graft-replacement of the > grafts. So instead of turning the monadic value into a derivation > beforehand, this could be done here instead, right? Considering a > gexp-compiler returns a store monad value of a derivation (IIUC), my > question was whether it is possible to use such a value in a gexp > directly (i.e. without the 'return' in the gexp-compiler for a > derivation, such that the evaluation of the derivation is delayed until > the gexp is lowered to a derivation). These values would still need to > be identified by 'procedure?' I guess, so it would not be better than > currently. > Does this make sense or did I misunderstand something? Oh, got it. Yes, we could keep a in the =E2=80=98replacement=E2= =80=99 field instead of explicitly calling =E2=80=98package->derivation=E2=80=99. It=E2= =80=99s much simpler, that=E2=80=99s a good idea. I gave it a try, see patch attached. Let me know what you think! Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable diff --git a/guix/gexp.scm b/guix/gexp.scm index e44aea6420..d6a429e60e 100644 --- a/guix/gexp.scm +++ b/guix/gexp.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright =C2=A9 2014-2024 Ludovic Court=C3=A8s +;;; Copyright =C2=A9 2014-2025 Ludovic Court=C3=A8s ;;; Copyright =C2=A9 2018 Cl=C3=A9ment Lassieur ;;; Copyright =C2=A9 2018 Jan Nieuwenhuizen ;;; Copyright =C2=A9 2019, 2020 Mathieu Othacehe @@ -747,7 +747,12 @@ (define-gexp-compiler compile-parameterized (target (if (memq %current-target-system parameters) (%current-target-system) target))) - (lower-object (thunk) system #:target target)))))))) + (match (thunk) + ((? struct? obj) + (lower-object obj system #:target target)) + (obj + (with-monad %store-monad + (return obj))))))))))) =20 expander =3D> (lambda (parameterized lowered output) (match (parameterized-bindings parameterized) @@ -758,10 +763,13 @@ (define-gexp-compiler compile-parameterized (with-fluids* fluids (map (lambda (thunk) (thunk)) values) (lambda () - ;; Delegate to the expander of the wrapped object. - (let* ((base (thunk)) - (expand (lookup-expander base))) - (expand base lowered output))))))))) + (match (thunk) + ((? struct? base) + ;; Delegate to the expander of the wrapped obj= ect. + (let ((expand (lookup-expander base))) + (expand base lowered output))) + (obj + obj))))))))) =20 ;;; diff --git a/guix/grafts.scm b/guix/grafts.scm index 7636df9267..98ef1e4058 100644 --- a/guix/grafts.scm +++ b/guix/grafts.scm @@ -101,9 +101,11 @@ (define* (graft-derivation/shallow drv grafts ;; List of store item pairs. (map (lambda (graft) (gexp - ((ungexp (graft-origin graft) + ((ungexp (with-parameters ((%graft? #f)) + (graft-origin graft)) (graft-origin-output graft)) - . (ungexp (graft-replacement graft) + . (ungexp (with-parameters ((%graft? #t)) + (graft-replacement graft)) (graft-replacement-output graft))))) grafts)) =20 @@ -275,20 +277,6 @@ (define* (cumulative-grafts store drv grafts #:system system))))) (reference-origins drv items))) =20 - ;; If the 'replacement' field of the record is a procedure, - ;; this means that it is a value in the store monad and the actual - ;; derivation needs to be computed here. - (define (finalize-graft item) - (let ((replacement (graft-replacement item))) - (if (procedure? replacement) - (graft - (inherit item) - (replacement - (run-with-store store replacement - #:guile-for-build guile - #:system system))) - item))) - (with-cache (list (derivation-file-name drv) outputs grafts) (match (non-self-references store drv outputs) (() ;no dependencies @@ -305,8 +293,7 @@ (define* (cumulative-grafts store drv grafts ;; Use APPLICABLE, the subset of GRAFTS that is really ;; applicable to DRV, to avoid creating several identical ;; grafted variants of DRV. - (let* ((new (graft-derivation/shallow* store drv - (map finalize-graf= t applicable) + (let* ((new (graft-derivation/shallow* store drv applicab= le #:outputs outputs #:guile guile #:system system)) diff --git a/guix/packages.scm b/guix/packages.scm index d266805ba8..78726b089a 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -1818,15 +1818,13 @@ (define (input-graft system) (if replacement (mcached eq? (=3D> %package-graft-cache) (mlet %store-monad ((orig (package->derivation packa= ge system - #:gra= ft? #f)) - (new -> (package->derivation rep= lacement system - #:g= raft? #t))) - ;; Keep NEW as a monadic value so that its computa= tion - ;; is delayed until necessary. + #:gra= ft? #f))) + ;; Keep REPLACEMENT as a package so that its + ;; derivation is computed only when necessary. (return (graft (origin orig) (origin-output output) - (replacement new) + (replacement replacement) (replacement-output output)))) package output system) (return #f)))) @@ -1842,16 +1840,13 @@ (define (input-cross-graft target system) (if replacement (mlet %store-monad ((orig (package->cross-derivation package target s= ystem - #:graft?= #f)) - (new -> (package->cross-derivation replac= ement - target= system - #:graf= t? #t))) - ;; Keep NEW as a monadic value so that its computation - ;; is delayed until necessary. + #:graft?= #f))) + ;; Keep REPLACEMENT as a package so that its derivation is + ;; computed only when necessary. (return (graft (origin orig) (origin-output output) - (replacement new) + (replacement replacement) (replacement-output output)))) (return #f)))) (_ diff --git a/tests/packages.scm b/tests/packages.scm index a4a0e2c3e8..2863fb5991 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1095,9 +1095,7 @@ (define compressors '(("gzip" . "gz") ((graft) (and (eq? (graft-origin graft) (package-derivation %store dep)) - (eq? (run-with-store %store - (graft-replacement graft)) - (package-derivation %store new))))))) + (eq? (graft-replacement graft) new)))))) =20 ;; XXX: This test would require building the cross toolchain just to see i= f it ;; needs grafting, which is obviously too expensive, and thus disabled. @@ -1134,9 +1132,7 @@ (define compressors '(("gzip" . "gz") ((graft) (and (eq? (graft-origin graft) (package-derivation %store dep)) - (eq? (run-with-store %store - (graft-replacement graft)) - (package-derivation %store new))))))) + (eq? (graft-replacement graft) new)))))) =20 (test-assert "package-grafts, same replacement twice" (let* ((new (dummy-package "dep" @@ -1161,9 +1157,7 @@ (define compressors '(("gzip" . "gz") (package-derivation %store (package (inherit dep) (replacement #f)))) - (eq? (run-with-store %store - (graft-replacement graft)) - (package-derivation %store new))))))) + (eq? (graft-replacement graft) new)))))) =20 (test-assert "package-grafts, dependency on several outputs" ;; Make sure we get one graft per output; see . @@ -1183,9 +1177,9 @@ (define compressors '(("gzip" . "gz") ((graft1 graft2) (and (eq? (graft-origin graft1) (graft-origin graft2) (package-derivation %store p0)) - (eq? (run-with-store %store (graft-replacement graft1)) - (run-with-store %store (graft-replacement graft2)) - (package-derivation %store p0*)) + (eq? (graft-replacement graft1) + (graft-replacement graft2) + p0*) (string=3D? "lib" (graft-origin-output graft1) (graft-replacement-output graft1)) @@ -1262,14 +1256,10 @@ (define compressors '(("gzip" . "gz") ((graft1 graft2) (and (eq? (graft-origin graft1) (package-derivation %store p1 #:graft? #f)) - (eq? (run-with-store %store - (graft-replacement graft1)) - (package-derivation %store p1r)) + (eq? (graft-replacement graft1) p1r) (eq? (graft-origin graft2) (package-derivation %store p2 #:graft? #f)) - (eq? (run-with-store %store - (graft-replacement graft2)) - (package-derivation %store p2r #:graft? #t))))))) + (eq? (graft-replacement graft2) p2r)))))) =20 ;;; XXX: Nowadays 'graft-derivation' needs to build derivations beforehand= to ;;; find out about their run-time dependencies, so this test is no longer --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 21 16:11:38 2025 Received: (at 70895) by debbugs.gnu.org; 21 Jan 2025 21:11:38 +0000 Received: from localhost ([127.0.0.1]:57327 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1taLX7-0006L4-LA for submit@debbugs.gnu.org; Tue, 21 Jan 2025 16:11:38 -0500 Received: from mout02.posteo.de ([185.67.36.66]:50859) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1taLX2-0006Kh-7t for 70895@debbugs.gnu.org; Tue, 21 Jan 2025 16:11:36 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 84B96240101 for <70895@debbugs.gnu.org>; Tue, 21 Jan 2025 22:11:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1737493885; bh=qp/enXVxoTp1N66OcCUpqgzmhxsbJZEYOQ0u5vloRx0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: From; b=on28xO9niwnSJWzjrKEo4QL6mFiHdNlLyqNOWOc/WTtTP+EJvjpst1FINPH5fZy89 GDFR534p6OhmB1AycNVy45sv4iwlXXqPFp32Q5wTAkNwUZRvltcl91IjwZKS8PhtSO KiuYHtq1IZRkUilQo+sb1U5JuCpwDhscOso5u6qZCTcgDqL0JrAc7L//ke9oCLNgOL MKaTtEF29hc3NQHIV/WLKLc4GmdRtAuu694McuVvu1PaOCinAHn7Bu7+GJnYGoRFG7 fI/HhzVlAsTxVdg2CHg2iPI5cVOw7SoCavQQaPaX5qB0RuzbqVM6ypO5USIZOihjW3 r5puOvQtrqJTA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Yd0Hv0YT8z9rxB; Tue, 21 Jan 2025 22:11:22 +0100 (CET) From: David Elsing To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <87h65tkqvx.fsf@gnu.org> References: <20240512134301.2234-1-david.elsing@posteo.net> <87zfs1921p.fsf@gnu.org> <7ycyov5amj.fsf@posteo.net> <87ikqbsp58.fsf@gnu.org> <86sepetrtn.fsf@posteo.net> <87h65tkqvx.fsf@gnu.org> Date: Tue, 21 Jan 2025 21:11:22 +0000 Message-ID: <7yldv3q3d1.fsf@posteo.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo', Ludovic Court=C3=A8s writes: > Oh, got it. Yes, we could keep a in the =E2=80=98replacement= =E2=80=99 field > instead of explicitly calling =E2=80=98package->derivation=E2=80=99. It= =E2=80=99s much simpler, > that=E2=80=99s a good idea. Oh nice, that's really neat! I didn't know about , that achieves exactly what I was hoping for. > I gave it a try, see patch attached. Let me know what you think! Is there a reason you use 'with-parameters' in 'graft-derivation/shallow' and not in 'input-graft' and 'input-cross-graft'? I attached a patch below where I do that and also set %current-system and %current-target-system (although I'm not sure they are strictly necessary because of the 'parameterize' in 'bag-grafts'). Then, the changes to the gexp-compiler of are not required to allow for strings in the 'replacement' field in tests/grafts.scm and the tests still pass. Cheers, David --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=grafts.patch diff --git a/guix/grafts.scm b/guix/grafts.scm index 7636df9267..e93a5e60bb 100644 --- a/guix/grafts.scm +++ b/guix/grafts.scm @@ -275,20 +275,6 @@ (define (dependency-grafts items) #:system system))))) (reference-origins drv items))) - ;; If the 'replacement' field of the record is a procedure, - ;; this means that it is a value in the store monad and the actual - ;; derivation needs to be computed here. - (define (finalize-graft item) - (let ((replacement (graft-replacement item))) - (if (procedure? replacement) - (graft - (inherit item) - (replacement - (run-with-store store replacement - #:guile-for-build guile - #:system system))) - item))) - (with-cache (list (derivation-file-name drv) outputs grafts) (match (non-self-references store drv outputs) (() ;no dependencies @@ -305,8 +291,7 @@ (define (finalize-graft item) ;; Use APPLICABLE, the subset of GRAFTS that is really ;; applicable to DRV, to avoid creating several identical ;; grafted variants of DRV. - (let* ((new (graft-derivation/shallow* store drv - (map finalize-graft applicable) + (let* ((new (graft-derivation/shallow* store drv applicable #:outputs outputs #:guile guile #:system system)) diff --git a/guix/packages.scm b/guix/packages.scm index d266805ba8..c9e441ffeb 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -1817,16 +1817,19 @@ (define (input-graft system) (let ((replacement (package-replacement package))) (if replacement (mcached eq? (=> %package-graft-cache) - (mlet %store-monad ((orig (package->derivation package system - #:graft? #f)) - (new -> (package->derivation replacement system - #:graft? #t))) - ;; Keep NEW as a monadic value so that its computation - ;; is delayed until necessary. + (mlet %store-monad + ((orig (package->derivation package system + #:graft? #f)) + ;; Do not compute the derivation of REPLACEMENT + ;; yet, as it might not be needed. + (replacement -> (with-parameters + ((%graft? #t) + (%current-system system)) + replacement))) (return (graft (origin orig) (origin-output output) - (replacement new) + (replacement replacement) (replacement-output output)))) package output system) (return #f)))) @@ -1840,18 +1843,21 @@ (define (input-cross-graft target system) (((? package? package) output) (let ((replacement (package-replacement package))) (if replacement - (mlet %store-monad ((orig (package->cross-derivation package - target system - #:graft? #f)) - (new -> (package->cross-derivation replacement - target system - #:graft? #t))) - ;; Keep NEW as a monadic value so that its computation - ;; is delayed until necessary. + (mlet %store-monad + ((orig (package->cross-derivation package + target system + #:graft? #f)) + ;; Do not compuate the derivation of REPLACEMENT + ;; yet, as it might not be needed. + (replacement -> (with-parameters + ((%graft? #t) + (%current-system system) + (%current-target-system target)) + replacement))) (return (graft (origin orig) (origin-output output) - (replacement new) + (replacement replacement) (replacement-output output)))) (return #f)))) (_ diff --git a/tests/packages.scm b/tests/packages.scm index a4a0e2c3e8..2863fb5991 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1095,9 +1095,7 @@ (define right-system? ((graft) (and (eq? (graft-origin graft) (package-derivation %store dep)) - (eq? (run-with-store %store - (graft-replacement graft)) - (package-derivation %store new))))))) + (eq? (graft-replacement graft) new)))))) ;; XXX: This test would require building the cross toolchain just to see if it ;; needs grafting, which is obviously too expensive, and thus disabled. @@ -1134,9 +1132,7 @@ (define right-system? ((graft) (and (eq? (graft-origin graft) (package-derivation %store dep)) - (eq? (run-with-store %store - (graft-replacement graft)) - (package-derivation %store new))))))) + (eq? (graft-replacement graft) new)))))) (test-assert "package-grafts, same replacement twice" (let* ((new (dummy-package "dep" @@ -1161,9 +1157,7 @@ (define right-system? (package-derivation %store (package (inherit dep) (replacement #f)))) - (eq? (run-with-store %store - (graft-replacement graft)) - (package-derivation %store new))))))) + (eq? (graft-replacement graft) new)))))) (test-assert "package-grafts, dependency on several outputs" ;; Make sure we get one graft per output; see . @@ -1183,9 +1177,9 @@ (define right-system? ((graft1 graft2) (and (eq? (graft-origin graft1) (graft-origin graft2) (package-derivation %store p0)) - (eq? (run-with-store %store (graft-replacement graft1)) - (run-with-store %store (graft-replacement graft2)) - (package-derivation %store p0*)) + (eq? (graft-replacement graft1) + (graft-replacement graft2) + p0*) (string=? "lib" (graft-origin-output graft1) (graft-replacement-output graft1)) @@ -1262,14 +1256,10 @@ (define right-system? ((graft1 graft2) (and (eq? (graft-origin graft1) (package-derivation %store p1 #:graft? #f)) - (eq? (run-with-store %store - (graft-replacement graft1)) - (package-derivation %store p1r)) + (eq? (graft-replacement graft1) p1r) (eq? (graft-origin graft2) (package-derivation %store p2 #:graft? #f)) - (eq? (run-with-store %store - (graft-replacement graft2)) - (package-derivation %store p2r #:graft? #t))))))) + (eq? (graft-replacement graft2) p2r)))))) ;;; XXX: Nowadays 'graft-derivation' needs to build derivations beforehand to ;;; find out about their run-time dependencies, so this test is no longer --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 28 10:58:51 2025 Received: (at 70895) by debbugs.gnu.org; 28 Jan 2025 15:58:52 +0000 Received: from localhost ([127.0.0.1]:38155 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tcnzH-0005QU-Es for submit@debbugs.gnu.org; Tue, 28 Jan 2025 10:58:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37260) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tcnyi-0005P9-UK for 70895@debbugs.gnu.org; Tue, 28 Jan 2025 10:58:17 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tcnyd-0000eR-5T; Tue, 28 Jan 2025 10:58:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=pO6qan918bc4rCz6eUhBbtSIuiJf2ZzLpPrWkrnO1lA=; b=MB2UwhIZyq/LmIDHF8UW 5DT9PMMJela7JvqcJ7+6Hj6gcuYj+aILEGmyQBWJpc9WKl/DDQxPtHlh/mNA1lnz1KlJHXtnUahXI KBTuE9JwSvkhxzWspZgBtbH6w2ZOM/fl31qKVrdmVGNwURrmNsXiyJ+2IDvrgu5XVvYQ2H7FD1x/r yX3WQSz0oMZrzeVZVUjpvcQ+fFDBUeh9M0w7F16n0zrgi7hVB3dy4qyR3HBvj8+eRsqmdgG1zOJZR XC5pQz2dcMC+Rgi56zwashcuc2KkuBFFhudoIDwYqmfWeWKLsXmnFzRgmSclqbZ0SDksSfHZTo25B 2i77AgUB1TpFZw==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: David Elsing Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <7yldv3q3d1.fsf@posteo.net> (David Elsing's message of "Tue, 21 Jan 2025 21:11:22 +0000") References: <20240512134301.2234-1-david.elsing@posteo.net> <87zfs1921p.fsf@gnu.org> <7ycyov5amj.fsf@posteo.net> <87ikqbsp58.fsf@gnu.org> <86sepetrtn.fsf@posteo.net> <87h65tkqvx.fsf@gnu.org> <7yldv3q3d1.fsf@posteo.net> Date: Tue, 28 Jan 2025 16:58:08 +0100 Message-ID: <87plk7aq27.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, David Elsing skribis: > Is there a reason you use 'with-parameters' in 'graft-derivation/shallow' > and not in 'input-graft' and 'input-cross-graft'? I attached a patch > below where I do that and also set %current-system and > %current-target-system (although I'm not sure they are strictly > necessary because of the 'parameterize' in 'bag-grafts'). > Then, the changes to the gexp-compiler of are not > required to allow for strings in the 'replacement' field in > tests/grafts.scm and the tests still pass. My thought was that the fact that grafts must be enabled on the replacement, etc., are implementation details. I=E2=80=99d like producers = of records to just be able to drop whatever is relevant to them in the =E2=80=98replacement=E2=80=99 field. I committed these the patch I sent before as two commits, and I added a test in =E2=80=98tests/gexp.scm=E2=80=99 for =E2=80=98with-parameters=E2=80= =99: 28e4018e59 grafts: Allow file-like objects in the =E2=80=98replacement=E2= =80=99 field of . 3ad2d21671 gexp: =E2=80=98with-parameters=E2=80=99 accepts plain store it= ems in its body. Let me know if you notice something wrong. Thanks a lot for suggesting this, it=E2=80=99s much nicer this way! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 29 14:31:14 2025 Received: (at 70895) by debbugs.gnu.org; 29 Jan 2025 19:31:14 +0000 Received: from localhost ([127.0.0.1]:43112 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tdDmL-0001GF-MW for submit@debbugs.gnu.org; Wed, 29 Jan 2025 14:31:14 -0500 Received: from mout02.posteo.de ([185.67.36.66]:47467) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tdDmI-0001Fy-FW for 70895@debbugs.gnu.org; Wed, 29 Jan 2025 14:31:12 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id CB92B240104 for <70895@debbugs.gnu.org>; Wed, 29 Jan 2025 20:31:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1738179062; bh=9J0V1Z8w2VUJijMdAa5QtoU4Ve94or8lJKyRa0iUbZI=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding:From; b=fkuq7T2+PJjCeKpt+EqywjHlZXYEuRHQ0rslP1vh1X0u7nsL82wSgyiA4Sex2tHMR CFlrEay1Mw43JZmjPsV1v+vC7libiTaQaD3tpkey+EM15Xka1+5cPhulUOYWIRJbw1 4D0lbed3ho8tZ9eVjolwp2JQsVVWajUsqxG/jmpm1MYedqRJA4fUAx80ZX5oZqoFYM NScEgBK3aih7c0NgNfcbdUCqGrhjs9rHzN8dqr54SVZ7lT8V97bLY0Bh21sppa/E8m BkZbn+3l9IFhhYCnMRRX1r9o6z3gJ1Itj9NgA5e4WPBw7oiTR7/djbRft7RlRHRTe5 j5TtzfTzv9YWg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4YjshP1gWHz6v0s; Wed, 29 Jan 2025 20:30:59 +0100 (CET) From: David Elsing To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <87plk7aq27.fsf@gnu.org> References: <20240512134301.2234-1-david.elsing@posteo.net> <87zfs1921p.fsf@gnu.org> <7ycyov5amj.fsf@posteo.net> <87ikqbsp58.fsf@gnu.org> <86sepetrtn.fsf@posteo.net> <87h65tkqvx.fsf@gnu.org> <7yldv3q3d1.fsf@posteo.net> <87plk7aq27.fsf@gnu.org> Date: Wed, 29 Jan 2025 19:30:59 +0000 Message-ID: <7yfrl1pgcs.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Ludovic Court=C3=A8s writes: > My thought was that the fact that grafts must be enabled on the > replacement, etc., are implementation details. I=E2=80=99d like producer= s of > records to just be able to drop whatever is relevant to them in > the =E2=80=98replacement=E2=80=99 field. Ah sure, that makes sense to me. > Let me know if you notice something wrong. I noticed that 'with-parameters' doesn't actually do anything here. :) The 'graft-origin' is already a derivation anyway, so there it has obviously no effect. For the 'graft-replacement', I noticed that when setting %graft? to #f, it still results in the same grafted derivation. The same was the case in my version using 'with-parameters' in 'input-graft' of course. I also wanted to use 'with-parameters' for the ROCm packages, but found that it generally does not work for packages [1], this seems to be also the case here. IIUC, this is because 'lower-object' returns a monadic procedure, which is evaluated outside the influence of 'with-fluids' in the gexp-compiler of . The grafting still works correctly however, I think because the %graft? parameter is also set when the derivations are actually calculated, while in 'input-graft', 'package->derivation' is explicitely called with #:graft? set to #f. What do you think? Best, David [1] https://issues.guix.gnu.org/75879 From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 31 11:26:25 2025 Received: (at 70895) by debbugs.gnu.org; 31 Jan 2025 16:26:25 +0000 Received: from localhost ([127.0.0.1]:53973 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tdtqb-0000E4-Ct for submit@debbugs.gnu.org; Fri, 31 Jan 2025 11:26:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42892) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tdtqY-0000Do-Ne for 70895@debbugs.gnu.org; Fri, 31 Jan 2025 11:26:23 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tdtqT-0002za-9R; Fri, 31 Jan 2025 11:26:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=Y/AF/9xmC+4ib7cazDFcmZLXSPdhpAO7sBhsjLmrMO8=; b=hwR20YZj/gPBsAl6ikvm LMQLxE2lKZFgd3oavEPcQ4CLfGofkrzlEC7eKaXOnQBz7/DahuI18ID/wMMhmPYsyou/eYiymGT0g e9VKOoOFfugYFrh6OxrSR4SqxsVzaTJ0fWVJbIm5M6TQbPzRt8bhlLE910jyvOf0avaKDfJL4/tzA HEh42VRO3r0x62Uva7zkGj4hOJyl/iJlaLroeApz7/UA8aCnGE6od/15ctNbbhzZW2E2JOOrYzYIo l9bwlZskDj1yUi2Tz8djza+OJ0NEK0z3wH9tO1hIfoLrN+dk+14IMaXsOJQJCFOkTsFp+H86lMbpT 6OclQJMtja8ZBg==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: David Elsing Subject: Re: [bug#70895] [PATCH] grafts: Only compute necessary graft derivations. In-Reply-To: <7yfrl1pgcs.fsf@posteo.net> (David Elsing's message of "Wed, 29 Jan 2025 19:30:59 +0000") References: <20240512134301.2234-1-david.elsing@posteo.net> <87zfs1921p.fsf@gnu.org> <7ycyov5amj.fsf@posteo.net> <87ikqbsp58.fsf@gnu.org> <86sepetrtn.fsf@posteo.net> <87h65tkqvx.fsf@gnu.org> <7yldv3q3d1.fsf@posteo.net> <87plk7aq27.fsf@gnu.org> <7yfrl1pgcs.fsf@posteo.net> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Duodi 12 =?utf-8?Q?Pluvi=C3=B4se?= an 233 de la =?utf-8?Q?R=C3=A9volution=2C?= jour du Brocoli X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 31 Jan 2025 17:26:13 +0100 Message-ID: <874j1fc5lm.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 70895 Cc: 70895@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi David, David Elsing skribis: >> My thought was that the fact that grafts must be enabled on the >> replacement, etc., are implementation details. I=E2=80=99d like produce= rs of >> records to just be able to drop whatever is relevant to them in >> the =E2=80=98replacement=E2=80=99 field. > > Ah sure, that makes sense to me. > >> Let me know if you notice something wrong. > > I noticed that 'with-parameters' doesn't actually do anything here. :) > The 'graft-origin' is already a derivation anyway, so there it has > obviously no effect. For the 'graft-replacement', I noticed that when > setting %graft? to #f, it still results in the same grafted derivation. > The same was the case in my version using 'with-parameters' in > 'input-graft' of course. Hmm. For =E2=80=98graft-origin=E2=80=99, it doesn=E2=80=99t really matter;= but the replacement, it does. > The grafting still works correctly however, I think because the %graft? > parameter is also set when the derivations are actually calculated, > while in 'input-graft', 'package->derivation' is explicitely called with > #:graft? set to #f. OK, pfew. > [1] https://issues.guix.gnu.org/75879 Uh, looks like this is a real bug. I=E2=80=99m surprised because we do have tests for that in =E2=80=98tests/gexp.scm=E2=80=99 (and it=E2=80=99s actual= ly used in a few important places), but maybe they=E2=80=99re not exercising the right thing. Ludo=E2=80=99. From unknown Tue Jun 17 01:48:45 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 01 Mar 2025 12:24:24 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator