From unknown Mon Jun 23 20:20:33 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#55316] [PATCH] scripts: package: Transform before creating manifest entries. Resent-From: Josselin Poiret Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 08 May 2022 14:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 55316 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 55316@debbugs.gnu.org Cc: Josselin Poiret X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.165201939919056 (code B ref -1); Sun, 08 May 2022 14:17:02 +0000 Received: (at submit) by debbugs.gnu.org; 8 May 2022 14:16:39 +0000 Received: from localhost ([127.0.0.1]:54976 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnhiA-0004wl-O4 for submit@debbugs.gnu.org; Sun, 08 May 2022 10:16:39 -0400 Received: from lists.gnu.org ([209.51.188.17]:35582) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnhi8-0004uh-ET for submit@debbugs.gnu.org; Sun, 08 May 2022 10:16:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nnhi8-0000G9-7G for guix-patches@gnu.org; Sun, 08 May 2022 10:16:36 -0400 Received: from jpoiret.xyz ([206.189.101.64]:38898) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nnhi6-0001kU-68 for guix-patches@gnu.org; Sun, 08 May 2022 10:16:35 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id C8898184D6A; Sun, 8 May 2022 14:16:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1652019390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=A67dAy3wWxjOorffQG3HZsQVeQhnwWJewn4fF5gbDpc=; b=oswEZ+urVi5zakk/ilCsXngeDmPaEFXbiSqbHOXuNUr/wyixG5Q3B0WJp9Ze1MMuamOGfr Ev7PRM6H8jkj0Rp0rigw+WRQXJKRzFPbRkvzRkJGNNeQs/Af9JYp3CYtuBgRtbYd++t6UP pP6YKat48TAOcR092LUuUfYIby0XCYPMn8sWsp9bD96UWF9S89pWS1alcVXZEk8CqL4XFY WOjdPEZIE3Yw8DhaueOC36TFAP9ICrnSgtDjQw/68vYocLHNxLFcN82uZ5kFzuuMdo0Yi2 U2AkWnZDP0hSfSirM+GITfvR8yertTaHGsH9q/zlS3fRqlWUhwV2JLAiWqMwCw== From: Josselin Poiret Date: Sun, 8 May 2022 16:16:06 +0200 Message-Id: <20220508141606.14085-1-dev@jpoiret.xyz> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spamd-Bar: / Received-SPF: pass client-ip=206.189.101.64; envelope-from=dev@jpoiret.xyz; helo=jpoiret.xyz X-Spam_score_int: 24 X-Spam_score: 2.4 X-Spam_bar: ++ X-Spam_report: (2.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_SUSPICIOUS_NTLD=0.499, FROM_SUSPICIOUS_NTLD_FP=1.997, PDS_OTHER_BAD_TLD=1.997, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 2.7 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/scripts/package.scm (options->installable): Add TRANSFORM argument, to be able to directly transform the new packages before creating their manifest entries. (process-actions): Remove transform [...] Content analysis details: (2.7 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.17 listed in list.dnswl.org] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.9 SPF_FAIL SPF: sender does not match SPF record (fail) [SPF failed: Please see http://www.openspf.org/Why?s=mfrom; id=dev%40jpoiret.xyz; ip=209.51.188.17; r=debbugs.gnu.org] 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.51.188.17 listed in wl.mailspike.net] 2.0 FROM_SUSPICIOUS_NTLD_FP From abused NTLD 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.3 (/) * guix/scripts/package.scm (options->installable): Add TRANSFORM argument, to be able to directly transform the new packages before creating their manifest entries. (process-actions): Remove transform-entry, and step3. --- Hello everyone, Someone on IRC [1] reported that they couldn't run guix install emacs-avy emacs-embark --with-branch=emacs-avy=master --with-branch=emacs-embark=master without guix complaining about conflicting entries of emacs-avy and the propagated one from emacs-embark, even in a profile without either of them. The issue was that package transformations were applied to the contents of the generated manifest entries, but weren't applied to any of their dependencies. To solve this, I figured it would be simpler to just apply the transformation from the start, only creating the manifest entries after they have been applied. Also, the default `package->manifest-entry` preserves transformations properties by default, so this doesn't lose any info either. With this patch applied, the above command works as expected for me. Best, Josselin Poiret [1] https://logs.guix.gnu.org/guix/2022-05-08.log#134728 guix/scripts/package.scm | 36 ++++++++++++------------------------ 1 file changed, 12 insertions(+), 24 deletions(-) diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index d007005607..4673cf18b2 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -10,6 +10,7 @@ ;;; Copyright © 2020 Ricardo Wurmus ;;; Copyright © 2020 Simon Tournier ;;; Copyright © 2018 Steve Sprang +;;; Copyright © 2022 Josselin Poiret ;;; ;;; This file is part of GNU Guix. ;;; @@ -694,10 +695,10 @@ (define (package->manifest-entry* package output) (manifest-entry-with-provenance (package->manifest-entry package output))) -(define (options->installable opts manifest transaction) - "Given MANIFEST, the current manifest, and OPTS, the result of 'args-fold', -return an variant of TRANSACTION that accounts for the specified installations -and upgrades." +(define (options->installable opts manifest transform transaction) + "Given MANIFEST, the current manifest, OPTS, and TRANSFORM, the result of +'args-fold', return an variant of TRANSACTION that accounts for the specified +installations, upgrades and transformations." (define upgrade? (options->upgrade-predicate opts)) @@ -714,13 +715,14 @@ (define to-install (('install . (? package? p)) ;; When given a package via `-e', install the first of its ;; outputs (XXX). - (package->manifest-entry* p "out")) + (package->manifest-entry* (transform p) "out")) (('install . (? string? spec)) (if (store-path? spec) (store-item->manifest-entry spec) (let-values (((package output) (specification->package+output spec))) - (package->manifest-entry* package output)))) + (package->manifest-entry* (transform package) + output)))) (('install . obj) (leave (G_ "cannot install non-package object: ~s~%") obj)) @@ -979,16 +981,6 @@ (define allow-collisions? (assoc-ref opts 'allow-collisions?)) (define profile (or (assoc-ref opts 'profile) %current-profile)) (define transform (options->transformation opts)) - (define (transform-entry entry) - (let ((item (transform (manifest-entry-item entry)))) - (manifest-entry-with-transformations - (manifest-entry - (inherit entry) - (item item) - (version (if (package? item) - (package-version item) - (manifest-entry-version entry))))))) - (when (equal? profile %current-profile) ;; Normally the daemon created %CURRENT-PROFILE when we connected, unless ;; it's a version that lacks the fix for @@ -1021,16 +1013,12 @@ (define (transform-entry entry) (map load-manifest files)))))) (step1 (options->removable opts manifest (manifest-transaction))) - (step2 (options->installable opts manifest step1)) - (step3 (manifest-transaction - (inherit step2) - (install (map transform-entry - (manifest-transaction-install step2))))) - (new (manifest-perform-transaction manifest step3)) + (step2 (options->installable opts manifest transform step1)) + (new (manifest-perform-transaction manifest step2)) (trans (if (null? files) - step3 + step2 (fold manifest-transaction-install-entry - step3 + step2 (manifest-entries manifest))))) (warn-about-old-distro) -- 2.35.1 From unknown Mon Jun 23 20:20:33 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#55316] [PATCH] scripts: package: Transform before creating manifest entries. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 09 May 2022 09:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 55316 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Josselin Poiret Cc: 55316@debbugs.gnu.org Received: via spool by 55316-submit@debbugs.gnu.org id=B55316.165208992920437 (code B ref 55316); Mon, 09 May 2022 09:53:02 +0000 Received: (at 55316) by debbugs.gnu.org; 9 May 2022 09:52:09 +0000 Received: from localhost ([127.0.0.1]:56154 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1no03k-0005JY-Nf for submit@debbugs.gnu.org; Mon, 09 May 2022 05:52:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49332) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1no03h-0005J4-W8 for 55316@debbugs.gnu.org; Mon, 09 May 2022 05:52:06 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48012) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1no03c-0004xO-9i; Mon, 09 May 2022 05:52:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=DptBGxhxGMHgblwT0ax1xBoOIba/X3/4YmMfkfc1+3s=; b=e/feirmxv+UyIVZ2HWpF kRcD7eLf79op2kiJJNZhdjhdjt1WRtGM9Kl6L95cVdFdAh/zoK6bQRTwRxFafYR2qOpzjJ80ZdFDL DZPXWbSu7FPNKmCNm0DKh+DElVI6r1L8SQEPrdOvUkZv49Vyl/0+Bi5SeNBylM4eAAyfLtYrSYM7E jH0mCX1MYMgWqExhRmw5xZ+hrUX4/afnPTgOLAoftx8mOLOmXihHey1uE2R8XHu4TFGNNhNtMmCuA 2XyHYlhfzibiKVAeMZj7wW6HONO5g/9XaHmlpJhE2D6EnHaHu3AJyWznLzj1S6UKTE2GHEVA4/cEv T7tdLUBvaJjmkQ==; Received: from [193.50.110.244] (port=52676 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1no03b-0000cU-OF; Mon, 09 May 2022 05:51:59 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20220508141606.14085-1-dev@jpoiret.xyz> Date: Mon, 09 May 2022 11:51:58 +0200 In-Reply-To: <20220508141606.14085-1-dev@jpoiret.xyz> (Josselin Poiret's message of "Sun, 8 May 2022 16:16:06 +0200") Message-ID: <874k1zrq2p.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hello! Josselin Poiret skribis: > * guix/scripts/package.scm (options->installable): Add TRANSFORM > argument, to be able to directly transform the new packages before > creating their manifest entries. > (process-actions): Remove transform-entry, and step3. > --- > Hello everyone, > > Someone on IRC [1] reported that they couldn't run > > guix install emacs-avy emacs-embark --with-branch=3Demacs-avy=3Dmaster --= with-branch=3Demacs-embark=3Dmaster > > without guix complaining about conflicting entries of emacs-avy and > the propagated one from emacs-embark, even in a profile without either > of them. The issue was that package transformations were applied to > the contents of the generated manifest entries, but weren't applied to > any of their dependencies. Interesting. > To solve this, I figured it would be simpler to just apply the > transformation from the start, only creating the manifest entries > after they have been applied. Also, the default > `package->manifest-entry` preserves transformations properties by > default, so this doesn't lose any info either. As discussed on IRC, could you make sure transformation properties are preserved in the manifest? I only see one test that explicitly checks that, "transaction-upgrade-entry, transformation options preserved" in tests/packages.scm, and then there=E2=80=99s "options->transformation + package->manifest-entry" in tests/transformations.scm. Also, could you add a test that reproduces the problem being fixed? Thanks! Ludo=E2=80=99. From unknown Mon Jun 23 20:20:33 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#55316] [PATCH v2] scripts: package: Transform before creating manifest entries. Resent-From: Josselin Poiret Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 09 May 2022 14:55:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 55316 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Josselin Poiret , 55316@debbugs.gnu.org Received: via spool by 55316-submit@debbugs.gnu.org id=B55316.165210808111875 (code B ref 55316); Mon, 09 May 2022 14:55:01 +0000 Received: (at 55316) by debbugs.gnu.org; 9 May 2022 14:54:41 +0000 Received: from localhost ([127.0.0.1]:59176 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1no4mW-00035S-Az for submit@debbugs.gnu.org; Mon, 09 May 2022 10:54:41 -0400 Received: from jpoiret.xyz ([206.189.101.64]:34958) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1no4mF-000354-HK for 55316@debbugs.gnu.org; Mon, 09 May 2022 10:54:38 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id 71822184BF2; Mon, 9 May 2022 14:54:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1652108060; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cXRDT04PPHTWUq5I1f09RzyarUFtYd+01K7FsLwRzxw=; b=n+S9QgzRnmhpPrR+KibRv/TWYgEP7PXvHV+ItgvldFGYJwYYnzw9iPKWqBzZ4JnJ5UlKW2 JyQlAXCOEkdv6QFa+HhH/c1PN3KfbXxQFLV5QmRFhcprwDgSPxza3O75lf0uflJsVXCh0V xJhf36d5vSGy7XcTryao0uZljDWN7ZW4QVLXs3o2xAe/g5n4sbbdJ2SdgGvJi79ufdD7fx eeEeo1iPWjmztvUzbm1TMhGl8/5O7xMlwQHPbTLDlInxAgpfdis6OrwpTEXHXoIFGo27tf 7sON17w+hpIKm9Y2bMi1d6wkg0kHgo5WM3QQFIloY+KFifuHHbT4344B8bPDiw== From: Josselin Poiret Date: Mon, 9 May 2022 16:54:10 +0200 Message-Id: <20220509145410.11678-1-dev@jpoiret.xyz> In-Reply-To: <874k1zrq2p.fsf@gnu.org> References: <874k1zrq2p.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spamd-Bar: / X-Spam-Score: 2.1 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/scripts/package.scm (options->installable): Add TRANSFORM argument, to be able to directly transform the new packages before creating their manifest entries. (process-actions): Remove transform [...] Content analysis details: (2.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 T_SPF_HELO_TEMPERROR SPF: test of HELO record failed (temperror) -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.1 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/scripts/package.scm (options->installable): Add TRANSFORM argument, to be able to directly transform the new packages before creating their manifest entries. (process-actions): Remove transform [...] Content analysis details: (1.1 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager * guix/scripts/package.scm (options->installable): Add TRANSFORM argument, to be able to directly transform the new packages before creating their manifest entries. (process-actions): Remove transform-entry, and step3, transforming directly in step2. * tests/guix-package.sh: Add test. --- Hello Ludo, Thanks for your review. As mentioned on IRC, those tests don't test process-actions, which is where the logic resides. I added a test to guix-package.sh that follows the original issue instead, note although that between yesterday and today, emacs-consult has changed enough to not compile anymore with `--with-branch=emacs-consult=main` (branch rename notwithstanding). Best, Josselin guix/scripts/package.scm | 36 ++++++++++++------------------------ tests/guix-package.sh | 30 ++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 24 deletions(-) diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index d007005607..4673cf18b2 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -10,6 +10,7 @@ ;;; Copyright © 2020 Ricardo Wurmus ;;; Copyright © 2020 Simon Tournier ;;; Copyright © 2018 Steve Sprang +;;; Copyright © 2022 Josselin Poiret ;;; ;;; This file is part of GNU Guix. ;;; @@ -694,10 +695,10 @@ (define (package->manifest-entry* package output) (manifest-entry-with-provenance (package->manifest-entry package output))) -(define (options->installable opts manifest transaction) - "Given MANIFEST, the current manifest, and OPTS, the result of 'args-fold', -return an variant of TRANSACTION that accounts for the specified installations -and upgrades." +(define (options->installable opts manifest transform transaction) + "Given MANIFEST, the current manifest, OPTS, and TRANSFORM, the result of +'args-fold', return an variant of TRANSACTION that accounts for the specified +installations, upgrades and transformations." (define upgrade? (options->upgrade-predicate opts)) @@ -714,13 +715,14 @@ (define to-install (('install . (? package? p)) ;; When given a package via `-e', install the first of its ;; outputs (XXX). - (package->manifest-entry* p "out")) + (package->manifest-entry* (transform p) "out")) (('install . (? string? spec)) (if (store-path? spec) (store-item->manifest-entry spec) (let-values (((package output) (specification->package+output spec))) - (package->manifest-entry* package output)))) + (package->manifest-entry* (transform package) + output)))) (('install . obj) (leave (G_ "cannot install non-package object: ~s~%") obj)) @@ -979,16 +981,6 @@ (define allow-collisions? (assoc-ref opts 'allow-collisions?)) (define profile (or (assoc-ref opts 'profile) %current-profile)) (define transform (options->transformation opts)) - (define (transform-entry entry) - (let ((item (transform (manifest-entry-item entry)))) - (manifest-entry-with-transformations - (manifest-entry - (inherit entry) - (item item) - (version (if (package? item) - (package-version item) - (manifest-entry-version entry))))))) - (when (equal? profile %current-profile) ;; Normally the daemon created %CURRENT-PROFILE when we connected, unless ;; it's a version that lacks the fix for @@ -1021,16 +1013,12 @@ (define (transform-entry entry) (map load-manifest files)))))) (step1 (options->removable opts manifest (manifest-transaction))) - (step2 (options->installable opts manifest step1)) - (step3 (manifest-transaction - (inherit step2) - (install (map transform-entry - (manifest-transaction-install step2))))) - (new (manifest-perform-transaction manifest step3)) + (step2 (options->installable opts manifest transform step1)) + (new (manifest-perform-transaction manifest step2)) (trans (if (null? files) - step3 + step2 (fold manifest-transaction-install-entry - step3 + step2 (manifest-entries manifest))))) (warn-about-old-distro) diff --git a/tests/guix-package.sh b/tests/guix-package.sh index d1b383d2ad..c2f5e39de0 100644 --- a/tests/guix-package.sh +++ b/tests/guix-package.sh @@ -1,6 +1,7 @@ # GNU Guix --- Functional package management for GNU # Copyright © 2012-2022 Ludovic Courtès # Copyright © 2013 Nikita Karetnikov +# Copyright © 2022 Josselin Poiret # # This file is part of GNU Guix. # @@ -210,6 +211,35 @@ test "$(readlink -f "$profile/bin/guile")" \ test ! -f "$profile/bin/sed" rm "$profile" "$profile"-[0-9]-link +# Make sure transformations apply to propagated inputs and don't lead to +# conflicts when installing them alongside, see bug +# +mkdir "$module_dir" +cat > "$module_dir/test.scm" <. -- 2.36.0 From unknown Mon Jun 23 20:20:33 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Josselin Poiret Subject: bug#55316: closed (Re: bug#55316: [PATCH] scripts: package: Transform before creating manifest entries.) Message-ID: References: <87tu9g75ky.fsf_-_@gnu.org> <20220508141606.14085-1-dev@jpoiret.xyz> X-Gnu-PR-Message: they-closed 55316 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 55316@debbugs.gnu.org Date: Mon, 23 May 2022 13:17:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1653311822-26945-1" This is a multi-part message in MIME format... ------------=_1653311822-26945-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #55316: [PATCH] scripts: package: Transform before creating manifest entrie= s. which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 55316@debbugs.gnu.org. --=20 55316: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D55316 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1653311822-26945-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 55316-done) by debbugs.gnu.org; 23 May 2022 13:16:06 +0000 Received: from localhost ([127.0.0.1]:47624 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nt7un-0006z4-RO for submit@debbugs.gnu.org; Mon, 23 May 2022 09:16:06 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33992) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nt7ui-0006yE-K0 for 55316-done@debbugs.gnu.org; Mon, 23 May 2022 09:16:04 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:33874) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nt7ud-0005IJ-2o; Mon, 23 May 2022 09:15:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=v/19uzld6gEGhcQp4F+7lzB9XHB2ng758f905o15WMI=; b=ZiFMqMywQy+MOn8uBgMP yXtAHfNsgnsK8k7jUnr6E+nNaYEYGtpHPTwFMu5Hv8StN/tntyTUd7nIK577TnQ3+AhXySGdqMuUJ SX+iLkppVGLen64gh1ga8YPFJ3lyXz6sy7IX/P6z1b0h0/QWmg5ccKApXw3VIk+s88CJ+CDiCf64m SQUy+OXd1TZxV1Qffyaqp2bP9pT1KrtBiKPMO6mHoSePt36xDX+1l+yVbF+CetGPzdjP2mh7JLUPG 2lG4hCQEfkLxYZrOIthrknD2te1DJ+gHlI4r09MVkmIrtgwKsfL8pQ33olTVoVYxgDInSu21PHcCn csNTkqO+ew9ouA==; Received: from [193.50.110.143] (port=52548 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nt7uR-0001m8-FM; Mon, 23 May 2022 09:15:49 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Josselin Poiret Subject: Re: bug#55316: [PATCH] scripts: package: Transform before creating manifest entries. References: <874k1zrq2p.fsf@gnu.org> <20220509145410.11678-1-dev@jpoiret.xyz> Date: Mon, 23 May 2022 15:15:41 +0200 In-Reply-To: <20220509145410.11678-1-dev@jpoiret.xyz> (Josselin Poiret's message of "Mon, 9 May 2022 16:54:10 +0200") Message-ID: <87tu9g75ky.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 55316-done Cc: 55316-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Josselin, Josselin Poiret skribis: > * guix/scripts/package.scm (options->installable): Add TRANSFORM > argument, to be able to directly transform the new packages before > creating their manifest entries. > (process-actions): Remove transform-entry, and step3, transforming > directly in step2. > * tests/guix-package.sh: Add test. Finally applied with the cosmetic change below, thank you! Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/tests/guix-package.sh b/tests/guix-package.sh index c2f5e39de0..dedba2fd74 100644 --- a/tests/guix-package.sh +++ b/tests/guix-package.sh @@ -212,8 +212,8 @@ test ! -f "$profile/bin/sed" rm "$profile" "$profile"-[0-9]-link # Make sure transformations apply to propagated inputs and don't lead to -# conflicts when installing them alongside, see bug -# +# conflicts when installing them alongside, see +# . mkdir "$module_dir" cat > "$module_dir/test.scm" <) id 1nnhiA-0004wl-O4 for submit@debbugs.gnu.org; Sun, 08 May 2022 10:16:39 -0400 Received: from lists.gnu.org ([209.51.188.17]:35582) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nnhi8-0004uh-ET for submit@debbugs.gnu.org; Sun, 08 May 2022 10:16:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nnhi8-0000G9-7G for guix-patches@gnu.org; Sun, 08 May 2022 10:16:36 -0400 Received: from jpoiret.xyz ([206.189.101.64]:38898) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nnhi6-0001kU-68 for guix-patches@gnu.org; Sun, 08 May 2022 10:16:35 -0400 Received: from authenticated-user (jpoiret.xyz [206.189.101.64]) by jpoiret.xyz (Postfix) with ESMTPA id C8898184D6A; Sun, 8 May 2022 14:16:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpoiret.xyz; s=dkim; t=1652019390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=A67dAy3wWxjOorffQG3HZsQVeQhnwWJewn4fF5gbDpc=; b=oswEZ+urVi5zakk/ilCsXngeDmPaEFXbiSqbHOXuNUr/wyixG5Q3B0WJp9Ze1MMuamOGfr Ev7PRM6H8jkj0Rp0rigw+WRQXJKRzFPbRkvzRkJGNNeQs/Af9JYp3CYtuBgRtbYd++t6UP pP6YKat48TAOcR092LUuUfYIby0XCYPMn8sWsp9bD96UWF9S89pWS1alcVXZEk8CqL4XFY WOjdPEZIE3Yw8DhaueOC36TFAP9ICrnSgtDjQw/68vYocLHNxLFcN82uZ5kFzuuMdo0Yi2 U2AkWnZDP0hSfSirM+GITfvR8yertTaHGsH9q/zlS3fRqlWUhwV2JLAiWqMwCw== From: Josselin Poiret To: guix-patches@gnu.org Subject: [PATCH] scripts: package: Transform before creating manifest entries. Date: Sun, 8 May 2022 16:16:06 +0200 Message-Id: <20220508141606.14085-1-dev@jpoiret.xyz> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Authentication-Results: jpoiret.xyz; auth=pass smtp.auth=jpoiret@jpoiret.xyz smtp.mailfrom=dev@jpoiret.xyz X-Spamd-Bar: / Received-SPF: pass client-ip=206.189.101.64; envelope-from=dev@jpoiret.xyz; helo=jpoiret.xyz X-Spam_score_int: 24 X-Spam_score: 2.4 X-Spam_bar: ++ X-Spam_report: (2.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_SUSPICIOUS_NTLD=0.499, FROM_SUSPICIOUS_NTLD_FP=1.997, PDS_OTHER_BAD_TLD=1.997, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 2.7 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/scripts/package.scm (options->installable): Add TRANSFORM argument, to be able to directly transform the new packages before creating their manifest entries. (process-actions): Remove transform [...] Content analysis details: (2.7 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.17 listed in list.dnswl.org] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.9 SPF_FAIL SPF: sender does not match SPF record (fail) [SPF failed: Please see http://www.openspf.org/Why?s=mfrom; id=dev%40jpoiret.xyz; ip=209.51.188.17; r=debbugs.gnu.org] 1.6 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: jpoiret.xyz (xyz)] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.51.188.17 listed in wl.mailspike.net] 2.0 FROM_SUSPICIOUS_NTLD_FP From abused NTLD 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: submit Cc: Josselin Poiret X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.3 (/) * guix/scripts/package.scm (options->installable): Add TRANSFORM argument, to be able to directly transform the new packages before creating their manifest entries. (process-actions): Remove transform-entry, and step3. --- Hello everyone, Someone on IRC [1] reported that they couldn't run guix install emacs-avy emacs-embark --with-branch=emacs-avy=master --with-branch=emacs-embark=master without guix complaining about conflicting entries of emacs-avy and the propagated one from emacs-embark, even in a profile without either of them. The issue was that package transformations were applied to the contents of the generated manifest entries, but weren't applied to any of their dependencies. To solve this, I figured it would be simpler to just apply the transformation from the start, only creating the manifest entries after they have been applied. Also, the default `package->manifest-entry` preserves transformations properties by default, so this doesn't lose any info either. With this patch applied, the above command works as expected for me. Best, Josselin Poiret [1] https://logs.guix.gnu.org/guix/2022-05-08.log#134728 guix/scripts/package.scm | 36 ++++++++++++------------------------ 1 file changed, 12 insertions(+), 24 deletions(-) diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm index d007005607..4673cf18b2 100644 --- a/guix/scripts/package.scm +++ b/guix/scripts/package.scm @@ -10,6 +10,7 @@ ;;; Copyright © 2020 Ricardo Wurmus ;;; Copyright © 2020 Simon Tournier ;;; Copyright © 2018 Steve Sprang +;;; Copyright © 2022 Josselin Poiret ;;; ;;; This file is part of GNU Guix. ;;; @@ -694,10 +695,10 @@ (define (package->manifest-entry* package output) (manifest-entry-with-provenance (package->manifest-entry package output))) -(define (options->installable opts manifest transaction) - "Given MANIFEST, the current manifest, and OPTS, the result of 'args-fold', -return an variant of TRANSACTION that accounts for the specified installations -and upgrades." +(define (options->installable opts manifest transform transaction) + "Given MANIFEST, the current manifest, OPTS, and TRANSFORM, the result of +'args-fold', return an variant of TRANSACTION that accounts for the specified +installations, upgrades and transformations." (define upgrade? (options->upgrade-predicate opts)) @@ -714,13 +715,14 @@ (define to-install (('install . (? package? p)) ;; When given a package via `-e', install the first of its ;; outputs (XXX). - (package->manifest-entry* p "out")) + (package->manifest-entry* (transform p) "out")) (('install . (? string? spec)) (if (store-path? spec) (store-item->manifest-entry spec) (let-values (((package output) (specification->package+output spec))) - (package->manifest-entry* package output)))) + (package->manifest-entry* (transform package) + output)))) (('install . obj) (leave (G_ "cannot install non-package object: ~s~%") obj)) @@ -979,16 +981,6 @@ (define allow-collisions? (assoc-ref opts 'allow-collisions?)) (define profile (or (assoc-ref opts 'profile) %current-profile)) (define transform (options->transformation opts)) - (define (transform-entry entry) - (let ((item (transform (manifest-entry-item entry)))) - (manifest-entry-with-transformations - (manifest-entry - (inherit entry) - (item item) - (version (if (package? item) - (package-version item) - (manifest-entry-version entry))))))) - (when (equal? profile %current-profile) ;; Normally the daemon created %CURRENT-PROFILE when we connected, unless ;; it's a version that lacks the fix for @@ -1021,16 +1013,12 @@ (define (transform-entry entry) (map load-manifest files)))))) (step1 (options->removable opts manifest (manifest-transaction))) - (step2 (options->installable opts manifest step1)) - (step3 (manifest-transaction - (inherit step2) - (install (map transform-entry - (manifest-transaction-install step2))))) - (new (manifest-perform-transaction manifest step3)) + (step2 (options->installable opts manifest transform step1)) + (new (manifest-perform-transaction manifest step2)) (trans (if (null? files) - step3 + step2 (fold manifest-transaction-install-entry - step3 + step2 (manifest-entries manifest))))) (warn-about-old-distro) -- 2.35.1 ------------=_1653311822-26945-1--