From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 15 01:51:52 2021 Received: (at submit) by debbugs.gnu.org; 15 Sep 2021 05:51:52 +0000 Received: from localhost ([127.0.0.1]:50581 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mQNpn-0000m0-QJ for submit@debbugs.gnu.org; Wed, 15 Sep 2021 01:51:52 -0400 Received: from lists.gnu.org ([209.51.188.17]:46250) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mQNpl-0000ls-H5 for submit@debbugs.gnu.org; Wed, 15 Sep 2021 01:51:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQNpl-0000Q3-8t for bug-guix@gnu.org; Wed, 15 Sep 2021 01:51:49 -0400 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]:33664) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mQNpa-0000wY-Rp for bug-guix@gnu.org; Wed, 15 Sep 2021 01:51:48 -0400 Received: by mail-wr1-x42d.google.com with SMTP id t18so1996466wrb.0 for ; Tue, 14 Sep 2021 22:51:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=5QT7DiKBHUgV/rwJu7/p4XVqbULr0QezsjCjHnhPKhw=; b=gIbPUpxAmhJtXbewlZpNyRvziPMHEPNac1q78pHR2K6DOHEu9CibOonwT0js+DQV3l XM9HKKPmY6vAWqM3Nx16HUVbKQjwNGWs6YjbCPxEe1+J8ujS2wYMIu2mob4KFVrh3sTs nZQDfnKZiC/cogw/R2xlt1TXTxGh1rg8qGnGTReMyBot1Q7pRBkL7RT0/TZxqHPUevE8 PVUGbYSK62VHrB0fPvnz41Cz58jbKlkNtPduG22Wk/Kbm2uepRji7Fm7OvN5k3KYmlIM mhjSGZpKmLhS6tsFEswqlepbyP2OsdujrWP0VkfuE9WBZEZ8RLTHLlX66il89cQRDuQZ AxZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=5QT7DiKBHUgV/rwJu7/p4XVqbULr0QezsjCjHnhPKhw=; b=LfODnec66+4SpnC7UqHOylhHz7dn9UvQP2o5USxdJSU0wvWf5GlfJ6hfcuU+K1q3UI gL7kI4KwWamSN8RnLPVPYtcy+HM1n9tqKtvCE33RkZDVm7r8ezUsLgtzGgs6+TU1YAhN 98dPWbgT2Pr1DTdVv4+eWiTPj05L6AzQqIL+rPYqSfFpcvEVbgI7R2Lvybu7nUXRAvMK QOKqJXJG96yr43HHH1m7VI3yGDqA+3yKnLPLmLWPrFcd/nuQAncHg6bzKT5IMtm1OCf4 9SCPrE9/0YMK25iJCwVFeluFZZCk3RGcVO4YG68tsX/41vDnupedVYG2Ku62S6yIq/2e lFIA== X-Gm-Message-State: AOAM532S3dcvqaHVaTFmG3OserMRcxHyTm06jr5lWDdm8R6hB+vr+wD/ xELwFKvpc+VJyz+sH+wOsaJRUMaRwpnzAw== X-Google-Smtp-Source: ABdhPJzBXz0E/Tz8GEHYdLyA5eHlBJWvklR8o6yBy/vDs2MdRpFajC8s1SiXd7QoLXiIgTJ5om4eHg== X-Received: by 2002:a5d:4608:: with SMTP id t8mr2887746wrq.136.1631685096563; Tue, 14 Sep 2021 22:51:36 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id z5sm3123546wmf.33.2021.09.14.22.51.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Sep 2021 22:51:36 -0700 (PDT) From: zimoun To: Bug Guix Subject: different fixed-output when using =?utf-8?Q?=E2=80=99--with-sourc?= =?utf-8?Q?e=E2=80=99?= transformation Date: Wed, 15 Sep 2021 07:44:36 +0200 Message-ID: <86zgsecrzv.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::42d; envelope-from=zimon.toutoune@gmail.com; helo=mail-wr1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_QUOTING=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Hi, If the package source tarball is provided to =E2=80=99--with-source=E2=80= =99, then the hash differs. I would expect a fixed point. Let investigate: --8<---------------cut here---------------start------------->8--- $ guix build hello@2.10 --with-source=3Dhello=3Dhttps://ftpmirror.gnu.org/g= nu/hello/hello-2.10.tar.gz /gnu/store/zfwhbbknkhxi3yqmp0qgh1l1crljgbm6-hello-2.10 $ guix build hello@2.10=20 /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 --8<---------------cut here---------------end--------------->8--- Where https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz is an official mirror. I expect the same store item. Why are they different? It is because the derivations are different. The first call contains the source tarball as an =E2=80=9Cinput=E2=80=9D; t= hat=E2=80=99s fine. The second call contains the source tarball as a derivation; again that=E2=80=99s fine. The corresponding first and second derivations are: --8<---------------cut here---------------start------------->8--- Derive ([("out","/gnu/store/zfwhbbknkhxi3yqmp0qgh1l1crljgbm6-hello-2.10","","")] ,[("/gnu/store/101ny738l311p4fm3cas7jgnkzagjv6a-module-import-compiled.drv= ",["out"]) [...] ,("/gnu/store/z4s27gwacbw8f38andfsh21r8v330dag-xz-5.2.4.drv",["out"])] ,["/gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import", "/gnu/store/chariqd6k0sli3s7vcl4q3al0crirz5v-hello-2.10.tar.gz", "/gnu/store/rndq9g8877l29ha41dvsl3aj1z0gw0ng-hello-2.10-guile-builder"] [...] --8<---------------cut here---------------end--------------->8--- and --8<---------------cut here---------------start------------->8--- Derive ([("out","/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10","","")] ,[("/gnu/store/101ny738l311p4fm3cas7jgnkzagjv6a-module-import-compiled.drv= ",["out"]) ,("/gnu/store/1a7xfcqcxj0pqi4f81x1agcxa46v2bbm-hello-2.10.tar.gz.drv",["= out"]) [...] ,("/gnu/store/z4s27gwacbw8f38andfsh21r8v330dag-xz-5.2.4.drv",["out"])] ,["/gnu/store/8a0wry8cvr405ha8d8bpjyzj5dzghigd-module-import", "/gnu/store/kql8b2hbsabcmany4m3hfm3wzdiymliy-hello-2.10-guile-builder"] [...] --8<---------------cut here---------------end--------------->8--- Therefore, I understand why the two store items have different hashes. Although, it should be better that the derivations would be the same; another story. :-) What puzzled me are the corresponding =E2=80=99guile-builder=E2=80=99s: --8<---------------cut here---------------start------------->8--- (define %build-inputs (quote (("source" . "/gnu/store/chariqd6k0sli3s7vcl4q3al0crirz5v-hello-2.10.t= ar.gz") --8<---------------cut here---------------end--------------->8--- and --8<---------------cut here---------------start------------->8--- (define %build-inputs (quote (("source" . "/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.t= ar.gz") --8<---------------cut here---------------end--------------->8--- but these 2 items have the same hash: --8<---------------cut here---------------start------------->8--- $ guix hash /gnu/store/chariqd6k0sli3s7vcl4q3al0crirz5v-hello-2.10.tar.gz 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i $ guix hash /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i --8<---------------cut here---------------end--------------->8--- Why? It is fixed-output so I was expecting the same thing. And the same as: --8<---------------cut here---------------start------------->8--- $ guix download https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz Starting download of /tmp/guix-file.qkhxkp >From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz... following redirection to `https://mirrors.sarata.com/gnu/hello/hello-2.10.t= ar.gz'... =E2=80=A610.tar.gz 709KiB 549Ki= B/s 00:01 [##################] 100.0% /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i --8<---------------cut here---------------end--------------->8--- Why the store item of source tarball is it different when applying the transformation? Maxime provided a clue [1]. All the best, simon 1: From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 18 12:06:51 2021 Received: (at 50597) by debbugs.gnu.org; 18 Sep 2021 16:06:51 +0000 Received: from localhost ([127.0.0.1]:36490 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mRcrb-00043J-Iv for submit@debbugs.gnu.org; Sat, 18 Sep 2021 12:06:51 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47404) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mRcrW-000432-Lu for 50597@debbugs.gnu.org; Sat, 18 Sep 2021 12:06:50 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56266) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mRcrR-0006PI-BR; Sat, 18 Sep 2021 12:06:41 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=35810 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mRcrR-00089w-2E; Sat, 18 Sep 2021 12:06:41 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zimoun Subject: Re: bug#50597: different fixed-output when using =?utf-8?Q?=E2=80=99--with-source=E2=80=99?= transformation References: <86zgsecrzv.fsf@gmail.com> Date: Sat, 18 Sep 2021 18:06:38 +0200 In-Reply-To: <86zgsecrzv.fsf@gmail.com> (zimoun's message of "Wed, 15 Sep 2021 07:44:36 +0200") Message-ID: <87fsu1uaup.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50597 Cc: 50597@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello! zimoun skribis: > Why the store item of source tarball is it different when applying the > transformation? > > Maxime provided a clue [1]. Maxime is right: =E2=80=98--with-source=E2=80=99 passes #:recursive? #t unc= onditionally, whereas it=E2=80=99s usually #f for a =E2=80=98url-fetch=E2=80=99 kind of U= RI. What we could do is make it #t if and only if the URL denotes a local file and that file is a directory. I think that would achieve what you want without any observable regression in practice. WDYT? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 18 13:35:31 2021 Received: (at 50597) by debbugs.gnu.org; 18 Sep 2021 17:35:31 +0000 Received: from localhost ([127.0.0.1]:36794 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mReFP-0000Jl-Jl for submit@debbugs.gnu.org; Sat, 18 Sep 2021 13:35:31 -0400 Received: from baptiste.telenet-ops.be ([195.130.132.51]:37086) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mReFN-0000Jb-2T for 50597@debbugs.gnu.org; Sat, 18 Sep 2021 13:35:30 -0400 Received: from ptr-bvsjgyjmffd7q9timvx.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:aaf1:9810:a0b8:a55d]) by baptiste.telenet-ops.be with bizsmtp id vVbT250020mfAB401VbToR; Sat, 18 Sep 2021 19:35:27 +0200 Message-ID: Subject: Re: bug#50597: different fixed-output when using =?UTF-8?Q?=E2=80=99--with-source=E2=80=99?= transformation From: Maxime Devos To: Ludovic =?ISO-8859-1?Q?Court=E8s?= , zimoun Date: Sat, 18 Sep 2021 19:35:12 +0200 In-Reply-To: <87fsu1uaup.fsf@gnu.org> References: <86zgsecrzv.fsf@gmail.com> <87fsu1uaup.fsf@gnu.org> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-I69KWV7MdjqfMp8DB29z" User-Agent: Evolution 3.34.2 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r21; t=1631986527; bh=JDz6OXXfjEqeGcPrh9jsewfxcvaUu8zNxubCI9BcNz0=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=OSoulndPDcGE69vrdnHzEY8zQBV6BCeY1uY2y8Q9CmGv3h2XJZhggJcmDaQgqeePu AEw95W4qtcwHPDbRnqtBZf/Wzs2onD8UDfubKJlCWYnbEiVfC43njpZ0XVV3s93+Iw VK709eJMvfxsw2oGZePNYdUWHDqL0xC9m1TCrNmhvL33fSJGN+hOFYITQpFfcGyIgk u2WGfagixC22zRzH40tzsTkpFUexBpq4DQBucbEdSTkCz8xrMD3BpP6c9Y86GUMxak 6vRUWb2N671W3a6c9w4Q+GSdMRGBOg+XROaKT9PJh99VDvPim/00wyVFnk2L+OxTWv YsCQ2y7r7q26Q== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 50597 Cc: 50597@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-I69KWV7MdjqfMp8DB29z Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s schreef op za 18-09-2021 om 18:06 [+0200]: > Hello! >=20 > zimoun skribis: >=20 > > Why the store item of source tarball is it different when applying the > > transformation? > >=20 > > Maxime provided a clue [1]. >=20 > Maxime is right: =E2=80=98--with-source=E2=80=99 passes #:recursive? #t u= nconditionally, > whereas it=E2=80=99s usually #f for a =E2=80=98url-fetch=E2=80=99 kind of= URI. >=20 > What we could do is make it #t if and only if the URL denotes a local > file and that file is a directory. I think that would achieve what you > want without any observable regression in practice. > > WDYT? This seems reasonable to me. Greetings, Maxime. --=-I69KWV7MdjqfMp8DB29z Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYUYjURccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7oHOAQCg5aZN+D+ze0P5pr7U5H1WZM52 m2AS+jL24v5H/nSX4QD/fIPYjX4kXQ/IcB3OzPjeNFKnfCrqIDS57zgVDlKdBAA= =OtFz -----END PGP SIGNATURE----- --=-I69KWV7MdjqfMp8DB29z-- From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 21 03:47:20 2021 Received: (at 50597) by debbugs.gnu.org; 21 Sep 2021 07:47:20 +0000 Received: from localhost ([127.0.0.1]:44767 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mSaUq-0005dt-2y for submit@debbugs.gnu.org; Tue, 21 Sep 2021 03:47:20 -0400 Received: from mail-wr1-f53.google.com ([209.85.221.53]:44705) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mSaUn-0005dY-Ly for 50597@debbugs.gnu.org; Tue, 21 Sep 2021 03:47:18 -0400 Received: by mail-wr1-f53.google.com with SMTP id d6so36273980wrc.11 for <50597@debbugs.gnu.org>; Tue, 21 Sep 2021 00:47:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=6EpbMIa9enDN2KRaY2ADz3qzqi0vI1CFDvWdXWaFe5A=; b=jItn4/mqd7TCP+Svo9XlygwjzcxItLt9/JvSWBuJwpSsGODcHwVdMMrv5MzYo9nWFP zhYYWTG/5g8+p1gzdcmDdnMDa2JBQRd/xvE9Obr5p72gHYVg9CdYgBBm2w3Sfov+vHiF NwtfACJuE4Obuiwa3WJMN7uQaih2nn0KS61RRohWnlHXEBI2GZ7lEowBCXaHeT6vQ5oA 273GyfpUJQO7WdXWGskMQJakYObR26ulXyLM2XjaPnHxtxjn3QSAZ1Cf5UpQj76zuKbS 295BRII8WFCIc6X8iJraQBI6FXKAFRmsM3BC/K8sNmRblSORByMHEpubyTjrMkOEKiQT pkUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=6EpbMIa9enDN2KRaY2ADz3qzqi0vI1CFDvWdXWaFe5A=; b=3biJM8kNxVJODV4OAtGmzMWiCOknjio5ee8RyyXA2Oo8xc2E/paIOv058vzNCUpWEz aH0bOISZ6r3w/tlYjA9/n/3ciUUPWSjMHD+neViFXkrle1QwH3WeePEj/WQpBIbR+Mwq 7Tco+Z3iYp9YDl18IlppFkQ3uHKwmQKYrCaqsVbkmnMoZ1epW6LkIznll4oBk0Lu+oHV kknMtWyZxjO//dzsYZJbYcxkSeUtVN0+s7jMiSfpOPMdihqx48TmvxfMufAreyWzgSra Em6bPGfEXZrBwvBE5DEOVltwpFL82yzJtMDyeCD+DQ+oMzOm2w/eqiCEkdZ4pXeW2Rg0 2gBw== X-Gm-Message-State: AOAM530SQbiGmf6Ob66WFku+30ObNt8FRnvHBGFf5VKAyWS84VtB2nKO FVcqsRRgD7T1xywXlo7aIFkpRqteEVXubA== X-Google-Smtp-Source: ABdhPJwsoILTnZvEQXFC+/6A3Dp/AABfPoeo0IxNRtHt+Bn/l631RqCg+pvwlOs4ZoAs9gdLgUpcXQ== X-Received: by 2002:a7b:c8da:: with SMTP id f26mr2965583wml.109.1632210431458; Tue, 21 Sep 2021 00:47:11 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id m18sm18212314wrn.85.2021.09.21.00.47.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Sep 2021 00:47:11 -0700 (PDT) From: zimoun To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#50597: different fixed-output when using =?utf-8?Q?=E2=80=99--with-source=E2=80=99?= transformation In-Reply-To: <87fsu1uaup.fsf@gnu.org> References: <86zgsecrzv.fsf@gmail.com> <87fsu1uaup.fsf@gnu.org> Date: Tue, 21 Sep 2021 09:40:36 +0200 Message-ID: <86czp2cr63.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 50597 Cc: 50597@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludo, On Sat, 18 Sep 2021 at 18:06, Ludovic Court=C3=A8s wrote: > zimoun skribis: > >> Why the store item of source tarball is it different when applying the >> transformation? >> >> Maxime provided a clue [1]. > > Maxime is right: =E2=80=98--with-source=E2=80=99 passes #:recursive? #t u= nconditionally, > whereas it=E2=80=99s usually #f for a =E2=80=98url-fetch=E2=80=99 kind of= URI. Thanks for confirming. Why is it different? > What we could do is make it #t if and only if the URL denotes a local > file and that file is a directory. I think that would achieve what you > want without any observable regression in practice. What I expect is that 2 (fixed-output) files with the same hash (0ssi1w=E2=80=A6) lives at the same store location=E2=80=A6 --8<---------------cut here---------------start------------->8--- $ guix hash /gnu/store/chariqd6k0sli3s7vcl4q3al0crirz5v-hello-2.10.tar.gz 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i $ guix hash /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i --8<---------------cut here---------------end--------------->8--- =E2=80=A6whatever if these (fixed-output) files come from =E2=80=9Cguix dow= nload=E2=80=9D, =E2=80=9Cguix build -S=E2=80=9D or =E2=80=9C--with-source=3Dhttps://=E2=80=9D or etc. The mapping from hash (0ssi1w=E2=80=A6) to store location should be 1:1 for fixed-output, IMHO. Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 22 06:26:12 2021 Received: (at 50597) by debbugs.gnu.org; 22 Sep 2021 10:26:12 +0000 Received: from localhost ([127.0.0.1]:48924 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mSzS8-0007pc-80 for submit@debbugs.gnu.org; Wed, 22 Sep 2021 06:26:12 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58088) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mSzS4-0007pL-H7 for 50597@debbugs.gnu.org; Wed, 22 Sep 2021 06:26:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58834) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mSzRo-0007wN-7w; Wed, 22 Sep 2021 06:25:56 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=35966 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mSzRI-0007J6-Vr; Wed, 22 Sep 2021 06:25:45 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: zimoun Subject: Re: bug#50597: different fixed-output when using =?utf-8?Q?=E2=80=99--with-source=E2=80=99?= transformation References: <86zgsecrzv.fsf@gmail.com> <87fsu1uaup.fsf@gnu.org> <86czp2cr63.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 1 =?utf-8?Q?Vend=C3=A9miaire?= an 230 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 22 Sep 2021 12:25:12 +0200 In-Reply-To: <86czp2cr63.fsf@gmail.com> (zimoun's message of "Tue, 21 Sep 2021 09:40:36 +0200") Message-ID: <87bl4kj4af.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50597 Cc: 50597@debbugs.gnu.org, Maxime Devos X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, zimoun skribis: > On Sat, 18 Sep 2021 at 18:06, Ludovic Court=C3=A8s wrote: >> zimoun skribis: >> >>> Why the store item of source tarball is it different when applying the >>> transformation? >>> >>> Maxime provided a clue [1]. >> >> Maxime is right: =E2=80=98--with-source=E2=80=99 passes #:recursive? #t = unconditionally, >> whereas it=E2=80=99s usually #f for a =E2=80=98url-fetch=E2=80=99 kind o= f URI. > > Thanks for confirming. Why is it different? Because we want =E2=80=98--with-source=E2=80=99 to also work with directori= es (see comment in =E2=80=98package-with-source=E2=80=99). >> What we could do is make it #t if and only if the URL denotes a local >> file and that file is a directory. I think that would achieve what you >> want without any observable regression in practice. > > What I expect is that 2 (fixed-output) files with the same hash > (0ssi1w=E2=80=A6) lives at the same store location=E2=80=A6 > > $ guix hash /gnu/store/chariqd6k0sli3s7vcl4q3al0crirz5v-hello-2.10.tar.gz > 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i > > $ guix hash /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz > 0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i The difference here is that one is marked as =E2=80=9Crecursive=E2=80=9D an= d the other is not. The patch below does one I suggested earlier: passing #:recursive? #t if and only if we know the source is a directory. However it=E2=80=99s not enough to get the same derivation: we=E2=80=99d al= so need to return a fixed-output derivation rather than a plain store file name. (See also .) To be continued=E2=80=A6 Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/guix/transformations.scm b/guix/transformations.scm index 5ae1977cb2..e0f5579c28 100644 --- a/guix/transformations.scm +++ b/guix/transformations.scm @@ -46,6 +46,7 @@ #:use-module (srfi srfi-37) #:use-module (ice-9 match) #:use-module (ice-9 vlist) + #:autoload (web uri) (string->uri uri-scheme uri-path) #:export (options->transformation manifest-entry-with-transformations @@ -106,15 +107,25 @@ extensions." (define* (package-with-source p uri #:optional version) "Return a package based on P but with its source taken from URI. Extract the new package's version number from URI." - (let ((base (tarball-base-name (basename uri)))) + (let ((base (tarball-base-name (basename uri))) + (file (match (string->uri uri) + (#f uri) + (uri (and (eq? 'file (uri-scheme uri)) + (uri-path uri)))))) (let-values (((_ version*) (hyphen-package-name->name+version base))) (package (inherit p) (version (or version version* (package-version p))) - ;; Use #:recursive? #t to allow for directories. - (source (downloaded-file uri #t)))))) + ;; Default to #:recursive? #f to match what 'url-fetch' does, + ;; but use #t when URI denotes a directory. + (source (let ((recursive? + (and file + (match (stat file #f) + (#f #t) + (st (eq? 'directory (stat:type st))))))) + (downloaded-file uri recursive?))))))) ;;; --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 13 05:38:00 2021 Received: (at 50597) by debbugs.gnu.org; 13 Oct 2021 09:38:00 +0000 Received: from localhost ([127.0.0.1]:58382 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maai0-0007K8-G1 for submit@debbugs.gnu.org; Wed, 13 Oct 2021 05:38:00 -0400 Received: from mail-wr1-f50.google.com ([209.85.221.50]:35381) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maahx-0007Jv-Oq for 50597@debbugs.gnu.org; Wed, 13 Oct 2021 05:37:59 -0400 Received: by mail-wr1-f50.google.com with SMTP id g25so6233802wrb.2 for <50597@debbugs.gnu.org>; Wed, 13 Oct 2021 02:37:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=R/aAIFPNIchAmj1SNKKgmf6Zp/fa36nWqWyGBiIRQGw=; b=Tt2xS3BguSHF5jwuBIaJ81d/UbrbliK6YawvYUkxVg2467rEWg9Q70mnmYwrP8wi/M 6BoU/1JA/bc3mFbvQqx+JW/pYMK3lqpUNHZ3Ehr7EaVJudnNnXgwtga8FYoeCrhp0RV6 yr5F/CQfuqBYTmCffQs0pkEAAzheoWM3Ys2h9lKTugSFRdziKDcpEozonJWm9bp+xgy4 XMo8zkxOtjPe9OMJD1a5Kygyc/8Cf6qppkv1YCgx8vX8O8EMbEy4mx9o+I7+RzAPCXVj qbHnwq/7wjA8NNDzk5zZnJel4vLDj99c5zDKPWCNcn3ekMOfaHqeyUNwPuLIhNpYRodF q6dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=R/aAIFPNIchAmj1SNKKgmf6Zp/fa36nWqWyGBiIRQGw=; b=wKjAdbVyU86ITiQylbb0HN4tRFcj27c0zLeF4EVfS1iwBY0HuPsQmvvK8Dh/3/B7bt N3Ekfi9gRHdxkIlle+nkJrqOh35onWftjT7yrwZ37+3lp1V4Uxm6gLrU8iW31HhtT2k1 qwggQqPI0RY9fFP0zZaNPieTXyYMvTPumfMIrti2ukiRUK7Lq6nECicUi6f3EP/ctOY0 A5sYgtwbzjO3S+ZOlsAnFJHiry1w2NdMjR5dwn838MVgCJgAISOGDBwJfJRYNjHM0bLK SO4cQqi2UqS5tqr/nQbdkcajhtrVeHCS5dpNmYu5XnyjQpuW88MORy6W0jaOynv49g/1 qASw== X-Gm-Message-State: AOAM530wunrUOjcEKe4JGgWP4kvFpfw9nISLpUt5tD795PTOqk5DgtIx silniB5dIwvmh+KGopsayJE= X-Google-Smtp-Source: ABdhPJzd6qIs+5RaP7NtGHV1OUwVwYla3pJLqc2haAbc9H5URxd4OgWNvynQ9M9Ocxyvnc/ktIOxSA== X-Received: by 2002:adf:a15c:: with SMTP id r28mr38862021wrr.287.1634117871783; Wed, 13 Oct 2021 02:37:51 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id l6sm5214397wmg.10.2021.10.13.02.37.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Oct 2021 02:37:51 -0700 (PDT) From: zimoun To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#50597: different fixed-output when using =?utf-8?Q?=E2=80=99--with-source=E2=80=99?= transformation In-Reply-To: <87bl4kj4af.fsf@gnu.org> References: <86zgsecrzv.fsf@gmail.com> <87fsu1uaup.fsf@gnu.org> <86czp2cr63.fsf@gmail.com> <87bl4kj4af.fsf@gnu.org> Date: Wed, 13 Oct 2021 11:31:37 +0200 Message-ID: <86wnmhi83q.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 50597 Cc: 50597@debbugs.gnu.org, Maxime Devos X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Wed, 22 Sep 2021 at 12:25, Ludovic Court=C3=A8s wrote: > However it=E2=80=99s not enough to get the same derivation: we=E2=80=99d = also need > to return a fixed-output derivation rather than a > plain store file name. (See also > .) Thanks for the explanations and the pointer. Indeed, to be continued=E2=80= =A6 > diff --git a/guix/transformations.scm b/guix/transformations.scm > index 5ae1977cb2..e0f5579c28 100644 > --- a/guix/transformations.scm > +++ b/guix/transformations.scm > @@ -46,6 +46,7 @@ > #:use-module (srfi srfi-37) > #:use-module (ice-9 match) > #:use-module (ice-9 vlist) > + #:autoload (web uri) (string->uri uri-scheme uri-path) > #:export (options->transformation > manifest-entry-with-transformations >=20=20 > @@ -106,15 +107,25 @@ extensions." > (define* (package-with-source p uri #:optional version) > "Return a package based on P but with its source taken from URI. Extr= act > the new package's version number from URI." > - (let ((base (tarball-base-name (basename uri)))) > + (let ((base (tarball-base-name (basename uri))) > + (file (match (string->uri uri) > + (#f uri) > + (uri (and (eq? 'file (uri-scheme uri)) > + (uri-path uri)))))) > (let-values (((_ version*) > (hyphen-package-name->name+version base))) > (package (inherit p) > (version (or version version* > (package-version p))) >=20=20 > - ;; Use #:recursive? #t to allow for directories. > - (source (downloaded-file uri #t)))))) > + ;; Default to #:recursive? #f to match what 'url-fetch' d= oes, > + ;; but use #t when URI denotes a directory. > + (source (let ((recursive? > + (and file > + (match (stat file #f) > + (#f #t) > + (st (eq? 'directory (stat:type st))= ))))) > + (downloaded-file uri recursive?))))))) LGTM! Cheers, simon