From unknown Sun Jun 15 08:59:20 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#32942 <32942@debbugs.gnu.org> To: bug#32942 <32942@debbugs.gnu.org> Subject: Status: 'guix offload' fails to restore non-ASCII file names when running in C locale Reply-To: bug#32942 <32942@debbugs.gnu.org> Date: Sun, 15 Jun 2025 15:59:20 +0000 retitle 32942 'guix offload' fails to restore non-ASCII file names when run= ning in C locale reassign 32942 guix submitter 32942 Julien Lepiller severity 32942 important thanks From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 05 04:18:37 2018 Received: (at submit) by debbugs.gnu.org; 5 Oct 2018 08:18:37 +0000 Received: from localhost ([127.0.0.1]:37374 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g8LJj-0002yV-Sx for submit@debbugs.gnu.org; Fri, 05 Oct 2018 04:18:37 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51801) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g8LJi-0002yG-37 for submit@debbugs.gnu.org; Fri, 05 Oct 2018 04:18:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g8LJV-0002At-TJ for submit@debbugs.gnu.org; Fri, 05 Oct 2018 04:18:27 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:37872) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g8LJQ-0001m0-QS for submit@debbugs.gnu.org; Fri, 05 Oct 2018 04:18:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33106) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g8LJP-0007hB-S2 for bug-guix@gnu.org; Fri, 05 Oct 2018 04:18:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g8LJH-00017z-W7 for bug-guix@gnu.org; Fri, 05 Oct 2018 04:18:14 -0400 Received: from lepiller.eu ([2a00:5884:8208::1]:45394) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g8LJH-0008Of-Hy for bug-guix@gnu.org; Fri, 05 Oct 2018 04:18:07 -0400 Received: from webmail.lepiller.eu (static-176-182-42-79.ncc.abo.bbox.fr [176.182.42.79]) by lepiller.eu (OpenSMTPD) with ESMTPSA id 37a73608 (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO) for ; Fri, 5 Oct 2018 08:19:43 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 05 Oct 2018 10:17:45 +0200 From: Julien Lepiller To: bug-guix@gnu.org Subject: nss-certs not deterministic Message-ID: <3974e5005881951012bb5e55a5bfabe2@lepiller.eu> X-Sender: julien@lepiller.eu User-Agent: Roundcube Webmail/1.3.6 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) While updating a profile, I found that nss-certs was not deterministic. From ludo: $ wget -O - -q https://mirror.hydra.gnu.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo | grep Hash NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla $ wget -O - -q https://berlin.guixsd.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo | grep Hash NarHash: sha256:08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 05 06:48:21 2018 Received: (at control) by debbugs.gnu.org; 5 Dec 2018 11:48:21 +0000 Received: from localhost ([127.0.0.1]:33171 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gUVfA-0002fH-Vf for submit@debbugs.gnu.org; Wed, 05 Dec 2018 06:48:21 -0500 Received: from eggs.gnu.org ([208.118.235.92]:55646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gUVf7-0002ev-QD for control@debbugs.gnu.org; Wed, 05 Dec 2018 06:48:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gUVf1-0004Kl-Uo for control@debbugs.gnu.org; Wed, 05 Dec 2018 06:48:12 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:51912) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gUVf1-0004KT-Ok for control@debbugs.gnu.org; Wed, 05 Dec 2018 06:48:11 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38650 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gUVf1-0001Dt-EN for control@debbugs.gnu.org; Wed, 05 Dec 2018 06:48:11 -0500 Date: Wed, 05 Dec 2018 12:48:10 +0100 Message-Id: <87lg54yz2d.fsf@gnu.org> To: control@debbugs.gnu.org From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: control message for bug #32942 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) severity 32942 important From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 05 09:01:43 2018 Received: (at 32942) by debbugs.gnu.org; 5 Dec 2018 14:01:43 +0000 Received: from localhost ([127.0.0.1]:33251 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gUXkC-0001RS-VQ for submit@debbugs.gnu.org; Wed, 05 Dec 2018 09:01:43 -0500 Received: from eggs.gnu.org ([208.118.235.92]:48725) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gUXkA-0001R8-5r for 32942@debbugs.gnu.org; Wed, 05 Dec 2018 09:01:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gUXk0-0006y0-0E for 32942@debbugs.gnu.org; Wed, 05 Dec 2018 09:01:33 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54403) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gUXjz-0006xg-RZ; Wed, 05 Dec 2018 09:01:27 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40576 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gUXjz-0002Dz-Iv; Wed, 05 Dec 2018 09:01:27 -0500 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Julien Lepiller Subject: Re: bug#32942: nss-certs not deterministic References: <3974e5005881951012bb5e55a5bfabe2@lepiller.eu> Date: Wed, 05 Dec 2018 15:01:25 +0100 In-Reply-To: <3974e5005881951012bb5e55a5bfabe2@lepiller.eu> (Julien Lepiller's message of "Fri, 05 Oct 2018 10:17:45 +0200") Message-ID: <87woooxebu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 32942 Cc: 32942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello, Julien Lepiller skribis: > While updating a profile, I found that nss-certs was not > deterministic. From ludo: > > $ wget -O - -q=20 > https://mirror.hydra.gnu.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo| gr= ep Hash > NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla > $ wget -O - -q=20 > https://berlin.guixsd.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo | > grep Hash > NarHash: sha256:08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s As shown above, berlin and hydra disagree on nss-certs. The difference is an encoding bug: --8<---------------cut here---------------start------------->8--- $ wget -O - https://berlin.guixsd.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8= ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.berlin $ wget -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqb= ls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.hydra $ diff -ru /tmp/nss-certs.{hydra,berlin} Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Ra=C3=ADz_Certic=C3=A1mara_S= .A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem Only in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.= 7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem Only in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=3DClass_Gold=3D_= F=C5=91tan=C3=BAs=C3=ADtv=C3=A1ny:2.6.73.65.44.228.0.16.pem Only in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=3DClass_Gold=3D= _F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem --8<---------------cut here---------------end--------------->8--- The problem was already reported as and since commit 412701b0e5e073e6767eed162c14698db99df69c (July 2017) =E2=80=98= guix publish=E2=80=99 on GuixSD runs in a UTF-8 locale to avoid that problem. The faulty narinfo/nar on berlin were generated on Oct. 17, 2018, so clearly the above commit was in effect. Indeed, after removing them and regenerating them, I=E2=80=99m still getting 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s (aka. the wrong hash). On closer inspection the problem is elsewhere: the /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 directory on berlin has question marks in file names, so =E2=80=98guix publish=E2=80=99 = is not to blame; instead the problem likely comes from =E2=80=98guix offload=E2=80=99. Indeed =E2=80=98guix-daemon=E2=80=99 and its child processes such as =E2=80= =98guix offload=E2=80=99 run with an empty environment, and thus in the C locale. Specifically, =E2=80=98restore-file-set=E2=80=99 on the build farm front-end must be the = one substituting question marks to the non-ASCII characters. If this analysis is correct, the patch below should fix it. I=E2=80=99ll t= ry it later. Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/gnu/services/base.scm b/gnu/services/base.scm index cee9898d79..9fe64e8087 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -1603,7 +1603,15 @@ failed to register public key '~a': ~a~%" key status)))))))) '()) #$@(if tmpdir (list (string-append "TMPDIR=" tmpdir)) - '())) + '()) + + ;; Make sure we run in a UTF-8 locale so that 'guix + ;; offload' correctly restores nars that contain UTF-8 + ;; file names such as 'nss-certs'. See + ;; . + (string-append "GUIX_LOCPATH=" + #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") #:log-file #$log-file)) (stop #~(make-kill-destructor)))))) --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 05 09:02:37 2018 Received: (at control) by debbugs.gnu.org; 5 Dec 2018 14:02:37 +0000 Received: from localhost ([127.0.0.1]:33258 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gUXl6-0001TG-U4 for submit@debbugs.gnu.org; Wed, 05 Dec 2018 09:02:37 -0500 Received: from eggs.gnu.org ([208.118.235.92]:48999) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gUXl5-0001T4-Mx for control@debbugs.gnu.org; Wed, 05 Dec 2018 09:02:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gUXkv-0007cZ-9Z for control@debbugs.gnu.org; Wed, 05 Dec 2018 09:02:30 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54417) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gUXkv-0007cT-5s for control@debbugs.gnu.org; Wed, 05 Dec 2018 09:02:25 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40582 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gUXku-000753-Ro for control@debbugs.gnu.org; Wed, 05 Dec 2018 09:02:25 -0500 Date: Wed, 05 Dec 2018 15:02:23 +0100 Message-Id: <87va48xea8.fsf@gnu.org> To: control@debbugs.gnu.org From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: control message for bug #32942 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) retitle 32942 'guix offload' fails to restore non-ASCII file names when running in C locale From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 09 18:29:46 2018 Received: (at 32942-done) by debbugs.gnu.org; 9 Dec 2018 23:29:46 +0000 Received: from localhost ([127.0.0.1]:40379 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gW8WA-0000t7-Dk for submit@debbugs.gnu.org; Sun, 09 Dec 2018 18:29:46 -0500 Received: from eggs.gnu.org ([208.118.235.92]:47759) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gW8W9-0000sw-Cc for 32942-done@debbugs.gnu.org; Sun, 09 Dec 2018 18:29:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gW8W1-0005D0-2w for 32942-done@debbugs.gnu.org; Sun, 09 Dec 2018 18:29:40 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50101) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gW8W0-0005Cv-VE; Sun, 09 Dec 2018 18:29:37 -0500 Received: from laubervilliers-656-1-271-96.w193-248.abo.wanadoo.fr ([193.248.46.96]:45966 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gW8W0-0003SE-JR; Sun, 09 Dec 2018 18:29:36 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Julien Lepiller Subject: Re: bug#32942: nss-certs not deterministic References: <3974e5005881951012bb5e55a5bfabe2@lepiller.eu> <87woooxebu.fsf@gnu.org> Date: Mon, 10 Dec 2018 00:29:28 +0100 In-Reply-To: <87woooxebu.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Wed, 05 Dec 2018 15:01:25 +0100") Message-ID: <87y38yuvmv.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 32942-done Cc: 32942-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hello, ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > The difference is an encoding bug: > > $ wget -O - https://berlin.guixsd.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbl= s8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.berlin > $ wget -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gy= qbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.hydra > $ diff -ru /tmp/nss-certs.{hydra,berlin} > Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Ra=C3=ADz_Certic=C3=A1mara= _S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem > Only in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.1= 5.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem > Only in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=3DClass_Gold= =3D_F=C5=91tan=C3=BAs=C3=ADtv=C3=A1ny:2.6.73.65.44.228.0.16.pem > Only in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=3DClass_Gold= =3D_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem [...] > On closer inspection the problem is elsewhere: the > /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 directory on > berlin has question marks in file names, so =E2=80=98guix publish=E2=80= =99 is not to > blame; instead the problem likely comes from =E2=80=98guix offload=E2=80= =99. > > Indeed =E2=80=98guix-daemon=E2=80=99 and its child processes such as =E2= =80=98guix offload=E2=80=99 run > with an empty environment, and thus in the C locale. Specifically, > =E2=80=98restore-file-set=E2=80=99 on the build farm front-end must be th= e one > substituting question marks to the non-ASCII characters. > > If this analysis is correct, the patch below should fix it. I=E2=80=99ll= try it > later. Pushed as 7e4bc215098f334bc2a11737f2665dd4992fc2da. Thanks, Ludo'. From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 19 12:35:52 2018 Received: (at 32942) by debbugs.gnu.org; 19 Dec 2018 17:35:52 +0000 Received: from localhost ([127.0.0.1]:55052 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gZflA-0006c2-C7 for submit@debbugs.gnu.org; Wed, 19 Dec 2018 12:35:52 -0500 Received: from mx1.riseup.net ([198.252.153.129]:35625) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gZfl8-0006bu-Ou for 32942@debbugs.gnu.org; Wed, 19 Dec 2018 12:35:51 -0500 Received: from piha.riseup.net (piha-pn.riseup.net [10.0.1.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.riseup.net (Postfix) with ESMTPS id E9DFB1A243B; Wed, 19 Dec 2018 09:35:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1545240950; bh=uNMMttr+Oh/WeOmvD9awTGKLJAa/beYH674zhjdcExE=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=sy4CKe57e3sNS0hZShG2osi0Bp36lLJjCDMrNMbFzBF3S8oHppADLJrR+7MeUOh64 06jnSLfdNUatmOZtqnCUt+ARCwP8LgU53xnAvTvsCRmj5BvVkjUfQSBcHV5tWSOp26 5F5/GcY0hR9eabYwQoPWKjZSt1ztNnKcmOAmitrk= X-Riseup-User-ID: DC5AC0AF614DC81E8122E5A3223E471E9C24A0FED3E5EF032A6A0A3B1A4B8E5D Received: from [127.0.0.1] (localhost [127.0.0.1]) by piha.riseup.net with ESMTPSA id C6D886BDB0; Wed, 19 Dec 2018 09:35:48 -0800 (PST) Subject: Re: bug#32942: nss-certs not deterministic To: =?UTF-8?Q?Ludovic_Court=c3=a8s?= , Julien Lepiller References: <3974e5005881951012bb5e55a5bfabe2@lepiller.eu> <87woooxebu.fsf@gnu.org> From: swedebugia Message-ID: <0fc840ed-c560-79ea-c765-7ab66af17d97@riseup.net> Date: Wed, 19 Dec 2018 18:42:19 +0100 MIME-Version: 1.0 In-Reply-To: <87woooxebu.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 32942 Cc: 32942@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On 2018-12-05 15:01, Ludovic Courtès wrote: > Hello, > > Julien Lepiller skribis: > >> While updating a profile, I found that nss-certs was not >> deterministic. From ludo: >> >> $ wget -O - -q >> https://mirror.hydra.gnu.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo| grep Hash >> NarHash: sha256:101v69xp1qzw9v6pgmbhw7gfdaic8vvs4v5l567lx7f2mjp25rla >> $ wget -O - -q >> https://berlin.guixsd.org/mbs5mavs3gi4y7xkywcwwjj9g3p1yjmv.narinfo | >> grep Hash >> NarHash: sha256:08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s > > As shown above, berlin and hydra disagree on nss-certs. > > The difference is an encoding bug: > > --8<---------------cut here---------------start------------->8--- > $ wget -O - https://berlin.guixsd.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.berlin > $ wget -O - https://mirror.hydra.gnu.org/nar/gzip/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 |gunzip -c |guix archive -x /tmp/nss-certs.hydra > $ diff -ru /tmp/nss-certs.{hydra,berlin} > Only in /tmp/nss-certs.hydra/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem > Only in /tmp/nss-certs.berlin/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem > Only in /tmp/nss-certs.hydra/etc/ssl/certs: NetLock_Arany_=Class_Gold=_Főtanúsítvány:2.6.73.65.44.228.0.16.pem > Only in /tmp/nss-certs.berlin/etc/ssl/certs: NetLock_Arany_=Class_Gold=_F?tan?s?tv?ny:2.6.73.65.44.228.0.16.pem > --8<---------------cut here---------------end--------------->8--- > > The problem was already reported as and > since commit 412701b0e5e073e6767eed162c14698db99df69c (July 2017) ‘guix > publish’ on GuixSD runs in a UTF-8 locale to avoid that problem. > > The faulty narinfo/nar on berlin were generated on Oct. 17, 2018, so > clearly the above commit was in effect. Indeed, after removing them and > regenerating them, I’m still getting > 08ziz714diyfq2klxy1nc0nhr5wa2vd356n9vizlq913a7an9a9s (aka. the wrong > hash). > > On closer inspection the problem is elsewhere: the > /gnu/store/xbj4fhad0lnz0ziflwi90gyqbls8ains-nss-certs-3.39 directory on > berlin has question marks in file names, so ‘guix publish’ is not to > blame; instead the problem likely comes from ‘guix offload’. > > Indeed ‘guix-daemon’ and its child processes such as ‘guix offload’ run > with an empty environment, and thus in the C locale. Specifically, > ‘restore-file-set’ on the build farm front-end must be the one > substituting question marks to the non-ASCII characters. > > If this analysis is correct, the patch below should fix it. I’ll try it > later. > > Thanks, > Ludo’. > > > > diff --git a/gnu/services/base.scm b/gnu/services/base.scm > index cee9898d79..9fe64e8087 100644 > --- a/gnu/services/base.scm > +++ b/gnu/services/base.scm > @@ -1603,7 +1603,15 @@ failed to register public key '~a': ~a~%" key status)))))))) > '()) > #$@(if tmpdir > (list (string-append "TMPDIR=" tmpdir)) > - '())) > + '()) > + > + ;; Make sure we run in a UTF-8 locale so that 'guix > + ;; offload' correctly restores nars that contain UTF-8 > + ;; file names such as 'nss-certs'. See > + ;; . > + (string-append "GUIX_LOCPATH=" > + #$glibc-utf8-locales "/lib/locale") > + "LC_ALL=en_US.utf8") > > #:log-file #$log-file)) > (stop #~(make-kill-destructor)))))) > Congratulations with the succeded hunt and thanks a lot for showing all the steps you took so I can improve my hunting skills and eventually begin helping by hunting on my own :D -- Cheers Swedebugia From unknown Sun Jun 15 08:59:20 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 17 Jan 2019 12:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator