From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 23 Dec 2018 14:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 33848@debbugs.gnu.org Cc: Pierre Neidhardt , Andy Patterson X-Debbugs-Original-To: Bug Guix Received: via spool by submit@debbugs.gnu.org id=B.15455748555877 (code B ref -1); Sun, 23 Dec 2018 14:21:02 +0000 Received: (at submit) by debbugs.gnu.org; 23 Dec 2018 14:20:55 +0000 Received: from localhost ([127.0.0.1]:60618 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb4cg-0001Wj-OS for submit@debbugs.gnu.org; Sun, 23 Dec 2018 09:20:55 -0500 Received: from eggs.gnu.org ([208.118.235.92]:35153) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb4cd-0001WV-QJ for submit@debbugs.gnu.org; Sun, 23 Dec 2018 09:20:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gb4c6-0006x6-G2 for submit@debbugs.gnu.org; Sun, 23 Dec 2018 09:20:46 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:42664) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gb4c5-0006wI-Lq for submit@debbugs.gnu.org; Sun, 23 Dec 2018 09:20:17 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53071) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gb4bi-00018j-WD for bug-guix@gnu.org; Sun, 23 Dec 2018 09:20:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gb4bO-0006IA-KZ for bug-guix@gnu.org; Sun, 23 Dec 2018 09:19:54 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:41666) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gb4bO-0006Ge-8W for bug-guix@gnu.org; Sun, 23 Dec 2018 09:19:34 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id B1233126E; Sun, 23 Dec 2018 15:19:32 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vknIypg0FXuW; Sun, 23 Dec 2018 15:19:31 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 779FB1D8; Sun, 23 Dec 2018 15:19:31 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 3 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 23 Dec 2018 15:19:30 +0100 Message-ID: <87r2e8jpfx.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hello, As discussed with Pierre at the R-B Summit, =E2=80=98sbcl-next=E2=80=99 lac= ks a reference to =E2=80=98next-gtk-webkit=E2=80=99 even though is invokes it: --8<---------------cut here---------------start------------->8--- $ guix gc --references $(type -P next) | grep next- /gnu/store/9d66xb8wvggsp0x9pxj61mzqy007978f-sbcl-next-1.1.0 /gnu/store/pqy064fw3vkfld6lw95vi0zavj19zvrc-sbcl-next-1.1.0-lib $ ./pre-inst-env guix run next WARNING: Setting locale failed. Check the following variables for correct values: LANG=3Den_US.utf8 Unhandled SIMPLE-ERROR in thread #: Couldn't execute "/gnu/store/7p6pbcmdgr53dff6033gcfl2jq0d762h-next-gtk-we= bkit-1.1.0/bin/next-gtk-webkit": No such file or directory --8<---------------cut here---------------end--------------->8--- (Here =E2=80=98guix run=E2=80=99 runs =E2=80=98next=E2=80=99 in a container= with exactly the closure of =E2=80=98next=E2=80=99, nothing more, and the =E2=80=98next=E2=80=99 binary= is grafted.) So the problem looks a lot like that this GCC issue we fixed a while back: . Looking at the =E2=80=98sbcl-next=E2=80=99 package, the reference to =E2=80= =98next-gtk-webkit=E2=80=99 is inserted in gtk-webkit.lisp: --8<---------------cut here---------------start------------->8--- (defvar *gtk-webkit-command* "next-gtk-webkit" "Path to the GTK-Webkit platform port executable.") --8<---------------cut here---------------end--------------->8--- Through hexl-mode on the =E2=80=98next=E2=80=99 binary, we can find that re= ference: --8<---------------cut here---------------start------------->8--- 01d0bac0: 2f00 0000 6700 0000 6e00 0000 7500 0000 /...g...n...u... 01d0bad0: 2f00 0000 7300 0000 7400 0000 6f00 0000 /...s...t...o... 01d0bae0: 7200 0000 6500 0000 2f00 0000 3700 0000 r...e.../...7... 01d0baf0: 7000 0000 3600 0000 7000 0000 6200 0000 p...6...p...b... 01d0bb00: 6300 0000 6d00 0000 6400 0000 6700 0000 c...m...d...g... 01d0bb10: 7200 0000 3500 0000 3300 0000 6400 0000 r...5...3...d... 01d0bb20: 6600 0000 6600 0000 3600 0000 3000 0000 f...f...6...0... 01d0bb30: 3300 0000 3300 0000 6700 0000 6300 0000 3...3...g...c... 01d0bb40: 6600 0000 6c00 0000 3200 0000 6a00 0000 f...l...2...j... 01d0bb50: 7100 0000 3000 0000 6400 0000 3700 0000 q...0...d...7... 01d0bb60: 3600 0000 3200 0000 6800 0000 2d00 0000 6...2...h...-... 01d0bb70: 6e00 0000 6500 0000 7800 0000 7400 0000 n...e...x...t... 01d0bb80: 2d00 0000 6700 0000 7400 0000 6b00 0000 -...g...t...k... 01d0bb90: 2d00 0000 7700 0000 6500 0000 6200 0000 -...w...e...b... 01d0bba0: 6b00 0000 6900 0000 7400 0000 2d00 0000 k...i...t...-... 01d0bbb0: 3100 0000 2e00 0000 3100 0000 2e00 0000 1.......1....... 01d0bbc0: 3000 0000 2f00 0000 6200 0000 6900 0000 0.../...b...i... 01d0bbd0: 6e00 0000 2f00 0000 6e00 0000 6500 0000 n.../...n...e... 01d0bbe0: 7800 0000 7400 0000 2d00 0000 6700 0000 x...t...-...g... 01d0bbf0: 7400 0000 6b00 0000 2d00 0000 7700 0000 t...k...-...w... 01d0bc00: 6500 0000 6200 0000 6b00 0000 6900 0000 e...b...k...i... 01d0bc10: 7400 0000 0000 0000 0000 0000 0000 0000 t............... 01d0bc20: e100 0100 0000 0000 2800 0000 0000 0000 ........(....... 01d0bc30: 2a47 544b 2d57 4542 4b49 542d 434f 4d4d *GTK-WEBKIT-COMM 01d0bc40: 414e 442a 0000 0000 0000 0000 0000 0000 AND*............ --8<---------------cut here---------------end--------------->8--- Apparently this string literal is stored as UTF-32 (UCS-4) or similar, which prevents the reference scanner and the grafting code from finding it, and problems ensue. :-) Pierre, Andy: is there any way to tell SBCL to store this literal as ASCII/UTF-8? That would be an easy fix, though we should discuss the pros and cons and whether to enable that globally. Thanks in advance! Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 23 Dec 2018 15:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: ajpatter@uwaterloo.ca, 33848@debbugs.gnu.org X-Debbugs-Original-Cc: Andy Patterson , Bug Guix Received: via spool by submit@debbugs.gnu.org id=B.154557757411407 (code B ref -1); Sun, 23 Dec 2018 15:07:01 +0000 Received: (at submit) by debbugs.gnu.org; 23 Dec 2018 15:06:14 +0000 Received: from localhost ([127.0.0.1]:33890 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb5KY-0002xv-C4 for submit@debbugs.gnu.org; Sun, 23 Dec 2018 10:06:14 -0500 Received: from eggs.gnu.org ([208.118.235.92]:40163) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb5KW-0002xj-RA for submit@debbugs.gnu.org; Sun, 23 Dec 2018 10:06:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gb5KQ-0001eY-QF for submit@debbugs.gnu.org; Sun, 23 Dec 2018 10:06:07 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:52557) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gb5KQ-0001eK-NL for submit@debbugs.gnu.org; Sun, 23 Dec 2018 10:06:06 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48343) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gb5KP-0003W7-Qs for bug-guix@gnu.org; Sun, 23 Dec 2018 10:06:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gb5KM-0001YX-I2 for bug-guix@gnu.org; Sun, 23 Dec 2018 10:06:05 -0500 Received: from relay4-d.mail.gandi.net ([217.70.183.196]:50737) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gb5KM-0001Wl-Bp; Sun, 23 Dec 2018 10:06:02 -0500 X-Originating-IP: 78.199.129.170 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id 7BDFBE0008; Sun, 23 Dec 2018 15:05:58 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87r2e8jpfx.fsf@gnu.org> Date: Sun, 23 Dec 2018 16:05:57 +0100 Message-ID: <87a7kwjnai.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Thanks for looking into this, Ludo. At first glance, I'd say that this is not a compilation option but the way strings are encoded by default. It seems that multibyte encoding is used a= ll over the place by a few compilers including SBCL (and CCL I think). One way I know around this (I'm by no mean a Common Lisp expert) is the flexi-streams package for re-encoding. More generally, shouldn't we make the reference scanner a bit smarter? In particular, how does it handle non-ASCII references? Maybe it would not be unreasonable to handle UTF-8 and UCS-4 for instance? =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwfpFUACgkQm9z0l6S7 zH/rdggAnQg4EGjzFVDFFNwofXuTBydyu+uLV6FHjagMA1ijxaEPtL27NhmdWUdM oFgRKIKabQpixyDSWJhAPGPIv2JHdrqiBwNRUfaDWSKhoh8/qA654QF8NiSFCHaf JpXMcdYVGvQ92Fo9OUFls8CWeWSpaEgQcTeIeeTNLwDCid8ob5gFW8doaqxraGw6 dkLeqIOuenB7jI/7cBs4yD4e+r8V/IAY/mVvuTZ+gRFGu+StbMo01KRX2X1xOVaL L4WiaGhJHyDYqw3otCPfZduOvsOyfhSrov3HpPT6vocQ11Wb8tW7t+JipO+w1DLi FMG7JYdAN3ibsDXXncsKcSxgeszj1g== =oKMR -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 23 Dec 2018 16:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154558360120607 (code B ref 33848); Sun, 23 Dec 2018 16:47:02 +0000 Received: (at 33848) by debbugs.gnu.org; 23 Dec 2018 16:46:41 +0000 Received: from localhost ([127.0.0.1]:33947 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb6tk-0005MJ-Lx for submit@debbugs.gnu.org; Sun, 23 Dec 2018 11:46:40 -0500 Received: from world.peace.net ([64.112.178.59]:32792) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb6tj-0005M6-1x for 33848@debbugs.gnu.org; Sun, 23 Dec 2018 11:46:39 -0500 Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gb6tb-0006Wd-M3; Sun, 23 Dec 2018 11:46:31 -0500 From: Mark H Weaver References: <87r2e8jpfx.fsf@gnu.org> Date: Sun, 23 Dec 2018 11:45:25 -0500 In-Reply-To: <87r2e8jpfx.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sun, 23 Dec 2018 15:19:30 +0100") Message-ID: <877eg0i43j.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludovic, Ludovic Court=C3=A8s writes: > As discussed with Pierre at the R-B Summit, =E2=80=98sbcl-next=E2=80=99 l= acks a > reference to =E2=80=98next-gtk-webkit=E2=80=99 even though is invokes it: > > $ guix gc --references $(type -P next) | grep next- > /gnu/store/9d66xb8wvggsp0x9pxj61mzqy007978f-sbcl-next-1.1.0 > /gnu/store/pqy064fw3vkfld6lw95vi0zavj19zvrc-sbcl-next-1.1.0-lib > $ ./pre-inst-env guix run next > > WARNING: Setting locale failed. > Check the following variables for correct values: > LANG=3Den_US.utf8 > Unhandled SIMPLE-ERROR in thread # {10005885B3}>: > Couldn't execute "/gnu/store/7p6pbcmdgr53dff6033gcfl2jq0d762h-next-gtk-= webkit-1.1.0/bin/next-gtk-webkit": No such file or directory > > > (Here =E2=80=98guix run=E2=80=99 runs =E2=80=98next=E2=80=99 in a contain= er with exactly the closure of > =E2=80=98next=E2=80=99, nothing more, and the =E2=80=98next=E2=80=99 bina= ry is grafted.) > > So the problem looks a lot like that this GCC issue we fixed a while > back: . > > Looking at the =E2=80=98sbcl-next=E2=80=99 package, the reference to =E2= =80=98next-gtk-webkit=E2=80=99 > is inserted in gtk-webkit.lisp: > > (defvar *gtk-webkit-command* "next-gtk-webkit" > "Path to the GTK-Webkit platform port executable.") > > > Through hexl-mode on the =E2=80=98next=E2=80=99 binary, we can find that = reference: > > 01d0bac0: 2f00 0000 6700 0000 6e00 0000 7500 0000 /...g...n...u... > 01d0bad0: 2f00 0000 7300 0000 7400 0000 6f00 0000 /...s...t...o... > 01d0bae0: 7200 0000 6500 0000 2f00 0000 3700 0000 r...e.../...7... > 01d0baf0: 7000 0000 3600 0000 7000 0000 6200 0000 p...6...p...b... > 01d0bb00: 6300 0000 6d00 0000 6400 0000 6700 0000 c...m...d...g... > 01d0bb10: 7200 0000 3500 0000 3300 0000 6400 0000 r...5...3...d... > 01d0bb20: 6600 0000 6600 0000 3600 0000 3000 0000 f...f...6...0... > 01d0bb30: 3300 0000 3300 0000 6700 0000 6300 0000 3...3...g...c... > 01d0bb40: 6600 0000 6c00 0000 3200 0000 6a00 0000 f...l...2...j... > 01d0bb50: 7100 0000 3000 0000 6400 0000 3700 0000 q...0...d...7... > 01d0bb60: 3600 0000 3200 0000 6800 0000 2d00 0000 6...2...h...-... > 01d0bb70: 6e00 0000 6500 0000 7800 0000 7400 0000 n...e...x...t... > 01d0bb80: 2d00 0000 6700 0000 7400 0000 6b00 0000 -...g...t...k... > 01d0bb90: 2d00 0000 7700 0000 6500 0000 6200 0000 -...w...e...b... > 01d0bba0: 6b00 0000 6900 0000 7400 0000 2d00 0000 k...i...t...-... > 01d0bbb0: 3100 0000 2e00 0000 3100 0000 2e00 0000 1.......1....... > 01d0bbc0: 3000 0000 2f00 0000 6200 0000 6900 0000 0.../...b...i... > 01d0bbd0: 6e00 0000 2f00 0000 6e00 0000 6500 0000 n.../...n...e... > 01d0bbe0: 7800 0000 7400 0000 2d00 0000 6700 0000 x...t...-...g... > 01d0bbf0: 7400 0000 6b00 0000 2d00 0000 7700 0000 t...k...-...w... > 01d0bc00: 6500 0000 6200 0000 6b00 0000 6900 0000 e...b...k...i... > 01d0bc10: 7400 0000 0000 0000 0000 0000 0000 0000 t............... > 01d0bc20: e100 0100 0000 0000 2800 0000 0000 0000 ........(....... > 01d0bc30: 2a47 544b 2d57 4542 4b49 542d 434f 4d4d *GTK-WEBKIT-COMM > 01d0bc40: 414e 442a 0000 0000 0000 0000 0000 0000 AND*............ > > Apparently this string literal is stored as UTF-32 (UCS-4) or similar, > which prevents the reference scanner and the grafting code from finding > it, and problems ensue. :-) IMO, we should consider modifying Guix to search for store references encoded in UTF-32 and/or UTF-16. I wouldn't be surprised if some other programs use those encodings. I'd be willing to work on it. What do you think? Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 23 Dec 2018 17:33:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154558636025014 (code B ref 33848); Sun, 23 Dec 2018 17:33:02 +0000 Received: (at 33848) by debbugs.gnu.org; 23 Dec 2018 17:32:40 +0000 Received: from localhost ([127.0.0.1]:33981 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb7cF-0006VM-MU for submit@debbugs.gnu.org; Sun, 23 Dec 2018 12:32:40 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:52410) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb7cE-0006VE-44 for 33848@debbugs.gnu.org; Sun, 23 Dec 2018 12:32:38 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 7CD6312BB; Sun, 23 Dec 2018 18:32:37 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qtk9Zqe9FfNj; Sun, 23 Dec 2018 18:32:36 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 6997F1282; Sun, 23 Dec 2018 18:32:36 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 3 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 23 Dec 2018 18:32:35 +0100 In-Reply-To: <877eg0i43j.fsf@netris.org> (Mark H. Weaver's message of "Sun, 23 Dec 2018 11:45:25 -0500") Message-ID: <87d0psi1xo.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi Mark, Mark H Weaver skribis: > Ludovic Court=C3=A8s writes: [...] >> Apparently this string literal is stored as UTF-32 (UCS-4) or similar, >> which prevents the reference scanner and the grafting code from finding >> it, and problems ensue. :-) > > IMO, we should consider modifying Guix to search for store references > encoded in UTF-32 and/or UTF-16. I wouldn't be surprised if some other > programs use those encodings. I'd be willing to work on it. I don=E2=80=99t think we=E2=80=99ve encountered the problem before. This w= ould require fixing both the scanner and the grafting code (though eventually that might be a single code base when the Scheme-implemented daemon is merged) in non-trivial ways. One issue is that users of an old daemon would get a different behavior than users of a new daemon. It would be the first time we introduce such a significant change in the daemon since Guix was started. For now I lean towards looking for a way to address the issue specifically for SBCL. I=E2=80=99d be tempted to generalize if and only if= we find other occurrences of the problem that would make the benefits outweigh the development and maintenance costs. WDYT? I remember discussing in the past some sort of =E2=80=9Cpluggable=E2=80=9D = reference scanning mechanism that could also work for compressed archives, etc. That also looks like the right thing, but it has a development and maintenance cost that=E2=80=99s pretty high whereas we might be able to add= ress the same problems in much simpler ways. Thanks, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 23 Dec 2018 22:02:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154560246920087 (code B ref 33848); Sun, 23 Dec 2018 22:02:02 +0000 Received: (at 33848) by debbugs.gnu.org; 23 Dec 2018 22:01:09 +0000 Received: from localhost ([127.0.0.1]:34100 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbBo5-0005Du-6q for submit@debbugs.gnu.org; Sun, 23 Dec 2018 17:01:09 -0500 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:46911) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbBo4-0005Dn-7D for 33848@debbugs.gnu.org; Sun, 23 Dec 2018 17:01:08 -0500 X-Originating-IP: 78.199.129.170 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id 48A421BF208; Sun, 23 Dec 2018 22:01:03 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87d0psi1xo.fsf@gnu.org> Date: Sun, 23 Dec 2018 23:01:01 +0100 Message-ID: <874lb3kin6.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > I don=E2=80=99t think we=E2=80=99ve encountered the problem before. Actually it does ring a bell for me. Didn't we have a similar issue with F= ish, or some dependency? > For now I lean towards looking for a way to address the issue > specifically for SBCL. Don't forget that we currently have 5 Lisp compilers. Besides, it's not clear that this can be fixed on the compiler's side, it c= ould very well be that patches will be required on a per-project basis. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwgBZ0ACgkQm9z0l6S7 zH8XGAgAqUkpkfyLkBTmGkB0E4yhMQ2Mo5elI/PvQQN4NGPHl/VDJysQMZJPnYDS N4FAooCf3v5oenfr+VgZKr7NDDkDgVIdZbUCjzEw0La7FFl8DpB4+riJ0WqtghiB jCr4KRNfuSn1tgIenvMFsswH3otTaAllMIlfqMhxYJDtGYTzcjP059xgDQ0rPlKF PoAOv839rILx0AfdXAp7knIV+q4iN623ZEiGFIJQ3K2JuaPoBkBBVUHkk/lJSOvW naGrIE56gqOnjMKJuTx9FuhhYPtN8ieNYj/VLV3y5V9v5JyvO3zwV49ahemBEK84 n9PxAlU4A0D66Gy6ZmCBN38Ewgfhkw== =s6io -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 24 09:55:10 2018 Received: (at control) by debbugs.gnu.org; 24 Dec 2018 14:55:10 +0000 Received: from localhost ([127.0.0.1]:34464 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbRdO-0003TD-5D for submit@debbugs.gnu.org; Mon, 24 Dec 2018 09:55:10 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:58916) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbRdM-0003Sj-NI for control@debbugs.gnu.org; Mon, 24 Dec 2018 09:55:09 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 9EB971468 for ; Mon, 24 Dec 2018 15:55:07 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J4tsYUKcAqaR for ; Mon, 24 Dec 2018 15:55:07 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id B9FFB1102 for ; Mon, 24 Dec 2018 15:55:06 +0100 (CET) Date: Mon, 24 Dec 2018 15:55:06 +0100 Message-Id: <87a7kvgek5.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #33848 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) severity 33848 important From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 24 Dec 2018 14:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: ajpatter@uwaterloo.ca, 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154566347514709 (code B ref 33848); Mon, 24 Dec 2018 14:58:02 +0000 Received: (at 33848) by debbugs.gnu.org; 24 Dec 2018 14:57:55 +0000 Received: from localhost ([127.0.0.1]:35532 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbRg2-0003pB-Ny for submit@debbugs.gnu.org; Mon, 24 Dec 2018 09:57:54 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:58924) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbRg1-0003p2-55 for 33848@debbugs.gnu.org; Mon, 24 Dec 2018 09:57:53 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 9035E1468; Mon, 24 Dec 2018 15:57:52 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OWrLy6bCjPYf; Mon, 24 Dec 2018 15:57:52 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id B71BA1102; Mon, 24 Dec 2018 15:57:51 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <87a7kwjnai.fsf@ambrevar.xyz> Date: Mon, 24 Dec 2018 15:57:50 +0100 In-Reply-To: <87a7kwjnai.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Sun, 23 Dec 2018 16:05:57 +0100") Message-ID: <875zvjgefl.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi! Pierre Neidhardt skribis: > Thanks for looking into this, Ludo. > > At first glance, I'd say that this is not a compilation option but the way > strings are encoded by default. It seems that multibyte encoding is used= all > over the place by a few compilers including SBCL (and CCL I think). > > One way I know around this (I'm by no mean a Common Lisp expert) is the > flexi-streams package for re-encoding. OK, we need to investigate. > More generally, shouldn't we make the reference scanner a bit smarter? In > particular, how does it handle non-ASCII references? Maybe it would not = be > unreasonable to handle UTF-8 and UCS-4 for instance? Store file names are always ASCII so problems arise when they are stored as UTF-16 or UTF-32/UCS-4. Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 24 Dec 2018 15:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154566397323268 (code B ref 33848); Mon, 24 Dec 2018 15:07:01 +0000 Received: (at 33848) by debbugs.gnu.org; 24 Dec 2018 15:06:13 +0000 Received: from localhost ([127.0.0.1]:35540 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbRo5-00063E-01 for submit@debbugs.gnu.org; Mon, 24 Dec 2018 10:06:13 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:58974) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbRo3-000636-Iy for 33848@debbugs.gnu.org; Mon, 24 Dec 2018 10:06:11 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 0079C1494; Mon, 24 Dec 2018 16:06:11 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 57lQzFcVz4IN; Mon, 24 Dec 2018 16:06:10 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id C72FA138F; Mon, 24 Dec 2018 16:06:09 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> Date: Mon, 24 Dec 2018 16:06:09 +0100 In-Reply-To: <874lb3kin6.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Sun, 23 Dec 2018 23:01:01 +0100") Message-ID: <87sgynezha.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi Pierre, Pierre Neidhardt skribis: >> I don=E2=80=99t think we=E2=80=99ve encountered the problem before. > > Actually it does ring a bell for me. Didn't we have a similar issue with= Fish, > or some dependency? We did have a problem with Fish but I can no longer find it. Do you remember what it was? Something with C++, no? >> For now I lean towards looking for a way to address the issue >> specifically for SBCL. > > Don't forget that we currently have 5 Lisp compilers. > Besides, it's not clear that this can be fixed on the compiler's side, it= could > very well be that patches will be required on a per-project basis. I know little about CL but maybe we can find a solution that works for all five compilers. At least that would be the first approach I would suggest following. Thanks, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 24 Dec 2018 17:10:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15456713442793 (code B ref 33848); Mon, 24 Dec 2018 17:10:01 +0000 Received: (at 33848) by debbugs.gnu.org; 24 Dec 2018 17:09:04 +0000 Received: from localhost ([127.0.0.1]:35589 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbTiy-0000iy-71 for submit@debbugs.gnu.org; Mon, 24 Dec 2018 12:09:04 -0500 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:58059) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbTiw-0000id-4i for 33848@debbugs.gnu.org; Mon, 24 Dec 2018 12:09:02 -0500 X-Originating-IP: 78.199.129.170 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 09C0EFF808; Mon, 24 Dec 2018 17:08:59 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87sgynezha.fsf@gnu.org> Date: Mon, 24 Dec 2018 18:08:59 +0100 Message-ID: <87r2e6j1hw.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable > Store file names are always ASCII so problems arise when they are stored > as UTF-16 or UTF-32/UCS-4. I understand that most programs stick to ASCII filenames, but what about th= e odd one using non-English, special characters? > We did have a problem with Fish but I can no longer find it. Do you > remember what it was? Something with C++, no? I think bug #30265. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwhEqsACgkQm9z0l6S7 zH8nhAf9Hv2U1ajhnsl50XrKSr629VR3LFtu6whoiU3WJOygmulOIdlaWJ2IRFSR mwCvD8I/pE+BokAgT28BQNpvyG78+vgJeevb4adTD8eUxQsS2aRLPvJ9Js3B4epY tTDxtm6xp/kFKmxk/9WFYX/lxuyXfSYv/A7m8q3qWfngzvizZjCZVY0iQHrDLlfS xP1TVlUoiudIUo9BCjLmQQuyAkgxgDln9idzgXZKWXZMrW6HcK3Q4Ji2ymowCUf0 vHRGj2mjBHo+QSYhOz/NduJPG717THk9C+9xG6eOyFa712VIwEJZc5dPKA50J/s0 OaeNE8fk/mCyJ2y3yyE2V61wqhgRvA== =E6l9 -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 24 Dec 2018 18:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15456752189302 (code B ref 33848); Mon, 24 Dec 2018 18:14:01 +0000 Received: (at 33848) by debbugs.gnu.org; 24 Dec 2018 18:13:38 +0000 Received: from localhost ([127.0.0.1]:35618 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbUjR-0002Px-Ot for submit@debbugs.gnu.org; Mon, 24 Dec 2018 13:13:38 -0500 Received: from world.peace.net ([64.112.178.59]:37592) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbUjQ-0002Pl-Im for 33848@debbugs.gnu.org; Mon, 24 Dec 2018 13:13:36 -0500 Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gbUjJ-0004Wz-Ok; Mon, 24 Dec 2018 13:13:29 -0500 From: Mark H Weaver References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> Date: Mon, 24 Dec 2018 13:12:23 -0500 In-Reply-To: <87sgynezha.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 24 Dec 2018 16:06:09 +0100") Message-ID: <87tvj2yesd.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludovic, Ludovic Court=C3=A8s writes: > Pierre Neidhardt skribis: > >>> For now I lean towards looking for a way to address the issue >>> specifically for SBCL. >> >> Don't forget that we currently have 5 Lisp compilers. >> Besides, it's not clear that this can be fixed on the compiler's side, i= t could >> very well be that patches will be required on a per-project basis. > > I know little about CL but maybe we can find a solution that works for > all five compilers. At least that would be the first approach I would > suggest following. I can't imagine a solution that would work for all five compilers, but perhaps that's a failure of imagination on my part. Of course, you're welcome to search for such a solution. Can you give me a rough outline of what you have in mind? Of course, the usual reason to choose UTF-32 is to support non-ASCII characters while retaining fixed-width code points, so that string lookups are straightforward and efficient. Using UTF-8 improves space efficiency, but at the cost of extra code complexity. That extra complexity is what I guess we would need to add to each program that currently uses UTF-32. Alternatively, we could extend the on-disk format to support UTF-8 and then add some kind of "load hook" that converts the string to UTF-32 at load time. Either way, it's likely to be a can of worms. Consider the case of Guile. Years ago we agreed to switch to UTF-8 as its sole internal string encoding, but it hasn't yet been done because it's a big job, even for those of us already intimately familiar with the code. Now imagine how hard it would be for someone who barely uses Guile, but nevertheless felt compelled to change our internal string representation to use UTF-8. Moreover, imagine that they hoped to find a single solution that would work for several different Scheme implementations. What would you say to them if they proposed to find a general solution to convert several Scheme implementations to use UTF-8 as their string representation, to save themselves the trouble of having to understand each implementation individually? I really think it would be a mistake to try to force every program and language implementation to use our preferred string representation. I suspect it would be vastly easier to compromise and support a few other popular string representations in Guix, namely UTF-16 and UTF-32. If you don't want to change the daemon, it could be worked around in our build-side code as follows: we could add a new phase to certain build systems (or possibly gnu-build-system) that scans each output for UTF-16/32 encoded store references that are never referenced in UTF-8. If such references exist, a file with an unobtrusive name would be added to that output containing those references encoded in UTF-8. This would enable our daemon's existing reference scanner to find all of the references. Our grafting code would then need to be extended to recognize and transform store references encoded in UTF-16/32 as well as UTF-8. What do you think? Regards, Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 24 Dec 2018 23:59:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15456959248943 (code B ref 33848); Mon, 24 Dec 2018 23:59:01 +0000 Received: (at 33848) by debbugs.gnu.org; 24 Dec 2018 23:58:44 +0000 Received: from localhost ([127.0.0.1]:35668 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gba7M-0002K7-Av for submit@debbugs.gnu.org; Mon, 24 Dec 2018 18:58:44 -0500 Received: from relay12.mail.gandi.net ([217.70.178.232]:57635) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gba7H-0002Jt-5P for 33848@debbugs.gnu.org; Mon, 24 Dec 2018 18:58:39 -0500 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay12.mail.gandi.net (Postfix) with ESMTPSA id 0C571200004; Mon, 24 Dec 2018 23:58:32 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87tvj2yesd.fsf@netris.org> Date: Tue, 25 Dec 2018 00:58:31 +0100 Message-ID: <87pntqiijc.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable I find Mark's points reasonable, although to be honest I have very little knowledge of the daemon. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwhcqcACgkQm9z0l6S7 zH/OWAf9EAHvRknh8vP7fQlEH0koh30BL0A62gzcn25Aw/Un5zPR7ZZ5vBhm7FqQ fOS3fGF+ZYwTd5QWHz/49sNMJes4caaNqxN9x1xm/IBU374/MEkxtvnJqNctYL0z MAIXlruvch+cBYFfAyuCDjkNqFBHuqlFPP1lZCbal6xHvirMLLzNfRhQcFtYXD0T y3YN0D5T9KfgQcrDEf78ShJSBto7lyBMKe9PqJBeKJexrzkD1XsY+sZB0PXiSrTN CT/tC2MqT8QMRrfGNpEepQIHMqowjVheJ3vcC5NDEKT7IFdY5art5d96+QpFlXNl RrCgC+3b8NGpxoWSR4oGqpR3FrjzEw== =Apxv -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 26 Dec 2018 16:08:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15458404604254 (code B ref 33848); Wed, 26 Dec 2018 16:08:01 +0000 Received: (at 33848) by debbugs.gnu.org; 26 Dec 2018 16:07:40 +0000 Received: from localhost ([127.0.0.1]:37878 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcBie-00016Y-4U for submit@debbugs.gnu.org; Wed, 26 Dec 2018 11:07:40 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:44310) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcBia-00016N-Ka for 33848@debbugs.gnu.org; Wed, 26 Dec 2018 11:07:38 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id AE20C1753; Wed, 26 Dec 2018 17:07:35 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N7ShGo81cct7; Wed, 26 Dec 2018 17:07:34 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 65F6CAAB; Wed, 26 Dec 2018 17:07:34 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87r2e6j1hw.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 6 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 26 Dec 2018 17:07:33 +0100 In-Reply-To: <87r2e6j1hw.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Mon, 24 Dec 2018 18:08:59 +0100") Message-ID: <87r2e4e0fu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Pierre Neidhardt skribis: >> Store file names are always ASCII so problems arise when they are stored >> as UTF-16 or UTF-32/UCS-4. > > I understand that most programs stick to ASCII filenames, but what about = the odd > one using non-English, special characters? That=E2=80=99s a separate debate. :-) Essentially this restriction on sto= re file names has always been there in Guix (and Nix before that). If we were to change it, that would raise compatibility issues. >> We did have a problem with Fish but I can no longer find it. Do you >> remember what it was? Something with C++, no? > > I think bug #30265. Oh I see, UCS-4 as well. (I can=E2=80=99t believe this bug is still open g= iven the relatively simple solutions outlined at . :-)) Thanks, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 26 Dec 2018 16:15:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15458408564903 (code B ref 33848); Wed, 26 Dec 2018 16:15:01 +0000 Received: (at 33848) by debbugs.gnu.org; 26 Dec 2018 16:14:16 +0000 Received: from localhost ([127.0.0.1]:37888 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcBp0-0001Gz-E6 for submit@debbugs.gnu.org; Wed, 26 Dec 2018 11:14:15 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:44360) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcBoy-0001Gr-0t for 33848@debbugs.gnu.org; Wed, 26 Dec 2018 11:14:12 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 7B83917CE; Wed, 26 Dec 2018 17:14:11 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DDHfqvVGlezg; Wed, 26 Dec 2018 17:14:10 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id BF6031753; Wed, 26 Dec 2018 17:14:09 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 6 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 26 Dec 2018 17:14:09 +0100 In-Reply-To: <87tvj2yesd.fsf@netris.org> (Mark H. Weaver's message of "Mon, 24 Dec 2018 13:12:23 -0500") Message-ID: <877efwe04u.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hello! Mark H Weaver skribis: > Ludovic Court=C3=A8s writes: > >> Pierre Neidhardt skribis: >> >>>> For now I lean towards looking for a way to address the issue >>>> specifically for SBCL. >>> >>> Don't forget that we currently have 5 Lisp compilers. >>> Besides, it's not clear that this can be fixed on the compiler's side, = it could >>> very well be that patches will be required on a per-project basis. >> >> I know little about CL but maybe we can find a solution that works for >> all five compilers. At least that would be the first approach I would >> suggest following. > > I can't imagine a solution that would work for all five compilers, but > perhaps that's a failure of imagination on my part. Of course, you're > welcome to search for such a solution. Can you give me a rough outline > of what you have in mind? I have nothing specific in mind, I=E2=80=99m just brainstorming with everyo= ne here. :-) For a similar situation in C++, there=E2=80=99s a fairly simple and local workaround: https://issues.guix.info/issue/30265#8 I=E2=80=99m not familiar with CL but I thought that it we could achieve something similar, that would be great=E2=80=94I=E2=80=99m not suggesting t= o change the CL compilers in any non-trivial way. For example I guess we could always store the file name as a literal byte vector/list and add a call to turn that into a string. Does that make sense? Thanks, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 10:38:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154590704531975 (code B ref 33848); Thu, 27 Dec 2018 10:38:01 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 10:37:25 +0000 Received: from localhost ([127.0.0.1]:38203 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcT2b-0008Jf-8V for submit@debbugs.gnu.org; Thu, 27 Dec 2018 05:37:25 -0500 Received: from relay10.mail.gandi.net ([217.70.178.230]:51675) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcT2Y-0008JW-Hp for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 05:37:23 -0500 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 0AC2B240002; Thu, 27 Dec 2018 10:37:19 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <877efwe04u.fsf@gnu.org> Date: Thu, 27 Dec 2018 11:37:18 +0100 Message-ID: <8736qji7c1.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > : > Store file names are always ASCII so problems arise when they are sto= red > : > as UTF-16 or UTF-32/UCS-4. > :=20 > : I understand that most programs stick to ASCII filenames, but what abou= t the odd > : one using non-English, special characters? >=20 > That=E2=80=99s a separate debate. :-) Essentially this restriction on s= tore > file names has always been there in Guix (and Nix before that). If we > were to change it, that would raise compatibility issues. But what happens if we attempt to store "=C3=A1" in the store? > For example I guess we could always store the file name as a literal > byte vector/list and add a call to turn that into a string. In the case of Next, that would be a simple patch, but other programs could= get much more complicated. In the end, this approach requires a linear amount = of work. Conversely, adding UCS-* support to the scanner would fix this issue= once and for all. > : > We did have a problem with Fish but I can no longer find it. Do you > : > remember what it was? Something with C++, no? > :=20 > : I think bug #30265. >=20 > Oh I see, UCS-4 as well. (I can=E2=80=99t believe this bug is still open= given > the relatively simple solutions outlined at > . :-)) Well, if currently only two packages out of 8500+ suffer from this, then I = think it's easier to go with Ludo's suggestion of patching the code to use ASCII strings. Does anyone know about more packages with this issue? It could also be that more packages suffer from this, unbeknownst to us. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwkq14ACgkQm9z0l6S7 zH9zbAf8C7alkC/FiNu4pb3HkuSWZXKkZ/pccOIXH0ErCiND6SwQC9pBXTgxoYew p9Y3J0SrKyUMVKHidWERkA1EnVR6wBUT3sru6idmiNF2JIBw5JC+UiNdiS5RqvXd Ka3eHjqxVXfL2kEINOOSoiB1t6P6chQsxHJjxOs9TTk+8UgFgDMF9VhtYubiaLYf oBOP7FVAIojHHGxth14ekyohT65TD4mgRqK3mTsLxPjrQ43/nAayo6aJWilx5BB1 YoRe8bjUNzHS1G0JSsM6E8ZRwwUfwBBhwdqFml2O76LpJoWi/xi358JNldRqD7j/ eV0ZNuJAZjONvVJZ9qtfJDifLPJkJQ== =IU/3 -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 13:54:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15459187931764 (code B ref 33848); Thu, 27 Dec 2018 13:54:01 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 13:53:13 +0000 Received: from localhost ([127.0.0.1]:38272 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcW64-0000SO-JX for submit@debbugs.gnu.org; Thu, 27 Dec 2018 08:53:12 -0500 Received: from dd26836.kasserver.com ([85.13.145.193]:50892) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcW61-0000SE-8u for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 08:53:11 -0500 Received: from localhost (77.116.200.150.wireless.dyn.drei.com [77.116.200.150]) by dd26836.kasserver.com (Postfix) with ESMTPSA id E30FC3360147; Thu, 27 Dec 2018 14:53:06 +0100 (CET) Date: Thu, 27 Dec 2018 14:52:58 +0100 From: Danny Milosavljevic Message-ID: <20181227145258.0c420eac@scratchpost.org> In-Reply-To: <87tvj2yesd.fsf@netris.org> References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> X-Mailer: Claws Mail 3.17.1 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/psKMAB3Tik_KCO1Se691Qt1"; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --Sig_/psKMAB3Tik_KCO1Se691Qt1 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi Mark, On Mon, 24 Dec 2018 13:12:23 -0500 Mark H Weaver wrote: > Of course, the usual reason to choose UTF-32 is to support non-ASCII > characters while retaining fixed-width code points, so that string > lookups are straightforward and efficient. This kind of lookup is almost never what is necessary. There are many people who assume character is the same as codepoint and to those people UTF-32 brings something to the table, but it's really not useful if people do text processing correctly, see below. (Of course whether packages actually do this remains to be seen) > Using UTF-8 improves space efficiency, but at the cost of extra code >complexity. I agree. > That extra > complexity is what I guess we would need to add to each program that > currently uses UTF-32. Yes, but they usually have to do stream processing even with UTF-32 (because a character can be composed of possibly infinite number of codepoints), so the infrastructure should be already there and the effort should be minimal. > Alternatively, we could extend the on-disk > format to support UTF-8 and then add some kind of "load hook" that > converts the string to UTF-32 at load time. Either way, it's likely to > be a can of worms. If it ever came to that, a pluggable reference scanner would be=20 preferrable. But really, it would irk me to have so much complexity in something so basic (the reference scanner) for no end-user gain (as a distribution we could just mandate UTF-8 for references and the problem would be gone for the user with no loss of functionality). It's always easy to add special cases - but more code means more bugs and I think if possible it's best to have only the simple case implemented in the core - because it's less complicated which means more likely to be correct (for the case it does handle). In the end it depends on what would be more code, and more widely used. Also, if we wanted to debug reference errors, we couldn't use grep anymore because it can't handle utf-32 either (neither can any of the other UNIX to= ols). Also, I really don't want to return to the time where I had to call iconv once every three commands to be able to do anything useful on UNIX. Also, the build daemon is written in C++ and C++ strings are widely known to have very very bad codepoint awareness (to say nothing about the horrible conversion facilities). Also, if both UTF-32 and UTF-8 are used on disk, care needs to not misdetect an UTF-8 sequence as an UTF-32 sequence of different text - or the other way around -, but that's unlikely for ASCII strings. > I really think it would be a mistake to try to force every program and > language implementation to use our preferred string representation. I > suspect it would be vastly easier to compromise and support a few other > popular string representations in Guix, namely UTF-16 and UTF-32. In 1992, UTF-8 was invented. Subsequently, most of the Internet, all new GNU Linux distributions etc, all UNIX GUI frameworks, Subversion etc standardized on UTF-8, with the eventual goal of standardizing all network transfer and storage to UTF-8. I think that by now the outliers are the ones who need to change, otherwise these senseless encoding conversions will never cease. It's not like different encodings allow for better expression of writings or anything useful to the end user. As a distribution we can't force upstream to change, but just filing bug reports upstream would make us see where they stand on this. > If you don't want to change the daemon, it could be worked around in our > build-side code as follows: we could add a new phase to certain build > systems (or possibly gnu-build-system) that scans each output for > UTF-16/32 encoded store references that are never referenced in UTF-8. > If such references exist, a file with an unobtrusive name would be added > to that output containing those references encoded in UTF-8. This would > enable our daemon's existing reference scanner to find all of the > references. I agree that that would be nice. As a first step, even just detecting problems like that and erroring out would be okay - in order to find them in the first place. Right now, it's difficult to detect and so also diffic= ult to say how wide-spread the problem is. If the problem is wide-spread enough my tune could change very quickly. What you propose is similar to what I did in Java in Guix, only it gives us even more advantages in the Java case (faster class loading and eventual non-propagated inputs). --Sig_/psKMAB3Tik_KCO1Se691Qt1 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlwk2ToACgkQ5xo1VCww uqUUzAgApbxUHv/XlbjYMXvV4cOY0maxbx92ndZlJiukCN+bIiMqhuCd7PdEoL7Z 1d9ABxe+2oXO4Nkjpez71nhK8ym8KwRYNDuTkCSZbzUJwNEee2pF/OlU2Y+Jugz5 ICSlYGCFfwx6Buf9bZReYq1e5qjO//QSytgYC061gYURw/abtGSEyvllHWv4qrl6 DFfQuQilycHAOqrT/ACBtMgFFnsV7miHs6CrKTSPPBWKKuA3BM4STNUfHlMeb8un gNUH3ijbDviqBgRiqDy50dZ0kbFv8zSm1LytoySX0qZ7j5oidDJGHATGbEXp4sDU 1dPmBSYeasLAVfJn4RQSQFAKba6TuA== =gkV6 -----END PGP SIGNATURE----- --Sig_/psKMAB3Tik_KCO1Se691Qt1-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 14:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15459194753110 (code B ref 33848); Thu, 27 Dec 2018 14:05:02 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 14:04:35 +0000 Received: from localhost ([127.0.0.1]:38275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcWH1-0000nz-Ql for submit@debbugs.gnu.org; Thu, 27 Dec 2018 09:04:35 -0500 Received: from world.peace.net ([64.112.178.59]:50574) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcWGw-0000na-J5 for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 09:04:30 -0500 Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gcWGp-0003PY-UJ; Thu, 27 Dec 2018 09:04:19 -0500 From: Mark H Weaver References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> Date: Thu, 27 Dec 2018 09:03:12 -0500 In-Reply-To: <8736qji7c1.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Thu, 27 Dec 2018 11:37:18 +0100") Message-ID: <87tvizvzgk.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Pierre Neidhardt writes: >> : > Store file names are always ASCII so problems arise when they are st= ored >> : > as UTF-16 or UTF-32/UCS-4. >> :=20 >> : I understand that most programs stick to ASCII filenames, but what abo= ut the odd >> : one using non-English, special characters? >>=20 >> That=E2=80=99s a separate debate. :-) Essentially this restriction on = store >> file names has always been there in Guix (and Nix before that). If we >> were to change it, that would raise compatibility issues. > > But what happens if we attempt to store "=C3=A1" in the store? Indeed. Although we might restrict the immediate entries within /gnu/store to ASCII characters, file names deeper within those directories may have non-ASCII characters. More generally, store references may occur within larger strings which might include non-ASCII characters. Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 14:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Danny Milosavljevic Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15459210575744 (code B ref 33848); Thu, 27 Dec 2018 14:31:02 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 14:30:57 +0000 Received: from localhost ([127.0.0.1]:38280 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcWga-0001Ua-Oa for submit@debbugs.gnu.org; Thu, 27 Dec 2018 09:30:57 -0500 Received: from world.peace.net ([64.112.178.59]:50632) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcWgY-0001UM-MY for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 09:30:55 -0500 Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gcWgS-0003Wu-St; Thu, 27 Dec 2018 09:30:48 -0500 From: Mark H Weaver References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <20181227145258.0c420eac@scratchpost.org> Date: Thu, 27 Dec 2018 09:29:42 -0500 In-Reply-To: <20181227145258.0c420eac@scratchpost.org> (Danny Milosavljevic's message of "Thu, 27 Dec 2018 14:52:58 +0100") Message-ID: <87pntnvy8e.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Danny, Danny Milosavljevic writes: > On Mon, 24 Dec 2018 13:12:23 -0500 > Mark H Weaver wrote: > >> Of course, the usual reason to choose UTF-32 is to support non-ASCII >> characters while retaining fixed-width code points, so that string >> lookups are straightforward and efficient. > > This kind of lookup is almost never what is necessary. There are many > people who assume character is the same as codepoint and to those people > UTF-32 brings something to the table, but it's really not useful if people > do text processing correctly, see below. > > (Of course whether packages actually do this remains to be seen) > >> That extra >> complexity is what I guess we would need to add to each program that >> currently uses UTF-32. > > Yes, but they usually have to do stream processing even with UTF-32 (because > a character can be composed of possibly infinite number of codepoints), I agree with you. However, as silly as it might be, the fact remains that almost every modern programming language and string library uses code points as the base units by which to index strings. > so the infrastructure should be already there and the effort should be > minimal. The infrastructure might or might not be there, depending on the sophistication of the program's unicode support, but even if it _is_ there, it will most likely be a layer that expects to iterate over strings indexed by code point to find graphemes, etc. Anyway, if you truly believe the effort should be minimal, feel free to investigate and propose patches to fix our 5 common lisp compilers and Fish to avoid storing UTF-32 in the object code. > Also, if both UTF-32 and UTF-8 are used on disk, care needs to not misdetect > an UTF-8 sequence as an UTF-32 sequence of different text - or the other way > around -, but that's unlikely for ASCII strings. This is not an issue because the substrings that the reference scanner and grafter are looking for are ASCII-only, even if they are part of a larger non-ASCII string. Specifically, they only need to look for the nix hashes. >> I really think it would be a mistake to try to force every program and >> language implementation to use our preferred string representation. I >> suspect it would be vastly easier to compromise and support a few other >> popular string representations in Guix, namely UTF-16 and UTF-32. > > In 1992, UTF-8 was invented. Subsequently, most of the Internet, > all new GNU Linux distributions etc, all UNIX GUI frameworks, Subversion > etc standardized on UTF-8, with the eventual goal of standardizing all > network transfer and storage to UTF-8. I think that by now the outliers > are the ones who need to change, I agree that we need to standardize on Unicode. However, given the perhaps unfortunate fact that almost everyone has standardized on code points as the units by which to index strings, choosing UTF-32 as an internal representation is a very reasonable choice, IMO. Anyway, feel free to engage with the developers of the Common Lisp implementations that use UTF-32 and try to convince them to change. The remaining question is: what to do if upstream refuses to change? Do we exclude that software in Guix, or do we maintain our own patches to override upstream's decision? >> If you don't want to change the daemon, it could be worked around in our >> build-side code as follows: we could add a new phase to certain build >> systems (or possibly gnu-build-system) that scans each output for >> UTF-16/32 encoded store references that are never referenced in UTF-8. >> If such references exist, a file with an unobtrusive name would be added >> to that output containing those references encoded in UTF-8. This would >> enable our daemon's existing reference scanner to find all of the >> references. > > I agree that that would be nice. As a first step, even just detecting > problems like that and erroring out would be okay - in order to find them > in the first place. Right now, it's difficult to detect and so also difficult > to say how wide-spread the problem is. If the problem is wide-spread enough > my tune could change very quickly. Sure, it would be useful to have more data on what packages are currently affected by this issue. Regards, Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 14:46:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15459219377132 (code B ref 33848); Thu, 27 Dec 2018 14:46:01 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 14:45:37 +0000 Received: from localhost ([127.0.0.1]:38283 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcWun-0001qy-A7 for submit@debbugs.gnu.org; Thu, 27 Dec 2018 09:45:37 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:51388) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcWul-0001qn-4y for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 09:45:35 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 2AE99140C; Thu, 27 Dec 2018 15:45:34 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5f68Bm7pGLpS; Thu, 27 Dec 2018 15:45:33 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 49669B27; Thu, 27 Dec 2018 15:45:33 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 27 Dec 2018 15:45:32 +0100 In-Reply-To: <87tvizvzgk.fsf@netris.org> (Mark H. Weaver's message of "Thu, 27 Dec 2018 09:03:12 -0500") Message-ID: <87o9979gfn.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hello, Mark H Weaver skribis: > Pierre Neidhardt writes: > >>> : > Store file names are always ASCII so problems arise when they are s= tored >>> : > as UTF-16 or UTF-32/UCS-4. >>> :=20 >>> : I understand that most programs stick to ASCII filenames, but what ab= out the odd >>> : one using non-English, special characters? >>>=20 >>> That=E2=80=99s a separate debate. :-) Essentially this restriction on= store >>> file names has always been there in Guix (and Nix before that). If we >>> were to change it, that would raise compatibility issues. >> >> But what happens if we attempt to store "=C3=A1" in the store? > > Indeed. Although we might restrict the immediate entries within > /gnu/store to ASCII characters, file names deeper within those > directories may have non-ASCII characters. More generally, store > references may occur within larger strings which might include non-ASCII > characters. Right. For example =E2=80=98nss-certs=E2=80=99 contains non-ASCII, UTF-8-e= ncoded file names. For =E2=80=9Ctop-level=E2=80=9D store file names, the restriction is enforc= ed by =E2=80=98checkStoreName=E2=80=99 in libstore/store-api.cc. Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 15:03:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.15459229509822 (code B ref 33848); Thu, 27 Dec 2018 15:03:01 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 15:02:30 +0000 Received: from localhost ([127.0.0.1]:39442 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcXB7-0002YM-RC for submit@debbugs.gnu.org; Thu, 27 Dec 2018 10:02:30 -0500 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:46739) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcXB6-0002YE-Qr for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 10:02:29 -0500 X-Originating-IP: 78.199.129.170 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id CD9721BF206; Thu, 27 Dec 2018 15:02:26 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87o9979gfn.fsf@gnu.org> Date: Thu, 27 Dec 2018 16:02:23 +0100 Message-ID: <87tvizgghs.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Just to be sure I understand: non-toplevel, non-ASCII file names will not be scanned properly, right? =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwk6X8ACgkQm9z0l6S7 zH/cXQgAnsjU66YtPvV2m1Mu2mRYRZfOFYicrKrjPiUkHKeQDY+CnBfPZQFNbMSL EICwx7JBZ86X48q0jJgJQ2PggQ7L17/4IhLKgL1brVTkuMIWyYXAj6hDF7qJ/ZK1 mkUve432gbMkavmBWSqddENa38T/XdUxij3SxeYyDp0YjhRjbeLddZH2eIji+B8m qavVmR/wsyTb0u8+xdScTGaB5QoOiYKiE58g02lrZTF2PUkNXG58LhbmPwl1hpfs 0abt5p4k/wvY5d9V8baXLlW9s4NQVE3/vpiE+ycQqPj90iUIummb83trbAshfUHH whUGRZvYwnWAXVebivC8jWmR2UQdog== =3pew -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 16:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154592735916728 (code B ref 33848); Thu, 27 Dec 2018 16:16:02 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 16:15:59 +0000 Received: from localhost ([127.0.0.1]:39472 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcYKF-0004Lk-BQ for submit@debbugs.gnu.org; Thu, 27 Dec 2018 11:15:59 -0500 Received: from relay10.mail.gandi.net ([217.70.178.230]:43167) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcYKE-0004Lc-3q for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 11:15:58 -0500 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 97CC2240004; Thu, 27 Dec 2018 16:15:55 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87tvizgghs.fsf@ambrevar.xyz> Date: Thu, 27 Dec 2018 17:15:51 +0100 Message-ID: <87r2e3gd3c.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Danny Milosavljevic writes: > In 1992, UTF-8 was invented. Subsequently, most of the Internet, > all new GNU Linux distributions etc, all UNIX GUI frameworks, Subversion > etc standardized on UTF-8, with the eventual goal of standardizing all > network transfer and storage to UTF-8. I think that by now the outliers > are the ones who need to change, otherwise these senseless encoding > conversions will never cease. It's not like different encodings allow for > better expression of writings or anything useful to the end user. >=20 > As a distribution we can't force upstream to change, but just filing > bug reports upstream would make us see where they stand on this. I agree with this. Reporting upstream should be a first step. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwk+rcACgkQm9z0l6S7 zH9gUgf+Nns9ka7lqwHLI14NoJfKk77nUli0A0ANr+I8yQqyll3u0KdiqulNStOc KnJtLQYny/co27/mMIMllL8im2pwoVhZ6hUxvwyp1AetR4CW5ArPqma2aFKpDFKx d+T5W1ZUA/fwyB3S1hc3qVIVOzxAHSKQp/Ik/tb++ZDmoHCEg5qlAFxJovlcsCPU nOlee9bqMXfweqZhckl+97xXmK9mJ3tZ3ijZKQ/ceBmvJvcf7t+XEOSOQQ3FQxsq YlkUh0jB39NrSTH/HbLxRzPUaihuwZRCEXJu0c29E6S8u+MmXHF04wdH9TXeHoWB BEAnq4txR6tjKKMEjpDAAeCJfHqnOw== =QuoM -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 17:04:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154593020321142 (code B ref 33848); Thu, 27 Dec 2018 17:04:01 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 17:03:23 +0000 Received: from localhost ([127.0.0.1]:39496 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcZ44-0005Us-VS for submit@debbugs.gnu.org; Thu, 27 Dec 2018 12:03:23 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:52386) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcZ3y-0005Uc-6m for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 12:03:18 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 28B921FA; Thu, 27 Dec 2018 18:03:13 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jrtGguFx35Uq; Thu, 27 Dec 2018 18:03:12 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 50E12170; Thu, 27 Dec 2018 18:03:12 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 27 Dec 2018 18:03:11 +0100 In-Reply-To: <87tvizgghs.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Thu, 27 Dec 2018 16:02:23 +0100") Message-ID: <87k1juaomo.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Pierre Neidhardt skribis: > Just to be sure I understand: non-toplevel, non-ASCII file names will > not be scanned properly, right? Every file in the store is properly scanned for references. It=E2=80=99s j= ust that users cannot create top-level items with a non-ASCII file name. I hope this clarifies things! Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 18:58:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154593707332239 (code B ref 33848); Thu, 27 Dec 2018 18:58:01 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 18:57:53 +0000 Received: from localhost ([127.0.0.1]:39541 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcaqr-0008Nq-C9 for submit@debbugs.gnu.org; Thu, 27 Dec 2018 13:57:53 -0500 Received: from relay12.mail.gandi.net ([217.70.178.232]:52773) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcaqm-0008Nd-Iv for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 13:57:48 -0500 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay12.mail.gandi.net (Postfix) with ESMTPSA id 508F2200003; Thu, 27 Dec 2018 18:57:41 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87k1juaomo.fsf@gnu.org> Date: Thu, 27 Dec 2018 19:57:41 +0100 Message-ID: <87muoqhk62.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > Every file in the store is properly scanned for references. It=E2=80=99s= just > that users cannot create top-level items with a non-ASCII file name. So if '/gnu/store/...-foo/=C3=A1' is stored as UTF-8 in a binary, then it w= ill be found? Is it because the filesystem encoding is also UTF-8 and Guix scans = over byte arrays? Sorry for dragging on this, I guess I should look at the code at this point= but I have very little time these days. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwlIKUACgkQm9z0l6S7 zH+Efgf/TSVtek2CEmPI+/oF8lD2xe6oSUhog4zvhirSXvsrDvLpY/R7i8lqjhw0 ZSqU/yqkPiz/b/sFZsWOdyjWUsniVjcOjujuy7tzwZynifj0RF9sCCnjJcM3j8Dm 3ioKwO5ppyAZBQRwt+UbjBoOC9NNyT3oDrjs9DWsWEL9cXBvkBbzoKzh/9kH5aaP vY1HoCx7mVuHeRsmKDR+YnaclArbux5jAseNCnWszqUJjkFvuDgGlmup5supsO3H t+tFPlVeeEKu414jynZUktyKrJZdRplgmpfqBpuB+6TlQTCG9AW4gqvjoqrgfGbR VrM1kAU01xtEILl9Zx7C2wRE2+RAtg== =oyjv -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 21:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154594768224694 (code B ref 33848); Thu, 27 Dec 2018 21:55:02 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 21:54:42 +0000 Received: from localhost ([127.0.0.1]:39598 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcdc2-0006QE-K9 for submit@debbugs.gnu.org; Thu, 27 Dec 2018 16:54:42 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:54502) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcdbz-0006Q4-ER for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 16:54:39 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 4C8EC1665; Thu, 27 Dec 2018 22:54:38 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1N__u0rUgLmU; Thu, 27 Dec 2018 22:54:37 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 522671639; Thu, 27 Dec 2018 22:54:37 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 27 Dec 2018 22:54:36 +0100 In-Reply-To: <87muoqhk62.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Thu, 27 Dec 2018 19:57:41 +0100") Message-ID: <87zhsq8wkj.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Pierre Neidhardt skribis: >> Every file in the store is properly scanned for references. It=E2=80=99= s just >> that users cannot create top-level items with a non-ASCII file name. > > So if '/gnu/store/...-foo/=C3=A1' is stored as UTF-8 in a binary, then it= will be > found? Is it because the filesystem encoding is also UTF-8 and Guix scan= s over > byte arrays? The reference scanner, currently written in C++, traverses whole directory trees. Being C++ it treats file names as byte arrays so it doesn=E2=80=99t matter what the file name encoding is. Note also that the reference scanner only looks for =E2=80=9Cxyz=E2=80=A6-f= oo=E2=80=9D; what comes before and after doesn=E2=80=99t matter. So for example if you have =E2=80=9C/gnu/store/xyz=E2=80=A6-foo/=C3=A0=E2=80=9D, what=E2=80=99s import= ant is the =E2=80=9Cxyz=E2=80=A6-foo=E2=80=9D bit. This is all happening in libstore/references.cc (which is surprisingly small) and in (guix build graft) for the grafting part, which Mark wrote a while back. HTH, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 22:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154594835325786 (code B ref 33848); Thu, 27 Dec 2018 22:06:02 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 22:05:53 +0000 Received: from localhost ([127.0.0.1]:39601 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcdmq-0006hq-OK for submit@debbugs.gnu.org; Thu, 27 Dec 2018 17:05:52 -0500 Received: from relay10.mail.gandi.net ([217.70.178.230]:53759) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcdmp-0006hh-6j for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 17:05:51 -0500 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay10.mail.gandi.net (Postfix) with ESMTPSA id 301A8240007; Thu, 27 Dec 2018 22:05:47 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87zhsq8wkj.fsf@gnu.org> Date: Thu, 27 Dec 2018 23:05:44 +0100 Message-ID: <87d0pmhbgn.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > The reference scanner, currently written in C++, traverses whole > directory trees. Being C++ it treats file names as byte arrays so it > doesn=E2=80=99t matter what the file name encoding is. But what matters then is that the filename encodings on the filesystem and = in the binary match, right? > Note also that the reference scanner only looks for =E2=80=9Cxyz=E2=80=A6= -foo=E2=80=9D; what > comes before and after doesn=E2=80=99t matter. So for example if you have > =E2=80=9C/gnu/store/xyz=E2=80=A6-foo/=C3=A0=E2=80=9D, what=E2=80=99s impo= rtant is the =E2=80=9Cxyz=E2=80=A6-foo=E2=80=9D bit. OK, makes sense, then my main worry is just moot :) =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwlTLgACgkQm9z0l6S7 zH/S3Qf/W9Oy7e1p3LkCiKM2l9t7jW3TezaUlHflLGmVd1zdiaTq/aeLdTfY2r+i +/aEweAHmQGD1oHWmSbnDMyBOQalzBNAQi8dg+oOSVNiMASWk+aHCj5OohE1mxrd dSLwTxk0a4BIM03GbEc/qFtI2nOZEJGjphkHJGjSKHB/5gzilsLVXRjajYWXVh/P qVaIH3xJzjA4zIyg711PQTKMqB8qIAKpr0OKA23vpZ1FaRKoNY5NRx/g1wQpFTfi 4ZdwgJwPwh3bDkU61C5mPiHcVARi8X3M65h96Aj+9RX9TCZ/+3oQTWexKTv2o0xH ZfSYbHpvtAb8xkg9sKLaN47TFoM2Zg== =djyT -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 27 Dec 2018 23:00:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154595155630525 (code B ref 33848); Thu, 27 Dec 2018 23:00:01 +0000 Received: (at 33848) by debbugs.gnu.org; 27 Dec 2018 22:59:16 +0000 Received: from localhost ([127.0.0.1]:39612 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcecW-0007wH-94 for submit@debbugs.gnu.org; Thu, 27 Dec 2018 17:59:16 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:54844) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcecV-0007wA-4M for 33848@debbugs.gnu.org; Thu, 27 Dec 2018 17:59:15 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 127AEE16; Thu, 27 Dec 2018 23:59:14 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mkus5lLJDbeT; Thu, 27 Dec 2018 23:59:13 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id DBAD91CF; Thu, 27 Dec 2018 23:59:12 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 27 Dec 2018 23:59:12 +0100 In-Reply-To: <87d0pmhbgn.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Thu, 27 Dec 2018 23:05:44 +0100") Message-ID: <87r2e28tkv.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Pierre Neidhardt skribis: >> The reference scanner, currently written in C++, traverses whole >> directory trees. Being C++ it treats file names as byte arrays so it >> doesn=E2=80=99t matter what the file name encoding is. > > But what matters then is that the filename encodings on the filesystem an= d in the > binary match, right? I=E2=80=99m not sure what you call =E2=80=9Cthe binary=E2=80=9D. Do you me= an the nar? Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 28 Dec 2018 07:48:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.154598326122571 (code B ref 33848); Fri, 28 Dec 2018 07:48:01 +0000 Received: (at 33848) by debbugs.gnu.org; 28 Dec 2018 07:47:41 +0000 Received: from localhost ([127.0.0.1]:39690 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcmrt-0005rz-Js for submit@debbugs.gnu.org; Fri, 28 Dec 2018 02:47:41 -0500 Received: from relay2-d.mail.gandi.net ([217.70.183.194]:51625) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gcmrr-0005rr-OW for 33848@debbugs.gnu.org; Fri, 28 Dec 2018 02:47:40 -0500 X-Originating-IP: 78.199.129.170 Received: from mimimi (moi44-1-78-199-129-170.fbx.proxad.net [78.199.129.170]) (Authenticated sender: mail@ambrevar.xyz) by relay2-d.mail.gandi.net (Postfix) with ESMTPSA id A7DFB40002; Fri, 28 Dec 2018 07:47:37 +0000 (UTC) References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> User-agent: mu4e 1.0; emacs 26.1 From: Pierre Neidhardt In-reply-to: <87r2e28tkv.fsf@gnu.org> Date: Fri, 28 Dec 2018 08:47:33 +0100 Message-ID: <874laygkiy.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > I=E2=80=99m not sure what you call =E2=80=9Cthe binary=E2=80=9D. Do you = mean the nar? No, in this case I referred to "/bin/next" in sbcl-next. So any file in th= e nar passed to the reference scanner. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAlwl1RUACgkQm9z0l6S7 zH/vtAgAq1KOgRvdEXvdvjPy//H0f/nhzu9Z1qYhwxV9tGOoeBcHiG/vUo6mc7/h Rkht5zFI35d6L1o09XB6oJKfsVSi0P5VE0APDZzlc3y1pifNcjFekBHUuiva5jR4 zCYeOsXc8AKZSSC7Lf+OPYb2CTFV2nvoANo9dhl5OUPZHZ0B7GcCyHZDtnk4jtiZ D3p/FjlLcmZ9hom0CbvHjffkDlq/0nmO22kqY5tuKqY/p6UDyJDXROeMdYi2u2lb hM3kgrw5j7MQrQthmt00PUmqDRPoudjnjc4btoGUjUIjZlVgfZidv1qygC4Bxi/C 9LZ3luopQVjPL/m2is4gwUp2DctYjg== =3PX/ -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 30 Mar 2021 10:16:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161709933226763 (code B ref 33848); Tue, 30 Mar 2021 10:16:01 +0000 Received: (at 33848) by debbugs.gnu.org; 30 Mar 2021 10:15:32 +0000 Received: from localhost ([127.0.0.1]:50447 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRBPI-0006xb-Au for submit@debbugs.gnu.org; Tue, 30 Mar 2021 06:15:32 -0400 Received: from relay6-d.mail.gandi.net ([217.70.183.198]:37487) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRBPE-0006xJ-8e for 33848@debbugs.gnu.org; Tue, 30 Mar 2021 06:15:30 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 19E22C000D; Tue, 30 Mar 2021 10:15:20 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <874laygkiy.fsf@ambrevar.xyz> References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> Date: Tue, 30 Mar 2021 12:15:20 +0200 Message-ID: <87lfa5eymf.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Many grafting issues with Nyxt have been reported recently: https://github.com/atlas-engineer/nyxt/issues/1257 Seems that glib is being grafted, which breaks cl-cffi-gtk and thus Nyxt. Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.198 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.198 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Many grafting issues with Nyxt have been reported recently: https://github.com/atlas-engineer/nyxt/issues/1257 Seems that glib is being grafted, which breaks cl-cffi-gtk and thus Nyxt. Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.198 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.198 listed in list.dnswl.org] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Many grafting issues with Nyxt have been reported recently: https://github.com/atlas-engineer/nyxt/issues/1257 Seems that glib is being grafted, which breaks cl-cffi-gtk and thus Nyxt. Is there a way to disable grafting for a given package? This would at least be a local workaround for Nyxt. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBi+jgSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/zX4IAKAVBaFbpQMrdhbgmIZ8p3izeO0ZKSrB IWYn8cys+gZ1nLS7zIRwqY+WVQjRPSP7pOTlP72hcXfEEC3hLg2h/wcKEXHc1Tkl oTUfKfoJ5pZ6lfKRan53BjCwPQwTk66HB+0IdGXYXn4NISY4Fn3oep9dYVC4fDv6 8OTlbl1tIFrydEnr4ZKxEX92fJbAkeP4gzCUzfjkcScQi0G6dQW3YB0nNjG2MFsm SENUdvrb+tD+ltxask7Iv/cPlcRB2Igz72vVEQMdunGRoOymi2X47kQGwPO1Z9Y2 PhKe8vUWu5a2uC6daqXGHkwU/fUmOAfSvV8aghPgey9PPxKJVq3qxo0= =86zr -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 30 Mar 2021 20:10:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161713496213067 (code B ref 33848); Tue, 30 Mar 2021 20:10:02 +0000 Received: (at 33848) by debbugs.gnu.org; 30 Mar 2021 20:09:22 +0000 Received: from localhost ([127.0.0.1]:52930 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRKfy-0003Og-Dg for submit@debbugs.gnu.org; Tue, 30 Mar 2021 16:09:22 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44392) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRKfw-0003OT-JU for 33848@debbugs.gnu.org; Tue, 30 Mar 2021 16:09:20 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:45865) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRKfq-0004Av-KV; Tue, 30 Mar 2021 16:09:14 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=58270 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lRKfp-0007AJ-Ef; Tue, 30 Mar 2021 16:09:13 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 10 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 30 Mar 2021 22:09:10 +0200 In-Reply-To: <87lfa5eymf.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Tue, 30 Mar 2021 12:15:20 +0200") Message-ID: <87tuoscsk9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, Pierre Neidhardt skribis: > Many grafting issues with Nyxt have been reported recently: > > https://github.com/atlas-engineer/nyxt/issues/1257 > > Seems that glib is being grafted, which breaks cl-cffi-gtk and thus > Nyxt. > > [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [209.51.188.92 listed in list.dnswl.org] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) Hi, Pierre Neidhardt skribis: > Many grafting issues with Nyxt have been reported recently: > > https://github.com/atlas-engineer/nyxt/issues/1257 > > Seems that glib is being grafted, which breaks cl-cffi-gtk and thus > Nyxt. > > Is there a way to disable grafting for a given package? This would at > least be a local workaround for Nyxt. No. I provided guidance in 2018 on how to address this issue: https://issues.guix.gnu.org/33848 Did anyone talk to the SBCL folks? Please, let=E2=80=99s address this rath= er than look for workarounds. Thanks, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 31 Mar 2021 07:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161717465630986 (code B ref 33848); Wed, 31 Mar 2021 07:11:02 +0000 Received: (at 33848) by debbugs.gnu.org; 31 Mar 2021 07:10:56 +0000 Received: from localhost ([127.0.0.1]:53384 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRV0B-00083i-Tc for submit@debbugs.gnu.org; Wed, 31 Mar 2021 03:10:56 -0400 Received: from relay4-d.mail.gandi.net ([217.70.183.196]:54969) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRV0A-00083T-12 for 33848@debbugs.gnu.org; Wed, 31 Mar 2021 03:10:54 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id D6BB1E0006; Wed, 31 Mar 2021 07:10:47 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87tuoscsk9.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> Date: Wed, 31 Mar 2021 09:10:47 +0200 Message-ID: <87sg4bdci0.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > I provided guidance in 2018 on how to address this issue: > > https://issues.guix.gnu.org/33848 Looks like you are linking to this very same thread :) > Did anyone talk to the SBCL folks? Please, =?UTF-8?Q?let=E2=80=99s?= address this rather > than look for workarounds. Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.196 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.196 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: > I provided guidance in 2018 on how to address this issue: > > https://issues.guix.gnu.org/33848 Looks like you are linking to this very same thread :) > Did anyone talk to the SBCL folks? Please, =?UTF-8?Q?let=E2=80=99s?= address this rather > than look for workarounds. Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.196 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.196 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > I provided guidance in 2018 on how to address this issue: > > https://issues.guix.gnu.org/33848 Looks like you are linking to this very same thread :) > Did anyone talk to the SBCL folks? Please, let=E2=80=99s address this ra= ther > than look for workarounds. I've just asked them for advice: https://bugs.launchpad.net/sbcl/+bug/1922011 =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBkIHcSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/fpUIAKTjnfzzgCtdSXezcntyxl2O9z30+gVc i5tdkmx+rPKPhj0hThw/GhWWyJrKXMuxh9QB5ZfHYQhEImUBlmnmasIZRcX5O15D i5jejNA5rnaCIsxmszlCd5Zv/LT5JBbN6rCONX9L1baYnc0NZ+drar39aULASIWI /cDxcl1eF5+dJz4uXdPoVfDyZVj5u8PqVbP6HJ2wkOpoYjj5pA1w7vgbUNPkSRva ZnjRvXoWHxcMxzzR5/bxEFWgc3Kh6TupvMUOAo0AOK1kdjfmul+jSadcSfUOVe2L TYnwIZGr0zX8nKYPM3g8ZNVmTPMgWdKjZJRPeEZ88+ur9BWyeb9aa3g= =TqV5 -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 31 Mar 2021 16:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Guillaume Le Vaillant Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16172071774419 (code B ref 33848); Wed, 31 Mar 2021 16:13:01 +0000 Received: (at 33848) by debbugs.gnu.org; 31 Mar 2021 16:12:57 +0000 Received: from localhost ([127.0.0.1]:55304 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRdSi-00019C-Ax for submit@debbugs.gnu.org; Wed, 31 Mar 2021 12:12:56 -0400 Received: from relay7-d.mail.gandi.net ([217.70.183.200]:41569) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRdSh-00018w-8A for 33848@debbugs.gnu.org; Wed, 31 Mar 2021 12:12:55 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay7-d.mail.gandi.net (Postfix) with ESMTPSA id AD07620003; Wed, 31 Mar 2021 16:12:48 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87tuoscsk9.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> Date: Wed, 31 Mar 2021 18:12:48 +0200 Message-ID: <87im57b8u7.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: A brief discussion has ensued on SBCL bugtracker: - It's unlikely that SBCL will change its internal string representation. - The main recommendation for an easy fix without updating the scanner is that we tweaked our build system to dump the store r [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.200 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [217.70.183.200 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: A brief discussion has ensued on SBCL bugtracker: - It's unlikely that SBCL will change its internal string representation. - The main recommendation for an easy fix without updating the scanner is that we tweaked our build system to dump the store r [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.200 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [217.70.183.200 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable A brief discussion has ensued on SBCL bugtracker: =2D It's unlikely that SBCL will change its internal string representation. =2D The main recommendation for an easy fix without updating the scanner is that we tweaked our build system to dump the store reference to a separate ASCII file. Thoughts? Guillaume? =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBkn4ASHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/CfQH/RoPIrdxAYdX2vbm+UqjYzGXZUbB4rCf QWfiUDvMOZF4afaGkXVBCqR24/UsuxlAz6doT8mtG9ADPtIkgCLWESF7XvN0OWmi kcUOh5UCDnE+A/QOIVwpLCvUVVtdN09XPQysiYVAxgvDdzwlEVdSXvZsGYe/YoLs 0PKIW/ddj0SXk/rXvAAm+EU8519TQ6s9oxEVuG/D8ZN6DpjZoQk33kQ/oCt2kWfG SWc15JHqqUotQvBsBgBL86Bu69S61KGOy8aDwQ5MAwbaS6Sic8T6KbUFEh8b1wsg 8xqTIzLfq2mpZMJKrBvZ+LhjUV3sFBdsYnBR1SeB8HS4RzUeKdbUsRk= =p0DJ -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 31 Mar 2021 20:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Guillaume Le Vaillant , Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161722335430073 (code B ref 33848); Wed, 31 Mar 2021 20:43:02 +0000 Received: (at 33848) by debbugs.gnu.org; 31 Mar 2021 20:42:34 +0000 Received: from localhost ([127.0.0.1]:55640 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRhfd-0007oz-VY for submit@debbugs.gnu.org; Wed, 31 Mar 2021 16:42:34 -0400 Received: from eggs.gnu.org ([209.51.188.92]:60782) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRhfd-0007om-2u for 33848@debbugs.gnu.org; Wed, 31 Mar 2021 16:42:33 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40181) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRhfW-0001fI-33; Wed, 31 Mar 2021 16:42:26 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=55542 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lRhfV-00089i-Hu; Wed, 31 Mar 2021 16:42:25 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 11 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 31 Mar 2021 22:42:24 +0200 In-Reply-To: <87im57b8u7.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Wed, 31 Mar 2021 18:12:48 +0200") Message-ID: <87zgyj5a33.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, Pierre Neidhardt skribis: > A brief discussion has ensued on SBCL bugtracker: Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [209.51.188.92 listed in list.dnswl.org] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) Hi, Pierre Neidhardt skribis: > A brief discussion has ensued on SBCL bugtracker: Nice, thanks for starting the discussion! > - It's unlikely that SBCL will change its internal string > representation. Of course, I would not suggest that. What could have been nice is if there=E2=80=99s a way to mark specific stri= ngs as being ASCII, or if there=E2=80=99s a =E2=80=9Cbyte vector=E2=80=9D data = type compatible with strings, for instance. > - The main recommendation for an easy fix without updating the scanner > is that we tweaked our build system to dump the store reference to a > separate ASCII file. That=E2=80=99s a good idea: simple and efficient. We do that for the initr= d, for instance (commit b36e06c2b0889f1d0f939465589d36887ff24d33). This could be done in =E2=80=98asdf-build-system/sbcl=E2=80=99 I suppose. = I can see two drawbacks: 1. Some packages like =E2=80=98nyxt=E2=80=99 don=E2=80=99t use it, so we= =E2=80=99d have to duplicate the phase there. 2. It may be coarse-grain compared to scanning binaries for references (for example, we might retain references to build-only tools, such as libraries used only for tests). That=E2=80=99s probably acceptable though, and certainly better than the st= atus quo. Thanks, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 31 Mar 2021 20:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Guillaume Le Vaillant , Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161722426331380 (code B ref 33848); Wed, 31 Mar 2021 20:58:02 +0000 Received: (at 33848) by debbugs.gnu.org; 31 Mar 2021 20:57:43 +0000 Received: from localhost ([127.0.0.1]:55644 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRhuJ-0008A4-Fx for submit@debbugs.gnu.org; Wed, 31 Mar 2021 16:57:43 -0400 Received: from relay11.mail.gandi.net ([217.70.178.231]:36663) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRhuI-00089q-4G for 33848@debbugs.gnu.org; Wed, 31 Mar 2021 16:57:42 -0400 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay11.mail.gandi.net (Postfix) with ESMTPSA id C35CE100003; Wed, 31 Mar 2021 20:57:34 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87zgyj5a33.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87zgyj5a33.fsf@gnu.org> Date: Wed, 31 Mar 2021 22:57:33 +0200 Message-ID: <874kgravnm.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Ludo, > What could have been nice is if =?UTF-8?Q?there=E2=80=99s?= a way to mark specific strings > as being ASCII, or if =?UTF-8?Q?there=E2=80=99s?= a =?UTF-8?Q?=E2=80=9Cbyte_?= =?UTF-8?Q?vector=E2=80=9D?= data type compatible with > strings, for instance. It does and it could work, but according to upstream it's just simpler to dump deps in a separate file. Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.178.231 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [217.70.178.231 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Ludo, > What could have been nice is if =?UTF-8?Q?there=E2=80=99s?= a way to mark specific strings > as being ASCII, or if =?UTF-8?Q?there=E2=80=99s?= a =?UTF-8?Q?=E2=80=9Cbyte_?= =?UTF-8?Q?vector=E2=80=9D?= data type compatible with > strings, for instance. It does and it could work, but according to upstream it's just simpler to dump deps in a separate file. Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.178.231 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [217.70.178.231 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, > What could have been nice is if there=E2=80=99s a way to mark specific st= rings > as being ASCII, or if there=E2=80=99s a =E2=80=9Cbyte vector=E2=80=9D dat= a type compatible with > strings, for instance. It does and it could work, but according to upstream it's just simpler to dump deps in a separate file. > 1. Some packages like =E2=80=98nyxt=E2=80=99 don=E2=80=99t use it, so w= e=E2=80=99d have to duplicate > the phase there. In the case of Nyxt, the reason it does not use it is because the asdf-build-system/sbcl binary production has some drawbacks. I could work on overhauling the build system to fix the uncanny behaviour, then Nyxt would use it. > 2. It may be coarse-grain compared to scanning binaries for references > (for example, we might retain references to build-only tools, such > as libraries used only for tests). The build system could easily leave out Lisp native-inputs, no? Cheers! =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBk4j0SHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/DwQIAI18KsfbdGJk3Ok5nAtdF5BzhN1QdWuP NjnRbL+kMtlvEVoYXEkh6a7m8h6AMi4mKZ0e2ju6eEIYfj/TUoBgE+vz/ksKySuK hD7c8x1l4M1v4qFfLUPkPyNZ1EQ0TAHhcVtNFl1QjWRHZPsSSBDmi5jW/zVEh6c1 6nY3q0M+ey03Rox3+RDd3kgNLjNA//G7gmF5xl7T9CLZ1jmpQ7VbTHGht52ina0F X2ryBD2CPxCdivyap67skzAaQMLTZUB8ZXpnxRsUb1ZxG7fkRygXxoIBM84DnIsH s53HWRx3AUf5eCJ+SKrQElhERh31oStF3nMNbU9L+PdReKELsSow6B0= =OGKR -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 06:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Guillaume Le Vaillant Cc: 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161725710918182 (code B ref 33848); Thu, 01 Apr 2021 06:06:02 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 06:05:09 +0000 Received: from localhost ([127.0.0.1]:55936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRqS5-0004jC-FY for submit@debbugs.gnu.org; Thu, 01 Apr 2021 02:05:09 -0400 Received: from world.peace.net ([64.112.178.59]:60272) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRqS3-0004ia-NR for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 02:05:08 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lRqRx-0001mg-8r; Thu, 01 Apr 2021 02:05:01 -0400 From: Mark H Weaver In-Reply-To: <87im57b8u7.fsf@ambrevar.xyz> References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> Date: Thu, 01 Apr 2021 02:03:06 -0400 Message-ID: <87czvebky2.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Pierre Neidhardt writes: > - The main recommendation for an easy fix without updating the scanner > is that we tweaked our build system to dump the store reference to a > separate [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Pierre Neidhardt writes: > - The main recommendation for an easy fix without updating the scanner > is that we tweaked our build system to dump the store reference to a > separate ASCII file. Sounds good. I made a similar proposal in Dec 2018, earlier in this thread . I wrote: If you don't want to change the daemon, it could be worked around in our build-side code as follows: we could add a new phase to certain build systems (or possibly gnu-build-system) that scans each output for UTF-16/32 encoded store references that are never referenced in UTF-8. If such references exist, a file with an unobtrusive name would be added to that output containing those references encoded in UTF-8. This would enable our daemon's existing reference scanner to find all of the references. Our grafting code would then need to be extended to recognize and transform store references encoded in UTF-16/32 as well as UTF-8. Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 07:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Guillaume Le Vaillant Cc: 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161726120424356 (code B ref 33848); Thu, 01 Apr 2021 07:14:01 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 07:13:24 +0000 Received: from localhost ([127.0.0.1]:55996 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRrW8-0006Km-Gu for submit@debbugs.gnu.org; Thu, 01 Apr 2021 03:13:24 -0400 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:49897) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRrW7-0006KY-3J for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 03:13:23 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id D275E1C000E; Thu, 1 Apr 2021 07:13:15 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87czvebky2.fsf@netris.org> References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> Date: Thu, 01 Apr 2021 09:13:15 +0200 Message-ID: <87sg4aa35g.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Thank you for the reminder, Mark, I had forgotten about your suggestion. If we are going for a SBCL-specific solution, I believe that all references are stored in plain text files at some points (the .asd files and the .lisp files) which are often encoded in ASCII / UTF-8, [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.197 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.183.197 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Thank you for the reminder, Mark, I had forgotten about your suggestion. If we are going for a SBCL-specific solution, I believe that all references are stored in plain text files at some points (the .asd files and the .lisp files) which are often encoded in ASCII / UTF-8, [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.197 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.183.197 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain Thank you for the reminder, Mark, I had forgotten about your suggestion. If we are going for a SBCL-specific solution, I believe that all references are stored in plain text files at some points (the .asd files and the .lisp files) which are often encoded in ASCII / UTF-8, so searching these files without having to deal with their encoding might be enough. But of course this is less general and more brittle. Mark, Guillaume, would you have time to give this a try? I'm a bit busy at the moment. Let me know if you can't work on it, I'll try to find time to work on a patch. Cheers! -- Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBlcosSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/ieQIAJVk4okboi89JzT/9KpPbx+HNgf3ks7v FfVyWnUCJeq5NBDKVGWmM9LZ230/rXPU63n7y7ZTYiauSL0UTap3tXtygWfgtDM8 zHWS4uc5KBybz3RoNNH6rMiPp5bGLbs15Y1NatjltpUFEiQp/EOw15my3ZjVxpcY FaUDwQSDaBRmemExSqT03E4vo7qrI7tLNfAmcW3iE3UKDECgs93WmcliUf0nuQL2 YmFHefgOdSScdNOZBO80yIyDmrkWe7C51c22y4pHu86sNLaJ3mVA1N+kNAtjG/qv pUw9E34Q1mSZY/1jbraa9f0J11PR6eJNU43dQdmOUeA/QdqBDCoOXb4= =uiP6 -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 07:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161726383729060 (code B ref 33848); Thu, 01 Apr 2021 07:58:02 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 07:57:17 +0000 Received: from localhost ([127.0.0.1]:56069 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRsCb-0007Yd-8x for submit@debbugs.gnu.org; Thu, 01 Apr 2021 03:57:17 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37332) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRsCZ-0007YP-Sw for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 03:57:16 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49696) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRsCT-0005Zj-PI; Thu, 01 Apr 2021 03:57:09 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=57342 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lRsCS-0006eW-HF; Thu, 01 Apr 2021 03:57:09 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 01 Apr 2021 09:57:07 +0200 In-Reply-To: <87czvebky2.fsf@netris.org> (Mark H. Weaver's message of "Thu, 01 Apr 2021 02:03:06 -0400") Message-ID: <87eefu30a4.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, Mark H Weaver skribis: > Pierre Neidhardt writes: >> - The main recommendation for an easy fix without updating the scanner >> is that we tweaked our build system to dump the store reference to a >> sepa [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [209.51.188.92 listed in list.dnswl.org] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) Hi, Mark H Weaver skribis: > Pierre Neidhardt writes: >> - The main recommendation for an easy fix without updating the scanner >> is that we tweaked our build system to dump the store reference to a >> separate ASCII file. > > Sounds good. I made a similar proposal in Dec 2018, earlier in this > thread . I wrote: > > If you don't want to change the daemon, it could be worked around in our > build-side code as follows: we could add a new phase to certain build > systems (or possibly gnu-build-system) that scans each output for > UTF-16/32 encoded store references that are never referenced in UTF-8. > If such references exist, a file with an unobtrusive name would be added > to that output containing those references encoded in UTF-8. This would > enable our daemon's existing reference scanner to find all of the > references. > > Our grafting code would then need to be extended to recognize and > transform store references encoded in UTF-16/32 as well as UTF-8. Oh thanks for the reminder. The separate ASCII file doesn=E2=80=99t solve it all because, as you write,= we=E2=80=99d need to change the grafting code as well. Then it might be simpler to use a =E2=80=9Cbyte vector=E2=80=9D data type f= or those strings. We=E2=80=99ll have to wait for Pierre=E2=80=99s patches to get a = better idea. :-) Thanks, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 08:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mark H Weaver Cc: Guillaume Le Vaillant , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16172669312241 (code B ref 33848); Thu, 01 Apr 2021 08:49:02 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 08:48:51 +0000 Received: from localhost ([127.0.0.1]:56266 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRt0U-0000a5-UC for submit@debbugs.gnu.org; Thu, 01 Apr 2021 04:48:51 -0400 Received: from relay7-d.mail.gandi.net ([217.70.183.200]:39747) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRt0T-0000Zl-Dr for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 04:48:49 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay7-d.mail.gandi.net (Postfix) with ESMTPSA id E79D320018; Thu, 1 Apr 2021 08:48:42 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87eefu30a4.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> Date: Thu, 01 Apr 2021 10:48:41 +0200 Message-ID: <87eefu9yqe.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi! > The separate ASCII file =?UTF-8?Q?doesn=E2=80=99t?= solve it all because, as you write, =?UTF-8?Q?we=E2=80=99d?= > need to change the grafting code as well. > > Then it might be simpler to use a =?UTF-8?Q?=E2=80=9Cbyte_?= =?UTF-8?Q?vector=E2=80=9D?= data type for those > [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.200 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [217.70.183.200 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi! > The separate ASCII file =?UTF-8?Q?doesn=E2=80=99t?= solve it all because, as you write, =?UTF-8?Q?we=E2=80=99d?= > need to change the grafting code as well. > > Then it might be simpler to use a =?UTF-8?Q?=E2=80=9Cbyte_?= =?UTF-8?Q?vector=E2=80=9D?= data type for those > [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.200 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [217.70.183.200 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi! > The separate ASCII file doesn=E2=80=99t solve it all because, as you writ= e, we=E2=80=99d > need to change the grafting code as well. > > Then it might be simpler to use a =E2=80=9Cbyte vector=E2=80=9D data type= for those > strings. Which strings and where would we use byte vectors? > We=E2=80=99ll have to wait for Pierre=E2=80=99s patches to get a better i= dea. > :-) By "Pierre's patches", you mean a patch to add a build phase that generates an file listings all references? Cheers! =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBliOkSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/NlgIAKW2OYmihOC2vAPTom6G7OEh7DU4xzU5 h+e9acIDKKjBR4PwbxTBwWJdF4Hq5i9rRNBM/Vr5OU8lN/0ikpTmh/2TCxzHPBC0 1Ln+8AJmIKaIc6VIXW24IhsT0F3CBkrbz0ch+iQYFlV3HJ8Vi/V4pa62865Y7jC0 mI4evp3puwutJjE4cPchBh6NJyF/0IGkbuS3f4E1ErrtcaFDUX3bCSsTZ29Jk/8U qEE67epxAgZYUj2OeH/464JtGWmrr8j6XQaTrmYWxeAWLDrozjDjtD0yBWxAX5Oy gcxOVbDcOXJbf4K3OLXoCj+jdIIrorf6y2nuPYqcoJ33h8XfcB1IdYk= =4p1L -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Guillaume Le Vaillant Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 09:08:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16172680584320 (code B ref 33848); Thu, 01 Apr 2021 09:08:01 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 09:07:38 +0000 Received: from localhost ([127.0.0.1]:56289 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRtIg-00017c-Fn for submit@debbugs.gnu.org; Thu, 01 Apr 2021 05:07:38 -0400 Received: from mout01.posteo.de ([185.67.36.65]:55957) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRtIe-00017M-G5 for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 05:07:37 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 34E82160060 for <33848@debbugs.gnu.org>; Thu, 1 Apr 2021 11:07:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1617268050; bh=pFX2Hdgq596McSZNUd1m2x4kbOv/dl3wNhLj0AS/cRA=; h=From:To:Cc:Subject:Date:From; b=JKb9yiqao5+HNkGlxV95zwKrHSeyw0WRcMfvZTAuyb+zX2evsRKpuvbJK39N5MNmy yaGTbrn64lEvNDzSUT/yb9DwkXzR3TGCHFpFbNMTZI0l+Z5IAZMlmDl/1RjHOr1DAm 3ytGOapAUM3OzpqAktv+nkbEasB+Nspnrdy1XIHECQR2nOU8Iy1hpOos8KD2TJ0qeA yRlxTVd4ccHn1ttAWFNQzZuUgzJfT7jxO7mRt1One4XfRqocfV6kpJi6Xi+Me8wfgn KuMS0DMngRaerWjWtRhJljy08HkyZIbJNLx00/nXosu3bEqG3Kvpyr6moYRYZyVcmM IqGa7zWXy4WXg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4F9y4c5vw3z9rxV; Thu, 1 Apr 2021 11:07:28 +0200 (CEST) References: <87r2e8jpfx.fsf@gnu.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> User-agent: mu4e 1.4.15; emacs 27.2 From: Guillaume Le Vaillant In-reply-to: <87eefu30a4.fsf@gnu.org> Date: Thu, 01 Apr 2021 11:07:23 +0200 Message-ID: <87im56l6es.fsf@yamatai> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.3 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.3 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Pierre Neidhardt skribis: > If we are going for a SBCL-specific solution, I believe that all > references are stored in plain text files at some points (the .asd files > and the .lisp files) which are often encoded in ASCII / UTF-8, so > searching these files without having to deal with their encoding might > be enough. But of course this is less general and more brittle. A store reference to a C library in a standalone Lisp binary can come either from the current package or from some dependencies (cl+ssl, cl-cffi-gtk, etc.). So we would need to scan the source code of all the Lisp dependencies recursively to get the full list of store refrences. And as Mark wrote below, with the current grafting code, this list of store references will not solve grafting for paths that are in UTF-32 or both in ASCII and UTF-32 in the binary file. Ludovic Court=C3=A8s skribis: > Mark H Weaver skribis: > >> Pierre Neidhardt writes: >>> - The main recommendation for an easy fix without updating the scanner >>> is that we tweaked our build system to dump the store reference to a >>> separate ASCII file. >> >> Sounds good. I made a similar proposal in Dec 2018, earlier in this >> thread . I wrote: >> >> If you don't want to change the daemon, it could be worked around in o= ur >> build-side code as follows: we could add a new phase to certain build >> systems (or possibly gnu-build-system) that scans each output for >> UTF-16/32 encoded store references that are never referenced in UTF-8. >> If such references exist, a file with an unobtrusive name would be add= ed >> to that output containing those references encoded in UTF-8. This wou= ld >> enable our daemon's existing reference scanner to find all of the >> references. >> >> Our grafting code would then need to be extended to recognize and >> transform store references encoded in UTF-16/32 as well as UTF-8. > > Oh thanks for the reminder. > > The separate ASCII file doesn=E2=80=99t solve it all because, as you writ= e, we=E2=80=99d > need to change the grafting code as well. > > Then it might be simpler to use a =E2=80=9Cbyte vector=E2=80=9D data type= for those > strings. We=E2=80=99ll have to wait for Pierre=E2=80=99s patches to get = a better idea. > :-) I'm not sure what you mean with the "byte vector" data type here. Could you explain what you're thinking about with a few more details? --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYGWNSw8cZ2x2QHBvc3Rl by5uZXQACgkQa+ggit8h/j8KtgEAoyvmikRXpUHbT3CLPgewYs1QE7dXwa/fSb96 bR8rM6YBAJpnLOUegNHW8lTLfJu8ats/gdkdL+TFysYRBtjIU6Sz =5hTk -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 09:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Guillaume Le Vaillant , Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16172683964913 (code B ref 33848); Thu, 01 Apr 2021 09:14:01 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 09:13:16 +0000 Received: from localhost ([127.0.0.1]:56298 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRtO8-0001HB-FA for submit@debbugs.gnu.org; Thu, 01 Apr 2021 05:13:16 -0400 Received: from relay6-d.mail.gandi.net ([217.70.183.198]:45993) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRtO6-0001Gv-Ed for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 05:13:14 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 16076C000F; Thu, 1 Apr 2021 09:13:07 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87im56l6es.fsf@yamatai> References: <87r2e8jpfx.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> Date: Thu, 01 Apr 2021 11:13:07 +0200 Message-ID: <87wntm8j18.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Guillaume! > A store reference to a C library in a standalone Lisp binary can come > either from the current package or from some dependencies > (cl+ssl, cl-cffi-gtk, etc.). So we would need to scan the source > [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.198 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.198 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Guillaume! > A store reference to a C library in a standalone Lisp binary can come > either from the current package or from some dependencies > (cl+ssl, cl-cffi-gtk, etc.). So we would need to scan the source > [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.198 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.198 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi Guillaume! > A store reference to a C library in a standalone Lisp binary can come > either from the current package or from some dependencies > (cl+ssl, cl-cffi-gtk, etc.). So we would need to scan the source > code of all the Lisp dependencies recursively to get the full list of > store refrences. I don't think there is need to scan recursively: if package A depends on B which depends on C, then A can lists the dependency on B in a file, and B can do the same for C. This way the relationship between A and C is properly stored. Am I missing something? > And as Mark wrote below, with the current grafting code, this list of > store references will not solve grafting for paths that are in UTF-32 or > both in ASCII and UTF-32 in the binary file. Indeed, and that's the core of the issue here I believe, since grafting is what breaks Nyxt in practice. Cheers! =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBljqMSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/Dy8IAJqkvs/eVUHm+1JUqFEfkMIzMMOB4vZN i6SVjs6lIlBgj33X5KhzmvYXh8VAZYSylDh/O9HUH3lhCmBueqyUUTfBDtoPm4fj snBU9J+QsDS9zl3VRvzdjyZUARAf5KxRrREYx/FpMMFhZUB9/lcczISkTpnH/5qc XxpRDdGGNbzJYmOH+KBZNCWD+wKIdXJmY9AncfwApA2ZlSfvEymKEf3ddCT8Mc2N liaE/gPFk9f+ws+iQOzwnOXcsLVtsAXXLt2Cufm5Fw8Z9Su2UR+pfaS0acdIImey xFVmnFC5OaBuStfHU0YmACpx9Im/om6Xam1BAvYEBWVd2fjdizfFizk= =s3Yo -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Guillaume Le Vaillant Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 09:54:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Mark H Weaver , Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16172707858746 (code B ref 33848); Thu, 01 Apr 2021 09:54:01 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 09:53:05 +0000 Received: from localhost ([127.0.0.1]:56369 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRu0f-0002H0-As for submit@debbugs.gnu.org; Thu, 01 Apr 2021 05:53:05 -0400 Received: from mout02.posteo.de ([185.67.36.66]:60271) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRu0d-0002GV-II for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 05:53:04 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 47B3F240101 for <33848@debbugs.gnu.org>; Thu, 1 Apr 2021 11:52:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1617270777; bh=RRXLf7jBYSTvtg9eM9+DsQDQz15Hw4vkQJGVzUWzAPI=; h=From:To:Cc:Subject:Date:From; b=OD67JY/307+iocvpzXG4GDAQtYqeE8o3GQMpirwX25V0+girw9V1PEnOOP1eS8wQI HF3AZwFc1/itjOymL35xkHDWOgCF1QaTe9wUiLHaQpnINkNacdoLYGL1hW15IMDZ8F Gwg8K0Rj1WOwiZpixrV8L77O4Iz82KAuvjLWU/zU4f0U9jomPyMFrI6tNmi95wg1Ou OJ0UKOEDvKQDJmYVD12HIlRwxsaPfk1FVT0znGVjAypAYqWxPuF27soi32NuMi2VAz 87E/oHzv7Azc1m5VTQlG4hfR6lHkgm5KP2F713fGcolRhW9xoVBfm/2IzE0u3OGme/ WDvekdL0z0y7A== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4F9z531H5lz9rxS; Thu, 1 Apr 2021 11:52:54 +0200 (CEST) References: <87r2e8jpfx.fsf@gnu.org> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> User-agent: mu4e 1.4.15; emacs 27.2 From: Guillaume Le Vaillant In-reply-to: <87wntm8j18.fsf@ambrevar.xyz> Date: Thu, 01 Apr 2021 11:52:50 +0200 Message-ID: <87a6qil4b1.fsf@yamatai> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.3 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.3 (-) --=-=-= Content-Type: text/plain Pierre Neidhardt skribis: > Hi Guillaume! > >> A store reference to a C library in a standalone Lisp binary can come >> either from the current package or from some dependencies >> (cl+ssl, cl-cffi-gtk, etc.). So we would need to scan the source >> code of all the Lisp dependencies recursively to get the full list of >> store refrences. > > I don't think there is need to scan recursively: if package A depends on > B which depends on C, then A can lists the dependency on B in a file, > and B can do the same for C. This way the relationship between A and C > is properly stored. > > Am I missing something? Oh, you say this file would be created for every Lisp package. I thought it would only be for the standalone binary case, because the "regular" asdf-build-system/sbcl used for Lisp libraries ships the sources and its make-asdf-configuration phase creates links to the required Lisp dependencies in '/gnu/store/...', so there should not be missing references. >> And as Mark wrote below, with the current grafting code, this list of >> store references will not solve grafting for paths that are in UTF-32 or >> both in ASCII and UTF-32 in the binary file. > > Indeed, and that's the core of the issue here I believe, since grafting > is what breaks Nyxt in practice. > > Cheers! I just wondered: does the grafting code take '.fasl' files into consideration? If yes, I guess a Lisp library could also end up in a strange half-grafted state if the grafting code modifies ASCII references and not UTF-32 ones... --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYGWX8g8cZ2x2QHBvc3Rl by5uZXQACgkQa+ggit8h/j+FZAD+NJBBrb4xTqeC8wYQb956Wv1WiGyWm9LTFln5 dXAIqfEA/2O1IKrJYZG/fItBPNLo2UR5PYJwolfNVIL4fwwBHLlk =lw/O -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 10:08:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Guillaume Le Vaillant Cc: Mark H Weaver , Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161727162610037 (code B ref 33848); Thu, 01 Apr 2021 10:08:02 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 10:07:06 +0000 Received: from localhost ([127.0.0.1]:56380 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRuED-0002bp-Sg for submit@debbugs.gnu.org; Thu, 01 Apr 2021 06:07:06 -0400 Received: from relay1-d.mail.gandi.net ([217.70.183.193]:21993) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRuEB-0002bI-7i for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 06:07:04 -0400 X-Originating-IP: 92.169.147.163 Received: from mimimi (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id ABF9024000D; Thu, 1 Apr 2021 10:06:55 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87a6qil4b1.fsf@yamatai> References: <87r2e8jpfx.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> Date: Thu, 01 Apr 2021 12:06:54 +0200 Message-ID: <87czvez5c1.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Guillaume Le Vaillant writes: > Oh, you say this file would be created for every Lisp package. I thought > it would only be for the standalone binary case, because the "regular" > asdf-build-system/sbcl used for Lisp libraries shi [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.193 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.193 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Guillaume Le Vaillant writes: > Oh, you say this file would be created for every Lisp package. I thought > it would only be for the standalone binary case, because the "regular" > asdf-build-system/sbcl used for Lisp libraries shi [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.193 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.193 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Guillaume Le Vaillant writes: > Oh, you say this file would be created for every Lisp package. I thought > it would only be for the standalone binary case, because the "regular" > asdf-build-system/sbcl used for Lisp libraries ships the sources and its > make-asdf-configuration phase creates links to the required Lisp > dependencies in '/gnu/store/...', so there should not be missing > references. Right. The only case where there could be a missing reference is if the source code contains an FFI reference stored in non-ASCII / UTF-8. So we need to parse other encodings too as Mark suggested if I understand correctly. > I just wondered: does the grafting code take '.fasl' files into > consideration? > If yes, I guess a Lisp library could also end up in a strange > half-grafted state if the grafting code modifies ASCII references and > not UTF-32 ones... Absolutely, .fasls suffer from the same problem since they may encode strings as UTF-32. =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBlmz4SHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/ZFsH/A5lZeTZQWXph2ZYsOjwv6mZMRrBY+sh H1Gu5mYoU+RIJv742j3gsrActW0Nb3jOY8VDlzcXV4iTCZz7/iOTg52r0wkMB9gQ QvJ3aXYa23G3Yoj7bvOkYB6NN+FGSdVBEIzeBA43jU7gLijDQvw6bWKWN50rkb9p Az+8vZexu7htL5bXQvYvQ8REPN7XtejLwHqDPfJZwZnlDBL00gVJHbQqwucg3yTm iHt3e4rYHmRubKiUxtNDGdvZeQAGW0CY7JzHLEzcZSa6HvTptU9t/xNEoaXmDaQl c1G1am0XD3JZvayzPHm8YUcf8UdlGif5QbyXTNQC5+ZQEwp53CLQ1vg= =rgOr -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 10:08:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Guillaume Le Vaillant Cc: Mark H Weaver , Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161727165810093 (code B ref 33848); Thu, 01 Apr 2021 10:08:02 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 10:07:38 +0000 Received: from localhost ([127.0.0.1]:56383 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRuEk-0002cj-8W for submit@debbugs.gnu.org; Thu, 01 Apr 2021 06:07:38 -0400 Received: from relay7-d.mail.gandi.net ([217.70.183.200]:49539) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRuEi-0002cR-CV for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 06:07:36 -0400 X-Originating-IP: 92.169.147.163 Received: from mimimi (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay7-d.mail.gandi.net (Postfix) with ESMTPSA id B001020006; Thu, 1 Apr 2021 10:07:29 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87a6qil4b1.fsf@yamatai> References: <87r2e8jpfx.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> Date: Thu, 01 Apr 2021 12:07:28 +0200 Message-ID: <87a6qiz5b3.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: I'm not familiar with the grafting code, so anyone who is (Mark? Ludo?) might be able to fix this much quicker than me! :) -- Pierre Neidhardt https://ambrevar.xyz/ Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.200 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [217.70.183.200 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: I'm not familiar with the grafting code, so anyone who is (Mark? Ludo?) might be able to fix this much quicker than me! :) -- Pierre Neidhardt https://ambrevar.xyz/ Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.200 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [217.70.183.200 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable I'm not familiar with the grafting code, so anyone who is (Mark? Ludo?) might be able to fix this much quicker than me! :) =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBlm2ASHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/jxMH/09oBJdVeh7HoBNEXMph1n7P1x+s2+j3 Ez/ICMW//2RExhoYZoyVVz0ehNep9j8I7ha4SCV7KCkua82HMYB533bjhJjd6Z3e nAz2txha/N+QVGpeUyZ7tIHGI69oKcHjF80OzQ1uKkMTJ5GmnVBKm6LvzK6QFOfn 6+SlVWSmDBoFi1GF3/BRrGh+WggO8XFdTdaVQeww2o6aBKTsIqVQCDR2HbkFHxUn //d5w0B3OmhEtWePZ6TNuQ3albz+d3I8Wve3F/HzOVCDREGHvJw55bbyyWkn8EJQ MZaKySjutHtz4ablW3pJEJsf1aOl6xYNdmmc5SjG8Tax1cWIQiY3L6k= =cC1E -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 15:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt Cc: Guillaume Le Vaillant , Mark H Weaver , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161729069112265 (code B ref 33848); Thu, 01 Apr 2021 15:25:02 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 15:24:51 +0000 Received: from localhost ([127.0.0.1]:58310 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRzBj-0003Bl-AZ for submit@debbugs.gnu.org; Thu, 01 Apr 2021 11:24:51 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53186) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lRzBh-0003BZ-L4 for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 11:24:50 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56521) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRzBY-0007Wc-PU; Thu, 01 Apr 2021 11:24:40 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=57130 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lRzBY-0007ql-7N; Thu, 01 Apr 2021 11:24:40 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 01 Apr 2021 17:24:37 +0200 In-Reply-To: <87a6qiz5b3.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Thu, 01 Apr 2021 12:07:28 +0200") Message-ID: <87pmze58p6.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Pierre Neidhardt skribis: > I'm not familiar with the grafting code, so anyone who is (Mark? Ludo?) > might be able to fix this much quicker than me! :) =?UTF-8?Q?There=E2=80=99s_?= =?UTF-8?Q?=E2=80=98%graft-hooks=E2=80=99?= in (guix build graft). One could add a hook that would do nothing except for SBCL packages; for SBCL packages, it would to the UCS-4 rewriting =?UTF-8?Q?=E2=80=9Csomehow=E2=80=9D.?= Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [209.51.188.92 listed in list.dnswl.org] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) Pierre Neidhardt skribis: > I'm not familiar with the grafting code, so anyone who is (Mark? Ludo?) > might be able to fix this much quicker than me! :) There=E2=80=99s =E2=80=98%graft-hooks=E2=80=99 in (guix build graft). One = could add a hook that would do nothing except for SBCL packages; for SBCL packages, it would to the UCS-4 rewriting =E2=80=9Csomehow=E2=80=9D. The other option, which might be easier, would be to arrange to not use UCS-4 in the first place, by choosing a bytevector data type for string literals known to contain a store reference. It can be done if we know where those references come from=E2=80=94e.g., they=E2=80=99re introduced by =E2=80=98substitute*=E2=80=99 on the source. I hope this makes sense! Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 17:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Pierre Neidhardt Cc: Guillaume Le Vaillant , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161729795324999 (code B ref 33848); Thu, 01 Apr 2021 17:26:01 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 17:25:53 +0000 Received: from localhost ([127.0.0.1]:58499 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lS14r-0006V9-Ja for submit@debbugs.gnu.org; Thu, 01 Apr 2021 13:25:53 -0400 Received: from world.peace.net ([64.112.178.59]:33142) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lS14q-0006Uv-3u for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 13:25:52 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lS14j-0007aS-6L; Thu, 01 Apr 2021 13:25:45 -0400 From: Mark H Weaver In-Reply-To: <87zgyj5a33.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <877eg0i43j.fsf@netris.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87zgyj5a33.fsf@gnu.org> Date: Thu, 01 Apr 2021 13:23:51 -0400 Message-ID: <875z15c3zx.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludovic, Ludovic Court=C3=A8s writes: > What could have been nice is if there=E2=80=99s a way to mark specific st= rings > as being ASCII, or if there=E2=80=99s a =E2=80=9Cbyte vector=E2=80=9D dat= a type compatible with > strings, for instance. Do we know that all strings containing store references will be representable in ASCII? Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 01 Apr 2021 17:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt , Guillaume Le Vaillant Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161729854526057 (code B ref 33848); Thu, 01 Apr 2021 17:36:02 +0000 Received: (at 33848) by debbugs.gnu.org; 1 Apr 2021 17:35:45 +0000 Received: from localhost ([127.0.0.1]:58506 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lS1EP-0006mD-H0 for submit@debbugs.gnu.org; Thu, 01 Apr 2021 13:35:45 -0400 Received: from world.peace.net ([64.112.178.59]:33168) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lS1EO-0006m0-0z for 33848@debbugs.gnu.org; Thu, 01 Apr 2021 13:35:44 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lS1EH-0008L3-9D; Thu, 01 Apr 2021 13:35:37 -0400 From: Mark H Weaver In-Reply-To: <87a6qiz5b3.fsf@ambrevar.xyz> References: <87r2e8jpfx.fsf@gnu.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> Date: Thu, 01 Apr 2021 13:33:55 -0400 Message-ID: <871rbtc3j5.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Pierre Neidhardt writes: > I'm not familiar with the grafting code, so anyone who is (Mark? Ludo?) > might be able to fix this much quicker than me! :) I'll think about what would be required to modify our grafting code to support UCS-4. Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Pierre Neidhardt writes: > I'm not familiar with the grafting code, so anyone who is (Mark? Ludo?) > might be able to fix this much quicker than me! :) I'll think about what would be required to modify our grafting code to support UCS-4. Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 02 Apr 2021 15:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161737641713968 (code B ref 33848); Fri, 02 Apr 2021 15:14:02 +0000 Received: (at 33848) by debbugs.gnu.org; 2 Apr 2021 15:13:37 +0000 Received: from localhost ([127.0.0.1]:60882 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lSLUP-0003dD-0K for submit@debbugs.gnu.org; Fri, 02 Apr 2021 11:13:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:54590) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lSLUL-0003cz-Tc for 33848@debbugs.gnu.org; Fri, 02 Apr 2021 11:13:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:33334) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lSLUG-0003rB-7F; Fri, 02 Apr 2021 11:13:28 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=34176 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lSLUE-0007Cb-0a; Fri, 02 Apr 2021 11:13:27 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <87d0psi1xo.fsf@gnu.org> <874lb3kin6.fsf@ambrevar.xyz> <87sgynezha.fsf@gnu.org> <87tvj2yesd.fsf@netris.org> <877efwe04u.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87zgyj5a33.fsf@gnu.org> <875z15c3zx.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 13 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 02 Apr 2021 17:13:23 +0200 In-Reply-To: <875z15c3zx.fsf@netris.org> (Mark H. Weaver's message of "Thu, 01 Apr 2021 13:23:51 -0400") Message-ID: <87im541zzg.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Mark, Mark H Weaver skribis: > Ludovic Court=C3=A8s writes: >> What could have been nice is if there=E2=80=99s a way to mark specific s= trings >> as being ASCII, or if there=E2=80=99s a =E2=80=9Cbyte vector=E2=80=9D da= ta type compatible with >> strings, for instance. > > Do we know that all strings containing store references will be > representable in ASCII? The basename part of /gnu/store/* is guaranteed to be pure ASCII. Now, you=E2=80=99re right that file names that come after could be non-ASCII, th= ough that=E2=80=99s very unlikely. We=E2=80=99d have to look at concrete exampl= es I guess. Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 02 Apr 2021 22:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt , Guillaume Le Vaillant Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161740371824554 (code B ref 33848); Fri, 02 Apr 2021 22:49:02 +0000 Received: (at 33848) by debbugs.gnu.org; 2 Apr 2021 22:48:38 +0000 Received: from localhost ([127.0.0.1]:32991 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lSSaj-0006Ny-S8 for submit@debbugs.gnu.org; Fri, 02 Apr 2021 18:48:38 -0400 Received: from world.peace.net ([64.112.178.59]:36528) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lSSah-0006Nl-NB for 33848@debbugs.gnu.org; Fri, 02 Apr 2021 18:48:36 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lSSaa-0003hD-Uc; Fri, 02 Apr 2021 18:48:29 -0400 From: Mark H Weaver In-Reply-To: <871rbtc3j5.fsf@netris.org> References: <87r2e8jpfx.fsf@gnu.org> <8736qji7c1.fsf@ambrevar.xyz> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> Date: Fri, 02 Apr 2021 18:46:41 -0400 Message-ID: <87r1js9udv.fsf@netris.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Here's a preliminary draft patch to add support for UTF-32 and UTF-16 references to our grafting code. I haven't yet measured the efficiency impact of these changes, but I suspect it's not too bad. I'd be curious to know whether it fixes the Nyxt graft. Mark --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-DRAFT-grafts-Add-support-for-UTF-16-and-UTF-32-store.patch Content-Transfer-Encoding: quoted-printable Content-Description: [PATCH] DRAFT: grafts: Add support for UTF-16 and UTF-32 store references >From 0fcfd804570fd1c07ffb1f6c176d6ec3430907df Mon Sep 17 00:00:00 2001 From: Mark H Weaver Date: Fri, 2 Apr 2021 18:36:23 -0400 Subject: [PATCH] DRAFT: grafts: Add support for UTF-16 and UTF-32 store references. --- guix/build/graft.scm | 138 +++++++++++++++++++++++++++++-------------- tests/grafts.scm | 68 +++++++++++++++++++++ 2 files changed, 162 insertions(+), 44 deletions(-) diff --git a/guix/build/graft.scm b/guix/build/graft.scm index c119ee71d1..6e7f3859cb 100644 --- a/guix/build/graft.scm +++ b/guix/build/graft.scm @@ -1,6 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2014, 2015, 2016, 2018 Ludovic Court=C3=A8s -;;; Copyright =C2=A9 2016 Mark H Weaver +;;; Copyright =C2=A9 2016, 2021 Mark H Weaver ;;; ;;; This file is part of GNU Guix. ;;; @@ -55,6 +55,36 @@ (string->char-set "0123456789abcdfghijklmnpqrsvwxyz") <>)) =20 +(define (nix-base32-char-or-nul? byte) + (or (nix-base32-char? byte) + (char=3D? byte #\nul))) + +(define (has-utf16-zeroes? buffer i) + (let loop ((j (+ 1 (- i (* 2 hash-length))))) + (or (>=3D j i) + (and (zero? (bytevector-u8-ref buffer j)) + (loop (+ j 2)))))) + +(define (has-utf32-zeroes? buffer i) + (let loop ((j (+ 1 (- i (* 4 hash-length))))) + (or (>=3D j i) + (and (zero? (bytevector-u8-ref buffer j)) + (zero? (bytevector-u8-ref buffer (+ j 1))) + (zero? (bytevector-u8-ref buffer (+ j 2))) + (loop (+ j 4)))))) + +(define (expand-bytevector bv char-size) + (let* ((len (bytevector-length bv)) + (bv* (make-bytevector (+ 1 (* char-size + (- len 1))) + 0))) + (let loop ((i 0)) + (when (< i len) + (bytevector-u8-set! bv* (* i char-size) + (bytevector-u8-ref bv i)) + (loop (+ i 1)))) + bv*)) + (define* (replace-store-references input output replacement-table #:optional (store (%store-directory))) "Read data from INPUT, replacing store references according to @@ -76,15 +106,16 @@ bytevectors to the same value." (list->vector (map pred (iota 256))) <>)) =20 - (define nix-base32-byte? + (define nix-base32-byte-or-nul? (optimize-u8-predicate - (compose nix-base32-char? + (compose nix-base32-char-or-nul? integer->char))) =20 (define (dash? byte) (=3D byte 45)) =20 (define request-size (expt 2 20)) ; 1 MiB =20 + ;; XXX This comment is no longer accurate! ;; We scan the file for the following 33-byte pattern: 32 bytes of ;; nix-base32 characters followed by a dash. To accommodate large files, ;; we do not read the entire file, but instead work on buffers of up to @@ -116,43 +147,61 @@ bytevectors to the same value." ;; written. (if (< i end) (let ((byte (bytevector-u8-ref buffer i))) - (cond ((and (dash? byte) - ;; We've found a dash. Note that we do not k= now - ;; whether the preceeding 32 bytes are nix-ba= se32 - ;; characters, but we do not need to know. If - ;; they are not, the following lookup will fa= il. - (lookup-replacement - (string-tabulate (lambda (j) - (integer->char - (bytevector-u8-ref buffer - (+ j (- i hash-length))= ))) - hash-length))) - =3D> (lambda (replacement) - ;; We've found a hash that needs to be replac= ed. - ;; First, write out all bytes preceding the h= ash - ;; that have not yet been written. - (put-bytevector output buffer written - (- i hash-length written)) - ;; Now write the replacement string. - (put-bytevector output replacement) - ;; Since the byte at position 'i' is a dash, - ;; which is not a nix-base32 char, the earlie= st - ;; position where the next hash might start is - ;; i+1, and the earliest position where the - ;; following dash might start is (+ i 1 - ;; hash-length). Also, increase the write - ;; position to account for REPLACEMENT. - (let ((len (bytevector-length replacement))) - (scan-from (+ i 1 len) - (+ i (- len hash-length)))))) - ;; If the byte at position 'i' is a nix-base32 char, + (cond ((dash? byte) + (let* ((char-size + (if (zero? (bytevector-u8-ref buffer (- i = 1))) + (if (zero? (bytevector-u8-ref buffer (= - i 2))) + (if (and (<=3D (* 4 hash-length) + (- i written)) + (has-utf32-zeroes? buffer= i)) + 4 + 1) + (if (and (<=3D (* 2 hash-length) + (- i written)) + (has-utf16-zeroes? buffer= i)) + 2 + 1)) + 1)) + (replacement* + (lookup-replacement + (string-tabulate (lambda (j) + (integer->char + (bytevector-u8-ref bu= ffer + (- i (* char-size + (- hash-leng= th j)))))) + hash-length))) + (replacement + (and replacement* + (expand-bytevector replacement* + char-size)))) + (if replacement + (begin + ;; We've found a hash that needs to be rep= laced. + ;; First, write out all bytes preceding th= e hash + ;; that have not yet been written. + (put-bytevector output buffer written + (- i (* char-size hash-len= gth) written)) + ;; Now write the replacement string. + (put-bytevector output replacement) + ;; Now compute the new value of 'written' = and + ;; the new value of 'i', and iterate. + (let ((written (+ (- i (* char-size hash-l= ength)) + (bytevector-length repla= cement)))) + (scan-from (+ written hash-length) writt= en))) + ;; The byte at position 'i' is a dash, which= is + ;; not a nix-base32 char, so the earliest + ;; position where the next hash might start = is + ;; i+1, with the following dash at position = (+ i + ;; 1 hash-length). + (scan-from (+ i 1 hash-length) written)))) + ;; If the byte at position 'i' is a nix-base32 char= or nul, ;; then the dash we're looking for might be as earl= y as ;; the following byte, so we can only advance by 1. - ((nix-base32-byte? byte) + ((nix-base32-byte-or-nul? byte) (scan-from (+ i 1) written)) - ;; If the byte at position 'i' is NOT a nix-base32 - ;; char, then the earliest position where the next = hash - ;; might start is i+1, with the following dash at + ;; If the byte at position 'i' is NOT a nix-base32 = char + ;; or nul, then the earliest position where the next + ;; hash might start is i+1, with the following dash= at ;; position (+ i 1 hash-length). (else (scan-from (+ i 1 hash-length) written)))) @@ -162,18 +211,19 @@ bytevectors to the same value." ;; "unget". If 'end' is less than 'request-size' then we r= ead ;; less than we asked for, which indicates that we are at E= OF, ;; so we needn't unget anything. Otherwise, we unget up to - ;; 'hash-length' bytes (32 bytes). However, we must be car= eful - ;; not to unget bytes that have already been written, becau= se - ;; that would cause them to be written again from the next - ;; buffer. In practice, this case occurs when a replacemen= t is - ;; made near or beyond the end of the buffer. When REPLACE= MENT - ;; went beyond END, we consume the extra bytes from INPUT. + ;; (* 4 hash-length) bytes. However, we must be careful no= t to + ;; unget bytes that have already been written, because that + ;; would cause them to be written again from the next buffe= r. + ;; In practice, this case occurs when a replacement is made + ;; near or beyond the end of the buffer. When REPLACEMENT = went + ;; beyond END, we consume the extra bytes from INPUT. (begin (if (> written end) (get-bytevector-n! input buffer 0 (- written end)) (let* ((unwritten (- end written)) (unget-size (if (=3D end request-size) - (min hash-length unwritten) + (min (* 4 hash-length) + unwritten) 0)) (write-size (- unwritten unget-size))) (put-bytevector output buffer written write-size) diff --git a/tests/grafts.scm b/tests/grafts.scm index a12c6a5911..0e1c7355b1 100644 --- a/tests/grafts.scm +++ b/tests/grafts.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Court=C3= =A8s +;;; Copyright =C2=A9 2021 Mark H Weaver ;;; ;;; This file is part of GNU Guix. ;;; @@ -468,4 +469,71 @@ replacement "/gnu/store"))))) =20 +(define (nul-expand str char-size) + (string-join (map string (string->list str)) + (make-string (- char-size 1) #\nul))) + +(for-each + (lambda (char-size1) + (for-each + (lambda (char-size2) + (for-each + (lambda (gap) + (for-each + (lambda (offset) + (test-equal (format #f "replace-store-references, char-sizes ~a= ~a, gap ~s, offset ~a" + char-size1 char-size2 gap offset) + (string-append (make-string offset #\=3D) + (nul-expand (string-append "/gnu/store/" + (make-string 32 #\6) + "-BlahBlaH") + char-size1) + gap + (nul-expand (string-append "/gnu/store/" + (make-string 32 #\8) + "-SoMeTHiNG") + char-size2) + (list->string (map integer->char (iota 77 33))= )) + + ;; Create input data where the right-hand-size of the dash ("= -something" + ;; here) goes beyond the end of the internal buffer of + ;; 'replace-store-references'. + (let* ((content (string-append (make-string offset #\=3D) + (nul-expand (string-append= "/gnu/store/" + = (make-string 32 #\5) + = "-blahblah") + char-size1) + gap + (nul-expand (string-append= "/gnu/store/" + = (make-string 32 #\7) + = "-something") + char-size2) + (list->string + (map integer->char (iota = 77 33))))) + (replacement (alist->vhash + `((,(make-string 32 #\5) + . ,(string->utf8 (string-append + (make-string 32 #\6) + "-BlahBlaH"))) + (,(make-string 32 #\7) + . ,(string->utf8 (string-append + (make-string 32 #\8) + "-SoMeTHiNG"))))))) + (call-with-output-string + (lambda (output) + ((@@ (guix build graft) replace-store-references) + (open-input-string content) output + replacement + "/gnu/store")))))) + ;; offsets to test + (map (lambda (i) (- buffer-size (* 40 char-size1) i)) + (iota 30)))) + ;; gaps + '("" "-" " " "a"))) + ;; char-size2 values to test + '(1 2))) + ;; char-size1 values to test + '(1 2 4)) + + (test-end) --=20 2.31.1 --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 03 Apr 2021 06:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver , Guillaume Le Vaillant Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161743271912476 (code B ref 33848); Sat, 03 Apr 2021 06:52:02 +0000 Received: (at 33848) by debbugs.gnu.org; 3 Apr 2021 06:51:59 +0000 Received: from localhost ([127.0.0.1]:33192 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lSa8V-0003FA-2Q for submit@debbugs.gnu.org; Sat, 03 Apr 2021 02:51:59 -0400 Received: from relay4-d.mail.gandi.net ([217.70.183.196]:50393) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lSa8T-0003Ey-Iy for 33848@debbugs.gnu.org; Sat, 03 Apr 2021 02:51:57 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id 171B4E0007; Sat, 3 Apr 2021 06:51:50 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87r1js9udv.fsf@netris.org> References: <87r2e8jpfx.fsf@gnu.org> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> Date: Sat, 03 Apr 2021 08:51:49 +0200 Message-ID: <87sg47vp16.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Wow, that was fast, thank you Mark! Any idea how I can test this, i.e. how I can force a graft? -- Pierre Neidhardt https://ambrevar.xyz/ Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.196 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.196 listed in wl.mailspike.net] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Wow, that was fast, thank you Mark! Any idea how I can test this, i.e. how I can force a graft? -- Pierre Neidhardt https://ambrevar.xyz/ Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.196 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [217.70.183.196 listed in wl.mailspike.net] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Wow, that was fast, thank you Mark! Any idea how I can test this, i.e. how I can force a graft? =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBoEIUSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/YPIH/R5/zbjQwOwOV+eL0ihxT3kJmJesqDAx qU3wOOFYvYSDSA3sD20CJxEUMgf/ucBOMWdBmjVgFvQimi7IkFcpGUECnEPQ0sgP P0ubBFI1tl0qMI/q2FnfmYCR1AgoboEccSeRnPJX4N4zvOS1gBaIP1eY6zoppobN mfXnSjK+J/oXHBYnXbJoy/ih8o019dDa5j1aMl0dDjxpfxCpCv2qzbPLQAGeM9AQ PhJU53dVRRBtzvCVVLlkmokdng0sfydWB/kOp90qH/jifZiylch6Qnn1AhQ/nr9q t0k0J+Niih4/wF5jsMX8QXVBQAgXhjYFu1kL04ShFeMS3UeNxu+Q/lc= =AVI2 -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 03 Apr 2021 20:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt , Guillaume Le Vaillant Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161748073623141 (code B ref 33848); Sat, 03 Apr 2021 20:13:02 +0000 Received: (at 33848) by debbugs.gnu.org; 3 Apr 2021 20:12:16 +0000 Received: from localhost ([127.0.0.1]:34475 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lSmcx-00061B-Tl for submit@debbugs.gnu.org; Sat, 03 Apr 2021 16:12:16 -0400 Received: from world.peace.net ([64.112.178.59]:38716) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lSmcw-00060w-9h for 33848@debbugs.gnu.org; Sat, 03 Apr 2021 16:12:15 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lSmcp-0000H2-Hq; Sat, 03 Apr 2021 16:12:08 -0400 From: Mark H Weaver In-Reply-To: <87sg47vp16.fsf@ambrevar.xyz> References: <87r2e8jpfx.fsf@gnu.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> Date: Sat, 03 Apr 2021 16:10:18 -0400 Message-ID: <875z139liy.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Pierre Neidhardt writes: > Wow, that was fast, thank you Mark! > > Any idea how I can test this, i.e. how I can force a graft? Just apply the patch to a git checkout of Guix, build it, and then use it to build anything you like, e.g. "./pre-inst-env guix build nyxt". Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Pierre Neidhardt writes: > Wow, that was fast, thank you Mark! > > Any idea how I can test this, i.e. how I can force a graft? Just apply the patch to a git checkout of Guix, build it, and then use it to build anything you like, e.g. "./pre-inst-env guix build nyxt". With this patch applied, all graft derivations will be rebuilt, but *only* grafts. When it's ready (i.e. when it has better comments, docstrings, etc), this change is perfectly appropriate for 'master'. FYI, I've already applied this new grafting code to my private branch of Guix, switched to a system and user profile built using it, rebooted into the new system, and I'm typing this email on it. I've also checked that none of the processes running on my system include executables or shared libraries from ungrafted store items (after removing my ibus .cache files, see ). Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 05 Apr 2021 19:46:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161765195412806 (code B ref 33848); Mon, 05 Apr 2021 19:46:01 +0000 Received: (at 33848) by debbugs.gnu.org; 5 Apr 2021 19:45:54 +0000 Received: from localhost ([127.0.0.1]:38604 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTVAX-0003KT-Nu for submit@debbugs.gnu.org; Mon, 05 Apr 2021 15:45:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34616) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTVAV-0003KG-V7 for 33848@debbugs.gnu.org; Mon, 05 Apr 2021 15:45:52 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36212) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lTVAP-00078T-HN; Mon, 05 Apr 2021 15:45:45 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=46772 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lTVAO-0001uU-Ui; Mon, 05 Apr 2021 15:45:45 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 16 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 05 Apr 2021 21:45:43 +0200 In-Reply-To: <875z139liy.fsf@netris.org> (Mark H. Weaver's message of "Sat, 03 Apr 2021 16:10:18 -0400") Message-ID: <87ft04sefs.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Mark, Mark H Weaver skribis: > Pierre Neidhardt writes: > >> Wow, that was fast, thank you Mark! Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [209.51.188.92 listed in list.dnswl.org] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) Hi Mark, Mark H Weaver skribis: > Pierre Neidhardt writes: > >> Wow, that was fast, thank you Mark! Yup, thumbs up! >> Any idea how I can test this, i.e. how I can force a graft? > > Just apply the patch to a git checkout of Guix, build it, and then use > it to build anything you like, e.g. "./pre-inst-env guix build nyxt". > > With this patch applied, all graft derivations will be rebuilt, but > *only* grafts. When it's ready (i.e. when it has better comments, > docstrings, etc), this change is perfectly appropriate for 'master'. The GC=E2=80=99s scanner still gets it wrong though. I wonder whether havi= ng the grafting code more capable than the scanner could lead to bad surprises. WDYT? Thank you for looking into this! Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 05 Apr 2021 23:07:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16176639998477 (code B ref 33848); Mon, 05 Apr 2021 23:07:02 +0000 Received: (at 33848) by debbugs.gnu.org; 5 Apr 2021 23:06:39 +0000 Received: from localhost ([127.0.0.1]:38862 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTYIo-0002Ce-Mk for submit@debbugs.gnu.org; Mon, 05 Apr 2021 19:06:38 -0400 Received: from world.peace.net ([64.112.178.59]:44152) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTYIm-0002CP-MV for 33848@debbugs.gnu.org; Mon, 05 Apr 2021 19:06:37 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTYIg-0005vv-9X; Mon, 05 Apr 2021 19:06:30 -0400 From: Mark H Weaver In-Reply-To: <87ft04sefs.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> <87ft04sefs.fsf@gnu.org> Date: Mon, 05 Apr 2021 19:04:44 -0400 Message-ID: <8735w472oo.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludovic, Ludovic Court=C3=A8s writes: > Mark H Weaver skribis: > >> With this patch applied, all graft derivations will be rebuilt, but >> *only* grafts. When it's ready (i.e. when it has better comments, >> docstrings, etc), this change is perfectly appropriate for 'master'. > > The GC=E2=80=99s scanner still gets it wrong though. I wonder whether ha= ving > the grafting code more capable than the scanner could lead to bad > surprises. WDYT? I've thought about it, and I've been unable to think of any disadvantage to making the grafter more capable than the scanner. It seems to me a pure win. That the scanner fails to find all references is clearly an important problem that should be fixed ASAP, but as far as I can tell, improving the grafter would not make that problem any worse or create any new problems. Improving the grafter should have the following effects: (1) Reducing the number of cases where ungrafted code with security flaws is being used on our systems. (2) Fixing problems in our Fish, Nyxt, and Common Lisp packages. Improving the scanner, or adding phases to selected packages or build systems to copy hidden references to an ASCII file, should have the following effects: (1) Reducing the number of cases where run-time dependencies are not known to Guix, which could lead to dependencies being prematurely GC'd or excluded from things like "guix pack". So, it seems to me that we should persue both of these improvements concurrently, and I see no practical advantage to postponing one for the sake of rolling them both out at the same time. Of course, I welcome other opinions on this. What do you think? Thanks, Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 06 Apr 2021 08:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161769699128538 (code B ref 33848); Tue, 06 Apr 2021 08:17:02 +0000 Received: (at 33848) by debbugs.gnu.org; 6 Apr 2021 08:16:31 +0000 Received: from localhost ([127.0.0.1]:39152 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTgsx-0007QE-B8 for submit@debbugs.gnu.org; Tue, 06 Apr 2021 04:16:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38536) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTgsv-0007Q2-T4 for 33848@debbugs.gnu.org; Tue, 06 Apr 2021 04:16:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:46292) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lTgsp-0004WI-8n; Tue, 06 Apr 2021 04:16:23 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48682 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lTgso-00014Z-PW; Tue, 06 Apr 2021 04:16:23 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> <87ft04sefs.fsf@gnu.org> <8735w472oo.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 17 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 06 Apr 2021 10:16:20 +0200 In-Reply-To: <8735w472oo.fsf@netris.org> (Mark H. Weaver's message of "Mon, 05 Apr 2021 19:04:44 -0400") Message-ID: <87v98zomjv.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Mark, Mark H Weaver skribis: > That the scanner fails to find all references is clearly an important > problem that should be fixed ASAP, but as far as I can tell, improving > the grafter would not make that problem any worse or create any new > problems. > > Improving the grafter should have the following effects: > > (1) Reducing the number of cases where ungrafted code with security > flaws is being used on our systems. > > (2) Fixing problems in our Fish, Nyxt, and Common Lisp packages. These are cases where the scanner may get the references wrong in the first place though. OTOH, there are also cases where the scanner gets the references right. For example, earlier Pierre mentioned that grafting breaks the reference of nyxt to glib: https://issues.guix.gnu.org/33848#26 It turns out that the scanner does find a reference to glib =E2=80=9Cby cha= nce=E2=80=9D: --8<---------------cut here---------------start------------->8--- $ guix gc --references $(guix build nyxt --no-grafts) | grep glib-2 /gnu/store/4vmhbc31cpbnlw3c96kcc094ihmaf7dv-glib-2.62.6 $ grep -r 4vmhbc31cpbnlw3c96kcc094ihmaf7dv $(guix build nyxt --no-grafts) Duuma dosiero /gnu/store/5pgh0cn1kzyajaanj7f1iyp91hd917d6-nyxt-2-pre-releas= e-6/bin/.nyxt-real kongruas --8<---------------cut here---------------end--------------->8--- So in this case, the fixed grafter is a net win. After applying the patch you sent, I confirm that Nyxt starts just fine when running: ./pre-inst-env guix environment --ad-hoc nyxt -CN -E ^DISPLAY --share=3D/= tmp/.X11-unix -- nyxt =E2=80=A6 whereas on master it fails to start with: --8<---------------cut here---------------start------------->8--- Unhandled SIMPLE-ERROR in thread #: Problem running initialization hook GLIB::RUN-INITIALIZERS: Unable to load any of the alternatives: ("/gnu/store/4vmhbc31cpbnlw3c96kcc094ihmaf7dv-glib-2.62.6/lib/libglib-2.= 0.so.0" "/gnu/store/4vmhbc31cpbnlw3c96kcc094ihmaf7dv-glib-2.62.6/lib/libglib-2.= 0.so") --8<---------------cut here---------------end--------------->8--- > Improving the scanner, or adding phases to selected packages or build > systems to copy hidden references to an ASCII file, should have the > following effects: > > (1) Reducing the number of cases where run-time dependencies are not > known to Guix, which could lead to dependencies being prematurely > GC'd or excluded from things like "guix pack". > > So, it seems to me that we should persue both of these improvements > concurrently, and I see no practical advantage to postponing one for the > sake of rolling them both out at the same time. Of course, I welcome > other opinions on this. As always I worry about added complexity. In this case, I think you=E2=80= =99re right: the UTF-{16,32}-capable grafter would most likely fix a number of issues right away, including this Nyxt issue. Thanks! Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 06 Apr 2021 08:24:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mark H Weaver Cc: Guillaume Le Vaillant , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161769741329192 (code B ref 33848); Tue, 06 Apr 2021 08:24:02 +0000 Received: (at 33848) by debbugs.gnu.org; 6 Apr 2021 08:23:33 +0000 Received: from localhost ([127.0.0.1]:39162 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTgzk-0007am-LQ for submit@debbugs.gnu.org; Tue, 06 Apr 2021 04:23:32 -0400 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:37607) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTgzj-0007aK-8D for 33848@debbugs.gnu.org; Tue, 06 Apr 2021 04:23:31 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id CC320FF808; Tue, 6 Apr 2021 08:23:24 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <87v98zomjv.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> <87ft04sefs.fsf@gnu.org> <8735w472oo.fsf@netris.org> <87v98zomjv.fsf@gnu.org> Date: Tue, 06 Apr 2021 10:23:22 +0200 Message-ID: <87wntfrfd1.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Mark, hi Ludo, > After applying the patch you sent, I confirm that Nyxt starts just fine > when running: > > ./pre-inst-env guix environment --ad-hoc nyxt -CN -E ^DISPLAY --share=/tmp/.X11-unix -- nyxt > > =?UTF-8?Q?=E2=80=A6?= where [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.183.199 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.199 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Mark, hi Ludo, > After applying the patch you sent, I confirm that Nyxt starts just fine > when running: > > ./pre-inst-env guix environment --ad-hoc nyxt -CN -E ^DISPLAY --share=/tmp/.X11-unix -- nyxt > > =?UTF-8?Q?=E2=80=A6?= where [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.183.199 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.199 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Mark, hi Ludo, > After applying the patch you sent, I confirm that Nyxt starts just fine > when running: > > ./pre-inst-env guix environment --ad-hoc nyxt -CN -E ^DISPLAY --share= =3D/tmp/.X11-unix -- nyxt > > =E2=80=A6 whereas on master it fails to start with: > > --8<---------------cut here---------------start------------->8--- > Unhandled SIMPLE-ERROR in thread # {10010D0103}>: > Problem running initialization hook GLIB::RUN-INITIALIZERS: > Unable to load any of the alternatives: > ("/gnu/store/4vmhbc31cpbnlw3c96kcc094ihmaf7dv-glib-2.62.6/lib/libglib-= 2.0.so.0" > "/gnu/store/4vmhbc31cpbnlw3c96kcc094ihmaf7dv-glib-2.62.6/lib/libglib-= 2.0.so") > --8<---------------cut here---------------end--------------->8--- Fantastic, this looks like it's fixing this grafting issue indeed! I also agree this looks like a net win even though the `guix gc` issue is not fix. The latter seems to be relatively easier to fix, doesn't it? Cheers! =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmBsGnoSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/BfoH/11cgUMkdFKk3mEPzFgwjwX0qk66UwQa 4MiBuWHZtvn4nS4cLRCzdyTk20bWk+w0hg7V5G2+EI5nKfrGGe3wqoH9MAASqG1S 14Q1aU6Q4QjBYdFwTy+oCKkCnPp0fL8dw95xHvYmPxRnbD2YTd0zPQ/Ei9cf6534 efkJ+zpyGc3h3ilO+BelQH+qSuscWBj6AyQNxvfdHPMZdJvY3jKW+P67kv8wqNhr ybX0IWD9lq+AkDdDiniS7MHmTpsOy+ZcBHOHGHT0Or7/07JRcraMU4Z3WLXQMbCQ MibQ9HiOkk15Cdx+/3RZZ+LhA738NZdLTznIU9PDVlUR0yuhxGUPFqw= =HVpO -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 06 Apr 2021 11:22:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Pierre Neidhardt , Guillaume Le Vaillant Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161770810831936 (code B ref 33848); Tue, 06 Apr 2021 11:22:02 +0000 Received: (at 33848) by debbugs.gnu.org; 6 Apr 2021 11:21:48 +0000 Received: from localhost ([127.0.0.1]:39385 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTjmF-0008J2-68 for submit@debbugs.gnu.org; Tue, 06 Apr 2021 07:21:48 -0400 Received: from world.peace.net ([64.112.178.59]:45226) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTjmD-0008Ip-Fd for 33848@debbugs.gnu.org; Tue, 06 Apr 2021 07:21:46 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTjm5-0006b5-Ur; Tue, 06 Apr 2021 07:21:38 -0400 From: Mark H Weaver In-Reply-To: <87r1js9udv.fsf@netris.org> References: <87r2e8jpfx.fsf@gnu.org> <87tvizvzgk.fsf@netris.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> Date: Tue, 06 Apr 2021 07:19:51 -0400 Message-ID: <87h7kj7j7x.fsf@netris.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Here's a revised draft of the patch, which updates the comments and refactors the code a bit to (hopefully) make it a bit more readable. Mark --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-DRAFT-grafts-Support-rewriting-UTF-16-and-UTF-32-sto.patch Content-Transfer-Encoding: quoted-printable Content-Description: [PATCH] DRAFT: grafts: Support rewriting UTF-16 and UTF-32 store references >From 6eec36e66d20d82fe02c6de793422875477b90d8 Mon Sep 17 00:00:00 2001 From: Mark H Weaver Date: Fri, 2 Apr 2021 18:36:23 -0400 Subject: [PATCH] DRAFT: grafts: Support rewriting UTF-16 and UTF-32 store references. * guix/build/graft.scm (replace-store-references): Add support for finding and rewriting UTF-16 and UTF-32 store references. * tests/grafts.scm: Add tests. --- guix/build/graft.scm | 247 +++++++++++++++++++++++++++---------------- tests/grafts.scm | 68 ++++++++++++ 2 files changed, 224 insertions(+), 91 deletions(-) diff --git a/guix/build/graft.scm b/guix/build/graft.scm index c119ee71d1..23fca8f29c 100644 --- a/guix/build/graft.scm +++ b/guix/build/graft.scm @@ -1,6 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2014, 2015, 2016, 2018 Ludovic Court=C3=A8s -;;; Copyright =C2=A9 2016 Mark H Weaver +;;; Copyright =C2=A9 2016, 2021 Mark H Weaver ;;; ;;; This file is part of GNU Guix. ;;; @@ -55,6 +55,40 @@ (string->char-set "0123456789abcdfghijklmnpqrsvwxyz") <>)) =20 +(define (nix-base32-char-or-nul? byte) + (or (nix-base32-char? byte) + (char=3D? byte #\nul))) + +(define (possible-utf16-hash? buffer i w) + (and (<=3D (* 2 hash-length) (- i w)) + (let loop ((j (+ 1 (- i (* 2 hash-length))))) + (or (>=3D j i) + (and (zero? (bytevector-u8-ref buffer j)) + (loop (+ j 2))))))) + +(define (possible-utf32-hash? buffer i w) + (and (<=3D (* 4 hash-length) (- i w)) + (let loop ((j (+ 1 (- i (* 4 hash-length))))) + (or (>=3D j i) + (and (zero? (bytevector-u8-ref buffer j)) + (zero? (bytevector-u8-ref buffer (+ j 1))) + (zero? (bytevector-u8-ref buffer (+ j 2))) + (loop (+ j 4))))))) + +(define (insert-nuls char-size bv) + (if (or (not bv) (=3D char-size 1)) + bv + (let* ((len (bytevector-length bv)) + (bv* (make-bytevector (+ 1 (* char-size + (- len 1))) + 0))) + (let loop ((i 0)) + (when (< i len) + (bytevector-u8-set! bv* (* i char-size) + (bytevector-u8-ref bv i)) + (loop (+ i 1)))) + bv*))) + (define* (replace-store-references input output replacement-table #:optional (store (%store-directory))) "Read data from INPUT, replacing store references according to @@ -76,9 +110,9 @@ bytevectors to the same value." (list->vector (map pred (iota 256))) <>)) =20 - (define nix-base32-byte? + (define nix-base32-byte-or-nul? (optimize-u8-predicate - (compose nix-base32-char? + (compose nix-base32-char-or-nul? integer->char))) =20 (define (dash? byte) (=3D byte 45)) @@ -86,100 +120,131 @@ bytevectors to the same value." (define request-size (expt 2 20)) ; 1 MiB =20 ;; We scan the file for the following 33-byte pattern: 32 bytes of - ;; nix-base32 characters followed by a dash. To accommodate large files, - ;; we do not read the entire file, but instead work on buffers of up to - ;; 'request-size' bytes. To ensure that every 33-byte sequence appears - ;; entirely within exactly one buffer, adjacent buffers must overlap, - ;; i.e. they must share 32 byte positions. We accomplish this by - ;; "ungetting" the last 32 bytes of each buffer before reading the next - ;; buffer, unless we know that we've reached the end-of-file. + ;; nix-base32 characters followed by a dash. When we find such a pattern + ;; whose hash is in REPLACEMENT-TABLE, we perform the required rewrite a= nd + ;; continue scanning. + ;; + ;; To support UTF-16 and UTF-32 store references, the 33 bytes comprising + ;; this hash+dash pattern may optionally be interspersed by extra NUL by= tes. + ;; This simple approach works because the characters we are looking for = are + ;; restricted to ASCII. UTF-16 hashes are interspersed with single NUL + ;; bytes ("\0"), and UTF-32 hashes are interspersed with triplets of NULs + ;; ("\0\0\0"). Note that we require NULs to be present only *between* t= he + ;; other bytes, and not at either end, in order to be insensitive to byte + ;; order. + ;; + ;; To accommodate large files, we do not read the entire file at once, b= ut + ;; instead work on buffers of up to 'request-size' bytes. To ensure that + ;; every hash+dash pattern appears in its entirety in at least one buffe= r, + ;; adjacent buffers must overlap by one byte less than the maximum size = of a + ;; hash+dash pattern. We accomplish this by "ungetting" a suffix of each + ;; buffer before reading the next buffer, unless we know that we've reac= hed + ;; the end-of-file. (let ((buffer (make-bytevector request-size))) - (let loop () - ;; Note: We avoid 'get-bytevector-n' to work around - ;; . + (define-syntax-rule (byte-at i) + (bytevector-u8-ref buffer i)) + (let outer-loop () (match (get-bytevector-n! input buffer 0 request-size) ((? eof-object?) 'done) (end - ;; We scan the buffer for dashes that might be preceded by a - ;; nix-base32 hash. The key optimization here is that whenever we - ;; find a NON-nix-base32 character at position 'i', we know that = it - ;; cannot be part of a hash, so the earliest position where the n= ext - ;; hash could start is i+1 with the following dash at position i+= 33. - ;; - ;; Since nix-base32 characters comprise only 1/8 of the 256 possi= ble - ;; byte values, and exclude some of the most common letters in - ;; English text (e t o u), in practice we can advance by 33 posit= ions - ;; most of the time. - (let scan-from ((i hash-length) (written 0)) - ;; 'i' is the first position where we look for a dash. 'writte= n' - ;; is the number of bytes in the buffer that have already been - ;; written. + (define (scan-from i w) + ;; Scan the buffer for dashes that might be preceded by nix has= hes, + ;; where 'i' is the minimum position where such a dash might be + ;; found, and 'w' is the number of bytes in the buffer that have + ;; been written so far. + ;; + ;; The key optimization here is that whenever we find a byte at + ;; position 'i' that cannot occur within a nix hash (because it= 's + ;; neither a nix-base32 character nor NUL), we can infer that t= he + ;; earliest position where the next hash could start is at i+1, + ;; and therefore the earliest position for the following dash is + ;; (+ i 1 hash-length), which is i+33. + ;; + ;; Since nix-base32-or-nul characters comprise only about 1/8 of + ;; the 256 possible byte values, and exclude some of the most + ;; common letters in English text (e t o u), we can advance 33 + ;; positions most of the time. (if (< i end) - (let ((byte (bytevector-u8-ref buffer i))) - (cond ((and (dash? byte) - ;; We've found a dash. Note that we do not k= now - ;; whether the preceeding 32 bytes are nix-ba= se32 - ;; characters, but we do not need to know. If - ;; they are not, the following lookup will fa= il. - (lookup-replacement - (string-tabulate (lambda (j) - (integer->char - (bytevector-u8-ref buffer - (+ j (- i hash-length))= ))) - hash-length))) - =3D> (lambda (replacement) - ;; We've found a hash that needs to be replac= ed. - ;; First, write out all bytes preceding the h= ash - ;; that have not yet been written. - (put-bytevector output buffer written - (- i hash-length written)) - ;; Now write the replacement string. - (put-bytevector output replacement) - ;; Since the byte at position 'i' is a dash, - ;; which is not a nix-base32 char, the earlie= st - ;; position where the next hash might start is - ;; i+1, and the earliest position where the - ;; following dash might start is (+ i 1 - ;; hash-length). Also, increase the write - ;; position to account for REPLACEMENT. - (let ((len (bytevector-length replacement))) - (scan-from (+ i 1 len) - (+ i (- len hash-length)))))) - ;; If the byte at position 'i' is a nix-base32 char, - ;; then the dash we're looking for might be as earl= y as - ;; the following byte, so we can only advance by 1. - ((nix-base32-byte? byte) - (scan-from (+ i 1) written)) - ;; If the byte at position 'i' is NOT a nix-base32 - ;; char, then the earliest position where the next = hash - ;; might start is i+1, with the following dash at - ;; position (+ i 1 hash-length). + (let ((byte (byte-at i))) + (cond ((dash? byte) + (found-dash i w)) + ((nix-base32-byte-or-nul? byte) + (scan-from (+ i 1) w)) (else - (scan-from (+ i 1 hash-length) written)))) - - ;; We have finished scanning the buffer. Now we determine = how - ;; many bytes have not yet been written, and how many bytes= to - ;; "unget". If 'end' is less than 'request-size' then we r= ead - ;; less than we asked for, which indicates that we are at E= OF, - ;; so we needn't unget anything. Otherwise, we unget up to - ;; 'hash-length' bytes (32 bytes). However, we must be car= eful - ;; not to unget bytes that have already been written, becau= se - ;; that would cause them to be written again from the next - ;; buffer. In practice, this case occurs when a replacemen= t is - ;; made near or beyond the end of the buffer. When REPLACE= MENT - ;; went beyond END, we consume the extra bytes from INPUT. - (begin - (if (> written end) - (get-bytevector-n! input buffer 0 (- written end)) - (let* ((unwritten (- end written)) - (unget-size (if (=3D end request-size) - (min hash-length unwritten) - 0)) - (write-size (- unwritten unget-size))) - (put-bytevector output buffer written write-size) - (unget-bytevector input buffer (+ written write-siz= e) - unget-size))) - (loop))))))))) + (not-part-of-hash i w)))) + (finish-buffer i w))) + + (define (not-part-of-hash i w) + ;; Position 'i' is known to not be within a nix hash. Therefor= e, + ;; the earliest position where the next hash might start is i+1, + ;; with the following dash at position (+ i 1 hash-length). + (scan-from (+ i 1 hash-length) w)) + + (define (found-dash i w) + (cond ((not (zero? (byte-at (- i 1)))) + (found-possible-hash 1 i w)) + ((not (zero? (byte-at (- i 2)))) + (if (possible-utf16-hash? buffer i w) + (found-possible-hash 2 i w) + (not-part-of-hash i w))) + (else + (if (possible-utf32-hash? buffer i w) + (found-possible-hash 4 i w) + (not-part-of-hash i w))))) + + (define (found-possible-hash char-size i w) + (let* ((hash (string-tabulate + (lambda (j) + (integer->char + (byte-at (- i (* char-size + (- hash-length j)))))) + hash-length)) + (replacement* (lookup-replacement hash)) + (replacement (and replacement* + (insert-nuls char-size replacement*)))) + (cond + ((not replacement) + (not-part-of-hash i w)) + (else + ;; We've found a hash that needs to be replaced. + ;; First, write out all bytes preceding the hash + ;; that have not yet been written. + (put-bytevector output buffer w + (- i (* char-size hash-length) w)) + ;; Now write the replacement string. + (put-bytevector output replacement) + ;; Now compute the new value of 'w' and + ;; the new value of 'i', and iterate. + (let ((w (+ (- i (* char-size hash-length)) + (bytevector-length replacement)))) + (scan-from (+ w hash-length) w)))))) + + (define (finish-buffer i w) + ;; We have finished scanning the buffer. Now we determine how + ;; many bytes have not yet been written, and how many bytes to + ;; "unget". If 'end' is less than 'request-size' then we read + ;; less than we asked for, which indicates that we are at EOF, + ;; so we needn't unget anything. Otherwise, we unget up to + ;; (* 4 hash-length) bytes. However, we must be careful not to + ;; unget bytes that have already been written, because that + ;; would cause them to be written again from the next buffer. + ;; In practice, this case occurs when a replacement is made + ;; near or beyond the end of the buffer. When REPLACEMENT went + ;; beyond END, we consume the extra bytes from INPUT. + (if (> w end) + (get-bytevector-n! input buffer 0 (- w end)) + (let* ((unwritten (- end w)) + (unget-size (if (=3D end request-size) + (min (* 4 hash-length) + unwritten) + 0)) + (write-size (- unwritten unget-size))) + (put-bytevector output buffer w write-size) + (unget-bytevector input buffer (+ w write-size) + unget-size))) + (outer-loop)) + + (scan-from hash-length 0)))))) =20 (define (rename-matching-files directory mapping) "Apply MAPPING to the names of all the files in DIRECTORY, where MAPPING= is diff --git a/tests/grafts.scm b/tests/grafts.scm index a12c6a5911..0e1c7355b1 100644 --- a/tests/grafts.scm +++ b/tests/grafts.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Court=C3= =A8s +;;; Copyright =C2=A9 2021 Mark H Weaver ;;; ;;; This file is part of GNU Guix. ;;; @@ -468,4 +469,71 @@ replacement "/gnu/store"))))) =20 +(define (nul-expand str char-size) + (string-join (map string (string->list str)) + (make-string (- char-size 1) #\nul))) + +(for-each + (lambda (char-size1) + (for-each + (lambda (char-size2) + (for-each + (lambda (gap) + (for-each + (lambda (offset) + (test-equal (format #f "replace-store-references, char-sizes ~a= ~a, gap ~s, offset ~a" + char-size1 char-size2 gap offset) + (string-append (make-string offset #\=3D) + (nul-expand (string-append "/gnu/store/" + (make-string 32 #\6) + "-BlahBlaH") + char-size1) + gap + (nul-expand (string-append "/gnu/store/" + (make-string 32 #\8) + "-SoMeTHiNG") + char-size2) + (list->string (map integer->char (iota 77 33))= )) + + ;; Create input data where the right-hand-size of the dash ("= -something" + ;; here) goes beyond the end of the internal buffer of + ;; 'replace-store-references'. + (let* ((content (string-append (make-string offset #\=3D) + (nul-expand (string-append= "/gnu/store/" + = (make-string 32 #\5) + = "-blahblah") + char-size1) + gap + (nul-expand (string-append= "/gnu/store/" + = (make-string 32 #\7) + = "-something") + char-size2) + (list->string + (map integer->char (iota = 77 33))))) + (replacement (alist->vhash + `((,(make-string 32 #\5) + . ,(string->utf8 (string-append + (make-string 32 #\6) + "-BlahBlaH"))) + (,(make-string 32 #\7) + . ,(string->utf8 (string-append + (make-string 32 #\8) + "-SoMeTHiNG"))))))) + (call-with-output-string + (lambda (output) + ((@@ (guix build graft) replace-store-references) + (open-input-string content) output + replacement + "/gnu/store")))))) + ;; offsets to test + (map (lambda (i) (- buffer-size (* 40 char-size1) i)) + (iota 30)))) + ;; gaps + '("" "-" " " "a"))) + ;; char-size2 values to test + '(1 2))) + ;; char-size1 values to test + '(1 2 4)) + + (test-end) --=20 2.31.1 --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Leo Famulari Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 06 Apr 2021 17:25:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161772984920734 (code B ref 33848); Tue, 06 Apr 2021 17:25:01 +0000 Received: (at 33848) by debbugs.gnu.org; 6 Apr 2021 17:24:09 +0000 Received: from localhost ([127.0.0.1]:41565 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTpQv-0005OM-JH for submit@debbugs.gnu.org; Tue, 06 Apr 2021 13:24:09 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:38363) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTpQu-0005Nz-0i for 33848@debbugs.gnu.org; Tue, 06 Apr 2021 13:24:08 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id F21E5132B; Tue, 6 Apr 2021 13:24:01 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Tue, 06 Apr 2021 13:24:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-transfer-encoding:in-reply-to; s=mesmtp; bh=5Doj1fgen/IcbItseEbwZpVbCCksPjjovesZ6vlRD/A=; b=qdp91RSyb0KR KgqS3aFFN0FsL+cLXbDPGR7G1c8cEJONy+6EsvyrwU05K90aTO6IfWdABRlK4D9y U4TfCejTSjMVwyQfvaJRCbaGBxs292+mrxDYE6Ni80gP7Pqov/DViJPL24H77Pzx BdwIysFBz64F0LEDC1MrvzhPkbX+Qms= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=5Doj1fgen/IcbItseEbwZpVbCCksPjjovesZ6vlRD /A=; b=NiStYwo6hfFRe3JSOHJUnltWgujhhZ4BR+sNuIyXET/lOa0ZRGGFIkCke uZyjbNuiU43b4Wu5Sby2kg6nzI5JCNV0Nr3DUEwsR1TYVq48EBF9ew/6CUenHvvD w7TSr/JQEmyBnsQ6OsfJzbdjIDJ8xfr6aVcpPRQCr2uXQ1qJsrH6a6dGakyVd1ON EyTxRSgBzY1hS7o3ySR/w4/OLC+rRcO5Q+L2/BWPJ1mo/Dpz0j4VdN/R7Sk1zHQr x1demWUZtLgJzF5UQk4J8ZavQZjdAnorOnbLy1eAV/7/K25n0MK0B9Kl7mhlPOFU LkCkPDEVc0HVBH5GQjWX80QqNwXrw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudejhedgheelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtugfgjgesthekredttddtjeenucfhrhhomhepnfgvohcu hfgrmhhulhgrrhhiuceolhgvohesfhgrmhhulhgrrhhirdhnrghmvgeqnecuggftrfgrth htvghrnhepgfduffettedtkeekudfhgfefgfeifeegueeitedujeffleeiudeuieffgfdu gfdunecuffhomhgrihhnpehgnhhurdhorhhgnecukfhppedutddtrdduuddrudeiledrud dukeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehl vghosehfrghmuhhlrghrihdrnhgrmhgv X-ME-Proxy: Received: from localhost (pool-100-11-169-118.phlapa.fios.verizon.net [100.11.169.118]) by mail.messagingengine.com (Postfix) with ESMTPA id 24F0424005D; Tue, 6 Apr 2021 13:24:00 -0400 (EDT) Date: Tue, 6 Apr 2021 13:23:58 -0400 From: Leo Famulari Message-ID: References: <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> <87ft04sefs.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <87ft04sefs.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Mon, Apr 05, 2021 at 09:45:43PM +0200, Ludovic Courtès wrote: > The GC’s scanner still gets it wrong though. I wonder whether having > the grafting code more capable than the scanner could lead to bad > surprises. WDYT? I'm going off-topic, but I've wished we had a generic fast string-search (and replace?) procedure. The go-build-system has a slow "one byte a time" implementation because I couldn't figure out how to re-use the code in (guix build grafts): https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/go-build-system.scm?h=v1.2.0#n269 There are probably some other places we could use a fast procedure. It might even be something to add to Guile. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 06 Apr 2021 23:24:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Leo Famulari , Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161775140816227 (code B ref 33848); Tue, 06 Apr 2021 23:24:02 +0000 Received: (at 33848) by debbugs.gnu.org; 6 Apr 2021 23:23:28 +0000 Received: from localhost ([127.0.0.1]:42461 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTv2e-0004Df-Cp for submit@debbugs.gnu.org; Tue, 06 Apr 2021 19:23:28 -0400 Received: from world.peace.net ([64.112.178.59]:46890) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lTv2V-0004DG-MM for 33848@debbugs.gnu.org; Tue, 06 Apr 2021 19:23:27 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lTv2P-0006ro-FB; Tue, 06 Apr 2021 19:23:13 -0400 From: Mark H Weaver In-Reply-To: References: <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> <87ft04sefs.fsf@gnu.org> Date: Tue, 06 Apr 2021 19:21:30 -0400 Message-ID: <87mtubngmi.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Leo, Leo Famulari writes: > On Mon, Apr 05, 2021 at 09:45:43PM +0200, Ludovic Court=C3=A8s wrote: >> The GC=E2=80=99s scanner still gets it wrong though. I wonder whether h= aving >> the grafting code more capable than the scanner could lead to bad >> surprises. WDYT? > > I'm going off-topic, but I've wished we had a generic fast string-search > (and replace?) procedure. Guile has several functions to help with this, e.g. 'string-contains', 'string-replace', 'string-replace-substring', and of course the regexp functions. The grafting code can't use these things because (1) we are not searching for a single string, but rather for arbitrary nix hashes, and (2) we can't easily use the regexp functions because there could be NULs present. > The go-build-system has a slow "one byte a time" implementation because > I couldn't figure out how to re-use the code in (guix build grafts): > > https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/go-build-syste= m.scm?h=3Dv1.2.0#n269 >From a glance, I would think that 'replace-store-references' from (guix build grafts) is directly applicable here. Do you remember what the difficulty was in using it? Alternatively, since you are only searching for a single string to replace, I think you could read the entire file into a single string (e.g. using 'get-string-all' with "ISO-8859-1" encoding) and use 'string-replace-substring'. What do you think? Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 08 Apr 2021 10:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.1617876830333 (code B ref 33848); Thu, 08 Apr 2021 10:14:02 +0000 Received: (at 33848) by debbugs.gnu.org; 8 Apr 2021 10:13:50 +0000 Received: from localhost ([127.0.0.1]:46041 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lURfa-00005I-4u for submit@debbugs.gnu.org; Thu, 08 Apr 2021 06:13:50 -0400 Received: from eggs.gnu.org ([209.51.188.92]:46538) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lURfY-00004y-9z for 33848@debbugs.gnu.org; Thu, 08 Apr 2021 06:13:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37374) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lURfP-00005Z-UQ; Thu, 08 Apr 2021 06:13:40 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=58442 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lURfP-00057c-Gj; Thu, 08 Apr 2021 06:13:39 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <87o9979gfn.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87h7kj7j7x.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 19 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 08 Apr 2021 12:13:37 +0200 In-Reply-To: <87h7kj7j7x.fsf@netris.org> (Mark H. Weaver's message of "Tue, 06 Apr 2021 07:19:51 -0400") Message-ID: <87ft01axta.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Mark, Mark H Weaver skribis: > From 6eec36e66d20d82fe02c6de793422875477b90d8 Mon Sep 17 00:00:00 2001 > From: Mark H Weaver > Date: Fri, 2 Apr 2021 18:36:23 -0400 > Subject: [PATCH] DRAFT: grafts: Support rewriting UTF-16 and UTF-32 store > references. > > * guix/build/graft.scm (replace-store-references): Add support for > finding and rewriting UTF-16 and UTF-32 store references. > * tests/grafts.scm: Add tests. Please add a =E2=80=9CFixes=E2=80=9D line in the commit log. I=E2=80=99m not reviewing the code in depth and I trust your judgment. The risks of bugs I can think of are: missed ASCII references (a regression, whereby some ASCII references would not get rewritten), and unrelated UTF-{16,32}-base32-looking references getting rewritten. I guess the latter is very unlikely because only sequences found in the replacement table may be rewritten, right? The former should be caught by =E2=80=98tests/grafts.scm=E2=80=99 but we co= uld also check the closure of real-world systems, for instance, to make sure it doesn=E2=80=99t refer to ungrafted things. Do you know how this affects performance? Some superficial comments: > +(define (possible-utf16-hash? buffer i w) > + (and (<=3D (* 2 hash-length) (- i w)) > + (let loop ((j (+ 1 (- i (* 2 hash-length))))) > + (or (>=3D j i) > + (and (zero? (bytevector-u8-ref buffer j)) > + (loop (+ j 2))))))) > + > +(define (possible-utf32-hash? buffer i w) > + (and (<=3D (* 4 hash-length) (- i w)) > + (let loop ((j (+ 1 (- i (* 4 hash-length))))) > + (or (>=3D j i) > + (and (zero? (bytevector-u8-ref buffer j)) > + (zero? (bytevector-u8-ref buffer (+ j 1))) > + (zero? (bytevector-u8-ref buffer (+ j 2))) > + (loop (+ j 4))))))) > + > +(define (insert-nuls char-size bv) Perhaps add short docstrings for clarity. > +(for-each > + (lambda (char-size1) > + (for-each > + (lambda (char-size2) > + (for-each > + (lambda (gap) > + (for-each > + (lambda (offset) > + (test-equal (format #f "replace-store-references, char-sizes = ~a ~a, gap ~s, offset ~a" > + char-size1 char-size2 gap offset) > + (string-append (make-string offset #\=3D) > + (nul-expand (string-append "/gnu/store/" > + (make-string 32 #= \6) > + "-BlahBlaH") > + char-size1) > + gap > + (nul-expand (string-append "/gnu/store/" > + (make-string 32 #= \8) > + "-SoMeTHiNG") > + char-size2) > + (list->string (map integer->char (iota 77 33= )))) > + > + ;; Create input data where the right-hand-size of the dash = ("-something" > + ;; here) goes beyond the end of the internal buffer of > + ;; 'replace-store-references'. > + (let* ((content (string-append (make-string offset #\= =3D) > + (nul-expand (string-appe= nd "/gnu/store/" > + = (make-string 32 #\5) > + = "-blahblah") > + char-size1) > + gap > + (nul-expand (string-appe= nd "/gnu/store/" > + = (make-string 32 #\7) > + = "-something") > + char-size2) > + (list->string > + (map integer->char (iot= a 77 33))))) > + (replacement (alist->vhash > + `((,(make-string 32 #\5) > + . ,(string->utf8 (string-append > + (make-string 32 #= \6) > + "-BlahBlaH"))) > + (,(make-string 32 #\7) > + . ,(string->utf8 (string-append > + (make-string 32 #= \8) > + "-SoMeTHiNG")))))= )) > + (call-with-output-string > + (lambda (output) > + ((@@ (guix build graft) replace-store-references) > + (open-input-string content) output > + replacement > + "/gnu/store")))))) > + ;; offsets to test > + (map (lambda (i) (- buffer-size (* 40 char-size1) i)) > + (iota 30)))) > + ;; gaps > + '("" "-" " " "a"))) > + ;; char-size2 values to test > + '(1 2))) > + ;; char-size1 values to test > + '(1 2 4)) For clarity, perhaps you can define a top-level procedure for the test and call it from =E2=80=98for-each=E2=80=99. Modulo these very minor issues, it looks like it=E2=80=99s ready to go! Thank you, Ludo=E2=80=99. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 13 Apr 2021 20:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16183444957016 (code B ref 33848); Tue, 13 Apr 2021 20:09:02 +0000 Received: (at 33848) by debbugs.gnu.org; 13 Apr 2021 20:08:15 +0000 Received: from localhost ([127.0.0.1]:60911 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWPKY-0001p4-HV for submit@debbugs.gnu.org; Tue, 13 Apr 2021 16:08:15 -0400 Received: from world.peace.net ([64.112.178.59]:35660) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWPKV-0001oj-VF for 33848@debbugs.gnu.org; Tue, 13 Apr 2021 16:08:13 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lWPKN-00048N-PD; Tue, 13 Apr 2021 16:08:04 -0400 From: Mark H Weaver In-Reply-To: <87ft01axta.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <87tvizgghs.fsf@ambrevar.xyz> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87h7kj7j7x.fsf@netris.org> <87ft01axta.fsf@gnu.org> Date: Tue, 13 Apr 2021 16:06:19 -0400 Message-ID: <87k0p6rlt5.fsf@netris.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludovic, Ludovic Court=C3=A8s writes: > Please add a =E2=80=9CFixes=E2=80=9D line in the commit log. Done, thanks. > I=E2=80=99m not reviewing the code in depth and I trust your judgment. > > The risks of bugs I can think of are: missed ASCII references (a > regression, whereby some ASCII references would not get rewritten), This is indeed a risk whenever we modify the grafting code, which is written for efficiency, not clarity. I've tried to be careful, and have checked that my newly grafted system and user profiles do not retain references to ungrafted code, modulo the following pre-existing issues: (ibus-daemon launches ungrafted subprocesses) (Chunked store references in .zo files in Racket 8) > and unrelated UTF-{16,32}-base32-looking references getting rewritten. > > I guess the latter is very unlikely because only sequences found in the > replacement table may be rewritten, right? Yes, that's correct. > The former should be caught by =E2=80=98tests/grafts.scm=E2=80=99 but we = could also > check the closure of real-world systems, for instance, to make sure it > doesn=E2=80=99t refer to ungrafted things. As I mention above, I've already done so for my own GNOME-based Guix system. > Do you know how this affects performance? I have not yet measured it, but subjectively, I do not notice any obvious difference in speed. I expect that the main performance impact is that large blocks of NULs will be processed more slowly by the current draft version of the new grafting code. That's because NULs can now be part of a nix hash, and therefore the new grafting code can only advance 1 byte position when seeing a NUL, whereas previously it would skip ahead 33 positions in that case. If desired, the handling of NULs could be made more efficient, at the cost of a bit more complexity. When seeing a NUL, we could check the adjacent bytes to see if this is part of a nix-base32 character in UTF-16 or UTF-32 encoding. If not, we could skip ahead. > Perhaps add short docstrings for clarity. Done. >> +(for-each >> + (lambda (char-size1) >> + (for-each >> + (lambda (char-size2) >> + (for-each >> + (lambda (gap) >> + (for-each >> + (lambda (offset) >> + (test-equal (format #f "replace-store-references, char-sizes= ~a ~a, gap ~s, offset ~a" >> + char-size1 char-size2 gap offset) [...] >> + ;; offsets to test >> + (map (lambda (i) (- buffer-size (* 40 char-size1) i)) >> + (iota 30)))) >> + ;; gaps >> + '("" "-" " " "a"))) >> + ;; char-size2 values to test >> + '(1 2))) >> + ;; char-size1 values to test >> + '(1 2 4)) > > For clarity, perhaps you can define a top-level procedure for the test > and call it from =E2=80=98for-each=E2=80=99. Good idea. I'd like to optimize the tests a bit before pushing this. They take a fairly long time to run, and lead to a huge 1.5G grafts.log file. It might be sufficient to avoid 'test-equal' here. > Modulo these very minor issues, it looks like it=E2=80=99s ready to go! Sounds good. Thanks for the review! Below, I've attached my current revision of this draft patch, which incorporates your suggestions regarding the main code. What remains is to improve the tests. Regards, Mark --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-DRAFT-grafts-Support-rewriting-UTF-16-and-UTF-32-sto.patch Content-Transfer-Encoding: quoted-printable Content-Description: [PATCH] DRAFT: grafts: Support rewriting UTF-16 and UTF-32 store references >From f3141eae346a66ff52c70708c87f880938bdbb24 Mon Sep 17 00:00:00 2001 From: Mark H Weaver Date: Fri, 2 Apr 2021 18:36:23 -0400 Subject: [PATCH] DRAFT: grafts: Support rewriting UTF-16 and UTF-32 store references. Partially fixes . * guix/build/graft.scm (replace-store-references): Add support for finding and rewriting UTF-16 and UTF-32 store references. * tests/grafts.scm: Add tests. --- guix/build/graft.scm | 281 +++++++++++++++++++++++++++++-------------- tests/grafts.scm | 68 +++++++++++ 2 files changed, 258 insertions(+), 91 deletions(-) diff --git a/guix/build/graft.scm b/guix/build/graft.scm index c119ee71d1..30be988587 100644 --- a/guix/build/graft.scm +++ b/guix/build/graft.scm @@ -1,6 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2014, 2015, 2016, 2018 Ludovic Court=C3=A8s -;;; Copyright =C2=A9 2016 Mark H Weaver +;;; Copyright =C2=A9 2016, 2021 Mark H Weaver ;;; ;;; This file is part of GNU Guix. ;;; @@ -55,6 +55,52 @@ (string->char-set "0123456789abcdfghijklmnpqrsvwxyz") <>)) =20 +(define (nix-base32-char-or-nul? c) + "Return true if C is a nix-base32 character or NUL, otherwise return fal= se." + (or (nix-base32-char? c) + (char=3D? c #\nul))) + +(define (possible-utf16-hash? buffer i w) + "Return true if (I - W) is large enough to hold a UTF-16 encoded +nix-base32 hash and if BUFFER contains NULs in all positions where NULs +are to be expected in a UTF-16 encoded hash+dash pattern whose dash is +found at position I. Otherwise, return false." + (and (<=3D (* 2 hash-length) (- i w)) + (let loop ((j (+ 1 (- i (* 2 hash-length))))) + (or (>=3D j i) + (and (zero? (bytevector-u8-ref buffer j)) + (loop (+ j 2))))))) + +(define (possible-utf32-hash? buffer i w) + "Return true if (I - W) is large enough to hold a UTF-32 encoded +nix-base32 hash and if BUFFER contains NULs in all positions where NULs +are to be expected in a UTF-32 encoded hash+dash pattern whose dash is +found at position I. Otherwise, return false." + (and (<=3D (* 4 hash-length) (- i w)) + (let loop ((j (+ 1 (- i (* 4 hash-length))))) + (or (>=3D j i) + (and (zero? (bytevector-u8-ref buffer j)) + (zero? (bytevector-u8-ref buffer (+ j 1))) + (zero? (bytevector-u8-ref buffer (+ j 2))) + (loop (+ j 4))))))) + +(define (insert-nuls char-size bv) + "Given a bytevector BV, return a bytevector containing the same bytes but +with (CHAR-SIZE - 1) NULs inserted between every two adjacent bytes from B= V. +For example, (insert-nuls 4 #u8(1 2 3)) =3D> #u8(1 0 0 0 2 0 0 0 3)." + (if (=3D char-size 1) + bv + (let* ((len (bytevector-length bv)) + (bv* (make-bytevector (+ 1 (* char-size + (- len 1))) + 0))) + (let loop ((i 0)) + (when (< i len) + (bytevector-u8-set! bv* (* i char-size) + (bytevector-u8-ref bv i)) + (loop (+ i 1)))) + bv*))) + (define* (replace-store-references input output replacement-table #:optional (store (%store-directory))) "Read data from INPUT, replacing store references according to @@ -76,9 +122,9 @@ bytevectors to the same value." (list->vector (map pred (iota 256))) <>)) =20 - (define nix-base32-byte? + (define nix-base32-byte-or-nul? (optimize-u8-predicate - (compose nix-base32-char? + (compose nix-base32-char-or-nul? integer->char))) =20 (define (dash? byte) (=3D byte 45)) @@ -86,100 +132,153 @@ bytevectors to the same value." (define request-size (expt 2 20)) ; 1 MiB =20 ;; We scan the file for the following 33-byte pattern: 32 bytes of - ;; nix-base32 characters followed by a dash. To accommodate large files, - ;; we do not read the entire file, but instead work on buffers of up to - ;; 'request-size' bytes. To ensure that every 33-byte sequence appears - ;; entirely within exactly one buffer, adjacent buffers must overlap, - ;; i.e. they must share 32 byte positions. We accomplish this by - ;; "ungetting" the last 32 bytes of each buffer before reading the next - ;; buffer, unless we know that we've reached the end-of-file. + ;; nix-base32 characters followed by a dash. When we find such a pattern + ;; whose hash is in REPLACEMENT-TABLE, we perform the required rewrite a= nd + ;; continue scanning. + ;; + ;; To support UTF-16 and UTF-32 store references, the 33 bytes comprising + ;; this hash+dash pattern may optionally be interspersed by extra NUL by= tes. + ;; This simple approach works because the characters we are looking for = are + ;; restricted to ASCII. UTF-16 hashes are interspersed with single NUL + ;; bytes ("\0"), and UTF-32 hashes are interspersed with triplets of NULs + ;; ("\0\0\0"). Note that we require NULs to be present only *between* t= he + ;; other bytes, and not at either end, in order to be insensitive to byte + ;; order. + ;; + ;; To accommodate large files, we do not read the entire file at once, b= ut + ;; instead work on buffers of up to REQUEST-SIZE bytes. To ensure that + ;; every hash+dash pattern appears in its entirety in at least one buffe= r, + ;; adjacent buffers must overlap by one byte less than the maximum size = of a + ;; hash+dash pattern. We accomplish this by "ungetting" a suffix of each + ;; buffer before reading the next buffer, unless we know that we've reac= hed + ;; the end-of-file. (let ((buffer (make-bytevector request-size))) - (let loop () - ;; Note: We avoid 'get-bytevector-n' to work around - ;; . + (define-syntax-rule (byte-at i) + (bytevector-u8-ref buffer i)) + (let outer-loop () (match (get-bytevector-n! input buffer 0 request-size) ((? eof-object?) 'done) (end - ;; We scan the buffer for dashes that might be preceded by a - ;; nix-base32 hash. The key optimization here is that whenever we - ;; find a NON-nix-base32 character at position 'i', we know that = it - ;; cannot be part of a hash, so the earliest position where the n= ext - ;; hash could start is i+1 with the following dash at position i+= 33. - ;; - ;; Since nix-base32 characters comprise only 1/8 of the 256 possi= ble - ;; byte values, and exclude some of the most common letters in - ;; English text (e t o u), in practice we can advance by 33 posit= ions - ;; most of the time. - (let scan-from ((i hash-length) (written 0)) - ;; 'i' is the first position where we look for a dash. 'writte= n' - ;; is the number of bytes in the buffer that have already been - ;; written. + (define (scan-from i w) + ;; Scan the buffer for dashes that might be preceded by nix has= hes, + ;; where I is the minimum position where such a dash might be + ;; found, and W is the number of bytes in the buffer that have = been + ;; written so far. We assume that I - W >=3D HASH-LENGTH. + ;; + ;; The key optimization here is that whenever we find a byte at + ;; position I that cannot occur within a nix hash (because it's + ;; neither a nix-base32 character nor NUL), we can infer that t= he + ;; earliest position where the next hash could start is at I + = 1, + ;; and therefore the earliest position for the following dash is + ;; (+ I 1 HASH-LENGTH), which is I + 33. + ;; + ;; Since nix-base32-or-nul characters comprise only about 1/8 of + ;; the 256 possible byte values, and exclude some of the most + ;; common letters in English text (e t o u), we can advance 33 + ;; positions much of the time. (if (< i end) - (let ((byte (bytevector-u8-ref buffer i))) - (cond ((and (dash? byte) - ;; We've found a dash. Note that we do not k= now - ;; whether the preceeding 32 bytes are nix-ba= se32 - ;; characters, but we do not need to know. If - ;; they are not, the following lookup will fa= il. - (lookup-replacement - (string-tabulate (lambda (j) - (integer->char - (bytevector-u8-ref buffer - (+ j (- i hash-length))= ))) - hash-length))) - =3D> (lambda (replacement) - ;; We've found a hash that needs to be replac= ed. - ;; First, write out all bytes preceding the h= ash - ;; that have not yet been written. - (put-bytevector output buffer written - (- i hash-length written)) - ;; Now write the replacement string. - (put-bytevector output replacement) - ;; Since the byte at position 'i' is a dash, - ;; which is not a nix-base32 char, the earlie= st - ;; position where the next hash might start is - ;; i+1, and the earliest position where the - ;; following dash might start is (+ i 1 - ;; hash-length). Also, increase the write - ;; position to account for REPLACEMENT. - (let ((len (bytevector-length replacement))) - (scan-from (+ i 1 len) - (+ i (- len hash-length)))))) - ;; If the byte at position 'i' is a nix-base32 char, - ;; then the dash we're looking for might be as earl= y as - ;; the following byte, so we can only advance by 1. - ((nix-base32-byte? byte) - (scan-from (+ i 1) written)) - ;; If the byte at position 'i' is NOT a nix-base32 - ;; char, then the earliest position where the next = hash - ;; might start is i+1, with the following dash at - ;; position (+ i 1 hash-length). + (let ((byte (byte-at i))) + (cond ((dash? byte) + (found-dash i w)) + ((nix-base32-byte-or-nul? byte) + (scan-from (+ i 1) w)) (else - (scan-from (+ i 1 hash-length) written)))) - - ;; We have finished scanning the buffer. Now we determine = how - ;; many bytes have not yet been written, and how many bytes= to - ;; "unget". If 'end' is less than 'request-size' then we r= ead - ;; less than we asked for, which indicates that we are at E= OF, - ;; so we needn't unget anything. Otherwise, we unget up to - ;; 'hash-length' bytes (32 bytes). However, we must be car= eful - ;; not to unget bytes that have already been written, becau= se - ;; that would cause them to be written again from the next - ;; buffer. In practice, this case occurs when a replacemen= t is - ;; made near or beyond the end of the buffer. When REPLACE= MENT - ;; went beyond END, we consume the extra bytes from INPUT. - (begin - (if (> written end) - (get-bytevector-n! input buffer 0 (- written end)) - (let* ((unwritten (- end written)) - (unget-size (if (=3D end request-size) - (min hash-length unwritten) - 0)) - (write-size (- unwritten unget-size))) - (put-bytevector output buffer written write-size) - (unget-bytevector input buffer (+ written write-siz= e) - unget-size))) - (loop))))))))) + (not-part-of-hash i w)))) + (finish-buffer i w))) + + (define (not-part-of-hash i w) + ;; Position I is known to not be within a nix hash that we must + ;; rewrite. Therefore, the earliest position where the next ha= sh + ;; might start is I + 1, and therefore the earliest position of + ;; the following dash is (+ I 1 HASH-LENGTH). + (scan-from (+ i 1 hash-length) w)) + + (define (found-dash i w) + ;; We know that there is a dash '-' at position I, and that + ;; I >=3D HASH-LENGTH. The immediately preceding bytes *might* + ;; contain a nix-base32 hash, but that is not yet known. Here, + ;; we rule out all but one possible encoding (ASCII, UTF-16, + ;; UTF-32) by counting how many NULs precede the dash. + (cond ((not (zero? (byte-at (- i 1)))) + ;; The dash is *not* preceded by a NUL, therefore it + ;; cannot possibly be a UTF-16 or UTF-32 hash. Proceed + ;; to check for an ASCII hash. + (found-possible-hash 1 i w)) + + ((not (zero? (byte-at (- i 2)))) + ;; The dash is preceded by exactly one NUL, therefore it + ;; cannot be an ASCII or UTF-32 hash. Proceed to check + ;; for a UTF-16 hash. + (if (possible-utf16-hash? buffer i w) + (found-possible-hash 2 i w) + (not-part-of-hash i w))) + + (else + ;; The dash is preceded by at least two NULs, therefore + ;; it cannot be an ASCII or UTF-16 hash. Proceed to + ;; check for a UTF-32 hash. + (if (possible-utf32-hash? buffer i w) + (found-possible-hash 4 i w) + (not-part-of-hash i w))))) + + (define (found-possible-hash char-size i w) + ;; We know that there is a dash '-' at position I, that + ;; I >=3D CHAR-SIZE * HASH-LENGTH, and that the only possible + ;; encoding for the preceding hash is as indicated by + ;; CHAR-SIZE. Here we check to see if the given hash is in + ;; REPLACEMENT-TABLE, and if so, we perform the required + ;; rewrite. + (let* ((hash (string-tabulate + (lambda (j) + (integer->char + (byte-at (- i (* char-size + (- hash-length j)))))) + hash-length)) + (replacement* (lookup-replacement hash)) + (replacement (and replacement* + (insert-nuls char-size replacement*)))) + (cond + ((not replacement) + (not-part-of-hash i w)) + (else + ;; We've found a hash that needs to be replaced. + ;; First, write out all bytes preceding the hash + ;; that have not yet been written. + (put-bytevector output buffer w + (- i (* char-size hash-length) w)) + ;; Now write the replacement string. + (put-bytevector output replacement) + ;; Now compute the new values of W and I and continue. + (let ((w (+ (- i (* char-size hash-length)) + (bytevector-length replacement)))) + (scan-from (+ w hash-length) w)))))) + + (define (finish-buffer i w) + ;; We have finished scanning the buffer. Now we determine how = many + ;; bytes have not yet been written, and how many bytes to "unge= t". + ;; If END is less than REQUEST-SIZE then we read less than we a= sked + ;; for, which indicates that we are at EOF, so we needn't unget + ;; anything. Otherwise, we unget up to (* 4 HASH-LENGTH) bytes. + ;; However, we must be careful not to unget bytes that have alr= eady + ;; been written, because that would cause them to be written ag= ain + ;; from the next buffer. In practice, this case occurs when a + ;; replacement is made near or beyond the end of the buffer. W= hen + ;; REPLACEMENT went beyond END, we consume the extra bytes from + ;; INPUT. + (if (> w end) + (get-bytevector-n! input buffer 0 (- w end)) + (let* ((unwritten (- end w)) + (unget-size (if (=3D end request-size) + (min (* 4 hash-length) + unwritten) + 0)) + (write-size (- unwritten unget-size))) + (put-bytevector output buffer w write-size) + (unget-bytevector input buffer (+ w write-size) + unget-size))) + (outer-loop)) + + (scan-from hash-length 0)))))) =20 (define (rename-matching-files directory mapping) "Apply MAPPING to the names of all the files in DIRECTORY, where MAPPING= is diff --git a/tests/grafts.scm b/tests/grafts.scm index a12c6a5911..0e1c7355b1 100644 --- a/tests/grafts.scm +++ b/tests/grafts.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Court=C3= =A8s +;;; Copyright =C2=A9 2021 Mark H Weaver ;;; ;;; This file is part of GNU Guix. ;;; @@ -468,4 +469,71 @@ replacement "/gnu/store"))))) =20 +(define (nul-expand str char-size) + (string-join (map string (string->list str)) + (make-string (- char-size 1) #\nul))) + +(for-each + (lambda (char-size1) + (for-each + (lambda (char-size2) + (for-each + (lambda (gap) + (for-each + (lambda (offset) + (test-equal (format #f "replace-store-references, char-sizes ~a= ~a, gap ~s, offset ~a" + char-size1 char-size2 gap offset) + (string-append (make-string offset #\=3D) + (nul-expand (string-append "/gnu/store/" + (make-string 32 #\6) + "-BlahBlaH") + char-size1) + gap + (nul-expand (string-append "/gnu/store/" + (make-string 32 #\8) + "-SoMeTHiNG") + char-size2) + (list->string (map integer->char (iota 77 33))= )) + + ;; Create input data where the right-hand-size of the dash ("= -something" + ;; here) goes beyond the end of the internal buffer of + ;; 'replace-store-references'. + (let* ((content (string-append (make-string offset #\=3D) + (nul-expand (string-append= "/gnu/store/" + = (make-string 32 #\5) + = "-blahblah") + char-size1) + gap + (nul-expand (string-append= "/gnu/store/" + = (make-string 32 #\7) + = "-something") + char-size2) + (list->string + (map integer->char (iota = 77 33))))) + (replacement (alist->vhash + `((,(make-string 32 #\5) + . ,(string->utf8 (string-append + (make-string 32 #\6) + "-BlahBlaH"))) + (,(make-string 32 #\7) + . ,(string->utf8 (string-append + (make-string 32 #\8) + "-SoMeTHiNG"))))))) + (call-with-output-string + (lambda (output) + ((@@ (guix build graft) replace-store-references) + (open-input-string content) output + replacement + "/gnu/store")))))) + ;; offsets to test + (map (lambda (i) (- buffer-size (* 40 char-size1) i)) + (iota 30)))) + ;; gaps + '("" "-" " " "a"))) + ;; char-size2 values to test + '(1 2))) + ;; char-size1 values to test + '(1 2 4)) + + (test-end) --=20 2.31.1 --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 14 Apr 2021 10:56:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16183977555907 (code B ref 33848); Wed, 14 Apr 2021 10:56:01 +0000 Received: (at 33848) by debbugs.gnu.org; 14 Apr 2021 10:55:55 +0000 Received: from localhost ([127.0.0.1]:33692 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWdBa-0001XD-Lu for submit@debbugs.gnu.org; Wed, 14 Apr 2021 06:55:54 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47764) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWdBY-0001Wy-Mb for 33848@debbugs.gnu.org; Wed, 14 Apr 2021 06:55:53 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40213) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lWdBS-0003j8-1W; Wed, 14 Apr 2021 06:55:46 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=34842 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lWdBQ-0005yR-Qr; Wed, 14 Apr 2021 06:55:45 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <87k1juaomo.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87h7kj7j7x.fsf@netris.org> <87ft01axta.fsf@gnu.org> <87k0p6rlt5.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 25 Germinal an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 14 Apr 2021 12:55:43 +0200 In-Reply-To: <87k0p6rlt5.fsf@netris.org> (Mark H. Weaver's message of "Tue, 13 Apr 2021 16:06:19 -0400") Message-ID: <87eefdf840.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Mark, Mark H Weaver skribis: >> The risks of bugs I can think of are: missed ASCII references (a >> regression, whereby some ASCII references would not get rewritten), > > This is indeed a risk whenever we modify the grafting code, which is > written for efficiency, not clarity. I've tried to be careful, and have > checked that my newly grafted system and user profiles do not retain > references to ungrafted code, modulo the following pre-existing issues: > > > (ibus-daemon launches ungrafted subprocesses) > > > (Chunked store references in .zo files in Racket 8) OK. >> and unrelated UTF-{16,32}-base32-looking references getting rewritten. >> >> I guess the latter is very unlikely because only sequences found in the >> replacement table may be rewritten, right? > > Yes, that's correct. > >> The former should be caught by =E2=80=98tests/grafts.scm=E2=80=99 but we= could also >> check the closure of real-world systems, for instance, to make sure it >> doesn=E2=80=99t refer to ungrafted things. > > As I mention above, I've already done so for my own GNOME-based Guix > system. Yes, we should be safe. >> Do you know how this affects performance? > > I have not yet measured it, but subjectively, I do not notice any > obvious difference in speed. > > I expect that the main performance impact is that large blocks of NULs > will be processed more slowly by the current draft version of the new > grafting code. That's because NULs can now be part of a nix hash, and > therefore the new grafting code can only advance 1 byte position when > seeing a NUL, whereas previously it would skip ahead 33 positions in > that case. > > If desired, the handling of NULs could be made more efficient, at the > cost of a bit more complexity. When seeing a NUL, we could check the > adjacent bytes to see if this is part of a nix-base32 character in > UTF-16 or UTF-32 encoding. If not, we could skip ahead. Let=E2=80=99s keep it this way for now; we can always revisit later if we f= eel the need for it. >> For clarity, perhaps you can define a top-level procedure for the test >> and call it from =E2=80=98for-each=E2=80=99. > > Good idea. I'd like to optimize the tests a bit before pushing this. > They take a fairly long time to run, and lead to a huge 1.5G grafts.log > file. It might be sufficient to avoid 'test-equal' here. Ah yes, rather use =E2=80=98test-assert=E2=80=99 or some such to avoid the = huge log file. :-) > Below, I've attached my current revision of this draft patch, which > incorporates your suggestions regarding the main code. What remains is > to improve the tests. LGTM! Feel free to push this version or an improved one. I think it=E2=80= =99s good to have it in the upcoming release, and if it=E2=80=99s pushed sooner, we=E2=80=99ll have more time to react in case something=E2=80=99s wrong. Thank you! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 14 11:19:44 2021 Received: (at control) by debbugs.gnu.org; 14 Apr 2021 15:19:44 +0000 Received: from localhost ([127.0.0.1]:35538 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWhIt-00079m-T5 for submit@debbugs.gnu.org; Wed, 14 Apr 2021 11:19:44 -0400 Received: from eggs.gnu.org ([209.51.188.92]:42570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWhIr-00079X-Jz for control@debbugs.gnu.org; Wed, 14 Apr 2021 11:19:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:45332) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lWhIl-0005nV-Ri for control@debbugs.gnu.org; Wed, 14 Apr 2021 11:19:36 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=35240 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lWhIl-00038J-9p for control@debbugs.gnu.org; Wed, 14 Apr 2021 11:19:35 -0400 Date: Wed, 14 Apr 2021 17:19:33 +0200 Message-Id: <871rbcgagq.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #47297 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) block 47297 by 33848 quit From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Leo Famulari Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 14 Apr 2021 22:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161843987728997 (code B ref 33848); Wed, 14 Apr 2021 22:38:02 +0000 Received: (at 33848) by debbugs.gnu.org; 14 Apr 2021 22:37:57 +0000 Received: from localhost ([127.0.0.1]:36013 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWo8y-0007Xc-D3 for submit@debbugs.gnu.org; Wed, 14 Apr 2021 18:37:57 -0400 Received: from out2-smtp.messagingengine.com ([66.111.4.26]:38175) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWo8u-0007XO-Pv for 33848@debbugs.gnu.org; Wed, 14 Apr 2021 18:37:55 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id A7D405C00D0; Wed, 14 Apr 2021 18:37:47 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Wed, 14 Apr 2021 18:37:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-transfer-encoding:in-reply-to; s=mesmtp; bh=BJZl/sDOcR1J34XSPGjDNMW6X5Bv74X+FH6Aj24Hxl4=; b=VIM/x65moZbD lLNiollMNBxwD4H0Ep9i0VGK23UZOzVFwo6MQALWAUl+U/4H9HyM5E4hvf9XzyVA +5RWBAMq8McajImSJMzfZcbb9LycAuyedDgB6ExSvC52OdeSwrV6gX4IbzPqVJD3 fYpzB8caEc1V5yq0MLaWgSmhChQTrWs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=BJZl/sDOcR1J34XSPGjDNMW6X5Bv74X+FH6Aj24Hx l4=; b=T2IlReOST7dWgEKOaob3GN2izSNVsOSxXyHtzqqvU4fhaZ5Ie+8nmIq5H km4IVOnV0+SuaA3hrtaOQxJ/TIs3zNS3Mud2G1q1iyyLqsf1X/T/fsusdzc1hTbw 0ugkm6UAFlYu+wiKlyF0z4OHzyyE6zK7Ley1TMC92P7j5oR0TQrGrJnNgkAi1Icz x0d1zRbMtFwkDcOlkI6OXXOUCpMqJCoSS01eWljQ3JyWfWv4TgkBI8jYhZ/v3AgX AIF+203O0SaKIIrTecaLTAW6Q5pcy2Z9EU26VWgHlfaOgRFlPCfvbhFAboDgaQnj Ucy/JuPm6XIzQ0mhbsIFNToQU6UvQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudelvddguddtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtugfgjgesthekredttddtjeenucfhrhhomhepnfgvohcu hfgrmhhulhgrrhhiuceolhgvohesfhgrmhhulhgrrhhirdhnrghmvgeqnecuggftrfgrth htvghrnhepgeejgeeghedtudfgffdutddvffefffejkeffffevffehgedvvdeutdffkeej jeejnecukfhppedutddtrdduuddrudeiledruddukeenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehlvghosehfrghmuhhlrghrihdrnhgrmhgv X-ME-Proxy: Received: from localhost (pool-100-11-169-118.phlapa.fios.verizon.net [100.11.169.118]) by mail.messagingengine.com (Postfix) with ESMTPA id 4E40B240069; Wed, 14 Apr 2021 18:37:47 -0400 (EDT) Date: Wed, 14 Apr 2021 18:37:45 -0400 From: Leo Famulari Message-ID: References: <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87h7kj7j7x.fsf@netris.org> <87ft01axta.fsf@gnu.org> <87k0p6rlt5.fsf@netris.org> <87eefdf840.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <87eefdf840.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On Wed, Apr 14, 2021 at 12:55:43PM +0200, Ludovic Courtès wrote: > LGTM! Feel free to push this version or an improved one. I think it’s > good to have it in the upcoming release, and if it’s pushed sooner, > we’ll have more time to react in case something’s wrong. Just FYI to everyone reading, we aim to release 1.2.1 this Sunday, April 18. From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 15 Apr 2021 07:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161847171415050 (code B ref 33848); Thu, 15 Apr 2021 07:29:02 +0000 Received: (at 33848) by debbugs.gnu.org; 15 Apr 2021 07:28:34 +0000 Received: from localhost ([127.0.0.1]:36345 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWwQU-0003ug-3o for submit@debbugs.gnu.org; Thu, 15 Apr 2021 03:28:34 -0400 Received: from world.peace.net ([64.112.178.59]:39706) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lWwQO-0003uJ-Ul for 33848@debbugs.gnu.org; Thu, 15 Apr 2021 03:28:32 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lWwQH-0003t1-Nj; Thu, 15 Apr 2021 03:28:21 -0400 From: Mark H Weaver In-Reply-To: <87eefdf840.fsf@gnu.org> References: <87r2e8jpfx.fsf@gnu.org> <87muoqhk62.fsf@ambrevar.xyz> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87h7kj7j7x.fsf@netris.org> <87ft01axta.fsf@gnu.org> <87k0p6rlt5.fsf@netris.org> <87eefdf840.fsf@gnu.org> Date: Thu, 15 Apr 2021 03:26:33 -0400 Message-ID: <871rbcj9dn.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Ludovic Court=C3=A8s writes: > LGTM! Feel free to push this version or an improved one. I think it=E2= =80=99s > good to have it in the upcoming release, and if it=E2=80=99s pushed soone= r, > we=E2=80=99ll have more time to react in case something=E2=80=99s wrong. I pushed an improved version of the patch to 'master' as commit 1bab9b9f17256a9e4f45f5b0cceb8b52e0a1b1ed. Thanks, Mark From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 16 Apr 2021 09:45:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver , Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Guillaume Le Vaillant , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161856627331678 (code B ref 33848); Fri, 16 Apr 2021 09:45:02 +0000 Received: (at 33848) by debbugs.gnu.org; 16 Apr 2021 09:44:33 +0000 Received: from localhost ([127.0.0.1]:40062 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lXL1d-0008Es-Fl for submit@debbugs.gnu.org; Fri, 16 Apr 2021 05:44:33 -0400 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:33155) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lXL1a-0008Ec-5p for 33848@debbugs.gnu.org; Fri, 16 Apr 2021 05:44:32 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 367A8FF810; Fri, 16 Apr 2021 09:44:22 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <871rbcj9dn.fsf@netris.org> References: <87r2e8jpfx.fsf@gnu.org> <87zhsq8wkj.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87h7kj7j7x.fsf@netris.org> <87ft01axta.fsf@gnu.org> <87k0p6rlt5.fsf@netris.org> <87eefdf840.fsf@gnu.org> <871rbcj9dn.fsf@netris.org> Date: Fri, 16 Apr 2021 11:44:22 +0200 Message-ID: <87v98mva15.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Just tested like Ludo did: Nyxt works now without the --no-grafts option! Thank you so much, Mark, this is a huge step forward for Nyxt (and Guix for course)! :) -- Pierre Neidhardt https://ambrevar.xyz/ Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.183.199 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.199 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.8 (/) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Just tested like Ludo did: Nyxt works now without the --no-grafts option! Thank you so much, Mark, this is a huge step forward for Nyxt (and Guix for course)! :) =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmB5XHYSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/WZ0H/Rlahh8Sq7sOjMiQMxmRBwLbx/ZixNrg MaDF4yiEIYBTixVTK1pXuDnbj460tp/VtgP0Dhsb0dZ49xz9FEkcY9h2X0eJD0l+ gaxXoRCc5b0ObaUqdtdAWnfv5++srz9OqCUN7Bxkhh5ydVVxWXHLEkuqHsVyolOv j7sA0d501KP230NaxhntAONQtxuQD19DwVmz0IuW1trFvvWZjpVAgHR/KBXKuZs0 uh8oJYRt94OE6ubVV+nY6eSOC+2V0XvhE6fqmSbvPA7R8EvRGdy9/+I07vWKUPw4 lGhhioT4Ss1lsUxmK1Ls4caurTzzXb8g/MerhDJlM6vbENqpogG3n2o= =EQ8X -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 22 09:36:15 2021 Received: (at control) by debbugs.gnu.org; 22 Apr 2021 13:36:15 +0000 Received: from localhost ([127.0.0.1]:33188 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZZV9-0006xj-1M for submit@debbugs.gnu.org; Thu, 22 Apr 2021 09:36:15 -0400 Received: from eggs.gnu.org ([209.51.188.92]:54810) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lZZV8-0006xV-0O for control@debbugs.gnu.org; Thu, 22 Apr 2021 09:36:14 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49244) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZZV2-0001Ym-Og for control@debbugs.gnu.org; Thu, 22 Apr 2021 09:36:08 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=33248 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lZZV0-0001r5-9r for control@debbugs.gnu.org; Thu, 22 Apr 2021 09:36:07 -0400 Date: Thu, 22 Apr 2021 15:36:04 +0200 Message-Id: <877dku1lx7.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #33848 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) tags 33848 fixed close 33848 quit From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 30 Apr 2021 20:05:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: fixed To: Pierre Neidhardt , Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Guillaume Le Vaillant , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16198130623582 (code B ref 33848); Fri, 30 Apr 2021 20:05:01 +0000 Received: (at 33848) by debbugs.gnu.org; 30 Apr 2021 20:04:22 +0000 Received: from localhost ([127.0.0.1]:59883 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcZN8-0000vi-9J for submit@debbugs.gnu.org; Fri, 30 Apr 2021 16:04:22 -0400 Received: from world.peace.net ([64.112.178.59]:45676) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcZN6-0000vQ-70 for 33848@debbugs.gnu.org; Fri, 30 Apr 2021 16:04:20 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lcZMw-0005X6-Lp; Fri, 30 Apr 2021 16:04:10 -0400 From: Mark H Weaver In-Reply-To: <87wntfrfd1.fsf@ambrevar.xyz> References: <87r2e8jpfx.fsf@gnu.org> <87d0pmhbgn.fsf@ambrevar.xyz> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> <87ft04sefs.fsf@gnu.org> <8735w472oo.fsf@netris.org> <87v98zomjv.fsf@gnu.org> <87wntfrfd1.fsf@ambrevar.xyz> Date: Fri, 30 Apr 2021 16:03:18 -0400 Message-ID: <878s4zo7z2.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Pierre, and Ludovic, Pierre Neidhardt writes: > I also agree this looks like a net win even though the `guix gc` issue > is not fix. > The latter seems to be relatively easier to fix, doesn't it? Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Hi Pierre, and Ludovic, Pierre Neidhardt writes: > I also agree this looks like a net win even though the `guix gc` issue > is not fix. > The latter seems to be relatively easier to fix, doesn't it? Yes. I now have a plan to finally fix this bug for good. I intend to write a scanner in Scheme that is able to find Nix hashes encoded in ASCII, UTF-16, or UTF-32. Using that, I'll write a procedure that, for each package output, finds all store references that are found encoded in UTF-16 or UTF-32 but never in ASCII, and write those references to a file (if that set is nonempty). This procedure can then be used by selected packages and/or build-systems. However, there's one thing I will need: the set of all transitive inputs (and native-inputs, including implicit inputs) available to the build, i.e. the set of possible hashes that might legitimately be found by the scanner. Ludovic: what's the best way to get that list from the build-side code? Thanks, Mark -- Disinformation flourishes because many people care deeply about injustice but very few check the facts. Ask me about . From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Pierre Neidhardt Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 01 May 2021 09:23:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: fixed To: Mark H Weaver , Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Guillaume Le Vaillant , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.161986095011311 (code B ref 33848); Sat, 01 May 2021 09:23:03 +0000 Received: (at 33848) by debbugs.gnu.org; 1 May 2021 09:22:30 +0000 Received: from localhost ([127.0.0.1]:60275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lclpV-0002wH-Oh for submit@debbugs.gnu.org; Sat, 01 May 2021 05:22:30 -0400 Received: from relay5-d.mail.gandi.net ([217.70.183.197]:54209) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lclpT-0002u2-1x for 33848@debbugs.gnu.org; Sat, 01 May 2021 05:22:27 -0400 X-Originating-IP: 92.169.147.163 Received: from bababa (lfbn-idf2-1-1335-163.w92-169.abo.wanadoo.fr [92.169.147.163]) (Authenticated sender: mail@ambrevar.xyz) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id D5DE01C0007; Sat, 1 May 2021 09:22:19 +0000 (UTC) From: Pierre Neidhardt In-Reply-To: <878s4zo7z2.fsf@netris.org> References: <87r2e8jpfx.fsf@gnu.org> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> <87ft04sefs.fsf@gnu.org> <8735w472oo.fsf@netris.org> <87v98zomjv.fsf@gnu.org> <87wntfrfd1.fsf@ambrevar.xyz> <878s4zo7z2.fsf@netris.org> Date: Sat, 01 May 2021 11:22:19 +0200 Message-ID: <87fsz6st9w.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Mark, thanks for the update and thanks for getting your hands dirty, this is a very valuable contribution to the core of Guix! Cheers! Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.183.197 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.197 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Mark, thanks for the update and thanks for getting your hands dirty, this is a very valuable contribution to the core of Guix! Cheers! Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [217.70.183.197 listed in wl.mailspike.net] -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [217.70.183.197 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: ambrevar.xyz (xyz)] -0.0 SPF_PASS SPF: sender matches SPF record -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager 1.0 BULK_RE_SUSP_NTLD Precedence bulk and RE: from a suspicious TLD 0.5 FROM_SUSPICIOUS_NTLD From abused NTLD --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi Mark, thanks for the update and thanks for getting your hands dirty, this is a very valuable contribution to the core of Guix! Cheers! =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmCNHcsSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/e7UIAJth7AUZl+5Y0is/vcfuMijMwswE0x2E 1IYnWiB9HVnsf2GCp/RoSVwNZzowbH4ehflXCfwS92LD2LlYtv8UzTDL4qT3bbpU CUckgLXi6uSeSdUPnGqmsAozvq0hOMjTCvhgaVgdiKgEDVb4yupkdwLKgeOooBBU Ghrnh4YXN/YVOXGB55PFpu5GW8ZUZvMfaav3SrOHKrvvlZZO7PbYA6dfXM/yFL6G tZH/Iicgjfy+TwgZcIfBPKjRWh9GiIzE7C5EgQ2SJkYWxRPsGEPwU81GclibR3a7 IYeCpn41MbbVqmsXoD7O0GGkskZ7TPmdzKcmSocagvRo8sRVVtiu+O8= =gs+Y -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 15 08:43:37 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33848: Store references in SBCL-compiled code are "invisible" Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 11 May 2021 08:47:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33848 X-GNU-PR-Package: guix X-GNU-PR-Keywords: fixed To: Mark H Weaver Cc: Guillaume Le Vaillant , Pierre Neidhardt , 33848@debbugs.gnu.org Received: via spool by 33848-submit@debbugs.gnu.org id=B33848.16207227788871 (code B ref 33848); Tue, 11 May 2021 08:47:01 +0000 Received: (at 33848) by debbugs.gnu.org; 11 May 2021 08:46:18 +0000 Received: from localhost ([127.0.0.1]:34894 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lgO1y-0002J0-Bk for submit@debbugs.gnu.org; Tue, 11 May 2021 04:46:18 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lgO1w-0002Ii-Ce for 33848@debbugs.gnu.org; Tue, 11 May 2021 04:46:16 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:42378) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgO1o-00024w-AK; Tue, 11 May 2021 04:46:09 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40030 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgO1n-000526-LL; Tue, 11 May 2021 04:46:08 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87r2e8jpfx.fsf@gnu.org> <87r2e28tkv.fsf@gnu.org> <874laygkiy.fsf@ambrevar.xyz> <87lfa5eymf.fsf@ambrevar.xyz> <87tuoscsk9.fsf@gnu.org> <87im57b8u7.fsf@ambrevar.xyz> <87czvebky2.fsf@netris.org> <87eefu30a4.fsf@gnu.org> <87im56l6es.fsf@yamatai> <87wntm8j18.fsf@ambrevar.xyz> <87a6qil4b1.fsf@yamatai> <87a6qiz5b3.fsf@ambrevar.xyz> <871rbtc3j5.fsf@netris.org> <87r1js9udv.fsf@netris.org> <87sg47vp16.fsf@ambrevar.xyz> <875z139liy.fsf@netris.org> <87ft04sefs.fsf@gnu.org> <8735w472oo.fsf@netris.org> <87v98zomjv.fsf@gnu.org> <87wntfrfd1.fsf@ambrevar.xyz> <878s4zo7z2.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 22 =?UTF-8?Q?Flor=C3=A9al?= an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 11 May 2021 10:46:05 +0200 In-Reply-To: <878s4zo7z2.fsf@netris.org> (Mark H. Weaver's message of "Fri, 30 Apr 2021 16:03:18 -0400") Message-ID: <878s4l4q0i.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Mark, Mark H Weaver skribis: > I intend to write a scanner in Scheme that is able to find Nix hashes > encoded in ASCII, UTF-16, or UTF-32. Using that, I'll write a procedure > that, for each package output, finds all store references that are found > encoded in UTF-16 or UTF-32 but never in ASCII, and write those > references to a file (if that set is nonempty). This procedure can then > be used by selected packages and/or build-systems. > > However, there's one thing I will need: the set of all transitive inputs > (and native-inputs, including implicit inputs) available to the build, > i.e. the set of possible hashes that might legitimately be found by the > scanner. > > Ludovic: what's the best way to get that list from the build-side code? You can use #:references-graphs for that. Sorry for the delay! Ludo=E2=80=99.