From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 14 11:47:20 2018 Received: (at submit) by debbugs.gnu.org; 14 Mar 2018 15:47:20 +0000 Received: from localhost ([127.0.0.1]:33459 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ew8ca-0000lj-0E for submit@debbugs.gnu.org; Wed, 14 Mar 2018 11:47:20 -0400 Received: from eggs.gnu.org ([208.118.235.92]:46811) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ew8cY-0000lV-Jo for submit@debbugs.gnu.org; Wed, 14 Mar 2018 11:47:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ew8cS-00065l-4D for submit@debbugs.gnu.org; Wed, 14 Mar 2018 11:47:13 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:44257) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ew8cS-00065c-0P for submit@debbugs.gnu.org; Wed, 14 Mar 2018 11:47:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37776) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ew8cQ-0008Rr-MP for bug-guix@gnu.org; Wed, 14 Mar 2018 11:47:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ew8cN-00063B-Hx for bug-guix@gnu.org; Wed, 14 Mar 2018 11:47:10 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:49686) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ew8cN-00062O-C6 for bug-guix@gnu.org; Wed, 14 Mar 2018 11:47:07 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id E791112A35; Wed, 14 Mar 2018 16:47:05 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IuyLKScanBAy; Wed, 14 Mar 2018 16:47:04 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id A55EF12A33; Wed, 14 Mar 2018 16:47:04 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: bug-guix@gnu.org Subject: Chunked store references in compiled code break grafting (again) X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 24 =?utf-8?Q?Vent=C3=B4se?= an 226 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 14 Mar 2018 16:47:04 +0100 Message-ID: <87o9jq7j7r.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit Cc: Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hello Guix, The recently added glibc grafts triggered issues that, in the end, show the return of (=E2=80=9CStore references in 8-b= yte chunks in compiled code=E2=80=9D). Specifically on commit 2b5c5f03c2f0a84f84a5517e2e6f5fa9f276ffa5: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix build -e '((@ (guix packages) package-replacement) (@= @ (gnu packages commencement) glibc-final))' --no-grafts /gnu/store/m07pz38dvizwx2bl9aik6wcbbqbhz6j6-glibc-2.26.105-g0890d5379c-debug /gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c /gnu/store/m8m005z2jbvqrj3s5b052camzk2qxpz5-glibc-2.26.105-g0890d5379c-stat= ic $ ./pre-inst-env guix build -e '((@ (guix packages) package-replacement) (@= @ (gnu packages commencement) glibc-final))'=20 /gnu/store/bdgcd723b8l1h8cg8wx4vjfypip29dsn-glibc-2.26.105-g0890d5379c-debug /gnu/store/l2wzs674z5ac5ccrvp73gdqw02mmzr22-glibc-2.26.105-g0890d5379c /gnu/store/2rp8zmymxi32wrw4s44f2dc67ci9kxgs-glibc-2.26.105-g0890d5379c-stat= ic $ grep -r 4sqai /gnu/store/l2wzs674z5ac5ccrvp73gdqw02mmzr22-glibc-2.26.105-= g0890d5379c Duuma dosiero /gnu/store/l2wzs674z5ac5ccrvp73gdqw02mmzr22-glibc-2.26.105-g0= 890d5379c/sbin/sln kongruas Duuma dosiero /gnu/store/l2wzs674z5ac5ccrvp73gdqw02mmzr22-glibc-2.26.105-g0= 890d5379c/sbin/nscd kongruas --8<---------------cut here---------------end--------------->8--- If we look with hexl-mode, we see that libc-2.26.so contains some of these too: --8<---------------cut here---------------start------------->8--- 000236a0: b92f 676e 752f 7374 6f48 8d50 01c6 003a ./gnu/stoH.P...: 000236b0: 4889 4801 48b8 7265 2f34 7371 6169 31f6 H.H.H.re/4sqai1. 000236c0: 4889 4208 48b8 6237 6332 6466 6a76 31ff H.B.H.b7c2dfjv1. 000236d0: 4889 4210 48b8 3632 6976 7267 3962 c642 H.B.H.62ivrg9b.B 000236e0: 5000 4889 4218 48b8 3877 6137 6268 3232 P.H.B.H.8wa7bh22 000236f0: 4889 4220 48b8 366c 612d 676c 6962 4889 H.B H.6la-glibH. 00023700: 4228 48b8 632d 322e 3236 2e31 4889 4230 B(H.c-2.26.1H.B0 00023710: 48b8 3035 2d67 3038 3930 4889 4238 48b8 H.05-g0890H.B8H. 00023720: 6435 3337 3963 2f6c 4889 4240 48b8 6962 d5379c/lH.B@H.ib 00023730: 2f67 636f 6e76 4889 4248 e881 f70b 0048 /gconvH.BH.....H --8<---------------cut here---------------end--------------->8--- That in turns means that gconv-modules won=E2=80=99t be found, and that gui= le with crash during startup with =E2=80=9CUncaught exception=E2=80=9D (becaus= e early on it fails to convert file names to UTF-8 through iconv). To be continued=E2=80=A6 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 14 12:31:24 2018 Received: (at control) by debbugs.gnu.org; 14 Mar 2018 16:31:24 +0000 Received: from localhost ([127.0.0.1]:33512 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ew9JD-0001z9-TW for submit@debbugs.gnu.org; Wed, 14 Mar 2018 12:31:24 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:45842) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ew9JC-0001yw-Eu for control@debbugs.gnu.org; Wed, 14 Mar 2018 12:31:22 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 3150D12A50 for ; Wed, 14 Mar 2018 17:31:21 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yAwVn0lYU7wP for ; Wed, 14 Mar 2018 17:31:20 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 64D1C12A4C for ; Wed, 14 Mar 2018 17:31:20 +0100 (CET) Date: Wed, 14 Mar 2018 17:31:19 +0100 Message-Id: <87muza7h60.fsf@gnu.org> To: control@debbugs.gnu.org From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: control message for bug #30820 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) severity 30820 serious From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 14 13:24:57 2018 Received: (at 30820) by debbugs.gnu.org; 14 Mar 2018 17:24:57 +0000 Received: from localhost ([127.0.0.1]:33536 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ewA92-0005Lz-Ox for submit@debbugs.gnu.org; Wed, 14 Mar 2018 13:24:57 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:46332) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ewA91-0005Lp-9R for 30820@debbugs.gnu.org; Wed, 14 Mar 2018 13:24:55 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 0DD8112A68; Wed, 14 Mar 2018 18:24:54 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bSqmYiGq6JsF; Wed, 14 Mar 2018 18:24:49 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 1E20312A3D; Wed, 14 Mar 2018 18:24:49 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: 30820@debbugs.gnu.org Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> Date: Wed, 14 Mar 2018 18:24:48 +0100 In-Reply-To: <87o9jq7j7r.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Wed, 14 Mar 2018 16:47:04 +0100") Message-ID: <87efkm7eov.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820 Cc: Ricardo Wurmus , Mark H Weaver X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) There are two issues. First the gcc-strmov-store-file-names.patch stopped working. When we introduced it, we were at 5.3.0 and 6.2.0 (see ). Today, with 5.5.0 and 6.3.0 it seems to have no effect (I use =E2=80=98ltra= ce=E2=80=99 here, which shows the getenv("NIX_STORE") call, which confirms that the =E2=80=98store_reference_p=E2=80=99 function in that patch gets called): --8<---------------cut here---------------start------------->8--- $ cat strmov.c #define _GNU_SOURCE #include static const char str[] =3D "MEMpCPY /gnu/store/THIS IS A LONG STRING, A VE= RY, VERY, VERY LOOOOONG STRING"; extern char *p, *q; #ifndef MEMCPY # define MEMCPY memcpy #endif void foo (char *x, char *y) { /* MEMCPY (x, str, sizeof str); */ MEMCPY (y, "this is a literal /gnu/store string", 35); } $ guix environment --no-grafts --pure --ad-hoc -C ltrace gcc-toolchain@5 --= ltrace -f -e getenv gcc -O2 -c strmov.c=20 [pid 2] gcc->getenv("COLUMNS") = =3D nil [pid 2] gcc->getenv("TERM") = =3D nil [pid 2] gcc->getenv("GCC_EXEC_PREFIX") = =3D nil [pid 2] gcc->getenv("PATH") = =3D "/gnu/store/c312mxd0ykdh3zi3zj13m".= .. [pid 2] gcc->getenv("PATH") = =3D "/gnu/store/c312mxd0ykdh3zi3zj13m".= .. [pid 2] gcc->getenv("COMPILER_PATH") = =3D nil [pid 2] gcc->getenv("LIBRARY_PATH") = =3D "/gnu/store/c312mxd0ykdh3zi3zj13m".= .. [pid 2] gcc->getenv("LPATH") = =3D nil [pid 2] gcc->getenv("GCC_COMPARE_DEBUG") = =3D nil [pid 2] gcc->getenv("GCC_ROOT") = =3D nil [pid 2] gcc->getenv("BINUTILS_ROOT") = =3D nil [pid 2] gcc->getenv("TMPDIR") = =3D "/tmp" [pid 2] gcc->getenv("TMP") = =3D "/tmp" [pid 2] gcc->getenv("TEMP") = =3D "/tmp" [pid 3] --- Called exec() --- [pid 3] gcc->getenv("COLUMNS") = =3D nil [pid 3] gcc->getenv("TERM") = =3D nil [pid 3] gcc->getenv("DEPENDENCIES_OUTPUT") = =3D nil [pid 3] gcc->getenv("SUNPRO_DEPENDENCIES") = =3D nil [pid 3] gcc->getenv("CPATH") = =3D nil [pid 3] gcc->getenv("C_INCLUDE_PATH") = =3D "/gnu/store/c312mxd0ykdh3zi3zj13m".= .. [pid 3] gcc->getenv("GCC_EXEC_PREFIX") = =3D nil [pid 3] gcc->getenv("PWD") = =3D nil [pid 3] gcc->getenv("NIX_STORE") = =3D nil [pid 3] +++ exited (status 0) +++ [pid 2] --- SIGCHLD (Child exited) --- [pid 4] --- Called exec() --- [pid 4] +++ exited (status 0) +++ [pid 2] --- SIGCHLD (Child exited) --- [pid 2] +++ exited (status 0) +++ $ objdump -S strmov.o | grep movabs 0: 48 b8 74 68 69 73 20 movabs $0x2073692073696874,%rax 11: 48 b8 61 20 6c 69 74 movabs $0x61726574696c2061,%rax 1f: 48 b8 6c 20 2f 67 6e movabs $0x732f756e672f206c,%rax 2d: 48 b8 74 6f 72 65 20 movabs $0x7274732065726f74,%rax $ guix environment --no-grafts --pure --ad-hoc -C ltrace gcc-toolchain@6 --= ltrace -f -e getenv gcc -O2 -c strmov.c=20 [pid 2] gcc->getenv("COLUMNS") = =3D nil [pid 2] gcc->getenv("TERM") = =3D nil [pid 2] gcc->getenv("GCC_EXEC_PREFIX") = =3D nil [pid 2] gcc->getenv("PATH") = =3D "/gnu/store/5141337yvhhjj5fhdx660".= .. [pid 2] gcc->getenv("PATH") = =3D "/gnu/store/5141337yvhhjj5fhdx660".= .. [pid 2] gcc->getenv("COMPILER_PATH") = =3D nil [pid 2] gcc->getenv("LIBRARY_PATH") = =3D "/gnu/store/5141337yvhhjj5fhdx660".= .. [pid 2] gcc->getenv("LPATH") = =3D nil [pid 2] gcc->getenv("GCC_COMPARE_DEBUG") = =3D nil [pid 2] gcc->getenv("GCC_ROOT") = =3D nil [pid 2] gcc->getenv("BINUTILS_ROOT") = =3D nil [pid 2] gcc->getenv("TMPDIR") = =3D "/tmp" [pid 2] gcc->getenv("TMP") = =3D "/tmp" [pid 2] gcc->getenv("TEMP") = =3D "/tmp" [pid 3] --- Called exec() --- [pid 3] gcc->getenv("COLUMNS") = =3D nil [pid 3] gcc->getenv("TERM") = =3D nil [pid 3] gcc->getenv("DEPENDENCIES_OUTPUT") = =3D nil [pid 3] gcc->getenv("SUNPRO_DEPENDENCIES") = =3D nil [pid 3] gcc->getenv("CPATH") = =3D nil [pid 3] gcc->getenv("C_INCLUDE_PATH") = =3D "/gnu/store/5141337yvhhjj5fhdx660".= .. [pid 3] gcc->getenv("GCC_EXEC_PREFIX") = =3D nil [pid 3] gcc->getenv("PWD") = =3D nil [pid 3] gcc->getenv("NIX_STORE") = =3D nil [pid 3] +++ exited (status 0) +++ [pid 2] --- SIGCHLD (Child exited) --- [pid 4] --- Called exec() --- [pid 4] +++ exited (status 0) +++ [pid 2] --- SIGCHLD (Child exited) --- [pid 2] +++ exited (status 0) +++ $ objdump -S strmov.o | grep movabs 0: 48 b8 74 68 69 73 20 movabs $0x2073692073696874,%rax 11: 48 b8 61 20 6c 69 74 movabs $0x61726574696c2061,%rax 1f: 48 b8 6c 20 2f 67 6e movabs $0x732f756e672f206c,%rax 2d: 48 b8 74 6f 72 65 20 movabs $0x7274732065726f74,%rax --8<---------------cut here---------------end--------------->8--- On GCC 7.3.0, it works as intended: --8<---------------cut here---------------start------------->8--- $ guix environment --no-grafts --pure --ad-hoc -C ltrace gcc-toolchain@7 --= ltrace -f -e getenv gcc -O2 -c strmov.c=20 [pid 2] gcc->getenv("COLUMNS") = =3D nil [pid 2] gcc->getenv("TERM") = =3D nil [pid 2] gcc->getenv("GCC_EXEC_PREFIX") = =3D nil [pid 2] gcc->getenv("PATH") = =3D "/gnu/store/zahi3qjfnfq5z0bsxkggq".= .. [pid 2] gcc->getenv("PATH") = =3D "/gnu/store/zahi3qjfnfq5z0bsxkggq".= .. [pid 2] gcc->getenv("COMPILER_PATH") = =3D nil [pid 2] gcc->getenv("LIBRARY_PATH") = =3D "/gnu/store/zahi3qjfnfq5z0bsxkggq".= .. [pid 2] gcc->getenv("LPATH") = =3D nil [pid 2] gcc->getenv("GCC_COMPARE_DEBUG") = =3D nil [pid 2] gcc->getenv("GCC_ROOT") = =3D nil [pid 2] gcc->getenv("BINUTILS_ROOT") = =3D nil [pid 2] gcc->getenv("TMPDIR") = =3D "/tmp" [pid 2] gcc->getenv("TMP") = =3D "/tmp" [pid 2] gcc->getenv("TEMP") = =3D "/tmp" [pid 3] --- Called exec() --- [pid 3] gcc->getenv("COLUMNS") = =3D nil [pid 3] gcc->getenv("TERM") = =3D nil [pid 3] gcc->getenv("DEPENDENCIES_OUTPUT") = =3D nil [pid 3] gcc->getenv("SUNPRO_DEPENDENCIES") = =3D nil [pid 3] gcc->getenv("CPATH") = =3D nil [pid 3] gcc->getenv("C_INCLUDE_PATH") = =3D "/gnu/store/zahi3qjfnfq5z0bsxkggq".= .. [pid 3] gcc->getenv("GCC_EXEC_PREFIX") = =3D nil [pid 3] gcc->getenv("PWD") = =3D nil [pid 3] gcc->getenv("NIX_STORE") = =3D nil [pid 3] +++ exited (status 0) +++ [pid 2] --- SIGCHLD (Child exited) --- [pid 4] --- Called exec() --- [pid 4] +++ exited (status 0) +++ [pid 2] --- SIGCHLD (Child exited) --- [pid 2] +++ exited (status 0) +++ $ objdump -S strmov.o | grep movabs $ guix environment --no-grafts --pure --ad-hoc -C ltrace gcc-toolchain@6 --= /bin/sh -c 'export NIX_STORE=3D/foo ; ltrace -f -e getenv gcc -O2 -c str= mov.c ' [pid 3] gcc->getenv("COLUMNS") = =3D nil [pid 3] gcc->getenv("TERM") = =3D nil [pid 3] gcc->getenv("GCC_EXEC_PREFIX") = =3D nil [pid 3] gcc->getenv("PATH") = =3D "/gnu/store/5141337yvhhjj5fhdx660".= .. [pid 3] gcc->getenv("PATH") = =3D "/gnu/store/5141337yvhhjj5fhdx660".= .. [pid 3] gcc->getenv("COMPILER_PATH") = =3D nil [pid 3] gcc->getenv("LIBRARY_PATH") = =3D "/gnu/store/5141337yvhhjj5fhdx660".= .. [pid 3] gcc->getenv("LPATH") = =3D nil [pid 3] gcc->getenv("GCC_COMPARE_DEBUG") = =3D nil [pid 3] gcc->getenv("GCC_ROOT") = =3D nil [pid 3] gcc->getenv("BINUTILS_ROOT") = =3D nil [pid 3] gcc->getenv("TMPDIR") = =3D "/tmp" [pid 3] gcc->getenv("TMP") = =3D "/tmp" [pid 3] gcc->getenv("TEMP") = =3D "/tmp" [pid 4] --- Called exec() --- [pid 4] gcc->getenv("COLUMNS") = =3D nil [pid 4] gcc->getenv("TERM") = =3D nil [pid 4] gcc->getenv("DEPENDENCIES_OUTPUT") = =3D nil [pid 4] gcc->getenv("SUNPRO_DEPENDENCIES") = =3D nil [pid 4] gcc->getenv("CPATH") = =3D nil [pid 4] gcc->getenv("C_INCLUDE_PATH") = =3D "/gnu/store/5141337yvhhjj5fhdx660".= .. [pid 4] gcc->getenv("GCC_EXEC_PREFIX") = =3D nil [pid 4] gcc->getenv("PWD") = =3D "/home/ludo/src/guix" [pid 4] gcc->getenv("NIX_STORE") = =3D "/foo" [pid 4] gcc->getenv("NIX_STORE") = =3D "/foo" [pid 4] +++ exited (status 0) +++ [pid 3] --- SIGCHLD (Child exited) --- [pid 5] --- Called exec() --- [pid 5] +++ exited (status 0) +++ [pid 3] --- SIGCHLD (Child exited) --- [pid 3] +++ exited (status 0) +++ $ objdump -S strmov.o | grep movabs 0: 48 b8 74 68 69 73 20 movabs $0x2073692073696874,%rax 11: 48 b8 61 20 6c 69 74 movabs $0x61726574696c2061,%rax 1f: 48 b8 6c 20 2f 67 6e movabs $0x732f756e672f206c,%rax 2d: 48 b8 74 6f 72 65 20 movabs $0x7274732065726f74,%rax --8<---------------cut here---------------end--------------->8--- The second issue is that the patch only ever worked with literal strings. It does not =E2=80=9Csee=E2=80=9D strings in constant arrays like= the =E2=80=98str=E2=80=99 array in the example above. The gconv-module file name mentioned in the first message in this bug report is an example of a string assigned to a static array, in iconv/gconv_conf.c: /* This is the default path where we look for module lists. */ static const char default_gconv_path[] =3D GCONV_PATH; Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 14 13:37:23 2018 Received: (at control) by debbugs.gnu.org; 14 Mar 2018 17:37:23 +0000 Received: from localhost ([127.0.0.1]:33557 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ewAL4-0005iG-S5 for submit@debbugs.gnu.org; Wed, 14 Mar 2018 13:37:22 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:46424) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ewAL3-0005i7-80 for control@debbugs.gnu.org; Wed, 14 Mar 2018 13:37:21 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id B7AF512A68 for ; Wed, 14 Mar 2018 18:37:20 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jVSVrzG-8lek for ; Wed, 14 Mar 2018 18:37:20 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 1E1C312A33 for ; Wed, 14 Mar 2018 18:37:20 +0100 (CET) Date: Wed, 14 Mar 2018 18:37:19 +0100 Message-Id: <87a7va7e40.fsf@gnu.org> To: control@debbugs.gnu.org From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: control message for bug #30395 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) merge 30395 30820 From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 16 04:54:37 2018 Received: (at 30820) by debbugs.gnu.org; 16 Mar 2018 08:54:37 +0000 Received: from localhost ([127.0.0.1]:35713 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ewl8H-0007vl-2m for submit@debbugs.gnu.org; Fri, 16 Mar 2018 04:54:37 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:35486) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ewl8E-0007vX-VD; Fri, 16 Mar 2018 04:54:35 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 990DF12E55; Fri, 16 Mar 2018 09:54:33 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TI8QioaJ-joE; Fri, 16 Mar 2018 09:54:32 +0100 (CET) Received: from ribbon (vpn-0-27.aquilenet.fr [IPv6:2a0c:e300:4:27::]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 573B212E37; Fri, 16 Mar 2018 09:54:32 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: 30820@debbugs.gnu.org Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87efkm7eov.fsf@gnu.org> Date: Fri, 16 Mar 2018 09:54:31 +0100 In-Reply-To: <87efkm7eov.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Wed, 14 Mar 2018 18:24:48 +0100") Message-ID: <87in9wxuwo.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820 Cc: Ricardo Wurmus , 30395@debbugs.gnu.org, Mark H Weaver X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > void foo (char *x, char *y) > { > /* MEMCPY (x, str, sizeof str); */ > MEMCPY (y, "this is a literal /gnu/store string", 35); > } This was not a correct example because =E2=80=9C/gnu/store=E2=80=9D must be= followed by at least 34 chars for the patch to work. And indeed, it does work in this case: --8<---------------cut here---------------start------------->8--- $ cat strmov.c #define _GNU_SOURCE #include static const char str[] =3D "MEMpCPY /gnu/store/THIS IS A LONG STRING, A VE= RY, VERY, VERY LOOOOONG STRING"; extern char *p, *q; #ifndef MEMCPY # define MEMCPY memcpy #endif void foo (char *x, char *y) { /* MEMCPY (x, str, sizeof str); */ MEMCPY (y, "this is a literal /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee= ee string", 35); } $ guix environment --ad-hoc gcc-toolchain@5 -C -- gcc -O2 -c strmov.c $ objdump -S strmov.o |grep movabs --8<---------------cut here---------------end--------------->8--- So the real issue is this: > The second issue is that the patch only ever worked with literal > strings. It does not =E2=80=9Csee=E2=80=9D strings in constant arrays li= ke the =E2=80=98str=E2=80=99 > array in the example above. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 15:06:22 2018 Received: (at 30820) by debbugs.gnu.org; 19 Mar 2018 19:06:22 +0000 Received: from localhost ([127.0.0.1]:42149 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey06w-0000ap-7E for submit@debbugs.gnu.org; Mon, 19 Mar 2018 15:06:22 -0400 Received: from world.peace.net ([50.252.239.5]:60682) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey06v-0000ad-48 for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 15:06:21 -0400 Received: from pool-72-93-34-38.bstnma.east.verizon.net ([72.93.34.38] helo=jojen) by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1ey06o-0001Wo-O9; Mon, 19 Mar 2018 15:06:14 -0400 From: Mark H Weaver To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> Date: Mon, 19 Mar 2018 15:05:26 -0400 In-Reply-To: <87o9jq7j7r.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Wed, 14 Mar 2018 16:47:04 +0100") Message-ID: <87muz3dgy1.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) ludo@gnu.org (Ludovic Court=C3=A8s) writes: > The recently added glibc grafts triggered issues that, in the end, show > the return of (=E2=80=9CStore references in 8= -byte > chunks in compiled code=E2=80=9D). I think that we should generalize our reference scanning and grafting code to support store references broken into pieces, as long as each piece containing part of the hash is at least 8 bytes long. Here's my preliminary proposal: (1) The reference scanner should recognize any 8-byte substring of a hash as a valid reference to that hash. (2) To enable reliable grafting of chunked references, we should impose the following new restrictions: (a) the store prefix must be at least 6 bytes, (b) grafting can change only the hash, not the readable part of the store name, and (c) the readable part of the store name must be at least 6 bytes. (3) The grafter should recognize and replace any 8-byte subsequence of the absolute store file name. The rationale for the restrictions is to ensure that any byte that needs to be modified by the grafter should be part of an 8-byte substring of the absolute store file name. This requires that there be at least 7 bytes of known text before the first changed byte and after the last changed byte. This is needed to provide a reasonable upper bound on the probability of grafting a matching sequence of bytes that is not a store reference. I'd be willing to work on implementing this soon. What do you think? Mark From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 15:17:28 2018 Received: (at 30820) by debbugs.gnu.org; 19 Mar 2018 19:17:28 +0000 Received: from localhost ([127.0.0.1]:42159 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey0Hg-0000qz-I1 for submit@debbugs.gnu.org; Mon, 19 Mar 2018 15:17:28 -0400 Received: from world.peace.net ([50.252.239.5]:60702) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey0Hf-0000qn-1x for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 15:17:27 -0400 Received: from pool-72-93-34-38.bstnma.east.verizon.net ([72.93.34.38] helo=jojen) by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1ey0HW-0001dn-47; Mon, 19 Mar 2018 15:17:18 -0400 From: Mark H Weaver To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> Date: Mon, 19 Mar 2018 15:16:30 -0400 In-Reply-To: <87muz3dgy1.fsf@netris.org> (Mark H. Weaver's message of "Mon, 19 Mar 2018 15:05:26 -0400") Message-ID: <87in9rdgfl.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Mark H Weaver writes: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> The recently added glibc grafts triggered issues that, in the end, show >> the return of (=E2=80=9CStore references in = 8-byte >> chunks in compiled code=E2=80=9D). > > I think that we should generalize our reference scanning and grafting > code to support store references broken into pieces, as long as each > piece containing part of the hash is at least 8 bytes long. > > Here's my preliminary proposal: > > (1) The reference scanner should recognize any 8-byte substring of a > hash as a valid reference to that hash. To facilitate the transition: to support older versions of the Guix daemon (or Nix daemon), we could add a final phase to gnu-build-system which would "unhide" these references as follows: It would scan each output directory for 8-byte substrings of hashes. If it finds any references that the old scanner is unable to see, it would install a file to the output directory containing the full store names of the hidden references. This only works for output directories, though. I don't know what to do about output files containing hidden references. I guess the final phase should raise an error in this case, and hopefully it rarely happens. Mark From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 17:22:19 2018 Received: (at 30820) by debbugs.gnu.org; 19 Mar 2018 21:22:19 +0000 Received: from localhost ([127.0.0.1]:42278 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey2EV-000414-3i for submit@debbugs.gnu.org; Mon, 19 Mar 2018 17:22:19 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:55180) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey2EQ-00040q-Ku for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 17:22:17 -0400 Received: from localhost (77.118.190.249.wireless.dyn.drei.com [77.118.190.249]) by dd26836.kasserver.com (Postfix) with ESMTPSA id BF00933613DE; Mon, 19 Mar 2018 22:22:12 +0100 (CET) Date: Mon, 19 Mar 2018 22:22:05 +0100 From: Danny Milosavljevic To: ludo@gnu.org (Ludovic =?ISO-8859-1?Q?Court=E8s?=) Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) Message-ID: <20180319222205.2e35f26f@scratchpost.org> In-Reply-To: <87efkm7eov.fsf@gnu.org> References: <87o9jq7j7r.fsf@gnu.org> <87efkm7eov.fsf@gnu.org> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.31; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/3pd6DzVOf2uaP+Xy_IAApRS"; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --Sig_/3pd6DzVOf2uaP+Xy_IAApRS Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, > The second issue is that the patch only ever worked with literal > strings. It does not =E2=80=9Csee=E2=80=9D strings in constant arrays li= ke the =E2=80=98str=E2=80=99 > array in the example above. >=20 > The gconv-module file name mentioned in the first message in this bug > report is an example of a string assigned to a static array, in > iconv/gconv_conf.c: >=20 > /* This is the default path where we look for module lists. */ > static const char default_gconv_path[] =3D GCONV_PATH; I don't understand why this is a problem. Grafting would just mutate default_gconv_path, right? Who cares how the runtime memcpy works (if there's no literal as source)? Or do you mean that it memcpys at compile time? --Sig_/3pd6DzVOf2uaP+Xy_IAApRS Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlqwKf0ACgkQ5xo1VCww uqUYdwgAleW7oQUOv9JYQlQUy9qQtR7eJNFwmndTLjqcOlaJ2PE9Yb0jg7X1ZTGm A29BKyL14YhV+2GmUeirPZfk4XJlUXMqYiOtqlYIa23h5uYfAtnYjl0Jfln9PJG0 vYewh1Wr6sAHcUghjmy09Vg0Fy1cEj/ydA2WzzeKHJ+qftYWOxctO75cuDsRxdCj adF3zNMKi25Ro2ovBwf1wYO3vF/UIcwvNCnSQO+rEZlWYcjawCV9WCz8TlRRgfEK 7ZqCMu45So1HDjSAm48pqD195IMNnA142MrcHJPvfc7VU3+fQvQphh4+g5aXqZHX v8uodHrRZxIefQFE2EFTHz87+beRxw== =2Gaa -----END PGP SIGNATURE----- --Sig_/3pd6DzVOf2uaP+Xy_IAApRS-- From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 17:34:10 2018 Received: (at 30820) by debbugs.gnu.org; 19 Mar 2018 21:34:10 +0000 Received: from localhost ([127.0.0.1]:42282 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey2Py-0004J7-99 for submit@debbugs.gnu.org; Mon, 19 Mar 2018 17:34:10 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:55990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey2Px-0004Iz-0V for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 17:34:09 -0400 Received: from localhost (77.118.190.249.wireless.dyn.drei.com [77.118.190.249]) by dd26836.kasserver.com (Postfix) with ESMTPSA id A2A053360CA2; Mon, 19 Mar 2018 22:34:07 +0100 (CET) Date: Mon, 19 Mar 2018 22:34:02 +0100 From: Danny Milosavljevic To: Mark H Weaver Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) Message-ID: <20180319223402.1ba0a369@scratchpost.org> In-Reply-To: <87muz3dgy1.fsf@netris.org> References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.31; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/nDHACSvVxixn.UDyI5uLVE+"; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30820 Cc: Ludovic =?ISO-8859-1?Q?Court=E8s?= , 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --Sig_/nDHACSvVxixn.UDyI5uLVE+ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi Mark, On Mon, 19 Mar 2018 15:05:26 -0400 Mark H Weaver wrote: > I think that we should generalize our reference scanning and grafting > code to support store references broken into pieces, as long as each > piece containing part of the hash is at least 8 bytes long. I don't think it's possible to get that to work reliably over all possible optimizations. I mean why 8 Byte? That's probably because of the 8 Byte registers on x86_64. What would happen on ARM? (32 bit registers)? And on MIPS (immediates only smaller than 32 bits)? What if gcc finds some repetition in the hash and decides not to load the register again the second time? I think the best way - since we have control over the entire toolchain anyw= ay - is to make gcc not do the data inlining in the first place. As far as I understand the gcc patches work after all. Ludo just made a mistake testing it. Although when we do this patching of gcc we should add a test to gcc that makes sure that the data inlining is indeed not there anymore for store paths. > Here's my preliminary proposal: >=20 > (1) The reference scanner should recognize any 8-byte substring of a > hash as a valid reference to that hash. >=20 > (2) To enable reliable grafting of chunked references, we should impose > the following new restrictions: (a) the store prefix must be at > least 6 bytes, (b) grafting can change only the hash, not the > readable part of the store name, and (c) the readable part of the > store name must be at least 6 bytes. >=20 > (3) The grafter should recognize and replace any 8-byte subsequence of > the absolute store file name. >=20 > The rationale for the restrictions is to ensure that any byte that needs > to be modified by the grafter should be part of an 8-byte substring of > the absolute store file name. This requires that there be at least 7 > bytes of known text before the first changed byte and after the last > changed byte. This is needed to provide a reasonable upper bound on the > probability of grafting a matching sequence of bytes that is not a store > reference. I think that is a good way to get the risk down - but it will not actually fix the problem for good. Also, complexity like the above makes the reference scanner more brittle and it's easier for bugs to hide in it (and it's slower). I think the gcc patch is actually a better way. --Sig_/nDHACSvVxixn.UDyI5uLVE+ Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlqwLMoACgkQ5xo1VCww uqXPpQgAlFrKx8F34L34+KkDcib8705k0QtyrrsWlzQn7DrgqwfIZl/SIiciMOj6 0qPVCBF6+qxdd6SXL8/CbCHn/7J3wHY9QDlTIHpIbVB79IvWQ0BK8cEj1VbfK/l+ 2Z6OXT+huec0Ej6HO3NPfI6sYjcEFz0VsQyAcsddb+CgWTdaeYpsaPgg3NIyuP2B uMbvu6DF3kKZU3o50hKNoV8rl54LohvaTE6agp+ahWdjq8uKsWbygXe4QJbRNc74 OWtrujIicfRshG9LUulnmOaMAUixTLSLx0Rr6GpxQdkcEhEytG6Qp3DyPa/6lAUy LLhNAhTvp2S8c/WPVYVyJ0dxbed+Eg== =IOCW -----END PGP SIGNATURE----- --Sig_/nDHACSvVxixn.UDyI5uLVE+-- From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 18:27:44 2018 Received: (at 30820) by debbugs.gnu.org; 19 Mar 2018 22:27:44 +0000 Received: from localhost ([127.0.0.1]:42315 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey3Fo-0005a7-6I for submit@debbugs.gnu.org; Mon, 19 Mar 2018 18:27:44 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:37708) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey3Fm-0005Zz-9o for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 18:27:42 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 2C6CD12478; Mon, 19 Mar 2018 23:27:41 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l2ltlBkChu1J; Mon, 19 Mar 2018 23:27:40 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id DFE6112474; Mon, 19 Mar 2018 23:27:39 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Danny Milosavljevic Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> <20180319223402.1ba0a369@scratchpost.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 29 =?utf-8?Q?Vent=C3=B4se?= an 226 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 19 Mar 2018 23:27:39 +0100 In-Reply-To: <20180319223402.1ba0a369@scratchpost.org> (Danny Milosavljevic's message of "Mon, 19 Mar 2018 22:34:02 +0100") Message-ID: <87h8pbg0pw.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820 Cc: Mark H Weaver , 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Danny Milosavljevic skribis: > As far as I understand the gcc patches work after all. Ludo just made > a mistake testing it. They work except when the string is not directly a literal, as in: static const char foo[] =3D "/gnu/store/=E2=80=A6"; =E2=80=A6 strcpy (x, foo); I think that=E2=80=99s workable though. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 18:29:13 2018 Received: (at 30820) by debbugs.gnu.org; 19 Mar 2018 22:29:13 +0000 Received: from localhost ([127.0.0.1]:42319 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey3HF-0005cb-Gd for submit@debbugs.gnu.org; Mon, 19 Mar 2018 18:29:13 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:37726) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey3HD-0005cT-Ld for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 18:29:11 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 2D31412478; Mon, 19 Mar 2018 23:29:11 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id woBpSkPVlJvu; Mon, 19 Mar 2018 23:29:10 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 65AC912474; Mon, 19 Mar 2018 23:29:10 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Danny Milosavljevic Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87efkm7eov.fsf@gnu.org> <20180319222205.2e35f26f@scratchpost.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 29 =?utf-8?Q?Vent=C3=B4se?= an 226 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 19 Mar 2018 23:29:10 +0100 In-Reply-To: <20180319222205.2e35f26f@scratchpost.org> (Danny Milosavljevic's message of "Mon, 19 Mar 2018 22:22:05 +0100") Message-ID: <87d0zzg0nd.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Heya, Danny Milosavljevic skribis: >> The second issue is that the patch only ever worked with literal >> strings. It does not =E2=80=9Csee=E2=80=9D strings in constant arrays l= ike the =E2=80=98str=E2=80=99 >> array in the example above. >>=20 >> The gconv-module file name mentioned in the first message in this bug >> report is an example of a string assigned to a static array, in >> iconv/gconv_conf.c: >>=20 >> /* This is the default path where we look for module lists. */ >> static const char default_gconv_path[] =3D GCONV_PATH; > > I don't understand why this is a problem. Grafting would just > mutate default_gconv_path, right? Who cares how the runtime memcpy > works (if there's no literal as source)? At compile-time, GCC finds out that =E2=80=98default_gconv_path=E2=80=99 is= used only in one place, in an strcpy call. Thus, it chooses to use the movabs optimization, and as a consequence, to split =E2=80=98default_gconv_path=E2= =80=99 in 8-byte chunks. It can do so because it=E2=80=99s =E2=80=98static=E2=80=99. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 18:34:43 2018 Received: (at 30820) by debbugs.gnu.org; 19 Mar 2018 22:34:43 +0000 Received: from localhost ([127.0.0.1]:42327 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey3MZ-0005mS-Fq for submit@debbugs.gnu.org; Mon, 19 Mar 2018 18:34:43 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:37768) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey3MX-0005mL-PE for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 18:34:42 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 2EA3012478; Mon, 19 Mar 2018 23:34:41 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vtksfxkI31PP; Mon, 19 Mar 2018 23:34:39 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id D9A7F12474; Mon, 19 Mar 2018 23:34:38 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Mark H Weaver Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 29 =?utf-8?Q?Vent=C3=B4se?= an 226 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 19 Mar 2018 23:34:38 +0100 In-Reply-To: <87muz3dgy1.fsf@netris.org> (Mark H. Weaver's message of "Mon, 19 Mar 2018 15:05:26 -0400") Message-ID: <87370vg0e9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Hi Mark, Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> The recently added glibc grafts triggered issues that, in the end, show >> the return of (=E2=80=9CStore references in = 8-byte >> chunks in compiled code=E2=80=9D). > > I think that we should generalize our reference scanning and grafting > code to support store references broken into pieces, as long as each > piece containing part of the hash is at least 8 bytes long. > > Here's my preliminary proposal: > > (1) The reference scanner should recognize any 8-byte substring of a > hash as a valid reference to that hash. > > (2) To enable reliable grafting of chunked references, we should impose > the following new restrictions: (a) the store prefix must be at > least 6 bytes, (b) grafting can change only the hash, not the > readable part of the store name, and (c) the readable part of the > store name must be at least 6 bytes. > > (3) The grafter should recognize and replace any 8-byte subsequence of > the absolute store file name. I=E2=80=99m quite reluctant because it would add complexity, it will probab= ly slow things down, and yet it may not handle all the cases, as Danny suggests. Mind you, the GCC patches are not perfect either, but they=E2=80=99re relat= ively easy to deal with (well, so far at least). In theory we would need similar patches for LLVM and maybe a couple other native compilers, though, which is obviously a downside, although we haven=E2=80=99t had any problems so far. WDYT? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 20:53:33 2018 Received: (at 30820) by debbugs.gnu.org; 20 Mar 2018 00:53:33 +0000 Received: from localhost ([127.0.0.1]:42408 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey5Wu-0004ZJ-VJ for submit@debbugs.gnu.org; Mon, 19 Mar 2018 20:53:33 -0400 Received: from world.peace.net ([50.252.239.5]:33826) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey5Wt-0004Z8-Js for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 20:53:31 -0400 Received: from pool-72-93-34-155.bstnma.east.verizon.net ([72.93.34.155] helo=jojen) by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1ey5Wn-0003p9-M8; Mon, 19 Mar 2018 20:53:25 -0400 From: Mark H Weaver To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> <87370vg0e9.fsf@gnu.org> Date: Mon, 19 Mar 2018 20:52:37 -0400 In-Reply-To: <87370vg0e9.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 19 Mar 2018 23:34:38 +0100") Message-ID: <87in9rbmay.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Mark H Weaver skribis: > >> ludo@gnu.org (Ludovic Court=C3=A8s) writes: >> >>> The recently added glibc grafts triggered issues that, in the end, show >>> the return of (=E2=80=9CStore references in= 8-byte >>> chunks in compiled code=E2=80=9D). >> >> I think that we should generalize our reference scanning and grafting >> code to support store references broken into pieces, as long as each >> piece containing part of the hash is at least 8 bytes long. >> >> Here's my preliminary proposal: >> >> (1) The reference scanner should recognize any 8-byte substring of a >> hash as a valid reference to that hash. >> >> (2) To enable reliable grafting of chunked references, we should impose >> the following new restrictions: (a) the store prefix must be at >> least 6 bytes, (b) grafting can change only the hash, not the >> readable part of the store name, and (c) the readable part of the >> store name must be at least 6 bytes. >> >> (3) The grafter should recognize and replace any 8-byte subsequence of >> the absolute store file name. > > I=E2=80=99m quite reluctant because it would add complexity, it will prob= ably > slow things down, and yet it may not handle all the cases, as Danny > suggests. > > Mind you, the GCC patches are not perfect either, but they=E2=80=99re rel= atively > easy to deal with (well, so far at least). In theory we would need > similar patches for LLVM and maybe a couple other native compilers, > though, which is obviously a downside, although we haven=E2=80=99t had any > problems so far. We would also need to find a solution to the problem described in the thread "broken references in jar manifests" on guix-devel started by Ricardo, which still has not found a satifactory solution. https://lists.gnu.org/archive/html/guix-devel/2018-03/msg00006.html My opinion is that I consider Guix's current expectations for how software must store its data on disk to be far too onerous, in cases where that data might include a store reference. I don't see sufficient justification for imposing such an onerous requirement on the software in Guix. Ultimately, I would prefer to see the scanning and grafting operations completely generalized, so that in general each package can specify how to scan and graft that particular package, making use of libraries in (guix build ...) to cover the usual cases. In most cases, that code would be within build-systems. Mark From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 21:05:09 2018 Received: (at 30820) by debbugs.gnu.org; 20 Mar 2018 01:05:09 +0000 Received: from localhost ([127.0.0.1]:42412 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey5i9-0004sY-3S for submit@debbugs.gnu.org; Mon, 19 Mar 2018 21:05:09 -0400 Received: from world.peace.net ([50.252.239.5]:33858) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey5i5-0004ry-H0 for 30820@debbugs.gnu.org; Mon, 19 Mar 2018 21:05:05 -0400 Received: from pool-72-93-34-155.bstnma.east.verizon.net ([72.93.34.155] helo=jojen) by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1ey5hy-0003tO-KV; Mon, 19 Mar 2018 21:04:58 -0400 From: Mark H Weaver To: Danny Milosavljevic Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> <20180319223402.1ba0a369@scratchpost.org> Date: Mon, 19 Mar 2018 21:04:09 -0400 In-Reply-To: <20180319223402.1ba0a369@scratchpost.org> (Danny Milosavljevic's message of "Mon, 19 Mar 2018 22:34:02 +0100") Message-ID: <87efkfblrq.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30820 Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Danny Milosavljevic writes: > On Mon, 19 Mar 2018 15:05:26 -0400 > Mark H Weaver wrote: > >> I think that we should generalize our reference scanning and grafting >> code to support store references broken into pieces, as long as each >> piece containing part of the hash is at least 8 bytes long. > > I don't think it's possible to get that to work reliably over all possible > optimizations. I mean why 8 Byte? That's probably because of the > 8 Byte registers on x86_64. > > What would happen on ARM? (32 bit registers)? > > And on MIPS (immediates only smaller than 32 bits)? It wouldn't make sense to do this kind of optimization with 4-byte chunks, because the overhead for the instructions would be too large to justify it. In any case, I've not seen any reports of any compiler generating code like this with 4-byte chunks. > What if gcc finds some repetition in the hash and decides not to load > the register again the second time? The probability of that happening with 8-byte chunks is on the order of 1 in 10^14, even if the GCC developers implemented such a thing. > I think the best way - since we have control over the entire toolchain anyway - > is to make gcc not do the data inlining in the first place. > > As far as I understand the gcc patches work after all. Ludo just made > a mistake testing it. > > Although when we do this patching of gcc we should add a test to gcc > that makes sure that the data inlining is indeed not there anymore > for store paths. > >> Here's my preliminary proposal: >> >> (1) The reference scanner should recognize any 8-byte substring of a >> hash as a valid reference to that hash. >> >> (2) To enable reliable grafting of chunked references, we should impose >> the following new restrictions: (a) the store prefix must be at >> least 6 bytes, (b) grafting can change only the hash, not the >> readable part of the store name, and (c) the readable part of the >> store name must be at least 6 bytes. >> >> (3) The grafter should recognize and replace any 8-byte subsequence of >> the absolute store file name. >> >> The rationale for the restrictions is to ensure that any byte that needs >> to be modified by the grafter should be part of an 8-byte substring of >> the absolute store file name. This requires that there be at least 7 >> bytes of known text before the first changed byte and after the last >> changed byte. This is needed to provide a reasonable upper bound on the >> probability of grafting a matching sequence of bytes that is not a store >> reference. > > I think that is a good way to get the risk down - but it will not actually > fix the problem for good. Nothing that we do will ever fix this problem for good. Don't pretend that patching GCC would fix the problem for good. There are problems in other software as well (e.g. in JAR manifests), and we already patched GCC once, and it broke some time later without anyone noticing. > Also, complexity like the above makes the reference scanner more brittle and > it's easier for bugs to hide in it (and it's slower). I think it could be implemented about as fast in practice, although it would require more memory. The hash table would be bigger, containing all 8-byte substrings of the hashes. As for bugs in the reference scanner: it's still simple enough that it could be formally proved correct without much difficulty. Mark From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 20 04:50:33 2018 Received: (at 30820) by debbugs.gnu.org; 20 Mar 2018 08:50:33 +0000 Received: from localhost ([127.0.0.1]:42698 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyCyW-0001TF-Vs for submit@debbugs.gnu.org; Tue, 20 Mar 2018 04:50:33 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:40490) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyCyV-0001T7-8P for 30820@debbugs.gnu.org; Tue, 20 Mar 2018 04:50:31 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 955D91209B; Tue, 20 Mar 2018 09:50:30 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AZRsCUNqw1hB; Tue, 20 Mar 2018 09:50:29 +0100 (CET) Received: from ribbon (unknown [193.50.110.92]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 9D46E12083; Tue, 20 Mar 2018 09:50:29 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Mark H Weaver Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> <20180319223402.1ba0a369@scratchpost.org> <87efkfblrq.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 30 =?utf-8?Q?Vent=C3=B4se?= an 226 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 20 Mar 2018 09:50:29 +0100 In-Reply-To: <87efkfblrq.fsf@netris.org> (Mark H. Weaver's message of "Mon, 19 Mar 2018 21:04:09 -0400") Message-ID: <87zi33w2p6.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820 Cc: Danny Milosavljevic , 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Mark H Weaver skribis: > Nothing that we do will ever fix this problem for good. Don't pretend > that patching GCC would fix the problem for good. There are problems in > other software as well (e.g. in JAR manifests), and we already patched > GCC once, and it broke some time later without anyone noticing. My initial message spread some confusion: the GCC patch still works as before, but there=E2=80=99s always been a corner case that was improperly handled. Now, we currently don=E2=80=99t have tests that would allow us to detect breakage before it bites, which is a problem. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 20 04:56:54 2018 Received: (at 30820) by debbugs.gnu.org; 20 Mar 2018 08:56:54 +0000 Received: from localhost ([127.0.0.1]:42702 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyD4f-0001cA-NL for submit@debbugs.gnu.org; Tue, 20 Mar 2018 04:56:53 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:40558) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyD4c-0001bz-W7 for 30820@debbugs.gnu.org; Tue, 20 Mar 2018 04:56:51 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 5E3B11258F; Tue, 20 Mar 2018 09:56:50 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lzSl2sebBw5Z; Tue, 20 Mar 2018 09:56:49 +0100 (CET) Received: from ribbon (unknown [193.50.110.92]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 37F6512572; Tue, 20 Mar 2018 09:56:49 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Mark H Weaver Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> <87370vg0e9.fsf@gnu.org> <87in9rbmay.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 30 =?utf-8?Q?Vent=C3=B4se?= an 226 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 20 Mar 2018 09:56:48 +0100 In-Reply-To: <87in9rbmay.fsf@netris.org> (Mark H. Weaver's message of "Mon, 19 Mar 2018 20:52:37 -0400") Message-ID: <87in9rw2en.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> Mark H Weaver skribis: >> >>> ludo@gnu.org (Ludovic Court=C3=A8s) writes: >>> >>>> The recently added glibc grafts triggered issues that, in the end, show >>>> the return of (=E2=80=9CStore references i= n 8-byte >>>> chunks in compiled code=E2=80=9D). >>> >>> I think that we should generalize our reference scanning and grafting >>> code to support store references broken into pieces, as long as each >>> piece containing part of the hash is at least 8 bytes long. >>> >>> Here's my preliminary proposal: >>> >>> (1) The reference scanner should recognize any 8-byte substring of a >>> hash as a valid reference to that hash. >>> >>> (2) To enable reliable grafting of chunked references, we should impose >>> the following new restrictions: (a) the store prefix must be at >>> least 6 bytes, (b) grafting can change only the hash, not the >>> readable part of the store name, and (c) the readable part of the >>> store name must be at least 6 bytes. >>> >>> (3) The grafter should recognize and replace any 8-byte subsequence of >>> the absolute store file name. >> >> I=E2=80=99m quite reluctant because it would add complexity, it will pro= bably >> slow things down, and yet it may not handle all the cases, as Danny >> suggests. >> >> Mind you, the GCC patches are not perfect either, but they=E2=80=99re re= latively >> easy to deal with (well, so far at least). In theory we would need >> similar patches for LLVM and maybe a couple other native compilers, >> though, which is obviously a downside, although we haven=E2=80=99t had a= ny >> problems so far. > > We would also need to find a solution to the problem described in the > thread "broken references in jar manifests" on guix-devel started by > Ricardo, which still has not found a satifactory solution. > > https://lists.gnu.org/archive/html/guix-devel/2018-03/msg00006.html > > My opinion is that I consider Guix's current expectations for how > software must store its data on disk to be far too onerous, in cases > where that data might include a store reference. I don't see sufficient > justification for imposing such an onerous requirement on the software > in Guix. In practice Guix and Nix have been living fine under these constraints, and with almost no modifications to upstream software, so it=E2=80=99s not = that bad. Nix doesn=E2=80=99t have grafts though, which is why this problem was= less visible there. > Ultimately, I would prefer to see the scanning and grafting operations > completely generalized, so that in general each package can specify how > to scan and graft that particular package, making use of libraries in > (guix build ...) to cover the usual cases. In most cases, that code > would be within build-systems. That would be precise GC instead of conservative GC in a way, right? So in essence we=E2=80=99d have, say, a scanner for ELF files (like =E2=80= =98dh_shdep=E2=80=99 in Debian or whatever it=E2=80=99s called), a scanner for jars, and so on? Still, how would we deal with strings embedded in the middle of binaries, as in this case? It seems to remain an open issue, no? I=E2=80=99m interested in experiments in that direction. I think that=E2= =80=99s a longer-term goal, though, and there are open questions: we have no idea how well that would work in practice. Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 20 19:07:36 2018 Received: (at 30820-done) by debbugs.gnu.org; 20 Mar 2018 23:07:36 +0000 Received: from localhost ([127.0.0.1]:44362 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyQLv-0004gF-RA for submit@debbugs.gnu.org; Tue, 20 Mar 2018 19:07:36 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:46562) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyQLs-0004g1-PH; Tue, 20 Mar 2018 19:07:34 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id DEBC11272A; Wed, 21 Mar 2018 00:07:31 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3lHELofH03Xv; Wed, 21 Mar 2018 00:07:30 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id A2D9812714; Wed, 21 Mar 2018 00:07:30 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: 30820-done@debbugs.gnu.org Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87efkm7eov.fsf@gnu.org> <87in9wxuwo.fsf@gnu.org> Date: Wed, 21 Mar 2018 00:07:30 +0100 In-Reply-To: <87in9wxuwo.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Fri, 16 Mar 2018 09:54:31 +0100") Message-ID: <87fu4uibwt.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820-done Cc: Ricardo Wurmus , Mark H Weaver , 30395-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Hello, ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > So the real issue is this: > >> The second issue is that the patch only ever worked with literal >> strings. It does not =E2=80=9Csee=E2=80=9D strings in constant arrays l= ike the =E2=80=98str=E2=80=99 >> array in the example above. Good news! Commit e288572710250bcd2aa0f69ce88154d98ac69b29 adjusts =E2=80=98gcc-strmov-store-file-names.patch=E2=80=99 in =E2=80=98core-update= s=E2=80=99 to correctly deal with this case: --8<---------------cut here---------------start------------->8--- $ cat strmov.c=20 #define _GNU_SOURCE #include static const char str[] =3D "This is a /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee string in a glob= al variable."; extern char *p, *q; #ifndef MEMCPY # define MEMCPY memcpy #endif void foo (char *x, char *y) { MEMCPY (x, str, sizeof str); MEMCPY (y, "this is a literal /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee= ee string", 35); } $ ./pre-inst-env guix build -e '(@@ (gnu packages commencement) gcc-final)' /gnu/store/wzdyqkdslk1s6f0vi9qw1xha8cniijzs-gcc-5.5.0-lib /gnu/store/46ww5s9zvsw04id438c4drpnwd9m6vl8-gcc-5.5.0 $ /gnu/store/46ww5s9zvsw04id438c4drpnwd9m6vl8-gcc-5.5.0/bin/gcc -O2 -c strm= ov.c $ objdump -S strmov.o |grep movabs $ NIX_STORE=3D/foo /gnu/store/46ww5s9zvsw04id438c4drpnwd9m6vl8-gcc-5.5.0/bi= n/gcc -O2 -c strmov.c $ objdump -S strmov.o |grep movabs 0: 48 b8 54 68 69 73 20 movabs $0x2073692073696854,%rax a: 48 ba 74 6f 72 65 2f movabs $0x6565652f65726f74,%rdx 1e: 48 b8 61 20 2f 67 6e movabs $0x732f756e672f2061,%rax 30: 48 b8 65 65 65 65 65 movabs $0x6565656565656565,%rax 4a: 48 b8 65 65 65 65 65 movabs $0x2065656565656565,%rax 58: 48 b8 73 74 72 69 6e movabs $0x6920676e69727473,%rax 66: 48 b8 6e 20 61 20 67 movabs $0x626f6c672061206e,%rax 74: 48 b8 61 6c 20 76 61 movabs $0x6169726176206c61,%rax 82: 48 b8 74 68 69 73 20 movabs $0x2073692073696874,%rax 93: 48 b8 61 20 6c 69 74 movabs $0x61726574696c2061,%rax a5: 48 b8 6c 20 2f 67 6e movabs $0x732f756e672f206c,%rax --8<---------------cut here---------------end--------------->8--- I built everything about to =E2=80=98gcc-final=E2=80=99 in =E2=80=98core-up= dates=E2=80=99. I checked manually that none of the /gnu/store references in libc-2.26.so were chunked. For the record, what the patch initially did was to skip code that would otherwise emit a =E2=80=9Cblock move=E2=80=9D when expanding __builtin_memc= py & co. This patch additionally skips similar code that would replace __builtin_memcpy calls with memory moves early on, in =E2=80=98gimple_fold_builtin_memory_op=E2=80=99, before =E2=80=98expand_bui= ltin=E2=80=99 is called. In the example above, this transformation would lead to the code below (as seen with =E2=80=98-fdump-tree-all=E2=80=99 in the =E2=80=98gimple=E2= =80=99 phase output): --8<---------------cut here---------------start------------->8--- foo (char * x, char * y) { MEM[(char * {ref-all})x] =3D MEM[(char * {ref-all})&str]; memcpy (y, "this is a literal /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee= ee string", 35); } --8<---------------cut here---------------end--------------->8--- With the patch we get: --8<---------------cut here---------------start------------->8--- foo (char * x, char * y) { memcpy (x, &str, 85); memcpy (y, "this is a literal /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee= ee string", 35); } --8<---------------cut here---------------end--------------->8--- Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 21 00:19:47 2018 Received: (at 30820) by debbugs.gnu.org; 21 Mar 2018 04:19:47 +0000 Received: from localhost ([127.0.0.1]:44689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyVE3-0007Md-G7 for submit@debbugs.gnu.org; Wed, 21 Mar 2018 00:19:47 -0400 Received: from world.peace.net ([50.252.239.5]:38204) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyVE1-0007MP-AC for 30820@debbugs.gnu.org; Wed, 21 Mar 2018 00:19:45 -0400 Received: from [172.16.1.2] (helo=jojen) by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1eyVDu-0007Hs-H1; Wed, 21 Mar 2018 00:19:38 -0400 From: Mark H Weaver To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> <87370vg0e9.fsf@gnu.org> <87in9rbmay.fsf@netris.org> <87in9rw2en.fsf@gnu.org> Date: Wed, 21 Mar 2018 00:17:53 -0400 In-Reply-To: <87in9rw2en.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Tue, 20 Mar 2018 09:56:48 +0100") Message-ID: <87woy62ham.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Hi Ludovic, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Mark H Weaver skribis: > >> We would also need to find a solution to the problem described in the >> thread "broken references in jar manifests" on guix-devel started by >> Ricardo, which still has not found a satifactory solution. >> >> https://lists.gnu.org/archive/html/guix-devel/2018-03/msg00006.html Okay, do you have a proposed fix for the issue of jar manifests? There's a specification for that file format which mandates that "No line may be longer than 72 bytes (not characters), in its UTF8-encoded form. If a value would make the initial line longer than this, it should be continued on extra lines (each starting with a single SPACE)." >> My opinion is that I consider Guix's current expectations for how >> software must store its data on disk to be far too onerous, in cases >> where that data might include a store reference. I don't see sufficient >> justification for imposing such an onerous requirement on the software >> in Guix. > > In practice Guix and Nix have been living fine under these constraints, > and with almost no modifications to upstream software, so it=E2=80=99s no= t that > bad. Nix doesn=E2=80=99t have grafts though, which is why this problem w= as less > visible there. > >> Ultimately, I would prefer to see the scanning and grafting operations >> completely generalized, so that in general each package can specify how >> to scan and graft that particular package, making use of libraries in >> (guix build ...) to cover the usual cases. In most cases, that code >> would be within build-systems. > > That would be precise GC instead of conservative GC in a way, right? > So in essence we=E2=80=99d have, say, a scanner for ELF files (like =E2= =80=98dh_shdep=E2=80=99 > in Debian or whatever it=E2=80=99s called), a scanner for jars, and so on? No, I wasn't thinking along those lines. While I'd very much prefer precise GC, it seems wholly infeasible for us to write precise scanners and grafters for every file format of every package in Guix. My thought was that supporting scanning and grafting of 8-byte-or-longer substrings of hashes would cover both GCC's inlined strings and jar manifests, the two issues that we currently know about, and that it would be nice if we could add further methods in the future. For example, some software might store its data in UTF-16, or compressed. > Still, how would we deal with strings embedded in the middle of > binaries, as in this case? It seems to remain an open issue, no? I believe that I addressed that case in my original proposal, no? > I=E2=80=99m interested in experiments in that direction. I think that=E2= =80=99s a > longer-term goal, though, and there are open questions: we have no idea > how well that would work in practice. Thanks for discussing it. I'm willing to drop it and go with your decision for now, but the "jar manifest" issue still needs a solution. Regards, Mark From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 21 01:44:04 2018 Received: (at 30820) by debbugs.gnu.org; 21 Mar 2018 05:44:04 +0000 Received: from localhost ([127.0.0.1]:44732 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyWXc-00018v-ET for submit@debbugs.gnu.org; Wed, 21 Mar 2018 01:44:04 -0400 Received: from world.peace.net ([50.252.239.5]:38350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyWXa-00018J-Cw for 30820@debbugs.gnu.org; Wed, 21 Mar 2018 01:44:02 -0400 Received: from [172.16.1.2] (helo=jojen) by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1eyWXU-0007wj-IG; Wed, 21 Mar 2018 01:43:56 -0400 From: Mark H Weaver To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=), Danny Milosavljevic Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87muz3dgy1.fsf@netris.org> Date: Wed, 21 Mar 2018 01:43:15 -0400 In-Reply-To: <87muz3dgy1.fsf@netris.org> (Mark H. Weaver's message of "Mon, 19 Mar 2018 15:05:26 -0400") Message-ID: <87sh8u2dcc.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30820 Cc: 30820@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) I just realized that my proposal is unworkable. If we allow strings containing store paths to be split into pieces, then some of those pieces may contain as little as one character of the hash. For example, the grafter might find "/store/c", which is likely not enough to determine which of the transitive inputs is being referenced, and therefore the grafter cannot decide what to write in place of the "c". Sorry for the noise. Mark From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 21 02:39:46 2018 Received: (at 30820-done) by debbugs.gnu.org; 21 Mar 2018 06:39:46 +0000 Received: from localhost ([127.0.0.1]:44765 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyXPW-0002Qe-Kx for submit@debbugs.gnu.org; Wed, 21 Mar 2018 02:39:46 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21021) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eyXPT-0002QO-Ul for 30820-done@debbugs.gnu.org; Wed, 21 Mar 2018 02:39:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1521614363; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:In-reply-to:Date:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; l=565; bh=S24ta7n2M1pxUlLdNyW4z7LRBFtPQF2AUbm5kwSIFLI=; b=RA8GzkSFW0XySrBu4yPZ+8F8cSAB8wrphXQUoBOy7VRt7jF+2zsKkmGLq9WM5Nvm LzjbfKlojWzzDAo3Fb5hhI2LboANMwo5EMqAr/lQH5XaxGfFNre54wBkYWRi6w692Dg h48gqfZlA4yg4VMXQQleV48oR+CAWZjbfCmWBvQY= Received: from localhost (port-92-200-42-42.dynamic.qsc.de [92.200.42.42]) by mx.zohomail.com with SMTPS id 1521614363597790.1984456585794; Tue, 20 Mar 2018 23:39:23 -0700 (PDT) References: <87o9jq7j7r.fsf@gnu.org> <87efkm7eov.fsf@gnu.org> <87in9wxuwo.fsf@gnu.org> <87fu4uibwt.fsf@gnu.org> User-agent: mu4e 1.0; emacs 25.3.1 From: Ricardo Wurmus To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) In-reply-to: <87fu4uibwt.fsf@gnu.org> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Wed, 21 Mar 2018 07:39:20 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-ID: <87bmfic4pz.fsf@elephly.net> X-ZohoMailClient: External X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30820-done Cc: Mark H Weaver , 30395-done@debbugs.gnu.org, 30820-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi Ludo, > Good news! Commit e288572710250bcd2aa0f69ce88154d98ac69b29 adjusts > =E2=80=98gcc-strmov-store-file-names.patch=E2=80=99 in =E2=80=98core-upda= tes=E2=80=99 to correctly deal > with this case: [=E2=80=A6] > I built everything about to =E2=80=98gcc-final=E2=80=99 in =E2=80=98core-= updates=E2=80=99. I checked > manually that none of the /gnu/store references in libc-2.26.so were > chunked. Wow, thank you so much for fixing this! Is this an option that we can suggest to the GCC developers to officially support? --=20 Ricardo From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 21 16:59:44 2018 Received: (at 30820-done) by debbugs.gnu.org; 21 Mar 2018 20:59:44 +0000 Received: from localhost ([127.0.0.1]:46321 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eykpk-0001b8-DY for submit@debbugs.gnu.org; Wed, 21 Mar 2018 16:59:44 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:54844) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eykpi-0001au-3b; Wed, 21 Mar 2018 16:59:42 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 5D41B12962; Wed, 21 Mar 2018 21:59:41 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yhFDScpINZ36; Wed, 21 Mar 2018 21:59:40 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:6a6c:dc17:fc02:cfda]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 19B471295F; Wed, 21 Mar 2018 21:59:39 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Ricardo Wurmus Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again) References: <87o9jq7j7r.fsf@gnu.org> <87efkm7eov.fsf@gnu.org> <87in9wxuwo.fsf@gnu.org> <87fu4uibwt.fsf@gnu.org> <87bmfic4pz.fsf@elephly.net> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 1 Germinal an 226 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 21 Mar 2018 21:59:39 +0100 In-Reply-To: <87bmfic4pz.fsf@elephly.net> (Ricardo Wurmus's message of "Wed, 21 Mar 2018 07:39:20 +0100") Message-ID: <87605pf8lg.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30820-done Cc: Mark H Weaver , 30395-done@debbugs.gnu.org, 30820-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Hello, Ricardo Wurmus skribis: >> Good news! Commit e288572710250bcd2aa0f69ce88154d98ac69b29 adjusts >> =E2=80=98gcc-strmov-store-file-names.patch=E2=80=99 in =E2=80=98core-upd= ates=E2=80=99 to correctly deal >> with this case: > [=E2=80=A6] >> I built everything about to =E2=80=98gcc-final=E2=80=99 in =E2=80=98core= -updates=E2=80=99. I checked >> manually that none of the /gnu/store references in libc-2.26.so were >> chunked. > > Wow, thank you so much for fixing this! It turned out to be easier than the first time. ;-) > Is this an option that we can suggest to the GCC developers to > officially support? I don=E2=80=99t know, it=E2=80=99s a Guix-specific hack, and just explainin= g the rationale to GCC people may be tricky: they=E2=80=99ll think we=E2=80=99re = all mad. ;-) Ludo=E2=80=99. From unknown Fri Aug 15 18:06:21 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 19 Apr 2018 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator