Subject: bug#30820: closed (Re: bug#30820: Chunked store references in compiled code break grafting (again))

From: [email-masked] (GNU bug Tracking System)
Date: Tue, 20 Mar 2018 23:08:03 +0000

Your bug report

#30395: Chunked store references in compiled code break grafting (again)

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
30395: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=30395
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#30820: Chunked store references in compiled code break grafting (again)
From: [email-masked] (Ludovic Courtès)
To: [email-masked]
Cc: Ricardo Wurmus <[email-masked]>, Mark H Weaver <[email-masked]>, [email-masked]
Date: Wed, 21 Mar 2018 00:07:30 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Hello,

[email-masked] (Ludovic Courtès) skribis:

> So the real issue is this:
>
>> The second issue is that the patch only ever worked with literal
>> strings.  It does not “see” strings in constant arrays like the ‘str’
>> array in the example above.

Good news!  Commit e288572710250bcd2aa0f69ce88154d98ac69b29 adjusts
‘gcc-strmov-store-file-names.patch’ in ‘core-updates’ to correctly deal
with this case:

--8<---------------cut here---------------start------------->8---
$ cat strmov.c 
#define _GNU_SOURCE
#include <string.h>
static const char str[] =
  "This is a /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee string in a global variable.";

extern char *p, *q;

#ifndef MEMCPY
# define MEMCPY memcpy
#endif

void foo (char *x, char *y)
{
  MEMCPY (x, str, sizeof str);
  MEMCPY (y, "this is a literal /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee string", 35);
}
$ ./pre-inst-env guix build -e '(@@ (gnu packages commencement) gcc-final)'
/gnu/store/wzdyqkdslk1s6f0vi9qw1xha8cniijzs-gcc-5.5.0-lib
/gnu/store/46ww5s9zvsw04id438c4drpnwd9m6vl8-gcc-5.5.0
$ /gnu/store/46ww5s9zvsw04id438c4drpnwd9m6vl8-gcc-5.5.0/bin/gcc -O2 -c strmov.c
$ objdump -S strmov.o |grep movabs
$ NIX_STORE=/foo /gnu/store/46ww5s9zvsw04id438c4drpnwd9m6vl8-gcc-5.5.0/bin/gcc -O2 -c strmov.c
$ objdump -S strmov.o |grep movabs
   0:	48 b8 54 68 69 73 20 	movabs $0x2073692073696854,%rax
   a:	48 ba 74 6f 72 65 2f 	movabs $0x6565652f65726f74,%rdx
  1e:	48 b8 61 20 2f 67 6e 	movabs $0x732f756e672f2061,%rax
  30:	48 b8 65 65 65 65 65 	movabs $0x6565656565656565,%rax
  4a:	48 b8 65 65 65 65 65 	movabs $0x2065656565656565,%rax
  58:	48 b8 73 74 72 69 6e 	movabs $0x6920676e69727473,%rax
  66:	48 b8 6e 20 61 20 67 	movabs $0x626f6c672061206e,%rax
  74:	48 b8 61 6c 20 76 61 	movabs $0x6169726176206c61,%rax
  82:	48 b8 74 68 69 73 20 	movabs $0x2073692073696874,%rax
  93:	48 b8 61 20 6c 69 74 	movabs $0x61726574696c2061,%rax
  a5:	48 b8 6c 20 2f 67 6e 	movabs $0x732f756e672f206c,%rax
--8<---------------cut here---------------end--------------->8---

I built everything about to ‘gcc-final’ in ‘core-updates’.  I checked
manually that none of the /gnu/store references in libc-2.26.so were
chunked.

For the record, what the patch initially did was to skip code that would
otherwise emit a “block move” when expanding __builtin_memcpy & co.
This patch additionally skips similar code that would replace
__builtin_memcpy calls with memory moves early on, in
‘gimple_fold_builtin_memory_op’, before ‘expand_builtin’ is called.

In the example above, this transformation would lead to the code below
(as seen with ‘-fdump-tree-all’ in the ‘gimple’ phase output):

--8<---------------cut here---------------start------------->8---
foo (char * x, char * y)
{
  MEM[(char * {ref-all})x] = MEM[(char * {ref-all})&str];
  memcpy (y, "this is a literal /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee string", 35);
}
--8<---------------cut here---------------end--------------->8---

With the patch we get:

--8<---------------cut here---------------start------------->8---
foo (char * x, char * y)
{
  memcpy (x, &str, 85);
  memcpy (y, "this is a literal /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee string", 35);
}
--8<---------------cut here---------------end--------------->8---

Ludo’.

Subject: Chunked store references in compiled code break grafting (again)
From: [email-masked] (Ludovic Courtès)
To: [email-masked]
Cc: Ricardo Wurmus <[email-masked]>
Date: Wed, 14 Mar 2018 16:47:04 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Hello Guix,

The recently added glibc grafts triggered issues that, in the end, show
the return of <http://bugs.gnu.org/24703> (“Store references in 8-byte
chunks in compiled code”).

Specifically on commit 2b5c5f03c2f0a84f84a5517e2e6f5fa9f276ffa5:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -e '((@ (guix packages) package-replacement) (@@ (gnu packages commencement) glibc-final))' --no-grafts
/gnu/store/m07pz38dvizwx2bl9aik6wcbbqbhz6j6-glibc-2.26.105-g0890d5379c-debug
/gnu/store/4sqaib7c2dfjv62ivrg9b8wa7bh226la-glibc-2.26.105-g0890d5379c
/gnu/store/m8m005z2jbvqrj3s5b052camzk2qxpz5-glibc-2.26.105-g0890d5379c-static
$ ./pre-inst-env guix build -e '((@ (guix packages) package-replacement) (@@ (gnu packages commencement) glibc-final))' 
/gnu/store/bdgcd723b8l1h8cg8wx4vjfypip29dsn-glibc-2.26.105-g0890d5379c-debug
/gnu/store/l2wzs674z5ac5ccrvp73gdqw02mmzr22-glibc-2.26.105-g0890d5379c
/gnu/store/2rp8zmymxi32wrw4s44f2dc67ci9kxgs-glibc-2.26.105-g0890d5379c-static
$ grep -r 4sqai /gnu/store/l2wzs674z5ac5ccrvp73gdqw02mmzr22-glibc-2.26.105-g0890d5379c
Duuma dosiero /gnu/store/l2wzs674z5ac5ccrvp73gdqw02mmzr22-glibc-2.26.105-g0890d5379c/sbin/sln kongruas
Duuma dosiero /gnu/store/l2wzs674z5ac5ccrvp73gdqw02mmzr22-glibc-2.26.105-g0890d5379c/sbin/nscd kongruas
--8<---------------cut here---------------end--------------->8---

If we look with hexl-mode, we see that libc-2.26.so contains some of
these too:

--8<---------------cut here---------------start------------->8---
000236a0: b92f 676e 752f 7374 6f48 8d50 01c6 003a  ./gnu/stoH.P...:
000236b0: 4889 4801 48b8 7265 2f34 7371 6169 31f6  H.H.H.re/4sqai1.
000236c0: 4889 4208 48b8 6237 6332 6466 6a76 31ff  H.B.H.b7c2dfjv1.
000236d0: 4889 4210 48b8 3632 6976 7267 3962 c642  H.B.H.62ivrg9b.B
000236e0: 5000 4889 4218 48b8 3877 6137 6268 3232  P.H.B.H.8wa7bh22
000236f0: 4889 4220 48b8 366c 612d 676c 6962 4889  H.B H.6la-glibH.
00023700: 4228 48b8 632d 322e 3236 2e31 4889 4230  B(H.c-2.26.1H.B0
00023710: 48b8 3035 2d67 3038 3930 4889 4238 48b8  H.05-g0890H.B8H.
00023720: 6435 3337 3963 2f6c 4889 4240 48b8 6962  d5379c/[email-masked]
00023730: 2f67 636f 6e76 4889 4248 e881 f70b 0048  /gconvH.BH.....H
--8<---------------cut here---------------end--------------->8---

That in turns means that gconv-modules won’t be found, and that guile
with crash during startup with “Uncaught exception” (because early on it
fails to convert file names to UTF-8 through iconv).

To be continued…

Ludo’.