GNU bug report logs - #48064
texlive-* packages fail to build non-deterministically

Previous Next

To reply to this bug, email your comments to 48064 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: <bug-guix <at> gnu.org>
Subject: texlive-* packages fail to build non-deterministically
Date: Tue, 27 Apr 2021 17:39:05 +0200

Some texlive- packages fail to build non-deterministically.  This one is
on powerpc64le-linux from current master (commit
d904abe0768293b2322dbf355b6e41d94e769d78), but ISTR we saw the same kind
of problem on other architectures before:

--8<---------------cut here---------------start------------->8---
Processing file fileerr.dtx (help) -> h.tex
                            (scroll) -> s.tex
                            (edit) -> e.tex
                            (batch) -> q.tex
                            (run) -> r.tex
                            (exit) -> x.tex
File fileerr.dtx ended by \endinput.
Lines  processed: 119
Comments removed: 100
Comments  passed: 0
Codelines passed: 10

***********************************************************
*
* To finish the installation you have to move the following
* files into a directory searched by TeX:
*
* All the files with extension `.sty' and `.tex'
* Note there also may be a file .tex which is `invisible'
* on some operating systems.
*
* To produce the documentation run the .dtx files through LaTeX.
*
* Happy TeXing
***********************************************************

* Finally trying to make a file `.tex'.
* (Placed at the end of this run, as this
* may fail on some operating systems.)

Generating file(s) .tex

realloc(): invalid next size
command "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "tools.ins" failed with signal 6
builder for `/gnu/store/k9v4w4vc9q22yrrl5ggmpcymidwcbamf-texlive-latex-tools-51265.drv' failed with exit code 1
--8<---------------cut here---------------end--------------->8---

Notice the “realloc” message (from glibc) suggesting heap corruption.

Often, simply retrying yields a successful build.

Ludo’.

Message #10 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: 48064 <at> debbugs.gnu.org
Subject: texlive-* packages fail to build non-deterministically
Date: Sat, 05 Jun 2021 22:36:51 -0300

[Message part 1 (text/plain, inline)]

I just tried building core-updates after the upgrade to TeX Live 2020
(core-updates commit f72a125) and texlive-amscls failed with a slightly
different error coming from glibc, but still indicative of heap corruption:
“corrupted size vs. prev_size”

Full build output attached, and relevant snippet below:

```
This is LuaTeX, Version 1.12.0 (TeX Live 2020) 
 restricted system commands enabled.
(./amsdtx.ins (/gnu/store/6w056ryry98gw1h6hl1xnh725l7k5jg9-texlive-latex-base-54632/share/texmf-dist/tex/latex/base/docstrip.tex
Utility: `docstrip' 2.5g <2018/05/03>
English documentation    <2018/05/03>

**********************************************************
* This program converts documented macro-files into fast *
* loadable files by stripping off (nearly) all comments! *
**********************************************************

********************************************************
* No Configuration file found, using default settings. *
********************************************************

)

Generating file(s) amsdtx.cls amsldoc.cls 

Processing file amsdtx.dtx (amsdtx) -> amsdtx.cls
                           (amsldoc) -> amsldoc.cls
File amsdtx.dtx ended by \endinput.
Lines  processed: 1196
Comments removed: 573 
Comments  passed: 1
Codelines passed: 584 

)
warning  (pdf backend): no pages of output.
Transcript written on amsdtx.log.
corrupted size vs. prev_size
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "luatex" arguments: ("-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsdtx.ins") exit-status: #f term-signal: 6 stop-signal: #f> 
phase `build' failed after 6.5 seconds
command "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsdtx.ins" failed with signal 6
```

I’ll keep trying to run valgrind on it to see where the corruption happens.
-- 
Thanks,
Thiago

[kl1j9qpdpplhavyjwpk1sgrafcl5kx-texlive-amscls-54632.drv (text/plain, attachment)]

Message #13 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: 48064 <at> debbugs.gnu.org
Subject: texlive-* packages fail to build non-deterministically
Date: Tue, 08 Jun 2021 21:28:25 -0300

Hello,

I was able to do a build of texlive-amscls with `export MALLOC_CHECK_=2`, 
and a core dump was generated. I managed to get a guix environment with 
debuginfo for texlive-bin, but for some reason it still doesn’t have 
debug info for glibc available. FYI, this was the command line I used:

$ guix environment texlive-amscls --with-debug-info=texlive-bin \
	--ad-hoc valgrind gdb glibc:debug

Valgrind isn’t working because of the lack of glibc debug info, but I was 
able to get the backtrace below from the core file, using GDB. I am yet to
analyse it and see if it provides any clue:

Core was generated by `luatex amsclass.ins'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f50ad3d5a7a in raise () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
(gdb) backtrace
#0  0x00007f50ad3d5a7a in raise () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#1  0x00007f50ad3c0523 in abort () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#2  0x00007f50ad415d28 in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#3  0x00007f50ad41d54a in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#4  0x00007f50ad421bd7 in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#5  0x0000000000486c7e in my_luaalloc (ud=<optimized out>, ptr=<optimized out>, osize=1104, nsize=<optimized out>)
    at ../../../texlive-20190410-source/texk/web2c/luatexdir/lua/luastuff.c:115
#6  0x00007f50ad754b81 in luaM_realloc_ (L=L <at> entry=0x12212a8, block=block <at> entry=0x1316cc0,
    osize=osize <at> entry=1104, nsize=nsize <at> entry=2208)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lmem.c:86
#7  0x00007f50ad74d8b3 in luaD_reallocstack (L=0x12212a8, newsize=138)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:182
#8  0x00007f50ad74e121 in luaD_precall (L=L <at> entry=0x12212a8, func=<optimized out>, nresults=0)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:424
#9  0x00007f50ad74e393 in luaD_precall (nresults=<optimized out>, func=<optimized out>, L=0x12212a8)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:413
#10 luaD_call (L=L <at> entry=0x12212a8, func=<optimized out>, nResults=<optimized out>)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:498
#11 0x00007f50ad74e3f1 in luaD_callnoyield (L=0x12212a8, func=<optimized out>, nResults=<optimized out>)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:509
#12 0x00007f50ad74d83f in luaD_rawrunprotected (L=L <at> entry=0x12212a8, f=f <at> entry=0x7f50ad74f6d0 <dothecall>,
    ud=ud <at> entry=0x0) at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:142
#13 0x00007f50ad74e6fb in luaD_pcall (L=L <at> entry=0x12212a8, func=func <at> entry=0x7f50ad74f6d0 <dothecall>,
    u=u <at> entry=0x0, old_top=1200, ef=ef <at> entry=0)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:729
#14 0x00007f50ad74f5dd in GCTM (L=L <at> entry=0x12212a8, propagateerrors=propagateerrors <at> entry=0)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:823
#15 0x00007f50ad750eca in callallpendingfinalizers (L=<optimized out>)
--Type <RET> for more, q to quit, c to continue without paging--
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:862
#16 luaC_freeallobjects (L=L <at> entry=0x12212a8)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:971
#17 0x00007f50ad75a93e in close_state (L=0x12212a8)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lstate.c:245
#18 0x00007f50ad75ae20 in lua_close (L=<optimized out>)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lstate.c:344
#19 0x00000000004999a6 in do_final_end ()
    at ../../../texlive-20190410-source/texk/web2c/luatexdir/tex/errors.c:257
#20 0x000000000044ea9e in main (ac=<optimized out>, av=<optimized out>)
    at ../../../texlive-20190410-source/texk/web2c/luatexdir/luatex.c:609
(gdb)
-- 
Thanks,
Thiago

Message #18 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: dev-luatex <at> ntg.nl
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Tue, 29 Jun 2021 16:02:34 +0200

Hello,

While investigating luatex crashes in the TeX Live 2020 package of
GNU Guix¹, we identified the following heap corruption reported by
Valgrind (this is on GNU/Linux, with glibc 2.33):

--8<---------------cut here---------------start------------->8---
sh-5.0$  ~ludo/.guix-profile/bin/valgrind --extra-debuginfo-path=/gnu/store/f933bvd6ab6l2lg6xl6k1a6jwvcls0z6-glibc-2.33-debug/lib/debug "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsbsy.dtx"
==28531== Memcheck, a memory error detector
==28531== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==28531== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==28531== Command: luatex -interaction=nonstopmode -output-directory=build &luatex amsbsy.dtx
==28531== 
This is LuaTeX, Version 1.12.0 (TeX Live 2020) 
 restricted system commands enabled.
==28531== Invalid write of size 8
==28531==    at 0x485C691: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x45118D: main (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==  Address 0xae14170 is 0 bytes after a block of size 1,168 alloc'd
==28531==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28531==    by 0x46695D: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x486D932: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x48660F2: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x4868BE7: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x4868FCF: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x486988B: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x485C6BB: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== 
==28531== Invalid write of size 4
==28531==    at 0x485C6A2: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x45118D: main (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==  Address 0xae14178 is 8 bytes after a block of size 1,168 alloc'd
==28531==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28531==    by 0x46695D: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x486D932: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x48660F2: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x4868BE7: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x4868FCF: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x486988B: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x485C6BB: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== 

[...]

valgrind: m_mallocfree.c:305 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1232, hi = 68.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.
--8<---------------cut here---------------end--------------->8---

Does that ring a bell?  Is there a chance this problem was fixed in the
meantime?

Thanks in advance,
Ludovic.

¹ https://issues.guix.gnu.org/48064

Message #21 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: dev-luatex <at> ntg.nl
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Tue, 29 Jun 2021 17:10:16 +0200

Hi,

Ludovic Courtès <ludo <at> gnu.org> skribis:

> While investigating luatex crashes in the TeX Live 2020 package of
> GNU Guix¹, we identified the following heap corruption reported by
> Valgrind (this is on GNU/Linux, with glibc 2.33):

This time with debug info for luatex:

--8<---------------cut here---------------start------------->8---
sh-5.0$  ~ludo/.guix-profile/bin/valgrind --extra-debuginfo-path=/gnu/store/f933bvd6ab6l2lg6xl6k1a6jwvcls0z6-glibc-2.33-debug/lib/debug "/gnu/store/00addvl34y6qj8k9k0bnx9jrgxqwry6q-texlive-bin-20200406/bin/luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsbsy.dtx"
==21562== Memcheck, a memory error detector
==21562== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==21562== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==21562== Command: /gnu/store/00addvl34y6qj8k9k0bnx9jrgxqwry6q-texlive-bin-20200406/bin/luatex -interaction=nonstopmode -output-directory=build &luatex amsbsy.dtx
==21562== 
This is LuaTeX, Version 1.12.0 (TeX Live 2020) 
 restricted system commands enabled.
==21562== Invalid write of size 8
==21562==    at 0x485C691: lua_pushlstring (lapi.c:483)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562==    by 0x45118D: main (luatex.c:609)
==21562==  Address 0xac0fc30 is 0 bytes after a block of size 1,168 alloc'd
==21562==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562==    by 0x46695D: my_luaalloc (luastuff.c:115)
==21562==    by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562==    by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562==    by 0x4868BE7: traversethread (lgc.c:549)
==21562==    by 0x4868BE7: propagatemark (lgc.c:588)
==21562==    by 0x4868FCF: singlestep (lgc.c:1057)
==21562==    by 0x486988B: luaC_step (lgc.c:1137)
==21562==    by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562== 
==21562== Invalid write of size 4
==21562==    at 0x485C6A2: lua_pushlstring (lapi.c:483)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562==    by 0x45118D: main (luatex.c:609)
==21562==  Address 0xac0fc38 is 8 bytes after a block of size 1,168 alloc'd
==21562==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562==    by 0x46695D: my_luaalloc (luastuff.c:115)
==21562==    by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562==    by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562==    by 0x4868BE7: traversethread (lgc.c:549)
==21562==    by 0x4868BE7: propagatemark (lgc.c:588)
==21562==    by 0x4868FCF: singlestep (lgc.c:1057)
==21562==    by 0x486988B: luaC_step (lgc.c:1137)
==21562==    by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562== 
==21562== Invalid read of size 16
==21562==    at 0x485D269: lua_rawset (lapi.c:809)
==21562==    by 0x568E14: load_hyphenation (texlang.c:307)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562==    by 0x45118D: main (luatex.c:609)
==21562==  Address 0xac0fc30 is 0 bytes after a block of size 1,168 alloc'd
==21562==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562==    by 0x46695D: my_luaalloc (luastuff.c:115)
==21562==    by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562==    by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562==    by 0x4868BE7: traversethread (lgc.c:549)
==21562==    by 0x4868BE7: propagatemark (lgc.c:588)
==21562==    by 0x4868FCF: singlestep (lgc.c:1057)
==21562==    by 0x486988B: luaC_step (lgc.c:1137)
==21562==    by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
--8<---------------cut here---------------end--------------->8---

Ludo’.

Message #24 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: 48064 <at> debbugs.gnu.org
Subject: texlive-* packages fail to build non-deterministically
Date: Tue, 29 Jun 2021 16:59:48 -0300

I have bad news and good news. :-)

Unfortunately, TeX Live 2021 still has this bug. I tested version 20210325 
(which is the latest on the historic TeX Live FTP site), with subversion 
tag texlive-2021.3 (which is the latest tag in the TeX Live repo).

The good news is that I found a simple workaround: use pdftex instead of 
luatex to build the affected packages. I am currently building all packages 
matching ‘^texlive’ a few times to find the ones needing this workaround.
So far, I found these:

• texlive-amsfonts
• texlive-amscls
• texlive-babel
• texlive-babel-swedish
• texlive-latex-amsmath
• texlive-generic-babel-english
• texlive-latex-cyrillic
• texlive-latex-graphics
• texlive-latex-tools

I will submit a patch to the guix-patches mailing list (with this bug in 
Cc:) as soon as the process finishes.

Message #27 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Wed, 30 Jun 2021 12:05:03 +0200

Hi Thiago,

Thiago Jung Bauermann <bauermann <at> kolabnow.com> skribis:

> I was able to do a build of texlive-amscls with `export MALLOC_CHECK_=2`, 
> and a core dump was generated. I managed to get a guix environment with 
> debuginfo for texlive-bin, but for some reason it still doesn’t have 
> debug info for glibc available. FYI, this was the command line I used:
>
> $ guix environment texlive-amscls --with-debug-info=texlive-bin \
> 	--ad-hoc valgrind gdb glibc:debug

FWIW I managed to get a Valgrind report for TeX Live on core-updates,
using ‘--extra-debuginfo-path’ to help it a bit:

  https://issues.guix.gnu.org/48064#6

Not that it helps a whole lot yet!

Ludo’.

Message #30 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: luigi scarso <luigi.scarso <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: luatex development list <dev-luatex <at> ntg.nl>, 48064 <at> debbugs.gnu.org
Subject: Re: [Dev-luatex] bug#48064: texlive-* packages fail to build
 non-deterministically
Date: Wed, 30 Jun 2021 13:53:41 +0200

[Message part 1 (text/plain, inline)]

On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo <at> gnu.org> wrote:

> Hi,
>
> Ludovic Courtès <ludo <at> gnu.org> skribis:
>
> > While investigating luatex crashes in the TeX Live 2020 package of
> > GNU Guix¹, we identified the following heap corruption reported by
> > Valgrind (this is on GNU/Linux, with glibc 2.33):
>
> This time with debug info for luatex:
>

Thank you for the report, I will check asap.


-- 
luigi

[Message part 2 (text/html, inline)]

Message #33 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: Ludovic Courtès <ludo <at> gnu.org>, 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Wed, 30 Jun 2021 11:54:45 -0300

Em quarta-feira, 30 de junho de 2021, às 09:46:01 -03, Thiago Jung
Bauermann escreveu:
> I will use your command line to get a Valgrind report for TeX Live 2021
> and post it to the dev-luatex mailing list as well.

Unfortunately I’m still having problems with glibc debug info and thus
Valgrind doesn’t work for me.

When I use this command:

$ guix environment texlive-amscls --pure	\
	--with-debug-info=texlive-bin	\
	--ad-hoc valgrind glibc:debug gdb 

I get luatex linked with this glibc:

$ /bin/which luatex
/gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex
$ ldd /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex | grep libc.so
        libc.so.6 => /gnu/store/rhlm1nixi98x09xbyjc4i38gl9xs01f2-glibc-2.33/lib/libc.so.6 (0x00007fec0ac3d000)

But for some reason I get debug info for a different glibc:

$ ls -1 /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/lib/debug/gnu/store/
5ahv6zfv64jm5y7sficll0k2scr7cxvz-glibc-2.33
d9r40z2klf9l25pjk9k71z3dvxrsifzs-glibc-2.33-static

-- 
Thanks,
Thiago

Message #36 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <thiago.bauermann <at> kolabnow.com>
To: Ludovic Courtès <ludo <at> gnu.org>,
 Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Wed, 30 Jun 2021 09:46:01 -0300

Hello Ludo,

Em 30 de junho de 2021 07:05:03 BRT, "Ludovic Courtès" <ludo <at> gnu.org> escreveu:
>Hi Thiago,
>
>FWIW I managed to get a Valgrind report for TeX Live on core-updates,
>using ‘--extra-debuginfo-path’ to help it a bit:
>
>  https://issues.guix.gnu.org/48064#6
>
>Not that it helps a whole lot yet!

I saw it, but after I posted my message above. Thank you!

I will use your command line to get a Valgrind report for TeX Live 2021 and post it to the dev-luatex mailing list as well.

-- 
Enviado de meu dispositivo Android com K-9 mail. Desculpe-me pela brevidade.

Message #39 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Thu, 01 Jul 2021 15:07:27 +0200

Thiago Jung Bauermann <bauermann <at> kolabnow.com> skribis:

> Em quarta-feira, 30 de junho de 2021, às 09:46:01 -03, Thiago Jung
> Bauermann escreveu:
>> I will use your command line to get a Valgrind report for TeX Live 2021
>> and post it to the dev-luatex mailing list as well.
>
> Unfortunately I’m still having problems with glibc debug info and thus
> Valgrind doesn’t work for me.
>
> When I use this command:
>
> $ guix environment texlive-amscls --pure	\
> 	--with-debug-info=texlive-bin	\
> 	--ad-hoc valgrind glibc:debug gdb 
>
> I get luatex linked with this glibc:
>
> $ /bin/which luatex
> /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex
> $ ldd /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex | grep libc.so
>         libc.so.6 => /gnu/store/rhlm1nixi98x09xbyjc4i38gl9xs01f2-glibc-2.33/lib/libc.so.6 (0x00007fec0ac3d000)
>
> But for some reason I get debug info for a different glibc:
>
> $ ls -1 /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/lib/debug/gnu/store/
> 5ahv6zfv64jm5y7sficll0k2scr7cxvz-glibc-2.33
> d9r40z2klf9l25pjk9k71z3dvxrsifzs-glibc-2.33-static

Yes, you need debug info for (@@ (gnu packages commencement)
glibc-final), which is not what you get when you type “glibc:debug”.

HTH!

Ludo’.

Message #42 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: luigi scarso <luigi.scarso <at> gmail.com>
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 luatex development list <dev-luatex <at> ntg.nl>, 48064 <at> debbugs.gnu.org
Subject: Re: [Dev-luatex] bug#48064: texlive-* packages fail to build
 non-deterministically
Date: Fri, 02 Jul 2021 12:11:00 -0300

Em quarta-feira, 30 de junho de 2021, às 08:53:41 -03, luigi scarso escreveu:
> On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo <at> gnu.org> wrote:
> > Hi,
> > 
> > Ludovic Courtès <ludo <at> gnu.org> skribis:
> > > While investigating luatex crashes in the TeX Live 2020 package of
> > > GNU Guix¹, we identified the following heap corruption reported by
> > 
> > > Valgrind (this is on GNU/Linux, with glibc 2.33):
> > This time with debug info for luatex:
> Thank you for the report, I will check asap.

Thanks! I was able to run Valgrind on LuaTeX 1.13.0, which is the latest 
one in TeX Live 2021.

The invalid reads and writes don’t happen on every run. I had to re-run the 
command 3 or 4 times until I got the result below (which matches our 
experience with the build failures in Guix packages)

-- 
Thanks,
Thiago


$ valgrind --extra-debuginfo-path=/gnu/store/rkhx3pj1qi7fx6pi9p2cg2sb9zn59qmg-profile/lib/debug luatex amsclass.ins
==239904== Memcheck, a memory error detector
==239904== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==239904== Using Valgrind-3.17.0 and LibVEX; rerun with -h for copyright info
==239904== Command: luatex amsclass.ins
==239904==
This is LuaTeX, Version 1.13.0 (TeX Live 2021)
 restricted system commands enabled.
==239904== Invalid write of size 8
==239904==    at 0x4860691: lua_pushlstring (lapi.c:483)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904==    at 0x48606A2: lua_pushlstring (lapi.c:483)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 16
==239904==    at 0x4861269: lua_rawset (lapi.c:809)
==239904==    by 0x56A974: load_hyphenation (texlang.c:307)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 1
==239904==    at 0x486127D: lua_rawset (lapi.c:811)
==239904==    by 0x56A974: load_hyphenation (texlang.c:307)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 8
==239904==    at 0x485F068: auxgetstr (lapi.c:596)
==239904==    by 0x463955: check_texconfig_init (luainit.c:1198)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904==    at 0x485F07A: auxgetstr (lapi.c:596)
==239904==    by 0x463955: check_texconfig_init (luainit.c:1198)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904==    at 0x4880608: luaV_finishget (lvm.c:176)
==239904==    by 0x485F089: auxgetstr (lapi.c:598)
==239904==    by 0x463955: check_texconfig_init (luainit.c:1198)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 4
==239904==    at 0x485F092: auxgetstr (lapi.c:601)
==239904==    by 0x463955: check_texconfig_init (luainit.c:1198)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 4
==239904==    at 0x485F6D9: lua_type (lapi.c:253)
==239904==    by 0x463966: check_texconfig_init (luainit.c:1199)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
(./amsclass.ins (/gnu/store/rkhx3pj1qi7fx6pi9p2cg2sb9zn59qmg-profile/share/texmf-dist/tex/latex/base/docstrip.tex==239904== Conditional jump or move depends on uninitialised value(s)
==239904==    at 0x4647BD: tprint (printing.c:484)
==239904==    by 0x4E5E6B: write_out (extensions.c:583)
==239904==    by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904==    by 0x4E62EA: do_extension (extensions.c:423)
==239904==    by 0x4F4860: main_control (maincontrol.c:1030)
==239904==    by 0x4F0537: main_body (mainbody.c:577)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==

==239904== Conditional jump or move depends on uninitialised value(s)
==239904==    at 0x4647BD: tprint (printing.c:484)
==239904==    by 0x4E5DED: write_out (extensions.c:585)
==239904==    by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904==    by 0x4E62EA: do_extension (extensions.c:423)
==239904==    by 0x4F4860: main_control (maincontrol.c:1030)
==239904==    by 0x4F0537: main_body (mainbody.c:577)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==
Utility: `docstrip' v2.6a <2020-07-07>
English documentation    <2020-07-11>
==239904== Conditional jump or move depends on uninitialised value(s)
==239904==    at 0x464523: tprint (printing.c:512)
==239904==    by 0x4E5DED: write_out (extensions.c:585)
==239904==    by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904==    by 0x4E62EA: do_extension (extensions.c:423)
==239904==    by 0x4F4860: main_control (maincontrol.c:1030)
==239904==    by 0x4F0537: main_body (mainbody.c:577)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==

**********************************************************
* This program converts documented macro-files into fast *
* loadable files by stripping off (nearly) all comments! *
**********************************************************

********************************************************
* No Configuration file found, using default settings. *
********************************************************

)

Generating file(s) amsthm.sty amsart.cls amsbook.cls amsproc.cls

Processing file amsclass.dtx (amsthm) -> amsthm.sty
                             (amsart,classes) -> amsart.cls
                             (amsbook,classes) -> amsbook.cls
                             (amsproc,classes) -> amsproc.cls
File amsclass.dtx ended by \endinput.
Lines  processed: 5197
Comments removed: 2926
Comments  passed: 21
Codelines passed: 2062

)
warning  (pdf backend): no pages of output.
Transcript written on amsclass.log.
==239904== Invalid write of size 8
==239904==    at 0x486C013: GCTM (lgc.c:819)
==239904==    by 0x486D779: callallpendingfinalizers (lgc.c:862)
==239904==    by 0x486D779: luaC_freeallobjects (lgc.c:971)
==239904==    by 0x4877A0B: close_state (lstate.c:245)
==239904==    by 0x4E33A5: do_final_end (errors.c:257)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa40 is 16 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==

valgrind: m_mallocfree.c:303 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1248, hi = 102.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.


host stacktrace:
==239904==    at 0x5803F050: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x5803F157: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x5803F2DE: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x58048742: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x58037DCB: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x58036637: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x5803AAB2: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x58035988: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x100417A3ED: ???
==239904==    by 0x1002CB9F2F: ???
==239904==    by 0xBF0E: ???

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 239904)
==239904==    at 0x486C01B: GCTM (lgc.c:820)
==239904==    by 0x486D779: callallpendingfinalizers (lgc.c:862)
==239904==    by 0x486D779: luaC_freeallobjects (lgc.c:971)
==239904==    by 0x4877A0B: close_state (lstate.c:245)
==239904==    by 0x4E33A5: do_final_end (errors.c:257)
==239904==    by 0x45118D: main (luatex.c:609)
client stack range: [0x1FFEFB0000 0x1FFF000FFF] client SP: 0x1FFF000130
valgrind stack range: [0x1002BBA000 0x1002CB9FFF] top usage: 9624 of 1048576


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

$ echo $?
1

Message #45 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Fri, 02 Jul 2021 12:17:32 -0300

Em quinta-feira, 1 de julho de 2021, às 10:07:27 -03, Ludovic Courtès 
escreveu:
> Yes, you need debug info for (@@ (gnu packages commencement)
> glibc-final), which is not what you get when you type “glibc:debug”.
> 
> HTH!

It did, thanks for the tip! For the record, this is the command line that 
worked for me:

$ guix environment texlive-amscls --pure	\
	--with-debug-info=texlive-bin	\
	--ad-hoc			\
	-e '(list (@@ (gnu packages commencement) glibc-final) “debug”)’  \
	valgrind gdb

Then I pointed valgrind at the /lib/debug directory in the environment’s 
profile, as seen in my previous message.

-- 
Thanks,
Thiago

Message #48 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: 48064 <at> debbugs.gnu.org
Cc: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Subject: [PATCH core-updates] build-system/texlive: Change default format to
 pdftex
Date: Fri,  2 Jul 2021 13:00:10 -0300

LuaTeX has a bug where sometimes it corrupts the heap and aborts. This
causes the build of texlive packages to fail at random. The problem is
being tracked at https://issues.guix.gnu.org/48064.

While a fix isn't found, switch the default TeX format (and consequently
also the engine) to pdftex to avoid the issue.

* guix/build-system/texlive.scm (texlive-build): Change default value of
the ‘tex-format’ key parameter to “pdftex”.
---
 guix/build-system/texlive.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Hello,

Originally I had a bigger patch which changed packages individually to
build with pdfTeX, because I thought the bug in LuaTeX was triggered only
by specific packages. As I changed more and more packages though, I
realized that any package can trigger the problem and the heap corruption
happens at random.

Therefore, to work around the LuaTeX bug we need to build all packages with
pdfTeX. I tested the patch with this script:

--8<---------------cut here---------------start------------->8---
#!/bin/bash

set -e

LOG_FILE="$1"
ROUNDS=$2

function verify_package() {
    local package="$1"
    local log_file="$2"

    if ! guix build "$package"; then
        echo "failure while building $package" >> "$log_file"
        return
    fi

    echo "success while building $package" >> "$log_file"

    if ! guix build --check --rounds=$ROUNDS "$package"; then
        echo "failure while checking $package" >> "$log_file"
        return
    fi

    echo "success while checking $package" >> "$log_file"

    return
}

guix describe >> "$LOG_FILE"
echo rounds = "$ROUNDS" >> "$LOG_FILE"
echo >> "$LOG_FILE"

for package in $(guix package --list-available='^texlive' | cut -f1)
do
    verify_package $package "$LOG_FILE"
done
--8<---------------cut here---------------end--------------->8---

and called it with `./verify-texlive-packages.sh verify-packages.log 5`.

I see some packages fail during `guix build --check` because they don't
build deterministically for some reason (which is a separate problem), but
I don't see any package failing to build anymore.

diff --git a/guix/build-system/texlive.scm b/guix/build-system/texlive.scm
index 0efa139fc124..00a36d5862d4 100644
--- a/guix/build-system/texlive.scm
+++ b/guix/build-system/texlive.scm
@@ -128,7 +128,7 @@ level package ID."
                         (tests? #f)
                         tex-directory
                         (build-targets #f)
-                        (tex-format "luatex")
+                        (tex-format "pdftex")
                         (phases '(@ (guix build texlive-build-system)
                                     %standard-phases))
                         (outputs '("out"))

Message #51 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Mon, 05 Jul 2021 10:42:55 +0200

Hello!

Thiago Jung Bauermann <bauermann <at> kolabnow.com> skribis:

> I have bad news and good news. :-)
>
> Unfortunately, TeX Live 2021 still has this bug. I tested version 20210325 
> (which is the latest on the historic TeX Live FTP site), with subversion 
> tag texlive-2021.3 (which is the latest tag in the TeX Live repo).

Bah.  Still, if you have the whole texlive upgrade, we should apply it!

> The good news is that I found a simple workaround: use pdftex instead of 
> luatex to build the affected packages. I am currently building all packages 
> matching ‘^texlive’ a few times to find the ones needing this workaround.
> So far, I found these:
>
> • texlive-amsfonts
> • texlive-amscls
> • texlive-babel
> • texlive-babel-swedish
> • texlive-latex-amsmath
> • texlive-generic-babel-english
> • texlive-latex-cyrillic
> • texlive-latex-graphics
> • texlive-latex-tools

We haven’t heard from dev-luatex yet.  I think we should go with this
workaround for now (those build failures are frequently preventing
evaluations at <https://ci.guix.gnu.org/jobset/core-updates> from
completing, which is a real bummer).  Does using ‘pdftex’ rather than
‘luatex’ have an impact on the output of these packages?

Thanks!

Ludo’.

Message #54 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Mon, 05 Jul 2021 11:20:20 +0200

Thiago Jung Bauermann <bauermann <at> kolabnow.com> skribis:

> LuaTeX has a bug where sometimes it corrupts the heap and aborts. This
> causes the build of texlive packages to fail at random. The problem is
> being tracked at https://issues.guix.gnu.org/48064.
>
> While a fix isn't found, switch the default TeX format (and consequently
> also the engine) to pdftex to avoid the issue.
>
> * guix/build-system/texlive.scm (texlive-build): Change default value of
> the ‘tex-format’ key parameter to “pdftex”.

Pushed as 04f9f9158da348e8299e9ab90ec389ba81be46b0 with the text above
inlined as a FIXME comment.

On IRC there were concerns about Unicode support, which LuaTeX provides
but pdftex doesn’t (IIUC), but it would seem that the worst that can
happen is that documentation of the packages themselves would be
mangled, which is okay.

Anyway, it’d be ideal to get feedback from the LuaTeX folks!

Thanks,
Ludo’.

Message #57 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 48064 <at> debbugs.gnu.org
Subject: Re: bug#48064: texlive-* packages fail to build non-deterministically
Date: Mon, 05 Jul 2021 14:27:16 -0300

Hi Ludo,

Em segunda-feira, 5 de julho de 2021, às 06:20:20 -03, Ludovic Courtès 
escreveu:
> Thiago Jung Bauermann <bauermann <at> kolabnow.com> skribis:
> > LuaTeX has a bug where sometimes it corrupts the heap and aborts. This
> > causes the build of texlive packages to fail at random. The problem is
> > being tracked at https://issues.guix.gnu.org/48064.
> > 
> > While a fix isn't found, switch the default TeX format (and
> > consequently
> > also the engine) to pdftex to avoid the issue.
> > 
> > * guix/build-system/texlive.scm (texlive-build): Change default value
> > of
> > the ‘tex-format’ key parameter to “pdftex”.
> 
> Pushed as 04f9f9158da348e8299e9ab90ec389ba81be46b0 with the text above
> inlined as a FIXME comment.

Thank you!

> On IRC there were concerns about Unicode support, which LuaTeX provides
> but pdftex doesn’t (IIUC), but it would seem that the worst that can
> happen is that documentation of the packages themselves would be
> mangled, which is okay.

I chose pdfTeX for the workaround because it’s the direct “predecessor” to 
LuaTeX, so I thought it would behave most similarly to it. But it’s just an 
uneducated guess.

Searching around a bit¹²³, XeTeX also has native Unicode support, so we 
could also switch to it. Either as the default, or for specific packages 
that need it.

NB: I last used TeX more than 15 years ago, and even then just lightly and 
sporadically. Don’t trust my judgement on TeX-related issues. :-)

> Anyway, it’d be ideal to get feedback from the LuaTeX folks!

Agreed!

-- 
Thanks,
Thiago

¹ https://tex.stackexchange.com/questions/13593/the-differences-between-tex-engines
² https://tex.stackexchange.com/questions/36/differences-between-luatex-context-and-xetex
³ https://www.tug.org/texlive/doc/texlive-en/texlive-en.html#x1-120002.4

Message #60 received at 48064 <at> debbugs.gnu.org (full text, mbox):

From: luigi scarso <luigi.scarso <at> gmail.com>
To: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 luatex development list <dev-luatex <at> ntg.nl>, 48064 <at> debbugs.gnu.org
Subject: Re: [Dev-luatex] bug#48064: texlive-* packages fail to build
 non-deterministically
Date: Fri, 31 Dec 2021 18:40:46 +0100

[Message part 1 (text/plain, inline)]

On Fri, Jul 2, 2021 at 5:11 PM Thiago Jung Bauermann <bauermann <at> kolabnow.com>
wrote:

> Em quarta-feira, 30 de junho de 2021, às 08:53:41 -03, luigi scarso
> escreveu:
> > On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo <at> gnu.org> wrote:
> > > Hi,
> > >
> > > Ludovic Courtès <ludo <at> gnu.org> skribis:
> > > > While investigating luatex crashes in the TeX Live 2020 package of
> > > > GNU Guix¹, we identified the following heap corruption reported by
> > >
> > > > Valgrind (this is on GNU/Linux, with glibc 2.33):
> > > This time with debug info for luatex:
> > Thank you for the report, I will check asap.
>
> Thanks! I was able to run Valgrind on LuaTeX 1.13.0, which is the latest
> one in TeX Live 2021.
>
> The invalid reads and writes don’t happen on every run. I had to re-run
> the
> command 3 or 4 times until I got the result below (which matches our
> experience with the build failures in Guix packages)
>
> --
> Thanks,
> Thiago
>

Until now I was not able to reproduce the issue.

-- 
luigi

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.