From unknown Sun Jun 15 01:08:14 2025 X-Loop: help-debbugs@gnu.org Subject: bug#38041: crypto with gnutls aka nettle (libhogweed) and scm_realloc Resent-From: Linas Vepstas Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Sun, 03 Nov 2019 01:12:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 38041 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: 38041@debbugs.gnu.org X-Debbugs-Original-To: bug-guile@gnu.org Reply-To: linasvepstas@gmail.com Received: via spool by submit@debbugs.gnu.org id=B.157274347324879 (code B ref -1); Sun, 03 Nov 2019 01:12:01 +0000 Received: (at submit) by debbugs.gnu.org; 3 Nov 2019 01:11:13 +0000 Received: from localhost ([127.0.0.1]:33280 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iR4QB-0006T9-QG for submit@debbugs.gnu.org; Sat, 02 Nov 2019 21:11:13 -0400 Received: from lists.gnu.org ([209.51.188.17]:37786) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iR4Q9-0006T1-Gq for submit@debbugs.gnu.org; Sat, 02 Nov 2019 21:11:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45648) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iR4Q7-0003rz-PT for bug-guile@gnu.org; Sat, 02 Nov 2019 21:11:09 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iR4Q5-00041q-VW for bug-guile@gnu.org; Sat, 02 Nov 2019 21:11:07 -0400 Received: from mail-lj1-x22a.google.com ([2a00:1450:4864:20::22a]:32847) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iR4Q5-0003z5-KJ for bug-guile@gnu.org; Sat, 02 Nov 2019 21:11:05 -0400 Received: by mail-lj1-x22a.google.com with SMTP id t5so13928826ljk.0 for ; Sat, 02 Nov 2019 18:11:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:from:date:message-id:subject:to; bh=2r1voYUG5OkDOa4hieMxekp7KjspMeFodQU8k4WQynE=; b=se5AzzO/kUcjzhy5+wnfxa64xRs4yAy4fan7YniSVkfmIrsls1D+JsC0LdLhOEFfaP AVzXZV6RNwdbJsJYCU8yT/n1JgRRVD6JxgS5SbLF4i4Ja5OXoRRY1qAj3uUHvc17WoK3 Byoll8N5KwEVArcrng2RHMNgSQthbal9B10CokGNnhl1NS77eIfc4ddRLRQ83igjka/i XUpqCp9msWWK9BIS9Ama0yuUoyfKQcXdmhOY2p1Vh5IxBnqDn46dmj5Cx3E30KWFAeCc EFDzWPEf1f4cu0af6hcgq5Kg+ic2poZIgzBngMMRRsLIr1xdYSoG37TjepY14rmgOZj/ 02zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:from:date:message-id :subject:to; bh=2r1voYUG5OkDOa4hieMxekp7KjspMeFodQU8k4WQynE=; b=ke3zuxLHnn2BRrp4Lg7MBk5oGvDWFCKAgTIP0bKu9g1bYP/zIFvF+rzXhijZDvhF9v weIIEo8v9G7nhPADZfwTmNsiO7YedrJOe2Nkyx9iE2c2fdnAren1XynDnIjpUiQ1PZaD bH4BexzuC7SSELd3NA/Ja40gZIllRDcK7wPvolCSbbfiyiv/FZK2G00Wei7ErP4eQwFk wLgF74odTfJRiGNOG7PvZGCNwD2jkLAcXWhV4xiQ4UYtQwWgIOpMuofIU/0dnp4fDSuy 7kOVk91U2UcCCMxBlut8rPR8IXvD9ouz8vVgF6mXZhiThfCfdfgW7tx5ihQmkz3S5ZNR OfLw== X-Gm-Message-State: APjAAAUYPd/y9XTh491LvA9T+ensE7a0neRaZ/am+lGI1wri7AeWnSJP 9yOIBT1+0tsliaEOjyNxQL7yZrX2PDXVYJxDLHVRBfQi X-Google-Smtp-Source: APXvYqzwrGm/irhXW5nHLNoseurUEByZU5Z8Fk1lWXeLAi1X2fVD+TnoSr2bLQ0m8aE/kYcDV17hgR+ihRP1+RsZDEM= X-Received: by 2002:a2e:3313:: with SMTP id d19mr6588595ljc.240.1572743463348; Sat, 02 Nov 2019 18:11:03 -0700 (PDT) MIME-Version: 1.0 From: Linas Vepstas Date: Sat, 2 Nov 2019 20:10:50 -0500 Message-ID: Content-Type: multipart/alternative; boundary="00000000000074aee5059666dfcb" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::22a X-Spam-Score: 0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --00000000000074aee5059666dfcb Content-Type: text/plain; charset="UTF-8" I've got an app that links gnutls (for crypto code) which links nettle (libhogweed) with is a GMP-using crypto library which seems like it wanted to call plain-old realloc, and ended up calling scm_realloc instead. Note that nettle does NOT use guile, so there's no plausible way that I know of to end up in guile code. This only seems to happen when nettle is used from multiple threads (so is maybe a nettle bug??) but the stack trace is so bizarre, I thought I'd report it here. It would seem that someone, somewhere, is doing some low-level thunking or trampolining of realloc(). First, the crazy stack trace: It's currently highly reproducible and exact: (gdb) r Starting program: /home/linas/src/novamente/src/atomspace-dht/build/tests/persist/dht/MultiUserUTest [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Running cxxtest tests (2 tests)Start creating 6 user sessions [2019-11-03 00:46:03:350] [DEBUG] BEGIN TEST: test_multiuser Collecting from unknown thread Thread 13 "MultiUserUTest" received signal SIGABRT, Aborted. [Switching to Thread 0x7fffe0ff9700 (LWP 3844)] __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007ffff6834535 in __GI_abort () at abort.c:79 #2 0x00007ffff5c80ded in GC_push_all_stacks () at pthread_stop_world.c:585 #3 0x00007ffff5c777df in GC_mark_some ( cold_gc_frame=0x7fffe0ff59d0 "\274\327\354\365\377\177") at mark.c:322 #4 0x00007ffff5c6d15d in GC_stopped_mark ( stop_func=stop_func@entry=0x7ffff5c6cbf0 ) at alloc.c:698 #5 0x00007ffff5c6dc69 in GC_try_to_collect_inner ( stop_func=0x7ffff5c6cbf0 ) at alloc.c:486 #6 0x00007ffff5c6deea in GC_try_to_collect_general ( stop_func=stop_func@entry=0x0, force_unmap=force_unmap@entry=0) at alloc.c:1065 #7 0x00007ffff5c6dfbd in GC_gcollect () at alloc.c:1089 #8 0x00007ffff6df3e5e in scm_gc_register_allocation (size=size@entry=136) at ../../libguile/gc.c:596 #9 0x00007ffff6df3554 in do_realloc (new_size=136, from=0x0) at ../../libguile/gc-malloc.c:70 #10 scm_realloc (mem=0x0, size=136) at ../../libguile/gc-malloc.c:117 #11 0x00007ffff630431f in _nettle_gmp_alloc () from /usr/lib/x86_64-linux-gnu/libhogweed.so.4 #12 0x00007ffff62fc968 in nettle_mpz_random_size () from /usr/lib/x86_64-linux-gnu/libhogweed.so.4 #13 0x00007ffff62fc9f4 in nettle_mpz_random () from /usr/lib/x86_64-linux-gnu/libhogweed.so.4 #14 0x00007ffff62fcd63 in _nettle_generate_pocklington_prime () from /usr/lib/x86_64-linux-gnu/libhogweed.so.4 #15 0x00007ffff62fd2ce in nettle_random_prime () from /usr/lib/x86_64-linux-gnu/libhogweed.so.4 #16 0x00007ffff6300b53 in nettle_rsa_generate_keypair () from /usr/lib/x86_64-linux-gnu/libhogweed.so.4 #17 0x00007ffff7e0729e in ?? () from /usr/lib/x86_64-linux-gnu/libgnutls.so.30 #18 0x00007ffff7da8f07 in gnutls_x509_privkey_generate2 () from /usr/lib/x86_64-linux-gnu/libgnutls.so.30 #19 0x00007ffff7f16990 in dht::crypto::PrivateKey::generate(unsigned int) () from /home/linas/src/novamente/src/atomspace-dht/build/opencog/persist/dht/libpersist-dht.so Next, verify that nettle does not use scm: $ nm /usr/lib/x86_64-linux-gnu/libhogweed.a |grep scm (nothing printed) $ nm /usr/lib/x86_64-linux-gnu/libhogweed.a |grep GC (nothing printed) $ nm /usr/lib/x86_64-linux-gnu/libhogweed.a |grep alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc U _nettle_gmp_alloc 00000000000005f0 T _nettle_gmp_alloc 00000000000004e0 T _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs U _nettle_gmp_alloc_limbs Debugging suggestions? -- cassette tapes - analog TV - film cameras - you --00000000000074aee5059666dfcb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I've got an app that links gnutls (for crypto cod= e) which links nettle (libhogweed) with is a GMP-using crypto library which= seems like it wanted to call plain-old realloc, and ended up calling scm_r= ealloc instead.=C2=A0 Note that nettle does NOT use guile, so there's n= o plausible way that I know of to end up in guile code.=C2=A0 This only see= ms to happen when nettle is used from multiple threads (so is maybe a nettl= e bug??) but the stack trace is so bizarre, I thought I'd report it her= e.

It would seem that someone, somewhere, is doing= some low-level thunking or trampolining of realloc().=C2=A0 First, the cra= zy stack trace:

It's currently highly repr= oducible and exact:
(gdb) r
Starting program: /home/linas/src/= novamente/src/atomspace-dht/build/tests/persist/dht/MultiUserUTest
[Thr= ead debugging using libthread_db enabled]
Using host libthread_db librar= y "/lib/x86_64-linux-gnu/libthread_db.so.1".
Running cxxtest t= ests (2 tests)Start creating 6 user sessions
[2019-11-03 00:46:03:350] [= DEBUG] BEGIN TEST: test_multiuser
Collecting from unknown thread

= Thread 13 "MultiUserUTest" received signal SIGABRT, Aborted.
[= Switching to Thread 0x7fffe0ff9700 (LWP 3844)]
__GI_raise (sig=3Dsig@ent= ry=3D6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 ../sysdeps/unix/sysv= /linux/raise.c: No such file or directory.
(gdb) bt
#0 =C2=A0__GI_rai= se (sig=3Dsig@entry=3D6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 =C2= =A00x00007ffff6834535 in __GI_abort () at abort.c:79
#2 =C2=A00x00007fff= f5c80ded in GC_push_all_stacks () at pthread_stop_world.c:585
#3 =C2=A00= x00007ffff5c777df in GC_mark_some (
=C2=A0 =C2=A0 cold_gc_frame=3D0x7fff= e0ff59d0 "\274\327\354\365\377\177") at mark.c:322
#4 =C2=A00x= 00007ffff5c6d15d in GC_stopped_mark (
=C2=A0 =C2=A0 stop_func=3Dstop_fun= c@entry=3D0x7ffff5c6cbf0 <GC_never_stop_func>)
=C2=A0 =C2=A0 at al= loc.c:698
#5 =C2=A00x00007ffff5c6dc69 in GC_try_to_collect_inner (
= =C2=A0 =C2=A0 stop_func=3D0x7ffff5c6cbf0 <GC_never_stop_func>) at all= oc.c:486
#6 =C2=A00x00007ffff5c6deea in GC_try_to_collect_general (
= =C2=A0 =C2=A0 stop_func=3Dstop_func@entry=3D0x0, force_unmap=3Dforce_unmap@= entry=3D0)
=C2=A0 =C2=A0 at alloc.c:1065
#7 =C2=A00x00007ffff5c6dfbd = in GC_gcollect () at alloc.c:1089
#8 =C2=A00x00007ffff6df3e5e in scm_gc_= register_allocation (size=3Dsize@entry=3D136)
=C2=A0 =C2=A0 at ../../lib= guile/gc.c:596
#9 =C2=A00x00007ffff6df3554 in do_realloc (new_size=3D136= , from=3D0x0)
=C2=A0 =C2=A0 at ../../libguile/gc-malloc.c:70
#10 scm_= realloc (mem=3D0x0, size=3D136) at ../../libguile/gc-malloc.c:117
#11 0x= 00007ffff630431f in _nettle_gmp_alloc ()
=C2=A0 =C2=A0from /usr/lib/x86_= 64-linux-gnu/libhogweed.so.4
#12 0x00007ffff62fc968 in nettle_mpz_random= _size ()
=C2=A0 =C2=A0from /usr/lib/x86_64-linux-gnu/libhogweed.so.4
= #13 0x00007ffff62fc9f4 in nettle_mpz_random ()
=C2=A0 =C2=A0from /usr/li= b/x86_64-linux-gnu/libhogweed.so.4
#14 0x00007ffff62fcd63 in _nettle_gen= erate_pocklington_prime ()
=C2=A0 =C2=A0from /usr/lib/x86_64-linux-gnu/l= ibhogweed.so.4
#15 0x00007ffff62fd2ce in nettle_random_prime ()
=C2= =A0 =C2=A0from /usr/lib/x86_64-linux-gnu/libhogweed.so.4
#16 0x00007ffff= 6300b53 in nettle_rsa_generate_keypair ()
=C2=A0 =C2=A0from /usr/lib/x86= _64-linux-gnu/libhogweed.so.4
#17 0x00007ffff7e0729e in ?? () from /usr/= lib/x86_64-linux-gnu/libgnutls.so.30
#18 0x00007ffff7da8f07 in gnutls_x5= 09_privkey_generate2 ()
=C2=A0 =C2=A0from /usr/lib/x86_64-linux-gnu/libg= nutls.so.30
#19 0x00007ffff7f16990 in dht::crypto::PrivateKey::generate(= unsigned int) ()
=C2=A0 =C2=A0from /home/linas/src/novamente/src/atomspa= ce-dht/build/opencog/persist/dht/libpersist-dht.so

=
Next, verify that nettle does not use scm:

$ = nm /usr/lib/x86_64-linux-gnu/libhogweed.a |grep scm
(nothing = printed)
$ nm /usr/lib/x86_64-linux-gnu/libhogweed.a |grep GC
(nothing printed)
$ nm /usr/lib/x86_64-linux-gnu/libho= gweed.a |grep alloc
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0U _nettle_gmp_alloc
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp= _alloc
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _= nettle_gmp_alloc
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0U _nettle_gmp_alloc
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0U _nettle_gmp_alloc
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_al= loc
00000000000005f0 T _nettle_gmp_alloc
00000000000004e0 T _nettle_g= mp_alloc_limbs
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0U _nettle_gmp_alloc_limbs
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc_limbs
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc_limbs
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_al= loc_limbs
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= U _nettle_gmp_alloc_limbs
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0U _nettle_gmp_alloc_limbs
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc_limbs
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nettle_gmp_alloc_lim= bs
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0U _nett= le_gmp_alloc_limbs

Debugging suggestions?
--
cassette tapes - analog TV - film cameras = - you
--00000000000074aee5059666dfcb--