From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 17 05:11:08 2017 Received: (at submit) by debbugs.gnu.org; 17 Nov 2017 10:11:08 +0000 Received: from localhost ([127.0.0.1]:44298 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFdc4-0000mJ-6C for submit@debbugs.gnu.org; Fri, 17 Nov 2017 05:11:08 -0500 Received: from eggs.gnu.org ([208.118.235.92]:52797) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFdc2-0000lF-RF for submit@debbugs.gnu.org; Fri, 17 Nov 2017 05:11:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eFdbw-0003fi-M9 for submit@debbugs.gnu.org; Fri, 17 Nov 2017 05:11:01 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:55797) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eFdbw-0003fN-Is for submit@debbugs.gnu.org; Fri, 17 Nov 2017 05:11:00 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43766) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eFdbv-0001U5-5K for bug-guix@gnu.org; Fri, 17 Nov 2017 05:11:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eFdbp-0003NG-8S for bug-guix@gnu.org; Fri, 17 Nov 2017 05:10:59 -0500 Received: from hera.aquilenet.fr ([141.255.128.1]:58220) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eFdbo-0003KZ-UI for bug-guix@gnu.org; Fri, 17 Nov 2017 05:10:53 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id BC231F8D0 for ; Fri, 17 Nov 2017 11:10:52 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2OQnBLMQBc_4 for ; Fri, 17 Nov 2017 11:10:52 +0100 (CET) Received: from ribbon (unknown [193.50.110.175]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 0361EF8CD for ; Fri, 17 Nov 2017 11:10:51 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: bug-guix@gnu.org Subject: 'guix publish' workers occasionally crash X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 27 Brumaire an 226 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 17 Nov 2017 11:10:49 +0100 Message-ID: <878tf55i6u.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hello, On berlin I=E2=80=99ve noticed that the =E2=80=98guix publish=E2=80=99 work= ers would occasionally stop working: the main thread would keep replying to HTTP requests, but the worker threads would no longer do anything, and would leave behind them a bunch of .tmp files in /var/cache/guix/publish. I captured the output of =E2=80=98guix publish=E2=80=99 (guix-0.13.0-8.357a= b93) and the only clue I have is this: --8<---------------cut here---------------start------------->8--- GET /6kl9ydqmgklcqhxswg6v5isq5n1ih5gp.narinfo In guix/workers.scm: 74:9 2 (_) 78:32 1 (_ srfi-34 #) In unknown file: 0 (make-stack #t) ERROR: In procedure make-stack: ERROR: Throw to key `srfi-34' with args `(#)'. GET /fgiih42mg2sr82mbmzf56grvrf021im6.narinfo --8<---------------cut here---------------end--------------->8--- =E2=80=A6 where 9 is EBADF. The open files are: --8<---------------cut here---------------start------------->8--- ludo@berlin ~$ sudo ls -l /proc/4105/fd total 0 l-wx------ 1 root root 64 Nov 17 04:00 0 -> /dev/null l-wx------ 1 root root 64 Nov 17 04:00 1 -> /home/ludo/nohup.out lr-x------ 1 root root 64 Nov 17 04:00 10 -> pipe:[9911558] l-wx------ 1 root root 64 Nov 17 04:00 11 -> pipe:[9911558] lr-x------ 1 root root 64 Nov 17 04:00 12 -> pipe:[9900923] l-wx------ 1 root root 64 Nov 17 04:00 13 -> pipe:[9900923] lrwx------ 1 root root 64 Nov 17 04:00 14 -> socket:[9911560] lrwx------ 1 root root 64 Nov 17 04:00 15 -> socket:[9911563] lr-x------ 1 root root 64 Nov 17 04:00 16 -> pipe:[9912323] l-wx------ 1 root root 64 Nov 17 04:00 17 -> pipe:[9912323] lr-x------ 1 root root 64 Nov 17 04:00 18 -> pipe:[9909649] l-wx------ 1 root root 64 Nov 17 04:00 19 -> pipe:[9909649] l-wx------ 1 root root 64 Nov 17 04:00 2 -> /home/ludo/nohup.out lr-x------ 1 root root 64 Nov 17 04:00 20 -> pipe:[9908165] l-wx------ 1 root root 64 Nov 17 04:00 21 -> pipe:[9908165] lr-x------ 1 root root 64 Nov 17 04:00 22 -> pipe:[9899432] l-wx------ 1 root root 64 Nov 17 04:00 23 -> pipe:[9899432] lr-x------ 1 root root 64 Nov 17 04:00 24 -> pipe:[9911564] l-wx------ 1 root root 64 Nov 17 04:00 25 -> pipe:[9911564] lr-x------ 1 root root 64 Nov 17 04:00 26 -> pipe:[9908166] l-wx------ 1 root root 64 Nov 17 04:00 27 -> pipe:[9908166] l-wx------ 1 root root 64 Nov 17 04:00 28 -> /var/cache/guix/publish/gzip/1= 1vik22bwzzvwksd1l08frhcm3v3hhmm-dealii-8.5.1.nar.tmp l-wx------ 1 root root 64 Nov 17 04:00 29 -> /var/cache/guix/publish/gzip/k= 90h1bkmsiq8qxrhk31l5c8maiqmy5k0-zathura-pdf-mupdf-0.3.1.nar.tmp lr-x------ 1 root root 64 Nov 17 04:00 3 -> pipe:[9902378] lr-x------ 1 root root 64 Nov 17 04:00 30 -> /gnu/store/2xp8n6k6mbfgi8972lj= 0vvk52gfx4576-linux-libre-4.9.61/lib/modules/4.9.61-gnu/kernel/drivers/net/= ethernet/intel/i40evf lr-x------ 1 root root 64 Nov 17 04:00 31 -> /gnu/store/k90h1bkmsiq8qxrhk31= l5c8maiqmy5k0-zathura-pdf-mupdf-0.3.1/share l-wx------ 1 root root 64 Nov 17 04:00 32 -> /var/cache/guix/publish/gzip/x= a6bwlgm82xwfg9icl4d9bs2bhj8cb1a-perl-safe-isa-1.000008.nar.tmp l-wx------ 1 root root 64 Nov 17 04:00 33 -> /var/cache/guix/publish/gzip/2= xp8n6k6mbfgi8972lj0vvk52gfx4576-linux-libre-4.9.61.nar.tmp lr-x------ 1 root root 64 Nov 17 04:00 34 -> /gnu/store/11vik22bwzzvwksd1l0= 8frhcm3v3hhmm-dealii-8.5.1/share l-wx------ 1 root root 64 Nov 17 04:00 4 -> pipe:[9902378] lr-x------ 1 root root 64 Nov 17 04:00 5 -> pipe:[9902379] l-wx------ 1 root root 64 Nov 17 04:00 6 -> pipe:[9902379] lr-x------ 1 root root 64 Nov 17 04:00 7 -> /gnu/store/sk1w5whws8xhvb03sm2h= 44f9fm6c2gkc-guix-0.13.0-8.357ab93/bin/.guix-real lr-x------ 1 root root 64 Nov 17 04:00 8 -> pipe:[9910685] l-wx------ 1 root root 64 Nov 17 04:00 9 -> pipe:[9910685] --8<---------------cut here---------------end--------------->8--- Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 17 07:23:04 2017 Received: (at 29335) by debbugs.gnu.org; 17 Nov 2017 12:23:04 +0000 Received: from localhost ([127.0.0.1]:44370 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFffj-0003xc-Kb for submit@debbugs.gnu.org; Fri, 17 Nov 2017 07:23:04 -0500 Received: from hera.aquilenet.fr ([141.255.128.1]:48412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFffh-0003x0-J9 for 29335@debbugs.gnu.org; Fri, 17 Nov 2017 07:23:03 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 84980F912 for <29335@debbugs.gnu.org>; Fri, 17 Nov 2017 13:23:02 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O3F1vpbRaSFI for <29335@debbugs.gnu.org>; Fri, 17 Nov 2017 13:23:00 +0100 (CET) Received: from ribbon (unknown [193.50.110.175]) by hera.aquilenet.fr (Postfix) with ESMTPSA id AD2C9F910 for <29335@debbugs.gnu.org>; Fri, 17 Nov 2017 13:23:00 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: 29335@debbugs.gnu.org Subject: Re: bug#29335: 'guix publish' workers occasionally crash References: <878tf55i6u.fsf@gnu.org> Date: Fri, 17 Nov 2017 13:22:58 +0100 In-Reply-To: <878tf55i6u.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Fri, 17 Nov 2017 11:10:49 +0100") Message-ID: <87vai93xi5.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 29335 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) If we look at our worker threads, they=E2=80=99re all stuck in mutex_lock: --8<---------------cut here---------------start------------->8--- (gdb) info threads Id Target Id Frame * 1 Thread 0x7f7ee99d5f00 (LWP 4105) "guix publish" 0x00007f7ee79c60ad i= n poll () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/l= ibc.so.6 2 Thread 0x7f7ee78e6700 (LWP 4106) ".guix-real" 0x00007f7ee900d58f in = pthread_cond_wait@@GLIBC_2.3.2 () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5s= ilnakh6-glibc-2.25/lib/libpthread.so.0 3 Thread 0x7f7ee70e5700 (LWP 4107) ".guix-real" 0x00007f7ee900d58f in = pthread_cond_wait@@GLIBC_2.3.2 () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5s= ilnakh6-glibc-2.25/lib/libpthread.so.0 4 Thread 0x7f7ee68e4700 (LWP 4108) ".guix-real" 0x00007f7ee900d58f in = pthread_cond_wait@@GLIBC_2.3.2 () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5s= ilnakh6-glibc-2.25/lib/libpthread.so.0 5 Thread 0x7f7ee60e3700 (LWP 4109) ".guix-real" 0x00007f7ee900d58f in = pthread_cond_wait@@GLIBC_2.3.2 () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5s= ilnakh6-glibc-2.25/lib/libpthread.so.0 6 Thread 0x7f7ee58e2700 (LWP 4110) ".guix-real" 0x00007f7ee900d58f in = pthread_cond_wait@@GLIBC_2.3.2 () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5s= ilnakh6-glibc-2.25/lib/libpthread.so.0 7 Thread 0x7f7ee50e1700 (LWP 4111) ".guix-real" 0x00007f7ee900d58f in = pthread_cond_wait@@GLIBC_2.3.2 () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5s= ilnakh6-glibc-2.25/lib/libpthread.so.0 8 Thread 0x7f7ee48e0700 (LWP 4112) ".guix-real" 0x00007f7ee900d58f in = pthread_cond_wait@@GLIBC_2.3.2 () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5s= ilnakh6-glibc-2.25/lib/libpthread.so.0 9 Thread 0x7f7ee3a4f700 (LWP 4113) ".guix-real" 0x00007f7ee9010aed in = read () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/lib= pthread.so.0 10 Thread 0x7f7ee114b700 (LWP 4114) ".guix-real" 0x00007f7ee9010aed in = read () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/lib= pthread.so.0 11 Thread 0x7f7edf892700 (LWP 4116) "publish worker" 0x00007f7ee90108fc= in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-gli= bc-2.25/lib/libpthread.so.0 12 Thread 0x7f7edf090700 (LWP 4117) "publish worker" 0x00007f7ee90108fc= in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-gli= bc-2.25/lib/libpthread.so.0 13 Thread 0x7f7ede88e700 (LWP 4118) "publish worker" 0x00007f7ee90108fc= in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-gli= bc-2.25/lib/libpthread.so.0 14 Thread 0x7f7ede08c700 (LWP 4119) "publish worker" 0x00007f7ee90108fc= in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-gli= bc-2.25/lib/libpthread.so.0 15 Thread 0x7f7edd88a700 (LWP 4120) "publish worker" 0x00007f7ee90108fc= in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-gli= bc-2.25/lib/libpthread.so.0 16 Thread 0x7f7edd088700 (LWP 4121) "publish worker" 0x00007f7ee90108fc= in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-gli= bc-2.25/lib/libpthread.so.0 (gdb) thread apply 11-16 bt Thread 11 (Thread 0x7f7edf892700 (LWP 4116)): #0 0x00007f7ee90108fc in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysff= jh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #1 0x00007f7ee9009b05 in pthread_mutex_lock () from /gnu/store/n6nvxlk2j8y= sffjh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #2 0x00007f7ee9545815 in scm_pthread_mutex_lock (mutex=3D) = at threads.c:1600 #3 0x00007f7ee9545969 in scm_dynwind_pthread_mutex_lock (mutex=3D0x7f7ee97= bbfa0 ) at threads.c:1613 #4 0x00007f7ee94e2927 in scm_readdir (port=3D#) at fi= lesys.c:1749 #5 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bea0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #6 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #7 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #8 0x00007f7ee94cbef8 in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:400 #9 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bea0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #10 0x00007f7ee95572aa in scm_call_n (proc=3D#, argv= =3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #11 0x00007f7ee94da7d9 in scm_call_0 (proc=3D) at eval.c:481 #12 0x00007f7ee9544fe6 in really_launch (d=3D0x266ab80) at threads.c:793 #13 0x00007f7ee94d4cba in c_body (d=3D0x7f7edf891e60) at continuations.c:422 #14 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bea0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #15 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #16 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #17 0x00007f7ee95464e6 in catch (tag=3Dtag@entry=3D#t, thunk=3D#, handler=3D#, pre_unwind_handler=3D#) at throw.c:137 #18 0x00007f7ee9546825 in scm_catch_with_pre_unwind_handler (key=3Dkey@entr= y=3D#t, thunk=3D, handler=3D, pre_unwind_hand= ler=3D) at throw.c:254 #19 0x00007f7ee95469df in scm_c_catch (tag=3Dtag@entry=3D#t, body=3Dbody@en= try=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7edf891e60= , handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7edf891e60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at throw.c:377 #20 0x00007f7ee94d52a0 in scm_i_with_continuation_barrier (body=3Dbody@entr= y=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7edf891e60, = handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7edf891e60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at continuations.c:360 #21 0x00007f7ee94d5335 in scm_c_with_continuation_barrier (func=3D, data=3D) at continuations.c:456 #22 0x00007f7ee954510c in with_guile (base=3D0x7f7edf891ec0, data=3D0x7f7ed= f891ef0) at threads.c:660 #23 0x00007f7ee923a732 in GC_call_with_stack_base () from /gnu/store/9blidf= 28xz0a0glnv6a4xcw6nxadg427-libgc-7.6.0/lib/libgc.so.1 #24 0x00007f7ee95447dd in scm_i_with_guile (dynamic_state=3D= , data=3D0x266ab80, func=3D0x7f7ee9544f70 ) at threads.c:703 #25 launch_thread (d=3D0x266ab80) at threads.c:802 #26 0x00007f7ee9007454 in start_thread () from /gnu/store/n6nvxlk2j8ysffjh3= jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #27 0x00007f7ee79cf7bf in clone () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5= silnakh6-glibc-2.25/lib/libc.so.6 Thread 12 (Thread 0x7f7edf090700 (LWP 4117)): #0 0x00007f7ee90108fc in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysff= jh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #1 0x00007f7ee9009b05 in pthread_mutex_lock () from /gnu/store/n6nvxlk2j8y= sffjh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #2 0x00007f7ee9545815 in scm_pthread_mutex_lock (mutex=3D) = at threads.c:1600 #3 0x00007f7ee9545969 in scm_dynwind_pthread_mutex_lock (mutex=3D0x7f7ee97= bbfa0 ) at threads.c:1613 #4 0x00007f7ee94d8dbb in scm_strerror (err=3Derr@entry=3D17) at error.c:128 #5 0x00007f7ee94d8e73 in scm_syserror (subr=3Dsubr@entry=3D0x7f7ee956b07a = "mkdir") at error.c:169 #6 0x00007f7ee94e2189 in scm_mkdir (path=3D, mode=3D) at filesys.c:1274 #7 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211be10, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #8 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #9 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #10 0x00007f7ee94cbef8 in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:400 #11 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211be10, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #12 0x00007f7ee95572aa in scm_call_n (proc=3D#, argv= =3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #13 0x00007f7ee94da7d9 in scm_call_0 (proc=3D) at eval.c:481 #14 0x00007f7ee9544fe6 in really_launch (d=3D0x266ab40) at threads.c:793 #15 0x00007f7ee94d4cba in c_body (d=3D0x7f7edf08fe60) at continuations.c:422 #16 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211be10, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #17 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #18 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #19 0x00007f7ee95464e6 in catch (tag=3Dtag@entry=3D#t, thunk=3D#, handler=3D#, pre_unwind_handler=3D#) at throw.c:137 #20 0x00007f7ee9546825 in scm_catch_with_pre_unwind_handler (key=3Dkey@entr= y=3D#t, thunk=3D, handler=3D, pre_unwind_hand= ler=3D) at throw.c:254 #21 0x00007f7ee95469df in scm_c_catch (tag=3Dtag@entry=3D#t, body=3Dbody@en= try=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7edf08fe60= , handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7edf08fe60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at throw.c:377 #22 0x00007f7ee94d52a0 in scm_i_with_continuation_barrier (body=3Dbody@entr= y=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7edf08fe60, = handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7edf08fe60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at continuations.c:360 #23 0x00007f7ee94d5335 in scm_c_with_continuation_barrier (func=3D, data=3D) at continuations.c:456 #24 0x00007f7ee954510c in with_guile (base=3D0x7f7edf08fec0, data=3D0x7f7ed= f08fef0) at threads.c:660 #25 0x00007f7ee923a732 in GC_call_with_stack_base () from /gnu/store/9blidf= 28xz0a0glnv6a4xcw6nxadg427-libgc-7.6.0/lib/libgc.so.1 #26 0x00007f7ee95447dd in scm_i_with_guile (dynamic_state=3D= , data=3D0x266ab40, func=3D0x7f7ee9544f70 ) at threads.c:703 #27 launch_thread (d=3D0x266ab40) at threads.c:802 #28 0x00007f7ee9007454 in start_thread () from /gnu/store/n6nvxlk2j8ysffjh3= jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #29 0x00007f7ee79cf7bf in clone () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5= silnakh6-glibc-2.25/lib/libc.so.6 Thread 13 (Thread 0x7f7ede88e700 (LWP 4118)): #0 0x00007f7ee90108fc in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysff= jh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #1 0x00007f7ee9009b05 in pthread_mutex_lock () from /gnu/store/n6nvxlk2j8y= sffjh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #2 0x00007f7ee9545815 in scm_pthread_mutex_lock (mutex=3D) = at threads.c:1600 #3 0x00007f7ee9545969 in scm_dynwind_pthread_mutex_lock (mutex=3D0x7f7ee97= bbfa0 ) at threads.c:1613 #4 0x00007f7ee94e2927 in scm_readdir (port=3D#) at fi= lesys.c:1749 #5 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bd80, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #6 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #7 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #8 0x00007f7ee94cbef8 in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:400 #9 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bd80, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #10 0x00007f7ee95572aa in scm_call_n (proc=3D#, argv= =3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #11 0x00007f7ee94da7d9 in scm_call_0 (proc=3D) at eval.c:481 #12 0x00007f7ee9544fe6 in really_launch (d=3D0x266ab00) at threads.c:793 #13 0x00007f7ee94d4cba in c_body (d=3D0x7f7ede88de60) at continuations.c:422 #14 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bd80, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #15 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #16 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #17 0x00007f7ee95464e6 in catch (tag=3Dtag@entry=3D#t, thunk=3D#, handler=3D#, pre_unwind_handler=3D#) at throw.c:137 #18 0x00007f7ee9546825 in scm_catch_with_pre_unwind_handler (key=3Dkey@entr= y=3D#t, thunk=3D, handler=3D, pre_unwind_hand= ler=3D) at throw.c:254 #19 0x00007f7ee95469df in scm_c_catch (tag=3Dtag@entry=3D#t, body=3Dbody@en= try=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7ede88de60= , handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7ede88de60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at throw.c:377 #20 0x00007f7ee94d52a0 in scm_i_with_continuation_barrier (body=3Dbody@entr= y=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7ede88de60, = handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7ede88de60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at continuations.c:360 #21 0x00007f7ee94d5335 in scm_c_with_continuation_barrier (func=3D, data=3D) at continuations.c:456 #22 0x00007f7ee954510c in with_guile (base=3D0x7f7ede88dec0, data=3D0x7f7ed= e88def0) at threads.c:660 #23 0x00007f7ee923a732 in GC_call_with_stack_base () from /gnu/store/9blidf= 28xz0a0glnv6a4xcw6nxadg427-libgc-7.6.0/lib/libgc.so.1 #24 0x00007f7ee95447dd in scm_i_with_guile (dynamic_state=3D= , data=3D0x266ab00, func=3D0x7f7ee9544f70 ) at threads.c:703 #25 launch_thread (d=3D0x266ab00) at threads.c:802 #26 0x00007f7ee9007454 in start_thread () from /gnu/store/n6nvxlk2j8ysffjh3= jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #27 0x00007f7ee79cf7bf in clone () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5= silnakh6-glibc-2.25/lib/libc.so.6 Thread 14 (Thread 0x7f7ede08c700 (LWP 4119)): #0 0x00007f7ee90108fc in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysff= jh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #1 0x00007f7ee9009b05 in pthread_mutex_lock () from /gnu/store/n6nvxlk2j8y= sffjh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #2 0x00007f7ee9545815 in scm_pthread_mutex_lock (mutex=3D) = at threads.c:1600 #3 0x00007f7ee9545969 in scm_dynwind_pthread_mutex_lock (mutex=3D0x7f7ee97= bbfa0 ) at threads.c:1613 #4 0x00007f7ee94d8dbb in scm_strerror (err=3Derr@entry=3D9) at error.c:128 #5 0x00007f7ee94d8e73 in scm_syserror (subr=3Dsubr@entry=3D0x7f7ee956afa0 = "readdir") at error.c:169 #6 0x00007f7ee94e29b4 in scm_readdir (port=3D#) at fi= lesys.c:1754 #7 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bcf0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #8 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #9 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #10 0x00007f7ee94cbef8 in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:400 #11 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bcf0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #12 0x00007f7ee95572aa in scm_call_n (proc=3D#, argv= =3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #13 0x00007f7ee94da7d9 in scm_call_0 (proc=3D) at eval.c:481 #14 0x00007f7ee9544fe6 in really_launch (d=3D0x266aac0) at threads.c:793 #15 0x00007f7ee94d4cba in c_body (d=3D0x7f7ede08be60) at continuations.c:422 #16 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bcf0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #17 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #18 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #19 0x00007f7ee95464e6 in catch (tag=3Dtag@entry=3D#t, thunk=3D#, handler=3D#, pre_unwind_handler=3D#) at throw.c:137 #20 0x00007f7ee9546825 in scm_catch_with_pre_unwind_handler (key=3Dkey@entr= y=3D#t, thunk=3D, handler=3D, pre_unwind_hand= ler=3D) at throw.c:254 #21 0x00007f7ee95469df in scm_c_catch (tag=3Dtag@entry=3D#t, body=3Dbody@en= try=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7ede08be60= , handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7ede08be60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at throw.c:377 #22 0x00007f7ee94d52a0 in scm_i_with_continuation_barrier (body=3Dbody@entr= y=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7ede08be60, = handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7ede08be60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at continuations.c:360 #23 0x00007f7ee94d5335 in scm_c_with_continuation_barrier (func=3D, data=3D) at continuations.c:456 #24 0x00007f7ee954510c in with_guile (base=3D0x7f7ede08bec0, data=3D0x7f7ed= e08bef0) at threads.c:660 #25 0x00007f7ee923a732 in GC_call_with_stack_base () from /gnu/store/9blidf= 28xz0a0glnv6a4xcw6nxadg427-libgc-7.6.0/lib/libgc.so.1 #26 0x00007f7ee95447dd in scm_i_with_guile (dynamic_state=3D= , data=3D0x266aac0, func=3D0x7f7ee9544f70 ) at threads.c:703 #27 launch_thread (d=3D0x266aac0) at threads.c:802 #28 0x00007f7ee9007454 in start_thread () from /gnu/store/n6nvxlk2j8ysffjh3= jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #29 0x00007f7ee79cf7bf in clone () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5= silnakh6-glibc-2.25/lib/libc.so.6 Thread 15 (Thread 0x7f7edd88a700 (LWP 4120)): #0 0x00007f7ee90108fc in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysff= jh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #1 0x00007f7ee9009b05 in pthread_mutex_lock () from /gnu/store/n6nvxlk2j8y= sffjh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #2 0x00007f7ee9545815 in scm_pthread_mutex_lock (mutex=3D) = at threads.c:1600 #3 0x00007f7ee9545969 in scm_dynwind_pthread_mutex_lock (mutex=3D0x7f7ee97= bbfa0 ) at threads.c:1613 #4 0x00007f7ee94e2927 in scm_readdir (port=3D#) at fi= lesys.c:1749 #5 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bc60, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #6 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #7 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #8 0x00007f7ee94cbef8 in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:400 #9 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bc60, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #10 0x00007f7ee95572aa in scm_call_n (proc=3D#, argv= =3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #11 0x00007f7ee94da7d9 in scm_call_0 (proc=3D) at eval.c:481 ---Type to continue, or q to quit--- #12 0x00007f7ee9544fe6 in really_launch (d=3D0x266aa80) at threads.c:793 #13 0x00007f7ee94d4cba in c_body (d=3D0x7f7edd889e60) at continuations.c:422 #14 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bc60, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #15 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #16 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #17 0x00007f7ee95464e6 in catch (tag=3Dtag@entry=3D#t, thunk=3D#, handler=3D#, pre_unwind_handler=3D#) at throw.c:137 #18 0x00007f7ee9546825 in scm_catch_with_pre_unwind_handler (key=3Dkey@entr= y=3D#t, thunk=3D, handler=3D, pre_unwind_hand= ler=3D) at throw.c:254 #19 0x00007f7ee95469df in scm_c_catch (tag=3Dtag@entry=3D#t, body=3Dbody@en= try=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7edd889e60= , handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7edd889e60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at throw.c:377 #20 0x00007f7ee94d52a0 in scm_i_with_continuation_barrier (body=3Dbody@entr= y=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7edd889e60, = handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7edd889e60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at continuations.c:360 #21 0x00007f7ee94d5335 in scm_c_with_continuation_barrier (func=3D, data=3D) at continuations.c:456 #22 0x00007f7ee954510c in with_guile (base=3D0x7f7edd889ec0, data=3D0x7f7ed= d889ef0) at threads.c:660 #23 0x00007f7ee923a732 in GC_call_with_stack_base () from /gnu/store/9blidf= 28xz0a0glnv6a4xcw6nxadg427-libgc-7.6.0/lib/libgc.so.1 #24 0x00007f7ee95447dd in scm_i_with_guile (dynamic_state=3D= , data=3D0x266aa80, func=3D0x7f7ee9544f70 ) at threads.c:703 #25 launch_thread (d=3D0x266aa80) at threads.c:802 #26 0x00007f7ee9007454 in start_thread () from /gnu/store/n6nvxlk2j8ysffjh3= jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #27 0x00007f7ee79cf7bf in clone () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5= silnakh6-glibc-2.25/lib/libc.so.6 Thread 16 (Thread 0x7f7edd088700 (LWP 4121)): #0 0x00007f7ee90108fc in __lll_lock_wait () from /gnu/store/n6nvxlk2j8ysff= jh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #1 0x00007f7ee9009b05 in pthread_mutex_lock () from /gnu/store/n6nvxlk2j8y= sffjh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #2 0x00007f7ee9545815 in scm_pthread_mutex_lock (mutex=3D) = at threads.c:1600 #3 0x00007f7ee9545969 in scm_dynwind_pthread_mutex_lock (mutex=3D0x7f7ee97= bbfa0 ) at threads.c:1613 #4 0x00007f7ee94d8dbb in scm_strerror (err=3Derr@entry=3D9) at error.c:128 #5 0x00007f7ee94d8e73 in scm_syserror (subr=3Dsubr@entry=3D0x7f7ee956bbd9 = "fport_close") at error.c:169 #6 0x00007f7ee94e7a85 in fport_close (port=3D) at fports.c:= 677 #7 0x00007f7ee9514785 in release_port (port=3D#) at ports.= c:158 #8 0x00007f7ee9517a0b in scm_close_port (port=3D#) at port= s.c:887 #9 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bbd0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #10 0x00007f7ee95572aa in scm_call_n (proc=3D#, argv= =3D, nargs=3D0) at vm.c:1257 #11 0x00007f7ee9514785 in release_port (port=3D#) at ports.= c:158 #12 0x00007f7ee9517a0b in scm_close_port (port=3D#) at port= s.c:887 #13 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bbd0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #14 0x00007f7ee95572aa in scm_call_n (proc=3D#, argv= =3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #15 0x00007f7ee94da7d9 in scm_call_0 (proc=3D) at eval.c:481 #16 0x00007f7ee94d7ba0 in scm_dynstack_unwind_1 (dynstack=3Ddynstack@entry= =3D0x1fd9088) at dynstack.c:407 #17 0x00007f7ee94d7cd0 in scm_dynstack_unwind (dynstack=3Ddynstack@entry=3D= 0x1fd9088, base=3D0x29e1720) at dynstack.c:437 #18 0x00007f7ee94d4b02 in scm_c_abort (vp=3D0x211bbd0, tag=3D, n=3D5, argv=3D0x7f7edd087200, current_registers=3D) at co= ntrol.c:162 #19 0x00007f7ee9548df1 in vm_abort (vp=3D, tag=3D, nargs=3D, current_registers=3D) at vm.c= :340 #20 0x00007f7ee9553cc6 in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bbd0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:1004 #21 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3D, nargs=3D5) at vm.c:1257 #22 0x00007f7ee94dab6b in scm_apply_0 (proc=3D#, args= =3D()) at eval.c:594 #23 0x00007f7ee94db76d in scm_apply_1 (proc=3D, arg1=3Darg1@= entry=3Dsystem-error, args=3D) at eval.c:600 #24 0x00007f7ee954686c in scm_throw (key=3Dkey@entry=3Dsystem-error, args= =3D) at throw.c:266 #25 0x00007f7ee9546de5 in scm_ithrow (key=3Dkey@entry=3Dsystem-error, args= =3D, no_return=3Dno_return@entry=3D1) at throw.c:611 #26 0x00007f7ee94d8d75 in scm_error_scm (key=3Dkey@entry=3Dsystem-error, su= br=3D, message=3Dmessage@entry=3D"~A", args=3Dargs@entry=3D(= "Invalid argument"), data=3Ddata@entry=3D(22)) at error.c:94 #27 0x00007f7ee94d8e2f in scm_error (key=3Dsystem-error, subr=3Dsubr@entry= =3D0x7f7ee956bbf1 "fport_read", message=3Dmessage@entry=3D0x7f7ee95732db "~= A", args=3D("Invalid argument"), rest=3Drest@entry=3D(22)) at error.c:59 #28 0x00007f7ee94d8eaa in scm_syserror (subr=3Dsubr@entry=3D0x7f7ee956bbf1 = "fport_read") at error.c:169 #29 0x00007f7ee94e7c75 in fport_read (port=3D, dst=3D, start=3D, count=3D58928) at fports.c:614 #30 0x00007f7ee9514d77 in scm_i_read_bytes (port=3Dport@entry=3D#, dst=3Ddst@entry=3D"#" =3D {...}, start=3Dstart@entry=3D0= , count=3Dcount@entry=3D58928) at ports.c:1544 #31 0x00007f7ee95193fc in scm_c_read_bytes (port=3Dport@entry=3D#, dst=3Ddst@entry=3D"#" =3D {...}, start=3Dstart@entry=3D0= , count=3D58928) at ports.c:1624 #32 0x00007f7ee951eb78 in scm_get_bytevector_n_x (port=3D#,= bv=3D"#" =3D {...}, start=3D, count=3D58928) at = r6rs-ports.c:461 #33 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bbd0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #34 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #35 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #36 0x00007f7ee94cbef8 in scm_call_with_unblocked_asyncs (proc=3D#) at async.c:400 #37 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bbd0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #38 0x00007f7ee95572aa in scm_call_n (proc=3D#, argv= =3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #39 0x00007f7ee94da7d9 in scm_call_0 (proc=3D) at eval.c:481 #40 0x00007f7ee9544fe6 in really_launch (d=3D0x266aa40) at threads.c:793 #41 0x00007f7ee94d4cba in c_body (d=3D0x7f7edd087e60) at continuations.c:422 #42 0x00007f7ee9553c4d in vm_regular_engine (thread=3D0x7f7ee97bbfa0 , vp=3D0x211bbd0, registers=3D0x0, resume=3D-385808132) at vm-e= ngine.c:784 #43 0x00007f7ee95572aa in scm_call_n (proc=3Dproc@entry=3D#, argv=3Dargv@entry=3D0x0, nargs=3Dnargs@entry=3D0) at vm.c:1257 #44 0x00007f7ee94da7d9 in scm_call_0 (proc=3Dproc@entry=3D#) at eval.c:481 #45 0x00007f7ee95464e6 in catch (tag=3Dtag@entry=3D#t, thunk=3D#, handler=3D#, pre_unwind_handler=3D#) at throw.c:137 #46 0x00007f7ee9546825 in scm_catch_with_pre_unwind_handler (key=3Dkey@entr= y=3D#t, thunk=3D, handler=3D, pre_unwind_hand= ler=3D) at throw.c:254 #47 0x00007f7ee95469df in scm_c_catch (tag=3Dtag@entry=3D#t, body=3Dbody@en= try=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7edd087e60= , handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7edd087e60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at throw.c:377 #48 0x00007f7ee94d52a0 in scm_i_with_continuation_barrier (body=3Dbody@entr= y=3D0x7f7ee94d4cb0 , body_data=3Dbody_data@entry=3D0x7f7edd087e60, = handler=3Dhandler@entry=3D0x7f7ee94d4f40 , handler_data=3Dhandler_data@entry=3D0x7f7edd087e60, pre_unwind_handler= =3Dpre_unwind_handler@entry=3D0x7f7ee94d4da0 , pre_unwi= nd_handler_data=3D0x2079ce0) at continuations.c:360 #49 0x00007f7ee94d5335 in scm_c_with_continuation_barrier (func=3D, data=3D) at continuations.c:456 #50 0x00007f7ee954510c in with_guile (base=3D0x7f7edd087ec0, data=3D0x7f7ed= d087ef0) at threads.c:660 #51 0x00007f7ee923a732 in GC_call_with_stack_base () from /gnu/store/9blidf= 28xz0a0glnv6a4xcw6nxadg427-libgc-7.6.0/lib/libgc.so.1 #52 0x00007f7ee95447dd in scm_i_with_guile (dynamic_state=3D= , data=3D0x266aa40, func=3D0x7f7ee9544f70 ) at threads.c:703 #53 launch_thread (d=3D0x266aa40) at threads.c:802 #54 0x00007f7ee9007454 in start_thread () from /gnu/store/n6nvxlk2j8ysffjh3= jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 #55 0x00007f7ee79cf7bf in clone () from /gnu/store/n6nvxlk2j8ysffjh3jphn1k5= silnakh6-glibc-2.25/lib/libc.so.6 --8<---------------cut here---------------end--------------->8--- Thread 14 is in scm_readdir, so it holds the terrible =E2=80=98scm_i_misc_mutex=E2=80=99, but calling SCM_SYSERROR there causes i= t to try and lock it again, leading to a deadlock: --8<---------------cut here---------------start------------->8--- #3 0x00007f7ee9545969 in scm_dynwind_pthread_mutex_lock (mutex=3D0x7f7ee97= bbfa0 ) at threads.c:1613 #4 0x00007f7ee94d8dbb in scm_strerror (err=3Derr@entry=3D9) at error.c:128 #5 0x00007f7ee94d8e73 in scm_syserror (subr=3Dsubr@entry=3D0x7f7ee956afa0 = "readdir") at error.c:169 #6 0x00007f7ee94e29b4 in scm_readdir (port=3D#) at fi= lesys.c:1754 --8<---------------cut here---------------end--------------->8--- Thread 16 got EINVAL from read(2), which sounds fishy: --8<---------------cut here---------------start------------->8--- #29 0x00007f7ee94e7c75 in fport_read (port=3D, dst=3D, start=3D, count=3D58928) at fports.c:614 614 in fports.c (gdb) p *fp $1 =3D {fdes =3D 35, revealed =3D 0, options =3D 0} (gdb) shell ls -l /proc/4105/fd/35 ls: cannot access '/proc/4105/fd/35': No such file or directory --8<---------------cut here---------------end--------------->8--- To be continued=E2=80=A6 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 17 17:10:47 2017 Received: (at 29335) by debbugs.gnu.org; 17 Nov 2017 22:10:47 +0000 Received: from localhost ([127.0.0.1]:45410 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFoqT-0004gK-Sv for submit@debbugs.gnu.org; Fri, 17 Nov 2017 17:10:47 -0500 Received: from hera.aquilenet.fr ([141.255.128.1]:49671) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFoqS-0004gC-3y for 29335@debbugs.gnu.org; Fri, 17 Nov 2017 17:10:44 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 5ED9AF43D for <29335@debbugs.gnu.org>; Fri, 17 Nov 2017 23:10:44 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rtRqLUWCUtZt for <29335@debbugs.gnu.org>; Fri, 17 Nov 2017 23:10:43 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 13267E418 for <29335@debbugs.gnu.org>; Fri, 17 Nov 2017 23:10:42 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: 29335@debbugs.gnu.org Subject: Re: bug#29335: 'guix publish' workers occasionally crash References: <878tf55i6u.fsf@gnu.org> <87vai93xi5.fsf@gnu.org> Date: Fri, 17 Nov 2017 23:10:39 +0100 In-Reply-To: <87vai93xi5.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Fri, 17 Nov 2017 13:22:58 +0100") Message-ID: <87vai8zhcw.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 29335 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > Thread 14 is in scm_readdir, so it holds the terrible > =E2=80=98scm_i_misc_mutex=E2=80=99, but calling SCM_SYSERROR there causes= it to try and > lock it again, leading to a deadlock: > > #3 0x00007f7ee9545969 in scm_dynwind_pthread_mutex_lock (mutex=3D0x7f7ee= 97bbfa0 ) at threads.c:1613 > #4 0x00007f7ee94d8dbb in scm_strerror (err=3Derr@entry=3D9) at error.c:1= 28 > #5 0x00007f7ee94d8e73 in scm_syserror (subr=3Dsubr@entry=3D0x7f7ee956afa= 0 "readdir") at error.c:169 > #6 0x00007f7ee94e29b4 in scm_readdir (port=3D#) at = filesys.c:1754 This particular issue is now fixed in Guile: https://git.savannah.gnu.org/cgit/guile.git/commit/?h=3Dstable-2.2&id=3D1= 87655cedb42cc4b01088179649b02295cd35260 Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 17 17:10:52 2017 Received: (at control) by debbugs.gnu.org; 17 Nov 2017 22:10:52 +0000 Received: from localhost ([127.0.0.1]:45413 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFoqa-0004gh-9F for submit@debbugs.gnu.org; Fri, 17 Nov 2017 17:10:52 -0500 Received: from hera.aquilenet.fr ([141.255.128.1]:49674) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eFoqY-0004gV-51 for control@debbugs.gnu.org; Fri, 17 Nov 2017 17:10:50 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id D7CD7F43D for ; Fri, 17 Nov 2017 23:10:51 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AJKdLAq6Lk3k for ; Fri, 17 Nov 2017 23:10:51 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id E85C5E418 for ; Fri, 17 Nov 2017 23:10:50 +0100 (CET) Date: Fri, 17 Nov 2017 23:10:47 +0100 Message-Id: <87tvxszhco.fsf@gnu.org> To: control@debbugs.gnu.org From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: control message for bug #29335 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) severity 29335 important From debbugs-submit-bounces@debbugs.gnu.org Sun Nov 19 17:48:51 2017 Received: (at 29335-done) by debbugs.gnu.org; 19 Nov 2017 22:48:51 +0000 Received: from localhost ([127.0.0.1]:48019 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eGYOR-0004P2-LS for submit@debbugs.gnu.org; Sun, 19 Nov 2017 17:48:51 -0500 Received: from hera.aquilenet.fr ([141.255.128.1]:53775) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eGYOP-0004Ou-TZ for 29335-done@debbugs.gnu.org; Sun, 19 Nov 2017 17:48:50 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 6F3FAF3EF for <29335-done@debbugs.gnu.org>; Sun, 19 Nov 2017 23:48:51 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PZJhn_xoyiL0 for <29335-done@debbugs.gnu.org>; Sun, 19 Nov 2017 23:48:50 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 83096DA3F for <29335-done@debbugs.gnu.org>; Sun, 19 Nov 2017 23:48:50 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: 29335-done@debbugs.gnu.org Subject: Re: bug#29335: 'guix publish' workers occasionally crash References: <878tf55i6u.fsf@gnu.org> Date: Sun, 19 Nov 2017 23:48:47 +0100 In-Reply-To: <878tf55i6u.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Fri, 17 Nov 2017 11:10:49 +0100") Message-ID: <87d14dzxyo.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 29335-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > On berlin I=E2=80=99ve noticed that the =E2=80=98guix publish=E2=80=99 wo= rkers would > occasionally stop working: the main thread would keep replying to HTTP > requests, but the worker threads would no longer do anything, and would > leave behind them a bunch of .tmp files in /var/cache/guix/publish. > > I captured the output of =E2=80=98guix publish=E2=80=99 (guix-0.13.0-8.35= 7ab93) and the > only clue I have is this: > > GET /6kl9ydqmgklcqhxswg6v5isq5n1ih5gp.narinfo > In guix/workers.scm: > 74:9 2 (_) > 78:32 1 (_ srfi-34 #) > In unknown file: > 0 (make-stack #t) > ERROR: In procedure make-stack: > ERROR: Throw to key `srfi-34' with args `(#)'. > GET /fgiih42mg2sr82mbmzf56grvrf021im6.narinfo Good news, this is fixed in 85f4f7b79040d982c6a655c898b4cd00d868fa9c. This could be reproduced by running =E2=80=98guix publish=E2=80=99 with 10 = workers or more, and then triggering nar compression en masse with =E2=80=98guix weath= er=E2=80=99. EBADF was due to a race condition in zlib.scm when closing gzip output ports: ;; 'gzclose' closes the underlying file descriptor. 'close-port' ;; calls close(2) and gets EBADF, which we swallow. (gzclose gzfile) (ignore-EBADF (close-port port))) There was a window after the =E2=80=98gzclose=E2=80=99 call during which th= e file descriptor for GZFILE and PORT above could be reused for something else, and then =E2=80=98close-port=E2=80=99 would close it. Thanks, Ludo=E2=80=99. From unknown Tue Sep 23 03:23:58 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 18 Dec 2017 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator