From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 27 04:22:41 2018 Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 09:22:41 +0000 Received: from localhost ([127.0.0.1]:34291 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqbT7-0000PM-1B for submit@debbugs.gnu.org; Tue, 27 Feb 2018 04:22:41 -0500 Received: from eggs.gnu.org ([208.118.235.92]:42421) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqbT5-0000P9-CM for submit@debbugs.gnu.org; Tue, 27 Feb 2018 04:22:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqbSz-00050h-8o for submit@debbugs.gnu.org; Tue, 27 Feb 2018 04:22:34 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:51747) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eqbSz-00050d-5r for submit@debbugs.gnu.org; Tue, 27 Feb 2018 04:22:33 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33377) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqbSx-0000Te-St for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 04:22:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqbSu-0004yt-PD for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 04:22:31 -0500 Received: from mout.web.de ([212.227.15.4]:45033) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eqbSu-0004yL-FY for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 04:22:28 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb003 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LwYtb-1eeyho0Ytb-018NMU; Tue, 27 Feb 2018 10:22:27 +0100 From: Michael Heerdegen To: bug-gnu-emacs@gnu.org Subject: 26.0.91; Crash when traversing a `stream-of-directory-files' Date: Tue, 27 Feb 2018 10:22:26 +0100 Message-ID: <87inaiss6l.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:hvKP+gAsqC03NREGBt//pGSayUGuY2ABDOvOb0RAPGYDzRGAkn3 OrMKuHQrTkTNg7jzTEdzLgvvdAVgVK6Tld3XZMWqySQZ85+rXbAhykqNt/Fds+xIQxW15Cn /9VuSBPGiVcKz4nKWvHs74ulsjl7SiYo4g1gA0zfY6zhVJMBIBj9sQITs6gXlielMmg9QP3 yzjsb2q7ARqgf5ppcDzfA== X-UI-Out-Filterresults: notjunk:1;V01:K0:7KfwDX0AHwg=:UpjL3/WxjTtZPIH7JA/Szp ItxGqosS+uSkBXcqf5VoTIKdewxotSfcCE8J+S8RRsYWYXJ0tgxSrBcT4cKocP9gOoBEZuNyS qWAcvc3gEAgXDY/WH11QH8ZDmskgjI3f4fuWY4iDow/waiF0N22H+z12Q1YhSBIohCWTvQv97 pvluiC4L5TzJBT+CIN29YdZlx/kEAAAgbSWyIrDXrRr6KMn6mIUzOsQNU806lq2nLxscqww8Q sCCYBmcp6ydc+mHcHMvUcsjb3aa0/q83YFKoNATA4v/Sz5Ytg2cQb+O2dSLzLcNdNH3a/h6fI wpO/tMNTGKiF4X6a1WSS7zQ8sI6XZYFnMJaFEnJrirnFhqLuvCmzjA5IfxR+2OIItXvh+03dh sj11c3K5bL1qdEcnoQA56wwsAnNbftbk0WEjiQ3UIveO1wtxPnpcpE3CEqaAesZ9RejGjFkEd F/HHDCrPjVwob7Imv3aNIsZAWcWopDtHQnmQ4jaVSJwofIo0L+Fu9qhUBSEPW6sqyNIne9GIJ DXQQSqQUVCM6Lry575QbNk0gU3bGd8wKNOzfbLCxjB678TfJJNFfROAzzDw0SQUR0KbUDnX8Q 10BSrTZfpdZhHBWQjsJRtJlDQE8l+CneDZkVV5VT99jckZwFHn20Tm4hW+gTiFTWP6wme6r4U pTKJldw7KvnBtPmjFxIQnpyqFI4U5uIiW3e8CezEU4OyN0f8VsPQyyS1J1c49Zq+GaCVkUISn qtGB0UE9lbjJ/FUyIkex+t/XW+g8L/MV1JiIjeKJoVFknzATgsyV+BwshHeBo5mDCugMOA6zh lyViz7nj0Sinv+AbcbXfzUfpy/VDGGYfXldX8J1lrLwqYzBFzQ= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.1 (----) Hello, Traversing a `stream-of-directory-files' over a huge directory hierarchy crashes my Emacs. Here is a recipe: I have a file "/home/micha/Treasure/today/stream-crash.el" with these contents: #+begin_src emacs-lisp ;; -*- lexical-binding: t -*- (require 'stream) (seq-doseq (_file (stream-of-directory-files "/home/micha" t nil t nil (lambda (file) (and (file-readable-p file) (file-regular-p file))))) nil) #+end_src Then I micha> emacs -Q -L /home/micha/software/elpa/packages/stream\ -l /home/micha/Treasure/today/stream-crash.el and I get a segfault after ~ 1 minute. This kind of crash only seems to occur when the traversed directory is sufficiently large. Thanks in advance, Michael. In GNU Emacs 26.0.91 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.28) of 2018-02-26 built on drachen Repository revision: ea8f9fd7be138708a52aad418d09d5d4ca6b2a7e Windowing system distributor 'The X.Org Foundation', version 11.0.11906000 System Description: Debian GNU/Linux testing (buster) From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 27 06:21:49 2018 Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 11:21:49 +0000 Received: from localhost ([127.0.0.1]:34412 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqdKP-0005BJ-3q for submit@debbugs.gnu.org; Tue, 27 Feb 2018 06:21:49 -0500 Received: from eggs.gnu.org ([208.118.235.92]:43685) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqdKN-0005B5-57 for submit@debbugs.gnu.org; Tue, 27 Feb 2018 06:21:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqdKH-00029U-2V for submit@debbugs.gnu.org; Tue, 27 Feb 2018 06:21:41 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:34101) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eqdKG-00029L-V4 for submit@debbugs.gnu.org; Tue, 27 Feb 2018 06:21:40 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34649) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqdKF-0000hU-RU for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 06:21:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqdKB-000281-N5 for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 06:21:39 -0500 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35957) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqdKB-00027w-It; Tue, 27 Feb 2018 06:21:35 -0500 Received: from [176.12.189.185] (port=20609 helo=[10.163.207.70]) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1eqdKA-00009e-3I; Tue, 27 Feb 2018 06:21:35 -0500 Date: Tue, 27 Feb 2018 13:21:30 +0200 User-Agent: K-9 Mail for Android In-Reply-To: <87inaiss6l.fsf@web.de> References: <87inaiss6l.fsf@web.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' To: bug-gnu-emacs@gnu.org, Michael Heerdegen , 30626@debbugs.gnu.org From: Eli Zaretskii Message-ID: <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) On February 27, 2018 11:22:26 AM GMT+02:00, Michael Heerdegen wrote: >=20 > Hello, >=20 > Traversing a `stream-of-directory-files' over a huge directory > hierarchy > crashes my Emacs=2E >=20 > Here is a recipe: I have a file > "/home/micha/Treasure/today/stream-crash=2Eel" with these contents: >=20 > #+begin_src emacs-lisp > ;; -*- lexical-binding: t -*- >=20 > (require 'stream) >=20 > (seq-doseq (_file (stream-of-directory-files > "/home/micha" t nil t nil > (lambda (file) (and (file-readable-p file) (file-regular-p file))))) > nil) > #+end_src >=20 > Then I >=20 > micha> emacs -Q -L /home/micha/software/elpa/packages/stream\ > -l /home/micha/Treasure/today/stream-crash=2Eel >=20 > and I get a segfault after ~ 1 minute=2E >=20 > This kind of crash only seems to occur when the traversed directory is > sufficiently large=2E I guess it's a stack overflow: that function recurses into subdirectories= =2E To avoid such problems, the function should be rewritten to work by BFS, n= ot DFS=2E From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 27 06:39:58 2018 Received: (at 30626) by debbugs.gnu.org; 27 Feb 2018 11:39:58 +0000 Received: from localhost ([127.0.0.1]:34439 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqdbw-0005gL-9f for submit@debbugs.gnu.org; Tue, 27 Feb 2018 06:39:56 -0500 Received: from mout.web.de ([212.227.15.4]:50267) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqdbu-0005g7-TT for 30626@debbugs.gnu.org; Tue, 27 Feb 2018 06:39:55 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb001 [213.165.67.108]) with ESMTPSA (Nemesis) id 0MZDga-1f9HUO0K44-00KueF; Tue, 27 Feb 2018 12:39:48 +0100 From: Michael Heerdegen To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> Date: Tue, 27 Feb 2018 12:39:47 +0100 In-Reply-To: <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> (Eli Zaretskii's message of "Tue, 27 Feb 2018 13:21:30 +0200") Message-ID: <87fu5moe4c.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:X5EY5LVRgdjRC8Wib1/u1h8L92V/TRjeExzyCsXYPpox+hHXHtE rwruZ2J9w6prrViJrm6vwpxsSQVK2Y2d8c4Wou3q2dHagrnEGCnFHBq1doPhM1OTccTV+Uk IAeHTNYhnrTTl1YNVLq/+DjYOpBRwdFUthUXQ4sfUsEDhltkT8SLou4seNc3fEPYEq2hPw5 LwphsktDq5HpPH17nFElg== X-UI-Out-Filterresults: notjunk:1;V01:K0:qhxycHH1bkI=:Pwx8O02HntoQwvU2K9Q2cc ubJW1BWUb+lcP+i2hka97ltM6Xyio/5EZUiYOFpHgoQszFJ8//m3wys0cP1kjoB++BuyW7o47 x86xAEEKREGwxHEXvTRAnQrDJbWM9ZfSCvT69npuGQvjqo+TsbXgFlFsL0C4CeJztAbTjYdLn +b6EhglRNcNXzdSVYZh4M5jh2i4PXnnWOO6y/4TeoZa7rdTJkxIkdJ+BmQtkxYqnSd3z5GgFx 6DVS7CTxJb+y+sO5DwZpyng3uh8QWYDE141zvrj9MckAzf8xpi6WvoJBig/nMo47g2JrEsFz9 jElT6+4AIAtdCr9YrvOFDUPoQMJ8Uavsil1zXbDA2pAalM0+g/M9f1nzoXOebyuQR9Z2qrEiV vXfFUcxAR/tw0Kknql8JZdzYDI3OGA9ujs+nU1UUydEeB+ophGU1jy22TNB5S6giOS/zQ6818 NJToa4t2pWDmgJeymTxDrTA8rxcyyL64w5J/Z7BZpLz3McaDrm3AT2trQSjn2SdBDaefW/2tQ GZ1ebGSImULP0oAB/zCFI+0FSpkHXsQn2TZ5jY9OfWpi8QgZ3NUPUtrhSPeFVV0jR72IcVv5n ywBPjyWPt8dMML0+6/8nvV3tKT0pryJbt6A1e/rCya+JCjopK+rKxr+wqwkRYcrzd4RnU5RmA 6ePmPOM5tPgYiKa8QxTD3xCgedBwsLdSrXiEa7TETFz8tZcakohdd0ozcv7BzjdUyoToL9+Pt HrqhxRB1enzs8bbD9iKN9g2bqVJNcY4//mX7808rXckY474ncwYUMl/QzQlkhG1U8ixVoKxlt pBxIetKjCoAmjImW9D5iRj/j3fS90L3YUWXCMNGwPvTnovsneg= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Eli Zaretskii writes: > I guess it's a stack overflow: that function recurses into > subdirectories. > > To avoid such problems, the function should be rewritten to work by > BFS, not DFS. Here is the backtrace I'm now able to produce. Looks like the crash happens in gc: | [lots of more lines like these cut] | #104019 0x00000000005e0526 in mark_object (arg=XIL(0x5a15413)) at alloc.c:6624 | #104020 0x00000000005dfab5 in mark_vectorlike (ptr=0x5a584e0) at alloc.c:6227 | #104021 0x00000000005e0526 in mark_object (arg=XIL(0x5bd7cf3)) at alloc.c:6624 | #104022 0x00000000005dfab5 in mark_vectorlike (ptr=0x5873a80) at alloc.c:6227 | #104023 0x00000000005e0526 in mark_object (arg=XIL(0x5bd7cb3)) at alloc.c:6624 | #104024 0x00000000006070ea in mark_specpdl (first=0x58a1348, ptr=0x58a1b90) at eval.c:3868 | #104025 0x00000000006856b2 in mark_one_thread (thread=0xcd5340 ) at thread.c:614 | #104026 0x00000000006857c7 in mark_threads_callback (ignore=0x0) at thread.c:649 | #104027 0x00000000005dda03 in flush_stack_call_func (func=0x685781 , arg=0x0) at alloc.c:5220 | #104028 0x00000000006857f5 in mark_threads () at thread.c:656 | #104029 0x00000000005df2d2 in garbage_collect_1 (end=0x7fffffff63f0) at alloc.c:5997 | #104030 0x00000000005df929 in Fgarbage_collect () at alloc.c:6168 | #104031 0x000000000055746c in maybe_gc () at lisp.h:4749 | #104032 0x00000000006047cb in Ffuncall (nargs=4, args=0x7fffffff64d0) at eval.c:2751 | #104033 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83c4), vector=XIL(0x3255bb5), maxdepth=make_number(13), args_template=make_number(128), nargs=2, args=0x7fffffff6b38) at bytecode.c:629 | #104034 0x000000000060529c in funcall_lambda (fun=XIL(0x3255c15), nargs=2, arg_vector=0x7fffffff6b38) at eval.c:2970 | #104035 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffff6b30) at eval.c:2771 | #104036 0x000000000060395b in Fapply (nargs=3, args=0x7fffffff6b30) at eval.c:2346 | #104037 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 , numargs=3, args=0x7fffffff6b30) at eval.c:2824 | #104038 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffff6b28) at eval.c:2769 | #104039 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x15372775), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff6ff8) at bytecode.c:629 | #104040 0x000000000060529c in funcall_lambda (fun=XIL(0x153727c5), nargs=0, arg_vector=0x7fffffff6ff8) at eval.c:2970 | #104041 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff6ff0) at eval.c:2771 | #104042 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff7460) at bytecode.c:629 | #104043 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff7458) at eval.c:2970 | #104044 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff7450) at eval.c:2771 | #104045 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff78e0) at bytecode.c:629 | #104046 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff78d8) at eval.c:2970 | #104047 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff78d0) at eval.c:2771 | #104048 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x153727f5), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff7da8) at bytecode.c:629 | #104049 0x000000000060529c in funcall_lambda (fun=XIL(0x15372845), nargs=0, arg_vector=0x7fffffff7da8) at eval.c:2970 | #104050 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff7da0) at eval.c:2771 | #104051 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff8210) at bytecode.c:629 | #104052 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff8208) at eval.c:2970 | #104053 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8200) at eval.c:2771 | #104054 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff8690) at bytecode.c:629 | #104055 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff8688) at eval.c:2970 | #104056 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8680) at eval.c:2771 | #104057 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x15372875), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff8b58) at bytecode.c:629 | #104058 0x000000000060529c in funcall_lambda (fun=XIL(0x153728c5), nargs=0, arg_vector=0x7fffffff8b58) at eval.c:2970 | #104059 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff8b50) at eval.c:2771 | #104060 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff8fc0) at bytecode.c:629 | #104061 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff8fb8) at eval.c:2970 | #104062 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff8fb0) at eval.c:2771 | #104063 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffff9440) at bytecode.c:629 | #104064 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffff9438) at eval.c:2970 | #104065 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff9430) at eval.c:2771 | #104066 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f83e4), vector=XIL(0x153728f5), maxdepth=make_number(8), args_template=make_number(256), nargs=0, args=0x7fffffff9908) at bytecode.c:629 | #104067 0x000000000060529c in funcall_lambda (fun=XIL(0x15372945), nargs=0, arg_vector=0x7fffffff9908) at eval.c:2970 | #104068 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffff9900) at eval.c:2771 | #104069 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffff9d70) at bytecode.c:629 | #104070 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffff9d68) at eval.c:2970 | #104071 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffff9d60) at eval.c:2771 | #104072 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffffa1e8) at bytecode.c:629 | #104073 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffffa1e0) at eval.c:2970 | #104074 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffa1d8) at eval.c:2771 | #104075 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f8c64), vector=XIL(0x153729a5), maxdepth=make_number(7), args_template=make_number(256), nargs=0, args=0x7fffffffa6a8) at bytecode.c:629 | #104076 0x000000000060529c in funcall_lambda (fun=XIL(0x153729f5), nargs=0, arg_vector=0x7fffffffa6a8) at eval.c:2970 | #104077 0x00000000006048d4 in Ffuncall (nargs=1, args=0x7fffffffa6a0) at eval.c:2771 | #104078 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42b7ad4), vector=XIL(0x930825), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffffab10) at bytecode.c:629 | #104079 0x000000000060529c in funcall_lambda (fun=XIL(0x3b73d75), nargs=1, arg_vector=0x7fffffffab08) at eval.c:2970 | #104080 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffab00) at eval.c:2771 | #104081 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f82f4), vector=XIL(0x3255aa5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffffaf88) at bytecode.c:629 | #104082 0x000000000060529c in funcall_lambda (fun=XIL(0x3255ab5), nargs=1, arg_vector=0x7fffffffaf80) at eval.c:2970 | #104083 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffaf78) at eval.c:2771 | #104084 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42f8c04), vector=XIL(0x331ede5), maxdepth=make_number(5), args_template=make_number(514), nargs=2, args=0x7fffffffb5d0) at bytecode.c:629 | #104085 0x000000000060529c in funcall_lambda (fun=XIL(0x331ee05), nargs=2, arg_vector=0x7fffffffb5c0) at eval.c:2970 | #104086 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffffb5b8) at eval.c:2771 | #104087 0x000000000060390d in Fapply (nargs=4, args=0x7fffffffb5b8) at eval.c:2342 | #104088 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 , numargs=4, args=0x7fffffffb5b8) at eval.c:2824 | #104089 0x0000000000604890 in Ffuncall (nargs=5, args=0x7fffffffb5b0) at eval.c:2769 | #104090 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x42fb574), vector=XIL(0x425f285), maxdepth=make_number(15), args_template=make_number(642), nargs=2, args=0x7fffffffba40) at bytecode.c:629 | #104091 0x000000000060529c in funcall_lambda (fun=XIL(0x425f305), nargs=2, arg_vector=0x7fffffffba30) at eval.c:2970 | #104092 0x000000000060500d in apply_lambda (fun=XIL(0x425f305), args=XIL(0x5bd89c3), count=30) at eval.c:2906 | #104093 0x0000000000603602 in eval_sub (form=XIL(0x5bd89d3)) at eval.c:2279 | #104094 0x0000000000632c69 in readevalloop_eager_expand_eval (val=XIL(0x5bd8a73), macroexpand=XIL(0xda5d0)) at lread.c:1850 | #104095 0x00000000006335f0 in readevalloop (readcharfun=XIL(0x5b75ce5), infile0=0x0, sourcename=XIL(0x55c2e14), printflag=false, unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at lread.c:2036 | #104096 0x0000000000633a0a in Feval_buffer (buffer=XIL(0), printflag=XIL(0), filename=XIL(0x56ea3c4), unibyte=XIL(0), do_allow_print=XIL(0)) at lread.c:2103 | #104097 0x0000000000604d25 in funcall_subr (subr=0xc40240 , numargs=0, args=0x7fffffffbfa0) at eval.c:2856 | #104098 0x0000000000604890 in Ffuncall (nargs=1, args=0x7fffffffbf98) at eval.c:2769 | #104099 0x00000000005fc8f7 in Ffuncall_interactively (nargs=1, args=0x7fffffffbf98) at callint.c:252 | #104100 0x0000000000604b87 in funcall_subr (subr=0xc3cfc0 , numargs=1, args=0x7fffffffbf98) at eval.c:2824 | #104101 0x0000000000604890 in Ffuncall (nargs=2, args=0x7fffffffbf90) at eval.c:2769 | #104102 0x00000000005fec50 in Fcall_interactively (function=XIL(0xad320), record_flag=XIL(0xaad0), keys=XIL(0x5a55745)) at callint.c:854 | #104103 0x0000000000604cac in funcall_subr (subr=0xc3d000 , numargs=3, args=0x7fffffffc300) at eval.c:2849 | #104104 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffc2f8) at eval.c:2769 | #104105 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f4bec), vector=XIL(0x9f4c0d), maxdepth=make_number(13), args_template=make_number(1025), nargs=2, args=0x7fffffffc868) at bytecode.c:629 | #104106 0x000000000060529c in funcall_lambda (fun=XIL(0x9f4bbd), nargs=2, arg_vector=0x7fffffffc858) at eval.c:2970 | #104107 0x00000000006048d4 in Ffuncall (nargs=3, args=0x7fffffffc850) at eval.c:2771 | #104108 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f489c), vector=XIL(0x9f48bd), maxdepth=make_number(15), args_template=make_number(769), nargs=3, args=0x7fffffffce00) at bytecode.c:629 | #104109 0x000000000060529c in funcall_lambda (fun=XIL(0x9f485d), nargs=3, arg_vector=0x7fffffffcde8) at eval.c:2970 | #104110 0x00000000006048d4 in Ffuncall (nargs=4, args=0x7fffffffcde0) at eval.c:2771 | #104111 0x0000000000603cdb in Fapply (nargs=2, args=0x7fffffffcfc0) at eval.c:2389 | #104112 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 , numargs=2, args=0x7fffffffcfc0) at eval.c:2824 | #104113 0x0000000000604890 in Ffuncall (nargs=3, args=0x7fffffffcfb8) at eval.c:2769 | #104114 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x505da94), vector=XIL(0x5044095), maxdepth=make_number(3), args_template=XIL(0), nargs=0, args=0x0) at bytecode.c:629 | #104115 0x0000000000605615 in funcall_lambda (fun=XIL(0x50c3185), nargs=5, arg_vector=0x5044095) at eval.c:3052 | #104116 0x00000000006048d4 in Ffuncall (nargs=6, args=0x7fffffffd450) at eval.c:2771 | #104117 0x0000000000603cdb in Fapply (nargs=3, args=0x7fffffffd648) at eval.c:2389 | #104118 0x0000000000604b87 in funcall_subr (subr=0xc3d7c0 , numargs=3, args=0x7fffffffd648) at eval.c:2824 | #104119 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffd640) at eval.c:2769 | #104120 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x14402c4), vector=XIL(0x50441c5), maxdepth=make_number(5), args_template=make_number(128), nargs=4, args=0x7fffffffdbf0) at bytecode.c:629 | #104121 0x000000000060529c in funcall_lambda (fun=XIL(0x50441f5), nargs=4, arg_vector=0x7fffffffdbf0) at eval.c:2970 | #104122 0x00000000006048d4 in Ffuncall (nargs=5, args=0x7fffffffdbe8) at eval.c:2771 | #104123 0x00000000005fc8f7 in Ffuncall_interactively (nargs=5, args=0x7fffffffdbe8) at callint.c:252 | #104124 0x0000000000604b87 in funcall_subr (subr=0xc3cfc0 , numargs=5, args=0x7fffffffdbe8) at eval.c:2824 | #104125 0x0000000000604890 in Ffuncall (nargs=6, args=0x7fffffffdbe0) at eval.c:2769 | #104126 0x0000000000603cdb in Fapply (nargs=3, args=0x7fffffffdd90) at eval.c:2389 | #104127 0x00000000005fcd7d in Fcall_interactively (function=XIL(0xda3c0), record_flag=XIL(0), keys=XIL(0x5a3e0a5)) at callint.c:389 | #104128 0x0000000000604cac in funcall_subr (subr=0xc3d000 , numargs=3, args=0x7fffffffdfe0) at eval.c:2849 | #104129 0x0000000000604890 in Ffuncall (nargs=4, args=0x7fffffffdfd8) at eval.c:2769 | #104130 0x000000000064cc41 in exec_byte_code (bytestr=XIL(0x9f4bec), vector=XIL(0x9f4c0d), maxdepth=make_number(13), args_template=make_number(1025), nargs=1, args=0x7fffffffe520) at bytecode.c:629 | #104131 0x000000000060529c in funcall_lambda (fun=XIL(0x9f4bbd), nargs=1, arg_vector=0x7fffffffe518) at eval.c:2970 | #104132 0x00000000006048d4 in Ffuncall (nargs=2, args=0x7fffffffe510) at eval.c:2771 | #104133 0x00000000006042a0 in call1 (fn=XIL(0x3f00), arg1=XIL(0xda3c0)) at eval.c:2620 | #104134 0x000000000055ec02 in command_loop_1 () at keyboard.c:1482 | #104135 0x00000000006013b2 in internal_condition_case (bfun=0x55e45e , handlers=XIL(0x5250), hfun=0x55dc1d ) at eval.c:1332 | #104136 0x000000000055e151 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110 | #104137 0x0000000000600c94 in internal_catch (tag=XIL(0xc6f0), func=0x55e124 , arg=XIL(0)) at eval.c:1097 | #104138 0x000000000055e0ef in command_loop () at keyboard.c:1089 | #104139 0x000000000055d7ef in recursive_edit_1 () at keyboard.c:695 | #104140 0x000000000055d970 in Frecursive_edit () at keyboard.c:766 | #104141 0x000000000055b5f0 in main (argc=1, argv=0x7fffffffe9f8) at emacs.c:1713 | Cannot access memory at address 0x7fffff66ff4f Michael. From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 27 07:09:24 2018 Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 12:09:24 +0000 Received: from localhost ([127.0.0.1]:34462 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqe4S-0006Q2-5k for submit@debbugs.gnu.org; Tue, 27 Feb 2018 07:09:24 -0500 Received: from eggs.gnu.org ([208.118.235.92]:56604) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqe4R-0006Pq-3L for submit@debbugs.gnu.org; Tue, 27 Feb 2018 07:09:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqe4L-00064H-5d for submit@debbugs.gnu.org; Tue, 27 Feb 2018 07:09:17 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:49960) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eqe4L-00064D-1f for submit@debbugs.gnu.org; Tue, 27 Feb 2018 07:09:17 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47566) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqe4K-0003gj-23 for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 07:09:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqe4J-00063T-9q for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 07:09:16 -0500 Received: from mout.web.de ([212.227.15.3]:59789) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eqe4C-00060S-IY; Tue, 27 Feb 2018 07:09:08 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb001 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LfzgJ-1eOVBX1lGd-00pgjc; Tue, 27 Feb 2018 13:09:00 +0100 From: Michael Heerdegen To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> Date: Tue, 27 Feb 2018 13:08:59 +0100 In-Reply-To: <87fu5moe4c.fsf@web.de> (Michael Heerdegen's message of "Tue, 27 Feb 2018 12:39:47 +0100") Message-ID: <877eqyocro.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:bxTtByJhimEv05aXorB6dgYWWI5CxcjWxfvhjzLjahV1FhFYFNA T1BSRBISXpfR++tkkYh8KMv/YpFKpT8Bag5I9ZZI8Vck42LSKCESQaCarfKPHHGgOSA9/nE iewu5TIdrCh7pWctYVGeOJAQ6ETjviH/te/yjFxqa+Dk8BsmS0GGtgxJWSlY2Va2/frcaMo ddTL6YvGsUZVFBkKg9seA== X-UI-Out-Filterresults: notjunk:1;V01:K0:5ht05sbwaVY=:YMhtPfUxSe0lsYXrfC3g0I lp+b54KumohKGoMlp9KAsLkVjC7Z4Pzifm4LlFr8pIPqlMa+4NxCGggMQMq5jD1p0fKq3DNww O3J6kZrEJi1QL/6RYZ+iplDmqqJxJ/XhznPPWnp4+BaZ9/siI5IjFAuaozvPGEXaavEE5omzN L0ndlZnJI4EOGm/IEL66CW0dfTdSVJWmU7z8va7W+ZL+ogphw9ULfk89+LIJkmN8t0jEj7pYK GjwRvc15QxX7hdHB62DwLu0OQpfjoZnYM1sRE0qoUUBX9aTwvV/G0xy6BbjIZqcezF+J1V+rm yRHy0QOK01u1MdrwBIvABQUBwCcZn5ZdWiZRhPnW4CINT7QEZOA+vtvACUD3UvlCiU+GVu4M3 AhZf8RM6x3D2I1veC12D4sR2h6I5d4HcGmXyWQf3O6anb4Ja5v5OXc7dBOGbyE7cdfwC8vzfN n2MsoUZSpm81oxFZD6Ygsue8eVL/lx95ax3FEdk2yDIKZc5ZgvoRZFbcgryKheFuDUk+ucOz7 UQfLR2Z5H9arnNJ50EmyFKNY+0PjCdwv/czkI1PCqhebZWlNG5ZV6bXVgjjQkIOU83TSDO2Ix +LMWnqwn13jPxLHJ/a4cb/Ol8o33EblCMPKJR0+g0VHofgNa41KeA/WaIcw+D6uT86Gcif3Tz GKFiFrsVxx+fgABpg99gHAGAgQlNc5yECuZIC+dcs9L8LjMsYdyXaK+buPc/wgIFbFJ5Z81QS RuWqyojhIRJzjHTIBQWM5e4FxrNy2LjeHAHYoVYs51lIx0vBdQhrdCR06uYFoTOggByhADrxD ylglMvVN2syigdWcgugSJipT6S0b75fro8FhluRmrm5/iOg3kg= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.1 (----) Michael Heerdegen writes: > Here is the backtrace I'm now able to produce. Looks like the crash > happens in gc. Here is a much simpler example that crashes as well: #+begin_src emacs-lisp (seq-doseq (_ (stream-range 1 1000000)) nil) #+end_src Note that this is executed as a loop due how to streams are implemented, although the definition of `seq-doseq' looks recursive. But it seems that gc has a problem with the large number of conses created when processing that. Michael. From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 27 13:00:57 2018 Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 18:00:57 +0000 Received: from localhost ([127.0.0.1]:35899 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqjYe-0002du-Sm for submit@debbugs.gnu.org; Tue, 27 Feb 2018 13:00:57 -0500 Received: from eggs.gnu.org ([208.118.235.92]:47382) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqjYd-0002dh-G9 for submit@debbugs.gnu.org; Tue, 27 Feb 2018 13:00:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqjYX-0007yO-QJ for submit@debbugs.gnu.org; Tue, 27 Feb 2018 13:00:50 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:39218) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eqjYX-0007yD-M7 for submit@debbugs.gnu.org; Tue, 27 Feb 2018 13:00:49 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38341) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqjYW-00067P-9a for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 13:00:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqjYS-0007up-J4 for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 13:00:48 -0500 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42529) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqjYJ-0007qX-JK; Tue, 27 Feb 2018 13:00:35 -0500 Received: from [176.228.60.248] (port=2484 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1eqjYI-0003P6-SW; Tue, 27 Feb 2018 13:00:35 -0500 Date: Tue, 27 Feb 2018 20:00:39 +0200 Message-Id: <831sh61feg.fsf@gnu.org> From: Eli Zaretskii To: Michael Heerdegen In-reply-to: <87fu5moe4c.fsf@web.de> (message from Michael Heerdegen on Tue, 27 Feb 2018 12:39:47 +0100) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Michael Heerdegen > Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org > Date: Tue, 27 Feb 2018 12:39:47 +0100 > > Eli Zaretskii writes: > > > I guess it's a stack overflow: that function recurses into > > subdirectories. > > > > To avoid such problems, the function should be rewritten to work by > > BFS, not DFS. > > Here is the backtrace I'm now able to produce. Looks like the crash > happens in gc: GC is deeply-recursive, and you have exacerbated that by using up a lot of stack. From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 27 13:09:16 2018 Received: (at submit) by debbugs.gnu.org; 27 Feb 2018 18:09:17 +0000 Received: from localhost ([127.0.0.1]:35922 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqjgi-0002ri-NF for submit@debbugs.gnu.org; Tue, 27 Feb 2018 13:09:16 -0500 Received: from eggs.gnu.org ([208.118.235.92]:50179) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqjgg-0002rV-SY for submit@debbugs.gnu.org; Tue, 27 Feb 2018 13:09:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqjga-0003PT-S6 for submit@debbugs.gnu.org; Tue, 27 Feb 2018 13:09:09 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:49711) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eqjga-0003PF-Oz for submit@debbugs.gnu.org; Tue, 27 Feb 2018 13:09:08 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41102) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqjgV-0000D0-Nq for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 13:09:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqjgU-0003N1-Qp for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 13:09:03 -0500 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42715) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqjgK-0003F4-CH; Tue, 27 Feb 2018 13:08:52 -0500 Received: from [176.228.60.248] (port=2496 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1eqjgI-0008NF-UI; Tue, 27 Feb 2018 13:08:51 -0500 Date: Tue, 27 Feb 2018 20:08:56 +0200 Message-Id: <83zi3uz4nb.fsf@gnu.org> From: Eli Zaretskii To: Michael Heerdegen In-reply-to: <877eqyocro.fsf@web.de> (message from Michael Heerdegen on Tue, 27 Feb 2018 13:08:59 +0100) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Michael Heerdegen > Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org > Date: Tue, 27 Feb 2018 13:08:59 +0100 > > #+begin_src emacs-lisp > (seq-doseq (_ (stream-range 1 1000000)) nil) > #+end_src > > Note that this is executed as a loop due how to streams are implemented, > although the definition of `seq-doseq' looks recursive. But it seems > that gc has a problem with the large number of conses created when > processing that. What can we do instead in such cases? Stack-overflow protection cannot work in GC, so you are shooting yourself in the foot by creating such large recursive structures. By the time we get to GC, where the problem will happen, it's too late, because the memory was already allocated. Does anyone has a reasonable idea for avoiding the crash in such programs? From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 27 20:29:29 2018 Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 01:29:29 +0000 Received: from localhost ([127.0.0.1]:36267 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqqYj-00022v-Kw for submit@debbugs.gnu.org; Tue, 27 Feb 2018 20:29:29 -0500 Received: from mail-it0-f49.google.com ([209.85.214.49]:40704) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqqYi-00022i-0I for 30626@debbugs.gnu.org; Tue, 27 Feb 2018 20:29:28 -0500 Received: by mail-it0-f49.google.com with SMTP id v186so1509606itc.5 for <30626@debbugs.gnu.org>; Tue, 27 Feb 2018 17:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=jU8JUjE6W55VY6I37oKWn09TjMLw2bqetg/bhOTki3g=; b=b1sFUlg2c/0p86qMHVI0A6G4Cuef0V0lN92r+W1JoLkvH50AEj+HUz32gBIuq0I0IU lBG5i1MJV19FATD6xjV6VeXoVgG8g8gPQDxdPeoDWXmX+il7V1uYjNxaCtmtIOgUA1sZ atyOtXm0D5rCBufkwp0Hh9iCaxHHPfUI9QwbLYmUoTZynNhjkeqIAv92HCcgwui2mDWw PrZ7agsTYoN98tf9jSqu4OsLpdlDOs5j+vE+ugmXUeyPtjqlA7V93p76z0o1O+c75SCq 37meonZOa+8XSKN5ajzIgISwh0GhCBMoPesoltFYyQNDxg+a+/okl4IxkL6h1TZBs4xk NQyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=jU8JUjE6W55VY6I37oKWn09TjMLw2bqetg/bhOTki3g=; b=Uijut9rvzW1/hMqIIgycpcaVSg7/oBu0WdRxzKn2jpt848bwe7xFpADkhIuSjIfJ9x LPjj0e/5EA2zjrdw/BBcXKzKMyLJ8pPY4/mghcld1UYs0ZoQpY72MEkm5XziHXWxQzf3 XWLu5G8CDa9injIwlLT9gKLmmLZsb1d2Zp4gOrloRQOmOHGG/tpHBE88xSQqxtbCLHeS VeAUir7+XrBznZhAnl2dnYggac22/x3DG0ywc54svKe3uxyo8dXb4W6C0nIe4H2f2QUw 03UwY7XeuiY03UE/sHOXlFB+imbz2kgp2oQZNvAnjNj2Dzxno1FsOwIOhm4glEb8OHMa FlWg== X-Gm-Message-State: APf1xPBmX0JPUyobkCYKRubPmLFhpeFFzsF8ju62DUT6VALi64FIMVZ5 CEqjxH5S4s3dHAQzIsl9j6/2HQ== X-Google-Smtp-Source: AG47ELtuM8/FMdTKLAc+W7C0nSbU3hvIe8XbADqTWfqrkJJM6x1QT7EIXEhSndLFCCFGyq5ZCnSrfw== X-Received: by 10.36.144.7 with SMTP id x7mr7071226itd.128.1519781362550; Tue, 27 Feb 2018 17:29:22 -0800 (PST) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id m32sm9372369iti.3.2018.02.27.17.29.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 27 Feb 2018 17:29:22 -0800 (PST) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> Date: Tue, 27 Feb 2018 20:29:21 -0500 In-Reply-To: <83zi3uz4nb.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 27 Feb 2018 20:08:56 +0200") Message-ID: <87lgfd52by.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Michael Heerdegen , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Eli Zaretskii writes: >> From: Michael Heerdegen >> Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org >> Date: Tue, 27 Feb 2018 13:08:59 +0100 >> >> #+begin_src emacs-lisp >> (seq-doseq (_ (stream-range 1 1000000)) nil) >> #+end_src >> >> Note that this is executed as a loop due how to streams are implemented, >> although the definition of `seq-doseq' looks recursive. Doesn't look recursive to me, it expands to a call to seq-do, which uses a simple loop. >> But it seems that gc has a problem with the large number of conses >> created when processing that. > > What can we do instead in such cases? Stack-overflow protection > cannot work in GC, so you are shooting yourself in the foot by > creating such large recursive structures. By the time we get to GC, > where the problem will happen, it's too late, because the memory was > already allocated. > > Does anyone has a reasonable idea for avoiding the crash in such > programs? I don't have a quick answer for the general case, but I think it's a bug in stream.el that it's creating such large structures in the first place. As far as I understand it, the point of streams is to handle long lists by encoding them as (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST) so as to avoid allocating large amounts of memory. Is there an easy way to find out what the large structures are, and where they are coming from? From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 28 05:58:59 2018 Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 10:58:59 +0000 Received: from localhost ([127.0.0.1]:36478 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqzRr-0000cM-Cb for submit@debbugs.gnu.org; Wed, 28 Feb 2018 05:58:59 -0500 Received: from mout.web.de ([212.227.17.12]:50553) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqzRp-0000c7-1h for 30626@debbugs.gnu.org; Wed, 28 Feb 2018 05:58:57 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MOzoR-1elEYY3buB-006PZ8; Wed, 28 Feb 2018 11:58:50 +0100 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> Date: Wed, 28 Feb 2018 11:58:49 +0100 In-Reply-To: <87lgfd52by.fsf@gmail.com> (Noam Postavsky's message of "Tue, 27 Feb 2018 20:29:21 -0500") Message-ID: <87bmg91ity.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:XkQ6Dh1kC+Kyw/7Jvu7xd3wRLLW9ULo+7fVx1kw4mvGWdh2aLb1 q3PukxVyQ9NfQcU2PS0sYCxUiVLO9mY+OND2o38Vdc8M3FAdlyD/KrLFtwWUzNquEShTK8q hREyRTtNdfzP4lXRz2Fsk3e6s/xqYgbUzHQEXHJ+QnkCWPixefVAKI3DNdczItkD7RBtaqV 2Q1/uUGfFUtVi4rB+jzaw== X-UI-Out-Filterresults: notjunk:1;V01:K0:MSX261JF4Nk=:9myiQqNSVT3vSAAAi0VHVA tYUYEh9PtaTrt7p9roq3PwsyHA6JZYsa4u+ebygitFxrQiCeTMk0xnTdVAs0wy1buaWaGzrWr ieNaGej1jwYjoAqqg6aOT0KnUfT8jtkywqLB0ozCGMX7OUpOEgELMGzfKclB35AixyyR3VFk7 YiRjB43ZUBo22pPMQ+Xv4LV+Kd7N3DgQ1Aiz3BfC2mNMKV8VfKJuuLnfbLAvKKYlaOhc5a+TB drcEEImqQ7p7c/tto1AAXWHW4e2D8TNS9u7++Z/eD9xWiZnAYZ9jXSzfrwpcQGYuT1tbHzZWw G/GVa6uIWS62KC7kxSOMSUYPSLIPzV1QwgNzfsFg1ajRvVrhXwJcMWwBJpeP/kPxZl/2jZ7UC Jk+X8fxCOKEKmHS3vr+2FmRX8uk6xj8CXSf8XLeMA75vJfRuF+A5ILVFTGKvzIB5/VFd/AllF WuVZLXEqtq63ao3CxCe0haMz/qLx82DaXmKLvrn3cE7dJ5PeQOcNK9bR16qOW7ycV+RQ93K6C sYf7MBypSjQ78RXSxlbR81/qxRIeBVJ6rJsm8ytUp7R+FR7xZs2i+tSdEiOZVS7FBOG/gJZjo ZxlnAmCk4xdYm6n4kthT2YZsrqMzepq4I+2uKUSZDtMu+TY7+MJvymrXHEkBGAqbZokmUhzDG 08KSOlsJeEwxM/3/tc2j1NYMg502+KiWW9qaUunZ3k856/5HSuqZBS1NiHJZI+TOfUFhDws2k fG85Wda9azYAA0gsETqlzOzActd6tOjMkEtaqLH73DEtOllCYqLrxauV4ZDaw93bLIAn2t2Y0 yM9oT7bdcTarpRSkNQP4lLZbUDFMFsCaQ8yFYQAWE5vsI862sI= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Noam Postavsky writes: > >> #+begin_src emacs-lisp > >> (seq-doseq (_ (stream-range 1 1000000)) nil) > >> #+end_src > >> > >> Note that this is executed as a loop due how to streams are > >> implemented, although the definition of `seq-doseq' looks > >> recursive. > > Doesn't look recursive to me, it expands to a call to seq-do, which uses > a simple loop. I was imprecise, I meant that the streams are defined recursively (most of the time). Though, it's a delayed type of recursion. Anyway, I think that this doesn't matter here. > > Does anyone has a reasonable idea for avoiding the crash in such > > programs? > > I don't have a quick answer for the general case, but I think it's a bug > in stream.el that it's creating such large structures in the first > place. As far as I understand it, the point of streams is to handle > long lists by encoding them as > > (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST) Yes, that's exactly how it's implemented. When requesting more elements from the stream, that becomes (FIRST-VALUE . (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST)) When we loop over the string, the cons whose car is the FIRST-VALUE, let's call it cons1, is immediately thrown away, and we continue with (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST) etc. AFAIU the problem is that the cons1 still exists in memory until garbage collection kicks in. When that happens, the cons1 points to a largely recursive cons structure, though this structure is never referenced from Lisp in this form. > so as to avoid allocating large amounts of memory. Is there an easy way > to find out what the large structures are, and where they are coming > from? I think I've answered that. At least, I think so. What I don't understand is that when I force the `seq-doseq' to call `garbace-collect' explicitly every 1000 cycles, or so, it doesn't help: the crash still happens after generating ~ 70 000 elements, or some more, but I can't avoid the crash, no matter how often I call gc. So I'm not sure whether these long lists are the problem or something else. The FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST looks harmless when I print it, even after thousands of iterations, so I would not understand why that could be problematic. streams.el implements things in a way that these rest functions are not deeply nested lambdas. Michael. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 28 06:05:38 2018 Received: (at submit) by debbugs.gnu.org; 28 Feb 2018 11:05:38 +0000 Received: from localhost ([127.0.0.1]:36508 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqzYH-0000oE-Rf for submit@debbugs.gnu.org; Wed, 28 Feb 2018 06:05:37 -0500 Received: from eggs.gnu.org ([208.118.235.92]:49897) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqzYG-0000o1-Gi for submit@debbugs.gnu.org; Wed, 28 Feb 2018 06:05:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqzY9-0000b1-5n for submit@debbugs.gnu.org; Wed, 28 Feb 2018 06:05:31 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:54930) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eqzY9-0000au-2B for submit@debbugs.gnu.org; Wed, 28 Feb 2018 06:05:29 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40861) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqzY7-0006Fk-Sn for bug-gnu-emacs@gnu.org; Wed, 28 Feb 2018 06:05:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqzY6-0000aN-Vu for bug-gnu-emacs@gnu.org; Wed, 28 Feb 2018 06:05:27 -0500 Received: from mout.web.de ([212.227.15.4]:56337) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eqzY0-0000Vg-5X; Wed, 28 Feb 2018 06:05:20 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb003 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LsyOA-1ejnga1o8w-012ajU; Wed, 28 Feb 2018 12:05:10 +0100 From: Michael Heerdegen To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> Date: Wed, 28 Feb 2018 12:05:09 +0100 In-Reply-To: <83zi3uz4nb.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 27 Feb 2018 20:08:56 +0200") Message-ID: <877eqx1ije.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:JP1ZoJ4r0VJOg7NYJUFhT1Xf+OPMlaSkBQNECuVUQ9QcYHn46TX vNsQ6+d0lwpOVExciCBiMClF34B3vE67VDPKVup+A/6cSp1gfo83DZtLBnl1UA13EFZDL38 cdGzLgtZWLaXa+88kPoRdM7iTn67tCBrFqJIaU0w5p30ULMXPzkqteX6gj9q6/1ULfpnw1n vWJbEE96Rqy/yJCwT50nA== X-UI-Out-Filterresults: notjunk:1;V01:K0:rR/WE+bKzkM=:9oRVS4zbtmFjh/aqHtnhxe z+CPZGipHXLDaxBenf/In8FJ3d0oicNdPy6Sl/CosY09ppS3xkszYrTWtdL5UTxxcFouN2J0v MBpNtKmP+Zk3nQKTg8a/JnVFHFDclxOtwrdDE0awnjI6eXsoqia2WvoPJA2j5SHRjvyFJxop9 rblnf275a2ZuLgK6R+5wcnANktKgNcTMh0+PD6QQGYbbp8NlDBo4ExfQa2E69EPoq77rbYsIT 40U3Apqkuylp8D1v0ckRMqLbZoiF/1tL9eTEFJzuM5vNEdLpW6HltPuAL6qqlnrxVDgvVYFxu mEkeDIyfgd1de1oI6bPKeZ4TMfRL+EQJBg4Tj8asl3SNWcM+TZc4fud+7HhajVPl7IC46CR+y 7XjuhmAiuyVfIVTB8WUuJVPs27Z8ZaDLeW0DVxQyyCXI5c4GUNgBRGyMhQNDzOPNG0fShceVn Rdraxh8izw3M4A6DTo580oiSwO39F9b47iY5Gqdpl865FQA+Xvrbzi2+L7hgw79X/Y3iKPzuG 4TxbtNb9Ej5RksyhHJfScPgVe7Dv7Fo/p7ghcTnTjDD1N+Rb77u82xDJ4lIkIVBN7uV0PiZZO BgcLAbyD/mRNqLeKfHZtVtk03ApUlo4KwSsnYhLskEU8M71UTYRIsh8z/dmlbdQIU4ffP1Lo5 x3t48iszmHtRsl8+VJAnHOwCCHH0yhNZy1if4rCfdo2c9CQoa/p7KR84WbIU5TG/kZ/GZC9rH p9lxo702COHPfyNaE1FLjLkCRWNXSwMSm3joGAA6uAaBLBY/e40tRJC5y2rm8Gueaxkt7/Vno xfEUwMqVCbBy5o/0wHNgkkC08PyPldD6eyGM1R+1RxQRZXNMC8= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit Cc: bug-gnu-emacs@gnu.org, Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.1 (----) Hello, I had written that > > (seq-doseq (_ (stream-range 1 1000000)) nil) crashes. CC'ing Nicolas, the author of stream.el. > What can we do instead in such cases? Stack-overflow protection > cannot work in GC, so you are shooting yourself in the foot by > creating such large recursive structures. By the time we get to GC, > where the problem will happen, it's too late, because the memory was > already allocated. > > Does anyone has a reasonable idea for avoiding the crash in such > programs? I would appreciate any effort to fix that, because it seems that currently streams are broken by design, and there is no way to fix that from the Lisp implementation. Yes, we could implement iterators instead of streams - that's what we get when we avoid the consing. But it's something different and not always an alternative, depending on what you want to do. Michael. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 28 08:20:56 2018 Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 13:20:56 +0000 Received: from localhost ([127.0.0.1]:36594 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er1fE-00068q-Bs for submit@debbugs.gnu.org; Wed, 28 Feb 2018 08:20:56 -0500 Received: from petton.fr ([89.234.186.68]:54004) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er1fC-00068a-0n for 30626@debbugs.gnu.org; Wed, 28 Feb 2018 08:20:54 -0500 From: Nicolas Petton To: Michael Heerdegen , Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' In-Reply-To: <877eqx1ije.fsf@web.de> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <877eqx1ije.fsf@web.de> Date: Wed, 28 Feb 2018 14:20:42 +0100 Message-ID: <87lgfdgsid.fsf@petton.fr> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail; t=1519824047; bh=QVJE2Hb70cRVPaubh5SkfPcFC94tDbC9dN4Qi21ophg=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type; b=WnmSe/ZLvi1EdJVngoRzUZ24AtXAiJN+60+4b8f/9i8ksB3WnyEqG8qT0DsTvu5jWSCpB8mUB+9dB9J52VEoBbGfyZso6/68guFTdz9Qvj5WcO84IY4iLIPE/AmHJpL3rJ0Lb3daUEwI2QrEOSR5pZ04rPdi0CDOYYMcTjgjhMk= X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --=-=-= Content-Type: text/plain Michael Heerdegen writes: > Hello, Hi, > I had written that > >> > (seq-doseq (_ (stream-range 1 1000000)) nil) > > crashes. CC'ing Nicolas, the author of stream.el. Thanks, I'll look into it. Cheers, Nico --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqWrKoACgkQ6LzXhmr8 +XgcqQgAzgY9bRiXsLRNLB+SxUyuC/b4+hHlfkaYzRq6M9gw5X1slBHNaiTJErpM Hf9NWnw8HCQ2qCrcyLQrrvYnQ9OgPJic1nxf4znNWHhH/y+ryKpNhKloCwyuzRVW 44z6TFIM5tsowpYoYeY9yjSNlGu/VOxAu55wPX6gMPnNwsxyZoonzxOsgwehej/F snJTG2EUlUTwVp0FRRl1qES+OUvZ3QGTPRX0r9kMNfjFj36UH0jCaETRTvh76uMR 3WM949/8AuPNfH81V4zmpjgH3z+1ODwoXmBtpBJcV09hqs457502BnlENCOE2pK5 z+P6t3oSy3KMSTZGyFfzBWVaycN/nA== =mII8 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 28 11:00:06 2018 Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 16:00:06 +0000 Received: from localhost ([127.0.0.1]:37686 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er49G-0001eb-8T for submit@debbugs.gnu.org; Wed, 28 Feb 2018 11:00:06 -0500 Received: from eggs.gnu.org ([208.118.235.92]:42348) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er49C-0001d2-Rw for 30626@debbugs.gnu.org; Wed, 28 Feb 2018 11:00:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1er492-0006dq-OJ for 30626@debbugs.gnu.org; Wed, 28 Feb 2018 10:59:57 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59666) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1er492-0006dh-LR; Wed, 28 Feb 2018 10:59:52 -0500 Received: from [176.228.60.248] (port=3444 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1er491-0007wm-Vu; Wed, 28 Feb 2018 10:59:52 -0500 Date: Wed, 28 Feb 2018 18:00:00 +0200 Message-Id: <83h8q1yuin.fsf@gnu.org> From: Eli Zaretskii To: Michael Heerdegen In-reply-to: <87bmg91ity.fsf@web.de> (message from Michael Heerdegen on Wed, 28 Feb 2018 11:58:49 +0100) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: npostavs@gmail.com, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Michael Heerdegen > Cc: Eli Zaretskii , 30626@debbugs.gnu.org > Date: Wed, 28 Feb 2018 11:58:49 +0100 > > > (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST) > > Yes, that's exactly how it's implemented. When requesting more elements > from the stream, that becomes > > (FIRST-VALUE . > (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST)) > > When we loop over the string, the cons whose car is the FIRST-VALUE, > let's call it cons1, is immediately thrown away, and we continue with > > (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST) > > etc. How do you see that the car is immediately thrown away? > AFAIU the problem is that the cons1 still exists in memory until garbage > collection kicks in. When that happens, the cons1 points to a largely > recursive cons structure, though this structure is never referenced from > Lisp in this form. What is "cons1" in this context? From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 28 11:21:03 2018 Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 16:21:03 +0000 Received: from localhost ([127.0.0.1]:37713 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er4TX-0002B1-3V for submit@debbugs.gnu.org; Wed, 28 Feb 2018 11:21:03 -0500 Received: from mout.web.de ([212.227.17.12]:47611) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er4TV-0002AA-2w for 30626@debbugs.gnu.org; Wed, 28 Feb 2018 11:21:01 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103 [213.165.67.124]) with ESMTPSA (Nemesis) id 0Lh6M7-1eKlv70MxL-00oa3f; Wed, 28 Feb 2018 17:20:54 +0100 From: Michael Heerdegen To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> Date: Wed, 28 Feb 2018 17:20:53 +0100 In-Reply-To: <83h8q1yuin.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 28 Feb 2018 18:00:00 +0200") Message-ID: <87po4pnl0a.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:Ohahmb63bDqc7lJ9F6Fsbm80aC/ngB0OcuC0WGRBvRADeWZDWfP InjDCSJwUoKwxkUrUTMLiZxgklaH3wrPyOz1qieA86r1EqYXlrcAlVrHSVj0xwUOQRfTpjm w1x0ai9u5caXOg6nrcekOn8Nve3yim4LtTHC5QhdRsulyx5s7FPuN80r15ng9mFeiEMvAoI gkg3rDcHrAvWKhzw7yt0A== X-UI-Out-Filterresults: notjunk:1;V01:K0:O7gTCDHOFlw=:O8LtxidCs/Yr/pJOSMYZ+S f+5g/57N+32P6GyqmX5CZXiLbTWSAQVTdxiPwN5YvD3BQEuZSbq0P58wxxQ0DiiKGU0ixh63y TUEGWba6hr8U012vMXWZKxMemREPwUBomY/J5TjV1nBZhpg/6hVnr5Q2F6U+oU2OUeInj+lHS cZYdS4JAnhmxA8vOxgHNCch2W3IGCTEhDnbnuCxjYK22c7xnxBHh4y7yMCPFQxUSj4avzd+ju 283gf+NGVhb5ZMFSy/WRTHOkmeL2q1dQiYjD++XeBFNrugVs5nNpPyJuRqscfbFdZ1q4il+lb /jfFdtDAYzsqXY0sz6mVd8RTKK5Cd9/xqC1AZEOFhkObY12zJAnvQqkKNTUia8rTwgdwFEWqh eIvxhHJk5veBXr5U7J8zjbc5+a0Yanw9hS/5r1McUgYBwbEus18vNZOoGe0sNemaDyF7LBklS cqNKrGKE1vaHKJrCpNuy9AZmHWdeXXG/ncOSvQ+vv8qJ7G9enumcgiFmVscxZcKBJOyHGiIJo CvQgauXzGSHH3vr2hQBhWta/t22QHZkRcbd32EDkYHjfxjZ1HzVgsxSnHK18zO131oP7Edeq1 7YbfsNuHpGhfAlVDKiRf/P7LM8Tt+EopYti2oXy1XWw9UDHxTg1ppXuXGCNksSm4UTYA+SrVL QGwbVRVGIEYuvrS5teomlf7r54wWiq5UPyEIgyOQdwz0AutD+zC2wloddjBChLtCY+YrnHusX Io3TA3TKo8TD/IMSK27S/8OGoBgTMvot7/S/oNfhlpErb11vaG4UsTA6BHyW4EMjewO0Fd0jD FPVw/4UCdSuiJIG+5YDXYVtpJ8jTLE3plpRtFJGAQxY2rQrbKM= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: npostavs@gmail.com, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Eli Zaretskii writes: > How do you see that the car is immediately thrown away? After expansion and dispatch, this is what gets executed: #+begin_src emacs-lisp (let ((stream (stream-range 1 1000000))) (while (not (stream-empty-p stream)) (funcall #'ignore (stream-first stream)) (setq stream (stream-rest stream)))) #+end_src Creating an element and throwing away the outermost cons alternate. > > AFAIU the problem is that the cons1 still exists in memory until > > garbage collection kicks in. When that happens, the cons1 points to > > a largely recursive cons structure, though this structure is never > > referenced from Lisp in this form. > What is "cons1" in this context? I said it some lines above: I defined it as name of the "first cons", i.e. the original stream. The return value of `stream-range' in the above example. Michael. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 28 12:23:01 2018 Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 17:23:02 +0000 Received: from localhost ([127.0.0.1]:37745 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er5RV-0003bb-BV for submit@debbugs.gnu.org; Wed, 28 Feb 2018 12:23:01 -0500 Received: from eggs.gnu.org ([208.118.235.92]:34823) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er5RT-0003bK-Fp for 30626@debbugs.gnu.org; Wed, 28 Feb 2018 12:22:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1er5RK-000087-8e for 30626@debbugs.gnu.org; Wed, 28 Feb 2018 12:22:54 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33348) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1er5RK-00007y-52; Wed, 28 Feb 2018 12:22:50 -0500 Received: from [176.228.60.248] (port=3566 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1er5RJ-0004Mq-Fm; Wed, 28 Feb 2018 12:22:49 -0500 Date: Wed, 28 Feb 2018 19:22:57 +0200 Message-Id: <837eqxyqoe.fsf@gnu.org> From: Eli Zaretskii To: Michael Heerdegen In-reply-to: <87po4pnl0a.fsf@web.de> (message from Michael Heerdegen on Wed, 28 Feb 2018 17:20:53 +0100) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: npostavs@gmail.com, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Michael Heerdegen > Cc: npostavs@gmail.com, 30626@debbugs.gnu.org > Date: Wed, 28 Feb 2018 17:20:53 +0100 > > Eli Zaretskii writes: > > > How do you see that the car is immediately thrown away? > > After expansion and dispatch, this is what gets executed: > > #+begin_src emacs-lisp > (let ((stream (stream-range 1 1000000))) > (while (not (stream-empty-p stream)) > (funcall #'ignore (stream-first stream)) > (setq stream (stream-rest stream)))) > #+end_src > > Creating an element and throwing away the outermost cons alternate. I don't think it's thrown away from the POV of GC. But you can easily see what is going on if you trace the GC on the C level. You should be able to see which object causes recursion in mark_object. I didn't look long enough, but what I did see looks very much like the entire unwound stream. From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 28 13:25:55 2018 Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 18:25:55 +0000 Received: from localhost ([127.0.0.1]:37823 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er6QM-00075X-Ta for submit@debbugs.gnu.org; Wed, 28 Feb 2018 13:25:55 -0500 Received: from mout.web.de ([212.227.17.11]:47525) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1er6QL-00075H-A1 for 30626@debbugs.gnu.org; Wed, 28 Feb 2018 13:25:53 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103 [213.165.67.124]) with ESMTPSA (Nemesis) id 0LcPWs-1eOsfc0tz4-00jt5s; Wed, 28 Feb 2018 19:25:46 +0100 From: Michael Heerdegen To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> Date: Wed, 28 Feb 2018 19:25:45 +0100 In-Reply-To: <837eqxyqoe.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 28 Feb 2018 19:22:57 +0200") Message-ID: <87lgfdnf86.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:FQOoquDeG4pc/xAgxhTQ+VVq2u7gGBHnFkClfxYgWmEVuFFO/R9 qGgJaD9VT2yFVYHaSGE6mFKZf51li+x5GsGLS7opNr5LRUAIsmofd6IZqlOFWVnaMS5iXuo CnN9eeauaCB3yR4NNMDTSs4kmWwOydqBoY5+rbEWOtc07wAvqvO0ESSubip1nwXyK41Emm+ lgh5fv7zsVziXkDUiM3sw== X-UI-Out-Filterresults: notjunk:1;V01:K0:wbTGeAldK6k=:TN5Noghk3nCdfVvZxzU2nY +44piVPayzpMlqAZ+NnR52oUGfqzszrFNI2h4y1lV7Uq6dlEA1OErERe4vkHENsCWIX0xK+s/ ZPVN1noaU4PzKSzLmMAp9bZmSkDKXMxGYf1DGJi0WI2ZNg7n0Yu4+rvPNCHuQY890ZTxQk4EA etCvceIpq4/oQgZtscjFFakO89ct31MNj+ytwSQ/laRYcBpkzJBApVjuuF0EINUGOtfks/BaW C2SZsNZIPon1RZXbxmlE+mmVWmLMu04xLXHoYsOhNaM2kZjgRaourf1AQJMan8DS/mLaf0vii p2H3Ya1BVdPmENYeUPevv+f9GqPeau4BhPZGC79+ECt0axPLNOOcl2+16Mb/9DsKuynT1m9eO Yxeeqyg7x9GAMZIjdTDUQdsCVFcaPR3+TKUpZaK5MURCQuiE+cjrqYrqeNXuY3oEcTWJsp/Ln hHGuVsVRzKgLyiqJx/T7x09yd8p4A0+ER4v/T3XYkaq2o/V7Sl44fWpQvYQwUWGusAMlql8JJ g0Mn6SrIMfzH/9yvaMYMA0M4HiSU8a3RBl01LBzP409YDEjuMWM7rK7VQWsMKxU6t0a4DFTR1 j41CZAPXv1AeKsaU9N9hv1ItFCK4B9mS4+IfObH249nM/e09fIrBBXugFIcDuMqB+rNvlsYjQ T1gW9/0uRYzhaYhOymYGwNVdvJc8HAxhobDsC4tsDj0xc+9vm5oI0AEDfcsEccMqQClISGGeg MPuc7V/yhjAgN1jEl9L6vbBbMIs3BpdbsZEkJBxpyxMFNqqfK+Hoz/GM/7xbryTLAbWlhUcz0 2fgpTsb6nIvgh2u1rRxyFlQ1C/QRBScgDqo2vZzA2R67ZivJh4= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: npostavs@gmail.com, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Eli Zaretskii writes: > I don't think it's thrown away from the POV of GC. But you can easily > see what is going on if you trace the GC on the C level. You should > be able to see which object causes recursion in mark_object. I didn't > look long enough, but what I did see looks very much like the entire > unwound stream. I have an idea what could be going on. In the stream-range example, this is how the stream is build: #+begin_src emacs-lisp (list stream--identifier (let (forced val) (lambda (&optional check) (if check forced (unless forced (setf val (progn (cons start (stream-range (+ start step) end step)))) (setf forced t)) val)))) #+end_src The inner `stream-range' call results in a closure, and I guess that this closure includes a reference to the outside VAL, which is the stream from one step back (though there isn't a lexical reference to the variable...does that make sense?) So there could be a chain of references via closure variables back to the first cons. Michael. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 01 05:45:06 2018 Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 10:45:06 +0000 Received: from localhost ([127.0.0.1]:38255 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erLhw-00005r-T4 for submit@debbugs.gnu.org; Thu, 01 Mar 2018 05:45:05 -0500 Received: from dancol.org ([96.126.100.184]:50764) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erLhu-00005j-Sg for 30626@debbugs.gnu.org; Thu, 01 Mar 2018 05:45:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject; bh=47QOjUHy9W4RZVnnFpob9g/wL0IisNhoraMPdnKwYx8=; b=EKGrQ9UcEqHk9Pr+BMTmoF9v/n+/nqogSBWcd186s4GxOMGPZugy0ajJsTQQ+s8D2D2p9pEHWrZ/PT6uMInuzzrh4ydbTnQAsoe5rbu32zLi8QIyXspqezqB3//dY76JiDqhaLjJiuLb4LCfE2p+0ntEYh8LPHMSjQwEqNs3mzYTisP4uHK4dLrcuUVTA9WweHL+WGBa9tVW5BiceBh4ON5MsElyfx6KoiT5byfnpwewg4I6UFjwpk7sVOyg1POGtGwRHZvpv+/djkEaNjkuBR704RTZq3kqZ34mVFfpbMprEgqq5tD3xEHsIEJ8Zk/RcfBH0QafHaEmjpfCXLRjlw==; Received: from [172.92.145.124] (helo=[192.168.86.27]) by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1erLht-00015c-7W; Thu, 01 Mar 2018 02:45:01 -0800 Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' To: Eli Zaretskii , Michael Heerdegen References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> From: Daniel Colascione Message-ID: <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> Date: Thu, 1 Mar 2018 02:44:54 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <83zi3uz4nb.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) On 02/27/2018 10:08 AM, Eli Zaretskii wrote: >> From: Michael Heerdegen >> Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org >> Date: Tue, 27 Feb 2018 13:08:59 +0100 >> >> #+begin_src emacs-lisp >> (seq-doseq (_ (stream-range 1 1000000)) nil) >> #+end_src >> >> Note that this is executed as a loop due how to streams are implemented, >> although the definition of `seq-doseq' looks recursive. But it seems >> that gc has a problem with the large number of conses created when >> processing that. > > What can we do instead in such cases? Stack-overflow protection > cannot work in GC, so you are shooting yourself in the foot by > creating such large recursive structures. By the time we get to GC, > where the problem will happen, it's too late, because the memory was > already allocated. > > Does anyone has a reasonable idea for avoiding the crash in such > programs? We need to fix GC being deeply recursive once and for all. Tweaking stack sizes on various platforms and trying to spot-fix GC for the occasional deeply recursive structure is annoying. Here's my proposal: Turn garbage_collect_1 into a queue-draining loop, initializing the object queue with the GC roots before draining it. We'll make mark_object put an object on this queue, turning the existing mark_object code into a mark_queued_object function. garbage_collect_1 will just call mark_queued_object in a loop; mark_queued_object can call mark_object, but since mark_object just enqueues an object and doesn't recurse, we can't exhaust the stack with deep object graphs. (We'll repurpose the mark bit to mean that the object is on the to-mark queue; by the time we fully drain the queue, just before we sweep, the mark bit will have the same meaning it does now.) We can't allocate memory to hold the queue during GC, so we'll have to pre-allocate it. We can implement the queue as a list of queue blocks, where each queue block is an array of 16k or so Lisp_Objects. During allocation, we'll just make sure we have one Lisp_Object queue-block slot for every non-self-representing Lisp object we allocate. Since we know that we'll have enough queue blocks for the worst GC case, we can have mark_object pull queue blocks from a free list, aborting if for some reason it ever runs out of queue blocks. (The previous paragraph guarantees we won't.) garbage_collect_1 will churn through these heap blocks and place each back on the free list after it's called mark_queued_object on every Lisp_Object in the queue block. In this way, in non-pathological cases of GC, we'll end up using the same few queue blocks over and over. That's a nice optimization, because we can MADV_DONTNEED unused queue blocks so the OS doesn't actually have to remember their contents. In this way, I think we can make the current GC model recursion-proof without drastically changing how we allocate Lisp objects. The additional memory requirements should be modest: it's basically one Lisp_Object per Lisp object allocated. The naive version of this scheme needs about 4.6MB of overhead on my current 20MB Emacs heap, but it should be possible to reduce the overhead significantly by taking advantage of the block allocation we do for conses and other types --- we can put whole blocks on the queue instead of pointers to individual block parts, so we can get away with a much smaller queue. Under this approach, the reserved-queue-block scheme would impose an overhead of somewhere around 1MB on the same heap. This amount of overhead seems reasonable. We may end up actually using less memory that we would for recursive mark_object stack invocation. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 01 06:25:54 2018 Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 11:25:54 +0000 Received: from localhost ([127.0.0.1]:38262 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erMLS-00013F-48 for submit@debbugs.gnu.org; Thu, 01 Mar 2018 06:25:54 -0500 Received: from mout.web.de ([212.227.17.12]:39465) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erMLP-000131-R4 for 30626@debbugs.gnu.org; Thu, 01 Mar 2018 06:25:52 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MEmgA-1exOjq0v1O-00G584; Thu, 01 Mar 2018 12:25:44 +0100 From: Michael Heerdegen To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> Date: Thu, 01 Mar 2018 12:25:43 +0100 In-Reply-To: <87lgfdnf86.fsf@web.de> (Michael Heerdegen's message of "Wed, 28 Feb 2018 19:25:45 +0100") Message-ID: <87muzs11hk.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:ZA3UyJrqrPYXyGGH6nkUim5onRakIxD7uo7ZlAFiMl5lZVvqLkI gwNnEZ4gzk4nNZ2vIkiD56E+Xe68ymt5YphYli7fEbLmX1b/LpO9MLG7FnVl2Mdi68m1OQ5 6jLVq+/3B36Xt0SPyfke/8l0mQMQJn6T8zs9XM+dnkflUKuEQ5Do3sqkdYT/dru9WEjn2PT ZDDnVQu/ll6NbwdoW0bog== X-UI-Out-Filterresults: notjunk:1;V01:K0:pixoC40cy78=:1khWpvoXDTEKOgOQZBuVQr 3XbXhZNRqFzyTfb4NwgsGwJSF1zaf+YPHgR1lD16ex7Vo1FWY2k2kQ7QPq33oKszVngVYE/jQ PPMtQU2FYt+5T/x7et5E+XSB2rc1hhw4Ky9/n0Lo1AItd1LZ15kQOF9dLiPAfzMXV3f3ZFYf+ 1Hy8nN5MPRlnyA8iSrQwBLUtWWVGBmhLHWAyE/A4fQQp09fW9NhtWpLNwP6w5q2sy7TLxAE9B gTVYZQOftPJjoKM2xkku8+OX3osXmwMcuZjDnVJFWfOuYkpSCE8HhPIZkQBEVyhSs4s6H+toV 5zC8jjjj34ny5PqaHn0zZ5AZXSRLushH8jXnw30QiH8+0/G1rgVj2GkTktNlC1zKMGRz3O22Y WlzT9RKR0Y8n3Fqey/DHwwbEB4JT7zoYxqMUTE1Zn9Q4q44SDHHoHmg9sBKzl1Nw918139lrP nW+z9vMucDSW7fO5mH69JG48rK1T1dtZZ5gklnF9wL5ZUNAujLskh3GKzDQyVsfyiFqweyu1Z iU5dwaElCznD4EFWA9I7ER14IrSKPzzaUKdVitBvO21iW80zdxEksTYaVUeYkVB6jETJrjf69 NXYeLCFuMhqTL6FPzNf1pXgZSf5ttBDC943/K6wJAJ31vbVvJlPkUevW3g9lL0fQZ32KSFzbi aKx3VllAVLUbcV2OsfI8yj8PXY4FTl3UX+CprFbhwEb3wyn70MU1qpkByskiHwZ4jSZRvh/sI 9JPoACVNoYpm/6mztu+BMiXRuwxw0iN4j+s7rc7lodof6U8bG7mA7irBt2p6/zFwgvwc5ZeNk ZDmWEH1bSoPvCtJQCSd8JUWxs+6sofIaVlY7VNcjC+3+q+7rlY= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: npostavs@gmail.com, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Michael Heerdegen writes: > The inner `stream-range' call results in a closure, and I guess that > this closure includes a reference to the outside VAL, which is the > stream from one step back (though there isn't a lexical reference to the > variable...does that make sense?) This is probably only a part of the puzzle. Some examples: test1.el: This is more or less the `stream-range' example reimplemented without dependencies so that it works in emacs -Q: #+begin_src emacs-lisp ;; -*- lexical-binding: t -*- (defun my-range (start) (let (forced val) (lambda () (unless forced (setq val (cons start (my-range (1+ start))) forced t)) val))) (defun my-test () (let ((range-object (my-range 1)) current-cons) (while (< (car (setq current-cons (funcall range-object))) (* 1000 1000)) (message "%d" (car current-cons)) (setq range-object (cdr current-cons))))) (my-test) #+end_src Loading this file test1.el crashes Emacs - but if you compile it, loading the compiled file doesn't crash. This is what I expected from my previous thoughts, because only the uncompiled closures include a reference to the outer VALs. test2.el: #+begin_src emacs-lisp ;; -*- lexical-binding: t -*- (require 'stream) (let ((stream (stream-range 1 1000000))) (stream-flush stream)) #+end_src This traverses the whole stream ignoring the elements. Loading this file crashes Emacs, no matter if compiled or not. I'm surprised it doesn't work even when compiled. test3.el: #+begin_src emacs-lisp ;; -*- lexical-binding: t -*- (require 'stream) (let ((stream (stream-range 1 1000000))) (while (not (stream-empty-p stream)) (cl-callf stream-rest stream))) #+end_src This is semantically exactly like test2.el, only the call to `stream-flush' has been replaced by literally writing out the definition. Nonetheless, the compiled file suddenly doesn't crash Emacs when loaded. Loading the uncompiled file test3.el still crashes. Seems that whether we get a crash or not depends on details in the implementation of lexical-binding. Michael. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 01 10:00:37 2018 Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 15:00:37 +0000 Received: from localhost ([127.0.0.1]:39444 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erPhE-0001rB-IG for submit@debbugs.gnu.org; Thu, 01 Mar 2018 10:00:36 -0500 Received: from eggs.gnu.org ([208.118.235.92]:50466) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erPhA-0001qw-H5 for 30626@debbugs.gnu.org; Thu, 01 Mar 2018 10:00:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erPh1-0000lH-A8 for 30626@debbugs.gnu.org; Thu, 01 Mar 2018 10:00:27 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54060) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erPh1-0000ku-6Y; Thu, 01 Mar 2018 10:00:23 -0500 Received: from [176.228.60.248] (port=4586 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1erPh0-0004G8-JZ; Thu, 01 Mar 2018 10:00:23 -0500 Date: Thu, 01 Mar 2018 17:00:33 +0200 Message-Id: <83k1uvyh66.fsf@gnu.org> From: Eli Zaretskii To: Michael Heerdegen In-reply-to: <87muzs11hk.fsf@web.de> (message from Michael Heerdegen on Thu, 01 Mar 2018 12:25:43 +0100) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: npostavs@gmail.com, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Michael Heerdegen > Cc: npostavs@gmail.com, 30626@debbugs.gnu.org > Date: Thu, 01 Mar 2018 12:25:43 +0100 > > Seems that whether we get a crash or not depends on details in the > implementation of lexical-binding. Byte compilation doesn't just produce byte code, it also changes the code into an equivalent one, but "equivalence" in this context doesn't include side effects like stack usage. As an extreme example, consider a tail-recursive program that the byte compiler converts (or at least might convert theoretically) into a loop. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 01 10:51:39 2018 Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 15:51:39 +0000 Received: from localhost ([127.0.0.1]:39492 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erQUd-00031u-Cu for submit@debbugs.gnu.org; Thu, 01 Mar 2018 10:51:39 -0500 Received: from mail-ot0-f182.google.com ([74.125.82.182]:45940) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erQUb-00031g-8D for 30626@debbugs.gnu.org; Thu, 01 Mar 2018 10:51:37 -0500 Received: by mail-ot0-f182.google.com with SMTP id f11so5952623otj.12 for <30626@debbugs.gnu.org>; Thu, 01 Mar 2018 07:51:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=TRJxXkM1tXJ+dLfVZFaZMeVWzYo5CKqeawx7lr1svJw=; b=jaRWJrFFxbCIh+Z4GLspNH7BN80O79E0Qva3svqwW9WjdTJUGetp6Xn2mkXJ8PVMgB d91BQytRUsCsQ9VNvcF6SaWArTF4wIpNFgClo6ayfITufoR+cFv2sDAREy5jB2AmuX5A fosfppQ9VZhkFyzi0auheExhOqGuB8w52h6BS4CGCZ6HTwVqaj1NcdkHJiXCiA5BiueS KvUi2aeJKEnEEC63YwVD3l5hr3M9s6yrLRWt291jVIXJ/XbGJXLkzhgj3vUKjcEC+qyA 0xRCHCIifB5anMfnE+UTi+Fri87kuGmkleNkeQUdsFkv4TRmigFxsAzNQXZtTC3M+dOE 2rHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=TRJxXkM1tXJ+dLfVZFaZMeVWzYo5CKqeawx7lr1svJw=; b=NupoKZEa8ncUvm4FX53xlnYthM6uazjB3DiMMD+xEuY9bdaB0uvfM6LLPgBEcMfEOr sjlR+SwpMLjWCxZczBbfQ6He8pVDwfS2hOZpfTwIw/osDFJ/nT9R2o6qD4dMi6M6oPd2 2oHsUdR20KW6GYkkqeyAXA8zmCKNB5fXXKmYpNpPDB0dpAy1xVIa01Di8wCvSos84QTa TpONcK/B+Qecm9cc+1Y+1z0NGf0x+nioT1rBVsrLK9qNMXV33db/RePqEqY4bApkQU01 zhkwyYdkfwqg6qws5+O05X9dKClXbfYD7shw+RmDYeMrDEJRsghYlCeu5NHfnsabi/7q h/RQ== X-Gm-Message-State: AElRT7E0is54+D2yYiyhHbmYZZfQR7LX1XsPl2wL9d97/zur/CcuM9sm BNFoQY1+1YBaJmwZjz0C96vqM4sWh9ySMNv16mc= X-Google-Smtp-Source: AG47ELt6wXBqzVdgeIcgQ8zHTN/aDmwqXD4B3/UkxQl8iDguLmcebvGrORNfpH4Y89SfEcSy+Fn+ALwuNcpgQgn2UcI= X-Received: by 10.157.47.178 with SMTP id r47mr1569664otb.335.1519919491455; Thu, 01 Mar 2018 07:51:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.74.59.65 with HTTP; Thu, 1 Mar 2018 07:51:31 -0800 (PST) In-Reply-To: <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> From: Noam Postavsky Date: Thu, 1 Mar 2018 10:51:31 -0500 Message-ID: Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' To: Daniel Colascione Content-Type: text/plain; charset="UTF-8" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Michael Heerdegen , Eli Zaretskii , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) On Thu, Mar 1, 2018 at 5:44 AM, Daniel Colascione wrote: > We need to fix GC being deeply recursive once and for all. Tweaking stack > sizes on various platforms and trying to spot-fix GC for the occasional > deeply recursive structure is annoying. Agreed, but could you please open a new thread for it? AFAICT, this bug thread is about streams.el functions producing structures proportional in size to the length of the entire stream, which is a bug in itself, regardless of whether or not the GC can handle the result. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 01 11:55:18 2018 Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 16:55:18 +0000 Received: from localhost ([127.0.0.1]:39561 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erRUE-0006To-6K for submit@debbugs.gnu.org; Thu, 01 Mar 2018 11:55:18 -0500 Received: from mout.web.de ([212.227.17.12]:46719) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erRUC-0006Ta-QA for 30626@debbugs.gnu.org; Thu, 01 Mar 2018 11:55:17 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb103 [213.165.67.124]) with ESMTPSA (Nemesis) id 0M5wzV-1eTiZO0FyL-00xqC3; Thu, 01 Mar 2018 17:54:59 +0100 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> Date: Thu, 01 Mar 2018 17:54:57 +0100 In-Reply-To: (Noam Postavsky's message of "Thu, 1 Mar 2018 10:51:31 -0500") Message-ID: <87muzrg2hq.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:yPftk7d2EJsspEQ1VzbZAiK8W3WYDB0tMe0SjnK7mAx8VApwXk2 Yy6IY5XbyrPsDeaVZ8ZQs9AXskrPkdqI8hWzu8fivqJq/1/IWWg16hZIsmq8AVf/bj8nC91 QaEYsxKN+HOYHpZ+tH4/uoBjbZz2AMZSnmWMHszPIU9XjoDvAodis6U/Gnq6mLU1NM10pfV kjC8oZYUwOdJl4TGw/hiw== X-UI-Out-Filterresults: notjunk:1;V01:K0:if2sPxsvh1Y=:jmJUn1S5k8cJ3VsZfiISA5 LBPjdqqPmNHQxBxxCqhu41CzIiuEYeY2508ncofIOrlifpxyiGlFpVsPoK8evUpzjHYoJgU+5 AFHUU1jBibr2humfYinJydnOMVCAfftSPOUcyuAxxw9JW8sVEb8cMI6MTv+HFaE/kRjlLiwji ar1SNevn1n3T5wnVnFhQMsWqafc13E0Zac+0KJdBqSRNWrDBn4hRs2cA5NvlJIzcwYNetC/HW WYBquIW4FG5DF31lOMCwOlmB5hIXcT5eZRs1U3zcjuWYy3wX169TXvdwZCg1R9H8h0+KL/QGP pFfnBp3LO20Ofx7/YOXjWQfaKFMwyTOzMu0E2cJmb1DlKNGj5fJ3WqlCGn14FvNAQ5h5G+GFw xq3GEhw7c2w7roYJffcWKh/qtaRwqGDcIKTe7LcF77+7gRYlTynEbBiPjk07SyPJfe3Ij9SvD gsGOCOwSMGk7QczLJznXNMj5xJ9bMLxdgYxekcCUZqTLAevY7sFUcObQG9wAM3UVFTmFsmt2I RZADnY0dnGB0MeY9PBFvo5tyHx8TuE2R50FsiEtFnfQLR8yFqwfFzXoNFhuXHRdZfxc45i+NZ lB64z7SCi0pL9+dxuDdoMjWeo5jOS8QYZXSCM84JjPSHmppXTHmXMWmXZZ4orejPJnkY2O02C /gfAtSBYkJfja2OVJPMLRzelXu5JgoHHwpD2Br/j6DBwAgnyIMrUaZ2LOAwHTtPvTqIO/A+gI x9R9c/Knf/afOnLuLq1/tSfG6dZvbilapz5ZbbKkpc6Ps3WnFtwReYh2T5dMNppvAaTCJrAmO 70POWhd1CrKoOO2Uii/ixYO4MO5djxAaXmtk5eagTLpv+08sdc= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , Daniel Colascione , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Noam Postavsky writes: > Agreed, but could you please open a new thread for it? AFAICT, this > bug thread is about streams.el functions producing structures > proportional in size to the length of the entire stream, which is a > bug in itself, regardless of whether or not the GC can handle the > result. No, it is a feature: we want streams to produce these structures, streams are delayed lists, not just iterators. Lists are also potentially unlimited nested conses, and that's not a bug. Streams are the same realized with delayed conses. Michael. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 01 12:15:51 2018 Received: (at 30626) by debbugs.gnu.org; 1 Mar 2018 17:15:51 +0000 Received: from localhost ([127.0.0.1]:39571 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erRo7-00071d-Cr for submit@debbugs.gnu.org; Thu, 01 Mar 2018 12:15:51 -0500 Received: from mail-ot0-f181.google.com ([74.125.82.181]:46008) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erRo6-00071O-6T for 30626@debbugs.gnu.org; Thu, 01 Mar 2018 12:15:50 -0500 Received: by mail-ot0-f181.google.com with SMTP id f11so6227625otj.12 for <30626@debbugs.gnu.org>; Thu, 01 Mar 2018 09:15:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=KOLq+21D6dCWMJ5tcp6BoMnKTCUue4G7LPcxu//TEjk=; b=U7euIbGT8ezO1Hp7mWWqDElrZ4JrbNPShyiayDoDvtdY9fiFcmmDNLetlm/eMUVml2 CD5i0OWXtP9P7jOfi+tWM9G416OHU1IvRAaB8LdB2mmCx5xEEuEuSUl4oygpr1TsKGNJ qbO87DIASh6Bz/aFTgyRNiRb/l3k3scT6MjS+fKBOGF0dhyrv1N03ZaiaHz2bn7lJUyR IoT/UdYxwV+pnv5D6MwZs3ZM8wZpMpOU2NWJlGiKaGrdfudeliw813v7ATXEZ5cTGqK/ bqSCYlGWol9AQ2D+sb0TiUscjP4sorCwzP5854vRS38wzI5e8X9sJKGBO4aa70vGS/lf QbHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=KOLq+21D6dCWMJ5tcp6BoMnKTCUue4G7LPcxu//TEjk=; b=T6oq6IEsY0XfACJY8pWSf3uNwEMiXYA1CtKl140iQOTjB8DScjool04IhpbIIdtbPd BaX5R9dit4z+wQvOTbLB934Vphb5I/DCFjSz/+4zQ/uYCfFbvy5o3q04Sm7oDMcPvSBJ xM+Jgl5M/Dlv/ELn21fxFeEBAuhY7pSEHFZR2LJojKORKx7MUK7OihipE+zzXLX6FRtc kQoATXNNDHzMlju/Jvqx06k+rBypm1nGsHcPhugFZzAOSQFoTswg3qxIN6xDfTT3ngW2 z4Yk6gjtf6fzqrn2tXFCmzIrbM0RUWJASp8wnSvrQ53N3okXRMFB0pNeANFS8MgR4Cun 1CoQ== X-Gm-Message-State: AElRT7FdHpPWT9wHSTLkKVWaxhvCgZ6xRqvg+ebD+2BYq1VjHawPIc24 iHSGP3lNsk72Zgs9zB6EfBaz9qLosB690TErJUU= X-Google-Smtp-Source: AG47ELtmQ34KbyS6Z0HTqfKg8PgSEYjQTxHOXB33weUD2Z7PvaujV14gMxvpt6QB2zoUtVQ+UcT5enEUlCtHEv73/lM= X-Received: by 10.157.29.218 with SMTP id w26mr1813973otw.24.1519924544545; Thu, 01 Mar 2018 09:15:44 -0800 (PST) MIME-Version: 1.0 Received: by 10.74.59.65 with HTTP; Thu, 1 Mar 2018 09:15:43 -0800 (PST) In-Reply-To: <87muzrg2hq.fsf@web.de> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> <87muzrg2hq.fsf@web.de> From: Noam Postavsky Date: Thu, 1 Mar 2018 12:15:43 -0500 Message-ID: Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' To: Michael Heerdegen Content-Type: text/plain; charset="UTF-8" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , Daniel Colascione , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) On Thu, Mar 1, 2018 at 11:54 AM, Michael Heerdegen wrote: > Noam Postavsky writes: > >> Agreed, but could you please open a new thread for it? AFAICT, this >> bug thread is about streams.el functions producing structures >> proportional in size to the length of the entire stream, which is a >> bug in itself, regardless of whether or not the GC can handle the >> result. > > No, it is a feature: we want streams to produce these structures, > streams are delayed lists, not just iterators. Lists are also > potentially unlimited nested conses, and that's not a bug. Streams are > the same realized with delayed conses. Maybe I've misunderstood, but is it not the case that iterating over (stream-range 1 n) should require only a constant amount of memory, regardless of the value of n? From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 02:09:16 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 07:09:17 +0000 Received: from localhost ([127.0.0.1]:40031 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ereoe-0004DN-MC for submit@debbugs.gnu.org; Fri, 02 Mar 2018 02:09:16 -0500 Received: from mout.web.de ([212.227.15.4]:43147) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ereoc-0004DA-Jg for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 02:09:15 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb001 [213.165.67.108]) with ESMTPSA (Nemesis) id 0M9oW8-1eyDKL2iQy-00B33Q; Fri, 02 Mar 2018 08:08:57 +0100 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> <87muzrg2hq.fsf@web.de> Date: Fri, 02 Mar 2018 08:08:55 +0100 In-Reply-To: (Noam Postavsky's message of "Thu, 1 Mar 2018 12:15:43 -0500") Message-ID: <87fu5jeyyg.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:TcsKnI4YUJlrmbQHDHf/m1ZQH2l6yFnjF2srPHAjjejXlVhgu9g By45ZiN9GuwnzYcwfkhC1qMkPk3pcmBlbw268XGIJLgQygoWn5dezhTWP6nN4PZDOASdD8s VmsKYzBti2SAUPTjHJFy2RJbQ0TUNotjxMtpfkOjvQA09CzRGAzKuN4/vpPJ51ABjsyqrB9 V+xzozGaa6Ci3nJibCSiA== X-UI-Out-Filterresults: notjunk:1;V01:K0:BAnyNq6AhdQ=:OcBQAAh0SlVJCscBw2wX5K xcALWDLlkSV6xSQkRKnefwOQvAWVvwka2yD8Nkn1bVA5/Zi0/sXygyDDrl9P0InUewhq7r33n FbaSdXE6NfQAwqsiszp0Bz2hjrVAd4rMc8H466Ch6Q1RJRVSRZmCdg70e2al52uIhkk/xgz2b BxuIJ9zBXAmPyPTor0/RNQpkUfGnoi2MNcsZLYh+Unnn1ASijL1FW3jS+Et1e9YPHDHUcA0fL zoq46j087QwEqbFhOyCtKTo+BdwdvrQHGE3+vCzkvJTC2IYGCnPd1PPxMxb6juuwP3eQOJ+MU EBt/cpyBCi/uBUp9db3/uRtpHBXkY2sP8Z3d/75OfgYSRV5L1OjzyLLesIAc7alw4TIlkCwVH Yy8iySVYAB3mYtR4cicXfxWa66rqdDTnkUS1uRF51GkVKaUtrdOd0s/90cERD3eEJKXuVviH0 tX08gcK6dIB5+Mg+F26kv6R/OwG7z9f0z0zkwxXUDuhoDWegXS2H06M7Lf4cetIAYNM+joa9Z c03hmmOEdsygQbMtAwSJu5AmzPkvPsiYW+3L+4bQAU9bW1NCYwCdh7jyZW6IWM0XS//aw0q/P q+zWiWaSELINRooZicjQUcfGo8SOuzbOsDEJcV1evV1+58SDDPOEAb9bYwnz24DTCa4wFgAR8 qUlB+RxxqMkIBl25IVVlQtCLjIGqMVSXfiDToE91Gxa/FPX5qqBenZ3xU2YJpr/uba7B4Ukcf 35XyN/M6XHRMANr799qXFA3Zar0hr3CdqF1dQ9p8Pa+Lakr2sQ06HLFTGB4cA+N2fATnisb6l 0z6sTjBrO4BM0zyk2DTQLtBr/n90SppmJFwLvXs6dS7aeY1280= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , Daniel Colascione , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Noam Postavsky writes: > Maybe I've misunderstood, but is it not the case that iterating over > (stream-range 1 n) should require only a constant amount of memory, > regardless of the value of n? Depends on your definition of `require'. Like, for example, (dolist (i 1000000) (message "%S" (cons i (1+ i)))) each iteration step creates and discards a new cons (or a constant number of conses). Not a exceptional thing in Lisp. When iterating over (stream-range 1 n), the garbage collector seems to have problems with how the garbage is structured. Michael. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 08:01:27 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 13:01:27 +0000 Received: from localhost ([127.0.0.1]:40228 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erkJT-0007ay-Bd for submit@debbugs.gnu.org; Fri, 02 Mar 2018 08:01:27 -0500 Received: from mail-io0-f176.google.com ([209.85.223.176]:34433) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erkJS-0007Us-7L for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 08:01:26 -0500 Received: by mail-io0-f176.google.com with SMTP id e7so10552545ioj.1 for <30626@debbugs.gnu.org>; Fri, 02 Mar 2018 05:01:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=YmEavTgt1rdL9FdqjtV5K2D2HbFinXHIPeUDCwyhqCQ=; b=UStxfH5q48Z42aNUuenCp4aVPkitvVkNL7H6gVL0FiGjlrcTjLuxyVKzNyXAJ7f5rT Gpd2qZubzNA0SdBPypECjM57maNxFv7JRbR/8YvLUifbMVnUydNF5PIVclQF2FhNJtLR u9reQMWu5tvmXFvr3vPpbjxguq9wgoB3x+TdMFrA+BgvFk/pLUq5rZEMxxbTkOFAcZdf aLr+v0pXVLTnO/wgK87/sDc4+JwflvkUW7DIoki1dLMIwjFksBXBHh3VoBZusrEcGgwH D5w2i2+NV/BaVl16jd8xxDdRAuKbdfLY/jQ8mqWnE1FwmLdlZ3U/JY+NXW8sjLEoHkfS nU+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=YmEavTgt1rdL9FdqjtV5K2D2HbFinXHIPeUDCwyhqCQ=; b=WZEDTQwjspweakbq/VDdOJDZj1XKrOvHr53hD4jAB5ZyIuSD0tSSWEieQr9y6mE4L1 27lkIjnfrTfEzEjz0OPjw2vUUWWEMh1NgE9zgGyrqXwXX9mMzXZyFpfkh03gTQkD952H 8e1VhTU447i58H6nRHSekked3wUrj/2KjSaF331S8yYovw92LUntNINhdD47zFwa5NZs ZnVgz33qCBgqzz92wXUkhoWXvi/6t8Kv5SFowv3rB7SFLCL5LObrotNPic1xPDju7BpY VpS4YCXxx4t1T0velvuAG7V+LOQmImLqPb59ToxuAoqZM93q8CYotAChY4e/Py9SSZi5 Nuag== X-Gm-Message-State: AElRT7GPGLJuL1CpO5pcp8YOEjzLS6Jht5eu46uK+5fQ1qvkAHSkwBQ0 +eP4sagiNl8eaaIZor9qutA6kA== X-Google-Smtp-Source: AG47ELtdyf5b4JoyzIjI+Q5HQK34ZBQs+KQIWajDhqCHv/y8mJSgiFweQcs+wFNSoES+FO5NlU5uMg== X-Received: by 10.107.134.95 with SMTP id i92mr6332864iod.210.1519995680384; Fri, 02 Mar 2018 05:01:20 -0800 (PST) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id c91sm4425920iod.16.2018.03.02.05.01.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Mar 2018 05:01:18 -0800 (PST) From: Noam Postavsky To: Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> <87muzrg2hq.fsf@web.de> <87fu5jeyyg.fsf@web.de> Date: Fri, 02 Mar 2018 08:01:16 -0500 In-Reply-To: <87fu5jeyyg.fsf@web.de> (Michael Heerdegen's message of "Fri, 02 Mar 2018 08:08:55 +0100") Message-ID: <878tba4oo3.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , Daniel Colascione , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Michael Heerdegen writes: > Noam Postavsky writes: > >> Maybe I've misunderstood, but is it not the case that iterating over >> (stream-range 1 n) should require only a constant amount of memory, >> regardless of the value of n? > > Depends on your definition of `require'. Like, for example, > > (dolist (i 1000000) (message "%S" (cons i (1+ i)))) > > each iteration step creates and discards a new cons (or a constant > number of conses). Not a exceptional thing in Lisp. When iterating > over (stream-range 1 n), the garbage collector seems to have problems > with how the garbage is structured. Ah, so let me be more precise. Iterating over (stream-range 1 n) should require only a constant amount of *reachable* memory at any particular instant. So your example above is okay, but the following one would be not acceptable (I mean, it's fine if some random lisp code does that, but stream-range should not be creating such long lists, or equivalently large structures): (let ((list nil)) (dotimes (i 1000000) (push i list) (message "%S" (car list)))) From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 08:05:03 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 13:05:03 +0000 Received: from localhost ([127.0.0.1]:40232 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erkMw-0008Dn-R4 for submit@debbugs.gnu.org; Fri, 02 Mar 2018 08:05:03 -0500 Received: from mout.web.de ([212.227.15.14]:44443) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erkMv-0008DG-5d for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 08:05:01 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb004 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LyOuM-1egJqC1fks-015trc; Fri, 02 Mar 2018 14:04:54 +0100 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> <87muzrg2hq.fsf@web.de> <87fu5jeyyg.fsf@web.de> Date: Fri, 02 Mar 2018 14:04:53 +0100 In-Reply-To: <87fu5jeyyg.fsf@web.de> (Michael Heerdegen's message of "Fri, 02 Mar 2018 08:08:55 +0100") Message-ID: <878tbafx1m.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:cWXk8iPvsSVTcDvL0N/M8rcHMg7OiCp6R3Eg+I/2VMb/FxbRuPk 5ipAiVKAvQgYXAPnPgs6oqQ4NVF6pLOSlKQJyaUFAOOypoWP44KJ56ts5HgtA4tX1XOZUC+ 4E9gWm7mtjSm4sHNW3RfNJdxJmlSwI4EFMZnteZvxMTSWoVJXtyht3ffheHSrK615n99gDx DF0cYKUIIi6Ehi33J8MXQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:hd7cvNf9LOc=:wGZYuS8l5LA1aXWpkBP0vx wdv22GGa7Wxr0r+hm93jzJPuLgOOW+aGL+o4gbGgl0T8Vs/CFVcm/8hRDALkYfmJHDbz3/acq 07yxuoJoBTx0pkmEZR3YaCmwX1GuVlt/hLBtJsWgiuOqEMBaKqEBidn5CT0Nk/aGqmxKKyVYG 5wNXtykmWpRFi66D1i3102wtoMckxZqCE4hlKhNQbFZdIW0sWV2bRDXUZcHBMRr0o6KmGCWzA AI9uBQIaq5pd/o+QGC/3hvobKgYCgDimaOVDY+xtuEWZnXa0T+oagWXrlzKSKoHzMHYfnDICI B1vUuTunYdAoknkqGavdaXpAD2sLAuxx+zqJ9CoiQ507u8cznqLIfRklhWHab3Zmd/I/nbsrL WyxcS+EWdlWcVlTLBXr5Ujkb6Iwx1uHapLhZu1EsWX06FYQRWScFoBiXA9PEty+zRpHIsIeEb v/xYKL0Xu4tmhLuWzXgy1iL+yDx8XQU7GVQAGUazPsuG1RweFxqVloEg3cRf+jkJJhqMEhNto VbOoPmTybZGIbpAi+aMwnCZMSgqQ+LgTxfBVo5gHKg4A+nRBD5lRPXjCoPx0kJ6zV3Oi66qBv CnA6FRAq3xWEsr/8ibBcf3uNitK7QDQNg8HyTo9fTZVWFoB3d2a1MLtTlXWojG1QF807XFVDF V+MyPOMasBHmY1mBVPXICrHyYl34jFKwU8Eh97mMeOFXy849wj+dd9cYxlz6uMaUHlcbu/++P VdZ81CpOiBBvN17VhWt3s+/JhSDiimld1iYD0N3tKKIvaOIATtiKN/b+3ZLreBOXzAhltl2ND 2yH/tS8/ovUgJ1Dkcz5R7vuNjoOD9ymxX0X7mBdJ5+vt5rZnho= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Michael Heerdegen writes: > Noam Postavsky writes: > > > Maybe I've misunderstood, but is it not the case that iterating over > > (stream-range 1 n) should require only a constant amount of memory, > > regardless of the value of n? But in this regard, we have a problem with how lexical-binding is implemented for interpreted code. Nested thunks (as implemented in "thunk.el") accumulate useless variable bindings - e.g. (defun test () (thunk-force (thunk-delay (thunk-force (thunk-delay (thunk-force (thunk-delay (lambda () 1)))))))) (test) ==> #1=(closure ((check) (#:val . #1#) (#:forced . t) (check) (#:val . #1#) (#:forced . t) (check) (#:val . #1#) (#:forced . t) t) nil 1) The length of the variable list is equivalent to the number of thunk wrappers. I believe that these useless variable lists are responsible for the crashes of the uncompiled versions of the test files I had posted. I think this problem is different from the gc issue. Streams use nested thunks. Of course does thunk.el not explicitly add such variable lists to the result - this is how closures are built in interpreted code. For nested thunks these just add up. BTW, if you byte-compile the above `test' function, then (disassemble (test)) ==> byte code: args: nil 0 constant 1 1 return and this problem is gone. Michael. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 08:14:19 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 13:14:19 +0000 Received: from localhost ([127.0.0.1]:40269 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erkVv-0008TJ-Ho for submit@debbugs.gnu.org; Fri, 02 Mar 2018 08:14:19 -0500 Received: from mout.web.de ([212.227.15.3]:32991) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erkVt-0008T3-5w for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 08:14:17 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb002 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LnjNH-1eDT4Z3Ll1-00htRs; Fri, 02 Mar 2018 14:14:00 +0100 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <5d5ccb32-434a-cda9-67c4-c60abeb450df@dancol.org> <87muzrg2hq.fsf@web.de> <87fu5jeyyg.fsf@web.de> <878tba4oo3.fsf@gmail.com> Date: Fri, 02 Mar 2018 14:13:59 +0100 In-Reply-To: <878tba4oo3.fsf@gmail.com> (Noam Postavsky's message of "Fri, 02 Mar 2018 08:01:16 -0500") Message-ID: <874llyfwmg.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:bhxg7g7AFV7bzh2b+niKz0SiGo/IjU5NAPX8cOHBsl6XrgUap5c uJDNjM/3aEwu0G2juRGpXYapvqr50eIgTVwF5SqoMVh9GbwkLHTH8euhrUGw+VJG1K8liti kbPP78NleekpgXfGzSxc57Ekru2861cb9rFsnQQZYQQRNSz1X41ePTzoOcxvogXn9aJY8Ft n4CqAVaZ0fpo06BSnrvRg== X-UI-Out-Filterresults: notjunk:1;V01:K0:0GJg6FnsdjM=:hrkjCuZBmuhWW7blc0dYtz KzSWnia32oxOGC71qfMiKnyuvLKVLwvyGlLlln8ZD4H7vV/wCLETamKo2AFpeB9nxJ06aq2ph 7TsaRh6NWBFAKD7wnC8wZoruKrlHC+6cbVsdQTA5RUNeWBxBpxpDPB2M8ZWeQEP5YECqQFj2e YM7vvYD0M4PUbTgkmD7LwroN8VfE0LzDCINZ7wLUNJvWj6Qsx3t0+f4k6GtMU3mtoUONZaBOW nGBP1BYo49M/XMmufoRURIIqQdYSiLpFZzE3qalLMWKDa5vkPxWBQU7SZz0aotIGaxCEiTb5S uSuQpOtm9VYgy1xo5aBCnL4EaTRbW5vnxWtt+gfCnTliU3HhycXBhxlfkBoUFtDlbHqAxM8iZ 4tSGfAe4Im/7JUq5Fq6oGiQ0XPTLR6b7eR2N6ZpIUeJUkHUkjkQWcqb/tZ1UDgFuBKmWLjgC0 EPOC6sy2t71Y9FN3S1poBzZmn7VyprQDAF/+jdXIQYCEn34HXw6ArXdfC+NpxxgkuS/rYQUn5 Pu2SHF4z+CbXpg3xqU7cPloriQUYoRyS2tvT6tqcO4qKh27VhwLvRnGMO8zrfT+TkZKRIbi3T OrqUGOVAFeaCtbpj0MyawCns3LBKsVQuI4CTZYF72CzZNb8JD+6Ce/YChILi4CfckWUwp4JXi lAVfuXLa5+hYKrksP9sU8MZtgbJe5k3ztoEkYirfgsHV/gajkc4NGOED4kFCVzC47wfonn/Nz FHeLzDGKvfEPU3C2pfWPxP8khOKvCORZcDVhmib0a5uN6H/uDtsXjNEnYoIheTK6nGjBDeZ+N TpqXIvqbxKOZ1OGNIXYlqa58gihtNo/gtS9gB7Jca/oSKMmG7E= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , Daniel Colascione , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Noam Postavsky writes: > Ah, so let me be more precise. Iterating over (stream-range 1 n) should > require only a constant amount of *reachable* memory at any particular > instant. So your example above is okay, but the following one would be > not acceptable (I mean, it's fine if some random lisp code does that, > but stream-range should not be creating such long lists, or equivalently > large structures): > > (let ((list nil)) > (dotimes (i 1000000) > (push i list) > (message "%S" (car list)))) Yes, absolutely agreed. Using streams, you can logically refer to the complete list of elements as one object, but the programmer must ensure that the referable list of generated elements doesn't get too large. Michael. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 09:11:54 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 14:11:55 +0000 Received: from localhost ([127.0.0.1]:40312 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erlPe-0001P9-Nj for submit@debbugs.gnu.org; Fri, 02 Mar 2018 09:11:54 -0500 Received: from mail-it0-f50.google.com ([209.85.214.50]:34644) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1erlPd-0001Ov-LI for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 09:11:53 -0500 Received: by mail-it0-f50.google.com with SMTP id n128so2142822ith.1 for <30626@debbugs.gnu.org>; Fri, 02 Mar 2018 06:11:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=lHPt3axmMeemcCSiyLyeUQOwJu9A0/wp1ogB44e7ADI=; b=IW0X5TM/sgI/QKtfdFXTIAwYtgZSDsDiH2YLm7vbJyUG4Dv5dR0GT8v3AyXcggTl3s hTWwv0q5zXkl0Bq7olytpv5laQPwt7AA7pxacNZherNnSfFUuhOX6bAG2kcqTjiswrtC 8Tto1DNxMudPXBev4Loet2ADuP1grhE+5p2H2OhVhTOZkJRhS9FwSTkxYvhjcuxBKvr9 Txa4Z/6g2Jttb3TW3GKHM3V6zTtvT0dE8yG3uZXCvYdKzYWfw2ECh/cSK8uReVN13VYr /MSGWPB+m+2YPzr/L/Zj5AvGg2WN7gPSlYPsnwSdg73SmGeMwkXEgpUHvf/eqGOa02Et beeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=lHPt3axmMeemcCSiyLyeUQOwJu9A0/wp1ogB44e7ADI=; b=B4w/fMOuHWEd8xQRp5Xma7xk6xzMpq2wkx2t+9drEmK9RveUImQ98gVmi5+WFLHCGx BNfiTqZS28AJAmN8nh0BlA0Inka3EJtSN8+X2QJXoYAj2ggFvdnRSd5SsPL78Cna24Om f0hU7G518BIrfhBl7k18+I+n27Q19ImRxEDXBgxFmegb5besIvb6Cv5YHwwj0HCJ3G43 /UgYqxvKKTDYantx9KOiF7+zlyCb7LSx8fl/ULB0lp87mlQGDu1jyEE377v5bV7wWp70 aDgG6t43dpyW3LyK3NEPkvUTaKoA3QrJn9BisADIMJkNSQxPmybldOIcO40uM4zeULcm gh7g== X-Gm-Message-State: AElRT7GJf3T7noOUBTjwgizokdKjK9UN07E87j2ZrK2YnRLdq9fk19P1 YTqSr/nSysWh6r37u6yvuLw= X-Google-Smtp-Source: AG47ELuKUlUGdjpbtSYPJin2WtBpXrPwFiH+KAuB7HTF5w1n1iT5engUK1YWokS+rRI6lmeOuGO+gQ== X-Received: by 10.36.55.70 with SMTP id r67mr2688149itr.40.1519999907945; Fri, 02 Mar 2018 06:11:47 -0800 (PST) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id 12sm875293itm.1.2018.03.02.06.11.47 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Mar 2018 06:11:47 -0800 (PST) From: Noam Postavsky To: Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> Date: Fri, 02 Mar 2018 09:11:46 -0500 In-Reply-To: <87muzs11hk.fsf@web.de> (Michael Heerdegen's message of "Thu, 01 Mar 2018 12:25:43 +0100") Message-ID: <87606e4lel.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Michael Heerdegen writes: > #+begin_src emacs-lisp > ;; -*- lexical-binding: t -*- > > (require 'stream) > > (let ((stream (stream-range 1 1000000))) > (while (not (stream-empty-p stream)) > (cl-callf stream-rest stream))) > #+end_src > > This is semantically exactly like test2.el, only the call to > `stream-flush' has been replaced by literally writing out the > definition. Nonetheless, the compiled file suddenly doesn't crash Emacs > when loaded. Loading the uncompiled file test3.el still crashes. Aha, but the following also crashes, whether compiled or not: ;; -*- lexical-binding: t -*- (require 'stream) (let* ((stream0 (stream-range 1 1000000)) (stream stream0)) (while (not (stream-empty-p stream)) (cl-callf stream-rest stream))) So the problem is that the initial stream0 object can reach the entire unfolding stream as it goes, and just holding on to that reference is enough to keep the whole thing in memory. Now, I can see that letting stream0 automagically get access to the unfolded result can be an optimization in some cases, although in this case it's a pessimization. It could also affect the semantics if unfolding the stream has side effects, not sure if stream.el makes guarantees about that though. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 10:07:09 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 15:07:09 +0000 Received: from localhost ([127.0.0.1]:41391 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ermH7-0002yQ-IY for submit@debbugs.gnu.org; Fri, 02 Mar 2018 10:07:09 -0500 Received: from mout.web.de ([212.227.15.14]:50735) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ermH6-0002y9-B9 for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 10:07:08 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb003 [213.165.67.108]) with ESMTPSA (Nemesis) id 0MSJH1-1fG8v31vEh-00TYNT; Fri, 02 Mar 2018 16:06:50 +0100 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> Date: Fri, 02 Mar 2018 16:06:49 +0100 In-Reply-To: <87606e4lel.fsf@gmail.com> (Noam Postavsky's message of "Fri, 02 Mar 2018 09:11:46 -0500") Message-ID: <87o9k6ms8m.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:mXgsQUGMor4X7g8Xtu6LZ2Cw+FVkMbVGg9PG2Iri+X+D5M41h1m 9Nud3A61rEX7eBV6V5aMTpt0gqoohzAGm2rilx373sBoJaLb4S75CGctxoCsa5E3G0j1wNL 2dBJ7iomA2g9tKGplUy/y34b5B+Db6qEmP+xjzxLTAakyEw94Ri3z11jw6uINPqgtGsLggU ZZ61sT8Ds0wgKNR/TU4Ow== X-UI-Out-Filterresults: notjunk:1;V01:K0:dEX4lT+N6XI=:zwQt9nETGEoCLYKFrDovrn nGoUVbeXgdJBsaNlGJokjlh9KZ2BRKgLdiUko9/mq4ezTNsbXJrfBr/KKgfeRKiNjmnGV3VBi seg8QbISjShgqbqj3Gxb7HfEJgzu1MrCBAWsoyILsDSEzRpVfyeQobbda6+jizwFSI4OetqwN 1G+tFXCb6Vy9VvFgKjqK7+7gtOScxdswizVcVOXN689I5v4/D+xTHxdTbfsuFNkhpyc+o6aQb i/AhACfH0cPfn6pbDYiTYUUoboDbOEMd/xRgmETrQlJFGBuY8Hxh9OoqGpShtEn7F8Mg39jW8 drNIplzncK+oMQNYMvuCw+yP4+3/WlNvIJhyoRrwZEM7Wk7rartsgZWvG5nsqTABVjob0pj1z VC9AK/aINdk+inVavLdfinGuDJjHYYnUVIqYlCyQ3u2DgKu1mxCd0/36dfW8DrFctkr4O34+0 WYLZgv5xwr4nONLoGXd4zouSXzcdtxd4Ay8H8DRyU1tcU7/4lb9rLxNr3HxF/qU19bv1cKrDp 11BHDvmv2JiHo/LnhiLWHSRkc/AvDFdG23qa4DeSDwlIGuSHtHUuVaMuGyj3tOOP5qk5UQ/eX f+1NjSxHNL3CaXoyekDWL5q8/ZPHuXxTlyjsyPcD7ra21+AXZUdZnn9y88f1AUtptiQbFQW7q 2dXYENcSqJWJ/V6K/89UhTvyQYr4WLZSVilx2mleC+n3i5nrySCgazAYpzixWTL0DOZir/TPO 083UDFdZNiL/azbbqj95/JqRun3vslsiTCWc/ZQqbB6GNSyTGF2B+n78MnIzHh+4rUCUe/y4a pRCWAuueOdxNe9PjsjFtaW6t2K/9xE+feFADHbzBLWXlITPlfY= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Noam Postavsky writes: > Aha, but the following also crashes, whether compiled or not: > > ;; -*- lexical-binding: t -*- > > (require 'stream) > > (let* ((stream0 (stream-range 1 1000000)) > (stream stream0)) > (while (not (stream-empty-p stream)) > (cl-callf stream-rest stream))) I guess you should have some awe if you write something like (stream-range 1 1000000) and keep in mind what happens with this gigantic thing. Though, could the object `stream0' references not already be garbage-collected when the loop has been entered, since there is no lexical reference to that variable there? BTW, it gets even worse if you append streams, since the original streams are not copied and magically unfolded when you generate elements from the concatenation. > Now, I can see that letting stream0 automagically get access to the > unfolded result can be an optimization in some cases, although in this > case it's a pessimization. It could also affect the semantics if > unfolding the stream has side effects, not sure if stream.el makes > guarantees about that though. AFAICT we make no such guarantees at all. When generating stream elements has side effects (which is not ideal, but it's not forbidden), then you must know what you are doing. In my experience, side effects often directly correlate with element generation - e.g. for a stream of search matches, a side effect is to set a variable to the position where to continue searching. These kind of side effects are not problematic. For non-trivial side-effects, dunno, never wanted something like that. But this problem also concerns other forms of delayed evaluation, including thunks, and generally everything you can do with closures. Michael. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 10:43:24 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 15:43:24 +0000 Received: from localhost ([127.0.0.1]:41435 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ermqC-0003qD-Hy for submit@debbugs.gnu.org; Fri, 02 Mar 2018 10:43:24 -0500 Received: from eggs.gnu.org ([208.118.235.92]:42103) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ermqA-0003pz-NB for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 10:43:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ermq2-0005CB-I2 for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 10:43:17 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59173) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ermq2-0005C7-Eh; Fri, 02 Mar 2018 10:43:14 -0500 Received: from [176.228.60.248] (port=3460 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ermq1-0007GF-Qo; Fri, 02 Mar 2018 10:43:14 -0500 Date: Fri, 02 Mar 2018 17:43:27 +0200 Message-Id: <837equwkio.fsf@gnu.org> From: Eli Zaretskii To: Michael Heerdegen In-reply-to: <87o9k6ms8m.fsf@web.de> (message from Michael Heerdegen on Fri, 02 Mar 2018 16:06:49 +0100) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87o9k6ms8m.fsf@web.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: nicolas@petton.fr, npostavs@gmail.com, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Michael Heerdegen > Date: Fri, 02 Mar 2018 16:06:49 +0100 > Cc: Nicolas Petton , 30626@debbugs.gnu.org > > > (let* ((stream0 (stream-range 1 1000000)) > > (stream stream0)) > > (while (not (stream-empty-p stream)) > > (cl-callf stream-rest stream))) > > I guess you should have some awe if you write something like > (stream-range 1 1000000) and keep in mind what happens with this > gigantic thing. Though, could the object `stream0' references not > already be garbage-collected when the loop has been entered, since there > is no lexical reference to that variable there? It's still referenced by the let* form, I think. But even if it didn't, you cannot rely on it being GC'ed, because Emacs triggers GC at times which are hard or even impossible to predict. From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 15:16:50 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 20:16:50 +0000 Received: from localhost ([127.0.0.1]:41558 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1err6o-0001b4-CX for submit@debbugs.gnu.org; Fri, 02 Mar 2018 15:16:50 -0500 Received: from petton.fr ([89.234.186.68]:45614) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1err6m-0001ap-Qr for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 15:16:49 -0500 From: Nicolas Petton To: Noam Postavsky , Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' In-Reply-To: <87606e4lel.fsf@gmail.com> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> Date: Fri, 02 Mar 2018 21:16:36 +0100 Message-ID: <87vaeefd23.fsf@petton.fr> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail; t=1520021801; bh=chxFAIhvtLFP0jp1QV2a47jvta88xwGqiavG0MfX0PI=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type; b=bJCNNn6mvu6fkPS7ilvQ5tr9VFnMDlXt4oEqvFD3O1p5PYL/ETSAZgCzuL50ptBNP3+JsFk9OMdUUga6t4mhMhQDQld/ewo6ywCYef7C78A3Al2y6MV8620Z+GCTr6iggPXo2QGDKXkXCw6pbPBvTxSHm3d/erJ9QD34hyxUaok= X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --=-=-= Content-Type: text/plain Noam Postavsky writes: > Now, I can see that letting stream0 automagically get access to the > unfolded result can be an optimization in some cases, although in this > case it's a pessimization. stream.el is at its core just an implementation of lazy-cons cells, so not letting stream0 get access to the result would mean changing the core implementation (I'm not necessarily against it). If we accept that `seq-elt', and other positional functions of seq.el should not work on streams, then I could rewrite stream.el to make it a positioned stream where previous elements are discarded after each element generation. However the list of supported functions from seq.el API would be significantly reduced. > It could also affect the semantics if unfolding the stream has side > effects, not sure if stream.el makes guarantees about that though. No, it doesn't. Nico --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqZsSQACgkQ6LzXhmr8 +XjObQf/bNEvOwmzGP7PM1qqjrIJcwBmzYi1s2FdhKp8TVjfOdtsc2Ovt+yBBnm+ 7OT9F1v9H6jyiQyzakcKi8O/TdtleZmmJQqGeGoWDOfJcQ0Ijwbtb7BwA7218bdT 9epzevOcmdc7XxQtQpf79SV+KSQq7OiHbAO4i2tBTU3h8QTtZzopXAfNVwpXaQsL BZBi4y36jpOZcHT18S81hAp9ADJRksnhAXkWSRyXBjiXzcigdB3Jf5NrSBNV/SEr F/uBHzD8RHG4rCHFVGzbv4CtDetgRwVRfGUEkl1IbHJyV9Ot3oMCY0GQB3mcGoOT fRf13nEhfGqlVFR1+JE6VydyVIDkQg== =r4Sx -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 15:58:56 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 20:58:57 +0000 Received: from localhost ([127.0.0.1]:41563 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1errlY-0002Yn-LT for submit@debbugs.gnu.org; Fri, 02 Mar 2018 15:58:56 -0500 Received: from petton.fr ([89.234.186.68]:55950) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1errlW-0002Ya-R1 for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 15:58:55 -0500 From: Nicolas Petton To: Noam Postavsky , Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' In-Reply-To: <87vaeefd23.fsf@petton.fr> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87vaeefd23.fsf@petton.fr> Date: Fri, 02 Mar 2018 21:58:43 +0100 Message-ID: <87sh9ifb3w.fsf@petton.fr> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail; t=1520024328; bh=pCuSUfM/JR20t69Ko1sc5/gGVnBXN+Zut5DaPsxviBA=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type; b=Hta5hPEjUqpNcrFoB4oYRWNJAiTkuffh45Mu9zcdc8txjqSrAAjaJg9O3osKm3hor128mWpUcH6VBpZwqzHAEBB/uqYTZXbF2xkeNrdrUjRTUnqHSk/+rgacvhVrxhTOfkNxJF2mH51AvHL8oZbTGzsei3bLFhX9iA2yUItOBhE= X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Nicolas Petton writes: > If we accept that `seq-elt', and other positional functions of seq.el > should not work on streams, then I could rewrite stream.el to make it a > positioned stream where previous elements are discarded after each > element generation. However the list of supported functions from seq.el > API would be significantly reduced. I had something like the following in mind: (cl-defstruct nstream current next-function) =20=20 (cl-defmethod nstream-next ((stream nstream)) (setf (nstream-current stream) (funcall (nstream-next-function stream) (nstream-current stream)))) =20=20 (defun nstream-range (&optional start end step) (unless start (setq start 0)) (unless step (setq step 1)) (make-nstream :current start :next-function (lambda (cur) (if (equal cur end) nil (+ cur step))))) Cheers, Nico --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqZuwMACgkQ6LzXhmr8 +Xh6vwf9GP46u+qqF2r5QY4/X4PK6TDmpvRnKBtsJ6g1Mw1x6E3ZXwrRSz10v9sK Dn9sexvOxJb8gqK6B5jqgbAQP0H0qGkW66liRR/Hls5It3rKZ9yHAGLFEEKaMMUp 90mfKFdSQzpyaeZLoBWIAVuoO30mcxq1DgpoJN+yJ3RkYgBz1Vbp4+eS3UI9OU2+ zzYA7SXeKBU64hzfjwrK0oF89cPvz4kyeRLSLkynFCEX/oRMiQyVWy1BSegAxXHI 3oVtpsqMc703rcioWdyla/JkulVSlhlmRVwPb2Nwf6ufy9IzEz9WhIKoBZAwWNlp Zzn0DJWfH1khIwBjr25i48YtuOG8RQ== =DErk -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 02 16:48:51 2018 Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 21:48:51 +0000 Received: from localhost ([127.0.0.1]:41569 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ersXr-0003gV-Iu for submit@debbugs.gnu.org; Fri, 02 Mar 2018 16:48:51 -0500 Received: from mail-qk0-f175.google.com ([209.85.220.175]:45622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ersXq-0003gH-BP for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 16:48:50 -0500 Received: by mail-qk0-f175.google.com with SMTP id g2so13768810qkd.12 for <30626@debbugs.gnu.org>; Fri, 02 Mar 2018 13:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=EFpfmREqGYP06GiSZ4rWHh17p3ivcCfe3pTRKT5DfBI=; b=ZzL5yBjDsDgJ34syzAgv56Y/2OPhI2rLLHcniPjlbldqJRXOJjF7zynNRm0ywP2AyG 6VcU0tYzN7Lvw5N8lq75JSGO34zRQN7SdlX8MtL5C7kfW1Inq5qh5WPRMjxd5IWhuT1K TSlJ+fBcIDUbIc9pcX7/s8hisxqigM7PmQIfK+2hsAK6yIb+DiXReG6QjHCuJwiKoVNp 8uY/5S4KBCb9t+l0CqO/vhAC52/HAxQiayG9YDMQZc7SObRD8FVCdKepE6ZGcOgcDHmB cmqucT/LlbMe2RHN7N6jEwEB/AsHU82RJh/CtGZqHe4cvwgNlgkobA2uuex5DR7vuHuV PqBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=EFpfmREqGYP06GiSZ4rWHh17p3ivcCfe3pTRKT5DfBI=; b=E49jvqKaS+VdmW08M4dNOAyw1mRHJzC/g5xqdrKE1P3Bx4KhSmMkDMK0lrVtdQqSHy tdjPFCYncn4LmYMxtSCfVbF7l3DRZoI/mYXIhxLGzjrI0uV9sE0TTlkZTyhHxRDzn7RO x6/454i1I0Ua17VZuU+D10SZ8egMu2ziKthUQBAR26CNe+PZtdJhE/eztZOIU7ELg9pR K66X5BQnVeoZzk2+NDDiuvu/ZE6X2/Uwzua016l7gRHIKTgw/GbWe7TvfYcmmLnFxW6f Ktq+x+ULY0Y2W71qIBVcXijrcEUiDlkysPEhlIAK1OKzNwcSNxw4nc2GnkaDBkMD8n1m Gr5w== X-Gm-Message-State: AElRT7E7fvZUH8I+88KfisdWcVo3xhjqEZ4Gk5XSDoJm2U2I88U0OoD1 UMAnvK77QCvI5Fc0HUS+dD9a1djZo8eJlU6WxO8KSloc X-Google-Smtp-Source: AG47ELvC+LL7nH/Q27qTI11GnXmLnSZm1YnevYgCXL4Px/Qwec6/Eq46vGFh7o9FPLJeU6DUXC+1D8kYBKg0/5ZSEeE= X-Received: by 10.55.78.212 with SMTP id c203mr10596248qkb.351.1520027324330; Fri, 02 Mar 2018 13:48:44 -0800 (PST) MIME-Version: 1.0 Received: by 10.200.52.16 with HTTP; Fri, 2 Mar 2018 13:48:23 -0800 (PST) In-Reply-To: <87606e4lel.fsf@gmail.com> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> From: John Mastro Date: Fri, 2 Mar 2018 13:48:23 -0800 Message-ID: Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' To: 30626@debbugs.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Michael Heerdegen , Nicolas Petton , Noam Postavsky X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Noam Postavsky wrote: > Aha, but the following also crashes, whether compiled or not: > > ;; -*- lexical-binding: t -*- > > (require 'stream) > > (let* ((stream0 (stream-range 1 1000000)) > (stream stream0)) > (while (not (stream-empty-p stream)) > (cl-callf stream-rest stream))) > > So the problem is that the initial stream0 object can reach the entire > unfolding stream as it goes, and just holding on to that reference is > enough to keep the whole thing in memory. > > Now, I can see that letting stream0 automagically get access to the > unfolded result can be an optimization in some cases, although in this > case it's a pessimization. It could also affect the semantics if > unfolding the stream has side effects, not sure if stream.el makes > guarantees about that though. Clojure has/had a similar issue. They describe this scenario (having a live reference to the beginning of the stream, preventing GC from collecting it) as "holding onto the head" of the stream (in Clojure they're called lazy seqs). Their solution was what they call "locals clearing". The compiler tracks the lifetimes of local bindings and "clears" them (by setting them to nil/null) after their point of last use, e.g.: (let* ((stream0 (stream-range 1 1000000)) (stream stream0)) (setq stream0 nil) ;; <<< Inserted by compiler (while (not (stream-empty-p stream)) (cl-callf stream-rest stream))) If the code does reference stream0 later, locals clearing can't help you, but that's considered a "if it hurts, don't do it" situation. This probably isn't practical for Emacs, especially since it could only work for byte-compiled code, but thought the prior art may be of interest. John From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 03 02:54:13 2018 Received: (at 30626) by debbugs.gnu.org; 3 Mar 2018 07:54:13 +0000 Received: from localhost ([127.0.0.1]:41773 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1es1zh-0000NK-9M for submit@debbugs.gnu.org; Sat, 03 Mar 2018 02:54:13 -0500 Received: from mout.web.de ([212.227.17.11]:44135) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1es1zf-0000N8-UN for 30626@debbugs.gnu.org; Sat, 03 Mar 2018 02:54:12 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb101 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MK1s1-1etP2M1zwD-001VNg; Sat, 03 Mar 2018 08:54:03 +0100 From: Michael Heerdegen To: Nicolas Petton Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87vaeefd23.fsf@petton.fr> Date: Sat, 03 Mar 2018 08:54:02 +0100 In-Reply-To: <87vaeefd23.fsf@petton.fr> (Nicolas Petton's message of "Fri, 02 Mar 2018 21:16:36 +0100") Message-ID: <87k1utmw6d.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:DIMhMu9sNbyoXsZAiYoAAM2He6/zd4KOtTWvC1mYRmZgvmiEpqX UuPJlFkfFTuQ0OxbftkBQajNoT4SL8o1O+kNqeZxcmAewz4dTZd9PZo7Vx9yWWnQSI0pUDj mxOnBWM0bAryGrGryyGPJSnWWmnHnzBqQiqtHX6RUR03bf1nE3kWFGfkdnlGhiPYnnF22ha wxjIXmhnDN/C5AdyPnKRQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:GY/Jl4bcwaY=:C0KRnQXqd/DIMvXwe86/En BRrgFVsYVJ8ee4yRJMOdHZxJbjmucnmsZOukIkjCQZ+pxXdohkyENbASMBYAHT3kCKpDxKhyU oeBuNeOKn6I7pKMrepQdhF61pL2e3FkVo1xFcfvcMgwbpBmGHH3WmsazC4NljFiXcFHZy4eoo FHHRtoO8yg/uXyZG0yqdZwBTE3dI0Ww1JTk/Fbi5V5iUL554sDsok2wRyAkRrI/NnmPkAw8PH t/ec/O2O/BG4ms29XQ0Ba2JYVEE1j+MThzOp7plItZauPWqOVCWHRt4AgmPCgwD39CtquTG79 0dKg4UFobxn9Ex/Mn5BLlEKXMFPv0nTaiU77NM1s2PQjJ6tPrBZDScgaqXswHTi1lZD+/ObfH sFVWHpxUtFBnmOLZ4TmsoRx8WsBdJpOTGgOKiqRMbL3AW/1SFhA4Px9Nu+05vQqrsL6JRY2Hu aiYXfOPuZYTKZ5/UMvL+iUq7W3HhNsAwpmlbB5LiXtSAY8bUja4d+Uc+1x4F8cRM4w4yQ73lF 5zAMbIUs3h+CP1b2cLciKL8GP4WolkOMA4F1//Np8UH0cO1s0TXfaUrrukL+EiEDeCiV2FTeC Pi1ZU4uM6F1vmgV6MMVwXbT1vkyUafsYn3ZqKi+RfO1mtmmyyETx0CJ3PAgWnSiFPgmj1+hri 0EvX9h3nURhfXCXKqP15T5TL2RmPyoE1e5EQD/cMLWSLwVADWy/u6UOpbeA371JDc/a360tA0 nv5xF+ZPhOiTVw/9XO0/eBWg8euarHrdzi0O2ODhd2rwc6O4b1HIvtE6mx3yMUNaCnqWfPbyi Kt0FQ2F7HsBoQYQR0rAQt3rrOw2dc/jHQ2l0m5tvKq5OZhfg0Y= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Noam Postavsky , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Nicolas Petton writes: > stream.el is at its core just an implementation of lazy-cons cells, so > not letting stream0 get access to the result would mean changing the > core implementation (I'm not necessarily against it). Please don't do this. The semantics of streams is worth keeping. I make use of it in el-search and other stuff. Michael. From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 03 02:56:15 2018 Received: (at 30626) by debbugs.gnu.org; 3 Mar 2018 07:56:15 +0000 Received: from localhost ([127.0.0.1]:41777 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1es21f-0000QD-MC for submit@debbugs.gnu.org; Sat, 03 Mar 2018 02:56:15 -0500 Received: from mout.web.de ([212.227.15.14]:35665) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1es21d-0000Pz-Sx for 30626@debbugs.gnu.org; Sat, 03 Mar 2018 02:56:14 -0500 Received: from drachen.dragon ([188.99.169.170]) by smtp.web.de (mrweb002 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LpOKb-1eFvNY40ub-00fCeT; Sat, 03 Mar 2018 08:56:06 +0100 From: Michael Heerdegen To: Nicolas Petton Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87vaeefd23.fsf@petton.fr> <87sh9ifb3w.fsf@petton.fr> Date: Sat, 03 Mar 2018 08:56:05 +0100 In-Reply-To: <87sh9ifb3w.fsf@petton.fr> (Nicolas Petton's message of "Fri, 02 Mar 2018 21:58:43 +0100") Message-ID: <87fu5hmw2y.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K0:Rzp4CMhQkfV47snGTbmF6fcPxYF8dM/OBBaCT/DYZVjWlNyfkKv I/NUzewDcnYh1WPat/vN7bSMueL2ogfShPzMBA8sbQT9Uak8qktNJdxJVQhZnUQDBqwFDw2 QQk1b/kVclNmY73I9T/o45uky2Q6EhBMI7o546efwZdWeliW76XgBCPJsLLBjhiRbW/dt6/ +pU7cNdRt3aXeSNS+x4Ww== X-UI-Out-Filterresults: notjunk:1;V01:K0:+wcrWG0NSK0=:YgV8zoWeZVlKA6MEWfo+vc ImZDFSGueBliJbSa38VLdEaVlR2GRjJV+1GqTeg5WCCsk0sviYS+tTMgPDB/aasqkEYvprIWR AnUeLGhhiHfHN3chraQMoMODDgz2IOJPsw0KoEj7FOShUMC1YBum4UvZoHOImEBPWyZ84NM1y ZFpFh6JQs7guDDxf9Y8MFYBS4kp17QQEKMyQ0KzGNYWdZIjIEhf//WHwLZDpdfi3WelBoEI8e fAWz2ODibeVBTOKmFjffA6sFBk29w2QBa6lKDG0UnKaxmZHTbbhVdwiUMkE+CIUL5kAbziJEq i8b6y1RgIR+nNP3NAAL0p0r0oOY2nF8j/cHb61tH/6RdWA8/UGWO5blvIeZDdzp5Cf82tBwKO LgmBoeyQqgGccyilEgt9ZI4UcNZJnB1iNimIqBF3H6Q0zWqngXc1bg9wI/fRq9mRwBBiQ7zJN +rdrslWDmrn0SkJdG62NbDv02/pV/EkjMU0fzZc9DJDM/g+oBeytWJ9m/VebJxfpeKqFPsiqJ YBg8WjhtwsANH6m4Fkl3J8Ir7rOmutVOWaqmq1AkA3KhB1ICTN8GdsqmEDb7u5ZaClQ7bOyQM V56M0bH530Q3VrgwwrkW6OzPON/+T0Lc9dcVEtmoHF6KDDFpnWsAG1pkqRxSjiHsOk640SpO5 Uuf6MYSiTns0A8uXrKTDNY1SWgLXiv9WY7LNFQ9rZ4VS22mOhy40Rbt1Z5gQ6hJMzN3U+c/Al JsjbDYikGmiknv1lVYyKEuyIQ8HsZUy8CgZolHqbQ1hSZg75oWaK4SWzT4axloSkXQ0vIVXIi rHBuPPqGs9+92exFB9IMeapdA1iUv//bOOO8dkP1n+At1kG+y4= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Noam Postavsky , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Nicolas Petton writes: > I had something like the following in mind: > > (cl-defstruct nstream current next-function) > > (cl-defmethod nstream-next ((stream nstream)) > (setf (nstream-current stream) (funcall (nstream-next-function stream) > (nstream-current stream)))) > > (defun nstream-range (&optional start end step) > (unless start (setq start 0)) > (unless step (setq step 1)) > (make-nstream :current start > :next-function (lambda (cur) > (if (equal cur end) > nil > (+ cur step))))) This is an implementation of iterators, not streams. We already have an implementation of iterators in Emacs. Michael. From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 03 03:47:53 2018 Received: (at 30626) by debbugs.gnu.org; 3 Mar 2018 08:47:53 +0000 Received: from localhost ([127.0.0.1]:41812 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1es2pc-0001lC-Ti for submit@debbugs.gnu.org; Sat, 03 Mar 2018 03:47:53 -0500 Received: from petton.fr ([89.234.186.68]:36072) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1es2pa-0001kv-LV for 30626@debbugs.gnu.org; Sat, 03 Mar 2018 03:47:51 -0500 From: Nicolas Petton To: Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' In-Reply-To: <87k1utmw6d.fsf@web.de> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87vaeefd23.fsf@petton.fr> <87k1utmw6d.fsf@web.de> Date: Sat, 03 Mar 2018 09:47:41 +0100 Message-ID: <87po4lfsuq.fsf@petton.fr> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=petton.fr; s=mail; t=1520066864; bh=61xDQ6GPfIPUCdgRMLK9molyPFF4tGvL/Dm5Q/1YpYM=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID:MIME-Version:Content-Type; b=KgugRkGp0NJEWT+iLNGEYp8jZOoonUA4ZJtvExGc9je7hO6sI9yVrlk/iq0KgQF5/2fALLpMSu4XuYewSfUjfuFchZSr9SvOzrU+LDhE0PzGuZUV6wGC3DrpKYauYg277Bfl/o4Bbh/cZ0xhsnQUDYx/pSP56Ve0uB7xdRx8rnI= X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Noam Postavsky , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --=-=-= Content-Type: text/plain Michael Heerdegen writes: > Please don't do this. The semantics of streams is worth keeping. I > make use of it in el-search and other stuff. I'm not saying that I want this :) Nico --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEE1AWqLIYsVPF+7mvg6LzXhmr8+XgFAlqaYS0ACgkQ6LzXhmr8 +Xhh+wgAij2rvDtqdsxRxlgZCl+uCLAYzoJdMwkAbJPhm7ygxgULP4uBgQlXNIA3 2Ozp4IAWkcQhXPjnoGDcmoLvYyy4Wh08dTWyBpdkyI0mzdasf2LZqxVUkl0pMh0f mEbTcnSg4pMlWK28ZRHtKv6jx2oA3YWf4iBCNoYdaty1yqvtlHdxQJWY4rWsNYLq 7zHgTYH1vdSolGoWlSHbNJb8hEf5SaJQGsb39Z5ycmUABEMjdjKWILFXGcEGMCPz cmmKoTphsVo2sCAW/3VbiS7asvL69sAHw/k/2q6Lbs6nLnBxGwVacS5V8frXU8Gf mp3SUGWOmmNt1lWfXUcg+EveTnCg3Q== =6t13 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 03 18:00:10 2018 Received: (at 30626) by debbugs.gnu.org; 3 Mar 2018 23:00:10 +0000 Received: from localhost ([127.0.0.1]:43301 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1esG8P-00088f-Uw for submit@debbugs.gnu.org; Sat, 03 Mar 2018 18:00:10 -0500 Received: from mail-io0-f178.google.com ([209.85.223.178]:37514) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1esG8N-00087K-NX for 30626@debbugs.gnu.org; Sat, 03 Mar 2018 18:00:08 -0500 Received: by mail-io0-f178.google.com with SMTP id d71so14256626iog.4 for <30626@debbugs.gnu.org>; Sat, 03 Mar 2018 15:00:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=DUfm0OkrVRdeSHZoneFuVeZ5D0A9J3+UdOZKfnY5gpc=; b=sv566DU+GR4oXhDGdSoEJctKszSECUMmKKBw46a9bitk2rFD6vC2dvGuBE4eKekhbb zhDxsM86CeZI4uIQwem3DUAZ99rwn4kKHS09y9j5R8uCi4mGJpiXFBzggpZ0X/4mhRT+ UptKcDIz0Q2qKcM613F8U0cApIqAEX3dVXOQZjC80jQzJaaUvfqXQe4vwjjwy7x9Kqjn K4CTYtyCmK/QpEdUhGyrCDrB25WJ2AZrGM96nWPO8jWrmiJNA66pI0k8m72HafC4eUFY 1Unn/zM6y5JGF9TJZuIaN8RdGyHOVZrKI8A0FrkiPQ7L9kMkY/vsZS5HtctiWkL8pHJk Xkbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=DUfm0OkrVRdeSHZoneFuVeZ5D0A9J3+UdOZKfnY5gpc=; b=Su0nFojlQF1d7aH9fYMKuBHXbDVdwiG2R2Cq7hkmTH/udNtkSXIqnfORvU70dl0h5R Xys2dnQ2vwqwHEvNQCCCHJRcMb6e7eZJ4lDaOQw1f15P+G70MQ6RqPJVY7+2/Wn8XC+y n3CUdOxpKRAjO5J/EP5ve01jVaQnRVYRSW7outh6BY+dZHztcXp8cjNCWfm5dOVd4ZvW DyHJK/o7SQI3iO0b5ZdLN8qtmDFyHE23Fonmg4zcLmh78wu66fEyEZbQOY1EmXj1JN3l AyBSDy2/c07/zYp+bhwAk2gy/Geb0CmtfBJrUgSqcfsAtXIYuye2J2I9KH20jocxyybX VTuw== X-Gm-Message-State: APf1xPAH4kj3/oGh615aTf8mB5u9rEENp4cUrAnqtKLfl9mgE1gMREeE 8jyEADpU98PUPQALK/mu9+M= X-Google-Smtp-Source: AG47ELuMBJP1D2ev1ea4P+3rH/O/txVTJ9XeDRE4iffYGcwTOVM/7yY2vags7yTWB4EXZEBjajTlKw== X-Received: by 10.107.89.13 with SMTP id n13mr12302420iob.154.1520118002193; Sat, 03 Mar 2018 15:00:02 -0800 (PST) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id 33sm5973960ioj.71.2018.03.03.15.00.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 03 Mar 2018 15:00:01 -0800 (PST) From: Noam Postavsky To: John Mastro Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> Date: Sat, 03 Mar 2018 18:00:00 -0500 In-Reply-To: (John Mastro's message of "Fri, 2 Mar 2018 13:48:23 -0800") Message-ID: <87lgf83gun.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Michael Heerdegen , Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) John Mastro writes: > (let* ((stream0 (stream-range 1 1000000)) > (stream stream0)) > (setq stream0 nil) ;; <<< Inserted by compiler > (while (not (stream-empty-p stream)) > (cl-callf stream-rest stream))) > > If the code does reference stream0 later, locals clearing can't help > you, but that's considered a "if it hurts, don't do it" situation. > > This probably isn't practical for Emacs, especially since it could only > work for byte-compiled code, but thought the prior art may be of > interest. Not sure how doable this solution is, but the fact that it works only for byte-compiled code seems fine to me. The interpreted case is doomed to fail anyway since the interpreter doesn't prune redundant variables from closures. From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 04 10:56:45 2018 Received: (at 30626) by debbugs.gnu.org; 4 Mar 2018 15:56:45 +0000 Received: from localhost ([127.0.0.1]:44576 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1esW0D-0004El-34 for submit@debbugs.gnu.org; Sun, 04 Mar 2018 10:56:45 -0500 Received: from mail-it0-f46.google.com ([209.85.214.46]:34035) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1esW0B-0004EZ-Qh for 30626@debbugs.gnu.org; Sun, 04 Mar 2018 10:56:44 -0500 Received: by mail-it0-f46.google.com with SMTP id n128so6696261ith.1 for <30626@debbugs.gnu.org>; Sun, 04 Mar 2018 07:56:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=eyn57TeEDhnHwGD7zIvu7QkupeY0DBtiGRSMiKhHPPU=; b=czSEQfqZNGauRwWDRUTYpsmHXgie2hcGv+RngJFlFZgMgF9Ffj7iqzemiu6tliNMY4 re3C5C0b/kqxL2fUKN38t5Fz4zlRARqAkJZVVeygzkelrYdzG6z9aSXRBfHLCkrHpF4M 4Y4VTWNzfuOHta+vEsRuyYeaz4/x1/CX1qWNLL1smK8BcWuVPQS6CfFca4oxoLjOjj7q MjTndPe6dx/V8YoTCwYcCK7Vdc0w0wUPJ7meYCQ/SDN0TcNul03kviuZXkmHRtw9JNBw dnAeSUfJxVpxVu+EQ1XrF4bYWIQ1P+3yv77NlF7Jq236IRXOql6+/B0yyGof5j7nvow+ mW1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=eyn57TeEDhnHwGD7zIvu7QkupeY0DBtiGRSMiKhHPPU=; b=Xqg7/cp42WIFI9k7W2tIq4G65ovTKF9/l4Fb8HKobfLy/PFC3+dIpGzblddwU1cika YFu4peFa1v+1WVHT1aMq2wq3xIPlybBlVnvulGAx4+o/qobSOZllez066kGMqjEqCv0J KbQekoeaPUG3TD0WJEjoJlv3bJrneQm7eHBzqFt5KgvY6j5n8g3ZFFr4TZYzFKvbh/J2 YXx8+/QEbp6Tlb89A20wliZLR76k5mLt6QG+YT1PamJfzm799ALPSxNhbzO/2umDmFds GaWpckq3VIOra0S0TS08nyo/mfHt0hi1qDEO+UlOPUqloOgJ12sOifndf1uHtaJHnHL2 2EDQ== X-Gm-Message-State: AElRT7GlXAtMfyht2n9hYxCOl4rkxvqqvVnLUso28yavMkq9c3L5fSoc PIC/5fwKZM1kYiRfieaB8w8Q4Q== X-Google-Smtp-Source: AG47ELspReNFHR/AoJ8KOvTds/Ho+M/xnlSWvCk+tYln4F8erP6qI8bd65mOrh7oiGDXyKB8FIurxg== X-Received: by 10.36.121.195 with SMTP id z186mr10477649itc.148.1520178998116; Sun, 04 Mar 2018 07:56:38 -0800 (PST) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id h63sm3803821ioa.81.2018.03.04.07.56.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 04 Mar 2018 07:56:37 -0800 (PST) From: Noam Postavsky To: John Mastro Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> Date: Sun, 04 Mar 2018 10:56:36 -0500 In-Reply-To: <87lgf83gun.fsf@gmail.com> (Noam Postavsky's message of "Sat, 03 Mar 2018 18:00:00 -0500") Message-ID: <87fu5f3kcr.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Michael Heerdegen , Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Noam Postavsky writes: > John Mastro writes: > >> (let* ((stream0 (stream-range 1 1000000)) >> (stream stream0)) >> (setq stream0 nil) ;; <<< Inserted by compiler >> (while (not (stream-empty-p stream)) >> (cl-callf stream-rest stream))) >> >> If the code does reference stream0 later, locals clearing can't help >> you, but that's considered a "if it hurts, don't do it" situation. >> >> This probably isn't practical for Emacs, especially since it could only >> work for byte-compiled code, but thought the prior art may be of >> interest. > > Not sure how doable this solution is, but the fact that it works only > for byte-compiled code seems fine to me. The interpreted case is doomed > to fail anyway since the interpreter doesn't prune redundant variables > from closures. Hmm, I think it won't work by itself though, just doing (stream-flush (stream-range 1 1000000)) also crashes, due to the head of the stream being referenced from the C stack somewhere (I can get the address from gdb, but I can't figure out how to get to the corresponding C variable from there). From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 04 12:02:29 2018 Received: (at 30626) by debbugs.gnu.org; 4 Mar 2018 17:02:29 +0000 Received: from localhost ([127.0.0.1]:44639 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1esX1p-0007kZ-It for submit@debbugs.gnu.org; Sun, 04 Mar 2018 12:02:29 -0500 Received: from eggs.gnu.org ([208.118.235.92]:33156) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1esX1n-0007kM-Vf for 30626@debbugs.gnu.org; Sun, 04 Mar 2018 12:02:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1esX1f-0004Di-SD for 30626@debbugs.gnu.org; Sun, 04 Mar 2018 12:02:22 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:39337) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1esX1f-0004Dc-Nq; Sun, 04 Mar 2018 12:02:19 -0500 Received: from [176.228.60.248] (port=3419 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1esX1f-00082i-3f; Sun, 04 Mar 2018 12:02:19 -0500 Date: Sun, 04 Mar 2018 19:02:03 +0200 Message-Id: <83lgf7u644.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <87fu5f3kcr.fsf@gmail.com> (message from Noam Postavsky on Sun, 04 Mar 2018 10:56:36 -0500) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Noam Postavsky > Date: Sun, 04 Mar 2018 10:56:36 -0500 > Cc: Michael Heerdegen , > Nicolas Petton , 30626@debbugs.gnu.org > > Hmm, I think it won't work by itself though, just doing > > (stream-flush (stream-range 1 1000000)) > > also crashes, due to the head of the stream being referenced from the C > stack somewhere (I can get the address from gdb, but I can't figure out > how to get to the corresponding C variable from there). Did you try "info symbol ADDRESS"? (I'm not sure this will work for automatic variables, though.) You could also try "info locals" after "set print address on" and/or "set print symbol on". From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 11 14:52:33 2018 Received: (at 30626) by debbugs.gnu.org; 11 Mar 2018 18:52:33 +0000 Received: from localhost ([127.0.0.1]:55687 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ev65A-0003OR-Sk for submit@debbugs.gnu.org; Sun, 11 Mar 2018 14:52:33 -0400 Received: from mail-io0-f171.google.com ([209.85.223.171]:37087) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ev659-0003OA-DY for 30626@debbugs.gnu.org; Sun, 11 Mar 2018 14:52:31 -0400 Received: by mail-io0-f171.google.com with SMTP id d71so8913980iog.4 for <30626@debbugs.gnu.org>; Sun, 11 Mar 2018 11:52:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=jAwd9ts5z2FRPNc1JrEUPpVJo8HaPiVEH4B2S67eGN0=; b=LfYcUkA2Y1p8uK1yn9zqhTP4ft5H3h/rd6giROraU3+DQsdSGlpvmOoW7PkIDE7aRA r64OmDJZ964yfnQl92mV5yqXDAeJHIzoYWmNeGCIqjJo6skzsP5Yd/arJishRznc+pZM cgaJChrdJFwCtwLyZTcUqIgqzV4ZuiKyne8ixzrWj6ngijfv/Xc8fKnVd/xbCc67YhdR WfB0IbS6xtSA849b3EkCO1YHvTxI5sruALeCjZrmGcix3kjLseQk06bfOVEr2cZQjkg1 4Tw0ZgAmPaT3C1shi3at1houKhyhdZPzyRO4Q0TnkiWvywph2Fa7fZI18YPwZ3mllNWl 6M7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=jAwd9ts5z2FRPNc1JrEUPpVJo8HaPiVEH4B2S67eGN0=; b=MeILTuYLrzAYwmxvj1jH4AUAZNHaN+bOS9ZSu+jQqfVi5AXPo+SGmxKxAuu0jSkNXl 3QoWHPbr1UK0RO79TfFVkao8nfVzrEE5DRCnujck53Ji1cuOkGQG606e78zz5au1Fmhe UaR/cMxYDLJ5/IjqVQIM16MspUrYNMxViP8gaPZW4NSKoIE426xadnwTC683zj/1jI2I r9DAzubX2QIE7e9oJHieuUg2KEbtQ/V6ArQADoXq73NzlPsTI5IgQqEmxRU+BO9HIUKJ p2VxfjHBUad8ytb2llnFcaKJJwtsYTnw15osgpWbdkNtPdv1flrmtuFm00eIBAAtwsgE WE5A== X-Gm-Message-State: AElRT7ER1XjPGLd4w4FC4675LM1D6X87mPHn7GD50U7LmvO69wPm7ds7 mxvyLe4TcuE7+c+/hB4cw3scww== X-Google-Smtp-Source: AG47ELvXdKheNs94mALmGXvph4HZGY/Z7q7nzN29tWCrM6/x33eM4CEC8GHX2f442ILeV+rPZUpN/g== X-Received: by 10.107.183.65 with SMTP id h62mr6214180iof.278.1520794345456; Sun, 11 Mar 2018 11:52:25 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id q21sm2195593itb.2.2018.03.11.11.52.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 11 Mar 2018 11:52:24 -0700 (PDT) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> Date: Sun, 11 Mar 2018 14:52:22 -0400 In-Reply-To: <83lgf7u644.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 04 Mar 2018 19:02:03 +0200") Message-ID: <87bmfuzbq1.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) --=-=-= Content-Type: text/plain Eli Zaretskii writes: >> also crashes, due to the head of the stream being referenced from the C >> stack somewhere (I can get the address from gdb, but I can't figure out >> how to get to the corresponding C variable from there). > > Did you try "info symbol ADDRESS"? (I'm not sure this will work for > automatic variables, though.) Doesn't seem to work. I guess it wouldn't work if the address was in the middle of an array either. > You could also try "info locals" after "set print address on" and/or > "set print symbol on". Those settings don't seem to help. I first guessed that the problem is due to saving function arguments during funcall, so I tried the following to check it: --- i/src/bytecode.c +++ w/src/bytecode.c @@ -387,7 +387,10 @@ exec_byte_code (Lisp_Object bytestr, Lisp_Object vector, Lisp_Object maxdepth, make_number (nargs))); ptrdiff_t pushedargs = min (nonrest, nargs); for (ptrdiff_t i = 0; i < pushedargs; i++, args++) - PUSH (*args); + { + PUSH (*args); + *args = Qnil; + } if (nonrest < nargs) PUSH (Flist (nargs - nonrest, args)); else This did change the backtrace (from starting with mark_specpdl to mark_stack), meaning I did find one reference, but it still crashes, so there must be more. --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=stream-crash.gdb.log Content-Description: gdb log excerpts No stack. Starting program: /home/npostavs/src/emacs/emacs-26/lucid/src/emacs -Q -batch -l elpa/packages/stream/stream.elc -l bug-30626-stream-crash/test5.elc [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x000000000060e9b5 in live_cons_holding (m=0x2f6e290, p=0x2f77620) at ../../src/alloc.c:4613 4613 return make_lisp_ptr (s, Lisp_Cons); #0 0x000000000060e9b5 in live_cons_holding (m=0x2f6e290, p=0x2f77620) at ../../src/alloc.c:4613 #1 0x000000000060e9ec in live_cons_p (m=0x2f6e290, p=0x2f77620) at ../../src/alloc.c:4622 #2 0x0000000000612f94 in mark_object (arg=XIL(0x2f77623)) at ../../src/alloc.c:6717 #3 0x0000000000611d4f in mark_vectorlike (ptr=0x2f78b50) at ../../src/alloc.c:6227 [...] #4850 0x0000000000612b42 in mark_object (arg=XIL(0x2efcb83)) at ../../src/alloc.c:6624 #4851 0x0000000000611d4f in mark_vectorlike (ptr=0x2e64c90) at ../../src/alloc.c:6227 #4852 0x0000000000612b42 in mark_object (arg=XIL(0x2e64c95)) at ../../src/alloc.c:6624 #4853 0x000000000060f3ce in mark_maybe_pointer (p=0x2e64c90) at ../../src/alloc.c:4936 #4854 0x000000000060f452 in mark_memory (start=0x7fffffffa520, end=0x7fffffffe868) at ../../src/alloc.c:4985 #4855 0x000000000060f493 in mark_stack (bottom=0x7fffffffe868 "a\036h\364\377\177", end=0x7fffffffa520 "0\245\377\377\377\177") at ../../src/alloc.c:5193 #4856 0x00000000006cdd75 in mark_one_thread (thread=0xe103e0 ) at ../../src/thread.c:616 #4857 0x00000000006cdf0a in mark_threads_callback (ignore=0x0) at ../../src/thread.c:649 #4858 0x000000000060f4da in flush_stack_call_func (func=0x6cde77 , arg=0x0) at ../../src/alloc.c:5220 #4859 0x00000000006cdf3c in mark_threads () at ../../src/thread.c:656 #4860 0x00000000006114a7 in garbage_collect_1 (end=0x7fffffffa700) at ../../src/alloc.c:5997 #4861 0x0000000000611b94 in Fgarbage_collect () at ../../src/alloc.c:6168 #4862 0x000000000057999d in maybe_gc () at ../../src/lisp.h:4749 #4863 0x000000000063c2cb in Ffuncall (nargs=6, args=0x7fffffffa7f8) at ../../src/eval.c:2751 #4864 0x000000000068d950 in exec_byte_code (bytestr=XIL(0x2e7aad4), vector=XIL(0x2e72715), maxdepth=make_number(18), args_template=make_number(768), nargs=3, args=0x7fffffffad20) at ../../src/bytecode.c:632 #4865 0x000000000063cfef in funcall_lambda (fun=XIL(0x1450e45), nargs=3, arg_vector=0x7fffffffad08) at ../../src/eval.c:2970 #4866 0x000000000063c402 in Ffuncall (nargs=4, args=0x7fffffffad00) at ../../src/eval.c:2771 #4867 0x000000000068d950 in exec_byte_code (bytestr=XIL(0x2e7b2d4), vector=XIL(0x2fe2945), maxdepth=make_number(7), args_template=make_number(256), nargs=0, args=0x7fffffffb1d8) at ../../src/bytecode.c:632 #4868 0x000000000063cfef in funcall_lambda (fun=XIL(0x2fe2985), nargs=0, arg_vector=0x7fffffffb1d8) at ../../src/eval.c:2970 #4869 0x000000000063c402 in Ffuncall (nargs=1, args=0x7fffffffb1d0) at ../../src/eval.c:2771 #4870 0x000000000068d950 in exec_byte_code (bytestr=XIL(0x2e769d4), vector=XIL(0x9d00a5), maxdepth=make_number(2), args_template=make_number(257), nargs=1, args=0x7fffffffb670) at ../../src/bytecode.c:632 #4871 0x000000000063cfef in funcall_lambda (fun=XIL(0x1626f05), nargs=1, arg_vector=0x7fffffffb668) at ../../src/eval.c:2970 #4872 0x000000000063c402 in Ffuncall (nargs=2, args=0x7fffffffb660) at ../../src/eval.c:2771 #4873 0x000000000068d950 in exec_byte_code (bytestr=XIL(0x2e7f7b4), vector=XIL(0x1622fe5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffffbb10) at ../../src/bytecode.c:632 #4874 0x000000000063cfef in funcall_lambda (fun=XIL(0x1450f35), nargs=1, arg_vector=0x7fffffffbb08) at ../../src/eval.c:2970 #4875 0x000000000063c402 in Ffuncall (nargs=2, args=0x7fffffffbb00) at ../../src/eval.c:2771 #4876 0x000000000068d950 in exec_byte_code (bytestr=XIL(0x2ee3c94), vector=XIL(0x2e664f5), maxdepth=make_number(3), args_template=make_number(257), nargs=1, args=0x7fffffffbfa8) at ../../src/bytecode.c:632 #4877 0x000000000063cfef in funcall_lambda (fun=XIL(0x2e66515), nargs=1, arg_vector=0x7fffffffbfa0) at ../../src/eval.c:2970 #4878 0x000000000063c402 in Ffuncall (nargs=2, args=0x7fffffffbf98) at ../../src/eval.c:2771 #4879 0x000000000068d950 in exec_byte_code (bytestr=XIL(0x2e91054), vector=XIL(0x1699ef5), maxdepth=make_number(4), args_template=XIL(0), nargs=0, args=0x0) at ../../src/bytecode.c:632 #4880 0x000000000068cb20 in Fbyte_code (bytestr=XIL(0x2e91054), vector=XIL(0x1699ef5), maxdepth=make_number(4)) at ../../src/bytecode.c:321 #4881 0x000000000063ac83 in eval_sub (form=XIL(0x2eeb413)) at ../../src/eval.c:2240 #4882 0x0000000000672b34 in readevalloop (readcharfun=XIL(0x68a0), infile0=0x7fffffffc6b0, sourcename=XIL(0x2e90f94), printflag=false, unibyte=XIL(0), readfun=XIL(0), start=XIL(0), end=XIL(0)) at ../../src/lread.c:2038 #4883 0x0000000000670e85 in Fload (file=XIL(0x2e8d884), noerror=XIL(0), nomessage=XIL(0xc090), nosuffix=XIL(0), must_suffix=XIL(0)) at ../../src/lread.c:1425 #4884 0x000000000063c945 in funcall_subr (subr=0xd7a420 , numargs=3, args=0x7fffffffc8e8) at ../../src/eval.c:2856 #4885 0x000000000063c3be in Ffuncall (nargs=4, args=0x7fffffffc8e0) at ../../src/eval.c:2769 #4886 0x000000000068d950 in exec_byte_code (bytestr=XIL(0xadcc2c), vector=XIL(0xadcc4d), maxdepth=make_number(23), args_template=make_number(257), nargs=1, args=0x7fffffffd248) at ../../src/bytecode.c:632 #4887 0x000000000063cfef in funcall_lambda (fun=XIL(0xadcbfd), nargs=1, arg_vector=0x7fffffffd240) at ../../src/eval.c:2970 #4888 0x000000000063c402 in Ffuncall (nargs=2, args=0x7fffffffd238) at ../../src/eval.c:2771 #4889 0x000000000068d950 in exec_byte_code (bytestr=XIL(0xad7424), vector=XIL(0xad7445), maxdepth=make_number(21), args_template=make_number(0), nargs=0, args=0x7fffffffde58) at ../../src/bytecode.c:632 #4890 0x000000000063cfef in funcall_lambda (fun=XIL(0xad73f5), nargs=0, arg_vector=0x7fffffffde58) at ../../src/eval.c:2970 #4891 0x000000000063c402 in Ffuncall (nargs=1, args=0x7fffffffde50) at ../../src/eval.c:2771 #4892 0x000000000068d950 in exec_byte_code (bytestr=XIL(0xad6614), vector=XIL(0xad6635), maxdepth=make_number(12), args_template=make_number(0), nargs=0, args=0x7fffffffe480) at ../../src/bytecode.c:632 #4893 0x000000000063cfef in funcall_lambda (fun=XIL(0xad65e5), nargs=0, arg_vector=0x7fffffffe480) at ../../src/eval.c:2970 #4894 0x000000000063cc29 in apply_lambda (fun=XIL(0xad65e5), args=XIL(0), count=4) at ../../src/eval.c:2906 #4895 0x000000000063ae2a in eval_sub (form=XIL(0x149a8e3)) at ../../src/eval.c:2279 #4896 0x000000000063a19f in Feval (form=XIL(0x149a8e3), lexical=XIL(0)) at ../../src/eval.c:2054 #4897 0x000000000058150f in top_level_2 () at ../../src/keyboard.c:1119 #4898 0x00000000006384ce in internal_condition_case (bfun=0x5814ec , handlers=XIL(0x5250), hfun=0x580ef1 ) at ../../src/eval.c:1332 #4899 0x0000000000581554 in top_level_1 (ignore=XIL(0)) at ../../src/keyboard.c:1127 #4900 0x00000000006379d7 in internal_catch (tag=XIL(0xc6f0), func=0x581511 , arg=XIL(0)) at ../../src/eval.c:1097 #4901 0x000000000058143e in command_loop () at ../../src/keyboard.c:1088 #4902 0x00000000005809db in recursive_edit_1 () at ../../src/keyboard.c:695 #4903 0x0000000000580bd0 in Frecursive_edit () at ../../src/keyboard.c:766 #4904 0x000000000057e76b in main (argc=7, argv=0x7fffffffe9e8) at ../../src/emacs.c:1713 Cannot access memory at address 0x7ffffff7ef7f (gdb) frame 4854 #4854 0x000000000060f452 in mark_memory (start=0x7fffffffa520, end=0x7fffffffe868) at ../../src/alloc.c:4985 4985 mark_maybe_pointer (*(void **) pp); (gdb) info locals pp = 0x7fffffffa968 "\220L\346\002" (gdb) info symbol 0x7fffffffa968 No symbol matches 0x7fffffffa968. --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 11 16:31:17 2018 Received: (at 30626) by debbugs.gnu.org; 11 Mar 2018 20:31:17 +0000 Received: from localhost ([127.0.0.1]:55720 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ev7cj-0005yP-AN for submit@debbugs.gnu.org; Sun, 11 Mar 2018 16:31:17 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42162) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ev7ch-0005yC-QW for 30626@debbugs.gnu.org; Sun, 11 Mar 2018 16:31:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ev7cZ-0002UP-Fz for 30626@debbugs.gnu.org; Sun, 11 Mar 2018 16:31:10 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:45381) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ev7cZ-0002UH-D7; Sun, 11 Mar 2018 16:31:07 -0400 Received: from [176.228.60.248] (port=2034 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ev7cY-0006xq-Fr; Sun, 11 Mar 2018 16:31:06 -0400 Date: Sun, 11 Mar 2018 22:31:09 +0200 Message-Id: <83vae2s6b6.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <87bmfuzbq1.fsf@gmail.com> (message from Noam Postavsky on Sun, 11 Mar 2018 14:52:22 -0400) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Noam Postavsky > Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org > Date: Sun, 11 Mar 2018 14:52:22 -0400 > > This did change the backtrace (from starting with mark_specpdl to > mark_stack), meaning I did find one reference, but it still crashes, so > there must be more. If you have the address, you could first find the stack frame to which it belongs, right? Then go to that stack frame and type "info locals", which should give you the locals in that frame. One of them is your variable. It could also be a temporary slot, in which case disassembly of that frame's function should show it. From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 11 17:51:29 2018 Received: (at 30626) by debbugs.gnu.org; 11 Mar 2018 21:51:29 +0000 Received: from localhost ([127.0.0.1]:55779 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ev8sK-0007z1-Db for submit@debbugs.gnu.org; Sun, 11 Mar 2018 17:51:29 -0400 Received: from mail-it0-f54.google.com ([209.85.214.54]:34525) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ev8sI-0007yo-Iz for 30626@debbugs.gnu.org; Sun, 11 Mar 2018 17:51:26 -0400 Received: by mail-it0-f54.google.com with SMTP id n128-v6so7828966ith.1 for <30626@debbugs.gnu.org>; Sun, 11 Mar 2018 14:51:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=3qqM39Am8OcHFJkzp96x+iZLjNLs1YJBJRktHUfek7c=; b=SJZFwAjpwVIQRWEiafv9ZGrf+PB2T/Tn92Q0UGu041hUVxfgEgLRQiXUfmfodXfWqx QRaUM/Ih89vFQ7GymTRF7B4sYLCVP2eemyJ/kbzEOAO7oJd2bqRpjYX+M5F61Ph+e/Lb l1nk8U4tWsxwOMS6TPCM4h7SYumvWfbhGqTxyPAJn0NWcpI2EbW1Cz2ILG4yj9HzUkLt jCrMjE3cPSuF5mOxkuK/8lpUCMgSb16PdctaQhRGCFp2YqtdCCgj9apCgWMe8qAe4hyc W4k6+WSMVNs1I0sxKMUdkNlx/MCwHL5UF1uNiAIUnPgHhVoPVx2QARXb/xGnspkFlcF7 FazA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=3qqM39Am8OcHFJkzp96x+iZLjNLs1YJBJRktHUfek7c=; b=NvLdqgAriHARHD3GxB18s9vYx808t6GvIXm3ZoPJlaXo73q5rGKnhUEPz7A7ExhUGT 0fG0zSpxi8xxCq++GOJtuaH0s8RNuexGqDX7uSCaxSwMw8/B2vXUVupL63e40N+fc2EC 3KFW6jCJysr1vp/yUeRlz2KDTw4uyhmkqbD4SO8g/m+Jd+6lPlRHhvaXGKKaFQyaUSer 1x88HiJiLdeCBZ7fvtBsBNwS6cf/KsRL/BEuqC0i5gWBoIUUfbH0uTb3pwMNOri0F8j0 q/no5VpXAsg9B5yTBs4Da57XNdlygmp6P5q8dJ8OAd2ONEPe3q6uxDKzYEkYMgtcE0d+ HaGw== X-Gm-Message-State: AElRT7ESe5QU+0sGSXAIH0iVKLbY17Nl5e8VD78m7hrMuLfjGdHn5UaR +9ngn75EGzgcatZrUxbncoOCGA== X-Google-Smtp-Source: AG47ELusd2PHE+PeCZdK2sHLpjS8hMfN/GYelH49oHEZbx+m0+LBYx87o+C0jSsU4XB3VCNja1adiQ== X-Received: by 10.36.22.85 with SMTP id a82mr6158089ita.33.1520805080991; Sun, 11 Mar 2018 14:51:20 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id v198sm2272349ita.29.2018.03.11.14.51.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 11 Mar 2018 14:51:20 -0700 (PDT) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> Date: Sun, 11 Mar 2018 17:51:19 -0400 In-Reply-To: <83vae2s6b6.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 11 Mar 2018 22:31:09 +0200") Message-ID: <876062z3fs.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Eli Zaretskii writes: >> From: Noam Postavsky >> Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org >> Date: Sun, 11 Mar 2018 14:52:22 -0400 >> >> This did change the backtrace (from starting with mark_specpdl to >> mark_stack), meaning I did find one reference, but it still crashes, so >> there must be more. > > If you have the address, you could first find the stack frame to which > it belongs, right? Um, how do I do that part? From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 11 23:28:25 2018 Received: (at 30626) by debbugs.gnu.org; 12 Mar 2018 03:28:25 +0000 Received: from localhost ([127.0.0.1]:55919 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1evE8P-0001cC-L7 for submit@debbugs.gnu.org; Sun, 11 Mar 2018 23:28:25 -0400 Received: from eggs.gnu.org ([208.118.235.92]:40230) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1evE8O-0001by-Kz for 30626@debbugs.gnu.org; Sun, 11 Mar 2018 23:28:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1evE8G-0001cH-3h for 30626@debbugs.gnu.org; Sun, 11 Mar 2018 23:28:19 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50482) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1evE8G-0001c3-01; Sun, 11 Mar 2018 23:28:16 -0400 Received: from [176.228.60.248] (port=2299 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1evE8E-0003nw-Cw; Sun, 11 Mar 2018 23:28:15 -0400 Date: Mon, 12 Mar 2018 05:28:19 +0200 Message-Id: <83sh96rmzw.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <876062z3fs.fsf@gmail.com> (message from Noam Postavsky on Sun, 11 Mar 2018 17:51:19 -0400) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Noam Postavsky > Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org > Date: Sun, 11 Mar 2018 17:51:19 -0400 > > > If you have the address, you could first find the stack frame to which > > it belongs, right? > > Um, how do I do that part? By comparing the address with the value of $bp in each frame, I'd say. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 12 22:00:08 2018 Received: (at 30626) by debbugs.gnu.org; 13 Mar 2018 02:00:08 +0000 Received: from localhost ([127.0.0.1]:57869 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1evZEW-0002mR-7l for submit@debbugs.gnu.org; Mon, 12 Mar 2018 22:00:08 -0400 Received: from mail-io0-f170.google.com ([209.85.223.170]:44569) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1evZET-0002ka-Di for 30626@debbugs.gnu.org; Mon, 12 Mar 2018 22:00:05 -0400 Received: by mail-io0-f170.google.com with SMTP id h23so13800911iob.11 for <30626@debbugs.gnu.org>; Mon, 12 Mar 2018 19:00:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=3SPelqAolOwa3cnA+kc79+QW+cLGhdoK0ZWUE2IMGaM=; b=UyaKE7G1eKYcak7n17Y9J/W2kLBODDKvnzN3bt6e1Fp401C0IzEWgJOLvIc9c8TL4/ mhl6sjRGSKrSPpE9UQ4hnIrPzIugUmqkxA5GwVS+EZROw73RhsULvgcv0Yf/p2KQBPFr DiYAYioD1KZTnIJrZU33wG3qIFdHLXy1ZdgfhIp0raGpZrTrT5xOoQpnOCe46bZWOxOp PRVbHCDf8S9qGBJk4Rn1BZHzjbugflCkzDlHeEerGdHUjamOJyYidp3uZtZH13S2jjGa ajOkXjqXSdZ/vFF/EyMFpG9wy8g+UjCvvf4f+Oh4QjuI/5CEk5fSm+NP/Vn04guUnd+p lKjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=3SPelqAolOwa3cnA+kc79+QW+cLGhdoK0ZWUE2IMGaM=; b=eebRXvNg2yo7zl7VJqnfpA/P4F60dO47+8bQGpi736MWHltfl/hkVcnP24aJ6Y/ikw u5eMsl8AyRd5SqGvIxjD+TrY76vJxHMFjqN3p2JTnAQ87QLg4QnvL1nx5t6QK865GEJ8 bnlqiXrqQ+G/hPJVLezGS9y7ozpU76glZVKuP0nwK+Q/Dw2tmxWGiqTv+faqx0C8yJ/S U2d9eTUgQsNStxyYTroL0PaCAU6h9QRETwbLpYlzpcRWQi1yJw/LOQGJN9qAtC6dOZYx QdKa6kWpUl2CQtqzz6TbO9f0R6+xbeAbzpry0qFPySw0Dgt9vLuTnG5dpy76U9i5AzL+ mN1g== X-Gm-Message-State: AElRT7GE7KzN1bmlj0FxHhcZlo4KxyvrpMftH5UwlCsq4avJ+Aq0yts1 BhtMQHg8D4g901NBO8WHUqpzZA== X-Google-Smtp-Source: AG47ELs081nW3FfPb7zn4DZufnZyizDiBGIFU8olACh/FXfkKR6yr0Vtq8g/gOf8UfC8lOkIVUlL0g== X-Received: by 10.107.68.26 with SMTP id r26mr10960590ioa.169.1520906399677; Mon, 12 Mar 2018 18:59:59 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id v133sm4344853itc.17.2018.03.12.18.59.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Mar 2018 18:59:58 -0700 (PDT) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> Date: Mon, 12 Mar 2018 21:59:57 -0400 In-Reply-To: <83sh96rmzw.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 12 Mar 2018 05:28:19 +0200") Message-ID: <87lgewybtu.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Eli Zaretskii writes: >> From: Noam Postavsky >> Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org >> Date: Sun, 11 Mar 2018 17:51:19 -0400 >> >> > If you have the address, you could first find the stack frame to which >> > it belongs, right? >> >> Um, how do I do that part? > > By comparing the address with the value of $bp in each frame, I'd say. Hmm, I found a match, but it doesn't make any sense. #4851 0x0000000000611d4f in mark_vectorlike (ptr=0x2e64c90) at ../../src/alloc.c:6227 #4852 0x0000000000612b42 in mark_object (arg=XIL(0x2e64c95)) at ../../src/alloc.c:6624 #4853 0x000000000060f3ce in mark_maybe_pointer (p=0x2e64c90) at ../../src/alloc.c:4936 #4854 0x000000000060f452 in mark_memory (start=0x7fffffffa520, end=0x7fffffffe868) at ../../src/alloc.c:4985 #4855 0x000000000060f493 in mark_stack (bottom=0x7fffffffe868 "a\036h\364\377\177", end=0x7fffffffa520 "0\245\377\377\377\177") at ../../src/alloc.c:5193 (gdb) frame 4854 #4854 0x000000000060f452 in mark_memory (start=0x7fffffffa520, end=0x7fffffffe868) at ../../src/alloc.c:4985 4985 mark_maybe_pointer (*(void **) pp); (gdb) p pp $28 = 0x7fffffffa968 "\220L\346\002" (gdb) frame 4864 #4864 0x000000000068d950 in exec_byte_code (bytestr=XIL(0x2e7aad4), vector=XIL(0x2e72715), maxdepth=make_number(18), args_template=make_number(768), nargs=3, args=0x7fffffffad20) at ../../src/bytecode.c:632 632 TOP = Ffuncall (op + 1, &TOP); (gdb) p $rbp $29 = (void *) 0x7fffffffabd0 (gdb) p/x $rbp - $28 $32 = 0x268 (gdb) disas /s [...] 1180 CASE (Bbuffer_substring): 1181 { 1182 Lisp_Object v1 = POP; 0x000000000068fea4 <+13154>: mov -0x40(%rbp),%rax 0x000000000068fea8 <+13158>: lea -0x8(%rax),%rdx 0x000000000068feac <+13162>: mov %rdx,-0x40(%rbp) 0x000000000068feb0 <+13166>: mov (%rax),%rax 0x000000000068feb3 <+13169>: mov %rax,-0x268(%rbp) 1183 TOP = Fbuffer_substring (TOP, v1); 0x000000000068feba <+13176>: mov -0x268(%rbp),%rdx 0x000000000068fec1 <+13183>: mov -0x40(%rbp),%rax 0x000000000068fec5 <+13187>: mov %rdx,%rsi 0x000000000068fec8 <+13190>: mov (%rax),%rdi 0x000000000068fecb <+13193>: callq 0x627e0a It can't be a buffer-substring arg, but that's the only reference to -0x268(%rbp) in that function. From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 13 12:06:30 2018 Received: (at 30626) by debbugs.gnu.org; 13 Mar 2018 16:06:30 +0000 Received: from localhost ([127.0.0.1]:59733 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1evmRZ-0005pW-Pp for submit@debbugs.gnu.org; Tue, 13 Mar 2018 12:06:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:36863) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1evmRV-0005pD-5U for 30626@debbugs.gnu.org; Tue, 13 Mar 2018 12:06:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1evmRL-0006FY-B9 for 30626@debbugs.gnu.org; Tue, 13 Mar 2018 12:06:20 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37226) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1evmRL-0006FS-7V; Tue, 13 Mar 2018 12:06:15 -0400 Received: from [176.228.60.248] (port=4365 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1evmRK-00054Z-Ll; Tue, 13 Mar 2018 12:06:15 -0400 Date: Tue, 13 Mar 2018 18:06:23 +0200 Message-Id: <834llkrmdc.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <87lgewybtu.fsf@gmail.com> (message from Noam Postavsky on Mon, 12 Mar 2018 21:59:57 -0400) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> <87lgewybtu.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Noam Postavsky > Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org > Date: Mon, 12 Mar 2018 21:59:57 -0400 > > 4985 mark_maybe_pointer (*(void **) pp); > (gdb) p pp > $28 = 0x7fffffffa968 "\220L\346\002" Should you look at pp or at *pp? Also note that for Lisp objects that are marked you need to reset their mark bit before trying to determine their type and value. If none of the above helps, please walk me through the steps that led you to look at -0x268(%rbp), because I'm not sure I follow. From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 13 20:09:27 2018 Received: (at 30626) by debbugs.gnu.org; 14 Mar 2018 00:09:27 +0000 Received: from localhost ([127.0.0.1]:60162 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1evtyx-0005qs-3U for submit@debbugs.gnu.org; Tue, 13 Mar 2018 20:09:27 -0400 Received: from mail-io0-f180.google.com ([209.85.223.180]:44175) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1evtyu-0005qZ-R3 for 30626@debbugs.gnu.org; Tue, 13 Mar 2018 20:09:25 -0400 Received: by mail-io0-f180.google.com with SMTP id h23so2215482iob.11 for <30626@debbugs.gnu.org>; Tue, 13 Mar 2018 17:09:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=DM76bdRqaYZIRDq5++xLxu2hjzi4WOErOdTLdte5Heg=; b=Suk7BNBp1DbZC2wHN1ISCF51546kC2e+8SS8dQ3vrFDc0DpUR1ILWu1yYhkWhudNCa JW6P2xXPIj/gmtzmvCe/AwDyqko/o2KE2DmSUbEB922gkHmZPEfSdqH2/y08PdExgesq eIsQ6wYAVND0LRueagQKC28tS1D8cMlLGoSW5GJ3qpQJPitOrY6IfdIlszmuc+q08580 ZfjA3r8FKYssP3HeCcJ8gD+5TK24gMz6aambpiaZIavtIkn6T6DqH0y0w8LAW9tHCmWK Q+JIe+1+1S41gr3yOLM0+sCe8YuBgSvRUEhMt8OvrlPoAhcxULmbMqVbJa2uw9fCdd8Q 4SWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=DM76bdRqaYZIRDq5++xLxu2hjzi4WOErOdTLdte5Heg=; b=cQM2/41QJZ1U1IUWO0+xu3+ejTtSdw+oOgFDSMhFl/Z836Xie52YZso8TFXTkOtOnO p9A79enlfTbRRnkSqZ8esEf1EkrhVKNerqJLAXn19OH69ao1KCpO5o9qfk10usEky+Yl XLGRPI9WFbj18K0Ui+125ffqA0TtIaCnEgtbiR8TwiFbS+2pTc81xQZY8hZBaaG7mIwt ioOeN64tNU95pSOtse07Zz/9Yfhp9LXxkPZSddVu8xiUsJaH49GsigHI7qLrrxH1US2I QwxoSKGZ5K67hAYDLkia3OKAuk78LB8nDgk1dfW+9gDFNWqUba6tEC2Pzvm6WFHwREQJ fB2A== X-Gm-Message-State: AElRT7GEOXR7Oc/MbUq7h7gK+3sqi69qXsj6DwQXdvaczBTIfoIEqTx7 +fEMRS73XwoWalBFJgHHVVn2Ng== X-Google-Smtp-Source: AG47ELuIDv7IPAcI2A3Y/mMfKDoY3KKIhQlTPldQp/GDOR97vz2M6w9UPJPbBTMcBD6r2gGgDyWt4A== X-Received: by 10.107.188.194 with SMTP id m185mr2850914iof.21.1520986159043; Tue, 13 Mar 2018 17:09:19 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id f74sm663853ioe.47.2018.03.13.17.09.17 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 13 Mar 2018 17:09:18 -0700 (PDT) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> <87lgewybtu.fsf@gmail.com> <834llkrmdc.fsf@gnu.org> Date: Tue, 13 Mar 2018 20:09:17 -0400 In-Reply-To: <834llkrmdc.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 13 Mar 2018 18:06:23 +0200") Message-ID: <87in9zy0uq.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Eli Zaretskii writes: > Should you look at pp or at *pp? I think it should be pp, but I'm not sure. The context: #4854 0x000000000060f452 in mark_memory (start=0x7fffffffa520, end=0x7fffffffe868) at ../../src/alloc.c:4985 #4855 0x000000000060f493 in mark_stack (bottom=0x7fffffffe868 "a\036h\364\377\177", end=0x7fffffffa520 "0\245\377\377\377\177") at ../../src/alloc.c:5193 mark_memory (void *start, void *end) { ... for (pp = start; (void *) pp < end; pp += GC_POINTER_ALIGNMENT) { mark_maybe_pointer (*(void **) pp); mark_maybe_object (*(Lisp_Object *) pp); } So the value of pp ranges over stack addresses and *pp would be the contents of the stack location. > Also note that for Lisp objects that are marked you need to reset > their mark bit before trying to determine their type and value. I think I'm looking for a C variable, and not a Lisp object (although the C variable presumably contains/points to a Lisp object). > If none of the above helps, please walk me through the steps that led > you to look at -0x268(%rbp), because I'm not sure I follow. Starting with the value of pp, I then go up looking for a close value of $rbp: (gdb) p pp $39 = 0x7fffffffa968 "\220L\346\002" (gdb) up #4855 0x000000000060f493 in mark_stack (bottom=0x7fffffffe868 "a\036h\364\377\177", end=0x7fffffffa520 "0\245\377\377\377\177") at ../../src/alloc.c:5193 5193 mark_memory (bottom, end); (gdb) p $rbp $40 = (void *) 0x7fffffffa420 (gdb) up #4856 0x00000000006cdd75 in mark_one_thread (thread=0xe103e0 ) at ../../src/thread.c:616 616 mark_stack (thread->m_stack_bottom, stack_top); (gdb) p $rbp $41 = (void *) 0x7fffffffa470 [...] (gdb) up #4863 0x000000000063c2cb in Ffuncall (nargs=6, args=0x7fffffffa7f8) at ../../src/eval.c:2751 2751 maybe_gc (); (gdb) p $rbp $48 = (void *) 0x7fffffffa780 (gdb) up #4864 0x000000000068d950 in exec_byte_code (bytestr=XIL(0x2e7aad4), vector=XIL(0x2e72715), maxdepth=make_number(18), args_template=make_number(768), nargs=3, args=0x7fffffffad20) at ../../src/bytecode.c:632 632 TOP = Ffuncall (op + 1, &TOP); (gdb) p $rbp $49 = (void *) 0x7fffffffabd0 Now I see that $rbp is higher than the target address, and the difference is 0x268, so the target location should be -0x268(%rbp). (gdb) p $rbp - 0x7fffffffa968 $52 = (void *) 0x268 Except something must be wrong in my reasoning, since the only ocurrences of -0x268(%rbp) are the buffer-string args, which could only hold integers or markers (neither of which could further point to long chains of objects). From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 15 12:34:29 2018 Received: (at 30626) by debbugs.gnu.org; 15 Mar 2018 16:34:29 +0000 Received: from localhost ([127.0.0.1]:35331 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ewVpl-00021F-A7 for submit@debbugs.gnu.org; Thu, 15 Mar 2018 12:34:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ewVpk-000213-CI for 30626@debbugs.gnu.org; Thu, 15 Mar 2018 12:34:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewVpa-0005Wc-VF for 30626@debbugs.gnu.org; Thu, 15 Mar 2018 12:34:22 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50939) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewVpa-0005WW-Qv; Thu, 15 Mar 2018 12:34:18 -0400 Received: from [176.228.60.248] (port=3575 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ewVpa-0003sb-7m; Thu, 15 Mar 2018 12:34:18 -0400 Date: Thu, 15 Mar 2018 18:34:16 +0200 Message-Id: <83605xqovr.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <87in9zy0uq.fsf@gmail.com> (message from Noam Postavsky on Tue, 13 Mar 2018 20:09:17 -0400) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> <87lgewybtu.fsf@gmail.com> <834llkrmdc.fsf@gnu.org> <87in9zy0uq.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Noam Postavsky > Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org > Date: Tue, 13 Mar 2018 20:09:17 -0400 > > Eli Zaretskii writes: > > > Should you look at pp or at *pp? > > I think it should be pp, but I'm not sure. The context: > > #4854 0x000000000060f452 in mark_memory (start=0x7fffffffa520, end=0x7fffffffe868) > at ../../src/alloc.c:4985 > #4855 0x000000000060f493 in mark_stack (bottom=0x7fffffffe868 "a\036h\364\377\177", > end=0x7fffffffa520 "0\245\377\377\377\177") at ../../src/alloc.c:5193 > > mark_memory (void *start, void *end) > { > ... > for (pp = start; (void *) pp < end; pp += GC_POINTER_ALIGNMENT) > { > mark_maybe_pointer (*(void **) pp); > mark_maybe_object (*(Lisp_Object *) pp); > } > > So the value of pp ranges over stack addresses and *pp would be the > contents of the stack location. But the call to mark_maybe_pointer means that we consider pp to be a pointer (in)to a Lisp object. Anyway, wouldn't it be easier to look one frame lower? We have this: #4850 0x0000000000612b42 in mark_object (arg=XIL(0x2efcb83)) at ../../src/alloc.c:6624 #4851 0x0000000000611d4f in mark_vectorlike (ptr=0x2e64c90) at ../../src/alloc.c:6227 #4852 0x0000000000612b42 in mark_object (arg=XIL(0x2e64c95)) at ../../src/alloc.c:6624 #4853 0x000000000060f3ce in mark_maybe_pointer (p=0x2e64c90) at ../../src/alloc.c:4936 #4854 0x000000000060f452 in mark_memory (start=0x7fffffffa520, end=0x7fffffffe868) at ../../src/alloc.c:4985 #4855 0x000000000060f493 in mark_stack (bottom=0x7fffffffe868 "a\036h\364\377\177", end=0x7fffffffa520 "0\245\377\377\377\177") at ../../src/alloc.c:5193 In frame #4852, we have found an object, and we are marking it. Did you try looking at that object? With these caveats: > > Also note that for Lisp objects that are marked you need to reset > > their mark bit before trying to determine their type and value. From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 17 11:53:47 2018 Received: (at 30626) by debbugs.gnu.org; 17 Mar 2018 15:53:47 +0000 Received: from localhost ([127.0.0.1]:38591 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1exE9S-0001wa-Ru for submit@debbugs.gnu.org; Sat, 17 Mar 2018 11:53:47 -0400 Received: from mail-it0-f44.google.com ([209.85.214.44]:34151) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1exE9R-0001wM-J4 for 30626@debbugs.gnu.org; Sat, 17 Mar 2018 11:53:45 -0400 Received: by mail-it0-f44.google.com with SMTP id z7-v6so4951199iti.1 for <30626@debbugs.gnu.org>; Sat, 17 Mar 2018 08:53:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=LkNdYmPm66yTQdhd4zr65hFhej8U6D6p3ttpR62iIyI=; b=fwxyykGp4Td9mH25rcSs0ag8YKy3ucrc49sVPFVUC9edon3DmbK7yL28ROqhAgje/k MK1wu5ZoFCk3gcW9vHjC2ZOtJ2v4S8hQy6TEA5mKhsHedtglM26ttraTtVO00WmQmyqS 8PYubRIrR4YW9XeOH8Jb0L4r6tN97esyINg6z9FYWTzLxuSwvKMEKYFojW4UeOVR7+6b B5RwpXcbgomnYQf8oy+r0ff64KhXfR7vtK2QzTOaUrSlI1pZHQuBZ39YJviqfYKb4Vfs ijch6Y4GAWCd45JnP0JDeSN6TpSgYJ5LBOSu03qng1N878s8GVAskv7u1+iTdyGCDy+S 1eLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=LkNdYmPm66yTQdhd4zr65hFhej8U6D6p3ttpR62iIyI=; b=TfsYDRE3FoMBlrrfo8ynGG8t/sIEsihEbeTxjLb8irymDlZKPwvQjeLqSotl67Woli chMV5b6gbT5+5igKeJKJlNjX5MtxYAc3VFwPTzqO90eWg/uhYphLYInnNc/CQU68+7zx 8lgaofFFNMNjslICLLpJDBioMrLn0TLOpuuqNxAlSgxQAbTfWJ/bBCgM9P3wfPGHgocJ EntkzIoIAjRnNW5LyaW9J6khz3Sv9KehAlkSEX63Zxs1TsJU9nHU1DTJAuM5S5Cea1sn 9Nsm7Be5IEM7vasSHXewQpv7WnlJn5o5eV5TeE6qLmMrNKgkvTAj/TsYRoOOPHOsx8b4 GVrA== X-Gm-Message-State: AElRT7EvfRXOOpPFGCMlOo6YmZ/Lf+8m3O8uLejRxhm8vSVMsHOyhlcq POhOvJEOd/ix8PpJQJ6mGt9xf9y9 X-Google-Smtp-Source: AG47ELv7h53nfadErxbTHnmnMBavZd9b6yUxnWnKsZG6MKav2ohjMmlwcsJSMaK+a6JNehkI0GbvRg== X-Received: by 2002:a24:7143:: with SMTP id n64-v6mr6171787itc.4.1521302019573; Sat, 17 Mar 2018 08:53:39 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id f201-v6sm5973324itc.12.2018.03.17.08.53.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 17 Mar 2018 08:53:38 -0700 (PDT) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> <87lgewybtu.fsf@gmail.com> <834llkrmdc.fsf@gnu.org> <87in9zy0uq.fsf@gmail.com> <83605xqovr.fsf@gnu.org> Date: Sat, 17 Mar 2018 11:53:36 -0400 In-Reply-To: <83605xqovr.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 15 Mar 2018 18:34:16 +0200") Message-ID: <871sgivgu7.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Eli Zaretskii writes: > In frame #4852, we have found an object, and we are marking it. Did > you try looking at that object? With these caveats: > >> > Also note that for Lisp objects that are marked you need to reset >> > their mark bit before trying to determine their type and value. Okay, I think xpr takes care of that, right? (I've restarted the debug sessions a few times, so numbers may not match exactly) #4853 0x000000000060f429 in mark_maybe_pointer (p=0x2e64c90) at ../../src/alloc.c:4936 4936 mark_object (obj); (gdb) p obj $60 = XIL(0x2e64c95) (gdb) xpr Lisp_Vectorlike PVEC_NORMAL_VECTOR $61 = (struct Lisp_Vector *) 0x2e64c90 {XIL(0x2efcb63), make_number(1000000), XIL(0x2efcb53), XIL(0x2efcb73), XIL(0x2efcb83), XIL(0x20ab5b0), XIL(0xc090)} (gdb) p $61->contents[0] $62 = XIL(0x2efcb63) (gdb) xpr Lisp_Cons $63 = (struct Lisp_Cons *) 0x2efcb60 { [...] car = make_number(2369), [...] chain = 0x0 [...] (gdb) p $61->contents[1] $64 = make_number(1000000) (gdb) p $61->contents[2] $65 = XIL(0x2efcb53) (gdb) xpr Lisp_Cons [...] car = make_number(1), [...] chain = 0x0 [...] (gdb) p $61->contents[3] $67 = XIL(0x2efcb73) (gdb) xpr [...] car = XIL(0xc090), [...] chain = 0x0 [...] (gdb) p $61->contents[4] [...] car = XIL(0x2efc443), [...] chain = 0x0 [...] (gdb) p $61->contents[5] [...] $72 = (struct Lisp_Symbol *) 0x2e97ef0 "stream-range" (gdb) p $61->contents[6] [...] $74 = (struct Lisp_Symbol *) 0xdf89d0 "t" It looks like a the lexical environment of a bytecode function, probably the initial stream, e.g., (stream-range 1 1000000) gives: (--stream-- #[256 "\211\203\303\242\207\303\242\204 \304\300\242\305\300\242\302\242\\\301\302\242#B\240\210\303\306\240\210\304\242\207" [(1) 1000000 (1) (nil) (nil) stream-range t] 7 " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (fn &optional CHECK)"]) Not sure where to go next with this. From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 17 12:10:32 2018 Received: (at 30626) by debbugs.gnu.org; 17 Mar 2018 16:10:32 +0000 Received: from localhost ([127.0.0.1]:38595 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1exEPg-0002MI-CM for submit@debbugs.gnu.org; Sat, 17 Mar 2018 12:10:32 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51572) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1exEPe-0002M3-3W for 30626@debbugs.gnu.org; Sat, 17 Mar 2018 12:10:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1exEPV-0006JZ-IT for 30626@debbugs.gnu.org; Sat, 17 Mar 2018 12:10:24 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:51504) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1exEPV-0006JM-CU; Sat, 17 Mar 2018 12:10:21 -0400 Received: from [176.228.60.248] (port=3153 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1exEPU-0004lc-Qi; Sat, 17 Mar 2018 12:10:21 -0400 Date: Sat, 17 Mar 2018 18:10:24 +0200 Message-Id: <83d102ptsf.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <871sgivgu7.fsf@gmail.com> (message from Noam Postavsky on Sat, 17 Mar 2018 11:53:36 -0400) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> <87lgewybtu.fsf@gmail.com> <834llkrmdc.fsf@gnu.org> <87in9zy0uq.fsf@gmail.com> <83605xqovr.fsf@gnu.org> <871sgivgu7.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Noam Postavsky > Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org > Date: Sat, 17 Mar 2018 11:53:36 -0400 > > It looks like a the lexical environment of a bytecode function, probably > the initial stream, e.g., (stream-range 1 1000000) gives: > > (--stream-- #[256 "\211\203\303\242\207\303\242\204 \304\300\242\305\300\242\302\242\\\301\302\242#B\240\210\303\306\240\210\304\242\207" > [(1) 1000000 (1) (nil) (nil) stream-range t] 7 " > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > (fn &optional CHECK)"]) > > Not sure where to go next with this. The goal was to find out which variable holds a reference to the entire long stream, right? So it sounds like a pointer to it is kept in an automatic variable on the stack of exec_byte_code, right? Which kinda makes sense, since the stream is still being processed, I think. Or am I confused? From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 17 12:27:45 2018 Received: (at 30626) by debbugs.gnu.org; 17 Mar 2018 16:27:45 +0000 Received: from localhost ([127.0.0.1]:38602 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1exEgK-0002mC-Tz for submit@debbugs.gnu.org; Sat, 17 Mar 2018 12:27:45 -0400 Received: from eggs.gnu.org ([208.118.235.92]:54269) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1exEgJ-0002lz-Sr for 30626@debbugs.gnu.org; Sat, 17 Mar 2018 12:27:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1exEgB-00045m-DL for 30626@debbugs.gnu.org; Sat, 17 Mar 2018 12:27:38 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:51813) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1exEgB-00045W-AE; Sat, 17 Mar 2018 12:27:35 -0400 Received: from [176.228.60.248] (port=3368 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1exEgA-00068F-N3; Sat, 17 Mar 2018 12:27:35 -0400 Date: Sat, 17 Mar 2018 18:27:38 +0200 Message-Id: <83bmfmpszp.fsf@gnu.org> From: Eli Zaretskii To: npostavs@gmail.com In-reply-to: <83d102ptsf.fsf@gnu.org> (message from Eli Zaretskii on Sat, 17 Mar 2018 18:10:24 +0200) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> <87lgewybtu.fsf@gmail.com> <834llkrmdc.fsf@gnu.org> <87in9zy0uq.fsf@gmail.com> <83605xqovr.fsf@gnu.org> <871sgivgu7.fsf@gmail.com> <83d102ptsf.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > Date: Sat, 17 Mar 2018 18:10:24 +0200 > From: Eli Zaretskii > Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, > 30626@debbugs.gnu.org > > > (--stream-- #[256 "\211\203\303\242\207\303\242\204 \304\300\242\305\300\242\302\242\\\301\302\242#B\240\210\303\306\240\210\304\242\207" > > [(1) 1000000 (1) (nil) (nil) stream-range t] 7 " > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > (fn &optional CHECK)"]) > > > > Not sure where to go next with this. > > The goal was to find out which variable holds a reference to the > entire long stream, right? So it sounds like a pointer to it is kept > in an automatic variable on the stack of exec_byte_code, right? Which > kinda makes sense, since the stream is still being processed, I think. > > Or am I confused? Actually, there's still some mystery: if this object is a 7-element vector, where do all the other GC frame come from? Hmm... how long/deep is each of the cons cells in elements 1 through 4 of the vector? If they are deeply nested, then that's the answer we've been looking for, I think. From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 17 13:28:32 2018 Received: (at 30626) by debbugs.gnu.org; 17 Mar 2018 17:28:32 +0000 Received: from localhost ([127.0.0.1]:38613 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1exFd9-0004IQ-NU for submit@debbugs.gnu.org; Sat, 17 Mar 2018 13:28:31 -0400 Received: from mail-io0-f170.google.com ([209.85.223.170]:46757) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1exFd8-0004IB-1C for 30626@debbugs.gnu.org; Sat, 17 Mar 2018 13:28:30 -0400 Received: by mail-io0-f170.google.com with SMTP id g14so7249483iob.13 for <30626@debbugs.gnu.org>; Sat, 17 Mar 2018 10:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=rotUGgFAkrC/uSFnRYSgOiyyyTgUJbmE2pNmgesDGiw=; b=rIqJHFNYGmYVbY3hEI8ddXXBPNGAytLgWxeWHXDxrPfdJb6zYk4+vK6CmJajl71yD4 hKE4b0MHXh36vQHLqtHXR5cLYPGAFcwb2Vnzaj+0A/i74tCRxZarwuAufm3xxp2juc+j 1ztIhaybP3KVjPL0rs6eei+XcXB15fRP6qxYismVDAPBRy73X4ESPK62zzNIYnj07y8a k0NwWSQLjJQQQTFPlnzwxRT93RjlKOnFQ8xRUfnG6z9MnMLXCfJ0kV0dz/CPT208sypz +akoMktp0noCm2X0tO7GhO0CM9k/yQZa/s1Bq8qceMObqMjuBsfIj9Ek1OW8QoxSC1k4 sLHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=rotUGgFAkrC/uSFnRYSgOiyyyTgUJbmE2pNmgesDGiw=; b=IuBGFbpWIAStL8+mEBiF3ZhDwkbUeHn1CtxaWfjFvg1HZi3DEqUpGs5YZ9Xx3YA4yw mUpIUBWx3lAu4ikXV6jYc8ZHkoDzi1XvxstNrx5xpDTrrV6JbNX0dabN/df9vALhpH3m +bG15R8aTHnzLifm2pv+Z8PyZoIXf7/FlnVtg67jszz9SI08uPJH0/027gUjXyTpqZgB 7XRz6k47727lUOCzXmHS/typ3KHuE5MMGckp3Atv1hP+jQzFq8QNq6hE1DUy48U/hyoJ zSYAnvQ1sZ/OJj1BrHM5o56TLYMxHMcPfJ1vs0T5E4F6FdKoMCMWCRcbc88dIPTvshlk fdYQ== X-Gm-Message-State: AElRT7HUSDjukflhfMyPKrrwtvKJgrZMIQlhxuPxbghhyjPaZl3kpsTO 9tMUSsA1MtDihTNIpFXN2JcQ/6KS X-Google-Smtp-Source: AG47ELuZuFV72WAJatudTQBi+5iLq+yFNroqcRx9SclTtz0rHB80lj5NdvoBAQWEqFoYLiAcUkFifQ== X-Received: by 10.107.51.78 with SMTP id z75mr6629859ioz.291.1521307704229; Sat, 17 Mar 2018 10:28:24 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id 12-v6sm2846047itm.0.2018.03.17.10.28.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 17 Mar 2018 10:28:23 -0700 (PDT) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> <87lgewybtu.fsf@gmail.com> <834llkrmdc.fsf@gnu.org> <87in9zy0uq.fsf@gmail.com> <83605xqovr.fsf@gnu.org> <871sgivgu7.fsf@gmail.com> <83d102ptsf.fsf@gnu.org> <83bmfmpszp.fsf@gnu.org> Date: Sat, 17 Mar 2018 13:28:22 -0400 In-Reply-To: <83bmfmpszp.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 17 Mar 2018 18:27:38 +0200") Message-ID: <87y3iqtxvt.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) Eli Zaretskii writes: >> The goal was to find out which variable holds a reference to the >> entire long stream, right? So it sounds like a pointer to it is kept >> in an automatic variable on the stack of exec_byte_code, right? Which >> kinda makes sense, since the stream is still being processed, I >> think. Yeah, but which variable exactly? I'd like to find it and add 'X = Qnil;' to confirm we've found where it is. > Actually, there's still some mystery: if this object is a 7-element > vector, where do all the other GC frame come from? Hmm... how > long/deep is each of the cons cells in elements 1 through 4 of the > vector? If they are deeply nested, then that's the answer we've been > looking for, I think. It's a bit confusing because of the indirection: stream-range uses the stream-cons macro, which uses the stream-make macro, which uses the thunk-delay macro. I believe the end result is that the lexical environment of the resulting closure has access to the next stream-element in the chain, so the nesting depth is the length of the stream (i.e., 100000 in the example). Perhaps this example makes it clearer: (setq print-circle t) (let* ((s0 (stream-range 1 2)) (s1 (stream-rest s0))) (list s0 s1)) ;=> ((--stream-- #[256 "\211\203..." [(1) 2 (1) (t) ((1 . #1=(--stream-- #[256 "\211\203..." [(nil) (nil) nil t] 3 "\n\n(fn &optional CHECK)"]))) stream-range t] 7 "\n\n(fn &optional CHECK)"]) #1#) From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 19 16:06:00 2018 Received: (at 30626) by debbugs.gnu.org; 19 Mar 2018 20:06:00 +0000 Received: from localhost ([127.0.0.1]:42211 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey12e-00024r-3J for submit@debbugs.gnu.org; Mon, 19 Mar 2018 16:06:00 -0400 Received: from eggs.gnu.org ([208.118.235.92]:49928) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ey12c-00024c-Nx for 30626@debbugs.gnu.org; Mon, 19 Mar 2018 16:05:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ey12S-0000ch-Np for 30626@debbugs.gnu.org; Mon, 19 Mar 2018 16:05:53 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35438) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ey12S-0000cU-KS; Mon, 19 Mar 2018 16:05:48 -0400 Received: from [176.228.60.248] (port=3455 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ey12S-0007op-4b; Mon, 19 Mar 2018 16:05:48 -0400 Date: Mon, 19 Mar 2018 22:05:58 +0200 Message-Id: <83vadrn849.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <87y3iqtxvt.fsf@gmail.com> (message from Noam Postavsky on Sat, 17 Mar 2018 13:28:22 -0400) Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> <87lgf83gun.fsf@gmail.com> <87fu5f3kcr.fsf@gmail.com> <83lgf7u644.fsf@gnu.org> <87bmfuzbq1.fsf@gmail.com> <83vae2s6b6.fsf@gnu.org> <876062z3fs.fsf@gmail.com> <83sh96rmzw.fsf@gnu.org> <87lgewybtu.fsf@gmail.com> <834llkrmdc.fsf@gnu.org> <87in9zy0uq.fsf@gmail.com> <83605xqovr.fsf@gnu.org> <871sgivgu7.fsf@gmail.com> <83d102ptsf.fsf@gnu.org> <83bmfmpszp.fsf@gnu.org> <87y3iqtxvt.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 30626 Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Eli Zaretskii Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) > From: Noam Postavsky > Cc: michael_heerdegen@web.de, john.b.mastro@gmail.com, nicolas@petton.fr, 30626@debbugs.gnu.org > Date: Sat, 17 Mar 2018 13:28:22 -0400 > > Eli Zaretskii writes: > > >> The goal was to find out which variable holds a reference to the > >> entire long stream, right? So it sounds like a pointer to it is kept > >> in an automatic variable on the stack of exec_byte_code, right? Which > >> kinda makes sense, since the stream is still being processed, I > >> think. > > Yeah, but which variable exactly? I'd like to find it and add 'X = > Qnil;' to confirm we've found where it is. I don't think you will be able to tell without stepping through the byte-code interpreter code, keeping track of what it stores where. From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 24 23:20:55 2019 Received: (at 30626) by debbugs.gnu.org; 25 Apr 2019 03:20:55 +0000 Received: from localhost ([127.0.0.1]:57072 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hJUwQ-0003He-Qf for submit@debbugs.gnu.org; Wed, 24 Apr 2019 23:20:55 -0400 Received: from mail-qk1-f179.google.com ([209.85.222.179]:41866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hJUwP-0003HL-4C for 30626@debbugs.gnu.org; Wed, 24 Apr 2019 23:20:53 -0400 Received: by mail-qk1-f179.google.com with SMTP id l199so2523173qke.8 for <30626@debbugs.gnu.org>; Wed, 24 Apr 2019 20:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=B613M59uLOwonp+Ve9ApLfmN8duCQrbn/dSoIbyLdFs=; b=n329/+92cyWcithXwjj8PYvleBj8uot47XBEru/vy73diyjpl4upokWiNQ3jjwekst RbEtIuY0wX5CVH2Fps2CfvdXX5eS+Pn9tQH3c0nekWzjt6BX8/92aSsAMZOapeNlUyPx FWdtbA1fwKt7teW+0Q65bqdQVDur4HL8Mi2v+hfhuvu+aLPZLhyrNQFI0F3AE0r5zt4Y TNLfl84L7RpViv49MBAFkycfriQQor6AMvTz7aUrAlImC4gRN3iFHeue/YWmtzSGob2q Hf8ej5SOKF+AU792CVaCOQAVb5TF3wAPdtA87D0Xidy+n6lznjs8o9qfN0NhI1jFJ7n3 TmqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=B613M59uLOwonp+Ve9ApLfmN8duCQrbn/dSoIbyLdFs=; b=JTfOuRFbPHIWZj2D7RjEwn1c8GjZkpqmL1ckN4XFB/hjI0nM3YVDLF3gonET7bR/bR 3k28Da6Ocbzo07RcQClhIbJ27bVmzcPvyXsWWuUxt6gJILNCYT/HVDbDNPZcRh8Fk/WC pc/2F9wLbEzqTkkdnAFTboYPyp2hBxTG72hr/grMjaRfY0FogfB1qXJU7XpG3ZxUpnNc GDamfXplYT4baQQ6WimAYG9Sz1I4PXlJiHwDZWyHyVxuXKDGIOpoVuqlLfF/rkwFsNln eUWXV5cF1DPpAwAbDUW/z+1xl8AjbtplJikG1lyMn1G4kdgiTTlpAOs3CVnDc3qi5Pz0 7QYg== X-Gm-Message-State: APjAAAXgnOpH8vq+/stiKFGzBnpqncR5MbMkm3lf7cTgD6KJdl++m9cw UG+jUz5u6rCXFIVFBxs7bp4= X-Google-Smtp-Source: APXvYqz56Ri/ufllS++XelCK3FVVU32bTPQlSLNOEunK87KKHnnP3hyLrrtdNdEWjmEGrfvLZwfSDg== X-Received: by 2002:a37:4ed5:: with SMTP id c204mr27811819qkb.68.1556162447473; Wed, 24 Apr 2019 20:20:47 -0700 (PDT) Received: from minid (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id b187sm1216445qkd.73.2019.04.24.20.20.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Apr 2019 20:20:45 -0700 (PDT) From: Noam Postavsky To: Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> Date: Wed, 24 Apr 2019 23:20:44 -0400 In-Reply-To: <87bmg91ity.fsf@web.de> (Michael Heerdegen's message of "Wed, 28 Feb 2018 11:58:49 +0100") Message-ID: <87imv2rcs3.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.2 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.2 (-) --=-=-= Content-Type: text/plain Michael Heerdegen writes: >> I don't have a quick answer for the general case, but I think it's a bug >> in stream.el that it's creating such large structures in the first >> place. As far as I understand it, the point of streams is to handle >> long lists by encoding them as >> >> (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST) > > Yes, that's exactly how it's implemented. When requesting more elements > from the stream, that becomes > > (FIRST-VALUE . > (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST)) > > When we loop over the string, the cons whose car is the FIRST-VALUE, > let's call it cons1, is immediately thrown away, and we continue with > > (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST) Coming back to this again. I think I got lost in the weeds of the byte-code function objects before. The core problem is that streams are not exactly encoded like the above, because even after forcing it you don't have just a plain SECOND-VALUE stored in the stream. The original FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST stays around and keeps referencing all the code and all the following elements. So a possible solution is change the stream to get rid of the lambda part and just leave the computed value after it's forced. With the following patch (stream-flush (stream-range 1 1000000)) succeeds: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-WIP-Drop-forced-lambda-s-from-stream-Bug-30626.patch Content-Description: patch >From 5e7139618f1b4cebbe9785ea5a9303b80d1b2c92 Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Wed, 24 Apr 2019 22:51:19 -0400 Subject: [PATCH] [WIP] Drop forced lambda's from stream (Bug#30626) Let the stream id distinguish between forced and unforced stream values, replacing (think-delay BODY) with just (lambda () BODY). When the value is forced, replace the lambda with its result. --- packages/stream/stream.el | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/packages/stream/stream.el b/packages/stream/stream.el index 3f6bc4b5b..fa7a3b520 100644 --- a/packages/stream/stream.el +++ b/packages/stream/stream.el @@ -65,18 +65,28 @@ (eval-when-compile (require 'cl-lib)) (require 'seq) -(require 'thunk) (eval-and-compile - (defconst stream--identifier '--stream-- - "Symbol internally used to identify streams.")) + (defconst stream--fresh-identifier '--stream-fresh-- + "Symbol internally used to streams whose head was not evaluated.") + (defconst stream--evald-identifier '--stream-evald-- + "Symbol internally used to streams whose head was evaluated.")) (defmacro stream-make (&rest body) "Return a stream built from BODY. BODY must return nil or a cons cell whose cdr is itself a stream." (declare (debug t)) - `(list ',stream--identifier (thunk-delay ,@body))) + `(list ',stream--fresh-identifier (lambda () ,@body))) + +(defun stream-force (stream) + (cond + ((eq (car-safe stream) stream--evald-identifier) + (cadr stream)) + ((eq (car-safe stream) stream--fresh-identifier) + (setf (car stream) stream--evald-identifier) + (setf (cadr stream) (funcall (cadr stream)))) + (t (signal 'wrong-type-argument (list 'streamp stream))))) (defmacro stream-cons (first rest) "Return a stream built from the cons of FIRST and REST. @@ -159,24 +169,26 @@ (defun stream-range (&optional start end step) (defun streamp (stream) "Return non-nil if STREAM is a stream, nil otherwise." - (eq (car-safe stream) stream--identifier)) + (let ((car (car-safe stream))) + (or (eq car stream--fresh-identifier) + (eq car stream--evald-identifier)))) (defun stream-empty () "Return a new empty stream." - (list stream--identifier (thunk-delay nil))) + (list stream--evald-identifier nil)) (defun stream-empty-p (stream) "Return non-nil if STREAM is empty, nil otherwise." - (null (thunk-force (cadr stream)))) + (null (cadr (stream-force stream)))) (defun stream-first (stream) "Return the first element of STREAM. Return nil if STREAM is empty." - (car (thunk-force (cadr stream)))) + (car (stream-force stream))) (defun stream-rest (stream) "Return a stream of all but the first element of STREAM." - (or (cdr (thunk-force (cadr stream))) + (or (cdr (stream-force stream)) (stream-empty))) (defun stream-append (&rest streams) -- 2.11.0 --=-=-= Content-Type: text/plain The reason I didn't do this in thunk.el is that thunks are just bare lambdas, so there is no way to get rid it "from the inside", as it were. Whereas for streams, it's just a matter of list editing. Some additional things that I thought of changing, but I didn't yet: - stream--identifier vs just using '--stream-- directly, I don't see what's the benefit of indirection here. - stream-make should use cons instead of list (or maybe a struct?). - stream-empty should just be a constant. --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 25 01:19:45 2019 Received: (at 30626) by debbugs.gnu.org; 25 Apr 2019 05:19:45 +0000 Received: from localhost ([127.0.0.1]:57190 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hJWnR-0006AX-1Y for submit@debbugs.gnu.org; Thu, 25 Apr 2019 01:19:45 -0400 Received: from mout.web.de ([212.227.17.12]:57307) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hJWnN-0006AI-OU for 30626@debbugs.gnu.org; Thu, 25 Apr 2019 01:19:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1556169572; bh=v6n8kdiJ/RtjD9IUkmoXiexax+3YXqsXRQrWOAczMw4=; h=X-UI-Sender-Class:From:To:Cc:Subject:References:Date:In-Reply-To; b=gRbPvWiSpqkMnV5UJ0ZU5YMOs/nSda0/pZ8oxPxwJ39H4HGhVsBGTEcM4JrCAUWFR 7t8lZBQmCh2kJKhAH2tkCjQHPmKidUeBGzCgQKoFTf1v6adSSZPKlJ8Pyxpvi6Gid9 vH7MmnbRF8cI+fjJRnJbBl/vqs5uwXylArYEET3w= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from drachen.dragon ([92.208.87.27]) by smtp.web.de (mrweb103 [213.165.67.124]) with ESMTPSA (Nemesis) id 0LsyRS-1geOG93ogE-012WbG; Thu, 25 Apr 2019 07:19:32 +0200 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <87imv2rcs3.fsf@gmail.com> Date: Thu, 25 Apr 2019 07:19:29 +0200 In-Reply-To: <87imv2rcs3.fsf@gmail.com> (Noam Postavsky's message of "Wed, 24 Apr 2019 23:20:44 -0400") Message-ID: <87y33ybr1a.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:GZ3hp7V2dZLnUhpXb2T5jiArBSLqmaVMGCbLHpFwqxjnxn6mFo+ CbbGP/dPmIB+gVRVvrWpBfEkRJA9VGX6B2zsPGHU2lg4sC+uAMLMoaosw+2gLWDCfNOVBcZ /gmUDAiLjuYdGqQcBpvZudCIBWeyUJrSjvVL7wz0ZS7QJFKY6j4pm7v0QNpAHKfHhhIVolM JlOz8WiYiOp4pIfnajZAA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:PKtHVc7jjCs=:CHM/QdDvhyGNWgwcNULUkU hT5WCP7sWuBE7ywIt5QxDM067dXN6M3n1V9mzMg2YbcoepOXObooIk/tRG274V5r+CyZigeDu Eu/IO7BiI522OgMfXXz6L7qFOHKsz1nuJPW+y/lzT5qlKtUgw+65i6ug5aqvCjzyAZQLpS4tq Rh7OAp6N7Gf5Z9rkgAtpAA41WNYSanu2h/dn/FJ7rmCy+8/Gr8Aal+8IAHeJEtCAmZJp9TCoN 8vrzusGjOMyshy2AGnRr5OmEUJDuiQpexQgKRUpH6ETKTnKgWlnbbgHKZ1fVInKosAm8A99z6 ZO8PsvoXbgiyKhWGnlW4ggOOJGNGBtDuiEonzswTZZ32dtO7QuV6AZoEV37CrY4i7SdyQa7fz lb2Cr4AgJ5jxe4d3aitav1TrGoftkUVBQomCvlcoZxPpDHspvC02cARWAZjqawyxytDsa/qDo hW0lE25TAQs6UOniHu9NMJBEoR44UkHOlb+PAil46e9guW74CsG6DcguxbSn7gkuRSVahzLEy cDGfsVDFTI1JZVtkBES3Eg6ja/E2/DZ/I2k0JJ4vM704N3Jdag1Zo9zgxCnI25In3ECibnqRR Q9WMKlCG3YFWT77CEDgmdImhjMmrmiG1QPzRYCvZw44d0i1941xEIcLLzAlFdPwC8rUOhPZMp Ydl2o3fWHitZcsRXWFC/U7z9j9A2TW7AbeMio+3yTmvFRfWyK8sHv2UzYhWVY53HbEGcyBnU8 7N8X9Rr0MUN5RIW9ZcpZDGOeK1JculBSdGXjzh5Y9BK+UUU28sLjU47enUfQ/NiYljGBih4JX EzdFc0tNBZeFWsK+xFSvmwIwZd8WnCSRm2kezKOCWdm6cqFd2MKdyZBSgkS+NMPRN5uiaonX3 +gRUhSvsAsNopvrbBtzfNKwLvMc3Ih0ZRtLNEg8Okwm9dYydRjLJTVe/vvzIEF81RcKzjnRn4 pZAUKQ4671Q== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Eli Zaretskii , Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Noam Postavsky writes: > Coming back to this again. I think I got lost in the weeds of the > byte-code function objects before. The core problem is that streams > are not exactly encoded like the above, because even after forcing it > you don't have just a plain SECOND-VALUE stored in the stream. The > original FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST stays around and keeps > referencing all the code and all the following elements. So a > possible solution is change the stream to get rid of the lambda part > and just leave the computed value after it's forced. With the > following patch (stream-flush (stream-range 1 1000000)) succeeds: > > [Patch...] Works for me, and it makes sense. As a test case I recompiled el-search.el (it uses streams for several things) with your patch applied to stream.el, and it worked well. > Some additional things that I thought of changing, but I didn't yet: > - stream--identifier vs just using '--stream-- directly, I don't see > what's the benefit of indirection here. A matter of taste I guess. > - stream-make should use cons instead of list (or maybe a struct?). I think cons would be ok. Would a struct make things slower? > - stream-empty should just be a constant. Dunno if there are cases where this would be problematic, but I guess we could do this as well. Anyway, thanks for looking into this again, I like your solution. Maybe Nicolas can also chime in. Michael. From debbugs-submit-bounces@debbugs.gnu.org Fri May 10 09:20:23 2019 Received: (at 30626) by debbugs.gnu.org; 10 May 2019 13:20:23 +0000 Received: from localhost ([127.0.0.1]:37453 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hP5Rn-0007ye-9I for submit@debbugs.gnu.org; Fri, 10 May 2019 09:20:23 -0400 Received: from mout.web.de ([212.227.17.12]:45395) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hP5Rl-0007yR-ML for 30626@debbugs.gnu.org; Fri, 10 May 2019 09:20:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1557494409; bh=K4D9Gh2feNYNupGhQ1p9r1/L5cE5Rn0kit0R3ctaIzw=; h=X-UI-Sender-Class:From:To:Cc:Subject:References:Date:In-Reply-To; b=j+njh/dB4cGNTUKCINU964/AcaKEs/eWs6LqjH6F+/juGzgVRgI3qpVkX8KTJLPJA 70Vdi4FTJ5yS2pwc1dLvA9tyTO/0ao/yJpzco7OiyX/16Y8GicWaZNaSMQ345r03LZ rh00MtKmHfMgHucVeWe4vFf4kZ+Z7zLnLoBTrXOc= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from drachen.dragon ([88.66.186.107]) by smtp.web.de (mrweb103 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MNtLj-1hN8RG0j1f-007W0w; Fri, 10 May 2019 15:20:09 +0200 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <87imv2rcs3.fsf@gmail.com> <87y33ybr1a.fsf@web.de> Date: Fri, 10 May 2019 15:20:08 +0200 In-Reply-To: <87y33ybr1a.fsf@web.de> (Michael Heerdegen's message of "Thu, 25 Apr 2019 07:19:29 +0200") Message-ID: <877eay30qf.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:Dsr2GPXDjBABRy9lJnQAJrawe0ppzpo9pyTzq5jvPKG9WxvPr9F OScyGbLxfwxtgfLRW1LRbaYdeInvC5B+wlyok7htPfWeHhZ/U6HeqDoYQlLcjjwEVmDpEa1 0rORTezudomEJYlblJW1SvPixpbp1Gq8G2lYb/0UxTYRp7OfqP5hegONRMhRQNQAdwmamou 7MRh50si+7cQ+BJ7ssmzg== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:7ATEk20CoZo=:8+ncqBShXBxUz2CqD2nY0y FxJS+BQhxhH+V+Q9kCJFytRpnkzu2H40fiQc/FbYkCbF0D3DuzVA2CHQrailChTWzuwbAak1R vU1sS9V9vmmqGwPDl2dawUzJodF+EoWE/thAnIu+If216HjTx2uAJTuMr6hyZNqy0jtwKZnjn +ZRMS8KpSP/B/hdd5pjbD81PVUCTde9rvRSIFkzFCK86yJ9PEFmq5UptINEkF0uEx9Dws8m7J smfMqf9C8wMJfPprx7b2yI0R1AMjxKwZnKlISYzog5H/g8QAgF2FN4fd+zvAZnPPu8k8RStig vjRxX7+ofOx1nnpoAYxosEzFJ1lk9riHWSUP5FiCbeUna5n1xfgoP6TCcTuqRAn7DRBFoo4N4 YCdr37xlU8y7H9cVTmSoyRmmicczeVviieoINisVaRLia85y4XP3nqOG9slKIVw4H2NxcU/I3 8c0qqNrrejYy/DJqrxSv4fvFYsJz6oTqqTOeMKoXT42qHWOXnap/0pM4rmMhHxjbyHTgMF2li 1uLbqCByt6HnXLhXHekzZ4Z2af1B8tJx8DXXwSZt0jQq7MPKPGo3S5L1JyPGnLCxXbp5AdoJY +zzJ8rL575iWhtTicVUhuSJUVBZPZQVQlRWhbNIzzdOZ5QTlMn1Ko+RZBFICvbC+Cgg5KsVCM 6A8acS/ionhq62H3tRayN+w4/fYBAqoQBHXMpGxKaXYPEyI4Vg/0SyRG/Kay0anv47gGat+XK itH/L5EEYYwGX3ttWAZV6uFWGmwMcHUicVd+AfniW7UUs6hBH4udAn5JF0MSekhTRZqfBS0nM pv7sMjxvqycVP2q6Dy/ZGCEPTHPpcZ02VxJhRWdjA8dyklgG7uh4hNg38JGgxk/NYxAr2iRpe B5+Zlgb1Z6YfALcrWMONcC1+91llHY3p9SOm7aMW1cvVQZH84Myt3lXhI2NutM6xqiO6xMzsA OIAoA7BciTQ== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Michael Heerdegen writes: > > [Patch...] > > Works for me, and it makes sense. As a test case I recompiled > el-search.el (it uses streams for several things) with your patch > applied to stream.el, and it worked well. @Nicolas: If you are short on time, can we just install this patch? I've tested it for a while and it works well, and I'm quite sure it is harmless (doesn't change any semantics apart from fixing the crashes). > > - stream-make should use cons instead of list (or maybe a struct?). > > I think cons would be ok. Would a struct make things slower? > > > - stream-empty should just be a constant. > > Dunno if there are cases where this would be problematic, but I guess we > could do this as well. @Nicolas: Do you want us to care about this or do you want to have a look yourself? I don't want to hurry, I just don't want this to be forgotten. If you say you have time in four months, it's still ok. Thanks, Michael. From debbugs-submit-bounces@debbugs.gnu.org Sat May 25 16:29:52 2019 Received: (at 30626) by debbugs.gnu.org; 25 May 2019 20:29:52 +0000 Received: from localhost ([127.0.0.1]:49739 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUdIb-0001ov-4C for submit@debbugs.gnu.org; Sat, 25 May 2019 16:29:49 -0400 Received: from mail-it1-f173.google.com ([209.85.166.173]:34065) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUdIY-0001oT-0j; Sat, 25 May 2019 16:29:47 -0400 Received: by mail-it1-f173.google.com with SMTP id g23so13858614iti.1; Sat, 25 May 2019 13:29:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=kIM6Fa9//Vdn1wdYuCP6FshnC1yhszyFFhcTByrSYvI=; b=VZS5qZeCe94dd97f6rHUFoB+5kNy8/OmHW30bVaah3hX7MdEgjYTdYiD3ueRfAzFQg /VImI5aPExMeNUVLaAP+iaRcTA9/EF1ovLjEHMznWJK1CniV+zsH5B9AP6MwManDsAg3 Dx63m0d/icHxaIjQUmTzluDziE+E9SoimNUUlQNeQxnDe4qA8q22lyXqPU0B8iB3VIqp RQrp3dHqfv7OmDBXrZbEizj2dhk6WsZuvqM3ncS4hDAGeEZy23XXnuBpbo0nqLVBWTv6 YilVL4kWRZlZSdIobvO7kEcKXIiX3iWxuL5TKXCKvW+9DY3AntNN/d/uB68H1FW+C8U6 /1PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=kIM6Fa9//Vdn1wdYuCP6FshnC1yhszyFFhcTByrSYvI=; b=Ujp2UTeGvgQCpXkrY8ovDu+Fqa9ykrfuY0n7j8iHWoP612JMboOCfCz8OXAjyFK8Pr sO3JMN3jXW2pKpt0xYlvT5NHl0C2W+ardBGeFsZlx8QWw3Fxe7YKHrsFiUyS4we+KBlQ C3Vn1Oh2Es9kGGoFXaZSOHDHp1nWT8OdmP3jTPGC99T9GTIZoz5ZnHUOz/Xs0fATqXd3 ueLi5oQBr6IPBRCAIoOBPLX1rSV7VNpv/J9i8FxBPMQFI02PgXNnb9CQnB5lqN7gvjXG LzstbaZzB81nnoFFBBZsgCMby5mKAwproa7ndPDi+fXfotriCYSByLRfWu/bH1wxDPJ3 86Gw== X-Gm-Message-State: APjAAAVhTSKIZePqUrZ/UXBOO2psYjQda98Cu3hGYhxtKVApv75ufpi5 zpKDrxRfT02oum8xjqqQ2jl6JFoQ X-Google-Smtp-Source: APXvYqxjFf/zCTsd/RLvppzB/C9rcf+KS2lYmbBcl9bjLkCFFZlXJvvgdw/2g61NFJ/JYTTdREJTNw== X-Received: by 2002:a05:660c:917:: with SMTP id s23mr22663146itj.166.1558816180112; Sat, 25 May 2019 13:29:40 -0700 (PDT) Received: from minid (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.gmail.com with ESMTPSA id b6sm712411iok.71.2019.05.25.13.29.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 25 May 2019 13:29:39 -0700 (PDT) From: Noam Postavsky To: Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <87imv2rcs3.fsf@gmail.com> <87y33ybr1a.fsf@web.de> <877eay30qf.fsf@web.de> Date: Sat, 25 May 2019 16:29:38 -0400 In-Reply-To: <877eay30qf.fsf@web.de> (Michael Heerdegen's message of "Fri, 10 May 2019 15:20:08 +0200") Message-ID: <87muja8eh9.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain tags 30626 + patch quit Michael Heerdegen writes: >> > - stream-make should use cons instead of list (or maybe a struct?). >> >> I think cons would be ok. Would a struct make things slower? A struct might be slower, and cons has the advantage that the print output is more readable for humans too. E.g., with this code: (let ((s (stream-range 1 5))) (stream-flush s) s) ;; Using cons (patch in this message): (--stream-evald-- 1 --stream-evald-- 2 --stream-evald-- 3 --stream-evald-- 4 --stream-evald--) ;; Using list (previous patch): (--stream-evald-- (1 --stream-evald-- (2 --stream-evald-- (3 --stream-evald-- (4 --stream-evald-- nil))))) ;; I guess using a struct would look something like this: #(--stream-evald-- (1 . #(--stream-evald-- (2 . #(--stream-evald-- (3 . #(--stream-evald-- (4 . #(--stream-evald-- nil))))))))) ;; Using list with thunk (current, v2.2.4) (--stream-- #[256 "\211\203\007\0\303\242\207\303\242\204 \0\304\300\242\305\300\242\302\242\\\301\302\242#B\240\210\303\306\240\210\304\242\207" [(1) 5 (1) (t) ((1 --stream-- #[256 "\211\203\007\0\303\242\207\303\242\204 \0\304\300\242\305\300\242\302\242\\\301\302\242#B\240\210\303\306\240\210\304\242\207" [(2) 5 (1) (t) ((2 --stream-- #[256 "\211\203\007\0\303\242\207\303\242\204 \0\304\300\242\305\300\242\302\242\\\301\302\242#B\240\210\303\306\240\210\304\242\207" [(3) 5 (1) (t) ((3 --stream-- #[256 "\211\203\007\0\303\242\207\303\242\204 \0\304\300\242\305\300\242\302\242\\\301\302\242#B\240\210\303\306\240\210\304\242\207" [(4) 5 (1) (t) ((4 --stream-- #[256 "\211\203\007\0\300\242\207\300\242\204\024\0\301\302\240\210\300\303\240\210\301\242\207" [(t) (nil) nil t] 3 " (fn &optional CHECK)"])) stream-range t] 7 " (fn &optional CHECK)"])) stream-range t] 7 " (fn &optional CHECK)"])) stream-range t] 7 " (fn &optional CHECK)"])) stream-range t] 7 " (fn &optional CHECK)"]) >> > - stream-empty should just be a constant. >> >> Dunno if there are cases where this would be problematic, but I guess we >> could do this as well. I've done this in the patch below. Passes all the tests, and I can't see why it would be problematic. > @Nicolas: Do you want us to care about this or do you want to have a > look yourself? I don't want to hurry, I just don't want this to be > forgotten. If you say you have time in four months, it's still ok. Not getting any response; I'll wait another week for comments and then push. --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=0001-Drop-forced-lambda-s-from-stream-Bug-30626.patch Content-Description: patch >From 7b126616c87bf034c933de711befcd80a7ada3bb Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Wed, 24 Apr 2019 22:51:19 -0400 Subject: [PATCH] Drop forced lambda's from stream (Bug#30626) Let the stream id distinguish between forced and unforced stream values. When the value is forced, replace the lambda with its result. This lets the lambda and anything it references be garbage collected. Change the representation of a stream from (--stream-- THUNK) to (--stream-fresh-- . (lambda () VALUE)) or (--stream-evald . VALUE). * packages/stream/stream.el (stream--identifier): Remove. (stream--fresh-identifier, stream--evald-identifier): New constants to replace it. (streamp): Check for new constants. (stream-make): Use cons and lambda instead of list and thunk-delay. (stream--force): New function. (stream-empty-p, stream-first, stream-rest): Use it. (stream-empty): New constant, return it from the function instead of creating a new one each time. * packages/stream/tests/stream-tests.el (stream-to-list): Remove. (stream-list-test): Use seq-into instead. --- packages/stream/stream.el | 41 +++++++++++++++++++++++++---------- packages/stream/tests/stream-tests.el | 12 ++-------- 2 files changed, 31 insertions(+), 22 deletions(-) diff --git a/packages/stream/stream.el b/packages/stream/stream.el index 3f6bc4b5b..9f73e8b86 100644 --- a/packages/stream/stream.el +++ b/packages/stream/stream.el @@ -1,6 +1,6 @@ ;;; stream.el --- Implementation of streams -*- lexical-binding: t -*- -;; Copyright (C) 2016 Free Software Foundation, Inc. +;; Copyright (C) 2016-2019 Free Software Foundation, Inc. ;; Author: Nicolas Petton ;; Keywords: stream, laziness, sequences @@ -41,7 +41,7 @@ ;; - ... ;; ;; All functions are prefixed with "stream-". -;; All functions are tested in test/automated/stream-tests.el +;; All functions are tested in tests/stream-tests.el ;; ;; Here is an example implementation of the Fibonacci numbers ;; implemented as in infinite stream: @@ -65,18 +65,30 @@ (eval-when-compile (require 'cl-lib)) (require 'seq) -(require 'thunk) (eval-and-compile - (defconst stream--identifier '--stream-- - "Symbol internally used to identify streams.")) + (defconst stream--fresh-identifier '--stream-fresh-- + "Symbol internally used to streams whose head was not evaluated.") + (defconst stream--evald-identifier '--stream-evald-- + "Symbol internally used to streams whose head was evaluated.")) (defmacro stream-make (&rest body) "Return a stream built from BODY. BODY must return nil or a cons cell whose cdr is itself a stream." (declare (debug t)) - `(list ',stream--identifier (thunk-delay ,@body))) + `(cons ',stream--fresh-identifier (lambda () ,@body))) + +(defun stream--force (stream) + "Evaluate and return the first cons cell of STREAM. +That value is the one passed to `stream-make'." + (cond + ((eq (car-safe stream) stream--evald-identifier) + (cdr stream)) + ((eq (car-safe stream) stream--fresh-identifier) + (setf (car stream) stream--evald-identifier) + (setf (cdr stream) (funcall (cdr stream)))) + (t (signal 'wrong-type-argument (list 'streamp stream))))) (defmacro stream-cons (first rest) "Return a stream built from the cons of FIRST and REST. @@ -159,24 +171,29 @@ (defun stream-range (&optional start end step) (defun streamp (stream) "Return non-nil if STREAM is a stream, nil otherwise." - (eq (car-safe stream) stream--identifier)) + (let ((car (car-safe stream))) + (or (eq car stream--fresh-identifier) + (eq car stream--evald-identifier)))) + +(defconst stream-empty (cons stream--evald-identifier nil) + "The empty stream.") (defun stream-empty () - "Return a new empty stream." - (list stream--identifier (thunk-delay nil))) + "Return the empty stream." + stream-empty) (defun stream-empty-p (stream) "Return non-nil if STREAM is empty, nil otherwise." - (null (thunk-force (cadr stream)))) + (null (cdr (stream--force stream)))) (defun stream-first (stream) "Return the first element of STREAM. Return nil if STREAM is empty." - (car (thunk-force (cadr stream)))) + (car (stream--force stream))) (defun stream-rest (stream) "Return a stream of all but the first element of STREAM." - (or (cdr (thunk-force (cadr stream))) + (or (cdr (stream--force stream)) (stream-empty))) (defun stream-append (&rest streams) diff --git a/packages/stream/tests/stream-tests.el b/packages/stream/tests/stream-tests.el index 021ed65cf..7487ef69b 100644 --- a/packages/stream/tests/stream-tests.el +++ b/packages/stream/tests/stream-tests.el @@ -1,6 +1,6 @@ ;;; stream-tests.el --- Unit tests for stream.el -*- lexical-binding: t -*- -;; Copyright (C) 2015, 2017 Free Software Foundation, Inc. +;; Copyright (C) 2015, 2017-2019 Free Software Foundation, Inc. ;; Author: Nicolas Petton @@ -29,14 +29,6 @@ (require 'stream) (require 'generator) (require 'cl-lib) -(defun stream-to-list (stream) - "Eagerly traverse STREAM and return a list of its elements." - (let (result) - (seq-do (lambda (elt) - (push elt result)) - stream) - (reverse result))) - (ert-deftest stream-empty-test () (should (streamp (stream-empty))) (should (stream-empty-p (stream-empty)))) @@ -240,7 +232,7 @@ (ert-deftest stream-range-test () (ert-deftest stream-list-test () (dolist (list '(nil '(1 2 3) '(a . b))) - (should (equal list (stream-to-list (stream list)))))) + (should (equal list (seq-into (stream list) 'list))))) (ert-deftest stream-seq-subseq-test () (should (stream-empty-p (seq-subseq (stream-range 2 10) 0 0))) -- 2.11.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat May 25 20:33:08 2019 Received: (at 30626) by debbugs.gnu.org; 26 May 2019 00:33:08 +0000 Received: from localhost ([127.0.0.1]:49961 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUh64-0006aB-Hl for submit@debbugs.gnu.org; Sat, 25 May 2019 20:33:08 -0400 Received: from mout.web.de ([212.227.15.4]:44899) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUh62-0006Zh-Hl for 30626@debbugs.gnu.org; Sat, 25 May 2019 20:33:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1558830773; bh=fBwcwLrUXCDmognS3spfb8utWoRRSQI78+6B7uDoQco=; h=X-UI-Sender-Class:From:To:Cc:Subject:References:Date:In-Reply-To; b=bcmMuKu3Za49Yeb7t9frWawRmC3G8r5Vm9T2z07w/S6uuFZrLWJAkz0M/VC5wj2cg L0dTxCZ5vmOpda1B22L2+n+QDrSR2vtj9/SIENbUDQSikuslSni3JbZfk092hLF+at 1sMrN5oW2Vb5u9F1EYZ1HvRVGmL/J8LT0vjA6XX0= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from drachen.dragon ([188.110.143.34]) by smtp.web.de (mrweb002 [213.165.67.108]) with ESMTPSA (Nemesis) id 0MUEsc-1h4oEw3tAo-00R59n; Sun, 26 May 2019 02:32:53 +0200 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <87imv2rcs3.fsf@gmail.com> <87y33ybr1a.fsf@web.de> <877eay30qf.fsf@web.de> <87muja8eh9.fsf@gmail.com> Date: Sun, 26 May 2019 02:32:49 +0200 In-Reply-To: <87muja8eh9.fsf@gmail.com> (Noam Postavsky's message of "Sat, 25 May 2019 16:29:38 -0400") Message-ID: <87h89iukb2.fsf@web.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:hHd2iUrvoWq16foRCw39CBKt749GK0P/LxxIDb9JbRzXeOjaRC0 rTyVXTsQEVg46psa1ZO2XbiM5ee7lI1gcy0kIFg7URvmjVpczAX2+WjG22Kd0MiBudNEQ43 RQ/Gaxe4IAY1MV5+r5z3HmaJbPF6RYrMG1XnnMS3F9Fc5f4ra6Gh6pg8+gK0Sdmfi4BmZ6r D4a8ajqw9NvYLpstJL06w== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:c3WSHbCanu0=:8HVVLxpKbmG87vuldxQv/j 3rNikAhemqr45tTnN/kDtVYA05FCelNc7bLf4yGxMauI/1t3TU40TWEDbTPL12VeKnYvTOaPi njUbmzA5NZZwgt8PrDGAcV/t+0IGcjEfRxKI9uU/ezEXUk2ksRZp3Qx2lvhLfZXiRGZlh3Yd7 dyGBGJu8/j+tINCjgPkg+LNfdQcgLbE1x0+7QSLX92Lr5wpb98GqH0dG+JsQI9S3+gXQBEQwI UlgbgNjValUOMSooOVlzw7ucU79eN/stioqwqMd9T7bgqK+5IJnnEogPnU0hjlVEUtancxr/s RS5n/mw4q5aGQGT6SFsJI9HpV1btbeE9lPQ+9hGMka5ZmgDbqkpRvl/e0WbZznqjCptdykAz/ mwxkGsz2LQLkVQCINRhXF9pyv5o8PM9ECUs6uEKr26grZkPuINaeJkrOHR15+K6gAV+ae4CiY XlKvT414N3qlSEFiFD8eZTaxKtKDK8IZmgFzbddtGAd7oZVCmY68R77AoBnL3iGFJo+S+5+7h ujBtdUOtzA45zTxpgTAVGo/eb8lI/IV7R6qYTkwwHZy3JrPBPLgMKLFfYqnELsjUz9EDB7gBm 1YfYOIQuyHwaV9miZn6twl2Yjq/2Gx69Rvcr9ganmeXbryBKCB26uF2tLqUUnDNuF8x8x+Fbd 9C3V368AdzmEVPWvKk85xwb+Mwf5qwhTirIbZWGJ3WqF30hFqvdnpgC6DgQQTjZ1jE6h2mo60 CGdgK3WAfIVUs1NQrklaW041YDGlQBf7J+E+fQNU0DlRdTYBwBg/NsNP9czu7mfQSz+XBPE8w fQTTNP9bEWBQp2TYceD3C7wCCEYFm0Nqb+KITEFN1SmT46As4mByd5v9VxMKZKlF0XJFf2ANO e368P/KxCc9Y+BqK8UYaeWHLCqQpmo/45mxn3v+2WZRAilJHAwNdFy2saFv+MNhsgAg5YhFyB 5jtesIZjLLdHlWLAGUs4aTeuIpVLWlkgZ8U8L/JAJElr4miUNWYbQ X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Noam Postavsky writes: > >> > - stream-make should use cons instead of list (or maybe a struct?). > >> > >> I think cons would be ok. Would a struct make things slower? > > A struct might be slower, and cons has the advantage that the print > output is more readable for humans too. E.g., with this code: [...] I see no reason not to switch to cons. Better readable, and less cons garbage than with `list'. Would you like to do this? > >> > - stream-empty should just be a constant. > >> > >> Dunno if there are cases where this would be problematic, but I > >> guess we > >> could do this as well. > > I've done this in the patch below. Passes all the tests, and I can't > see why it would be problematic. Looks good to me. Also passes my tests (which are all my stream.el uses, including el-search.el). > > @Nicolas: Do you want us to care about this or do you want to have a > > look yourself? I don't want to hurry, I just don't want this to be > > forgotten. If you say you have time in four months, it's still ok. > > Not getting any response; I'll wait another week for comments and then > push. Ok, let's do this. Thanks, Michael. From debbugs-submit-bounces@debbugs.gnu.org Sat May 25 20:40:10 2019 Received: (at 30626) by debbugs.gnu.org; 26 May 2019 00:40:10 +0000 Received: from localhost ([127.0.0.1]:49968 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUhCs-0006lW-Ar for submit@debbugs.gnu.org; Sat, 25 May 2019 20:40:10 -0400 Received: from mail-it1-f172.google.com ([209.85.166.172]:34548) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUhCp-0006lE-VW for 30626@debbugs.gnu.org; Sat, 25 May 2019 20:40:08 -0400 Received: by mail-it1-f172.google.com with SMTP id g23so14157189iti.1 for <30626@debbugs.gnu.org>; Sat, 25 May 2019 17:40:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=g7lS5QtZhaNsOB1GBLjiIYv1YOckIzgHzYUUELPQBZE=; b=h3Bt8y0NaPnFhhhU4MuSIyroHMzzeoPOpjtiksqtWo//xsQ/0m+zCDdCXlJ6qRvmGy LmiSFfHoMSZ2pxmqSxwA6q9sj20gUUfCTxjbuX308nCV2hlrz/PgBAll5GmbGoKHG4rz cq3e+PupufyRJ1nkTdRWJ4l7O4GXN/9IEvtHpXpkoT+iN4HQs5LYmoKMB3YEcGmifOfz hsLCZedejjCb7YDloMEpQ9dg7ci277OA+ASQUTHfkOV+eOnzsCGDAZP6xR+xnPoP13hN jAvpspb85fSQ8eVyTFHQvEu0JSpB+EU3ve0GQ55nXS8atLwvC3Xv6I0wDgU0l6nGzBQK xjrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=g7lS5QtZhaNsOB1GBLjiIYv1YOckIzgHzYUUELPQBZE=; b=uSr/cvI0W+cn+MQ8eyfM/4zdhReLGul98LqMkdLT6eIOror49f8+bZL8UKfkbYzej+ 5nVhwWKPlrJ2Of7BvQOMj5IEYjS+mjy74Ks2/hCZy5Px9YFU80sz1P2PiVrYME5lQbtq v7M8sT551UJXKQktauTaALAohBdAuBO9VPKWeV/JUMzErKxPJkizUlYu8tkBeHmdR6Tg D23Alh3xLas9ods3k6UWNRXl3hwARnHjFWPoJaovuR+jT5gZ88NBRqoj5IRV5s9qEyHE JUReqVaYL/GaqDAxGNlkU5lnV8kfaUkRCA/OsNxYiqEPoRgcpA7KftHaO/gi5GPZew7n VjsA== X-Gm-Message-State: APjAAAX4aJrKJBluLZ1oN0gBVbALa8MMGdEp7DEZkBliOTo7tB0uHwXO Rld9RUCHDI/BTrOtvEVOKrYxtO3N X-Google-Smtp-Source: APXvYqzTy+tkR35gJtrpfakiXxp6J56C6cmnYqRcob/TSaCzo7sbKfGmf5lSUTJItRKTaI3KYV9Ttw== X-Received: by 2002:a02:e0a:: with SMTP id 10mr3315186jae.98.1558831202086; Sat, 25 May 2019 17:40:02 -0700 (PDT) Received: from minid (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.gmail.com with ESMTPSA id p20sm2444474ioj.63.2019.05.25.17.40.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 25 May 2019 17:40:01 -0700 (PDT) From: Noam Postavsky To: Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <87imv2rcs3.fsf@gmail.com> <87y33ybr1a.fsf@web.de> <877eay30qf.fsf@web.de> <87muja8eh9.fsf@gmail.com> <87h89iukb2.fsf@web.de> Date: Sat, 25 May 2019 20:40:00 -0400 In-Reply-To: <87h89iukb2.fsf@web.de> (Michael Heerdegen's message of "Sun, 26 May 2019 02:32:49 +0200") Message-ID: <87blzq82vz.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Michael Heerdegen writes: > I see no reason not to switch to cons. Better readable, and less cons > garbage than with `list'. > > Would you like to do this? Yes, just to be clear, the patch in my previous message did already include the switch to cons. From debbugs-submit-bounces@debbugs.gnu.org Sat May 25 21:16:22 2019 Received: (at 30626) by debbugs.gnu.org; 26 May 2019 01:16:22 +0000 Received: from localhost ([127.0.0.1]:50008 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUhls-0007j9-8e for submit@debbugs.gnu.org; Sat, 25 May 2019 21:16:21 -0400 Received: from mout.web.de ([212.227.15.14]:39057) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUhlq-0007ir-5U for 30626@debbugs.gnu.org; Sat, 25 May 2019 21:16:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1558833363; bh=Rmm5n86zMy43ODpsyyD+8SP1UgdzSljo4HLNW4IpxMY=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=lprLBzO+rQ0o0oZq57QjQr1oPp9V04H2FXb/dXl6lkUq2lg7d0/JqBCpAzpMufvT9 bRPmJmAz1c+ewHzugeunK2Jnyo9yBn1PWLExHgaWyIcQ/L0NOKRehc53c96WIdb5y0 Wk+AzpaHlLGvdjT4iWtRVDeWLWAg3lmxDHbieKPQ= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from drachen.dragon ([188.110.143.34]) by smtp.web.de (mrweb002 [213.165.67.108]) with ESMTPSA (Nemesis) id 0LqlF4-1h0L1V0VbW-00eKvZ; Sun, 26 May 2019 03:16:03 +0200 From: Michael Heerdegen To: Noam Postavsky Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' In-Reply-To: <87blzq82vz.fsf@gmail.com> (Noam Postavsky's message of "Sat, 25 May 2019 20:40:00 -0400") References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <87imv2rcs3.fsf@gmail.com> <87y33ybr1a.fsf@web.de> <877eay30qf.fsf@web.de> <87muja8eh9.fsf@gmail.com> <87h89iukb2.fsf@web.de> <87blzq82vz.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Date: Sun, 26 May 2019 03:15:58 +0200 Message-ID: <87a7fauib5.fsf@web.de> MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:21bfgv7kGJMczFHwGf+phbbV3hxrS8o49MGIf4X3nPBtvRCV46F N09aF76XXcQb69EYT72zh59PAw7X7p7wFiQ50fhk1ckFlw5CtbD92ozDNyRPWsSMFg0DEr8 FdVQ/8FPUAxLfsL8dEBf1cX57YUrDzWBD7BEokaQNUPLUuuJALg9KOpEFXHs8yTS12yGNbU wtuu3K+VMMXgJmniI3afQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:5OhPCvrEsKk=:CjuyEKCj4KgsbN2KiewIUN 47ZH2Z/lSlC1DuBNZq0jmDAXwm4Zv+bU7w+Ck26yVa5yU1nSE55rDgSl+y/IBN6vFzKVNPlPb yiR7kYdqZ/8HKiRbRmthcMW6ggBGmHdT+PJBMPYZ0gEXGwi+3Pegc31w+gHc6ZmEQ2GeGnsvP G3yFWE7lpKUDpN6l2qs4xmAAzLxMmnpe4U/DAtbsLvYWImfZGrcHogjfC9V1MC6fQcc6k0h4h jnb8cIX0qn2A9lZZzDcZSbcrEMy+M3+AdKUkYftu3nvFTqnNGyn7dPpyVcGDLoYoCgJNAQVE9 xp11Ywg++46++NUR6x+eFbi7cptx59fGkx70NgMypifOIytod/tvBTB6w+StlDbhDIzcFNtl/ ujUCpKp0/0ERTpJJTQ4yfTcPez9qOL9w66QOtRsGImuAXabtjK2Ystajp4P1rEWTuNUXM2Ji9 QG/2e67yQjN0KqleqYm7CtPSOfg5fW4BXI94YAqBh0Ldt2KyUy7QR4SWLifqm6MrrysGqKK+b dDPJp7VT78omxuo/cYV8l5R7fHzS5cze5DpPzgmQypDomKleWW3qspN88nnnYxhLtkjG9PZ9C YEw8nvma+eMvJIvHbckic8nKV7omh2WZ7Y2am2a83jRtRrhJER1REEqnbIqpTPXmD2OvSVrCh H/HZA7ZL0p/H5kYThIfKX4RKBmjulmdS/t7REhG2eHd2GPVucZlcT5JCIco99uQU1HAu/ExtB X8qYm9zwEU6fqMl/2YRAhPkHEj1XOPT/7I7FXMQ+S+vyBZzBloxNKNAyeGURBsUMOIcHjsBlI aggFjIiglipIZtDe1ylgp7vgbmxYoPRk+GRJ+ukY8wh8CWbTsQN15akrWkQOZTPL7agrzclFs HVXG2OauqrzAc99ecL/N0cad2nh0zal8o0CadX9J4Za7ggH6x78PR1pN2VCbY44wQ+6G0Hxs6 Xy4nmAQYAr0EHLQgoJE2FQwIqT0ePYgbGp3NsIsT94aC+1AyLRToY X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 30626 Cc: Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Noam Postavsky writes: > Yes, just to be clear, the patch in my previous message did already > include the switch to cons. Ah - of course - sorry, I was already a bit tired, I read it but wasn't aware of it. Sorry, I had spent the time finding a bug in your patch that turned out was just my oversight of some file to recompile. Michael. From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 03 20:26:59 2019 Received: (at 30626) by debbugs.gnu.org; 4 Jun 2019 00:26:59 +0000 Received: from localhost ([127.0.0.1]:43457 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hXxI3-0007Ys-5H for submit@debbugs.gnu.org; Mon, 03 Jun 2019 20:26:59 -0400 Received: from mail-it1-f169.google.com ([209.85.166.169]:55238) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hXxI1-0007Ya-0j; Mon, 03 Jun 2019 20:26:57 -0400 Received: by mail-it1-f169.google.com with SMTP id h20so30643643itk.4; Mon, 03 Jun 2019 17:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=z0uD1fHB/xsPuRmh5XeFsOAo1z/HqWBDa5/zH7gxAm8=; b=j/AFuPCpqv4ayYtoLJwIyRlvlPNjmAx683LJQ0vKfBEKDU+m5byTcN4fgJc1Q6tvYO jNoTgmExJAjkwk4AkXudXVQisS8GH9g31JA7T62V/o8xODd1FHZvnJ0Xu0ZH4YVJKX7B hWVuMENAHcQrvPN8rQLtR+jVT+vJeL6ryJpBoS44myovGhtTQuqeT7bZ7t/qNyGE3oZ4 cOexT9bAu1e/5rPrjrIc7+txKO1u4lZjch5JdwTMjA/tXeinFK/4AaBvIUb8pe6VkNEi XJ0cHkrpEMHWwNqmV81zbiCVH250aLkIR7YH6foGsyjhZUCdFOusvUFzY0TN4RIB6R4u /yrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=z0uD1fHB/xsPuRmh5XeFsOAo1z/HqWBDa5/zH7gxAm8=; b=TWSU9ZFvR3eeVKPwfinW8+LTK4Z2fQZaqjDdWysB/Mx+vMQScJ5sbmu4AorPSKf3h3 X+aoPg8PT+B0JdEZ/aMNbqovva/QpCB/hqqzxu7eqkD6dDV4Omq8jlKgiORnKMeYYdWd RDa3draViXPK65r+B1oIIXw/asAUtGacctoqgxilmSPhR8fFzLvGBMoGvAiu2kZof9u4 2NJRBZ4Be7GVZIUhT5RVh7WRr4wSgXBfq8AOhPtxJziGMgVomttL3bnfNzGlkn4ITU/Y JIrYQOh5eumr5mCd5MDaUyKabUiajFyowF1labYFYN3gARphEUi2Px16hOYR8L8N4yY7 1qpg== X-Gm-Message-State: APjAAAUSG6XpN5J8CT0txP3E8kyD04GgjczEk2Q8IMh5gVIfmC4oiJxv UoTe5hC93+l02e0wUD4niVg/er3f X-Google-Smtp-Source: APXvYqzujwPG9xe9sGWQAo3ybyNpd4FPPvgzv0nNG8/+GH0R2kTseCKPYVSkn8Bk+33yaY+WhZOVgA== X-Received: by 2002:a24:5491:: with SMTP id t139mr18732168ita.173.1559608011113; Mon, 03 Jun 2019 17:26:51 -0700 (PDT) Received: from minid (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.gmail.com with ESMTPSA id t19sm5528259iog.41.2019.06.03.17.26.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 03 Jun 2019 17:26:50 -0700 (PDT) From: Noam Postavsky To: Michael Heerdegen Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <87imv2rcs3.fsf@gmail.com> <87y33ybr1a.fsf@web.de> <877eay30qf.fsf@web.de> <87muja8eh9.fsf@gmail.com> <87h89iukb2.fsf@web.de> <87blzq82vz.fsf@gmail.com> <87a7fauib5.fsf@web.de> Date: Mon, 03 Jun 2019 20:26:49 -0400 In-Reply-To: <87a7fauib5.fsf@web.de> (Michael Heerdegen's message of "Sun, 26 May 2019 03:15:58 +0200") Message-ID: <87v9xm42ly.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30626 Cc: Nicolas Petton , 30626@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) tags 30626 fixed close 30626 quit Pushed to elpa. b5f4061db 2019-06-03T20:23:03-04:00 "Drop forced lambda's from stream (Bug#30626)" https://git.savannah.gnu.org/cgit/emacs/elpa.git/commit/?id=b5f4061db4226d1d49fcb0ff53db0424f359e344 > I had spent the time finding a bug in your patch that turned out was > just my oversight of some file to recompile. Oh, I hate it when that happens. From unknown Sat Aug 16 00:34:25 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 02 Jul 2019 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator