GNU bug report logs - #27585
segfault when evaluating a file containing only backticks

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 27585 in the body.
You can then email your comments to 27585 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Steve Kemp <steve <at> steve.org.uk>
To: bug-gnu-emacs <at> gnu.org
Subject: segfault when evaluating a file containing only backticks
Date: Wed, 05 Jul 2017 06:21:10 +0000

  I've recently started fuzzing GNU Emacs, using the current git sources.
 During the course of that work I stumbled upon this easily reproduced bug:

   deagol ~ $ perl -e 'print "`" x ( 1024 * 1024  * 12);' > t.el
   deagol ~ $ /usr/bin/emacs --batch --script ./t.el
   ..
   Segmentation fault (core dumped)

  (So I'm trying to call "emacs --batch --script $file" where the file
 contains thousands of repeated backtick-characters.)

  Because I've built from source I can see this backtrace:

#5  handle_sigsegv (sig=11, siginfo=<optimized out>, arg=<optimized out>)
    at sysdep.c:1811
#6  <signal handler called>
#7  read1 (readcharfun=readcharfun <at> entry=35581829, 
    pch=pch <at> entry=0x7ffcc661e010, first_in_list=first_in_list <at> entry=false)
    at lread.c:2923
#8  0x0000000000ad8cda in read0 (readcharfun=35581829) at lread.c:2220
#9  read1 (readcharfun=readcharfun <at> entry=35581829, 
    pch=pch <at> entry=0x7ffcc66220c0, first_in_list=first_in_list <at> entry=false)
    at lread.c:3149
#10 0x0000000000ad8cda in read0 (readcharfun=35581829) at lread.c:2220
#11 read1 (readcharfun=readcharfun <at> entry=35581829, 
    pch=pch <at> entry=0x7ffcc6626170, first_in_list=first_in_list <at> entry=false)
    at lread.c:3149
#12 0x0000000000ad8cda in read0 (readcharfun=35581829) at lread.c:2220
#13 read1 (readcharfun=readcharfun <at> entry=35581829, 
    pch=pch <at> entry=0x7ffcc662a220, first_in_list=first_in_list <at> entry=false)
    at lread.c:3149
#14 0x0000000000ad8cda in read0 (readcharfun=35581829) at lread.c:2220
#15 read1 (readcharfun=readcharfun <at> entry=35581829, 
    pch=pch <at> entry=0x7ffcc662e2d0, first_in_list=first_in_list <at> entry=false)
    at lread.c:3149
#16 0x0000000000ad8cda in read0 (readcharfun=35581829) at lread.c:2220
#17 read1 (readcharfun=readcharfun <at> entry=35581829, 
    pch=pch <at> entry=0x7ffcc6632380, first_in_list=first_in_list <at> entry=false)
    at lread.c:3149
#18 0x0000000000ad8cda in read0 (readcharfun=35581829) at lread.c:2220
#19 read1 (readcharfun=readcharfun <at> entry=35581829, 
    pch=pch <at> entry=0x7ffcc6636430, first_in_list=first_in_list <at> entry=false)
    at lread.c:3149
#20 0x0000000000ad8cda in read0 (readcharfun=35581829) at lread.c:2220
#21 read1 (readcharfun=readcharfun <at> entry=35581829, 
    pch=pch <at> entry=0x7ffcc663a4e0, first_in_list=first_in_list <at> entry=false)
    at lread.c:3149
....


  I've replicated this upon the package of GNU Emacs as available to
 the old-stable/jessie release of Debian GNU/Linux, which identifies
 itself as:


In GNU Emacs 24.5.1 (x86_64-pc-linux-gnu, GTK+ Version 3.14.5)
 of 2016-03-19 on trouble, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11604000
System Description: Debian GNU/Linux 8.8 (jessie)

Configured using:
 `configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.5/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.5/site-lisp:/usr/share/emacs/site-lisp
 --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.5/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.5/site-lisp:/usr/share/emacs/site-lisp
 --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars
 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat
 -Werror=format-security -Wall' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-z,relro'
 

  My current-git build reports as:

In GNU Emacs 26.0.50 (build 1, x86_64-pc-linux-gnu)
 of 2017-07-05 built on kernel.default.skx.uk0.bigv.io
Repository revision: 5d62247323f53f3ae9c7d9f51e951635887b2fb6
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...

Configured using:
 'configure --prefix=/tmp/emacs/ --without-makeinfo --with-gnutls=no'

Configured features:
SOUND NOTIFY ZLIB

Important settings:
  value of $LC_ALL: en_US.UTF8
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

  "Obviously" this same bug can be reproduced inside emacs:

    1.  Open Emacs.
    2.  Create a new buffer.
    3.  Fill the buffer with `
    4.  Ctrl-x h
    5.  M-x eval-region



Steve
-- 
https://www.steve.org.uk/

Message #8 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Steve Kemp <steve <at> steve.org.uk>
Cc: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Wed, 05 Jul 2017 09:51:46 +0200

On Jul 05 2017, Steve Kemp <steve <at> steve.org.uk> wrote:

>   Because I've built from source I can see this backtrace:
>
> #5  handle_sigsegv (sig=11, siginfo=<optimized out>, arg=<optimized out>)
>     at sysdep.c:1811
> #6  <signal handler called>
> #7  read1 (readcharfun=readcharfun <at> entry=35581829, 
>     pch=pch <at> entry=0x7ffcc661e010, first_in_list=first_in_list <at> entry=false)
>     at lread.c:2923

Most likely just a stack overflow.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #11 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Steve Kemp <steve <at> steve.org.uk>
To: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Wed, 05 Jul 2017 08:26:53 +0000

> > #5  handle_sigsegv (sig=11, siginfo=<optimized out>, arg=<optimized out>)
> >     at sysdep.c:1811
> > #6  <signal handler called>
> > #7  read1 (readcharfun=readcharfun <at> entry=35581829, 
> >     pch=pch <at> entry=0x7ffcc661e010, first_in_list=first_in_list <at> entry=false)
> >     at lread.c:2923
> 
> Most likely just a stack overflow.

  Agreed, but still I think a segfault is unexpected and could be
 prevented.

Steve
-- 
https://www.steve.org.uk/

Message #14 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Kemp <steve <at> steve.org.uk>
Cc: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Wed, 05 Jul 2017 21:41:45 +0300

> From: Steve Kemp <steve <at> steve.org.uk>
> Date: Wed, 05 Jul 2017 06:21:10 +0000
> 
> 
>   I've recently started fuzzing GNU Emacs, using the current git sources.
>  During the course of that work I stumbled upon this easily reproduced bug:
> 
>    deagol ~ $ perl -e 'print "`" x ( 1024 * 1024  * 12);' > t.el
>    deagol ~ $ /usr/bin/emacs --batch --script ./t.el
>    ..
>    Segmentation fault (core dumped)

Here it says:

  Re-entering top level after C stack overflow

and doesn't crash.

> > Most likely just a stack overflow.
> 
>   Agreed, but still I think a segfault is unexpected and could be
>  prevented.

See above: the machinery to try and prevent it exists, but it doesn't
always succeed.  And it really can't be 100% reliable.  So I'm unsure
what did you expect, and why.  Emacs generally gives you enough rope
to hang yourself; it's up to you not to be tempted to do so...

IOW: why would someone want to run such a silly "program"?

Message #17 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Steve Kemp <steve <at> steve.org.uk>
To: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Wed, 05 Jul 2017 18:55:31 +0000

> >    deagol ~ $ perl -e 'print "`" x ( 1024 * 1024  * 12);' > t.el
> >    deagol ~ $ /usr/bin/emacs --batch --script ./t.el
> >    ..
> >    Segmentation fault (core dumped)
> 
> Here it says:
> 
>   Re-entering top level after C stack overflow
> 
> and doesn't crash.

  That's great.  For me it crashes on all the versions I've tested.
 Sometimes with a shorter input, others need longer, but it had seemed
 universal.

> See above: the machinery to try and prevent it exists, but it doesn't
> always succeed.  And it really can't be 100% reliable.  So I'm unsure
> what did you expect, and why.

  Honestly?  I expect Emacs to not crash.  The mechanics might be
 simple, or they might be complex but as a mere user I shouldn't
 be expected to know or care about the details.  I expect evaluating
 lisp to not kill the editor - although I appreciate that executing
 unknown could code do all kinds of nasty things, from uploading
 my images to the internet, to running "rm -rf $HOME".

  The fine manual, in "54.2 When Is There a Bug", agrees.  When
 paraphrased as "segfault == bug".

> IOW: why would someone want to run such a silly "program"?

  In the real world?  Nobody.  It was just the first crash that
 came out of fuzz-testing.  Perhaps there will be more subtle
 and interesting ones to report in the future when the fuzzer
 has ran for a few more days/weeks.  Perhaps not.  Either way
 I'd regard it as a bug that should be fixed, even if it is
 not a security hole, and not something that is going to surprise
 users in practice.

Steve
-- 
https://steve.fi/

Message #20 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Kemp <steve <at> steve.org.uk>
Cc: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Wed, 05 Jul 2017 22:47:22 +0300

> From: Steve Kemp <steve <at> steve.org.uk>
> Date: Wed, 05 Jul 2017 18:55:31 +0000
> 
> > See above: the machinery to try and prevent it exists, but it doesn't
> > always succeed.  And it really can't be 100% reliable.  So I'm unsure
> > what did you expect, and why.
> 
>   Honestly?  I expect Emacs to not crash.

You wrote a program that triggers infinite recursion.  Such programs
will crash in most, if not all, languages.  So your expectations are
unrealistic.

>  I expect evaluating lisp to not kill the editor

Valid Lisp, I agree.  But yours isn't.

Moreover, there are those among us (I'm not one of them) who thinks
Emacs shouldn't even try to recover from stack overflow, they say it
should crash hard right there and then.

So your expectations are not necessarily shared, even as aspirations,
by some developers.

> > IOW: why would someone want to run such a silly "program"?
> 
>   In the real world?  Nobody.

Then why are we discussing this use case?  Let's talk about
more practical and interesting cases.

Message #23 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Steve Kemp <steve <at> steve.org.uk>
To: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 03:46:35 +0000

> >  I expect evaluating lisp to not kill the editor
>
> Valid Lisp, I agree.  But yours isn't.

  Here we disagree.

  I enjoy reporting bugs when invalid things are tried, because I
 expect software to be robust, or as much as possible.  If you feed
 invalid script to gawk, perl, emacs, etc, etc, I expect an error
 not a segfault.

  I appreciate there are often difficult cases, such as infinite
 recursion, division by zero, and memory exhaustion, but the idea
 that ones programming environment is free to crash if your code
 is wrong is .. unpleasant.

> >   In the real world?  Nobody.
>
> Then why are we discussing this use case?  Let's talk about
> more practical and interesting cases.

  I read from this that I should not bother running any more
 fuzzing, after all if it does result in any crashes they're a result
 of bogus-coding that would never be hit in the real world.

  That's unfortunate, but I've no particular wish to argue
 in public.    I will follow your suggestion.

  Bug left open because it crashes for me, if it fails to crash
 for others with a somewhat reasonable error message I guess
 that is good.

Steve
--

Message #26 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Steve Kemp <steve <at> steve.org.uk>, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 08:46:14 +0200

On Jul 05 2017, Eli Zaretskii <eliz <at> gnu.org> wrote:

> You wrote a program that triggers infinite recursion.

This isn't infinite recursion since the input is finite.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #29 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Kemp <steve <at> steve.org.uk>
Cc: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 18:16:45 +0300

> From: Steve Kemp <steve <at> steve.org.uk>
> Date: Thu, 06 Jul 2017 03:46:35 +0000
> 
>   I enjoy reporting bugs when invalid things are tried, because I
>  expect software to be robust, or as much as possible.  If you feed
>  invalid script to gawk, perl, emacs, etc, etc, I expect an error
>  not a segfault.

I enjoy seeing _any_ bugs reported about Emacs, because we want to
make Emacs as robust as possible.  Emacs tries very hard not to crash,
but sometimes the techniques we use cannot work reliably.  As in this
case.  There's a difference between "as robust as possible" and "100%
robust", and expecting not to see segfault _at_all_ from a system with
a full-fledged programming language built into it is unrealistic,
especially when a program deliberately tries to cause Emacs to run out
of memory.

>   I appreciate there are often difficult cases, such as infinite
>  recursion, division by zero, and memory exhaustion, but the idea
>  that ones programming environment is free to crash if your code
>  is wrong is .. unpleasant.

Emacs is not a programming environment.  Emacs is an editor and a
text-processing environment with a built-in extension language.  The
extension language is provided for extending the editor, not for
running arbitrary programs that have no relation to the editor.  You
can, of course, do the latter if you wish, but that is not the main
use case for Emacs, and not the one for which it is optimized.  Making
such a marginal use case more important than it is will tax the much
more important uses of Emacs as an editor and text-processing tool.

Emacs is required not to crash when used in its main roles, or when it
runs one of the packages provided with it.  It should also try very
hard not to crash in other cases as well, but it is not _required_ to
do so, not at all costs anyway.

What that means is that each crash should first be carefully analyzed
in order to understand the underlying reasons and factors which led to
the crash.  Having found the reasons, if there are reasonable ways of
resolving them to prevent the crash, that shall be done.  A crash that
happens during some use case that belongs to the main Emacs use
patterns increases our motivation to seek for a solution, even if the
solutions are hard to find or require complicated changes.  Crashes
outside of the patterns we care about don't produce such an added
motivation, but are still solved if the solution is reasonably
practical and doesn't adversely affects other important uses.

I'm sorry I have to describe all this, which I have no doubt you
understand very well.  It's just that this discussion until now seems
to somehow ignore these simple and clear-cut considerations, without
which I feel the discussion loses its important context, and you
somehow feel that this project is not interested in hearing about
bugs, which is simply not true.

> > >   In the real world?  Nobody.
> >
> > Then why are we discussing this use case?  Let's talk about
> > more practical and interesting cases.
> 
>   I read from this that I should not bother running any more
>  fuzzing, after all if it does result in any crashes they're a result
>  of bogus-coding that would never be hit in the real world.

I suggested nothing of the kind.  Fuzzing can uncover any number of
problems of different nature.  Each one of those should be analyzed
first, before the decision is made whether it should be fixed and with
what priority.  I presume that the above is due to some offense you
took from what I wrote, which is why I thought it was important to
explain what I think should be the way of handling bug reports -- any
bug reports -- submitted against Emacs.

>   That's unfortunate, but I've no particular wish to argue
>  in public.    I will follow your suggestion.

Once again, there was no suggestion from me that fuzzing will never
uncover any bugs we will consider important to fix, or even
unimportant but easy enough to fix.  Each bug report is analyzed on
its own right, and the decision whether to fix it and at what priority
is made independently for each one of them.  It doesn't matter what
method was used to trigger the problem, the analysis and the
conclusion consider the problem itself, not the method used to find
it.

In this case, Emacs already does everything we knew was possible with
modern operating systems (and some of us think we do too much).  Emacs
tries to avoid stack overflow both on the Lisp level and on the C
level, with some non-negligible degree of success.  It is entirely
clear to us that a cleverly constructed Lisp program could circumvent
these defenses and cause a segfault anyway, but Emacs is not supposed
to be a 100% safe environment for running such malicious programs;
crashing for them is IMO way better than some other possible outcomes,
like wiping out the filesystem or wedging the OS kernel.

In sum, I hope you will continue trying to break Emacs and will report
any "successes", so we could improve Emacs in the future.

Thanks.

Message #32 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 18:19:14 +0300

> From: Andreas Schwab <schwab <at> suse.de>
> Cc: Steve Kemp <steve <at> steve.org.uk>,  27585 <at> debbugs.gnu.org
> Date: Thu, 06 Jul 2017 08:46:14 +0200
> 
> On Jul 05 2017, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> > You wrote a program that triggers infinite recursion.
> 
> This isn't infinite recursion since the input is finite.

If we want to be pedantically accurate, then no "infinite" recursion
is ever truly infinite, since it will always stop when the system is
shut down, or the power fails, or the world ends, whichever happens
first.

So we are actually always talking about recursions deep enough to blow
up the stack.

Message #35 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 17:31:35 +0200

On Jul 06 2017, Eli Zaretskii <eliz <at> gnu.org> wrote:

>> From: Andreas Schwab <schwab <at> suse.de>
>> Cc: Steve Kemp <steve <at> steve.org.uk>,  27585 <at> debbugs.gnu.org
>> Date: Thu, 06 Jul 2017 08:46:14 +0200
>> 
>> On Jul 05 2017, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> 
>> > You wrote a program that triggers infinite recursion.
>> 
>> This isn't infinite recursion since the input is finite.
>
> If we want to be pedantically accurate, then no "infinite" recursion
> is ever truly infinite, since it will always stop when the system is
> shut down, or the power fails, or the world ends, whichever happens
> first.

This is not a useful way to categorize infinite recursion.  In this
case, the recursion is always bounded by the size of the input.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #38 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Steve Kemp <steve <at> steve.org.uk>
To: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 15:33:42 +0000

> I presume that the above is due to some offense you
> took from what I wrote, which is why I thought it was important to
> explain what I think should be the way of handling bug reports -- any
> bug reports -- submitted against Emacs.

  I appreciate the time you took to do so, but no, no offense was
 taken.

  I only thought stopping was sensible because my experience is
 that fuzzing tends to discover are things that are malformed, broken,
 and almost never the kind of things that a user would intend to do.

  In short your initial response to this particular case seems
 like it would apply to any similar issue which  is liable to
 be discovered - they'd be genuine bugs, but they'd also be things
 that would never happen "in the wild", so while they wouldn't
 be ignored, it would be easy to push them to the back of the queue.

> In sum, I hope you will continue trying to break Emacs and will report
> any "successes", so we could improve Emacs in the future.

  THanks.  I'm sitting on a couple of hundred crashing cases, just
 trying to simplify them and see if they share the same cause (they
 probably do).

Steve
-- 
https://steve.fi/

Message #41 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andreas Schwab <schwab <at> suse.de>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 18:37:37 +0300

> From: Andreas Schwab <schwab <at> suse.de>
> Cc: steve <at> steve.org.uk,  27585 <at> debbugs.gnu.org
> Date: Thu, 06 Jul 2017 17:31:35 +0200
> 
> > If we want to be pedantically accurate, then no "infinite" recursion
> > is ever truly infinite, since it will always stop when the system is
> > shut down, or the power fails, or the world ends, whichever happens
> > first.
> 
> This is not a useful way to categorize infinite recursion.

Neither is this hair-splitting.

Message #44 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 17:41:39 +0200

On Jul 06 2017, Eli Zaretskii <eliz <at> gnu.org> wrote:

>> From: Andreas Schwab <schwab <at> suse.de>
>> Cc: steve <at> steve.org.uk,  27585 <at> debbugs.gnu.org
>> Date: Thu, 06 Jul 2017 17:31:35 +0200
>> 
>> > If we want to be pedantically accurate, then no "infinite" recursion
>> > is ever truly infinite, since it will always stop when the system is
>> > shut down, or the power fails, or the world ends, whichever happens
>> > first.
>> 
>> This is not a useful way to categorize infinite recursion.
>
> Neither is this hair-splitting.

Nothing of this is hair-splitting.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #47 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Eli Zaretskii <eliz <at> gnu.org>,Steve Kemp <steve <at> steve.org.uk>
Cc: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 08:48:44 -0700

On July 5, 2017 11:41:45 AM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> From: Steve Kemp <steve <at> steve.org.uk>
>> Date: Wed, 05 Jul 2017 06:21:10 +0000
>> 
>> 
>>   I've recently started fuzzing GNU Emacs, using the current git
>sources.
>>  During the course of that work I stumbled upon this easily
>reproduced bug:
>> 
>>    deagol ~ $ perl -e 'print "`" x ( 1024 * 1024  * 12);' > t.el
>>    deagol ~ $ /usr/bin/emacs --batch --script ./t.el
>>    ..
>>    Segmentation fault (core dumped)
>
>Here it says:
>
>  Re-entering top level after C stack overflow
>
>and doesn't crash.
>
>> > Most likely just a stack overflow.
>> 
>>   Agreed, but still I think a segfault is unexpected and could be
>>  prevented.
>
>See above: the machinery to try and prevent it exists, but it doesn't
>always succeed.  And it really can't be 100% reliable.  So I'm unsure
>what did you expect, and why.  Emacs generally gives you enough rope
>to hang yourself; it's up to you not to be tempted to do so...

This argument doesn't make sense to me. If we're happy letting elisp segfault, why bounds check AREF? 

Other managed runtimes --- Java, C# --- are perfectly capable of reliably detecting and recovering from stack exhaustion. There is absolutely no reason, aside from an implementation defect, for the elisp runtime not to do the same.

Stack overflow detection could be made perfectly reliable.


>
>IOW: why would someone want to run such a silly "program"?

Message #50 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Eli Zaretskii <eliz <at> gnu.org>,Steve Kemp <steve <at> steve.org.uk>
Cc: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 08:52:44 -0700

On July 5, 2017 12:47:22 PM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> From: Steve Kemp <steve <at> steve.org.uk>
>> Date: Wed, 05 Jul 2017 18:55:31 +0000
>> 
>> > See above: the machinery to try and prevent it exists, but it
>doesn't
>> > always succeed.  And it really can't be 100% reliable.  So I'm
>unsure
>> > what did you expect, and why.
>> 
>>   Honestly?  I expect Emacs to not crash.
>
>You wrote a program that triggers infinite recursion.  Such programs
>will crash in most, if not all, languages.  So your expectations are
>unrealistic.
>
>>  I expect evaluating lisp to not kill the editor
>
>Valid Lisp, I agree.  But yours isn't.
>
>Moreover, there are those among us (I'm not one of them) who thinks
>Emacs shouldn't even try to recover from stack overflow, they say it
>should crash hard right there and then.

Native stack? Certainly. The current approach, a signal handler that longjmps to top-level, cannot possibly work reliability, since it interrupts and abandons whatever the code is doing. If it has some kind of lock held and you try to take that lock again, you deadlock. Data structures might be in completely incoherent states. The last time we had this discussion, someone asserted that the worst that could happen might be a "memory leak". That's very wrong.

This signal handler is a huge, ticking time bomb, and I completely turn it off in my Emacs. Everyone else should too.

Recovering when elisp blows the stack is a different matter.


>
>So your expectations are not necessarily shared, even as aspirations,
>by some developers.
>
>> > IOW: why would someone want to run such a silly "program"?
>> 
>>   In the real world?  Nobody.
>
>Then why are we discussing this use case?  Let's talk about
>more practical and interesting cases.

Message #53 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 19:19:40 +0300

> Date: Thu, 06 Jul 2017 08:52:44 -0700
> CC: 27585 <at> debbugs.gnu.org
> From: Daniel Colascione <dancol <at> dancol.org>
> 
> >Moreover, there are those among us (I'm not one of them) who thinks
> >Emacs shouldn't even try to recover from stack overflow, they say it
> >should crash hard right there and then.
> 
> Native stack? Certainly.

Yes, this discussion is about the native stack, not the ELisp stack.

> Recovering when elisp blows the stack is a different matter.

I believe we already do that.

Message #56 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Steve Kemp <steve <at> steve.org.uk>
Cc: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 19:24:23 +0300

> From: Steve Kemp <steve <at> steve.org.uk>
> Date: Thu, 06 Jul 2017 15:33:42 +0000
> 
>   In short your initial response to this particular case seems
>  like it would apply to any similar issue which  is liable to
>  be discovered - they'd be genuine bugs, but they'd also be things
>  that would never happen "in the wild", so while they wouldn't
>  be ignored, it would be easy to push them to the back of the queue.

That wasn't my intent, and I apologize if my wording somehow implied
that.  I was only talking about the particular crash you reported.
Any other crash should be analyzed separately, and the conclusion
could very well be different.

>   THanks.  I'm sitting on a couple of hundred crashing cases, just
>  trying to simplify them and see if they share the same cause (they
>  probably do).

Well, I hope some of them are for different reasons.

It is strange you cannot trigger the stack-overflow protection in your
build, though.  If you write a trivial infinite-recursion Lisp
function, and then run it after lifting max-lisp-eval-depth and
max-specpdl-size to the largest positive number, don't you see what I
saw on my system?

Message #59 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Colascione <dancol <at> dancol.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 09:37:30 -0700

On July 6, 2017 9:19:40 AM PDT, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> Date: Thu, 06 Jul 2017 08:52:44 -0700
>> CC: 27585 <at> debbugs.gnu.org
>> From: Daniel Colascione <dancol <at> dancol.org>
>> 
>> >Moreover, there are those among us (I'm not one of them) who thinks
>> >Emacs shouldn't even try to recover from stack overflow, they say it
>> >should crash hard right there and then.
>> 
>> Native stack? Certainly.
>
>Yes, this discussion is about the native stack, not the ELisp stack.

Sort of --- an elisp file is driving the native code to blow its stack. I really don't think that any valid syntax should cause Emacs to segfault.

In this instance, couldn't we add a hard threshold to limit recursion?


>
>> Recovering when elisp blows the stack is a different matter.
>
>I believe we already do that.

We have a threshold. It's conservative, but it works most of the time. IMHO, explicit stack probing would be both less conservative and note robust.

Message #62 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Daniel Colascione <dancol <at> dancol.org>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Thu, 06 Jul 2017 20:27:59 +0300

> Date: Thu, 06 Jul 2017 09:37:30 -0700
> CC: steve <at> steve.org.uk,27585 <at> debbugs.gnu.org
> From: Daniel Colascione <dancol <at> dancol.org>
> 
> In this instance, couldn't we add a hard threshold to limit recursion?

I'm not sure I understand the proposal.  Could you elaborate?

> >> Recovering when elisp blows the stack is a different matter.
> >
> >I believe we already do that.
> 
> We have a threshold. It's conservative, but it works most of the time. IMHO, explicit stack probing would be both less conservative and note robust.

Won't it be expensive?  Or maybe I don't have a clear idea what you
meant by "stack probing".

Message #67 received at 27585-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Steve Kemp <steve <at> steve.org.uk>
Cc: 27585-done <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Fri, 14 Jul 2017 05:09:34 -0700

[Message part 1 (text/plain, inline)]

Thanks for reporting the bug. I reproduced the problem on Fedora 26 x86-64, 
fixed it in master by applying the attached patch, and am boldly marking the bug 
as fixed.

As Eli and Daniel mentioned, this area of Emacs cannot be 100% reliable and to 
some extent is indeed a "ticking time bomb". That being said, the problem in 
this particular case was that Emacs had a bad heuristic for guessing whether a 
segmentation violation address was due to stack overflow on GNU/Linux. This bad 
heuristic has been in place for years without anybody reporting it. It's good 
that we fixed this bug (though I hope "normal" users never notice the bug fix :-).

[0001-Improve-stack-overflow-heuristic-on-GNU-Linux.patch (text/x-patch, attachment)]

Message #70 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Fri, 14 Jul 2017 16:30:16 +0300

> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Fri, 14 Jul 2017 05:09:34 -0700
> Cc: 27585-done <at> debbugs.gnu.org
> 
> It's good that we fixed this bug (though I hope "normal" users never
> notice the bug fix :-).

Indeed, thanks.

Message #73 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Steve Kemp <steve <at> steve.org.uk>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Sat, 15 Jul 2017 05:03:08 +0000

On Fri Jul 14, 2017 at 16:30:16 +0300, Eli Zaretskii wrote:

> > It's good that we fixed this bug (though I hope "normal" users never
> > notice the bug fix :-).
> 
> Indeed, thanks.

  Thanks again.  I  do note one behaviour change though, which is
 perhaps worth noting.

  Previously this segfaulted:

    $ perl -e 'print "," x ( 1024 * 1024  * 12);' > t.el
    $ /tmp/emacs/bin/emacs -Q --batch --script ./t.el

  Now it does not, as the bug is fixed, but I'd expect the
 emacs process to terminate when the script is loaded. 

  That doesn't happen though:

    $ /tmp/emacs/bin/emacs -Q --batch --script ./t.el
    Re-entering top level after C stack overflow
    [emacs still running]

  Compare that with the normal example:

    $ echo "(defun foo() "Test")" > t.el
    $ /tmp/emacs/bin/emacs -Q --batch --script ./t.el
    $ 

  Perhaps not a huge problem, but it is a change I think?

Steve
-- 
https://steve.fi/

Message #76 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Steve Kemp <steve <at> steve.org.uk>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Fri, 14 Jul 2017 22:12:08 -0700

Steve Kemp wrote:
>      $ /tmp/emacs/bin/emacs -Q --batch --script ./t.el
>      Re-entering top level after C stack overflow
>      [emacs still running]
> 
>    Compare that with the normal example:
> 
>      $ echo "(defun foo() "Test")" > t.el
>      $ /tmp/emacs/bin/emacs -Q --batch --script ./t.el
>      $
> 
>    Perhaps not a huge problem, but it is a change I think?

After a stack overflow, Emacs drops what it's doing and goes to the top level, 
regardless of its command-line arguments. Perhaps it would be better if Emacs 
did something else for this particular case. It's low priority, though.

Message #79 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: steve <at> steve.org.uk, 27585 <at> debbugs.gnu.org
Subject: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Sat, 15 Jul 2017 10:15:46 +0300

> Cc: 27585 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Date: Fri, 14 Jul 2017 22:12:08 -0700
> 
> Steve Kemp wrote:
> >      $ /tmp/emacs/bin/emacs -Q --batch --script ./t.el
> >      Re-entering top level after C stack overflow
> >      [emacs still running]
> > 
> >    Compare that with the normal example:
> > 
> >      $ echo "(defun foo() "Test")" > t.el
> >      $ /tmp/emacs/bin/emacs -Q --batch --script ./t.el
> >      $
> > 
> >    Perhaps not a huge problem, but it is a change I think?
> 
> After a stack overflow, Emacs drops what it's doing and goes to the top level, 
> regardless of its command-line arguments. Perhaps it would be better if Emacs 
> did something else for this particular case. It's low priority, though.

I think the idea is to let the user do whatever is necessary to end
the session as quickly as possible.  Granted, it is optimized for
interactive usage, since the goal of the stack-overflow protection is
to avoid losing unrelated edits due to some code that's gone awry, and
that makes little sense in non-interactive sessions.  So batch
invocations might indeed do something else, like exit.  I agree that
the priority is not too high, because again, stack overflow is most
harmful in interactive sessions.

Message #86 received at 27585 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 27585 <at> debbugs.gnu.org
Subject: Fwd: Re: bug#27585: segfault when evaluating a file containing only
 backticks
Date: Sun, 24 Sep 2017 15:47:32 -0700

[Sending this again since the bug was archived and read-only earlier today.]

-------- Forwarded Message --------
Subject: Re: bug#27585: segfault when evaluating a file containing only backticks
Date: Sun, 24 Sep 2017 11:32:44 -0700
From: Paul Eggert <eggert <at> cs.ucla.edu>

Philipp Stephani wrote:
> Paul Eggert <eggert <at> cs.ucla.edu> schrieb am Fr., 14. Juli 2017 um 14:10 Uhr:
> 
>> As Eli and Daniel mentioned, this area of Emacs cannot be 100% reliable
>>
> 
> Why? Surely Emacs could switch away from recursion for the reader and the
> evaluator to an explicitly-managed stack. That would be a bit of work, but
> certainly not impossible.

You are right, on both points. By "this area of Emacs" I was assuming the 
current recursive implementation. It's not the only place Emacs recurses, though.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.