GNU bug report logs - #7876
Crash: infinite recursion in next_element_from_buffer

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 7876 in the body.
You can then email your comments to 7876 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: bug-gnu-emacs <at> gnu.org
Subject: Crash: infinite recursion in next_element_from_buffer
Date: Thu, 20 Jan 2011 14:49:31 -0500

[Message part 1 (text/plain, inline)]

Package: Emacs
Version: 24.0.50

I'm working on moving compile.el's parsing away from font-lock and using
the new code, I triggered a bug somewhere in the redisplay, that results
in a crash because of an infinite recursion:

The stack trace shows (other than the top items which vary depending on
the exact moment the maximum stack size was hit) that the recursion has
the following trivial pattern:

  [...]
  #8  0x080a8ea8 in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6655
  #9  0x080a8ebd in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6656
  #10 0x080a8ebd in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6656
  [...]

where 6656 is the second line of:

	  handle_stop (it);
	  return GET_NEXT_DISPLAY_ELEMENT (it);

I can reproduce it easily, but not reliably.  The changed code in
compile.el has not been heavily tested, so the trigger for this bug may
be something like an odd value of the `face' or
`font-lock-face' properties.

Here's what I do to reproduce it:

   % src/emacs -Q -l "newcompile.el"
   M-x grep TAB TAB RET
   Fset_window_config src/*.c RET

I've attached the "offending" new compile.el.  This does not always
crash, but almost.  The TAB TAB is on purpose to bring up the
*completion* buffer, because it seemed to make the bug show up
more frequently.
   
  


In GNU Emacs 24.0.50.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2011-01-20 on ceviche
Windowing system distributor `The X.Org Foundation', version 11.0.10707000
configured using `configure  'CFLAGS=-Wall -Wno-pointer-sign -DUSE_LISP_UNION_TYPE -DSYNC_INPUT -DENABLE_CHECKING -DXASSERTS -DFONTSET_DEBUG -g -O1 -I/usr/include/GNUstep' '--enable-maintainer-mode' '--with-x-toolkit=lucid''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: fr_CH.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Minibuffer-Area

Minor modes in effect:
  electric-pair-mode: t
  electric-indent-mode: t
  url-handler-mode: t
  global-reveal-mode: t
  reveal-mode: t
  auto-insert-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<switch-frame> <switch-frame> M-x r e p o <tab> r <tab> 
<return>

Recent messages:
test42
test5
test6
Loading /home/monnier/etc/emacs/X11.el (source)...done
Loading /home/monnier/etc/emacs/custom.el (source)...done
Ispell-kill: nil american
Starting new Ispell process [american] ...
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...
Quit

Load-path shadows:
None found.

Features:
(shadow sort mail-extr message sendmail rfc822 mml mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mailabbrev mail-utils gmm-utils mailheader emacsbug server noutline
outline easy-mmode flyspell ispell eldoc checkdoc regexp-opt thingatpt
help-mode easymenu view prog-mode electric url-handlers url-parse
auth-source netrc gnus-util url-vars mm-util mail-prsvr reveal
autoinsert uniquify advice help-fns advice-preload savehist
minibuf-eldef cl cl-loaddefs proof-site proof-autoloads pg-vars
bbdb-autoloads agda2 tooltip ediff-hook vc-hooks lisp-float-type mwheel
x-win x-dnd tool-bar dnd fontset image fringe lisp-mode register page
newcomment menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs
button faces cus-face files text-properties overlay md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting system-font-setting
font-render-setting x-toolkit x multi-tty emacs)

[compile.el (application/emacs-lisp, attachment)]

Message #8 received at 7876 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: 7876 <at> debbugs.gnu.org
Subject: Re: bug#7876: Crash: infinite recursion in next_element_from_buffer
Date: Thu, 20 Jan 2011 16:14:10 -0500

[Message part 1 (text/plain, inline)]

found 7876 23.2.92
thanks

>    % src/emacs -Q -l "newcompile.el"
>    M-x grep TAB TAB RET
>    Fset_window_config src/*.c RET

I've just reproduced this same crash in the emacs-23 banch, after
changing my compile.el to not rely on syntax-propertize (see new
version attached).
In case you use a different grep pattern, it seems that the crash
happens mostly when you reach the end of the *grep* buffer.  And if the
crash doesn't happen first time around, you can `g' to re-run grep.


        Stefan

[compile.el (application/emacs-lisp, attachment)]

Message #13 received at 7876 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 7876 <at> debbugs.gnu.org
Subject: Re: bug#7876: Crash: infinite recursion in next_element_from_buffer
Date: Thu, 20 Jan 2011 18:36:51 -0500

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Date: Thu, 20 Jan 2011 16:14:10 -0500
> Cc: 
> 
> I've just reproduced this same crash in the emacs-23 banch, after
> changing my compile.el to not rely on syntax-propertize (see new
> version attached).

That at least means the crash is unrelated to the bidirectional
display.

Can you show where it crashes in Emacs 23 (a portion of the
backtrace)?

Also, what are the few calls before (in the callstack) the first call
to next_element_from_buffer that recurses?

Message #16 received at 7876 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 7876 <at> debbugs.gnu.org
Subject: Re: bug#7876: Crash: infinite recursion in next_element_from_buffer
Date: Thu, 20 Jan 2011 18:50:34 -0500

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Date: Thu, 20 Jan 2011 14:49:31 -0500
> Cc: 
> 
>   #8  0x080a8ea8 in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6655
>   #9  0x080a8ebd in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6656
>   #10 0x080a8ebd in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6656
>   [...]
> 
> where 6656 is the second line of:
> 
> 	  handle_stop (it);
> 	  return GET_NEXT_DISPLAY_ELEMENT (it);

Looks like handle_stop is not updating it->stop_charpos, for some
reason.  Could you verify that this is the case?  The GDB command
"pit" will display the important fields of the iterator object,
including stop_charpos.

If this is true, please trace through handle_stop, and see why this
happens.  It has this code near the end:

  /* Determine where to stop next.  */
  if (handled == HANDLED_NORMALLY)
    compute_stop_pos (it);

If compute_stop_pos is never called, it could explain the infinite
recursion, but then we will have to find out why `handled' never gets
set to HANDLED_NORMALLY.

I will be unable to try debugging this myself, at least for the next
few days.  But I can explain the logic behind the
next_element_from_buffer & handle_stop dance, if you don't know that
already.  The idea is quite simple.  Let me know if you need this.

HTH

Message #21 received at 7876-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 7876-done <at> debbugs.gnu.org
Subject: Re: bug#7876: Crash: infinite recursion in next_element_from_buffer
Date: Mon, 24 Jan 2011 17:09:58 -0500

Thanks a lot, Eli, I think I've found the problem:

the stop_pos is stuck there, because it's the end of the buffer, but
end_charpos points further than ZV.

The reason for that is that grep.el's font-lock code doesn't just
add/remove text properties but also removes text.  In the old
compile.el, jit-lock is disabled so font-lock is run when the text is
inserted, but with my new compile.el, jit-lock can be (and is, by
default) enabled, so the end_charpos becomes invalid after running
jit-lock, and the redisplay code doesn't seem to pay attention to
this possibility.

I'm not very happy with grep.el's modifying the buffer like that from
font-lock-keywords, but neither is it OK for the redisplay to get stuck
in such an inf-loop.

I've installed the patch below in the emacs-23 branch, to account for
this particular case, although the jit-lock code could do nastier things
(e.g. add/remove text before the current position, thus rendering
IT_CHARPOS invalid).


        Stefan


=== modified file 'src/xdisp.c'
--- src/xdisp.c	2011-01-02 23:50:46 +0000
+++ src/xdisp.c	2011-01-24 22:02:26 +0000
@@ -3337,6 +3337,8 @@
       val = Vfontification_functions;
       specbind (Qfontification_functions, Qnil);
 
+      xassert (it->end_charpos == ZV);
+      
       if (!CONSP (val) || EQ (XCAR (val), Qlambda))
 	safe_call1 (val, pos);
       else
@@ -3376,6 +3378,13 @@
 
       unbind_to (count, Qnil);
 
+      /* The fontification code may have added/removed text.
+	 It could do even a lot worse, but let's at least protect against
+	 the most obvious case where only the text past `pos' gets changed',
+	 as is/was done in grep.el where some escapes sequences are turned
+	 into face properties (bug#7876).  */
+      it->end_charpos = ZV;
+      
       /* Return HANDLED_RECOMPUTE_PROPS only if function fontified
 	 something.  This avoids an endless loop if they failed to
 	 fontify the text for which reason ever.  */

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.