Subject: bug#7876: closed (Re: bug#7876: Crash: infinite recursion in next_element_from_buffer)

From: [email-masked] (GNU bug Tracking System)
Date: Mon, 24 Jan 2011 22:02:02 +0000

Your bug report

#7876: Crash: infinite recursion in next_element_from_buffer

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
7876: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=7876
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#7876: Crash: infinite recursion in next_element_from_buffer
From: Stefan Monnier <[email-masked]>
To: Eli Zaretskii <[email-masked]>
Cc: [email-masked]
Date: Mon, 24 Jan 2011 17:09:58 -0500
List-Id: <debbugs-submit.debbugs.gnu.org>

Thanks a lot, Eli, I think I've found the problem:

the stop_pos is stuck there, because it's the end of the buffer, but
end_charpos points further than ZV.

The reason for that is that grep.el's font-lock code doesn't just
add/remove text properties but also removes text.  In the old
compile.el, jit-lock is disabled so font-lock is run when the text is
inserted, but with my new compile.el, jit-lock can be (and is, by
default) enabled, so the end_charpos becomes invalid after running
jit-lock, and the redisplay code doesn't seem to pay attention to
this possibility.

I'm not very happy with grep.el's modifying the buffer like that from
font-lock-keywords, but neither is it OK for the redisplay to get stuck
in such an inf-loop.

I've installed the patch below in the emacs-23 branch, to account for
this particular case, although the jit-lock code could do nastier things
(e.g. add/remove text before the current position, thus rendering
IT_CHARPOS invalid).

        Stefan

=== modified file 'src/xdisp.c'
--- src/xdisp.c	2011-01-02 23:50:46 +0000
+++ src/xdisp.c	2011-01-24 22:02:26 +0000
@@ -3337,6 +3337,8 @@
       val = Vfontification_functions;
       specbind (Qfontification_functions, Qnil);
 
+      xassert (it->end_charpos == ZV);
+      
       if (!CONSP (val) || EQ (XCAR (val), Qlambda))
 	safe_call1 (val, pos);
       else
@@ -3376,6 +3378,13 @@
 
       unbind_to (count, Qnil);
 
+      /* The fontification code may have added/removed text.
+	 It could do even a lot worse, but let's at least protect against
+	 the most obvious case where only the text past `pos' gets changed',
+	 as is/was done in grep.el where some escapes sequences are turned
+	 into face properties (bug#7876).  */
+      it->end_charpos = ZV;
+      
       /* Return HANDLED_RECOMPUTE_PROPS only if function fontified
 	 something.  This avoids an endless loop if they failed to
 	 fontify the text for which reason ever.  */

Subject: Crash: infinite recursion in next_element_from_buffer
From: Stefan Monnier <[email-masked]>
To: [email-masked]
Date: Thu, 20 Jan 2011 14:49:31 -0500
List-Id: <debbugs-submit.debbugs.gnu.org>

Package: Emacs
Version: 24.0.50

I'm working on moving compile.el's parsing away from font-lock and using
the new code, I triggered a bug somewhere in the redisplay, that results
in a crash because of an infinite recursion:

The stack trace shows (other than the top items which vary depending on
the exact moment the maximum stack size was hit) that the recursion has
the following trivial pattern:

  [...]
  #8  0x080a8ea8 in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6655
  #9  0x080a8ebd in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6656
  #10 0x080a8ebd in next_element_from_buffer (it=0xbfffbd40) at xdisp.c:6656
  [...]

where 6656 is the second line of:

	  handle_stop (it);
	  return GET_NEXT_DISPLAY_ELEMENT (it);

I can reproduce it easily, but not reliably.  The changed code in
compile.el has not been heavily tested, so the trigger for this bug may
be something like an odd value of the `face' or
`font-lock-face' properties.

Here's what I do to reproduce it:

   % src/emacs -Q -l "newcompile.el"
   M-x grep TAB TAB RET
   Fset_window_config src/*.c RET

I've attached the "offending" new compile.el.  This does not always
crash, but almost.  The TAB TAB is on purpose to bring up the
*completion* buffer, because it seemed to make the bug show up
more frequently.

In GNU Emacs 24.0.50.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2011-01-20 on ceviche
Windowing system distributor `The X.Org Foundation', version 11.0.10707000
configured using `configure  'CFLAGS=-Wall -Wno-pointer-sign -DUSE_LISP_UNION_TYPE -DSYNC_INPUT -DENABLE_CHECKING -DXASSERTS -DFONTSET_DEBUG -g -O1 -I/usr/include/GNUstep' '--enable-maintainer-mode' '--with-x-toolkit=lucid''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: fr_CH.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Minibuffer-Area

Minor modes in effect:
  electric-pair-mode: t
  electric-indent-mode: t
  url-handler-mode: t
  global-reveal-mode: t
  reveal-mode: t
  auto-insert-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<switch-frame> <switch-frame> M-x r e p o <tab> r <tab> 
<return>

Recent messages:
test42
test5
test6
Loading /home/monnier/etc/emacs/X11.el (source)...done
Loading /home/monnier/etc/emacs/custom.el (source)...done
Ispell-kill: nil american
Starting new Ispell process [american] ...
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...
Quit

Load-path shadows:
None found.

Features:
(shadow sort mail-extr message sendmail rfc822 mml mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mailabbrev mail-utils gmm-utils mailheader emacsbug server noutline
outline easy-mmode flyspell ispell eldoc checkdoc regexp-opt thingatpt
help-mode easymenu view prog-mode electric url-handlers url-parse
auth-source netrc gnus-util url-vars mm-util mail-prsvr reveal
autoinsert uniquify advice help-fns advice-preload savehist
minibuf-eldef cl cl-loaddefs proof-site proof-autoloads pg-vars
bbdb-autoloads agda2 tooltip ediff-hook vc-hooks lisp-float-type mwheel
x-win x-dnd tool-bar dnd fontset image fringe lisp-mode register page
newcomment menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs
button faces cus-face files text-properties overlay md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting system-font-setting
font-render-setting x-toolkit x multi-tty emacs)