Subject: bug#28850: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

From: Eli Zaretskii <[email-masked]>
Date: Sun, 15 Oct 2017 19:07:50 +0300

This bug is bugging me for quite some time now, and my hopes for it to
be resolved are now gone, so I finally sat down to debug it.

I have jit-lock-stealth turned on in my sessions, so whenever I
restart Emacs (e.g., when I build a new binary, or after a system
restart), and restore my session using desktop.el, Emacs starts
fontifying in the background.  At some point, sometimes more than
once, I get this error:

  Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

Today I ran Emacs under a debugger, and caught this error.  The
details are below, but in a nutshell, CC mode's fontification
functions call re-search-forward with BOUND that is before point.  I
hope the data below is enough to understand why that happens and fix
it; if not, please tell what additional data is needed to diagnose the
problem.

Here're the C and Lisp backtraces from the error, and some relevant
data that explains why the error happened:

  Thread 1 hit Breakpoint 5, search_command (string=...,
      bound=make_number(123806), noerror=..., count=...,
      direction=direction@entry=1, RE=RE@entry=1, posix=posix@entry=false)
      at search.c:1046
  1046            error ("Invalid search bound (wrong side of point)");
  (gdb) bt
  #0  search_command (string=..., bound=make_number(123806), noerror=...,
      count=..., direction=direction@entry=1, RE=RE@entry=1,
      posix=posix@entry=false) at search.c:1046
  #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000ad97598),
      bound=..., noerror=..., count=...) at search.c:2271
  #2  0x0121f9f0 in funcall_subr (subr=<optimized out>,
      subr@entry=0x137fc48 <Sre_search_forward>, numargs=<optimized out>,
      numargs@entry=3, args=<optimized out>, args@entry=0x8898d0) at eval.c:2849
  #3  0x0121ddc6 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x8898c8)
      at eval.c:2766
  #4  0x0129445c in exec_byte_code (bytestr=XIL(0x800000000ad526b8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #5  0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=4,
      arg_vector=arg_vector@entry=0x889ed0) at eval.c:3049
  #6  0x0121de35 in Ffuncall (nargs=nargs@entry=5, args=args@entry=0x889ec8)
      at eval.c:2768
  #7  0x0129445c in exec_byte_code (bytestr=XIL(0x800000000ad97558),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #8  0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88a410) at eval.c:3049
  #9  0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88a408)
      at eval.c:2768
  #10 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000add7b70),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #11 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=5,
      arg_vector=arg_vector@entry=0x88a980) at eval.c:3049
  #12 0x0121de35 in Ffuncall (nargs=nargs@entry=6, args=args@entry=0x88a978)
      at eval.c:2768
  #13 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000add8368),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #14 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=5,
      arg_vector=arg_vector@entry=0x88ae50) at eval.c:3049
  #15 0x0121de35 in Ffuncall (nargs=nargs@entry=6, args=args@entry=0x88ae48)
      at eval.c:2768
  #16 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000add83e8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #17 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88b310) at eval.c:3049
  #18 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88b308)
      at eval.c:2768
  #19 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000ad54fe0),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #20 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=4,
      arg_vector=arg_vector@entry=0x88c040) at eval.c:3049
  #21 0x0121de35 in Ffuncall (nargs=nargs@entry=5, args=args@entry=0x88c038)
      at eval.c:2768
  #22 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000add83c8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #23 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88c410) at eval.c:3049
  #24 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88c408)
      at eval.c:2768
  #25 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146ae20),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #26 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88ca70) at eval.c:3049
  #27 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88ca68)
      at eval.c:2768
  #28 0x0129445c in exec_byte_code (bytestr=XIL(0x8000000001469d78),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #29 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88ce60) at eval.c:3049
  #30 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88ce58)
      at eval.c:2768
  #31 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000adad4c0),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #32 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88d230) at eval.c:3049
  #33 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88d228)
      at eval.c:2768
  #34 0x0129445c in exec_byte_code (bytestr=XIL(0x8000000001469798),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #35 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88d578) at eval.c:3049
  #36 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88d570)
      at eval.c:2768
  #37 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146d9a8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88daf8)
      at bytecode.c:629
  #38 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88daf8) at eval.c:2967
  #39 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88daf0)
      at eval.c:2768
  #40 0x0121dfff in run_hook_wrapped_funcall (nargs=2, args=0x88daf0)
      at eval.c:2493
  #41 0x0121b1e2 in run_hook_with_args (nargs=nargs@entry=2,
      args=args@entry=0x88daf0,
      funcall=funcall@entry=0x121dfcf <run_hook_wrapped_funcall>) at eval.c:2574
  #42 0x0121b38b in Frun_hook_wrapped (nargs=2, args=0x88daf0) at eval.c:2508
  #43 0x0121f8f8 in funcall_subr (
      subr=subr@entry=0x167d5c8 <Srun_hook_wrapped>, numargs=numargs@entry=2,
      args=args@entry=0x88daf0) at eval.c:2821
  #44 0x0121ddc6 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88dae8)
      at eval.c:2766
  #45 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146d938),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=2, args=<optimized out>, args@entry=0x88dee0)
      at bytecode.c:629
  #46 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88dee0) at eval.c:2967
  #47 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88ded8)
      at eval.c:2768
  #48 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146da00),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=2, args=<optimized out>, args@entry=0x88e3e0)
      at bytecode.c:629
  #49 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88e3e0) at eval.c:2967
  #50 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88e3d8)
      at eval.c:2768
  #51 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146dc90),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88e9d0)
      at bytecode.c:629
  #52 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88e9d0) at eval.c:2967
  #53 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88e9c8)
      at eval.c:2768
  #54 0x01220d8a in Fapply (nargs=2, args=0x88e9c8) at eval.c:2343
  #55 0x0121f8f8 in funcall_subr (subr=subr@entry=0x167d668 <Sapply>,
      numargs=numargs@entry=2, args=args@entry=0x88e9c8) at eval.c:2821
  #56 0x0121ddc6 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88e9c0)
      at eval.c:2766
  #57 0x0129445c in exec_byte_code (bytestr=XIL(0x80000000014770f8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88edb8)
      at bytecode.c:629
  #58 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88edb8) at eval.c:2967
  #59 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88edb0)
      at eval.c:2768
  #60 0x0121e088 in call1 (fn=XIL(0xf1f0), arg1=...) at eval.c:2617
  #61 0x0114a180 in timer_check_2 (timers=..., idle_timers=...)
      at keyboard.c:4462
  #62 0x01150644 in timer_check () at keyboard.c:4524
  #63 0x0115069a in readable_events (flags=flags@entry=1) at keyboard.c:3340
  #64 0x011594d2 in get_input_pending (flags=flags@entry=1) at keyboard.c:6824
  #65 0x011596b9 in detect_input_pending_run_timers (
      do_display=do_display@entry=true) at keyboard.c:9951
  #66 0x012a9223 in wait_reading_process_output (time_limit=<optimized out>,
      nsecs=<optimized out>, nsecs@entry=0, read_kbd=<optimized out>,
      read_kbd@entry=-1, do_display=<optimized out>, wait_for_cell=...,
      wait_proc=<optimized out>, wait_proc@entry=0x0,
      just_wait_proc=<optimized out>, just_wait_proc@entry=0) at process.c:5504
  #67 0x01159b56 in kbd_buffer_get_event (kbp=kbp@entry=0x88f47c,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3,
      end_time=end_time@entry=0x0) at keyboard.c:3831
  #68 0x0115aab4 in read_event_from_main_queue (end_time=end_time@entry=0x0,
      local_getcjmp=local_getcjmp@entry=0x88f670,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3) at keyboard.c:2151
  #69 0x0115ae38 in read_decoded_event_from_main_queue (
      end_time=end_time@entry=0x0, local_getcjmp=local_getcjmp@entry=0x88f670,
      prev_event=XIL(0), used_mouse_menu=used_mouse_menu@entry=0x88f7c3)
      at keyboard.c:2214
  #70 0x0115c9ff in read_char (commandflag=1, map=..., prev_event=...,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3,
      end_time=end_time@entry=0x0) at keyboard.c:2802
  #71 0x0115e6be in read_key_sequence (keybuf=keybuf@entry=0x88f870,
      bufsize=bufsize@entry=30, prompt=XIL(0xfb938800000000),
      dont_downcase_last=dont_downcase_last@entry=false,
      can_return_switch_frame=can_return_switch_frame@entry=true,
      fix_current_buffer=fix_current_buffer@entry=true,
      prevent_redisplay=prevent_redisplay@entry=false) at keyboard.c:9147
  #72 0x01161825 in command_loop_1 () at keyboard.c:1368
  #73 0x0121a98e in internal_condition_case (
      bfun=bfun@entry=0x1161517 <command_loop_1>, handlers=...,
      hfun=hfun@entry=0x114d246 <cmd_error>) at eval.c:1332
  #74 0x01142ce3 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110
  #75 0x0121a8f0 in internal_catch (tag=XIL(0xf538),
      func=func@entry=0x1142cbc <command_loop_2>, arg=...) at eval.c:1097
  #76 0x01142c8b in command_loop () at keyboard.c:1089
  #77 0x0114cba6 in recursive_edit_1 () at keyboard.c:695
  #78 0x0114cff7 in Frecursive_edit () at keyboard.c:766
  #79 0x01141ab3 in main (argc=<optimized out>, argv=<optimized out>)
      at emacs.c:1713

  Lisp Backtrace:
  "re-search-forward" (0x8898d0)
  "c-syntactic-re-search-forward" (0x889ed0)
  "c-forward-declarator" (0x88a410)
  "c-font-lock-declarators" (0x88a980)
  "c-font-lock-single-decl" (0x88ae50)
  0xad881a0 PVEC_COMPILED
  "c-find-decl-spots" (0x88c040)
  "c-font-lock-declarations" (0x88c410)
  "font-lock-fontify-keywords-region" (0x88ca70)
  "font-lock-default-fontify-region" (0x88ce60)
  "c-font-lock-fontify-region" (0x88d230)
  "font-lock-fontify-region" (0x88d578)
  0x83d89a8 PVEC_COMPILED
  "run-hook-wrapped" (0x88daf0)
  "jit-lock--run-functions" (0x88dee0)
  "jit-lock-fontify-now" (0x88e3e0)
  "jit-lock-stealth-fontify" (0x88e9d0)
  "apply" (0x88e9c8)
  "timer-event-handler" (0x88edb8)
  (gdb) p n
  $1 = 1
  (gdb) p lim
  $2 = <optimized out>
  (gdb) pp bound
  123806
  (gdb) p PT
  $3 = 123811

So point is 123811 and the BOUND argument of re-search-forward is
123806, too small.

  (gdb) up
  #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000ad97598),
      bound=..., noerror=..., count=...) at search.c:2271
  2271      return search_command (regexp, bound, noerror, count, 1, 1, 0);
  (gdb) pp regexp
  "[;:,]\\|\\s)\\|\\(=\\|\\s(\\)"
  (gdb) p current_buffer
  $4 = (struct buffer *) 0xb362590
  (gdb) pp current_buffer->name_
  "process.c"

These are the regexp argument to re-search-forward and the buffer
which was being fontified.

The problem happens in c-syntactic-re-search-forward in this snippet:

    (condition-case err
	(while
	    (and
	     (progn
	       (setq search-pos (point))
	       (if (re-search-forward regexp bound noerror)  <<<<<<<<<<<
		   t
		 ;; Without the following, when PAREN-LEVEL is non-nil, and
		 ;; NOERROR is not nil or t, and the very first search above
		 ;; has just failed, point would end up at BOUND rather than
		 ;; just before the next close paren.
		 (when (and (eq search-pos start)
			    paren-level
			    (not (memq noerror '(nil t))))
		   (setq state (parse-partial-sexp start bound -1))
		   (if (eq (car state) -1)
		       (setq bound (1- (point)))))
		 nil))

This is called from c-forward-declarator:

	 ;; Search syntactically to the end of the declarator (";",
	 ;; ",", a closing paren, eob etc) or to the beginning of an
	 ;; initializer or function prototype ("=" or "\\s\(").
	 ;; Note that square brackets are now not also treated as
	 ;; initializers, since this broke when there were also
	 ;; initializing brace lists.
	 (let (found)
	   (while
	       (and (progn
		      ;; In the next loop, we keep searching forward whilst
		      ;; we find ":"s which aren't single colons inside C++
		      ;; "for" statements.
		      (while
			  (and
			   (setq found
				 (c-syntactic-re-search-forward   <<<<<<<<<<<
				  "[;:,]\\|\\s)\\|\\(=\\|\\s(\\)"
				  limit t t))

It looks like c-syntactic-re-search-forward calls re-search-forward in
a loop, but perhaps it fails to update the limit to be in sync with
point that moves as the search proceeds?

Let me know what other data I can provide to help fix this annoying
problem.

In GNU Emacs 26.0.90 (build 1, i686-pc-mingw32)
 of 2017-10-12 built on HOME-C4E4A596F7
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --prefix=/d/usr --with-wide-int --with-modules
 --enable-checking=yes,glyphs 'CFLAGS=-Og -gdwarf-4 -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS MODULES LCMS2

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote w32notify w32 lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 100974 11112)
 (symbols 56 21280 1)
 (miscs 48 40 107)
 (strings 16 31517 1999)
 (string-bytes 1 759703)
 (vectors 16 14009)
 (vector-slots 8 645650 16134)
 (floats 8 51 226)
 (intervals 40 268 103)
 (buffers 880 11))