Subject: Re: 26.0.90; Error running timer 'jit-lock-stealth-fontify': (error "Invalid search bound (wrong side of point)")

From: Eli Zaretskii <[email-masked]>
Date: Tue, 24 Oct 2017 17:46:08 +0300

> Date: Sun, 22 Oct 2017 20:13:40 +0000
> Cc: [email-masked]
> From: Alan Mackenzie <[email-masked]>
> 
> > So point is 123811 and the BOUND argument of re-search-forward is
> > 123806, too small.
> 
> What I think's happening is that c-forward-declarator has found a "["
> which is before BOUND, but then sets point to the matching "]" which is
> after BOUND.  It then calls c-syntactic-re-search-forward again,
> resulting in the error.
> 
> In master's process.c, there is a "]" very close to 123811.

For the record, here's the place where this happened, with the two
locations shown by "^":

      char namebuf[sizeof (ifq->ifr_name) + 1];
                                         ^    ^
The reason you don't see this in process.c you have is that the
version I used was edited wrt to what you have.

> I haven't reproduced the problem, but I admit I haven't tried all that
> hard.  Could you please try out the patch below, and let me know if it
> fixes the bug.

Thanks, this fixes a very large part of the problem, so I think you
should install this on the release branch.

It doesn't solve all of it, though, because I got that breakpoint hit
again.  This time it took much longer before that happened (a sign
that most of the problem is indeed solved), and the backtrace is
different.  Here's the C and the Lisp backtraces, followed by some
relevant values:

  Thread 1 hit Breakpoint 3, search_command (string=...,
      bound=make_number(2425), noerror=..., count=...,
      direction=direction@entry=1, RE=RE@entry=1, posix=posix@entry=false)
      at search.c:1046
  1046            error ("Invalid search bound (wrong side of point)");
  (gdb) bt
  #0  search_command (string=..., bound=make_number(2425), noerror=...,
      count=..., direction=direction@entry=1, RE=RE@entry=1,
      posix=posix@entry=false) at search.c:1046
  #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000adcb1f0),
      bound=..., noerror=..., count=...) at search.c:2271
  #2  0x0121f9f0 in funcall_subr (subr=<optimized out>,
      subr@entry=0x137fc48 <Sre_search_forward>, numargs=<optimized out>,
      numargs@entry=3, args=<optimized out>, args@entry=0x88c050) at eval.c:2849
  #3  0x0121ddc6 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88c048)
      at eval.c:2766
  #4  0x0129445c in exec_byte_code (bytestr=XIL(0x800000000adcb1c0),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #5  0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88c410) at eval.c:3049
  #6  0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88c408)
      at eval.c:2768
  #7  0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146ae20),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #8  0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88ca70) at eval.c:3049
  #9  0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88ca68)
      at eval.c:2768
  #10 0x0129445c in exec_byte_code (bytestr=XIL(0x8000000001469d78),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #11 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88ce60) at eval.c:3049
  #12 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88ce58)
      at eval.c:2768
  #13 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000ad9d5a8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #14 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=3,
      arg_vector=arg_vector@entry=0x88d230) at eval.c:3049
  #15 0x0121de35 in Ffuncall (nargs=nargs@entry=4, args=args@entry=0x88d228)
      at eval.c:2768
  #16 0x0129445c in exec_byte_code (bytestr=XIL(0x8000000001469798),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=0, args=<optimized out>, args@entry=0x0) at bytecode.c:629
  #17 0x0121d94f in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88d578) at eval.c:3049
  #18 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88d570)
      at eval.c:2768
  #19 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146d9a8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88daf8)
      at bytecode.c:629
  #20 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88daf8) at eval.c:2967
  #21 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88daf0)
      at eval.c:2768
  #22 0x0121dfff in run_hook_wrapped_funcall (nargs=2, args=0x88daf0)
      at eval.c:2493
  #23 0x0121b1e2 in run_hook_with_args (nargs=nargs@entry=2,
      args=args@entry=0x88daf0,
      funcall=funcall@entry=0x121dfcf <run_hook_wrapped_funcall>) at eval.c:2574
  #24 0x0121b38b in Frun_hook_wrapped (nargs=2, args=0x88daf0) at eval.c:2508
  #25 0x0121f8f8 in funcall_subr (
      subr=subr@entry=0x167d5c8 <Srun_hook_wrapped>, numargs=numargs@entry=2,
      args=args@entry=0x88daf0) at eval.c:2821
  #26 0x0121ddc6 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88dae8)
      at eval.c:2766
  #27 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146d938),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=2, args=<optimized out>, args@entry=0x88dee0)
      at bytecode.c:629
  #28 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88dee0) at eval.c:2967
  #29 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88ded8)
      at eval.c:2768
  #30 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146da00),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=2, args=<optimized out>, args@entry=0x88e3e0)
      at bytecode.c:629
  #31 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=2,
      arg_vector=arg_vector@entry=0x88e3e0) at eval.c:2967
  #32 0x0121de35 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88e3d8)
      at eval.c:2768
  #33 0x0129445c in exec_byte_code (bytestr=XIL(0x800000000146dc90),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88e9d0)
      at bytecode.c:629
  #34 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88e9d0) at eval.c:2967
  #35 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88e9c8)
      at eval.c:2768
  #36 0x01220d8a in Fapply (nargs=2, args=0x88e9c8) at eval.c:2343
  #37 0x0121f8f8 in funcall_subr (subr=subr@entry=0x167d668 <Sapply>,
      numargs=numargs@entry=2, args=args@entry=0x88e9c8) at eval.c:2821
  #38 0x0121ddc6 in Ffuncall (nargs=nargs@entry=3, args=args@entry=0x88e9c0)
      at eval.c:2766
  #39 0x0129445c in exec_byte_code (bytestr=XIL(0x80000000014770f8),
      vector=..., maxdepth=..., args_template=..., nargs=<optimized out>,
      nargs@entry=1, args=<optimized out>, args@entry=0x88edb8)
      at bytecode.c:629
  #40 0x0121d467 in funcall_lambda (fun=..., nargs=nargs@entry=1,
      arg_vector=arg_vector@entry=0x88edb8) at eval.c:2967
  #41 0x0121de35 in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x88edb0)
      at eval.c:2768
  #42 0x0121e088 in call1 (fn=XIL(0xf1f0), arg1=...) at eval.c:2617
  #43 0x0114a180 in timer_check_2 (timers=..., idle_timers=...)
      at keyboard.c:4462
  #44 0x01150644 in timer_check () at keyboard.c:4524
  #45 0x0115069a in readable_events (flags=flags@entry=1) at keyboard.c:3340
  #46 0x011594d2 in get_input_pending (flags=flags@entry=1) at keyboard.c:6824
  #47 0x011596b9 in detect_input_pending_run_timers (
      do_display=do_display@entry=true) at keyboard.c:9951
  #48 0x012a9223 in wait_reading_process_output (time_limit=<optimized out>,
      nsecs=<optimized out>, nsecs@entry=0, read_kbd=<optimized out>,
      read_kbd@entry=-1, do_display=<optimized out>, wait_for_cell=...,
      wait_proc=<optimized out>, wait_proc@entry=0x0,
      just_wait_proc=<optimized out>, just_wait_proc@entry=0) at process.c:5504
  #49 0x01159b56 in kbd_buffer_get_event (kbp=kbp@entry=0x88f47c,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3,
      end_time=end_time@entry=0x0) at keyboard.c:3831
  #50 0x0115aab4 in read_event_from_main_queue (end_time=end_time@entry=0x0,
      local_getcjmp=local_getcjmp@entry=0x88f670,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3) at keyboard.c:2151
  #51 0x0115ae38 in read_decoded_event_from_main_queue (
      end_time=end_time@entry=0x0, local_getcjmp=local_getcjmp@entry=0x88f670,
      prev_event=XIL(0), used_mouse_menu=used_mouse_menu@entry=0x88f7c3)
      at keyboard.c:2214
  #52 0x0115c9ff in read_char (commandflag=1, map=..., prev_event=...,
      used_mouse_menu=used_mouse_menu@entry=0x88f7c3,
      end_time=end_time@entry=0x0) at keyboard.c:2802
  #53 0x0115e6be in read_key_sequence (keybuf=keybuf@entry=0x88f870,
      bufsize=bufsize@entry=30, prompt=XIL(0x9b938800000000),
      dont_downcase_last=dont_downcase_last@entry=false,
      can_return_switch_frame=can_return_switch_frame@entry=true,
      fix_current_buffer=fix_current_buffer@entry=true,
      prevent_redisplay=prevent_redisplay@entry=false) at keyboard.c:9147
  #54 0x01161825 in command_loop_1 () at keyboard.c:1368
  #55 0x0121a98e in internal_condition_case (
      bfun=bfun@entry=0x1161517 <command_loop_1>, handlers=...,
      hfun=hfun@entry=0x114d246 <cmd_error>) at eval.c:1332
  #56 0x01142ce3 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1110
  #57 0x0121a8f0 in internal_catch (tag=XIL(0xf538),
      func=func@entry=0x1142cbc <command_loop_2>, arg=...) at eval.c:1097
  #58 0x01142c8b in command_loop () at keyboard.c:1089
  #59 0x0114cba6 in recursive_edit_1 () at keyboard.c:695
  #60 0x0114cff7 in Frecursive_edit () at keyboard.c:766
  #61 0x01141ab3 in main (argc=<optimized out>, argv=<optimized out>)
      at emacs.c:1713

  Lisp Backtrace:
  "re-search-forward" (0x88c050)
  0xad7f328 PVEC_COMPILED
  "font-lock-fontify-keywords-region" (0x88ca70)
  "font-lock-default-fontify-region" (0x88ce60)
  "c-font-lock-fontify-region" (0x88d230)
  "font-lock-fontify-region" (0x88d578)
  0x13810840 PVEC_COMPILED
  "run-hook-wrapped" (0x88daf0)
  "jit-lock--run-functions" (0x88dee0)
  "jit-lock-fontify-now" (0x88e3e0)
  "jit-lock-stealth-fontify" (0x88e9d0)
  "apply" (0x88e9c8)
  "timer-event-handler" (0x88edb8)
  (gdb) pp current_buffer->name_
  "platform.h"
  (gdb) pp current_buffer->directory_
  "d:/utils/lz4-1.7.5/programs/"
  (gdb) p PT
  $1 = 2645
  (gdb) p bound
  $2 = make_number(2425)
  (gdb) up
  #1  0x011cd2c9 in Fre_search_forward (regexp=XIL(0x800000000adcb1f0),
      bound=..., noerror=..., count=...) at search.c:2271
  2271      return search_command (regexp, bound, noerror, count, 1, 1, 0);
  (gdb) pp regexp
  "\\(\\=\\|\\(\\=\\|[^\\]\\)[
  ]\\)\\s *#\\s *\\(\\(?:\\(?:el\\)?if\\)\\)\\([^[:alnum:]_$]\\|$\\)\\(\\\\\\(.\\|
  [
  ]\\)\\|[^
  ]\\)*"

As you see, point is at 2645, whereas BOUND is at 2425.  This happens
in the file platform.h from the lz4-1.7.5 distribution.  Here's the
relevant part of platform.h with the two locations shown (I added an
empty line for each "^" marker):

/* **************************************
*  Detect 64-bit OS
*  http://nadeausoftware.com/articles/2012/02/c_c_tip_how_detect_processor_type_using_compiler_predefined_macros
****************************************/
#if defined __ia64 || defined _M_IA64                                                                               /* Intel Itanium */ \
  || defined __powerpc64__ || defined __ppc64__ || defined __PPC64__                                                /* POWER 64-bit */  \
  || (defined __sparc && (defined __sparcv9 || defined __sparc_v9__ || defined __arch64__)) || defined __sparc64__  /* SPARC 64-bit */  \
  || defined __x86_64__s || defined _M_X64                                                                          /* x86 64-bit */    \
  || defined __arm64__ || defined __aarch64__ || defined __ARM64_ARCH_8__                                           /* ARM 64-bit */    \
  || (defined __mips  && (__mips == 64 || __mips == 4 || __mips == 3))                                              /* MIPS 64-bit */   \
                                                                      ^
  || defined _LP64 || defined __LP64__ /* NetBSD, OpenBSD */ || defined __64BIT__ /* AIX */ || defined _ADDR64 /* Cray */               \
  || (defined __SIZEOF_POINTER__ && __SIZEOF_POINTER__ == 8) /* gcc */
              ^
#  if !defined(__64BIT__)
#    define __64BIT__  1
#  endif
#endif

As you see, BOUND is after the closing paren, after "3))", and point
is at the beginning of "__SIZEOF_POINTER__" a couple of lines further.

To tell the truth, the Lisp backtrace puzzles me a bit, because the
only call to re-search-forward in font-lock-fontify-keywords-region is
protected by a condition that should have prevented this problem from
happening:

      (while (and (< (point) end)
		  (if (stringp matcher)
		      (re-search-forward matcher end t)
		    (funcall matcher end))

So maybe I'm missing something, or maybe the problematic call to
re-search-forward comes from some macro expansion I didn't identify.

Let me know if I can provide any more details for your analysis.

Thanks.