GNU bug report logs - #71681
29.3.50; tree-sitter crash

Previous Next

To reply to this bug, email your comments to 71681 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.3.50; tree-sitter crash
Date: Thu, 20 Jun 2024 19:33:29 +0300

Evaluating this expression causes a crash:

(progn
  (find-file (expand-file-name "src/treesit.c" installation-directory))
  (c-ts-mode)
  (font-lock-ensure 63209 63387))

in latest master, but not in latest emacs-29 (only in 5-months old emacs-29).

If this is not reproducible, I could provide more details.

libtree-sitter is at the latest version.

Thread 1 "emacs" received signal SIGSEGV, Segmentation fault.
0x00007ffff3f88f41 in ts_language_public_symbol () from /usr/local/lib/libtree-sitter.so.0
(gdb) bt
#0  0x00007ffff3f88f41 in ts_language_public_symbol () at /usr/local/lib/libtree-sitter.so.0
#1  0x00007ffff3f9fe9c in ts_query_cursor.advance () at /usr/local/lib/libtree-sitter.so.0
#2  0x00007ffff3fa117f in ts_query_cursor_next_match () at /usr/local/lib/libtree-sitter.so.0
#3  0x00005555557f0f8f in Ftreesit_query_capture (node=<optimized out>, query=<optimized out>, beg=<optimized out>, end=<optimized out>, node_only=XIL(0)) at treesit.c:3014
#4  0x00007fffec125106 in F747265657369742d2d666f6e742d6c6f636b2d666f6e746966792d726567696f6e2d31_treesit__font_lock_fontify_region_1_0 ()
#5  0x000055555575faf7 in Ffuncall (nargs=7, args=0x7fffffffcc00) at eval.c:3093
#6  0x00007fffec124e28 in F747265657369742d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_treesit_font_lock_fontify_region_0 ()
#7  0x000055555575faf7 in Ffuncall (nargs=4, args=0x7fffffffccb0) at eval.c:3093
#8  0x00007fffef266534 in F666f6e742d6c6f636b2d666f6e746966792d73796e746163746963616c6c792d726567696f6e_font_lock_fontify_syntactically_region_0 ()
#9  0x000055555575faf7 in Ffuncall (nargs=4, args=0x7fffffffce10) at eval.c:3093
#10 0x00007fffef26427f in F666f6e742d6c6f636b2d64656661756c742d666f6e746966792d726567696f6e_font_lock_default_fontify_region_0 ()
#11 0x000055555575faf7 in Ffuncall (nargs=4, args=0x7fffffffceb0) at eval.c:3093
#12 0x00007fffef2630c5 in F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0 ()
#13 0x00005555557a8b38 in exec_byte_code (fun=<optimized out>, args_template=<optimized out>, nargs=<optimized out>, args=<optimized out>) at lisp.h:2243
#14 0x000055555575faf7 in Ffuncall (nargs=2, args=0x7fffffffd030) at eval.c:3093
#15 0x00005555557602a0 in run_hook_wrapped_funcall (nargs=<optimized out>, args=0x7fffffffd030) at eval.c:2872
#16 0x000055555575e9fb in run_hook_with_args (nargs=2, args=0x7fffffffd030, funcall=0x555555760280 <run_hook_wrapped_funcall>) at eval.c:2953
#17 0x00007fffef236115 in F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0 ()
#18 0x000055555575faf7 in Ffuncall (nargs=3, args=0x7fffffffd150) at eval.c:3093
#19 0x00007fffef2369e9 in F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0 ()
#20 0x000055555575faf7 in Ffuncall (nargs=3, args=0x7fffffffd250) at eval.c:3093
#21 0x00007fffef263482 in F666f6e742d6c6f636b2d656e73757265_font_lock_ensure_0 ()
#22 0x00005555557631da in eval_sub (form=<optimized out>) at lisp.h:2243
#23 0x0000555555763381 in Fprogn (body=<optimized out>) at eval.c:439
#24 0x0000555555763066 in eval_sub (form=<optimized out>) at lisp.h:2243
#25 0x0000555555763381 in Fprogn (body=<optimized out>) at eval.c:439
#26 0x0000555555763066 in eval_sub (form=<optimized out>) at lisp.h:2243
#27 0x0000555555764bc1 in Fprogn (body=<optimized out>) at eval.c:439
#28 Flet (args=<optimized out>) at eval.c:1109
#29 0x0000555555763066 in eval_sub (form=<optimized out>) at lisp.h:2243
#30 0x0000555555763437 in Fsetq (args=<optimized out>) at eval.c:486
#31 0x0000555555763066 in eval_sub (form=<optimized out>) at lisp.h:2243
#32 0x000055555578ce3a in readevalloop_eager_expand_eval (val=<optimized out>, macroexpand=XIL(0xadd0)) at lisp.h:1192
#33 0x0000555555794ba0 in readevalloop (readcharfun=XIL(0x7ffff02e33d5), infile0=0x0, sourcename=XIL(0), printflag=true, unibyte=<optimized out>, readfun=XIL(0x5555560bc1f5), start=make_fixnum(202), end=XIL(0x5555560bc285)) at lread.c:2538
#34 0x000055555579601a in Feval_region (start=make_fixnum(202), end=make_fixnum(328), printflag=XIL(0x30), read_function=XIL(0x5555560bc1f5)) at lisp.h:752
#35 0x00007fffefacbbf6 in F656c6973702d2d6576616c2d646566756e_elisp__eval_defun_0 ()
#36 0x000055555575faf7 in Ffuncall (nargs=1, args=0x7fffffffd9f8) at eval.c:3093
#37 0x00007fffefacbcb1 in F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_38 ()
#38 0x000055555575faf7 in Ffuncall (nargs=1, args=0x7fffffffda40) at eval.c:3093
#39 0x0000555555760f09 in call0 (fn=<optimized out>) at lisp.h:3515
#40 Fhandler_bind_1 (nargs=<optimized out>, args=0x7fffffffda90) at eval.c:1478
#41 0x00007fffefacbd7a in F6576616c2d646566756e_eval_defun_0 ()
#42 0x000055555575faf7 in Ffuncall (nargs=nargs <at> entry=2, args=args <at> entry=0x7fffffffdb58) at eval.c:3093
#43 0x000055555575b4f3 in Ffuncall_interactively (nargs=2, args=0x7fffffffdb58) at callint.c:250
#44 0x000055555575faf7 in Ffuncall (nargs=nargs <at> entry=3, args=args <at> entry=0x7fffffffdb50) at eval.c:3093
#45 0x000055555575cc53 in Fcall_interactively (function=<optimized out>, record_flag=<optimized out>, keys=<optimized out>) at callint.c:789
#46 0x00007fffef9330cd in F636f6d6d616e642d65786563757465_command_execute_0 ()
#47 0x000055555575faf7 in Ffuncall (nargs=nargs <at> entry=2, args=args <at> entry=0x7fffffffde50) at eval.c:3093
#48 0x00005555556e2247 in command_loop_1 () at lisp.h:1192
#49 0x000055555575e0d7 in internal_condition_case (bfun=bfun <at> entry=0x5555556e1e40 <command_loop_1>, handlers=handlers <at> entry=XIL(0x90), hfun=hfun <at> entry=0x5555556d63c0 <cmd_error>) at eval.c:1613
#50 0x00005555556ce07a in command_loop_2 (handlers=handlers <at> entry=XIL(0x90)) at keyboard.c:1168
#51 0x000055555575e019 in internal_catch (tag=tag <at> entry=XIL(0x11d30), func=func <at> entry=0x5555556ce050 <command_loop_2>, arg=arg <at> entry=XIL(0x90)) at eval.c:1292
#52 0x00005555556ce016 in command_loop () at lisp.h:1192
#53 0x00005555556d5f25 in recursive_edit_1 () at keyboard.c:754
#54 0x00005555556d62d4 in Frecursive_edit () at keyboard.c:837
#55 0x00005555555aebf4 in main (argc=<optimized out>, argv=<optimized out>) at emacs.c:2629

Lisp Backtrace:
"treesit--font-lock-fontify-region-1" (0xffffcc08)
"treesit-font-lock-fontify-region" (0xffffccb8)
"font-lock-fontify-syntactically-region" (0xffffce18)
"font-lock-default-fontify-region" (0xffffceb8)
"font-lock-fontify-region" (0xedea4040)
0x5681b288 PVEC_CLOSURE
"jit-lock--run-functions" (0xffffd158)
"jit-lock-fontify-now" (0xffffd258)
"font-lock-ensure" (0xffffd2d0)
"progn" (0xffffd3a0)
"progn" (0xffffd480)
"let" (0xffffd5d0)
"setq" (0xffffd6d0)
"elisp--eval-defun" (0xffffda00)
0xf060f638 PVEC_SUBR
"eval-defun" (0xffffdb60)
"funcall-interactively" (0xffffdb58)
"command-execute" (0xffffde58)

Message #8 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sat, 22 Jun 2024 16:55:38 -0700

> On Jun 20, 2024, at 9:33 AM, Juri Linkov <juri <at> linkov.net> wrote:
> 
> Evaluating this expression causes a crash:
> 
> (progn
>  (find-file (expand-file-name "src/treesit.c" installation-directory))
>  (c-ts-mode)
>  (font-lock-ensure 63209 63387))
> 
> in latest master, but not in latest emacs-29 (only in 5-months old emacs-29).
> 
> If this is not reproducible, I could provide more details.
> 
> libtree-sitter is at the latest version.

Hmm, I can’t reproduce with latest master and libtree-sitter. Maybe you can send me the exact commits that you used?

Here’s mine:

Emacs: 72f2b01e318
Tree-sitter: 6ec478c1

Yuan

Message #11 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 71681 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sun, 23 Jun 2024 08:32:47 +0300

> Cc: 71681 <at> debbugs.gnu.org
> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 22 Jun 2024 16:55:38 -0700
> 
> 
> 
> > On Jun 20, 2024, at 9:33 AM, Juri Linkov <juri <at> linkov.net> wrote:
> > 
> > Evaluating this expression causes a crash:
> > 
> > (progn
> >  (find-file (expand-file-name "src/treesit.c" installation-directory))
> >  (c-ts-mode)
> >  (font-lock-ensure 63209 63387))
> > 
> > in latest master, but not in latest emacs-29 (only in 5-months old emacs-29).
> > 
> > If this is not reproducible, I could provide more details.
> > 
> > libtree-sitter is at the latest version.
> 
> Hmm, I can’t reproduce with latest master and libtree-sitter. Maybe you can send me the exact commits that you used?
> 
> Here’s mine:
> 
> Emacs: 72f2b01e318
> Tree-sitter: 6ec478c1

I can reproduce this with tree-sitter version 0.20.8.  Can you try
building Emacs with that version?  I know it's somewhat old, but given
the ABI breakage issue, I expect quite a few people avoid upgrading to
a later version (I didn't).

Message #14 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sun, 23 Jun 2024 09:46:58 +0300

>> Evaluating this expression causes a crash:
>>
>> (progn
>>  (find-file (expand-file-name "src/treesit.c" installation-directory))
>>  (c-ts-mode)
>>  (font-lock-ensure 63209 63387))
>>
>> in latest master, but not in latest emacs-29 (only in 5-months old emacs-29).
>>
>> If this is not reproducible, I could provide more details.
>>
>> libtree-sitter is at the latest version.
>
> Hmm, I can’t reproduce with latest master and libtree-sitter.
> Maybe you can send me the exact commits that you used?
>
> Here’s mine:
>
> Emacs: 72f2b01e318
> Tree-sitter: 6ec478c1

The commit are:

Emacs: 6f2036243f2 (2024-06-23, latest master)
Tree-sitter: 3da7deed (2024-06-08, version 0.22.6)

Also fails on old commits:
Emacs: ef01b634d21 (2024-01-18, emacs-29)
Tree-sitter: 870fb877 (2022-11-16, version 0.6.3)

But doesn't fail on:
Emacs: ce85d3811da (2024-06-18, recent emacs-29)
Tree-sitter: 3da7deed (2024-06-08, version 0.22.6)

Maybe it doesn't fail on recent emacs-29 because of the fix in 20af58d3a13?
But the same fix exists in latest master as well.

Message #17 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sun, 23 Jun 2024 20:38:08 +0300

>> Evaluating this expression causes a crash:
>>
>> (progn
>>  (find-file (expand-file-name "src/treesit.c" installation-directory))
>>  (c-ts-mode)
>>  (font-lock-ensure 63209 63387))
>>
>> in latest master, but not in latest emacs-29 (only in 5-months old emacs-29).
>>
>> If this is not reproducible, I could provide more details.
>>
>> libtree-sitter is at the latest version.
>
> Hmm, I can’t reproduce with latest master and libtree-sitter. Maybe you can send me the exact commits that you used?
>
> Here’s mine:
>
> Emacs: 72f2b01e318
> Tree-sitter: 6ec478c1

Probably reproducibility depends on the content of the src/treesit.c file.
Then the most reliable way to reproduce it is this:

0. emacs -Q
1. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
2. C-x v L
3. in the *vc-change-log* buffer move point to the commit 20af58d3a13
4. type D
5. crash caused by diff-font-lock-syntax fontification that uses treesit

The numbers in (font-lock-ensure 63209 63387) above were extracted
from diff hunk boundaries that might be different when the file was edited.

Message #20 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Mon, 24 Jun 2024 00:46:22 -0700

> On Jun 23, 2024, at 10:38 AM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>>> Evaluating this expression causes a crash:
>>> 
>>> (progn
>>> (find-file (expand-file-name "src/treesit.c" installation-directory))
>>> (c-ts-mode)
>>> (font-lock-ensure 63209 63387))
>>> 
>>> in latest master, but not in latest emacs-29 (only in 5-months old emacs-29).
>>> 
>>> If this is not reproducible, I could provide more details.
>>> 
>>> libtree-sitter is at the latest version.
>> 
>> Hmm, I can’t reproduce with latest master and libtree-sitter. Maybe you can send me the exact commits that you used?
>> 
>> Here’s mine:
>> 
>> Emacs: 72f2b01e318
>> Tree-sitter: 6ec478c1
> 
> Probably reproducibility depends on the content of the src/treesit.c file.
> Then the most reliable way to reproduce it is this:
> 
> 0. emacs -Q
> 1. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
> 2. C-x v L
> 3. in the *vc-change-log* buffer move point to the commit 20af58d3a13
> 4. type D
> 5. crash caused by diff-font-lock-syntax fontification that uses treesit
> 
> The numbers in (font-lock-ensure 63209 63387) above were extracted
> from diff hunk boundaries that might be different when the file was edited.

I reproduce it once with the first set of commits you provided, but for some reason couldn’t reproduce it again. I’m sure it’s something wrong that I did. I’ll report back when I make progress. TBH it seems like something wrong with tree-sitter itself, but I’ll make sure to figure out what’s the problem exactly.

Yuan

Message #23 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Tue, 25 Jun 2024 23:04:56 -0700

> On Jun 24, 2024, at 12:46 AM, Yuan Fu <casouri <at> gmail.com> wrote:
> 
> 
> 
>> On Jun 23, 2024, at 10:38 AM, Juri Linkov <juri <at> linkov.net> wrote:
>> 
>>>> Evaluating this expression causes a crash:
>>>> 
>>>> (progn
>>>> (find-file (expand-file-name "src/treesit.c" installation-directory))
>>>> (c-ts-mode)
>>>> (font-lock-ensure 63209 63387))
>>>> 
>>>> in latest master, but not in latest emacs-29 (only in 5-months old emacs-29).
>>>> 
>>>> If this is not reproducible, I could provide more details.
>>>> 
>>>> libtree-sitter is at the latest version.
>>> 
>>> Hmm, I can’t reproduce with latest master and libtree-sitter. Maybe you can send me the exact commits that you used?
>>> 
>>> Here’s mine:
>>> 
>>> Emacs: 72f2b01e318
>>> Tree-sitter: 6ec478c1
>> 
>> Probably reproducibility depends on the content of the src/treesit.c file.
>> Then the most reliable way to reproduce it is this:
>> 
>> 0. emacs -Q
>> 1. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
>> 2. C-x v L
>> 3. in the *vc-change-log* buffer move point to the commit 20af58d3a13
>> 4. type D
>> 5. crash caused by diff-font-lock-syntax fontification that uses treesit
>> 
>> The numbers in (font-lock-ensure 63209 63387) above were extracted
>> from diff hunk boundaries that might be different when the file was edited.
> 
> I reproduce it once with the first set of commits you provided, but for some reason couldn’t reproduce it again. I’m sure it’s something wrong that I did. I’ll report back when I make progress. TBH it seems like something wrong with tree-sitter itself, but I’ll make sure to figure out what’s the problem exactly.
> 
> Yuan

Ok, I can reproduce it now. Looking into it…

Yuan

Message #26 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sat, 29 Jun 2024 16:54:39 -0700

> On Jun 25, 2024, at 11:04 PM, Yuan Fu <casouri <at> gmail.com> wrote:
> 
> 
> 
>> On Jun 24, 2024, at 12:46 AM, Yuan Fu <casouri <at> gmail.com> wrote:
>> 
>> 
>> 
>>> On Jun 23, 2024, at 10:38 AM, Juri Linkov <juri <at> linkov.net> wrote:
>>> 
>>>>> Evaluating this expression causes a crash:
>>>>> 
>>>>> (progn
>>>>> (find-file (expand-file-name "src/treesit.c" installation-directory))
>>>>> (c-ts-mode)
>>>>> (font-lock-ensure 63209 63387))
>>>>> 
>>>>> in latest master, but not in latest emacs-29 (only in 5-months old emacs-29).
>>>>> 
>>>>> If this is not reproducible, I could provide more details.
>>>>> 
>>>>> libtree-sitter is at the latest version.
>>>> 
>>>> Hmm, I can’t reproduce with latest master and libtree-sitter. Maybe you can send me the exact commits that you used?
>>>> 
>>>> Here’s mine:
>>>> 
>>>> Emacs: 72f2b01e318
>>>> Tree-sitter: 6ec478c1
>>> 
>>> Probably reproducibility depends on the content of the src/treesit.c file.
>>> Then the most reliable way to reproduce it is this:
>>> 
>>> 0. emacs -Q
>>> 1. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
>>> 2. C-x v L
>>> 3. in the *vc-change-log* buffer move point to the commit 20af58d3a13
>>> 4. type D
>>> 5. crash caused by diff-font-lock-syntax fontification that uses treesit
>>> 
>>> The numbers in (font-lock-ensure 63209 63387) above were extracted
>>> from diff hunk boundaries that might be different when the file was edited.
>> 
>> I reproduce it once with the first set of commits you provided, but for some reason couldn’t reproduce it again. I’m sure it’s something wrong that I did. I’ll report back when I make progress. TBH it seems like something wrong with tree-sitter itself, but I’ll make sure to figure out what’s the problem exactly.
>> 
>> Yuan
> 
> Ok, I can reproduce it now. Looking into it…

Finally figured out why. It’s not tree-sitter’s problem, but ours. I reduced the crash to a signal and pushed the fix to emacs-30. Next I’ll make sure the signal is properly handled. Below quoting the commit message:

The immediate cause of the crash is that tree-sitter accessed a node's
tree, but the tree is already deleted.

What happended, I think, is this:

1. Buffer modified, parser->need_reparse set to true,
parser->timestamp incremented.
2. A node is created from the parser, this node has the old tree but
the _new_ timestamp (bad!).
3. Parser re-parses (treesit_ensure_parsed), new tree created, old
tree deleted.
4. Ftreesit_query_capture accessed the old node, and the old tree,
crash.

We shouldn't bump the parser timestamp when we set
parser->need_reparse to true; instead, we should bump the timestamp
when we actually reparsed and created a new tree.

Yuan

Message #29 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>, bug-gnu-emacs <at> gnu.org
Cc: Yuan Fu <casouri <at> gmail.com>, 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sun, 30 Jun 2024 16:28:53 +0200

Today I did a git pull of the master branch.
Is it possible that this patch is the cause of this error?

Error muted by safe_call: (treesit--font-lock-mark-ranges-to-fontify ((1 . 
8867)) #<treesit-parser for php>) signaled (wrong-type-argument treesit-node-p 
#<treesit-parser for php>)
Error during redisplay: (jit-lock-function 1) signaled (wrong-type-argument 
treesit-node-p #<treesit-parser for php>)

Vincenzo

In data domenica 30 giugno 2024 01:54:39 CEST, Yuan Fu ha scritto:
> > On Jun 25, 2024, at 11:04 PM, Yuan Fu <casouri <at> gmail.com> wrote:
> >> On Jun 24, 2024, at 12:46 AM, Yuan Fu <casouri <at> gmail.com> wrote:
> >>> On Jun 23, 2024, at 10:38 AM, Juri Linkov <juri <at> linkov.net> wrote:
> >>>>> Evaluating this expression causes a crash:
> >>>>> 
> >>>>> (progn
> >>>>> (find-file (expand-file-name "src/treesit.c" installation-directory))
> >>>>> (c-ts-mode)
> >>>>> (font-lock-ensure 63209 63387))
> >>>>> 
> >>>>> in latest master, but not in latest emacs-29 (only in 5-months old
> >>>>> emacs-29).
> >>>>> 
> >>>>> If this is not reproducible, I could provide more details.
> >>>>> 
> >>>>> libtree-sitter is at the latest version.
> >>>> 
> >>>> Hmm, I can’t reproduce with latest master and libtree-sitter. Maybe you
> >>>> can send me the exact commits that you used?
> >>>> 
> >>>> Here’s mine:
> >>>> 
> >>>> Emacs: 72f2b01e318
> >>>> Tree-sitter: 6ec478c1
> >>> 
> >>> Probably reproducibility depends on the content of the src/treesit.c
> >>> file.
> >>> Then the most reliable way to reproduce it is this:
> >>> 
> >>> 0. emacs -Q
> >>> 1. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
> >>> 2. C-x v L
> >>> 3. in the *vc-change-log* buffer move point to the commit 20af58d3a13
> >>> 4. type D
> >>> 5. crash caused by diff-font-lock-syntax fontification that uses treesit
> >>> 
> >>> The numbers in (font-lock-ensure 63209 63387) above were extracted
> >>> from diff hunk boundaries that might be different when the file was
> >>> edited.
> >> 
> >> I reproduce it once with the first set of commits you provided, but for
> >> some reason couldn’t reproduce it again. I’m sure it’s something wrong
> >> that I did. I’ll report back when I make progress. TBH it seems like
> >> something wrong with tree-sitter itself, but I’ll make sure to figure
> >> out what’s the problem exactly.
> >> 
> >> Yuan
> > 
> > Ok, I can reproduce it now. Looking into it…
> 
> Finally figured out why. It’s not tree-sitter’s problem, but ours. I reduced
> the crash to a signal and pushed the fix to emacs-30. Next I’ll make sure
> the signal is properly handled. Below quoting the commit message:
> 
> The immediate cause of the crash is that tree-sitter accessed a node's
> tree, but the tree is already deleted.
> 
> What happended, I think, is this:
> 
> 1. Buffer modified, parser->need_reparse set to true,
> parser->timestamp incremented.
> 2. A node is created from the parser, this node has the old tree but
> the _new_ timestamp (bad!).
> 3. Parser re-parses (treesit_ensure_parsed), new tree created, old
> tree deleted.
> 4. Ftreesit_query_capture accessed the old node, and the old tree,
> crash.
> 
> We shouldn't bump the parser timestamp when we set
> parser->need_reparse to true; instead, we should bump the timestamp
> when we actually reparsed and created a new tree.
> 
> Yuan

Message #35 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sun, 30 Jun 2024 19:15:28 +0300

> Finally figured out why. It’s not tree-sitter’s problem, but
> ours. I reduced the crash to a signal and pushed the fix to
> emacs-30. Next I’ll make sure the signal is properly handled. Below
> quoting the commit message:
>
> The immediate cause of the crash is that tree-sitter accessed a node's
> tree, but the tree is already deleted.
>
> What happended, I think, is this:
>
> 1. Buffer modified, parser->need_reparse set to true,
> parser->timestamp incremented.
> 2. A node is created from the parser, this node has the old tree but
> the _new_ timestamp (bad!).
> 3. Parser re-parses (treesit_ensure_parsed), new tree created, old
> tree deleted.
> 4. Ftreesit_query_capture accessed the old node, and the old tree,
> crash.
>
> We shouldn't bump the parser timestamp when we set
> parser->need_reparse to true; instead, we should bump the timestamp
> when we actually reparsed and created a new tree.

Thank you very much.  I confirm there are no crashes anymore.

Message #38 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>, bug-gnu-emacs <at> gnu.org
Cc: Yuan Fu <casouri <at> gmail.com>, 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sun, 30 Jun 2024 21:22:07 +0200

Today I did a git pull of the master branch.
Is it possible that this patch is the cause of this error?

Error muted by safe_call: (treesit--font-lock-mark-ranges-to-fontify ((1 . 
8867)) #<treesit-parser for php>) signaled (wrong-type-argument treesit-node-p 
#<treesit-parser for php>)
Error during redisplay: (jit-lock-function 1) signaled (wrong-type-argument 
treesit-node-p #<treesit-parser for php>)

Vincenzo

In data domenica 30 giugno 2024 01:54:39 CEST, Yuan Fu ha scritto:
> > On Jun 25, 2024, at 11:04 PM, Yuan Fu <casouri <at> gmail.com> wrote:
> >> On Jun 24, 2024, at 12:46 AM, Yuan Fu <casouri <at> gmail.com> wrote:
> >>> On Jun 23, 2024, at 10:38 AM, Juri Linkov <juri <at> linkov.net> wrote:
> >>>>> Evaluating this expression causes a crash:
> >>>>> 
> >>>>> (progn
> >>>>> (find-file (expand-file-name "src/treesit.c" installation-directory))
> >>>>> (c-ts-mode)
> >>>>> (font-lock-ensure 63209 63387))
> >>>>> 
> >>>>> in latest master, but not in latest emacs-29 (only in 5-months old
> >>>>> emacs-29).
> >>>>> 
> >>>>> If this is not reproducible, I could provide more details.
> >>>>> 
> >>>>> libtree-sitter is at the latest version.
> >>>> 
> >>>> Hmm, I can’t reproduce with latest master and libtree-sitter. Maybe you
> >>>> can send me the exact commits that you used?
> >>>> 
> >>>> Here’s mine:
> >>>> 
> >>>> Emacs: 72f2b01e318
> >>>> Tree-sitter: 6ec478c1
> >>> 
> >>> Probably reproducibility depends on the content of the src/treesit.c
> >>> file.
> >>> Then the most reliable way to reproduce it is this:
> >>> 
> >>> 0. emacs -Q
> >>> 1. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
> >>> 2. C-x v L
> >>> 3. in the *vc-change-log* buffer move point to the commit 20af58d3a13
> >>> 4. type D
> >>> 5. crash caused by diff-font-lock-syntax fontification that uses treesit
> >>> 
> >>> The numbers in (font-lock-ensure 63209 63387) above were extracted
> >>> from diff hunk boundaries that might be different when the file was
> >>> edited.
> >> 
> >> I reproduce it once with the first set of commits you provided, but for
> >> some reason couldn’t reproduce it again. I’m sure it’s something wrong
> >> that I did. I’ll report back when I make progress. TBH it seems like
> >> something wrong with tree-sitter itself, but I’ll make sure to figure
> >> out what’s the problem exactly.
> >> 
> >> Yuan
> > 
> > Ok, I can reproduce it now. Looking into it…
> 
> Finally figured out why. It’s not tree-sitter’s problem, but ours. I reduced
> the crash to a signal and pushed the fix to emacs-30. Next I’ll make sure
> the signal is properly handled. Below quoting the commit message:
> 
> The immediate cause of the crash is that tree-sitter accessed a node's
> tree, but the tree is already deleted.
> 
> What happended, I think, is this:
> 
> 1. Buffer modified, parser->need_reparse set to true,
> parser->timestamp incremented.
> 2. A node is created from the parser, this node has the old tree but
> the _new_ timestamp (bad!).
> 3. Parser re-parses (treesit_ensure_parsed), new tree created, old
> tree deleted.
> 4. Ftreesit_query_capture accessed the old node, and the old tree,
> crash.
> 
> We shouldn't bump the parser timestamp when we set
> parser->need_reparse to true; instead, we should bump the timestamp
> when we actually reparsed and created a new tree.
> 
> Yuan

Message #44 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Vincenzo Pupillo <v.pupillo <at> gmail.com>
Cc: Bug Report Emacs <bug-gnu-emacs <at> gnu.org>, 71681 <at> debbugs.gnu.org,
 Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Sun, 30 Jun 2024 22:37:40 -0700

> On Jun 30, 2024, at 12:22 PM, Vincenzo Pupillo <v.pupillo <at> gmail.com> wrote:
> 
> Today I did a git pull of the master branch.
> Is it possible that this patch is the cause of this error?
> 
> Error muted by safe_call: (treesit--font-lock-mark-ranges-to-fontify ((1 . 
> 8867)) #<treesit-parser for php>) signaled (wrong-type-argument treesit-node-p 
> #<treesit-parser for php>)
> Error during redisplay: (jit-lock-function 1) signaled (wrong-type-argument 
> treesit-node-p #<treesit-parser for php>)
> 
> Vincenzo

Embarrassingly, yes. I made a typo and didn’t catch it. It should be fixed now (on emacs-30).

Yuan

Message #50 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Mon, 01 Jul 2024 09:49:09 +0300

> I reduced the crash to a signal and pushed the fix to emacs-30.
> Next I’ll make sure the signal is properly handled.

Now with the latest emacs-30 at the commit b2c966f8396
there is another problem:

0. emacs -Q
1. eval: (setq backtrace-on-redisplay-error t)
2. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
3. C-x v L
4. in the *vc-change-log* buffer move point to the commit b2c966f8396
5. type D

Warning (error): Error in a redisplay Lisp hook.
See buffer *Redisplay-trace*

Error: treesit-node-outdated (#<treesit-node-outdated>)
  treesit--font-lock-fontify-region-1(#<treesit-node-outdated> #<treesit-compiled-query> 99654 99975 nil nil)
  treesit-font-lock-fontify-region(99654 99975 nil)
  font-lock-fontify-syntactically-region(99654 99975 nil)
  font-lock-default-fontify-region(99654 99974 nil)
  font-lock-fontify-region(99654 99974)
  font-lock-ensure(99654 99974)
  diff-syntax-fontify-hunk(122 539 t)
  diff-syntax-fontify(122 539)
  diff--font-lock-syntax(539)
  font-lock-fontify-keywords-region(1 539 nil)
  font-lock-default-fontify-region(1 539 nil)
  font-lock-fontify-region(1 539)
  jit-lock--run-functions(1 539)
  jit-lock-fontify-now(1 539)
  jit-lock-function(1)
  vc-diff-finish(#<buffer *vc-diff*> nil nil)
  vc-exec-after(#f(compiled-function () #<bytecode -0xcd6e6e57937525e>))
  log-view-diff-common(1 1 t)
  log-view-diff-changeset(1 1)
  funcall-interactively(log-view-diff-changeset 1 1)
  command-execute(log-view-diff-changeset)

Message #53 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: 71681 <at> debbugs.gnu.org
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Mon, 1 Jul 2024 00:01:23 -0700

> On Jun 30, 2024, at 11:49 PM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>> I reduced the crash to a signal and pushed the fix to emacs-30.
>> Next I’ll make sure the signal is properly handled.
> 
> Now with the latest emacs-30 at the commit b2c966f8396
> there is another problem:
> 
> 0. emacs -Q
> 1. eval: (setq backtrace-on-redisplay-error t)
> 2. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
> 3. C-x v L
> 4. in the *vc-change-log* buffer move point to the commit b2c966f8396
> 5. type D
> 
> Warning (error): Error in a redisplay Lisp hook.
> See buffer *Redisplay-trace*
> 
> Error: treesit-node-outdated (#<treesit-node-outdated>)
>  treesit--font-lock-fontify-region-1(#<treesit-node-outdated> #<treesit-compiled-query> 99654 99975 nil nil)
>  treesit-font-lock-fontify-region(99654 99975 nil)
>  font-lock-fontify-syntactically-region(99654 99975 nil)
>  font-lock-default-fontify-region(99654 99974 nil)
>  font-lock-fontify-region(99654 99974)
>  font-lock-ensure(99654 99974)
>  diff-syntax-fontify-hunk(122 539 t)
>  diff-syntax-fontify(122 539)
>  diff--font-lock-syntax(539)
>  font-lock-fontify-keywords-region(1 539 nil)
>  font-lock-default-fontify-region(1 539 nil)
>  font-lock-fontify-region(1 539)
>  jit-lock--run-functions(1 539)
>  jit-lock-fontify-now(1 539)
>  jit-lock-function(1)
>  vc-diff-finish(#<buffer *vc-diff*> nil nil)
>  vc-exec-after(#f(compiled-function () #<bytecode -0xcd6e6e57937525e>))
>  log-view-diff-common(1 1 t)
>  log-view-diff-changeset(1 1)
>  funcall-interactively(log-view-diff-changeset 1 1)
>  command-execute(log-view-diff-changeset)

Yes, that’s what meant by “reduced crash to signal”. The crash is fixed, but I need to fix the font-lock code so it can handle the signal gracefully (or don’t cause the signal from the first place). It’s not yet clear to me why does treesit-font-lock-fontify-region end up using an outdated node for query.

Yuan

Message #56 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Bug Report Emacs <bug-gnu-emacs <at> gnu.org>, 71681 <at> debbugs.gnu.org,
 Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Mon, 01 Jul 2024 12:20:51 +0200

Thank you Yuan!
Vincenzo

In data lunedì 1 luglio 2024 07:37:40 CEST, Yuan Fu ha scritto:
> 
> > On Jun 30, 2024, at 12:22 PM, Vincenzo Pupillo <v.pupillo <at> gmail.com> wrote:
> > 
> > Today I did a git pull of the master branch.
> > Is it possible that this patch is the cause of this error?
> > 
> > Error muted by safe_call: (treesit--font-lock-mark-ranges-to-fontify ((1 . 
> > 8867)) #<treesit-parser for php>) signaled (wrong-type-argument treesit-node-p 
> > #<treesit-parser for php>)
> > Error during redisplay: (jit-lock-function 1) signaled (wrong-type-argument 
> > treesit-node-p #<treesit-parser for php>)
> > 
> > Vincenzo
> 
> Embarrassingly, yes. I made a typo and didn’t catch it. It should be fixed now (on emacs-30).
> 
> Yuan
> 

Message #62 received at 71681 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 71681 <at> debbugs.gnu.org, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#71681: 29.3.50; tree-sitter crash
Date: Fri, 28 Feb 2025 18:00:39 -0800

Yuan Fu <casouri <at> gmail.com> writes:

>> On Jun 30, 2024, at 11:49 PM, Juri Linkov <juri <at> linkov.net> wrote:
>>
>>> I reduced the crash to a signal and pushed the fix to emacs-30.
>>> Next I’ll make sure the signal is properly handled.
>>
>> Now with the latest emacs-30 at the commit b2c966f8396
>> there is another problem:
>>
>> 0. emacs -Q
>> 1. eval: (setq backtrace-on-redisplay-error t)
>> 2. eval: (add-to-list 'major-mode-remap-alist '(c-mode . c-ts-mode))
>> 3. C-x v L
>> 4. in the *vc-change-log* buffer move point to the commit b2c966f8396
>> 5. type D
>>
>> Warning (error): Error in a redisplay Lisp hook.
>> See buffer *Redisplay-trace*
>>
>> Error: treesit-node-outdated (#<treesit-node-outdated>)
>>  treesit--font-lock-fontify-region-1(#<treesit-node-outdated> #<treesit-compiled-query> 99654 99975 nil nil)
>>  treesit-font-lock-fontify-region(99654 99975 nil)
>>  font-lock-fontify-syntactically-region(99654 99975 nil)
>>  font-lock-default-fontify-region(99654 99974 nil)
>>  font-lock-fontify-region(99654 99974)
>>  font-lock-ensure(99654 99974)
>>  diff-syntax-fontify-hunk(122 539 t)
>>  diff-syntax-fontify(122 539)
>>  diff--font-lock-syntax(539)
>>  font-lock-fontify-keywords-region(1 539 nil)
>>  font-lock-default-fontify-region(1 539 nil)
>>  font-lock-fontify-region(1 539)
>>  jit-lock--run-functions(1 539)
>>  jit-lock-fontify-now(1 539)
>>  jit-lock-function(1)
>>  vc-diff-finish(#<buffer *vc-diff*> nil nil)
>>  vc-exec-after(#f(compiled-function () #<bytecode -0xcd6e6e57937525e>))
>>  log-view-diff-common(1 1 t)
>>  log-view-diff-changeset(1 1)
>>  funcall-interactively(log-view-diff-changeset 1 1)
>>  command-execute(log-view-diff-changeset)
>
> Yes, that’s what meant by “reduced crash to signal”. The crash is fixed, but I
> need to fix the font-lock code so it can handle the signal gracefully (or don’t
> cause the signal from the first place). It’s not yet clear to me why does
> treesit-font-lock-fontify-region end up using an outdated node for query.

Did you make any progress with the second part of this fix?

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.