GNU bug report logs - #62333
30.0.50; Issue with tree-sitter syntax tree during certain changes

Previous Next

Full log

View this message in rfc822 format

From: Yuan Fu <casouri <at> gmail.com>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: Wilhelm Kirschbaum <wkirschbaum <at> gmail.com>, 62333 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: bug#62333: 30.0.50; Issue with tree-sitter syntax tree during certain changes
Date: Wed, 22 Mar 2023 20:03:20 -0700

[Message part 1 (text/plain, inline)]

> On Mar 22, 2023, at 6:03 PM, Dmitry Gutov <dgutov <at> yandex.ru> wrote:
> 
> On 21/03/2023 16:00, Wilhelm Kirschbaum wrote:
>> With the following text in a buffer with elixir-ts-mode:
>> def foo() do
>> end
>> When I delete the `)` and re-insert it, the tree-sitter syntax tree does
>> not fully recover and the `do end` part is absent when observing using
>> treesit-explore-mode.
>> Before removing the `)`:
>> (source
>> (call target: (identifier)
>>  (arguments
>>   (call target: (identifier)
>>    (arguments ( ))))
>>  (do_block do end)))
>> after re-inserting `)`:
>> (source
>> (call target: (identifier)
>>  (arguments
>>   (call target: (identifier)
>>    (arguments ( ))))))
>> I can see the same behaviour using ruby-ts-mode when I remove the
>> following expression:
>> (setq-local syntax-propertize-function #'ruby-ts--syntax-propertize)
>> The ruby syntax I tested with:
>> def foo()
>> end
>> When I add a local `syntax-propertize-function' function to elixir-ts-mode
>> it also solves the problem of not fully recovering the tree-sitter
>> syntax tree.  It does not seem to matter what the function achieves
>> thought, so am a bit confused.
> 
> I can repro both scenarios. Except with ruby-ts-mode, the difference in printed AST is the added wrapping in (program ...). But the visible result is similar: "end" not being highlighted.

The problem seems to be caused by narrowing during fontification: when parsing, tree-sitter sees that BUF_ZV_BYTE is smaller than it’s visible_end, and removed the part after the parenthesis from the parse tree. I suspect this has to do with the narrowed fontification we added for dealing with long lines. 

CC’ed Eli because I don’t really know anything about the locked narrow thing besides that it exists. In particular, how does it determine when and where to narrow?


To see it in action, apply this patch, and repeat Wilhelm’s experiment: remove the “)” and reinsert it. You should see

start: 9, oend: 10, nend: 9
start: 9, oend: 10, nend: 9
start: 9, oend: 9, nend: 10
start: 9, oend: 9, nend: 10
2.5 start: 10, oend: 18, nend: 10 —> culprit

(Make sure to turn off electric-pair-mode to reduce noise.)

Yuan

[print.diff (application/octet-stream, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.