Subject: Re: bug#78402: treesit after-change-functions

From: Yuan Fu <[email-masked]>
Date: Sat, 12 Jul 2025 22:34:01 -0700

> On Jul 12, 2025, at 2:02 AM, Eli Zaretskii <[email-masked]> wrote:
> 
>> Cc: Stefan Monnier <[email-masked]>, [email-masked]
>> From: Yuan Fu <[email-masked]>
>> Date: Tue, 8 Jul 2025 23:42:21 -0700
>> 
>> 
>> 
>>> On Jun 6, 2025, at 8:42 AM, Juri Linkov <[email-masked]> wrote:
>>> 
>>>>> 1. M-: (setq outline-minor-mode-cycle t)
>>>>> 2. M-: (setq outline-minor-mode-use-buttons t)
>>>>> 3. M-: (setq debug-on-error t)
>>>>> 4. C-x p f test/manual/indent/html-multi.html
>>>>> 5. M-x load-library RET treesit-x
>>>>> 6. M-x liquid-generic-ts-mode
>>>>> 7. M-x outline-minor-mode
>>>>> 8. move point to the first 's' in 'script' on line 19
>>>>> 9. type 'M-d' ('kill-word')
>>>> 
>>>> Aha, I reproduced it. It didn’t happen when I kill-word, but happened when
>>>> I undo.
>>> 
>>> This is what I observe too: when it doesn't fail on edit,
>>> then in any case it fails on undo.
>>> 
>>>> I’m still trying to understand why the node-outdated error can
>>>> happen. Each parser has a tick, which is incremented whenever it
>>>> re-parses. When we create a node from the parser, the node inherits the
>>>> current tick number of the parser. We only consider a node outdated when
>>>> its tick is less than its parser’s tick, meaning the parser re-parsed after
>>>> the node is created. For a parser to re-parse, it has to a) receive
>>>> a buffer edit or b) change it’s ranges; and then someone needs to request
>>>> a node from it.
>>>> 
>>>> That means in the following backtrace, the parser received a buffer
>>>> edit or changed its ranges after the `prev` node is created. And
>>>> that’s baffling to me.
>>> 
>>> Maybe calling this from after-change-functions affects the order of
>>> calling parser functions.
>> 
>> Ok, after much struggle, the bug should be fixed now. Quoting my commit message:
>> 
>> Right now in treesit-outline-search -> treesit-navigate-thing, a
>> freshly created tree-sitter node becomes outdated within the
>> function.  I'm not sure _exactly_ how it happend, but it might
>> look like this: we first get a node from, say, html parser, then
>> get another node from, say, liquid parser.  Creating the node
>> from liquid parser causes a reparse which updated the range of
>> the html parser, which rendered the html node outdated.
>> 
>> There're several problems with the current design, let's start
>> with the most obvious one: we add
>> treesit--font-lock-mark-ranges-to-fontify as a notifier of the
>> primar parser in treesit-major-mode-setup.  Now, if a ts major
>> mode inherits another major mode, treesit-major-mode-setup will
>> be called twice, once in the parent mode and once in the child
>> mode, and two parsers will have the notifier.  But
>> treesit--font-lock-mark-ranges-to-fontify is designed to run
>> only once.
>> 
>> I believe this bug, together with some mysterious async
>> execution order, led to the problems we saw in the bug report.
>> My solution is to just make everything synchronous.
>> 
>> So I added treesit-parser-changed-regions, and modified
>> treesit--font-lock-mark-ranges-to-fontify to use it.  Now we
>> don't need to add the notifier to the primary parser anymore.
>> 
>> I also applied the tree-sitter-outline change we discussed in
>> the bug report.  (Change to treesit-outline-search, and remove
>> treesit--after-change.)
> 
> Thanks.  Should this bug be closed now?

I closed it. If it somehow happens again (unlikely) we can reopen it.

Yuan