GNU bug report logs - #76398
treesit-aggregated-outline-predicate

Previous Next

Full log

View this message in rfc822 format

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: 76398 <at> debbugs.gnu.org, Vincenzo Pupillo <v.pupillo <at> gmail.com>
Subject: bug#76398: treesit-aggregated-outline-predicate
Date: Thu, 27 Feb 2025 19:10:00 -0800

> On Feb 24, 2025, at 11:37 AM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>>>>>>> So this patch helps 'treesit-outline-search' to get out of the local parser
>>>>>>> to the primary parser to continue search for the next outline predicate.
>>>>>>> 
>>>>>>> 'treesit-outline-level' should do the same, but currently I can't find
>>>>>>> a suitable function to break out of embedded confinement
>>>>>>> and get the host node that contains the guest ranges.
>>>>>>> I mean that e.g. (treesit-parser-root-node (treesit-node-parser node))
>>>>>>> can get the root node of the local parser, but how to get its parent node
>>>>>>> in the primary parser?  It's understandable that treesit-node-parent
>>>>>>> doesn't go out of its parser.  But maybe there is another function?
>>>>>>> If such function doesn't exist, this is fine, then could find that
>>>>>>> node manually by calculating from treesit-parser-included-ranges.
>>>>>> 
>>>>>> Maybe we need two new primitives:
>>>>>> 
>>>>>> (treesit-next-parser-boundary POS)
>>>>>> (treesit-prev-parser-boundary POS)
>>>>> 
>>>>> Now pushed as 'treesit-closest-parser-boundary'.
>>>> 
>>>> Hold on, let’s not get ahead of ourselves. First of all, the name is not
>>>> very descriptive IMO it actually finds range boundary, not parser boundary;
>>>> and the docstring mentions local parser while the function itself doesn’t
>>>> really involve local parsers—it just checks parser ranges. It can be used
>>>> for getting out of local parsers, yes, but that’s a use-case, not what it
>>>> does. So if we want to add this function to the public API set for
>>>> tree-sitter, it needs a better docstring. (And at the moment I have doubt
>>>> on its general usefulness.)
>>> 
>>> I assumed that this function will not remain in the final
>>> implementation of this feature.  I added it temporarily
>>> to get the embedded outlines into a working state.
>> 
>> Great. Then let’s use double dash and maybe even add some comments to explain that, WDYT?
> 
> Maybe a better name would be 'treesit-outline--closest-range-boundary'?
> 
> It's needed to prevent skipping the range boundaries that
> treesit-navigate-thing does by default, e.g. instead of
> 
> from (the starting point of the navigation)
> range_1_beg
> range_1_end
> to (the ending point of the navigation)
> 
> we need to stop inside the next range and search inside from its beginning:
> 
> from
> range_1_beg
> to
> range_1_end
> 
> Another case it that when inside a local range, treesit-navigate-thing
> returns nil, but need to go outside the local range, and continue the search:
> 
> range_1_beg
> from
> range_1_end
> to
> 
>>>> More over, is this even necessary?  Why do we need to go over all the
>>>> ranges for all the parsers to get out of a local parser?  I thought we
>>>> can just get the local parser and get it’s range?
>>> 
>>> I tried to use treesit-local-parsers-at, but it always returns nil.
>> 
>> That’s probably because the major mode you’re testing with doesn’t use
>> local parser for the embedded language. I’ll add a function to get the
>> “parent node”, that should solve your problem here.
> 
> Like you noticed, we need to check ranges, not parsers anyway,
> so treesit-local-parsers-at can't help here.

Combine with your reply above, it seems we need some information attached to the ranges (with either overlay or text property), rather than to the parser. Is that right? When updating ranges, we can mark the ranges covered by each embed parser, and perhaps link the text prop or overlay to the “parent node”.


>>>>>> (treesit-upper-parser-node POS)
>>>>> 
>>>>> Addition of 'treesit-upper-parser-node' is underway that should
>>>>> be used in 'treesit-up-list' as well.
>>>> 
>>>> And if we need to get the “parent node” of a local parser, we can do it in
>>>> much nicer ways. We can record the parent node when creating the local
>>>> parser, by either adding a field to the parser object, or record it in
>>>> a local database, or even just save it in the text property alongside the
>>>> local parser. Let’s take some time and think of the best way to solve
>>>> this.
>>> 
>>> Now I improved treesit-outline-level as well without adding
>>> new functions.  Everything works now, so we can do more
>>> refactoring without introducing regressions.
>>> 
>>>> Whatever you have in mind, I suspect that it wouldn’t work if there
>>>> are more than one layer of nesting of parsers, ie, what if you want to get
>>>> out of a embedded (local) parser’s embedded parser?
>>> 
>>> The current implementation in treesit-outline-level works with any depth
>>> of nested parsers.  But the current solution is quite fragile.  So we need
>>> to find a better way such as recording the parent parser node somewhere.
> 
> Actually, here as well we need a parent node for the range, not the parser.
> Maybe the range should be a special object like the parser is?
> 
>>>> On the same note, we actually need some proper tree structure for the
>>>> primary parser - local parser relationship, because there can be more than
>>>> one layer. What we currently have doesn’t handle this well (font-lock and
>>>> indentation). It’s a real use case, someone requested for this for the Perl
>>>> (or Haskell?) mode, and imagine a rust buffer embeds a markdown comment
>>>> which embeds rust code examples. I bring this up because this tree
>>>> structure would’ve solved your problem here as well, once we have it.
>>> 
>>> Agreed, this needs a better design.
>> 
>> Cool, working on it.
> 
> Thanks.

I’ve mostly worked out arbitrary nesting of embedded parsers. I tested with markdown -> javascript -> jsdoc, and it seems to work fine. What I’m not sure about right now is the navigation part.

Yuan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.