GNU bug report logs - #59426
29.0.50; [tree-sitter] Some functions exceed maximum recursion limit

Previous Next

Full log

View this message in rfc822 format

From: Yuan Fu <casouri <at> gmail.com>
To: Mattias Engdegård <mattiase <at> acm.org>
Cc: Po Lu <luangruo <at> yahoo.com>, 59426 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, Stefan Kangas <stefankangas <at> gmail.com>
Subject: bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit
Date: Thu, 24 Nov 2022 01:17:02 -0800

> On Nov 23, 2022, at 12:01 PM, Mattias Engdegård <mattiase <at> acm.org> wrote:
> 
> 23 nov. 2022 kl. 19.46 skrev Yuan Fu <casouri <at> gmail.com>:
> 
>> It shouldn’t, but tree-sitter thinks some closing brackets are erroneous and skips them when parsing (it skips erroneous tokens in the hope to parse the rest of the file despite local errors). So a 10k wide tree becomes 10k tall.
>> 
>> We can submit a bug repot to tree-sitter-c (“maybe don’t skip closing brackets even there is error, or somthing”), but that’s another story.
> 
> Thanks for the explanation. In this case it seems that it's the #line directive that throws a spanner in the works. You probably already discovered that, but for the record, here is a cut-down example:
> 
> static hf_register_info hf[] = {
> #line 1 "./asn1/rrc/packet-rrc-hfarr.c"
>    { &hf_rrc_DL_DCCH_Message_PDU,
>      { "DL-DCCH-Message", "rrc.DL_DCCH_Message_element",
>        FT_NONE, BASE_NONE, NULL, 0,
>        NULL, HFILL }},
>    { &hf_rrc_cellIdentity_c_id,
>       {"Cell Identifier", "rrc.cellIdentity.c_id",
>       FT_UINT32, BASE_DEC, NULL, 0,
>       "The Cell Identifier (C-Id) part of the Cell Identity", HFILL }}
>  };
> 
> Note how the warning colour of the curly brackets vanishes once the #line line is removed.
> Even if this snag is corrected, there will always be cases where preprocessor use causes trouble of this or a similar kind. It seems quite convincing that we should void C recursion in favour of explicit stacks where possible.
> 

Does it worth the complexity tho? We only need a stack if we want to support this scenario, in which case tree-sitter has a wrong parse tree. Instead of spending the time and resource to go down that deep tree, it’s better to fail early, and let the user decide to either give up on weird files, or try some other approximation.

It’s too early to tell if being able to go down arbitrarily deep into a deep tree is useful. The only use of traversing the whole tree right now is to generate the imenu indexes, which don’t really need to go down more than 10 levels, since most defun nodes we are interested in are either top-level or near top-level.

So I’d prefer we keep it simple and have a hard limit for now. If we later find that a stack is favorable we can always add it in.

Yuan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.