GNU bug report logs - #77906
31.0.50; php-js-mode and js-ts-mode: treesit-show-paren-data--categorize: peculiar error: ("Cannot find the definition of the predicate in `treesit-thing-settings'" list)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 77906 in the body.
You can then email your comments to 77906 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
To: Bug Emacs <bug-gnu-emacs <at> gnu.org>
Subject: 31.0.50;
 php-js-mode and js-ts-mode: treesit-show-paren-data--categorize: peculiar
 error: ("Cannot find the definition of the predicate in
 `treesit-thing-settings'" list)
Date: Fri, 18 Apr 2025 20:57:01 +0200

Ciao,
both php-ts-mode and js-ts-mode show the same error. It is similar to bug 
#75456. It was fixed by me at the time, for js-ts-mode, with a patch to 
js--treesit-language-at-point. The problem arises when there are languages 
other than the main language in the buffer.
The following snippet shows the problem, just put the cursor at the end of the 
comment (jsdoc parser required):

function MyBigFunc(param1,
                   param2) {
    const abc = 'def';
}
/**
 * Rapresent
 * @param {int} a dada
 */| <-- cursor here
var variablename = function(a) {
    console.log("text"); // dada
    /** dada */
    // dada
};

With php it is even worse because of the number of embedded languages.

Any idea?

Thanks.
Vincenzo

Message #8 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Vincenzo Pupillo <v.pupillo <at> gmail.com>
Cc: Yuan Fu <casouri <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Sun, 20 Apr 2025 09:19:18 +0300

> both php-ts-mode and js-ts-mode show the same error. It is similar to bug 
> #75456. It was fixed by me at the time, for js-ts-mode, with a patch to 
> js--treesit-language-at-point. The problem arises when there are languages 
> other than the main language in the buffer.
> The following snippet shows the problem, just put the cursor at the end of the 
> comment (jsdoc parser required):
>
> function MyBigFunc(param1,
>                    param2) {
>     const abc = 'def';
> }
> /**
>  * Rapresent
>  * @param {int} a dada
>  */| <-- cursor here
> var variablename = function(a) {
>     console.log("text"); // dada
>     /** dada */
>     // dada
> };
>
> With php it is even worse because of the number of embedded languages.
>
> Any idea?

I noticed the same problem that the end of a range
unexpectedly belongs to its host that breaks many things
such as navigation to the start of the embedded root node
from its end.  Instead it uses the host node.

Maybe point at the end of the range should belong
to the embedded parser?

Message #11 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Sun, 20 Apr 2025 07:55:04 -0700

> On Apr 19, 2025, at 11:19 PM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>> both php-ts-mode and js-ts-mode show the same error. It is similar to bug 
>> #75456. It was fixed by me at the time, for js-ts-mode, with a patch to 
>> js--treesit-language-at-point. The problem arises when there are languages 
>> other than the main language in the buffer.
>> The following snippet shows the problem, just put the cursor at the end of the 
>> comment (jsdoc parser required):
>> 
>> function MyBigFunc(param1,
>>                   param2) {
>>    const abc = 'def';
>> }
>> /**
>> * Rapresent
>> * @param {int} a dada
>> */| <-- cursor here
>> var variablename = function(a) {
>>    console.log("text"); // dada
>>    /** dada */
>>    // dada
>> };
>> 
>> With php it is even worse because of the number of embedded languages.
>> 
>> Any idea?
> 
> I noticed the same problem that the end of a range
> unexpectedly belongs to its host that breaks many things
> such as navigation to the start of the embedded root node
> from its end.  Instead it uses the host node.
> 
> Maybe point at the end of the range should belong
> to the embedded parser?

The end position of a region has always been considered not part of a region, not only in Emacs, but pretty much everywhere. Let’s fix treesit-show-paren-data—categorize instead. We can add optional arguments to treesit-language-at and friends to make it easier to handle the end-of-range case.

Yuan

Message #14 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Sun, 20 Apr 2025 21:29:47 +0300

close 77906 31.0.50
thanks

> The end position of a region has always been considered not part of
> a region, not only in Emacs, but pretty much everywhere. Let’s fix
> treesit-show-paren-data—categorize instead.

Ah, I missed that this bug report is about treesit-show-paren-data--categorize
because it was mentioned only in the subject, and who reads subjects 😄

So now fixed the off-by-one error in treesit-show-paren-data--categorize.

Message #19 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Vincenzo Pupillo <v.pupillo <at> gmail.com>
To: Yuan Fu <casouri <at> gmail.com>, Juri Linkov <juri <at> linkov.net>
Cc: 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50;
 php-js-mode and js-ts-mode: treesit-show-paren-data--categorize: peculiar
 error: ("Cannot find the definition of the predicate in
 `treesit-thing-settings'" list)
Date: Sun, 20 Apr 2025 20:58:05 +0200

In data domenica 20 aprile 2025 20:29:47 Ora legale dell’Europa centrale, Juri 
Linkov ha scritto:
> close 77906 31.0.50
> thanks
> 
> > The end position of a region has always been considered not part of
> > a region, not only in Emacs, but pretty much everywhere. Let’s fix
> > treesit-show-paren-data—categorize instead.
> 
> Ah, I missed that this bug report is about
> treesit-show-paren-data--categorize because it was mentioned only in the
> subject, and who reads subjects 😄
:-D

> 
> So now fixed the off-by-one error in treesit-show-paren-data--categorize.
Thanks Juri.

Vincenzo

Message #22 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Sun, 20 Apr 2025 19:00:54 -0700

> On Apr 20, 2025, at 11:29 AM, Juri Linkov <juri <at> linkov.net> wrote:
> 
> close 77906 31.0.50
> thanks
> 
>> The end position of a region has always been considered not part of
>> a region, not only in Emacs, but pretty much everywhere. Let’s fix
>> treesit-show-paren-data—categorize instead.
> 
> Ah, I missed that this bug report is about treesit-show-paren-data--categorize
> because it was mentioned only in the subject, and who reads subjects 😄
> 
> So now fixed the off-by-one error in treesit-show-paren-data—categorize.

Thanks!

Yuan

Message #25 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Tue, 22 Apr 2025 20:36:03 +0300

>> So now fixed the off-by-one error in treesit-show-paren-data—categorize.
>
> Thanks!

Now added this docstring to the internal function
'treesit-show-paren-data--categorize':

    "Return a list suitable for `show-paren-data-function' (which see).
  If the optional argument END-P is non-nil, interpret the position POS
  as belonging to the node that ends before POS (by subtracting 1 from POS)."

I guess more functions will need such flag, e.g. 'backward-sexp'
at the end of the range doesn't skip to the embedded node's start.

Message #28 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Wed, 23 Apr 2025 19:55:34 +0300

> Now added this docstring to the internal function
> 'treesit-show-paren-data--categorize':
>
>     "Return a list suitable for `show-paren-data-function' (which see).
>   If the optional argument END-P is non-nil, interpret the position POS
>   as belonging to the node that ends before POS (by subtracting 1 from POS)."
>
> I guess more functions will need such flag, e.g. 'backward-sexp'
> at the end of the range doesn't skip to the embedded node's start.

This patch is not for immediate installation, but
it demonstrates how this problem can be fixed:

  diff --git a/lisp/treesit.el b/lisp/treesit.el
  index bf5c1ed5f6c..c73b380f38c 100644
  --- a/lisp/treesit.el
  +++ b/lisp/treesit.el
  @@ -869,7 +869,7 @@ treesit-parsers-at
     (let ((res nil))
       ;; Refer to (ref:local-parser-overlay) for more explanation of local
       ;; parser overlays.
  -    (dolist (ov (overlays-at (or pos (point))))
  +    (dolist (ov (overlays-at (1- (or pos (point)))))
         (when-let* ((parser (overlay-get ov 'treesit-parser))
                     (host-parser (or (null with-host)
                                      (overlay-get ov 'treesit-host-parser)))

With this patch the following case works correctly in test.heex:

  <%= if true do %>
    <p>hello, tree-sitter!</p>
  <% end| %>

With point at "|" the command 'backward-sexp' (C-M-b)
correctly jumps to the beginning of "if".

Message #31 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Juri Linkov <juri <at> linkov.net>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Wed, 23 Apr 2025 21:56:54 -0700

> On Apr 23, 2025, at 9:55 AM, Juri Linkov <juri <at> linkov.net> wrote:
> 
>> Now added this docstring to the internal function
>> 'treesit-show-paren-data--categorize':
>> 
>>    "Return a list suitable for `show-paren-data-function' (which see).
>>  If the optional argument END-P is non-nil, interpret the position POS
>>  as belonging to the node that ends before POS (by subtracting 1 from POS)."
>> 
>> I guess more functions will need such flag, e.g. 'backward-sexp'
>> at the end of the range doesn't skip to the embedded node's start.
> 
> This patch is not for immediate installation, but
> it demonstrates how this problem can be fixed:
> 
>  diff --git a/lisp/treesit.el b/lisp/treesit.el
>  index bf5c1ed5f6c..c73b380f38c 100644
>  --- a/lisp/treesit.el
>  +++ b/lisp/treesit.el
>  @@ -869,7 +869,7 @@ treesit-parsers-at
>     (let ((res nil))
>       ;; Refer to (ref:local-parser-overlay) for more explanation of local
>       ;; parser overlays.
>  -    (dolist (ov (overlays-at (or pos (point))))
>  +    (dolist (ov (overlays-at (1- (or pos (point)))))
>         (when-let* ((parser (overlay-get ov 'treesit-parser))
>                     (host-parser (or (null with-host)
>                                      (overlay-get ov 'treesit-host-parser)))
> 
> With this patch the following case works correctly in test.heex:
> 
>  <%= if true do %>
>    <p>hello, tree-sitter!</p>
>  <% end| %>
> 
> With point at "|" the command 'backward-sexp' (C-M-b)
> correctly jumps to the beginning of "if".

Right. I think it can be done in backward-sexp since there we know we’re moving back. I assume that’s what you mean.

Yuan

Message #34 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Thu, 24 Apr 2025 09:47:08 +0300

>>  @@ -869,7 +869,7 @@ treesit-parsers-at
>>     (let ((res nil))
>>       ;; Refer to (ref:local-parser-overlay) for more explanation of local
>>       ;; parser overlays.
>>  -    (dolist (ov (overlays-at (or pos (point))))
>>  +    (dolist (ov (overlays-at (1- (or pos (point)))))
>>         (when-let* ((parser (overlay-get ov 'treesit-parser))
>>                     (host-parser (or (null with-host)
>>                                      (overlay-get ov 'treesit-host-parser)))
>> 
>> With this patch the following case works correctly in test.heex:
>> 
>>  <%= if true do %>
>>    <p>hello, tree-sitter!</p>
>>  <% end| %>
>> 
>> With point at "|" the command 'backward-sexp' (C-M-b)
>> correctly jumps to the beginning of "if".
>
> Right.  I think it can be done in backward-sexp since there we know
> we’re moving back.  I assume that’s what you mean.

The problem is that the last time when the arg is still -1
is in 'treesit-forward-sexp', and then it calls
'treesit-beginning-of-thing' with '(abs arg)'.

The complete call chain:

- treesit-forward-sexp
- treesit-beginning-of-thing
- treesit-navigate-thing
- treesit-thing-at
- treesit-node-at
- treesit-parsers-at
- overlays-at

So not clear how to propagate the flag for decrementing the offset.
Maybe by let-binding a new dynamically bound variable?

Message #37 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Sun, 04 May 2025 19:50:29 +0300

[Message part 1 (text/plain, inline)]

>>> With this patch the following case works correctly in test.heex:
>>> 
>>>  <%= if true do %>
>>>    <p>hello, tree-sitter!</p>
>>>  <% end| %>
>>> 
>>> With point at "|" the command 'backward-sexp' (C-M-b)
>>> correctly jumps to the beginning of "if".
>>
>> Right.  I think it can be done in backward-sexp since there we know
>> we’re moving back.  I assume that’s what you mean.
>
> The problem is that the last time when the arg is still -1
> is in 'treesit-forward-sexp', and then it calls
> 'treesit-beginning-of-thing' with '(abs arg)'.
>
> The complete call chain:
>
> - treesit-forward-sexp
> - treesit-beginning-of-thing
> - treesit-navigate-thing
> - treesit-thing-at
> - treesit-node-at
> - treesit-parsers-at
> - overlays-at
>
> So not clear how to propagate the flag for decrementing the offset.
> Maybe by let-binding a new dynamically bound variable?

Ok, here is the patch:

[treesit--parser-overlay-offset.patch (text/x-diff, inline)]

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 69cc28b0cec..af33ae06610 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -839,6 +839,11 @@ treesit--clip-ranges
            if (<= start (car range) (cdr range) end)
            collect range))
 
+(defvar treesit--parser-overlay-offset 0
+  "Defines at which position to get the parser overlay.
+The commands that move backward need to set it to -1 to be
+able to use the range that ends immediately before point.")
+
 (defun treesit-parsers-at (&optional pos language with-host only)
   "Return all parsers at POS.
 
@@ -869,7 +874,8 @@ treesit-parsers-at
   (let ((res nil))
     ;; Refer to (ref:local-parser-overlay) for more explanation of local
     ;; parser overlays.
-    (dolist (ov (overlays-at (or pos (point))))
+    (dolist (ov (overlays-at (+ (or pos (point))
+                                treesit--parser-overlay-offset)))
       (when-let* ((parser (overlay-get ov 'treesit-parser))
                   (host-parser (or (null with-host)
                                    (overlay-get ov 'treesit-host-parser)))
@@ -3021,7 +3027,8 @@ treesit-forward-sexp
           t)
         (if (> arg 0)
             (treesit-end-of-thing pred (abs arg) 'restricted)
-          (treesit-beginning-of-thing pred (abs arg) 'restricted))
+          (let ((treesit--parser-overlay-offset -1))
+            (treesit-beginning-of-thing pred (abs arg) 'restricted)))
         ;; If we couldn't move, we should signal an error and report
         ;; the obstacle, like `forward-sexp' does.  If we couldn't
         ;; find a parent, we simply return nil without moving point,
@@ -3036,6 +3043,7 @@ treesit--forward-list-with-default
 ARG is described in the docstring of `forward-list'."
   (let* ((pred (or treesit-sexp-type-regexp 'list))
          (arg (or arg 1))
+         (treesit--parser-overlay-offset (if (> arg 0) 0 -1))
          (cnt arg)
          (inc (if (> arg 0) 1 -1)))
     (while (/= cnt 0)
@@ -3163,6 +3171,7 @@ treesit-up-list
   (interactive "^p")
   (let* ((pred (or treesit-sexp-type-regexp 'list))
          (arg (or arg 1))
+         (treesit--parser-overlay-offset -1)
          (cnt arg)
          (inc (if (> arg 0) 1 -1)))
     (while (/= cnt 0)

Message #40 received at 77906 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Yuan Fu <casouri <at> gmail.com>
Cc: Vincenzo Pupillo <v.pupillo <at> gmail.com>, 77906 <at> debbugs.gnu.org
Subject: Re: bug#77906: 31.0.50; php-js-mode and js-ts-mode:
 treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
 definition of the predicate in `treesit-thing-settings'" list)
Date: Mon, 12 May 2025 21:23:29 +0300

>>>> With this patch the following case works correctly in test.heex:
>>>> 
>>>>  <%= if true do %>
>>>>    <p>hello, tree-sitter!</p>
>>>>  <% end| %>
>>>> 
>>>> With point at "|" the command 'backward-sexp' (C-M-b)
>>>> correctly jumps to the beginning of "if".
>>>
>>> Right.  I think it can be done in backward-sexp since there we know
>>> we’re moving back.  I assume that’s what you mean.
>>
>> The problem is that the last time when the arg is still -1
>> is in 'treesit-forward-sexp', and then it calls
>> 'treesit-beginning-of-thing' with '(abs arg)'.
>>
>> The complete call chain:
>>
>> - treesit-forward-sexp
>> - treesit-beginning-of-thing
>> - treesit-navigate-thing
>> - treesit-thing-at
>> - treesit-node-at
>> - treesit-parsers-at
>> - overlays-at
>>
>> So not clear how to propagate the flag for decrementing the offset.
>> Maybe by let-binding a new dynamically bound variable?
>
> Ok, here is the patch:

Now pushed to master.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.