GNU bug report logs - #72388
31.0.50; Use tree-sitter-cuda grammar but with tree-sitter-cpp's font-lock/indentation rules

Previous Next

To reply to this bug, email your comments to 72388 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Bug Report Emacs <bug-gnu-emacs <at> gnu.org>
Subject: 31.0.50; Use tree-sitter-cuda grammar but with tree-sitter-cpp's
 font-lock/indentation rules
Date: Tue, 30 Jul 2024 17:05:48 -0700

Originated from "Implementing cuda-ts-mode” on emacs-devel [1]. Basically we want to create a major mode for tree-sitter-cuda. Tree-sitter-cuda grammar is basically cpp grammar plus some cuda rules. So it makes sense to reuse cpp mode’s font-lock/indentation rules. But those rules directs to the ‘cpp grammar. A simple solution is to allow users to map one language to another; then when ever lisp tries to create or get a parser for lang A, lang B is used.

Yuan

[1] Message-ID: <v2a2js5ssoz7d3hquenaieubgmj263tz3ejbogordd5t3ys4v2 <at> efpjojlxrlv4>

Message #8 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: 72388 <at> debbugs.gnu.org
Cc: Yuan Fu <casouri <at> gmail.com>
Subject: 31.0.50; Use tree-sitter-cuda grammar but with tree-sitter-cpp's
 font-lock/indentation rules
Date: Tue, 13 Aug 2024 01:10:57 +0200

Hi Yuan:

I am just trying to use the new treesit-language-remap-alist and so far
I understand your idea of how to make it work for the basics. But it is
not clear how can I use the new var to add the new syntax entries.

I mean, looking at the grammar:

https://github.com/tree-sitter-grammars/tree-sitter-cuda/blob/master/grammar.js

You can see that it is the inherited C++ syntax, with extra entries in
some fields (like _declaration_modifiers or delete_expression). For this
part the new variable may work perfectly.

However there are a few "new" entries like: kernel_call_expression,
kernel_call_syntax and launch_bounds that will need special handling.

What's your suggestion?

I mean, for the latest what we really need is to extend (partially
redefine) the c-ts-mode--indent-styles, c-ts-mode--keywords and so on.

Is that the intended approach?

Best,
Ergus

Message #11 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Ergus <spacibba <at> aol.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Mon, 19 Aug 2024 20:44:41 -0700

Hey Ergus, sorry for the delay.

> On Aug 12, 2024, at 4:10 PM, Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors <bug-gnu-emacs <at> gnu.org> wrote:
> 
> Hi Yuan:
> 
> I am just trying to use the new treesit-language-remap-alist and so far
> I understand your idea of how to make it work for the basics. But it is
> not clear how can I use the new var to add the new syntax entries.
> 
> I mean, looking at the grammar:
> 
> https://github.com/tree-sitter-grammars/tree-sitter-cuda/blob/master/grammar.js
> 
> You can see that it is the inherited C++ syntax, with extra entries in
> some fields (like _declaration_modifiers or delete_expression). For this
> part the new variable may work perfectly.
> 
> However there are a few "new" entries like: kernel_call_expression,
> kernel_call_syntax and launch_bounds that will need special handling.
> 
> What's your suggestion?
> 
> I mean, for the latest what we really need is to extend (partially
> redefine) the c-ts-mode--indent-styles, c-ts-mode--keywords and so on.
> 
> Is that the intended approach?

Yeah, you can define your own cuda-indent-styles by grabbing c-ts-mode—indent-styles and adding your rules on top of it, something like

(append your-styles
        (c-ts-mode--indent-styles 'c++))

For font-lock, something like

(append (treesit-font-lock-rules
         :language 'cuda
         :faeture 'xxx
         '(queries))
        (c-ts-mode--font-lock-settings 'c++))

The new treesit-language-remap-alist will ensure that even though the config you borrow from c-ts-mode calls for c++, Emacs will actually use cuda grammar in your major mode.

Yuan

Message #14 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Thu, 16 Jan 2025 20:35:39 +0100

Hi Yuan:

It has been a long time since this emails.. But today finally I made my
serious attempt to make a `cuda-ts-mode`.

While the treesit-language-remap-alist helped to reuse an important part
of the code, there were a few issues I needed to manage manually and
maybe they could be improved somehow:

1. The indentation rules still uses the 'cpp prefix, so I had to
manually tune the code:

```
(defun cuda-ts-mode--simple-indent-rules ()
  (let ((cpp-rules (c-ts-mode--simple-indent-rules
		    'cpp c-ts-mode-indent-style)))
    `((cuda . ,(alist-get 'cpp cpp-rules)))))


(setq-local treesit-simple-indent-rules
		(cuda-ts-mode--simple-indent-rules))
```

In order to manage it properly

2. I see that font-lock is working, but the fontlock rules check is
using the cpp parser.

That's because the c-ts-mode--font-lock-settings still receives cpp as input.
I am trying to find a way to get it from c++ and then change the
:language cpp
with
:language cuda
because if I call
(c-ts-mode--font-lock-settings 'cuda) I obviously get an error.

Is there some way to go around these in a cleaner way?

Thanks in advance and thanks,
Ergus


On Mon, Aug 19, 2024 at 08:44:41PM -0700, Yuan Fu wrote:
>
>Hey Ergus, sorry for the delay.
>
>> On Aug 12, 2024, at 4:10 PM, Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors <bug-gnu-emacs <at> gnu.org> wrote:
>>
>> Hi Yuan:
>>
>> I am just trying to use the new treesit-language-remap-alist and so far
>> I understand your idea of how to make it work for the basics. But it is
>> not clear how can I use the new var to add the new syntax entries.
>>
>> I mean, looking at the grammar:
>>
>> https://github.com/tree-sitter-grammars/tree-sitter-cuda/blob/master/grammar.js
>>
>> You can see that it is the inherited C++ syntax, with extra entries in
>> some fields (like _declaration_modifiers or delete_expression). For this
>> part the new variable may work perfectly.
>>
>> However there are a few "new" entries like: kernel_call_expression,
>> kernel_call_syntax and launch_bounds that will need special handling.
>>
>> What's your suggestion?
>>
>> I mean, for the latest what we really need is to extend (partially
>> redefine) the c-ts-mode--indent-styles, c-ts-mode--keywords and so on.
>>
>> Is that the intended approach?
>
>Yeah, you can define your own cuda-indent-styles by grabbing c-ts-mode—indent-styles and adding your rules on top of it, something like
>
>(append your-styles
>        (c-ts-mode--indent-styles 'c++))
>
>For font-lock, something like
>
>(append (treesit-font-lock-rules
>         :language 'cuda
>         :faeture 'xxx
>         '(queries))
>        (c-ts-mode--font-lock-settings 'c++))
>
>The new treesit-language-remap-alist will ensure that even though the config you borrow from c-ts-mode calls for c++, Emacs will actually use cuda grammar in your major mode.
>
>Yuan

Message #17 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Fri, 17 Jan 2025 03:21:44 +0100

At the moment I have a fully functional version on github:

https://github.com/Ergus/cuda-ts-mode

I don't really like it because the font-lock rules are a copy-paste from
c-ts-mode--font-lock-settings, but that function makes so many checks
like (eq mode 'cpp) that I cannot avoid.
Also the c-ts-mode--test-virtual-named-p hardcodes 'cpp internally.

Apart from that everything is more or less working I think. But strong
modification suggestions are very welcome ;)

Best,
Ergus



On Thu, Jan 16, 2025 at 08:35:43PM +0100, Ergus wrote:
>Hi Yuan:
>
>It has been a long time since this emails.. But today finally I made my
>serious attempt to make a `cuda-ts-mode`.
>
>While the treesit-language-remap-alist helped to reuse an important part
>of the code, there were a few issues I needed to manage manually and
>maybe they could be improved somehow:
>
>1. The indentation rules still uses the 'cpp prefix, so I had to
>manually tune the code:
>
>```
>(defun cuda-ts-mode--simple-indent-rules ()
>  (let ((cpp-rules (c-ts-mode--simple-indent-rules
>		    'cpp c-ts-mode-indent-style)))
>    `((cuda . ,(alist-get 'cpp cpp-rules)))))
>
>
>(setq-local treesit-simple-indent-rules
>		(cuda-ts-mode--simple-indent-rules))
>```
>
>In order to manage it properly
>
>2. I see that font-lock is working, but the fontlock rules check is
>using the cpp parser.
>
>That's because the c-ts-mode--font-lock-settings still receives cpp as input.
>I am trying to find a way to get it from c++ and then change the
>:language cpp
>with
>:language cuda
>because if I call
>(c-ts-mode--font-lock-settings 'cuda) I obviously get an error.
>
>Is there some way to go around these in a cleaner way?
>
>Thanks in advance and thanks,
>Ergus
>
>
>On Mon, Aug 19, 2024 at 08:44:41PM -0700, Yuan Fu wrote:
>>
>>Hey Ergus, sorry for the delay.
>>
>>>On Aug 12, 2024, at 4:10 PM, Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors <bug-gnu-emacs <at> gnu.org> wrote:
>>>
>>>Hi Yuan:
>>>
>>>I am just trying to use the new treesit-language-remap-alist and so far
>>>I understand your idea of how to make it work for the basics. But it is
>>>not clear how can I use the new var to add the new syntax entries.
>>>
>>>I mean, looking at the grammar:
>>>
>>>https://github.com/tree-sitter-grammars/tree-sitter-cuda/blob/master/grammar.js
>>>
>>>You can see that it is the inherited C++ syntax, with extra entries in
>>>some fields (like _declaration_modifiers or delete_expression). For this
>>>part the new variable may work perfectly.
>>>
>>>However there are a few "new" entries like: kernel_call_expression,
>>>kernel_call_syntax and launch_bounds that will need special handling.
>>>
>>>What's your suggestion?
>>>
>>>I mean, for the latest what we really need is to extend (partially
>>>redefine) the c-ts-mode--indent-styles, c-ts-mode--keywords and so on.
>>>
>>>Is that the intended approach?
>>
>>Yeah, you can define your own cuda-indent-styles by grabbing c-ts-mode—indent-styles and adding your rules on top of it, something like
>>
>>(append your-styles
>>       (c-ts-mode--indent-styles 'c++))
>>
>>For font-lock, something like
>>
>>(append (treesit-font-lock-rules
>>        :language 'cuda
>>        :faeture 'xxx
>>        '(queries))
>>       (c-ts-mode--font-lock-settings 'c++))
>>
>>The new treesit-language-remap-alist will ensure that even though the config you borrow from c-ts-mode calls for c++, Emacs will actually use cuda grammar in your major mode.
>>
>>Yuan

Message #20 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Ergus <spacibba <at> aol.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Fri, 17 Jan 2025 17:04:23 -0800

> On Jan 16, 2025, at 11:35 AM, Ergus <spacibba <at> aol.com> wrote:
> 
> Hi Yuan:
> 
> It has been a long time since this emails.. But today finally I made my
> serious attempt to make a `cuda-ts-mode`.
> 
> While the treesit-language-remap-alist helped to reuse an important part
> of the code, there were a few issues I needed to manage manually and
> maybe they could be improved somehow:
> 
> 1. The indentation rules still uses the 'cpp prefix, so I had to
> manually tune the code:
> 
> ```
> (defun cuda-ts-mode--simple-indent-rules ()
>  (let ((cpp-rules (c-ts-mode--simple-indent-rules
>    'cpp c-ts-mode-indent-style)))
>    `((cuda . ,(alist-get 'cpp cpp-rules)))))
> 
> 
> (setq-local treesit-simple-indent-rules
> (cuda-ts-mode--simple-indent-rules))
> ```
> 
> In order to manage it properly
> 
> 2. I see that font-lock is working, but the fontlock rules check is
> using the cpp parser.
> 
> That's because the c-ts-mode--font-lock-settings still receives cpp as input.
> I am trying to find a way to get it from c++ and then change the
> :language cpp
> with
> :language cuda
> because if I call
> (c-ts-mode--font-lock-settings 'cuda) I obviously get an error.
> 
> Is there some way to go around these in a cleaner way?

Thanks Ergus, once you map a language to another, it should be transparent, and you shouldn’t need to change cpp to cuda. It’s most likely a bug somewhere. I’ll fix it.

Yuan

Message #23 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Ergus <spacibba <at> aol.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Fri, 17 Jan 2025 17:29:00 -0800

> On Jan 17, 2025, at 5:04 PM, Yuan Fu <casouri <at> gmail.com> wrote:
> 
> 
> 
>> On Jan 16, 2025, at 11:35 AM, Ergus <spacibba <at> aol.com> wrote:
>> 
>> Hi Yuan:
>> 
>> It has been a long time since this emails.. But today finally I made my
>> serious attempt to make a `cuda-ts-mode`.
>> 
>> While the treesit-language-remap-alist helped to reuse an important part
>> of the code, there were a few issues I needed to manage manually and
>> maybe they could be improved somehow:
>> 
>> 1. The indentation rules still uses the 'cpp prefix, so I had to
>> manually tune the code:
>> 
>> ```
>> (defun cuda-ts-mode--simple-indent-rules ()
>> (let ((cpp-rules (c-ts-mode--simple-indent-rules
>>   'cpp c-ts-mode-indent-style)))
>>   `((cuda . ,(alist-get 'cpp cpp-rules)))))
>> 
>> 
>> (setq-local treesit-simple-indent-rules
>> (cuda-ts-mode--simple-indent-rules))
>> ```
>> 
>> In order to manage it properly
>> 
>> 2. I see that font-lock is working, but the fontlock rules check is
>> using the cpp parser.
>> 
>> That's because the c-ts-mode--font-lock-settings still receives cpp as input.
>> I am trying to find a way to get it from c++ and then change the
>> :language cpp
>> with
>> :language cuda
>> because if I call
>> (c-ts-mode--font-lock-settings 'cuda) I obviously get an error.
>> 
>> Is there some way to go around these in a cleaner way?
> 
> Thanks Ergus, once you map a language to another, it should be transparent, and you shouldn’t need to change cpp to cuda. It’s most likely a bug somewhere. I’ll fix it.
> 
> Yuan

Ok, now using c-ts-mode’s font-lock and indentation rules verbatim should work. BTW, you should have

(setq-local treesit-language-remap-alist '((cpp . cuda)))

in cuda-ts-mode’s body.

Yuan

Message #26 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Sat, 18 Jan 2025 13:42:47 +0100

On Fri, Jan 17, 2025 at 05:29:00PM -0800, Yuan Fu wrote:
>
>
>> On Jan 17, 2025, at 5:04 PM, Yuan Fu <casouri <at> gmail.com> wrote:
>>
>>
>>
>>> On Jan 16, 2025, at 11:35 AM, Ergus <spacibba <at> aol.com> wrote:
>>>
>>> Hi Yuan:
>>>
>>> It has been a long time since this emails.. But today finally I made my
>>> serious attempt to make a `cuda-ts-mode`.
>>>
>>> While the treesit-language-remap-alist helped to reuse an important part
>>> of the code, there were a few issues I needed to manage manually and
>>> maybe they could be improved somehow:
>>>
>>> 1. The indentation rules still uses the 'cpp prefix, so I had to
>>> manually tune the code:
>>>
>>> ```
>>> (defun cuda-ts-mode--simple-indent-rules ()
>>> (let ((cpp-rules (c-ts-mode--simple-indent-rules
>>>   'cpp c-ts-mode-indent-style)))
>>>   `((cuda . ,(alist-get 'cpp cpp-rules)))))
>>>
>>>
>>> (setq-local treesit-simple-indent-rules
>>> (cuda-ts-mode--simple-indent-rules))
>>> ```
>>>
>>> In order to manage it properly
>>>
>>> 2. I see that font-lock is working, but the fontlock rules check is
>>> using the cpp parser.
>>>
>>> That's because the c-ts-mode--font-lock-settings still receives cpp as input.
>>> I am trying to find a way to get it from c++ and then change the
>>> :language cpp
>>> with
>>> :language cuda
>>> because if I call
>>> (c-ts-mode--font-lock-settings 'cuda) I obviously get an error.
>>>
>>> Is there some way to go around these in a cleaner way?
>>
>> Thanks Ergus, once you map a language to another, it should be transparent, and you shouldn’t need to change cpp to cuda. It’s most likely a bug somewhere. I’ll fix it.
>>
>> Yuan
>
>Ok, now using c-ts-mode’s font-lock and indentation rules verbatim should work. BTW, you should have
>
>(setq-local treesit-language-remap-alist '((cpp . cuda)))
>
>in cuda-ts-mode’s body.
>
>Yuan
>

Hi Yuan:

Very thanks for your answer and quick fix ;)

However when I try this:

```
(define-derived-mode cuda-ts-mode c-ts-base-mode "Cuda"
  (when (treesit-ready-p 'cuda)
    (setq-local treesit-primary-parser (treesit-parser-create 'cuda))
    (setq-local treesit-language-remap-alist '((cpp . cuda)))))
```

`treesit-simple-indent-rules' and `treesit-font-lock-settings' are nil.

On the other hand:

If I do instead:

```
(define-derived-mode cuda-ts-mode c++-ts-mode "Cuda" ...
```

Then I have fontlock, but it seems taken from C++ grammar and
indentation still doesn't work. It actually creates a 'cpp grammar,
which I think is not desired in this case.

Indentation doesn't work in this case because:

```
(treesit-node-language (treesit-node-at (point)))
```

returns 'cuda, but `treesit-simple-indent-rules' has 'cpp key; and
the `treesit-simple-indent' function has:

```
(language (treesit-node-language parent))
(rules (alist-get language treesit-simple-indent-rules))
```

The code in `c-ts-mode--simple-indent-rules' ends with:

(pcase mode
      ('c `((c . ,rules)))
      ('cpp `((cpp . ,rules))))

So it can only create 'cpp or 'c keys and `treesit-node-language' uses
`treesit-parser-language' (which is correct). So I still need to remap
things manually.

In font-lock I cannot see in such details because the var
`treesit-font-lock-settings' is more opaque, but I suppose that there
may be happening something similar.

Message #29 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Ergus <spacibba <at> aol.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Sun, 19 Jan 2025 22:57:16 -0800

> On Jan 18, 2025, at 4:42 AM, Ergus <spacibba <at> aol.com> wrote:
> 
> On Fri, Jan 17, 2025 at 05:29:00PM -0800, Yuan Fu wrote:
>> 
>> 
>>> On Jan 17, 2025, at 5:04 PM, Yuan Fu <casouri <at> gmail.com> wrote:
>>> 
>>> 
>>> 
>>>> On Jan 16, 2025, at 11:35 AM, Ergus <spacibba <at> aol.com> wrote:
>>>> 
>>>> Hi Yuan:
>>>> 
>>>> It has been a long time since this emails.. But today finally I made my
>>>> serious attempt to make a `cuda-ts-mode`.
>>>> 
>>>> While the treesit-language-remap-alist helped to reuse an important part
>>>> of the code, there were a few issues I needed to manage manually and
>>>> maybe they could be improved somehow:
>>>> 
>>>> 1. The indentation rules still uses the 'cpp prefix, so I had to
>>>> manually tune the code:
>>>> 
>>>> ```
>>>> (defun cuda-ts-mode--simple-indent-rules ()
>>>> (let ((cpp-rules (c-ts-mode--simple-indent-rules
>>>>  'cpp c-ts-mode-indent-style)))
>>>>  `((cuda . ,(alist-get 'cpp cpp-rules)))))
>>>> 
>>>> 
>>>> (setq-local treesit-simple-indent-rules
>>>> (cuda-ts-mode--simple-indent-rules))
>>>> ```
>>>> 
>>>> In order to manage it properly
>>>> 
>>>> 2. I see that font-lock is working, but the fontlock rules check is
>>>> using the cpp parser.
>>>> 
>>>> That's because the c-ts-mode--font-lock-settings still receives cpp as input.
>>>> I am trying to find a way to get it from c++ and then change the
>>>> :language cpp
>>>> with
>>>> :language cuda
>>>> because if I call
>>>> (c-ts-mode--font-lock-settings 'cuda) I obviously get an error.
>>>> 
>>>> Is there some way to go around these in a cleaner way?
>>> 
>>> Thanks Ergus, once you map a language to another, it should be transparent, and you shouldn’t need to change cpp to cuda. It’s most likely a bug somewhere. I’ll fix it.
>>> 
>>> Yuan
>> 
>> Ok, now using c-ts-mode’s font-lock and indentation rules verbatim should work. BTW, you should have
>> 
>> (setq-local treesit-language-remap-alist '((cpp . cuda)))
>> 
>> in cuda-ts-mode’s body.
>> 
>> Yuan
>> 
> 
> Hi Yuan:
> 
> Very thanks for your answer and quick fix ;)
> 
> However when I try this:
> 
> ```
> (define-derived-mode cuda-ts-mode c-ts-base-mode "Cuda"
>  (when (treesit-ready-p 'cuda)
>    (setq-local treesit-primary-parser (treesit-parser-create 'cuda))
>    (setq-local treesit-language-remap-alist '((cpp . cuda)))))
> ```
> 
> `treesit-simple-indent-rules' and `treesit-font-lock-settings' are nil.
> 
> On the other hand:
> 
> If I do instead:
> 
> ```
> (define-derived-mode cuda-ts-mode c++-ts-mode "Cuda" ...
> ```
> 
> Then I have fontlock, but it seems taken from C++ grammar and
> indentation still doesn't work. It actually creates a 'cpp grammar,
> which I think is not desired in this case.
> 
> Indentation doesn't work in this case because:
> 
> ```
> (treesit-node-language (treesit-node-at (point)))
> ```
> 
> returns 'cuda, but `treesit-simple-indent-rules' has 'cpp key; and
> the `treesit-simple-indent' function has:
> 
> ```
> (language (treesit-node-language parent))
> (rules (alist-get language treesit-simple-indent-rules))
> ```
> 
> The code in `c-ts-mode--simple-indent-rules' ends with:
> 
> (pcase mode
>      ('c `((c . ,rules)))
>      ('cpp `((cpp . ,rules))))
> 
> So it can only create 'cpp or 'c keys and `treesit-node-language' uses
> `treesit-parser-language' (which is correct). So I still need to remap
> things manually.
> 
> In font-lock I cannot see in such details because the var
> `treesit-font-lock-settings' is more opaque, but I suppose that there
> may be happening something similar.
> 

Thanks Ergus. I was trying to avoid using the heavy hammer but it seems we have to make it completely transparent. I pushed a change to master. Now when you create a cuda parser using the remapping from cpp, it should completely look and talk like a cpp parser.

Font-lock should work though. It worked for me when I tested it locally.

Yuan

Message #32 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Mon, 20 Jan 2025 18:09:27 +0100

Hi Yuan:

Very thanks for your reply and patches :) I think I am bothering too
much :p

On Sun, Jan 19, 2025 at 10:57:16PM -0800, Yuan Fu wrote:
>
>
>> On Jan 18, 2025, at 4:42 AM, Ergus <spacibba <at> aol.com> wrote:
>>
>> On Fri, Jan 17, 2025 at 05:29:00PM -0800, Yuan Fu wrote:
>>
>> Hi Yuan:
>>
>> Very thanks for your answer and quick fix ;)
>>
>> However when I try this:
>>
>> ```
>> (define-derived-mode cuda-ts-mode c-ts-base-mode "Cuda"
>>  (when (treesit-ready-p 'cuda)
>>    (setq-local treesit-primary-parser (treesit-parser-create 'cuda))
>>    (setq-local treesit-language-remap-alist '((cpp . cuda)))))
>> ```
>>
>> `treesit-simple-indent-rules' and `treesit-font-lock-settings' are nil.
>>
>> On the other hand:
>>
>> If I do instead:
>>
>> ```
>> (define-derived-mode cuda-ts-mode c++-ts-mode "Cuda" ...
>> ```
>>
>> Then I have fontlock, but it seems taken from C++ grammar and
>> indentation still doesn't work. It actually creates a 'cpp grammar,
>> which I think is not desired in this case.
>>
>> Indentation doesn't work in this case because:
>>
>> ```
>> (treesit-node-language (treesit-node-at (point)))
>> ```
>>
>> returns 'cuda, but `treesit-simple-indent-rules' has 'cpp key; and
>> the `treesit-simple-indent' function has:
>>
>> ```
>> (language (treesit-node-language parent))
>> (rules (alist-get language treesit-simple-indent-rules))
>> ```
>>
>> The code in `c-ts-mode--simple-indent-rules' ends with:
>>
>> (pcase mode
>>      ('c `((c . ,rules)))
>>      ('cpp `((cpp . ,rules))))
>>
>> So it can only create 'cpp or 'c keys and `treesit-node-language' uses
>> `treesit-parser-language' (which is correct). So I still need to remap
>> things manually.
>>
>> In font-lock I cannot see in such details because the var
>> `treesit-font-lock-settings' is more opaque, but I suppose that there
>> may be happening something similar.
>>
>
>Thanks Ergus. I was trying to avoid using the heavy hammer but it seems we have to make it completely transparent. I pushed a change to master. Now when you create a cuda parser using the remapping from cpp, it should completely look and talk like a cpp parser.
>
>Font-lock should work though. It worked for me when I tested it locally.
>
>Yuan

I have exactly the same error as before.

This is the code:

```
(define-derived-mode cuda-ts-mode c-ts-base-mode "Cuda"
  "Major mode for editing Cuda, powered by tree-sitter.

This mode is independent from the classic cuda-mode.el, but inherits
most of the properties from c++-ts-mode like `c-ts-mode-indent-style',
`c-ts-mode-indent-offset' or `c-ts-mode-enable-doxygen'."

  (when (treesit-ready-p 'cuda)

    (setq treesit-language-remap-alist '((cpp . cuda)))
    (setq-local treesit-primary-parser (treesit-parser-create 'cuda))

    (treesit-major-mode-setup)))
```

(treesit-node-language (treesit-node-at (point))) => cuda
treesit-simple-indent-rules => nil
treesit-font-lock-settings => nil

What I am doing wrong?

...

Should I derive from c++-ts-mode instead?

In that case, font-lock works but I don't know if it is with the cuda or
cpp parser.

However, indentation doesn't work at all for the same explication in the
last email.

(treesit-node-language (treesit-node-at (point))) returns cuda, but keys
in indent-rules is cpp.

Once again, sorry for bothering so much,
Best,
Ergus

Message #35 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Ergus <spacibba <at> aol.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Mon, 20 Jan 2025 15:29:45 -0800

[Message part 1 (text/plain, inline)]

> On Jan 20, 2025, at 9:09 AM, Ergus <spacibba <at> aol.com> wrote:
> 
> Hi Yuan:
> 
> Very thanks for your reply and patches :) I think I am bothering too
> much :p

No no, sorry, I didn’t explain it clear enough.

> 
> On Sun, Jan 19, 2025 at 10:57:16PM -0800, Yuan Fu wrote:
>> 
>> 
>>> On Jan 18, 2025, at 4:42 AM, Ergus <spacibba <at> aol.com> wrote:
>>> 
>>> On Fri, Jan 17, 2025 at 05:29:00PM -0800, Yuan Fu wrote:
>>> 
>>> Hi Yuan:
>>> 
>>> Very thanks for your answer and quick fix ;)
>>> 
>>> However when I try this:
>>> 
>>> ```
>>> (define-derived-mode cuda-ts-mode c-ts-base-mode "Cuda"
>>> (when (treesit-ready-p 'cuda)
>>>   (setq-local treesit-primary-parser (treesit-parser-create 'cuda))
>>>   (setq-local treesit-language-remap-alist '((cpp . cuda)))))
>>> ```
>>> 
>>> `treesit-simple-indent-rules' and `treesit-font-lock-settings' are nil.
>>> 
>>> On the other hand:
>>> 
>>> If I do instead:
>>> 
>>> ```
>>> (define-derived-mode cuda-ts-mode c++-ts-mode "Cuda" ...
>>> ```
>>> 
>>> Then I have fontlock, but it seems taken from C++ grammar and
>>> indentation still doesn't work. It actually creates a 'cpp grammar,
>>> which I think is not desired in this case.
>>> 
>>> Indentation doesn't work in this case because:
>>> 
>>> ```
>>> (treesit-node-language (treesit-node-at (point)))
>>> ```
>>> 
>>> returns 'cuda, but `treesit-simple-indent-rules' has 'cpp key; and
>>> the `treesit-simple-indent' function has:
>>> 
>>> ```
>>> (language (treesit-node-language parent))
>>> (rules (alist-get language treesit-simple-indent-rules))
>>> ```
>>> 
>>> The code in `c-ts-mode--simple-indent-rules' ends with:
>>> 
>>> (pcase mode
>>>     ('c `((c . ,rules)))
>>>     ('cpp `((cpp . ,rules))))
>>> 
>>> So it can only create 'cpp or 'c keys and `treesit-node-language' uses
>>> `treesit-parser-language' (which is correct). So I still need to remap
>>> things manually.
>>> 
>>> In font-lock I cannot see in such details because the var
>>> `treesit-font-lock-settings' is more opaque, but I suppose that there
>>> may be happening something similar.
>>> 
>> 
>> Thanks Ergus. I was trying to avoid using the heavy hammer but it seems we have to make it completely transparent. I pushed a change to master. Now when you create a cuda parser using the remapping from cpp, it should completely look and talk like a cpp parser.
>> 
>> Font-lock should work though. It worked for me when I tested it locally.
>> 
>> Yuan
> 
> I have exactly the same error as before.
> 
> This is the code:
> 
> ```
> (define-derived-mode cuda-ts-mode c-ts-base-mode "Cuda"
>  "Major mode for editing Cuda, powered by tree-sitter.
> 
> This mode is independent from the classic cuda-mode.el, but inherits
> most of the properties from c++-ts-mode like `c-ts-mode-indent-style',
> `c-ts-mode-indent-offset' or `c-ts-mode-enable-doxygen'."
> 
>  (when (treesit-ready-p 'cuda)
> 
>    (setq treesit-language-remap-alist '((cpp . cuda)))
>    (setq-local treesit-primary-parser (treesit-parser-create 'cuda))

I think if you replace ‘cuda with ‘cpp here, everything should work. This parser will be a cuda parser but labeled as cpp parser. That also means if you want to add coda-specific font-lock rules or indentation, you need to use ‘cpp’ as the language key.

> 
>    (treesit-major-mode-setup)))
> ```
> 
> (treesit-node-language (treesit-node-at (point))) => cuda
> treesit-simple-indent-rules => nil
> treesit-font-lock-settings => nil
> 
> What I am doing wrong?
> 
> ...
> 
> Should I derive from c++-ts-mode instead?

You don’t need to, c-ts-base-mode is fine.

> 
> In that case, font-lock works but I don't know if it is with the cuda or
> cpp parser.
> 
> However, indentation doesn't work at all for the same explication in the
> last email.
> 
> (treesit-node-language (treesit-node-at (point))) returns cuda, but keys
> in indent-rules is cpp.

Please see this patch. With this patch I can get cuda work for both font-lock and indentation.

Yuan

[cuda-ts-mode.patch (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

Message #38 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Wed, 22 Jan 2025 07:23:28 +0100

Hi Yuan:

When I try your patch I get an error:

Debugger entered--Lisp error: (treesit-no-parser cpp)
  (treesit-buffer-root-node cpp)
  (treesit-node-at 3091)
  (treesit-show-paren-data--categorize 3091)
  (treesit-show-paren-data)
  (show-paren-function)
  (apply show-paren-function nil)
  (timer-event-handler [t 0 0 125000 t show-paren-function nil idle 0 nil])

When I try:
(treesit-parser-list nil 'cpp) -> nil

So it it like the

I tried manually (setq-local treesit-primary-parser
(treesit-parser-create 'cpp))

but nothing.

On Mon, Jan 20, 2025 at 03:29:45PM -0800, Yuan Fu wrote:
>
>
>> On Jan 20, 2025, at 9:09 AM, Ergus <spacibba <at> aol.com> wrote:
>>
>> Hi Yuan:
>>
>> Very thanks for your reply and patches :) I think I am bothering too
>> much :p
>
>No no, sorry, I didn’t explain it clear enough.
>
>>
>> On Sun, Jan 19, 2025 at 10:57:16PM -0800, Yuan Fu wrote:
>>>
>>>
>>>> On Jan 18, 2025, at 4:42 AM, Ergus <spacibba <at> aol.com> wrote:
>>>>
>>>> On Fri, Jan 17, 2025 at 05:29:00PM -0800, Yuan Fu wrote:
>>>>
>>>> Hi Yuan:
>>>>
>>>> Very thanks for your answer and quick fix ;)
>>>>
>>>> However when I try this:
>>>>
>>>> ```
>>>> (define-derived-mode cuda-ts-mode c-ts-base-mode "Cuda"
>>>> (when (treesit-ready-p 'cuda)
>>>>   (setq-local treesit-primary-parser (treesit-parser-create 'cuda))
>>>>   (setq-local treesit-language-remap-alist '((cpp . cuda)))))
>>>> ```
>>>>
>>>> `treesit-simple-indent-rules' and `treesit-font-lock-settings' are nil.
>>>>
>>>> On the other hand:
>>>>
>>>> If I do instead:
>>>>
>>>> ```
>>>> (define-derived-mode cuda-ts-mode c++-ts-mode "Cuda" ...
>>>> ```
>>>>
>>>> Then I have fontlock, but it seems taken from C++ grammar and
>>>> indentation still doesn't work. It actually creates a 'cpp grammar,
>>>> which I think is not desired in this case.
>>>>
>>>> Indentation doesn't work in this case because:
>>>>
>>>> ```
>>>> (treesit-node-language (treesit-node-at (point)))
>>>> ```
>>>>
>>>> returns 'cuda, but `treesit-simple-indent-rules' has 'cpp key; and
>>>> the `treesit-simple-indent' function has:
>>>>
>>>> ```
>>>> (language (treesit-node-language parent))
>>>> (rules (alist-get language treesit-simple-indent-rules))
>>>> ```
>>>>
>>>> The code in `c-ts-mode--simple-indent-rules' ends with:
>>>>
>>>> (pcase mode
>>>>     ('c `((c . ,rules)))
>>>>     ('cpp `((cpp . ,rules))))
>>>>
>>>> So it can only create 'cpp or 'c keys and `treesit-node-language' uses
>>>> `treesit-parser-language' (which is correct). So I still need to remap
>>>> things manually.
>>>>
>>>> In font-lock I cannot see in such details because the var
>>>> `treesit-font-lock-settings' is more opaque, but I suppose that there
>>>> may be happening something similar.
>>>>
>>>
>>> Thanks Ergus. I was trying to avoid using the heavy hammer but it seems we have to make it completely transparent. I pushed a change to master. Now when you create a cuda parser using the remapping from cpp, it should completely look and talk like a cpp parser.
>>>
>>> Font-lock should work though. It worked for me when I tested it locally.
>>>
>>> Yuan
>>
>> I have exactly the same error as before.
>>
>> This is the code:
>>
>> ```
>> (define-derived-mode cuda-ts-mode c-ts-base-mode "Cuda"
>>  "Major mode for editing Cuda, powered by tree-sitter.
>>
>> This mode is independent from the classic cuda-mode.el, but inherits
>> most of the properties from c++-ts-mode like `c-ts-mode-indent-style',
>> `c-ts-mode-indent-offset' or `c-ts-mode-enable-doxygen'."
>>
>>  (when (treesit-ready-p 'cuda)
>>
>>    (setq treesit-language-remap-alist '((cpp . cuda)))
>>    (setq-local treesit-primary-parser (treesit-parser-create 'cuda))
>
>I think if you replace ‘cuda with ‘cpp here, everything should work. This parser will be a cuda parser but labeled as cpp parser. That also means if you want to add coda-specific font-lock rules or indentation, you need to use ‘cpp’ as the language key.
>
>>
>>    (treesit-major-mode-setup)))
>> ```
>>
>> (treesit-node-language (treesit-node-at (point))) => cuda
>> treesit-simple-indent-rules => nil
>> treesit-font-lock-settings => nil
>>
>> What I am doing wrong?
>>
>> ...
>>
>> Should I derive from c++-ts-mode instead?
>
>You don’t need to, c-ts-base-mode is fine.
>
>>
>> In that case, font-lock works but I don't know if it is with the cuda or
>> cpp parser.
>>
>> However, indentation doesn't work at all for the same explication in the
>> last email.
>>
>> (treesit-node-language (treesit-node-at (point))) returns cuda, but keys
>> in indent-rules is cpp.
>
>Please see this patch. With this patch I can get cuda work for both font-lock and indentation.
>
>Yuan
>


>
>

Message #41 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Ergus <spacibba <at> aol.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Wed, 29 Jan 2025 23:27:59 -0800

> On Jan 21, 2025, at 10:23 PM, Ergus <spacibba <at> aol.com> wrote:
> 
> Hi Yuan:
> 
> When I try your patch I get an error:
> 
> Debugger entered--Lisp error: (treesit-no-parser cpp)
>  (treesit-buffer-root-node cpp)
>  (treesit-node-at 3091)
>  (treesit-show-paren-data--categorize 3091)
>  (treesit-show-paren-data)
>  (show-paren-function)
>  (apply show-paren-function nil)
>  (timer-event-handler [t 0 0 125000 t show-paren-function nil idle 0 nil])
> 
> When I try:
> (treesit-parser-list nil 'cpp) -> nil
> 
> So it it like the
> 
> I tried manually (setq-local treesit-primary-parser
> (treesit-parser-create 'cpp))
> 
> but nothing.

Apologies, the previous change I made is indeed insufficient. Please try the latest master, the remapping should be truly transparent now.

Yuan

Message #44 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Fri, 31 Jan 2025 01:06:06 +0100

Hi Yuan:

The cuda mode seems to work now. I am still testing and fixing details,
but so far now it works.

However I have detected another issue that I thought was related with
the remap, but it is not because I got it also in normal c++-ts-mode:

When c-ts-mode-enable-doxygen is enabled and the cursor moves inside a
doxygen comment I get this error:

Debugger entered--Lisp error: (treesit-predicate-not-found ("Cannot find the definition of the predicate in `treesit-thing-settings'" list))
  (treesit-parent-until #<treesit-node description in 7380-7452> list)
  (treesit-show-paren-data--categorize 7376)
  (treesit-show-paren-data)
  (show-paren-function)
  (apply show-paren-function nil)
  (timer-event-handler [t 0 0 125000 t show-paren-function nil idle 0 nil])

If you prefer I can open a new issue for this, but maybe it is not needed.

Best,
Ergus

On Wed, Jan 29, 2025 at 11:27:59PM -0800, Yuan Fu wrote:
>
>
>> On Jan 21, 2025, at 10:23 PM, Ergus <spacibba <at> aol.com> wrote:
>>
>> Hi Yuan:
>>
>> When I try your patch I get an error:
>>
>> Debugger entered--Lisp error: (treesit-no-parser cpp)
>>  (treesit-buffer-root-node cpp)
>>  (treesit-node-at 3091)
>>  (treesit-show-paren-data--categorize 3091)
>>  (treesit-show-paren-data)
>>  (show-paren-function)
>>  (apply show-paren-function nil)
>>  (timer-event-handler [t 0 0 125000 t show-paren-function nil idle 0 nil])
>>
>> When I try:
>> (treesit-parser-list nil 'cpp) -> nil
>>
>> So it it like the
>>
>> I tried manually (setq-local treesit-primary-parser
>> (treesit-parser-create 'cpp))
>>
>> but nothing.
>
>Apologies, the previous change I made is indeed insufficient. Please try the latest master, the remapping should be truly transparent now.
>
>Yuan

Message #47 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 72388 <at> debbugs.gnu.org
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Fri, 31 Jan 2025 01:20:32 +0100

Hi Yuan:

Another detail.

I just detected that when I call treesit-inspect-node-at-point  I always
get No node at point

However: (treesit-node-at (point)) provides right information.

On Wed, Jan 29, 2025 at 11:27:59PM -0800, Yuan Fu wrote:
>
>
>> On Jan 21, 2025, at 10:23 PM, Ergus <spacibba <at> aol.com> wrote:
>>
>> Hi Yuan:
>>
>> When I try your patch I get an error:
>>
>> Debugger entered--Lisp error: (treesit-no-parser cpp)
>>  (treesit-buffer-root-node cpp)
>>  (treesit-node-at 3091)
>>  (treesit-show-paren-data--categorize 3091)
>>  (treesit-show-paren-data)
>>  (show-paren-function)
>>  (apply show-paren-function nil)
>>  (timer-event-handler [t 0 0 125000 t show-paren-function nil idle 0 nil])
>>
>> When I try:
>> (treesit-parser-list nil 'cpp) -> nil
>>
>> So it it like the
>>
>> I tried manually (setq-local treesit-primary-parser
>> (treesit-parser-create 'cpp))
>>
>> but nothing.
>
>Apologies, the previous change I made is indeed insufficient. Please try the latest master, the remapping should be truly transparent now.
>
>Yuan

Message #50 received at 72388 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Ergus <spacibba <at> aol.com>
Cc: 72388 <at> debbugs.gnu.org, Yuan Fu <casouri <at> gmail.com>
Subject: Re: bug#72388: 31.0.50; Use tree-sitter-cuda grammar but with
 tree-sitter-cpp's font-lock/indentation rules
Date: Fri, 31 Jan 2025 09:05:35 +0200

> When c-ts-mode-enable-doxygen is enabled and the cursor moves inside a
> doxygen comment I get this error:
>
> Debugger entered--Lisp error: (treesit-predicate-not-found ("Cannot find the definition of the predicate in `treesit-thing-settings'" list))
>   (treesit-parent-until #<treesit-node description in 7380-7452> list)
>   (treesit-show-paren-data--categorize 7376)
>   (treesit-show-paren-data)
>   (show-paren-function)
>   (apply show-paren-function nil)
>   (timer-event-handler [t 0 0 125000 t show-paren-function nil idle 0 nil])
>
> If you prefer I can open a new issue for this, but maybe it is not needed.

This is a separate issue: bug#75198 (closed) and bug#75456 (still open).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.