Subject: Re: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes

From: Stefan Monnier <[email-masked]>
Date: Mon, 06 Jan 2025 09:14:58 -0500

>> However, deciding on what makes sense is hard, which is why I'm
>> suggesting we try to define a "spec". It needn't really be much more
>> than reference suggestions for mode implementors at first, then maybe
>> some intersection of features crystallize across all modes.
>
> I don't think it's possible to find a spec better than Stefan already
> defined as an AST node delimited by an opening and a closing "keyword":
> (...), [...], if...fi, begin...end, ...

IIUC, Theodor was talking more generally than just about "list" operations.
I think "list" operations are among the clearest case (BTW, I wonder if
someone has already tried to implement that functionality for Haskell or
Python, where the opening and a closing "keywords" are basically newlines).

> Or when looking at the commands with the 'forward-' prefix:
>
>   forward-sexp (C-M-f)
>   forward-list (C-M-n)

OK, we already talked about these two.  We could probably start drafting
a kind of guideline.

>   forward-symbol

This one doesn't look too hard either.

>   forward-sentence (M-e)

This one is trickier.  You just provided a reasonably good start with
"In languages with statements a sentence corresponds to a statement",
but there are some non-trivial questions about language without
statements, about use within comments, about how to handle nesting
(e.g. statements within statements) and more generally the use of `M-e`
from outside of statements.

>   forward-paragraph (M-})

This one is also tied to filling, so it's important for it to behave
well w.r.t to comments to allow filling actual text paragraphs
within comments.

As for its behavior outside of comments ...

>   forward-page (C-x ])
>   forward-word (M-f)

I think these should be oblivious to the syntax (i.e. line `forward-line`
and `forward-char`).

Your list missed the "defun" unit of navigation.

One other potentially useful unit of navigation that depends on the
syntax would be "list element", which would do something like skip to
the next/previous separator delimiting elements of the nearest enclosing
paired delimiters.  For {...;...;...} blocks, this would behave like
`forward-sentence`, except that when used from within the args of
a function call like a `foo (..., ..., ...)`, it would jump
between arguments instead of skipping all the way to the next statement.

        Stefan