Subject: bug#73404: 30.0.50; [forward/kill/etc]-sexp commands do not behave as expected in tree-sitter modes

From: Juri Linkov <[email-masked]>
Date: Sun, 12 Jan 2025 09:34:52 +0200

> Let's take a piece of Haskell code like:
>
>     foo x y = do
>         a <- readsomething
>                x (bar y)
>         b <- readsomethingelse
>                x (baz y)
>         return (a, b)
>
> A) If the cursor is right before `a <- ...`, I definitely expect `C-M-f` to
>    move just over the identifier.

Is the cursor at the beginning of the line or after indentation?

It seems this scheme is whitespace-sensitive since moving the cursor
back over the newline to the end of the previous line with `do`
will change whether `C-M-f` should move over the next `bind` node
or just over the next symbol.

> B) If the cursor is right after `do` (i.e. looking-at EOL), I could see
>    `C-M-f` moving either to the end of the `do` block (i.e. the end of the
>    `foo` function, here), or to right after the closing paren of `(bar y)`.

Usually `do` designates the beginning of the node, so I'd expect the cursor
to be before `do` to be able to move to the end of the `do` block.
Unless explicit curly braces are used in which case the cursor
after `do` is expected to move after the closing curly brace.

> C) If the cursor is right after the closing paren of `(bar y)`, then I'd
>    expect `C-M-f` to move to the closing paren of `(baz y)`.

This heuristics looks whitespace-sensitive too: if the cursor was
after the newline at the beginning of the next line then `C-M-f`
would move over the next symbol.

> If we make the block/list syntax more explicit (which Haskell allows),
> the above code is:
>
>     foo x y = do{
>         a <- readsomething
>                x (bar y);
>         b <- readsomethingelse
>                x (baz y);
>         return (a, b)
>     }
>
> In (A) it's clear we're already inside the block.

Explicit syntax removes a lot of ambiguity indeed.

> In (B), we have an ambiguity where the cursor can be considered to be
> looking either at the whole {...} block, or just at the first statement
> within it, but I think it's clearly not looking at just the `a` identifier.

When the cursor is after `{` then it's inside the {...} block.

> In (C) what I propose is the behavior that SMIE uses in such cases.

Is SMIE sensitive to the position of the cursor inside whitespace region
including newlines?