Subject: Re: bug#76791: 31.0.50; forward-list doesn't work in bash-ts-mode

From: Juri Linkov <[email-masked]>
Date: Sat, 15 Mar 2025 21:15:52 +0200

> check_dirs_file() {
>     if [ "$(wc -l "$dirs_file")" -gt 10 ]; then
> tmp_dirs_file="$(mktemp)"
> sed '1d' "$dirs_file" > "$tmp_dirs_file"
> cp "$tmp_dirs_file" "$dirs_file"
>     fi
> }
>
> 3. M-x bash-ts-mode
> 4. Move the cursor to the opening bracket of '$(mktemp)' and execute
> 'forward-list'. It gives "No next group" message.

The problem is that many tree-sitter grammars are so imperfect that even
the current heuristics in 'treesit-forward-list' can't help to handle
many cases.  Here are a few examples:

1. bash-ts-mode

  $(a)
=>
  (command_substitution "$("
   (command
    name: (command_name (word)))
   ")")

Here the open paren is inside the node "$(".

2. ruby-ts-mode

  "#{a}"
=>
  (string " (interpolation "#{" (identifier) "}") ")

Here the open curly brace is inside the node "#{".

3. elixir-ts-mode

  &(a)
=>
  (unary_operator operator: "&" operand: "(" (identifier) ")")

Here the open paren is the second node "(" after "&".

4. go-ts-mode

  switch a { }
=>
  (expression_switch_statement "switch" value: (identifier) "{" "}")

Here the open curly brace is far from the sibling "switch" node.

(A similar case in 'for_statement' is already handled
surprisingly well by the current heuristics in 'treesit-forward-list'.)

I see only 2 variants how to allow 'forward-list' to handle all cases:

1. Improve the current heuristics to decide when to fall back
   to the default syntax-based navigation;

2. Explicitly specify the positions that should fall back
   to the default syntax-based navigation.

I can't find a better heuristics, so probably we need to have
another thing 'sexp-list' that specifies when to use
'forward-sexp-default-function' and 'forward-list-default-function'.

This patch solves all the above problems:

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 46332cb1e4b..060686e8dde 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -3015,6 +3015,12 @@ treesit--forward-list-with-default
         ;; Use the default function only if it doesn't go
         ;; over the sibling and doesn't go out of the current group.
         (or (when (and default-pos
+                       (treesit-node-match-p
+                        (treesit-node-at (if (> arg 0) (point)
+                                           (max (1- (point)) (point-min))))
+                        'sexp-list t))
+              (goto-char default-pos))
+            (when (and default-pos
                        (or (null sibling)
                            (if (> arg 0)
                                (<= default-pos (treesit-node-start sibling))
diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index c4b7d0837a4..0c493e3cd77 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -1662,6 +1662,11 @@ bash-ts-mode
                                  "command_substitution"
                                  "process_substitution")
                          eos))
+                   (sexp-list
+                    ("$(" .
+                     ,(lambda (node)
+                        (equal (treesit-node-type (treesit-node-parent node))
+                               "command_substitution"))))
                    (sentence
                     ,(rx bos (or "redirected_statement"
                                  "declaration_command"
diff --git a/lisp/progmodes/ruby-ts-mode.el b/lisp/progmodes/ruby-ts-mode.el
index 1594f301641..84a3842d9ab 100644
--- a/lisp/progmodes/ruby-ts-mode.el
+++ b/lisp/progmodes/ruby-ts-mode.el
@@ -1253,6 +1253,12 @@ ruby-ts-mode
                                 "hash")
                                eos)
                               #'ruby-ts--list-p))
+                 (sexp-list
+                  ("#{" . ,(lambda (node)
+                             ;; for C-M-f in "abc #{ghi} def"
+                             (and (eq (char-after (point)) ?{)
+                                  (equal (treesit-node-type (treesit-node-parent node))
+                                         "interpolation")))))
                  (sentence ,(rx bos (or "return"
                                         "body_statement"
                                         "call"
@@ -1260,19 +1273,8 @@ ruby-ts-mode
                                 eos))
                  (text ,(lambda (node)
                           (or (member (treesit-node-type node)
-                                      '("comment" "string_content" "heredoc_content"))
-                              ;; for C-M-f in hash[:key] and hash['key']
-                              (and (member (treesit-node-text node)
-                                           '("[" "]"))
-                                   (equal (treesit-node-type
-                                           (treesit-node-parent node))
-                                          "element_reference"))
-                              ;; for C-M-f in "abc #{ghi} def"
-                              (and (member (treesit-node-text node)
-                                           '("#{" "}"))
-                                   (equal (treesit-node-type
-                                           (treesit-node-parent node))
-                                          "interpolation"))))))))
+                                      '("comment" "string_content"
+                                        "heredoc_content"))))))))
 
   ;; Imenu.
   (setq-local imenu-create-index-function #'ruby-ts--imenu)
diff --git a/lisp/progmodes/elixir-ts-mode.el b/lisp/progmodes/elixir-ts-mode.el
index d50692d87c0..3d50e85f41b 100644
--- a/lisp/progmodes/elixir-ts-mode.el
+++ b/lisp/progmodes/elixir-ts-mode.el
@@ -722,6 +712,13 @@ elixir-ts-mode
     (setq-local treesit-simple-indent-rules elixir-ts--indent-rules)
 
     ;; Navigation.
+    (setq-local treesit-thing-settings
+                `((elixir
+                   (sexp-list
+                    ("(" . ,(lambda (node)
+                              ;; for C-M-f in "&(&1)"
+                              (equal (treesit-node-type (treesit-node-parent node))
+                                     "unary_operator"))))
     (setq-local treesit-defun-type-regexp
                 '("call" . elixir-ts--defun-p))
 
diff --git a/lisp/progmodes/go-ts-mode.el b/lisp/progmodes/go-ts-mode.el
index d117fe21590..1932ee69606 100644
--- a/lisp/progmodes/go-ts-mode.el
+++ b/lisp/progmodes/go-ts-mode.el
@@ -306,6 +306,10 @@ go-ts-mode
                                  "argument_list"
                                  "literal_value")
                          eos))
+                   (sexp-list
+                    (lambda (node)
+                      (equal (treesit-node-type (treesit-node-parent node))
+                             "expression_switch_statement")))
                    (sentence
                     (or "declaration" "statement")))))