GNU bug report logs - #72525
31.0.50; Forward sexp inconsistency issue c++-ts-mode

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 72525 in the body.
You can then email your comments to 72525 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 31.0.50; Forward sexp inconsistency issue c++-ts-mode
Date: Thu, 08 Aug 2024 16:45:42 +0200

Hi:

Using this code:

```
int main()
{
  abort(); /* 1 */
  abort(); /* 1 */
}
```

There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.

When there is a comment at the end of the line, if I do `mark-sexp`
(C-M-SPC) consecutively I get this selected regions:

-----------------------------
1.
abort();

2.
abort(); /* 1 */

3.
abort(); /* 1 */
abort

4.
abort(); /* 1 */
abort()

5.
abort(); /* 1 */
abort();

6.
abort(); /* 1 */
abort(); /* 1 */

-------------------------------



But when there is NOT trailing comment

```
int main()
{
  abort();
  abort();
}
```

-------------------------------
1.
abort();

2.
abort();
abort();
-------------------------------


It looks like in the fist example after 3 the sexp definition is more fine
grained (similar to the previous c++-mode behavior) and it selects
separately:
 the function name,
 the arguments
 the semicolon
 the comment

But if there is no comment at the end, it always considers the complete
line as a sexp (including the ;).

For my use case I would prefer the old behavior because it is consistent
with the current sexp definition in all emacs (with maybe the exception
of python-mode).  Because it is easier to copy function names or
function calls with a few movements.

However, if it is too difficult to reproduce the old behavior; then the
new one may be implemented consistently.

Best,
Ergus



In GNU Emacs 31.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version
 3.24.43, cairo version 1.18.0) of 2024-08-07 built on RTX
Repository revision: dd93c32c82c65f7c7b26e8c7a8e4fa8425cdbfe0
Repository branch: project
System Description: Arch Linux

Configured using:
 'configure --prefix=/home/ergo/.local/ --with-mailutils --with-pgtk
 --with-modules --with-cairo --with-harfbuzz
 --with-native-compilation=aot
 '--program-transform-name=s/^ctags$/ctags.emacs/''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: C++//

Minor modes in effect:
  fancy-compilation-mode: t
  global-auto-revert-mode: t
  electric-pair-mode: t
  whitespace-mode: t
  flyspell-mode: t
  completion-preview-mode: t
  diff-hl-margin-local-mode: t
  diff-hl-margin-mode: t
  diff-hl-mode: t
  corfu-terminal-mode: t
  global-corfu-mode: t
  corfu-mode: t
  project-multi-mode: t
  gtags-mode: t
  repeat-mode: t
  xterm-mouse-mode: t
  xclip-mode: t
  override-global-mode: t
  winner-mode: t
  save-place-mode: t
  delete-selection-mode: t
  savehist-mode: t
  global-display-fill-column-indicator-mode: t
  display-fill-column-indicator-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  which-key-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/mnt/casa/gits/emacs_clones/cuda-mode/cuda-mode hides /home/ergo/.config/emacs/elpa/cuda-mode-20240716.1936/cuda-mode
/mnt/casa/gits/emacs_clones/gtags-mode/gtags-mode hides /home/ergo/.config/emacs/elpa/gtags-mode-1.8/gtags-mode
/home/ergo/.config/emacs/elpa/transient-20240805.1231/transient hides /home/ergo/.local/share/emacs/31.0.50/lisp/transient

Features:
(shadow sort mail-extr fancy-compilation compile comint ansi-osc
ansi-color comp-run comp-common emacsbug message mailcap yank-media puny
rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util
text-property-search time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils autorevert filenotify ffap
thingatpt url-parse auth-source eieio eieio-core icons password-cache
json map url-vars elec-pair whitespace flyspell-correct flyspell ispell
completion-preview diff-hl-margin diff-hl-dired citre-lang-fileref
citre-tags citre-ctags citre-readtags citre-readtags-tables
citre-backend-interface citre-common-tag rx citre-common-util dired-x
dired dired-loaddefs diff-hl log-view pcvs-util vc-dir ewoc vc
vc-dispatcher diff-mode track-changes corfu-terminal popon corfu
project-multi-mode gtags-mode cl-macs files-x xref project c++-ts-mode
c-ts-mode c-ts-common term/tmux term/xterm xterm init repeat
markdown-ts-mode subr-x treesit cape compat use-package-ensure
use-package-diminish xt-mouse xclip edmacro kmacro byte-opt gv
use-package-bind-key bind-key cl-extra help-mode simple-16-theme winner
ring saveplace delsel savehist easy-mmode display-fill-column-indicator
display-line-numbers diminish which-key cl-seq use-package-core
cl-loaddefs cl-lib bytecomp byte-compile disp-table info
0blayout-autoloads ac-emoji-autoloads ac-haskell-process-autoloads
ac-html-autoloads arduino-cli-mode-autoloads auctex-autoloads tex-site
auto-complete-autoloads avy-zap-autoloads avy-autoloads
better-shell-autoloads caml-autoloads cape-autoloads citre-autoloads
clang-format-autoloads cobol-mode-autoloads compile-multi-autoloads
corfu-terminal-autoloads corfu-autoloads crdt-autoloads
csv-mode-autoloads cuda-mode-autoloads d-mode-autoloads
deadgrep-autoloads debbugs-autoloads diff-hl-autoloads
diminish-autoloads dired-sidebar-autoloads dired-subtree-autoloads
dired-hacks-utils-autoloads dumb-jump-autoloads e2ansi-autoloads
emamux-autoloads esup-autoloads evil-collection-autoloads
annalist-autoloads evil-leader-autoloads evil-autoloads
face-explorer-autoloads fancy-compilation-autoloads flx-autoloads
flycheck-julia-autoloads flycheck-rust-autoloads flycheck-autoloads
flymake-nasm-autoloads flymake-quickdef-autoloads
flyspell-correct-autoloads git-modes-autoloads git-timemachine-autoloads
gnuplot-autoloads google-c-style-autoloads goto-chg-autoloads
groovy-mode-autoloads gtags-mode-autoloads haskell-mode-autoloads
highlight-indent-guides-autoloads i3wm-config-mode-autoloads
ibuffer-sidebar-autoloads iedit-autoloads imenu-list-autoloads
julia-ts-mode-autoloads julia-mode-autoloads languagetool-autoloads
lice-autoloads lorem-ipsum-autoloads lua-mode-autoloads magit-autoloads
git-commit-autoloads magit-section-autoloads markdown-mode-autoloads
markdown-ts-mode-autoloads modern-cpp-font-lock-autoloads
move-dup-autoloads multiple-cursors-autoloads mutt-mode-autoloads
nasm-mode-autoloads neotree-autoloads nftables-mode-autoloads
nginx-mode-autoloads notmuch-autoloads objed-autoloads
opencl-mode-autoloads paradox-autoloads phi-search-autoloads
pkg-info-autoloads epl-autoloads pkgbuild-mode-autoloads
platformio-mode-autoloads async-autoloads popon-autoloads
popup-autoloads projectile-autoloads projection-autoloads
protobuf-mode-autoloads protobuf-ts-mode-autoloads
ptemplate-templates-autoloads ptemplate-autoloads scopeline-autoloads
shell-command+-autoloads slime-autoloads macrostep-autoloads
sphinx-mode-autoloads f-autoloads dash-autoloads s-autoloads
spinner-autoloads ssh-config-mode-autoloads string-inflection-autoloads
sudo-edit-autoloads systemd-autoloads tmux-mode-autoloads
transient-autoloads tsc-autoloads urgrep-autoloads vdiff-autoloads
hydra-autoloads lv-autoloads vterm-toggle-autoloads vterm-autoloads
vundo-autoloads with-editor-autoloads xclip-autoloads
yasnippet-snippets-autoloads yasnippet-autoloads early-init rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win
term/common-win touch-screen pgtk-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty move-toolbar make-network-process native-compile emacs)

Memory information:
((conses 16 185298 34784) (symbols 48 15017 0)
 (strings 32 50296 12386) (string-bytes 1 1766229) (vectors 16 19239)
 (vector-slots 8 220612 9451) (floats 8 95 112) (intervals 56 967 0)
 (buffers 992 13))

Message #8 received at 72525 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ergus <spacibba <at> aol.com>, Yuan Fu <casouri <at> gmail.com>
Cc: 72525 <at> debbugs.gnu.org
Subject: Re: bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode
Date: Sat, 10 Aug 2024 10:56:48 +0300

> Date: Thu, 08 Aug 2024 16:45:42 +0200
> From:  Ergus via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> 
> Hi:
> 
> Using this code:
> 
> ```
> int main()
> {
>   abort(); /* 1 */
>   abort(); /* 1 */
> }
> ```
> 
> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
> 
> When there is a comment at the end of the line, if I do `mark-sexp`
> (C-M-SPC) consecutively I get this selected regions:
> 
> -----------------------------
> 1.
> abort();
> 
> 2.
> abort(); /* 1 */
> 
> 3.
> abort(); /* 1 */
> abort
> 
> 4.
> abort(); /* 1 */
> abort()
> 
> 5.
> abort(); /* 1 */
> abort();
> 
> 6.
> abort(); /* 1 */
> abort(); /* 1 */
> 
> -------------------------------
> 
> 
> 
> But when there is NOT trailing comment
> 
> ```
> int main()
> {
>   abort();
>   abort();
> }
> ```
> 
> -------------------------------
> 1.
> abort();
> 
> 2.
> abort();
> abort();
> -------------------------------
> 
> 
> It looks like in the fist example after 3 the sexp definition is more fine
> grained (similar to the previous c++-mode behavior) and it selects
> separately:
>  the function name,
>  the arguments
>  the semicolon
>  the comment
> 
> But if there is no comment at the end, it always considers the complete
> line as a sexp (including the ;).
> 
> For my use case I would prefer the old behavior because it is consistent
> with the current sexp definition in all emacs (with maybe the exception
> of python-mode).  Because it is easier to copy function names or
> function calls with a few movements.
> 
> However, if it is too difficult to reproduce the old behavior; then the
> new one may be implemented consistently.

Yuan, any comments or suggestions?

FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
source code is not well-defined.

Message #11 received at 72525 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Ergus <spacibba <at> aol.com>, 72525 <at> debbugs.gnu.org
Subject: Re: bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode
Date: Sat, 10 Aug 2024 22:10:01 -0700

> On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> Date: Thu, 08 Aug 2024 16:45:42 +0200
>> From:  Ergus via "Bug reports for GNU Emacs,
>> the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>> 
>> 
>> Hi:
>> 
>> Using this code:
>> 
>> ```
>> int main()
>> {
>>  abort(); /* 1 */
>>  abort(); /* 1 */
>> }
>> ```
>> 
>> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
>> 
>> When there is a comment at the end of the line, if I do `mark-sexp`
>> (C-M-SPC) consecutively I get this selected regions:
>> 
>> -----------------------------
>> 1.
>> abort();
>> 
>> 2.
>> abort(); /* 1 */
>> 
>> 3.
>> abort(); /* 1 */
>> abort
>> 
>> 4.
>> abort(); /* 1 */
>> abort()
>> 
>> 5.
>> abort(); /* 1 */
>> abort();
>> 
>> 6.
>> abort(); /* 1 */
>> abort(); /* 1 */
>> 
>> -------------------------------
>> 
>> 
>> 
>> But when there is NOT trailing comment
>> 
>> ```
>> int main()
>> {
>>  abort();
>>  abort();
>> }
>> ```
>> 
>> -------------------------------
>> 1.
>> abort();
>> 
>> 2.
>> abort();
>> abort();
>> -------------------------------
>> 
>> 
>> It looks like in the fist example after 3 the sexp definition is more fine
>> grained (similar to the previous c++-mode behavior) and it selects
>> separately:
>> the function name,
>> the arguments
>> the semicolon
>> the comment
>> 
>> But if there is no comment at the end, it always considers the complete
>> line as a sexp (including the ;).
>> 
>> For my use case I would prefer the old behavior because it is consistent
>> with the current sexp definition in all emacs (with maybe the exception
>> of python-mode).  Because it is easier to copy function names or
>> function calls with a few movements.
>> 
>> However, if it is too difficult to reproduce the old behavior; then the
>> new one may be implemented consistently.
> 
> Yuan, any comments or suggestions?
> 
> FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
> source code is not well-defined.

Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.

Yuan

Message #14 received at 72525 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: spacibba <at> aol.com, 72525 <at> debbugs.gnu.org
Subject: Re: bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode
Date: Sat, 24 Aug 2024 11:28:12 +0300

Ping!  Any progress with this?

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Sat, 10 Aug 2024 22:10:01 -0700
> Cc: Ergus <spacibba <at> aol.com>,
>  72525 <at> debbugs.gnu.org
> 
> 
> 
> > On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> >> Date: Thu, 08 Aug 2024 16:45:42 +0200
> >> From:  Ergus via "Bug reports for GNU Emacs,
> >> the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> >> 
> >> 
> >> Hi:
> >> 
> >> Using this code:
> >> 
> >> ```
> >> int main()
> >> {
> >>  abort(); /* 1 */
> >>  abort(); /* 1 */
> >> }
> >> ```
> >> 
> >> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
> >> 
> >> When there is a comment at the end of the line, if I do `mark-sexp`
> >> (C-M-SPC) consecutively I get this selected regions:
> >> 
> >> -----------------------------
> >> 1.
> >> abort();
> >> 
> >> 2.
> >> abort(); /* 1 */
> >> 
> >> 3.
> >> abort(); /* 1 */
> >> abort
> >> 
> >> 4.
> >> abort(); /* 1 */
> >> abort()
> >> 
> >> 5.
> >> abort(); /* 1 */
> >> abort();
> >> 
> >> 6.
> >> abort(); /* 1 */
> >> abort(); /* 1 */
> >> 
> >> -------------------------------
> >> 
> >> 
> >> 
> >> But when there is NOT trailing comment
> >> 
> >> ```
> >> int main()
> >> {
> >>  abort();
> >>  abort();
> >> }
> >> ```
> >> 
> >> -------------------------------
> >> 1.
> >> abort();
> >> 
> >> 2.
> >> abort();
> >> abort();
> >> -------------------------------
> >> 
> >> 
> >> It looks like in the fist example after 3 the sexp definition is more fine
> >> grained (similar to the previous c++-mode behavior) and it selects
> >> separately:
> >> the function name,
> >> the arguments
> >> the semicolon
> >> the comment
> >> 
> >> But if there is no comment at the end, it always considers the complete
> >> line as a sexp (including the ;).
> >> 
> >> For my use case I would prefer the old behavior because it is consistent
> >> with the current sexp definition in all emacs (with maybe the exception
> >> of python-mode).  Because it is easier to copy function names or
> >> function calls with a few movements.
> >> 
> >> However, if it is too difficult to reproduce the old behavior; then the
> >> new one may be implemented consistently.
> > 
> > Yuan, any comments or suggestions?
> > 
> > FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
> > source code is not well-defined.
> 
> Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.
> 
> Yuan

Message #17 received at 72525 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Ergus <spacibba <at> aol.com>, 72525 <at> debbugs.gnu.org
Subject: Re: bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode
Date: Mon, 26 Aug 2024 19:53:51 -0700

> On Aug 24, 2024, at 1:28 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> Ping!  Any progress with this?
> 
>> From: Yuan Fu <casouri <at> gmail.com>
>> Date: Sat, 10 Aug 2024 22:10:01 -0700
>> Cc: Ergus <spacibba <at> aol.com>,
>> 72525 <at> debbugs.gnu.org
>> 
>> 
>> 
>>> On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>>> 
>>>> Date: Thu, 08 Aug 2024 16:45:42 +0200
>>>> From:  Ergus via "Bug reports for GNU Emacs,
>>>> the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>>>> 
>>>> 
>>>> Hi:
>>>> 
>>>> Using this code:
>>>> 
>>>> ```
>>>> int main()
>>>> {
>>>> abort(); /* 1 */
>>>> abort(); /* 1 */
>>>> }
>>>> ```
>>>> 
>>>> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
>>>> 
>>>> When there is a comment at the end of the line, if I do `mark-sexp`
>>>> (C-M-SPC) consecutively I get this selected regions:
>>>> 
>>>> -----------------------------
>>>> 1.
>>>> abort();
>>>> 
>>>> 2.
>>>> abort(); /* 1 */
>>>> 
>>>> 3.
>>>> abort(); /* 1 */
>>>> abort
>>>> 
>>>> 4.
>>>> abort(); /* 1 */
>>>> abort()
>>>> 
>>>> 5.
>>>> abort(); /* 1 */
>>>> abort();
>>>> 
>>>> 6.
>>>> abort(); /* 1 */
>>>> abort(); /* 1 */
>>>> 
>>>> -------------------------------
>>>> 
>>>> 
>>>> 
>>>> But when there is NOT trailing comment
>>>> 
>>>> ```
>>>> int main()
>>>> {
>>>> abort();
>>>> abort();
>>>> }
>>>> ```
>>>> 
>>>> -------------------------------
>>>> 1.
>>>> abort();
>>>> 
>>>> 2.
>>>> abort();
>>>> abort();
>>>> -------------------------------
>>>> 
>>>> 
>>>> It looks like in the fist example after 3 the sexp definition is more fine
>>>> grained (similar to the previous c++-mode behavior) and it selects
>>>> separately:
>>>> the function name,
>>>> the arguments
>>>> the semicolon
>>>> the comment
>>>> 
>>>> But if there is no comment at the end, it always considers the complete
>>>> line as a sexp (including the ;).
>>>> 
>>>> For my use case I would prefer the old behavior because it is consistent
>>>> with the current sexp definition in all emacs (with maybe the exception
>>>> of python-mode).  Because it is easier to copy function names or
>>>> function calls with a few movements.
>>>> 
>>>> However, if it is too difficult to reproduce the old behavior; then the
>>>> new one may be implemented consistently.
>>> 
>>> Yuan, any comments or suggestions?
>>> 
>>> FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
>>> source code is not well-defined.
>> 
>> Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.
>> 
>> Yuan

I know the reason for the inconsistency now. Treesit-forward-sexp first checks whether point is in a “text” node, ie, comment or string; if so, it uses the default/normal forward-sexp function; if not, it uses the parse tree to go over sexp.

In the first example, because there’s a comment right before point, treesit-forward-sexp thinks it’s in a text node, and used the normal forward-sexp function, which moved point after the next symbol.

In the second example, because there’s no comment anymore, treesit-forward-sexp uses the parse tree to move the point; and since the next node after point is the statement line, it moves point over the whole line. When using the parse tree to move point, treesit-forward-sexp always moves in the same “level” in which that point is. Eg, when point is between two lines, treesit-forward-sexp moves over lines; if point is inside an argument list between two arguments, treesit-forward-sexp moves over each argument.

If you want to just select the identifier or other more fine-grained movement, IMHO it’s probably better to use forward-word.

I fixed the inconsistency so now treesit-forward-sexp in both example moves over the whole line. The fix is pushed to emacs-30.

Yuan

Message #22 received at 72525-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: spacibba <at> aol.com, 72525-done <at> debbugs.gnu.org
Subject: Re: bug#72525: 31.0.50; Forward sexp inconsistency issue c++-ts-mode
Date: Sat, 07 Sep 2024 10:36:42 +0300

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Mon, 26 Aug 2024 19:53:51 -0700
> Cc: Ergus <spacibba <at> aol.com>,
>  72525 <at> debbugs.gnu.org
> 
> 
> 
> > On Aug 24, 2024, at 1:28 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > 
> > Ping!  Any progress with this?
> > 
> >> From: Yuan Fu <casouri <at> gmail.com>
> >> Date: Sat, 10 Aug 2024 22:10:01 -0700
> >> Cc: Ergus <spacibba <at> aol.com>,
> >> 72525 <at> debbugs.gnu.org
> >> 
> >> 
> >> 
> >>> On Aug 10, 2024, at 12:56 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >>> 
> >>>> Date: Thu, 08 Aug 2024 16:45:42 +0200
> >>>> From:  Ergus via "Bug reports for GNU Emacs,
> >>>> the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> >>>> 
> >>>> 
> >>>> Hi:
> >>>> 
> >>>> Using this code:
> >>>> 
> >>>> ```
> >>>> int main()
> >>>> {
> >>>> abort(); /* 1 */
> >>>> abort(); /* 1 */
> >>>> }
> >>>> ```
> >>>> 
> >>>> There is an inconsistency in the c++-ts-mode behavior of `forward-sexp`.
> >>>> 
> >>>> When there is a comment at the end of the line, if I do `mark-sexp`
> >>>> (C-M-SPC) consecutively I get this selected regions:
> >>>> 
> >>>> -----------------------------
> >>>> 1.
> >>>> abort();
> >>>> 
> >>>> 2.
> >>>> abort(); /* 1 */
> >>>> 
> >>>> 3.
> >>>> abort(); /* 1 */
> >>>> abort
> >>>> 
> >>>> 4.
> >>>> abort(); /* 1 */
> >>>> abort()
> >>>> 
> >>>> 5.
> >>>> abort(); /* 1 */
> >>>> abort();
> >>>> 
> >>>> 6.
> >>>> abort(); /* 1 */
> >>>> abort(); /* 1 */
> >>>> 
> >>>> -------------------------------
> >>>> 
> >>>> 
> >>>> 
> >>>> But when there is NOT trailing comment
> >>>> 
> >>>> ```
> >>>> int main()
> >>>> {
> >>>> abort();
> >>>> abort();
> >>>> }
> >>>> ```
> >>>> 
> >>>> -------------------------------
> >>>> 1.
> >>>> abort();
> >>>> 
> >>>> 2.
> >>>> abort();
> >>>> abort();
> >>>> -------------------------------
> >>>> 
> >>>> 
> >>>> It looks like in the fist example after 3 the sexp definition is more fine
> >>>> grained (similar to the previous c++-mode behavior) and it selects
> >>>> separately:
> >>>> the function name,
> >>>> the arguments
> >>>> the semicolon
> >>>> the comment
> >>>> 
> >>>> But if there is no comment at the end, it always considers the complete
> >>>> line as a sexp (including the ;).
> >>>> 
> >>>> For my use case I would prefer the old behavior because it is consistent
> >>>> with the current sexp definition in all emacs (with maybe the exception
> >>>> of python-mode).  Because it is easier to copy function names or
> >>>> function calls with a few movements.
> >>>> 
> >>>> However, if it is too difficult to reproduce the old behavior; then the
> >>>> new one may be implemented consistently.
> >>> 
> >>> Yuan, any comments or suggestions?
> >>> 
> >>> FWIW, I'm not sure this is a bug: what constitutes a "sexp" in C++
> >>> source code is not well-defined.
> >> 
> >> Yeah I’ll look into this. And yeah there were some discussion around how should we define sexp in c++-ts-mode but there wasn’t a concrete conclusion (I don’t think it’s possible to come up with a concrete one anyway.) Still, if it can be made more convenient for common use-cases I’m more than happy to improve it. Just be aware that I’ll be super busy next week (and I still haven’t done the parse string feature) so it might take me a while to get back.
> >> 
> >> Yuan
> 
> I know the reason for the inconsistency now. Treesit-forward-sexp first checks whether point is in a “text” node, ie, comment or string; if so, it uses the default/normal forward-sexp function; if not, it uses the parse tree to go over sexp.
> 
> In the first example, because there’s a comment right before point, treesit-forward-sexp thinks it’s in a text node, and used the normal forward-sexp function, which moved point after the next symbol.
> 
> In the second example, because there’s no comment anymore, treesit-forward-sexp uses the parse tree to move the point; and since the next node after point is the statement line, it moves point over the whole line. When using the parse tree to move point, treesit-forward-sexp always moves in the same “level” in which that point is. Eg, when point is between two lines, treesit-forward-sexp moves over lines; if point is inside an argument list between two arguments, treesit-forward-sexp moves over each argument.
> 
> If you want to just select the identifier or other more fine-grained movement, IMHO it’s probably better to use forward-word.
> 
> I fixed the inconsistency so now treesit-forward-sexp in both example moves over the whole line. The fix is pushed to emacs-30.

Thanks.

No further comments, so I assume the bug is fixed, and I'm therefore
closing it.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.