Subject: bug#78703: beginning-of-defun and friends still wrong in typescript-ts-mode

From: Daniel Colascione <[email-masked]>
Date: Wed, 11 Jun 2025 05:59:28 -0700

Eli Zaretskii <[email-masked]> writes:

>> Date: Tue, 10 Jun 2025 09:55:59 -0700
>> From: Daniel Colascione <[email-masked]>
>> CC: [email-masked], [email-masked], [email-masked]
>> 
>> >Not really, not when you look closely.  The tools we've built before
>> >tree-sitter are ad-hoc, so they allow us to provide information that
>> >parsers don't have and don't need to have.  Our syntax tables are not
>> >exactly "less efficient parsing", and regular expressions allow us to
>> >match whatever we want and call that anything we want.
>> 
>> There is nothing one bit of information available to c-mode not available to c-ts-mode.
>
> Sure, if you want c-ts-mode do everything c-mode did, plus what it
> does based on the parser.  But the idea is to toss as much of the
> ad-hoc "parsing" and "matching" we have, so that we don't need to keep
> updating that forever, as the languages evolve and add/redefine
> features.  That was the main purpose of integrating tree-sitter in
> Emacs, after all.  So from where I stand, going back to all the stuff
> we have in c-mode would mean we wasted a lot of efforts for nothing.
> People who just want the c-mode behavior 1:1 should simply use c-mode,
> it will never be removed from Emacs.
>
>> > We still lack some useful
>> >functionalities that are present in CC Mode, forrea that very reason;
>> >what was easy to implement was done long ago.
>> 
>> That's the opposite of reality. Alan and others have spent years
>> building a flexible and fast backtracking syntactic analyser for CC
>> mode. Tree sitter does the same thing but in a more general way, in
>> native code for better performance. Its availability makes doing
>> what cc-mode does easier, not harder.
>
> We expected that, yes.  The reality disappointed us to some extent.

What specific information is missing from the TS AST?

>> >We've been doing that since day one: you write code that looks at the
>> >variables to figure out what behavior to expect, or you write code
>> >that is general enough to not care.
>> 
>> I can't wait to program against our glorious new dwim function. The
>> problem with let binding the world is that the set of dynamic inputs
>> becomes unbounded. What if I just bind the strategy option and one
>> day TS introduces, say, a new sub-strategy option that makes the
>> function I call behave differently?
>
> If we decide it's a good idea, yes.  But these additions are not a
> force of nature: we decide whether to add or not to add, and we should
> consider the adverse effects each such addition has when we do.  If we
> do a good job, there won't be unbounded set of dynamic inputs to
> consider in each case.
>
>> >> Yet we use the concept of command names to express different
>> >> concepts elsewhere. And if the concept is ambiguously defined,
>> >> provide a minimal knob to adjust that concept, not change the
>> >> operation of primitives to be inconsistent with each other.
>> >
>> >I think we do the former, or at least we try.
>> 
>> Then let's make separate commands to express moving to a defun boundary one way versus another way and let users express their preference for connecting input to action using keymaps.
>
> I'm not saying we should never do that.  I was responding to your much
> more general remarks, not to this particular case.

By default, commands should do one thing.  One can rebut the presumption
that a command should do one simple thing for certain special cases, I
guess, but I don't see a strong argument for for beginning-of-defun
being one of these cases when it's easy to provide commands for the
various supported behaviors.

> IOW, if you argue about this specific issue, and with arguments
> specific to it, it's possible we will eventually agree.  Just let's
> not introduce general arguments like "this should never be done",
> because I don't think I agree, based on our experiences and practices.
> So such arguments will not convince.
>
>> >> No it isn't. If the defun navigation functions in TS modes had
>> >> their traditional behavior, they'd continue to work for higher
>> >> level constructs built on top of them like the prog-mode reindent
>> >> and mark defun. TS modes broke a whole bunch of things that had
>> >> worked fine for decades, and instead of fixing them, they just
>> >> made even more abstractions to plug inconsistent tree sitter
>> >> things in place of the broken things.
>> >
>> >Indentation is a lot more than just navigation.  And I disagree with
>> >you extreme interpretation of the current state of indentation and
>> >navigation support in TS-based modes.
>> 
>> I'm right.
>
> I beg to differ.
>
>> >Sorry, this is simplification.  A typical declaration breaks down into
>> >smaller parts, and we have expectations and ideas about indentation of
>> >each one of them.  But the tree-sitter classification of the AST
>> >constituents does not necessarily make that easy, because you could
>> >have the same syntactic symbol both inside a declaration and in other
>> >places.  So having an AST does not always immediately tell you how to
>> >indent correctly.
>> 
>> No, but it gives you more information than looking-at does, and cc-mode does its job admirable given only that simple tool. You can look at nesting and context in the TS AST to figure out what to do.
>
> Once again, going the looking-at way means we bring back all the
> ad-hoc stuff that attempts to "parse" the code using heuristic
> regexps, with the serious disadvantage that these heuristics need to
> be well understood by someone who has a good knowledge of the
> underlying language, and updated as the language evolves.  So I'd like
> to do that only when absolutely necessary, and as little as possible
> even then.

I'm not saying c++-ts-mode should parse using regexps like cc-mode does.
I am saying that the AST should contain all the information cc-mode
would instead get from this parsing and can be put to the same use.
What information is the AST missing?

>> >> Yes, and because it's ambiguous we get annoyances like python-mode's default sexp movement. Now every mode is like that, and you can't turn it off half the time?
>> >
>> >What else did you expect?  Some users like one style, others like the
>> >other.  Are we supposed to say "my way or the highway"?  And that's
>> >even before we consider that the disagreement cuts through the
>> >developers themselves.
>> 
>> No. I'm expecting a generally consistent experience, and if we want
>> to provide a configuration knob, it should affect everything
>> consistently. One shouldn't have to form independent and different
>> muscle memory for each language mode because the whims of their
>> authors were different.
>
> Consistency is problematic here, because "sexp" doesn't translate
> consistently between languages, and even "defun" not always does
> (cf. the "namespace" case).

That "sexp" and "defun" mean different things in different languages
doesn't mean movement commands between defuns should do different things
in different languages.  Movement _between_ defuns should work the same
way everywhere even if the _definition_ of a defun is language-specific.

I can write with a pen or a pencil.  I don't have to switch from
left-handed to right-handed writing when I want to write with ink
instead of graphite.

>> >I find continuing this kind of argument not constructive, so I will
>> >stop here.  Let me just say that I think you are looking at this stuff
>> >from some semi-abstract, almost idealistic, aspect.  As if we didn't
>> >have 40 years of development and user experience and expectations to
>> >keep and uphold.
>> 
>> I'll never understand the mindset that holds that things making sense and having a structure is bad actually because sense and structure are "academic" and "idealistic".
>
> There's no such mindset.  I have no disagreement with you on that
> level.  The disagreement is on a very practical level: we did try to
> do things that way at the beginning, two years ago.  We are where we
> are because that didn't work well enough.

Maybe we didn't try hard enough.  If there's something missing in the
AST, perhaps it could be added.

>> What 40 years of development and user experience holds is that if
>> I'm four pages deep into a nasty TypeScript function and hit
>> beginning-of-defun, I want to go to the beginning of the four page
>> defun I am editing and not some random place two pages up that I
>> didn't even know about in which someone scribbled out some kind of
>> nested lambda irrelevant to my present task.
>
> It turns out other users have other preferences and expectations,
> especially if we include other languages in the context.  We cannot
> tell them to get lost.

Users really prefer go-to-sibling behavior for beginning-of-defun?
Says who?