Subject: Re: bug#77872: [PATCH 1/2] ansi-term: ignore CSI commands with subparams

From: Johannes Altmanninger <[email-masked]>
Date: Sun, 20 Apr 2025 10:03:00 +0200

On Fri, Apr 18, 2025 at 10:33:36PM -0700, Jared Finder wrote:
> On 2025-04-18 02:00, Johannes Altmanninger wrote:
> > If it's not acceptable to ignore curly underline sequences, I can
> > probably also implement them in term.el (i.e. actually recognize
> > \e[4:3m).
> 
> I think this is a much preferred option. Emacs 30 in a terminal already
> supports displaying wavy underlines. Adding support for wavy underlines to
> ansi-term looks like it wouldn't be harder than the proposed patch.

It would be a step in the right direction, yeah
(though unrelated to my troubles with terminfo)

Given that we don't want to break this specific implicitly-supported command,
we should probably do the same for others, if any.
So the question is, is there any other command of the form
"CSI Ps : Ps X" where ansi-term's current interpretation as "CSI Ps X" is
reasonable. 

We'd need to do a bit of research, based on the commands that ansi-term
supports today.

Not sure when I can get to that (either way, I'll likely not be able
to enable curly underlines by default; there are various issues left
with other terminals).

> > That would give us curly underlines in graphical emacs, but it
> > would barely change the "emacs -nw" behavior in practice, because,
> > as described above, asking terminfo for Smulx/Su has a lot of false
> > negatives.  Patch 2/2 would change "emacs -nw" to not
> > emit anything if the capability is not advertised.
> > So my perspective is: if we want to implement \e[4:3m now, we should
> > also take the second patch to actually fix my use case for wrong
> > terminfo.
> 
> (Note for other readers, patch 2/2 is in another bug:
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=77871)

Yeah sorry I missed the "we prefer the 'git format-patch' method with
attachment" part from the CONTRIBUTE file.

> > In future, to get out of this mess, we should consider not asking
> > for Smulx/Su and simply emit the styled sequences unconditionally.
> > Unless Emacs still needs to run on hardware terminals that misinterpret
> > those, that would be the sanest option going forward.  It will
> > essentially force all terminal emulators to be compatible with this,
> > which is fine I think?  From the terminals I have tested, emitting
> > \e[4:3m would only cause regressions on these terminals: abduco, dvtm,
> > JetBrains IDE terminals and urxvt.  It should be easy to fix them or
> > add workarounds; in a few years it should no longer be an issue.
> 
> If I'm understanding this right, you're suggesting to have Emacs always emit
> the wavy underline escape sequence for :underline (:style wave), even on
> terminals it believes does not support wavy underlines.
> This seems like a significant behavior change.

Yes, deferring to the underlying terminal here would be a noticeable
change. I'd argue it would be simpler overall to have the middle-man 
be less opinionated but of course that's subjective. It's not super
obvious how nested terminals should behave. I personally tend to
avoid them nowadays though they are very useful sometimes.

> While libraries packaged within Emacs look to
> be good about using a face supports spec when defining their faces, the same
> can't be said for external Emacs libraries.

Right. The fallback behavior is helpful to some users/libraries and
the opposite to others.

I realized today even if I queried those nestable terminals (like tmux and ansi-term),
and they always returned the correct answer as to whether Smulx/Su is supported,
it wouldn't really help me because they this doesn't tell me anything
about the parent terminal.
I guess I can work around it for tmux (which does tell me the parent
terminal's name and pty) but it sounds pretty difficult to standardize
an interface for this.

> I also don't understand how this makes things better. Are there terminals
> that you encounter that Emacs does not accurately report
> (display-supports-face-attributes-p '(:underline (:style wave))) for? Local
> testing on a Mac with iTerm2, kitty, Alacritty, Apple Terminal.app, Rio, and
> WezTerm correctly reported t vs nil.

Couple problems.

1.

I took a naïve approach and installed Emacs 29.4 as well as these
terminals via Homebrew on a (mostly vanilla) macOS. I've been running
"emacs -nw" with no config, evaluating that expression gave nil for
all of them.

This result is consistent with

	{ infocmp -x $TERM; infocmp -x $TERM_PROGRAM } | grep -E '\b(Su|Smulx)\b'

which gave no results on this.

I guess it's my bad to use macOS's system libraries that are probably not updated often.

After `brew install ncurses` version 6.5, my $(find /opt/homebrew -name infocmp) did manage
to find a terminfo entry for say kitty, but not others like iTerm.app.
Presumably "TERMINFO=/opt/homebrew/opt/ncurses/share/terminfo emacs -nw" would work too.

On my Linux system only kitty, alacritty and rio have it.

	$ for term in iterm2 kitty xterm-kitty alacritty Apple_Terminal rio wezterm; do
	    echo $term
	    infocmp -x $term | grep -E '\b(Su|Smulx)\b'
	done
	iterm2
	kitty
		Smulx=\E[4:%p1%dm, Ss=\E[%p1%d q, TS=\E]2;,
	xterm-kitty
		am, ccc, hs, km, mc5i, mir, msgr, npc, xenl, Su, Tc, XF, fullkbd,
		Smulx=\E[4:%p1%dm, Ss=\E[%p1%d q, Sync=\EP=%p1%ds\E\\,
	alacritty
		PE=\E[201~, PS=\E[200~, Se=\E[0 q, Smulx=\E[4:%p1%dm,
	Apple_Terminal
	rio
		PE=\E[201~, PS=\E[200~, Se=\E[0 q, Smulx=\E[4:%p1%dm,
	wezterm

Of course we can and should add Smulx/Su to the others.
But the bigger problems is:

2.

Out of the ones you mention,
alacritty, Apple Terminal.app, iTerm and wezterm
impersonate XTerm by setting `TERM=xterm256color`
(as do others like gnome-terminal and Konsole).

As far as I can tell this means that
/usr/share/terminfo/x/xterm+256color should be used, which does *not*
support curly and colored underlines.

I'd be very curious to hear what kind of magic Emacs and/or your
system uses to somehow still get Su/Smulx support for those terminals.

Especially for Terminal.app, which does not even support XTVERSION;
the only way to work around its quirks is by checking if $TERM_PROGRAM
is "Apple_Terminal", which I don't see Emacs doing.

3.

However way you got Emacs to think Terminal.app supports curly
underlines, it definitely does not support the de-facto standard
control sequence here. Even worse: it interprets that command as
"paint the background yellow". To reproduce:

	printf '\033[4:3mcurly\033[m'