GNU bug report logs - #77846
[PATCH ]31.0.50; C# dotnet compiler output incorrectly handled by compilation-mode

Previous Next

To reply to this bug, email your comments to 77846 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: bug-gnu-emacs <at> gnu.org
Subject: [PATCH ]31.0.50; C# dotnet compiler output incorrectly handled by
 compilation-mode
Date: Wed, 16 Apr 2025 20:24:44 +0200

[Message part 1 (text/plain, inline)]

Hey everyone.

When using Emacs to compile a modern C# project, one uses the dotnet CLI to do so (dotnet build).

The output from recent dotnet CLI versions are (unfortunately) optmized for "fancyness" these days, which means it contains various escape sequences.

The current regexp in compilation-mode incorrectly includes this escape sequence for the first error reported, making navigation to the first error found fail.

Attached is a patch which excludes this escape sequence from the captured value, fixing navigation again.

—
Kind Regards
Jostein Kjønigsen



[Message part 2 (text/html, inline)]

[0001-compile.el-Handle-dotnet-compiler-escape-sequences.patch (application/octet-stream, attachment)]

[Message part 4 (text/html, inline)]

Message #8 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: 77846 <at> debbugs.gnu.org
Subject: Re: bug#77846: [PATCH ]31.0.50;
 C# dotnet compiler output incorrectly handled by compilation-mode
Date: Thu, 17 Apr 2025 08:52:42 +0300

> From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
> Date: Wed, 16 Apr 2025 20:24:44 +0200
> 
> When using Emacs to compile a modern C# project, one uses the dotnet CLI to do so (dotnet build).
> 
> The output from recent dotnet CLI versions are (unfortunately) optmized for "fancyness" these days, which means it contains various escape sequences.
> 
> The current regexp in compilation-mode incorrectly includes this escape sequence for the first error reported, making navigation to the first error found fail.
> 
> Attached is a patch which excludes this escape sequence from the captured value, fixing navigation again.

Thanks, but please also update etc/compilation.txt accordingly.

Message #11 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 77846 <at> debbugs.gnu.org
Subject: bug#77846: [PATCH ]31.0.50; C# dotnet compiler output incorrectly
 handled by compilation-mode 
Date: Fri, 18 Apr 2025 17:14:54 +0200

+          (optional (seq "{ESC}=")) ; escape sequence sometimes found on first line of output

Please keep lines within 80 columns, and don't put actual control chars in source code; use escape sequences. You don't need the `seq`.
Something like

+          (opt "\e=")  ; sequence sometimes found on first line of output

perhaps.

As Eli remarked we'd like to see an example in compilation.txt, and just as important is adding a test case in compile-tests.el.

It would also be useful to know precisely what that escape sequence is supposed to be doing. As far as I can tell, `ESC =` is the DECKPAM sequence that puts the keypad in application mode, which makes no sense at all in this context.

Perhaps it's compilation-filter-hook that really should be used to filter it out, and other sequences like it?

Message #14 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 77846 <at> debbugs.gnu.org
Subject: Re: bug#77846: [PATCH ]31.0.50; C# dotnet compiler output incorrectly
 handled by compilation-mode 
Date: Fri, 18 Apr 2025 18:57:39 +0200

[Message part 1 (text/plain, inline)]

> On 18 Apr 2025, at 17:14, Mattias Engdegård <mattias.engdegard <at> gmail.com> wrote:
> 
> +          (optional (seq "{ESC}=")) ; escape sequence sometimes found on first line of output
> 
> Please keep lines within 80 columns, and don't put actual control chars in source code; use escape sequences. You don't need the `seq`.
> Something like
> 
> +          (opt "\e=")  ; sequence sometimes found on first line of output
> 
> perhaps.
> 

I've made some changes like that, and that seems to work fine.

> As Eli remarked we'd like to see an example in compilation.txt

Done. Didn't know about this one.

> , and just as important is adding a test case in compile-tests.el.

I'll defer that slightly based on comment below.

> 
> It would also be useful to know precisely what that escape sequence is supposed to be doing. As far as I can tell, `ESC =` is the DECKPAM sequence that puts the keypad in application mode, which makes no sense at all in this context.

I'll admit to not know too much in depth stuff about escape codes and how they're used.

From trying to read the MSBuild source code, my best guess so far is that the escape-code seems to be used in an elaborate output-mode detection process:

https://github.com/dotnet/msbuild/blob/a9d68ab58eab3c25b6378a90be8060dd1429a6ef/src/Framework/NativeMethods.cs#L1588-L1642

Someone else might know more about this, or are better able to parse this code.

> 
> Perhaps it's compilation-filter-hook that really should be used to filter it out, and other sequences like it?
> 

That also seems like an appropriate solution.

Is there precedence for solution like in place already?

--
Jostein

[Message part 2 (text/html, inline)]

Message #17 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 77846 <at> debbugs.gnu.org
Subject: Re: bug#77846: [PATCH ]31.0.50; C# dotnet compiler output incorrectly
 handled by compilation-mode 
Date: Sat, 19 Apr 2025 21:28:22 +0200

18 apr. 2025 kl. 18.57 skrev Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>:

> From trying to read the MSBuild source code, my best guess so far is that the escape-code seems to be used in an elaborate output-mode detection process:
> 
> https://github.com/dotnet/msbuild/blob/a9d68ab58eab3c25b6378a90be8060dd1429a6ef/src/Framework/NativeMethods.cs#L1588-L1642

Not sure if that particular piece of code is relevant but you probably know MSBuild (and C#) better. I spent all of five minutes searching the msbuild project for insertion or detection of `<ESC> =` but nothing obvious showed up, so there is probably something we don't quite understand here.

It makes no sense to emit that sequence from a non-interactive program to a terminal because it doesn't affect the output in any way. Does it come from the C# compiler proper, from MSBuild, or some other software component (no idea how this all fits together, as you can tell).

>> Perhaps it's compilation-filter-hook that really should be used to filter it out, and other sequences like it?
> 
> That also seems like an appropriate solution.
> 
> Is there precedence for solution like in place already?

This hook would be the most obvious thing to use if the stray sequence is emitted by a different layer and risks appearing in different contexts. For example, one could write something like

> (defun stray-msbuild-seq-compilation-filter ()
>   (defvar compilation-filter-start)
>   (let ((begin compilation-filter-start)
>         (end (point))
>         (inhibit-read-only t))
>     (save-excursion
>       (goto-char begin)
>       (while (re-search-forward (rx bol "\e=") end t)
>         (delete-region (match-beginning 0) (match-end 0))))))

and then add that function to compilation-filter-hook.

Message #20 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 77846 <at> debbugs.gnu.org
Subject: Re: bug#77846: [PATCH ]31.0.50; C# dotnet compiler output incorrectly
 handled by compilation-mode
Date: Wed, 23 Apr 2025 08:29:36 +0200

[Message part 1 (text/plain, inline)]

> On 19 Apr 2025, at 21:28, Mattias Engdegård <mattias.engdegard <at> gmail.com> wrote:
> 
> 
>>> Perhaps it's compilation-filter-hook that really should be used to filter it out, and other sequences like it?
>> 
>> That also seems like an appropriate solution.
>> 
>> Is there precedence for solution like in place already?
> 
> This hook would be the most obvious thing to use if the stray sequence is emitted by a different layer and risks appearing in different contexts. For example, one could write something like
> 
>> (defun stray-msbuild-seq-compilation-filter ()
>>  (defvar compilation-filter-start)
>>  (let ((begin compilation-filter-start)
>>        (end (point))
>>        (inhibit-read-only t))
>>    (save-excursion
>>      (goto-char begin)
>>      (while (re-search-forward (rx bol "\e=") end t)
>>        (delete-region (match-beginning 0) (match-end 0))))))
> 
> and then add that function to compilation-filter-hook.
> 

Just to be clear and to make sure I understand: Do you suggest I test that hook locally and if it works, propose that as a patch? If so, should I add it to compile.el? Or were you suggesting something else?

I see that in the entirety of the current Emacs code-base, there are only 4 implemented hooks of this kind, and they are kinda spread around all over (follow.el, grep.el, so-long.el).

--
Jostein

[Message part 2 (text/html, inline)]

Message #23 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: mattias.engdegard <at> gmail.com, Jostein Kjønigsen
 <jostein <at> secure.kjonigsen.net>
Cc: 77846 <at> debbugs.gnu.org
Subject: Re: bug#77846: [PATCH ]31.0.50; C# dotnet compiler output incorrectly
 handled by compilation-mode
Date: Sat, 10 May 2025 12:15:27 +0300

Ping!  Mattias, could you please answer Jostein's question?

> From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
> Date: Wed, 23 Apr 2025 08:29:36 +0200
> Cc: Eli Zaretskii <eliz <at> gnu.org>,
>  77846 <at> debbugs.gnu.org
> 
> 
>  On 19 Apr 2025, at 21:28, Mattias Engdegård <mattias.engdegard <at> gmail.com> wrote:
> 
>  Perhaps it's compilation-filter-hook that really should be used to filter it out, and other sequences
>  like it?
> 
>  That also seems like an appropriate solution.
> 
>  Is there precedence for solution like in place already?
> 
>  This hook would be the most obvious thing to use if the stray sequence is emitted by a different layer
>  and risks appearing in different contexts. For example, one could write something like
> 
>  (defun stray-msbuild-seq-compilation-filter ()
>   (defvar compilation-filter-start)
>   (let ((begin compilation-filter-start)
>         (end (point))
>         (inhibit-read-only t))
>     (save-excursion
>       (goto-char begin)
>       (while (re-search-forward (rx bol "\e=") end t)
>         (delete-region (match-beginning 0) (match-end 0))))))
> 
>  and then add that function to compilation-filter-hook.
> 
> Just to be clear and to make sure I understand: Do you suggest I test that hook locally and if it works,
> propose that as a patch? If so, should I add it to compile.el? Or were you suggesting something else?
> 
> I see that in the entirety of the current Emacs code-base, there are only 4 implemented hooks of this kind,
> and they are kinda spread around all over (follow.el, grep.el, so-long.el).

Message #26 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 77846 <at> debbugs.gnu.org
Subject: Re: bug#77846: [PATCH ]31.0.50; C# dotnet compiler output incorrectly
 handled by compilation-mode 
Date: Thu, 15 May 2025 19:04:02 +0200

Jostein and Eli, sorry about the delay.

23 apr. 2025 kl. 08.29 skrev Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>:

> Just to be clear and to make sure I understand: Do you suggest I test that hook locally and if it works, propose that as a patch? If so, should I add it to compile.el? Or were you suggesting something else?

What I meant is that `compilation-filter-hook` seems to be the proper level for filtering out stuff that isn't part of the messages emitted by a compiler, or in-band signalling common to otherwise unrelated tools. An example of the former would be unwanted annotation by the build system or terminal emulation, of the latter is SGR escape sequences for colouring the output.

I just don't understand where the 'ESC =' sequence comes from (the C# compiler?) or what it means. You said it was for 'fanciness', but I can't find any reference to it anywhere.

Could it be the result of Emacs mutilating the message somehow? What happens if you send the compiler errors to a file and visit it with Emacs in fundamental-mode?

Message #29 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: jostein <at> secure.kjonigsen.net
Cc: Mattias Engdegård <mattias.engdegard <at> gmail.com>,
 77846 <at> debbugs.gnu.org
Subject: Re: bug#77846: [PATCH ]31.0.50; C# dotnet compiler output incorrectly
 handled by compilation-mode
Date: Sat, 24 May 2025 12:11:14 +0300

> From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
> Date: Thu, 15 May 2025 19:04:02 +0200
> Cc: Eli Zaretskii <eliz <at> gnu.org>,
>  77846 <at> debbugs.gnu.org
> 
> Jostein and Eli, sorry about the delay.
> 
> 23 apr. 2025 kl. 08.29 skrev Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>:
> 
> > Just to be clear and to make sure I understand: Do you suggest I test that hook locally and if it works, propose that as a patch? If so, should I add it to compile.el? Or were you suggesting something else?
> 
> What I meant is that `compilation-filter-hook` seems to be the proper level for filtering out stuff that isn't part of the messages emitted by a compiler, or in-band signalling common to otherwise unrelated tools. An example of the former would be unwanted annotation by the build system or terminal emulation, of the latter is SGR escape sequences for colouring the output.
> 
> I just don't understand where the 'ESC =' sequence comes from (the C# compiler?) or what it means. You said it was for 'fanciness', but I can't find any reference to it anywhere.
> 
> Could it be the result of Emacs mutilating the message somehow? What happens if you send the compiler errors to a file and visit it with Emacs in fundamental-mode?

Jostein, any followups or comments?

Message #32 received at 77846 <at> debbugs.gnu.org (full text, mbox):

From: Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Mattias Engdegård <mattias.engdegard <at> gmail.com>,
 77846 <at> debbugs.gnu.org
Subject: Re: bug#77846: [PATCH ]31.0.50; C# dotnet compiler output incorrectly
 handled by compilation-mode
Date: Mon, 26 May 2025 10:11:10 +0200

> On 24 May 2025, at 11:11, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
>> Date: Thu, 15 May 2025 19:04:02 +0200
>> Cc: Eli Zaretskii <eliz <at> gnu.org>,
>> 77846 <at> debbugs.gnu.org
>> 
>> Jostein and Eli, sorry about the delay.
>> 
>> 23 apr. 2025 kl. 08.29 skrev Jostein Kjønigsen <jostein <at> secure.kjonigsen.net>:
>> 
>>> Just to be clear and to make sure I understand: Do you suggest I test that hook locally and if it works, propose that as a patch? If so, should I add it to compile.el? Or were you suggesting something else?
>> 
>> What I meant is that `compilation-filter-hook` seems to be the proper level for filtering out stuff that isn't part of the messages emitted by a compiler, or in-band signalling common to otherwise unrelated tools. An example of the former would be unwanted annotation by the build system or terminal emulation, of the latter is SGR escape sequences for colouring the output.
>> 
>> I just don't understand where the 'ESC =' sequence comes from (the C# compiler?) or what it means. You said it was for 'fanciness', but I can't find any reference to it anywhere.
>> 
>> Could it be the result of Emacs mutilating the message somehow? What happens if you send the compiler errors to a file and visit it with Emacs in fundamental-mode?
> 
> Jostein, any followups or comments?

I'm afraid not.

The feedback so far has been not as straightforward as it usually is when there's a bug which needs fixing.

I'm still on the fence on what I think is the best solution for me personally & locally only, vs what is a good fix to push upstream for all Emacs-users to consume.

If open bugs in the tracker is an issue (administrative noise, etc) It's OK for me to close/suspend this bug as it is, while I work around it locally until I have a proper lightbulb-moment.

Right now, I have other more important fixes I'd rather put my attention instead ;)

--
Jostein

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.