Subject: bug#27270: closed (Re: bug#27270: display-raw-bytes-as-hex generates ambiguous output for Emacs strings)

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 28 Apr 2022 17:59:02 +0000

Your bug report

#27270: display-raw-bytes-as-hex generates ambiguous output for Emacs strings

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
27270: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27270
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#27270: display-raw-bytes-as-hex generates ambiguous output for Emacs strings
From: Paul Eggert <[email-masked]>
To: Lars Ingebrigtsen <[email-masked]>
Cc: Eli Zaretskii <[email-masked]>, [email-masked], [email-masked], [email-masked]
Date: Thu, 28 Apr 2022 10:58:33 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

On 4/27/22 10:22, Lars Ingebrigtsen wrote:
> Paul Eggert <[email-masked]> writes:
> 
>> Could you explain what you mean by "change the output"? (Sorry, I'm
>> not seeing it.)
> 
> I said earlier:
> 
>> display-raw-bytes-as-hex is a display setting.  You want to change it so
>> that the data output will be different

Still not quite following, as I had been thinking more recently of 
changing only how display-raw-bytes-as-hex displays.

That being said, I looked into the code and found that what I was asking 
for would be quite a pain to implement - more trouble than it's worth, 
anyway - so I withdraw the suggestion and am closing the bug report.

I installed the attached, which documents the situation.

From d501db962eae2b831a2497adc85a94e98064e969 Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Thu, 28 Apr 2022 10:51:01 -0700
Subject: [PATCH] Document \807 etc. in raw byte display

* doc/emacs/display.texi (Display Custom): Mention potential
confusion in raw byte display.
---
 doc/emacs/display.texi | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/doc/emacs/display.texi b/doc/emacs/display.texi
index 2ac0dca622..7a6c7f391b 100644
--- a/doc/emacs/display.texi
+++ b/doc/emacs/display.texi
@@ -2097,3 +2097,14 @@ Display Custom
 byte with a decimal value of 128 is displayed as @code{\200}.  To
 change display to the hexadecimal format of @code{\x80}, set the
 variable @code{display-raw-bytes-as-hex} to @code{t}.
+Care may be needed when interpreting a raw byte when copying
+text from a terminal containing an Emacs session, or when a terminal's
+@code{escape-glyph} face looks like the default face.  For example, by
+default Emacs displays the four characters @samp{\}, @samp{2},
+@samp{0}, @samp{0} with the same characters it displays a byte with
+decimal value 128.  The problem can be worse with hex displays, where
+the raw byte 128 followed by the character @samp{7} is displayed as
+@code{\x807}, which Emacs Lisp reads as the single character U+0807
+SAMARITAN LETTER IT; this confusion does not occur with the
+corresponding octal display @code{\2007} because octal escapes contain
+at most three digits.
-- 
2.35.1

Subject: display-raw-bytes-as-hex generates ambiguous output for Emacs strings
From: Paul Eggert <[email-masked]>
To: Emacs bug reports and feature requests <[email-masked]>
Cc: Vasilij Schneidermann <[email-masked]>
Date: Tue, 6 Jun 2017 20:57:51 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

With the default octal display format one can copy text out of a terminal window 
and into an Emacs string, reliably. With the new hex display this doesn't work 
any more, unfortunately. For example, if I run this shell script:

printf 'x\2205y\n' >foo.txt
LC_ALL=C emacs -nw --color=no --eval '(progn (setq display-raw-bytes-as-hex t) 
(find-file-literally "foo.txt"))'

then on the terminal display I see:

x\x905y

If I cut and paste this (using my windowing system) into an Emacs string, like this:

"x\x905y"

and then evaluate the string, the result is the string "xअy", that is, a 
3-character string with the characters "x", "अ", and "y", where the middle 
character is U+090F DEVANAGARI LETTER A. This is an incorrect representation, as 
the buffer actually contains the four characters "x", "\x90", "5", and "y". The 
problem is that the string has glued together the representation of the 
character "\x90" to the representation of the character "5", resulting in the 
representation of the character "\x905" which is not accurate.

Please change the behavior of display-raw-bytes-as-hex so that it is not 
ambiguous in this way.

A simple solution would be to display this instead:

x\x90\x35y

though that is awkward because it means the ASCII 0-9, a-f, A-F would be 
displayed as hexadecimal escapes when they follow another hexadecimal escape. 
Perhaps we can think of a better approach. One possibility would be to define 
and use a new string escape \Xxx that contains at most two hex digits.

By the way, I expected display-raw-bytes-as-hex to affect how Emacs displays 
Emacs strings, too. Shouldn't it?