Subject: bug#27270: display-raw-bytes-as-hex generates ambiguous output for Emacs strings

From: Paul Eggert <[email-masked]>
Date: Tue, 6 Jun 2017 20:57:51 -0700

With the default octal display format one can copy text out of a terminal window 
and into an Emacs string, reliably. With the new hex display this doesn't work 
any more, unfortunately. For example, if I run this shell script:

printf 'x\2205y\n' >foo.txt
LC_ALL=C emacs -nw --color=no --eval '(progn (setq display-raw-bytes-as-hex t) 
(find-file-literally "foo.txt"))'

then on the terminal display I see:

x\x905y

If I cut and paste this (using my windowing system) into an Emacs string, like this:

"x\x905y"

and then evaluate the string, the result is the string "xअy", that is, a 
3-character string with the characters "x", "अ", and "y", where the middle 
character is U+090F DEVANAGARI LETTER A. This is an incorrect representation, as 
the buffer actually contains the four characters "x", "\x90", "5", and "y". The 
problem is that the string has glued together the representation of the 
character "\x90" to the representation of the character "5", resulting in the 
representation of the character "\x905" which is not accurate.

Please change the behavior of display-raw-bytes-as-hex so that it is not 
ambiguous in this way.

A simple solution would be to display this instead:

x\x90\x35y

though that is awkward because it means the ASCII 0-9, a-f, A-F would be 
displayed as hexadecimal escapes when they follow another hexadecimal escape. 
Perhaps we can think of a better approach. One possibility would be to define 
and use a new string escape \Xxx that contains at most two hex digits.

By the way, I expected display-raw-bytes-as-hex to affect how Emacs displays 
Emacs strings, too. Shouldn't it?