Subject: Re: bug#27270: display-raw-bytes-as-hex generates ambiguous output for Emacs strings

From: Paul Eggert <[email-masked]>
Date: Thu, 8 Jun 2017 09:24:56 -0700

On 06/08/2017 09:11 AM, Eli Zaretskii wrote:
>> (setq display-raw-bytes-as-hex t) C-j
>> (format "%c%c" ?\u0090 ?5) C-j
>>
>> Emacs displays this:
>>
>> "\x905"
>>
>> which is the wrong string visually.
> How is that different from "\2205" you get under the default settings?

When I cut and paste "\2205" into another Emacs, it evaluates to the 
same two-character string that I started off with because octal escapes 
are limited to 3 octal digits. When I cut and paste "\x905" I get a 
one-character string because there is no limit to the length of 
hexadecimal escapes. This is a problem, because cut-and-paste should 
continue to copy text accurately even when I'm using terminal windows.

>> The string should be
>> displayed unambiguously, either like this:
>>
>> "\x80\ 5"
>>
>> or via some other means.
> We do use "some other means": the raw byte has a different face.

That doesn't help when --color=no is specified, or in terminal sessions 
that do not support colors. And the colors, even when present, do not 
survive cutting and pasting, which copies the text without colors. So 
this is a real problem.

> But if you evaluate the above in*scratch*, you won't see that because of
> font-lock.  Turn off font-lock-mode, and you will clearly see where
> the raw byte ends and "normal" text begins.

Turning off font-lock-mode doesn't help when colors are disabled. I 
often run with colors disabled, since my terminal color scheme disagrees 
with Emacs's and I prefer monochrome anyway. So this ambiguity will be a 
real pain for me.