Subject: Re: bug#44486: 27.1; C-@ chars corrupt elisp buffer

From: Eli Zaretskii <[email-masked]>
Date: Sat, 14 Nov 2020 18:13:49 +0200

> From: Stefan Monnier <[email-masked]>
> Cc: [email-masked],  [email-masked],  [email-masked],
>   [email-masked],  [email-masked]
> Date: Sat, 14 Nov 2020 10:19:57 -0500
> 
> >> Actually, for prefer-utf-8 files, I think we never want to automatically
> >> fallback to binary.
> > I think you are assuming prefer-utf-8 is something other than what it
> > is.  It is not a variant of UTF-8, it is a variant of 'undecided'
> > (i.e. it starts by detecting the encoding), which prefers UTF-8 if
> > that can decode the text.
> 
> My position is not based on principles but on pragmatic concerns.
> AFAIK `prefer-utf-8` is only ever used for files which are known to
> contain text and should almost always contain UTF-8 text.

For those, we should use utf-8, not prefer-utf-8.

> I believe if there's a NUL byte in such a files but it otherwise doesn't
> contain any invalid UTF-8 byte sequence, it will result in better
> behavior if we treat it as UFT-8 than as binary.

We treat null bytes as the _single_ telltale sign of a binary file.
If we disable that in coding-systems that are supposed to _detect_
encoding, we will never be able to detect binary files.