GNU bug report logs - #21780
25.0.50; Saving *Help* results in bad encoding because of curly quotes

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21780 in the body.
You can then email your comments to 21780 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.0.50; Saving *Help* results in bad encoding because of curly quotes
Date: Wed, 28 Oct 2015 18:50:49 -0700 (PDT)

emacs -Q
M-x load-library isearch.el
C-h f isearch-forward
In buffer *Help*: C-x C-w foo.txt

You get a coding-system warning.  I tried saving it as utf-8 and as raw
text.

In both cases, when I open that file in a new Emacs session, I see octal
escapes where there were curly quotes.

Why were there curly quotes?  Because `C-h f' produces curly quotes.

This is a regression - no such problem exists with Emacs 24.5 (or prior).

In order to produce a reasonable, readable file from the *Help* buffer
that I attached to the following mail message to emacs-devel <at> gnu.org,
I had to resort to using Emacs 24.5:

 "RE: Exposing Isearch toggleable options", 2015-10-28, ~21:20


(BTW, it is apparently *not* the case, in spite of what is stated at
http://lists.gnu.org/archive/html/emacs-devel/2015-10/index.html, that
the mailing list archive is updated every 30 minutes.  Far from it,
it seems.  That's why I didn't provide a URL to the emacs-devel post.
Got tired after 1/2 hour of waiting for it to show up.)

In GNU Emacs 25.0.50.1 (i686-pc-mingw32)
 of 2015-10-09
Bzr revision: af45926d66d303fdc4c2c3ebbc820b4a54d9e4a0
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --host=i686-pc-mingw32 --enable-checking=yes,glyphs'

Message #10 received at 21780-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 21780-done <at> debbugs.gnu.org
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Thu, 29 Oct 2015 19:41:38 +0200

> Date: Wed, 28 Oct 2015 18:50:49 -0700 (PDT)
> From: Drew Adams <drew.adams <at> oracle.com>
> 
> emacs -Q
> M-x load-library isearch.el
> C-h f isearch-forward
> In buffer *Help*: C-x C-w foo.txt
> 
> You get a coding-system warning.  I tried saving it as utf-8 and as raw
> text.
> 
> In both cases, when I open that file in a new Emacs session, I see octal
> escapes where there were curly quotes.

Thanks, I fixed the first part of this: Emacs should no longer ask
annoying questions when you save help buffers with curved quotes.

The second part, which happens when visiting the saved file, is not a
bug: you need to specify the encoding of files when visiting them in
locales whose default encoding is different.  (Actually, I expect this
to work automatically for you, at least in "emacs -Q", but that
doesn't happen in every locale.)

Message #13 received at 21780-done <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21780-done <at> debbugs.gnu.org
Subject: RE: bug#21780: 25.0.50; Saving *Help* results in bad encoding because
 of curly quotes
Date: Thu, 29 Oct 2015 10:58:24 -0700 (PDT)

> > emacs -Q
> > M-x load-library isearch.el
> > C-h f isearch-forward
> > In buffer *Help*: C-x C-w foo.txt
> >
> > You get a coding-system warning.  I tried saving it as utf-8 and as raw
> > text.
> >
> > In both cases, when I open that file in a new Emacs session, I see octal
> > escapes where there were curly quotes.
> 
> Thanks, I fixed the first part of this: Emacs should no longer ask
> annoying questions when you save help buffers with curved quotes.
> 
> The second part, which happens when visiting the saved file, is not a
> bug: you need to specify the encoding of files when visiting them in
> locales whose default encoding is different.  (Actually, I expect this
> to work automatically for you, at least in "emacs -Q", but that
> doesn't happen in every locale.)

I guess I should interpret this as meaning that the bug is fixed (?).

But I don't understand the second part.  What do I need to change, as
a user, to get this to work as I would expect?

In Emacs, before saving, the buffer looks fine.  When visiting the
resulting file it does not look right - it is unreadable.  There are
3 octal escapes for each opening curly quote and 3 of them for each
closing curly quote.  That can amount to quite a lot of noise.

Do I need to save the buffer using some other encoding?  If so, which?
Emacs proposed two encodings (one of which was raw text, which I tried;
and I tried also utf-8, which I would have thought would show curly
quotes OK.

I would think that Emacs would DTRT when opening the file, based on
the encoding used to save it.  Should users really need to do something
special each time they visit the file?  They've never had to do this
before, for basic, common *Help* output.

This still seems like a regression to me, as there is no such
annoyance in Emacs 24.5 or prior.  Then, Emacs did not use curly
quotes for `describe-*' command output, and saved *Help* buffers
were readable from the outset.

If readers have to jump through hoops (e.g. changing "locales"),
and there is no good fix for this regression in behavior, then I'd
suggest that maybe `describe-*' commands should not use curly quotes.

[Or could this perhaps be a font problem?  Might the default font
(e.g. on MS Windows) just need to be changed?]

Message #16 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 21780 <at> debbugs.gnu.org
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Thu, 29 Oct 2015 20:18:29 +0200

> Date: Thu, 29 Oct 2015 10:58:24 -0700 (PDT)
> From: Drew Adams <drew.adams <at> oracle.com>
> Cc: 21780-done <at> debbugs.gnu.org
> 
> > > emacs -Q
> > > M-x load-library isearch.el
> > > C-h f isearch-forward
> > > In buffer *Help*: C-x C-w foo.txt
> > >
> > > You get a coding-system warning.  I tried saving it as utf-8 and as raw
> > > text.
> > >
> > > In both cases, when I open that file in a new Emacs session, I see octal
> > > escapes where there were curly quotes.
> > 
> > Thanks, I fixed the first part of this: Emacs should no longer ask
> > annoying questions when you save help buffers with curved quotes.
> > 
> > The second part, which happens when visiting the saved file, is not a
> > bug: you need to specify the encoding of files when visiting them in
> > locales whose default encoding is different.  (Actually, I expect this
> > to work automatically for you, at least in "emacs -Q", but that
> > doesn't happen in every locale.)
> 
> I guess I should interpret this as meaning that the bug is fixed (?).

Yes, I think so.

> But I don't understand the second part.  What do I need to change, as
> a user, to get this to work as I would expect?

It might work as you expect already.  You can try this:

  . After "C-h f some-function RET", switch to the *Help* buffer and
    type "C-x RET f utf-8 RET", then save the buffer as in your
    recipe.

  . Now visit the file where you saved the *Help* buffer: if the
    curved quotes display correctly, then "it works for you".

  . If the curved quotes look like raw bytes or, worse, pairs of
    non-ASCII characters, you need to visit such file like this:

      C-x RET c utf-8 RET C-x C-f FILE-NAME RET

> In Emacs, before saving, the buffer looks fine.

It looks fine, but the encoding mnemonic on the mode line is not "U"
(which stands for UTF-8), right?  That is why Emacs asks you for
encoding: it cannot save these characters using your locale's default
encoding (which is what the *Help* buffer uses by default).

> When visiting the resulting file it does not look right - it is
> unreadable.  There are 3 octal escapes for each opening curly quote
> and 3 of them for each closing curly quote.  That can amount to
> quite a lot of noise.

Yes, because you probably told Emacs to use raw-text or somesuch, when
it asked.

> Do I need to save the buffer using some other encoding?  If so, which?

Yes, you could tell it to use UTF-8 when it asked.  (After my change,
Emacs will do that automatically, no questions asked, when saving
*Help* buffers with curved quotes.)

> Emacs proposed two encodings (one of which was raw text, which I tried;
> and I tried also utf-8, which I would have thought would show curly
> quotes OK.

UTF-8 should have worked.  I wouldn't expect you to see octal escapes
after saving in UTF-8.

> I would think that Emacs would DTRT when opening the file, based on
> the encoding used to save it.

It cannot always do that.  To make sure it always does, there should
be a 'coding' cookie in the file or a file-local variable to the same
effect.  But you will have to add it manually; I don't think it's OK
for Emacs to insert such additions on its own, because Emacs has no
idea how the file will be used.

> Should users really need to do something special each time they
> visit the file?  They've never had to do this before, for basic,
> common *Help* output.

If you customize text-quoting-style to use ASCII characters for
quoting, Emacs will still behave as it did before: the file you
produce will be pure ASCII, so no decoding is necessary.

> If readers have to jump through hoops (e.g. changing "locales"),
> and there is no good fix for this regression in behavior, then I'd
> suggest that maybe `describe-*' commands should not use curly quotes.

Saving a *Help* buffer is not a frequent operation, and most users
nowadays live in UTF-8 locales anyway.  And even in some non-UTF-8
locales Emacs will succeed in displaying the file correctly when
visiting it, even without the need to type "C-x RET c".  So I think
this is not a catastrophe.

> [Or could this perhaps be a font problem?  Might the default font
> (e.g. on MS Windows) just need to be changed?]

No, it's not a font problem: Emacs did display those characters before
you saved the buffer, right?

Message #19 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Drew Adams <drew.adams <at> oracle.com>
Cc: 21780 <at> debbugs.gnu.org
Subject: RE: bug#21780: 25.0.50; Saving *Help* results in bad encoding because
 of curly quotes
Date: Thu, 29 Oct 2015 13:53:30 -0700 (PDT)

[Message part 1 (text/plain, inline)]

> It might work as you expect already.  You can try this:
> 
>   . After "C-h f some-function RET", switch to the *Help* buffer and
>     type "C-x RET f utf-8 RET", then save the buffer as in your
>     recipe.
> 
>   . Now visit the file where you saved the *Help* buffer: if the
>     curved quotes display correctly, then "it works for you".

That's exactly what I reported I did.  The result was seeing octal
escapes when I opened the file in a new Emacs session.

Or so I thought.  I just repeated it now with `emacs -Q' and it does
not seem to be a problem if utf-8 is used.

I think now that what I did earlier, when specified utf-8, I was
using my setup.  When I try that again (with my setup), I see the
problem I reported.

Also, with my setup the *Warning* text is different.  Instead of
providing lots of possible encoding choices and using
chinese-iso-8bit as the default (which is what I get with emacs
-Q - why is that, BTW?), it says:

Select one of the safe coding systems listed below,
or cancel the writing with C-g and edit the buffer
   to remove or modify the problematic characters,
or specify any other coding system (and risk losing
   the problematic characters).

  raw-text no-conversion

It was in that context that I anyway tried utf-8 and got the
awful result.  I also tried raw-text, since utf-8 was not in
the prompt (but was apparently accepted, and apparently did
not help).

The full warning is this, when I use my setup (which uses font
"-outline-Lucida Console-normal-normal-normal-mono-14-*-*-*-c-*-iso8859-1"):

These default coding systems were tried to encode text
in the buffer 'throw-isearch-help5.txt':
  (undecided-unix (489 . 8216) (491 . 8217) (499 . 8216) (503 . 8217)
  (577 . 8216) (583 . 8217) (875 . 8216) (892 . 8217) (912 . 8216)
  (931 . 8217) (963 . 8216)) (iso-latin-1-unix (489 . 8216) (491
  . 8217) (499 . 8216) (503 . 8217) (577 . 8216) (583 . 8217) (875
  . 8216) (892 . 8217) (912 . 8216) (931 . 8217) (963 . 8216))
However, each of them encountered characters it couldn't encode:
  undecided-unix cannot encode these: ' ' ' ' ' ' ' ' ' ' ...
  iso-latin-1-unix cannot encode these: ' ' ' ' ' ' ' ' ' ' ...

Click on a character (or switch to this window by 'C-x o'
and select the characters by RET) to jump to the place it appears,
where 'C-u C-x =' will give information about it.

Select one of the safe coding systems listed below,
or cancel the writing with C-g and edit the buffer
   to remove or modify the problematic characters,
or specify any other coding system (and risk losing
   the problematic characters).

  raw-text no-conversion

I got only a #...# file written with utf-8, as the Emacs 25 build
I have crashes all the time, and trying to select the minibuffer
frame after the popped-up *Warning* frame grabs the selection just
crashes Emacs.  And that happens now when I try again, with my setup.

I've attached those files from the first try I made for this,
with my setup.  I see now that the `#...#' one has UTF-8 encoding
and the other, `...' without the #s, has encoding `t' in the mode
line, which I guess means raw text.

To report the problem I then used `emacs -Q', and this time I
tried raw text, and I saw the octal escapes.  So I mistakenly
reported that I saw them after specifying both utf-8 and raw text.

>   . If the curved quotes look like raw bytes or, worse, pairs of
>     non-ASCII characters, you need to visit such file like this:
> 
>       C-x RET c utf-8 RET C-x C-f FILE-NAME RET

Users should not have to do that.  I thought they would, like me
have that annoyance.

> > In Emacs, before saving, the buffer looks fine.
> 
> It looks fine, but the encoding mnemonic on the mode line is not "U"
> (which stands for UTF-8), right?  That is why Emacs asks you for
> encoding: it cannot save these characters using your locale's default
> encoding (which is what the *Help* buffer uses by default).

> Yes, because you probably told Emacs to use raw-text or somesuch, when
> it asked.

See above.

> > Do I need to save the buffer using some other encoding?  If so, which?
> 
> Yes, you could tell it to use UTF-8 when it asked.  (After my change,
> Emacs will do that automatically, no questions asked, when saving
> *Help* buffers with curved quotes.)

Sounds good.

> > Emacs proposed two encodings (one of which was raw text, which I tried;
> > and I tried also utf-8, which I would have thought would show curly
> > quotes OK.
> 
> UTF-8 should have worked.  I wouldn't expect you to see octal escapes
> after saving in UTF-8.

See above for, I think, the explanation of what I did and saw.

Thx.

[throw-isearch-help.txt (text/plain, attachment)]

[#throw-isearch-help.txt# (application/octet-stream, attachment)]

Message #22 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 21780 <at> debbugs.gnu.org
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Fri, 30 Oct 2015 09:47:48 +0200

> Date: Thu, 29 Oct 2015 13:53:30 -0700 (PDT)
> From: Drew Adams <drew.adams <at> oracle.com>
> Cc: 21780 <at> debbugs.gnu.org
> 
> I think now that what I did earlier, when specified utf-8, I was
> using my setup.  When I try that again (with my setup), I see the
> problem I reported.

So the question now becomes what do you have in your setup that causes
this.  I'm guessing you do something that changes the defaults for
encoding/decoding text.

> Also, with my setup the *Warning* text is different.  Instead of
> providing lots of possible encoding choices and using
> chinese-iso-8bit as the default (which is what I get with emacs
> -Q - why is that, BTW?), it says:
> 
> Select one of the safe coding systems listed below,
> or cancel the writing with C-g and edit the buffer
>    to remove or modify the problematic characters,
> or specify any other coding system (and risk losing
>    the problematic characters).
> 
>   raw-text no-conversion

What its suggests depend on your customizations.

> The full warning is this, when I use my setup (which uses font
> "-outline-Lucida Console-normal-normal-normal-mono-14-*-*-*-c-*-iso8859-1"):

The font has nothing to do with this.
> >   . If the curved quotes look like raw bytes or, worse, pairs of
> >     non-ASCII characters, you need to visit such file like this:
> > 
> >       C-x RET c utf-8 RET C-x C-f FILE-NAME RET
> 
> Users should not have to do that.

They have been doing that since Emacs 20.1.  This is how you visit a
file whose encoding Emacs cannot guess correctly.  You just didn't
have the pleasure of bumping into this problem until now.

Another possibility is to visit the file normally, see that it wasn't
decoded correctly, then type "C-x RET r utf-8 RET", which will revisit
the file using the specified encoding.

> > UTF-8 should have worked.  I wouldn't expect you to see octal escapes
> > after saving in UTF-8.
> 
> See above for, I think, the explanation of what I did and saw.

It doesn't.  You have something in your customizations that runs afoul
of your expectations (which do work in "emacs -Q").

Message #25 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Drew Adams <drew.adams <at> oracle.com>
Cc: 21780 <at> debbugs.gnu.org
Subject: RE: bug#21780: 25.0.50; Saving *Help* results in bad encoding because
 of curly quotes
Date: Fri, 30 Oct 2015 08:07:54 -0700 (PDT)

> > I think now that what I did earlier, when specified utf-8, I was
> > using my setup.  When I try that again (with my setup), I see the
> > problem I reported.
> 
> So the question now becomes what do you have in your setup that causes
> this.  I'm guessing you do something that changes the defaults for
> encoding/decoding text.

I have tons of stuff in my setup.  Let me know what I might search
for that might affect encoding/decoding.

> > Also, with my setup the *Warning* text is different.  Instead of
> > providing lots of possible encoding choices and using
> > chinese-iso-8bit as the default (which is what I get with emacs
> > -Q - why is that, BTW?), it says:
> >
> > Select one of the safe coding systems listed below,
> > or cancel the writing with C-g and edit the buffer
> >    to remove or modify the problematic characters,
> > or specify any other coding system (and risk losing
> >    the problematic characters).
> >
> >   raw-text no-conversion
> 
> What its suggests depend on your customizations.

Such as?  Again, could it be affected by the default font choice?

> > > UTF-8 should have worked.  I wouldn't expect you to see octal escapes
> > > after saving in UTF-8.
> >
> > See above for, I think, the explanation of what I did and saw.
> 
> It doesn't.  You have something in your customizations that runs afoul
> of your expectations (which do work in "emacs -Q").

Please read again what I said about the crash.  I think it
at least explains why I saw octal escapes when I visited
the last-saved file - it was saved with raw text.  Only the
auto-save file was saved with utf-8.  When I visited the file
in a new session, it showed the octal escapes because it was
saved as raw text.

Why that happened I'm not sure.  Perhaps when I did `C-x C-w'
the *Help* buffer was first saved as raw text (?), then
autosaved as utf-8, then the crash interrupted finally
resaving the file itself as utf-8 (?).

You will need to give me an idea what to look for, if I am to
try hunting for something in my init file and all that it loads.

One thing I do is this, to get Unix-style line endings:
(setq-default buffer-file-coding-system 'undecided-unix)

And I do this:
(setq process-coding-system-alist
      (cons '("bash" . (raw-text-dos . raw-text-unix))
            process-coding-system-alist))

But I don't imagine that either of those is related to this.

Message #28 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 21780 <at> debbugs.gnu.org, drew.adams <at> oracle.com
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Fri, 30 Oct 2015 17:22:26 +0200

> Date: Fri, 30 Oct 2015 08:07:54 -0700 (PDT)
> From: Drew Adams <drew.adams <at> oracle.com>
> Cc: 21780 <at> debbugs.gnu.org
> 
> One thing I do is this, to get Unix-style line endings:
> (setq-default buffer-file-coding-system 'undecided-unix)

That's your problem, mot probably.  Can you try again after removing
it?  If that solves the problem, I can then tell you how to do what
you want without disrupting encoding/decoding defaults.

> And I do this:
> (setq process-coding-system-alist
>       (cons '("bash" . (raw-text-dos . raw-text-unix))
>             process-coding-system-alist))

This is not related, but it is also wrong.  Why do you do that?

Message #31 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21780 <at> debbugs.gnu.org
Subject: RE: bug#21780: 25.0.50; Saving *Help* results in bad encoding because
 of curly quotes
Date: Fri, 30 Oct 2015 09:02:15 -0700 (PDT)

[Message part 1 (text/plain, inline)]

> > (setq-default buffer-file-coding-system 'undecided-unix)
> 
> That's your problem, mot probably.  Can you try again after removing
> it?  If that solves the problem, I can then tell you how to do what
> you want without disrupting encoding/decoding defaults.
> 
> > And I do this:
> > (setq process-coding-system-alist
> >       (cons '("bash" . (raw-text-dos . raw-text-unix))
> >             process-coding-system-alist))
> 
> This is not related, but it is also wrong.  Why do you do that?

Why are these things "wrong"?  I do them as part of the setup
to use Cygwin.  I do them in `setup-cygwin.el', which is,
incidentally, used by quite a few people AFAIK.
http://www.emacswiki.org/emacs/download/setup-cygwin.el

Anyway, I tried commenting out the first of those.  That did
change the text of the *Warning* buffer, so that it mentioned
utf-8 as one of the possibilities.  (Unfortunately, I still
cannot get the file saved, because Emacs crashes.  Again, the
autosave file looks fine in a new session, and shows U(Unix)
in the mode line.)

Attached is a screenshot of the *Warning* text after commenting
out that line.

[throw-emacs-C-x-C-w-help-buf.png (image/png, attachment)]

Message #34 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21780 <at> debbugs.gnu.org
Subject: RE: bug#21780: 25.0.50; Saving *Help* results in bad encoding because
 of curly quotes
Date: Fri, 30 Oct 2015 09:17:58 -0700 (PDT)

> Anyway, I tried commenting out the first of those.  That did
> change the text of the *Warning* buffer, so that it mentioned
> utf-8 as one of the possibilities.  (Unfortunately, I still
> cannot get the file saved, because Emacs crashes.  Again, the
> autosave file looks fine in a new session, and shows U(Unix)
> in the mode line.)
> 
> Attached is a screenshot of the *Warning* text after commenting
> out that line.

Sorry, I was wrong about this and something I said earlier.

Even with the line present, I do not see the problem I thought
I saw earlier, of the *Warning* presenting only two encoding
options, one of which was raw text.  Dunno what build I saw
that in.

The screenshot I just sent is what I see even in the build
I reported about (and it mentions utf-8).  The rest of the
problems are as I reported them.

Message #37 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 21780 <at> debbugs.gnu.org
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Fri, 30 Oct 2015 22:50:57 +0200

> Date: Fri, 30 Oct 2015 09:02:15 -0700 (PDT)
> From: Drew Adams <drew.adams <at> oracle.com>
> Cc: 21780 <at> debbugs.gnu.org
> 
> > > (setq-default buffer-file-coding-system 'undecided-unix)
> > 
> > That's your problem, mot probably.  Can you try again after removing
> > it?  If that solves the problem, I can then tell you how to do what
> > you want without disrupting encoding/decoding defaults.
> > 
> > > And I do this:
> > > (setq process-coding-system-alist
> > >       (cons '("bash" . (raw-text-dos . raw-text-unix))
> > >             process-coding-system-alist))
> > 
> > This is not related, but it is also wrong.  Why do you do that?
> 
> Why are these things "wrong"?

Because they defeat some of the heuristics that decoding and encoding
needs to silently DTRT.  Whoever wrote them didn't understand what she
was doing, and most probably didn't understand what is the problem
that needed to be solved.

If you want to have Unix EOLs by default, the correct customization is
this:

  (setq-default buffer-file-coding-system
                (coding-system-change-eol-conversion
                 buffer-file-coding-system 'unix))

This modifies just the EOL type of the default encoding, leaving the
rest intact.

The other customization, for process-coding-system-alist, is a very
bad idea, if your Bash can sometimes report non-ASCII strings.  AFAIK,
Cygwin nowadays uses UTF-8 as its encoding, so the correct
customization would be to use utf-8 instead of raw-text there.

> I do them as part of the setup to use Cygwin.  I do them in
> `setup-cygwin.el', which is, incidentally, used by quite a few
> people AFAIK.
> http://www.emacswiki.org/emacs/download/setup-cygwin.el

That file needs this fixed ASAP.

> Anyway, I tried commenting out the first of those.  That did
> change the text of the *Warning* buffer, so that it mentioned
> utf-8 as one of the possibilities.  (Unfortunately, I still
> cannot get the file saved, because Emacs crashes.  Again, the
> autosave file looks fine in a new session, and shows U(Unix)
> in the mode line.)

The crash is some separate problem, it doesn't crash for me.

Anyway, this is all tangential to the problem.  After the file is
saved as UTF-8, does visiting it display it correctly, after you
correct your customizations as indicated above?  If not, you will have
to use "C-x RET c" or "C-x RET r", as I mentioned earlier.

Message #40 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: drew.adams <at> oracle.com
Cc: 21780 <at> debbugs.gnu.org
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Fri, 30 Oct 2015 22:57:17 +0200

> Date: Fri, 30 Oct 2015 22:50:57 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 21780 <at> debbugs.gnu.org
> 
> If you want to have Unix EOLs by default, the correct customization is
> this:
> 
>   (setq-default buffer-file-coding-system
>                 (coding-system-change-eol-conversion
>                  buffer-file-coding-system 'unix))

This variant is better:

  (setq-default buffer-file-coding-system
                (coding-system-change-eol-conversion
                 default-buffer-file-coding-system 'unix))

Message #43 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 21780 <at> debbugs.gnu.org
Subject: RE: bug#21780: 25.0.50; Saving *Help* results in bad encoding because
 of curly quotes
Date: Fri, 30 Oct 2015 14:27:43 -0700 (PDT)

> > If you want to have Unix EOLs by default, the correct customization is
> > this:
>   (setq-default buffer-file-coding-system
>                 (coding-system-change-eol-conversion
>                  default-buffer-file-coding-system 'unix))

OK. Thx.

Message #46 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Moreton <andrewjmoreton <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Fri, 30 Oct 2015 23:06:22 +0000

On Fri 30 Oct 2015, Eli Zaretskii wrote:

>> Date: Fri, 30 Oct 2015 22:50:57 +0200
>> From: Eli Zaretskii <eliz <at> gnu.org>
>> Cc: 21780 <at> debbugs.gnu.org
>> 
>> If you want to have Unix EOLs by default, the correct customization is
>> this:
>> 
>>   (setq-default buffer-file-coding-system
>>                 (coding-system-change-eol-conversion
>>                  buffer-file-coding-system 'unix))
>
> This variant is better:
>
>   (setq-default buffer-file-coding-system
>                 (coding-system-change-eol-conversion
>                  default-buffer-file-coding-system 'unix))

Why ? Help for `default-buffer-file-coding-system' says:

  This variable is obsolete since 23.2;
  use ‘buffer-file-coding-system’ instead.

I think Drew's version was fine.

    AndyM

Message #49 received at 21780 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andy Moreton <andrewjmoreton <at> gmail.com>
Cc: 21780 <at> debbugs.gnu.org
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Sat, 31 Oct 2015 09:28:11 +0200

> From: Andy Moreton <andrewjmoreton <at> gmail.com>
> Date: Fri, 30 Oct 2015 23:06:22 +0000
> 
> On Fri 30 Oct 2015, Eli Zaretskii wrote:
> 
> >> Date: Fri, 30 Oct 2015 22:50:57 +0200
> >> From: Eli Zaretskii <eliz <at> gnu.org>
> >> Cc: 21780 <at> debbugs.gnu.org
> >> 
> >> If you want to have Unix EOLs by default, the correct customization is
> >> this:
> >> 
> >>   (setq-default buffer-file-coding-system
> >>                 (coding-system-change-eol-conversion
> >>                  buffer-file-coding-system 'unix))
> >
> > This variant is better:
> >
> >   (setq-default buffer-file-coding-system
> >                 (coding-system-change-eol-conversion
> >                  default-buffer-file-coding-system 'unix))
> 
> Why ? Help for `default-buffer-file-coding-system' says:
> 
>   This variable is obsolete since 23.2;
>   use ‘buffer-file-coding-system’ instead.

Then use (default-value buffer-file-coding-system) instead.  The point
being to use the default value to modify the default value.  When this
code runs, the current buffer should have the same value as its
buffer-local value, but I preferred not to rely on that.

> I think Drew's version was fine.

What Drew's version?  The one that unconditionally used undecided-unix
as the default?  I very much disagree.

Message #52 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Moreton <andrewjmoreton <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#21780: 25.0.50;
 Saving *Help* results in bad encoding because of curly quotes
Date: Sat, 31 Oct 2015 18:10:11 +0000

On Sat 31 Oct 2015, Eli Zaretskii wrote:

>> From: Andy Moreton <andrewjmoreton <at> gmail.com>
>> Date: Fri, 30 Oct 2015 23:06:22 +0000
>> 
>> On Fri 30 Oct 2015, Eli Zaretskii wrote:
>> 
>> >> Date: Fri, 30 Oct 2015 22:50:57 +0200
>> >> From: Eli Zaretskii <eliz <at> gnu.org>
>> >> Cc: 21780 <at> debbugs.gnu.org
>> >> 
>> >> If you want to have Unix EOLs by default, the correct customization is
>> >> this:
>> >> 
>> >>   (setq-default buffer-file-coding-system
>> >>                 (coding-system-change-eol-conversion
>> >>                  buffer-file-coding-system 'unix))
>> >
>> > This variant is better:
>> >
>> >   (setq-default buffer-file-coding-system
>> >                 (coding-system-change-eol-conversion
>> >                  default-buffer-file-coding-system 'unix))
>> 
>> Why ? Help for `default-buffer-file-coding-system' says:
>> 
>>   This variable is obsolete since 23.2;
>>   use ‘buffer-file-coding-system’ instead.
>
> Then use (default-value buffer-file-coding-system) instead.  The point
> being to use the default value to modify the default value.  When this
> code runs, the current buffer should have the same value as its
> buffer-local value, but I preferred not to rely on that.

I see your point. So you prefer this:

(setq-default buffer-file-coding-system
              (coding-system-change-eol-conversion
               (default-value 'buffer-file-coding-system) 'unix))

Thanks for answering this - a useful addition to my init.el.

    AndyM

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.