GNU bug report logs - #876
^M in some Windows info files because of null-byte detection

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 876 in the body.
You can then email your comments to 876 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Juanma Barranquero" <lekktu <at> gmail.com>
To: "Emacs Bug Tracker" <submit <at> debbugs.gnu.org>
Subject: ^M in some Windows info files because of null-byte detection
Date: Wed, 3 Sep 2008 18:39:11 +0200

Some info files in Windows (for example, the Emacs FAQ) are displayed
with spurious ^M characters because of recent changes in null byte
detection.

Kenichi Handa proposed a new inhibit-null-byte-detection variable.
Stefan Monnier counter-proposed inhibit-coding-systems.


emacs-devel discussion:

  http://lists.gnu.org/archive/html/emacs-devel/2008-06/msg00739.html
  http://lists.gnu.org/archive/html/emacs-devel/2008-07/msg00215.html
  http://lists.gnu.org/archive/html/emacs-devel/2008-07/msg00225.html
  http://lists.gnu.org/archive/html/emacs-devel/2008-08/msg00293.html

Message #22 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: 876-done <at> debbugs.gnu.org
Cc: emacs-devel <at> gnu.org, bug-gnu-emacs <at> gnu.org, emacs-pretest-bug <at> gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 13:15:51 +0200

> Date: Sat, 29 Nov 2008 00:39:27 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: lekktu <at> gmail.com, emacs-devel <at> gnu.org, monnier <at> iro.umontreal.ca,
> 	handa <at> m17n.org
> 
> > From: "Drew Adams" <drew.adams <at> oracle.com>
> > Date: Fri, 28 Nov 2008 13:28:34 -0800
> > Cc: 'Juanma Barranquero' <lekktu <at> gmail.com>, emacs-devel <at> gnu.org
> > 
> > Whatever happened to this thread and the associated bugs: #876, #1117, #1284?
> > 
> > It sounds like there were alternative proposals about how to fix this, but there
> > was no discussion to try to reach a consensus or a decision. Is that where
> > things were left?
> > 
> > Meanwhile, it's still impossible to use the index in Info manuals on Windows,
> > and it's impossible to use some manuals (e.g. Viper) at all.
> 
> Don't worry, this will get fixed before Emacs 23 is ready for release.
> 
> I will work on it soon if no one beats me to it.

(Fore some value of "soon", sorry.)

I fixed this bug.

There were two suggestions for how to fix this: one by Handa-san in
this message:

  http://lists.gnu.org/archive/html/emacs-devel/2008-08/msg00293.html

followed by a tentative patch by Juanma here:

  http://lists.gnu.org/archive/html/emacs-devel/2008-08/msg00316.html

The other suggestion was by Stefan:

  http://lists.gnu.org/archive/html/emacs-devel/2008-08/msg00373.html

I decided I liked the first alternative better, since it has much
more local effect than the other one.  What Stefan suggested implied
messing with coding priorities, and I didn't feel that was TRT at this
late stage in Emacs 23.1 development.

The changes I installed are more thorough than what Juanma posted:
they change both detect_coding and detect_coding_system, and also bind
inhibit-null-byte-detection in a couple more places in info.el.

The result was tested on GNU/Linux, MS-Windows, and MS-DOS, both with
compressed and uncompressed Info files, with auto-compression-mode
both on and off.

Message #38 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Juanma Barranquero" <lekktu <at> gmail.com>
To: "Eli Zaretskii" <eliz <at> gnu.org>
Cc: 876-done <at> debbugs.gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 13:29:53 +0100

On Sat, Jan 10, 2009 at 12:15, Eli Zaretskii <eliz <at> gnu.org> wrote:

> followed by a tentative patch by Juanma here:
>
>  http://lists.gnu.org/archive/html/emacs-devel/2008-08/msg00316.html

[...]

> The changes I installed are more thorough than what Juanma posted:
> they change both detect_coding and detect_coding_system, and also bind
> inhibit-null-byte-detection in a couple more places in info.el.

Yeah, my patch was just proof-of-concept, and unfinished. Thanks for
fixing this!

    Juanma

Message #39 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Juanma Barranquero" <lekktu <at> gmail.com>
To: "Eli Zaretskii" <eliz <at> gnu.org>
Cc: 876-done <at> debbugs.gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 15:07:37 +0100

On Sat, Jan 10, 2009 at 12:15, Eli Zaretskii <eliz <at> gnu.org> wrote:

> I fixed this bug.

If the info files are compressed with gzip (using the Windows binary
from http://www.gzip.org), when visited from Info they still have
spurious ^M.

As an aside, when I use MSYS' gzip I get an error in decompressing:

  Error while executing "gzip -c -q -d < c:/emacs/info/efaq.gz"
  gzip: stdin: invalid compressed data--crc error

Could be related to CRLF issues in the redirection (but this is a gzip
problem, not Emacs', of course).

    Juanma

Message #40 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juanma Barranquero <lekktu <at> gmail.com>
Cc: 876-done <at> debbugs.gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 17:13:56 +0200

> Date: Sat, 10 Jan 2009 15:07:37 +0100
> From: "Juanma Barranquero" <lekktu <at> gmail.com>
> Cc: 876-done <at> emacsbugs.donarmstrong.com, emacs-devel <at> gnu.org
> 
> On Sat, Jan 10, 2009 at 12:15, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> > I fixed this bug.
> 
> If the info files are compressed with gzip (using the Windows binary
> from http://www.gzip.org), when visited from Info they still have
> spurious ^M.

I cannot reproduce this (tried with compressing info/emacs*).  Please
send a complete self-contained recipe.

Does "spurious" mean that every line has a ^M, like you saw before the
fix, or just some lines?

Also, does the above reference to http://www.gzip.org means that files
compressed by other versions of gzip do work as intended?

> As an aside, when I use MSYS' gzip I get an error in decompressing:
> 
>   Error while executing "gzip -c -q -d < c:/emacs/info/efaq.gz"
>   gzip: stdin: invalid compressed data--crc error
> 
> Could be related to CRLF issues in the redirection (but this is a gzip
> problem, not Emacs', of course).

Right, and I don't have MSYS anyway.

Message #41 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Eli Zaretskii'" <eliz <at> gnu.org>, <876-done <at> debbugs.gnu.org>
Cc: <emacs-pretest-bug <at> gnu.org>, <bug-gnu-emacs <at> gnu.org>,
        <emacs-devel <at> gnu.org>
Subject: RE: ^M in the info files
Date: Sat, 10 Jan 2009 08:21:34 -0800

Thanks for fixing this. 

Message #42 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Juanma Barranquero" <lekktu <at> gmail.com>
To: "Eli Zaretskii" <eliz <at> gnu.org>
Cc: 876-done <at> debbugs.gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 19:23:28 +0100

On Sat, Jan 10, 2009 at 16:13, Eli Zaretskii <eliz <at> gnu.org> wrote:

> I cannot reproduce this (tried with compressing info/emacs*).  Please
> send a complete self-contained recipe.

Sorry, apparently it depends on something in my .emacs. It's still a
bug, though.

Try with

  cd info
  gzip efaq
  emacs -Q -eval "(progn (set-language-environment \"UTF-8\") (info
\"(efaq)Top\"))"

> Does "spurious" mean that every line has a ^M, like you saw before the
> fix, or just some lines?

Every line, just like before the fix.

> Also, does the above reference to http://www.gzip.org means that files
> compressed by other versions of gzip do work as intended?

Means that using the gzip from www.gzip.org it fails, and using the
one from MSYS I get the error I reported. I don't have other versions
of gzip.

For additional fun,

  cd info
  gzip emacs
  emacs -Q -eval "(info \"(emacs)Top\")"
   =>
  gzip: illegal option -- Q
  usage: gzip [-acdfhlLnNrtvV19] [-S suffix] [file ...

(this with the standard www.gzip.org version).

    Juanma

Message #43 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juanma Barranquero <lekktu <at> gmail.com>
Cc: 876-done <at> debbugs.gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 21:08:02 +0200

Message #44 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juanma Barranquero <lekktu <at> gmail.com>
Cc: 876-done <at> debbugs.gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 21:12:06 +0200

> Date: Sat, 10 Jan 2009 19:23:28 +0100
> From: "Juanma Barranquero" <lekktu <at> gmail.com>
> Cc: 876-done <at> emacsbugs.donarmstrong.com, emacs-devel <at> gnu.org
> 
> On Sat, Jan 10, 2009 at 16:13, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> > I cannot reproduce this (tried with compressing info/emacs*).  Please
> > send a complete self-contained recipe.
> 
> Sorry, apparently it depends on something in my .emacs. It's still a
> bug, though.
> 
> Try with
> 
>   cd info
>   gzip efaq
>   emacs -Q -eval "(progn (set-language-environment \"UTF-8\") (info \"(efaq)Top\"))"

Yes, I see that now, but it has something to do with UTF-8 as the
language environment (it works correctly without it).  The Info buffer
has utf-8-unix as its buffer-file-coding-system, which is another
sign.  So I think this is a separate bug.  Please file a separate bug
report for it.

Message #45 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juanma Barranquero <lekktu <at> gmail.com>
Cc: 876-done <at> debbugs.gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 21:19:22 +0200

> Date: Sat, 10 Jan 2009 19:23:28 +0100
> From: "Juanma Barranquero" <lekktu <at> gmail.com>
> Cc: 876-done <at> emacsbugs.donarmstrong.com, emacs-devel <at> gnu.org
> 
>   cd info
>   gzip emacs
>   emacs -Q -eval "(info \"(emacs)Top\")"
>    =>
>   gzip: illegal option -- Q
>   usage: gzip [-acdfhlLnNrtvV19] [-S suffix] [file ...
> 
> (this with the standard www.gzip.org version).

Doesn't happen to me, neither with gzip 1.2.4 from gzip.org, nor with
v1.3.5 from GnuWin32.  I did replace "emacs" with a full file name,
though, to make sure it doesn't pick up some other version somewhere
on my INFOPATH.

Message #48 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Juanma Barranquero" <lekktu <at> gmail.com>
To: "Eli Zaretskii" <eliz <at> gnu.org>
Cc: 876-done <at> debbugs.gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Sat, 10 Jan 2009 22:04:18 +0100

On Sat, Jan 10, 2009 at 20:19, Eli Zaretskii <eliz <at> gnu.org> wrote:

>>   gzip: illegal option -- Q
>>   usage: gzip [-acdfhlLnNrtvV19] [-S suffix] [file ...
>>
>> (this with the standard www.gzip.org version).

My mistake. My command interpreter was trying to execute emacs.gz.
Sorry for the noise.

    Juanma

Message #49 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 876-done <at> debbugs.gnu.org, emacs-pretest-bug <at> gnu.org,
        bug-gnu-emacs <at> gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Mon, 12 Jan 2009 15:54:38 -0500

Thank you, Eli,

> I decided I liked the first alternative better, since it has much
> more local effect than the other one.  What Stefan suggested implied
> messing with coding priorities, and I didn't feel that was TRT at this
> late stage in Emacs 23.1 development.

[ Obviously my suggestion requires more work to implement, but.. ]
Hmm...curious, why does it involve coding priorities?


        Stefan

Message #52 received at 876-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 876-done <at> debbugs.gnu.org, emacs-pretest-bug <at> gnu.org,
        bug-gnu-emacs <at> gnu.org, emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Tue, 13 Jan 2009 00:13:04 +0200

> X-Spam-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_00,
> 	FORGED_RCVD_HELO,SPF_SOFTFAIL autolearn=no version=3.1.0
> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: 876-done <at> emacsbugs.donarmstrong.com,  emacs-pretest-bug <at> gnu.org,  bug-gnu-emacs <at> gnu.org,  emacs-devel <at> gnu.org
> Date: Mon, 12 Jan 2009 15:54:38 -0500
> 
> Hmm...curious, why does it involve coding priorities?

You suggested to introduce a way of controlling which possible
encodings will _not_ be acceptable by the caller of
detect-coding-region/string.  These primitives work by scanning the
coding_priorities[] array and checking them against those categories
that has been rejected based on the text in the region/string.  See
detect_coding and detect_coding_system.

Message #57 received at 876 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 876 <at> debbugs.gnu.org,
        emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Mon, 12 Jan 2009 17:27:03 -0500

Please stop cc'ing emacs-pretest-bug <at> gnu.org and bug-gnu-emacs <at> gnu.org
in this discussion, it messes up the bug tracker.

Message #64 received at 876 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: monnier <at> iro.umontreal.ca, 876 <at> debbugs.gnu.org,
        emacs-devel <at> gnu.org
Subject: Re: ^M in the info files
Date: Tue, 13 Jan 2009 06:01:23 +0200

> From: Glenn Morris <rgm <at> gnu.org>
> Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>, 876 <at> emacsbugs.donarmstrong.com,  emacs-devel <at> gnu.org
> Date: Mon, 12 Jan 2009 17:27:03 -0500
> 
> 
> Please stop cc'ing emacs-pretest-bug <at> gnu.org and bug-gnu-emacs <at> gnu.org
> in this discussion, it messes up the bug tracker.

Why won't the tracker get its act together and stop forcing us into
editing mail headers?  I use the Rmail's `r' command, which replies to
all the original "To" and "CC" addressees.  With the amounts of mail
I'm responding to, I cannot afford editing headers of every message I
send.  Even if I try, I will surely sometimes forget.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.