GNU bug report logs - #2416
23.0.60; decode-coding-region

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 2416 in the body.
You can then email your comments to 2416 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: mj <mj54590 <at> gmail.com>
To: emacs-pretest-bug <at> gnu.org
Subject: 23.0.60; decode-coding-region
Date: Fri, 20 Feb 2009 13:13:01 -0800 (PST)

I have been having this problem since I switched to Emacs 23 several
weeks ago. I'm using VM to read my mails. There seems to be a problem
in decode-coding-region when VM tries to decode a string. When VM
tries to decode a region or a string, it uses a temporary buffer and
basically runs the following lisp code:

(apply 'decode-coding-region (point-min) (point-max)  'us-ascii nil)

The original buffer  content would be something like this:

B7040400-12
some text here

after decode-coding-region is executed, the buffer content became:

^@7040450-12
some text here

Where ^@ is actually binary code \0 (not ascii ^ and @). There is another instance
that a string was decoded and the result is  ^@ prefixed. 

I could not reproduce this with "Emacs -Q". But it always happens when
thsoe particular messages were processed by VM. 

Strangely enough, if I inserted a few spaces at the beginning of
buffer: (one space in the following buffer)

 B7040400-12
some text here

And, the decoding was done correctly. In another instance mentioned
above, one space is not enough. I had to put several spaces to get the
decoding working. 

I saw another bug report just yesterday regarding decode-coding-region
crashing. I applied the patch, but it did not help in the
decoding. 

Please let me know if you need other information to help understand
the problem. Thanks. 

-----
Emacs version: "GNU Emacs 23.0.60.1 (i386-mingw-nt5.1.2600) of 2009-01-29 on T42"

Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4) --cflags -I../../GnuWin32/include'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: chinese-big5
  default-enable-multibyte-characters: t

Major mode: Shell

Minor modes in effect:
  auto-image-file-mode: t
  shell-dirtrack-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Message #10 received at 2416 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: mj <mj54590 <at> gmail.com>, 2416 <at> debbugs.gnu.org
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sat, 21 Feb 2009 11:16:11 +0200

> Date: Fri, 20 Feb 2009 13:13:01 -0800 (PST)
> From: mj <mj54590 <at> gmail.com>
> Cc: 
> 
> I have been having this problem since I switched to Emacs 23 several
> weeks ago. I'm using VM to read my mails. There seems to be a problem
> in decode-coding-region when VM tries to decode a string. When VM
> tries to decode a region or a string, it uses a temporary buffer and
> basically runs the following lisp code:
> 
> (apply 'decode-coding-region (point-min) (point-max)  'us-ascii nil)
> 
> The original buffer  content would be something like this:
> 
> B7040400-12
> some text here
> 
> after decode-coding-region is executed, the buffer content became:
> 
> ^@7040450-12
> some text here
> 
> Where ^@ is actually binary code \0 (not ascii ^ and @). There is another instance
> that a string was decoded and the result is  ^@ prefixed. 
> 
> I could not reproduce this with "Emacs -Q". But it always happens when
> thsoe particular messages were processed by VM. 

Could you please see if the problem still persists in the current CVS?
Your Emacs seems to be about a month old (Jan 29), and a couple of
related bugs were fixed in coding.c since then.

Message #15 received at 2416 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: MJ <mj54590 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 2416 <at> debbugs.gnu.org
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sat, 21 Feb 2009 08:20:12 -0500

[Message part 1 (text/plain, inline)]

Thanks for the reply. I just tried with the latest CVS version and the
problem still persists:

"GNU Emacs 23.0.90.1 (i386-mingw-nt5.1.2600) of 2009-02-21 on T42"

On Sat, Feb 21, 2009 at 4:16 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> > Date: Fri, 20 Feb 2009 13:13:01 -0800 (PST)
> > From: mj <mj54590 <at> gmail.com>
> > Cc:
> >
> > I have been having this problem since I switched to Emacs 23 several
> > weeks ago. I'm using VM to read my mails. There seems to be a problem
> > in decode-coding-region when VM tries to decode a string. When VM
> > tries to decode a region or a string, it uses a temporary buffer and
> > basically runs the following lisp code:
> >
> > (apply 'decode-coding-region (point-min) (point-max)  'us-ascii nil)
> >
> > The original buffer  content would be something like this:
> >
> > B7040400-12
> > some text here
> >
> > after decode-coding-region is executed, the buffer content became:
> >
> > ^@7040450-12
> > some text here
> >
> > Where ^@ is actually binary code \0 (not ascii ^ and @). There is another
> instance
> > that a string was decoded and the result is  ^@ prefixed.
> >
> > I could not reproduce this with "Emacs -Q". But it always happens when
> > thsoe particular messages were processed by VM.
>
> Could you please see if the problem still persists in the current CVS?
> Your Emacs seems to be about a month old (Jan 29), and a couple of
> related bugs were fixed in coding.c since then.
>

[Message part 2 (text/html, inline)]

Message #20 received at 2416 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: mj <mj54590 <at> gmail.com>
Cc: 2416 <at> debbugs.gnu.org
Subject: Re: 23.0.60; decode-coding-region
Date: Sat, 21 Feb 2009 12:21:46 -0500

Could you try to narrow down the bug, so that it can be reproduced?  One
way would be to copy out buffer contents before decoding into a separate
buffer, and see if decode-coding-region misbehaves when applied manually
to that bufer.

Message #25 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Juanma Barranquero <lekktu <at> gmail.com>
To: mj <mj54590 <at> gmail.com>, 2416 <at> debbugs.gnu.org
Cc: emacs-pretest-bug <at> gnu.org
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sun, 22 Feb 2009 03:47:03 +0100

On Fri, Feb 20, 2009 at 22:13, mj <mj54590 <at> gmail.com> wrote:

> (apply 'decode-coding-region (point-min) (point-max)  'us-ascii nil)
>
> The original buffer  content would be something like this:
>
> B7040400-12
> some text here
>
> after decode-coding-region is executed, the buffer content became:
>
> ^@7040450-12
> some text here
>
> Where ^@ is actually binary code \0 (not ascii ^ and @). There is another instance
> that a string was decoded and the result is  ^@ prefixed.

Could it be related to bug#1809?

(with-temp-buffer
  (insert (make-string 20 ?.))
  (decode-coding-region 1 (point-max) 'raw-text)
  (buffer-string))

=> "^@..................."

    Juanma

Message #35 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: MJ <mj54590 <at> gmail.com>
To: Juanma Barranquero <lekktu <at> gmail.com>
Cc: 2416 <at> debbugs.gnu.org, emacs-pretest-bug <at> gnu.org,
        Chong Yidong <cyd <at> stupidchicken.com>, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sun, 22 Feb 2009 00:07:33 -0500

[Message part 1 (text/plain, inline)]

Juanma, thank you for the lisp code that reproduces the same problem that I
am having.

If a space is inserted at the beginning of the buffer, then the result is
correct (as stated in my bug report):

(with-temp-buffer
 (insert " ")
 (insert (make-string 20 ?.))
 (decode-coding-region 2 (point-max) 'us-ascii)
 (buffer-string))
" ...................."

(I use 'us-ascii just to show the coding does not matter).

Now, hopefully emacs developers will  be able to understand and fix the
problem.


On Sat, Feb 21, 2009 at 9:47 PM, Juanma Barranquero <lekktu <at> gmail.com>wrote:

> On Fri, Feb 20, 2009 at 22:13, mj <mj54590 <at> gmail.com> wrote:
>
> > (apply 'decode-coding-region (point-min) (point-max)  'us-ascii nil)
> >
> > The original buffer  content would be something like this:
> >
> > B7040400-12
> > some text here
> >
> > after decode-coding-region is executed, the buffer content became:
> >
> > ^@7040450-12
> > some text here
> >
> > Where ^@ is actually binary code \0 (not ascii ^ and @). There is another
> instance
> > that a string was decoded and the result is  ^@ prefixed.
>
> Could it be related to bug#1809?
>
> (with-temp-buffer
>  (insert (make-string 20 ?.))
>  (decode-coding-region 1 (point-max) 'raw-text)
>  (buffer-string))
>
> => "^@..................."
>
>    Juanma
>

[Message part 2 (text/html, inline)]

Message #45 received at 2416 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Juanma Barranquero <lekktu <at> gmail.com>
To: MJ <mj54590 <at> gmail.com>
Cc: 2416 <at> debbugs.gnu.org, Chong Yidong <cyd <at> stupidchicken.com>,
        Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sun, 22 Feb 2009 06:13:59 +0100

On Sun, Feb 22, 2009 at 06:07, MJ <mj54590 <at> gmail.com> wrote:

> If a space is inserted at the beginning of the buffer, then the result is
> correct (as stated in my bug report):

If you look at #1809, you'll see that the problem seems to be
sensitive to string length. Assuming you're using ASCII, do you see it
with strings of length != 20?

    Juanma

Message #50 received at 2416 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: MJ <mj54590 <at> gmail.com>
To: Juanma Barranquero <lekktu <at> gmail.com>
Cc: 2416 <at> debbugs.gnu.org, Chong Yidong <cyd <at> stupidchicken.com>,
        Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sun, 22 Feb 2009 00:23:31 -0500

[Message part 1 (text/plain, inline)]

Indeed, including newline, the buffer length is exactly 20 bytes in two
cases where they are ascii. In another case, it's BIG5 Chinese (each BIG5
code is two bytes), and the total is also 20 bytes. Thanks for pointing it
out.

On Sun, Feb 22, 2009 at 12:13 AM, Juanma Barranquero <lekktu <at> gmail.com>wrote:

> On Sun, Feb 22, 2009 at 06:07, MJ <mj54590 <at> gmail.com> wrote:
>
> > If a space is inserted at the beginning of the buffer, then the result is
> > correct (as stated in my bug report):
>
> If you look at #1809, you'll see that the problem seems to be
> sensitive to string length. Assuming you're using ASCII, do you see it
> with strings of length != 20?
>
>    Juanma
>

[Message part 2 (text/html, inline)]

Message #55 received at 2416 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Juanma Barranquero <lekktu <at> gmail.com>
To: MJ <mj54590 <at> gmail.com>
Cc: 2416 <at> debbugs.gnu.org, Chong Yidong <cyd <at> stupidchicken.com>,
        Eli Zaretskii <eliz <at> gnu.org>, control <at> debbugs.gnu.org
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sun, 22 Feb 2009 06:44:35 +0100

merge 1809 2416
quit

> Indeed, including newline, the buffer length is exactly 20 bytes in two
> cases where they are ascii. In another case, it's BIG5 Chinese (each BIG5
> code is two bytes), and the total is also 20 bytes. Thanks for pointing it
> out.

OK, so this is #1809 then. I'm merging the bugs.

  Juanma

Message #62 received at 2416 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Juanma Barranquero <lekktu <at> gmail.com>
To: MJ <mj54590 <at> gmail.com>
Cc: 2416 <at> debbugs.gnu.org, Chong Yidong <cyd <at> stupidchicken.com>,
        Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sun, 22 Feb 2009 07:07:41 +0100

On Sun, Feb 22, 2009 at 06:13, Juanma Barranquero <lekktu <at> gmail.com> wrote:

> If you look at #1809, you'll see that the problem seems to be
> sensitive to string length. Assuming you're using ASCII, do you see it
> with strings of length != 20?

A clue: the bug is sensitive to the value of the BUF_GAP_SIZE set in
`get-buffer-create', at buffer.c:364.

If you change

  BUF_GAP_SIZE (b) = 20;

to another size, so does the bug's triggering length.

    Juanma

Message #67 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Juanma Barranquero <lekktu <at> gmail.com>
Cc: 2416 <at> debbugs.gnu.org, mj <mj54590 <at> gmail.com>,
        emacs-pretest-bug <at> gnu.org
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sun, 22 Feb 2009 15:31:33 +0100

Juanma Barranquero <lekktu <at> gmail.com> writes:

> Could it be related to bug#1809?
>
> (with-temp-buffer
>   (insert (make-string 20 ?.))
>   (decode-coding-region 1 (point-max) 'raw-text)
>   (buffer-string))
>
> => "^@..................."

I've installed a fix.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab <at> suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #77 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: MJ <mj54590 <at> gmail.com>
To: Andreas Schwab <schwab <at> suse.de>
Cc: Juanma Barranquero <lekktu <at> gmail.com>, 2416 <at> debbugs.gnu.org,
        emacs-pretest-bug <at> gnu.org
Subject: Re: bug#2416: 23.0.60; decode-coding-region
Date: Sun, 22 Feb 2009 21:23:05 -0500

[Message part 1 (text/plain, inline)]

Andreas, thank you for the quick fix, which indeed solves the problem I was
having. -- mj

On Sun, Feb 22, 2009 at 9:31 AM, Andreas Schwab <schwab <at> suse.de> wrote:

> Juanma Barranquero <lekktu <at> gmail.com> writes:
>
> > Could it be related to bug#1809?
> >
> > (with-temp-buffer
> >   (insert (make-string 20 ?.))
> >   (decode-coding-region 1 (point-max) 'raw-text)
> >   (buffer-string))
> >
> > => "^@..................."
>
> I've installed a fix.
>
> Andreas.
>
> --
> Andreas Schwab, SuSE Labs, schwab <at> suse.de
> SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
> PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
> "And now for something completely different."
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.