GNU bug report logs - #1187
23.0.60; Cannot read vline.el - invalid read syntax

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 1187 in the body.
You can then email your comments to 1187 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: <emacs-pretest-bug <at> gnu.org>
Subject: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Thu, 16 Oct 2008 14:22:45 -0700

[Message part 1 (text/plain, inline)]

Library vline.el can be read fine in Emacs 22, but Emacs 23 raises an
error, saying Invalid read syntax: "?".
 

In GNU Emacs 23.0.60.1 (i386-mingw-nt5.1.2600)
 of 2008-10-03 on LENNART-69DE564
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4) --no-opt --cflags -Ic:/g/include
-fno-crossjumping'
 
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: cp1252
  default-enable-multibyte-characters: t
 
Major mode: Emacs-Lisp
 
Minor modes in effect:
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
 
Recent input:
C-x C-f c o n r r <backspace> <backspace> t r i b / 
v l i n e <tab> <return> M-x l o a d - f <return> <return> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <menu-bar> <help-menu> 
<send-emacs-bug-report>
 
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Loading c:/drews-lisp-20/CONTRIB/vline.el (source)...
load-with-code-conversion: Invalid read syntax: "?"

[Message part 2 (text/html, inline)]

[vline.el (application/octet-stream, attachment)]

Message #10 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lennart Borgman (gmail)" <lennart.borgman <at> gmail.com>
To: Drew Adams <drew.adams <at> oracle.com>, 1187 <at> debbugs.gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 01:41:04 +0200

Drew Adams wrote:
> Library vline.el can be read fine in Emacs 22, but Emacs 23 raises an
> error, saying Invalid read syntax: "?".

Just for info: I had some similar problems and decided to set

 current-language-environment to "UTF-8"

Message #15 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Lennart Borgman \(gmail\)'" <lennart.borgman <at> gmail.com>,
        <1187 <at> debbugs.gnu.org>
Subject: RE: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Thu, 16 Oct 2008 17:00:40 -0700

> From: Lennart Borgman Sent: Thursday, October 16, 2008 4:41 PM
> > Library vline.el can be read fine in Emacs 22, but Emacs 23 
> > raises an error, saying Invalid read syntax: "?".
> 
> Just for info: I had some similar problems and decided to set
>  current-language-environment to "UTF-8"

OK, thanks for the workaround.

But that should not be necessary. In Emacs 22, it just DTRT. Users should be
able to load the file in both Emacs versions without fiddling with the language
environment.

Message #20 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>, 1187 <at> debbugs.gnu.org
Cc: emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 10:15:28 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Date: Thu, 16 Oct 2008 14:22:45 -0700
> Cc: 
> 
> Library vline.el can be read fine in Emacs 22, but Emacs 23 raises an
> error, saying Invalid read syntax: "?".

I cannot reproduce this, perhaps because when I saved the vline.el you
attached, I selected a wrong encoding.  Please visit the file with
"M-x find-file-literally" and tell me what 8-bit bytes you see on the
line that begins with "((memq char '(?\t".  Then visit that file
normally with Emacs 22 and tell what non-ASCII character(s) you see on
that line (use "C-u C-x =" to describe those characters).

Message #30 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>, 1187 <at> debbugs.gnu.org
Cc: lennart.borgman <at> gmail.com, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 10:18:01 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Date: Thu, 16 Oct 2008 17:00:40 -0700
> Cc: 
> 
> > Just for info: I had some similar problems and decided to set
> >  current-language-environment to "UTF-8"
> 
> OK, thanks for the workaround.
> 
> But that should not be necessary. In Emacs 22, it just DTRT. Users should be
> able to load the file in both Emacs versions without fiddling with the language
> environment.

But that could be a problem with vline.el itself, you know.  It uses
non-ASCII characters, but does not include any "coding:" cookies, so
Emacs is left with its guesswork for how to interpret the 8-bit bytes
included in the file.  And that guesswork is not fool-proof.

I will look at this closer after you report the details about the
bytes that I requested in my other message.

Message #45 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lennart Borgman (gmail)" <lennart.borgman <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Drew Adams <drew.adams <at> oracle.com>, 1187 <at> debbugs.gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 10:36:47 +0200

Eli Zaretskii wrote:
>> From: "Drew Adams" <drew.adams <at> oracle.com>
>> Date: Thu, 16 Oct 2008 17:00:40 -0700
>> Cc: 
>>
>>> Just for info: I had some similar problems and decided to set
>>>  current-language-environment to "UTF-8"
>> OK, thanks for the workaround.
>>
>> But that should not be necessary. In Emacs 22, it just DTRT. Users should be
>> able to load the file in both Emacs versions without fiddling with the language
>> environment.
> 
> But that could be a problem with vline.el itself, you know.  It uses
> non-ASCII characters, but does not include any "coding:" cookies, so
> Emacs is left with its guesswork for how to interpret the 8-bit bytes
> included in the file.  And that guesswork is not fool-proof.
> 
> I will look at this closer after you report the details about the
> bytes that I requested in my other message.

I do not remember; could the defaults be changed so that reading files
like vline.el will succeed?

Message #50 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Lennart Borgman (gmail)" <lennart.borgman <at> gmail.com>
Cc: drew.adams <at> oracle.com, 1187 <at> debbugs.gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 11:57:05 +0200

> Date: Fri, 17 Oct 2008 10:36:47 +0200
> From: "Lennart Borgman (gmail)" <lennart.borgman <at> gmail.com>
> CC: Drew Adams <drew.adams <at> oracle.com>, 1187 <at> emacsbugs.donarmstrong.com
> 
> Eli Zaretskii wrote:
> >> From: "Drew Adams" <drew.adams <at> oracle.com>
> >> Date: Thu, 16 Oct 2008 17:00:40 -0700
> >> Cc: 
> >>
> >>> Just for info: I had some similar problems and decided to set
> >>>  current-language-environment to "UTF-8"
> >> OK, thanks for the workaround.
> >>
> >> But that should not be necessary. In Emacs 22, it just DTRT. Users should be
> >> able to load the file in both Emacs versions without fiddling with the language
> >> environment.
> > 
> > But that could be a problem with vline.el itself, you know.  It uses
> > non-ASCII characters, but does not include any "coding:" cookies, so
> > Emacs is left with its guesswork for how to interpret the 8-bit bytes
> > included in the file.  And that guesswork is not fool-proof.
> > 
> > I will look at this closer after you report the details about the
> > bytes that I requested in my other message.
> 
> I do not remember; could the defaults be changed so that reading files
> like vline.el will succeed?

Obviously, that depends on the nature of the problem, which at least
to me is not yet clear.

Message #55 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Lawrence Mitchell <wence <at> gmx.li>
To: bug-gnu-emacs <at> gnu.org
Cc: emacs-pretest-bug <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 13:30:24 +0100

Drew Adams wrote:
> Library vline.el can be read fine in Emacs 22, but Emacs 23 raises an
> error, saying Invalid read syntax: "?".

The following sequence of events works for me in

GNU Emacs 23.0.60.4 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll
bars) of 2008-10-07 on lamacq.ph.ed.ac.uk

$ wget http://www.emacswiki.org/cgi-bin/emacs/download/vline.el \
-O /tmp/vline.el

$ LANG=C emacs -Q --eval '(load-file "/tmp/vline.el")'
LANG=C so that current-language-environment is "ASCII"
and default-buffer-file-coding-system is nil.

Successfully prints:
Loading /tmp/vline.el (source)...done

> In GNU Emacs 23.0.60.1 (i386-mingw-nt5.1.2600)
>  of 2008-10-03 on LENNART-69DE564
> Windowing system distributor `Microsoft Corp.', version 5.1.2600
> configured using `configure --with-gcc (3.4) --no-opt --cflags -Ic:/g/include
> -fno-crossjumping'

> Important settings:
>   value of $LC_ALL: nil
>   value of $LC_COLLATE: nil
>   value of $LC_CTYPE: nil
>   value of $LC_MESSAGES: nil
>   value of $LC_MONETARY: nil
>   value of $LC_NUMERIC: nil
>   value of $LC_TIME: nil
>   value of $LANG: ENU
                    ^^^ this seems to be the culprit.

LANG=ENU is an unknown locale here but if LANG=en_GB such that
default-buffer-file-coding-system is iso-latin-1-unix then the
section of the file that causes problems is:
(memq char '(?\t ?　))

where the second character is:

|         character: 　 (12288, #o30000, #x3000)
| preferred charset: unicode (Unicode (ISO10646))
|        code point: 0x3000
|            syntax: _ 	which means: symbol
|          category: c:Chinese h:Korean j:Japanese
| 		   |:While filling, we can break a line at this character.
|       buffer code: #xE3 #x80 #x80
|         file code: #xE3 #x80 #x80 (encoded by coding system utf-8-unix)
|           display: by this font (glyph code)
|     xft:-unknown-Kochi Gothic-normal-normal-normal-*-15-*-*-*-*-0-iso10646-1 (#x6F3)

| Character code properties: customize what to show
|   name: IDEOGRAPHIC SPACE
|   general-category: Zs (Separator, Space)
|   decomposition: (<wide> 32) (<wide> ' ')


The difference between emacs-21.3 and emacs 23 appears to be that
the former ignores the two extra bytes when reading in latin-1
but emacs 23 does not:

$ od -c /tmp/f
0000000   ? 343 200 200
0000004

Emacs 23:

(let ((coding-system-for-read 'iso-latin-1)
      ret)
  (with-temp-buffer
    (insert-file-contents "/tmp/f")
    (setq ret (read (current-buffer))))
  (insert ret))

=>

(invalid-read-syntax "?")
  read(#<buffer  *temp*>)

----
(let ((coding-system-for-read 'utf-8)
      ret)
  (with-temp-buffer
    (insert-file-contents "/tmp/f")
    (setq ret (read (current-buffer))))
  (insert ret))

=>

IDEOGRAPHIC SPACE is inserted correctly


Emacs 21.3

(let ((coding-system-for-read 'iso-latin-1)
      ret)
  (with-temp-buffer
    (insert-file-contents "/tmp/f")
    (setq ret (read (current-buffer))))
  (insert ret))

=>

ã ; LATIN SMALL LETTER A WITH TILDE is inserted.

-----
(let ((coding-system-for-read 'utf-8)
      ret)
  (with-temp-buffer
    (insert-file-contents "/tmp/f")
    (setq ret (read (current-buffer))))
  (insert ret))

=>

IDEOGRAPHIC SPACE is inserted correctly
-- 
Lawrence Mitchell <wence <at> gmx.li>

Message #65 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lawrence Mitchell <wence <at> gmx.li>, 1187 <at> debbugs.gnu.org
Cc: bug-gnu-emacs <at> gnu.org, bug-submit-list <at> donarmstrong.com,
        bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 15:07:12 +0200

> From: Lawrence Mitchell <wence <at> gmx.li>
> Date: Fri, 17 Oct 2008 13:30:24 +0100
> Cc: emacs-pretest-bug <at> gnu.org
> 
> Drew Adams wrote:
> > Library vline.el can be read fine in Emacs 22, but Emacs 23 raises an
> > error, saying Invalid read syntax: "?".
> 
> The following sequence of events works for me in
> 
> GNU Emacs 23.0.60.4 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll
> bars) of 2008-10-07 on lamacq.ph.ed.ac.uk
> 
> $ wget http://www.emacswiki.org/cgi-bin/emacs/download/vline.el \
> -O /tmp/vline.el
> 
> $ LANG=C emacs -Q --eval '(load-file "/tmp/vline.el")'
> LANG=C so that current-language-environment is "ASCII"
> and default-buffer-file-coding-system is nil.
> 
> Successfully prints:
> Loading /tmp/vline.el (source)...done

For me, the file vline.el downloaded from the above address loads
successfully even without setting LANG to "C".  Just a simple "C-x
C-f" works and doesn't throw any errors.

Either the version of vline.el Drew used is different, or something
else is at work here.  Drew, are you using the patched EmacsW32 binary
produced by Lennart?  If so, perhaps it's something that is being
triggered by the patches.  Or maybe something in Drew's .emacs
customizations?

Message #75 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Eli Zaretskii'" <eliz <at> gnu.org>, <1187 <at> debbugs.gnu.org>
Cc: <emacs-pretest-bug <at> gnu.org>, <bug-gnu-emacs <at> gnu.org>
Subject: RE: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 07:26:39 -0700

> From: Eli Zaretskii Sent: Friday, October 17, 2008 1:15 AM
> > From: "Drew Adams" Date: Thu, 16 Oct 2008 14:22:45 -0700
> > Library vline.el can be read fine in Emacs 22, but Emacs 23
> > raises an error, saying Invalid read syntax: "?".
>
> I cannot reproduce this, perhaps because when I saved the vline.el you
> attached, I selected a wrong encoding.  Please visit the file with
> "M-x find-file-literally" and tell me what 8-bit bytes you see on the
> line that begins with "((memq char '(?\t".

Not sure what you mean, but I see this (pasting):
((memq char '(?\t ?　))

which looks like this (typing this in):
((memq char '(?\t ?\343\200\200))

oops - as soon as I hit C-s to save what I'd typed so far, Outlook changed what
it looks like (I'm using plain text with encoding auto-select (the only
choice)). I'll type it again - this is what it looks like in Emacs:

((memq char '(?\t ?\343\200\200))

There, that time it didn't change - that's what I see: three octal sequences, of
343, 200, 200.

> Then visit that file
> normally with Emacs 22 and tell what non-ASCII character(s) you see on
> that line (use "C-u C-x =" to describe those characters).

The text looks like ((memq char '(?\t ?  )), and C-u C-x = on the char after the
second ? gives this:

  character: 　 (53409, #o150241, #xd0a1, U+3000)
    charset: japanese-jisx0208 (JISX0208.1983/1990 Japanese Kanji: ISO-IR-87.)
 code point: #x21 #x21
     syntax: _ 	which means: symbol
   category: j:Japanese |:While filling, we can break a line at this character.
buffer code: #x92 #xA1 #xA1
  file code: #xE3 #x80 #x80 (encoded by coding system mule-utf-8-unix)
    display: by this font (glyph code)
     -outline-Arial Unicode
MS-normal-r-normal-normal-13-97-96-96-p-*-jisx0208-sjis (#x3000)

There are text properties here:
  fontified            t

If I do the same thing in Emacs 23 (with find-file-literally), I see this:

        character:   (227, #o343, #xe3)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0xE3
           syntax: w 	which means: word
         category: j:Japanese l:Latin v:Vietnamese
      buffer code: #xC3 #xA3
        file code: #xC3 #xA3 (encoded by coding system no-conversion)
          display: by this font (glyph code)
    uniscribe:-outline-Courier
New-normal-normal-normal-mono-13-*-*-*-c-*-iso10646-1 (#x6D)

Character code properties: customize what to show
  name: LATIN SMALL LETTER A WITH TILDE
  general-category: Ll (Letter, Lowercase)
  decomposition: (97 771) ('a' '')
  old-name: LATIN SMALL LETTER A TILDE

Does this help? Thx - Drew

Message #90 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Eli Zaretskii'" <eliz <at> gnu.org>, <1187 <at> debbugs.gnu.org>,
        "'Lawrence Mitchell'" <wence <at> gmx.li>
Cc: <bug-gnu-emacs <at> gnu.org>, <bug-submit-list <at> donarmstrong.com>
Subject: RE: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 07:40:16 -0700

> > The following sequence of events works for me in
> > 
> > GNU Emacs 23.0.60.4 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll
> > bars) of 2008-10-07 on lamacq.ph.ed.ac.uk
> > 
> > $ wget http://www.emacswiki.org/cgi-bin/emacs/download/vline.el \
> > -O /tmp/vline.el
> > 
> > $ LANG=C emacs -Q --eval '(load-file "/tmp/vline.el")'
> > LANG=C so that current-language-environment is "ASCII"
> > and default-buffer-file-coding-system is nil.
> > 
> > Successfully prints:
> > Loading /tmp/vline.el (source)...done
> 
> For me, the file vline.el downloaded from the above address loads
> successfully even without setting LANG to "C".  Just a simple "C-x
> C-f" works and doesn't throw any errors.
> 
> Either the version of vline.el Drew used is different, or something
> else is at work here.  Drew, are you using the patched EmacsW32 binary
> produced by Lennart?  If so, perhaps it's something that is being
> triggered by the patches.  Or maybe something in Drew's .emacs
> customizations?

No, I'm not using Lennart's patched binary; I'm using his vanilla binary. But
it's possible I saved or modified the file somehow - I don't recall. But that
should be irrelevant for the bug report - it doesn't matter if the file I sent
is the original vline.el or something else. What matters is that I opened it
using emacs -Q with a vanilla Emacs binary - no customizations, nothing else
loaded, nada.

Message #100 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Lennart Borgman (gmail)" <lennart.borgman <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>, 1187 <at> debbugs.gnu.org
Cc: Lawrence Mitchell <wence <at> gmx.li>, Drew Adams <drew.adams <at> oracle.com>
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 16:58:17 +0200

Eli Zaretskii wrote:
>> From: Lawrence Mitchell <wence <at> gmx.li>
>> Date: Fri, 17 Oct 2008 13:30:24 +0100
>> Cc: emacs-pretest-bug <at> gnu.org
>>
>> Drew Adams wrote:
>>> Library vline.el can be read fine in Emacs 22, but Emacs 23 raises an
>>> error, saying Invalid read syntax: "?".
>> The following sequence of events works for me in
>>
>> GNU Emacs 23.0.60.4 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll
>> bars) of 2008-10-07 on lamacq.ph.ed.ac.uk
>>
>> $ wget http://www.emacswiki.org/cgi-bin/emacs/download/vline.el \
>> -O /tmp/vline.el
>>
>> $ LANG=C emacs -Q --eval '(load-file "/tmp/vline.el")'
>> LANG=C so that current-language-environment is "ASCII"
>> and default-buffer-file-coding-system is nil.
>>
>> Successfully prints:
>> Loading /tmp/vline.el (source)...done
> 
> For me, the file vline.el downloaded from the above address loads
> successfully even without setting LANG to "C".  Just a simple "C-x
> C-f" works and doesn't throw any errors.
> 
> Either the version of vline.el Drew used is different, or something
> else is at work here.  Drew, are you using the patched EmacsW32 binary
> produced by Lennart?  If so, perhaps it's something that is being
> triggered by the patches.  Or maybe something in Drew's .emacs
> customizations?


I think Drew is using the unpatched version. And I can see this problem
with both the unpatched and the patched version.

However there might be a misunderstanding. There is no error while
reading vline.el. The error comes when I do eval-buffer.

Message #105 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Lennart Borgman \(gmail\)'" <lennart.borgman <at> gmail.com>,
        "'Eli Zaretskii'" <eliz <at> gnu.org>, <1187 <at> debbugs.gnu.org>
Cc: "'Lawrence Mitchell'" <wence <at> gmx.li>
Subject: RE: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 08:18:06 -0700

> > For me, the file vline.el downloaded from the above address loads
> > successfully even without setting LANG to "C".  Just a simple "C-x
> > C-f" works and doesn't throw any errors.
> > 
> > Either the version of vline.el Drew used is different, or something
> > else is at work here.  Drew, are you using the patched 
> > EmacsW32 binary produced by Lennart?  If so, perhaps it's something
> > that is being triggered by the patches.  Or maybe something in Drew's
> > .emacs customizations?
> 
> I think Drew is using the unpatched version.

Correct.

> And I can see this problem with both the unpatched and the patched version.
> 
> However there might be a misunderstanding. There is no error while
> reading vline.el. The error comes when I do eval-buffer.

Yes, the problem arises when I load (so, eval) the library. But Lawrence and Eli
both mentioned loading also.

Message #110 received at 1187-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 1187-done <at> debbugs.gnu.org, Lawrence Mitchell <wence <at> gmx.li>
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 11:46:05 -0400

http://www.emacswiki.org/cgi-bin/emacs/download/vline.el contains
non-ascii chars, but does not contain any coding tag, so depending on
your coding settings, it will decode it in different ways, some of which
will lead to load-time errors, others to incorrect behavior.
There's nothing for Emacs to do here.
Such uses are OK for a file you wrote for your own use, but for
distribution to people who may use other locales, it's not.


        Stefan

Message #115 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: <1187 <at> debbugs.gnu.org>, "'Eli Zaretskii'" <eliz <at> gnu.org>,
        "'Lawrence Mitchell'" <wence <at> gmx.li>
Cc: <bug-gnu-emacs <at> gnu.org>, <bug-submit-list <at> donarmstrong.com>
Subject: RE: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 08:53:46 -0700

FWIW, my reply-all for this message:

> -----Original Message-----
> From: bug-gnu-emacs-bounces+drew.adams=oracle.com <at> gnu.org 
> [mailto:bug-gnu-emacs-bounces+drew.adams=oracle.com <at> gnu.org] 
> On Behalf Of Drew Adams
> Sent: Friday, October 17, 2008 7:40 AM
> To: 'Eli Zaretskii'; 1187 <at> emacsbugs.donarmstrong.com; 
> 'Lawrence Mitchell'
> Cc: bug-gnu-emacs <at> gnu.org; bug-submit-list <at> donarmstrong.com
> Subject: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax

led to this returned mail message from the postmaster:

 The original message was received at Fri, 17 Oct 2008 09:40:29 -0500
 from agminet01.oracle.com [141.146.126.228]

    ----- The following addresses had permanent fatal errors -----
 <bug-submit-list <at> donarmstrong.com>
     (reason: 550 5.1.1 <bug-submit-list <at> donarmstrong.com>... User unknown)

    ----- Transcript of session follows -----
 ... while talking to linnode.donarmstrong.com.:
 >>> DATA
 <<< 550 5.1.1 <bug-submit-list <at> donarmstrong.com>... User unknown
 550 5.1.1 <bug-submit-list <at> donarmstrong.com>... User unknown
 <<< 503 5.0.0 Need RCPT (recipient)

And the attached file details.txt has this:

 Reporting-MTA: dns; agminet02.oracle.com
 Arrival-Date: Fri, 17 Oct 2008 09:40:29 -0500

 Final-Recipient: RFC822; bug-submit-list <at> donarmstrong.com
 Action: failed
 Status: 5.1.1
 Remote-MTA: DNS; linnode.donarmstrong.com
 Diagnostic-Code: SMTP; 550 5.1.1 <bug-submit-list <at> donarmstrong.com>... User
unknown
 Last-Attempt-Date: Fri, 17 Oct 2008 10:43:28 -0500

Dunno if this indicates a bug in the bug-reporting system or not. I guess it was
Eli who added the address bug-submit-list <at> donarmstrong.com to the cc list. 

Message #125 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Lawrence Mitchell <wence <at> gmx.li>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 1187 <at> debbugs.gnu.org,
        "'Lennart Borgman \(gmail\)'" <lennart.borgman <at> gmail.com>,
        "'Eli Zaretskii'" <eliz <at> gnu.org>
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 16:58:19 +0100

Hi Drew:

Drew Adams wrote:
>>> For me, the file vline.el downloaded from the above address loads
>>> successfully even without setting LANG to "C".  Just a simple "C-x
>>> C-f" works and doesn't throw any errors.

>>> Either the version of vline.el Drew used is different, or something
>>> else is at work here.  Drew, are you using the patched
>>> EmacsW32 binary produced by Lennart?  If so, perhaps it's something
>>> that is being triggered by the patches.  Or maybe something in Drew's
>>> .emacs customizations?

>> I think Drew is using the unpatched version.

> Correct.

>> And I can see this problem with both the unpatched and the patched version.

>> However there might be a misunderstanding. There is no error while
>> reading vline.el. The error comes when I do eval-buffer.

> Yes, the problem arises when I load (so, eval) the library. But
> Lawrence and Eli both mentioned loading also.

I think the problem is a bad interaction between your language
environment and the file in question.

Does the following command load vline.el successfully?

emacs -Q --eval '(let ((coding-system-for-read (quote utf-8)))
(load-file "path/to/vline.el"))'

?

For me, the above works regardless of my LANG environment
variable however:

LANG=en_US emacs -Q --eval '(load-file "path/to/vline.el")'

Loading /Home/s0198183/tmp/vline.el (source)...
Invalid read syntax: "?"

In this case, emacs uses the iso-latin-1 coding-system to open
the file and barfs when trying to read the utf-8 character in it.

Cheers,
Lawrence
-- 
Lawrence Mitchell <wence <at> gmx.li>

Message #130 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: lennart.borgman <at> gmail.com, 1187 <at> debbugs.gnu.org, wence <at> gmx.li
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 18:00:08 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Cc: "'Lawrence Mitchell'" <wence <at> gmx.li>
> Date: Fri, 17 Oct 2008 08:18:06 -0700
> 
> > However there might be a misunderstanding. There is no error while
> > reading vline.el. The error comes when I do eval-buffer.
> 
> Yes, the problem arises when I load (so, eval) the library. But Lawrence and Eli
> both mentioned loading also.

No, I said visiting with "C-x C-f".  And your original report said:

  Library vline.el can be read fine in Emacs 22, but Emacs 23 raises an
  error, saying Invalid read syntax: "?".

It said nothing about evaluating it.

Anyway, does the problem go away if you visit the file with

  "C-x RET c utf-8 RET C-x C-f vline.el RET"

before evaluating it?

If this does the trick, then please talk to the author and ask him to
include an explicit coding: cookie in the file.

Message #135 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 1187 <at> debbugs.gnu.org, wence <at> gmx.li, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 18:04:07 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Cc: <bug-gnu-emacs <at> gnu.org>, <bug-submit-list <at> donarmstrong.com>
> Date: Fri, 17 Oct 2008 08:53:46 -0700
> 
> Dunno if this indicates a bug in the bug-reporting system or not. I guess it was
> Eli who added the address bug-submit-list <at> donarmstrong.com to the cc list. 

Yes, sorry.  It's error-prone to manually remove addresses from the
list when replying, and this time I goofed.  I wish the bug tracker
didn't add such addresses to messages it sends to humans.

Message #145 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Lawrence Mitchell'" <wence <at> gmx.li>
Cc: <1187 <at> debbugs.gnu.org>,
        "'Lennart Borgman \(gmail\)'" <lennart.borgman <at> gmail.com>,
        "'Eli Zaretskii'" <eliz <at> gnu.org>
Subject: RE: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 09:09:03 -0700

> I think the problem is a bad interaction between your language
> environment and the file in question.
> 
> Does the following command load vline.el successfully?
> 
> emacs -Q --eval '(let ((coding-system-for-read (quote utf-8)))
> (load-file "path/to/vline.el"))'

Yes, that works. (Actually, I just put the sexp in *scratch* and eval'd it
there.

> For me, the above works regardless of my LANG environment
> variable however:
> 
> LANG=en_US emacs -Q --eval '(load-file "path/to/vline.el")'
> 
> Loading /Home/s0198183/tmp/vline.el (source)...
> Invalid read syntax: "?"
> 
> In this case, emacs uses the iso-latin-1 coding-system to open
> the file and barfs when trying to read the utf-8 character in it.

OK, I guess you found the problem, but what is the solution? Are users expected
to set `coding-system-for-read'? Is vline.el expected to use a local variable to
state that it is UTF-8?

This seemed to work OK in Emacs 22 without users doing anything special. But, as
Eli said (or perhaps it was you), perhaps Emacs 22 was simply ignoring some of
the code.

Is this a vline.el bug or an Emacs 23 bug or an Emacs 22 bug or a user problem
(language environment)?

Message #150 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Eli Zaretskii'" <eliz <at> gnu.org>
Cc: <lennart.borgman <at> gmail.com>, <1187 <at> debbugs.gnu.org>,
        <wence <at> gmx.li>
Subject: RE: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 09:21:16 -0700

> > > However there might be a misunderstanding. There is no error while
> > > reading vline.el. The error comes when I do eval-buffer.
> > 
> > Yes, the problem arises when I load (so, eval) the library. 
> > But Lawrence and Eli both mentioned loading also.
> 
> No, I said visiting with "C-x C-f".

Not really. You said:

 For me, the file vline.el downloaded from the above address loads
                                                             ^^^^^
 successfully even without setting LANG to "C".

But you did add this, which confuses things (loading with visiting):

 Just a simple "C-x C-f" works and doesn't throw any errors.

I didn't notice that last part.

> And your original report said:
> 
>   Library vline.el can be read fine in Emacs 22, but Emacs 23 
>   raises an error, saying Invalid read syntax: "?".
> 
> It said nothing about evaluating it.

Yes, I too was not clear enough. I meant load, not visit.

> Anyway, does the problem go away if you visit the file with
> 
>   "C-x RET c utf-8 RET C-x C-f vline.el RET"
> 
> before evaluating it?

Yes and no. I did `C-x RET c utf-8 RET C-x C-f vline.el RET'. If I then do `M-x
load-file RET', then I get the same error. However, if instead of `M-x load-file
RET' I use `M-x eval-buffer RET', then I don't get the error.
 
> If this does the trick, then please talk to the author and ask him to
> include an explicit coding: cookie in the file.

I can do that, if you tell me exactly what to tell him needs to be done.

However, is it normal that Emacs 23 raises an error if the encoding is wrong?
Emacs 22 does not raise any error here.

Message #155 received at 1187 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: <1187 <at> debbugs.gnu.org>
Cc: <monnier <at> iro.umontreal.ca>
Subject: RE: bug#1187 closed by Stefan Monnier <monnier <at> iro.umontreal.ca> (Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax)
Date: Fri, 17 Oct 2008 09:48:08 -0700

> This is an automatic notification regarding your bug report
> which was filed against the emacs package:
> 
> #1187: 23.0.60; Cannot read vline.el - invalid read syntax
> 
> It has been closed by Stefan Monnier <monnier <at> iro.umontreal.ca>.
> 
> Their explanation is attached below along with your original report.
> If this explanation is unsatisfactory and you have not received a
> better one in a separate message then please contact Stefan 
> Monnier <monnier <at> iro.umontreal.ca> by replying to this email.

[Note: "replying to this email" - even Reply All, does *not* send anything to
monnier <at> iro.umontreal.ca. That address needs to be added by hand, apparently.]


Uh, why was this closed in the middle of debugging and a discussion about it?

 http://www.emacswiki.org/cgi-bin/emacs/download/vline.el contains
 non-ascii chars, but does not contain any coding tag, so depending on
 your coding settings, it will decode it in different ways, some of which
 will lead to load-time errors, others to incorrect behavior.
 There's nothing for Emacs to do here.
 Such uses are OK for a file you wrote for your own use, but for
 distribution to people who may use other locales, it's not.

Not too helpful. Please explain what the author needs to do to fix the file.

And does the Emacs doc perhaps need to be updated to explain that libraries that
worked in Emacs 22 might need to be modified by adding coding settings in order
to work in Emacs 23?

BTW, the URL you cite does not correspond to the file I sent, as someone pointed
out, even though the file names are the same. The problem might be the same for
both files (dunno), but the reference is incorrect. This bug report you closed
is not about the file you closed it for.

Message #160 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>, Kenichi Handa <handa <at> m17n.org>
Cc: 1187 <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Fri, 17 Oct 2008 20:24:36 +0200

> From: "Drew Adams" <drew.adams <at> oracle.com>
> Cc: <lennart.borgman <at> gmail.com>, <1187 <at> emacsbugs.donarmstrong.com>,
>         <wence <at> gmx.li>
> Date: Fri, 17 Oct 2008 09:21:16 -0700
> 
> > Anyway, does the problem go away if you visit the file with
> > 
> >   "C-x RET c utf-8 RET C-x C-f vline.el RET"
> > 
> > before evaluating it?
> 
> Yes and no. I did `C-x RET c utf-8 RET C-x C-f vline.el RET'. If I then do `M-x
> load-file RET', then I get the same error. However, if instead of `M-x load-file
> RET' I use `M-x eval-buffer RET', then I don't get the error.

I meant the latter, sorry for another vague request.

> > If this does the trick, then please talk to the author and ask him to
> > include an explicit coding: cookie in the file.
> 
> I can do that, if you tell me exactly what to tell him needs to be done.

Add a "coding: utf-8" cookie on the first line of the file.  See the
beginning of calendar/icalendar.el for an example.

> However, is it normal that Emacs 23 raises an error if the encoding is wrong?
> Emacs 22 does not raise any error here.

I don't know.  Perhaps Handa-san can tell if there's a better way of
handling this particular case, without a need for a coding cookie.

Message #170 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Kenichi Handa <handa <at> m17n.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: drew.adams <at> oracle.com, 1187 <at> debbugs.gnu.org,
        bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Mon, 20 Oct 2008 19:28:08 +0900

In article <uwsg7krkr.fsf <at> gnu.org>, Eli Zaretskii <eliz <at> gnu.org> writes:

> > However, is it normal that Emacs 23 raises an error if the encoding is wrong?
> > Emacs 22 does not raise any error here.

> I don't know.  Perhaps Handa-san can tell if there's a better way of
> handling this particular case, without a need for a coding cookie.

Please send me the vnline.el that causes this problem.  I'll
check what is wrong.

---
Kenichi Handa
handa <at> ni.aist.go.jp

Message #180 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Kenichi Handa <handa <at> m17n.org>
Cc: drew.adams <at> oracle.com, 1187 <at> debbugs.gnu.org,
        bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Mon, 20 Oct 2008 12:58:54 +0200

> From: Kenichi Handa <handa <at> m17n.org>
> CC: drew.adams <at> oracle.com, 1187 <at> emacsbugs.donarmstrong.com,
>         bug-gnu-emacs <at> gnu.org
> Date: Mon, 20 Oct 2008 19:28:08 +0900
> 
> In article <uwsg7krkr.fsf <at> gnu.org>, Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > > However, is it normal that Emacs 23 raises an error if the encoding is wrong?
> > > Emacs 22 does not raise any error here.
> 
> > I don't know.  Perhaps Handa-san can tell if there's a better way of
> > handling this particular case, without a need for a coding cookie.
> 
> Please send me the vnline.el that causes this problem.  I'll
> check what is wrong.

It's vline.el, and you can download it from here:

  http://www.emacswiki.org/cgi-bin/wiki/vline.el

(It is better to download, to avoid any possible transformations of
the file in mail transfer.)

Message #190 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Kenichi Handa <handa <at> m17n.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: drew.adams <at> oracle.com, 1187 <at> debbugs.gnu.org,
        bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Tue, 21 Oct 2008 11:40:42 +0900

In article <uy70jilch.fsf <at> gnu.org>, Eli Zaretskii <eliz <at> gnu.org> writes:

> It's vline.el, and you can download it from here:

>   http://www.emacswiki.org/cgi-bin/wiki/vline.el

> (It is better to download, to avoid any possible transformations of
> the file in mail transfer.)

Ok, I found what is the problem.  In Emacs 22, iso-8859-1 is
a coding of type `iso-2022', but in Emacs 23, it's of type
`charset', and I forgot to handle latin-extra-code-table
(which has nil for the byte 0x80) in the detector of that
kind of coding system.  As I've just installed a fix, the
latest code should detect vline.el correctly as utf-8.

But, the reason why it's detected as utf-8 is because the
file contains a byte 0x80.  If a file doesn't contain a byte
in the range 0x80..0x9F and nil in latin-extra-code-table,
it's impossible to distinguish iso-latin-1 from utf-8.

So, in general, it's a good idea to add coding-tag: utf-8
for utf-8 files.

---
Kenichi Handa
handa <at> ni.aist.go.jp

Message #200 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Drew Adams" <drew.adams <at> oracle.com>
To: "'Kenichi Handa'" <handa <at> m17n.org>, "'Eli Zaretskii'" <eliz <at> gnu.org>
Cc: <1187 <at> debbugs.gnu.org>, <bug-gnu-emacs <at> gnu.org>
Subject: RE: bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax
Date: Mon, 20 Oct 2008 21:46:07 -0700

> From: Kenichi Handa Sent: Monday, October 20, 2008 7:41 PM
> Ok, I found what is the problem.  In Emacs 22, iso-8859-1 is
> a coding of type `iso-2022', but in Emacs 23, it's of type
> `charset', and I forgot to handle latin-extra-code-table
> (which has nil for the byte 0x80) in the detector of that
> kind of coding system.  As I've just installed a fix, the
> latest code should detect vline.el correctly as utf-8.
> 
> But, the reason why it's detected as utf-8 is because the
> file contains a byte 0x80.  If a file doesn't contain a byte
> in the range 0x80..0x9F and nil in latin-extra-code-table,
> it's impossible to distinguish iso-latin-1 from utf-8.
> 
> So, in general, it's a good idea to add coding-tag: utf-8
> for utf-8 files.

Thanks very much for following up with this, especially after the bug had been
closed. I've sent your message to the author.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.