GNU bug report logs - #636
23.0.60; Read syntax error while byte-compiling

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 636 in the body.
You can then email your comments to 636 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Dmitry Dzhus <dima <at> sphinx.net.ru>
To: emacs-pretest-bug <at> gnu.org
Subject: 23.0.60; Read syntax error while byte-compiling
Date: Fri, 01 Aug 2008 11:43:43 +0400

The problem occurs when one tries to byte-compile a file with the
following contents (note the newline after `eval-when-compile`):

    (eval-when-compile
       (require 'cc-mode))

Using this command:

    emacs -q --batch -f batch-byte-compile file.el

Which fails with error:

    In toplevel form:
    test1.el:2:19:Error: Invalid read syntax: ")"

Visiting this file in Emacs and attempting to byte-compile it using the
`M-x byte-compile-file RET` command results in the same error being
shown in *Compile-Log* buffer.
    
In the same time, if there is no newline after `eval-when-compile`, e.g.

    (eval-when-compile (require 'cc-mode))

Byte-compiling proceeds successfully. The problem occurs only if one
requires `cc-mode`. Probably this is somehow related with `cc-bytecomp`
module.

In GNU Emacs 23.0.60.1 (i686-pc-linux-gnu, GTK+ Version 2.12.11)
 of 2008-07-30 on blizzard

Windowing system distributor `The X.Org Foundation', version
 11.0.10402000

configured using `configure '--prefix=/usr' '--host=i686-pc-linux-gnu'
'--mandir=/usr/share/man' '--infodir=/usr/share/info'
'--datadir=/usr/share' '--sysconfdir=/etc' '--localstatedir=/var/lib'
'--program-suffix=-emacs-23' '--infodir=/usr/share/info/emacs-23'
'--without-carbon' '--with-sound' '--with-x'
'--with-toolkit-scroll-bars' '--with-gif' '--with-jpeg' '--with-png'
'--with-rsvg' '--with-tiff' '--with-xpm' '--enable-font-backend'
'--with-freetype' '--with-xft' '--without-libotf' '--without-m17n-flt'
'--with-x-toolkit=gtk' '--without-hesiod' '--without-kerberos'
'--without-kerberos5' '--with-gpm' '--with-dbus'
'--build=i686-pc-linux-gnu' 'build_alias=i686-pc-linux-gnu'
'host_alias=i686-pc-linux-gnu' 'CFLAGS=-O2 -march=native -pipe'
'LDFLAGS=-Wl,-O1''

Important settings:
  value of $LC_ALL: 
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: C
  value of $LC_TIME: nil
  value of $LANG: ru_RU.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t
-- 
Happy Hacking.

http://sphinx.net.ru
む

Message #8 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: 636 <at> debbugs.gnu.org
Subject: Re: Please fix before the release of 23.1
Date: Sun, 10 Aug 2008 14:55:44 -0400

More info from

http://lists.gnu.org/archive/html/emacs-devel/2008-08/msg00416.html

Ulrich Mueller wrote:

> The problem goes away if I revert the following change:
>
> 2008-06-27  Alan Mackenzie  <acm <at> muc.de>
>
> 	* progmodes/cc-defs.el (c-emacs-features): New feature
> 	'argumentative-bod-function.
>
> --- progmodes/cc-defs.el	6 May 2008 07:18:12 -0000	1.57
> +++ progmodes/cc-defs.el	27 Jun 2008 21:12:46 -0000	1.58
> @@ -1440,6 +1440,14 @@
>  			 '1-bit)
>  		       list)))
>  
> +    ;; In Emacs >= 23, beginning-of-defun will passes its parameter to
> +    ;; beginning-of-defun-function.  Assume end-of-defun does the same.
> +    (let ((beginning-of-defun-function
> +	   (lambda (&optional arg)
> +	     (not (eq arg nil)))))
> +      (if (beginning-of-defun 1)
> +	  (setq list (cons 'argumentative-bod-function list))))
> +
>      (let ((buf (generate-new-buffer " test"))
>  	  parse-sexp-lookup-properties
>  	  parse-sexp-ignore-comments
> @@ -1539,6 +1547,9 @@
>  
>  '8-bit              8 bit syntax entry flags (XEmacs style).
>  '1-bit              1 bit syntax entry flags (Emacs style).
> +'argumentative-bod-function         beginning-of-defun passes ARG through
> +                    to a non-null beginning-of-defun-function.  It is assumed
> +		    the end-of-defun does the same thing.
>  'syntax-properties  It works to override the syntax for specific characters
>  		    in the buffer with the 'syntax-table property.  It's
>  		    always set - CC Mode no longer works in emacsen without

Message #13 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Ulrich Mueller <ulm <at> gentoo.org>
Cc: 636 <at> debbugs.gnu.org, emacs-devel <at> gnu.org,
        Glenn Morris <rgm <at> gnu.org>, Michael Olson <mwolson <at> gnu.org>,
        Alan Mackenzie <acm <at> muc.de>
Subject: Re: Please fix before the release of 23.1
Date: Sun, 10 Aug 2008 18:15:28 -0400

Ulrich Mueller <ulm <at> gentoo.org> writes:

>>>>>> On Sun, 10 Aug 2008, I wrote:
>
>> ,----[ test.el ]
>> | (progn
>> |   (require 'cc-mode))
>> `----
>
>> $ emacs -Q test.el
>> M-x eval-buffer
>
>> This will result in an error: Invalid read syntax: ")"
>
> The problem goes away if I revert the following change:
>
> 2008-06-27  Alan Mackenzie  <acm <at> muc.de>
>
> 	* progmodes/cc-defs.el (c-emacs-features): New feature
> 	'argumentative-bod-function.

Thanks for pinpointing this.

In fact, it's apparently not a problem with cc-mode, but with Emacs.
Here's a stripped-down recipe:

1. Create a file foo.el in your lisp path with the following contents:

(defconst foo
  (let ((beginning-of-defun-function
	 (lambda (&optional arg)
	   (not (eq arg nil)))))
    (progn
      (beginning-of-defun 1))))

(provide 'foo)

2. Byte-compile it (M-x byte-compile-file RET).

3. Create a file test.el with the following contents:

(progn
  (require 'foo))

4. emacs test.el

5. M-x eval-buffer

Debugger entered--Lisp error: (invalid-read-syntax ")")
  eval-buffer()  ; Reading at buffer position 25
  call-interactively(eval-buffer t nil)
  execute-extended-command(nil)
  call-interactively(execute-extended-command nil nil)


I don't know what causes this bug, though.  Anyone?

Message #18 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Ulrich Mueller <ulm <at> gentoo.org>
Cc: Glenn Morris <rgm <at> gnu.org>, Michael Olson <mwolson <at> gnu.org>,
        636 <at> debbugs.gnu.org, Alan Mackenzie <acm <at> muc.de>,
        emacs-devel <at> gnu.org
Subject: Re: Please fix before the release of 23.1
Date: Sun, 10 Aug 2008 18:18:14 -0400

Chong Yidong <cyd <at> stupidchicken.com> writes:

> (defconst foo
>   (let ((beginning-of-defun-function
> 	 (lambda (&optional arg)
> 	   (not (eq arg nil)))))
>     (progn
>       (beginning-of-defun 1))))

By the way, replacing `progn' with `save-excursion' makes the bug go
away (this makes cc-def.el work too).  Why?  I don't know.

Message #23 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Ulrich Mueller <ulm <at> gentoo.org>
To: Chong Yidong <cyd <at> stupidchicken.com>
Cc: 636 <at> debbugs.gnu.org, emacs-devel <at> gnu.org,
        Glenn Morris <rgm <at> gnu.org>, Michael Olson <mwolson <at> gnu.org>,
        Alan Mackenzie <acm <at> muc.de>
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Mon, 11 Aug 2008 09:18:55 +0200

>>>>> On Sun, 10 Aug 2008, Chong Yidong wrote:

> Thanks for pinpointing this.

> In fact, it's apparently not a problem with cc-mode, but with Emacs.
> Here's a stripped-down recipe:

> 1. Create a file foo.el in your lisp path with the following contents:

> (defconst foo
>   (let ((beginning-of-defun-function
> 	 (lambda (&optional arg)
> 	   (not (eq arg nil)))))
>     (progn
>       (beginning-of-defun 1))))

> (provide 'foo)

Calling beginning-of-defun will move point, right?

And since beginning-of-defun-function is essentially defined as a
no-op, point will just be moved to the beginning-of-line. (In fact,
eval-buffer on foo.el results in an infinite loop.)

> By the way, replacing `progn' with `save-excursion' makes the bug go
> away (this makes cc-def.el work too). Why? I don't know.

Indeed. If we agree that the bug is in cc-defs.el, then the following
patch should fix it:

--- progmodes/cc-defs.el	27 Jun 2008 21:12:46 -0000	1.58
+++ progmodes/cc-defs.el	11 Aug 2008 07:11:04 -0000
@@ -1445,7 +1445,7 @@
     (let ((beginning-of-defun-function
 	   (lambda (&optional arg)
 	     (not (eq arg nil)))))
-      (if (beginning-of-defun 1)
+      (if (save-excursion (beginning-of-defun 1))
 	  (setq list (cons 'argumentative-bod-function list))))
 
     (let ((buf (generate-new-buffer " test"))

Ulrich

Message #28 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Ulrich Mueller <ulm <at> gentoo.org>
Cc: Chong Yidong <cyd <at> stupidchicken.com>, Glenn Morris <rgm <at> gnu.org>,
        Michael Olson <mwolson <at> gnu.org>, 636 <at> debbugs.gnu.org,
        emacs-devel <at> gnu.org
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Mon, 11 Aug 2008 08:47:08 +0000

Hi, Ulrich and Yidong!

Thanks for CC'ing me in on this one.

On Mon, Aug 11, 2008 at 09:18:55AM +0200, Ulrich Mueller wrote:
> >>>>> On Sun, 10 Aug 2008, Chong Yidong wrote:

> > In fact, it's apparently not a problem with cc-mode, but with Emacs.
> > Here's a stripped-down recipe:

> > 1. Create a file foo.el in your lisp path with the following contents:

> > (defconst foo
> >   (let ((beginning-of-defun-function
> > 	 (lambda (&optional arg)
> > 	   (not (eq arg nil)))))
> >     (progn
> >       (beginning-of-defun 1))))

> > (provide 'foo)

> Calling beginning-of-defun will move point, right?

> And since beginning-of-defun-function is essentially defined as a
> no-op, point will just be moved to the beginning-of-line. (In fact,
> eval-buffer on foo.el results in an infinite loop.)

> > By the way, replacing `progn' with `save-excursion' makes the bug go
> > away (this makes cc-def.el work too). Why? I don't know.

> Indeed. If we agree that the bug is in cc-defs.el, then the following
> patch should fix it:

> --- progmodes/cc-defs.el	27 Jun 2008 21:12:46 -0000	1.58
> +++ progmodes/cc-defs.el	11 Aug 2008 07:11:04 -0000
> @@ -1445,7 +1445,7 @@
>      (let ((beginning-of-defun-function
>  	   (lambda (&optional arg)
>  	     (not (eq arg nil)))))
> -      (if (beginning-of-defun 1)
> +      (if (save-excursion (beginning-of-defun 1))
>  	  (setq list (cons 'argumentative-bod-function list))))

>      (let ((buf (generate-new-buffer " test"))

I protest against this "fix".  :-)  Not against the fix as such (though
it is somewhat ugly, and would need some comments), just that the bug it
fixes is not understood.  Therefore, it is NOT a fix.  It is still a bug,
which will cause pain somewhere else.

> Ulrich

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #33 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Alan Mackenzie <acm <at> muc.de>
Cc: Ulrich Mueller <ulm <at> gentoo.org>, Glenn Morris <rgm <at> gnu.org>,
        Chong Yidong <cyd <at> stupidchicken.com>, 636 <at> debbugs.gnu.org,
        Michael Olson <mwolson <at> gnu.org>, emacs-devel <at> gnu.org
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Mon, 11 Aug 2008 10:54:59 +0200

Alan Mackenzie <acm <at> muc.de> writes:

> it is somewhat ugly, and would need some comments), just that the bug it
> fixes is not understood.

It's pretty simple.  You are moving point while load/eval-buffer/etc. is
reading the buffer, where reading the buffer always means reading from
point.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab <at> suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #38 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Michael Olson <mwolson <at> gnu.org>
To: Ulrich Mueller <ulm <at> gentoo.org>
Cc: Chong Yidong <cyd <at> stupidchicken.com>, 636 <at> debbugs.gnu.org,
        emacs-devel <at> gnu.org, Glenn Morris <rgm <at> gnu.org>,
        Alan Mackenzie <acm <at> muc.de>
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Mon, 11 Aug 2008 08:26:04 -0700

Ulrich Mueller <ulm <at> gentoo.org> writes:

> Indeed. If we agree that the bug is in cc-defs.el, then the following
> patch should fix it:
>
> --- progmodes/cc-defs.el	27 Jun 2008 21:12:46 -0000	1.58
> +++ progmodes/cc-defs.el	11 Aug 2008 07:11:04 -0000
> @@ -1445,7 +1445,7 @@
>      (let ((beginning-of-defun-function
>  	   (lambda (&optional arg)
>  	     (not (eq arg nil)))))
> -      (if (beginning-of-defun 1)
> +      (if (save-excursion (beginning-of-defun 1))
>  	  (setq list (cons 'argumentative-bod-function list))))
>
>      (let ((buf (generate-new-buffer " test"))

This patch fixes the problem for me as well.  It's interesting that
evaluating a top-level defconst definition can screw up the
byte-compiler though ... would it make sense for the byte-compiler to
automatically do a save-excursion around the processing of each
top-level form in a given file?

-- 
|       Michael Olson  |  FSF Associate Member #652     |
| http://mwolson.org/  |  Hobbies: Lisp, HCoop          |
| Projects: Emacs, Muse, ERC, EMMS, ErBot, DVC, Planner |
`-------------------------------------------------------'

Message #43 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Ulrich Mueller <ulm <at> gentoo.org>
Cc: 636 <at> debbugs.gnu.org, emacs-devel <at> gnu.org,
        Glenn Morris <rgm <at> gnu.org>, Michael Olson <mwolson <at> gnu.org>,
        Alan Mackenzie <acm <at> muc.de>
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Mon, 11 Aug 2008 12:30:04 -0400

Ulrich Mueller <ulm <at> gentoo.org> writes:

>> (defconst foo
>>   (let ((beginning-of-defun-function
>> 	 (lambda (&optional arg)
>> 	   (not (eq arg nil)))))
>>     (progn
>>       (beginning-of-defun 1))))
>
>> (provide 'foo)
>
> Calling beginning-of-defun will move point, right?
>
> And since beginning-of-defun-function is essentially defined as a
> no-op, point will just be moved to the beginning-of-line. (In fact,
> eval-buffer on foo.el results in an infinite loop.)

Aha, I see.

So the question is, should we make the change to cc-defs.el, or (as
suggested by Michael Olson) make the byte-compiler wrap around each
variable definition with `save-excursion'?

Even with the latter solution, incorrectly written elisp in variable
definitions can still create problems.  For example, `save-excursion'
won't save us if the elisp runs `kill-buffer'.

This indicates that we fix cc-defs.el, and leave the rest of Emacs
alone.

Message #48 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Andreas Schwab <schwab <at> suse.de>
Cc: emacs-devel <at> gnu.org, 636 <at> debbugs.gnu.org
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Mon, 11 Aug 2008 18:50:55 +0000

Hi, Andreas

On Mon, Aug 11, 2008 at 10:54:59AM +0200, Andreas Schwab wrote:
> Alan Mackenzie <acm <at> muc.de> writes:

> > it is somewhat ugly, and would need some comments), just that the bug
> > it fixes is not understood.

> It's pretty simple.  You are moving point while load/eval-buffer/etc.
> is reading the buffer, where reading the buffer always means reading
> from point.

Ah!  I missed that, because it wasn't me (in the capacity of CC Mode
maintainer) that moved point.

Thanks!

> Andreas.

> -- 
> Andreas Schwab, SuSE Labs, schwab <at> suse.de
> SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
> PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
> "And now for something completely different."

Message #53 received at 636 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Chong Yidong <cyd <at> stupidchicken.com>
Cc: Ulrich Mueller <ulm <at> gentoo.org>, 636 <at> debbugs.gnu.org,
        emacs-devel <at> gnu.org, Glenn Morris <rgm <at> gnu.org>,
        Michael Olson <mwolson <at> gnu.org>
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Mon, 11 Aug 2008 19:01:24 +0000

Hi, everybody!

On Mon, Aug 11, 2008 at 12:30:04PM -0400, Chong Yidong wrote:
> Ulrich Mueller <ulm <at> gentoo.org> writes:

> >> (defconst foo
> >>   (let ((beginning-of-defun-function
> >> 	 (lambda (&optional arg)
> >> 	   (not (eq arg nil)))))
> >>     (progn
> >>       (beginning-of-defun 1))))

> >> (provide 'foo)

> > Calling beginning-of-defun will move point, right?

> > And since beginning-of-defun-function is essentially defined as a
> > no-op, point will just be moved to the beginning-of-line. (In fact,
> > eval-buffer on foo.el results in an infinite loop.)

> Aha, I see.

> So the question is, should we make the change to cc-defs.el, or (as
> suggested by Michael Olson) make the byte-compiler wrap around each
> variable definition with `save-excursion'?

The way I see it, it's a bug in `beginning-of-defun'.  The docstring
says that a non-nil `beginning-of-defun-function' finds the pertinent
place, but beginning-of-defun then moves point somewhere else.  I HATE
things that are ostensibly "helpful", but in reality are dumbing down,
and mainly just foul things up.

I would be in favour of NOT moving point after a BOD-function has done
its work.  However, I wouldn't campaign too hard about it, because ....

CC Mode should remain compatible with lots of Emacsen, including 21.n.
That means I'll have to modify CC Mode anyway.  So I will.

> Even with the latter solution, incorrectly written elisp in variable
> definitions can still create problems.  For example, `save-excursion'
> won't save us if the elisp runs `kill-buffer'.

Hey, that's an idea!  ;-)

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #58 received at 636-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Alan Mackenzie <acm <at> muc.de>
Cc: Ulrich Mueller <ulm <at> gentoo.org>, 636-done <at> debbugs.gnu.org,
        emacs-devel <at> gnu.org, Glenn Morris <rgm <at> gnu.org>,
        Michael Olson <mwolson <at> gnu.org>
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Mon, 11 Aug 2008 17:42:22 -0400

Alan Mackenzie <acm <at> muc.de> writes:

> The way I see it, it's a bug in `beginning-of-defun'.  The docstring
> says that a non-nil `beginning-of-defun-function' finds the pertinent
> place, but beginning-of-defun then moves point somewhere else.  I HATE
> things that are ostensibly "helpful", but in reality are dumbing down,
> and mainly just foul things up.

This is part of the baggage in beginning-of-defun regarding whether
column zero is the beginning of a defun.  Changing this now might be
unwise, so for the moment I've simply revised the docstring of
beginning-of-defun for extra clarity.

I've also changed cc-defs.el to use beginning-of-defun-raw, which is a
more direct and side-effect free test of beginning-of-defun-function,
and added a save-excursion for extra plus safety ;-)

[Also, closing bug#636.]

Message #59 received at 636-done <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Chong Yidong <cyd <at> stupidchicken.com>
Cc: Ulrich Mueller <ulm <at> gentoo.org>, 636-done <at> debbugs.gnu.org,
        emacs-devel <at> gnu.org, Glenn Morris <rgm <at> gnu.org>,
        Michael Olson <mwolson <at> gnu.org>
Subject: Re: bug#636: Please fix before the release of 23.1
Date: Wed, 13 Aug 2008 09:24:55 +0000

Hi, Yidong!

On Mon, Aug 11, 2008 at 05:42:22PM -0400, Chong Yidong wrote:
> Alan Mackenzie <acm <at> muc.de> writes:

> > The way I see it, it's a bug in `beginning-of-defun'.  The docstring
> > says that a non-nil `beginning-of-defun-function' finds the pertinent
> > place, but beginning-of-defun then moves point somewhere else.  I
> > HATE things that are ostensibly "helpful", but in reality are dumbing
> > down, and mainly just foul things up.

> This is part of the baggage in beginning-of-defun regarding whether
> column zero is the beginning of a defun.  Changing this now might be
> unwise, so for the moment I've simply revised the docstring of
> beginning-of-defun for extra clarity.

Thanks!

> I've also changed cc-defs.el to use beginning-of-defun-raw, which is a
> more direct and side-effect free test of beginning-of-defun-function,
> and added a save-excursion for extra plus safety ;-)

I've just added a bit to bind mark-ring, because BOD adds a mark.

I'm not actually convinced that using bod-raw is a good idea, since it
feels like using an internal interface (which we might want to refactor
away later).  But it's not that big a deal, so I've left it.

> [Also, closing bug#636.]

:-)

-- 
Alan Mackenzie (Nuremberg, Germany).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.