GNU bug report logs - #16184
24.3.50; edebug and eval-when-compiler don't work together

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 16184 in the body.
You can then email your comments to 16184 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Anders Lindgren <andlind <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3.50; edebug and eval-when-compiler don't work together
Date: Wed, 18 Dec 2013 11:58:23 +0100

[Message part 1 (text/plain, inline)]

Hi!

The construct 'eval-when-compile' seems to throw debug off. When pressing
C-u C-M-x on the following, an error is issued. This worked correctly in
24.3. This makes it hard to use edebug on functions in cc-mode
(like c-font-lock-declarations), as it makes heavy use of eval-when-compile.

(defun test (limit)
  (eval-when-compile
    (boundp 'parse-sexp-lookup-properties)))

Sincerely,
    Anders Lidgren


In GNU Emacs 24.3.50.2 (x86_64-apple-darwin13.0.0, NS apple-appkit-1265.00)
 of 2013-12-17 on macpro.lan
Bzr revision: 115421
monnier <at> iro.umontreal.ca-20131208081150-4bq2y1cgta14nfmj
Windowing system distributor `Apple', version 10.3.1265
Configured using:
 `configure --with-ns'

Important settings:
  value of $LC_CTYPE: UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-u C-M-x C-SPC <down> <down> <down> <escape> w <up>
<up> <up> <up> s-f s-f <escape> f <escape> f <escape>
d <escape> d <escape> d <escape> d <down> C-a C-x C-s
<up> <up> s-f <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <backspace>
<backspace> <backspace> t e s t <down> <down> <up>
C-u C-M-x C-x C-s <up> C-a C-SPC C-g C-SPC <down> <down>
<down> <escape> w <menu-bar> <help-menu> <send-ema
cs-bug-report>

Recent messages:
Saving file /Users/anders/emacs/src/misc/bug-edebug-eval-when-compile.el...
Wrote /Users/anders/emacs/src/misc/bug-edebug-eval-when-compile.el
Beginning of buffer
Edebug: test
edebug-enter: Symbol's value as variable is void: limit
Saving file /Users/anders/emacs/src/misc/bug-edebug-eval-when-compile.el...
Wrote /Users/anders/emacs/src/misc/bug-edebug-eval-when-compile.el
Mark activated
Quit
Mark activated

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils misearch multi-isearch edebug easymenu cl-loaddefs
cl-lib vc-dispatcher vc-svn time-date tooltip electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel ns-win tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
cocoa ns multi-tty emacs)

[Message part 2 (text/html, inline)]

Message #8 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: 16184 <at> debbugs.gnu.org
Cc: Anders Lindgren <andlind <at> gmail.com>
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 20 Dec 2013 09:01:35 +0900

[Message part 1 (text/plain, inline)]

Anders Lindgren wrote:
> The construct 'eval-when-compile' seems to throw debug off. When
> pressing C-u C-M-x on the following, an error is issued. This worked
> correctly in 24.3. This makes it hard to use edebug on functions in
> cc-mode (like c-font-lock-declarations), as it makes heavy use of
> eval-when-compile.

> (defun test (limit)
>   (eval-when-compile
>     (boundp 'parse-sexp-lookup-properties)))

I'm not sure of it but the patch below seems to solve it.
This works for the bug#14646 case[1], too.  WDYT?

[1] <http://thread.gmane.org/gmane.emacs.bugs/80778>

[Message part 2 (text/x-patch, inline)]

--- lisp/emacs-lisp/byte-run.el~	2013-11-21 22:12:14 +0000
+++ lisp/emacs-lisp/byte-run.el	2013-12-19 23:45:41 +0000
@@ -391,7 +391,7 @@
   "Like `progn', but evaluates the body at compile time if you're compiling.
 Thus, the result of the body appears to the compiler as a quoted constant.
 In interpreted code, this is entirely equivalent to `progn'."
-  (declare (debug (def-body)) (indent 0))
+  (declare (debug (&rest def-form)) (indent 0))
   (list 'quote (eval (cons 'progn body) lexical-binding)))
 
 (defmacro eval-and-compile (&rest body)
--- lisp/progmodes/cc-defs.el~	2013-06-02 22:06:34 +0000
+++ lisp/progmodes/cc-defs.el	2013-12-19 23:45:40 +0000
@@ -1137,7 +1137,7 @@
 ;; Make edebug understand the macros.
 ;(eval-after-load "edebug" ; 2006-07-09: def-edebug-spec is now in subr.el.
 ;  '(progn
-(def-edebug-spec cc-eval-when-compile t)
+(def-edebug-spec cc-eval-when-compile (&rest def-form))
 (def-edebug-spec c-point t)
 (def-edebug-spec c-set-region-active t)
 (def-edebug-spec c-safe t)

Message #11 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Anders Lindgren <andlind <at> gmail.com>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 16184 <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 20 Dec 2013 07:59:01 +0100

[Message part 1 (text/plain, inline)]

Hi!

Unfortunately, the problem still seems to be there. I tried this on latest
trunk (as of yesterday) with the patch applied and I still get an error
when I do C-u C-M-x on the code I originally supplied.

    -- Anders


On Fri, Dec 20, 2013 at 1:01 AM, Katsumi Yamaoka <yamaoka <at> jpl.org> wrote:

> Anders Lindgren wrote:
> > The construct 'eval-when-compile' seems to throw debug off. When
> > pressing C-u C-M-x on the following, an error is issued. This worked
> > correctly in 24.3. This makes it hard to use edebug on functions in
> > cc-mode (like c-font-lock-declarations), as it makes heavy use of
> > eval-when-compile.
>
> > (defun test (limit)
> >   (eval-when-compile
> >     (boundp 'parse-sexp-lookup-properties)))
>
> I'm not sure of it but the patch below seems to solve it.
> This works for the bug#14646 case[1], too.  WDYT?
>
> [1] <http://thread.gmane.org/gmane.emacs.bugs/80778>
>
>

[Message part 2 (text/html, inline)]

Message #14 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Anders Lindgren <andlind <at> gmail.com>
Cc: 16184 <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 20 Dec 2013 17:28:45 +0900

Anders Lindgren wrote:
> Unfortunately, the problem still seems to be there. I tried this on
> latest trunk (as of yesterday) with the patch applied and I still get
> an error when I do C-u C-M-x on the code I originally supplied.

Could you make sure that the edebug spec of the `eval-when-compile'
macro definition (i.e. in byte-run.el) has been changed into
`(&rest def-form)'?
Otherwise please try `C-M-x' on the patched `eval-when-compile' macro.

Message #17 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Anders Lindgren <andlind <at> gmail.com>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 16184 <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 20 Dec 2013 10:26:19 +0100

[Message part 1 (text/plain, inline)]

Hi!

I just tried this and the error is still there.

On an emacs with bzr revision 115609, I started Emacs with -Q, and ran C-u
C-M-x on the code, and I got the error. I inspected the eval-when-compile
macro and it did have the form suggested by the patch in the bug report you
reported, i.e. "(declare (debug (def-body)) (indent 0))". Just to make sure
I ran C-M-x on it and tried again, but the error is still there...

    -- Anders


On Fri, Dec 20, 2013 at 9:28 AM, Katsumi Yamaoka <yamaoka <at> jpl.org> wrote:

> Anders Lindgren wrote:
> > Unfortunately, the problem still seems to be there. I tried this on
> > latest trunk (as of yesterday) with the patch applied and I still get
> > an error when I do C-u C-M-x on the code I originally supplied.
>
> Could you make sure that the edebug spec of the `eval-when-compile'
> macro definition (i.e. in byte-run.el) has been changed into
> `(&rest def-form)'?
> Otherwise please try `C-M-x' on the patched `eval-when-compile' macro.
>

[Message part 2 (text/html, inline)]

Message #20 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Anders Lindgren <andlind <at> gmail.com>
Cc: 16184 <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 20 Dec 2013 18:53:40 +0900

Anders Lindgren wrote:
> I just tried this and the error is still there.

Well, if you tried `C-u C-M-x' on `c-font-lock-declarations' and
the like, you needed to have patched cc-defs.el in addition to
byte-run.el.

Message #23 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Anders Lindgren <andlind <at> gmail.com>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 16184 <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 20 Dec 2013 11:30:47 +0100

[Message part 1 (text/plain, inline)]

Sorry, I don't follow you here...

I get the problem when I do C-u C-M-x on both the small cut-down example of
my original bug report as well as on c-font-lock-declarations.

I haven't seen any patch to cc-defs.el in any of the mails I have received
on this matter.

I get the problem when I start Emacs with the "-Q" option, based on plain
build from a clean bzr checkout. I don't have any local patches applied, if
that is what you are suggesting.

    -- Anders


On Fri, Dec 20, 2013 at 10:53 AM, Katsumi Yamaoka <yamaoka <at> jpl.org> wrote:

> Anders Lindgren wrote:
> > I just tried this and the error is still there.
>
> Well, if you tried `C-u C-M-x' on `c-font-lock-declarations' and
> the like, you needed to have patched cc-defs.el in addition to
> byte-run.el.
>
>

[Message part 2 (text/html, inline)]

Message #26 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Anders Lindgren <andlind <at> gmail.com>
Cc: 16184 <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 20 Dec 2013 21:44:53 +0900

Anders Lindgren <andlind <at> gmail.com> wrote:
> I haven't seen any patch to cc-defs.el in any of the mails I have received on
> this matter.

<http://article.gmane.org/gmane.emacs.bugs/82252>
-> <http://cache.gmane.org//gmane/emacs/bugs/82252-001.bin>

Message #29 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Anders Lindgren <andlind <at> gmail.com>
To: Katsumi Yamaoka <yamaoka <at> jpl.org>
Cc: 16184 <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 20 Dec 2013 14:38:38 +0100

[Message part 1 (text/plain, inline)]

Hi!

OK, now I found the patch, and I now understand what you mean by the
"(&rest def-form)".

It seems to have solved one problem, but created another. When using C-u
C-M-x, everything works ok. However, the normal C-M-x command issues the
error "Too deep - perhaps infinite loop in spec?" When enabling
debug-on-error, it looks like a very deep call stack (754 entries).
Curiously, it works if I mark the function with the region and do
"eval-region", it works properly.

    -- Anders


On Fri, Dec 20, 2013 at 1:44 PM, Katsumi Yamaoka <yamaoka <at> jpl.org> wrote:

> Anders Lindgren <andlind <at> gmail.com> wrote:
> > I haven't seen any patch to cc-defs.el in any of the mails I have
> received on
> > this matter.
>
> <http://article.gmane.org/gmane.emacs.bugs/82252>
> -> <http://cache.gmane.org//gmane/emacs/bugs/82252-001.bin>
>

[Message part 2 (text/html, inline)]

Message #32 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: Anders Lindgren <andlind <at> gmail.com>
Cc: 16184 <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Tue, 24 Dec 2013 09:23:26 +0900

Anders Lindgren wrote:
> It seems to have solved one problem, but created another. When using
> C-u C-M-x, everything works ok. However, the normal C-M-x command
> issues the error "Too deep - perhaps infinite loop in spec?" When
> enabling debug-on-error, it looks like a very deep call stack (754
> entries). Curiously, it works if I mark the function with the region
> and do "eval-region", it works properly.

Sorry for the delay.  But
I'm unable to reproduce such an infloop-like error so far.
Could you show the name or the definition of the function
causing it?

Message #37 received at 16184-done <at> debbugs.gnu.org (full text, mbox):

From: Katsumi Yamaoka <yamaoka <at> jpl.org>
To: 16184-done <at> debbugs.gnu.org
Cc: Anders Lindgren <andlind <at> gmail.com>
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Thu, 26 Dec 2013 07:37:08 +0900

Katsumi Yamaoka wrote:
> Anders Lindgren wrote:
>> It seems to have solved one problem, but created another. When using
>> C-u C-M-x, everything works ok. However, the normal C-M-x command
>> issues the error "Too deep - perhaps infinite loop in spec?" When
>> enabling debug-on-error, it looks like a very deep call stack (754
>> entries). Curiously, it works if I mark the function with the region
>> and do "eval-region", it works properly.

> Sorry for the delay.  But
> I'm unable to reproduce such an infloop-like error so far.
> Could you show the name or the definition of the function
> causing it?

I've committed my patch since I think it's better at least than
not working.  Please reopen the bug if you get a reproducible error.

Message #44 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Gemini Lasswell <gazally <at> runbox.com>
To: 16184 <at> debbugs.gnu.org
Subject: incorrect Edebug spec for eval-and-compile
Date: Tue, 18 Oct 2016 17:06:07 -0700

This bug also exists in eval-and-compile, which needs the same fix as
eval-when-compile.

(defun test (limit)
  (eval-and-compile
    (boundp 'parse-sexp-lookup-properties)))

To reproduce:

1. emacs -Q
2. enter the above code into *scratch*
3. C-u C-M-x

Result: edebug-after: Wrong type argument: consp, nil

Message #47 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: David Bremner <david <at> tethera.net>
To: 16184 <at> debbugs.gnu.org
Subject: confirmed in 25.1
Date: Tue, 15 Nov 2016 21:47:40 -0400

[Message part 1 (text/plain, inline)]

I tried the example from message #44 and got the same failure.
it does say "Edebug: test" on the previous line of *Messages*, fwiw

d

[signature.asc (application/pgp-signature, inline)]

Message #50 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Benaiah Mischenko <benaiah <at> mischenko.com>
To: "16184 <at> debbugs.gnu.org" <16184 <at> debbugs.gnu.org>
Subject: confirmed on 25.1.1
Date: Wed, 16 Nov 2016 02:07:34 +0000

The eval-and-compile test case from message #44 fails for me in emacs -Q as well, with the following *Messages* output:

Edebug: test
edebug-after: Wrong type argument: consp, nil

The output of M-x emacs-version is as follows:

GNU Emacs 25.1.1 (x86_64-apple-darwin15.6.0, NS appkit-1404.47 Version 10.11.6 (Build 15G31)) of 2016-09-21

Turning on toggle-debug-on-error and running it again gave me the following backtrace:

Debugger entered--Lisp error: (wrong-type-argument consp nil)
  edebug-before(1)
  (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))
  (progn (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties))))
  eval((progn (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))) nil)
  #[128 "\301\302\303B\"D\207" [lexical-binding quote eval progn] 5 1783882]((edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties))))
  macroexpand((eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))) nil)
  macroexp-macroexpand((eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))) nil)
  macroexp--expand-all((eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties)))))
  macroexp--all-forms((edebug-after (edebug-before 0) 3 (eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties))))) 1)
  #[514 "\300
\301\"\211\204

Message #55 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Anders Lindgren <andlind <at> gmail.com>
Cc: Benaiah Mischenko <benaiah <at> mischenko.com>,
 Gemini Lasswell <gazally <at> runbox.com>, Katsumi Yamaoka <yamaoka <at> jpl.org>,
 16184 <at> debbugs.gnu.org, David Bremner <david <at> tethera.net>
Subject: Re: bug#16184: 24.3.50; edebug and eval-when-compiler don't work
 together
Date: Fri, 18 Nov 2016 22:47:13 +0000

Hello, Anders.

On Wed, Dec 18, 2013 at 11:58:23AM +0100, Anders Lindgren wrote:
> Hi!

> The construct 'eval-when-compile' seems to throw debug off. When pressing
> C-u C-M-x on the following, an error is issued. This worked correctly in
> 24.3. This makes it hard to use edebug on functions in cc-mode
> (like c-font-lock-declarations), as it makes heavy use of eval-when-compile.

> (defun test (limit)
>   (eval-when-compile
>     (boundp 'parse-sexp-lookup-properties)))

> Sincerely,
>     Anders Lidgren

[ .... ].

This has been bugging me off and on for years, too.  If I'd noticed your
bug report back in 2013, I might have got round to looking at it then,
but I thought it was just me that was suffering.  :-(

I think I now understand what's happening, and I have a tentative fix,
though this might be more of a workaround than fix, I'm not sure.

Anyhow, this is the form I've been testing with:

    (defun test (limit)
      (let ((foo
             (cc-eval-when-compile
               (boundp 'parse-sexp-lookup-properties))))))

, which clearly needs (require 'cc-defs) before it if you don't have CC
Mode loaded already.

#########################################################################

Essential background information
================================

When edebug instruments a function, it inserts a call to edebug-enter at
the start of the function.  Regularly throughout the function, it inserts
calls to edebug-before and edebug-after, in forms which look something
like:

    (edebug-before 2)
    (edebug-after (edebug-before 2) 3 ...)

.  edebug maintains a stack of numerical "offsets" (whatever they are)
called edebug-offset-indices, consing a 0 onto it in edebug-enter, each
time an instrumented function is entered.  At each edebug-before or
edebug-after, the top element of this stack is modified with:

    (setcar edebug-offset-indices before-index) ; where before-index is,
say 2.

The offset at the top of edebug-offset-indices is used to position point
each time edebug displays the buffer being debugged before waiting for
user input.

#########################################################################

The bug mechanism is this: The macro-expander working on
cc-eval-when-compile, and edebug's instrumentation produce a form
something like this:

    (edebug-enter 'test (limit)
                  (function (lambda ......
                   (cc-eval-when-compile
                    (edebug-after (edebug-before 2) 3
                                  (boundp 'parse-sexp-lookup-properties)
                   .....)))))

.  During this instrumentation, the inside of the cc-eval-when-compile
form gets run, in particular, the (edebug-before 2).  At this stage,
since edebug-enter hasn't been run at all, edebug-offset-indices is still
nil.  The (edebug-before 2) thus attempts (setcar nil 2), which throws
the error.

#########################################################################

My proposed solution/workaround is to initialise edebug-offset-indices to
'(0) rather than nil, giving edebug-before/after a cons to work on, even
when they're called "at the top level".

This seems to work, sort of.  In my test form above, when I do C-u C-M-x,
the function `test' gets successfully instrumented.  However, if I
replace cc-eval-when-compile with eval-when-compile and do C-u C-M-x, the
behaviour is different: during the instrumentation, edebug steps through
the `boundp' form.  I don't yet understand where this difference comes
from.

So, here's the patch, which I'd be grateful if you would try out.  It's
based on the savannah master:



diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index 66117b8..5dc6f89 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -2052,9 +2052,9 @@ edebug-stack
 (defvar edebug-stack-depth -1)
 ;; Index of last edebug-stack item.
 
-(defvar edebug-offset-indices nil)
+(defvar edebug-offset-indices (list 0))
 ;; Stack of offset indices of visited edebug sexps.
-;; Should be nil at the top level.
+;; Should be nil at the top level.  No longer!  Extensive comment needed here.
 ;; Each function adds one cons.  Top is modified with setcar.
 
 
This patch clearly needs commenting added to it, but I'm in much too
prolix a frame of mind to attempt that now.  ;-(

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #60 received at 16184-done <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: 16184-done <at> debbugs.gnu.org
Cc: Gemini Lasswell <gazally <at> runbox.com>, Katsumi Yamaoka <yamaoka <at> jpl.org>,
 Benaiah Mischenko <benaiah <at> mischenko.com>, Anders Lindgren <andlind <at> gmail.com>,
 David Bremner <david <at> tethera.net>
Subject: Re: bug#16184: 24.3.50; edebug and eval-when-compiler don't work
 together
Date: Thu, 29 Dec 2016 09:29:04 +0000

Bug fixed.

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #63 received at 16184-done <at> debbugs.gnu.org (full text, mbox):

From: Anders Lindgren <andlind <at> gmail.com>
To: Alan Mackenzie <acm <at> muc.de>
Cc: Gemini Lasswell <gazally <at> runbox.com>, Katsumi Yamaoka <yamaoka <at> jpl.org>,
 Benaiah Mischenko <benaiah <at> mischenko.com>, David Bremner <david <at> tethera.net>,
 16184-done <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 30 Dec 2016 21:02:31 +0100

[Message part 1 (text/plain, inline)]

Hi!

I just got around to test this. Now, `eval-when-comple' seems to be
working. Unfortunately, `eval-and-comple' is still broken.

If I do C-u M-C-x eval the following with `debug-on-error' set to t, I get
the backtrace below:

(defun test (limit)
  (eval-and-compile
    (boundp 'parse-sexp-lookup-properties)))


Debugger entered--Lisp error: (void-variable edebug-freq-count)
  edebug-before(1)
  (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties)))
  (progn (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))
  eval((progn (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties)))) nil)
  #[128 "\301\302\303 B \"D\207" [lexical-binding quote eval progn] 5
1802730]((edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))
  macroexpand((eval-and-compile (edebug-after (edebug-before 1) 2 (boundp
(quote parse-sexp-lookup-properties)))) nil)
  macroexp-macroexpand((eval-and-compile (edebug-after (edebug-before 1) 2
(boundp (quote parse-sexp-lookup-properties)))) nil)
  macroexp--expand-all((eval-and-compile (edebug-after (edebug-before 1) 2
(boundp (quote parse-sexp-lookup-properties)))))
  macroexp--all-forms((edebug-after (edebug-before 0) 3 (eval-and-compile
(edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))) 1)
  #[514 "\300 \301\"\211\204 \302 \242\303\"\202g\304 !\204+\3051$\306\307
! \"0\202*\310\311 \"\262 \210\312 \242\" \242 =\203b \242\302
\242\303\"\211\262 \232\203J \242\202e \312 \"\240\210\211
\242=\203\\\211\202e\313 !\202e\313 !\262 \207" [function-get
compiler-macro macroexp--all-forms 1 functionp (debug error)
autoload-do-load indirect-function message "macroexp--expand-all: %S"
macroexp--compiler-macro macroexp--expand-all] 8 "\n\n(fn FORM
FUNC)"](((edebug-after (edebug-before 0) 3 (eval-and-compile (edebug-after
(edebug-before 1) 2 (boundp (quote parse-sexp-lookup-properties))))))
edebug-after)
  macroexp--expand-all((edebug-after (edebug-before 0) 3 (eval-and-compile
(edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))))
  macroexp--all-forms((lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties)))))) 2)
  macroexp--expand-all((function (lambda nil (edebug-after (edebug-before
0) 3 (eval-and-compile (edebug-after (edebug-before 1) 2 (boundp (quote
parse-sexp-lookup-properties))))))))
  macroexp--all-forms((edebug-enter (quote test) (list limit) (function
(lambda nil (edebug-after (edebug-before 0) 3 (eval-and-compile
(edebug-after (edebug-before 1) 2 (boundp ...))))))) 1)
  #[514 "\300 \301\"\211\204 \302 \242\303\"\202g\304 !\204+\3051$\306\307
! \"0\202*\310\311 \"\262 \210\312 \242\" \242 =\203b \242\302
\242\303\"\211\262 \232\203J \242\202e \312 \"\240\210\211
\242=\203\\\211\202e\313 !\202e\313 !\262 \207" [function-get
compiler-macro macroexp--all-forms 1 functionp (debug error)
autoload-do-load indirect-function message "macroexp--expand-all: %S"
macroexp--compiler-macro macroexp--expand-all] 8 "\n\n(fn FORM
FUNC)"](((edebug-enter (quote test) (list limit) (function (lambda nil
(edebug-after (edebug-before 0) 3 (eval-and-compile (edebug-after ... 2
...))))))) edebug-enter)
  macroexp--expand-all((edebug-enter (quote test) (list limit) (function
(lambda nil (edebug-after (edebug-before 0) 3 (eval-and-compile
(edebug-after (edebug-before 1) 2 (boundp ...))))))))
  macroexp--all-forms((lambda (limit) (edebug-enter (quote test) (list
limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after ... 2 ...))))))) 2)
  macroexp--expand-all((function (lambda (limit) (edebug-enter (quote test)
(list limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile ...))))))))
  macroexp--all-forms((defalias (quote test) (function (lambda (limit)
(edebug-enter (quote test) (list limit) (function (lambda nil (edebug-after
... 3 ...))))))) 1)
  #[514 "\300 \301\"\211\204 \302 \242\303\"\202g\304 !\204+\3051$\306\307
! \"0\202*\310\311 \"\262 \210\312 \242\" \242 =\203b \242\302
\242\303\"\211\262 \232\203J \242\202e \312 \"\240\210\211
\242=\203\\\211\202e\313 !\202e\313 !\262 \207" [function-get
compiler-macro macroexp--all-forms 1 functionp (debug error)
autoload-do-load indirect-function message "macroexp--expand-all: %S"
macroexp--compiler-macro macroexp--expand-all] 8 "\n\n(fn FORM
FUNC)"](((defalias (quote test) (function (lambda (limit) (edebug-enter
(quote test) (list limit) (function (lambda nil ...))))))) defalias)
  macroexp--expand-all((defun test (limit) (edebug-enter (quote test) (list
limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after ... 2 ...))))))))
  macroexpand-all((defun test (limit) (edebug-enter (quote test) (list
limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after ... 2 ...))))))))
  eval-sexp-add-defvars((defun test (limit) (edebug-enter (quote test)
(list limit) (function (lambda nil (edebug-after (edebug-before 0) 3
(eval-and-compile (edebug-after ... 2 ...))))))))
  edebug-eval-defun((4))
  apply(edebug-eval-defun (4))
  eval-defun((4))
  funcall-interactively(eval-defun (4))
  call-interactively(eval-defun nil nil)
  command-execute(eval-defun)

    -- Anders


On Thu, Dec 29, 2016 at 10:29 AM, Alan Mackenzie <acm <at> muc.de> wrote:

> Bug fixed.
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>

[Message part 2 (text/html, inline)]

Message #66 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Gemini Lasswell <gazally <at> runbox.com>
To: Alan Mackenzie <acm <at> muc.de>
Cc: David Bremner <david <at> tethera.net>, Katsumi Yamaoka <yamaoka <at> jpl.org>,
 16184 <at> debbugs.gnu.org, Anders Lindgren <andlind <at> gmail.com>,
 Benaiah Mischenko <benaiah <at> mischenko.com>
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Fri, 30 Dec 2016 15:27:37 -0800

Alan Mackenzie <acm <at> muc.de> writes:

>
> .  During this instrumentation, the inside of the cc-eval-when-compile
> form gets run, in particular, the (edebug-before 2).  At this stage,
> since edebug-enter hasn't been run at all, edebug-offset-indices is still
> nil.  The (edebug-before 2) thus attempts (setcar nil 2), which throws
> the error.
>

I haven't able to reproduce the bug with cc-eval-when-compile,
just eval-and-compile. But the thing that is supposed to make Edebug
wrap a form in edebug-enter is the use of def-form or def-body in the
Edebug spec. It works for eval-when-compile which has the Edebug spec
(&rest def-form). The body of eval-and-compile doesn't get wrapped
because its Edebug spec is t, so the bug happens there.

cc-eval-when-compile has the same Edebug spec as eval-when-compile, so
its body should get wrapped by edebug-enter. If that's not happening in
your Emacs, it's a bug in Edebug which is different from the
eval-and-compile Edebug spec bug.

Message #69 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Gemini Lasswell <gazally <at> runbox.com>
Cc: David Bremner <david <at> tethera.net>, Katsumi Yamaoka <yamaoka <at> jpl.org>,
 16184 <at> debbugs.gnu.org, Anders Lindgren <andlind <at> gmail.com>,
 Benaiah Mischenko <benaiah <at> mischenko.com>
Subject: Re: bug#16184: 24.3.50; edebug and eval-when-compiler don't work
 together
Date: Sat, 31 Dec 2016 11:18:52 +0000

Hello, Gemini.

On Fri, Dec 30, 2016 at 03:27:37PM -0800, Gemini Lasswell wrote:
> Alan Mackenzie <acm <at> muc.de> writes:

> > .  During this instrumentation, the inside of the cc-eval-when-compile
> > form gets run, in particular, the (edebug-before 2).  At this stage,
> > since edebug-enter hasn't been run at all, edebug-offset-indices is still
> > nil.  The (edebug-before 2) thus attempts (setcar nil 2), which throws
> > the error.

> I haven't able to reproduce the bug with cc-eval-when-compile,
> just eval-and-compile. But the thing that is supposed to make Edebug
> wrap a form in edebug-enter is the use of def-form or def-body in the
> Edebug spec.

This is extremely poorly documented, even in the source code, if it is
documented at all.  I don't really understand the documentation of the
edebug spec in general.

> It works for eval-when-compile which has the Edebug spec (&rest
> def-form). The body of eval-and-compile doesn't get wrapped because
> its Edebug spec is t, so the bug happens there.

Ah!  I think I'm beginning to understand all this.  Looking at the
source code, it seems a def-form or def-body causes the form to be
wrapping by edebug-enter, but not much else does.

> cc-eval-when-compile has the same Edebug spec as eval-when-compile, so
> its body should get wrapped by edebug-enter. If that's not happening in
> your Emacs, it's a bug in Edebug which is different from the
> eval-and-compile Edebug spec bug.

I think you've persuaded me that my fix to this bug was wrong.  Rather
than adapting initialisations to cope with edebug-enter not being
called, I should have diagnosed _why_ an edebug-enter is not being
generated during instrumentation of a cc-eval-when-compile form.

I'm working on this now.  Thank you indeed for drawing my attention to
these edebug specs.

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #72 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Anders Lindgren <andlind <at> gmail.com>
Cc: Benaiah Mischenko <benaiah <at> mischenko.com>,
 Gemini Lasswell <gazally <at> runbox.com>, Katsumi Yamaoka <yamaoka <at> jpl.org>,
 16184 <at> debbugs.gnu.org, David Bremner <david <at> tethera.net>
Subject: Re: bug#16184: 24.3.50; edebug and eval-when-compiler don't work
 together
Date: Sat, 31 Dec 2016 16:22:54 +0000

Hello again, Anders.

On Fri, Dec 30, 2016 at 09:02:31PM +0100, Anders Lindgren wrote:
> Hi!

> I just got around to test this. Now, `eval-when-comple' seems to be
> working. Unfortunately, `eval-and-comple' is still broken.

> If I do C-u M-C-x eval the following with `debug-on-error' set to t, I get
> the backtrace below:

> (defun test (limit)
>   (eval-and-compile
>     (boundp 'parse-sexp-lookup-properties)))

[ snip backtrace ]

I got a tip from Gemini Lasswell last night.  What is broken is not
edebug.el itself, but the edebug specs applied to various macros.  For
macros like eval-\(when\|and\)-compile, there needs to be an
edebug-enter form compile around the macro.  Most edebug specs don't
compile this - only (I think) def-form and def-body do.  This is not
clearly documented.

Anyhow, I've changed the edebug spec of eval-and-compile to what it
should be (\(cc-\)?eval-when-compile already had it), and I think these
problems are all solved.

Would you please try out the following patch, and let me know sometime
that it's OK, then I can commit it.

Happy New Year again!



diff --git a/lisp/emacs-lisp/byte-run.el b/lisp/emacs-lisp/byte-run.el
index 9d2a048..e680ebb 100644
--- a/lisp/emacs-lisp/byte-run.el
+++ b/lisp/emacs-lisp/byte-run.el
@@ -474,7 +474,7 @@ eval-and-compile
 `progn', except that the value of the expression may be (but is
 not necessarily) computed at load time if eager macro expansion
 is enabled."
-  (declare (debug t) (indent 0))
+  (declare (debug (&rest def-form)) (indent 0))
   ;; When the byte-compiler expands code, this macro is not used, so we're
   ;; either about to run `body' (plain interpretation) or we're doing eager
   ;; macroexpansion.
diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
index 15b1389..fd3e48d 100644
--- a/lisp/emacs-lisp/edebug.el
+++ b/lisp/emacs-lisp/edebug.el
@@ -2052,15 +2052,10 @@ edebug-stack
 (defvar edebug-stack-depth -1)
 ;; Index of last edebug-stack item.
 
-(defvar edebug-offset-indices (list 0))
+(defvar edebug-offset-indices nil)
 ;; Stack of offset indices of visited edebug sexps.
 ;; Each function adds one cons.  Top is modified with setcar.
-;; Should be nil at the top level.  No longer!  There are occasions
-;; when this variable is modified by setcar before a cons has been
-;; pushed by `edebug-enter', e.g. when instrumenting
-;; `c-font-lock-declarations' in .../lisp/progmodes/cc-fonts.el.  So
-;; this variable must be initialized to a cons.
-
+;; Should be nil at the top level.
 (defvar edebug-entered nil
   ;; Non-nil if edebug has already been entered at this recursive edit level.
   ;; This should stay nil at the top level.

>     -- Anders

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #75 received at 16184 <at> debbugs.gnu.org (full text, mbox):

From: Anders Lindgren <andlind <at> gmail.com>
To: Alan Mackenzie <acm <at> muc.de>
Cc: Benaiah Mischenko <benaiah <at> mischenko.com>,
 Gemini Lasswell <gazally <at> runbox.com>, Katsumi Yamaoka <yamaoka <at> jpl.org>,
 16184 <at> debbugs.gnu.org, David Bremner <david <at> tethera.net>
Subject: Re: bug#16184: 24.3.50;
 edebug and eval-when-compiler don't work together
Date: Sun, 1 Jan 2017 01:30:44 +0100

[Message part 1 (text/plain, inline)]

Hi Alan!

This time it seems to be working properly with both `eval-when-compile' and
`eval-and-compile'!

Happy new year to you too!

    -- Anders

On Sat, Dec 31, 2016 at 5:22 PM, Alan Mackenzie <acm <at> muc.de> wrote:

> Hello again, Anders.
>
> On Fri, Dec 30, 2016 at 09:02:31PM +0100, Anders Lindgren wrote:
> > Hi!
>
> > I just got around to test this. Now, `eval-when-comple' seems to be
> > working. Unfortunately, `eval-and-comple' is still broken.
>
> > If I do C-u M-C-x eval the following with `debug-on-error' set to t, I
> get
> > the backtrace below:
>
> > (defun test (limit)
> >   (eval-and-compile
> >     (boundp 'parse-sexp-lookup-properties)))
>
> [ snip backtrace ]
>
> I got a tip from Gemini Lasswell last night.  What is broken is not
> edebug.el itself, but the edebug specs applied to various macros.  For
> macros like eval-\(when\|and\)-compile, there needs to be an
> edebug-enter form compile around the macro.  Most edebug specs don't
> compile this - only (I think) def-form and def-body do.  This is not
> clearly documented.
>
> Anyhow, I've changed the edebug spec of eval-and-compile to what it
> should be (\(cc-\)?eval-when-compile already had it), and I think these
> problems are all solved.
>
> Would you please try out the following patch, and let me know sometime
> that it's OK, then I can commit it.
>
> Happy New Year again!
>
>
>
> diff --git a/lisp/emacs-lisp/byte-run.el b/lisp/emacs-lisp/byte-run.el
> index 9d2a048..e680ebb 100644
> --- a/lisp/emacs-lisp/byte-run.el
> +++ b/lisp/emacs-lisp/byte-run.el
> @@ -474,7 +474,7 @@ eval-and-compile
>  `progn', except that the value of the expression may be (but is
>  not necessarily) computed at load time if eager macro expansion
>  is enabled."
> -  (declare (debug t) (indent 0))
> +  (declare (debug (&rest def-form)) (indent 0))
>    ;; When the byte-compiler expands code, this macro is not used, so we're
>    ;; either about to run `body' (plain interpretation) or we're doing
> eager
>    ;; macroexpansion.
> diff --git a/lisp/emacs-lisp/edebug.el b/lisp/emacs-lisp/edebug.el
> index 15b1389..fd3e48d 100644
> --- a/lisp/emacs-lisp/edebug.el
> +++ b/lisp/emacs-lisp/edebug.el
> @@ -2052,15 +2052,10 @@ edebug-stack
>  (defvar edebug-stack-depth -1)
>  ;; Index of last edebug-stack item.
>
> -(defvar edebug-offset-indices (list 0))
> +(defvar edebug-offset-indices nil)
>  ;; Stack of offset indices of visited edebug sexps.
>  ;; Each function adds one cons.  Top is modified with setcar.
> -;; Should be nil at the top level.  No longer!  There are occasions
> -;; when this variable is modified by setcar before a cons has been
> -;; pushed by `edebug-enter', e.g. when instrumenting
> -;; `c-font-lock-declarations' in .../lisp/progmodes/cc-fonts.el.  So
> -;; this variable must be initialized to a cons.
> -
> +;; Should be nil at the top level.
>  (defvar edebug-entered nil
>    ;; Non-nil if edebug has already been entered at this recursive edit
> level.
>    ;; This should stay nil at the top level.
>
> >     -- Anders
>
> --
> Alan Mackenzie (Nuremberg, Germany).
>

[Message part 2 (text/html, inline)]

Message #78 received at 16184-done <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Anders Lindgren <andlind <at> gmail.com>
Cc: Gemini Lasswell <gazally <at> runbox.com>, Katsumi Yamaoka <yamaoka <at> jpl.org>,
 Benaiah Mischenko <benaiah <at> mischenko.com>, David Bremner <david <at> tethera.net>,
 16184-done <at> debbugs.gnu.org
Subject: Re: bug#16184: 24.3.50; edebug and eval-when-compiler don't work
 together
Date: Sun, 1 Jan 2017 08:37:11 +0000

Hello, Anders.

On Sun, Jan 01, 2017 at 01:30:44AM +0100, Anders Lindgren wrote:
> Hi Alan!

> This time it seems to be working properly with both `eval-when-compile' and
> `eval-and-compile'!

> Happy new year to you too!

Thanks!  I've just committed the fix.

>     -- Anders

-- 
Alan Mackenzie (Nuremberg, Germany).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.