Subject: bug#8338: lexbind lisp error

From: "Eric M. Ludlam" <[email-masked]>
Date: Mon, 28 Mar 2011 21:49:57 -0400

Hi Stefan,

I don't think eieio is at fault here.  Instead, there is some semanticdb 
table that has an empty reference to it's parent database.

That seems to be caused because of a custom c-mode-hook which is adding 
the pre-processor symbol.  It turns out this is not needed because you 
can run that code outside the hook, even if c mode isn't available.

I suspect without testing anything that this is because some pretty 
serious machinery is getting used before semantic has initialized that 
buffer, since the mode-hook is used to bootstrap parser values. 
(Though, the Emacs version is different now, so I don't know how it 
works in core Emacs.)

What that has to do with lexbind I cannot guess, unless mode-local now 
initializes things in a different order?

Does that help?
Eric

On 03/28/2011 08:57 PM, Stefan Monnier wrote:
> Hi Eric,
>
> Could you help me figure out what's going on with cedet/eieio on the
> lexbind branch?  It's currently the only known bug on this branch which
> is otherwise pretty much "ready for merge".
>
>> After starting Emacs use gtags-find-tag to find a tag.  Here is the
>> back trace
>
> You can find the complete backtrace at
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8338, but I include
> a hand-edited version of it:
>
>> Debugger entered--Lisp error: (unbound-slot "#<class semanticdb-table>" ...)
>>    signal(unbound-slot ...)
>>    eieio-default-superclass(...)
>>    apply(eieio-default-superclass ...)
>>    eieio-generic-call-primary-only(slot-unbound ...)
>>    slot-unbound(...)
>>    eieio-barf-if-slot-unbound(unbound ...)
>>    eieio-oref(...)
>>    semanticdb-table(...)
>>    apply(semanticdb-table ...)
>>    eieio-generic-call-primary-only(semanticdb-full-filename ...)
>>    semanticdb-full-filename(...)
>>    #[(obj)<bytecode>  <csts>  4 "Return non-nil of OBJ's tag list is out of date.\nThe file associated with OBJ does not need to be in a buffer."](...)
>>    apply(...)
>>    semanticdb-needs-refresh-p(...)
>>    semantic-c-reset-preprocessor-symbol-map()
>>    byte-code(...)
>>    (semantic-c-add-preprocessor-symbol "__KERNEL__" "")
>      [...]
>>    run-hooks(c-mode-common-hook c-mode-hook)
>>    apply(run-hooks (c-mode-common-hook c-mode-hook))
>>    run-mode-hooks(c-mode-common-hook c-mode-hook)
>>    c-mode()
>
> The lexbind branch should normally be 100% backward compatible, but of
> course, this is not quite the case.  There are a few incompatible changes,
> most notably the `byte-compile' property works differently because of
> some modifications that were necessary in the byte-compiler to compile
> lexically scoped code.  So I had to make some changes to eieio and cedet
> to get the code to compile.  I've included them after my signature.
> Could you please take a look at these changes (which should work
> correctly in Emacs-23 as well), as well as take a look at the above
> backtrace and try to help me figure out what might have gone wrong (I
> don't know much if anything about eieio, so the above backtrace doesn't
> tell me much).
>
>
>          Stefan
>
>
> === modified file 'lisp/cedet/ChangeLog'
> --- lisp/cedet/ChangeLog	2011-03-12 19:19:47 +0000
> +++ lisp/cedet/ChangeLog	2011-02-21 23:38:07 +0000
> @@ -1,3 +1,8 @@
> +2011-02-21  Stefan Monnier<[email-masked]>
> +
> +	* semantic/wisent/comp.el (wisent-byte-compile-grammar):
> +	Macroexpand before passing to byte-compile-form.
> +
>   2011-01-13  Stefan Monnier<[email-masked]>
>
>   	* srecode/srt-mode.el (srecode-template-mode): Use define-derived-mode.
>
> === modified file 'lisp/cedet/semantic/wisent/comp.el'
> --- lisp/cedet/semantic/wisent/comp.el	2011-01-25 04:08:28 +0000
> +++ lisp/cedet/semantic/wisent/comp.el	2011-02-21 23:25:12 +0000
> @@ -3452,15 +3452,13 @@
>     (if (wisent-automaton-p grammar)
>         grammar ;; Grammar already compiled just return it
>       (wisent-with-context compile-grammar
> -      (let* ((gc-cons-threshold 1000000)
> -             automaton)
> +      (let* ((gc-cons-threshold 1000000))
>           (garbage-collect)
>   	(setq wisent-new-log-flag t)
>   	;; Parse input grammar
>   	(wisent-parse-grammar grammar start-list)
>   	;; Generate the LALR(1) automaton
> -	(setq automaton (wisent-parser-automaton))
> -	automaton))))
> +	(wisent-parser-automaton)))))
>   
>   ;;;; --------------------------
>   ;;;; Byte compile input grammar
> @@ -3476,7 +3474,15 @@
>     ;; automaton internal data structure.  Then, because the internal
>     ;; data structure contains an obarray, convert it to a lisp form so
>     ;; it can be byte-compiled.
> -  (byte-compile-form (wisent-automaton-lisp-form (eval form))))
> +  (byte-compile-form
> +   ;; FIXME: we macroexpand here since `byte-compile-form' expects
> +   ;; macroexpanded code, but that's just a workaround: for lexical-binding
> +   ;; the lisp form should have to pass through closure-conversion and
> +   ;; `wisent-byte-compile-grammar' is called much too late for that.
> +   ;; Why isn't this `wisent-automaton-lisp-form' performed at
> +   ;; macroexpansion time?  --Stef
> +   (macroexpand-all
> +    (wisent-automaton-lisp-form (eval form)))))
>
>   (put 'wisent-compile-grammar 'byte-compile 'wisent-byte-compile-grammar)
>
>
> === removed file 'lisp/emacs-lisp/eieio-comp.el'
> --- lisp/emacs-lisp/eieio-comp.el	2011-01-26 08:36:39 +0000
> +++ lisp/emacs-lisp/eieio-comp.el	1970-01-01 00:00:00 +0000
> @@ -1,142 +0,0 @@
> -;;; eieio-comp.el -- eieio routines to help with byte compilation
> -
> -;; Copyright (C) 1995-1996, 1998-2002, 2005, 2008-2011
> -;;   Free Software Foundation, Inc.
> -
> -;; Author: Eric M. Ludlam<[email-masked]>
> -;; Version: 0.2
> -;; Keywords: lisp, tools
> -;; Package: eieio
> -
> -;; This file is part of GNU Emacs.
> -
> -;; GNU Emacs is free software: you can redistribute it and/or modify
> -;; it under the terms of the GNU General Public License as published by
> -;; the Free Software Foundation, either version 3 of the License, or
> -;; (at your option) any later version.
> -
> -;; GNU Emacs is distributed in the hope that it will be useful,
> -;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> -;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> -;; GNU General Public License for more details.
> -
> -;; You should have received a copy of the GNU General Public License
> -;; along with GNU Emacs.  If not, see<http://www.gnu.org/licenses/>.
> -
> -;;; Commentary:
> -
> -;; Byte compiler functions for defmethod.  This will affect the new GNU
> -;; byte compiler for Emacs 19 and better.  This function will be called by
> -;; the byte compiler whenever a `defmethod' is encountered in a file.
> -;; It will output a function call to `eieio-defmethod' with the byte
> -;; compiled function as a parameter.
> -
> -;;; Code:
> -
> -(declare-function eieio-defgeneric-form "eieio" (method doc-string))
> -
> -;; Some compatibility stuff
> -(eval-and-compile
> -  (if (not (fboundp 'byte-compile-compiled-obj-to-list))
> -      (defun byte-compile-compiled-obj-to-list (moose) nil))
> -
> -  (if (not (boundp 'byte-compile-outbuffer))
> -      (defvar byte-compile-outbuffer nil))
> -  )
> -
> -;; This teaches the byte compiler how to do this sort of thing.
> -(put 'defmethod 'byte-hunk-handler 'byte-compile-file-form-defmethod)
> -
> -(defun byte-compile-file-form-defmethod (form)
> -  "Mumble about the method we are compiling.
> -This function is mostly ripped from `byte-compile-file-form-defun',
> -but it's been modified to handle the special syntax of the `defmethod'
> -command.  There should probably be one for `defgeneric' as well, but
> -that is called but rarely.  Argument FORM is the body of the method."
> -  (setq form (cdr form))
> -  (let* ((meth (car form))
> -	 (key (progn (setq form (cdr form))
> -		     (cond ((or (eq ':BEFORE (car form))
> -				(eq ':before (car form)))
> -			    (setq form (cdr form))
> -			    ":before ")
> -			   ((or (eq ':AFTER (car form))
> -				(eq ':after (car form)))
> -			    (setq form (cdr form))
> -			    ":after ")
> -			   ((or (eq ':PRIMARY (car form))
> -				(eq ':primary (car form)))
> -			    (setq form (cdr form))
> -			    ":primary ")
> -			   ((or (eq ':STATIC (car form))
> -				(eq ':static (car form)))
> -			    (setq form (cdr form))
> -			    ":static ")
> -			   (t ""))))
> -	 (params (car form))
> -	 (lamparams (byte-compile-defmethod-param-convert params))
> -	 (arg1 (car params))
> -	 (class (if (listp arg1) (nth 1 arg1) nil))
> -	 (my-outbuffer (if (eval-when-compile (featurep 'xemacs))
> -			   byte-compile-outbuffer
> -			 (cond ((boundp 'bytecomp-outbuffer)
> -				bytecomp-outbuffer) ; Emacs>= 23.2
> -			       ((boundp 'outbuffer) outbuffer)
> -			       (t (error "Unable to set outbuffer"))))))
> -    (let ((name (format "%s::%s" (or class "#<generic>") meth)))
> -      (if byte-compile-verbose
> -	  ;; #### filename used free
> -	  (message "Compiling %s... (%s)"
> -		   (cond ((boundp 'bytecomp-filename) bytecomp-filename)
> -			 ((boundp 'filename) filename)
> -			 (t ""))
> -		   name))
> -      (setq byte-compile-current-form name) ; for warnings
> -      )
> -    ;; Flush any pending output
> -    (byte-compile-flush-pending)
> -    ;; Byte compile the body.  For the byte compiled forms, add the
> -    ;; rest arguments, which will get ignored by the engine which will
> -    ;; add them later (I hope)
> -    (let* ((new-one (byte-compile-lambda
> -		     (append (list 'lambda lamparams)
> -			     (cdr form))))
> -	   (code (byte-compile-byte-code-maker new-one)))
> -      (princ "\n(eieio-defmethod '" my-outbuffer)
> -      (princ meth my-outbuffer)
> -      (princ " '(" my-outbuffer)
> -      (princ key my-outbuffer)
> -      (prin1 params my-outbuffer)
> -      (princ " " my-outbuffer)
> -      (prin1 code my-outbuffer)
> -      (princ "))" my-outbuffer)
> -      )
> -    ;; Now add this function to the list of known functions.
> -    ;; Don't bother with a doc string.   Not relevant here.
> -    (add-to-list 'byte-compile-function-environment
> -		 (cons meth
> -		       (eieio-defgeneric-form meth "")))
> -
> -    ;; Remove it from the undefined list if it is there.
> -    (let ((elt (assq meth byte-compile-unresolved-functions)))
> -      (if elt (setq byte-compile-unresolved-functions
> -		    (delq elt byte-compile-unresolved-functions))))
> -
> -    ;; nil prevents cruft from appearing in the output buffer.
> -    nil))
> -
> -(defun byte-compile-defmethod-param-convert (paramlist)
> -  "Convert method params into the params used by the `defmethod' thingy.
> -Argument PARAMLIST is the parameter list to convert."
> -  (let ((argfix nil))
> -    (while paramlist
> -      (setq argfix (cons (if (listp (car paramlist))
> -			     (car (car paramlist))
> -			   (car paramlist))
> -			 argfix))
> -      (setq paramlist (cdr paramlist)))
> -    (nreverse argfix)))
> -
> -(provide 'eieio-comp)
> -
> -;;; eieio-comp.el ends here
>
> === modified file 'lisp/emacs-lisp/eieio.el'
> --- lisp/emacs-lisp/eieio.el	2011-02-18 08:00:08 +0000
> +++ lisp/emacs-lisp/eieio.el	2011-02-24 16:37:15 +0000
> @@ -45,8 +45,7 @@
>   ;;; Code:
>
>   (eval-when-compile
> -  (require 'cl)
> -  (require 'eieio-comp))
> +  (require 'cl))
>
>   (defvar eieio-version "1.3"
>     "Current version of EIEIO.")
> @@ -123,6 +122,7 @@
>   ;; while it is being built itself.
>   (defvar eieio-default-superclass nil)
>
> +;; FIXME: The constants below should have a `eieio-' prefix added!!
>   (defconst class-symbol 1 "Class's symbol (self-referencing.).")
>   (defconst class-parent 2 "Class parent slot.")
>   (defconst class-children 3 "Class children class slot.")
> @@ -181,10 +181,6 @@
>   	(t key) ;; already generic.. maybe.
>   	))
>
> -;; How to specialty compile stuff.
> -(autoload 'byte-compile-file-form-defmethod "eieio-comp"
> -  "This function is used to byte compile methods in a nice way.")
> -(put 'defmethod 'byte-hunk-handler 'byte-compile-file-form-defmethod)
>   
>   ;;; Important macros used in eieio.
>   ;;
> @@ -1192,10 +1188,8 @@
>     ;; is faster to execute this for not byte-compiled.  ie, install this,
>     ;; then measure calls going through here.  I wonder why.
>     (require 'bytecomp)
> -  (let ((byte-compile-free-references nil)
> -	(byte-compile-warnings nil)
> -	)
> -    (byte-compile-lambda
> +  (let ((byte-compile-warnings nil))
> +    (byte-compile
>        `(lambda (&rest local-args)
>   	,doc-string
>   	;; This is a cool cheat.  Usually we need to look up in the
> @@ -1205,7 +1199,8 @@
>   	;; of that one implementation, then clearly, there is no method def.
>   	(if (not (eieio-object-p (car local-args)))
>   	    ;; Not an object.  Just signal.
> -	    (signal 'no-method-definition (list ,(list 'quote method) local-args))
> +	    (signal 'no-method-definition
> +                    (list ,(list 'quote method) local-args))
>
>   	  ;; We do have an object.  Make sure it is the right type.
>   	  (if ,(if (eq class eieio-default-superclass)
> @@ -1228,9 +1223,7 @@
>   		  )
>   	      (apply ,(list 'quote impl) local-args)
>   	      ;(,impl local-args)
> -	      ))))
> -     )
> -  ))
> +	      )))))))
>
>   (defsubst eieio-defgeneric-reset-generic-form-primary-only-one (method)
>     "Setup METHOD to call the generic form."
> @@ -1296,9 +1289,35 @@
>                        ((typearg class-name) arg2&optional opt&rest rest)
>       \"doc-string\"
>        body)"
> -  `(eieio-defmethod (quote ,method) (quote ,args)))
> +  (let* ((key (cond ((or (eq ':BEFORE (car args))
> +                         (eq ':before (car args)))
> +                     (setq args (cdr args))
> +                     :before)
> +                    ((or (eq ':AFTER (car args))
> +                         (eq ':after (car args)))
> +                     (setq args (cdr args))
> +                     :after)
> +                    ((or (eq ':PRIMARY (car args))
> +                         (eq ':primary (car args)))
> +                     (setq args (cdr args))
> +                     :primary)
> +                    ((or (eq ':STATIC (car args))
> +                         (eq ':static (car args)))
> +                     (setq args (cdr args))
> +                     :static)
> +                    (t nil)))
> +	 (params (car args))
> +	 (lamparams
> +          (mapcar (lambda (param) (if (listp param) (car param) param))
> +                  params))
> +	 (arg1 (car params))
> +	 (class (if (listp arg1) (nth 1 arg1) nil)))
> +    `(eieio-defmethod ',method
> +                      '(,@(if key (list key))
> +                        ,params)
> +                      (lambda ,lamparams ,@(cdr args)))))
>
> -(defun eieio-defmethod (method args)
> +(defun eieio-defmethod (method args&optional code)
>     "Work part of the `defmethod' macro defining METHOD with ARGS."
>     (let ((key nil) (body nil) (firstarg nil) (argfix nil) (argclass nil) loopa)
>       ;; find optional keys
> @@ -1352,10 +1371,7 @@
>         ;; generics are higher
>         (setq key (eieio-specialized-key-to-generic-key key)))
>       ;; Put this lambda into the symbol so we can find it
> -    (if (byte-code-function-p (car-safe body))
> -	(eieiomt-add method (car-safe body) key argclass)
> -      (eieiomt-add method (append (list 'lambda (reverse argfix)) body)
> -		   key argclass))
> +    (eieiomt-add method code key argclass)
>       )
>
>     (when eieio-optimize-primary-methods-flag
>