"J.D. Smith" <jdtsmith@gmail.com> writes:

> Here is an update on progress.  The attached is working reasonably well,
> reproducing lisp/loaddefs.el with a few changes (see attached).  I went
> with (declare ((autoload . macro-expand))) on the notion that there
> might be other autoload-related declarations in the future, but open to
> suggestion.

Please have a look at this latest patch, which I believe solves the
mentioned problems.

Changes:

1. I implemented the list 2 bug fix for autoload's INTERACTIVE argument
   described above, correctly keeping interactive arg specs out of the
   loaddefs files.

2. I moved the autoload macro-expand declare up front everywhere it's
   used[1], to ensure the macro-expand request will be included in
   *-loaddefs.el files.  This is important for packages which manage
   their own *-loaddefs.el files (e.g. `cl-macs').
   
3. I have kept list 2 around[2], and count presence on this list, or the
   list of 4 specially-handled macros (defgroup, defcustom, deftheme,
   defclass) as "we know about this car".  No file load will be
   attempted for such special macros, so only proper "user-defined
   UNKNOWN macros" will result in a load attempt.

With these, the loaddefs.el file changes only modestly (see attached),
and (almost) no files have load attempted.

About that almost.  There is one small order-of-operation issue yet to
be solved, for packages which do clever things.  Take any of the
`net/tramp-*.el` files.  They all require `tramp-compat`, which requires
`tramp-loaddefs', which is itself an autoload file the build process is
generating.  For example, tramp's autoload file eventually includes the
macro `tramp--with-startup'.  When one of tramp's autoloads use an
unknown macro like this, there will be an attempt to load the file,
implicitly requiring that yet-to-exist `tramp-loaddefs.el' file,
resulting in a (handled) error.  Classic chicken and egg.

    INFO     Scraping 1543 files for loaddefs...70% 
  loaddefs-gen: loading file tramp-adb (for tramp--with-startup)
  Loading /Users/jdsmith/code/emacs/emacs-mac/lisp/net/tramp-adb.el (source)...
  Warning (emacs): loaddefs-gen: load error
  	(file-missing Cannot open load file No such file or directory tramp-loaddefs)
  loaddefs-gen: loading file tramp-androidsu (for tramp--with-startup)
  Loading /Users/jdsmith/code/emacs/emacs-mac/lisp/net/tramp-androidsu.el (source)...
  Warning (emacs): loaddefs-gen: load error
  	(file-missing Cannot open load file No such file or directory tramp-loaddefs)
  ...

In this case these are harmless warnings, since in fact
`tramp--with-startup' is /not/ requesting macro-expand, so failing to
define it doesn't matter.  Subsequent builds do not show this error.
But this could be alarming to people, so we probably need a solution.

[1] Note that the order of declare entries already matters, because some
    of them generate forms which include `:autoload-end', and this
    token, once encountered _anywhere_, puts an absolute stop to
    emitting additional declare-generated forms into loaddefs.
    E.g. `(declare (debug ...))' does this.  This, by the way, is why
    /some/ macros have their (e.g.) `indent' declares passed through to
    the loaddefs files, and others do not.

[2] Some of these macros aren't yet suited for full expansion anyway.
    For example `define-derived-mode' includes a bunch of unwanted forms
    if it gets the expansion treatment.  So for now, leaving the
    shortcut list 2 intact seems to be the lesser evil.