GNU bug report logs - #79353
31.0.50; Eager macro expansion errors when building master

Previous Next

To reply to this bug, email your comments to 79353 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Cc: JD Smith <jdtsmith <at> gmail.com>
Subject: 31.0.50; Eager macro expansion errors when building master
Date: Sun, 31 Aug 2025 08:33:48 +0200

I am seeing errors while producing autoloads when building master from
git clean -xdf.

    INFO     Scraping 1518 files for loaddefs...70% 
  loaddefs-gen: loading file tramp-adb (for tramp--with-startup)
  Loading /Users/gerd/emacs/github/master/lisp/net/tramp-adb.el (source)...
  Warning (emacs): loaddefs-gen: load error
          (error Eager macro-expansion failure: (error "Eager macro-expansion failure: (void-function cl-subseq)"))

Bisected to (in CC):

  1f4e2e82649bb2a122b1406caf645ea06a933dc6 is the first bad commit
  commit 1f4e2e82649bb2a122b1406caf645ea06a933dc6
  Author: JD Smith <93749+jdtsmith <at> users.noreply.github.com>
  Date:   Sun Aug 10 17:17:47 2025 -0400
      Adapt tramp to new autoload-macro expand

      Bug #78995.

      * lisp/net/tramp-compat.el (tramp-loaddefs): suppress error on requiring
      tramp-loaddef.
      * lisp/net/tramp.el (tramp--with-startup): declare autoload-macro
      expand, and suppress warnings about this declare form on older versions
      of Emacs.
   lisp/net/tramp-compat.el | 2 +-
   lisp/net/tramp.el        | 7 +++++++
   2 files changed, 8 insertions(+), 1 deletion(-)

Message #8 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: "J.D. Smith" <jdtsmith <at> gmail.com>
To: Gerd Möllmann <gerd.moellmann <at> gmail.com>
Cc: luangruo <at> yahoo.com, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 monnier <at> iro.umontreal.ca
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Sun, 31 Aug 2025 09:19:38 -0400

Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:

> I am seeing errors while producing autoloads when building master from
> git clean -xdf.
>
>     INFO     Scraping 1518 files for loaddefs...70% 
>   loaddefs-gen: loading file tramp-adb (for tramp--with-startup)
>   Loading /Users/gerd/emacs/github/master/lisp/net/tramp-adb.el (source)...
>   Warning (emacs): loaddefs-gen: load error
>           (error Eager macro-expansion failure: (error "Eager macro-expansion failure: (void-function cl-subseq)"))
>
> Bisected to (in CC):
>
>   1f4e2e82649bb2a122b1406caf645ea06a933dc6 is the first bad commit
>   commit 1f4e2e82649bb2a122b1406caf645ea06a933dc6
>   Author: JD Smith <93749+jdtsmith <at> users.noreply.github.com>
>   Date:   Sun Aug 10 17:17:47 2025 -0400
>       Adapt tramp to new autoload-macro expand
>
>       Bug #78995.
>
>       * lisp/net/tramp-compat.el (tramp-loaddefs): suppress error on requiring
>       tramp-loaddef.
>       * lisp/net/tramp.el (tramp--with-startup): declare autoload-macro
>       expand, and suppress warnings about this declare form on older versions
>       of Emacs.
>    lisp/net/tramp-compat.el | 2 +-
>    lisp/net/tramp.el        | 7 +++++++
>    2 files changed, 8 insertions(+), 1 deletion(-)

This warning arises from the new dynamic `autoload-macro expand'
capability Stefan M. and I added.  I believe the chain of events is as
follows:

1. tramp-adb declares:

  ;;;###tramp-autoload
  (tramp--with-startup
  ...

2. During initial autoload generation, this macro is unknown.  So the
file tramp-adb.el is /loaded/, to give it a chance to define the macro.

3. The macro `tramp--with-startup' is indeed defined in tramp.el, which
tramp-adb requires.  So far so good.  But during the loading of
tramp.el, some other package it loads uses `cl-subseq' at the top level
without first requiring cl-extra.

5. Why has this not been an issue before?  The macro `cl-subseq' is
/itself/ an autoload in cl-extra.el.  Since the CL-specific loadefs file
`cl-loaddefs.el' does not yet exist (remember this is during autoload
generation), this causes the (handled) warning you see.

*Question*:

In the lisp/net/tramp-loaddefs.el file which was generated during this
build, what do you see under:

  ;;; Generated autoloads from tramp-adb.el

Can you try a build again with `make bootstrap' and see whether the
warning disappears and if that section of lisp/net/tramp-loaddefs.el
changes?  My guess is this is a harmless warning.

On a related issue, we still need to revert Po's commit e9800cabffa
which restored one of the hard-coded macro lists the autoload-macro
expand feature intentionally removed, possibly in an attempt to fix a
similar issue with recentf. Po, reminder to check that again with `make
bootstrap'.

I would like to use this report to test various flavors of "from
scratch" builds and see if there are any approaches to mitigate these
warnings (that's why I'm reporting them!).  Loading during autoload
generation should be rare, but is essential for external package authors
to request macro expansion during package install (think: modes which
provide custom convenience macros to wrap `define-minor-mode').

Message #11 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: "J.D. Smith" <jdtsmith <at> gmail.com>
Cc: luangruo <at> yahoo.com, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 monnier <at> iro.umontreal.ca
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Sun, 31 Aug 2025 16:03:39 +0200

[Message part 1 (text/plain, inline)]

"J.D. Smith" <jdtsmith <at> gmail.com> writes:

> *Question*:
>
> In the lisp/net/tramp-loaddefs.el file which was generated during this
> build, what do you see under:
>
>   ;;; Generated autoloads from tramp-adb.el

This is the tramp-loaddefs.el after building from git clean -xdf

[tramp-loaddefs.el-after-git-clean.el (application/emacs-lisp, attachment)]

[Message part 3 (text/plain, inline)]

> Can you try a build again with `make bootstrap' and see whether the
> warning disappears and if that section of lisp/net/tramp-loaddefs.el
> changes?  My guess is this is a harmless warning.

The errors are the same, and here's tramp-loaddefs.el after bootstrap

[tramp-loaddefs-after-bootstrap.el (application/emacs-lisp, attachment)]

Message #14 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "J.D. Smith" <jdtsmith <at> gmail.com>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 luangruo <at> yahoo.com, 79353 <at> debbugs.gnu.org, michael.albinus <at> gmx.de
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Sun, 31 Aug 2025 11:29:18 -0400

> 2. During initial autoload generation, this macro is unknown.  So the
> file tramp-adb.el is /loaded/, to give it a chance to define the macro.
>
> 3. The macro `tramp--with-startup' is indeed defined in tramp.el, which
> tramp-adb requires.  So far so good.  But during the loading of
> tramp.el, some other package it loads uses `cl-subseq' at the top level
> without first requiring cl-extra.
>
> 5. Why has this not been an issue before?  The macro `cl-subseq' is
> /itself/ an autoload in cl-extra.el.  Since the CL-specific loadefs file
> `cl-loaddefs.el' does not yet exist (remember this is during autoload
> generation), this causes the (handled) warning you see.

FWIW it sounds like a problem I've seen before in other circumstances
where I just added a `(require 'cl-extra)`, e.g.:

    lisp/emacs-lisp/comp-cstr.el:(require 'cl-extra) ;HACK: For `cl-find-class' when `cl-loaddefs' is missing.


- Stefan

Message #17 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: "J.D. Smith" <jdtsmith <at> gmail.com>
To: Gerd Möllmann <gerd.moellmann <at> gmail.com>
Cc: luangruo <at> yahoo.com, 79353 <at> debbugs.gnu.org, michael.albinus <at> gmx.de,
 monnier <at> iro.umontreal.ca
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Sun, 31 Aug 2025 13:43:57 -0400

Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:

> "J.D. Smith" <jdtsmith <at> gmail.com> writes:
>
>> *Question*:
>>
>> In the lisp/net/tramp-loaddefs.el file which was generated during this
>> build, what do you see under:
>>
>>   ;;; Generated autoloads from tramp-adb.el
>
> This is the tramp-loaddefs.el after building from git clean -xdf
>
>
>
>> Can you try a build again with `make bootstrap' and see whether the
>> warning disappears and if that section of lisp/net/tramp-loaddefs.el
>> changes?  My guess is this is a harmless warning.
>
> The errors are the same, and here's tramp-loaddefs.el after bootstrap

Thanks for checking.  These files are identical, and the error is
preventing tramp--with-startup from getting defined during autoload
generation, so autoloaded calls to it just get inlined as-is in
tramp-loaddefs.el.  If it were working as intended, most of those would
be replaced by `add-hook'.

Gerd, do you mind reverting e9800cabffa and seeing what if anything
changes (from a clean start)?  Probably nothing.

BTW, Stefan and Michael, I see that tramp actually autoloads more
complicated forms using `tramp--with-startup', e.g.:

 (when (featurep 'dbusbind)
   (tramp--with-startup ...))

which means the macro definition still needs to be included in the
autoload file as well.  For consistency, maybe it makes the best sense
NOT to delcare autoload-macro expand in tramp--with-startup.  Thoughts?

Message #20 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: "J.D. Smith" <jdtsmith <at> gmail.com>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 luangruo <at> yahoo.com, 79353 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Sun, 31 Aug 2025 19:47:20 +0200

"J.D. Smith" <jdtsmith <at> gmail.com> writes:

Hi,

> BTW, Stefan and Michael, I see that tramp actually autoloads more
> complicated forms using `tramp--with-startup', e.g.:
>
>  (when (featurep 'dbusbind)
>    (tramp--with-startup ...))
>
> which means the macro definition still needs to be included in the
> autoload file as well.  For consistency, maybe it makes the best sense
> NOT to delcare autoload-macro expand in tramp--with-startup.  Thoughts?

I've lost track long ago. No thoughts, therefore.

Best regards, Michael.

Message #23 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: "J.D. Smith" <jdtsmith <at> gmail.com>
Cc: luangruo <at> yahoo.com, 79353 <at> debbugs.gnu.org, michael.albinus <at> gmx.de,
 monnier <at> iro.umontreal.ca
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Sun, 31 Aug 2025 19:55:07 +0200

"J.D. Smith" <jdtsmith <at> gmail.com> writes:

> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>
>> "J.D. Smith" <jdtsmith <at> gmail.com> writes:
>>
>>> *Question*:
>>>
>>> In the lisp/net/tramp-loaddefs.el file which was generated during this
>>> build, what do you see under:
>>>
>>>   ;;; Generated autoloads from tramp-adb.el
>>
>> This is the tramp-loaddefs.el after building from git clean -xdf
>>
>>
>>
>>> Can you try a build again with `make bootstrap' and see whether the
>>> warning disappears and if that section of lisp/net/tramp-loaddefs.el
>>> changes?  My guess is this is a harmless warning.
>>
>> The errors are the same, and here's tramp-loaddefs.el after bootstrap
>
> Thanks for checking.  These files are identical, and the error is
> preventing tramp--with-startup from getting defined during autoload
> generation, so autoloaded calls to it just get inlined as-is in
> tramp-loaddefs.el.  If it were working as intended, most of those would
> be replaced by `add-hook'.
>
> Gerd, do you mind reverting e9800cabffa and seeing what if anything
> changes (from a clean start)?  Probably nothing.

Sure. Reverting

  e9800cabffa73018a83108de2bb4e1686f1e8385
  Author:     Po Lu <luangruo <at> yahoo.com>
  AuthorDate: Thu Aug 14 19:33:09 2025 +0800
  Commit:     Po Lu <luangruo <at> yahoo.com>
  CommitDate: Thu Aug 14 19:34:28 2025 +0800

  Parent:     02f5a075dc2 vc-hg-checkin-patch: Fix several issues
  Merged:     consult-problem
  Contained:  cl-packages emacs-mac-master feature/igc master
  Follows:    emacs-30.1 (178644)

  Fix generation of defvars defined by easy-mmode and company in loaddefs.el

changes nothing. 

Message #26 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: "J.D. Smith" <jdtsmith <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 luangruo <at> yahoo.com, 79353 <at> debbugs.gnu.org, michael.albinus <at> gmx.de
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Sun, 31 Aug 2025 21:18:12 -0400

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> 2. During initial autoload generation, this macro is unknown.  So the
>> file tramp-adb.el is /loaded/, to give it a chance to define the macro.
>>
>> 3. The macro `tramp--with-startup' is indeed defined in tramp.el, which
>> tramp-adb requires.  So far so good.  But during the loading of
>> tramp.el, some other package it loads uses `cl-subseq' at the top level
>> without first requiring cl-extra.
>>
>> 5. Why has this not been an issue before?  The macro `cl-subseq' is
>> /itself/ an autoload in cl-extra.el.  Since the CL-specific loadefs file
>> `cl-loaddefs.el' does not yet exist (remember this is during autoload
>> generation), this causes the (handled) warning you see.
>
> FWIW it sounds like a problem I've seen before in other circumstances
> where I just added a `(require 'cl-extra)`, e.g.:
>
>     lisp/emacs-lisp/comp-cstr.el:(require 'cl-extra) ;HACK: For `cl-find-class' when `cl-loaddefs' is missing.

Interesting.  `tramp.el' could probably just (require 'cl-extra) to fix
this.  I guess this is a new use of this cl-extra macro somewhere, which
wasn't caught in testing.

Message #29 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "J.D. Smith" <jdtsmith <at> gmail.com>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 luangruo <at> yahoo.com, 79353 <at> debbugs.gnu.org, michael.albinus <at> gmx.de
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Sun, 31 Aug 2025 23:53:32 -0400

>> FWIW it sounds like a problem I've seen before in other circumstances
>> where I just added a `(require 'cl-extra)`, e.g.:
>>
>>     lisp/emacs-lisp/comp-cstr.el:(require 'cl-extra) ;HACK: For
>> `cl-find-class' when `cl-loaddefs' is missing.
>
> Interesting.  `tramp.el' could probably just (require 'cl-extra) to fix
> this.  I guess this is a new use of this cl-extra macro somewhere, which
> wasn't caught in testing.

I wouldn't call it "new use".  It's probably just a use that can now
occur a bit earlier (before `cl-loaddefs.el` is built).

See also comment in cl-lib.el:

    (unless (load "cl-loaddefs" 'noerror 'quiet)
      ;; When bootstrapping, cl-loaddefs hasn't been built yet!
      (require 'cl-macs)
      (require 'cl-seq)
      ;; FIXME: Arguably we should also load `cl-extra', except that this
      ;; currently causes more bootstrap troubles, and `cl-extra' is
      ;; rarely used, so instead we explicitly (require 'cl-extra) at
      ;; those rare places where we do need it.
      )


- Stefan

Message #32 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: luangruo <at> yahoo.com, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 "J.D. Smith" <jdtsmith <at> gmail.com>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Tue, 02 Sep 2025 06:48:50 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>>> FWIW it sounds like a problem I've seen before in other circumstances
>>> where I just added a `(require 'cl-extra)`, e.g.:
>>>
>>>     lisp/emacs-lisp/comp-cstr.el:(require 'cl-extra) ;HACK: For
>>> `cl-find-class' when `cl-loaddefs' is missing.
>>
>> Interesting.  `tramp.el' could probably just (require 'cl-extra) to fix
>> this.  I guess this is a new use of this cl-extra macro somewhere, which
>> wasn't caught in testing.
>
> I wouldn't call it "new use".  It's probably just a use that can now
> occur a bit earlier (before `cl-loaddefs.el` is built).
>
> See also comment in cl-lib.el:
>
>     (unless (load "cl-loaddefs" 'noerror 'quiet)
>       ;; When bootstrapping, cl-loaddefs hasn't been built yet!
>       (require 'cl-macs)
>       (require 'cl-seq)
>       ;; FIXME: Arguably we should also load `cl-extra', except that this
>       ;; currently causes more bootstrap troubles, and `cl-extra' is
>       ;; rarely used, so instead we explicitly (require 'cl-extra) at
>       ;; those rare places where we do need it.
>       )
>
>
> - Stefan

Not sure if that's relevant, don't know what this is all about anyway.
When I

1 file changed, 1 insertion(+)
lisp/emacs-lisp/loaddefs-gen.el | 1 +

modified   lisp/emacs-lisp/loaddefs-gen.el
@@ -42,6 +42,7 @@
 (require 'radix-tree)
 (require 'lisp-mnt)
 (require 'generate-lisp-file)
+(require 'cl-extra)
 
 (defvar autoload-compute-prefixes t
   "If non-nil, autoload will add code to register the prefixes used in a file.

, just to see what happens, I get

  GEN      autoloads
  INFO     Scraping 1520 files for loaddefs... 
  INFO     Scraping 1520 files for loaddefs...10% 
  INFO     Scraping 1520 files for loaddefs...20% 
  INFO     Scraping 1520 files for loaddefs...30% 
  INFO     Scraping 1520 files for loaddefs...40% 
  INFO     Scraping 1520 files for loaddefs...50% 
  INFO     Scraping 1520 files for loaddefs...60% 
  INFO     Scraping 1520 files for loaddefs...70% 
loaddefs-gen: loading file tramp-adb (for tramp--with-startup)
Loading /Users/gerd/emacs/github/master/lisp/net/tramp-adb.el (source)...
Warning (emacs): loaddefs-gen: load error
	(invalid-read-syntax \N{COLON} 182 16)

which I think indicates that char-from-name doesn't work at that point
in the build process (tramp-compat.el has such character constants).

Message #35 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 Po Lu <luangruo <at> yahoo.com>
Cc: 79353 <at> debbugs.gnu.org, michael.albinus <at> gmx.de, monnier <at> iro.umontreal.ca
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Tue, 2 Sep 2025 09:50:46 -0400

> On Aug 31, 2025, at 1:55 PM, Gerd Möllmann <gerd.moellmann <at> gmail.com> wrote:
> 
> "J.D. Smith" <jdtsmith <at> gmail.com> writes:
> 
>> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
>> 
>>> "J.D. Smith" <jdtsmith <at> gmail.com> writes:
>>> 
>>>> *Question*:
>>>> 
>>>> In the lisp/net/tramp-loaddefs.el file which was generated during this
>>>> build, what do you see under:
>>>> 
>>>>  ;;; Generated autoloads from tramp-adb.el
>>> 
>>> This is the tramp-loaddefs.el after building from git clean -xdf
>>> 
>>> 
>>> 
>>>> Can you try a build again with `make bootstrap' and see whether the
>>>> warning disappears and if that section of lisp/net/tramp-loaddefs.el
>>>> changes?  My guess is this is a harmless warning.
>>> 
>>> The errors are the same, and here's tramp-loaddefs.el after bootstrap
>> 
>> Thanks for checking.  These files are identical, and the error is
>> preventing tramp--with-startup from getting defined during autoload
>> generation, so autoloaded calls to it just get inlined as-is in
>> tramp-loaddefs.el.  If it were working as intended, most of those would
>> be replaced by `add-hook'.
>> 
>> Gerd, do you mind reverting e9800cabffa and seeing what if anything
>> changes (from a clean start)?  Probably nothing.
> 
> Sure. Reverting
> 
>  e9800cabffa73018a83108de2bb4e1686f1e8385
>  Author:     Po Lu <luangruo <at> yahoo.com>
>  AuthorDate: Thu Aug 14 19:33:09 2025 +0800
>  Commit:     Po Lu <luangruo <at> yahoo.com>
>  CommitDate: Thu Aug 14 19:34:28 2025 +0800
> 
>  Parent:     02f5a075dc2 vc-hg-checkin-patch: Fix several issues
>  Merged:     consult-problem
>  Contained:  cl-packages emacs-mac-master feature/igc master
>  Follows:    emacs-30.1 (178644)
> 
>  Fix generation of defvars defined by easy-mmode and company in loaddefs.el
> 
> changes nothing. 

Thanks, Gerd.  Po, can you confirm this and, if the same, revert your e9800cabffa commit?

Message #38 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: Gerd Möllmann <gerd.moellmann <at> gmail.com>
Cc: Po Lu <luangruo <at> yahoo.com>, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Tue, 2 Sep 2025 09:51:47 -0400

> On Sep 2, 2025, at 12:48 AM, Gerd Möllmann <gerd.moellmann <at> gmail.com> wrote:
> 
> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
> 
>>>> FWIW it sounds like a problem I've seen before in other circumstances
>>>> where I just added a `(require 'cl-extra)`, e.g.:
>>>> 
>>>>    lisp/emacs-lisp/comp-cstr.el:(require 'cl-extra) ;HACK: For
>>>> `cl-find-class' when `cl-loaddefs' is missing.
>>> 
>>> Interesting.  `tramp.el' could probably just (require 'cl-extra) to fix
>>> this.  I guess this is a new use of this cl-extra macro somewhere, which
>>> wasn't caught in testing.
>> 
>> I wouldn't call it "new use".  It's probably just a use that can now
>> occur a bit earlier (before `cl-loaddefs.el` is built).
>> 
>> See also comment in cl-lib.el:
>> 
>>    (unless (load "cl-loaddefs" 'noerror 'quiet)
>>      ;; When bootstrapping, cl-loaddefs hasn't been built yet!
>>      (require 'cl-macs)
>>      (require 'cl-seq)
>>      ;; FIXME: Arguably we should also load `cl-extra', except that this
>>      ;; currently causes more bootstrap troubles, and `cl-extra' is
>>      ;; rarely used, so instead we explicitly (require 'cl-extra) at
>>      ;; those rare places where we do need it.
>>      )
>> 
>> 
>> - Stefan
> 
> Not sure if that's relevant, don't know what this is all about anyway.

I think the idea is to require 'cl-extra in tramp.el, after 'cl-lib.  The issue is we are loading tramp during the build of *-loaddefs.el files, and yet tramp is currently relying (indirectly) on the autoload of cl-subseq from cl-extra. 

Message #41 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: JD Smith <jdtsmith <at> gmail.com>
Cc: Po Lu <luangruo <at> yahoo.com>, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Tue, 02 Sep 2025 16:01:05 +0200

JD Smith <jdtsmith <at> gmail.com> writes:

>> Not sure if that's relevant, don't know what this is all about anyway.
>
> I think the idea is to require 'cl-extra in tramp.el, after 'cl-lib.
> The issue is we are loading tramp during the build of *-loaddefs.el
> files, and yet tramp is currently relying (indirectly) on the autoload
> of cl-subseq from cl-extra.

I meant more what your original change was for. Anyway, with this
change

1 file changed, 2 insertions(+)
lisp/net/tramp.el | 2 ++

modified   lisp/net/tramp.el
@@ -67,6 +67,8 @@
 (require 'tramp-message)
 (require 'tramp-integration)
 (require 'trampver)
+(require 'cl-lib)
+(require 'cl-extra)
 
 ;; Pacify byte-compiler.
 (require 'cl-lib)

I get the same errors

loaddefs-gen: loading file tramp-adb (for tramp--with-startup)
Loading /Users/gerd/emacs/github/master/lisp/net/tramp-adb.el (source)...
Warning (emacs): loaddefs-gen: load error
	(error Eager macro-expansion failure: (error "Eager macro-expansion failure: (void-function cl-subseq)"))

Message #44 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: Gerd Möllmann <gerd.moellmann <at> gmail.com>
Cc: Po Lu <luangruo <at> yahoo.com>, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Tue, 2 Sep 2025 10:08:29 -0400

> On Sep 2, 2025, at 10:01 AM, Gerd Möllmann <gerd.moellmann <at> gmail.com> wrote:
> 
> JD Smith <jdtsmith <at> gmail.com> writes:
> 
>>> Not sure if that's relevant, don't know what this is all about anyway.
>> 
>> I think the idea is to require 'cl-extra in tramp.el, after 'cl-lib.
>> The issue is we are loading tramp during the build of *-loaddefs.el
>> files, and yet tramp is currently relying (indirectly) on the autoload
>> of cl-subseq from cl-extra.
> 
> I meant more what your original change was for. Anyway, with this
> change
> 
> 1 file changed, 2 insertions(+)
> lisp/net/tramp.el | 2 ++
> 
> modified   lisp/net/tramp.el
> @@ -67,6 +67,8 @@
> (require 'tramp-message)
> (require 'tramp-integration)
> (require 'trampver)
> +(require 'cl-lib)
> +(require 'cl-extra)
> 
> ;; Pacify byte-compiler.
> (require 'cl-lib)
> 
> I get the same errors
> 
> loaddefs-gen: loading file tramp-adb (for tramp--with-startup)
> Loading /Users/gerd/emacs/github/master/lisp/net/tramp-adb.el (source)...
> Warning (emacs): loaddefs-gen: load error
> (error Eager macro-expansion failure: (error "Eager macro-expansion failure: (void-function cl-subseq)"))

I guess I don't understand where this is coming from then.  What if you take out the condition-case and let it generate a backtrace?

modified   lisp/emacs-lisp/loaddefs-gen.el
@@ -235,10 +235,7 @@ loaddefs-generate--make-autoload
                     (member file loaddefs--load-error-files))
           (let ((load-path (cons (file-name-directory file) load-path)))
             (message "loaddefs-gen: loading file %s (for %s)" file car)
-            (condition-case e (load file)
-              (error
-               (push file loaddefs--load-error-files) ; do not attempt again
-               (warn "loaddefs-gen: load error\n\t%s" e)))))
+            (load file)))
         (and (macrop car)
 	     (eq 'expand (function-get car 'autoload-macro))
 	     (setq expand (let ((load-true-file-name file)

Message #47 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: JD Smith <jdtsmith <at> gmail.com>
Cc: Po Lu <luangruo <at> yahoo.com>, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Tue, 02 Sep 2025 16:23:18 +0200

[Message part 1 (text/plain, inline)]

JD Smith <jdtsmith <at> gmail.com> writes:

> I guess I don't understand where this is coming from then.  What if you take out the condition-case and let it generate a backtrace?

With

1 file changed, 2 insertions(+), 2 deletions(-)
lisp/emacs-lisp/loaddefs-gen.el | 4 ++--

modified   lisp/emacs-lisp/loaddefs-gen.el
@@ -235,8 +235,8 @@ loaddefs-generate--make-autoload
                     (member file loaddefs--load-error-files))
           (let ((load-path (cons (file-name-directory file) load-path)))
             (message "loaddefs-gen: loading file %s (for %s)" file car)
-            (condition-case e (load file)
-              (error
+            (prog1 (load file)
+              '(error
                (push file loaddefs--load-error-files) ; do not attempt again
                (warn "loaddefs-gen: load error\n\t%s" e)))))
         (and (macrop car)

I get the attached compilation log, with a backtrace

[compilation.log (application/octet-stream, attachment)]

Message #50 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: Gerd Möllmann <gerd.moellmann <at> gmail.com>
Cc: Po Lu <luangruo <at> yahoo.com>, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Tue, 2 Sep 2025 12:56:34 -0400

> On Sep 2, 2025, at 10:23 AM, Gerd Möllmann <gerd.moellmann <at> gmail.com> wrote:
> 
> JD Smith <jdtsmith <at> gmail.com> writes:
> 
>> I guess I don't understand where this is coming from then.  What if you take out the condition-case and let it generate a backtrace?
> 
> I get the attached compilation log, with a backtrace
> 
> <compilation.log>

Thanks.  Looks like auth-source is the culprit to me.  Maybe give it a require 'cl-extra (with a note like Stefan's) and try?

  signal(error ("Eager macro-expansion failure: (error \"Eager macro-expansion failure: (void-function cl-subseq)\")"))
  error("Eager macro-expansion failure: %S" (error "Eager macro-expansion failure: (void-function cl-subseq)"))
  (condition-case err (let ((macroexp--pending-eager-loads (cons load-file-name macroexp--pending-eager-loads))) (if full-p (macroexpand--all-toplevel form) (macroexpand form))) ((debug error) (error "Eager macro-expansion failure: %S" err) form))
  (cond ((eq 'skip (car macroexp--pending-eager-loads)) form) ((and load-file-name (member load-file-name macroexp--pending-eager-loads)) (let* ((bt (delq nil (mapcar #'macroexp--trim-backtrace-frame (macroexp--backtrace)))) (elem (list 'load (file-name-nondirectory load-file-name))) (tail (member elem (cdr (member elem bt))))) (if tail (setcdr tail (list '…))) (if (eq (car-safe (car bt)) 'macroexpand-all) (setq bt (cdr bt))) (if macroexp--debug-eager (debug 'eager-macroexp-cycle) (error "Eager macro-expansion skipped due to cycle:\n  %s" (mapconcat #'prin1-to-string (nreverse bt) " => "))) (setq macroexp--pending-eager-loads (cons 'skip macroexp--pending-eager-loads)) form)) (t (condition-case err (let ((macroexp--pending-eager-loads (cons load-file-name macroexp--pending-eager-loads))) (if full-p (macroexpand--all-toplevel form) (macroexpand form))) ((debug error) (error "Eager macro-expansion failure: %S" err) form))))
  internal-macroexpand-for-load((eval-when-compile (require 'epg)) nil)
  eval-buffer(#<buffer  *load*-798440> nil "/Users/gerd/emacs/github/master/lisp/auth-source.el" nil t)
  (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) (eval-buffer buffer nil (if dump-mode file fullname) nil t))
  (let ((read-symbol-shorthands shorthands)) (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) (eval-buffer buffer nil (if dump-mode file fullname) nil t)))
  (let ((load-true-file-name fullname) (load-file-name fullname) (set-auto-coding-for-load t) (inhibit-file-name-operation nil) shorthands) (with-current-buffer buffer (set-buffer-multibyte t) (let (deactivate-mark) (insert-file-contents fullname)) (setq shorthands (and hack-read-symbol-shorthands-function (funcall hack-read-symbol-shorthands-function))) (if (and enable-multibyte-characters (or (eq (coding-system-type last-coding-system-used) 'raw-text))) (set-buffer-multibyte nil)) (set-buffer-modified-p nil)) (let ((read-symbol-shorthands shorthands)) (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) (eval-buffer buffer nil (if dump-mode file fullname) nil t))))
  (unwind-protect (let ((load-true-file-name fullname) (load-file-name fullname) (set-auto-coding-for-load t) (inhibit-file-name-operation nil) shorthands) (with-current-buffer buffer (set-buffer-multibyte t) (let (deactivate-mark) (insert-file-contents fullname)) (setq shorthands (and hack-read-symbol-shorthands-function (funcall hack-read-symbol-shorthands-function))) (if (and enable-multibyte-characters (or (eq (coding-system-type last-coding-system-used) 'raw-text))) (set-buffer-multibyte nil)) (set-buffer-modified-p nil)) (let ((read-symbol-shorthands shorthands)) (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) (eval-buffer buffer nil (if dump-mode file fullname) nil t)))) (let (kill-buffer-hook kill-buffer-query-functions) (kill-buffer buffer)))
  (let ((buffer (generate-new-buffer " *load*")) (load-in-progress t) (source (string-suffix-p ".el" fullname))) (unless nomessage (if source (message "Loading %s (source)..." file) (message "Loading %s..." file))) (when purify-flag (push file preloaded-file-list)) (unwind-protect (let ((load-true-file-name fullname) (load-file-name fullname) (set-auto-coding-for-load t) (inhibit-file-name-operation nil) shorthands) (with-current-buffer buffer (set-buffer-multibyte t) (let (deactivate-mark) (insert-file-contents fullname)) (setq shorthands (and hack-read-symbol-shorthands-function (funcall hack-read-symbol-shorthands-function))) (if (and enable-multibyte-characters (or (eq (coding-system-type last-coding-system-used) 'raw-text))) (set-buffer-multibyte nil)) (set-buffer-modified-p nil)) (let ((read-symbol-shorthands shorthands)) (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) (eval-buffer buffer nil (if dump-mode file fullname) nil t)))) (let (kill-buffer-hook kill-buffer-query-functions) (kill-buffer buffer))) (do-after-load-evaluation fullname) (unless (or nomessage noninteractive) (if source (message "Loading %s (source)...done" file) (message "Loading %s...done" file))) t)
  (if (null (file-readable-p fullname)) (and (null noerror) (signal 'file-error (list "Cannot open load file" file))) (let ((buffer (generate-new-buffer " *load*")) (load-in-progress t) (source (string-suffix-p ".el" fullname))) (unless nomessage (if source (message "Loading %s (source)..." file) (message "Loading %s..." file))) (when purify-flag (push file preloaded-file-list)) (unwind-protect (let ((load-true-file-name fullname) (load-file-name fullname) (set-auto-coding-for-load t) (inhibit-file-name-operation nil) shorthands) (with-current-buffer buffer (set-buffer-multibyte t) (let (deactivate-mark) (insert-file-contents fullname)) (setq shorthands (and hack-read-symbol-shorthands-function (funcall hack-read-symbol-shorthands-function))) (if (and enable-multibyte-characters (or (eq (coding-system-type last-coding-system-used) 'raw-text))) (set-buffer-multibyte nil)) (set-buffer-modified-p nil)) (let ((read-symbol-shorthands shorthands)) (if eval-function (funcall eval-function buffer (if dump-mode file fullname)) (eval-buffer buffer nil (if dump-mode file fullname) nil t)))) (let (kill-buffer-hook kill-buffer-query-functions) (kill-buffer buffer))) (do-after-load-evaluation fullname) (unless (or nomessage noninteractive) (if source (message "Loading %s (source)...done" file) (message "Loading %s...done" file))) t))
  load-with-code-conversion("/Users/gerd/emacs/github/master/lisp/auth-source.el" "/Users/gerd/emacs/github/master/lisp/auth-source.el" nil t)
  require(auth-source)

Message #53 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: JD Smith <jdtsmith <at> gmail.com>
Cc: Po Lu <luangruo <at> yahoo.com>, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Tue, 02 Sep 2025 19:58:26 +0200

JD Smith <jdtsmith <at> gmail.com> writes:

>> On Sep 2, 2025, at 10:23 AM, Gerd Möllmann <gerd.moellmann <at> gmail.com> wrote:
>> 
>> JD Smith <jdtsmith <at> gmail.com> writes:
>> 
>>> I guess I don't understand where this is coming from then.  What if you take out the condition-case and let it generate a backtrace?
>> 
>> I get the attached compilation log, with a backtrace
>> 
>> <compilation.log>
>
> Thanks. Looks like auth-source is the culprit to me. Maybe give it a
> require 'cl-extra (with a note like Stefan's) and try?

I'm sorry, but I'll have to pass. I think it's better if someone in the
know takes it from here.

Message #56 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: JD Smith <jdtsmith <at> gmail.com>
Cc: Po Lu <luangruo <at> yahoo.com>, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 03 Sep 2025 08:07:35 +0200

Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:

> JD Smith <jdtsmith <at> gmail.com> writes:
>
>>> On Sep 2, 2025, at 10:23 AM, Gerd Möllmann <gerd.moellmann <at> gmail.com> wrote:
>>> 
>>> JD Smith <jdtsmith <at> gmail.com> writes:
>>> 
>>>> I guess I don't understand where this is coming from then. What if
>>>> you take out the condition-case and let it generate a backtrace?
>>> 
>>> I get the attached compilation log, with a backtrace
>>> 
>>> <compilation.log>
>>
>> Thanks. Looks like auth-source is the culprit to me. Maybe give it a
>> require 'cl-extra (with a note like Stefan's) and try?
>
> I'm sorry, but I'll have to pass. I think it's better if someone in the
> know takes it from here.

Maybe an explanation is in order why I said that. Remember what I said
about the invalid-read-syntax once the macro-expansion errors are fixed?
That's of course exactly what's happening:

1 file changed, 1 insertion(+)
lisp/auth-source.el | 1 +

modified   lisp/auth-source.el
@@ -44,6 +44,7 @@
 (require 'icons)
 
 (require 'cl-lib)
+(require 'cl-extra)
 (require 'eieio)
 
 (declare-function gnutls-symmetric-decrypt "gnutls.c")

=>

  INFO     Scraping 1529 files for loaddefs...70% 
loaddefs-gen: loading file tramp-adb (for tramp--with-startup)
Loading /Users/gerd/emacs/github/master/lisp/net/tramp-adb.el (source)...
Warning (emacs): loaddefs-gen: load error
	(invalid-read-syntax \N{COLON} 182 16)

HTH

Message #59 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: Gerd Möllmann <gerd.moellmann <at> gmail.com>
Cc: Po Lu <luangruo <at> yahoo.com>, michael.albinus <at> gmx.de, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 3 Sep 2025 08:21:32 -0400

> On Sep 3, 2025, at 2:07 AM, Gerd Möllmann <gerd.moellmann <at> gmail.com> wrote:
> 
> Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:
> 
>> JD Smith <jdtsmith <at> gmail.com> writes:
>> 
>>>> On Sep 2, 2025, at 10:23 AM, Gerd Möllmann <gerd.moellmann <at> gmail.com> wrote:
>>>> 
>>>> JD Smith <jdtsmith <at> gmail.com> writes:
>>>> 
>>>>> I guess I don't understand where this is coming from then. What if
>>>>> you take out the condition-case and let it generate a backtrace?
>>>> 
>>>> I get the attached compilation log, with a backtrace
>>>> 
>>>> <compilation.log>
>>> 
>>> Thanks. Looks like auth-source is the culprit to me. Maybe give it a
>>> require 'cl-extra (with a note like Stefan's) and try?
>> 
>> I'm sorry, but I'll have to pass. I think it's better if someone in the
>> know takes it from here.
> 
> Maybe an explanation is in order why I said that. Remember what I said
> about the invalid-read-syntax once the macro-expansion errors are fixed?
> That's of course exactly what's happening:
> 
> 1 file changed, 1 insertion(+)
> lisp/auth-source.el | 1 +
> 
> modified   lisp/auth-source.el
> @@ -44,6 +44,7 @@
> (require 'icons)
> 
> (require 'cl-lib)
> +(require 'cl-extra)
> (require 'eieio)
> 
> (declare-function gnutls-symmetric-decrypt "gnutls.c")
> 
> =>
> 
>  INFO     Scraping 1529 files for loaddefs...70% 
> loaddefs-gen: loading file tramp-adb (for tramp--with-startup)
> Loading /Users/gerd/emacs/github/master/lisp/net/tramp-adb.el (source)...
> Warning (emacs): loaddefs-gen: load error
> (invalid-read-syntax \N{COLON} 182 16)

Thanks, Gerd.  I don't have a master build setup, so I haven't seen this issue.  Does anyone else?  

I'm not sure what this could mean; a corrupted file somewhere?  If someone who can reproduce this removes the condition-case mentioned earlier, the traceback may be informative. 

Message #62 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: JD Smith <jdtsmith <at> gmail.com>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 Po Lu <luangruo <at> yahoo.com>, 79353 <at> debbugs.gnu.org, michael.albinus <at> gmx.de
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 03 Sep 2025 09:36:39 -0400

> I'm not sure what this could mean; a corrupted file somewhere?

No, it's just that `tramp-compat.el` uses `\N{...}`:

    (if (boundp 'password-colon-equivalents)
        (defvaralias
          'tramp-compat-password-colon-equivalents
          'password-colon-equivalents)
      (defvar tramp-compat-password-colon-equivalents
        '(?\N{COLON}
          ?\N{FULLWIDTH COLON}
          ?\N{SMALL COLON}
          ?\N{PRESENTATION FORM FOR VERTICAL COLON}
          ?\N{KHMER SIGN CAMNUC PII KUUH})
        "List of characters equivalent to trailing colon in \"password\" prompts."))

but that syntax is not supported by the Lisp reader in the early part of
the bootstrap because it relies on a data structure built during Emacs
compilation (using ELisp code) from the Unicode tables.


        Stefan

Message #65 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 Po Lu <luangruo <at> yahoo.com>, 79353 <at> debbugs.gnu.org,
 JD Smith <jdtsmith <at> gmail.com>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 03 Sep 2025 16:15:40 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> I'm not sure what this could mean; a corrupted file somewhere?
>
> No, it's just that `tramp-compat.el` uses `\N{...}`:
>
>     (if (boundp 'password-colon-equivalents)
>         (defvaralias
>           'tramp-compat-password-colon-equivalents
>           'password-colon-equivalents)
>       (defvar tramp-compat-password-colon-equivalents
>         '(?\N{COLON}
>           ?\N{FULLWIDTH COLON}
>           ?\N{SMALL COLON}
>           ?\N{PRESENTATION FORM FOR VERTICAL COLON}
>           ?\N{KHMER SIGN CAMNUC PII KUUH})
>         "List of characters equivalent to trailing colon in \"password\" prompts."))
>
> but that syntax is not supported by the Lisp reader in the early part of
> the bootstrap because it relies on a data structure built during Emacs
> compilation (using ELisp code) from the Unicode tables.

tramp-compat-password-colon-equivalents is not autoloaded.
tramp-compat.el does not contain any autoload cookie. Why is it in
play here?

>         Stefan

Best regards, Michael.

Message #68 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: "J.D. Smith" <jdtsmith <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 Po Lu <luangruo <at> yahoo.com>, 79353 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 03 Sep 2025 10:50:49 -0400

Michael Albinus <michael.albinus <at> gmx.de> writes:

> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>
>>> I'm not sure what this could mean; a corrupted file somewhere?
>>
>> No, it's just that `tramp-compat.el` uses `\N{...}`:
>> but that syntax is not supported by the Lisp reader in the early part of
>> the bootstrap because it relies on a data structure built during Emacs
>> compilation (using ELisp code) from the Unicode tables.
>
> tramp-compat-password-colon-equivalents is not autoloaded.
> tramp-compat.el does not contain any autoload cookie. Why is it in
> play here?

Because the new autoload bootstrap actually /loads/ files with autoload
calls to `tramp--with-startup`, e.g. the file `tramp-adb`.  This file
requires `tramp`, which loads `tramp-compat`, which uses the (not yet
available) unicode syntax.  Turns out there are more footguns to early
loading of lisp files than expected.

Zooming back out, this entire feature was really created for /external
packages/, so they could provide their own macros which wrap
e.g. `define-minor-mode`, and have these autoloads handled correctly
without pre-loading anything:

 ;;;###autoload
 (my-fancy-mode-define 'foo 'bar)  ;; correctly expanded to define-minor-mode etc.

So far I think the only internal library which is unknown to the
bootstrap version of emacs used to build autoloads during compilation is
TRAMP.  For example, it knows about easy-mmode (and hence
`define-minor-mode' and friends).

I see two options:

1. Find and fix the footguns, making the bootstrap version of emacs used
   for autoload generation more robust.  E.g. move unicode table
   generation earlier in the build.

2. Stop loading internal files during bootstrap.  Add an argument to
   `loaddefs-generate--make-autoload' which inhibits loading absolutely.
   Pass this arg in during bootstrap autoload generation.

For option 2, we'd remove the expand declaration from
`tramp--with-startup', which is incomplete anyway, since some of its
autoloads are gated behind e.g. `(when ... (tramp--with-startup))'.
External package autoload generation would still work fine, since this
uses a "full" emacs version to generate.

Option 2 has the one disadvantage that external (ELPA) packages which
moved into core would need to forego and rework whatever auto-expansion
they had.  Since TRAMP has already happily existed in that state for
many years, I think this is fine.  How to document this is another
issue.  Kind of subtle and yucky.

Are there other good options I'm overlooking?

Message #71 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "J.D. Smith" <jdtsmith <at> gmail.com>
Cc: Gerd Möllmann
 <gerd.moellmann <at> gmail.com>, Po Lu <luangruo <at> yahoo.com>,
 Michael Albinus <michael.albinus <at> gmx.de>, 79353 <at> debbugs.gnu.org
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 03 Sep 2025 11:19:31 -0400

> available) unicode syntax.  Turns out there are more footguns to early
> loading of lisp files than expected.

Bootstrap is full of exciting surprises.

> Zooming back out, this entire feature was really created for /external
> packages/, so they could provide their own macros which wrap
> e.g. `define-minor-mode`, and have these autoloads handled correctly
> without pre-loading anything:

FWIW, it's best to reduce differences between "internal" and "external"
packages.

> 1. Find and fix the footguns, making the bootstrap version of Emacs used
>    for autoload generation more robust.  E.g. move unicode table
>    generation earlier in the build.

Worth a try, but as you've seen the bootstrap is ... interesting so it
may prove more trouble than its worth.
We could also change `tramp-compat.el` to refrain from using the \N{...} syntax.

One more thing: AFAIK we're talking about eliminating a compile-time
*warning*, right?  IOW, the new autoload-macro-expansion is sufficiently
careful to withstand errors that the resulting code still works, no?

> 2. Stop loading internal files during bootstrap.  Add an argument to
>    `loaddefs-generate--make-autoload' which inhibits loading absolutely.
>    Pass this arg in during bootstrap autoload generation.

I'd rather not go there if I can avoid it.

> For option 2, we'd remove the expand declaration from
> `tramp--with-startup', which is incomplete anyway, since some of its
> autoloads are gated behind e.g. `(when ... (tramp--with-startup))'.

We should probably remove it anyway: it was a good way to test the new
functionality, but it's not that important.


        Stefan

Message #74 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 Po Lu <luangruo <at> yahoo.com>, Michael Albinus <michael.albinus <at> gmx.de>,
 79353 <at> debbugs.gnu.org
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 3 Sep 2025 12:53:14 -0400

> On Sep 3, 2025, at 11:19 AM, Stefan Monnier <monnier <at> iro.umontreal.ca> wrote:
> 
>> available) unicode syntax.  Turns out there are more footguns to early
>> loading of lisp files than expected.
> 
> Bootstrap is full of exciting surprises.

:) 

>> Zooming back out, this entire feature was really created for /external
>> packages/, so they could provide their own macros which wrap
>> e.g. `define-minor-mode`, and have these autoloads handled correctly
>> without pre-loading anything:
> 
> FWIW, it's best to reduce differences between "internal" and "external"
> packages.

Yeah, that's my instinct too, since some packages are both, and some migrate in either direction.

>> 1. Find and fix the footguns, making the bootstrap version of Emacs used
>>   for autoload generation more robust.  E.g. move unicode table
>>   generation earlier in the build.
> 
> Worth a try, but as you've seen the bootstrap is ... interesting so it
> may prove more trouble than its worth.
> We could also change `tramp-compat.el` to refrain from using the \N{...} syntax.
> 
> One more thing: AFAIK we're talking about eliminating a compile-time
> *warning*, right?  IOW, the new autoload-macro-expansion is sufficiently
> careful to withstand errors that the resulting code still works, no?

Yes, the error is handled, and the result is that the TRAMP file which autoloads calls to `tramp--with-startup' simply fails to load, hence that macro remains undefined. This macro, and autoload calls to it, are then copied verbatim into the tramp-loaddefs.el file.  Which is how it has been for many years.

>> 2. Stop loading internal files during bootstrap.  Add an argument to
>>   `loaddefs-generate--make-autoload' which inhibits loading absolutely.
>>   Pass this arg in during bootstrap autoload generation.
> 
> I'd rather not go there if I can avoid it.
> 
>> For option 2, we'd remove the expand declaration from
>> `tramp--with-startup', which is incomplete anyway, since some of its
>> autoloads are gated behind e.g. `(when ... (tramp--with-startup))'.
> 
> We should probably remove it anyway: it was a good way to test the new
> functionality, but it's not that important.

It currently doesn't function due to the load error.  We could of course just catch and not report load errors. But they do provide insight into early-loading issues that may merit correction.  And this "silent approach" would be a problem for internal packages which begin to rely on `autoload-macro expand` and the early-loading behavior during autoload bootstrap generation, only later to find out loading no longer occurs because of the introduction of some new bootstrap-emacs problem, causing breakage.  Preventing /future/ problems seems important.

Summarizing, I guess option 3 is just:

3. Stop declaring autoload-macro expand in tramp--with-startup (since it doesn't function anyway), and ask people to ignore the warning.

I do think a real question could be asked: how early should it be possible to load a valid lisp file?  

Thanks

Message #77 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: JD Smith <jdtsmith <at> gmail.com>
Cc: Gerd Möllmann
 <gerd.moellmann <at> gmail.com>, Po Lu <luangruo <at> yahoo.com>,
 Michael Albinus <michael.albinus <at> gmx.de>, 79353 <at> debbugs.gnu.org
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 03 Sep 2025 13:24:47 -0400

> It currently doesn't function due to the load error.  We could of course
> just catch and not report load errors. But they do provide insight into
> early-loading issues that may merit correction.  And this "silent approach"

Yes, we should address the warning rather than mute all those warnings.

> I do think a real question could be asked: how early should it be possible
> to load a valid lisp file?

That one's easy: the whole of `loadup.el` is made of a delicate
ordering, so it's only after those files have been loaded that you're in
safe territory.

So, I guess this suggests we should build the \N{...} table before we
build the `loaddefs.el` and that should be doable without any new
bootstrap issue since we have `ldefs-boot.el` to solve that dependency.


        Stefan

Message #80 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Gerd Möllmann <gerd.moellmann <at> gmail.com>,
 Po Lu <luangruo <at> yahoo.com>, 79353 <at> debbugs.gnu.org,
 JD Smith <jdtsmith <at> gmail.com>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Wed, 03 Sep 2025 19:34:44 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

Hi,

> So, I guess this suggests we should build the \N{...} table before we
> build the `loaddefs.el` and that should be doable without any new
> bootstrap issue since we have `ldefs-boot.el` to solve that dependency.

The definition of tramp-compat-password-colon-equivalents is temporarily
in Tramp. It will disappear, one Tramp is backward compatible down to
Emacs 30 only.

For the time being: wouldn't it be sufficient to define

--8<---------------cut here---------------start------------->8---
(defvar tramp-compat-password-colon-equivalents '(58 65306 65109 65043 6102))
--8<---------------cut here---------------end--------------->8---

Not a general solution of course, but maybe sufficient for this problem.

>         Stefan

Best regards, Michael.

Message #83 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Po Lu <luangruo <at> yahoo.com>, 79353 <at> debbugs.gnu.org,
 Michael Albinus <michael.albinus <at> gmx.de>, JD Smith <jdtsmith <at> gmail.com>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Thu, 04 Sep 2025 09:03:06 +0200

[Message part 1 (text/plain, inline)]

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> It currently doesn't function due to the load error.  We could of course
>> just catch and not report load errors. But they do provide insight into
>> early-loading issues that may merit correction.  And this "silent approach"
>
> Yes, we should address the warning rather than mute all those warnings.
>
>> I do think a real question could be asked: how early should it be possible
>> to load a valid lisp file?
>
> That one's easy: the whole of `loadup.el` is made of a delicate
> ordering, so it's only after those files have been loaded that you're in
> safe territory.
>
> So, I guess this suggests we should build the \N{...} table before we
> build the `loaddefs.el` and that should be doable without any new
> bootstrap issue since we have `ldefs-boot.el` to solve that dependency.

Put some sturdy footwear on for the Makefile shrubbery, and came up with
the attached patch. It's for my Emacs only; I have no way of testing
this with all the other platforms, and all the ways Emacs can be built.
And I'm not sure that you would want the loaddefs-gen change either. And
blahblah :-).

[0001-Fix-autoload-generation-bug-79353.patch (text/x-patch, attachment)]

Message #86 received at 79353 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Möllmann <gerd.moellmann <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Po Lu <luangruo <at> yahoo.com>, 79353 <at> debbugs.gnu.org,
 Michael Albinus <michael.albinus <at> gmx.de>, JD Smith <jdtsmith <at> gmail.com>
Subject: Re: bug#79353: 31.0.50; Eager macro expansion errors when building
 master
Date: Thu, 04 Sep 2025 09:13:10 +0200

Gerd Möllmann <gerd.moellmann <at> gmail.com> writes:

> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>
>>> It currently doesn't function due to the load error.  We could of course
>>> just catch and not report load errors. But they do provide insight into
>>> early-loading issues that may merit correction.  And this "silent approach"
>>
>> Yes, we should address the warning rather than mute all those warnings.
>>
>>> I do think a real question could be asked: how early should it be possible
>>> to load a valid lisp file?
>>
>> That one's easy: the whole of `loadup.el` is made of a delicate
>> ordering, so it's only after those files have been loaded that you're in
>> safe territory.
>>
>> So, I guess this suggests we should build the \N{...} table before we
>> build the `loaddefs.el` and that should be doable without any new
>> bootstrap issue since we have `ldefs-boot.el` to solve that dependency.
>
> Put some sturdy footwear on for the Makefile shrubbery, and came up with
> the attached patch. It's for my Emacs only; I have no way of testing
> this with all the other platforms, and all the ways Emacs can be built.
> And I'm not sure that you would want the loaddefs-gen change either. And
> blahblah :-).

Forgot something: IMHO, the load in loaddefs-gen should not be in a
condition-case. The build should fail with a backtrace.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.