GNU bug report logs - #74382
`compile-first` Make rule is no longer using `load-prefer-newer`

Previous Next

Full log

Message #14 received at 74382 <at> debbugs.gnu.org (full text, mbox):

From: Konstantin Kharlamov <Hi-Angel <at> yandex.ru>
To: Alan Mackenzie <acm <at> muc.de>
Cc: 74382 <at> debbugs.gnu.org
Subject: Re: `compile-first` Make rule is no longer using `load-prefer-newer`
Date: Sat, 16 Nov 2024 20:45:47 +0300

On Sat, 2024-11-16 at 16:53 +0000, Alan Mackenzie wrote:
> Hello, Konstantin.
>
> On Sat, Nov 16, 2024 at 18:11:01 +0300, Konstantin Kharlamov wrote:
> > CC: Alan Mackenzie, author of the change in
> > 10083e788f7349fa363d100687dc3d94bea88f57
>
> > I've seen for a long time Emacs master builds fail from time to
> > time in spectacular
> > ways after updating the repo, sometimes so badly that `make clean`
> > doesn't help.
>
> > I never dug into that though, but I'm attributing this to the
> > occasional build
> > messages similar to:
>
> >     Source file ‘/home/constantine/Projects/emacs/lisp/emacs-
> > lisp/comp.el’ newer than byte-compiled file; using older file
> >     Source file ‘/home/constantine/Projects/emacs/lisp/emacs-
> > lisp/bytecomp.el’ newer than byte-compiled file; using older file
> >     Source file ‘/home/constantine/Projects/emacs/lisp/emacs-
> > lisp/comp-cstr.el’ newer than byte-compiled file; using older file
>
> > …which makes sense, because if the repo changed `comp.el` API and
> > Emacs during the
> > build of newer files is trying to make use of older `.elc` file and
> > hence the older
> > API, it may result in failure.
>
> The idea is that the byte compiler is first built from .el files,
> giving
> ..elc files.  The .elc byte compiler is then used for all the other
> files, since it's much faster.
>
> > Got some spare time today, dug into one of the messages. From what
> > I understand it's
> > caused by this line `lisp/Makefile.in`:
>
> >     # ... but we must prefer .elc files for those in the early
> > bootstrap.
> >     compile-first: BYTE_COMPILE_FLAGS = $(BYTE_COMPILE_EXTRA_FLAGS)
>
> > > From what I understand, this rewrites BYTE_COMPILE_FLAGS to be an
> > > empty variable,
> > which results in `(setq load-prefer-newer t)` being stripped off of
> > the build.
>
> Yes, this is to prefer the faster .elc byte compiler, whose file
> dates
> have been set back to the epoch (1970-01-01:00:00:00 UTC).  I think
> this
> was to ensure that when we come to native compilation, the .el source
> files of the compiler will be newer than the .elc files, hence
> triggering a native compilation of them.
>
> I honestly don't believe that this mechanism for speeding up the
> early
> build is the cause of the spectacular failures in some of your
> builds,
> but I could be wrong.
>
> > The straightforward solution is to remove this line. But since the
> > line's commentary
> > opposes to such solution, I'm starting up a discussion what exactly
> > should be the
> > behavior here 😊
>
> Perhaps if you could be more specific about the failures you see, we
> might be able to be of more help.

Sure, I just reproduced it after removing all `.elc` files in the repo,
here how:

1. `git checkout f2f13fa630b` (a commit from April)
2. `make -j$(nproc)` to compile. Note: you don't need to wait for build
to finish, I just waited for all files under `lisp/emacs-lisp`
directory to finish compilation, and then ^C'ed it.
3. `git checkout 29098a291f5` (a November commit).
4. `make -j$(nproc)`

## Expected

Build finishes

## Actual

It fails with stacktrace so huge that it goes beyond Konsole scrollback
and with the following message:


    Loading macroexp.elc (compiled; note, source file is newer)...
    Wrong type argument: "../../lisp/emacs-lisp/comp.el", hash-table-p, (#s(byte-to-native-top-level (byte-code "��!���!���!���!���!���!���!���!������%�����DD�����&        �����DD�����&   �����DD���&�����DD�����&������DD�����&������DD�����&������DD�����&�" [require bytecomp cl-lib gv rx subr-x warnings comp-common comp-cstr custom-declare-group ...] 10) t) #s(byte-to-native-top-level (#<symbol defvar at 4425> #<symbol no-native-compile at 4432> nil "Non-nil to prevent native-compiling of Emacs Lisp code.
    Note that when `no-byte-compile' is set to non-nil it overrides the value of
    `no-native-compile'.
    This is normally set in local file variables at the end of the
    Emacs Lisp file:

You can see it fails on the `comp.el`, and I presume the reason is it simply didn't rebuild the necessary .elc files, and instead loaded outdated bytecode.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.