GNU bug report logs - #48497
28.0.50; Native compile: excessive recompilation

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 48497 in the body.
You can then email your comments to 48497 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Michael Welsh Duggan <mwd <at> md5i.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; Native compile: excessive recompilation
Date: Tue, 18 May 2021 09:25:00 -0400

The problem is that it seems as if standard Emacs lisp files from the
installed base Emacs lisp library are getting recompiled in the user's
emacs eln cache despite there being an existing native-compiled version
installed.

First, I did a fresh out-of-tree build from master using the following
procedure:

The git worktree is in a directory called "master".

cd master
./autogen.sh
mkdir ../build
cd ../build
mkdir -p ~/tmp/emacs/config
../master/configure --without-toolkit-scroll-bars --with-x-toolkit=lucid \
  --with-native-compilation --prefix=~/tmp/emacs
make -j 32 NATIVE_FULL_AOT=1 bootstrap
make NATIVE_FULL_AOT=1 install

I verified that there was a directoy,
~/tmp/emacs/lib/emacs/28.0.50/native-lisp/28.0.50-6152138a,  which
contained 1382 .eln entries.

I then started emacs like this, after first making sure there is no
"~/.emacs.d" directory or symlink in my home directory:

XDG_CONFIG_HOME=~/tmp/emacs/config ~/tmp/emacs/bin/emacs -Q

At this point, ~/tmp/emacs/config was empty.  I then typed `C-h i',
starting info.  This started some native compilation in the background.
After waiting for the compilation (viewed in top) to settle, there was a
directory, ~/tmp/emacs/config/emacs/eln-cache/28.0.50-6152138a, which
contained the following .eln files:

bytecomp-12882072-1f232d0e.eln  comp-cstr-ef162ef7-c492cd97.eln
byte-opt-9c5f25f5-26e670d7.eln  gv-e0cf7478-ce8290a8.eln
cconv-3b1f1f98-ac4628a4.eln     help-mode-d4dbae3d-034e8cf5.eln
cl-extra-d7051cba-ff815ded.eln  info-ce12c0ca-3d2b00fd.eln
cl-lib-8b938900-fbcee53d.eln    rx-627d8c83-18b6598f.eln
cl-macs-7ae82f81-bc4ef696.eln   seq-c314b42f-4e726806.eln
cl-seq-f464c4e2-95d0eb23.eln    subr-x-02dfef32-04f2924b.eln
comp-7672a6ed-d617b299.eln      warnings-28e75f4d-c98d4e00.eln

I verified that versions of these files existed in
~/tmp/emacs/lib/emacs/28.0.50/native-lisp/28.0.50-6152138a, which they
did, but the first hash differs in each case, and the file sizes are
completely different.

Hopefully this is enough information to recreate the behavior.  I first
encountered this with a build that was installed without --prefix (in
/usr/local) and with a setup that used ~/.emacs.d/.  I changed these
portions for this report in order to make sure I was testing a fresh
build without, hopefully, introducing any of my existing setup into the
test (and without possibly mucking up my existing work environment).

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0)
 of 2021-05-17 built on miko
Repository revision: f572735c5105a84da3175ae6cdad807fa103dfe1
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure --without-toolkit-scroll-bars --with-x-toolkit=lucid
 --with-native-compilation --prefix=/home/md5i/tmp/emacs'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF X11 XAW3D
XDBE XIM XPM LUCID ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

-- 
Michael Welsh Duggan
(md5i <at> md5i.com)

Message #8 received at 48497 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Michael Welsh Duggan <mwd <at> md5i.com>
Cc: 48497 <at> debbugs.gnu.org
Subject: Re: bug#48497: 28.0.50; Native compile: excessive recompilation
Date: Wed, 19 May 2021 13:10:10 +0000

Michael Welsh Duggan <mwd <at> md5i.com> writes:

> The problem is that it seems as if standard Emacs lisp files from the
> installed base Emacs lisp library are getting recompiled in the user's
> emacs eln cache despite there being an existing native-compiled version
> installed.
>
> First, I did a fresh out-of-tree build from master using the following
> procedure:
>
> The git worktree is in a directory called "master".
>
> cd master
> ./autogen.sh
> mkdir ../build
> cd ../build
> mkdir -p ~/tmp/emacs/config
> ../master/configure --without-toolkit-scroll-bars --with-x-toolkit=lucid \
>   --with-native-compilation --prefix=~/tmp/emacs
> make -j 32 NATIVE_FULL_AOT=1 bootstrap
> make NATIVE_FULL_AOT=1 install
>
> I verified that there was a directoy,
> ~/tmp/emacs/lib/emacs/28.0.50/native-lisp/28.0.50-6152138a,  which
> contained 1382 .eln entries.
>
> I then started emacs like this, after first making sure there is no
> "~/.emacs.d" directory or symlink in my home directory:
>
> XDG_CONFIG_HOME=~/tmp/emacs/config ~/tmp/emacs/bin/emacs -Q
>
> At this point, ~/tmp/emacs/config was empty.  I then typed `C-h i',
> starting info.  This started some native compilation in the background.
> After waiting for the compilation (viewed in top) to settle, there was a
> directory, ~/tmp/emacs/config/emacs/eln-cache/28.0.50-6152138a, which
> contained the following .eln files:
>
> bytecomp-12882072-1f232d0e.eln  comp-cstr-ef162ef7-c492cd97.eln
> byte-opt-9c5f25f5-26e670d7.eln  gv-e0cf7478-ce8290a8.eln
> cconv-3b1f1f98-ac4628a4.eln     help-mode-d4dbae3d-034e8cf5.eln
> cl-extra-d7051cba-ff815ded.eln  info-ce12c0ca-3d2b00fd.eln
> cl-lib-8b938900-fbcee53d.eln    rx-627d8c83-18b6598f.eln
> cl-macs-7ae82f81-bc4ef696.eln   seq-c314b42f-4e726806.eln
> cl-seq-f464c4e2-95d0eb23.eln    subr-x-02dfef32-04f2924b.eln
> comp-7672a6ed-d617b299.eln      warnings-28e75f4d-c98d4e00.eln
>
> I verified that versions of these files existed in
> ~/tmp/emacs/lib/emacs/28.0.50/native-lisp/28.0.50-6152138a, which they
> did, but the first hash differs in each case, and the file sizes are
> completely different.
>
> Hopefully this is enough information to recreate the behavior.

Thanks it is, I can reproduce this.

  Andrea

Message #11 received at 48497 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Michael Welsh Duggan <mwd <at> md5i.com>
Cc: 48497 <at> debbugs.gnu.org
Subject: Re: bug#48497: 28.0.50; Native compile: excessive recompilation
Date: Fri, 21 May 2021 06:53:36 +0000

Hi Michael,

I've pushed 3f207753a0 to fix this, would you mind giving it a try?
(please rebuild all eln with like make bootstrap).

Thanks!

  Andrea

Message #14 received at 48497 <at> debbugs.gnu.org (full text, mbox):

From: Michael Welsh Duggan <mwd <at> md5i.com>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: Michael Welsh Duggan <mwd <at> md5i.com>, 48497 <at> debbugs.gnu.org
Subject: Re: bug#48497: 28.0.50; Native compile: excessive recompilation
Date: Fri, 21 May 2021 11:17:16 -0400

Andrea Corallo <akrl <at> sdf.org> writes:

> I've pushed 3f207753a0 to fix this, would you mind giving it a try?
> (please rebuild all eln with like make bootstrap).

Tested in both my isolated test case and in my day-to-day work emacs.
My observation is that this seems to have fixed this problem.  Thank
you!

-- 
Michael Welsh Duggan
(md5i <at> md5i.com)

Message #19 received at 48497-done <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Michael Welsh Duggan <mwd <at> md5i.com>
Cc: 48497-done <at> debbugs.gnu.org
Subject: Re: bug#48497: 28.0.50; Native compile: excessive recompilation
Date: Sat, 22 May 2021 05:35:10 +0000

Michael Welsh Duggan <mwd <at> md5i.com> writes:

> Andrea Corallo <akrl <at> sdf.org> writes:
>
>> I've pushed 3f207753a0 to fix this, would you mind giving it a try?
>> (please rebuild all eln with like make bootstrap).
>
> Tested in both my isolated test case and in my day-to-day work emacs.
> My observation is that this seems to have fixed this problem.  Thank
> you!

Very nice! Closing then.

Thanks for reporting

  Andrea

Message #22 received at 48497 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andrea Corallo <akrl <at> sdf.org>
Cc: mwd <at> md5i.com, 48497 <at> debbugs.gnu.org, akrl <at> sdf.org
Subject: Re: bug#48497: 28.0.50; Native compile: excessive recompilation
Date: Sat, 22 May 2021 09:26:08 +0300

> Cc: 48497-done <at> debbugs.gnu.org
> Date: Sat, 22 May 2021 05:35:10 +0000
> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> Michael Welsh Duggan <mwd <at> md5i.com> writes:
> 
> > Andrea Corallo <akrl <at> sdf.org> writes:
> >
> >> I've pushed 3f207753a0 to fix this, would you mind giving it a try?
> >> (please rebuild all eln with like make bootstrap).
> >
> > Tested in both my isolated test case and in my day-to-day work emacs.
> > My observation is that this seems to have fixed this problem.  Thank
> > you!
> 
> Very nice! Closing then.

Andrea, given the changes in this commit, why did you tell Michael to
rebuild all *.eln files after updating from Git.  Do everyone else
also need to recompile all *.eln files?

Message #25 received at 48497 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <akrl <at> sdf.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: mwd <at> md5i.com, 48497 <at> debbugs.gnu.org
Subject: Re: bug#48497: 28.0.50; Native compile: excessive recompilation
Date: Mon, 24 May 2021 06:17:32 +0000

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Cc: 48497-done <at> debbugs.gnu.org
>> Date: Sat, 22 May 2021 05:35:10 +0000
>> From:  Andrea Corallo via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>> 
>> Michael Welsh Duggan <mwd <at> md5i.com> writes:
>> 
>> > Andrea Corallo <akrl <at> sdf.org> writes:
>> >
>> >> I've pushed 3f207753a0 to fix this, would you mind giving it a try?
>> >> (please rebuild all eln with like make bootstrap).
>> >
>> > Tested in both my isolated test case and in my day-to-day work emacs.
>> > My observation is that this seems to have fixed this problem.  Thank
>> > you!
>> 
>> Very nice! Closing then.
>
> Andrea, given the changes in this commit, why did you tell Michael to
> rebuild all *.eln files after updating from Git.  Do everyone else
> also need to recompile all *.eln files?

On out of tree builds building ahead of time we named wrongly the eln
files so these were not recognized and picked-up triggering a new
automatic compilation.

  Andrea

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.