GNU bug report logs - #43532
[feature/native-comp] *.eln file name hashing, algorithm doesn't seem to play nice with NATIVE_FULL_AOT with self-contained builds

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Jim Myhrberg <contact <at> jimeh.me>
Subject: bug#43532: closed (Re: bug#43532: [feature/native-comp] *.eln
 file name hashing, algorithm doesn't seem to play nice with
 NATIVE_FULL_AOT and self-contained, Emacs.app builds for macOS)
Date: Thu, 15 Oct 2020 15:03:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#43532: [feature/native-comp] *.eln file name hashing, algorithm doesn't seem to play nice with NATIVE_FULL_AOT with self-contained builds

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 43532 <at> debbugs.gnu.org.

-- 
43532: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=43532
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Andrea Corallo <akrl <at> sdf.org>
To: Aloxaf Yin <aloxafx <at> gmail.com>
Cc: Andrew Whatson <whatson <at> gmail.com>, 43532-done <at> debbugs.gnu.org,
 Jim Myhrberg <contact <at> jimeh.me>
Subject: Re: bug#43532: [feature/native-comp] *.eln file name hashing,
 algorithm doesn't seem to play nice with NATIVE_FULL_AOT and
 self-contained, Emacs.app builds for macOS
Date: Thu, 15 Oct 2020 15:02:40 +0000

Aloxaf Yin <aloxafx <at> gmail.com> writes:

> It works well. Thank you all!

Excellent! I'm closing then :)

  Andrea

[Message part 3 (message/rfc822, inline)]

From: Jim Myhrberg <contact <at> jimeh.me>
To: bug-gnu-emacs <at> gnu.org
Subject: [feature/native-comp] *.eln file name hashing algorithm doesn't seem
 to play nice with NATIVE_FULL_AOT and self-contained Emacs.app builds
 for macOS
Date: Sun, 20 Sep 2020 14:02:44 +0100

Hi,

TL;DR:

If I've understood correctly, the filename of the cached *.eln files
includes a hash based on the absolute file path and content of the
source *.el/*.el.gz file. On macOS when producing a self-contained
Emacs.app bundle this means that the cached *.eln files bundled into
the app cannot be used, as their absolute path won't match what they
were during build time. And app bundles on macOS can be placed and
launched from anywhere on disk.

I'm not sure what the best solution here might be. But if the actual
content of the file is used to produce the hash, maybe there's no need
to use the full absolute file path, and instead just the filename
itself, along with the content?

The Long Version:

I'm not sure how familiar people here might be with macOS application
bundles, so here's a brief summary just in case; Application bundles
are self-contained macOS GUI applications which can be launched by for
example double clicking on them, among other ways. On a technical
level, they're simply a folder called "<AppName>.app" which contains
all assets needed to run the application. The main executable is
"<AppName>.app/Contents/MacOS/<AppName>".

In the case of building a self-contained Emacs.app on macOS, it looks
the *.eln caches are generated for the files located in
"<workdir>/nextstep/Emacs.app/Contents/Resources/lisp", and the *.eln
cache folder is
"<workdir>/nextstep/Emacs.app/Contents/MacOS/lib/emacs/28.0.50/native-lisp".
Which means the full set of *.eln caches can only be used when
launched from within the Emacs source directory where you built the
application.

Typical usage on macOS would have the Emacs.app movied/copied to the
"/Applications" folder, but there's no guarantee about that, as
applications are supposed work fine regardless of where it's located
on disk they're located.

I've done some tests while using NATIVE_FULL_AOT=1, which produces
1,572 *.eln files within Emacs.app. But when starting emacs with my
config, it starts by compiling a whole bunch of files from
"Emacs.app/Contents/Resources/lisp/", like "emacs-lisp/cconv.el.gz",
"emacs-lisp/bytecomp.el.gz", "help-mode.el.gz", and others. If I grab
the new *.eln files it produced in my user cache, and move them to
within Emacs.app, it doesn't compile them on launch again. If I then
move Emacs.app to another location on disk, it does compile them
again, as the absolute path has changed.

However there's 83 *.eln files which are loaded from Emacs.app,
because if they're removed, it fails to start at all with a dlopen
error instead. I don't know enough about the internals of Emacs, but
I'd assume they are referred to from the dump file, while the others
are loaded up due to my config requiring them, at which point a cache
miss happens due to the absolute path yielding a different change
filename.


Thanks :)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.