GNU bug report logs - #46256
[feature/native-comp] AOT eln files ignored if run from build tree

Previous Next

Full log

Message #244 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Moreton <andrewjmoreton <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#46256: [feature/native-comp] AOT eln files ignored if run
 from build tree
Date: Sat, 06 Mar 2021 01:47:09 +0000

On Fri 05 Mar 2021, Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" wrote:

> Pip Cet <pipcet <at> gmail.com> writes:
>
>> On Fri, Mar 5, 2021 at 9:33 AM Andrea Corallo via Bug reports for GNU
>> Emacs, the Swiss army knife of text editors <bug-gnu-emacs <at> gnu.org>
>> wrote:
>>> Eli Zaretskii <eliz <at> gnu.org> writes:
>>> > So currently the only way to fill up a newly created subdirectory of
>>> > native-lisp/ is to manually delete the *.elc files of all the files in
>>> > lisp.mk's $shortlisp list, is that sufficient?
>>>
>>> Yes I think so.
>>>
>>> The trouble of using make for building such a system is that make is not
>>> aware of the .eln filename, so it should be necessary to ask the Emacs
>>> binary about that to create dynamically the precise (multiple target)
>>> rule.  Not very practical IMO...
>>
>> I do wonder whether the whole filename scheme is really the best option.
>>
>> IIUC, and that's a big if in this case, the main motivation for using
>> hashes in the .eln filenames is that dlopen() is broken and may return
>> the same handle for subsequent dlopen()s of the same name, even if the
>> underlying file changed in between.
>
> Unfortunately this was only an unfortunate discover along the road...
> this design predates that.

Can you explain what the problem is with dlopen ? I have not found a
complete and precise description of the problem in earlier messages as a
reproducer.

Is the problem that dlopen resolves to use an unlinked file kept alive
by having open handles, rather than a new file with the filename used
by the old file before it was unlinked ?

>> Merely verifying that the ABI is correct could be done at runtime, so
>> that's no reason to keep a hash in the filename.
>>
>> So my vague idea is this:
>>
>> 1. implement fixed_dlopen(), which keeps track of filenames that have
>> been opened and, if necessary, creates a temporary file and loads that
>> instead of its argument.
>> 2. compile lisp/emacs-lisp/bytecomp.el to lisp/emacs-lisp/bytecomp.elc
>> and native-lisp/emacs-lisp/bytecomp.eln
>
> So it was at the beginning, I think we moved away from that before the
> odd dlopen behavior.

As above, this odd dlopen behaviour needs to be fully explained to
ensure that design choices are not driven by possible misunderstandings.

>> 3. add extra code in the top level function of each .eln to check that
>> the ABI is correct.
>>
>> This would allow us to use standard make rules. It would also make
>> .eln filenames predictable. It might even draw someone's attention to
>> the fact that dlopen() is broken and make them fix it.
>>
>> I'm probably missing other good reasons for the hashed filename scheme.
>
> Yep, this was discussed in length on emacs-devel, IIRC mainly on a long
> standing thread called "native compilation the bird-eye view" (or
> something close).

Thread "Native compilation: the bird-eye view" starts here:
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg02186.html

I agree with Pip that using standard make rules eases several development
pains and should be used if at all possible.

    AndyM

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.