Subject: bug#75812: 29.2.50; Support native-compiling package-install from batch Emacs

From: Spencer Baugh <[email-masked]>
Date: Tue, 29 Apr 2025 14:05:38 -0400

Eli Zaretskii <[email-masked]> writes:
>> > Or why not install a native-comp-async-all-done-hook that will notice
>> > when compilations are done and allow the batch job to exit?  All you
>> > need is a means to wait until all the async compilations are finished,
>> > no?
>> 
>> How do you propose to do that?  I guess I could call kill-emacs from the
>> hook, but I have more code I want to run after the native compilation is
>> finished, and I can't move it all into the hook.
>
> Then don't call kill-emacs from the hook; instead, use 'throw', or run
> the rest of the code from the hook directly.

So a function like this?

(defun native-comp-async-wait-for-all-done ()
  (unless (zerop (comp--async-runnings))
    (catch 'native-comp-done
      (let ((native-comp-async-all-done-hook (list (lambda () (throw 'native-comp-done)))))
        (while (sit-for 999))))))

Could we add that?

>> >> There's another related issue for the packaging use case I mentioned in
>> >> my original email: The .eln files are placed in ~/.emacs.d/eln-cache/ by
>> >> default when doing this native compilation.  That complicates building
>> >> and installing these packages independently.  Is there a way to put the
>> >> .eln files for a package in that individual package directory instead?
>> >
>> > You need to tweak native-comp-eln-load-path, e.g. prepend to the
>> > default-value list the directory where you want the *.eln files to be
>> > written.  But you are well advised to leave the other directories in
>> > the list alone.
>> 
>> Makes sense.
>> 
>> Does that mean that, if I want the .eln files to be loaded from the
>> individual package directory, I have to put every single package
>> directory on native-comp-eln-load-path at runtime?  That seems annoying,
>> and probably slow.
>
> I'm confused.  It was you who asked how to avoid placing the *.eln
> files in ~/.emacs.d/eln-cache/, but now you say it causes problems?
> "If it hurts, don't do that."  I don't understand why the default
> location is problematic in the first place, and you haven't provided
> any reasons.
>
> And why are you talking about a separate directory for each package?
> One could envision using a single directory for the *.eln files of all
> the packages, for example.  Whether that is reasonable again depends
> on your goals and specific cases, which you haven't described.

Yes, it's a matter of compiling each package separately in parallel,
which is annoying to do if they all write files to a single directory as
a side-effect.

But, I think what I could do is compile each package with a different
directory prepended to native-comp-eln-load-path, and then once all the
packages are compiled, merge those directories into one omnibus
directory, which is bundled with my installed Emacs.

Would it be reasonable to merge those directories into the last entry of
native-comp-eln-load-path?  (e.g. /usr/lib/emacs/30.1.50/native-lisp)

Or will it break something for there to be more than just bundled elisp
in that directory?

>> Is there a way to get an .eln file to be loaded from the directory the
>> .elc file was found in?
>
> No, and it doesn't make sense to do that, because the *.eln files are
> not in the directories mentioned in native-comp-eln-load-path, they
> are in versioned subdirectories of those directories, and those
> subdirectories must be verified by computing the hashes of the
> corresponding *.el files, to make sure we are not by chance loading
> incompatible machine code.

That's fair, makes sense.