Subject: Re: bug#70246: 30.0.50; Update INSTALL for native compilation

From: Stephen Berman <[email-masked]>
Date: Sun, 07 Apr 2024 21:14:08 +0200

On Sun, 07 Apr 2024 17:52:13 +0300 Eli Zaretskii <[email-masked]> wrote:

>> From: Stephen Berman <[email-masked]>
>> Cc: [email-masked]
>> Date: Sun, 07 Apr 2024 13:48:41 +0200
>>
>> On Sun, 07 Apr 2024 09:08:05 +0300 Eli Zaretskii <[email-masked]> wrote:
>>
>> > Thanks.  This LGTM, except the changes to item 6: there, I would only
>> > mention the possibility of building out of the source tree.  Whether
>> > running uninstalled is possible in this case depends where is the
>> > build tree relative to the source tree, so I'd rather not describe all
>> > the gory details there because they are too complex to explain (what
>> > you wrote is inaccurate);
>>
>> There was a mistake at the end of 6b; what I meant to write is "but the
>> ../lib-src and ../native-lisp directories in the build directory
>> (because the compiled files in these directories are
>> machine-dependent)."  If there were any other inaccuracies in what I
>> wrote, can you please point them out to me (for my own edification)?
>
> See below.
>
>> >                           I'd just say that running uninstalled is
>> > fully supported only when Emacs is built in the source tree.
>>
>> So instead of having 6{a,b,c} as in my patch, are the following
>> additions to the current version (at the end of the first paragraph of
>> 6) ok?
>>
>> =======================================================================
>> , ../native-lisp (if built with support for native compilation).
>>
>> If you build Emacs in a directory separate from the source directory
>> ("out-of-tree"), run 'make' in the build directory.  Then you can
>> execute the 'src/emacs' file under that directory "in place".  However,
>> running Emacs uninstalled is fully supported only when it is built in
>> the source tree.
>> =======================================================================
>
> Yes, but see below.
>
>> But, again for my own edification, can you elaborate what "fully
>> supported" means here?
>
> It means that (a) Emacs will find all the files it needs for running,
> which includes programs and files in lib-src/, Lisp *.el/*.elc/*.eln
> files, Info files, and files in the etc/ subdirectory in the tree from
> which it is run; and (b) when any of the source files are edited and
> Emacs is rebuilt, the next invocation will use the updated files,
> including recompiling the *.eln files as needed.

AFAIK all that happens with my uninstalled out-of-tree builds, but maybe
there's some subtlety I'm not aware of.

>> I've been running Emacs uninstalled from a build
>> directory outside of the source tree for many years, and that statement
>> suggests my builds are deficient (though perhaps only in unobvious ways
>> that are not crucial for using Emacs), and if so, I'd like to know how
>> and why.
>
> What Emacs does at startup to decide where its files are installed is
> complex and hard to explain without describing the entire algorithm
> (which is both inappropriate for INSTALL and subject to change without
> notice).

Yes, INSTALL is certainly not the place for such details.

>           If you are interested, look in emacs.c:init_cmdargs and
> lread.c:load_path_default.

Ok, thanks for the pointers.

> The bottom line is that the Lisp and Info files will be taken from the
> source tree, not from the build tree.  So if you have more than one
> build from the same source tree, then they all share the Lisp and Info
> files.  Therefore, if you modify anything in the source tree, you must
> rebuild all the out-of-tree builds, otherwise those you don't rebuild
> will be inconsistent with the modified sources.

Yes, but if you instead make copies of the sources (or different repo
branches) to build different configurations and make changes that you
want to test in all configurations, then you have to make those changes
in each copy or branch and rebuild them all, too.  So there are
trade-offs in both procedures.

>> >> +'/usr/local/share/emacs/VERSION/native-lisp' holds the natively compiled
>> >> +		pre-loaded Emacs Lisp files.
>> >
>> > This is incorrect: native-lisp is installed under /usr/lib/emacs/VERSION,
>> > not under /usr/share/emacs/VERSION.
>>
>> I guess you meant /usr/local/lib/emacs/VERSION for the default location?
>
> Yes.
>
>> +By default, Emacs natively compiles only pre-loaded Lisp files during
>> +the build process; other Lisp files are natively compiled
>> +"just-in-time", i.e, when they are loaded in the running Emacs.  If you
>                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> "when they are first time loaded into the running Emacs".
>
>>  6) Run 'make' in the top directory of the Emacs distribution to finish
>> -building Emacs in the standard way.  The final executable file is
>> -named 'src/emacs'.  You can execute this file "in place" without
>> -copying it, if you wish; then it automatically uses the sibling
>> -directories ../lisp, ../lib-src, ../info.
>> +building Emacs in the standard way.  The final executable file is named
>> +'src/emacs'.  You can execute this file "in place" without copying it,
>> +if you wish; then it automatically uses the sibling directories ../lisp,
>> +../lib-src, ../info, ../native-lisp (if built with support for native
>> +compilation).
>> +
>> +If you build Emacs in a directory separate from the source directory
>> +("out-of-tree"), run 'make' in the build directory.  Then you can
>> +execute the 'src/emacs' file under that directory "in place".  However,
>> +running Emacs uninstalled is fully supported only when it is built in
>> +the source tree.
>
> I think the last sentence should be better phrased as
>
>   However, running Emacs uninstalled in this case will still use the
>   ../lisp and ../info subdirectories of the source tree, while using
>   the ../lib-src and ../native-lisp subdirectories from the build tree.
>
>> +'/usr/local/lib/emacs/VERSION/native-lisp' holds the natively compiled
>> +		pre-loaded Emacs Lisp files.
>
> The above also contains the other Lisp files if using the
> "--with-native-compilation=aot" configure-time option.
>
>> +'~/.emacs.d/eln-cache/' holds the remaining natively compiled Lisp files.
>
> Instead of "remaining" I'd say "just-in-time natively compiled Lisp
> files".
>
> Thanks.

Thanks again for the review and suggestions, which I've incorporated
into the attached patch, and if you confirm it's good to go, I'll commit
it to master.

Steve Berman

diff --git a/INSTALL b/INSTALL
index 2aaa02f37d7..32802d428d2 100644
--- a/INSTALL
+++ b/INSTALL
@@ -264,6 +264,25 @@ to force GTK+ to run under Broadway, start Emacs like this:
 The GNUstep build also supports the Wayland window system.  If that is
 what you want, see nextstep/INSTALL.

+* Native compilation of Lisp files
+
+In addition to byte-compiling files of Lisp code, Emacs can also produce
+"native code", which usually runs several times faster than the
+corresponding byte-compiled code.  To build Emacs with this feature,
+your system must have not only GCC (the C compiler) and Binutils (the
+assembler and linker) but also the 'libgccjit' shared library, which is
+part of the GCC distribution.  If these components are available,
+building Emacs will automatically produce natively compiled Lisp code.
+
+By default, Emacs natively compiles only pre-loaded Lisp files during
+the build process; other Lisp files are natively compiled
+"just-in-time", i.e., the first time they are loaded into the running
+Emacs.  If you want Emacs to natively compile all Lisp files during the
+build ("ahead of time"), use the 'configure' option
+'--with-native-compilation=aot'.  If you do not want natively compiled
+Lisp code even if your system satisfies the build requirements, use the
+'configure' option '--with-native-compilation=no'.
+
 DETAILED BUILDING AND INSTALLATION:

 (This is for a Unix or Unix-like system.  For GNUstep and macOS,
@@ -553,10 +572,18 @@ need to create them if you have nothing to put in them.
 wish to add to various termcap entries.  (This is unlikely to be necessary.)

 6) Run 'make' in the top directory of the Emacs distribution to finish
-building Emacs in the standard way.  The final executable file is
-named 'src/emacs'.  You can execute this file "in place" without
-copying it, if you wish; then it automatically uses the sibling
-directories ../lisp, ../lib-src, ../info.
+building Emacs in the standard way.  The final executable file is named
+'src/emacs'.  You can execute this file "in place" without copying it,
+if you wish; then it automatically uses the sibling directories ../lisp,
+../lib-src, ../info, ../native-lisp (if built with support for native
+compilation).
+
+If you build Emacs in a directory separate from the source directory
+("out-of-tree"), run 'make' in the build directory.  Then you can
+execute the 'src/emacs' file under that directory "in place".  However,
+in this case the Emacs executable, while still using the ../lisp and
+../info subdirectories of the source tree, will use the ../lib-src and
+../native-lisp subdirectories from the build tree.

 Or you can "install" the executable and the other files into their
 installed locations, with 'make install'.  By default, Emacs's files
@@ -574,6 +601,11 @@ are installed in the following directories:
 		at the same time; in particular, you don't have to
 		make Emacs unavailable while installing a new version.

+'/usr/local/lib/emacs/VERSION/native-lisp' holds the natively compiled
+		pre-loaded Emacs Lisp files.
+
+'~/.emacs.d/eln-cache/' holds the remaining natively compiled Lisp files.
+
 '/usr/local/share/emacs/VERSION/etc' holds the Emacs tutorial, the DOC
 		file, and other architecture-independent files Emacs
 		might need while running.