GNU bug report logs - #72613
Lack of fast-install on Linux proves painful w/ LTO

Previous Next

To reply to this bug, email your comments to 72613 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sam James <sam <at> gentoo.org>
To: bug-libtool <at> gnu.org
Subject: Lack of fast-install on Linux proves painful w/ LTO
Date: Wed, 14 Aug 2024 04:18:04 +0100

[Message part 1 (text/plain, inline)]

Hi!

At the moment, it looks like fast-install is disabled on Linux:

# This must be glibc/ELF.
linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*)
  version_type=linux # correct to gnu/linux during the next big refactor
  need_lib_prefix=no
  need_version=no
  ...
  # This implies no fast_install, which is unacceptable.
  # Some rework will be needed to allow for fast_install
  # before this can be enabled.
  hardcode_into_libs=yes

This is particularly painful when using LTO, as you end up with a pretty
time-consuming relink for something which is in theory avoidable.

(I've tried to follow the logic through but failed. It's not clear to me
why Linux needs this opt-out. I'm sure there *is* a reason, I just can't
spot it yet.)

thanks,
sam

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 72613 <at> debbugs.gnu.org (full text, mbox):

From: Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
To: Sam James <sam <at> gentoo.org>, 72613 <at> debbugs.gnu.org
Subject: Re: bug#72613: Lack of fast-install on Linux proves painful w/ LTO
Date: Thu, 15 Aug 2024 18:51:53 +0300

[Message part 1 (text/plain, inline)]

Severity: wishlist

On 14/08/2024 06:18, Sam James wrote:
> Hi!
> 
> At the moment, it looks like fast-install is disabled on Linux:
> 
> # This must be glibc/ELF.
> linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*)
>    version_type=linux # correct to gnu/linux during the next big refactor
>    need_lib_prefix=no
>    need_version=no
>    ...
>    # This implies no fast_install, which is unacceptable.
>    # Some rework will be needed to allow for fast_install
>    # before this can be enabled.
>    hardcode_into_libs=yes
> 
> This is particularly painful when using LTO, as you end up with a pretty
> time-consuming relink for something which is in theory avoidable.
> 
> (I've tried to follow the logic through but failed. It's not clear to me
> why Linux needs this opt-out. I'm sure there *is* a reason, I just can't
> spot it yet.)

Looking through the documentation:

"""
@defvar fast_install
Determines whether libtool will privilege the installer or the
developer.  The assumption is that installers will seldom run programs
in the build tree, and the developer will seldom install.  This is only
meaningful on platforms where @code{shlibpath_overrides_runpath} is
not @samp{yes}, so @code{fast_install} will be set to @samp{needless} in
this case.  If @code{fast_install} set to @samp{yes}, libtool will create
programs that search for installed libraries, and, if a program is run
in the build tree, a new copy will be linked on-demand to use the
yet-to-be-installed libraries.  If set to @samp{no}, libtool will create
programs that use the yet-to-be-installed libraries, and will link
a new copy of the program at install time.  The default value is
@samp{yes} or @samp{needless}, depending on platform and configuration
flags, and it can be turned from @samp{yes} to @samp{no} with the
configure flag @option{--disable-fast-install}.

On some systems, the linker always hardcodes paths to dependent libraries
into the output.  In this case, @code{fast_install} is never set to 
@samp{yes},
and relinking at install time is triggered.  This also means that 
@env{DESTDIR}
installation does not work as expected.
@end defvar
[...]
@defvar hardcode_into_libs
Whether the platform supports hardcoding of run-paths into libraries.
If enabled, linking of programs will be much simpler but libraries will
need to be relinked during installation.  Set to @samp{yes} or @samp{no}.
@end defvar
"""

If I understand, hardcoding run-paths into libraries causes the
relinking during installation, which makes fast_install not possible.

I am not sure of all the implications of removing the hardcoding of
run-paths in libraries, except that linking of programs will be more
complex. For now, I think it would be best to look more into this
after the stable release, but if anyone has any ideas or corrections,
please add them here.

-- 
Ileana Dumitrescu

GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354

[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #11 received at 72613 <at> debbugs.gnu.org (full text, mbox):

From: Sam James <sam <at> gentoo.org>
To: Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
Cc: 72613 <at> debbugs.gnu.org
Subject: Re: bug#72613: Lack of fast-install on Linux proves painful w/ LTO
Date: Thu, 15 Aug 2024 17:04:54 +0100

[Message part 1 (text/plain, inline)]

Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com> writes:

> Severity: wishlist
>
> [...]
> For now, I think it would be best to look more into this
> after the stable release, but if anyone has any ideas or corrections,
> please add them here.

Waiting is completely fine with me until after the release. I will try
find time to digest the logic as well (but no promises as I've tried
already and it's complicated). I just wanted to file it so I didn't
forget.

Cheers.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 72613 <at> debbugs.gnu.org (full text, mbox):

From: Nick Bowler <nbowler <at> draconx.ca>
To: Sam James <sam <at> gentoo.org>
Cc: 72613 <at> debbugs.gnu.org
Subject: Re: bug#72613: Lack of fast-install on Linux proves painful w/ LTO
Date: Thu, 15 Aug 2024 13:12:18 -0400

On 2024-08-13 23:18, Sam James wrote:
> At the moment, it looks like fast-install is disabled on Linux:

I think the comment is very out of date.

> # This must be glibc/ELF.
> linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*)
>   version_type=linux # correct to gnu/linux during the next big refactor
>   need_lib_prefix=no
>   need_version=no
>   ...
>   # This implies no fast_install, which is unacceptable.
>   # Some rework will be needed to allow for fast_install
>   # before this can be enabled.
>   hardcode_into_libs=yes
> 
> This is particularly painful when using LTO, as you end up with a pretty
> time-consuming relink for something which is in theory avoidable.

On GNU/Linux, I believe relink should only be performed for uninstalled
shared libraries that link against other uninstalled shared libraries.

This is needed because the necessary runpaths are, as suggested by the
name hardcode_into_libs, hardcoded into libraries.  You can see this by
running readelf on such libraries in the build tree.  This has to be
changed to remove the build location when the libraries are ultimately
installed, and the way libtool currently does that is by relinking.

For normal libraries which do not depend on uninstalled shared libraries,
and for programs, no relinking is expected on GNU/Linux, so you should be
able to avoid this if you can restructure your project to avoid such
transitive dependencies between uninstalled shared libraries.

For the special case where libraries are located in the same build
directory and also installed into the same directory, on some platforms
(at least Solaris and GNU/Linux) Libtool could perhaps be changed to
generate runpaths using ${ORIGIN} which would not need to be changed
on installation.  That might help avoid relinks in certain cases.

Cheers,
  Nick

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.