Subject: bug#73979: closed (Re: bug#73979: validate-runpath phases fails when binaries linked to package's own libraries)

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 19 Dec 2024 05:12:02 +0000

Your bug report

#73979: validate-runpath phases fails when binaries linked to package's own libraries

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
73979: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=73979
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#73979: validate-runpath phases fails when binaries linked to package's own libraries
From: Maxim Cournoyer <[email-masked]>
To: Ludovic Courtès <[email-masked]>
Cc: [email-masked]
Date: Thu, 19 Dec 2024 14:10:21 +0900
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi Ludovic,

Ludovic Courtès <[email-masked]> writes:

> Hi,
>
> Maxim Cournoyer <[email-masked]> skribis:
>
>> There's a common pattern in packages where the validate-runpath phases
>> fail, which is when a binary is linked to libraries provided by the same
>> package.  In this case, our ld-wrapper script appears to not bake the
>> required runpath, which then fails the validate-runpath phase.
>>
>> When this happens, the common workaround is adding link directives such
>> as (string-append "-Wl,-rpath=" #$output "/lib/subdir") to LDFLAGS (see
>> for example the 'dmraid' package definition).
>
> I believe this is the exception, not the rule, and I see that as a bug
> in the build system of those packages.  (As a counterexample, any
> package built with Automake/Libtool is fine.)

Interesting.  I had not considered that point of view (that the build
system/usage of it could be at cause) enough; I think you are right.

[...]

>> One idea could be to allow creating rpaths to %BUILD-DIRECTORY prefixed
>> libraries, and have these entries refined in a phase that would run
>> after the package is installed, before the validate-runpath phase runs.
>> It could be called e.g. 'sanitize-runpath' and proceed along those
>> lines:
>>
>> Scan for RUNPATH entries being prefixed by %BUILD-DIRECTORY; attempt to
>> have them rewritten to libraries (.so) found under the package's own
>> libdir library prefix (at any depth), including a potential "lib"
>> output.  In case it couldn't, it would error.
>>
>> Does that sound feasible?
>
> It might be feasible, but I think it’s the wrong approach.  The problem
> here is in the build system itself; Guix is “not at fault”, so to speak.
>
> Nevertheless, from a practical viewpoint, whether or not Guix is at
> fault is largely irrelevant.  So the question becomes: how widespread is
> this issue?  If it’s widespread, can it be solved at a (guix
> build-system …) level?  (For instance, I think ‘cmake-build-system’ and
> ‘meson-build-system’ do the right things in that regard.)

I think CMake nowadays doesn't relink but uses some ELF-patching code to
patch the installed binaries in place (which is faster than relinking);
that would mean our ld-wrapper doesn't get involved in these case and we
depend on RPATH being enabled in the CMake build system, which it
normally is unless the package set CMAKE_SKIP_RPATH to false.  I guess
Meson is also patching binaries for speed.

> If it’s a per-package issue, which seems to be the case given what you
> describe, then I would fix it locally in this package, for instance by
> passing ‘-Wl,-rpath=$ORIGIN/../lib’ or whatever is convenient.

There doesn't seem to be so many occurrences; grepping for
'-Wl,-rpath.*output' I find 73 problematic packages.  I've found one
place where this happens is for Cythonized shared objects; this produced
shared objects linked to a library of the build directory, and is not
concerned with installing so nothing rewrites the runpath entries to
their correct installed location.  One such example is openpmix; the
issue was dismissed as "we expect people to set LD_LIBRARY_PATH" [0].

[0]  https://github.com/openpmix/openpmix/issues/3457

Anyway, sorry for the wall of text.  Closing!

-- 
Thanks,
Maxim

Subject: validate-runpath phases fails when binaries linked to package's own libraries
From: Maxim Cournoyer <[email-masked]>
To: bug-guix <[email-masked]>
Date: Thu, 24 Oct 2024 15:01:11 +0900
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

There's a common pattern in packages where the validate-runpath phases
fail, which is when a binary is linked to libraries provided by the same
package.  In this case, our ld-wrapper script appears to not bake the
required runpath, which then fails the validate-runpath phase.

When this happens, the common workaround is adding link directives such
as (string-append "-Wl,-rpath=" #$output "/lib/subdir") to LDFLAGS (see
for example the 'dmraid' package definition).

Here's a recent example, attempting to build TORCS:

--8<---------------cut here---------------start------------->8---
starting phase `validate-runpath'
validating RUNPATH of 32 binaries in "/gnu/store/q25qb3r6kzhn9w7mac9i9gn4gg45c7v1-torcs-1.3.7/lib"...

[...]

validating RUNPATH of 0 binaries in "/gnu/store/q25qb3r6kzhn9w7mac9i9gn4gg45c7v1-torcs-1.3.7/bin"...
error: in phase 'validate-runpath': uncaught exception:
misc-error #f "RUNPATH validation failed" () #f
--8<---------------cut here---------------end--------------->8---

Here's what the link command for the problematic accc-bin binary looks
like, when the GUIX_LD_WRAPPER_DEBUG environment variable is set during
its build:

--8<---------------cut here---------------start------------->8---
g++ mainaccc.o ../../linux/linuxspec.o ac3dload.o ac3dgroup.o -L/tmp/guix-build-torcs-1.3.7.drv-0/torcs-1.3.7/export/lib  -lopenal -lalut -lvorbisfile -L/usr/lib  -ltgf -lplibssg -lplibsg -lplibul -ltxml -lplibjs -lplibssgaux -lplibssg -lplibsm -lplibsl -lplibsg -lplibul -lglut -lGLU -lGL -lpng -lz -ldl -lXrandr -lXrender -lXxf86vm -lXmu -lXi -lXt -lSM -lICE -lXext -lX11 -lm  -o accc-bin

--8<---------------cut here---------------end--------------->8---

So we see that ld-wrapper saw the accc-bin binary being linked against
the package's own
/tmp/guix-build-torcs-1.3.7.drv-0/torcs-1.3.7/export/lib/libtgf.so, but
this is later dismissed for "not being in the store", by this code in
gnu/packages/ld-wrapper.in:

--8<---------------cut here---------------start------------->8---
(define (rpath-arguments library-files)
  ;; Return the `-rpath' argument list for each of LIBRARY-FILES, a list of
  ;; absolute file names.
  (fold-right (lambda (file args)
                ;; Add '-rpath' if and only if FILE is in the store; we don't
                ;; want to add '-rpath' for files under %BUILD-DIRECTORY or
                ;; %TEMPORARY-DIRECTORY because that could leak to installed
                ;; files.
                (cond ((and (not %disable-rpath?)
                            (store-file-name? file))
                       (cons* "-rpath" (dirname file) args))
                      ((or %allow-impurities?
                           (pure-file-name? file))
                       args)
                      (else
                       (begin
                         (format (current-error-port)
                                 "ld-wrapper: error: attempt to use \
library outside of ~a: ~s~%"
                                 %store-directory file)
                         (exit 1)))))
              '()
              library-files))
--8<---------------cut here---------------end--------------->8---

(pure-file-name? x) returns #t (it considers build directory files as
"pure"), but the args is not augmented with -rpath in this case, so the
link directives are simply skipped.

One idea could be to allow creating rpaths to %BUILD-DIRECTORY prefixed
libraries, and have these entries refined in a phase that would run
after the package is installed, before the validate-runpath phase runs.
It could be called e.g. 'sanitize-runpath' and proceed along those
lines:

Scan for RUNPATH entries being prefixed by %BUILD-DIRECTORY; attempt to
have them rewritten to libraries (.so) found under the package's own
libdir library prefix (at any depth), including a potential "lib"
output.  In case it couldn't, it would error.

Does that sound feasible?

-- 
Thanks,
Maxim