GNU bug report logs - #47221
Guile not in native-inputs when it should

Previous Next

To reply to this bug, email your comments to 47221 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: bug-guix <at> gnu.org
Subject: Guile not in native-inputs when it should
Date: Wed, 17 Mar 2021 22:58:57 +0100

[Message part 1 (text/plain, inline)]

Hi Guix,

(In response to bug#47027, but opened as a new bug.)

On Wed, 2021-03-17 at 21:52 +0100, Ludovic Courtès wrote:
> Hi,
> 
> Maxime Devos <maximedevos <at> telenet.be> skribis:
> [...]
> > Shouldn't the "guile" input be included in the native-inputs
> > as well (perhaps only native-inputs suffices), for cross-compilation?
> 
> Yes it should, good point.

FWIW, I tried to write a linter to catch these kind of issues.
(If there's a "guile" input, then there usually should also be
a "guile" native-input.)  Currently, it has too many false positives
for my taste.  I most likely won't be working on it in the near future
though.  (Preliminary patch attached)

> ./pre-int-env guix lint -t "check-inputs-should-also-be-native"

(Output attached)

Some suspicious things:
* guile-config & others are missing a "guile" in the native-inputs
* clipmenu & others use "wrap-script" to define wrapper scripts
  (in this case "guile" does not have to be in native-inputs).
  The "wrap-script" procedure from (guix build utils) uses the
  "which" procedure to determine where guile is located ...
  but this is incorrect when cross-compiling!

  (It is possible to override the "guile" binary used with a
  keyword argument).

  (I assume inputs in "inputs" do not contribute to the $PATH
  in a cross-compilation environment; only "native-inputs" should
  contribute to $PATH)

  idk if it is feasible or if there are complications, but
  IMHO the inputs in "inputs" shouldn't contribute to $PATH
  at all (not even when not cross-compilation), only inputs
  in $PATH.

There seems to be plenty of low-hanging cross-compilation fruit here!

Greetings,
Maxime

[0001-lint-Check-whether-guile-should-be-in-native-inputs.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 47221 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
Cc: 47221 <at> debbugs.gnu.org
Subject: Re: bug#47221: Guile not in native-inputs when it should
Date: Thu, 18 Mar 2021 08:18:38 +0100

[Message part 1 (text/plain, inline)]

I made a spelling error in the command:
./pre-inst-env guix lint -c "inputs-should-also-be-native"

I forgot to attach the output of "guix lint -c ..." (now attached).

[list-of-suspicious-packages (text/plain, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 47221 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 47221 <at> debbugs.gnu.org
Subject: Re: bug#47221: Guile not in native-inputs when it should
Date: Thu, 18 Mar 2021 10:29:59 +0100

[Message part 1 (text/plain, inline)]

On Wed, 2021-03-17 at 22:58 +0100, Maxime Devos wrote:
> [...]
> Some suspicious things:
> * [...]
> * clipmenu & others use "wrap-script" to define wrapper scripts

>   (in this case "guile" does not have to be in native-inputs).
>   The "wrap-script" procedure from (guix build utils) uses the
>   "which" procedure to determine where guile is located ...
>   but this is incorrect when cross-compiling!

Demonstration (host system: x86-64-linux with a childhurd running, without qemu binfmt),
using the "bats" package (the "bats" package is choosen because it doesn't have many
dependencies and it uses wrap-script).  ("bats" actually uses wrap-script correctly,
so first remove the following line
                                ":" (assoc-ref %build-inputs "guile") "/bin"
from the package definition to simulate a misbehaving package)

./pre-inst-env guix build --system=i586-gnu --target=x86-64-linux bats
--> tcl fails to build with plenty of failing test cases
^ TODO submit a bug report, for now try without tests

./pre-inst-env guix build --system=i586-gnu --target=x86-64-linux bats --without-tests=tcl
^ TODO this hangs the childhurd (something about paging?)

./pre-inst-env guix build --target=aarch64-linux bats
(warning: this takes some time building the cross-compiler)
--> install.sh: line 15: /gnu/store/...-coreutils-8.32/bin/install: cannot execute binary file: Exec format error

After adding "coreutils" to the native-inputs:

./pre-inst-env guix build --target=aarch64-linux bats
(success! --> some /gnu/store/something path $STORE_ITEM)

Let's look at $STORE_ITEM/bin/bats:

(start snip)
#!#f --no-auto-compile
#!#; Guix wrapper
#\-(begin (let ((current (getenv "PATH"))) (setenv "PATH" (if current (string-append "/gnu/store/qrj2w7a8ms7rkyvqhnrv8wrvqnbwv9bm-bash-
5.0.16/bin:/gnu/store/n8awazyldv9hbzb7pjcw76hiifmvrpvd-coreutils-8.32/bin:/gnu/store/3xi5vprn92r0jcb03lk9ykind5pi789j-grep-3.4/bin:/path-
not-set" ":" current) "/gnu/store/qrj2w7a8ms7rkyvqhnrv8wrvqnbwv9bm-bash-5.0.16/bin:/gnu/store/n8awazyldv9hbzb7pjcw76hiifmvrpvd-coreutils-
8.32/bin:/gnu/store/3xi5vprn92r0jcb03lk9ykind5pi789j-grep-3.4/bin:/path-not-set"))))
#\-(let ((cl (command-line))) (apply execl "/gnu/store/qrj2w7a8ms7rkyvqhnrv8wrvqnbwv9bm-bash-5.0.16/bin/bash" (car cl) (cons (car cl)
(append (quote ("")) cl))))
#!/gnu/store/qrj2w7a8ms7rkyvqhnrv8wrvqnbwv9bm-bash-5.0.16/bin/bash

set -e

BATS_READLINK='true'
[...]
(end snip)

I was worried for a moment that the inputs in "inputs" would contribute to
$PATH even when cross-compiling, but this turns out not to be the case.
However, I believe "wrap-script" should raise some kind of exception
instead of trying to use "#f" as interpreter.

--
Btw., "wrap-program" also uses "which" (but for finding the shell),
but fixing that would entail a world-rebuild as "wrap-program" doesn't
have a keyword argument

> Greetings,
> Maxime

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 47221 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 47221 <at> debbugs.gnu.org
Subject: Re: bug#47221: Guile not in native-inputs when it should
Date: Thu, 18 Mar 2021 15:01:59 +0100

[Message part 1 (text/plain, inline)]

This fixes some uses of wrap-script.

[0001-gnu-Explicitely-pass-the-guile-binary-to-wrap-script.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 47221 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 47221 <at> debbugs.gnu.org
Subject: Re: bug#47221: Guile not in native-inputs when it should
Date: Thu, 18 Mar 2021 20:08:11 +0100

[Message part 1 (text/plain, inline)]

On Thu, 2021-03-18 at 15:01 +0100, Maxime Devos wrote:
> This fixes some uses of wrap-script.

There's a bug in the patch (a missing #:input argument),
so don't apply yet.  Will be fixed in the next revision
(along with more cross-compilation fixes).

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 47221 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: 47221 <at> debbugs.gnu.org
Subject: [PATCH v2]: Correct some inputs / native-inputs issues with guile
Date: Sat, 20 Mar 2021 22:45:47 +0100

[Message part 1 (text/plain, inline)]

Hi Guix,

The first patch ‘gnu: Explicitely pass the guile binary to wrap-script.’
does what it say what it does.  This is important for picking up the guile
binary for the architecture in --target instead of the guile from native-inputs.

(The unpatched & patched package definitions do not have a guile native-input,
so before this patch they wouldn't pick up a guile at all -- packages in inputs
do not contribute to the $PATH when cross-compiling.)

The second patch ‘gnu: Make guile packages cross-compilable when the fix is trivial.’
only touches guile libraries.  It adds guile to the native-inputs when required,
sometimes it adds guile to inputs and sometimes it copies propagated-inputs & inputs
to native-inputs when cross-compiling.  It also fixes some other cross-compilations
issues like autoconf being in inputs instead of native-inputs.

The second patch only touches gnu/packages/guile-xyz.scm; other packages are ignored.
Also ignored are: emacsy-minimal (there have been patches and/or bug reports lately)
guile-bash (retired project) and guile-studio (it is an Emacs package).

Suggested testing method:

(start shell script)
# BAD_PACKAGES: fails to compile
BAD_PACKAGES="guile2.0-commonmark guile3.0-ncurses-with-gpm guile-dbi guile2.2-pfds"

# OK_PACKAGES: compiles & cross-compiles
OK_PACKAGES="guildhall guile-daemon guile-dsv guile-filesystem guile2.0-filesystem guile2.2-filesystem guile-syntax-highlight guile2.2-
syntax-highlight guile-sjson guile-sparql guile-email guile-mastodon guile-parted guile2.2-parted guile-config guile-hall guile-wisp
guile-commonmark guile-stis-parser guile-persist guile-file-names guile-srfi-158 guile-semver jupyter-guile-kernel guile-ics srfi-64-
driver guile-websocket g-wrap guile-webutils guile-srfi-159 guile-torrent guile-irc guile-machine-code guile2.2-sjson guile2.2-dsv
guile2.2-email guile2.2-config guile2.2-hall guile2.2-ics guile2.2-wisp guile2.2-commonmark guile2.2-semver guile2.2-webutils guile-redis
guile2.2-redis guile2.0-redis guile-irregex guile2.0-irregex guile2.2-irregex mcron guile2.2-mcron guile-srfi-89 guile-srfi-145 guile-
srfi-180 guile-jpeg guile-hashing guile2.2-hashing guile-packrat guile-ac-d-bus guile-lens guile2.2-lens guile-rdf guile-jsonld guile-
struct-pack guile-laesare guile-mkdir-p guile-jwt guile-r6rs-protobuf guile-shapefile schmutz guile-cbor guile-8sync guile-squee guile2.2-
squee guile-colorized guile2.2-colorized guile-pfds guile-prometheus guile-aa-tree guile-simple-zmq guile2.2-simple-zmq guile-debbugs
guile-email-latest guile-miniadapton guile-lib guile2.0-lib guile2.2-lib guile-minikanren guile2.0-minikanren guile2.2-minikanren python-
on-guile"

# NOT_CROSS_COMPILABLE: self-describing (compiles natively)
NOT_CROSS_COMPILABLE="guile-cv guile-gi guile-ncurses g-golf guile-picture-language guile-sly guile-aspell guile-fibers guile-sodium
guile-reader guile-udev haunt guile2.2-haunt guile2.0-haunt guile2.2-ncurses guile-ncurses-with-gpm guile-xosd artanis guile-xapian
guile2.2-xapian guile-newt guile2.2-newt guile2.2-reader guile-avahi guile2.0-pg guile-libyaml guile-eris guile-ffi-fftw "

make
# replace aarch64 with architecture of choice and maybe adjust -M and -c
./pre-inst-env guix build $OK_PACKAGES $NOT_CROSS_COMPILABLE -M6 -c1 --fallback
./pre-inst-env guix build $OK_PACKAGES -M6 -c1 --target=aarch64-linux-gnu --fallback
make as-derivation
(end shell script)

Greetings,
Maxime

[0001-gnu-Explicitely-pass-the-guile-binary-to-wrap-script.patch (text/x-patch, attachment)]

[0002-gnu-Make-guile-packages-cross-compilable-when-the-fi.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.