GNU bug report logs - #66801
[PATCH] mix-build-system: draft 1

Previous Next

Full log

Message #209 received at 66801 <at> debbugs.gnu.org (full text, mbox):

From: Pierre-Henry Fröhring <phfrohring <at> deeplinks.com>
To: Liliana Marie Prikler <liliana.prikler <at> gmail.com>
Cc: 66801 <at> debbugs.gnu.org
Subject: Re: [PATCH v3 01/14] build-system: Add mix-build-system.
Date: Fri, 17 Nov 2023 08:36:22 +0100

* WAITING Comment
** lilyp
> +(define* (install-hex #:key name inputs outputs #:allow-other-keys)
> +  "Install Hex."
> +  (let ((hex-archive-path (string-append (getenv "MIX_ARCHIVES")
> "/hex")))
> +    (mkdir-p hex-archive-path)
> +    (symlink (car (list-directories (elixir-libdir (assoc-ref inputs
> "elixir")
> +                                                   (assoc-ref inputs
> "elixir-hex"))))
> +             (string-append hex-archive-path "/hex"))))

Why do we need this?  It looks like we'll be pasting the same (native?)
input all over the store, which imho would be bad.


** phf
Without ~Hex~, you get this error:
#+begin_example
starting phase `build'
Could not find Hex, which is needed to build dependency :ex_doc
Shall I install Hex? (if running non-interactively, use "mix local.hex
--force") [Yn]
#+end_example
This message is called from ~handle_rebar_not_found~ in
~lib/mix/lib/mix/tasks/deps.compile.ex~ ; which is called from
~rebar_cmd~ because
a ~manager~ (~Hex~) could not be found. Hex must be present =>
install-hex must be
executed.

I thought that this should not be a problem since Hex is needed at build time,
and just symlinked. Maybe should it be copied instead?


** lilyp
Is hex not an (implicit) native-input in your build system?  Anything that
keeps it from functioning is a packaging bug imho.


** phf
If ~mix~ finds ~Hex~ under ~$MIX_ARCHIVES/hex/hex~, then ~mix compile~
does not emit
the message above. I'm not sure how could this be changed. I've tried to set
~MIX_PATH~ to ~/gnu/store/…-elixir-hex-2.0.5/lib/elixir/1.14~ without
success. So,
this is the reason why ~install-hex~ phase installs Hex like it does.


** lilyp
Look into mix and how it invokes hex.  There's hardcoding potential, I'm sure
of it.


** phf
:PROPERTIES:
:ID:       d9ffbbfa-3cd3-4a44-9acb-04a0627b90d7
:END:

- Observation: ~mix compile~ ⇒ "Could not find Hex, which is needed to
build dependency :ex_doc".
- We have the source
  - guix build --source
- We found from where this message is emitted.
  - lib/mix/lib/mix/hex.ex
- We now why the message was emitted.
  - Code.ensure_loaded?(Hex) failed.
  - https://hexdocs.pm/elixir/1.14.0/Code.html#ensure_loaded/1

Well, adding the store path of Hex to ~ERL_LIBS~ solves the problem. The
~install-hex~ phase can be deleted in favor of manipulating ~ERL_LIBS~ as
suggested with propagated inputs
[[id:d7cd6e3d-9802-499f-a157-7698aca942d4][below]].


* WAITING Comment
:PROPERTIES:
:ID:       d7cd6e3d-9802-499f-a157-7698aca942d4
:END:

** lilyp
> +  (define (install-inputs mix-env)
> +    (for-each (cut install-input mix-env <>)
> +              (append inputs native-inputs)))

Installing native inputs: probably a bad idea (inhibits cross-
compilation).


** phf
A slight variant that does not link things when it should not:
#+begin_src scheme
(define (install-inputs mix-env)
    (for-each (cut install-input mix-env <>)
              (cond
                ((string=? mix-env "prod") inputs)
                ((member mix-env '("shared" "test")) (append inputs
native-inputs))
                (else (error (format #f "Unexpected Mix env: ~a~%" mix-env))))))
#+end_src

I did not consider cross-compilation yet. The following might be wrong be here
we go. As far as I understand, Erlang applications are largely platform
independent. Once the code is compiled to BEAM bytecode, it can run on any
platform that has the Erlang VM installed. If an Erlang library uses native
extensions, then cross-compilation might be required. For a build to succeed
in a given environment (one of "prod", "test", "shared"), the BEAM files of
all dependencies should be present on the build machine. So, all dependencies
must be installed


** lilyp
Not an expert on elixir, but that sounds borked.


** phf
Yes. Did not have time to look into it as of now.


** lilyp
You might get around this with propagated-inputs maybe?  That being said,
native-inputs shouldn't blow up a build if missing.


** phf
If ~native-inputs~ are missing, it's fine. But wait, maybe there is a
misunderstanding here. Please, check the reasoning: in a cross-compilation
context, we have two machines A and B, and:
- native-inputs: dependencies that must be built on A for A ;
- inputs: dependencies that must be built on A for B ;
- propagated-inputs: like inputs but installed in the profile.


If installing Elixir (like Python) gathers all libraries in the profile with a
variable like ~GUIX_ERL_LIBS~, then, it would be enough to list dependencies (in
packages) in ~propagated-inputs~ instead of ~inputs~ and make them available to
Elixir through ~ERL_LIBS~ (like ~GUIX_PYTHONPATH~ and ~PYTHONPATH~). As a
consequence, the ~install-dependencies~ phase would be reduced to
~ERL_LIBS=$GUIX_ERL_LIBS~.


Is this an answer to: "You might get around this with propagated-inputs
maybe?"


** lilyp
If environment variables work, that is clearly to be preferred over any other
magic we've considered so far.  My hunch is that rebar-build- system was
written this way because environment variables didn't work, however.  Same
with Rust and Node, which are kinda yucky in this regard.

Other than that, yes, (ab)using propagated-inputs like that is the way to go
when there's no smarter alternative available.


** phf
As hinted [[id:d9ffbbfa-3cd3-4a44-9acb-04a0627b90d7][above]], abusing
propagated inputs works up to the ~elixir-machete~
package. Essentially, it is enough to add the code below to the
~elixir~ package:
#+begin_src scheme
(native-search-paths
 (list (search-path-specification
        (variable "GUIX_ERL_LIBS")
        (files (list "lib/erlang/lib"
                     (string-append "lib/elixir/" (version-major+minor
version)))))))
#+end_src


~install-dependencies~ has been deleted altogether for a single line added to
~set-mix-env~.
#+begin_src scheme
(setenv "ERL_LIBS" (getenv "GUIX_ERL_LIBS"))
#+end_src

So, if Erlang and Elixir dependencies are added to propagated inputs, then it
seems to work as expected.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.