Package: guix-patches;
Reported by: Pierre-Henry Fröhring <phfrohring <at> deeplinks.com>
Date: Sat, 28 Oct 2023 20:21:02 UTC
Severity: normal
Tags: patch
Done: Liliana Marie Prikler <liliana.prikler <at> gmail.com>
Bug is archived. No further changes may be made.
Message #140 received at 66801 <at> debbugs.gnu.org (full text, mbox):
From: Pierre-Henry Fröhring <phfrohring <at> deeplinks.com> To: 66801 <at> debbugs.gnu.org Cc: Pierre-Henry Fröhring <phfrohring <at> deeplinks.com> Subject: [PATCH 0/5] build Erlang packages with dependencies Date: Wed, 8 Nov 2023 10:22:34 +0100
Introduction
════════════
• The objective is to introduce a `mix-build-system' for building
Elixir packages.
• It is first necessary to have `rebar-build-system' to build Erlang
packages.
• We start by showing a build procedure using a package named `lager'
that operates without the need for network access.
• We generalize the build procedure to any Erlang package by using
[Checkout Dependencies].
• To align this build procedure with Guix, we ensure that for any
Erlang Guix package, all its dependencies, including the transitive
ones, can be identified.
• We show that the source code for each dependency is retrievable from
the store.
• We modify `rebar.scm' and `rebar-build-system.scm' to build
procedure into Guix.
• The packages defined in `erlang-xyz.scm' serve as the test cases for
this implementation.
• The changes to the Guix source code are minimal. While the primary
goal is to validate the approach, it may be necessary to resolve
stylistic issues or potential bugs.
[Checkout Dependencies]
<https://rebar3.org/docs/configuration/dependencies/#checkout-dependencies>
Bash
════
Rebar3 compiles and tests Lager without network access
──────────────────────────────────────────────────────
There is a directory and an environment such that `rebar3 compile' and
`rebar3 test' compile and test the [lager package] without network
access.
[lager package] <https://hex.pm/packages/lager>
Sources are fetched and unpacked
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
┌────
│ #! /usr/bin/env bash
│ set -euo pipefail
│ IFS=$'\n\t'
│
│ tmpdir=$(mktemp -d)
│ cd $tmpdir
│ wget -O source.tar https://repo.hex.pm/tarballs/lager-3.9.2.tar
│ tar xf source.tar
│ tar xf contents.tar.gz
└────
Dependencies are fetched
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
┌────
│ rebar3 compile
│ rebar3 eunit
└────
Checkouts are built
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
#+begin_src bash mkdir _checkouts mv _build/default/lib/goldrush
_checkouts rm -rf _build #+end_srcè
The directory is built
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
┌────
│ .
│ ├── _checkouts
│ │ └── goldrush
│ │ ├── ebin
│ │ │ ├── …
│ │ │ └── gr_sup.beam
│ │ ├── hex_metadata.config
│ │ ├── LICENSE
│ │ ├── priv
│ │ │ └── edoc.css
│ │ ├── README.org
│ │ ├── rebar.config
│ │ ├── rebar.lock
│ │ └── src
│ │ ├── …
│ │ └── gr_sup.erl
│ ├── CHECKSUM
│ ├── contents.tar.gz
│ ├── include
│ │ └── lager.hrl
│ ├── LICENSE
│ ├── metadata.config
│ ├── priv
│ │ └── edoc.css
│ ├── README.md
│ ├── rebar.config
│ ├── rebar.config.script
│ ├── rebar.lock
│ ├── source.tar
│ ├── src
│ │ ├── …
│ │ └── lager_util.erl
│ └── VERSION
│
│ 9 directories, 73 files
└────
The environment is defined
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
┌────
│ (specifications->manifest '("coreutils" "rebar3"))
└────
Lager is compiled and tested without network access
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
┌────
│ guix shell -C -m manifest.scm -- rebar3 compile
│ guix shell -C -m manifest.scm -- rebar3 eunit
└────
Erlang can find and use the compiled module
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
┌────
│ ERL_LIBS=_build/default/lib erl -noshell -eval 'io:format("~p~n", [code:which(lager)]).' -s init stop
└────
┌────
│ "_build/default/lib/lager/ebin/lager.beam"
└────
Discussion
══════════
Several observations can be made, and alternative approaches can be
proposed, as listed below. Meanwhile, with the implementation
described above, it becomes feasible to build Erlang packages with
Guix, which is an essential step towards building Elixir packages.
Wasted Computations
───────────────────
This build procedure needs to re-build of all Erlang dependencies for
an Erlang package, even if they have been previously built. This is
due to the `rebar3' build system's refusal to operate without network
access unless the dependencies are checked out. If checked out, it
insists on rebuilding them from sources.
Expensive computations
──────────────────────
For a given Erlang dependency, it is necessary to retreive all its
transitive dependencies which might be expensive to compute. See:
[all-transitive-inputs].
[all-transitive-inputs] <./guix/build-system/rebar.scm::(define
(all-transitive-inputs>
Wasted Space
────────────
The source code for all Erlang dependencies of a specific Erlang
package must be available on the build machine, whereas it might be
assumed that the build artifacts alone would suffice.
Alternatives
────────────
Liliana Marie Pinker has suggested several alternatives listed below.
Store sources along the build artifacts
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
From [issue/66801]:
keep the sources in lib/erlang/lib/lib-name/src so that it
gets symlinked by the phase we have. Though at that point
we can surely go with a less surprising install directory.
[issue/66801] <https://issues.guix.gnu.org/issue/66801>
Write the sources to a special source output
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
From [issue/66801]:
You could alternatively also write the sources to a
special source output if and only if one such output is
requested via the outputs field. Again, I'd hazard a
guess that this would be a very standard output for
anything that needs to go into mix-build-system and thus
not worth the split after all.
[issue/66801] <https://issues.guix.gnu.org/issue/66801>
Patch Rebar3
╌╌╌╌╌╌╌╌╌╌╌╌
From [issue/66801]:
you could patch mix to only look for compiled stuff and
not sources. That would tackle the issue at the root
instead of trying to work around it, with the caveat being
that we would need to maintain this patch ourselves if
upstream doesn't accept it.
[issue/66801] <https://issues.guix.gnu.org/issue/66801>
Pierre-Henry Fröhring (5):
guix: build-system: rebar: build Erlang packages with dependencies.
gnu: Add erlang-goldrush.
gnu: Add erlang-lager.
gnu: Add erlang-unicode-util-compat.
gnu: Add erlang-idna.
gnu/packages/erlang-xyz.scm | 111 +++++++++++++++
guix/build-system/rebar.scm | 223 ++++++++++++++++++++++--------
guix/build/rebar-build-system.scm | 43 +++---
3 files changed, 300 insertions(+), 77 deletions(-)
create mode 100644 gnu/packages/erlang-xyz.scm
base-commit: 4dfbc536689b07e56aead3dd864b8af54613d091
--
2.41.0
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.