Package: guix-patches;
Reported by: Hilton Chain <hako <at> ultrarare.space>
Date: Tue, 18 Mar 2025 07:18:02 UTC
Severity: normal
Tags: patch
View this message in rfc822 format
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: steve <at> futurile.net, 77093 <at> debbugs.gnu.org, divya <at> subvertising.org Subject: [bug#77093] [PATCH rust-team v2 00/17] New Rust packaging workflow based on lockfile importer. Date: Wed, 19 Mar 2025 08:52:19 +0200
[Message part 1 (text/plain, inline)]
+CC the other rust-team members, not sure how they got left out.
guix/import/crate/cargo-lock.scm should be added to etc/teams.scm.
I was thinking some more about the rust-crates.tmpl file and I think it
makes sense to keep it since it'll likely be useful for people who want
rust packages in their own channel or for working into a guix.scm file.
Some of the patches, specifically some of the importer ones, seem like
they could go straight to master.
On Tue, Mar 18, 2025 at 10:26:43PM +0800, Hilton Chain wrote:
> V1 -> V2:
> * Drop jemalloc environment variable patch.
> * Remove use of RUSTC_BOOTSTRAP.
> * Adjust documentation.
> Diff inserted at end.
>
> Efraim Flashner (1):
> build/cargo: Don't try to unpack sanity-check.py.
>
> Hilton Chain (16):
> build/cargo: Pass ‘--offline’ to cargo.
> build/cargo: Print out all non-empty binary files.
> build-system: cargo: Support packaging Cargo workspace.
> build-system: cargo: Support installing Cargo workspace.
> build/cargo: Set default value of arguments for build phases.
> build/cargo: Support non-workspace directory source inputs.
> scripts: import: Document argument for ‘--insert’ option in help
> message.
> scripts: import: Add two newlines for ‘--insert’ option.
> scripts: import: Support expressions defined by 'define.
> scripts: import: Pass "--insert" to importers.
> scripts: import: Skip existing definition for ‘--insert’ option.
> import: crate: crate-name->package-name: Move to (guix build-system
> cargo).
> build-system: cargo: Add ‘cargo-inputs’.
> import: crate: Add Cargo.lock parser.
> import: crate: Add ‘--lockfile’ option.
> doc: Document lockfile importer based Rust packaging workflow.
>
> Makefile.am | 1 +
> doc/contributing.texi | 415 ++++++++++++++++++++++++++++--
> doc/guix.texi | 24 ++
> etc/teams/rust/cleanup-crates.sh | 38 +++
> etc/teams/rust/rust-crates.tmpl | 42 +++
> gnu/local.mk | 2 +
> gnu/packages/rust-crates.scm | 42 +++
> gnu/packages/rust-sources.scm | 29 +++
> guix/build-system/cargo.scm | 57 +++-
> guix/build/cargo-build-system.scm | 151 ++++++++---
> guix/import/crate.scm | 81 +++++-
> guix/import/crate/cargo-lock.scm | 105 ++++++++
> guix/scripts/import.scm | 53 ++--
> guix/scripts/import/crate.scm | 58 ++++-
> guix/utils.scm | 27 +-
> tests/crate.scm | 88 +++++++
> 16 files changed, 1107 insertions(+), 106 deletions(-)
> create mode 100755 etc/teams/rust/cleanup-crates.sh
> create mode 100644 etc/teams/rust/rust-crates.tmpl
> create mode 100644 gnu/packages/rust-crates.scm
> create mode 100644 gnu/packages/rust-sources.scm
> create mode 100644 guix/import/crate/cargo-lock.scm
>
>
> base-commit: b54a9ca849f013300c633fb79d80bc754f6b28a2
> --
> 2.48.1
>
> --8<---------------cut here---------------start------------->8---
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index e7c5116a3d..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1607,15 +1607,15 @@ Rust Crates
> have switched to a different packaging model.
>
> Rust programs (binary crates) and dependencies (library crates) are treated
> -separately. We put main efforts into programs and only package Rust
> -dependencies as sources, utilizing automation with manual focus on unbundling
> +separately. We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> vendored dependencies. The following paragraphs will explain them and give
> several examples.
>
> Rust programs are treated like any other package and named using the lowercase
> upstream name. When using the Cargo build system (@pxref{Build Systems,
> @code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> -argument set to @code{#f}, as this argument only makes sense to dependencies.
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> be set to enable relevant support.
>
> @@ -1631,21 +1631,21 @@ Rust Crates
> contributing to Guix. Naturally, this module serves as a store for both sources
> and unbundling strategies.
>
> -This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
> -version containing all changes from other branches, so that the maintained
> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> version can be used directly in case of merge conflicts, thus coordination is
> required for other committers to modify it.
>
> Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
> -script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
> +script @file{etc/teams/rust/cleanup-crates.sh} for this module.
>
> @item
> @code{(gnu packages rust-sources)}, storing more complex definitions that need
> to be full packages. This includes Rust dependencies requiring external inputs
> to unbundle and Cargo workspaces.
>
> -These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
> -For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
>
> Since they are added manually, the following naming convention applies:
>
> @@ -1668,7 +1668,7 @@ Rust Crates
> Let's demonstrate the packaging workflow by examples, note that package-specific
> issues are not involved here.
>
> -In preparation, we'll adding the following packages to our environment:
> +In preparation, we'll add the following packages to our environment:
>
> @example
> guix shell rust rust:cargo cargo-audit cargo-license
> @@ -1747,7 +1747,7 @@ Rust Crates
> @code{check-for-pregenerated-files} phase:
>
> @example
> -$ guix build cargo-audit
> +$ ./pre-inst-env guix build cargo-audit
> @dots{}
> starting phase `check-for-pregenerated-files'
> Searching for binary files...
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 029c8cf59c..218c2ba630 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -9365,14 +9365,14 @@ Build Systems
> sources, to ease in future hacking on rust packages.
>
> This build system supports cargo workspaces. Parameter
> -@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
> -names of library crates to package in @code{package} phase. Specified
> -crates are packaged from left to right, in case there's dependency among
> -them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
> -package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
> -@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
> -paths of binary crates to install in @code{install} phase,
> -@code{''("crates/atuin")}, for example.
> +@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
> +library crates to package in the @code{package} phase. Specified crates are
> +packaged from left to right, in case there's dependency among them. For
> +example, specifying @code{''("pcre2-sys" "pcre2")} will package
> +@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
> +@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
> +binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
> +for example.
> @end defvar
>
> @defvar chicken-build-system
> @@ -14705,19 +14705,18 @@ Invoking guix import
> a comment.
> @item --lockfile=@var{file}
> @itemx -f @var{file}
> -When @option{--lockfile} is specified, importer will ignore other
> -options and won't output package definitions, instead generating source
> -definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
> -For example:
> +When @option{--lockfile} is specified, the importer will ignore other options
> +and won't output package definitions, instead generating source definitions for
> +all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
>
> @example
> guix import crate --lockfile=/path/to/Cargo.lock my-package
> @end example
>
> -generates sources from @file{/path/to/Cargo.lock}, and a list
> -@code{my-package-cargo-inputs} referencing these sources. The generated
> -list is intended for package's base @code{inputs}, replacing
> -@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
> +generates sources from @file{/path/to/Cargo.lock} and a list
> +@code{my-package-cargo-inputs} referencing these sources. The generated list is
> +intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
> +@code{#:cargo-development-inputs}.
> @end table
>
> @item elm
> diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
> index b547421b88..41adc03752 100644
> --- a/guix/build/cargo-build-system.scm
> +++ b/guix/build/cargo-build-system.scm
> @@ -248,12 +248,6 @@ (define* (configure #:key inputs
> (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
> (setenv "SODIUM_USE_PKG_CONFIG" "1")
> (setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
> - ;; This flag is needed when not using the bundled jemalloc.
> - ;; https://github.com/tikv/jemallocator/issues/19
> - (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
> - (when (assoc-ref inputs "jemalloc")
> - (setenv "JEMALLOC_OVERRIDE"
> - (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so")))
> (when (assoc-ref inputs "openssl")
> (setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
> (when (assoc-ref inputs "gettext")
> @@ -356,23 +350,17 @@ (define* (package #:key
>
> (if (null? cargo-package-crates)
> (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
> - ;; Use unstable feature ‘--registry’.
> (begin
> - (setenv "RUSTC_BOOTSTRAP" "1")
> (for-each
> (lambda (pkg)
> (apply invoke "cargo" "package" "--offline" "--package" pkg
> - "--registry" "crates-io" "-Z" "package-workspace"
> cargo-package-flags)
> (for-each
> (lambda (crate)
> (invoke "tar" "xzf" crate "-C" vendor-dir))
> - (begin
> - (delete-file-recursively "target/package/tmp-registry")
> - (find-files "target/package" "\\.crate$")))
> + (find-files "target/package" "\\.crate$"))
> (patch-cargo-checksums #:vendor-dir vendor-dir))
> - cargo-package-crates)
> - (unsetenv "RUSTC_BOOTSTRAP")))
> + cargo-package-crates)))
>
> ;; Then unpack the crate, reset the timestamp of all contained files, and
> ;; repack them. This is necessary to ensure that they are reproducible.
> diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> index d8e1ed3e6f..f6e2985ed9 100644
> --- a/guix/scripts/import.scm
> +++ b/guix/scripts/import.scm
> @@ -126,13 +126,12 @@ (define-command (guix-import . args)
> (show-version-and-exit "guix import"))
> ((or ("-i" file importer args ...)
> ("--insert" file importer args ...))
> - (let* ((definer?
> - (cut member
> - <>
> - `(,@(if (member importer '("crate"))
> - '(define)
> - '())
> - define-public)))
> + (let* ((importer-definers
> + `(,@(if (member importer '("crate"))
> + '(define)
> + '())
> + define-public))
> + (definer? (cut member <> importer-definers))
> (find-and-insert
> (lambda (expr)
> (match expr
> diff --git a/tests/crate.scm b/tests/crate.scm
> index d46f753f9c..9b7066c3b1 100644
> --- a/tests/crate.scm
> +++ b/tests/crate.scm
> @@ -478,7 +478,7 @@ (define rust-leaf-bob-3.0.2-yanked
> (license #f)))
>
> (define temp-file
> - (string-append "t-utils-" (number->string (getpid))))
> + (string-append "t-crate-" (number->string (getpid))))
>
>
> (test-begin "crate")
> --8<---------------cut here---------------end--------------->8---
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.