Package: guix-patches;
Reported by: Hilton Chain <hako <at> ultrarare.space>
Date: Tue, 18 Mar 2025 07:18:02 UTC
Severity: normal
Tags: patch
View this message in rfc822 format
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Subject: [bug#77093] [PATCH rust-team v3 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Sun, 23 Mar 2025 15:28:39 +0800
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
* doc/guix-cookbook.texi (Packaging)[Packaging Workflow]: New section.
Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
doc/contributing.texi | 91 ++++++++---
doc/guix-cookbook.texi | 352 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 418 insertions(+), 25 deletions(-)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..17f22390d5 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,75 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
+Rust programs (binary crates) and dependencies (library crates) are treated
+separately. We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
+vendored dependencies. The following paragraphs will explain them and give
+several examples.
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix. The name should be changed to lowercase as appropriate and
-dashes should remain in place.
+Rust programs are treated like any other package and named using the lowercase
+upstream name. When using the Cargo build system (@pxref{Build Systems,
+@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
+argument set to @code{#f}, as this argument only makes sense for dependencies.
+When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
+be set to enable relevant support.
+
+Rust dependencies are managed in two modules:
+
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, @code{crate}, @code{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored sources unbundled before
+being contributed to Guix. This module is managed by the Rust team
+(@pxref{Teams}).
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages. This includes Rust dependencies requiring external inputs
+to unbundle and Cargo workspaces.
+
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix. The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix. The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course). This follows the ``caret'' version scheme
-intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments. It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}. If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible. Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+package to be used at any given time, so all dependencies should have a
+versioned suffix. The versioned suffix is the left-most non-zero digit (and any
+leading zeros, of course). This follows the ``caret'' version scheme intended
+by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we are usually packaging development snapshots of Rust dependencies
+specifically for some Rust programs, and can't simply identity them by version.
+In this case we can use a @code{for-@var{program}} suffix, for example,
+@code{rust-pipewire-for-niri} and @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Rust dependencies are not referenced directly. @code{(guix build-sytem cargo)}
+provides a @code{cargo-inputs} procedure to create an input list, in combination
+with the lockfile importer.
+
+@cindex cargo-inputs
+@deffn {Procedure} cargo-inputs @var{name} @
+ [#:crates-module '(gnu packages rust-crates)] @
+ [#:sources-module '(gnu packages rust-sources)]
+Given symbol @code{'@var{name}}, resolve variable
+@code{@var{name}-cargo-inputs}, an input list, in @var{crates-module}, return
+its copy with @code{#f} removed and symbols resolved to variables defined in
+@var{sources-module} if the input list exists, otherwise return an empty list.
+
+@var{name} must be consistent with the one used in lockfile importer invocation.
+@end deffn
+
+For a more detailed packaging workflow, @pxref{Packaging Rust Crates,,,
+guix-cookbook, GNU Guix Cookbook}.
@node Elm Packages
diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index d9b98a2ab3..1f0e5eee54 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -103,6 +103,7 @@ Top
Packaging
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
Packaging Tutorial
@@ -127,6 +128,16 @@ Top
* Automatic update::
* Inheritance::
+Packaging Workflows
+
+* Packaging Rust Crates::
+
+Packaging Rust Crates
+
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+
System Configuration
* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
@@ -514,6 +525,7 @@ Packaging
@menu
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
@end menu
@node Packaging Tutorial
@@ -1593,6 +1605,346 @@ References
@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
@end itemize
+@node Packaging Workflows
+@section Packaging Workflows
+
+The following sections are real life examples on working with specific build
+systems, serving as extensions to the concise packaging guidelines
+(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
+
+@menu
+* Packaging Rust Crates::
+@end menu
+
+@node Packaging Rust Crates
+@subsection Packaging Rust Crates
+
+In preparation, add the following packages to our environment:
+
+@example
+$ guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+@menu
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+@end menu
+
+@node Common Workflow for Packaging Rust Crates
+@subsubsection Common Workflow for Packaging Rust Crates
+
+In this example, we'll package @code{cargo-audit}, which is published on the
+@uref{https://crates.io, crates.io} Rust package repository. All its
+dependencies are on crates.io as well.
+
+@enumerate
+@item
+Since @code{cargo-audit} is available on crates.io, We can generate a draft
+definition via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU
+Guix Reference Manual}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+The symbol used in @code{cargo-inputs}, @code{'cargo-audit} here, must be a
+unique identifier, usually matching the variable name of the package.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+$ cargo generate-lockfile
+$ cargo audit
+$ cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining
+Packages,,, guix, GNU Guix Reference Manual}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+$ guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, package @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them via the Guix command-line interface through expression:
+
+@example
+$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+reference it by a symbol in the imported definition.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to a matching symbol.
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+@end enumerate
+
+@node Development Snapshots and Cargo Workspaces
+@subsubsection Development Snapshots and Cargo Workspaces
+
+In this example, we'll package @code{niri}, which depends on development
+snapshots (also Cargo workspaces here).
+
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+to a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in a build phase, with a package-specific substitution
+pattern.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3+)))
+@end lisp
+
+@code{niri} has Cargo workspace dependencies. When packaging a Cargo workspace,
+argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+
+@node Rust Programs With Mixed Build Systems
+@subsubsection Rust Programs With Mixed Build Systems
+
+In this example, we'll package @code{fish}, which combines two build systems.
+
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+
+
@c *********************************************************************
@node System Configuration
@chapter System Configuration
--
2.49.0
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.