Subject: [PATCH rust-team v3 17/17] doc: Document lockfile importer based Rust packaging workflow.

From: Hilton Chain <[email-masked]>
Date: Sun, 23 Mar 2025 15:28:39 +0800

* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
* doc/guix-cookbook.texi (Packaging)[Packaging Workflow]: New section.

Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
 doc/contributing.texi  |  91 ++++++++---
 doc/guix-cookbook.texi | 352 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 418 insertions(+), 25 deletions(-)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..17f22390d5 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,75 @@ Rust Crates
 @subsection Rust Crates
 
 @cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
+Rust programs (binary crates) and dependencies (library crates) are treated
+separately.  We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
+vendored dependencies.  The following paragraphs will explain them and give
+several examples.
 
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix.  The name should be changed to lowercase as appropriate and
-dashes should remain in place.
+Rust programs are treated like any other package and named using the lowercase
+upstream name.  When using the Cargo build system (@pxref{Build Systems,
+@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
+argument set to @code{#f}, as this argument only makes sense for dependencies.
+When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
+be set to enable relevant support.
+
+Rust dependencies are managed in two modules:
+
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, @code{crate}, @code{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored sources unbundled before
+being contributed to Guix.  This module is managed by the Rust team
+(@pxref{Teams}).
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages.  This includes Rust dependencies requiring external inputs
+to unbundle and Cargo workspaces.
+
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}.  For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix.  The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
 
 In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix.  The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course).  This follows the ``caret'' version scheme
-intended by Cargo.  Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments.  It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}.  Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}.  If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible.  Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+package to be used at any given time, so all dependencies should have a
+versioned suffix.  The versioned suffix is the left-most non-zero digit (and any
+leading zeros, of course).  This follows the ``caret'' version scheme intended
+by Cargo.  Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we are usually packaging development snapshots of Rust dependencies
+specifically for some Rust programs, and can't simply identity them by version.
+In this case we can use a @code{for-@var{program}} suffix, for example,
+@code{rust-pipewire-for-niri} and @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Rust dependencies are not referenced directly.  @code{(guix build-sytem cargo)}
+provides a @code{cargo-inputs} procedure to create an input list, in combination
+with the lockfile importer.
+
+@cindex cargo-inputs
+@deffn {Procedure} cargo-inputs @var{name} @
+       [#:crates-module '(gnu packages rust-crates)] @
+       [#:sources-module '(gnu packages rust-sources)]
+Given symbol @code{'@var{name}}, resolve variable
+@code{@var{name}-cargo-inputs}, an input list, in @var{crates-module}, return
+its copy with @code{#f} removed and symbols resolved to variables defined in
+@var{sources-module} if the input list exists, otherwise return an empty list.
+
+@var{name} must be consistent with the one used in lockfile importer invocation.
+@end deffn
+
+For a more detailed packaging workflow, @pxref{Packaging Rust Crates,,,
+guix-cookbook, GNU Guix Cookbook}.

 @node Elm Packages
diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index d9b98a2ab3..1f0e5eee54 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -103,6 +103,7 @@ Top
 Packaging
 
 * Packaging Tutorial::          A tutorial on how to add packages to Guix.
+* Packaging Workflows::         Real life examples on working with specific build systems.
 
 Packaging Tutorial
 
@@ -127,6 +128,16 @@ Top
 * Automatic update::
 * Inheritance::
 
+Packaging Workflows
+
+* Packaging Rust Crates::
+
+Packaging Rust Crates
+
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+
 System Configuration
 
 * Auto-Login to a Specific TTY::  Automatically Login a User to a Specific TTY
@@ -514,6 +525,7 @@ Packaging
 
 @menu
 * Packaging Tutorial::          A tutorial on how to add packages to Guix.
+* Packaging Workflows::         Real life examples on working with specific build systems.
 @end menu
 
 @node Packaging Tutorial
@@ -1593,6 +1605,346 @@ References
 @uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
 @end itemize
 
+@node Packaging Workflows
+@section Packaging Workflows
+
+The following sections are real life examples on working with specific build
+systems, serving as extensions to the concise packaging guidelines
+(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
+
+@menu
+* Packaging Rust Crates::
+@end menu
+
+@node Packaging Rust Crates
+@subsection Packaging Rust Crates
+
+In preparation, add the following packages to our environment:
+
+@example
+$ guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+@menu
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+@end menu
+
+@node Common Workflow for Packaging Rust Crates
+@subsubsection Common Workflow for Packaging Rust Crates
+
+In this example, we'll package @code{cargo-audit}, which is published on the
+@uref{https://crates.io, crates.io} Rust package repository.  All its
+dependencies are on crates.io as well.
+
+@enumerate
+@item
+Since @code{cargo-audit} is available on crates.io, We can generate a draft
+definition via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU
+Guix Reference Manual}).  In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+  (package
+    (name "cargo-audit")
+    (version "0.21.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (crate-uri "cargo-audit" version))
+       (file-name (string-append name "-" version ".tar.gz"))
+       (sha256
+        (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+    (build-system cargo-build-system)
+    (arguments (list #:install-source? #f))
+    (inputs (cargo-inputs 'cargo-audit))
+    (home-page "https://rustsec.org/")
+    (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+    (description
+     "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+    (license (list license:asl2.0 license:expat))))
+@end lisp
+
+The symbol used in @code{cargo-inputs}, @code{'cargo-audit} here, must be a
+unique identifier, usually matching the variable name of the package.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+$ cargo generate-lockfile
+$ cargo audit
+$ cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining
+Packages,,, guix, GNU Guix Reference Manual}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+$ guix import --insert=gnu/packages/rust-crates.scm \
+      crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, package @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources.  The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them via the Guix command-line interface through expression:
+
+@example
+$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+  (crate-source "bzip2-sys" "0.1.13+1.0.8"
+                "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+                #:snippet
+                '(begin
+                   (delete-file-recursively "bzip2-1.0.8")
+                   (delete-file "build.rs")
+                   ;; Inspired by Debian's patch.
+                   (with-output-to-file "build.rs"
+                     (lambda _
+                       (format #t "fn main() @{~@@
+                        println!(\"cargo:rustc-link-lib=bz2\");~@@
+                        @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+reference it by a symbol in the imported definition.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to a matching symbol.
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+@end enumerate
+
+@node Development Snapshots and Cargo Workspaces
+@subsubsection Development Snapshots and Cargo Workspaces
+
+In this example, we'll package @code{niri}, which depends on development
+snapshots (also Cargo workspaces here).
+
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+to a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in a build phase, with a package-specific substitution
+pattern.
+
+@lisp
+(define-public niri
+  (package
+   (name "niri")
+   (version "25.02")
+   (source (origin
+             (method git-fetch)
+             (uri (git-reference
+                   (url "https://github.com/YaLTeR/niri")
+                   (commit (string-append "v" version))))
+             (file-name (git-file-name name version))
+             (sha256
+              (base32
+               "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+   (build-system cargo-build-system)
+   (arguments
+    (list #:install-source? #f
+          #:phases
+          #~(modify-phases %standard-phases
+              (add-after 'unpack 'use-guix-vendored-dependencies
+                (lambda _
+                  (substitute* "Cargo.toml"
+                    (("# version =.*")
+                     "version = \"*\"")
+                    (("git.*optional")
+                     "version = \"*\", optional")
+                    (("^git = .*")
+                     "")))))))
+   (native-inputs
+    (list pkg-config))
+   (inputs
+    (cons* clang
+           libdisplay-info
+           libinput-minimal
+           libseat
+           libxkbcommon
+           mesa
+           pango
+           pipewire
+           wayland
+           (cargo-inputs 'niri)))
+   (home-page "https://github.com/YaLTeR/niri")
+   (synopsis "Scrollable-tiling Wayland compositor")
+   (description
+    "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format.  It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+   (license license:gpl3+)))
+@end lisp
+
+@code{niri} has Cargo workspace dependencies.  When packaging a Cargo workspace,
+argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+  (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+        (revision "0"))
+    (package
+      (name "rust-pipewire")
+      (version (git-version "0.8.0" revision commit))
+      (source
+       (origin
+         (method git-fetch)
+         (uri (git-reference
+               (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+               (commit commit)))
+         (file-name (git-file-name name version))
+         (sha256
+          (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+      (build-system cargo-build-system)
+      (arguments
+       (list #:skip-build? #t
+             #:cargo-package-crates
+             ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+      (inputs (cargo-inputs 'rust-pipewire-for-niri))
+      (home-page "https://pipewire.org/")
+      (synopsis "Rust bindings for PipeWire")
+      (description "This package provides Rust bindings for PipeWire.")
+      (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+
+@node Rust Programs With Mixed Build Systems
+@subsubsection Rust Programs With Mixed Build Systems
+
+In this example, we'll package @code{fish}, which combines two build systems.
+
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+  (package
+    (name "fish")
+    (version "4.0.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/fish-shell/fish-shell/"
+                           "releases/download/" version "/"
+                           "fish-" version ".tar.xz"))
+       (sha256
+        (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+    (build-system cmake-build-system)
+    (inputs
+     (cons* fish-foreign-env ncurses pcre2
+            python                    ;for fish_config and manpage completions
+            (cargo-inputs 'fish)))
+    (native-inputs
+     (list doxygen groff                ;for 'fish --help'
+           pkg-config
+           procps                       ;for the test suite
+           rust
+           `(,rust "cargo")))
+    (arguments
+     (list #:imported-modules
+           (append %cargo-build-system-modules
+                   %cmake-build-system-modules)
+           #:modules
+           (((guix build cargo-build-system) #:prefix cargo:)
+            (guix build cmake-build-system)
+            (guix build utils))
+           #:phases
+           #~(modify-phases %standard-phases
+               (add-after 'unpack 'use-guix-vendored-dependencies
+                 (lambda _
+                   (substitute* "Cargo.toml"
+                     (("git.*tag.*,")
+                      "version = \"*\","))))
+               (add-after 'unpack 'prepare-cargo-build-system
+                 (lambda args
+                   (for-each
+                    (lambda (phase)
+                      (format #t "Running cargo phase: ~a~%" phase)
+                      (apply (assoc-ref cargo:%standard-phases phase)
+                             args))
+                    '(unpack-rust-crates
+                      configure
+                      check-for-pregenerated-files
+                      patch-cargo-checksums)))))))
+    (synopsis "The friendly interactive shell")
+    (description
+     "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness.  Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands.  It also
+has extensive and discoverable help.  A special @@command@{help@} command gives
+access to all the fish documentation in your web browser.  Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+    (home-page "https://fishshell.com/")
+    (license license:gpl2)))
+@end lisp
+
+
 @c *********************************************************************
 @node System Configuration
 @chapter System Configuration
-- 
2.49.0