Package: guix-patches;
Reported by: Hilton Chain <hako <at> ultrarare.space>
Date: Tue, 6 May 2025 12:50:01 UTC
Severity: normal
Tags: patch
Done: Andreas Enge <andreas <at> enge.fr>
Bug is archived. No further changes may be made.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: guix-patches <at> gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 6 May 2025 20:47:38 +0800
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
* doc/guix-cookbook.texi (Packaging)[Packaging Workflow]: New section.
Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
doc/contributing.texi | 97 +++++++----
doc/guix-cookbook.texi | 368 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 437 insertions(+), 28 deletions(-)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..148b4b0f71 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,75 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
-
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix. The name should be changed to lowercase as appropriate and
-dashes should remain in place.
-
-In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix. The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course). This follows the ``caret'' version scheme
-intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments. It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}. If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible. Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+Rust applications (binary crates) and libraries (library crates) are packaged
+separately. We put our main efforts into applications and only package
+libraries as sources, utilizing automation with a manual focus on unbundling
+vendored dependencies.
+
+Rust applications are treated like any other package and named using the
+lowercase upstream name. When using the Cargo build system (@pxref{Build
+Systems, @code{cargo-build-system}}), Rust applications should have the
+@code{#:install-source?} parameter set to @code{#f}, as this parameter only
+makes sense for libraries. When the package source is a Cargo workspace,
+@code{#:cargo-install-paths} must be set to enable relevant support.
+
+Rust libraries are hidden from the user interface and managed in two modules:
+
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+packages' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, @code{crate}, @option{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored dependencies unbundled
+before being contributed to Guix. This module is managed by the Rust team
+(@pxref{Teams}).
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages. This includes Rust libraries requiring external inputs to
+unbundle and Cargo workspaces.
+
+These libraries should have the @code{#:skip-build?} parameter set to @code{#t}.
+For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix. The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
+
+In the Rust community it is common for multiple incompatible versions of a
+package to be used at any given time, so all libraries should have a versioned
+suffix. The versioned suffix is the left-most non-zero digit (and any leading
+zeros, of course). This follows the ``caret'' version scheme intended by Cargo.
+Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we usually package development snapshots of Rust libraries
+specifically for some Rust applications, and can't simply identity them by
+version. In this case we can use a @code{for-@var{program}} suffix, for
+example, @code{rust-pipewire-for-niri} and @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Rust libraries are not referenced directly. @code{(guix build-sytem cargo)}
+provides a @code{cargo-inputs} procedure to lookup input lists generated by the
+lockfile importer.
+
+@cindex cargo inputs
+@findex cargo-inputs
+@deffn {Procedure} cargo-inputs name [#:module '(gnu packages rust-crates)]
+Lookup Cargo inputs for @var{name} in @var{module}, return an empty list if
+unavailable.
+
+@var{name} must be consistent with the one used in lockfile importer invocation.
+
+@var{module} must provide a public interface @code{lookup-cargo-inputs}, a
+template is available in the @file{etc/teams/rust} directory of Guix source
+tree.
+@end deffn
+
+@xref{Packaging Rust Crates,,, guix-cookbook, GNU Guix Cookbook}, for a more
+detailed packaging workflow.
@node Elm Packages
diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index fb58866d40..19379bf36f 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -105,6 +105,7 @@ Top
Packaging
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
Packaging Tutorial
@@ -129,6 +130,16 @@ Top
* Automatic update::
* Inheritance::
+Packaging Workflows
+
+* Packaging Rust Crates::
+
+Packaging Rust Crates
+
+* Common Workflow for Rust Packaging::
+* Cargo Workspaces and Development Snapshots::
+* Using Rust Libraries in Other Build Systems::
+
System Configuration
* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
@@ -518,6 +529,7 @@ Packaging
@menu
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
@end menu
@node Packaging Tutorial
@@ -1598,7 +1610,363 @@ References
@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
@end itemize
+@node Packaging Workflows
+@section Packaging Workflows
+
+The following sections are real life examples on working with specific build
+systems, serving as extensions to the concise packaging guidelines
+(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
+
+@menu
+* Packaging Rust Crates::
+@end menu
+
+@node Packaging Rust Crates
+@subsection Packaging Rust Crates
+
+In preparation, add the following packages to our environment:
+
+@example
+$ guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+@menu
+* Common Workflow for Rust Packaging::
+* Cargo Workspaces and Development Snapshots::
+* Using Rust Libraries in Other Build Systems::
+@end menu
+
+@node Common Workflow for Rust Packaging
+@subsubsection Common Workflow for Rust Packaging
+
+In this example, we'll package @code{cargo-audit}, which is published on the
+@uref{https://crates.io, crates.io} Rust package repository. All its
+dependencies are on crates.io as well.
+
+@enumerate
+@item
+Since @code{cargo-audit} is available on crates.io, we can generate a template
+via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU Guix
+Reference Manual}). After manual editing, we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+The identifier used to invoke @code{cargo-inputs}, @code{'cargo-audit} here,
+must be unique, usually matching the variable name of the package.
+
+@item
+Unpack package source and navigate to the unpacked directory, then run the
+following commands:
+
+@example
+$ cargo generate-lockfile
+$ cargo audit
+$ cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions.
+Applying it to all Rust applications helps reduce a great number of Rust
+libraries we need to check later. Although sometimes libraries may fail to
+follow the @uref{https://semver.org/, semantic versioning} scheme, it's still
+acceptable.
+
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses, for all the dependencies. We must have an acceptable output of
+@command{cargo audit} and ensure all dependencies are licensed with our
+supported licenses (@pxref{Defining Packages,,, guix, GNU Guix Reference
+Manual}).
+
+@item
+Import dependencies from the generated lockfile:
+
+@example
+$ guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+
+# Or use short options, in this case the shell processes pathes before passing
+# them to Guix, allowing expansion of @code{~}, for example.
+$ guix import -i gnu/packages/rust-crates.scm \
+ crate -f /path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} here must be consistent with the identifier used for
+@code{cargo-inputs} invocation in the package definition.
+
+At this stage, package @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored dependencies. The lockfile importer inserts
+@code{TODO:} comments for libraries with high probability of bundled
+dependencies. @code{cargo-build-system} also performs additional check for
+binary files, which usually indicate bundling, in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_aclui.a
+./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_activeds.a
+./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_asycfilt.a
+./guix-vendor/rust-winapi-i686-pc-windows-gnu-0.4.0.tar.gz/lib/libwinapi_amsi.a
+@dots{}
+@end example
+
+Although Rust libraries are not publicly exported, we can still select them via
+the Guix command-line interface through expression:
+
+@example
+$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+To unbundle most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-curl-sys-0.4.80+curl-8.12.1
+ (crate-source "curl-sys" "0.4.80+curl-8.12.1"
+ "0d7ppx4kq77hc5nyff6jydmfabpgd0i3ppjvn8x0q833mhpdzxsm"
+ #:snippet '(delete-file-recursively "curl")))
+@end lisp
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires a build process
+that involves other packages, we should make a full package in @code{(gnu
+packages rust-sources)} first and reference it in the imported definition.
+
+For example, we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to reference it.
+
+@lisp
+(define rust-ring-0.17.14 rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+
+To facility various tasks in the common workflow, several scripts are provided
+in the @file{etc/teams/rust} directory of Guix source tree.
+@end enumerate
+
+@node Cargo Workspaces and Development Snapshots
+@subsubsection Cargo Workspaces and Development Snapshots
+
+In this example, we'll package @code{niri}, which depends on development
+snapshots (also Cargo workspaces here).
+
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+to a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in a build phase, with a package-specific substitution
+pattern.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3+)))
+@end lisp
+
+@code{niri} has Cargo workspace dependencies. When packaging a Cargo workspace,
+parameter @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 rust-pipewire-for-niri)
+@end lisp
+
+@node Using Rust Libraries in Other Build Systems
+@subsubsection Using Rust Libraries in Other Build Systems
+
+In this example, we'll package @code{libchewing}, which combines two build
+systems.
+
+When building Rust packages in other build systems, we need to add @code{rust},
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+For cross-compilation support, we'll also add @code{rust-sysroot} created with
+@code{(make-rust-sysroot (%current-target-system))} to @code{native-inputs}, and
+set @code{#:cargo-target} for @code{cargo-build-system}'s build phases.
+
+@lisp
+(define-public libchewing
+ (package
+ (name "libchewing")
+ (version "0.9.1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/chewing/libchewing")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "0gh64wvrk5pn0fhmpvj1j99d5g7f7697rk96zbkc8l72yjr819z5"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list #:imported-modules
+ (append %cmake-build-system-modules
+ %cargo-build-system-modules)
+ #:modules
+ '(((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ ;; For cross-compilation.
+ #:cargo-target #$(cargo-triplet)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (native-inputs
+ (append
+ (list rust `(,rust "cargo") )
+ ;; For cross-compilation.
+ (or (and=> (%current-target-system)
+ (compose list make-rust-sysroot))
+ '())))
+ (inputs
+ (cons* corrosion ncurses sqlite (cargo-inputs 'libchewing)))
+ (synopsis "Chinese phonetic input method")
+ (description "Chewing is an intelligent phonetic (Zhuyin/Bopomofo) input
+method, one of the most popular choices for Traditional Chinese users.")
+ (home-page "https://chewing.im/")
+ (license license:lgpl2.1+)))
+@end lisp
+
+
@c *********************************************************************
@node System Configuration
@chapter System Configuration
--
2.49.0
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.