Package: guix-patches;
Reported by: Hilton Chain <hako <at> ultrarare.space>
Date: Tue, 18 Mar 2025 07:18:02 UTC
Severity: normal
Tags: patch
To reply to this bug, email your comments to 77093 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:18:02 GMT) Full text and rfc822 format available.Hilton Chain <hako <at> ultrarare.space>:ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org.
(Tue, 18 Mar 2025 07:18:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: guix-patches <at> gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 00/18] New Rust packaging workflow based on lockfile importer. Date: Tue, 18 Mar 2025 15:16:05 +0800
Hi Guix!
This series:
* integrates cargo2guix (https://git.sr.ht/~look/cargo2guix) as a crate importer
option, ‘-f / --lockfile=’.
* adds Cargo workspace support to cargo-build-system, via two new arguments:
#:cargo-package-crates and #:cargo-install-paths.
* introduces a new Rust packaging workflow, as described in contributing.texi
change.
This change is compatible with existing cargo-build-system users.
Thanks
Efraim Flashner (2):
build/cargo: Don't try to unpack sanity-check.py.
build/cargo: Use system jemalloc.
Hilton Chain (16):
build/cargo: Pass ‘--offline’ to cargo.
build/cargo: Print out all non-empty binary files.
build-system: cargo: Support packaging Cargo workspace.
build-system: cargo: Support installing Cargo workspace.
build/cargo: Set default value of arguments for build phases.
build/cargo: Support non-workspace directory source inputs.
scripts: import: Document argument for ‘--insert’ option in help
message.
scripts: import: Add two newlines for ‘--insert’ option.
scripts: import: Support expressions defined by 'define.
scripts: import: Pass "--insert" to importers.
scripts: import: Skip existing definition for ‘--insert’ option.
import: crate: crate-name->package-name: Move to (guix build-system
cargo).
build-system: cargo: Add ‘cargo-inputs’.
import: crate: Add Cargo.lock parser.
import: crate: Add ‘--lockfile’ option.
doc: Document lockfile importer based Rust packaging workflow.
Makefile.am | 1 +
doc/contributing.texi | 415 ++++++++++++++++++++++++++++--
doc/guix.texi | 25 ++
etc/teams/rust/cleanup-crates.sh | 38 +++
etc/teams/rust/rust-crates.tmpl | 42 +++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 42 +++
gnu/packages/rust-sources.scm | 29 +++
guix/build-system/cargo.scm | 57 +++-
guix/build/cargo-build-system.scm | 163 +++++++++---
guix/import/crate.scm | 81 +++++-
guix/import/crate/cargo-lock.scm | 105 ++++++++
guix/scripts/import.scm | 54 ++--
guix/scripts/import/crate.scm | 58 ++++-
guix/utils.scm | 27 +-
tests/crate.scm | 88 +++++++
16 files changed, 1121 insertions(+), 106 deletions(-)
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
create mode 100644 guix/import/crate/cargo-lock.scm
base-commit: b54a9ca849f013300c633fb79d80bc754f6b28a2
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:26:04 GMT) Full text and rfc822 format available.Message #8 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 01/18] build/cargo: Pass ‘--offline’ to cargo. Date: Tue, 18 Mar 2025 15:24:14 +0800
This will make error messages more helpful.
* guix/build/cargo-build-system.scm (build,check,package,install): Pass
‘--offline’ to cargo.
Change-Id: Ic95f603b793319f99c9c1fbce43f773bfc8126c0
---
guix/build/cargo-build-system.scm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 5ef6b23abd..41f54f42b7 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -246,7 +246,7 @@ (define* (build #:key
"Build a given Cargo package."
(or skip-build?
(apply invoke
- `("cargo" "build"
+ `("cargo" "build" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -264,7 +264,7 @@ (define* (check #:key
"Run tests for a given Cargo package."
(when tests?
(apply invoke
- `("cargo" "test"
+ `("cargo" "test" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -303,7 +303,7 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" ,@cargo-package-flags))
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
@@ -356,7 +356,8 @@ (define* (install #:key
;; otherwise cargo will raise an error.
(or skip-build?
(not (has-executable-target?))
- (invoke "cargo" "install" "--no-track" "--path" "." "--root" out
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" "." "--root" out
"--features" (string-join features)))
(when install-source?
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:26:06 GMT) Full text and rfc822 format available.Message #11 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 02/18] build/cargo: Print out all non-empty binary files. Date: Tue, 18 Mar 2025 15:24:15 +0800
* guix/build/cargo-build-system.scm (%standard-phases): Move
'unpack-rust-crates after 'unpack.
Move 'check-for-pregenerated-files after 'configure.
(check-for-pregenerated-files): Only check non-empty files.
Print out binary files.
Run in parallel.
Don't fail to keep compatibility for phase order change.
Change-Id: I0a332fe843e97687324bd908fa111422a63e475d
---
guix/build/cargo-build-system.scm | 31 +++++++++++++++++++++++++------
1 file changed, 25 insertions(+), 6 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41f54f42b7..58238992b8 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -29,9 +29,11 @@ (define-module (guix build cargo-build-system)
#:use-module ((guix build utils) #:hide (delete))
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (%standard-phases
@@ -111,12 +113,29 @@ (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
(define (rust-package? name)
(string-prefix? "rust-" name))
-(define* (check-for-pregenerated-files #:rest _)
+(define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
"Check the source code for files which are known to generally be bundled
libraries or executables."
- (let ((pregenerated-files (find-files "." "\\.(a|dll|dylib|exe|lib)$")))
- (when (not (null-list? pregenerated-files))
- (error "Possible pre-generated files found:" pregenerated-files))))
+ (format #t "Searching for binary files...~%")
+ (let ((known-pattern (make-regexp "\\.(a|dll|dylib|exe|lib)$"))
+ (empty-file?
+ (lambda (filename)
+ (call-with-ascii-input-file filename
+ (lambda (p)
+ (eqv? #\0 (read-char p)))))))
+ (n-par-for-each
+ (if parallel-build?
+ (parallel-job-count)
+ 1)
+ (lambda (file)
+ (unless (empty-file? file)
+ ;; Print out binary files.
+ (false-if-exception (invoke "grep" "-IL" "." file))
+ ;; Warn about known pre-generated files.
+ ;; Not failing here for compatibility with existing packages.
+ (when (regexp-exec known-pattern file)
+ (format #t "error: Possible pre-generated file found: ~a~%" file))))
+ (find-files "."))))
(define* (configure #:key inputs
target system
@@ -380,8 +399,8 @@ (define %standard-phases
(replace 'check check)
(replace 'install install)
(add-after 'build 'package package)
- (add-after 'unpack 'check-for-pregenerated-files check-for-pregenerated-files)
- (add-after 'check-for-pregenerated-files 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'unpack 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'configure 'check-for-pregenerated-files check-for-pregenerated-files)
(add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
(define* (cargo-build #:key inputs (phases %standard-phases)
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:26:08 GMT) Full text and rfc822 format available.Message #14 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 03/18] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 18 Mar 2025 15:24:16 +0800
* guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-package-crates]: New argument.
* guix/build/cargo-build-system.scm (package): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
---
doc/guix.texi | 7 +++++++
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 28 ++++++++++++++++++++++++++--
3 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0ca109a214..322512fd36 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9363,6 +9363,13 @@ Build Systems
the binaries defined by the crate. Unless @code{install-source? #f} is
defined it will also install a source crate repository of itself and unpacked
sources, to ease in future hacking on rust packages.
+
+This build system supports cargo workspaces. Parameter
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
+names of library crates to package in @code{package} phase. Specified
+crates are packaged from left to right, in case there's dependency among
+them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
+package @code{"pcre2-sys"} first and then @code{"pcre2"}.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 452f7f78d0..4f6d46e70c 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -94,6 +94,7 @@ (define* (cargo-build name inputs
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(features ''())
(skip-build? #f)
@@ -122,6 +123,7 @@ (define* (cargo-build name inputs
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
@@ -154,6 +156,7 @@ (define* (cargo-cross-build name
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(cargo-target (cargo-triplet (or target system)))
(features ''())
@@ -185,6 +188,7 @@ (define* (cargo-cross-build name
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 58238992b8..001e9c7adc 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -300,11 +300,16 @@ (define* (package #:key
source
skip-build?
install-source?
+ (cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
+ (vendor-dir "guix-vendor")
#:allow-other-keys)
"Run 'cargo-package' for a given Cargo package."
(if install-source?
- (if skip-build?
+ ;; NOTE: Cargo workspace packaging support:
+ ;; #:install-source? #t + #:cargo-package-crates.
+ (if (and (null? cargo-package-crates)
+ skip-build?)
(begin
(install-file source "target/package")
(with-directory-excursion "target/package"
@@ -322,7 +327,26 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+
+ (if (null? cargo-package-crates)
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+ ;; Use unstable feature ‘--registry’.
+ (begin
+ (setenv "RUSTC_BOOTSTRAP" "1")
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ "--registry" "crates-io" "-Z" "package-workspace"
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (begin
+ (delete-file-recursively "target/package/tmp-registry")
+ (find-files "target/package" "\\.crate$")))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates)
+ (unsetenv "RUSTC_BOOTSTRAP")))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:26:09 GMT) Full text and rfc822 format available.Message #17 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 04/18] build-system: cargo: Support installing Cargo workspace. Date: Tue, 18 Mar 2025 15:24:17 +0800
*guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-install-paths]: New argument.
* guix/build/cargo-build-system.scm (install): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
---
doc/guix.texi | 5 ++++-
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 17 +++++++++++++----
3 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 322512fd36..c20e1d7f9c 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9369,7 +9369,10 @@ Build Systems
names of library crates to package in @code{package} phase. Specified
crates are packaged from left to right, in case there's dependency among
them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
-package @code{"pcre2-sys"} first and then @code{"pcre2"}.
+package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
+paths of binary crates to install in @code{install} phase,
+@code{''("crates/atuin")}, for example.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4f6d46e70c..4486c706a1 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -96,6 +96,7 @@ (define* (cargo-build name inputs
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(features ''())
(skip-build? #f)
(parallel-build? #t)
@@ -125,6 +126,7 @@ (define* (cargo-build name inputs
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
@@ -158,6 +160,7 @@ (define* (cargo-cross-build name
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(cargo-target (cargo-triplet (or target system)))
(features ''())
(skip-build? #f)
@@ -190,6 +193,7 @@ (define* (cargo-cross-build name
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 001e9c7adc..bcee10837e 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -384,6 +384,7 @@ (define* (install #:key
skip-build?
install-source?
features
+ (cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
(let* ((out (assoc-ref outputs "out"))
@@ -398,10 +399,18 @@ (define* (install #:key
;; Only install crates which include binary targets,
;; otherwise cargo will raise an error.
(or skip-build?
- (not (has-executable-target?))
- (invoke "cargo" "install" "--offline" "--no-track"
- "--path" "." "--root" out
- "--features" (string-join features)))
+ ;; NOTE: Cargo workspace installation support:
+ ;; #:skip-build? #f + #:cargo-install-paths.
+ (and (null? cargo-install-paths)
+ (not (has-executable-target?)))
+ (for-each
+ (lambda (path)
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" path "--root" out
+ "--features" (string-join features)))
+ (if (null? cargo-install-paths)
+ '(".")
+ cargo-install-paths)))
(when install-source?
;; Install crate tarballs and unpacked sources for later use.
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:04 GMT) Full text and rfc822 format available.Message #20 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 05/18] build/cargo: Set default value of arguments for build phases. Date: Tue, 18 Mar 2025 15:24:18 +0800
This makes it easier to use these phases in other bulid systems.
* guix/build/cargo-build-system.scm (unpack-rust-crates,configure,build,package)
(install): Set default value of arguments.
Change-Id: I1dde1b063d8eee57967903abd2fce94574211a0a
---
guix/build/cargo-build-system.scm | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index bcee10837e..fdc9ae89c8 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -76,7 +76,8 @@ (define (crate-src? path)
" | cut -d/ -f2"
" | grep -q '^Cargo.toml$'")))))
-(define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
+(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
+ #:allow-other-keys)
(define (inputs->rust-inputs inputs)
"Filter using the label part from INPUTS."
(filter (lambda (input)
@@ -139,7 +140,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(define* (configure #:key inputs
target system
- cargo-target
+ (cargo-target #f)
(vendor-dir "guix-vendor")
#:allow-other-keys)
"Vendor Cargo.toml dependencies as guix inputs."
@@ -258,7 +259,7 @@ (define* (patch-cargo-checksums #:key
(define* (build #:key
parallel-build?
- skip-build?
+ (skip-build? #f)
(features '())
(cargo-build-flags '("--release"))
#:allow-other-keys)
@@ -298,8 +299,8 @@ (define* (check #:key
(define* (package #:key
source
- skip-build?
- install-source?
+ (skip-build? #f)
+ (install-source? #t)
(cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
(vendor-dir "guix-vendor")
@@ -381,9 +382,9 @@ (define* (package #:key
(define* (install #:key
inputs
outputs
- skip-build?
- install-source?
- features
+ (skip-build? #f)
+ (install-source? #t)
+ (features '())
(cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:06 GMT) Full text and rfc822 format available.Message #23 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team 06/18] build/cargo: Don't try to unpack sanity-check.py. Date: Tue, 18 Mar 2025 15:24:19 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (crate-src?): Also don't try to
check python scripts for Cargo.toml.
Change-Id: I001a89b83d2e472706b1263007be45d1153c140f
---
guix/build/cargo-build-system.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index fdc9ae89c8..ea00ff3fbe 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -64,6 +64,7 @@ (define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
(and (not (directory-exists? path)) ; not a tarball
+ (not (string-suffix? "py" path)) ; sanity-check.py
;; First we print out all file names within the tarball to see if it
;; looks like the source of a crate. However, the tarball will include
;; an extra path component which we would like to ignore (since we're
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:07 GMT) Full text and rfc822 format available.Message #26 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 07/18] build/cargo: Support non-workspace directory source inputs. Date: Tue, 18 Mar 2025 15:24:20 +0800
* guix/build/cargo-build-system.scm (single-crate?): New procedure.
(crate-src?): Support non-workspace directory source inputs.
(configure): Likewise.
Change-Id: If2bd318b44ed765baec69309f25d320edceee116
---
guix/build/cargo-build-system.scm | 60 ++++++++++++++++++++-----------
1 file changed, 39 insertions(+), 21 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index ea00ff3fbe..a1c4f556bb 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -33,6 +33,7 @@ (define-module (guix build cargo-build-system)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 textual-ports)
#:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -60,22 +61,35 @@ (define (has-executable-target?)
(bin-dep? (lambda (dep) (find bin? (get-kinds dep)))))
(find bin-dep? (manifest-targets))))
+(define (single-crate? dir)
+ "Check if directory DIR contains 'Cargo.toml' and is not a workspace."
+ (let ((manifest-file (in-vicinity dir "Cargo.toml")))
+ (and (file-exists? manifest-file)
+ (not (string-contains
+ (call-with-input-file manifest-file get-string-all)
+ "[workspace]")))))
+
(define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
- (and (not (directory-exists? path)) ; not a tarball
- (not (string-suffix? "py" path)) ; sanity-check.py
- ;; First we print out all file names within the tarball to see if it
- ;; looks like the source of a crate. However, the tarball will include
- ;; an extra path component which we would like to ignore (since we're
- ;; interested in checking if a Cargo.toml exists at the root of the
- ;; archive, but not nested anywhere else). We do this by cutting up
- ;; each output line and only looking at the second component. We then
- ;; check if it matches Cargo.toml exactly and short circuit if it does.
- (apply invoke (list "sh" "-c"
- (string-append "tar -tf " path
- " | cut -d/ -f2"
- " | grep -q '^Cargo.toml$'")))))
+ (if (directory-exists? path)
+ ;; The build system only handles sources containing single crate.
+ ;; Workspaces should be packaged into crates (via 'package phase)
+ ;; and used in inputs.
+ (single-crate? path)
+ (and (not (string-suffix? "py" path)) ;sanity-check.py
+ ;; First we print out all file names within the tarball to see
+ ;; if it looks like the source of a crate. However, the tarball
+ ;; will include an extra path component which we would like to
+ ;; ignore (since we're interested in checking if a Cargo.toml
+ ;; exists at the root of the archive, but not nested anywhere
+ ;; else). We do this by cutting up each output line and only
+ ;; looking at the second component. We then check if it matches
+ ;; Cargo.toml exactly and short circuit if it does.
+ (invoke "sh" "-c"
+ (string-append "tar -tf " path
+ " | cut -d/ -f2"
+ " | grep -q '^Cargo.toml$'")))))
(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
#:allow-other-keys)
@@ -159,14 +173,18 @@ (define* (configure #:key inputs
(and (crate-src? path)
;; Gracefully handle duplicate inputs
(not (file-exists? crate-dir))
- (mkdir-p crate-dir)
- ;; Cargo crates are simply gzipped tarballs but with a .crate
- ;; extension. We expand the source to a directory name we control
- ;; so that we can generate any cargo checksums.
- ;; The --strip-components argument is needed to prevent creating
- ;; an extra directory within `crate-dir`.
- (format #t "Unpacking ~a~%" name)
- (invoke "tar" "xf" path "-C" crate-dir "--strip-components" "1")))))
+ (if (directory-exists? path)
+ (copy-recursively path crate-dir)
+ (begin
+ (mkdir-p crate-dir)
+ ;; Cargo crates are simply gzipped tarballs but with a
+ ;; .crate extension. We expand the source to a directory
+ ;; name we control so that we can generate any cargo
+ ;; checksums. The --strip-components argument is needed to
+ ;; prevent creating an extra directory within `crate-dir`.
+ (format #t "Unpacking ~a~%" name)
+ (invoke "tar" "xf" path "-C" crate-dir
+ "--strip-components" "1")))))))
inputs)
;; For cross-building
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:07 GMT) Full text and rfc822 format available.Message #29 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team 08/18] build/cargo: Use system jemalloc. Date: Tue, 18 Mar 2025 15:24:21 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (configure): Set environment variables to
use system jemalloc.
Change-Id: Ibb1c025c7354ba2befd2770e3f1ccd785baa06c3
Modified-by: Hilton Chain <hako <at> ultrarare.space>
---
guix/build/cargo-build-system.scm | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index a1c4f556bb..b547421b88 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -248,6 +248,12 @@ (define* (configure #:key inputs
(setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
(setenv "SODIUM_USE_PKG_CONFIG" "1")
(setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
+ ;; This flag is needed when not using the bundled jemalloc.
+ ;; https://github.com/tikv/jemallocator/issues/19
+ (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
+ (when (assoc-ref inputs "jemalloc")
+ (setenv "JEMALLOC_OVERRIDE"
+ (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so")))
(when (assoc-ref inputs "openssl")
(setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
(when (assoc-ref inputs "gettext")
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:09 GMT) Full text and rfc822 format available.Message #32 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 09/18] scripts: import: Document argument for ‘--insert’ option in help message. Date: Tue, 18 Mar 2025 15:24:22 +0800
* guix/scripts/import.scm (show-help): Add missing ‘FILE’. Change-Id: I540d5feae3fe49c00e9bd6f7a8649ffe0d6e006d --- guix/scripts/import.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index bbf31baa15..4fad329cd3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -69,7 +69,7 @@ (define (show-help) (display (G_ " -h, --help display this help and exit")) (display (G_ " - -i, --insert insert packages into file alphabetically")) + -i, --insert=FILE insert packages into FILE alphabetically")) (display (G_ " -V, --version display version information and exit")) (newline) -- 2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:10 GMT) Full text and rfc822 format available.Message #35 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 11/18] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 15:24:24 +0800
* guix/utils.scm (find-definition-location): New procedure.
(find-definition-insertion-location): Define with it.
* guix/scripts/import.scm (import-as-definitions, guix-import): Support
expressions defined by 'define.
Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
---
guix/scripts/import.scm | 20 +++++++++++++++-----
guix/utils.scm | 27 +++++++++++++++++++--------
2 files changed, 34 insertions(+), 13 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 58a84d0db7..aaa3d26673 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -30,6 +30,7 @@ (define-module (guix scripts import)
#:use-module (guix read-print)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))
((expressions ...)
(for-each (lambda (expr)
@@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))))
expressions))
(x
@@ -117,13 +120,20 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let ((find-and-insert
+ (let* ((definer?
+ (cut member
+ <>
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public)))
+ (find-and-insert
(lambda (expr)
(match expr
- (('define-public term _ ...)
+ (((? definer? definer) term _ ...)
(let ((source-properties
(find-definition-insertion-location
- file term)))
+ file term #:definer definer)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index c7c23d9d5b..77ec6d992a 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -154,6 +154,7 @@ (define-module (guix utils)
edit-expression
delete-expression
insert-expression
+ find-definition-location
find-definition-insertion-location
filtered-port
@@ -520,24 +521,34 @@ (define (insert-expression source-properties expr)
(string-append expr "\n\n" str))))
(edit-expression source-properties insert)))
-(define (find-definition-insertion-location file term)
- "Search in FILE for a top-level public definition whose defined term
-alphabetically succeeds TERM. Return the location if found, or #f
-otherwise."
- (let ((search-term (symbol->string term)))
+(define* (find-definition-location file term
+ #:key (definer 'define-public)
+ (pred string=))
+ "Search in FILE for a top-level definition defined by DEFINER with defined
+term comparing to TERM through PRED. Return the location if PRED returns #t,
+or #f otherwise."
+ (let ((search-term (symbol->string term))
+ (definer? (cut eq? definer <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (('define-public current-term _ ...)
- (string> (symbol->string current-term)
- search-term))
+ (((? definer?) current-term _ ...)
+ (pred (symbol->string current-term)
+ search-term))
((? eof-object?) #t)
(_ #f))
(and (not (eof-object? syntax))
(syntax-source syntax))))))))
+(define* (find-definition-insertion-location file term
+ #:key (definer 'define-public))
+ "Search in FILE for a top-level definition defined by DEFINER with defined
+term alphabetically succeeds TERM. Return the location if found, or #f
+otherwise."
+ (find-definition-location file term #:definer definer #:pred string>))
+
;;;
;;; Keyword arguments.
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:12 GMT) Full text and rfc822 format available.Message #38 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 10/18] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 18 Mar 2025 15:24:23 +0800
For consistency with ‘insert-expression’.
* guix/scripts/import.scm (guix-import): Add two newlines when inserting.
Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d
---
guix/scripts/import.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 4fad329cd3..58a84d0db7 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -129,6 +129,7 @@ (define-command (guix-import . args)
(let ((port (open-file file "a")))
(pretty-print-with-comments port expr)
(newline port)
+ (newline port)
(close-port port)))))))))
(import-as-definitions importer args find-and-insert)))
((importer args ...)
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:12 GMT) Full text and rfc822 format available.Message #41 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 12/18] scripts: import: Pass "--insert" to importers. Date: Tue, 18 Mar 2025 15:24:25 +0800
So that importers can adjust the file before inserting.
* guix/scripts/import.scm (%standard-import-options): Add ‘--file-to-insert’.
(guix-import): Pass it to importers when ‘--insert’ is set.
Change-Id: I8e7a18ee8e0f96d7fc5688a207a7a5390ad2fa30
---
guix/scripts/import.scm | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index aaa3d26673..1b2c35f3b1 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -31,6 +31,7 @@ (define-module (guix scripts import)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-37)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -41,7 +42,12 @@ (define-module (guix scripts import)
;;; Command line options.
;;;
-(define %standard-import-options '())
+(define %standard-import-options
+ (list
+ ;; Hidden option for importer-specific file preprocessing.
+ (option '("file-to-insert") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'file-to-insert arg result)))))
;;;
@@ -141,7 +147,10 @@ (define-command (guix-import . args)
(newline port)
(newline port)
(close-port port)))))))))
- (import-as-definitions importer args find-and-insert)))
+ (import-as-definitions importer
+ (cons (string-append "--file-to-insert=" file)
+ args)
+ find-and-insert)))
((importer args ...)
(let ((print (lambda (expr)
(leave-on-EPIPE
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:13 GMT) Full text and rfc822 format available.Message #44 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 13/18] scripts: import: Skip existing definition for ‘--insert’ option. Date: Tue, 18 Mar 2025 15:24:26 +0800
* guix/scripts/import.scm (guix-import): Skip existing definition for
‘--insert’ option.
Change-Id: I2c4242669f974b263a018ab0cf56538bd7c81d06
---
guix/scripts/import.scm | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 1b2c35f3b1..d8e1ed3e6f 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -137,16 +137,18 @@ (define-command (guix-import . args)
(lambda (expr)
(match expr
(((? definer? definer) term _ ...)
- (let ((source-properties
- (find-definition-insertion-location
+ ;; Skip existing definition.
+ (unless (find-definition-location file term #:definer definer)
+ (let ((source-properties
+ (find-definition-insertion-location
file term #:definer definer)))
- (if source-properties
- (insert-expression source-properties expr)
- (let ((port (open-file file "a")))
- (pretty-print-with-comments port expr)
- (newline port)
- (newline port)
- (close-port port)))))))))
+ (if source-properties
+ (insert-expression source-properties expr)
+ (let ((port (open-file file "a")))
+ (pretty-print-with-comments port expr)
+ (newline port)
+ (newline port)
+ (close-port port))))))))))
(import-as-definitions importer
(cons (string-append "--file-to-insert=" file)
args)
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:15 GMT) Full text and rfc822 format available.Message #47 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 14/18] import: crate: crate-name->package-name: Move to (guix build-system cargo). Date: Tue, 18 Mar 2025 15:24:27 +0800
* guix/import/crate.scm (crate-name->package-name): Move to...
* guix/build-system/cargo.scm (crate-name->package-name): ...here and export.
Change-Id: Ie9813179d6c40d159956cc1e2ae59a74dea0a42d
---
guix/build-system/cargo.scm | 6 +++++-
guix/import/crate.scm | 5 -----
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4486c706a1..7a07003262 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -44,7 +44,8 @@ (define-module (guix build-system cargo)
%crate-base-url
crate-url
crate-url?
- crate-uri))
+ crate-uri
+ crate-name->package-name))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -58,6 +59,9 @@ (define (crate-uri name version)
to NAME and VERSION."
(string-append crate-url name "/" version "/download"))
+(define (crate-name->package-name name)
+ (downstream-package-name "rust-" name))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 5a8caeb3e1..14e6e28c5b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -481,10 +481,6 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
-(define (crate-name->package-name name)
- (downstream-package-name "rust-" name))
-
-
;;;
;;; Updater
@@ -528,4 +524,3 @@ (define %crate-updater
(description "Updater for crates.io packages")
(pred crate-package?)
(import import-release)))
-
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:17 GMT) Full text and rfc822 format available.Message #50 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 15/18] build-system: cargo: Add ‘cargo-inputs’. Date: Tue, 18 Mar 2025 15:24:28 +0800
* gnu/packages/rust-crates.scm: New file.
* gnu/packages/rust-sources.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them.
* guix/build-system/cargo.scm (crate-source,cargo-inputs): New procedures.
* guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo).
* etc/teams/rust/cleanup-crates.sh: New file.
* etc/teams/rust/rust-crates.tmpl: New file.
Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3
---
etc/teams/rust/cleanup-crates.sh | 38 +++++++++++++++++++++++++++
etc/teams/rust/rust-crates.tmpl | 42 +++++++++++++++++++++++++++++
gnu/local.mk | 2 ++
gnu/packages/rust-crates.scm | 42 +++++++++++++++++++++++++++++
gnu/packages/rust-sources.scm | 29 ++++++++++++++++++++
guix/build-system/cargo.scm | 45 +++++++++++++++++++++++++++++++-
guix/import/crate.scm | 2 +-
7 files changed, 198 insertions(+), 2 deletions(-)
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
new file mode 100755
index 0000000000..cd4c2462b9
--- /dev/null
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+FILE=gnu/packages/rust-crates.scm
+PATTERN='^(define rust-'
+
+grep "$PATTERN" $FILE | cut -d' ' -f2 | while IFS= read -r crate
+do
+ if [ "$(grep -wc "$crate" $FILE)" -eq 1 ]; then
+ echo "\
+(begin
+ (use-modules (guix utils))
+ (let ((source-properties
+ (find-definition-location \"$FILE\" '$crate #:definer 'define)))
+ (and=> source-properties delete-expression)))" |
+ guix repl -t machine
+ fi
+done
+
+# Delete extra newlines.
+sed --in-place ':a;N;$!ba;s/\n\n\+/\n\n/g' $FILE
diff --git a/etc/teams/rust/rust-crates.tmpl b/etc/teams/rust/rust-crates.tmpl
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/etc/teams/rust/rust-crates.tmpl
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/local.mk b/gnu/local.mk
index 02de02e65f..8d8bd6ebbe 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -616,6 +616,8 @@ GNU_SYSTEM_MODULES = \
%D%/packages/rush.scm \
%D%/packages/rust.scm \
%D%/packages/rust-apps.scm \
+ %D%/packages/rust-crates.scm \
+ %D%/packages/rust-sources.scm \
%D%/packages/samba.scm \
%D%/packages/sagemath.scm \
%D%/packages/sawfish.scm \
diff --git a/gnu/packages/rust-crates.scm b/gnu/packages/rust-crates.scm
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/gnu/packages/rust-crates.scm
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/packages/rust-sources.scm b/gnu/packages/rust-sources.scm
new file mode 100644
index 0000000000..bf9b91a671
--- /dev/null
+++ b/gnu/packages/rust-sources.scm
@@ -0,0 +1,29 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-sources)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; Cargo workspaces and Rust dependencies requiring external inputs to
+;;; unbundle.
+;;;
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 7a07003262..47d4f10969 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -31,6 +31,7 @@ (define-module (guix build-system cargo)
#:use-module (guix gexp)
#:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix download)
#:use-module (guix platform)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
@@ -45,7 +46,9 @@ (define-module (guix build-system cargo)
crate-url
crate-url?
crate-uri
- crate-name->package-name))
+ crate-name->package-name
+ crate-source
+ cargo-inputs))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -62,6 +65,46 @@ (define (crate-uri name version)
(define (crate-name->package-name name)
(downstream-package-name "rust-" name))
+(define* (crate-source name version hash #:key (patches '()) (snippet #f))
+ (origin
+ (method url-fetch)
+ (uri (crate-uri name version))
+ (file-name
+ (string-append (crate-name->package-name name) "-" version ".tar.gz"))
+ (sha256 (base32 hash))
+ (modules '((guix build utils)))
+ (patches patches)
+ (snippet snippet)))
+
+(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
+ (sources-module '(gnu packages rust-sources)))
+ "Given NAME, resolve input list 'NAME-cargo-inputs' in CRATES-MODULE, return
+its copy with #f removed and symbols resolved to variables defined in
+SOURCES-MODULE, if the list exists, otherwise return an empty list."
+ (let loop ((inputs
+ (catch #t
+ (lambda ()
+ (module-ref (resolve-interface crates-module)
+ (symbol-append name '-cargo-inputs)))
+ (const '())))
+ (result '()))
+ (if (null? inputs)
+ result
+ (match inputs
+ ((input . rest)
+ (loop rest
+ (cond
+ ;; #f, remove it.
+ ((not input)
+ result)
+ ;; Symbol, resolve it in SOURCES-MODULE.
+ ((symbol? input)
+ (cons (module-ref (resolve-interface sources-module) input)
+ result))
+ ;; Else: no change.
+ (else
+ (cons input result)))))))))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 14e6e28c5b..a6f247bbae 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -27,7 +27,7 @@
(define-module (guix import crate)
#:use-module (guix base32)
- #:use-module (guix build-system cargo)
+ #:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:17 GMT) Full text and rfc822 format available.Message #53 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Murilo <murilo <at> disroot.org>, Hilton Chain <hako <at> ultrarare.space>, Luis Guilherme Coelho <lgcoelho <at> disroot.org> Subject: [PATCH rust-team 16/18] import: crate: Add Cargo.lock parser. Date: Tue, 18 Mar 2025 15:24:29 +0800
* guix/import/crate/cargo-lock.scm: New file.
* Makefile.am (MODULES): Regisiter it.
* guix/import/crate.scm (cargo-lock->definitions): New procedure.
Co-authored-by: Murilo <murilo <at> disroot.org>
Co-authored-by: Luis Guilherme Coelho <lgcoelho <at> disroot.org>
Change-Id: I95421e9e2ba11a671b4bc4e1323c6d31a1b012c5
---
Makefile.am | 1 +
guix/import/crate.scm | 78 +++++++++++++++++++++++
guix/import/crate/cargo-lock.scm | 105 +++++++++++++++++++++++++++++++
3 files changed, 184 insertions(+)
create mode 100644 guix/import/crate/cargo-lock.scm
diff --git a/Makefile.am b/Makefile.am
index 4091a52387..a04e461ea8 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -295,6 +295,7 @@ MODULES = \
guix/import/cpan.scm \
guix/import/cran.scm \
guix/import/crate.scm \
+ guix/import/crate/cargo-lock.scm \
guix/import/egg.scm \
guix/import/elm.scm \
guix/import/elpa.scm \
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index a6f247bbae..ef3a6d0b4b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -9,6 +9,9 @@
;;; Copyright © 2023, 2024 David Elsing <david.elsing <at> posteo.net>
;;; Copyright © 2024 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;; Copyright © 2025 Herman Rimm <herman <at> rimm.ee>
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024-2025 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -26,12 +29,14 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix import crate)
+ #:use-module (guix base16)
#:use-module (guix base32)
#:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
#:use-module (guix i18n)
+ #:use-module (guix import crate cargo-lock)
#:use-module (guix import json)
#:use-module (guix import utils)
#:use-module (guix memoization)
@@ -39,9 +44,11 @@ (define-module (guix import crate)
#:use-module (guix read-print)
#:use-module (guix upstream)
#:use-module (guix utils)
+ #:use-module (guix scripts download)
#:use-module (gnu packages)
#:use-module (ice-9 match)
#:use-module (ice-9 regex)
+ #:use-module (ice-9 textual-ports)
#:use-module (json)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
@@ -52,6 +59,7 @@ (define-module (guix import crate)
guix-package->crate-name
string->license
crate-recursive-import
+ cargo-lock->definitions
%crate-updater))
@@ -481,6 +489,76 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
+
+;;;
+;;; Convert ‘Cargo.lock’ to Guix sources.
+;;;
+
+(define (cargo-lock->definitions lockfile package-name)
+ "Given LOCKFILE, a 'Cargo.lock' file, convert its content into source
+definitions. Return a list of sources and 'PACKAGE-NAME-cargo-inputs', an input
+list referencing all imported sources."
+ (define (crate->definition crate)
+ (match crate
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source _)
+ ('crate-checksum checksum))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,@(if (or (string-suffix? "src" name)
+ (string-suffix? "sys" name))
+ (list (comment ";; TODO: Check bundled sources.\n" #f))
+ '())
+ (crate-source ,name ,version
+ ,(bytevector->nix-base32-string
+ (base16-string->bytevector checksum)))))
+ ;; Git snapshot.
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source source))
+ (begin
+ (let* ((src (string-split source (char-set #\+ #\? #\#)))
+ (url (second src))
+ (commit (last src))
+ (version (string-append version "." (string-take commit 7)))
+ (checksum
+ (second
+ (string-split
+ (with-output-to-string
+ (lambda _
+ (guix-download "-g" url
+ (string-append "--commit=" commit))))
+ #\newline))))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,(comment
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url ,url)
+ (commit ,commit)))
+ (file-name
+ (git-file-name ,(crate-name->package-name name) ,version))
+ (sha256 (base32 ,checksum)))))))
+ ;; Workspace member.
+ (else #f)))
+
+ (let ((definitions
+ (filter-map crate->definition
+ (cargo-lock-string->scm
+ (call-with-input-file lockfile get-string-all)))))
+ (values definitions
+ `(define-public
+ ,(string->symbol (string-append package-name "-cargo-inputs"))
+ (list ,@(map second definitions))))))
+
;;;
;;; Updater
diff --git a/guix/import/crate/cargo-lock.scm b/guix/import/crate/cargo-lock.scm
new file mode 100644
index 0000000000..b8a6ab20c0
--- /dev/null
+++ b/guix/import/crate/cargo-lock.scm
@@ -0,0 +1,105 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import crate cargo-lock)
+ #:use-module (ice-9 peg)
+ #:export (cargo-lock-string->scm
+
+ crate-name
+ crate-version
+ crate-source
+ crate-checksum
+ crate-dependencies
+ cargo-lock))
+
+;;;
+;;; PEG parser for ‘Cargo.lock’.
+;;;
+
+(define (cargo-lock-string->scm str)
+ (peg:tree (search-for-pattern cargo-lock str)))
+
+;; Auxiliar peg patterns
+(define-peg-pattern numeric-char body
+ (range #\0 #\9))
+
+(define-peg-pattern lowercase-char body
+ (range #\a #\z))
+
+(define-peg-pattern uppercase-char body
+ (range #\A #\Z))
+
+(define-peg-pattern alphabetic-char body
+ (or lowercase-char uppercase-char))
+
+(define-peg-pattern alphanumeric-char body
+ (or alphabetic-char numeric-char))
+
+;; name
+(define-peg-pattern crate-name all
+ (+ (or "-" alphabetic-char
+ "_" numeric-char)))
+
+;; version
+(define-peg-pattern non-negative-integer body
+ (+ numeric-char))
+
+(define-peg-pattern crate-version all
+ (and non-negative-integer "."
+ non-negative-integer "."
+ non-negative-integer
+ (? (+ (or "-" lowercase-char
+ "." uppercase-char
+ "+" numeric-char "_")))))
+
+;; source
+(define-peg-pattern crate-source all
+ (and (or "registry" "git")
+ "+https://"
+ (+ (or "/" "." "?" "=" "-" "#" "_"
+ alphanumeric-char))))
+
+;; checksum
+(define-peg-pattern crate-checksum all
+ (+ (or lowercase-char numeric-char)))
+
+;; dependency specification
+(define-peg-pattern dependency-specification all
+ (and crate-name (? (and (ignore " ") crate-version))))
+
+;; dependencies
+(define-peg-pattern crate-dependencies all
+ (and (ignore "[\n")
+ (+ (and (ignore " \"")
+ (capture dependency-specification)
+ (ignore "\",\n")))
+ (ignore "]")))
+
+;; crates
+(define-peg-pattern crate all
+ (and (ignore "[[package]]\n")
+ (ignore "name = \"") (capture crate-name) (ignore "\"\n")
+ (ignore "version = \"") (capture crate-version) (ignore "\"\n")
+ (? (and (ignore "source = \"") (capture crate-source) (ignore "\"\n")))
+ (? (and (ignore "checksum = \"") (capture crate-checksum) (ignore "\"\n")))
+ (? (ignore (and "dependencies = " crate-dependencies "\n")))))
+
+;; Cargo.lock
+(define-peg-pattern cargo-lock all
+ (+ (and (ignore "\n") crate)))
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, divya <at> subvertising.org, efraim <at> flashner.co.il, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, maxim.cournoyer <at> gmail.com, zimon.toutoune <at> gmail.com, steve <at> futurile.net, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:18 GMT) Full text and rfc822 format available.Message #56 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 17/18] import: crate: Add ‘--lockfile’ option. Date: Tue, 18 Mar 2025 15:24:30 +0800
* guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
* guix/scripts/import/crate.scm (show-help): Add it.
(guix-import-crate): Use it.
* doc/guix.texi (Invoking guix import): Document it.
* tests/crate.scm (temp-file): New variable.
("crate-lockfile-import"): New test.
Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
---
doc/guix.texi | 15 ++++++
guix/scripts/import/crate.scm | 58 +++++++++++++++++++----
tests/crate.scm | 88 +++++++++++++++++++++++++++++++++++
3 files changed, 151 insertions(+), 10 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index c20e1d7f9c..029c8cf59c 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14703,6 +14703,21 @@ Invoking guix import
If a crate dependency is not (yet) packaged, make the corresponding
input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
a comment.
+@item --lockfile=@var{file}
+@itemx -f @var{file}
+When @option{--lockfile} is specified, importer will ignore other
+options and won't output package definitions, instead generating source
+definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
+For example:
+
+@example
+guix import crate --lockfile=/path/to/Cargo.lock my-package
+@end example
+
+generates sources from @file{/path/to/Cargo.lock}, and a list
+@code{my-package-cargo-inputs} referencing these sources. The generated
+list is intended for package's base @code{inputs}, replacing
+@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
@end table
@item elm
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 723cbb3665..3b536e135a 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -25,11 +25,13 @@
(define-module (guix scripts import crate)
#:use-module (guix ui)
#:use-module (guix utils)
+ #:use-module (guix read-print)
#:use-module (guix scripts)
#:use-module (guix import crate)
#:use-module (guix scripts import)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-26)
#:use-module (srfi srfi-37)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
@@ -60,6 +62,9 @@ (define (show-help)
sufficient package exists for it"))
(newline)
(display (G_ "
+ -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
+ (newline)
+ (display (G_ "
-h, --help display this help and exit"))
(display (G_ "
-V, --version display version information and exit"))
@@ -87,6 +92,9 @@ (define %options
(option '("mark-missing") #f #f
(lambda (opt name arg result)
(alist-cons 'mark-missing #t result)))
+ (option '(#\f "lockfile") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'lockfile arg result)))
%standard-import-options))
@@ -101,6 +109,8 @@ (define (guix-import-crate . args)
#:build-options? #f))
(let* ((opts (parse-options))
+ (lockfile (assoc-ref opts 'lockfile))
+ (file-to-insert (assoc-ref opts 'file-to-insert))
(args (filter-map (match-lambda
(('argument . value)
value)
@@ -111,16 +121,44 @@ (define (guix-import-crate . args)
(define-values (name version)
(package-name->name+version spec))
- (match (if (assoc-ref opts 'recursive)
- (crate-recursive-import
- name #:version version
- #:recursive-dev-dependencies?
- (assoc-ref opts 'recursive-dev-dependencies)
- #:allow-yanked? (assoc-ref opts 'allow-yanked))
- (crate->guix-package
- name #:version version #:include-dev-deps? #t
- #:allow-yanked? (assoc-ref opts 'allow-yanked)
- #:mark-missing? (assoc-ref opts 'mark-missing)))
+ (match (cond
+ ((and=> lockfile
+ (lambda (file)
+ (or (file-exists? file)
+ (leave (G_ "file '~a' does not exist~%") file))))
+ (let-values (((crate-definitions input-list)
+ (cargo-lock->definitions lockfile name)))
+ (if file-to-insert
+ ;; Adjusted from ‘--insert’ option of guix-import.
+ (let ((term (second input-list)))
+ (begin
+ ;; Remove existing input list definition.
+ (and=> (find-definition-location file-to-insert term)
+ delete-expression)
+ ;; Insert input list alphabetically.
+ (or (and=> (find-definition-insertion-location
+ file-to-insert term)
+ (cut insert-expression <> input-list))
+ (let ((port (open-file file-to-insert "a")))
+ (newline port)
+ (pretty-print-with-comments port input-list)
+ (newline port)
+ (newline port)
+ (close-port port))))
+ crate-definitions)
+ `(,@crate-definitions
+ ,input-list))))
+ ((assoc-ref opts 'recursive)
+ (crate-recursive-import
+ name #:version version
+ #:recursive-dev-dependencies?
+ (assoc-ref opts 'recursive-dev-dependencies)
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)))
+ (else
+ (crate->guix-package
+ name #:version version #:include-dev-deps? #t
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)
+ #:mark-missing? (assoc-ref opts 'mark-missing))))
((or #f '())
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
diff --git a/tests/crate.scm b/tests/crate.scm
index d0dc779cd2..d46f753f9c 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -34,6 +34,7 @@ (define-module (test-crate)
#:use-module (gnu packages)
#:use-module (ice-9 iconv)
#:use-module (ice-9 match)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-64))
@@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
(description #f)
(license #f)))
+(define temp-file
+ (string-append "t-utils-" (number->string (getpid))))
+
(test-begin "crate")
@@ -1178,4 +1182,88 @@ (define rust-leaf-bob-3.0.2-yanked
(x
(pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
+
+(test-assert "crate-lockfile-import"
+ (begin
+ (call-with-output-file temp-file
+ (lambda (port)
+ (display "\
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = \"adler2\"
+version = \"2.0.0\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
+
+[[package]]
+name = \"aho-corasick\"
+version = \"1.1.3\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
+dependencies = [
+ \"memchr\",
+]
+
+[[package]]
+name = \"smithay\"
+version = \"0.4.0\"
+source = \"git+https://github.com/Smithay/smithay.git?rev=\
+0cd3345c59f7cb139521f267956a1a4e33248393#\
+0cd3345c59f7cb139521f267956a1a4e33248393\"
+dependencies = [
+ \"appendlist\",
+]
+
+[[package]]
+name = \"test\"
+version = \"25.2.0\"\n" port)))
+ (mock
+ ((guix scripts download) guix-download
+ (lambda _
+ (format #t "~a~%~a~%"
+ "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
+ (let-values
+ (((crates-definitions input-list)
+ (cargo-lock->definitions temp-file "test")))
+ (and
+ (match crates-definitions
+ (((define 'rust-adler2-2.0.0
+ (crate-source
+ "adler2" "2.0.0"
+ "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
+ (define 'rust-aho-corasick-1.1.3
+ (crate-source
+ "aho-corasick" "1.1.3"
+ "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
+ (define 'rust-smithay-0.4.0.0cd3345
+ ($ <comment>
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/Smithay/smithay.git")
+ (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
+ (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
+ (sha256
+ (base32
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
+ #t)
+ (x
+ (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
+ (match input-list
+ ((define-public 'test-cargo-inputs
+ (list rust-adler2-2.0.0
+ rust-aho-corasick-1.1.3
+ rust-smithay-0.4.0.0cd3345))
+ #t)
+ (x
+ (pk 'fail x #f))))))))
+
(test-end "crate")
+
+(false-if-exception (delete-file temp-file))
--
2.48.1
hako <at> ultrarare.space, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:18 GMT) Full text and rfc822 format available.Message #59 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 18/18] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 18 Mar 2025 15:24:31 +0800
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
1 file changed, 390 insertions(+), 25 deletions(-)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..e7c5116a3d 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,399 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
+As currently there's no achievable way to reuse Rust packages as pre-compiled
+inputs for other packages at a distribution scale, and our previous approach on
+making all Rust dependencies as full packages doesn't cope well with Rust
+ecosystem thus causing issues on both contribution and maintenance sides, we
+have switched to a different packaging model.
+
+Rust programs (binary crates) and dependencies (library crates) are treated
+separately. We put main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with manual focus on unbundling
+vendored dependencies. The following paragraphs will explain them and give
+several examples.
+
+Rust programs are treated like any other package and named using the lowercase
+upstream name. When using the Cargo build system (@pxref{Build Systems,
+@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
+argument set to @code{#f}, as this argument only makes sense to dependencies.
+When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
+be set to enable relevant support.
+
+Rust dependencies are managed in two modules:
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix. The name should be changed to lowercase as appropriate and
-dashes should remain in place.
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, crate, @code{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored sources unbundled before
+contributing to Guix. Naturally, this module serves as a store for both sources
+and unbundling strategies.
+
+This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
+version containing all changes from other branches, so that the maintained
+version can be used directly in case of merge conflicts, thus coordination is
+required for other committers to modify it.
+
+Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
+script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages. This includes Rust dependencies requiring external inputs
+to unbundle and Cargo workspaces.
+
+These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
+For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix. The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix. The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course). This follows the ``caret'' version scheme
-intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments. It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}. If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible. Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+package to be used at any given time, so all dependencies should have a
+versioned suffix. The versioned suffix is the left-most non-zero digit (and any
+leading zeros, of course). This follows the ``caret'' version scheme intended
+by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we are usually packaging development snapshots of Rust dependencies
+specifically for some Rust programs, and can't simply identity them by version.
+In this case we can use a @code{for-@var{program}} suffix. Examples@:
+@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Let's demonstrate the packaging workflow by examples, note that package-specific
+issues are not involved here.
+
+In preparation, we'll adding the following packages to our environment:
+
+@example
+guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
+repository @uref{https://crates.io, crates.io}.
+
+@enumerate
+@item
+We can generate a draft definition via the crates.io importer (@pxref{Invoking
+guix import, crate}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
+facilitating dependency modification. @code{'cargo-audit} used here must be a
+unique identifier, usually same to the program's variable name.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+cargo generate-lockfile
+cargo audit
+cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining Packages,
+@code{license}}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them in Guix command-line interface through expression:
+
+@example
+guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+point the imported definition to a symbol with the same name.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to point to it:
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+
+Such modifications are processed by @code{cargo-inputs} procedure.
+
+@code{cargo-inputs} can also be configured by keywoard arguments
+@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
+resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
+packages rust-sources)}, module to resolve modified symbols), which are useful
+to maintain an independent channel.
+@end enumerate
+
+Example 2: @code{niri}, which is not available on crates.io and depends on
+development snapshots (also Cargo workspaces in this example).
+
+@enumerate
+@item
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+in a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in build environment, the substitution pattern is
+package-specific.
+
+@code{cargo-inputs} returns a list, all list operations apply to it too.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3)))
+@end lisp
+
+@item
+@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
+workspace, build argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+@end enumerate
+
+Example 3: @code{fish}, which combines two build systems.
+
+@enumerate
+@item
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+@end enumerate
@node Elm Packages
--
2.48.1
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:50:04 GMT) Full text and rfc822 format available.Message #62 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Steve George <steve <at> futurile.net>, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org> Subject: Re: [bug#77093] [PATCH rust-team 03/18] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 18 Mar 2025 13:49:15 +0200
[Message part 1 (text/plain, inline)]
I've attached a diff that removes the need for RUSTC_BOOTSTRAP that
seems to work when I rebuilt rust-pubgrub.
Some links I followed:
https://doc.rust-lang.org/cargo/commands/cargo-publish.html#package-selection
https://github.com/rust-lang/cargo/issues/10948#issuecomment-1229236664
"One background knowledge: cargo publish is roughly `cargo package` +
call crates.io API."
Before and after:
(ins)efraim <at> 3900XT ~/workspace/guix-rust-team$ tree /gnu/store/zk1n23l2wdca8snz1wqllmjxv83hccl5-rust-pubgrub-0.3.0-0.b70cf70 | head
/gnu/store/zk1n23l2wdca8snz1wqllmjxv83hccl5-rust-pubgrub-0.3.0-0.b70cf70
└── share
├── cargo
│ ├── registry
│ │ ├── pubgrub-0.3.0.crate
│ │ └── version-ranges-0.1.1.crate
│ └── src
│ ├── pubgrub-0.3.0
│ │ ├── benches
│ │ │ ├── backtracking.rs
(ins)efraim <at> 3900XT ~/workspace/guix-rust-team$ tree /gnu/store/9310sgn182cgqy4p1xkn2hi8x4wg0nn8-rust-pubgrub-0.3.0-0.b70cf70 | head
/gnu/store/9310sgn182cgqy4p1xkn2hi8x4wg0nn8-rust-pubgrub-0.3.0-0.b70cf70
└── share
├── cargo
│ ├── registry
│ │ ├── pubgrub-0.3.0.crate
│ │ └── version-ranges-0.1.1.crate
│ └── src
│ ├── pubgrub-0.3.0
│ │ ├── benches
│ │ │ ├── backtracking.rs
On Tue, Mar 18, 2025 at 03:24:16PM +0800, Hilton Chain wrote:
> * guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
> [#:cargo-package-crates]: New argument.
> * guix/build/cargo-build-system.scm (package): Use it.
> * doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
>
> Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
> ---
> doc/guix.texi | 7 +++++++
> guix/build-system/cargo.scm | 4 ++++
> guix/build/cargo-build-system.scm | 28 ++++++++++++++++++++++++++--
> 3 files changed, 37 insertions(+), 2 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 0ca109a214..322512fd36 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -9363,6 +9363,13 @@ Build Systems
> the binaries defined by the crate. Unless @code{install-source? #f} is
> defined it will also install a source crate repository of itself and unpacked
> sources, to ease in future hacking on rust packages.
> +
> +This build system supports cargo workspaces. Parameter
> +@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
> +names of library crates to package in @code{package} phase. Specified
> +crates are packaged from left to right, in case there's dependency among
> +them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
> +package @code{"pcre2-sys"} first and then @code{"pcre2"}.
> @end defvar
>
> @defvar chicken-build-system
> diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
> index 452f7f78d0..4f6d46e70c 100644
> --- a/guix/build-system/cargo.scm
> +++ b/guix/build-system/cargo.scm
> @@ -94,6 +94,7 @@ (define* (cargo-build name inputs
> (vendor-dir "guix-vendor")
> (cargo-build-flags ''("--release"))
> (cargo-test-flags ''())
> + (cargo-package-crates ''())
> (cargo-package-flags ''("--no-metadata" "--no-verify"))
> (features ''())
> (skip-build? #f)
> @@ -122,6 +123,7 @@ (define* (cargo-build name inputs
> #:vendor-dir #$vendor-dir
> #:cargo-build-flags #$(sexp->gexp cargo-build-flags)
> #:cargo-test-flags #$(sexp->gexp cargo-test-flags)
> + #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
> #:cargo-package-flags #$(sexp->gexp cargo-package-flags)
> #:cargo-target #$(cargo-triplet system)
> #:features #$(sexp->gexp features)
> @@ -154,6 +156,7 @@ (define* (cargo-cross-build name
> (vendor-dir "guix-vendor")
> (cargo-build-flags ''("--release"))
> (cargo-test-flags ''())
> + (cargo-package-crates ''())
> (cargo-package-flags ''("--no-metadata" "--no-verify"))
> (cargo-target (cargo-triplet (or target system)))
> (features ''())
> @@ -185,6 +188,7 @@ (define* (cargo-cross-build name
> #:vendor-dir #$vendor-dir
> #:cargo-build-flags #$(sexp->gexp cargo-build-flags)
> #:cargo-test-flags #$(sexp->gexp cargo-test-flags)
> + #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
> #:cargo-package-flags #$(sexp->gexp cargo-package-flags)
> #:cargo-target #$(cargo-triplet (or target system))
> #:features #$(sexp->gexp features)
> diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
> index 58238992b8..001e9c7adc 100644
> --- a/guix/build/cargo-build-system.scm
> +++ b/guix/build/cargo-build-system.scm
> @@ -300,11 +300,16 @@ (define* (package #:key
> source
> skip-build?
> install-source?
> + (cargo-package-crates '())
> (cargo-package-flags '("--no-metadata" "--no-verify"))
> + (vendor-dir "guix-vendor")
> #:allow-other-keys)
> "Run 'cargo-package' for a given Cargo package."
> (if install-source?
> - (if skip-build?
> + ;; NOTE: Cargo workspace packaging support:
> + ;; #:install-source? #t + #:cargo-package-crates.
> + (if (and (null? cargo-package-crates)
> + skip-build?)
> (begin
> (install-file source "target/package")
> (with-directory-excursion "target/package"
> @@ -322,7 +327,26 @@ (define* (package #:key
> ;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
> (when (file-exists? "Cargo.toml.orig")
> (delete-file "Cargo.toml.orig"))
> - (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
> +
> + (if (null? cargo-package-crates)
> + (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
> + ;; Use unstable feature ‘--registry’.
> + (begin
> + (setenv "RUSTC_BOOTSTRAP" "1")
> + (for-each
> + (lambda (pkg)
> + (apply invoke "cargo" "package" "--offline" "--package" pkg
> + "--registry" "crates-io" "-Z" "package-workspace"
> + cargo-package-flags)
> + (for-each
> + (lambda (crate)
> + (invoke "tar" "xzf" crate "-C" vendor-dir))
> + (begin
> + (delete-file-recursively "target/package/tmp-registry")
> + (find-files "target/package" "\\.crate$")))
> + (patch-cargo-checksums #:vendor-dir vendor-dir))
> + cargo-package-crates)
> + (unsetenv "RUSTC_BOOTSTRAP")))
>
> ;; Then unpack the crate, reset the timestamp of all contained files, and
> ;; repack them. This is necessary to ensure that they are reproducible.
> --
> 2.48.1
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[0003-no-unstable-flags (text/plain, attachment)]
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:50:06 GMT) Full text and rfc822 format available.Message #65 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team 08/18] build/cargo: Use system jemalloc. Date: Tue, 18 Mar 2025 13:49:21 +0200
[Message part 1 (text/plain, inline)]
I'm not sure how I feel about this one. On one hand the unprefixed_malloc flag is definitely necessary. On the other hand sometimes we use libjemalloc.so and sometimes libjemalloc_pic.a, and I don't remember when/why we use one over the other, other than "it just works" On Tue, Mar 18, 2025 at 03:24:21PM +0800, Hilton Chain wrote: > From: Efraim Flashner <efraim <at> flashner.co.il> > > * guix/build/cargo-build-system.scm (configure): Set environment variables to > use system jemalloc. > > Change-Id: Ibb1c025c7354ba2befd2770e3f1ccd785baa06c3 > Modified-by: Hilton Chain <hako <at> ultrarare.space> > --- > guix/build/cargo-build-system.scm | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm > index a1c4f556bb..b547421b88 100644 > --- a/guix/build/cargo-build-system.scm > +++ b/guix/build/cargo-build-system.scm > @@ -248,6 +248,12 @@ (define* (configure #:key inputs > (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1") > (setenv "SODIUM_USE_PKG_CONFIG" "1") > (setenv "ZSTD_SYS_USE_PKG_CONFIG" "1") > + ;; This flag is needed when not using the bundled jemalloc. > + ;; https://github.com/tikv/jemallocator/issues/19 > + (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1") > + (when (assoc-ref inputs "jemalloc") > + (setenv "JEMALLOC_OVERRIDE" > + (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so"))) > (when (assoc-ref inputs "openssl") > (setenv "OPENSSL_DIR" (assoc-ref inputs "openssl"))) > (when (assoc-ref inputs "gettext") > -- > 2.48.1 > -- Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:50:08 GMT) Full text and rfc822 format available.Message #68 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Steve George <steve <at> futurile.net>, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org> Subject: Re: [bug#77093] [PATCH rust-team 04/18] build-system: cargo: Support installing Cargo workspace. Date: Tue, 18 Mar 2025 13:49:27 +0200
[Message part 1 (text/plain, inline)]
I don't love the name cargo-install-paths. I had to build atuin to see
what it did. The last few lines of the 'install phase:
Compiling atuin-daemon v18.4.0 (/tmp/guix-build-atuin-18.4.0.drv-0/source/crates/atuin-daemon)
Compiling atuin v18.4.0 (/tmp/guix-build-atuin-18.4.0.drv-0/source/crates/atuin)
Finished `release` profile [optimized] target(s) in 22.73s
Installing /gnu/store/yqfi454wkc1lv455m7n5i7l5lwn1la8v-atuin-18.4.0/bin/atuin
Installed package `atuin v18.4.0 (/tmp/guix-build-atuin-18.4.0.drv-0/source/crates/atuin)` (executable `atuin`)
warning: be sure to add `/gnu/store/yqfi454wkc1lv455m7n5i7l5lwn1la8v-atuin-18.4.0/bin` to your PATH to be able to run the installed binaries
With #:cargo-install-paths removed I got an error that it was a virtual
manifest and to find the actual package. Looking at the Cargo.toml I
don't see an obvious way to figure out which package(s) in the workspace
has an installable executable.
So obviously 'paths-to-installable-crates-in-the-worktree' is too long.
Looking at the code it's obvious how the name cargo-install-path came to
be.
On Tue, Mar 18, 2025 at 03:24:17PM +0800, Hilton Chain wrote:
> *guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
> [#:cargo-install-paths]: New argument.
> * guix/build/cargo-build-system.scm (install): Use it.
> * doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
>
> Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
> ---
> doc/guix.texi | 5 ++++-
> guix/build-system/cargo.scm | 4 ++++
> guix/build/cargo-build-system.scm | 17 +++++++++++++----
> 3 files changed, 21 insertions(+), 5 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 322512fd36..c20e1d7f9c 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -9369,7 +9369,10 @@ Build Systems
> names of library crates to package in @code{package} phase. Specified
> crates are packaged from left to right, in case there's dependency among
> them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
> -package @code{"pcre2-sys"} first and then @code{"pcre2"}.
> +package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
> +@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
> +paths of binary crates to install in @code{install} phase,
paths of binary crates to install in the @code{install} phase,
> +@code{''("crates/atuin")}, for example.
> @end defvar
>
> @defvar chicken-build-system
> diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
> index 4f6d46e70c..4486c706a1 100644
> --- a/guix/build-system/cargo.scm
> +++ b/guix/build-system/cargo.scm
> @@ -96,6 +96,7 @@ (define* (cargo-build name inputs
> (cargo-test-flags ''())
> (cargo-package-crates ''())
> (cargo-package-flags ''("--no-metadata" "--no-verify"))
> + (cargo-install-paths ''())
> (features ''())
> (skip-build? #f)
> (parallel-build? #t)
> @@ -125,6 +126,7 @@ (define* (cargo-build name inputs
> #:cargo-test-flags #$(sexp->gexp cargo-test-flags)
> #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
> #:cargo-package-flags #$(sexp->gexp cargo-package-flags)
> + #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
> #:cargo-target #$(cargo-triplet system)
> #:features #$(sexp->gexp features)
> #:skip-build? #$skip-build?
> @@ -158,6 +160,7 @@ (define* (cargo-cross-build name
> (cargo-test-flags ''())
> (cargo-package-crates ''())
> (cargo-package-flags ''("--no-metadata" "--no-verify"))
> + (cargo-install-paths ''())
> (cargo-target (cargo-triplet (or target system)))
> (features ''())
> (skip-build? #f)
> @@ -190,6 +193,7 @@ (define* (cargo-cross-build name
> #:cargo-test-flags #$(sexp->gexp cargo-test-flags)
> #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
> #:cargo-package-flags #$(sexp->gexp cargo-package-flags)
> + #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
> #:cargo-target #$(cargo-triplet (or target system))
> #:features #$(sexp->gexp features)
> #:skip-build? #$skip-build?
> diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
> index 001e9c7adc..bcee10837e 100644
> --- a/guix/build/cargo-build-system.scm
> +++ b/guix/build/cargo-build-system.scm
> @@ -384,6 +384,7 @@ (define* (install #:key
> skip-build?
> install-source?
> features
> + (cargo-install-paths '())
> #:allow-other-keys)
> "Install a given Cargo package."
> (let* ((out (assoc-ref outputs "out"))
> @@ -398,10 +399,18 @@ (define* (install #:key
> ;; Only install crates which include binary targets,
> ;; otherwise cargo will raise an error.
> (or skip-build?
> - (not (has-executable-target?))
> - (invoke "cargo" "install" "--offline" "--no-track"
> - "--path" "." "--root" out
> - "--features" (string-join features)))
> + ;; NOTE: Cargo workspace installation support:
> + ;; #:skip-build? #f + #:cargo-install-paths.
> + (and (null? cargo-install-paths)
> + (not (has-executable-target?)))
> + (for-each
> + (lambda (path)
> + (invoke "cargo" "install" "--offline" "--no-track"
> + "--path" path "--root" out
> + "--features" (string-join features)))
> + (if (null? cargo-install-paths)
> + '(".")
> + cargo-install-paths)))
>
> (when install-source?
> ;; Install crate tarballs and unpacked sources for later use.
> --
> 2.48.1
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:50:12 GMT) Full text and rfc822 format available.Message #71 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team 10/18] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 18 Mar 2025 13:49:33 +0200
[Message part 1 (text/plain, inline)]
Is this only relevant for packages crated with 'guix import cargo'? If so we should probably just change the importer to add a newline to the misbehaving importer. On Tue, Mar 18, 2025 at 03:24:23PM +0800, Hilton Chain wrote: > For consistency with ‘insert-expression’. > > * guix/scripts/import.scm (guix-import): Add two newlines when inserting. > > Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d > --- > guix/scripts/import.scm | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm > index 4fad329cd3..58a84d0db7 100644 > --- a/guix/scripts/import.scm > +++ b/guix/scripts/import.scm > @@ -129,6 +129,7 @@ (define-command (guix-import . args) > (let ((port (open-file file "a"))) > (pretty-print-with-comments port expr) > (newline port) > + (newline port) > (close-port port))))))))) > (import-as-definitions importer args find-and-insert))) > ((importer args ...) > -- > 2.48.1 > > > > -- Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:51:03 GMT) Full text and rfc822 format available.Message #74 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team 11/18] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 13:49:39 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 03:24:24PM +0800, Hilton Chain wrote:
> * guix/utils.scm (find-definition-location): New procedure.
> (find-definition-insertion-location): Define with it.
> * guix/scripts/import.scm (import-as-definitions, guix-import): Support
> expressions defined by 'define.
>
> Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
> ---
> guix/scripts/import.scm | 20 +++++++++++++++-----
> guix/utils.scm | 27 +++++++++++++++++++--------
> 2 files changed, 34 insertions(+), 13 deletions(-)
>
> diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> index 58a84d0db7..aaa3d26673 100644
> --- a/guix/scripts/import.scm
> +++ b/guix/scripts/import.scm
> @@ -30,6 +30,7 @@ (define-module (guix scripts import)
> #:use-module (guix read-print)
> #:use-module (guix utils)
> #:use-module (srfi srfi-1)
> + #:use-module (srfi srfi-26)
> #:use-module (ice-9 format)
> #:use-module (ice-9 match)
> #:export (%standard-import-options
> @@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
> ((and expr (or ('package _ ...)
> ('let _ ...)))
> (proc (package->definition expr)))
> - ((and expr ('define-public _ ...))
> + ((and expr (or ('define-public _ ...)
> + ('define _ ...)))
> (proc expr))
> ((expressions ...)
> (for-each (lambda (expr)
> @@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
> ((and expr (or ('package _ ...)
> ('let _ ...)))
> (proc (package->definition expr)))
> - ((and expr ('define-public _ ...))
> + ((and expr (or ('define-public _ ...)
> + ('define _ ...)))
> (proc expr))))
> expressions))
> (x
> @@ -117,13 +120,20 @@ (define-command (guix-import . args)
> (show-version-and-exit "guix import"))
> ((or ("-i" file importer args ...)
> ("--insert" file importer args ...))
> - (let ((find-and-insert
> + (let* ((definer?
> + (cut member
> + <>
> + `(,@(if (member importer '("crate"))
> + '(define)
> + '())
> + define-public)))
This part above seems like it would break the option to use 'guix import
crate' with the --insert flag for "normal" packages. I question how
useful it currently is in that scenario since we normally have to strip
the rust- prefix from those packages anyway.
> + (find-and-insert
> (lambda (expr)
> (match expr
> - (('define-public term _ ...)
> + (((? definer? definer) term _ ...)
> (let ((source-properties
> (find-definition-insertion-location
> - file term)))
> + file term #:definer definer)))
> (if source-properties
> (insert-expression source-properties expr)
> (let ((port (open-file file "a")))
> diff --git a/guix/utils.scm b/guix/utils.scm
> index c7c23d9d5b..77ec6d992a 100644
> --- a/guix/utils.scm
> +++ b/guix/utils.scm
> @@ -154,6 +154,7 @@ (define-module (guix utils)
> edit-expression
> delete-expression
> insert-expression
> + find-definition-location
> find-definition-insertion-location
>
> filtered-port
> @@ -520,24 +521,34 @@ (define (insert-expression source-properties expr)
> (string-append expr "\n\n" str))))
> (edit-expression source-properties insert)))
>
> -(define (find-definition-insertion-location file term)
> - "Search in FILE for a top-level public definition whose defined term
> -alphabetically succeeds TERM. Return the location if found, or #f
> -otherwise."
> - (let ((search-term (symbol->string term)))
> +(define* (find-definition-location file term
> + #:key (definer 'define-public)
> + (pred string=))
> + "Search in FILE for a top-level definition defined by DEFINER with defined
> +term comparing to TERM through PRED. Return the location if PRED returns #t,
> +or #f otherwise."
> + (let ((search-term (symbol->string term))
> + (definer? (cut eq? definer <>)))
> (call-with-input-file file
> (lambda (port)
> (do ((syntax (read-syntax port)
> (read-syntax port)))
> ((match (syntax->datum syntax)
> - (('define-public current-term _ ...)
> - (string> (symbol->string current-term)
> - search-term))
> + (((? definer?) current-term _ ...)
> + (pred (symbol->string current-term)
> + search-term))
> ((? eof-object?) #t)
> (_ #f))
> (and (not (eof-object? syntax))
> (syntax-source syntax))))))))
>
> +(define* (find-definition-insertion-location file term
> + #:key (definer 'define-public))
> + "Search in FILE for a top-level definition defined by DEFINER with defined
> +term alphabetically succeeds TERM. Return the location if found, or #f
> +otherwise."
> + (find-definition-location file term #:definer definer #:pred string>))
> +
>
> ;;;
> ;;; Keyword arguments.
> --
> 2.48.1
>
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:51:04 GMT) Full text and rfc822 format available.Message #77 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Ludovic Courtès <ludo <at> gnu.org>, 77093 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Subject: Re: [bug#77093] [PATCH rust-team 18/18] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 18 Mar 2025 13:49:54 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 03:24:31PM +0800, Hilton Chain wrote:
> * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
> documentation.
>
> Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
> ---
> doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 390 insertions(+), 25 deletions(-)
>
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index ab4f30d54b..e7c5116a3d 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1600,34 +1600,399 @@ Rust Crates
> @subsection Rust Crates
>
> @cindex rust
> -Rust programs standing for themselves are named as any other package, using the
> -lowercase upstream name.
> +As currently there's no achievable way to reuse Rust packages as pre-compiled
> +inputs for other packages at a distribution scale, and our previous approach on
> +making all Rust dependencies as full packages doesn't cope well with Rust
> +ecosystem thus causing issues on both contribution and maintenance sides, we
> +have switched to a different packaging model.
I'm not sure the first paragraph is necessary.
> +
> +Rust programs (binary crates) and dependencies (library crates) are treated
> +separately. We put main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with manual focus on unbundling
> +vendored dependencies. The following paragraphs will explain them and give
> +several examples.
Rust programs (binary crates) and dependencies (library crates) are treated
separately. We put our main efforts into programs and only package Rust
dependencies as sources, utilizing automation with a manual focus on unbundling
vendored dependencies. The following paragraphs will explain them and give
several examples.
> +
> +Rust programs are treated like any other package and named using the lowercase
> +upstream name. When using the Cargo build system (@pxref{Build Systems,
> +@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> +argument set to @code{#f}, as this argument only makes sense to dependencies.
argument set to @code{#f}, as this argument only makes sense for dependencies.
> +When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> +be set to enable relevant support.
> +
> +Rust dependencies are managed in two modules:
>
> -To prevent namespace collisions we prefix all other Rust packages with the
> -@code{rust-} prefix. The name should be changed to lowercase as appropriate and
> -dashes should remain in place.
> +@enumerate
> +@item
> +@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
> +programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
> +import, crate, @code{--lockfile=@var{file}}}).
> +
> +Imported definitions must be checked and have vendored sources unbundled before
> +contributing to Guix. Naturally, this module serves as a store for both sources
> +and unbundling strategies.
> +
> +This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
This module is managed by the Rust team (@pxref{Teams}) to ensure there's always one
> +version containing all changes from other branches, so that the maintained
> +version can be used directly in case of merge conflicts, thus coordination is
> +required for other committers to modify it.
> +
> +Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
> +script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
script @file{etc/teams/rust/cleanup-crates.sh} for this module.
> +
> +@item
> +@code{(gnu packages rust-sources)}, storing more complex definitions that need
> +to be full packages. This includes Rust dependencies requiring external inputs
> +to unbundle and Cargo workspaces.
> +
> +These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
These dependencies should have the @code{#:skip-build?} argument set to @code{#t}.
> +For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +
> +Since they are added manually, the following naming convention applies:
> +
> +To prevent namespace collisions they are named with @code{rust-} prefix. The
> +name should be changed to lowercase as appropriate and dashes should remain in
> +place.
>
> In the rust ecosystem it is common for multiple incompatible versions of a
> -package to be used at any given time, so all package definitions should have a
> -versioned suffix. The versioned suffix is the left-most non-zero digit (and
> -any leading zeros, of course). This follows the ``caret'' version scheme
> -intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> -
> -Because of the difficulty in reusing rust packages as pre-compiled inputs for
> -other packages the Cargo build system (@pxref{Build Systems,
> -@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
> -@code{cargo-development-inputs} keywords as build system arguments. It would be
> -helpful to think of these as similar to @code{propagated-inputs} and
> -@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
> -should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
> -@code{#:cargo-development-inputs}. If a Rust package links to other libraries
> -then the standard placement in @code{inputs} and the like should be used.
> -
> -Care should be taken to ensure the correct version of dependencies are used; to
> -this end we try to refrain from skipping the tests or using @code{#:skip-build?}
> -when possible. Of course this is not always possible, as the package may be
> -developed for a different Operating System, depend on features from the Nightly
> -Rust compiler, or the test suite may have atrophied since it was released.
> +package to be used at any given time, so all dependencies should have a
> +versioned suffix. The versioned suffix is the left-most non-zero digit (and any
> +leading zeros, of course). This follows the ``caret'' version scheme intended
> +by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> +
> +In practice we are usually packaging development snapshots of Rust dependencies
> +specifically for some Rust programs, and can't simply identity them by version.
> +In this case we can use a @code{for-@var{program}} suffix. Examples@:
> +@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
> +@end enumerate
> +
> +Let's demonstrate the packaging workflow by examples, note that package-specific
> +issues are not involved here.
> +
> +In preparation, we'll adding the following packages to our environment:
> +
> +@example
> +guix shell rust rust:cargo cargo-audit cargo-license
> +@end example
> +
> +Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
> +repository @uref{https://crates.io, crates.io}.
> +
> +@enumerate
> +@item
> +We can generate a draft definition via the crates.io importer (@pxref{Invoking
> +guix import, crate}). In the end we'll have the following definiton:
> +
> +@lisp
> +(define-public cargo-audit
> + (package
> + (name "cargo-audit")
> + (version "0.21.2")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (crate-uri "cargo-audit" version))
> + (file-name (string-append name "-" version ".tar.gz"))
> + (sha256
> + (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
> + (build-system cargo-build-system)
> + (arguments (list #:install-source? #f))
> + (inputs (cargo-inputs 'cargo-audit))
> + (home-page "https://rustsec.org/")
> + (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
> + (description
> + "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
> +audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
> + (license (list license:asl2.0 license:expat))))
> +@end lisp
> +
> +@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
> +facilitating dependency modification. @code{'cargo-audit} used here must be a
> +unique identifier, usually same to the program's variable name.
> +
> +@item
> +Unpack package source and navigate to the unpacked directory, then execute the
> +following commands:
> +
> +@example
> +cargo generate-lockfile
> +cargo audit
> +cargo license
> +@end example
> +
> +@command{cargo generate-lockfile} updates dependencies to compatible versions,
> +@command{cargo audit} checks known vulnerabilities and @command{cargo license}
> +checks licenses of all dependencies.
> +
> +We must have an acceptable output of @command{cargo audit} and ensure all
> +dependencies are licensed with our supported licenses (@pxref{Defining Packages,
> +@code{license}}).
> +
> +@item
> +Import dependencies from previously generated lockfile:
> +
> +@example
> +guix import --insert=gnu/packages/rust-crates.scm \
> + crate --lockfile=/path/to/Cargo.lock cargo-audit
> +@end example
> +
> +@code{cargo-audit} used here must be consistent with the identifier used for
> +@code{cargo-inputs}.
> +
> +At this stage, @code{cargo-audit} is buildable.
> +
> +@item
> +Finally we'll unbundle vendored sources. The lockfile importer inserts
> +@code{TODO:} comments to dependencies with high probability of bundled sources.
> +@code{cargo-build-system} also performs additional check in its
> +@code{check-for-pregenerated-files} phase:
> +
> +@example
> +$ guix build cargo-audit
> +@dots{}
> +starting phase `check-for-pregenerated-files'
> +Searching for binary files...
> +./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
> +./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
> +@dots{}
> +@end example
> +
> +Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
> +can still select them in Guix command-line interface through expression:
> +
> +@example
> +guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
> +@end example
> +
> +For most dependencies, a snippet is sufficient:
> +
> +@lisp
> +(define rust-bzip2-sys-0.1.13+1.0.8
> + (crate-source "bzip2-sys" "0.1.13+1.0.8"
> + "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
> + #:snippet
> + '(begin
> + (delete-file-recursively "bzip2-1.0.8")
> + (delete-file "build.rs")
> + ;; Inspired by Debian's patch.
> + (with-output-to-file "build.rs"
> + (lambda _
> + (format #t "fn main() @{~@@
> + println!(\"cargo:rustc-link-lib=bz2\");~@@
> + @}~%"))))))
> +@end lisp
> +
> +In a more complex case, where unbundling one dependency requires other packages,
> +we should package the dependency in @code{(gnu packages rust-sources)} first and
> +point the imported definition to a symbol with the same name.
> +
> +For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
> +rust-sources)}, then the imported definition in @code{(gnu packages
> +rust-crates)} should be modified to point to it:
> +
> +@lisp
> +(define rust-ring-0.17.14 'rust-ring-0.17)
> +@end lisp
> +
> +When one dependency can be safely removed, modify it to @code{#f}.
> +
> +@lisp
> +(define rust-openssl-src-300.4.2+3.4.1 #f)
> +@end lisp
> +
> +Such modifications are processed by @code{cargo-inputs} procedure.
> +
> +@code{cargo-inputs} can also be configured by keywoard arguments
> +@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
> +resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
> +packages rust-sources)}, module to resolve modified symbols), which are useful
> +to maintain an independent channel.
> +@end enumerate
> +
> +Example 2: @code{niri}, which is not available on crates.io and depends on
> +development snapshots (also Cargo workspaces in this example).
> +
> +@enumerate
> +@item
> +As we can't ensure compatibility of a development snapshot, before executing
> +@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
> +in a known working revision.
> +
> +To use our packaged development snapshots, it's also necessary to modify
> +@file{Cargo.toml} in build environment, the substitution pattern is
> +package-specific.
> +
> +@code{cargo-inputs} returns a list, all list operations apply to it too.
> +
> +@lisp
> +(define-public niri
> + (package
> + (name "niri")
> + (version "25.02")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/YaLTeR/niri")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:install-source? #f
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("# version =.*")
> + "version = \"*\"")
> + (("git.*optional")
> + "version = \"*\", optional")
> + (("^git = .*")
> + "")))))))
> + (native-inputs
> + (list pkg-config))
> + (inputs
> + (cons* clang
> + libdisplay-info
> + libinput-minimal
> + libseat
> + libxkbcommon
> + mesa
> + pango
> + pipewire
> + wayland
> + (cargo-inputs 'niri)))
> + (home-page "https://github.com/YaLTeR/niri")
> + (synopsis "Scrollable-tiling Wayland compositor")
> + (description
> + "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
> +scrollable format. It is considered stable for daily use and performs most
> +functions expected of a Wayland compositor.")
> + (license license:gpl3)))
> +@end lisp
> +
> +@item
> +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> +workspace, build argument @code{#:cargo-package-crates} is required.
> +
> +@lisp
> +(define-public rust-pipewire-for-niri
> + (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> + (revision "0"))
> + (package
> + (name "rust-pipewire")
> + (version (git-version "0.8.0" revision commit))
> + (source
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> + (commit commit)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:skip-build? #t
> + #:cargo-package-crates
> + ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> + (inputs (cargo-inputs 'rust-pipewire-for-niri))
> + (home-page "https://pipewire.org/")
> + (synopsis "Rust bindings for PipeWire")
> + (description "This package provides Rust bindings for PipeWire.")
> + (license license:expat))))
> +@end lisp
> +
> +Don't forget to modify all workspace members in @code{(gnu packages
> +rust-crates)}:
> +
> +@lisp
> +(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@dots{}
> +(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@end lisp
> +@end enumerate
> +
> +Example 3: @code{fish}, which combines two build systems.
> +
> +@enumerate
> +@item
> +When building Rust packages in other build systems, we need to add @code{rust}
> +and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
> +build systems, and apply necessary build phases from @code{cargo-build-system}.
> +
> +@lisp
> +(define-public fish
> + (package
> + (name "fish")
> + (version "4.0.0")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (string-append "https://github.com/fish-shell/fish-shell/"
> + "releases/download/" version "/"
> + "fish-" version ".tar.xz"))
> + (sha256
> + (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
> + (build-system cmake-build-system)
> + (inputs
> + (cons* fish-foreign-env ncurses pcre2
> + python ;for fish_config and manpage completions
> + (cargo-inputs 'fish)))
> + (native-inputs
> + (list doxygen groff ;for 'fish --help'
> + pkg-config
> + procps ;for the test suite
> + rust
> + `(,rust "cargo")))
> + (arguments
> + (list #:imported-modules
> + (append %cargo-build-system-modules
> + %cmake-build-system-modules)
> + #:modules
> + (((guix build cargo-build-system) #:prefix cargo:)
> + (guix build cmake-build-system)
> + (guix build utils))
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("git.*tag.*,")
> + "version = \"*\","))))
> + (add-after 'unpack 'prepare-cargo-build-system
> + (lambda args
> + (for-each
> + (lambda (phase)
> + (format #t "Running cargo phase: ~a~%" phase)
> + (apply (assoc-ref cargo:%standard-phases phase)
> + args))
> + '(unpack-rust-crates
> + configure
> + check-for-pregenerated-files
> + patch-cargo-checksums)))))))
> + (synopsis "The friendly interactive shell")
> + (description
> + "Fish (friendly interactive shell) is a shell focused on interactive use,
> +discoverability, and friendliness. Fish has very user-friendly and powerful
> +tab-completion, including descriptions of every completion, completion of
> +strings with wildcards, and many completions for specific commands. It also
> +has extensive and discoverable help. A special @@command@{help@} command gives
> +access to all the fish documentation in your web browser. Other features
> +include smart terminal handling based on terminfo, an easy to search history,
> +and syntax highlighting.")
> + (home-page "https://fishshell.com/")
> + (license license:gpl2)))
> +@end lisp
> +@end enumerate
>
>
> @node Elm Packages
> --
> 2.48.1
>
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:51:05 GMT) Full text and rfc822 format available.Message #80 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Steve George <steve <at> futurile.net>, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team 17/18] import: crate: Add ‘--lockfile’ option. Date: Tue, 18 Mar 2025 13:49:46 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 03:24:30PM +0800, Hilton Chain wrote:
> * guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
> * guix/scripts/import/crate.scm (show-help): Add it.
> (guix-import-crate): Use it.
> * doc/guix.texi (Invoking guix import): Document it.
> * tests/crate.scm (temp-file): New variable.
> ("crate-lockfile-import"): New test.
>
> Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
> ---
> doc/guix.texi | 15 ++++++
> guix/scripts/import/crate.scm | 58 +++++++++++++++++++----
> tests/crate.scm | 88 +++++++++++++++++++++++++++++++++++
> 3 files changed, 151 insertions(+), 10 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index c20e1d7f9c..029c8cf59c 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -14703,6 +14703,21 @@ Invoking guix import
> If a crate dependency is not (yet) packaged, make the corresponding
> input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
> a comment.
> +@item --lockfile=@var{file}
> +@itemx -f @var{file}
> +When @option{--lockfile} is specified, importer will ignore other
> +options and won't output package definitions, instead generating source
> +definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
When @option{--lockfile} is specified, the importer will ignore other
options and won't output package definitions, instead generating source
definitions for all dependencies in @var{file}, a @file{Cargo.lock} file.
> +For example:
> +
> +@example
> +guix import crate --lockfile=/path/to/Cargo.lock my-package
> +@end example
> +
> +generates sources from @file{/path/to/Cargo.lock}, and a list
> +@code{my-package-cargo-inputs} referencing these sources. The generated
> +list is intended for package's base @code{inputs}, replacing
> +@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
generates sources from @file{/path/to/Cargo.lock} and a list
@code{my-package-cargo-inputs} referencing these sources. The generated
list is intended for the package's @code{inputs}, replacing
@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
> @end table
>
> @item elm
> diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
> index 723cbb3665..3b536e135a 100644
> --- a/guix/scripts/import/crate.scm
> +++ b/guix/scripts/import/crate.scm
> @@ -25,11 +25,13 @@
> (define-module (guix scripts import crate)
> #:use-module (guix ui)
> #:use-module (guix utils)
> + #:use-module (guix read-print)
> #:use-module (guix scripts)
> #:use-module (guix import crate)
> #:use-module (guix scripts import)
> #:use-module (srfi srfi-1)
> #:use-module (srfi srfi-11)
> + #:use-module (srfi srfi-26)
> #:use-module (srfi srfi-37)
> #:use-module (ice-9 match)
> #:use-module (ice-9 format)
> @@ -60,6 +62,9 @@ (define (show-help)
> sufficient package exists for it"))
> (newline)
> (display (G_ "
> + -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
> + (newline)
> + (display (G_ "
> -h, --help display this help and exit"))
> (display (G_ "
> -V, --version display version information and exit"))
> @@ -87,6 +92,9 @@ (define %options
> (option '("mark-missing") #f #f
> (lambda (opt name arg result)
> (alist-cons 'mark-missing #t result)))
> + (option '(#\f "lockfile") #f #t
> + (lambda (opt name arg result)
> + (alist-cons 'lockfile arg result)))
> %standard-import-options))
>
>
> @@ -101,6 +109,8 @@ (define (guix-import-crate . args)
> #:build-options? #f))
>
> (let* ((opts (parse-options))
> + (lockfile (assoc-ref opts 'lockfile))
> + (file-to-insert (assoc-ref opts 'file-to-insert))
> (args (filter-map (match-lambda
> (('argument . value)
> value)
> @@ -111,16 +121,44 @@ (define (guix-import-crate . args)
> (define-values (name version)
> (package-name->name+version spec))
>
> - (match (if (assoc-ref opts 'recursive)
> - (crate-recursive-import
> - name #:version version
> - #:recursive-dev-dependencies?
> - (assoc-ref opts 'recursive-dev-dependencies)
> - #:allow-yanked? (assoc-ref opts 'allow-yanked))
> - (crate->guix-package
> - name #:version version #:include-dev-deps? #t
> - #:allow-yanked? (assoc-ref opts 'allow-yanked)
> - #:mark-missing? (assoc-ref opts 'mark-missing)))
> + (match (cond
> + ((and=> lockfile
> + (lambda (file)
> + (or (file-exists? file)
> + (leave (G_ "file '~a' does not exist~%") file))))
> + (let-values (((crate-definitions input-list)
> + (cargo-lock->definitions lockfile name)))
> + (if file-to-insert
> + ;; Adjusted from ‘--insert’ option of guix-import.
> + (let ((term (second input-list)))
> + (begin
> + ;; Remove existing input list definition.
> + (and=> (find-definition-location file-to-insert term)
> + delete-expression)
> + ;; Insert input list alphabetically.
> + (or (and=> (find-definition-insertion-location
> + file-to-insert term)
> + (cut insert-expression <> input-list))
> + (let ((port (open-file file-to-insert "a")))
> + (newline port)
> + (pretty-print-with-comments port input-list)
> + (newline port)
> + (newline port)
> + (close-port port))))
> + crate-definitions)
> + `(,@crate-definitions
> + ,input-list))))
> + ((assoc-ref opts 'recursive)
> + (crate-recursive-import
> + name #:version version
> + #:recursive-dev-dependencies?
> + (assoc-ref opts 'recursive-dev-dependencies)
> + #:allow-yanked? (assoc-ref opts 'allow-yanked)))
> + (else
> + (crate->guix-package
> + name #:version version #:include-dev-deps? #t
> + #:allow-yanked? (assoc-ref opts 'allow-yanked)
> + #:mark-missing? (assoc-ref opts 'mark-missing))))
> ((or #f '())
> (leave (G_ "failed to download meta-data for package '~a'~%")
> (if version
> diff --git a/tests/crate.scm b/tests/crate.scm
> index d0dc779cd2..d46f753f9c 100644
> --- a/tests/crate.scm
> +++ b/tests/crate.scm
> @@ -34,6 +34,7 @@ (define-module (test-crate)
> #:use-module (gnu packages)
> #:use-module (ice-9 iconv)
> #:use-module (ice-9 match)
> + #:use-module (srfi srfi-11)
> #:use-module (srfi srfi-64))
>
>
> @@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
> (description #f)
> (license #f)))
>
> +(define temp-file
> + (string-append "t-utils-" (number->string (getpid))))
> +
>
> (test-begin "crate")
>
> @@ -1178,4 +1182,88 @@ (define rust-leaf-bob-3.0.2-yanked
> (x
> (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
>
> +
> +(test-assert "crate-lockfile-import"
> + (begin
> + (call-with-output-file temp-file
> + (lambda (port)
> + (display "\
> +# This file is automatically @generated by Cargo.
> +# It is not intended for manual editing.
> +version = 3
> +
> +[[package]]
> +name = \"adler2\"
> +version = \"2.0.0\"
> +source = \"registry+https://github.com/rust-lang/crates.io-index\"
> +checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
> +
> +[[package]]
> +name = \"aho-corasick\"
> +version = \"1.1.3\"
> +source = \"registry+https://github.com/rust-lang/crates.io-index\"
> +checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
> +dependencies = [
> + \"memchr\",
> +]
> +
> +[[package]]
> +name = \"smithay\"
> +version = \"0.4.0\"
> +source = \"git+https://github.com/Smithay/smithay.git?rev=\
> +0cd3345c59f7cb139521f267956a1a4e33248393#\
> +0cd3345c59f7cb139521f267956a1a4e33248393\"
> +dependencies = [
> + \"appendlist\",
> +]
> +
> +[[package]]
> +name = \"test\"
> +version = \"25.2.0\"\n" port)))
> + (mock
> + ((guix scripts download) guix-download
> + (lambda _
> + (format #t "~a~%~a~%"
> + "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
> + "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
> + (let-values
> + (((crates-definitions input-list)
> + (cargo-lock->definitions temp-file "test")))
> + (and
> + (match crates-definitions
> + (((define 'rust-adler2-2.0.0
> + (crate-source
> + "adler2" "2.0.0"
> + "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
> + (define 'rust-aho-corasick-1.1.3
> + (crate-source
> + "aho-corasick" "1.1.3"
> + "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
> + (define 'rust-smithay-0.4.0.0cd3345
> + ($ <comment>
> + ";; TODO: Define standalone package if this is a workspace.\n"
> + #f)
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/Smithay/smithay.git")
> + (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
> + (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
> + (sha256
> + (base32
> + "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
> + #t)
> + (x
> + (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
> + (match input-list
> + ((define-public 'test-cargo-inputs
> + (list rust-adler2-2.0.0
> + rust-aho-corasick-1.1.3
> + rust-smithay-0.4.0.0cd3345))
> + #t)
> + (x
> + (pk 'fail x #f))))))))
> +
> (test-end "crate")
> +
> +(false-if-exception (delete-file temp-file))
> --
> 2.48.1
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:02 GMT) Full text and rfc822 format available.Message #83 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Steve George <steve <at> futurile.net> Subject: Re: [bug#77093] [PATCH rust-team 03/18] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 18 Mar 2025 20:07:26 +0800
On Tue, 18 Mar 2025 19:49:15 +0800, Efraim Flashner wrote: > > [1 <multipart/mixed (7bit)>] > [1.1 <text/plain; utf-8 (quoted-printable)>] > I've attached a diff that removes the need for RUSTC_BOOTSTRAP that > seems to work when I rebuilt rust-pubgrub. > > Some links I followed: > https://doc.rust-lang.org/cargo/commands/cargo-publish.html#package-selection > https://github.com/rust-lang/cargo/issues/10948#issuecomment-1229236664 > "One background knowledge: cargo publish is roughly `cargo package` + > call crates.io API." > > Before and after: > > (ins)efraim <at> 3900XT ~/workspace/guix-rust-team$ tree /gnu/store/zk1n23l2wdca8snz1wqllmjxv83hccl5-rust-pubgrub-0.3.0-0.b70cf70 | head > /gnu/store/zk1n23l2wdca8snz1wqllmjxv83hccl5-rust-pubgrub-0.3.0-0.b70cf70 > └── share > ├── cargo > │ ├── registry > │ │ ├── pubgrub-0.3.0.crate > │ │ └── version-ranges-0.1.1.crate > │ └── src > │ ├── pubgrub-0.3.0 > │ │ ├── benches > │ │ │ ├── backtracking.rs > (ins)efraim <at> 3900XT ~/workspace/guix-rust-team$ tree /gnu/store/9310sgn182cgqy4p1xkn2hi8x4wg0nn8-rust-pubgrub-0.3.0-0.b70cf70 | head > /gnu/store/9310sgn182cgqy4p1xkn2hi8x4wg0nn8-rust-pubgrub-0.3.0-0.b70cf70 > └── share > ├── cargo > │ ├── registry > │ │ ├── pubgrub-0.3.0.crate > │ │ └── version-ranges-0.1.1.crate > │ └── src > │ ├── pubgrub-0.3.0 > │ │ ├── benches > │ │ │ ├── backtracking.rs Thanks! It works for rust-pipewire too.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:02 GMT) Full text and rfc822 format available.Message #86 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team 08/18] build/cargo: Use system jemalloc. Date: Tue, 18 Mar 2025 20:09:07 +0800
On Tue, 18 Mar 2025 19:49:21 +0800, Efraim Flashner wrote: > > [1 <text/plain; utf-8 (quoted-printable)>] > I'm not sure how I feel about this one. On one hand the > unprefixed_malloc flag is definitely necessary. On the other hand > sometimes we use libjemalloc.so and sometimes libjemalloc_pic.a, and I > don't remember when/why we use one over the other, other than "it just > works" > > On Tue, Mar 18, 2025 at 03:24:21PM +0800, Hilton Chain wrote: > > From: Efraim Flashner <efraim <at> flashner.co.il> > > > > * guix/build/cargo-build-system.scm (configure): Set environment variables to > > use system jemalloc. > > > > Change-Id: Ibb1c025c7354ba2befd2770e3f1ccd785baa06c3 > > Modified-by: Hilton Chain <hako <at> ultrarare.space> > > --- > > guix/build/cargo-build-system.scm | 6 ++++++ > > 1 file changed, 6 insertions(+) I'll drop this one for now then, I didn't search the source tree :P
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:03 GMT) Full text and rfc822 format available.Message #89 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org, Christopher Baines <guix <at> cbaines.net>, Josselin Poiret <dev <at> jpoiret.xyz>, Ludovic Courtès <ludo <at> gnu.org>, Mathieu Othacehe <othacehe <at> gnu.org>, Simon Tournier <zimon.toutoune <at> gmail.com>, Tobias Geerinckx-Rice <me <at> tobias.gr> Subject: Re: [bug#77093] [PATCH rust-team 10/18] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 18 Mar 2025 20:18:43 +0800
On Tue, 18 Mar 2025 19:49:33 +0800, Efraim Flashner wrote: > > [1 <text/plain; utf-8 (quoted-printable)>] > Is this only relevant for packages crated with 'guix import cargo'? If > so we should probably just change the importer to add a newline to the > misbehaving importer. > > On Tue, Mar 18, 2025 at 03:24:23PM +0800, Hilton Chain wrote: > > For consistency with ‘insert-expression’. > > > > * guix/scripts/import.scm (guix-import): Add two newlines when inserting. > > > > Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d > > --- > > guix/scripts/import.scm | 1 + > > 1 file changed, 1 insertion(+) This is a general issue, adding one newline doesn't work well with appending multiple definitions. For appending n definitions, (n - 1) lines need manual addition: --8<---------------cut here---------------start------------->8--- a b c --8<---------------cut here---------------end--------------->8--- By adding two newlines, only one line needs manual deletion: --8<---------------cut here---------------start------------->8--- a b c --8<---------------cut here---------------end--------------->8---
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:03 GMT) Full text and rfc822 format available.Message #92 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org, Christopher Baines <guix <at> cbaines.net>, Josselin Poiret <dev <at> jpoiret.xyz>, Ludovic Courtès <ludo <at> gnu.org>, Mathieu Othacehe <othacehe <at> gnu.org>, Simon Tournier <zimon.toutoune <at> gmail.com>, Tobias Geerinckx-Rice <me <at> tobias.gr> Subject: Re: [bug#77093] [PATCH rust-team 11/18] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 20:29:49 +0800
On Tue, 18 Mar 2025 19:49:39 +0800,
Efraim Flashner wrote:
>
> [1 <text/plain; utf-8 (quoted-printable)>]
> On Tue, Mar 18, 2025 at 03:24:24PM +0800, Hilton Chain wrote:
> > * guix/utils.scm (find-definition-location): New procedure.
> > (find-definition-insertion-location): Define with it.
> > * guix/scripts/import.scm (import-as-definitions, guix-import): Support
> > expressions defined by 'define.
> >
> > Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
> > ---
> > guix/scripts/import.scm | 20 +++++++++++++++-----
> > guix/utils.scm | 27 +++++++++++++++++++--------
> > 2 files changed, 34 insertions(+), 13 deletions(-)
> >
> > diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> > index 58a84d0db7..aaa3d26673 100644
> > --- a/guix/scripts/import.scm
> > +++ b/guix/scripts/import.scm
> > @@ -30,6 +30,7 @@ (define-module (guix scripts import)
> > #:use-module (guix read-print)
> > #:use-module (guix utils)
> > #:use-module (srfi srfi-1)
> > + #:use-module (srfi srfi-26)
> > #:use-module (ice-9 format)
> > #:use-module (ice-9 match)
> > #:export (%standard-import-options
> > @@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
> > ((and expr (or ('package _ ...)
> > ('let _ ...)))
> > (proc (package->definition expr)))
> > - ((and expr ('define-public _ ...))
> > + ((and expr (or ('define-public _ ...)
> > + ('define _ ...)))
> > (proc expr))
> > ((expressions ...)
> > (for-each (lambda (expr)
> > @@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
> > ((and expr (or ('package _ ...)
> > ('let _ ...)))
> > (proc (package->definition expr)))
> > - ((and expr ('define-public _ ...))
> > + ((and expr (or ('define-public _ ...)
> > + ('define _ ...)))
> > (proc expr))))
> > expressions))
> > (x
> > @@ -117,13 +120,20 @@ (define-command (guix-import . args)
> > (show-version-and-exit "guix import"))
> > ((or ("-i" file importer args ...)
> > ("--insert" file importer args ...))
> > - (let ((find-and-insert
> > + (let* ((definer?
> > + (cut member
> > + <>
> > + `(,@(if (member importer '("crate"))
> > + '(define)
> > + '())
> > + define-public)))
>
> This part above seems like it would break the option to use 'guix import
> crate' with the --insert flag for "normal" packages. I question how
> useful it currently is in that scenario since we normally have to strip
> the rust- prefix from those packages anyway.
Hmmm, since the crate importer now checks both ‘define’ and ‘define-public’,
there might be ordering issue when the module to insert uses ‘define’. But it's
still managable I think.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:04 GMT) Full text and rfc822 format available.Message #95 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Subject: Re: [bug#77093] [PATCH rust-team 18/18] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 18 Mar 2025 20:36:17 +0800
On Tue, 18 Mar 2025 19:49:54 +0800, Efraim Flashner wrote: > > [1 <text/plain; utf-8 (quoted-printable)>] > On Tue, Mar 18, 2025 at 03:24:31PM +0800, Hilton Chain wrote: > > * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update > > documentation. > > > > Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc > > --- > > doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++--- > > 1 file changed, 390 insertions(+), 25 deletions(-) > > > > diff --git a/doc/contributing.texi b/doc/contributing.texi > > index ab4f30d54b..e7c5116a3d 100644 > > --- a/doc/contributing.texi > > +++ b/doc/contributing.texi > > @@ -1600,34 +1600,399 @@ Rust Crates > > @subsection Rust Crates > > > > @cindex rust > > -Rust programs standing for themselves are named as any other package, using the > > -lowercase upstream name. > > +As currently there's no achievable way to reuse Rust packages as pre-compiled > > +inputs for other packages at a distribution scale, and our previous approach on > > +making all Rust dependencies as full packages doesn't cope well with Rust > > +ecosystem thus causing issues on both contribution and maintenance sides, we > > +have switched to a different packaging model. > > I'm not sure the first paragraph is necessary. I think we need to clarify why do we only do this for Rust, since similar workflow can be applied to other ecosystems with a package manager as well. Thanks for the review, I'll prepare V2 soon.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 13:17:04 GMT) Full text and rfc822 format available.Message #98 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team 11/18] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 15:16:18 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 08:29:49PM +0800, Hilton Chain wrote:
> On Tue, 18 Mar 2025 19:49:39 +0800,
> Efraim Flashner wrote:
> >
> > [1 <text/plain; utf-8 (quoted-printable)>]
> > On Tue, Mar 18, 2025 at 03:24:24PM +0800, Hilton Chain wrote:
> > > * guix/utils.scm (find-definition-location): New procedure.
> > > (find-definition-insertion-location): Define with it.
> > > * guix/scripts/import.scm (import-as-definitions, guix-import): Support
> > > expressions defined by 'define.
> > >
> > > Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
> > > ---
> > > guix/scripts/import.scm | 20 +++++++++++++++-----
> > > guix/utils.scm | 27 +++++++++++++++++++--------
> > > 2 files changed, 34 insertions(+), 13 deletions(-)
> > >
> > > diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> > > index 58a84d0db7..aaa3d26673 100644
> > > --- a/guix/scripts/import.scm
> > > +++ b/guix/scripts/import.scm
> > > @@ -30,6 +30,7 @@ (define-module (guix scripts import)
> > > #:use-module (guix read-print)
> > > #:use-module (guix utils)
> > > #:use-module (srfi srfi-1)
> > > + #:use-module (srfi srfi-26)
> > > #:use-module (ice-9 format)
> > > #:use-module (ice-9 match)
> > > #:export (%standard-import-options
> > > @@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
> > > ((and expr (or ('package _ ...)
> > > ('let _ ...)))
> > > (proc (package->definition expr)))
> > > - ((and expr ('define-public _ ...))
> > > + ((and expr (or ('define-public _ ...)
> > > + ('define _ ...)))
> > > (proc expr))
> > > ((expressions ...)
> > > (for-each (lambda (expr)
> > > @@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
> > > ((and expr (or ('package _ ...)
> > > ('let _ ...)))
> > > (proc (package->definition expr)))
> > > - ((and expr ('define-public _ ...))
> > > + ((and expr (or ('define-public _ ...)
> > > + ('define _ ...)))
> > > (proc expr))))
> > > expressions))
> > > (x
> > > @@ -117,13 +120,20 @@ (define-command (guix-import . args)
> > > (show-version-and-exit "guix import"))
> > > ((or ("-i" file importer args ...)
> > > ("--insert" file importer args ...))
> > > - (let ((find-and-insert
> > > + (let* ((definer?
> > > + (cut member
> > > + <>
> > > + `(,@(if (member importer '("crate"))
> > > + '(define)
> > > + '())
> > > + define-public)))
> >
> > This part above seems like it would break the option to use 'guix import
> > crate' with the --insert flag for "normal" packages. I question how
> > useful it currently is in that scenario since we normally have to strip
> > the rust- prefix from those packages anyway.
>
> Hmmm, since the crate importer now checks both ‘define’ and ‘define-public’,
> there might be ordering issue when the module to insert uses ‘define’. But it's
> still managable I think.
Reading it over again I think I was in need of more coffee. I thought
that the importer for cargo would only search for define, not for
define-public also.
As for needing to strip the rust- prefix, there's no change from our
current setup, so we can ignore that for now.
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:28:05 GMT) Full text and rfc822 format available.Message #101 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 01/17] build/cargo: Pass ‘--offline’ to cargo. Date: Tue, 18 Mar 2025 22:26:44 +0800
This will make error messages more helpful.
* guix/build/cargo-build-system.scm (build,check,package,install): Pass
‘--offline’ to cargo.
Change-Id: Ic95f603b793319f99c9c1fbce43f773bfc8126c0
---
guix/build/cargo-build-system.scm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 5ef6b23abd..41f54f42b7 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -246,7 +246,7 @@ (define* (build #:key
"Build a given Cargo package."
(or skip-build?
(apply invoke
- `("cargo" "build"
+ `("cargo" "build" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -264,7 +264,7 @@ (define* (check #:key
"Run tests for a given Cargo package."
(when tests?
(apply invoke
- `("cargo" "test"
+ `("cargo" "test" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -303,7 +303,7 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" ,@cargo-package-flags))
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
@@ -356,7 +356,8 @@ (define* (install #:key
;; otherwise cargo will raise an error.
(or skip-build?
(not (has-executable-target?))
- (invoke "cargo" "install" "--no-track" "--path" "." "--root" out
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" "." "--root" out
"--features" (string-join features)))
(when install-source?
--
2.48.1
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:28:07 GMT) Full text and rfc822 format available.Message #104 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 00/17] New Rust packaging workflow based on lockfile importer. Date: Tue, 18 Mar 2025 22:26:43 +0800
V1 -> V2:
* Drop jemalloc environment variable patch.
* Remove use of RUSTC_BOOTSTRAP.
* Adjust documentation.
Diff inserted at end.
Efraim Flashner (1):
build/cargo: Don't try to unpack sanity-check.py.
Hilton Chain (16):
build/cargo: Pass ‘--offline’ to cargo.
build/cargo: Print out all non-empty binary files.
build-system: cargo: Support packaging Cargo workspace.
build-system: cargo: Support installing Cargo workspace.
build/cargo: Set default value of arguments for build phases.
build/cargo: Support non-workspace directory source inputs.
scripts: import: Document argument for ‘--insert’ option in help
message.
scripts: import: Add two newlines for ‘--insert’ option.
scripts: import: Support expressions defined by 'define.
scripts: import: Pass "--insert" to importers.
scripts: import: Skip existing definition for ‘--insert’ option.
import: crate: crate-name->package-name: Move to (guix build-system
cargo).
build-system: cargo: Add ‘cargo-inputs’.
import: crate: Add Cargo.lock parser.
import: crate: Add ‘--lockfile’ option.
doc: Document lockfile importer based Rust packaging workflow.
Makefile.am | 1 +
doc/contributing.texi | 415 ++++++++++++++++++++++++++++--
doc/guix.texi | 24 ++
etc/teams/rust/cleanup-crates.sh | 38 +++
etc/teams/rust/rust-crates.tmpl | 42 +++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 42 +++
gnu/packages/rust-sources.scm | 29 +++
guix/build-system/cargo.scm | 57 +++-
guix/build/cargo-build-system.scm | 151 ++++++++---
guix/import/crate.scm | 81 +++++-
guix/import/crate/cargo-lock.scm | 105 ++++++++
guix/scripts/import.scm | 53 ++--
guix/scripts/import/crate.scm | 58 ++++-
guix/utils.scm | 27 +-
tests/crate.scm | 88 +++++++
16 files changed, 1107 insertions(+), 106 deletions(-)
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
create mode 100644 guix/import/crate/cargo-lock.scm
base-commit: b54a9ca849f013300c633fb79d80bc754f6b28a2
--
2.48.1
--8<---------------cut here---------------start------------->8---
diff --git a/doc/contributing.texi b/doc/contributing.texi
index e7c5116a3d..837074dead 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1607,15 +1607,15 @@ Rust Crates
have switched to a different packaging model.
Rust programs (binary crates) and dependencies (library crates) are treated
-separately. We put main efforts into programs and only package Rust
-dependencies as sources, utilizing automation with manual focus on unbundling
+separately. We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
vendored dependencies. The following paragraphs will explain them and give
several examples.
Rust programs are treated like any other package and named using the lowercase
upstream name. When using the Cargo build system (@pxref{Build Systems,
@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
-argument set to @code{#f}, as this argument only makes sense to dependencies.
+argument set to @code{#f}, as this argument only makes sense for dependencies.
When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
be set to enable relevant support.
@@ -1631,21 +1631,21 @@ Rust Crates
contributing to Guix. Naturally, this module serves as a store for both sources
and unbundling strategies.
-This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
-version containing all changes from other branches, so that the maintained
+This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
+one version containing all changes from other branches, so that the maintained
version can be used directly in case of merge conflicts, thus coordination is
required for other committers to modify it.
Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
-script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
+script @file{etc/teams/rust/cleanup-crates.sh} for this module.
@item
@code{(gnu packages rust-sources)}, storing more complex definitions that need
to be full packages. This includes Rust dependencies requiring external inputs
to unbundle and Cargo workspaces.
-These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
-For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
Since they are added manually, the following naming convention applies:
@@ -1668,7 +1668,7 @@ Rust Crates
Let's demonstrate the packaging workflow by examples, note that package-specific
issues are not involved here.
-In preparation, we'll adding the following packages to our environment:
+In preparation, we'll add the following packages to our environment:
@example
guix shell rust rust:cargo cargo-audit cargo-license
@@ -1747,7 +1747,7 @@ Rust Crates
@code{check-for-pregenerated-files} phase:
@example
-$ guix build cargo-audit
+$ ./pre-inst-env guix build cargo-audit
@dots{}
starting phase `check-for-pregenerated-files'
Searching for binary files...
diff --git a/doc/guix.texi b/doc/guix.texi
index 029c8cf59c..218c2ba630 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9365,14 +9365,14 @@ Build Systems
sources, to ease in future hacking on rust packages.
This build system supports cargo workspaces. Parameter
-@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
-names of library crates to package in @code{package} phase. Specified
-crates are packaged from left to right, in case there's dependency among
-them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
-package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
-@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
-paths of binary crates to install in @code{install} phase,
-@code{''("crates/atuin")}, for example.
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+library crates to package in the @code{package} phase. Specified crates are
+packaged from left to right, in case there's dependency among them. For
+example, specifying @code{''("pcre2-sys" "pcre2")} will package
+@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
+binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
+for example.
@end defvar
@defvar chicken-build-system
@@ -14705,19 +14705,18 @@ Invoking guix import
a comment.
@item --lockfile=@var{file}
@itemx -f @var{file}
-When @option{--lockfile} is specified, importer will ignore other
-options and won't output package definitions, instead generating source
-definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
-For example:
+When @option{--lockfile} is specified, the importer will ignore other options
+and won't output package definitions, instead generating source definitions for
+all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
@example
guix import crate --lockfile=/path/to/Cargo.lock my-package
@end example
-generates sources from @file{/path/to/Cargo.lock}, and a list
-@code{my-package-cargo-inputs} referencing these sources. The generated
-list is intended for package's base @code{inputs}, replacing
-@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
+generates sources from @file{/path/to/Cargo.lock} and a list
+@code{my-package-cargo-inputs} referencing these sources. The generated list is
+intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
+@code{#:cargo-development-inputs}.
@end table
@item elm
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index b547421b88..41adc03752 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -248,12 +248,6 @@ (define* (configure #:key inputs
(setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
(setenv "SODIUM_USE_PKG_CONFIG" "1")
(setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
- ;; This flag is needed when not using the bundled jemalloc.
- ;; https://github.com/tikv/jemallocator/issues/19
- (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
- (when (assoc-ref inputs "jemalloc")
- (setenv "JEMALLOC_OVERRIDE"
- (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so")))
(when (assoc-ref inputs "openssl")
(setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
(when (assoc-ref inputs "gettext")
@@ -356,23 +350,17 @@ (define* (package #:key
(if (null? cargo-package-crates)
(apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
- ;; Use unstable feature ‘--registry’.
(begin
- (setenv "RUSTC_BOOTSTRAP" "1")
(for-each
(lambda (pkg)
(apply invoke "cargo" "package" "--offline" "--package" pkg
- "--registry" "crates-io" "-Z" "package-workspace"
cargo-package-flags)
(for-each
(lambda (crate)
(invoke "tar" "xzf" crate "-C" vendor-dir))
- (begin
- (delete-file-recursively "target/package/tmp-registry")
- (find-files "target/package" "\\.crate$")))
+ (find-files "target/package" "\\.crate$"))
(patch-cargo-checksums #:vendor-dir vendor-dir))
- cargo-package-crates)
- (unsetenv "RUSTC_BOOTSTRAP")))
+ cargo-package-crates)))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index d8e1ed3e6f..f6e2985ed9 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -126,13 +126,12 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let* ((definer?
- (cut member
- <>
- `(,@(if (member importer '("crate"))
- '(define)
- '())
- define-public)))
+ (let* ((importer-definers
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public))
+ (definer? (cut member <> importer-definers))
(find-and-insert
(lambda (expr)
(match expr
diff --git a/tests/crate.scm b/tests/crate.scm
index d46f753f9c..9b7066c3b1 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -478,7 +478,7 @@ (define rust-leaf-bob-3.0.2-yanked
(license #f)))
(define temp-file
- (string-append "t-utils-" (number->string (getpid))))
+ (string-append "t-crate-" (number->string (getpid))))
(test-begin "crate")
--8<---------------cut here---------------end--------------->8---
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:28:10 GMT) Full text and rfc822 format available.Message #107 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 02/17] build/cargo: Print out all non-empty binary files. Date: Tue, 18 Mar 2025 22:26:45 +0800
* guix/build/cargo-build-system.scm (%standard-phases): Move
'unpack-rust-crates after 'unpack.
Move 'check-for-pregenerated-files after 'configure.
(check-for-pregenerated-files): Only check non-empty files.
Print out binary files.
Run in parallel.
Don't fail to keep compatibility for phase order change.
Change-Id: I0a332fe843e97687324bd908fa111422a63e475d
---
guix/build/cargo-build-system.scm | 31 +++++++++++++++++++++++++------
1 file changed, 25 insertions(+), 6 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41f54f42b7..58238992b8 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -29,9 +29,11 @@ (define-module (guix build cargo-build-system)
#:use-module ((guix build utils) #:hide (delete))
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (%standard-phases
@@ -111,12 +113,29 @@ (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
(define (rust-package? name)
(string-prefix? "rust-" name))
-(define* (check-for-pregenerated-files #:rest _)
+(define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
"Check the source code for files which are known to generally be bundled
libraries or executables."
- (let ((pregenerated-files (find-files "." "\\.(a|dll|dylib|exe|lib)$")))
- (when (not (null-list? pregenerated-files))
- (error "Possible pre-generated files found:" pregenerated-files))))
+ (format #t "Searching for binary files...~%")
+ (let ((known-pattern (make-regexp "\\.(a|dll|dylib|exe|lib)$"))
+ (empty-file?
+ (lambda (filename)
+ (call-with-ascii-input-file filename
+ (lambda (p)
+ (eqv? #\0 (read-char p)))))))
+ (n-par-for-each
+ (if parallel-build?
+ (parallel-job-count)
+ 1)
+ (lambda (file)
+ (unless (empty-file? file)
+ ;; Print out binary files.
+ (false-if-exception (invoke "grep" "-IL" "." file))
+ ;; Warn about known pre-generated files.
+ ;; Not failing here for compatibility with existing packages.
+ (when (regexp-exec known-pattern file)
+ (format #t "error: Possible pre-generated file found: ~a~%" file))))
+ (find-files "."))))
(define* (configure #:key inputs
target system
@@ -380,8 +399,8 @@ (define %standard-phases
(replace 'check check)
(replace 'install install)
(add-after 'build 'package package)
- (add-after 'unpack 'check-for-pregenerated-files check-for-pregenerated-files)
- (add-after 'check-for-pregenerated-files 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'unpack 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'configure 'check-for-pregenerated-files check-for-pregenerated-files)
(add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
(define* (cargo-build #:key inputs (phases %standard-phases)
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:04 GMT) Full text and rfc822 format available.Message #110 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 05/17] build/cargo: Set default value of arguments for build phases. Date: Tue, 18 Mar 2025 22:26:48 +0800
This makes it easier to use these phases in other bulid systems.
* guix/build/cargo-build-system.scm (unpack-rust-crates,configure,build,package)
(install): Set default value of arguments.
Change-Id: I1dde1b063d8eee57967903abd2fce94574211a0a
---
guix/build/cargo-build-system.scm | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 85f10a7ac4..85a9698680 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -76,7 +76,8 @@ (define (crate-src? path)
" | cut -d/ -f2"
" | grep -q '^Cargo.toml$'")))))
-(define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
+(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
+ #:allow-other-keys)
(define (inputs->rust-inputs inputs)
"Filter using the label part from INPUTS."
(filter (lambda (input)
@@ -139,7 +140,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(define* (configure #:key inputs
target system
- cargo-target
+ (cargo-target #f)
(vendor-dir "guix-vendor")
#:allow-other-keys)
"Vendor Cargo.toml dependencies as guix inputs."
@@ -258,7 +259,7 @@ (define* (patch-cargo-checksums #:key
(define* (build #:key
parallel-build?
- skip-build?
+ (skip-build? #f)
(features '())
(cargo-build-flags '("--release"))
#:allow-other-keys)
@@ -298,8 +299,8 @@ (define* (check #:key
(define* (package #:key
source
- skip-build?
- install-source?
+ (skip-build? #f)
+ (install-source? #t)
(cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
(vendor-dir "guix-vendor")
@@ -375,9 +376,9 @@ (define* (package #:key
(define* (install #:key
inputs
outputs
- skip-build?
- install-source?
- features
+ (skip-build? #f)
+ (install-source? #t)
+ (features '())
(cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:05 GMT) Full text and rfc822 format available.Message #113 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 03/17] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 18 Mar 2025 22:26:46 +0800
* guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-package-crates]: New argument.
* guix/build/cargo-build-system.scm (package): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
---
doc/guix.texi | 7 +++++++
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 22 ++++++++++++++++++++--
3 files changed, 31 insertions(+), 2 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0ca109a214..0089f9e60f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9363,6 +9363,13 @@ Build Systems
the binaries defined by the crate. Unless @code{install-source? #f} is
defined it will also install a source crate repository of itself and unpacked
sources, to ease in future hacking on rust packages.
+
+This build system supports cargo workspaces. Parameter
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+library crates to package in the @code{package} phase. Specified crates are
+packaged from left to right, in case there's dependency among them. For
+example, specifying @code{''("pcre2-sys" "pcre2")} will package
+@code{"pcre2-sys"} first and then @code{"pcre2"}.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 452f7f78d0..4f6d46e70c 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -94,6 +94,7 @@ (define* (cargo-build name inputs
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(features ''())
(skip-build? #f)
@@ -122,6 +123,7 @@ (define* (cargo-build name inputs
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
@@ -154,6 +156,7 @@ (define* (cargo-cross-build name
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(cargo-target (cargo-triplet (or target system)))
(features ''())
@@ -185,6 +188,7 @@ (define* (cargo-cross-build name
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 58238992b8..979f93aef4 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -300,11 +300,16 @@ (define* (package #:key
source
skip-build?
install-source?
+ (cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
+ (vendor-dir "guix-vendor")
#:allow-other-keys)
"Run 'cargo-package' for a given Cargo package."
(if install-source?
- (if skip-build?
+ ;; NOTE: Cargo workspace packaging support:
+ ;; #:install-source? #t + #:cargo-package-crates.
+ (if (and (null? cargo-package-crates)
+ skip-build?)
(begin
(install-file source "target/package")
(with-directory-excursion "target/package"
@@ -322,7 +327,20 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+
+ (if (null? cargo-package-crates)
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+ (begin
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (find-files "target/package" "\\.crate$"))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates)))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:05 GMT) Full text and rfc822 format available.Message #116 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 04/17] build-system: cargo: Support installing Cargo workspace. Date: Tue, 18 Mar 2025 22:26:47 +0800
*guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-install-paths]: New argument.
* guix/build/cargo-build-system.scm (install): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
---
doc/guix.texi | 5 ++++-
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 17 +++++++++++++----
3 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0089f9e60f..7fa986b4b9 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9369,7 +9369,10 @@ Build Systems
library crates to package in the @code{package} phase. Specified crates are
packaged from left to right, in case there's dependency among them. For
example, specifying @code{''("pcre2-sys" "pcre2")} will package
-@code{"pcre2-sys"} first and then @code{"pcre2"}.
+@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
+binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
+for example.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4f6d46e70c..4486c706a1 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -96,6 +96,7 @@ (define* (cargo-build name inputs
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(features ''())
(skip-build? #f)
(parallel-build? #t)
@@ -125,6 +126,7 @@ (define* (cargo-build name inputs
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
@@ -158,6 +160,7 @@ (define* (cargo-cross-build name
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(cargo-target (cargo-triplet (or target system)))
(features ''())
(skip-build? #f)
@@ -190,6 +193,7 @@ (define* (cargo-cross-build name
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 979f93aef4..85f10a7ac4 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -378,6 +378,7 @@ (define* (install #:key
skip-build?
install-source?
features
+ (cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
(let* ((out (assoc-ref outputs "out"))
@@ -392,10 +393,18 @@ (define* (install #:key
;; Only install crates which include binary targets,
;; otherwise cargo will raise an error.
(or skip-build?
- (not (has-executable-target?))
- (invoke "cargo" "install" "--offline" "--no-track"
- "--path" "." "--root" out
- "--features" (string-join features)))
+ ;; NOTE: Cargo workspace installation support:
+ ;; #:skip-build? #f + #:cargo-install-paths.
+ (and (null? cargo-install-paths)
+ (not (has-executable-target?)))
+ (for-each
+ (lambda (path)
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" path "--root" out
+ "--features" (string-join features)))
+ (if (null? cargo-install-paths)
+ '(".")
+ cargo-install-paths)))
(when install-source?
;; Install crate tarballs and unpacked sources for later use.
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:06 GMT) Full text and rfc822 format available.Message #119 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 06/17] build/cargo: Don't try to unpack sanity-check.py. Date: Tue, 18 Mar 2025 22:26:49 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (crate-src?): Also don't try to
check python scripts for Cargo.toml.
Change-Id: I001a89b83d2e472706b1263007be45d1153c140f
---
guix/build/cargo-build-system.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 85a9698680..32a756ad99 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -64,6 +64,7 @@ (define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
(and (not (directory-exists? path)) ; not a tarball
+ (not (string-suffix? "py" path)) ; sanity-check.py
;; First we print out all file names within the tarball to see if it
;; looks like the source of a crate. However, the tarball will include
;; an extra path component which we would like to ignore (since we're
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:06 GMT) Full text and rfc822 format available.Message #122 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 07/17] build/cargo: Support non-workspace directory source inputs. Date: Tue, 18 Mar 2025 22:26:50 +0800
* guix/build/cargo-build-system.scm (single-crate?): New procedure.
(crate-src?): Support non-workspace directory source inputs.
(configure): Likewise.
Change-Id: If2bd318b44ed765baec69309f25d320edceee116
---
guix/build/cargo-build-system.scm | 60 ++++++++++++++++++++-----------
1 file changed, 39 insertions(+), 21 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 32a756ad99..41adc03752 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -33,6 +33,7 @@ (define-module (guix build cargo-build-system)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 textual-ports)
#:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -60,22 +61,35 @@ (define (has-executable-target?)
(bin-dep? (lambda (dep) (find bin? (get-kinds dep)))))
(find bin-dep? (manifest-targets))))
+(define (single-crate? dir)
+ "Check if directory DIR contains 'Cargo.toml' and is not a workspace."
+ (let ((manifest-file (in-vicinity dir "Cargo.toml")))
+ (and (file-exists? manifest-file)
+ (not (string-contains
+ (call-with-input-file manifest-file get-string-all)
+ "[workspace]")))))
+
(define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
- (and (not (directory-exists? path)) ; not a tarball
- (not (string-suffix? "py" path)) ; sanity-check.py
- ;; First we print out all file names within the tarball to see if it
- ;; looks like the source of a crate. However, the tarball will include
- ;; an extra path component which we would like to ignore (since we're
- ;; interested in checking if a Cargo.toml exists at the root of the
- ;; archive, but not nested anywhere else). We do this by cutting up
- ;; each output line and only looking at the second component. We then
- ;; check if it matches Cargo.toml exactly and short circuit if it does.
- (apply invoke (list "sh" "-c"
- (string-append "tar -tf " path
- " | cut -d/ -f2"
- " | grep -q '^Cargo.toml$'")))))
+ (if (directory-exists? path)
+ ;; The build system only handles sources containing single crate.
+ ;; Workspaces should be packaged into crates (via 'package phase)
+ ;; and used in inputs.
+ (single-crate? path)
+ (and (not (string-suffix? "py" path)) ;sanity-check.py
+ ;; First we print out all file names within the tarball to see
+ ;; if it looks like the source of a crate. However, the tarball
+ ;; will include an extra path component which we would like to
+ ;; ignore (since we're interested in checking if a Cargo.toml
+ ;; exists at the root of the archive, but not nested anywhere
+ ;; else). We do this by cutting up each output line and only
+ ;; looking at the second component. We then check if it matches
+ ;; Cargo.toml exactly and short circuit if it does.
+ (invoke "sh" "-c"
+ (string-append "tar -tf " path
+ " | cut -d/ -f2"
+ " | grep -q '^Cargo.toml$'")))))
(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
#:allow-other-keys)
@@ -159,14 +173,18 @@ (define* (configure #:key inputs
(and (crate-src? path)
;; Gracefully handle duplicate inputs
(not (file-exists? crate-dir))
- (mkdir-p crate-dir)
- ;; Cargo crates are simply gzipped tarballs but with a .crate
- ;; extension. We expand the source to a directory name we control
- ;; so that we can generate any cargo checksums.
- ;; The --strip-components argument is needed to prevent creating
- ;; an extra directory within `crate-dir`.
- (format #t "Unpacking ~a~%" name)
- (invoke "tar" "xf" path "-C" crate-dir "--strip-components" "1")))))
+ (if (directory-exists? path)
+ (copy-recursively path crate-dir)
+ (begin
+ (mkdir-p crate-dir)
+ ;; Cargo crates are simply gzipped tarballs but with a
+ ;; .crate extension. We expand the source to a directory
+ ;; name we control so that we can generate any cargo
+ ;; checksums. The --strip-components argument is needed to
+ ;; prevent creating an extra directory within `crate-dir`.
+ (format #t "Unpacking ~a~%" name)
+ (invoke "tar" "xf" path "-C" crate-dir
+ "--strip-components" "1")))))))
inputs)
;; For cross-building
--
2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:08 GMT) Full text and rfc822 format available.Message #125 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 08/17] scripts: import: Document argument for ‘--insert’ option in help message. Date: Tue, 18 Mar 2025 22:26:51 +0800
* guix/scripts/import.scm (show-help): Add missing ‘FILE’. Change-Id: I540d5feae3fe49c00e9bd6f7a8649ffe0d6e006d --- guix/scripts/import.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index bbf31baa15..4fad329cd3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -69,7 +69,7 @@ (define (show-help) (display (G_ " -h, --help display this help and exit")) (display (G_ " - -i, --insert insert packages into file alphabetically")) + -i, --insert=FILE insert packages into FILE alphabetically")) (display (G_ " -V, --version display version information and exit")) (newline) -- 2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:03 GMT) Full text and rfc822 format available.Message #128 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 09/17] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 18 Mar 2025 22:26:52 +0800
For consistency with ‘insert-expression’.
* guix/scripts/import.scm (guix-import): Add two newlines when inserting.
Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d
---
guix/scripts/import.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 4fad329cd3..58a84d0db7 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -129,6 +129,7 @@ (define-command (guix-import . args)
(let ((port (open-file file "a")))
(pretty-print-with-comments port expr)
(newline port)
+ (newline port)
(close-port port)))))))))
(import-as-definitions importer args find-and-insert)))
((importer args ...)
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:05 GMT) Full text and rfc822 format available.Message #131 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 14/17] build-system: cargo: Add ‘cargo-inputs’. Date: Tue, 18 Mar 2025 22:26:57 +0800
* gnu/packages/rust-crates.scm: New file.
* gnu/packages/rust-sources.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them.
* guix/build-system/cargo.scm (crate-source,cargo-inputs): New procedures.
* guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo).
* etc/teams/rust/cleanup-crates.sh: New file.
* etc/teams/rust/rust-crates.tmpl: New file.
Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3
---
etc/teams/rust/cleanup-crates.sh | 38 +++++++++++++++++++++++++++
etc/teams/rust/rust-crates.tmpl | 42 +++++++++++++++++++++++++++++
gnu/local.mk | 2 ++
gnu/packages/rust-crates.scm | 42 +++++++++++++++++++++++++++++
gnu/packages/rust-sources.scm | 29 ++++++++++++++++++++
guix/build-system/cargo.scm | 45 +++++++++++++++++++++++++++++++-
guix/import/crate.scm | 2 +-
7 files changed, 198 insertions(+), 2 deletions(-)
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
new file mode 100755
index 0000000000..cd4c2462b9
--- /dev/null
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+FILE=gnu/packages/rust-crates.scm
+PATTERN='^(define rust-'
+
+grep "$PATTERN" $FILE | cut -d' ' -f2 | while IFS= read -r crate
+do
+ if [ "$(grep -wc "$crate" $FILE)" -eq 1 ]; then
+ echo "\
+(begin
+ (use-modules (guix utils))
+ (let ((source-properties
+ (find-definition-location \"$FILE\" '$crate #:definer 'define)))
+ (and=> source-properties delete-expression)))" |
+ guix repl -t machine
+ fi
+done
+
+# Delete extra newlines.
+sed --in-place ':a;N;$!ba;s/\n\n\+/\n\n/g' $FILE
diff --git a/etc/teams/rust/rust-crates.tmpl b/etc/teams/rust/rust-crates.tmpl
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/etc/teams/rust/rust-crates.tmpl
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/local.mk b/gnu/local.mk
index 02de02e65f..8d8bd6ebbe 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -616,6 +616,8 @@ GNU_SYSTEM_MODULES = \
%D%/packages/rush.scm \
%D%/packages/rust.scm \
%D%/packages/rust-apps.scm \
+ %D%/packages/rust-crates.scm \
+ %D%/packages/rust-sources.scm \
%D%/packages/samba.scm \
%D%/packages/sagemath.scm \
%D%/packages/sawfish.scm \
diff --git a/gnu/packages/rust-crates.scm b/gnu/packages/rust-crates.scm
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/gnu/packages/rust-crates.scm
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/packages/rust-sources.scm b/gnu/packages/rust-sources.scm
new file mode 100644
index 0000000000..bf9b91a671
--- /dev/null
+++ b/gnu/packages/rust-sources.scm
@@ -0,0 +1,29 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-sources)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; Cargo workspaces and Rust dependencies requiring external inputs to
+;;; unbundle.
+;;;
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 7a07003262..47d4f10969 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -31,6 +31,7 @@ (define-module (guix build-system cargo)
#:use-module (guix gexp)
#:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix download)
#:use-module (guix platform)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
@@ -45,7 +46,9 @@ (define-module (guix build-system cargo)
crate-url
crate-url?
crate-uri
- crate-name->package-name))
+ crate-name->package-name
+ crate-source
+ cargo-inputs))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -62,6 +65,46 @@ (define (crate-uri name version)
(define (crate-name->package-name name)
(downstream-package-name "rust-" name))
+(define* (crate-source name version hash #:key (patches '()) (snippet #f))
+ (origin
+ (method url-fetch)
+ (uri (crate-uri name version))
+ (file-name
+ (string-append (crate-name->package-name name) "-" version ".tar.gz"))
+ (sha256 (base32 hash))
+ (modules '((guix build utils)))
+ (patches patches)
+ (snippet snippet)))
+
+(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
+ (sources-module '(gnu packages rust-sources)))
+ "Given NAME, resolve input list 'NAME-cargo-inputs' in CRATES-MODULE, return
+its copy with #f removed and symbols resolved to variables defined in
+SOURCES-MODULE, if the list exists, otherwise return an empty list."
+ (let loop ((inputs
+ (catch #t
+ (lambda ()
+ (module-ref (resolve-interface crates-module)
+ (symbol-append name '-cargo-inputs)))
+ (const '())))
+ (result '()))
+ (if (null? inputs)
+ result
+ (match inputs
+ ((input . rest)
+ (loop rest
+ (cond
+ ;; #f, remove it.
+ ((not input)
+ result)
+ ;; Symbol, resolve it in SOURCES-MODULE.
+ ((symbol? input)
+ (cons (module-ref (resolve-interface sources-module) input)
+ result))
+ ;; Else: no change.
+ (else
+ (cons input result)))))))))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 14e6e28c5b..a6f247bbae 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -27,7 +27,7 @@
(define-module (guix import crate)
#:use-module (guix base32)
- #:use-module (guix build-system cargo)
+ #:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:07 GMT) Full text and rfc822 format available.Message #134 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Murilo <murilo <at> disroot.org>, Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il>, Luis Guilherme Coelho <lgcoelho <at> disroot.org> Subject: [PATCH rust-team v2 15/17] import: crate: Add Cargo.lock parser. Date: Tue, 18 Mar 2025 22:26:58 +0800
* guix/import/crate/cargo-lock.scm: New file.
* Makefile.am (MODULES): Regisiter it.
* guix/import/crate.scm (cargo-lock->definitions): New procedure.
Co-authored-by: Murilo <murilo <at> disroot.org>
Co-authored-by: Luis Guilherme Coelho <lgcoelho <at> disroot.org>
Change-Id: I95421e9e2ba11a671b4bc4e1323c6d31a1b012c5
---
Makefile.am | 1 +
guix/import/crate.scm | 78 +++++++++++++++++++++++
guix/import/crate/cargo-lock.scm | 105 +++++++++++++++++++++++++++++++
3 files changed, 184 insertions(+)
create mode 100644 guix/import/crate/cargo-lock.scm
diff --git a/Makefile.am b/Makefile.am
index 4091a52387..a04e461ea8 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -295,6 +295,7 @@ MODULES = \
guix/import/cpan.scm \
guix/import/cran.scm \
guix/import/crate.scm \
+ guix/import/crate/cargo-lock.scm \
guix/import/egg.scm \
guix/import/elm.scm \
guix/import/elpa.scm \
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index a6f247bbae..ef3a6d0b4b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -9,6 +9,9 @@
;;; Copyright © 2023, 2024 David Elsing <david.elsing <at> posteo.net>
;;; Copyright © 2024 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;; Copyright © 2025 Herman Rimm <herman <at> rimm.ee>
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024-2025 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -26,12 +29,14 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix import crate)
+ #:use-module (guix base16)
#:use-module (guix base32)
#:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
#:use-module (guix i18n)
+ #:use-module (guix import crate cargo-lock)
#:use-module (guix import json)
#:use-module (guix import utils)
#:use-module (guix memoization)
@@ -39,9 +44,11 @@ (define-module (guix import crate)
#:use-module (guix read-print)
#:use-module (guix upstream)
#:use-module (guix utils)
+ #:use-module (guix scripts download)
#:use-module (gnu packages)
#:use-module (ice-9 match)
#:use-module (ice-9 regex)
+ #:use-module (ice-9 textual-ports)
#:use-module (json)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
@@ -52,6 +59,7 @@ (define-module (guix import crate)
guix-package->crate-name
string->license
crate-recursive-import
+ cargo-lock->definitions
%crate-updater))
@@ -481,6 +489,76 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
+
+;;;
+;;; Convert ‘Cargo.lock’ to Guix sources.
+;;;
+
+(define (cargo-lock->definitions lockfile package-name)
+ "Given LOCKFILE, a 'Cargo.lock' file, convert its content into source
+definitions. Return a list of sources and 'PACKAGE-NAME-cargo-inputs', an input
+list referencing all imported sources."
+ (define (crate->definition crate)
+ (match crate
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source _)
+ ('crate-checksum checksum))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,@(if (or (string-suffix? "src" name)
+ (string-suffix? "sys" name))
+ (list (comment ";; TODO: Check bundled sources.\n" #f))
+ '())
+ (crate-source ,name ,version
+ ,(bytevector->nix-base32-string
+ (base16-string->bytevector checksum)))))
+ ;; Git snapshot.
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source source))
+ (begin
+ (let* ((src (string-split source (char-set #\+ #\? #\#)))
+ (url (second src))
+ (commit (last src))
+ (version (string-append version "." (string-take commit 7)))
+ (checksum
+ (second
+ (string-split
+ (with-output-to-string
+ (lambda _
+ (guix-download "-g" url
+ (string-append "--commit=" commit))))
+ #\newline))))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,(comment
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url ,url)
+ (commit ,commit)))
+ (file-name
+ (git-file-name ,(crate-name->package-name name) ,version))
+ (sha256 (base32 ,checksum)))))))
+ ;; Workspace member.
+ (else #f)))
+
+ (let ((definitions
+ (filter-map crate->definition
+ (cargo-lock-string->scm
+ (call-with-input-file lockfile get-string-all)))))
+ (values definitions
+ `(define-public
+ ,(string->symbol (string-append package-name "-cargo-inputs"))
+ (list ,@(map second definitions))))))
+
;;;
;;; Updater
diff --git a/guix/import/crate/cargo-lock.scm b/guix/import/crate/cargo-lock.scm
new file mode 100644
index 0000000000..b8a6ab20c0
--- /dev/null
+++ b/guix/import/crate/cargo-lock.scm
@@ -0,0 +1,105 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import crate cargo-lock)
+ #:use-module (ice-9 peg)
+ #:export (cargo-lock-string->scm
+
+ crate-name
+ crate-version
+ crate-source
+ crate-checksum
+ crate-dependencies
+ cargo-lock))
+
+;;;
+;;; PEG parser for ‘Cargo.lock’.
+;;;
+
+(define (cargo-lock-string->scm str)
+ (peg:tree (search-for-pattern cargo-lock str)))
+
+;; Auxiliar peg patterns
+(define-peg-pattern numeric-char body
+ (range #\0 #\9))
+
+(define-peg-pattern lowercase-char body
+ (range #\a #\z))
+
+(define-peg-pattern uppercase-char body
+ (range #\A #\Z))
+
+(define-peg-pattern alphabetic-char body
+ (or lowercase-char uppercase-char))
+
+(define-peg-pattern alphanumeric-char body
+ (or alphabetic-char numeric-char))
+
+;; name
+(define-peg-pattern crate-name all
+ (+ (or "-" alphabetic-char
+ "_" numeric-char)))
+
+;; version
+(define-peg-pattern non-negative-integer body
+ (+ numeric-char))
+
+(define-peg-pattern crate-version all
+ (and non-negative-integer "."
+ non-negative-integer "."
+ non-negative-integer
+ (? (+ (or "-" lowercase-char
+ "." uppercase-char
+ "+" numeric-char "_")))))
+
+;; source
+(define-peg-pattern crate-source all
+ (and (or "registry" "git")
+ "+https://"
+ (+ (or "/" "." "?" "=" "-" "#" "_"
+ alphanumeric-char))))
+
+;; checksum
+(define-peg-pattern crate-checksum all
+ (+ (or lowercase-char numeric-char)))
+
+;; dependency specification
+(define-peg-pattern dependency-specification all
+ (and crate-name (? (and (ignore " ") crate-version))))
+
+;; dependencies
+(define-peg-pattern crate-dependencies all
+ (and (ignore "[\n")
+ (+ (and (ignore " \"")
+ (capture dependency-specification)
+ (ignore "\",\n")))
+ (ignore "]")))
+
+;; crates
+(define-peg-pattern crate all
+ (and (ignore "[[package]]\n")
+ (ignore "name = \"") (capture crate-name) (ignore "\"\n")
+ (ignore "version = \"") (capture crate-version) (ignore "\"\n")
+ (? (and (ignore "source = \"") (capture crate-source) (ignore "\"\n")))
+ (? (and (ignore "checksum = \"") (capture crate-checksum) (ignore "\"\n")))
+ (? (ignore (and "dependencies = " crate-dependencies "\n")))))
+
+;; Cargo.lock
+(define-peg-pattern cargo-lock all
+ (+ (and (ignore "\n") crate)))
--
2.48.1
guix <at> cbaines.net, divya <at> subvertising.org, efraim <at> flashner.co.il, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, maxim.cournoyer <at> gmail.com, zimon.toutoune <at> gmail.com, steve <at> futurile.net, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:11 GMT) Full text and rfc822 format available.Message #137 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 16/17] import: crate: Add ‘--lockfile’ option. Date: Tue, 18 Mar 2025 22:26:59 +0800
* guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
* guix/scripts/import/crate.scm (show-help): Add it.
(guix-import-crate): Use it.
* doc/guix.texi (Invoking guix import): Document it.
* tests/crate.scm (temp-file): New variable.
("crate-lockfile-import"): New test.
Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
---
doc/guix.texi | 14 ++++++
guix/scripts/import/crate.scm | 58 +++++++++++++++++++----
tests/crate.scm | 88 +++++++++++++++++++++++++++++++++++
3 files changed, 150 insertions(+), 10 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 7fa986b4b9..218c2ba630 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14703,6 +14703,20 @@ Invoking guix import
If a crate dependency is not (yet) packaged, make the corresponding
input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
a comment.
+@item --lockfile=@var{file}
+@itemx -f @var{file}
+When @option{--lockfile} is specified, the importer will ignore other options
+and won't output package definitions, instead generating source definitions for
+all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
+
+@example
+guix import crate --lockfile=/path/to/Cargo.lock my-package
+@end example
+
+generates sources from @file{/path/to/Cargo.lock} and a list
+@code{my-package-cargo-inputs} referencing these sources. The generated list is
+intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
+@code{#:cargo-development-inputs}.
@end table
@item elm
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 723cbb3665..3b536e135a 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -25,11 +25,13 @@
(define-module (guix scripts import crate)
#:use-module (guix ui)
#:use-module (guix utils)
+ #:use-module (guix read-print)
#:use-module (guix scripts)
#:use-module (guix import crate)
#:use-module (guix scripts import)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-26)
#:use-module (srfi srfi-37)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
@@ -60,6 +62,9 @@ (define (show-help)
sufficient package exists for it"))
(newline)
(display (G_ "
+ -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
+ (newline)
+ (display (G_ "
-h, --help display this help and exit"))
(display (G_ "
-V, --version display version information and exit"))
@@ -87,6 +92,9 @@ (define %options
(option '("mark-missing") #f #f
(lambda (opt name arg result)
(alist-cons 'mark-missing #t result)))
+ (option '(#\f "lockfile") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'lockfile arg result)))
%standard-import-options))
@@ -101,6 +109,8 @@ (define (guix-import-crate . args)
#:build-options? #f))
(let* ((opts (parse-options))
+ (lockfile (assoc-ref opts 'lockfile))
+ (file-to-insert (assoc-ref opts 'file-to-insert))
(args (filter-map (match-lambda
(('argument . value)
value)
@@ -111,16 +121,44 @@ (define (guix-import-crate . args)
(define-values (name version)
(package-name->name+version spec))
- (match (if (assoc-ref opts 'recursive)
- (crate-recursive-import
- name #:version version
- #:recursive-dev-dependencies?
- (assoc-ref opts 'recursive-dev-dependencies)
- #:allow-yanked? (assoc-ref opts 'allow-yanked))
- (crate->guix-package
- name #:version version #:include-dev-deps? #t
- #:allow-yanked? (assoc-ref opts 'allow-yanked)
- #:mark-missing? (assoc-ref opts 'mark-missing)))
+ (match (cond
+ ((and=> lockfile
+ (lambda (file)
+ (or (file-exists? file)
+ (leave (G_ "file '~a' does not exist~%") file))))
+ (let-values (((crate-definitions input-list)
+ (cargo-lock->definitions lockfile name)))
+ (if file-to-insert
+ ;; Adjusted from ‘--insert’ option of guix-import.
+ (let ((term (second input-list)))
+ (begin
+ ;; Remove existing input list definition.
+ (and=> (find-definition-location file-to-insert term)
+ delete-expression)
+ ;; Insert input list alphabetically.
+ (or (and=> (find-definition-insertion-location
+ file-to-insert term)
+ (cut insert-expression <> input-list))
+ (let ((port (open-file file-to-insert "a")))
+ (newline port)
+ (pretty-print-with-comments port input-list)
+ (newline port)
+ (newline port)
+ (close-port port))))
+ crate-definitions)
+ `(,@crate-definitions
+ ,input-list))))
+ ((assoc-ref opts 'recursive)
+ (crate-recursive-import
+ name #:version version
+ #:recursive-dev-dependencies?
+ (assoc-ref opts 'recursive-dev-dependencies)
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)))
+ (else
+ (crate->guix-package
+ name #:version version #:include-dev-deps? #t
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)
+ #:mark-missing? (assoc-ref opts 'mark-missing))))
((or #f '())
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
diff --git a/tests/crate.scm b/tests/crate.scm
index d0dc779cd2..9b7066c3b1 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -34,6 +34,7 @@ (define-module (test-crate)
#:use-module (gnu packages)
#:use-module (ice-9 iconv)
#:use-module (ice-9 match)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-64))
@@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
(description #f)
(license #f)))
+(define temp-file
+ (string-append "t-crate-" (number->string (getpid))))
+
(test-begin "crate")
@@ -1178,4 +1182,88 @@ (define rust-leaf-bob-3.0.2-yanked
(x
(pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
+
+(test-assert "crate-lockfile-import"
+ (begin
+ (call-with-output-file temp-file
+ (lambda (port)
+ (display "\
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = \"adler2\"
+version = \"2.0.0\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
+
+[[package]]
+name = \"aho-corasick\"
+version = \"1.1.3\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
+dependencies = [
+ \"memchr\",
+]
+
+[[package]]
+name = \"smithay\"
+version = \"0.4.0\"
+source = \"git+https://github.com/Smithay/smithay.git?rev=\
+0cd3345c59f7cb139521f267956a1a4e33248393#\
+0cd3345c59f7cb139521f267956a1a4e33248393\"
+dependencies = [
+ \"appendlist\",
+]
+
+[[package]]
+name = \"test\"
+version = \"25.2.0\"\n" port)))
+ (mock
+ ((guix scripts download) guix-download
+ (lambda _
+ (format #t "~a~%~a~%"
+ "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
+ (let-values
+ (((crates-definitions input-list)
+ (cargo-lock->definitions temp-file "test")))
+ (and
+ (match crates-definitions
+ (((define 'rust-adler2-2.0.0
+ (crate-source
+ "adler2" "2.0.0"
+ "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
+ (define 'rust-aho-corasick-1.1.3
+ (crate-source
+ "aho-corasick" "1.1.3"
+ "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
+ (define 'rust-smithay-0.4.0.0cd3345
+ ($ <comment>
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/Smithay/smithay.git")
+ (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
+ (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
+ (sha256
+ (base32
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
+ #t)
+ (x
+ (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
+ (match input-list
+ ((define-public 'test-cargo-inputs
+ (list rust-adler2-2.0.0
+ rust-aho-corasick-1.1.3
+ rust-smithay-0.4.0.0cd3345))
+ #t)
+ (x
+ (pk 'fail x #f))))))))
+
(test-end "crate")
+
+(false-if-exception (delete-file temp-file))
--
2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:15 GMT) Full text and rfc822 format available.Message #140 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 10/17] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 22:26:53 +0800
* guix/utils.scm (find-definition-location): New procedure.
(find-definition-insertion-location): Define with it.
* guix/scripts/import.scm (import-as-definitions, guix-import): Support
expressions defined by 'define.
Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
---
guix/scripts/import.scm | 19 ++++++++++++++-----
guix/utils.scm | 27 +++++++++++++++++++--------
2 files changed, 33 insertions(+), 13 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 58a84d0db7..e53028e747 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -30,6 +30,7 @@ (define-module (guix scripts import)
#:use-module (guix read-print)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))
((expressions ...)
(for-each (lambda (expr)
@@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))))
expressions))
(x
@@ -117,13 +120,19 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let ((find-and-insert
+ (let* ((importer-definers
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public))
+ (definer? (cut member <> importer-definers))
+ (find-and-insert
(lambda (expr)
(match expr
- (('define-public term _ ...)
+ (((? definer? definer) term _ ...)
(let ((source-properties
(find-definition-insertion-location
- file term)))
+ file term #:definer definer)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index c7c23d9d5b..77ec6d992a 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -154,6 +154,7 @@ (define-module (guix utils)
edit-expression
delete-expression
insert-expression
+ find-definition-location
find-definition-insertion-location
filtered-port
@@ -520,24 +521,34 @@ (define (insert-expression source-properties expr)
(string-append expr "\n\n" str))))
(edit-expression source-properties insert)))
-(define (find-definition-insertion-location file term)
- "Search in FILE for a top-level public definition whose defined term
-alphabetically succeeds TERM. Return the location if found, or #f
-otherwise."
- (let ((search-term (symbol->string term)))
+(define* (find-definition-location file term
+ #:key (definer 'define-public)
+ (pred string=))
+ "Search in FILE for a top-level definition defined by DEFINER with defined
+term comparing to TERM through PRED. Return the location if PRED returns #t,
+or #f otherwise."
+ (let ((search-term (symbol->string term))
+ (definer? (cut eq? definer <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (('define-public current-term _ ...)
- (string> (symbol->string current-term)
- search-term))
+ (((? definer?) current-term _ ...)
+ (pred (symbol->string current-term)
+ search-term))
((? eof-object?) #t)
(_ #f))
(and (not (eof-object? syntax))
(syntax-source syntax))))))))
+(define* (find-definition-insertion-location file term
+ #:key (definer 'define-public))
+ "Search in FILE for a top-level definition defined by DEFINER with defined
+term alphabetically succeeds TERM. Return the location if found, or #f
+otherwise."
+ (find-definition-location file term #:definer definer #:pred string>))
+
;;;
;;; Keyword arguments.
--
2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:17 GMT) Full text and rfc822 format available.Message #143 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 11/17] scripts: import: Pass "--insert" to importers. Date: Tue, 18 Mar 2025 22:26:54 +0800
So that importers can adjust the file before inserting.
* guix/scripts/import.scm (%standard-import-options): Add ‘--file-to-insert’.
(guix-import): Pass it to importers when ‘--insert’ is set.
Change-Id: I8e7a18ee8e0f96d7fc5688a207a7a5390ad2fa30
---
guix/scripts/import.scm | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index e53028e747..e99796a430 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -31,6 +31,7 @@ (define-module (guix scripts import)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-37)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -41,7 +42,12 @@ (define-module (guix scripts import)
;;; Command line options.
;;;
-(define %standard-import-options '())
+(define %standard-import-options
+ (list
+ ;; Hidden option for importer-specific file preprocessing.
+ (option '("file-to-insert") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'file-to-insert arg result)))))
;;;
@@ -140,7 +146,10 @@ (define-command (guix-import . args)
(newline port)
(newline port)
(close-port port)))))))))
- (import-as-definitions importer args find-and-insert)))
+ (import-as-definitions importer
+ (cons (string-append "--file-to-insert=" file)
+ args)
+ find-and-insert)))
((importer args ...)
(let ((print (lambda (expr)
(leave-on-EPIPE
--
2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:20 GMT) Full text and rfc822 format available.Message #146 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 12/17] scripts: import: Skip existing definition for ‘--insert’ option. Date: Tue, 18 Mar 2025 22:26:55 +0800
* guix/scripts/import.scm (guix-import): Skip existing definition for
‘--insert’ option.
Change-Id: I2c4242669f974b263a018ab0cf56538bd7c81d06
---
guix/scripts/import.scm | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index e99796a430..f6e2985ed9 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -136,16 +136,18 @@ (define-command (guix-import . args)
(lambda (expr)
(match expr
(((? definer? definer) term _ ...)
- (let ((source-properties
- (find-definition-insertion-location
+ ;; Skip existing definition.
+ (unless (find-definition-location file term #:definer definer)
+ (let ((source-properties
+ (find-definition-insertion-location
file term #:definer definer)))
- (if source-properties
- (insert-expression source-properties expr)
- (let ((port (open-file file "a")))
- (pretty-print-with-comments port expr)
- (newline port)
- (newline port)
- (close-port port)))))))))
+ (if source-properties
+ (insert-expression source-properties expr)
+ (let ((port (open-file file "a")))
+ (pretty-print-with-comments port expr)
+ (newline port)
+ (newline port)
+ (close-port port))))))))))
(import-as-definitions importer
(cons (string-append "--file-to-insert=" file)
args)
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:22 GMT) Full text and rfc822 format available.Message #149 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 13/17] import: crate: crate-name->package-name: Move to (guix build-system cargo). Date: Tue, 18 Mar 2025 22:26:56 +0800
* guix/import/crate.scm (crate-name->package-name): Move to...
* guix/build-system/cargo.scm (crate-name->package-name): ...here and export.
Change-Id: Ie9813179d6c40d159956cc1e2ae59a74dea0a42d
---
guix/build-system/cargo.scm | 6 +++++-
guix/import/crate.scm | 5 -----
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4486c706a1..7a07003262 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -44,7 +44,8 @@ (define-module (guix build-system cargo)
%crate-base-url
crate-url
crate-url?
- crate-uri))
+ crate-uri
+ crate-name->package-name))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -58,6 +59,9 @@ (define (crate-uri name version)
to NAME and VERSION."
(string-append crate-url name "/" version "/download"))
+(define (crate-name->package-name name)
+ (downstream-package-name "rust-" name))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 5a8caeb3e1..14e6e28c5b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -481,10 +481,6 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
-(define (crate-name->package-name name)
- (downstream-package-name "rust-" name))
-
-
;;;
;;; Updater
@@ -528,4 +524,3 @@ (define %crate-updater
(description "Updater for crates.io packages")
(pred crate-package?)
(import import-release)))
-
--
2.48.1
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:24 GMT) Full text and rfc822 format available.Message #152 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 18 Mar 2025 22:27:00 +0800
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
1 file changed, 390 insertions(+), 25 deletions(-)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..837074dead 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,399 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
+As currently there's no achievable way to reuse Rust packages as pre-compiled
+inputs for other packages at a distribution scale, and our previous approach on
+making all Rust dependencies as full packages doesn't cope well with Rust
+ecosystem thus causing issues on both contribution and maintenance sides, we
+have switched to a different packaging model.
+
+Rust programs (binary crates) and dependencies (library crates) are treated
+separately. We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
+vendored dependencies. The following paragraphs will explain them and give
+several examples.
+
+Rust programs are treated like any other package and named using the lowercase
+upstream name. When using the Cargo build system (@pxref{Build Systems,
+@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
+argument set to @code{#f}, as this argument only makes sense for dependencies.
+When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
+be set to enable relevant support.
+
+Rust dependencies are managed in two modules:
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix. The name should be changed to lowercase as appropriate and
-dashes should remain in place.
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, crate, @code{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored sources unbundled before
+contributing to Guix. Naturally, this module serves as a store for both sources
+and unbundling strategies.
+
+This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
+one version containing all changes from other branches, so that the maintained
+version can be used directly in case of merge conflicts, thus coordination is
+required for other committers to modify it.
+
+Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
+script @file{etc/teams/rust/cleanup-crates.sh} for this module.
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages. This includes Rust dependencies requiring external inputs
+to unbundle and Cargo workspaces.
+
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix. The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix. The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course). This follows the ``caret'' version scheme
-intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments. It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}. If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible. Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+package to be used at any given time, so all dependencies should have a
+versioned suffix. The versioned suffix is the left-most non-zero digit (and any
+leading zeros, of course). This follows the ``caret'' version scheme intended
+by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we are usually packaging development snapshots of Rust dependencies
+specifically for some Rust programs, and can't simply identity them by version.
+In this case we can use a @code{for-@var{program}} suffix. Examples@:
+@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Let's demonstrate the packaging workflow by examples, note that package-specific
+issues are not involved here.
+
+In preparation, we'll add the following packages to our environment:
+
+@example
+guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
+repository @uref{https://crates.io, crates.io}.
+
+@enumerate
+@item
+We can generate a draft definition via the crates.io importer (@pxref{Invoking
+guix import, crate}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
+facilitating dependency modification. @code{'cargo-audit} used here must be a
+unique identifier, usually same to the program's variable name.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+cargo generate-lockfile
+cargo audit
+cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining Packages,
+@code{license}}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them in Guix command-line interface through expression:
+
+@example
+guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+point the imported definition to a symbol with the same name.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to point to it:
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+
+Such modifications are processed by @code{cargo-inputs} procedure.
+
+@code{cargo-inputs} can also be configured by keywoard arguments
+@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
+resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
+packages rust-sources)}, module to resolve modified symbols), which are useful
+to maintain an independent channel.
+@end enumerate
+
+Example 2: @code{niri}, which is not available on crates.io and depends on
+development snapshots (also Cargo workspaces in this example).
+
+@enumerate
+@item
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+in a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in build environment, the substitution pattern is
+package-specific.
+
+@code{cargo-inputs} returns a list, all list operations apply to it too.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3)))
+@end lisp
+
+@item
+@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
+workspace, build argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+@end enumerate
+
+Example 3: @code{fish}, which combines two build systems.
+
+@enumerate
+@item
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+@end enumerate
@node Elm Packages
--
2.48.1
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 06:53:03 GMT) Full text and rfc822 format available.Message #155 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: steve <at> futurile.net, 77093 <at> debbugs.gnu.org, divya <at> subvertising.org Subject: Re: [PATCH rust-team v2 00/17] New Rust packaging workflow based on lockfile importer. Date: Wed, 19 Mar 2025 08:52:19 +0200
[Message part 1 (text/plain, inline)]
+CC the other rust-team members, not sure how they got left out.
guix/import/crate/cargo-lock.scm should be added to etc/teams.scm.
I was thinking some more about the rust-crates.tmpl file and I think it
makes sense to keep it since it'll likely be useful for people who want
rust packages in their own channel or for working into a guix.scm file.
Some of the patches, specifically some of the importer ones, seem like
they could go straight to master.
On Tue, Mar 18, 2025 at 10:26:43PM +0800, Hilton Chain wrote:
> V1 -> V2:
> * Drop jemalloc environment variable patch.
> * Remove use of RUSTC_BOOTSTRAP.
> * Adjust documentation.
> Diff inserted at end.
>
> Efraim Flashner (1):
> build/cargo: Don't try to unpack sanity-check.py.
>
> Hilton Chain (16):
> build/cargo: Pass ‘--offline’ to cargo.
> build/cargo: Print out all non-empty binary files.
> build-system: cargo: Support packaging Cargo workspace.
> build-system: cargo: Support installing Cargo workspace.
> build/cargo: Set default value of arguments for build phases.
> build/cargo: Support non-workspace directory source inputs.
> scripts: import: Document argument for ‘--insert’ option in help
> message.
> scripts: import: Add two newlines for ‘--insert’ option.
> scripts: import: Support expressions defined by 'define.
> scripts: import: Pass "--insert" to importers.
> scripts: import: Skip existing definition for ‘--insert’ option.
> import: crate: crate-name->package-name: Move to (guix build-system
> cargo).
> build-system: cargo: Add ‘cargo-inputs’.
> import: crate: Add Cargo.lock parser.
> import: crate: Add ‘--lockfile’ option.
> doc: Document lockfile importer based Rust packaging workflow.
>
> Makefile.am | 1 +
> doc/contributing.texi | 415 ++++++++++++++++++++++++++++--
> doc/guix.texi | 24 ++
> etc/teams/rust/cleanup-crates.sh | 38 +++
> etc/teams/rust/rust-crates.tmpl | 42 +++
> gnu/local.mk | 2 +
> gnu/packages/rust-crates.scm | 42 +++
> gnu/packages/rust-sources.scm | 29 +++
> guix/build-system/cargo.scm | 57 +++-
> guix/build/cargo-build-system.scm | 151 ++++++++---
> guix/import/crate.scm | 81 +++++-
> guix/import/crate/cargo-lock.scm | 105 ++++++++
> guix/scripts/import.scm | 53 ++--
> guix/scripts/import/crate.scm | 58 ++++-
> guix/utils.scm | 27 +-
> tests/crate.scm | 88 +++++++
> 16 files changed, 1107 insertions(+), 106 deletions(-)
> create mode 100755 etc/teams/rust/cleanup-crates.sh
> create mode 100644 etc/teams/rust/rust-crates.tmpl
> create mode 100644 gnu/packages/rust-crates.scm
> create mode 100644 gnu/packages/rust-sources.scm
> create mode 100644 guix/import/crate/cargo-lock.scm
>
>
> base-commit: b54a9ca849f013300c633fb79d80bc754f6b28a2
> --
> 2.48.1
>
> --8<---------------cut here---------------start------------->8---
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index e7c5116a3d..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1607,15 +1607,15 @@ Rust Crates
> have switched to a different packaging model.
>
> Rust programs (binary crates) and dependencies (library crates) are treated
> -separately. We put main efforts into programs and only package Rust
> -dependencies as sources, utilizing automation with manual focus on unbundling
> +separately. We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> vendored dependencies. The following paragraphs will explain them and give
> several examples.
>
> Rust programs are treated like any other package and named using the lowercase
> upstream name. When using the Cargo build system (@pxref{Build Systems,
> @code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> -argument set to @code{#f}, as this argument only makes sense to dependencies.
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> be set to enable relevant support.
>
> @@ -1631,21 +1631,21 @@ Rust Crates
> contributing to Guix. Naturally, this module serves as a store for both sources
> and unbundling strategies.
>
> -This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
> -version containing all changes from other branches, so that the maintained
> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> version can be used directly in case of merge conflicts, thus coordination is
> required for other committers to modify it.
>
> Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
> -script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
> +script @file{etc/teams/rust/cleanup-crates.sh} for this module.
>
> @item
> @code{(gnu packages rust-sources)}, storing more complex definitions that need
> to be full packages. This includes Rust dependencies requiring external inputs
> to unbundle and Cargo workspaces.
>
> -These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
> -For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
>
> Since they are added manually, the following naming convention applies:
>
> @@ -1668,7 +1668,7 @@ Rust Crates
> Let's demonstrate the packaging workflow by examples, note that package-specific
> issues are not involved here.
>
> -In preparation, we'll adding the following packages to our environment:
> +In preparation, we'll add the following packages to our environment:
>
> @example
> guix shell rust rust:cargo cargo-audit cargo-license
> @@ -1747,7 +1747,7 @@ Rust Crates
> @code{check-for-pregenerated-files} phase:
>
> @example
> -$ guix build cargo-audit
> +$ ./pre-inst-env guix build cargo-audit
> @dots{}
> starting phase `check-for-pregenerated-files'
> Searching for binary files...
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 029c8cf59c..218c2ba630 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -9365,14 +9365,14 @@ Build Systems
> sources, to ease in future hacking on rust packages.
>
> This build system supports cargo workspaces. Parameter
> -@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
> -names of library crates to package in @code{package} phase. Specified
> -crates are packaged from left to right, in case there's dependency among
> -them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
> -package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
> -@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
> -paths of binary crates to install in @code{install} phase,
> -@code{''("crates/atuin")}, for example.
> +@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
> +library crates to package in the @code{package} phase. Specified crates are
> +packaged from left to right, in case there's dependency among them. For
> +example, specifying @code{''("pcre2-sys" "pcre2")} will package
> +@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
> +@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
> +binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
> +for example.
> @end defvar
>
> @defvar chicken-build-system
> @@ -14705,19 +14705,18 @@ Invoking guix import
> a comment.
> @item --lockfile=@var{file}
> @itemx -f @var{file}
> -When @option{--lockfile} is specified, importer will ignore other
> -options and won't output package definitions, instead generating source
> -definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
> -For example:
> +When @option{--lockfile} is specified, the importer will ignore other options
> +and won't output package definitions, instead generating source definitions for
> +all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
>
> @example
> guix import crate --lockfile=/path/to/Cargo.lock my-package
> @end example
>
> -generates sources from @file{/path/to/Cargo.lock}, and a list
> -@code{my-package-cargo-inputs} referencing these sources. The generated
> -list is intended for package's base @code{inputs}, replacing
> -@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
> +generates sources from @file{/path/to/Cargo.lock} and a list
> +@code{my-package-cargo-inputs} referencing these sources. The generated list is
> +intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
> +@code{#:cargo-development-inputs}.
> @end table
>
> @item elm
> diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
> index b547421b88..41adc03752 100644
> --- a/guix/build/cargo-build-system.scm
> +++ b/guix/build/cargo-build-system.scm
> @@ -248,12 +248,6 @@ (define* (configure #:key inputs
> (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
> (setenv "SODIUM_USE_PKG_CONFIG" "1")
> (setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
> - ;; This flag is needed when not using the bundled jemalloc.
> - ;; https://github.com/tikv/jemallocator/issues/19
> - (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
> - (when (assoc-ref inputs "jemalloc")
> - (setenv "JEMALLOC_OVERRIDE"
> - (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so")))
> (when (assoc-ref inputs "openssl")
> (setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
> (when (assoc-ref inputs "gettext")
> @@ -356,23 +350,17 @@ (define* (package #:key
>
> (if (null? cargo-package-crates)
> (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
> - ;; Use unstable feature ‘--registry’.
> (begin
> - (setenv "RUSTC_BOOTSTRAP" "1")
> (for-each
> (lambda (pkg)
> (apply invoke "cargo" "package" "--offline" "--package" pkg
> - "--registry" "crates-io" "-Z" "package-workspace"
> cargo-package-flags)
> (for-each
> (lambda (crate)
> (invoke "tar" "xzf" crate "-C" vendor-dir))
> - (begin
> - (delete-file-recursively "target/package/tmp-registry")
> - (find-files "target/package" "\\.crate$")))
> + (find-files "target/package" "\\.crate$"))
> (patch-cargo-checksums #:vendor-dir vendor-dir))
> - cargo-package-crates)
> - (unsetenv "RUSTC_BOOTSTRAP")))
> + cargo-package-crates)))
>
> ;; Then unpack the crate, reset the timestamp of all contained files, and
> ;; repack them. This is necessary to ensure that they are reproducible.
> diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> index d8e1ed3e6f..f6e2985ed9 100644
> --- a/guix/scripts/import.scm
> +++ b/guix/scripts/import.scm
> @@ -126,13 +126,12 @@ (define-command (guix-import . args)
> (show-version-and-exit "guix import"))
> ((or ("-i" file importer args ...)
> ("--insert" file importer args ...))
> - (let* ((definer?
> - (cut member
> - <>
> - `(,@(if (member importer '("crate"))
> - '(define)
> - '())
> - define-public)))
> + (let* ((importer-definers
> + `(,@(if (member importer '("crate"))
> + '(define)
> + '())
> + define-public))
> + (definer? (cut member <> importer-definers))
> (find-and-insert
> (lambda (expr)
> (match expr
> diff --git a/tests/crate.scm b/tests/crate.scm
> index d46f753f9c..9b7066c3b1 100644
> --- a/tests/crate.scm
> +++ b/tests/crate.scm
> @@ -478,7 +478,7 @@ (define rust-leaf-bob-3.0.2-yanked
> (license #f)))
>
> (define temp-file
> - (string-append "t-utils-" (number->string (getpid))))
> + (string-append "t-crate-" (number->string (getpid))))
>
>
> (test-begin "crate")
> --8<---------------cut here---------------end--------------->8---
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 06:54:07 GMT) Full text and rfc822 format available.Message #158 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Wed, 19 Mar 2025 08:52:32 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 10:27:00PM +0800, Hilton Chain wrote:
> * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
> documentation.
>
> Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
> ---
> doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 390 insertions(+), 25 deletions(-)
>
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index ab4f30d54b..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1600,34 +1600,399 @@ Rust Crates
> @subsection Rust Crates
>
> @cindex rust
> -Rust programs standing for themselves are named as any other package, using the
> -lowercase upstream name.
> +As currently there's no achievable way to reuse Rust packages as pre-compiled
> +inputs for other packages at a distribution scale, and our previous approach on
> +making all Rust dependencies as full packages doesn't cope well with Rust
> +ecosystem thus causing issues on both contribution and maintenance sides, we
> +have switched to a different packaging model.
> +
> +Rust programs (binary crates) and dependencies (library crates) are treated
> +separately. We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> +vendored dependencies. The following paragraphs will explain them and give
> +several examples.
> +
> +Rust programs are treated like any other package and named using the lowercase
> +upstream name. When using the Cargo build system (@pxref{Build Systems,
> +@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> +When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> +be set to enable relevant support.
> +
> +Rust dependencies are managed in two modules:
>
> -To prevent namespace collisions we prefix all other Rust packages with the
> -@code{rust-} prefix. The name should be changed to lowercase as appropriate and
> -dashes should remain in place.
> +@enumerate
> +@item
> +@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
> +programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
> +import, crate, @code{--lockfile=@var{file}}}).
> +
> +Imported definitions must be checked and have vendored sources unbundled before
> +contributing to Guix. Naturally, this module serves as a store for both sources
> +and unbundling strategies.
> +
> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> +version can be used directly in case of merge conflicts, thus coordination is
> +required for other committers to modify it.
> +
> +Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
> +script @file{etc/teams/rust/cleanup-crates.sh} for this module.
> +
> +@item
> +@code{(gnu packages rust-sources)}, storing more complex definitions that need
> +to be full packages. This includes Rust dependencies requiring external inputs
> +to unbundle and Cargo workspaces.
> +
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +
> +Since they are added manually, the following naming convention applies:
> +
> +To prevent namespace collisions they are named with @code{rust-} prefix. The
> +name should be changed to lowercase as appropriate and dashes should remain in
> +place.
>
> In the rust ecosystem it is common for multiple incompatible versions of a
> -package to be used at any given time, so all package definitions should have a
> -versioned suffix. The versioned suffix is the left-most non-zero digit (and
> -any leading zeros, of course). This follows the ``caret'' version scheme
> -intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> -
> -Because of the difficulty in reusing rust packages as pre-compiled inputs for
> -other packages the Cargo build system (@pxref{Build Systems,
> -@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
> -@code{cargo-development-inputs} keywords as build system arguments. It would be
> -helpful to think of these as similar to @code{propagated-inputs} and
> -@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
> -should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
> -@code{#:cargo-development-inputs}. If a Rust package links to other libraries
> -then the standard placement in @code{inputs} and the like should be used.
> -
> -Care should be taken to ensure the correct version of dependencies are used; to
> -this end we try to refrain from skipping the tests or using @code{#:skip-build?}
> -when possible. Of course this is not always possible, as the package may be
> -developed for a different Operating System, depend on features from the Nightly
> -Rust compiler, or the test suite may have atrophied since it was released.
> +package to be used at any given time, so all dependencies should have a
> +versioned suffix. The versioned suffix is the left-most non-zero digit (and any
> +leading zeros, of course). This follows the ``caret'' version scheme intended
> +by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> +
> +In practice we are usually packaging development snapshots of Rust dependencies
> +specifically for some Rust programs, and can't simply identity them by version.
> +In this case we can use a @code{for-@var{program}} suffix. Examples@:
> +@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
> +@end enumerate
> +
> +Let's demonstrate the packaging workflow by examples, note that package-specific
> +issues are not involved here.
> +
> +In preparation, we'll add the following packages to our environment:
> +
> +@example
> +guix shell rust rust:cargo cargo-audit cargo-license
> +@end example
I was able to run cargo-audit as 'cargo-audit audit ...' but I wouldn't
actually recommend that to anyone. I'm not sure we actually need rust
in the shell but I don't have a strong preference. More thinking aloud,
perhaps we should adjust rust:cargo to automatically pull in rust:out?
I also find it amusing to suggest using cargo-audit when it's not (yet)
in guix and we immediately follow up with importing it.
I could see an argument for putting this part in the cookbook instead of
here but I feel strongly that it should be here.
> +
> +Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
> +repository @uref{https://crates.io, crates.io}.
> +
> +@enumerate
> +@item
> +We can generate a draft definition via the crates.io importer (@pxref{Invoking
> +guix import, crate}). In the end we'll have the following definiton:
> +
> +@lisp
> +(define-public cargo-audit
> + (package
> + (name "cargo-audit")
> + (version "0.21.2")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (crate-uri "cargo-audit" version))
> + (file-name (string-append name "-" version ".tar.gz"))
> + (sha256
> + (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
> + (build-system cargo-build-system)
> + (arguments (list #:install-source? #f))
> + (inputs (cargo-inputs 'cargo-audit))
> + (home-page "https://rustsec.org/")
> + (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
> + (description
> + "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
> +audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
> + (license (list license:asl2.0 license:expat))))
> +@end lisp
> +
> +@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
> +facilitating dependency modification. @code{'cargo-audit} used here must be a
> +unique identifier, usually same to the program's variable name.
> +
> +@item
> +Unpack package source and navigate to the unpacked directory, then execute the
> +following commands:
> +
> +@example
> +cargo generate-lockfile
> +cargo audit
> +cargo license
> +@end example
> +
> +@command{cargo generate-lockfile} updates dependencies to compatible versions,
> +@command{cargo audit} checks known vulnerabilities and @command{cargo license}
> +checks licenses of all dependencies.
> +
> +We must have an acceptable output of @command{cargo audit} and ensure all
> +dependencies are licensed with our supported licenses (@pxref{Defining Packages,
> +@code{license}}).
> +
> +@item
> +Import dependencies from previously generated lockfile:
> +
> +@example
> +guix import --insert=gnu/packages/rust-crates.scm \
> + crate --lockfile=/path/to/Cargo.lock cargo-audit
> +@end example
> +
> +@code{cargo-audit} used here must be consistent with the identifier used for
> +@code{cargo-inputs}.
> +
> +At this stage, @code{cargo-audit} is buildable.
> +
> +@item
> +Finally we'll unbundle vendored sources. The lockfile importer inserts
> +@code{TODO:} comments to dependencies with high probability of bundled sources.
> +@code{cargo-build-system} also performs additional check in its
> +@code{check-for-pregenerated-files} phase:
> +
> +@example
> +$ ./pre-inst-env guix build cargo-audit
> +@dots{}
> +starting phase `check-for-pregenerated-files'
> +Searching for binary files...
> +./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
> +./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
> +@dots{}
> +@end example
> +
> +Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
> +can still select them in Guix command-line interface through expression:
> +
> +@example
> +guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
> +@end example
> +
> +For most dependencies, a snippet is sufficient:
> +
> +@lisp
> +(define rust-bzip2-sys-0.1.13+1.0.8
> + (crate-source "bzip2-sys" "0.1.13+1.0.8"
> + "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
> + #:snippet
> + '(begin
> + (delete-file-recursively "bzip2-1.0.8")
> + (delete-file "build.rs")
> + ;; Inspired by Debian's patch.
> + (with-output-to-file "build.rs"
> + (lambda _
> + (format #t "fn main() @{~@@
> + println!(\"cargo:rustc-link-lib=bz2\");~@@
> + @}~%"))))))
> +@end lisp
> +
> +In a more complex case, where unbundling one dependency requires other packages,
> +we should package the dependency in @code{(gnu packages rust-sources)} first and
> +point the imported definition to a symbol with the same name.
> +
> +For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
> +rust-sources)}, then the imported definition in @code{(gnu packages
> +rust-crates)} should be modified to point to it:
> +
> +@lisp
> +(define rust-ring-0.17.14 'rust-ring-0.17)
> +@end lisp
> +
> +When one dependency can be safely removed, modify it to @code{#f}.
> +
> +@lisp
> +(define rust-openssl-src-300.4.2+3.4.1 #f)
> +@end lisp
> +
> +Such modifications are processed by @code{cargo-inputs} procedure.
> +
> +@code{cargo-inputs} can also be configured by keywoard arguments
> +@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
> +resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
> +packages rust-sources)}, module to resolve modified symbols), which are useful
> +to maintain an independent channel.
> +@end enumerate
> +
> +Example 2: @code{niri}, which is not available on crates.io and depends on
> +development snapshots (also Cargo workspaces in this example).
> +
> +@enumerate
> +@item
> +As we can't ensure compatibility of a development snapshot, before executing
> +@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
> +in a known working revision.
> +
> +To use our packaged development snapshots, it's also necessary to modify
> +@file{Cargo.toml} in build environment, the substitution pattern is
> +package-specific.
> +
> +@code{cargo-inputs} returns a list, all list operations apply to it too.
> +
> +@lisp
> +(define-public niri
> + (package
> + (name "niri")
> + (version "25.02")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/YaLTeR/niri")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:install-source? #f
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("# version =.*")
> + "version = \"*\"")
> + (("git.*optional")
> + "version = \"*\", optional")
> + (("^git = .*")
> + "")))))))
> + (native-inputs
> + (list pkg-config))
> + (inputs
> + (cons* clang
> + libdisplay-info
> + libinput-minimal
> + libseat
> + libxkbcommon
> + mesa
> + pango
> + pipewire
> + wayland
> + (cargo-inputs 'niri)))
> + (home-page "https://github.com/YaLTeR/niri")
> + (synopsis "Scrollable-tiling Wayland compositor")
> + (description
> + "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
> +scrollable format. It is considered stable for daily use and performs most
> +functions expected of a Wayland compositor.")
> + (license license:gpl3)))
> +@end lisp
> +
> +@item
> +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> +workspace, build argument @code{#:cargo-package-crates} is required.
> +
> +@lisp
> +(define-public rust-pipewire-for-niri
> + (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> + (revision "0"))
> + (package
> + (name "rust-pipewire")
> + (version (git-version "0.8.0" revision commit))
> + (source
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> + (commit commit)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:skip-build? #t
> + #:cargo-package-crates
> + ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> + (inputs (cargo-inputs 'rust-pipewire-for-niri))
should these be inputs or propagated-inputs? in rust-ring-0.17 in
rust-sources they are propagated. I would think inputs would be enough
since the Cargo.lock file will let us know which other crates are
necessary for actually building it as a dependency of that package.
> + (home-page "https://pipewire.org/")
> + (synopsis "Rust bindings for PipeWire")
> + (description "This package provides Rust bindings for PipeWire.")
> + (license license:expat))))
> +@end lisp
> +
> +Don't forget to modify all workspace members in @code{(gnu packages
> +rust-crates)}:
> +
> +@lisp
> +(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@dots{}
> +(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@end lisp
> +@end enumerate
> +
> +Example 3: @code{fish}, which combines two build systems.
> +
> +@enumerate
> +@item
> +When building Rust packages in other build systems, we need to add @code{rust}
> +and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
> +build systems, and apply necessary build phases from @code{cargo-build-system}.
> +
> +@lisp
> +(define-public fish
> + (package
> + (name "fish")
> + (version "4.0.0")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (string-append "https://github.com/fish-shell/fish-shell/"
> + "releases/download/" version "/"
> + "fish-" version ".tar.xz"))
> + (sha256
> + (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
> + (build-system cmake-build-system)
> + (inputs
> + (cons* fish-foreign-env ncurses pcre2
> + python ;for fish_config and manpage completions
> + (cargo-inputs 'fish)))
> + (native-inputs
> + (list doxygen groff ;for 'fish --help'
> + pkg-config
> + procps ;for the test suite
> + rust
> + `(,rust "cargo")))
> + (arguments
> + (list #:imported-modules
> + (append %cargo-build-system-modules
> + %cmake-build-system-modules)
> + #:modules
> + (((guix build cargo-build-system) #:prefix cargo:)
> + (guix build cmake-build-system)
> + (guix build utils))
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("git.*tag.*,")
> + "version = \"*\","))))
> + (add-after 'unpack 'prepare-cargo-build-system
> + (lambda args
> + (for-each
> + (lambda (phase)
> + (format #t "Running cargo phase: ~a~%" phase)
> + (apply (assoc-ref cargo:%standard-phases phase)
> + args))
> + '(unpack-rust-crates
> + configure
> + check-for-pregenerated-files
> + patch-cargo-checksums)))))))
> + (synopsis "The friendly interactive shell")
> + (description
> + "Fish (friendly interactive shell) is a shell focused on interactive use,
> +discoverability, and friendliness. Fish has very user-friendly and powerful
> +tab-completion, including descriptions of every completion, completion of
> +strings with wildcards, and many completions for specific commands. It also
> +has extensive and discoverable help. A special @@command@{help@} command gives
> +access to all the fish documentation in your web browser. Other features
> +include smart terminal handling based on terminfo, an easy to search history,
> +and syntax highlighting.")
> + (home-page "https://fishshell.com/")
> + (license license:gpl2)))
> +@end lisp
> +@end enumerate
>
>
> @node Elm Packages
> --
> 2.48.1
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 06:54:11 GMT) Full text and rfc822 format available.Message #161 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team v2 10/17] scripts: import: Support expressions defined by 'define. Date: Wed, 19 Mar 2025 08:52:40 +0200
[Message part 1 (text/plain, inline)]
How about 'define-prefix' instead of 'definer'? It seems more
descriptive.
On Tue, Mar 18, 2025 at 10:26:53PM +0800, Hilton Chain wrote:
> * guix/utils.scm (find-definition-location): New procedure.
> (find-definition-insertion-location): Define with it.
> * guix/scripts/import.scm (import-as-definitions, guix-import): Support
> expressions defined by 'define.
>
> Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
> ---
> guix/scripts/import.scm | 19 ++++++++++++++-----
> guix/utils.scm | 27 +++++++++++++++++++--------
> 2 files changed, 33 insertions(+), 13 deletions(-)
>
> diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> index 58a84d0db7..e53028e747 100644
> --- a/guix/scripts/import.scm
> +++ b/guix/scripts/import.scm
> @@ -30,6 +30,7 @@ (define-module (guix scripts import)
> #:use-module (guix read-print)
> #:use-module (guix utils)
> #:use-module (srfi srfi-1)
> + #:use-module (srfi srfi-26)
> #:use-module (ice-9 format)
> #:use-module (ice-9 match)
> #:export (%standard-import-options
> @@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
> ((and expr (or ('package _ ...)
> ('let _ ...)))
> (proc (package->definition expr)))
> - ((and expr ('define-public _ ...))
> + ((and expr (or ('define-public _ ...)
> + ('define _ ...)))
> (proc expr))
> ((expressions ...)
> (for-each (lambda (expr)
> @@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
> ((and expr (or ('package _ ...)
> ('let _ ...)))
> (proc (package->definition expr)))
> - ((and expr ('define-public _ ...))
> + ((and expr (or ('define-public _ ...)
> + ('define _ ...)))
> (proc expr))))
> expressions))
> (x
> @@ -117,13 +120,19 @@ (define-command (guix-import . args)
> (show-version-and-exit "guix import"))
> ((or ("-i" file importer args ...)
> ("--insert" file importer args ...))
> - (let ((find-and-insert
> + (let* ((importer-definers
> + `(,@(if (member importer '("crate"))
> + '(define)
> + '())
> + define-public))
> + (definer? (cut member <> importer-definers))
> + (find-and-insert
> (lambda (expr)
> (match expr
> - (('define-public term _ ...)
> + (((? definer? definer) term _ ...)
> (let ((source-properties
> (find-definition-insertion-location
> - file term)))
> + file term #:definer definer)))
> (if source-properties
> (insert-expression source-properties expr)
> (let ((port (open-file file "a")))
> diff --git a/guix/utils.scm b/guix/utils.scm
> index c7c23d9d5b..77ec6d992a 100644
> --- a/guix/utils.scm
> +++ b/guix/utils.scm
> @@ -154,6 +154,7 @@ (define-module (guix utils)
> edit-expression
> delete-expression
> insert-expression
> + find-definition-location
> find-definition-insertion-location
>
> filtered-port
> @@ -520,24 +521,34 @@ (define (insert-expression source-properties expr)
> (string-append expr "\n\n" str))))
> (edit-expression source-properties insert)))
>
> -(define (find-definition-insertion-location file term)
> - "Search in FILE for a top-level public definition whose defined term
> -alphabetically succeeds TERM. Return the location if found, or #f
> -otherwise."
> - (let ((search-term (symbol->string term)))
> +(define* (find-definition-location file term
> + #:key (definer 'define-public)
> + (pred string=))
> + "Search in FILE for a top-level definition defined by DEFINER with defined
> +term comparing to TERM through PRED. Return the location if PRED returns #t,
> +or #f otherwise."
> + (let ((search-term (symbol->string term))
> + (definer? (cut eq? definer <>)))
> (call-with-input-file file
> (lambda (port)
> (do ((syntax (read-syntax port)
> (read-syntax port)))
> ((match (syntax->datum syntax)
> - (('define-public current-term _ ...)
> - (string> (symbol->string current-term)
> - search-term))
> + (((? definer?) current-term _ ...)
> + (pred (symbol->string current-term)
> + search-term))
> ((? eof-object?) #t)
> (_ #f))
> (and (not (eof-object? syntax))
> (syntax-source syntax))))))))
>
> +(define* (find-definition-insertion-location file term
> + #:key (definer 'define-public))
> + "Search in FILE for a top-level definition defined by DEFINER with defined
> +term alphabetically succeeds TERM. Return the location if found, or #f
> +otherwise."
> + (find-definition-location file term #:definer definer #:pred string>))
> +
>
> ;;;
> ;;; Keyword arguments.
> --
> 2.48.1
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 12:44:02 GMT) Full text and rfc822 format available.Message #164 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il> Cc: 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Wed, 19 Mar 2025 19:34:46 +0800
On Wed, 19 Mar 2025 14:52:32 +0800,
Efraim Flashner wrote:
>
> > +Let's demonstrate the packaging workflow by examples, note that package-specific
> > +issues are not involved here.
> > +
> > +In preparation, we'll add the following packages to our environment:
> > +
> > +@example
> > +guix shell rust rust:cargo cargo-audit cargo-license
> > +@end example
>
> I was able to run cargo-audit as 'cargo-audit audit ...' but I wouldn't
> actually recommend that to anyone. I'm not sure we actually need rust
> in the shell but I don't have a strong preference. More thinking aloud,
‘cargo generate-lockfile’ sometimes requires rust.
> perhaps we should adjust rust:cargo to automatically pull in rust:out?
Sounds reasonable.
> I also find it amusing to suggest using cargo-audit when it's not (yet)
> in guix and we immediately follow up with importing it.
XD yes, it's funny to use cargo-audit audit cargo-audit, and cargo-license to
check license of cargo-license.
> I could see an argument for putting this part in the cookbook instead of
> here but I feel strongly that it should be here.
> > [...]
> > +@item
> > +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> > +workspace, build argument @code{#:cargo-package-crates} is required.
> > +
> > +@lisp
> > +(define-public rust-pipewire-for-niri
> > + (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> > + (revision "0"))
> > + (package
> > + (name "rust-pipewire")
> > + (version (git-version "0.8.0" revision commit))
> > + (source
> > + (origin
> > + (method git-fetch)
> > + (uri (git-reference
> > + (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> > + (commit commit)))
> > + (file-name (git-file-name name version))
> > + (sha256
> > + (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> > + (build-system cargo-build-system)
> > + (arguments
> > + (list #:skip-build? #t
> > + #:cargo-package-crates
> > + ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> > + (inputs (cargo-inputs 'rust-pipewire-for-niri))
>
> should these be inputs or propagated-inputs? in rust-ring-0.17 in
> rust-sources they are propagated. I would think inputs would be enough
> since the Cargo.lock file will let us know which other crates are
> necessary for actually building it as a dependency of that package.
For rust-ring-0.17 the propagation is necessary, I think this is because our
packaged version is a bit older than the one used in generated lockfile.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 13:43:01 GMT) Full text and rfc822 format available.Message #167 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> To: Hilton Chain <hako <at> ultrarare.space> Cc: Ludovic Courtès <ludo <at> gnu.org>, 77093 <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il> Subject: Re: [bug#77093] [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Wed, 19 Mar 2025 22:41:39 +0900
Hi,
Hilton Chain <hako <at> ultrarare.space> writes:
> * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
> documentation.
>
> Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
> ---
> doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 390 insertions(+), 25 deletions(-)
>
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index ab4f30d54b..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1600,34 +1600,399 @@ Rust Crates
> @subsection Rust Crates
>
> @cindex rust
> -Rust programs standing for themselves are named as any other package, using the
> -lowercase upstream name.
> +As currently there's no achievable way to reuse Rust packages as pre-compiled
> +inputs for other packages at a distribution scale, and our previous approach on
> +making all Rust dependencies as full packages doesn't cope well with Rust
> +ecosystem thus causing issues on both contribution and maintenance sides, we
> +have switched to a different packaging model.
I'd drop the above paragraph; it won't age well.
> +Rust programs (binary crates) and dependencies (library crates) are treated
> +separately. We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> +vendored dependencies. The following paragraphs will explain them and give
> +several examples.
> +
> +Rust programs are treated like any other package and named using the lowercase
> +upstream name. When using the Cargo build system (@pxref{Build Systems,
> +@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> +When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> +be set to enable relevant support.
> +
> +Rust dependencies are managed in two modules:
>
> -To prevent namespace collisions we prefix all other Rust packages with the
> -@code{rust-} prefix. The name should be changed to lowercase as appropriate and
> -dashes should remain in place.
> +@enumerate
> +@item
> +@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
> +programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
> +import, crate, @code{--lockfile=@var{file}}}).
> +
> +Imported definitions must be checked and have vendored sources unbundled before
> +contributing to Guix. Naturally, this module serves as a store for both sources
s/contributing/being contributed/. The next sentence (Naturally, ...)
seems superfluous to me.
> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> +version can be used directly in case of merge conflicts, thus coordination is
> +required for other committers to modify it.
I think the above could be simplified like: This module is managed by
the Rust team, via the @file{etc/teams/rust/rust-crates.tmpl} template
file and @file{etc/teams/rust/rust-crates.tmpl} cleanup script.
[...]
> +@item
> +@code{(gnu packages rust-sources)}, storing more complex definitions that need
> +to be full packages. This includes Rust dependencies requiring external inputs
> +to unbundle and Cargo workspaces.
> +
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +
> +Since they are added manually, the following naming convention applies:
> +
> +To prevent namespace collisions they are named with @code{rust-} prefix. The
> +name should be changed to lowercase as appropriate and dashes should remain in
> +place.
>
> In the rust ecosystem it is common for multiple incompatible versions of a
> -package to be used at any given time, so all package definitions should have a
> -versioned suffix. The versioned suffix is the left-most non-zero digit (and
> -any leading zeros, of course). This follows the ``caret'' version scheme
> -intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> -
> -Because of the difficulty in reusing rust packages as pre-compiled inputs for
> -other packages the Cargo build system (@pxref{Build Systems,
> -@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
> -@code{cargo-development-inputs} keywords as build system arguments. It would be
> -helpful to think of these as similar to @code{propagated-inputs} and
> -@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
> -should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
> -@code{#:cargo-development-inputs}. If a Rust package links to other libraries
> -then the standard placement in @code{inputs} and the like should be used.
> -
> -Care should be taken to ensure the correct version of dependencies are used; to
> -this end we try to refrain from skipping the tests or using @code{#:skip-build?}
> -when possible. Of course this is not always possible, as the package may be
> -developed for a different Operating System, depend on features from the Nightly
> -Rust compiler, or the test suite may have atrophied since it was released.
> +package to be used at any given time, so all dependencies should have a
> +versioned suffix. The versioned suffix is the left-most non-zero digit (and any
> +leading zeros, of course). This follows the ``caret'' version scheme intended
> +by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
@: is not needed before a colon. Refer to (info "(texinfo) Not Ending a Sentence").
> +In practice we are usually packaging development snapshots of Rust dependencies
> +specifically for some Rust programs, and can't simply identity them by version.
> +In this case we can use a @code{for-@var{program}} suffix. Examples@:
I'd join the last sentence with the previous one, e.g. [...] suffix, for
example: [...]
> +@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
> +@end enumerate
> +
> +Let's demonstrate the packaging workflow by examples, note that package-specific
> +issues are not involved here.
I'd drop the part after examples.
> +In preparation, we'll add the following packages to our environment:
> +
> +@example
> +guix shell rust rust:cargo cargo-audit cargo-license
> +@end example
> +
> +Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
> +repository @uref{https://crates.io, crates.io}.
I'd replace the textual crates.io with the later @uref, end place the
period after 'repository'.
This verbose how-to Rust packaging, while useful, would better belong in
the cookbook, in my opinion. The GNU Guix Cookbook document section can
then be referenced from here, for example:
@ref{Rust Packaging Examples,,,guix-cookbook,GNU Guix Cookbook} for real
life examples of packaging Rust applications.
> +@enumerate
> +@item
> +We can generate a draft definition via the crates.io importer (@pxref{Invoking
> +guix import, crate}). In the end we'll have the following definiton:
> +
> +@lisp
> +(define-public cargo-audit
> + (package
> + (name "cargo-audit")
> + (version "0.21.2")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (crate-uri "cargo-audit" version))
> + (file-name (string-append name "-" version ".tar.gz"))
> + (sha256
> + (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
> + (build-system cargo-build-system)
> + (arguments (list #:install-source? #f))
> + (inputs (cargo-inputs 'cargo-audit))
> + (home-page "https://rustsec.org/")
> + (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
> + (description
> + "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
> +audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
> + (license (list license:asl2.0 license:expat))))
> +@end lisp
> +
> +@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
> +facilitating dependency modification. @code{'cargo-audit} used here must be a
> +unique identifier, usually same to the program's variable name.
I'd reword to: [...], usually matching the variable name of the package.
> +
> +@item
> +Unpack package source and navigate to the unpacked directory, then execute the
> +following commands:
> +
> +@example
> +cargo generate-lockfile
> +cargo audit
> +cargo license
> +@end example
> +
> +@command{cargo generate-lockfile} updates dependencies to compatible versions,
> +@command{cargo audit} checks known vulnerabilities and @command{cargo license}
> +checks licenses of all dependencies.
> +
> +We must have an acceptable output of @command{cargo audit} and ensure all
> +dependencies are licensed with our supported licenses (@pxref{Defining Packages,
> +@code{license}}).
> +
> +@item
> +Import dependencies from previously generated lockfile:
> +
> +@example
> +guix import --insert=gnu/packages/rust-crates.scm \
> + crate --lockfile=/path/to/Cargo.lock cargo-audit
> +@end example
> +
> +@code{cargo-audit} used here must be consistent with the identifier used for
> +@code{cargo-inputs}.
> +
> +At this stage, @code{cargo-audit} is buildable.
> +
> +@item
> +Finally we'll unbundle vendored sources. The lockfile importer inserts
> +@code{TODO:} comments to dependencies with high probability of bundled sources.
> +@code{cargo-build-system} also performs additional check in its
> +@code{check-for-pregenerated-files} phase:
> +
> +@example
> +$ ./pre-inst-env guix build cargo-audit
> +@dots{}
> +starting phase `check-for-pregenerated-files'
> +Searching for binary files...
> +./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
> +./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
> +@dots{}
> +@end example
> +
> +Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
> +can still select them in Guix command-line interface through expression:
select them *via* the Guix command-line interface [...]
> +
> +@example
> +guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
> +@end example
> +
> +For most dependencies, a snippet is sufficient:
> +
> +@lisp
> +(define rust-bzip2-sys-0.1.13+1.0.8
> + (crate-source "bzip2-sys" "0.1.13+1.0.8"
> + "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
> + #:snippet
> + '(begin
> + (delete-file-recursively "bzip2-1.0.8")
> + (delete-file "build.rs")
> + ;; Inspired by Debian's patch.
> + (with-output-to-file "build.rs"
> + (lambda _
> + (format #t "fn main() @{~@@
> + println!(\"cargo:rustc-link-lib=bz2\");~@@
> + @}~%"))))))
> +@end lisp
> +
> +In a more complex case, where unbundling one dependency requires other packages,
> +we should package the dependency in @code{(gnu packages rust-sources)} first and
> +point the imported definition to a symbol with the same name.
> +
> +For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
> +rust-sources)}, then the imported definition in @code{(gnu packages
> +rust-crates)} should be modified to point to it:
> +
> +@lisp
> +(define rust-ring-0.17.14 'rust-ring-0.17)
> +@end lisp
> +
> +When one dependency can be safely removed, modify it to @code{#f}.
> +
> +@lisp
> +(define rust-openssl-src-300.4.2+3.4.1 #f)
> +@end lisp
> +
> +Such modifications are processed by @code{cargo-inputs} procedure.
> +
> +@code{cargo-inputs} can also be configured by keywoard arguments
> +@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
> +resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
> +packages rust-sources)}, module to resolve modified symbols), which are useful
> +to maintain an independent channel.
> +@end enumerate
> +
> +Example 2: @code{niri}, which is not available on crates.io and depends on
It seems to me these examples should have their own structure node
(subsection or subsubsection, as appropriate).
> +development snapshots (also Cargo workspaces in this example).
> +
> +@enumerate
> +@item
> +As we can't ensure compatibility of a development snapshot, before executing
> +@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
> +in a known working revision.
> +
> +To use our packaged development snapshots, it's also necessary to modify
> +@file{Cargo.toml} in build environment, the substitution pattern is
> +package-specific.
The above sentence appears incomplete or disconnected. Perhaps,
To use our packaged development snapshots, it's also necessary to modify
@file{Cargo.toml} in a phase, with a package-specific substitution
pattern.
> +@code{cargo-inputs} returns a list, all list operations apply to it too.
This appears to be out of context?
> +
> +@lisp
> +(define-public niri
> + (package
> + (name "niri")
> + (version "25.02")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/YaLTeR/niri")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:install-source? #f
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("# version =.*")
> + "version = \"*\"")
> + (("git.*optional")
> + "version = \"*\", optional")
> + (("^git = .*")
> + "")))))))
> + (native-inputs
> + (list pkg-config))
> + (inputs
> + (cons* clang
> + libdisplay-info
> + libinput-minimal
> + libseat
> + libxkbcommon
> + mesa
> + pango
> + pipewire
> + wayland
> + (cargo-inputs 'niri)))
> + (home-page "https://github.com/YaLTeR/niri")
> + (synopsis "Scrollable-tiling Wayland compositor")
> + (description
> + "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
> +scrollable format. It is considered stable for daily use and performs most
> +functions expected of a Wayland compositor.")
> + (license license:gpl3)))
I think this should be gpl3+, as the license file carries the original
'or any later version' and apparently no further directions to limit
gpl3 only.
> +@end lisp
> +
> +@item
> +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> +workspace, build argument @code{#:cargo-package-crates} is required.
Hm. Does that mean we're still in a situation where some inputs are not
true Guix inputs, and some Rust-packaging oddity that our tooling
doesn't handle (e.g. 'guix graph' ?).
> +@lisp
> +(define-public rust-pipewire-for-niri
> + (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> + (revision "0"))
> + (package
> + (name "rust-pipewire")
> + (version (git-version "0.8.0" revision commit))
> + (source
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> + (commit commit)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:skip-build? #t
> + #:cargo-package-crates
> + ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> + (inputs (cargo-inputs 'rust-pipewire-for-niri))
> + (home-page "https://pipewire.org/")
> + (synopsis "Rust bindings for PipeWire")
> + (description "This package provides Rust bindings for PipeWire.")
> + (license license:expat))))
> +@end lisp
> +
> +Don't forget to modify all workspace members in @code{(gnu packages
> +rust-crates)}:
> +
> +@lisp
> +(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@dots{}
> +(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@end lisp
> +@end enumerate
> +
> +Example 3: @code{fish}, which combines two build systems.
> +
> +@enumerate
> +@item
> +When building Rust packages in other build systems, we need to add @code{rust}
> +and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
> +build systems, and apply necessary build phases from @code{cargo-build-system}.
> +
> +@lisp
> +(define-public fish
> + (package
> + (name "fish")
> + (version "4.0.0")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (string-append "https://github.com/fish-shell/fish-shell/"
> + "releases/download/" version "/"
> + "fish-" version ".tar.xz"))
> + (sha256
> + (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
> + (build-system cmake-build-system)
> + (inputs
> + (cons* fish-foreign-env ncurses pcre2
> + python ;for fish_config and manpage completions
> + (cargo-inputs 'fish)))
> + (native-inputs
> + (list doxygen groff ;for 'fish --help'
> + pkg-config
> + procps ;for the test suite
> + rust
> + `(,rust "cargo")))
> + (arguments
> + (list #:imported-modules
> + (append %cargo-build-system-modules
> + %cmake-build-system-modules)
> + #:modules
> + (((guix build cargo-build-system) #:prefix cargo:)
> + (guix build cmake-build-system)
> + (guix build utils))
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("git.*tag.*,")
> + "version = \"*\","))))
> + (add-after 'unpack 'prepare-cargo-build-system
> + (lambda args
> + (for-each
> + (lambda (phase)
> + (format #t "Running cargo phase: ~a~%" phase)
> + (apply (assoc-ref cargo:%standard-phases phase)
> + args))
> + '(unpack-rust-crates
> + configure
> + check-for-pregenerated-files
> + patch-cargo-checksums)))))))
> + (synopsis "The friendly interactive shell")
> + (description
> + "Fish (friendly interactive shell) is a shell focused on interactive use,
> +discoverability, and friendliness. Fish has very user-friendly and powerful
> +tab-completion, including descriptions of every completion, completion of
> +strings with wildcards, and many completions for specific commands. It also
> +has extensive and discoverable help. A special @@command@{help@} command gives
> +access to all the fish documentation in your web browser. Other features
> +include smart terminal handling based on terminfo, an easy to search history,
> +and syntax highlighting.")
> + (home-page "https://fishshell.com/")
> + (license license:gpl2)))
> +@end lisp
> +@end enumerate
Thanks for taking the time to author all of this!
--
Maxim
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Thu, 20 Mar 2025 12:04:02 GMT) Full text and rfc822 format available.Message #170 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Cc: Ludovic Courtès <ludo <at> gnu.org>, 77093 <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il> Subject: Re: [bug#77093] [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Thu, 20 Mar 2025 20:02:37 +0800
On Wed, 19 Mar 2025 21:41:39 +0800,
Maxim Cournoyer wrote:
>
> [...]
> > +@end lisp
> > +
> > +@item
> > +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> > +workspace, build argument @code{#:cargo-package-crates} is required.
>
> Hm. Does that mean we're still in a situation where some inputs are not
> true Guix inputs, and some Rust-packaging oddity that our tooling
> doesn't handle (e.g. 'guix graph' ?).
This argument is meant to specify a order, workspace members may depend on each
other, so the dependencies should be packaged and put into "guix-vendor" first.
The naming might be confusing but I can't come up with a better one :(
cargo- : cargo-build-system specific
package- : package phase
crates : list of crate names
Rust <package>s will have all their Rust dependencies in inputs field but most
of them are origins, so there won't be much difference in terms of ‘guix graph’.
--8<---------------cut here---------------start------------->8---
$ guix graph niri | grep '"rust-.*'
"140032307245408" [label = "rust-bindgen-cli <at> 0.71.1", shape = box, fontname = sans];
"140032307245936" [label = "rust-cbindgen <at> 0.26.0", shape = box, fontname = sans];
"140032240396352" [label = "rust-ring <at> 0.17.8", shape = box, fontname = sans];
"140032307276192" [label = "rust-cargo-c <at> 0.10.9+cargo-0.85.0", shape = box, fontname = sans];
"140032240396176" [label = "rust-smithay <at> 0.4.0-0.0cd3345", shape = box, fontname = sans];
"140032240396880" [label = "rust-pipewire <at> 0.8.0-0.fd3d8f7", shape = box, fontname = sans];
$ guix graph niri --type=bag-with-origins | grep '"rust-.*'
"/gnu/store/kzl3bv8yfczwg18kcx649p7nmf8n2m55-rust-bindgen-cli-0.71.1.drv" [label = "rust-bindgen-cli <at> 0.71.1", shape = box, fontname = sans];
"/gnu/store/d28511hifkbpzr8hrnngb5hlsy7js2na-rust-windows-x86-64-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/7kh3innc661lf8fcya6nziglv93hy2yl-rust-windows-x86-64-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/fc73asklfa1q29g46bc8gpkggqwpc83w-rust-windows-x86-64-gnu-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-gnu-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/ca8laa955hw96r49dg5pzsl5pjxqs37j-rust-windows-i686-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-i686-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/9czqdmfglvmag8xi69a4jyabblhff72j-rust-windows-i686-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-i686-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/wjmvcdbqjm1zxqaih1c5f52d7990h40p-rust-windows-i686-gnu-0.52.6.tar.zst.drv" [label = "rust-windows-i686-gnu-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/w16561c65nzsqmif3ww634mijfk835gl-rust-windows-aarch64-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-aarch64-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/aaw14iijajsxv5s3p6g58py1ippbr3ql-rust-windows-aarch64-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-aarch64-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/xp4i48sdij8qmq2xwpq8dq5h4ax0mzgs-rust-windows-targets-0.52.6.tar.gz.drv" [label = "rust-windows-targets-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/wlfg3iy55ph6ffwp96sqyzg22isx8573-rust-windows-sys-0.59.0.tar.gz.drv" [label = "rust-windows-sys-0.59.0.tar.gz", shape = box, fontname = sans];
"/gnu/store/q8x8721s5r6rb8nrb19jkwbszrqih26f-rust-winapi-util-0.1.9.tar.gz.drv" [label = "rust-winapi-util-0.1.9.tar.gz", shape = box, fontname = sans];
"/gnu/store/qynjbqr3md18m6q4ynrpvglfa8xjys0d-rust-utf8parse-0.2.2.tar.gz.drv" [label = "rust-utf8parse-0.2.2.tar.gz", shape = box, fontname = sans];
[...]
--8<---------------cut here---------------end--------------->8---
Thanks for the review! I'll prepare v3 soon :)
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, divya <at> subvertising.org, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:01 GMT) Full text and rfc822 format available.Message #173 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 01/17] build/cargo: Pass ‘--offline’ to cargo. Date: Sun, 23 Mar 2025 15:23:05 +0800
This will make error messages more helpful.
* guix/build/cargo-build-system.scm (build,check,package,install): Pass
‘--offline’ to cargo.
Change-Id: Ic95f603b793319f99c9c1fbce43f773bfc8126c0
---
guix/build/cargo-build-system.scm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 5ef6b23abd..41f54f42b7 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -246,7 +246,7 @@ (define* (build #:key
"Build a given Cargo package."
(or skip-build?
(apply invoke
- `("cargo" "build"
+ `("cargo" "build" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -264,7 +264,7 @@ (define* (check #:key
"Run tests for a given Cargo package."
(when tests?
(apply invoke
- `("cargo" "test"
+ `("cargo" "test" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -303,7 +303,7 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" ,@cargo-package-flags))
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
@@ -356,7 +356,8 @@ (define* (install #:key
;; otherwise cargo will raise an error.
(or skip-build?
(not (has-executable-target?))
- (invoke "cargo" "install" "--no-track" "--path" "." "--root" out
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" "." "--root" out
"--features" (string-join features)))
(when install-source?
--
2.49.0
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, divya <at> subvertising.org, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:02 GMT) Full text and rfc822 format available.Message #176 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 02/17] build/cargo: Print out all non-empty binary files. Date: Sun, 23 Mar 2025 15:23:06 +0800
* guix/build/cargo-build-system.scm (%standard-phases): Move
'unpack-rust-crates after 'unpack.
Move 'check-for-pregenerated-files after 'configure.
(check-for-pregenerated-files): Only check non-empty files.
Print out binary files.
Run in parallel.
Don't fail to keep compatibility for phase order change.
Change-Id: I0a332fe843e97687324bd908fa111422a63e475d
---
guix/build/cargo-build-system.scm | 31 +++++++++++++++++++++++++------
1 file changed, 25 insertions(+), 6 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41f54f42b7..58238992b8 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -29,9 +29,11 @@ (define-module (guix build cargo-build-system)
#:use-module ((guix build utils) #:hide (delete))
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (%standard-phases
@@ -111,12 +113,29 @@ (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
(define (rust-package? name)
(string-prefix? "rust-" name))
-(define* (check-for-pregenerated-files #:rest _)
+(define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
"Check the source code for files which are known to generally be bundled
libraries or executables."
- (let ((pregenerated-files (find-files "." "\\.(a|dll|dylib|exe|lib)$")))
- (when (not (null-list? pregenerated-files))
- (error "Possible pre-generated files found:" pregenerated-files))))
+ (format #t "Searching for binary files...~%")
+ (let ((known-pattern (make-regexp "\\.(a|dll|dylib|exe|lib)$"))
+ (empty-file?
+ (lambda (filename)
+ (call-with-ascii-input-file filename
+ (lambda (p)
+ (eqv? #\0 (read-char p)))))))
+ (n-par-for-each
+ (if parallel-build?
+ (parallel-job-count)
+ 1)
+ (lambda (file)
+ (unless (empty-file? file)
+ ;; Print out binary files.
+ (false-if-exception (invoke "grep" "-IL" "." file))
+ ;; Warn about known pre-generated files.
+ ;; Not failing here for compatibility with existing packages.
+ (when (regexp-exec known-pattern file)
+ (format #t "error: Possible pre-generated file found: ~a~%" file))))
+ (find-files "."))))
(define* (configure #:key inputs
target system
@@ -380,8 +399,8 @@ (define %standard-phases
(replace 'check check)
(replace 'install install)
(add-after 'build 'package package)
- (add-after 'unpack 'check-for-pregenerated-files check-for-pregenerated-files)
- (add-after 'check-for-pregenerated-files 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'unpack 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'configure 'check-for-pregenerated-files check-for-pregenerated-files)
(add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
(define* (cargo-build #:key inputs (phases %standard-phases)
--
2.49.0
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, ludo <at> gnu.org, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:02 GMT) Full text and rfc822 format available.Message #179 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 00/17] New Rust packaging workflow based on lockfile importer. Date: Sun, 23 Mar 2025 15:23:04 +0800
V2 -> V3:
* Add Efraim's etc/teams/rust/audit-rust-crates script.
* Add (guix import crate cargo-lock) to Rust team's scope.
* ‘definer’ -> ‘define-prefix’
* Adjust documentation and move examples to Guix Cookbook.
* Document ‘cargo-inputs’ and remove mentioning of files under etc/teams/rust.
Diff inserted at end.
Efraim Flashner (1):
build/cargo: Don't try to unpack sanity-check.py.
Hilton Chain (16):
build/cargo: Pass ‘--offline’ to cargo.
build/cargo: Print out all non-empty binary files.
build-system: cargo: Support packaging Cargo workspace.
build-system: cargo: Support installing Cargo workspace.
build/cargo: Set default value of arguments for build phases.
build/cargo: Support non-workspace directory source inputs.
scripts: import: Document argument for ‘--insert’ option in help
message.
scripts: import: Add two newlines for ‘--insert’ option.
scripts: import: Support expressions defined by 'define.
scripts: import: Pass "--insert" to importers.
scripts: import: Skip existing definition for ‘--insert’ option.
import: crate: crate-name->package-name: Move to (guix build-system
cargo).
build-system: cargo: Add ‘cargo-inputs’.
import: crate: Add Cargo.lock parser.
import: crate: Add ‘--lockfile’ option.
doc: Document lockfile importer based Rust packaging workflow.
Makefile.am | 1 +
doc/contributing.texi | 91 +++++---
doc/guix-cookbook.texi | 352 ++++++++++++++++++++++++++++++
doc/guix.texi | 24 ++
etc/teams.scm | 1 +
etc/teams/rust/audit-rust-crates | 70 ++++++
etc/teams/rust/cleanup-crates.sh | 37 ++++
etc/teams/rust/rust-crates.tmpl | 42 ++++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 42 ++++
gnu/packages/rust-sources.scm | 29 +++
guix/build-system/cargo.scm | 59 ++++-
guix/build/cargo-build-system.scm | 150 +++++++++----
guix/import/crate.scm | 81 ++++++-
guix/import/crate/cargo-lock.scm | 105 +++++++++
guix/scripts/import.scm | 54 +++--
guix/scripts/import/crate.scm | 58 ++++-
guix/utils.scm | 29 ++-
tests/crate.scm | 88 ++++++++
19 files changed, 1209 insertions(+), 106 deletions(-)
create mode 100755 etc/teams/rust/audit-rust-crates
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
create mode 100644 guix/import/crate/cargo-lock.scm
base-commit: 127e1e89a83785f4ad8362bdc76b0d850279cf2f
--
2.49.0
diff --git a/doc/contributing.texi b/doc/contributing.texi
index 837074dead..17f22390d5 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,12 +1600,6 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-As currently there's no achievable way to reuse Rust packages as pre-compiled
-inputs for other packages at a distribution scale, and our previous approach on
-making all Rust dependencies as full packages doesn't cope well with Rust
-ecosystem thus causing issues on both contribution and maintenance sides, we
-have switched to a different packaging model.
-
Rust programs (binary crates) and dependencies (library crates) are treated
separately. We put our main efforts into programs and only package Rust
dependencies as sources, utilizing automation with a manual focus on unbundling
@@ -1625,19 +1619,11 @@ Rust Crates
@item
@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
-import, crate, @code{--lockfile=@var{file}}}).
+import, @code{crate}, @code{--lockfile=@var{file}}}).
Imported definitions must be checked and have vendored sources unbundled before
-contributing to Guix. Naturally, this module serves as a store for both sources
-and unbundling strategies.
-
-This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
-one version containing all changes from other branches, so that the maintained
-version can be used directly in case of merge conflicts, thus coordination is
-required for other committers to modify it.
-
-Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
-script @file{etc/teams/rust/cleanup-crates.sh} for this module.
+being contributed to Guix. This module is managed by the Rust team
+(@pxref{Teams}).
@item
@code{(gnu packages rust-sources)}, storing more complex definitions that need
@@ -1661,338 +1647,28 @@ Rust Crates
In practice we are usually packaging development snapshots of Rust dependencies
specifically for some Rust programs, and can't simply identity them by version.
-In this case we can use a @code{for-@var{program}} suffix. Examples@:
-@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
-@end enumerate
-
-Let's demonstrate the packaging workflow by examples, note that package-specific
-issues are not involved here.
-
-In preparation, we'll add the following packages to our environment:
-
-@example
-guix shell rust rust:cargo cargo-audit cargo-license
-@end example
-
-Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
-repository @uref{https://crates.io, crates.io}.
-
-@enumerate
-@item
-We can generate a draft definition via the crates.io importer (@pxref{Invoking
-guix import, crate}). In the end we'll have the following definiton:
-
-@lisp
-(define-public cargo-audit
- (package
- (name "cargo-audit")
- (version "0.21.2")
- (source
- (origin
- (method url-fetch)
- (uri (crate-uri "cargo-audit" version))
- (file-name (string-append name "-" version ".tar.gz"))
- (sha256
- (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
- (build-system cargo-build-system)
- (arguments (list #:install-source? #f))
- (inputs (cargo-inputs 'cargo-audit))
- (home-page "https://rustsec.org/")
- (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
- (description
- "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
-audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
- (license (list license:asl2.0 license:expat))))
-@end lisp
-
-@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
-facilitating dependency modification. @code{'cargo-audit} used here must be a
-unique identifier, usually same to the program's variable name.
-
-@item
-Unpack package source and navigate to the unpacked directory, then execute the
-following commands:
-
-@example
-cargo generate-lockfile
-cargo audit
-cargo license
-@end example
-
-@command{cargo generate-lockfile} updates dependencies to compatible versions,
-@command{cargo audit} checks known vulnerabilities and @command{cargo license}
-checks licenses of all dependencies.
-
-We must have an acceptable output of @command{cargo audit} and ensure all
-dependencies are licensed with our supported licenses (@pxref{Defining Packages,
-@code{license}}).
-
-@item
-Import dependencies from previously generated lockfile:
-
-@example
-guix import --insert=gnu/packages/rust-crates.scm \
- crate --lockfile=/path/to/Cargo.lock cargo-audit
-@end example
-
-@code{cargo-audit} used here must be consistent with the identifier used for
-@code{cargo-inputs}.
-
-At this stage, @code{cargo-audit} is buildable.
-
-@item
-Finally we'll unbundle vendored sources. The lockfile importer inserts
-@code{TODO:} comments to dependencies with high probability of bundled sources.
-@code{cargo-build-system} also performs additional check in its
-@code{check-for-pregenerated-files} phase:
-
-@example
-$ ./pre-inst-env guix build cargo-audit
-@dots{}
-starting phase `check-for-pregenerated-files'
-Searching for binary files...
-./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
-./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
-./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
-./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
-./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
-./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
-@dots{}
-@end example
-
-Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
-can still select them in Guix command-line interface through expression:
-
-@example
-guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
-@end example
-
-For most dependencies, a snippet is sufficient:
-
-@lisp
-(define rust-bzip2-sys-0.1.13+1.0.8
- (crate-source "bzip2-sys" "0.1.13+1.0.8"
- "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
- #:snippet
- '(begin
- (delete-file-recursively "bzip2-1.0.8")
- (delete-file "build.rs")
- ;; Inspired by Debian's patch.
- (with-output-to-file "build.rs"
- (lambda _
- (format #t "fn main() @{~@@
- println!(\"cargo:rustc-link-lib=bz2\");~@@
- @}~%"))))))
-@end lisp
-
-In a more complex case, where unbundling one dependency requires other packages,
-we should package the dependency in @code{(gnu packages rust-sources)} first and
-point the imported definition to a symbol with the same name.
-
-For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
-rust-sources)}, then the imported definition in @code{(gnu packages
-rust-crates)} should be modified to point to it:
-
-@lisp
-(define rust-ring-0.17.14 'rust-ring-0.17)
-@end lisp
-
-When one dependency can be safely removed, modify it to @code{#f}.
-
-@lisp
-(define rust-openssl-src-300.4.2+3.4.1 #f)
-@end lisp
-
-Such modifications are processed by @code{cargo-inputs} procedure.
-
-@code{cargo-inputs} can also be configured by keywoard arguments
-@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
-resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
-packages rust-sources)}, module to resolve modified symbols), which are useful
-to maintain an independent channel.
+In this case we can use a @code{for-@var{program}} suffix, for example,
+@code{rust-pipewire-for-niri} and @code{rust-pubgrub-for-uv}.
@end enumerate
-Example 2: @code{niri}, which is not available on crates.io and depends on
-development snapshots (also Cargo workspaces in this example).
+Rust dependencies are not referenced directly. @code{(guix build-sytem cargo)}
+provides a @code{cargo-inputs} procedure to create an input list, in combination
+with the lockfile importer.
-@enumerate
-@item
-As we can't ensure compatibility of a development snapshot, before executing
-@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
-in a known working revision.
+@cindex cargo-inputs
+@deffn {Procedure} cargo-inputs @var{name} @
+ [#:crates-module '(gnu packages rust-crates)] @
+ [#:sources-module '(gnu packages rust-sources)]
+Given symbol @code{'@var{name}}, resolve variable
+@code{@var{name}-cargo-inputs}, an input list, in @var{crates-module}, return
+its copy with @code{#f} removed and symbols resolved to variables defined in
+@var{sources-module} if the input list exists, otherwise return an empty list.
-To use our packaged development snapshots, it's also necessary to modify
-@file{Cargo.toml} in build environment, the substitution pattern is
-package-specific.
-
-@code{cargo-inputs} returns a list, all list operations apply to it too.
-
-@lisp
-(define-public niri
- (package
- (name "niri")
- (version "25.02")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/YaLTeR/niri")
- (commit (string-append "v" version))))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
- (build-system cargo-build-system)
- (arguments
- (list #:install-source? #f
- #:phases
- #~(modify-phases %standard-phases
- (add-after 'unpack 'use-guix-vendored-dependencies
- (lambda _
- (substitute* "Cargo.toml"
- (("# version =.*")
- "version = \"*\"")
- (("git.*optional")
- "version = \"*\", optional")
- (("^git = .*")
- "")))))))
- (native-inputs
- (list pkg-config))
- (inputs
- (cons* clang
- libdisplay-info
- libinput-minimal
- libseat
- libxkbcommon
- mesa
- pango
- pipewire
- wayland
- (cargo-inputs 'niri)))
- (home-page "https://github.com/YaLTeR/niri")
- (synopsis "Scrollable-tiling Wayland compositor")
- (description
- "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
-scrollable format. It is considered stable for daily use and performs most
-functions expected of a Wayland compositor.")
- (license license:gpl3)))
-@end lisp
-
-@item
-@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
-workspace, build argument @code{#:cargo-package-crates} is required.
-
-@lisp
-(define-public rust-pipewire-for-niri
- (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
- (revision "0"))
- (package
- (name "rust-pipewire")
- (version (git-version "0.8.0" revision commit))
- (source
- (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
- (commit commit)))
- (file-name (git-file-name name version))
- (sha256
- (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
- (build-system cargo-build-system)
- (arguments
- (list #:skip-build? #t
- #:cargo-package-crates
- ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
- (inputs (cargo-inputs 'rust-pipewire-for-niri))
- (home-page "https://pipewire.org/")
- (synopsis "Rust bindings for PipeWire")
- (description "This package provides Rust bindings for PipeWire.")
- (license license:expat))))
-@end lisp
-
-Don't forget to modify all workspace members in @code{(gnu packages
-rust-crates)}:
-
-@lisp
-(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
-(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
-@dots{}
-(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
-(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
-@end lisp
-@end enumerate
-
-Example 3: @code{fish}, which combines two build systems.
-
-@enumerate
-@item
-When building Rust packages in other build systems, we need to add @code{rust}
-and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
-build systems, and apply necessary build phases from @code{cargo-build-system}.
+@var{name} must be consistent with the one used in lockfile importer invocation.
+@end deffn
-@lisp
-(define-public fish
- (package
- (name "fish")
- (version "4.0.0")
- (source
- (origin
- (method url-fetch)
- (uri (string-append "https://github.com/fish-shell/fish-shell/"
- "releases/download/" version "/"
- "fish-" version ".tar.xz"))
- (sha256
- (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
- (build-system cmake-build-system)
- (inputs
- (cons* fish-foreign-env ncurses pcre2
- python ;for fish_config and manpage completions
- (cargo-inputs 'fish)))
- (native-inputs
- (list doxygen groff ;for 'fish --help'
- pkg-config
- procps ;for the test suite
- rust
- `(,rust "cargo")))
- (arguments
- (list #:imported-modules
- (append %cargo-build-system-modules
- %cmake-build-system-modules)
- #:modules
- (((guix build cargo-build-system) #:prefix cargo:)
- (guix build cmake-build-system)
- (guix build utils))
- #:phases
- #~(modify-phases %standard-phases
- (add-after 'unpack 'use-guix-vendored-dependencies
- (lambda _
- (substitute* "Cargo.toml"
- (("git.*tag.*,")
- "version = \"*\","))))
- (add-after 'unpack 'prepare-cargo-build-system
- (lambda args
- (for-each
- (lambda (phase)
- (format #t "Running cargo phase: ~a~%" phase)
- (apply (assoc-ref cargo:%standard-phases phase)
- args))
- '(unpack-rust-crates
- configure
- check-for-pregenerated-files
- patch-cargo-checksums)))))))
- (synopsis "The friendly interactive shell")
- (description
- "Fish (friendly interactive shell) is a shell focused on interactive use,
-discoverability, and friendliness. Fish has very user-friendly and powerful
-tab-completion, including descriptions of every completion, completion of
-strings with wildcards, and many completions for specific commands. It also
-has extensive and discoverable help. A special @@command@{help@} command gives
-access to all the fish documentation in your web browser. Other features
-include smart terminal handling based on terminfo, an easy to search history,
-and syntax highlighting.")
- (home-page "https://fishshell.com/")
- (license license:gpl2)))
-@end lisp
-@end enumerate
+For a more detailed packaging workflow, @pxref{Packaging Rust Crates,,,
+guix-cookbook, GNU Guix Cookbook}.
@node Elm Packages
diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index d9b98a2ab3..1f0e5eee54 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -103,6 +103,7 @@ Top
Packaging
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
Packaging Tutorial
@@ -127,6 +128,16 @@ Top
* Automatic update::
* Inheritance::
+Packaging Workflows
+
+* Packaging Rust Crates::
+
+Packaging Rust Crates
+
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+
System Configuration
* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
@@ -514,6 +525,7 @@ Packaging
@menu
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
@end menu
@node Packaging Tutorial
@@ -1593,6 +1605,346 @@ References
@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
@end itemize
+@node Packaging Workflows
+@section Packaging Workflows
+
+The following sections are real life examples on working with specific build
+systems, serving as extensions to the concise packaging guidelines
+(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
+
+@menu
+* Packaging Rust Crates::
+@end menu
+
+@node Packaging Rust Crates
+@subsection Packaging Rust Crates
+
+In preparation, add the following packages to our environment:
+
+@example
+$ guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+@menu
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+@end menu
+
+@node Common Workflow for Packaging Rust Crates
+@subsubsection Common Workflow for Packaging Rust Crates
+
+In this example, we'll package @code{cargo-audit}, which is published on the
+@uref{https://crates.io, crates.io} Rust package repository. All its
+dependencies are on crates.io as well.
+
+@enumerate
+@item
+Since @code{cargo-audit} is available on crates.io, We can generate a draft
+definition via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU
+Guix Reference Manual}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+The symbol used in @code{cargo-inputs}, @code{'cargo-audit} here, must be a
+unique identifier, usually matching the variable name of the package.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+$ cargo generate-lockfile
+$ cargo audit
+$ cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining
+Packages,,, guix, GNU Guix Reference Manual}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+$ guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, package @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them via the Guix command-line interface through expression:
+
+@example
+$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+reference it by a symbol in the imported definition.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to a matching symbol.
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+@end enumerate
+
+@node Development Snapshots and Cargo Workspaces
+@subsubsection Development Snapshots and Cargo Workspaces
+
+In this example, we'll package @code{niri}, which depends on development
+snapshots (also Cargo workspaces here).
+
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+to a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in a build phase, with a package-specific substitution
+pattern.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3+)))
+@end lisp
+
+@code{niri} has Cargo workspace dependencies. When packaging a Cargo workspace,
+argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+
+@node Rust Programs With Mixed Build Systems
+@subsubsection Rust Programs With Mixed Build Systems
+
+In this example, we'll package @code{fish}, which combines two build systems.
+
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+
+
@c *********************************************************************
@node System Configuration
@chapter System Configuration
diff --git a/etc/teams.scm b/etc/teams.scm
index 6bddbd91fa..c4bee8d3d9 100755
--- a/etc/teams.scm
+++ b/etc/teams.scm
@@ -389,6 +389,7 @@ (define-team rust
"guix/build/cargo-utils.scm"
"guix/build-system/cargo.scm"
"guix/import/crate.scm"
+ "guix/import/crate/cargo-lock.scm"
"guix/scripts/import/crate.scm"
"tests/crate.scm")))
diff --git a/etc/teams/rust/audit-rust-crates b/etc/teams/rust/audit-rust-crates
new file mode 100755
index 0000000000..d5546fd1e1
--- /dev/null
+++ b/etc/teams/rust/audit-rust-crates
@@ -0,0 +1,70 @@
+#!/usr/bin/env -S gawk -f
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Efraim Flashner <efraim <at> flashner.co.il>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+# To run:
+# ./etc/teams/rust/audit-rust-crates ./path/to/file.scm
+# Prints the output of cargo-audit to the shell.
+
+# Make sure we have cargo-audit in our PATH
+BEGIN {
+ if (system("which cargo-audit 1> /dev/null"))
+ exit 1;
+ # Parse a record at a time.
+ RS = "\n\n"
+ cargoAudit = "cargo-audit audit --file -"
+}
+
+# Check the crate-source origin-only inputs
+/crate-source/ {
+ for(i=3; i <= NF-2; i++) {
+ if($i == "(crate-source") {
+ cargoLock = cargoLock "[[package]]\nname = " $(i+1) "\nversion = " $(i+2) "\n"
+ next
+ }
+ }
+}
+
+# Check the crates packaged from crates.io tarballs
+/crate-uri/ {
+ for(i=3; i <= NF; i++) {
+ if($i == "(version")
+ crateVersion = $(i+1)
+ if($i == "(crate-uri")
+ crateName = $(i+1)
+ }
+ gsub(/)/, "", crateVersion)
+ cargoLock = cargoLock "[[package]]\nname = " crateName "\nversion = " crateVersion "\n"
+}
+
+# The xxxx-cargo-input variables have a set style
+# TODO: Replace the last dash between the name and the version with a space!
+# This doesn't take into account swapping between "-" and "_" so we skip it.
+#( $2 ~ /-cargo-inputs/ ) {
+# sub(/-cargo-inputs/, "", $2)
+# gsub(/)/, "", $0)
+# gsub(/rust-/, "", $0)
+# #gensub(/([[:alpha:]])-([[:digit:]]+)/, "\\1 \\2", "g", $i)
+# print "[[package]]\nname = \"" $2 "\"\nversion = \"1.0.0\"\ndependencies = ["
+# for (i = 4; i <= NF; i++) {
+# print "\"" $i "\","
+# }
+# print "]"
+#}
+
+END { print cargoLock | cargoAudit }
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
index cd4c2462b9..eca37ca00c 100755
--- a/etc/teams/rust/cleanup-crates.sh
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -1,5 +1,4 @@
#!/bin/sh
-
# GNU Guix --- Functional package management for GNU
# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
#
@@ -28,7 +27,7 @@ do
(begin
(use-modules (guix utils))
(let ((source-properties
- (find-definition-location \"$FILE\" '$crate #:definer 'define)))
+ (find-definition-location \"$FILE\" '$crate #:define-prefix 'define)))
(and=> source-properties delete-expression)))" |
guix repl -t machine
fi
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 47d4f10969..bbfa2f933b 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -78,9 +78,11 @@ (define* (crate-source name version hash #:key (patches '()) (snippet #f))
(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
(sources-module '(gnu packages rust-sources)))
- "Given NAME, resolve input list 'NAME-cargo-inputs' in CRATES-MODULE, return
-its copy with #f removed and symbols resolved to variables defined in
-SOURCES-MODULE, if the list exists, otherwise return an empty list."
+
+ "Given symbol NAME, resolve variable 'NAME-cargo-inputs', an input list, in
+CRATES-MODULE, return its copy with #f removed and symbols resolved to
+variables defined in SOURCES-MODULE if the input list exists, otherwise return
+an empty list."
(let loop ((inputs
(catch #t
(lambda ()
@@ -101,7 +103,7 @@ (define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
((symbol? input)
(cons (module-ref (resolve-interface sources-module) input)
result))
- ;; Else: no change.
+ ;; Else: keep it.
(else
(cons input result)))))))))
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41adc03752..1012d7f401 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -350,17 +350,16 @@ (define* (package #:key
(if (null? cargo-package-crates)
(apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
- (begin
- (for-each
- (lambda (pkg)
- (apply invoke "cargo" "package" "--offline" "--package" pkg
- cargo-package-flags)
- (for-each
- (lambda (crate)
- (invoke "tar" "xzf" crate "-C" vendor-dir))
- (find-files "target/package" "\\.crate$"))
- (patch-cargo-checksums #:vendor-dir vendor-dir))
- cargo-package-crates)))
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (find-files "target/package" "\\.crate$"))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index f6e2985ed9..1bddac15c6 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -126,21 +126,22 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let* ((importer-definers
+ (let* ((define-prefixes
`(,@(if (member importer '("crate"))
'(define)
'())
define-public))
- (definer? (cut member <> importer-definers))
+ (define-prefix? (cut member <> define-prefixes))
(find-and-insert
(lambda (expr)
(match expr
- (((? definer? definer) term _ ...)
+ (((? define-prefix? define-prefix) term _ ...)
;; Skip existing definition.
- (unless (find-definition-location file term #:definer definer)
+ (unless (find-definition-location
+ file term #:define-prefix define-prefix)
(let ((source-properties
(find-definition-insertion-location
- file term #:definer definer)))
+ file term #:define-prefix define-prefix)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index 77ec6d992a..3f85320845 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -522,19 +522,19 @@ (define (insert-expression source-properties expr)
(edit-expression source-properties insert)))
(define* (find-definition-location file term
- #:key (definer 'define-public)
+ #:key (define-prefix 'define-public)
(pred string=))
- "Search in FILE for a top-level definition defined by DEFINER with defined
-term comparing to TERM through PRED. Return the location if PRED returns #t,
-or #f otherwise."
+ "Search in FILE for a top-level definition defined by DEFINE-PREFIX with
+defined term comparing to TERM through PRED. Return the location if PRED
+returns #t, or #f otherwise."
(let ((search-term (symbol->string term))
- (definer? (cut eq? definer <>)))
+ (define-prefix? (cut eq? define-prefix <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (((? definer?) current-term _ ...)
+ (((? define-prefix?) current-term _ ...)
(pred (symbol->string current-term)
search-term))
((? eof-object?) #t)
@@ -543,11 +543,13 @@ (define* (find-definition-location file term
(syntax-source syntax))))))))
(define* (find-definition-insertion-location file term
- #:key (definer 'define-public))
- "Search in FILE for a top-level definition defined by DEFINER with defined
-term alphabetically succeeds TERM. Return the location if found, or #f
+ #:key
+ (define-prefix 'define-public))
+ "Search in FILE for a top-level definition defined by DEFINE-PREFIX with
+defined term alphabetically succeeds TERM. Return the location if found, or #f
otherwise."
- (find-definition-location file term #:definer definer #:pred string>))
+ (find-definition-location
+ file term #:define-prefix define-prefix #:pred string>))
;;;
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, divya <at> subvertising.org, ludo <at> gnu.org, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:03 GMT) Full text and rfc822 format available.Message #182 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 03/17] build-system: cargo: Support packaging Cargo workspace. Date: Sun, 23 Mar 2025 15:23:07 +0800
* guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-package-crates]: New argument.
* guix/build/cargo-build-system.scm (package): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
---
doc/guix.texi | 7 +++++++
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 21 +++++++++++++++++++--
3 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0ca109a214..0089f9e60f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9363,6 +9363,13 @@ Build Systems
the binaries defined by the crate. Unless @code{install-source? #f} is
defined it will also install a source crate repository of itself and unpacked
sources, to ease in future hacking on rust packages.
+
+This build system supports cargo workspaces. Parameter
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+library crates to package in the @code{package} phase. Specified crates are
+packaged from left to right, in case there's dependency among them. For
+example, specifying @code{''("pcre2-sys" "pcre2")} will package
+@code{"pcre2-sys"} first and then @code{"pcre2"}.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 452f7f78d0..4f6d46e70c 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -94,6 +94,7 @@ (define* (cargo-build name inputs
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(features ''())
(skip-build? #f)
@@ -122,6 +123,7 @@ (define* (cargo-build name inputs
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
@@ -154,6 +156,7 @@ (define* (cargo-cross-build name
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(cargo-target (cargo-triplet (or target system)))
(features ''())
@@ -185,6 +188,7 @@ (define* (cargo-cross-build name
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 58238992b8..51a45b4eab 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -300,11 +300,16 @@ (define* (package #:key
source
skip-build?
install-source?
+ (cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
+ (vendor-dir "guix-vendor")
#:allow-other-keys)
"Run 'cargo-package' for a given Cargo package."
(if install-source?
- (if skip-build?
+ ;; NOTE: Cargo workspace packaging support:
+ ;; #:install-source? #t + #:cargo-package-crates.
+ (if (and (null? cargo-package-crates)
+ skip-build?)
(begin
(install-file source "target/package")
(with-directory-excursion "target/package"
@@ -322,7 +327,19 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+
+ (if (null? cargo-package-crates)
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (find-files "target/package" "\\.crate$"))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
--
2.49.0
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, divya <at> subvertising.org, ludo <at> gnu.org, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:03 GMT) Full text and rfc822 format available.Message #185 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 04/17] build-system: cargo: Support installing Cargo workspace. Date: Sun, 23 Mar 2025 15:23:08 +0800
*guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-install-paths]: New argument.
* guix/build/cargo-build-system.scm (install): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
---
doc/guix.texi | 5 ++++-
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 17 +++++++++++++----
3 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0089f9e60f..7fa986b4b9 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9369,7 +9369,10 @@ Build Systems
library crates to package in the @code{package} phase. Specified crates are
packaged from left to right, in case there's dependency among them. For
example, specifying @code{''("pcre2-sys" "pcre2")} will package
-@code{"pcre2-sys"} first and then @code{"pcre2"}.
+@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
+binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
+for example.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4f6d46e70c..4486c706a1 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -96,6 +96,7 @@ (define* (cargo-build name inputs
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(features ''())
(skip-build? #f)
(parallel-build? #t)
@@ -125,6 +126,7 @@ (define* (cargo-build name inputs
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
@@ -158,6 +160,7 @@ (define* (cargo-cross-build name
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(cargo-target (cargo-triplet (or target system)))
(features ''())
(skip-build? #f)
@@ -190,6 +193,7 @@ (define* (cargo-cross-build name
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 51a45b4eab..67311fc850 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -377,6 +377,7 @@ (define* (install #:key
skip-build?
install-source?
features
+ (cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
(let* ((out (assoc-ref outputs "out"))
@@ -391,10 +392,18 @@ (define* (install #:key
;; Only install crates which include binary targets,
;; otherwise cargo will raise an error.
(or skip-build?
- (not (has-executable-target?))
- (invoke "cargo" "install" "--offline" "--no-track"
- "--path" "." "--root" out
- "--features" (string-join features)))
+ ;; NOTE: Cargo workspace installation support:
+ ;; #:skip-build? #f + #:cargo-install-paths.
+ (and (null? cargo-install-paths)
+ (not (has-executable-target?)))
+ (for-each
+ (lambda (path)
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" path "--root" out
+ "--features" (string-join features)))
+ (if (null? cargo-install-paths)
+ '(".")
+ cargo-install-paths)))
(when install-source?
;; Install crate tarballs and unpacked sources for later use.
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:03 GMT) Full text and rfc822 format available.ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:04 GMT) Full text and rfc822 format available.divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:06 GMT) Full text and rfc822 format available.divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:08 GMT) Full text and rfc822 format available.divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:10 GMT) Full text and rfc822 format available.divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:11 GMT) Full text and rfc822 format available.Message #203 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 05/17] build/cargo: Set default value of arguments for build phases. Date: Sun, 23 Mar 2025 15:28:27 +0800
This makes it easier to use these phases in other bulid systems.
* guix/build/cargo-build-system.scm (unpack-rust-crates,configure,build,package)
(install): Set default value of arguments.
Change-Id: I1dde1b063d8eee57967903abd2fce94574211a0a
---
guix/build/cargo-build-system.scm | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 67311fc850..15eda8396c 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -76,7 +76,8 @@ (define (crate-src? path)
" | cut -d/ -f2"
" | grep -q '^Cargo.toml$'")))))
-(define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
+(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
+ #:allow-other-keys)
(define (inputs->rust-inputs inputs)
"Filter using the label part from INPUTS."
(filter (lambda (input)
@@ -139,7 +140,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(define* (configure #:key inputs
target system
- cargo-target
+ (cargo-target #f)
(vendor-dir "guix-vendor")
#:allow-other-keys)
"Vendor Cargo.toml dependencies as guix inputs."
@@ -258,7 +259,7 @@ (define* (patch-cargo-checksums #:key
(define* (build #:key
parallel-build?
- skip-build?
+ (skip-build? #f)
(features '())
(cargo-build-flags '("--release"))
#:allow-other-keys)
@@ -298,8 +299,8 @@ (define* (check #:key
(define* (package #:key
source
- skip-build?
- install-source?
+ (skip-build? #f)
+ (install-source? #t)
(cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
(vendor-dir "guix-vendor")
@@ -374,9 +375,9 @@ (define* (package #:key
(define* (install #:key
inputs
outputs
- skip-build?
- install-source?
- features
+ (skip-build? #f)
+ (install-source? #t)
+ (features '())
(cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:13 GMT) Full text and rfc822 format available.Message #206 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v3 06/17] build/cargo: Don't try to unpack sanity-check.py. Date: Sun, 23 Mar 2025 15:28:28 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (crate-src?): Also don't try to
check python scripts for Cargo.toml.
Change-Id: I001a89b83d2e472706b1263007be45d1153c140f
---
guix/build/cargo-build-system.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 15eda8396c..f2a7323748 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -64,6 +64,7 @@ (define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
(and (not (directory-exists? path)) ; not a tarball
+ (not (string-suffix? "py" path)) ; sanity-check.py
;; First we print out all file names within the tarball to see if it
;; looks like the source of a crate. However, the tarball will include
;; an extra path component which we would like to ignore (since we're
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:15 GMT) Full text and rfc822 format available.Message #209 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 07/17] build/cargo: Support non-workspace directory source inputs. Date: Sun, 23 Mar 2025 15:28:29 +0800
* guix/build/cargo-build-system.scm (single-crate?): New procedure.
(crate-src?): Support non-workspace directory source inputs.
(configure): Likewise.
Change-Id: If2bd318b44ed765baec69309f25d320edceee116
---
guix/build/cargo-build-system.scm | 60 ++++++++++++++++++++-----------
1 file changed, 39 insertions(+), 21 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index f2a7323748..1012d7f401 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -33,6 +33,7 @@ (define-module (guix build cargo-build-system)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 textual-ports)
#:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -60,22 +61,35 @@ (define (has-executable-target?)
(bin-dep? (lambda (dep) (find bin? (get-kinds dep)))))
(find bin-dep? (manifest-targets))))
+(define (single-crate? dir)
+ "Check if directory DIR contains 'Cargo.toml' and is not a workspace."
+ (let ((manifest-file (in-vicinity dir "Cargo.toml")))
+ (and (file-exists? manifest-file)
+ (not (string-contains
+ (call-with-input-file manifest-file get-string-all)
+ "[workspace]")))))
+
(define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
- (and (not (directory-exists? path)) ; not a tarball
- (not (string-suffix? "py" path)) ; sanity-check.py
- ;; First we print out all file names within the tarball to see if it
- ;; looks like the source of a crate. However, the tarball will include
- ;; an extra path component which we would like to ignore (since we're
- ;; interested in checking if a Cargo.toml exists at the root of the
- ;; archive, but not nested anywhere else). We do this by cutting up
- ;; each output line and only looking at the second component. We then
- ;; check if it matches Cargo.toml exactly and short circuit if it does.
- (apply invoke (list "sh" "-c"
- (string-append "tar -tf " path
- " | cut -d/ -f2"
- " | grep -q '^Cargo.toml$'")))))
+ (if (directory-exists? path)
+ ;; The build system only handles sources containing single crate.
+ ;; Workspaces should be packaged into crates (via 'package phase)
+ ;; and used in inputs.
+ (single-crate? path)
+ (and (not (string-suffix? "py" path)) ;sanity-check.py
+ ;; First we print out all file names within the tarball to see
+ ;; if it looks like the source of a crate. However, the tarball
+ ;; will include an extra path component which we would like to
+ ;; ignore (since we're interested in checking if a Cargo.toml
+ ;; exists at the root of the archive, but not nested anywhere
+ ;; else). We do this by cutting up each output line and only
+ ;; looking at the second component. We then check if it matches
+ ;; Cargo.toml exactly and short circuit if it does.
+ (invoke "sh" "-c"
+ (string-append "tar -tf " path
+ " | cut -d/ -f2"
+ " | grep -q '^Cargo.toml$'")))))
(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
#:allow-other-keys)
@@ -159,14 +173,18 @@ (define* (configure #:key inputs
(and (crate-src? path)
;; Gracefully handle duplicate inputs
(not (file-exists? crate-dir))
- (mkdir-p crate-dir)
- ;; Cargo crates are simply gzipped tarballs but with a .crate
- ;; extension. We expand the source to a directory name we control
- ;; so that we can generate any cargo checksums.
- ;; The --strip-components argument is needed to prevent creating
- ;; an extra directory within `crate-dir`.
- (format #t "Unpacking ~a~%" name)
- (invoke "tar" "xf" path "-C" crate-dir "--strip-components" "1")))))
+ (if (directory-exists? path)
+ (copy-recursively path crate-dir)
+ (begin
+ (mkdir-p crate-dir)
+ ;; Cargo crates are simply gzipped tarballs but with a
+ ;; .crate extension. We expand the source to a directory
+ ;; name we control so that we can generate any cargo
+ ;; checksums. The --strip-components argument is needed to
+ ;; prevent creating an extra directory within `crate-dir`.
+ (format #t "Unpacking ~a~%" name)
+ (invoke "tar" "xf" path "-C" crate-dir
+ "--strip-components" "1")))))))
inputs)
;; For cross-building
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:18 GMT) Full text and rfc822 format available.Message #212 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 08/17] scripts: import: Document argument for ‘--insert’ option in help message. Date: Sun, 23 Mar 2025 15:28:30 +0800
* guix/scripts/import.scm (show-help): Add missing ‘FILE’. Change-Id: I540d5feae3fe49c00e9bd6f7a8649ffe0d6e006d --- guix/scripts/import.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index bbf31baa15..4fad329cd3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -69,7 +69,7 @@ (define (show-help) (display (G_ " -h, --help display this help and exit")) (display (G_ " - -i, --insert insert packages into file alphabetically")) + -i, --insert=FILE insert packages into FILE alphabetically")) (display (G_ " -V, --version display version information and exit")) (newline) -- 2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:21 GMT) Full text and rfc822 format available.Message #215 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 09/17] scripts: import: Add two newlines for ‘--insert’ option. Date: Sun, 23 Mar 2025 15:28:31 +0800
For consistency with ‘insert-expression’.
* guix/scripts/import.scm (guix-import): Add two newlines when inserting.
Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d
---
guix/scripts/import.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 4fad329cd3..58a84d0db7 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -129,6 +129,7 @@ (define-command (guix-import . args)
(let ((port (open-file file "a")))
(pretty-print-with-comments port expr)
(newline port)
+ (newline port)
(close-port port)))))))))
(import-as-definitions importer args find-and-insert)))
((importer args ...)
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:23 GMT) Full text and rfc822 format available.Message #218 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 10/17] scripts: import: Support expressions defined by 'define. Date: Sun, 23 Mar 2025 15:28:32 +0800
* guix/utils.scm (find-definition-location): New procedure.
(find-definition-insertion-location): Define with it.
* guix/scripts/import.scm (import-as-definitions, guix-import): Support
expressions defined by 'define.
Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
---
guix/scripts/import.scm | 21 +++++++++++++++------
guix/utils.scm | 29 +++++++++++++++++++++--------
2 files changed, 36 insertions(+), 14 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 58a84d0db7..6f6354928e 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -30,6 +30,7 @@ (define-module (guix scripts import)
#:use-module (guix read-print)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))
((expressions ...)
(for-each (lambda (expr)
@@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))))
expressions))
(x
@@ -117,13 +120,19 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let ((find-and-insert
+ (let* ((define-prefixes
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public))
+ (define-prefix? (cut member <> define-prefixes))
+ (find-and-insert
(lambda (expr)
(match expr
- (('define-public term _ ...)
+ (((? define-prefix? define-prefix) term _ ...)
(let ((source-properties
- (find-definition-insertion-location
- file term)))
+ (find-definition-insertion-location
+ file term #:define-prefix define-prefix)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index c7c23d9d5b..3f85320845 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -154,6 +154,7 @@ (define-module (guix utils)
edit-expression
delete-expression
insert-expression
+ find-definition-location
find-definition-insertion-location
filtered-port
@@ -520,24 +521,36 @@ (define (insert-expression source-properties expr)
(string-append expr "\n\n" str))))
(edit-expression source-properties insert)))
-(define (find-definition-insertion-location file term)
- "Search in FILE for a top-level public definition whose defined term
-alphabetically succeeds TERM. Return the location if found, or #f
-otherwise."
- (let ((search-term (symbol->string term)))
+(define* (find-definition-location file term
+ #:key (define-prefix 'define-public)
+ (pred string=))
+ "Search in FILE for a top-level definition defined by DEFINE-PREFIX with
+defined term comparing to TERM through PRED. Return the location if PRED
+returns #t, or #f otherwise."
+ (let ((search-term (symbol->string term))
+ (define-prefix? (cut eq? define-prefix <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (('define-public current-term _ ...)
- (string> (symbol->string current-term)
- search-term))
+ (((? define-prefix?) current-term _ ...)
+ (pred (symbol->string current-term)
+ search-term))
((? eof-object?) #t)
(_ #f))
(and (not (eof-object? syntax))
(syntax-source syntax))))))))
+(define* (find-definition-insertion-location file term
+ #:key
+ (define-prefix 'define-public))
+ "Search in FILE for a top-level definition defined by DEFINE-PREFIX with
+defined term alphabetically succeeds TERM. Return the location if found, or #f
+otherwise."
+ (find-definition-location
+ file term #:define-prefix define-prefix #:pred string>))
+
;;;
;;; Keyword arguments.
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:25 GMT) Full text and rfc822 format available.Message #221 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 11/17] scripts: import: Pass "--insert" to importers. Date: Sun, 23 Mar 2025 15:28:33 +0800
So that importers can adjust the file before inserting.
* guix/scripts/import.scm (%standard-import-options): Add ‘--file-to-insert’.
(guix-import): Pass it to importers when ‘--insert’ is set.
Change-Id: I8e7a18ee8e0f96d7fc5688a207a7a5390ad2fa30
---
guix/scripts/import.scm | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 6f6354928e..38349bff3a 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -31,6 +31,7 @@ (define-module (guix scripts import)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-37)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -41,7 +42,12 @@ (define-module (guix scripts import)
;;; Command line options.
;;;
-(define %standard-import-options '())
+(define %standard-import-options
+ (list
+ ;; Hidden option for importer-specific file preprocessing.
+ (option '("file-to-insert") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'file-to-insert arg result)))))
;;;
@@ -140,7 +146,10 @@ (define-command (guix-import . args)
(newline port)
(newline port)
(close-port port)))))))))
- (import-as-definitions importer args find-and-insert)))
+ (import-as-definitions importer
+ (cons (string-append "--file-to-insert=" file)
+ args)
+ find-and-insert)))
((importer args ...)
(let ((print (lambda (expr)
(leave-on-EPIPE
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:26 GMT) Full text and rfc822 format available.Message #224 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 12/17] scripts: import: Skip existing definition for ‘--insert’ option. Date: Sun, 23 Mar 2025 15:28:34 +0800
* guix/scripts/import.scm (guix-import): Skip existing definition for
‘--insert’ option.
Change-Id: I2c4242669f974b263a018ab0cf56538bd7c81d06
---
guix/scripts/import.scm | 23 +++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 38349bff3a..1bddac15c6 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -136,16 +136,19 @@ (define-command (guix-import . args)
(lambda (expr)
(match expr
(((? define-prefix? define-prefix) term _ ...)
- (let ((source-properties
- (find-definition-insertion-location
- file term #:define-prefix define-prefix)))
- (if source-properties
- (insert-expression source-properties expr)
- (let ((port (open-file file "a")))
- (pretty-print-with-comments port expr)
- (newline port)
- (newline port)
- (close-port port)))))))))
+ ;; Skip existing definition.
+ (unless (find-definition-location
+ file term #:define-prefix define-prefix)
+ (let ((source-properties
+ (find-definition-insertion-location
+ file term #:define-prefix define-prefix)))
+ (if source-properties
+ (insert-expression source-properties expr)
+ (let ((port (open-file file "a")))
+ (pretty-print-with-comments port expr)
+ (newline port)
+ (newline port)
+ (close-port port))))))))))
(import-as-definitions importer
(cons (string-append "--file-to-insert=" file)
args)
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:26 GMT) Full text and rfc822 format available.Message #227 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 13/17] import: crate: crate-name->package-name: Move to (guix build-system cargo). Date: Sun, 23 Mar 2025 15:28:35 +0800
* guix/import/crate.scm (crate-name->package-name): Move to...
* guix/build-system/cargo.scm (crate-name->package-name): ...here and export.
Change-Id: Ie9813179d6c40d159956cc1e2ae59a74dea0a42d
---
guix/build-system/cargo.scm | 6 +++++-
guix/import/crate.scm | 5 -----
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4486c706a1..7a07003262 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -44,7 +44,8 @@ (define-module (guix build-system cargo)
%crate-base-url
crate-url
crate-url?
- crate-uri))
+ crate-uri
+ crate-name->package-name))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -58,6 +59,9 @@ (define (crate-uri name version)
to NAME and VERSION."
(string-append crate-url name "/" version "/download"))
+(define (crate-name->package-name name)
+ (downstream-package-name "rust-" name))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 5a8caeb3e1..14e6e28c5b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -481,10 +481,6 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
-(define (crate-name->package-name name)
- (downstream-package-name "rust-" name))
-
-
;;;
;;; Updater
@@ -528,4 +524,3 @@ (define %crate-updater
(description "Updater for crates.io packages")
(pred crate-package?)
(import import-release)))
-
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:27 GMT) Full text and rfc822 format available.Message #230 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 14/17] build-system: cargo: Add ‘cargo-inputs’. Date: Sun, 23 Mar 2025 15:28:36 +0800
* gnu/packages/rust-crates.scm: New file.
* gnu/packages/rust-sources.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them.
* guix/build-system/cargo.scm (crate-source,cargo-inputs): New procedures.
* guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo).
* etc/teams/rust/audit-rust-crates: New file.
* etc/teams/rust/cleanup-crates.sh: New file.
* etc/teams/rust/rust-crates.tmpl: New file.
Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3
---
etc/teams/rust/audit-rust-crates | 70 ++++++++++++++++++++++++++++++++
etc/teams/rust/cleanup-crates.sh | 37 +++++++++++++++++
etc/teams/rust/rust-crates.tmpl | 42 +++++++++++++++++++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 42 +++++++++++++++++++
gnu/packages/rust-sources.scm | 29 +++++++++++++
guix/build-system/cargo.scm | 47 ++++++++++++++++++++-
guix/import/crate.scm | 2 +-
8 files changed, 269 insertions(+), 2 deletions(-)
create mode 100755 etc/teams/rust/audit-rust-crates
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
diff --git a/etc/teams/rust/audit-rust-crates b/etc/teams/rust/audit-rust-crates
new file mode 100755
index 0000000000..d5546fd1e1
--- /dev/null
+++ b/etc/teams/rust/audit-rust-crates
@@ -0,0 +1,70 @@
+#!/usr/bin/env -S gawk -f
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Efraim Flashner <efraim <at> flashner.co.il>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+# To run:
+# ./etc/teams/rust/audit-rust-crates ./path/to/file.scm
+# Prints the output of cargo-audit to the shell.
+
+# Make sure we have cargo-audit in our PATH
+BEGIN {
+ if (system("which cargo-audit 1> /dev/null"))
+ exit 1;
+ # Parse a record at a time.
+ RS = "\n\n"
+ cargoAudit = "cargo-audit audit --file -"
+}
+
+# Check the crate-source origin-only inputs
+/crate-source/ {
+ for(i=3; i <= NF-2; i++) {
+ if($i == "(crate-source") {
+ cargoLock = cargoLock "[[package]]\nname = " $(i+1) "\nversion = " $(i+2) "\n"
+ next
+ }
+ }
+}
+
+# Check the crates packaged from crates.io tarballs
+/crate-uri/ {
+ for(i=3; i <= NF; i++) {
+ if($i == "(version")
+ crateVersion = $(i+1)
+ if($i == "(crate-uri")
+ crateName = $(i+1)
+ }
+ gsub(/)/, "", crateVersion)
+ cargoLock = cargoLock "[[package]]\nname = " crateName "\nversion = " crateVersion "\n"
+}
+
+# The xxxx-cargo-input variables have a set style
+# TODO: Replace the last dash between the name and the version with a space!
+# This doesn't take into account swapping between "-" and "_" so we skip it.
+#( $2 ~ /-cargo-inputs/ ) {
+# sub(/-cargo-inputs/, "", $2)
+# gsub(/)/, "", $0)
+# gsub(/rust-/, "", $0)
+# #gensub(/([[:alpha:]])-([[:digit:]]+)/, "\\1 \\2", "g", $i)
+# print "[[package]]\nname = \"" $2 "\"\nversion = \"1.0.0\"\ndependencies = ["
+# for (i = 4; i <= NF; i++) {
+# print "\"" $i "\","
+# }
+# print "]"
+#}
+
+END { print cargoLock | cargoAudit }
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
new file mode 100755
index 0000000000..eca37ca00c
--- /dev/null
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -0,0 +1,37 @@
+#!/bin/sh
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+FILE=gnu/packages/rust-crates.scm
+PATTERN='^(define rust-'
+
+grep "$PATTERN" $FILE | cut -d' ' -f2 | while IFS= read -r crate
+do
+ if [ "$(grep -wc "$crate" $FILE)" -eq 1 ]; then
+ echo "\
+(begin
+ (use-modules (guix utils))
+ (let ((source-properties
+ (find-definition-location \"$FILE\" '$crate #:define-prefix 'define)))
+ (and=> source-properties delete-expression)))" |
+ guix repl -t machine
+ fi
+done
+
+# Delete extra newlines.
+sed --in-place ':a;N;$!ba;s/\n\n\+/\n\n/g' $FILE
diff --git a/etc/teams/rust/rust-crates.tmpl b/etc/teams/rust/rust-crates.tmpl
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/etc/teams/rust/rust-crates.tmpl
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/local.mk b/gnu/local.mk
index 02de02e65f..8d8bd6ebbe 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -616,6 +616,8 @@ GNU_SYSTEM_MODULES = \
%D%/packages/rush.scm \
%D%/packages/rust.scm \
%D%/packages/rust-apps.scm \
+ %D%/packages/rust-crates.scm \
+ %D%/packages/rust-sources.scm \
%D%/packages/samba.scm \
%D%/packages/sagemath.scm \
%D%/packages/sawfish.scm \
diff --git a/gnu/packages/rust-crates.scm b/gnu/packages/rust-crates.scm
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/gnu/packages/rust-crates.scm
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/packages/rust-sources.scm b/gnu/packages/rust-sources.scm
new file mode 100644
index 0000000000..bf9b91a671
--- /dev/null
+++ b/gnu/packages/rust-sources.scm
@@ -0,0 +1,29 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-sources)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; Cargo workspaces and Rust dependencies requiring external inputs to
+;;; unbundle.
+;;;
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 7a07003262..bbfa2f933b 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -31,6 +31,7 @@ (define-module (guix build-system cargo)
#:use-module (guix gexp)
#:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix download)
#:use-module (guix platform)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
@@ -45,7 +46,9 @@ (define-module (guix build-system cargo)
crate-url
crate-url?
crate-uri
- crate-name->package-name))
+ crate-name->package-name
+ crate-source
+ cargo-inputs))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -62,6 +65,48 @@ (define (crate-uri name version)
(define (crate-name->package-name name)
(downstream-package-name "rust-" name))
+(define* (crate-source name version hash #:key (patches '()) (snippet #f))
+ (origin
+ (method url-fetch)
+ (uri (crate-uri name version))
+ (file-name
+ (string-append (crate-name->package-name name) "-" version ".tar.gz"))
+ (sha256 (base32 hash))
+ (modules '((guix build utils)))
+ (patches patches)
+ (snippet snippet)))
+
+(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
+ (sources-module '(gnu packages rust-sources)))
+
+ "Given symbol NAME, resolve variable 'NAME-cargo-inputs', an input list, in
+CRATES-MODULE, return its copy with #f removed and symbols resolved to
+variables defined in SOURCES-MODULE if the input list exists, otherwise return
+an empty list."
+ (let loop ((inputs
+ (catch #t
+ (lambda ()
+ (module-ref (resolve-interface crates-module)
+ (symbol-append name '-cargo-inputs)))
+ (const '())))
+ (result '()))
+ (if (null? inputs)
+ result
+ (match inputs
+ ((input . rest)
+ (loop rest
+ (cond
+ ;; #f, remove it.
+ ((not input)
+ result)
+ ;; Symbol, resolve it in SOURCES-MODULE.
+ ((symbol? input)
+ (cons (module-ref (resolve-interface sources-module) input)
+ result))
+ ;; Else: keep it.
+ (else
+ (cons input result)))))))))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 14e6e28c5b..a6f247bbae 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -27,7 +27,7 @@
(define-module (guix import crate)
#:use-module (guix base32)
- #:use-module (guix build-system cargo)
+ #:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:28 GMT) Full text and rfc822 format available.Message #233 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Murilo <murilo <at> disroot.org>, Hilton Chain <hako <at> ultrarare.space>, Luis Guilherme Coelho <lgcoelho <at> disroot.org> Subject: [PATCH rust-team v3 15/17] import: crate: Add Cargo.lock parser. Date: Sun, 23 Mar 2025 15:28:37 +0800
* guix/import/crate/cargo-lock.scm: New file.
* Makefile.am (MODULES): Regisiter it.
* etc/teams.scm (rust)[#:scope]: Add it.
* guix/import/crate.scm (cargo-lock->definitions): New procedure.
Co-authored-by: Murilo <murilo <at> disroot.org>
Co-authored-by: Luis Guilherme Coelho <lgcoelho <at> disroot.org>
Change-Id: I95421e9e2ba11a671b4bc4e1323c6d31a1b012c5
---
Makefile.am | 1 +
etc/teams.scm | 1 +
guix/import/crate.scm | 78 +++++++++++++++++++++++
guix/import/crate/cargo-lock.scm | 105 +++++++++++++++++++++++++++++++
4 files changed, 185 insertions(+)
create mode 100644 guix/import/crate/cargo-lock.scm
diff --git a/Makefile.am b/Makefile.am
index 4091a52387..a04e461ea8 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -295,6 +295,7 @@ MODULES = \
guix/import/cpan.scm \
guix/import/cran.scm \
guix/import/crate.scm \
+ guix/import/crate/cargo-lock.scm \
guix/import/egg.scm \
guix/import/elm.scm \
guix/import/elpa.scm \
diff --git a/etc/teams.scm b/etc/teams.scm
index 6bddbd91fa..c4bee8d3d9 100755
--- a/etc/teams.scm
+++ b/etc/teams.scm
@@ -389,6 +389,7 @@ (define-team rust
"guix/build/cargo-utils.scm"
"guix/build-system/cargo.scm"
"guix/import/crate.scm"
+ "guix/import/crate/cargo-lock.scm"
"guix/scripts/import/crate.scm"
"tests/crate.scm")))
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index a6f247bbae..ef3a6d0b4b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -9,6 +9,9 @@
;;; Copyright © 2023, 2024 David Elsing <david.elsing <at> posteo.net>
;;; Copyright © 2024 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;; Copyright © 2025 Herman Rimm <herman <at> rimm.ee>
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024-2025 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -26,12 +29,14 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix import crate)
+ #:use-module (guix base16)
#:use-module (guix base32)
#:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
#:use-module (guix i18n)
+ #:use-module (guix import crate cargo-lock)
#:use-module (guix import json)
#:use-module (guix import utils)
#:use-module (guix memoization)
@@ -39,9 +44,11 @@ (define-module (guix import crate)
#:use-module (guix read-print)
#:use-module (guix upstream)
#:use-module (guix utils)
+ #:use-module (guix scripts download)
#:use-module (gnu packages)
#:use-module (ice-9 match)
#:use-module (ice-9 regex)
+ #:use-module (ice-9 textual-ports)
#:use-module (json)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
@@ -52,6 +59,7 @@ (define-module (guix import crate)
guix-package->crate-name
string->license
crate-recursive-import
+ cargo-lock->definitions
%crate-updater))
@@ -481,6 +489,76 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
+
+;;;
+;;; Convert ‘Cargo.lock’ to Guix sources.
+;;;
+
+(define (cargo-lock->definitions lockfile package-name)
+ "Given LOCKFILE, a 'Cargo.lock' file, convert its content into source
+definitions. Return a list of sources and 'PACKAGE-NAME-cargo-inputs', an input
+list referencing all imported sources."
+ (define (crate->definition crate)
+ (match crate
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source _)
+ ('crate-checksum checksum))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,@(if (or (string-suffix? "src" name)
+ (string-suffix? "sys" name))
+ (list (comment ";; TODO: Check bundled sources.\n" #f))
+ '())
+ (crate-source ,name ,version
+ ,(bytevector->nix-base32-string
+ (base16-string->bytevector checksum)))))
+ ;; Git snapshot.
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source source))
+ (begin
+ (let* ((src (string-split source (char-set #\+ #\? #\#)))
+ (url (second src))
+ (commit (last src))
+ (version (string-append version "." (string-take commit 7)))
+ (checksum
+ (second
+ (string-split
+ (with-output-to-string
+ (lambda _
+ (guix-download "-g" url
+ (string-append "--commit=" commit))))
+ #\newline))))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,(comment
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url ,url)
+ (commit ,commit)))
+ (file-name
+ (git-file-name ,(crate-name->package-name name) ,version))
+ (sha256 (base32 ,checksum)))))))
+ ;; Workspace member.
+ (else #f)))
+
+ (let ((definitions
+ (filter-map crate->definition
+ (cargo-lock-string->scm
+ (call-with-input-file lockfile get-string-all)))))
+ (values definitions
+ `(define-public
+ ,(string->symbol (string-append package-name "-cargo-inputs"))
+ (list ,@(map second definitions))))))
+
;;;
;;; Updater
diff --git a/guix/import/crate/cargo-lock.scm b/guix/import/crate/cargo-lock.scm
new file mode 100644
index 0000000000..b8a6ab20c0
--- /dev/null
+++ b/guix/import/crate/cargo-lock.scm
@@ -0,0 +1,105 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import crate cargo-lock)
+ #:use-module (ice-9 peg)
+ #:export (cargo-lock-string->scm
+
+ crate-name
+ crate-version
+ crate-source
+ crate-checksum
+ crate-dependencies
+ cargo-lock))
+
+;;;
+;;; PEG parser for ‘Cargo.lock’.
+;;;
+
+(define (cargo-lock-string->scm str)
+ (peg:tree (search-for-pattern cargo-lock str)))
+
+;; Auxiliar peg patterns
+(define-peg-pattern numeric-char body
+ (range #\0 #\9))
+
+(define-peg-pattern lowercase-char body
+ (range #\a #\z))
+
+(define-peg-pattern uppercase-char body
+ (range #\A #\Z))
+
+(define-peg-pattern alphabetic-char body
+ (or lowercase-char uppercase-char))
+
+(define-peg-pattern alphanumeric-char body
+ (or alphabetic-char numeric-char))
+
+;; name
+(define-peg-pattern crate-name all
+ (+ (or "-" alphabetic-char
+ "_" numeric-char)))
+
+;; version
+(define-peg-pattern non-negative-integer body
+ (+ numeric-char))
+
+(define-peg-pattern crate-version all
+ (and non-negative-integer "."
+ non-negative-integer "."
+ non-negative-integer
+ (? (+ (or "-" lowercase-char
+ "." uppercase-char
+ "+" numeric-char "_")))))
+
+;; source
+(define-peg-pattern crate-source all
+ (and (or "registry" "git")
+ "+https://"
+ (+ (or "/" "." "?" "=" "-" "#" "_"
+ alphanumeric-char))))
+
+;; checksum
+(define-peg-pattern crate-checksum all
+ (+ (or lowercase-char numeric-char)))
+
+;; dependency specification
+(define-peg-pattern dependency-specification all
+ (and crate-name (? (and (ignore " ") crate-version))))
+
+;; dependencies
+(define-peg-pattern crate-dependencies all
+ (and (ignore "[\n")
+ (+ (and (ignore " \"")
+ (capture dependency-specification)
+ (ignore "\",\n")))
+ (ignore "]")))
+
+;; crates
+(define-peg-pattern crate all
+ (and (ignore "[[package]]\n")
+ (ignore "name = \"") (capture crate-name) (ignore "\"\n")
+ (ignore "version = \"") (capture crate-version) (ignore "\"\n")
+ (? (and (ignore "source = \"") (capture crate-source) (ignore "\"\n")))
+ (? (and (ignore "checksum = \"") (capture crate-checksum) (ignore "\"\n")))
+ (? (ignore (and "dependencies = " crate-dependencies "\n")))))
+
+;; Cargo.lock
+(define-peg-pattern cargo-lock all
+ (+ (and (ignore "\n") crate)))
--
2.49.0
guix <at> cbaines.net, divya <at> subvertising.org, efraim <at> flashner.co.il, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, maxim.cournoyer <at> gmail.com, zimon.toutoune <at> gmail.com, steve <at> futurile.net, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:28 GMT) Full text and rfc822 format available.Message #236 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 16/17] import: crate: Add ‘--lockfile’ option. Date: Sun, 23 Mar 2025 15:28:38 +0800
* guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
* guix/scripts/import/crate.scm (show-help): Add it.
(guix-import-crate): Use it.
* doc/guix.texi (Invoking guix import): Document it.
* tests/crate.scm (temp-file): New variable.
("crate-lockfile-import"): New test.
Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
---
doc/guix.texi | 14 ++++++
guix/scripts/import/crate.scm | 58 +++++++++++++++++++----
tests/crate.scm | 88 +++++++++++++++++++++++++++++++++++
3 files changed, 150 insertions(+), 10 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 7fa986b4b9..218c2ba630 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14703,6 +14703,20 @@ Invoking guix import
If a crate dependency is not (yet) packaged, make the corresponding
input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
a comment.
+@item --lockfile=@var{file}
+@itemx -f @var{file}
+When @option{--lockfile} is specified, the importer will ignore other options
+and won't output package definitions, instead generating source definitions for
+all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
+
+@example
+guix import crate --lockfile=/path/to/Cargo.lock my-package
+@end example
+
+generates sources from @file{/path/to/Cargo.lock} and a list
+@code{my-package-cargo-inputs} referencing these sources. The generated list is
+intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
+@code{#:cargo-development-inputs}.
@end table
@item elm
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 723cbb3665..3b536e135a 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -25,11 +25,13 @@
(define-module (guix scripts import crate)
#:use-module (guix ui)
#:use-module (guix utils)
+ #:use-module (guix read-print)
#:use-module (guix scripts)
#:use-module (guix import crate)
#:use-module (guix scripts import)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-26)
#:use-module (srfi srfi-37)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
@@ -60,6 +62,9 @@ (define (show-help)
sufficient package exists for it"))
(newline)
(display (G_ "
+ -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
+ (newline)
+ (display (G_ "
-h, --help display this help and exit"))
(display (G_ "
-V, --version display version information and exit"))
@@ -87,6 +92,9 @@ (define %options
(option '("mark-missing") #f #f
(lambda (opt name arg result)
(alist-cons 'mark-missing #t result)))
+ (option '(#\f "lockfile") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'lockfile arg result)))
%standard-import-options))
@@ -101,6 +109,8 @@ (define (guix-import-crate . args)
#:build-options? #f))
(let* ((opts (parse-options))
+ (lockfile (assoc-ref opts 'lockfile))
+ (file-to-insert (assoc-ref opts 'file-to-insert))
(args (filter-map (match-lambda
(('argument . value)
value)
@@ -111,16 +121,44 @@ (define (guix-import-crate . args)
(define-values (name version)
(package-name->name+version spec))
- (match (if (assoc-ref opts 'recursive)
- (crate-recursive-import
- name #:version version
- #:recursive-dev-dependencies?
- (assoc-ref opts 'recursive-dev-dependencies)
- #:allow-yanked? (assoc-ref opts 'allow-yanked))
- (crate->guix-package
- name #:version version #:include-dev-deps? #t
- #:allow-yanked? (assoc-ref opts 'allow-yanked)
- #:mark-missing? (assoc-ref opts 'mark-missing)))
+ (match (cond
+ ((and=> lockfile
+ (lambda (file)
+ (or (file-exists? file)
+ (leave (G_ "file '~a' does not exist~%") file))))
+ (let-values (((crate-definitions input-list)
+ (cargo-lock->definitions lockfile name)))
+ (if file-to-insert
+ ;; Adjusted from ‘--insert’ option of guix-import.
+ (let ((term (second input-list)))
+ (begin
+ ;; Remove existing input list definition.
+ (and=> (find-definition-location file-to-insert term)
+ delete-expression)
+ ;; Insert input list alphabetically.
+ (or (and=> (find-definition-insertion-location
+ file-to-insert term)
+ (cut insert-expression <> input-list))
+ (let ((port (open-file file-to-insert "a")))
+ (newline port)
+ (pretty-print-with-comments port input-list)
+ (newline port)
+ (newline port)
+ (close-port port))))
+ crate-definitions)
+ `(,@crate-definitions
+ ,input-list))))
+ ((assoc-ref opts 'recursive)
+ (crate-recursive-import
+ name #:version version
+ #:recursive-dev-dependencies?
+ (assoc-ref opts 'recursive-dev-dependencies)
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)))
+ (else
+ (crate->guix-package
+ name #:version version #:include-dev-deps? #t
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)
+ #:mark-missing? (assoc-ref opts 'mark-missing))))
((or #f '())
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
diff --git a/tests/crate.scm b/tests/crate.scm
index d0dc779cd2..9b7066c3b1 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -34,6 +34,7 @@ (define-module (test-crate)
#:use-module (gnu packages)
#:use-module (ice-9 iconv)
#:use-module (ice-9 match)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-64))
@@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
(description #f)
(license #f)))
+(define temp-file
+ (string-append "t-crate-" (number->string (getpid))))
+
(test-begin "crate")
@@ -1178,4 +1182,88 @@ (define rust-leaf-bob-3.0.2-yanked
(x
(pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
+
+(test-assert "crate-lockfile-import"
+ (begin
+ (call-with-output-file temp-file
+ (lambda (port)
+ (display "\
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = \"adler2\"
+version = \"2.0.0\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
+
+[[package]]
+name = \"aho-corasick\"
+version = \"1.1.3\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
+dependencies = [
+ \"memchr\",
+]
+
+[[package]]
+name = \"smithay\"
+version = \"0.4.0\"
+source = \"git+https://github.com/Smithay/smithay.git?rev=\
+0cd3345c59f7cb139521f267956a1a4e33248393#\
+0cd3345c59f7cb139521f267956a1a4e33248393\"
+dependencies = [
+ \"appendlist\",
+]
+
+[[package]]
+name = \"test\"
+version = \"25.2.0\"\n" port)))
+ (mock
+ ((guix scripts download) guix-download
+ (lambda _
+ (format #t "~a~%~a~%"
+ "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
+ (let-values
+ (((crates-definitions input-list)
+ (cargo-lock->definitions temp-file "test")))
+ (and
+ (match crates-definitions
+ (((define 'rust-adler2-2.0.0
+ (crate-source
+ "adler2" "2.0.0"
+ "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
+ (define 'rust-aho-corasick-1.1.3
+ (crate-source
+ "aho-corasick" "1.1.3"
+ "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
+ (define 'rust-smithay-0.4.0.0cd3345
+ ($ <comment>
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/Smithay/smithay.git")
+ (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
+ (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
+ (sha256
+ (base32
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
+ #t)
+ (x
+ (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
+ (match input-list
+ ((define-public 'test-cargo-inputs
+ (list rust-adler2-2.0.0
+ rust-aho-corasick-1.1.3
+ rust-smithay-0.4.0.0cd3345))
+ #t)
+ (x
+ (pk 'fail x #f))))))))
+
(test-end "crate")
+
+(false-if-exception (delete-file temp-file))
--
2.49.0
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:33:02 GMT) Full text and rfc822 format available.Message #239 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Sun, 23 Mar 2025 15:28:39 +0800
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
* doc/guix-cookbook.texi (Packaging)[Packaging Workflow]: New section.
Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
doc/contributing.texi | 91 ++++++++---
doc/guix-cookbook.texi | 352 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 418 insertions(+), 25 deletions(-)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..17f22390d5 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,75 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
+Rust programs (binary crates) and dependencies (library crates) are treated
+separately. We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
+vendored dependencies. The following paragraphs will explain them and give
+several examples.
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix. The name should be changed to lowercase as appropriate and
-dashes should remain in place.
+Rust programs are treated like any other package and named using the lowercase
+upstream name. When using the Cargo build system (@pxref{Build Systems,
+@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
+argument set to @code{#f}, as this argument only makes sense for dependencies.
+When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
+be set to enable relevant support.
+
+Rust dependencies are managed in two modules:
+
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, @code{crate}, @code{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored sources unbundled before
+being contributed to Guix. This module is managed by the Rust team
+(@pxref{Teams}).
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages. This includes Rust dependencies requiring external inputs
+to unbundle and Cargo workspaces.
+
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix. The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix. The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course). This follows the ``caret'' version scheme
-intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments. It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}. If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible. Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+package to be used at any given time, so all dependencies should have a
+versioned suffix. The versioned suffix is the left-most non-zero digit (and any
+leading zeros, of course). This follows the ``caret'' version scheme intended
+by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we are usually packaging development snapshots of Rust dependencies
+specifically for some Rust programs, and can't simply identity them by version.
+In this case we can use a @code{for-@var{program}} suffix, for example,
+@code{rust-pipewire-for-niri} and @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Rust dependencies are not referenced directly. @code{(guix build-sytem cargo)}
+provides a @code{cargo-inputs} procedure to create an input list, in combination
+with the lockfile importer.
+
+@cindex cargo-inputs
+@deffn {Procedure} cargo-inputs @var{name} @
+ [#:crates-module '(gnu packages rust-crates)] @
+ [#:sources-module '(gnu packages rust-sources)]
+Given symbol @code{'@var{name}}, resolve variable
+@code{@var{name}-cargo-inputs}, an input list, in @var{crates-module}, return
+its copy with @code{#f} removed and symbols resolved to variables defined in
+@var{sources-module} if the input list exists, otherwise return an empty list.
+
+@var{name} must be consistent with the one used in lockfile importer invocation.
+@end deffn
+
+For a more detailed packaging workflow, @pxref{Packaging Rust Crates,,,
+guix-cookbook, GNU Guix Cookbook}.
@node Elm Packages
diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index d9b98a2ab3..1f0e5eee54 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -103,6 +103,7 @@ Top
Packaging
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
Packaging Tutorial
@@ -127,6 +128,16 @@ Top
* Automatic update::
* Inheritance::
+Packaging Workflows
+
+* Packaging Rust Crates::
+
+Packaging Rust Crates
+
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+
System Configuration
* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
@@ -514,6 +525,7 @@ Packaging
@menu
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
@end menu
@node Packaging Tutorial
@@ -1593,6 +1605,346 @@ References
@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
@end itemize
+@node Packaging Workflows
+@section Packaging Workflows
+
+The following sections are real life examples on working with specific build
+systems, serving as extensions to the concise packaging guidelines
+(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
+
+@menu
+* Packaging Rust Crates::
+@end menu
+
+@node Packaging Rust Crates
+@subsection Packaging Rust Crates
+
+In preparation, add the following packages to our environment:
+
+@example
+$ guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+@menu
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+@end menu
+
+@node Common Workflow for Packaging Rust Crates
+@subsubsection Common Workflow for Packaging Rust Crates
+
+In this example, we'll package @code{cargo-audit}, which is published on the
+@uref{https://crates.io, crates.io} Rust package repository. All its
+dependencies are on crates.io as well.
+
+@enumerate
+@item
+Since @code{cargo-audit} is available on crates.io, We can generate a draft
+definition via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU
+Guix Reference Manual}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+The symbol used in @code{cargo-inputs}, @code{'cargo-audit} here, must be a
+unique identifier, usually matching the variable name of the package.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+$ cargo generate-lockfile
+$ cargo audit
+$ cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining
+Packages,,, guix, GNU Guix Reference Manual}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+$ guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, package @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them via the Guix command-line interface through expression:
+
+@example
+$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+reference it by a symbol in the imported definition.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to a matching symbol.
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+@end enumerate
+
+@node Development Snapshots and Cargo Workspaces
+@subsubsection Development Snapshots and Cargo Workspaces
+
+In this example, we'll package @code{niri}, which depends on development
+snapshots (also Cargo workspaces here).
+
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+to a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in a build phase, with a package-specific substitution
+pattern.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3+)))
+@end lisp
+
+@code{niri} has Cargo workspace dependencies. When packaging a Cargo workspace,
+argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+
+@node Rust Programs With Mixed Build Systems
+@subsubsection Rust Programs With Mixed Build Systems
+
+In this example, we'll package @code{fish}, which combines two build systems.
+
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+
+
@c *********************************************************************
@node System Configuration
@chapter System Configuration
--
2.49.0
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 25 Mar 2025 09:24:02 GMT) Full text and rfc822 format available.Message #242 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Hilton Chain <hako <at> ultrarare.space> Cc: 77093 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Subject: Re: [bug#77093] [PATCH rust-team 00/18] New Rust packaging workflow based on lockfile importer. Date: Tue, 25 Mar 2025 10:22:51 +0100
Hi Hilton, Hilton Chain <hako <at> ultrarare.space> skribis: > This series: > * integrates cargo2guix (https://git.sr.ht/~look/cargo2guix) as a crate importer > option, ‘-f / --lockfile=’. > > * adds Cargo workspace support to cargo-build-system, via two new arguments: > #:cargo-package-crates and #:cargo-install-paths. > > * introduces a new Rust packaging workflow, as described in contributing.texi > change. I hadn’t noticed this patch series but I’m really happy you’re working on it, I think it can help make Rust packaging more sustainable. 👍 Ludo’.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sat, 05 Apr 2025 02:58:01 GMT) Full text and rfc822 format available.Message #245 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Steve George <steve <at> futurile.net>, Efraim Flashner <efraim <at> flashner.co.il>, Divya Ranjan Pattanaik <divya <at> subvertising.org> Subject: Re: [PATCH rust-team v3 00/17] New Rust packaging workflow based on lockfile importer. Date: Sat, 05 Apr 2025 10:53:33 +0800
On Sun, 23 Mar 2025 15:23:04 +0800,
Hilton Chain wrote:
>
> V2 -> V3:
> * Add Efraim's etc/teams/rust/audit-rust-crates script.
> * Add (guix import crate cargo-lock) to Rust team's scope.
> * ‘definer’ -> ‘define-prefix’
> * Adjust documentation and move examples to Guix Cookbook.
> * Document ‘cargo-inputs’ and remove mentioning of files under etc/teams/rust.
>
> Diff inserted at end.
>
> Efraim Flashner (1):
> build/cargo: Don't try to unpack sanity-check.py.
>
> Hilton Chain (16):
> build/cargo: Pass ‘--offline’ to cargo.
> build/cargo: Print out all non-empty binary files.
> build-system: cargo: Support packaging Cargo workspace.
> build-system: cargo: Support installing Cargo workspace.
> build/cargo: Set default value of arguments for build phases.
> build/cargo: Support non-workspace directory source inputs.
> scripts: import: Document argument for ‘--insert’ option in help
> message.
> scripts: import: Add two newlines for ‘--insert’ option.
> scripts: import: Support expressions defined by 'define.
> scripts: import: Pass "--insert" to importers.
> scripts: import: Skip existing definition for ‘--insert’ option.
> import: crate: crate-name->package-name: Move to (guix build-system
> cargo).
> build-system: cargo: Add ‘cargo-inputs’.
> import: crate: Add Cargo.lock parser.
> import: crate: Add ‘--lockfile’ option.
> doc: Document lockfile importer based Rust packaging workflow.
>
> Makefile.am | 1 +
> doc/contributing.texi | 91 +++++---
> doc/guix-cookbook.texi | 352 ++++++++++++++++++++++++++++++
> doc/guix.texi | 24 ++
> etc/teams.scm | 1 +
> etc/teams/rust/audit-rust-crates | 70 ++++++
> etc/teams/rust/cleanup-crates.sh | 37 ++++
> etc/teams/rust/rust-crates.tmpl | 42 ++++
> gnu/local.mk | 2 +
> gnu/packages/rust-crates.scm | 42 ++++
> gnu/packages/rust-sources.scm | 29 +++
> guix/build-system/cargo.scm | 59 ++++-
> guix/build/cargo-build-system.scm | 150 +++++++++----
> guix/import/crate.scm | 81 ++++++-
> guix/import/crate/cargo-lock.scm | 105 +++++++++
> guix/scripts/import.scm | 54 +++--
> guix/scripts/import/crate.scm | 58 ++++-
> guix/utils.scm | 29 ++-
> tests/crate.scm | 88 ++++++++
> 19 files changed, 1209 insertions(+), 106 deletions(-)
> create mode 100755 etc/teams/rust/audit-rust-crates
> create mode 100755 etc/teams/rust/cleanup-crates.sh
> create mode 100644 etc/teams/rust/rust-crates.tmpl
> create mode 100644 gnu/packages/rust-crates.scm
> create mode 100644 gnu/packages/rust-sources.scm
> create mode 100644 guix/import/crate/cargo-lock.scm
I'll apply this series (except the documentation change, which will be tracked
here separately) to rust-team when I have checked an initial set of packages.
In case it's useful, I'm using this script to get a list of unstaged crates to
look through:
--8<---------------cut here---------------start------------->8---
for i in $(git diff gnu/packages/rust-crates.scm | grep -E '^\+\(define rust-' | cut -f2 -d' ')
do
guix build -e "(@@ (gnu packages rust-crates) $i)" -v0
done
--8<---------------cut here---------------end--------------->8---
Thanks
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sat, 12 Apr 2025 11:21:02 GMT) Full text and rfc822 format available.Message #248 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Steve George <steve <at> futurile.net>, Efraim Flashner <efraim <at> flashner.co.il>, Divya Ranjan Pattanaik <divya <at> subvertising.org> Subject: Re: [PATCH rust-team v3 00/17] New Rust packaging workflow based on lockfile importer. Date: Sat, 12 Apr 2025 19:20:36 +0800
On Sat, 05 Apr 2025 10:53:33 +0800,
Hilton Chain wrote:
>
> On Sun, 23 Mar 2025 15:23:04 +0800,
> Hilton Chain wrote:
> >
> I'll apply this series (except the documentation change, which will be tracked
> here separately) to rust-team when I have checked an initial set of packages.
>
> In case it's useful, I'm using this script to get a list of unstaged crates to
> look through:
> --8<---------------cut here---------------start------------->8---
> for i in $(git diff gnu/packages/rust-crates.scm | grep -E '^\+\(define rust-' | cut -f2 -d' ')
> do
> guix build -e "(@@ (gnu packages rust-crates) $i)" -v0
> done
> --8<---------------cut here---------------end--------------->8---
Adjusted this script to open editor automatically until all files have been
opened:
--8<---------------cut here---------------start------------->8---
for i in $(git diff gnu/packages/rust-crates.scm | grep -E '^\+\(define rust-' | cut -f2 -d' ')
do
file="$(guix build -e "(@@ (gnu packages rust-crates) $i)" -v0 2>/dev/null)"
if [[ -n $file ]]; then
$EDITOR $file
fi
done
--8<---------------cut here---------------end--------------->8---
I use this script for an extra check, after addressing files reported by
check-for-pregenerated-files phase first.
The phase can't check source, a large number of pregenerated bindings exist and
are included as before. Some crates are specificially made for this purpose as
well, I think we should have a clear policy for them.
I'll set a deadline (1 year?) to remove #:cargo-inputs support from
cargo-build-sytem. Here're remaining users in our codebase:
--8<---------------cut here---------------start------------->8---
$ git grep -wc '#:cargo-.*inputs' gnu/packages
gnu/packages/bioinformatics.scm:3 wait python-team merge
gnu/packages/crates-apple.scm:84 dependencies
gnu/packages/crates-audio.scm:68 ...
gnu/packages/crates-check.scm:42 ...
gnu/packages/crates-compression.scm:115 ...
gnu/packages/crates-crypto.scm:355 ...
gnu/packages/crates-database.scm:80 ...
gnu/packages/crates-graphics.scm:346 ...
gnu/packages/crates-gtk.scm:348 ...
gnu/packages/crates-io.scm:5234 ...
gnu/packages/crates-shell.scm:51 ...
gnu/packages/crates-tls.scm:130 ...
gnu/packages/crates-vcs.scm:429 ...
gnu/packages/crates-web.scm:404 ...
gnu/packages/crates-windows.scm:146 ...
gnu/packages/crypto.scm:3 ...
gnu/packages/machine-learning.scm:11 wait python-team merge
gnu/packages/python-compression.scm:1 ...
gnu/packages/python-crypto.scm:1 ...
gnu/packages/python-science.scm:1 ...
gnu/packages/python-web.scm:1 ...
gnu/packages/python-xyz.scm:8 ...
gnu/packages/rust-apps.scm:96 TODO
gnu/packages/sequoia.scm:46 TODO
--8<---------------cut here---------------end--------------->8---
I have rebased rust-team to a recent master, this issue (#77093) will be an
extra blocker for merging the branch.
Don't keep too many changes to rust-crates.scm locally :)
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sat, 12 Apr 2025 12:59:05 GMT) Full text and rfc822 format available.Message #251 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Hilton Chain <hako <at> ultrarare.space> Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Steve George <steve <at> futurile.net>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Efraim Flashner <efraim <at> flashner.co.il> Subject: Re: [PATCH rust-team v3 00/17] New Rust packaging workflow based on lockfile importer. Date: Sat, 12 Apr 2025 14:58:08 +0200
Hello, Hilton Chain <hako <at> ultrarare.space> skribis: > I'll set a deadline (1 year?) to remove #:cargo-inputs support from > cargo-build-sytem. Here're remaining users in our codebase: I think you could add an entry in ‘etc/news.scm’ and mention this plan. (Perhaps a blog post explaining the rationale and strategy would be in order, too! :-)) Kudos for all this work! Ludo’.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sat, 12 Apr 2025 13:55:03 GMT) Full text and rfc822 format available.Message #254 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Ludovic Courtès <ludo <at> gnu.org> Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Steve George <steve <at> futurile.net>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Efraim Flashner <efraim <at> flashner.co.il> Subject: Re: [PATCH rust-team v3 00/17] New Rust packaging workflow based on lockfile importer. Date: Sat, 12 Apr 2025 21:54:46 +0800
On Sat, 12 Apr 2025 20:58:08 +0800, Ludovic Courtès wrote: > > Hello, > > Hilton Chain <hako <at> ultrarare.space> skribis: > > > I'll set a deadline (1 year?) to remove #:cargo-inputs support from > > cargo-build-sytem. Here're remaining users in our codebase: > > I think you could add an entry in ‘etc/news.scm’ and mention this plan. > (Perhaps a blog post explaining the rationale and strategy would be in > order, too! :-)) > > Kudos for all this work! Thanks :) I'll add a news entry in this issue and prepare a blog post.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 22 Apr 2025 21:29:01 GMT) Full text and rfc822 format available.Message #257 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Hilton Chain <hako <at> ultrarare.space> Cc: Steve George <steve <at> futurile.net>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Efraim Flashner <efraim <at> flashner.co.il> Subject: Re: [bug#77093] [PATCH rust-team v3 14/17] build-system: cargo: Add ‘cargo-inputs’. Date: Tue, 22 Apr 2025 23:20:00 +0200
Hello,
Hilton Chain <hako <at> ultrarare.space> writes:
> * gnu/packages/rust-crates.scm: New file.
> * gnu/packages/rust-sources.scm: New file.
> * gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them.
> * guix/build-system/cargo.scm (crate-source,cargo-inputs): New procedures.
> * guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo).
> * etc/teams/rust/audit-rust-crates: New file.
> * etc/teams/rust/cleanup-crates.sh: New file.
> * etc/teams/rust/rust-crates.tmpl: New file.
>
> Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3
I’m late to the party and coming mostly to discuss the color of the
bikeshed—the bikeshed looks really great regardless of the color anyway.
:-)
> etc/teams/rust/audit-rust-crates | 70 ++++++++++++++++++++++++++++++++
> etc/teams/rust/cleanup-crates.sh | 37 +++++++++++++++++
Maybe not a blocker, but we should plan to move from awk + sh to Scheme.
> +;;; Rust dependencies fetched from crates.io and non-workspace development
> +;;; snapshots.
> +;;;
> +
> +(define qqqq-separator 'begin-of-crates)
> +
> +(define ssss-separator 'end-of-crates)
I couldn’t find where these markers are actually used. Would be nice if
it could be avoided, somehow.
> +(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
> + (sources-module '(gnu packages rust-sources)))
> +
> + "Given symbol NAME, resolve variable 'NAME-cargo-inputs', an input list, in
> +CRATES-MODULE, return its copy with #f removed and symbols resolved to
> +variables defined in SOURCES-MODULE if the input list exists, otherwise return
> +an empty list."
> + (let loop ((inputs
> + (catch #t
> + (lambda ()
> + (module-ref (resolve-interface crates-module)
> + (symbol-append name '-cargo-inputs)))
My preference would be to use a first-class hash table instead of
playing tricks with the module system at run time: that would be more
discoverable, more predictable (no computed variable names), and
future-proof (should the module system introspection capabilities vanish
or become less powerful/risky).
So I would arrange to have something like:
(define-cargo-inputs lookup-input
(NAME => (list …))
…)
and that would expand to something like:
(define lookup-input
(let ((table (make-hash-table)))
(hashq-set! 'NAME (list …))
(lambda (name)
"Return the inputs for NAME."
(hashq-ref table name))))
I was in the mood for macrology, so here’s a macro that does it:
(define-syntax define-cargo-inputs
(syntax-rules (=>)
((_ lookup inputs ...)
(define lookup
(let ((table (make-hash-table)))
(letrec-syntax ((record
(syntax-rules (=>)
((_) #t)
((_ (name => lst) rest (... ...))
(begin
(hashq-set! table 'name lst)
(record rest (... ...)))))))
(record inputs ...)
(lambda (name)
"Return the inputs for NAME."
(hashq-ref table name))))))))
Ludo’.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 22 Apr 2025 21:29:02 GMT) Full text and rfc822 format available.Message #260 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Efraim Flashner <efraim <at> flashner.co.il>, Steve George <steve <at> futurile.net>, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team v3 16/17] import: crate: Add ‘--lockfile’ option. Date: Tue, 22 Apr 2025 23:28:17 +0200
Hilton Chain <hako <at> ultrarare.space> writes:
> * guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
> * guix/scripts/import/crate.scm (show-help): Add it.
> (guix-import-crate): Use it.
> * doc/guix.texi (Invoking guix import): Document it.
> * tests/crate.scm (temp-file): New variable.
> ("crate-lockfile-import"): New test.
>
> Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
[...]
> + (match crates-definitions
> + (((define 'rust-adler2-2.0.0
> + (crate-source
> + "adler2" "2.0.0"
> + "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
Note that this doesn’t actually testwhat you’d like. For instance, it
would match:
((whatever rust-adler2-2.0.0
(nonsense "adler2" "2.0.0"
"09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
…)
But the bug is already here in ‘master’, uh.
So you need to quote every symbol that is meant to be matched literally,
or wrap the pattern in quasiquote, as is done in ‘tests/gem.scm’:
(match …
((`(define rust-adler2-2.0.0 (crate-source …))) #t))
Ludo’.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 23 Apr 2025 04:14:01 GMT) Full text and rfc822 format available.Message #263 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Ludovic Courtès <ludo <at> gnu.org> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Efraim Flashner <efraim <at> flashner.co.il>, Steve George <steve <at> futurile.net>, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team v3 16/17] import: crate: Add ‘--lockfile’ option. Date: Wed, 23 Apr 2025 12:12:56 +0800
On Wed, 23 Apr 2025 05:28:17 +0800,
Ludovic Courtès wrote:
>
> Hilton Chain <hako <at> ultrarare.space> writes:
>
> > * guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
> > * guix/scripts/import/crate.scm (show-help): Add it.
> > (guix-import-crate): Use it.
> > * doc/guix.texi (Invoking guix import): Document it.
> > * tests/crate.scm (temp-file): New variable.
> > ("crate-lockfile-import"): New test.
> >
> > Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
>
> [...]
>
> > + (match crates-definitions
> > + (((define 'rust-adler2-2.0.0
> > + (crate-source
> > + "adler2" "2.0.0"
> > + "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
>
> Note that this doesn’t actually testwhat you’d like. For instance, it
> would match:
>
> ((whatever rust-adler2-2.0.0
> (nonsense "adler2" "2.0.0"
> "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
> …)
>
> But the bug is already here in ‘master’, uh.
>
> So you need to quote every symbol that is meant to be matched literally,
> or wrap the pattern in quasiquote, as is done in ‘tests/gem.scm’:
>
> (match …
> ((`(define rust-adler2-2.0.0 (crate-source …))) #t))
Thanks! Changes here are not applied to master yet, they are currently
available in rust-team only.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 23 Apr 2025 04:29:02 GMT) Full text and rfc822 format available.Message #266 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Ludovic Courtès <ludo <at> gnu.org> Cc: Steve George <steve <at> futurile.net>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Efraim Flashner <efraim <at> flashner.co.il> Subject: Re: [bug#77093] [PATCH rust-team v3 14/17] build-system: cargo: Add ‘cargo-inputs’. Date: Wed, 23 Apr 2025 12:27:49 +0800
On Wed, 23 Apr 2025 05:20:00 +0800, Ludovic Courtès wrote: > > Hello, > > Hilton Chain <hako <at> ultrarare.space> writes: > > > * gnu/packages/rust-crates.scm: New file. > > * gnu/packages/rust-sources.scm: New file. > > * gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them. > > * guix/build-system/cargo.scm (crate-source,cargo-inputs): New procedures. > > * guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo). > > * etc/teams/rust/audit-rust-crates: New file. > > * etc/teams/rust/cleanup-crates.sh: New file. > > * etc/teams/rust/rust-crates.tmpl: New file. > > > > Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3 > > I’m late to the party and coming mostly to discuss the color of the > bikeshed—the bikeshed looks really great regardless of the color anyway. > :-) > > > etc/teams/rust/audit-rust-crates | 70 ++++++++++++++++++++++++++++++++ > > etc/teams/rust/cleanup-crates.sh | 37 +++++++++++++++++ > > Maybe not a blocker, but we should plan to move from awk + sh to Scheme. Yes, this is planned. > > +;;; Rust dependencies fetched from crates.io and non-workspace development > > +;;; snapshots. > > +;;; > > + > > +(define qqqq-separator 'begin-of-crates) > > + > > +(define ssss-separator 'end-of-crates) > > I couldn’t find where these markers are actually used. Would be nice if > it could be avoided, somehow. The values doesn't matter, I'm using these two variable names to limit insertion location of ‘guix import’ -- inserted definitions always start with "rust-", which is between Q and S. > > +(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates)) > > + (sources-module '(gnu packages rust-sources))) > > + > > + "Given symbol NAME, resolve variable 'NAME-cargo-inputs', an input list, in > > +CRATES-MODULE, return its copy with #f removed and symbols resolved to > > +variables defined in SOURCES-MODULE if the input list exists, otherwise return > > +an empty list." > > + (let loop ((inputs > > + (catch #t > > + (lambda () > > + (module-ref (resolve-interface crates-module) > > + (symbol-append name '-cargo-inputs))) > > My preference would be to use a first-class hash table instead of > playing tricks with the module system at run time: that would be more > discoverable, more predictable (no computed variable names), and > future-proof (should the module system introspection capabilities vanish > or become less powerful/risky). > > So I would arrange to have something like: > > (define-cargo-inputs lookup-input > (NAME => (list …)) > …) > > and that would expand to something like: > > (define lookup-input > (let ((table (make-hash-table))) > (hashq-set! 'NAME (list …)) > (lambda (name) > "Return the inputs for NAME." > (hashq-ref table name)))) > > I was in the mood for macrology, so here’s a macro that does it: > > (define-syntax define-cargo-inputs > (syntax-rules (=>) > ((_ lookup inputs ...) > (define lookup > (let ((table (make-hash-table))) > (letrec-syntax ((record > (syntax-rules (=>) > ((_) #t) > ((_ (name => lst) rest (... ...)) > (begin > (hashq-set! table 'name lst) > (record rest (... ...))))))) > (record inputs ...) > (lambda (name) > "Return the inputs for NAME." > (hashq-ref table name)))))))) Thanks! I'll see how I can use it.
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:03 GMT) Full text and rfc822 format available.Message #269 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 01/22] scripts: import: Correct behavior for ‘--insert’ option. Date: Tue, 29 Apr 2025 00:22:58 +0800
The info manual documents ‘--insert=FILE’, but it wasn't actually supported.
* guix/scripts/import.scm (show-help): Add missing ‘FILE’.
(guix-import): Accept ‘--insert=FILE’.
Change-Id: I540d5feae3fe49c00e9bd6f7a8649ffe0d6e006d
---
guix/scripts/import.scm | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index bbf31baa15..a85f6c6c49 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -69,7 +69,7 @@ (define (show-help)
(display (G_ "
-h, --help display this help and exit"))
(display (G_ "
- -i, --insert insert packages into file alphabetically"))
+ -i, --insert=FILE insert packages into FILE alphabetically"))
(display (G_ "
-V, --version display version information and exit"))
(newline)
@@ -106,7 +106,18 @@ (define-command (guix-import . args)
(category packaging)
(synopsis "import a package definition from an external repository")
- (match args
+ (define (process-args args)
+ (match args
+ ;; Workaround to accpet ‘--insert=FILE’, for the consistency of
+ ;; command-line interface.
+ ((arg . rest)
+ (if (string-prefix? "--insert=" arg)
+ (append (string-split arg #\=)
+ rest)
+ args))
+ (_ args)))
+
+ (match (process-args args)
(()
(format (current-error-port)
(G_ "guix import: missing importer name~%")))
--
2.49.0
guix <at> cbaines.net, divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, maxim.cournoyer <at> gmail.com, zimon.toutoune <at> gmail.com, steve <at> futurile.net, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:07 GMT) Full text and rfc822 format available.Message #272 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 00/22] Cargo.lock importer and build system changes. Date: Tue, 29 Apr 2025 00:22:57 +0800
Main changes since v3:
* ‘define-cargo-inputs’ macro suggested by Ludovic.
* ‘cargo-inputs-from-lockfile’ suggested by Zheng Junjie, which can be used in
the guix.scm of a Rust project.
* Rephrase docstring.
Some changes have been made on rust-team since v3, this revision also sends
all of them out for review.
I'll change the terms Rust program & dependency to application & library, btw.
Since there're too many patches in this issue, I think it would be better to
open a new one for documentation.
Thanks
Efraim Flashner (1):
build/cargo: Don't try to unpack sanity-check.py.
Hilton Chain (21):
scripts: import: Correct behavior for ‘--insert’ option.
scripts: import: Add two newlines for ‘--insert’ option.
scripts: import: Support expressions defined by 'define.
scripts: import: Pass "--insert" to importers.
scripts: import: Skip existing definition for ‘--insert’ option.
import: crate: crate-name->package-name: Move to (guix build-system
cargo).
build/cargo: Pass ‘--offline’ to cargo.
build/cargo: Print out all non-empty binary files.
build-system: cargo: Support packaging Cargo workspace.
build-system: cargo: Support installing Cargo workspace.
build/cargo: Set default value of arguments for build phases.
build/cargo: Support non-workspace directory source inputs.
build/cargo: Don't vendor self.
build/cargo: Use system libsqlite3.
build/cargo: Use sytsem oniguruma.
build/cargo: Unconditionally use system gettext.
build-system: cargo: Add ‘cargo-inputs’.
build-system: cargo: Export ‘cargo-triplet’.
build/cargo: Accept rust-sysroot from native-inputs.
import: crate: Add Cargo.lock parser.
import: crate: Add ‘--lockfile’ option.
Makefile.am | 1 +
doc/guix.texi | 15 +++
etc/teams.scm | 1 +
etc/teams/rust/audit-rust-crates | 70 +++++++++++
etc/teams/rust/cleanup-crates.sh | 37 ++++++
etc/teams/rust/rust-crates.tmpl | 44 +++++++
etc/teams/rust/unpack-new-crates.sh | 45 +++++++
gnu/local.mk | 2 +
gnu/packages/antivirus.scm | 5 +-
gnu/packages/gnome.scm | 5 -
gnu/packages/python-crypto.scm | 5 -
gnu/packages/python-xyz.scm | 5 -
gnu/packages/rust-crates.scm | 44 +++++++
gnu/packages/rust-sources.scm | 29 +++++
gnu/packages/syndication.scm | 5 -
guix/build-system/cargo.scm | 66 +++++++++-
guix/build/cargo-build-system.scm | 179 ++++++++++++++++++++--------
guix/import/crate.scm | 128 +++++++++++++++++++-
guix/import/crate/cargo-lock.scm | 105 ++++++++++++++++
guix/scripts/import.scm | 67 ++++++++---
guix/scripts/import/crate.scm | 57 +++++++--
guix/utils.scm | 29 +++--
tests/crate.scm | 87 ++++++++++++++
23 files changed, 914 insertions(+), 117 deletions(-)
create mode 100755 etc/teams/rust/audit-rust-crates
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100755 etc/teams/rust/unpack-new-crates.sh
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
create mode 100644 guix/import/crate/cargo-lock.scm
base-commit: 8f57b1dd2a544b882d365c5be17ebcf51d9594f6
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:10 GMT) Full text and rfc822 format available.Message #275 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 02/22] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 29 Apr 2025 00:22:59 +0800
For consistency with ‘insert-expression’.
* guix/scripts/import.scm (guix-import): Add two newlines after inserting.
Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d
---
guix/scripts/import.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index a85f6c6c49..4ce31b17ff 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -140,6 +140,7 @@ (define-command (guix-import . args)
(let ((port (open-file file "a")))
(pretty-print-with-comments port expr)
(newline port)
+ (newline port)
(close-port port)))))))))
(import-as-definitions importer args find-and-insert)))
((importer args ...)
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:10 GMT) Full text and rfc822 format available.Message #278 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 03/22] scripts: import: Support expressions defined by 'define. Date: Tue, 29 Apr 2025 00:23:00 +0800
* guix/utils.scm (find-definition-location): New procedure.
(find-definition-insertion-location): Define with it.
* guix/scripts/import.scm (import-as-definitions, guix-import): Support
expressions defined by 'define.
Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
---
guix/scripts/import.scm | 21 +++++++++++++++------
guix/utils.scm | 29 +++++++++++++++++++++--------
2 files changed, 36 insertions(+), 14 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 4ce31b17ff..1242fb0b66 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -30,6 +30,7 @@ (define-module (guix scripts import)
#:use-module (guix read-print)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))
((expressions ...)
(for-each (lambda (expr)
@@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))))
expressions))
(x
@@ -128,13 +131,19 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let ((find-and-insert
+ (let* ((define-prefixes
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public))
+ (define-prefix? (cut member <> define-prefixes))
+ (find-and-insert
(lambda (expr)
(match expr
- (('define-public term _ ...)
+ (((? define-prefix? define-prefix) term _ ...)
(let ((source-properties
- (find-definition-insertion-location
- file term)))
+ (find-definition-insertion-location
+ file term #:define-prefix define-prefix)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index 7ae98096c2..7f555e57f9 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -154,6 +154,7 @@ (define-module (guix utils)
edit-expression
delete-expression
insert-expression
+ find-definition-location
find-definition-insertion-location
filtered-port
@@ -520,24 +521,36 @@ (define (insert-expression source-properties expr)
(string-append expr "\n\n" str))))
(edit-expression source-properties insert)))
-(define (find-definition-insertion-location file term)
- "Search in FILE for a top-level public definition whose defined term
-alphabetically succeeds TERM. Return the location if found, or #f
-otherwise."
- (let ((search-term (symbol->string term)))
+(define* (find-definition-location file term
+ #:key (define-prefix 'define-public)
+ (pred string=))
+ "Search in FILE for a top-level definition created using DEFINE-PREFIX, with
+the defined term compared to TERM through PRED. Return the location if PRED
+returns #t, or #f otherwise."
+ (let ((search-term (symbol->string term))
+ (define-prefix? (cut eq? define-prefix <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (('define-public current-term _ ...)
- (string> (symbol->string current-term)
- search-term))
+ (((? define-prefix?) current-term _ ...)
+ (pred (symbol->string current-term)
+ search-term))
((? eof-object?) #t)
(_ #f))
(and (not (eof-object? syntax))
(syntax-source syntax))))))))
+(define* (find-definition-insertion-location file term
+ #:key
+ (define-prefix 'define-public))
+ "Search in FILE for a top-level definition created using DEFINE-PREFIX, with
+the defined term alphabetically succeeds TERM. Return the location if found,
+or #f otherwise."
+ (find-definition-location
+ file term #:define-prefix define-prefix #:pred string>))
+
;;;
;;; Keyword arguments.
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:11 GMT) Full text and rfc822 format available.Message #281 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 04/22] scripts: import: Pass "--insert" to importers. Date: Tue, 29 Apr 2025 00:23:01 +0800
So that importers can adjust the file before inserting.
* guix/scripts/import.scm (%standard-import-options): Add ‘--file-to-insert’.
(guix-import): Pass it to importers when ‘--insert’ is set.
Change-Id: I8e7a18ee8e0f96d7fc5688a207a7a5390ad2fa30
---
guix/scripts/import.scm | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 1242fb0b66..c620911893 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -31,6 +31,7 @@ (define-module (guix scripts import)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-37)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -41,7 +42,12 @@ (define-module (guix scripts import)
;;; Command line options.
;;;
-(define %standard-import-options '())
+(define %standard-import-options
+ (list
+ ;; Hidden option for importer-specific file preprocessing.
+ (option '("file-to-insert") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'file-to-insert arg result)))))
;;;
@@ -151,7 +157,10 @@ (define-command (guix-import . args)
(newline port)
(newline port)
(close-port port)))))))))
- (import-as-definitions importer args find-and-insert)))
+ (import-as-definitions importer
+ (cons (string-append "--file-to-insert=" file)
+ args)
+ find-and-insert)))
((importer args ...)
(let ((print (lambda (expr)
(leave-on-EPIPE
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:11 GMT) Full text and rfc822 format available.Message #284 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 05/22] scripts: import: Skip existing definition for ‘--insert’ option. Date: Tue, 29 Apr 2025 00:23:02 +0800
* guix/scripts/import.scm (guix-import): Skip existing definition for
‘--insert’ option.
Change-Id: I2c4242669f974b263a018ab0cf56538bd7c81d06
---
guix/scripts/import.scm | 23 +++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index c620911893..a80893ce5f 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -147,16 +147,19 @@ (define-command (guix-import . args)
(lambda (expr)
(match expr
(((? define-prefix? define-prefix) term _ ...)
- (let ((source-properties
- (find-definition-insertion-location
- file term #:define-prefix define-prefix)))
- (if source-properties
- (insert-expression source-properties expr)
- (let ((port (open-file file "a")))
- (pretty-print-with-comments port expr)
- (newline port)
- (newline port)
- (close-port port)))))))))
+ ;; Skip existing definition.
+ (unless (find-definition-location
+ file term #:define-prefix define-prefix)
+ (let ((source-properties
+ (find-definition-insertion-location
+ file term #:define-prefix define-prefix)))
+ (if source-properties
+ (insert-expression source-properties expr)
+ (let ((port (open-file file "a")))
+ (pretty-print-with-comments port expr)
+ (newline port)
+ (newline port)
+ (close-port port))))))))))
(import-as-definitions importer
(cons (string-append "--file-to-insert=" file)
args)
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:12 GMT) Full text and rfc822 format available.Message #287 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 06/22] import: crate: crate-name->package-name: Move to (guix build-system cargo). Date: Tue, 29 Apr 2025 00:23:03 +0800
* guix/import/crate.scm (crate-name->package-name): Move to...
* guix/build-system/cargo.scm (crate-name->package-name): ...here and export.
Change-Id: Ie9813179d6c40d159956cc1e2ae59a74dea0a42d
---
guix/build-system/cargo.scm | 6 +++++-
guix/import/crate.scm | 5 -----
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 452f7f78d0..f2ffc67879 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -44,7 +44,8 @@ (define-module (guix build-system cargo)
%crate-base-url
crate-url
crate-url?
- crate-uri))
+ crate-uri
+ crate-name->package-name))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -58,6 +59,9 @@ (define (crate-uri name version)
to NAME and VERSION."
(string-append crate-url name "/" version "/download"))
+(define (crate-name->package-name name)
+ (downstream-package-name "rust-" name))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 5a8caeb3e1..14e6e28c5b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -481,10 +481,6 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
-(define (crate-name->package-name name)
- (downstream-package-name "rust-" name))
-
-
;;;
;;; Updater
@@ -528,4 +524,3 @@ (define %crate-updater
(description "Updater for crates.io packages")
(pred crate-package?)
(import import-release)))
-
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:12 GMT) Full text and rfc822 format available.Message #290 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 07/22] build/cargo: Pass ‘--offline’ to cargo. Date: Tue, 29 Apr 2025 00:23:04 +0800
This will make error messages more helpful.
* guix/build/cargo-build-system.scm (build, check, package, install): Pass
‘--offline’ to cargo.
Change-Id: Ic95f603b793319f99c9c1fbce43f773bfc8126c0
---
guix/build/cargo-build-system.scm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 5ef6b23abd..41f54f42b7 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -246,7 +246,7 @@ (define* (build #:key
"Build a given Cargo package."
(or skip-build?
(apply invoke
- `("cargo" "build"
+ `("cargo" "build" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -264,7 +264,7 @@ (define* (check #:key
"Run tests for a given Cargo package."
(when tests?
(apply invoke
- `("cargo" "test"
+ `("cargo" "test" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -303,7 +303,7 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" ,@cargo-package-flags))
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
@@ -356,7 +356,8 @@ (define* (install #:key
;; otherwise cargo will raise an error.
(or skip-build?
(not (has-executable-target?))
- (invoke "cargo" "install" "--no-track" "--path" "." "--root" out
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" "." "--root" out
"--features" (string-join features)))
(when install-source?
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:13 GMT) Full text and rfc822 format available.Message #293 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 08/22] build/cargo: Print out all non-empty binary files. Date: Tue, 29 Apr 2025 00:23:05 +0800
* guix/build/cargo-build-system.scm (%standard-phases): Move
‘unpack-rust-crates’ after ‘unpack’.
Move ‘check-for-pregenerated-files’ after ‘configure’.
(check-for-pregenerated-files): Only check non-empty files.
Print out binary files.
Run in parallel.
Don't fail to keep compatibility for phase order change.
Change-Id: I0a332fe843e97687324bd908fa111422a63e475d
---
guix/build/cargo-build-system.scm | 33 +++++++++++++++++++++++++------
1 file changed, 27 insertions(+), 6 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41f54f42b7..c37fd70418 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -27,11 +27,14 @@ (define-module (guix build cargo-build-system)
#:use-module ((guix build gnu-build-system) #:prefix gnu:)
#:use-module (guix build json)
#:use-module ((guix build utils) #:hide (delete))
+ #:use-module (ice-9 binary-ports)
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (%standard-phases
@@ -111,12 +114,30 @@ (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
(define (rust-package? name)
(string-prefix? "rust-" name))
-(define* (check-for-pregenerated-files #:rest _)
+(define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
"Check the source code for files which are known to generally be bundled
libraries or executables."
- (let ((pregenerated-files (find-files "." "\\.(a|dll|dylib|exe|lib)$")))
- (when (not (null-list? pregenerated-files))
- (error "Possible pre-generated files found:" pregenerated-files))))
+ (format #t "Searching for binary files...~%")
+ (let ((known-pattern (make-regexp "\\.(a|dll|dylib|exe|lib)$"))
+ (empty-file?
+ (lambda (file stat)
+ (let ((size (stat:size stat)))
+ (or (zero? size)
+ (and (eqv? 1 size)
+ (eqv? #\newline
+ (call-with-ascii-input-file file read-char))))))))
+ (n-par-for-each
+ (if parallel-build?
+ (parallel-job-count)
+ 1)
+ (lambda (file)
+ ;; Print out binary files.
+ (false-if-exception (invoke "grep" "-IL" "." file))
+ ;; Warn about known pre-generated files.
+ ;; Not failing here for compatibility with existing packages.
+ (when (regexp-exec known-pattern file)
+ (format #t "error: Possible pre-generated file found: ~a~%" file)))
+ (find-files "." (negate empty-file?)))))
(define* (configure #:key inputs
target system
@@ -380,8 +401,8 @@ (define %standard-phases
(replace 'check check)
(replace 'install install)
(add-after 'build 'package package)
- (add-after 'unpack 'check-for-pregenerated-files check-for-pregenerated-files)
- (add-after 'check-for-pregenerated-files 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'unpack 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'configure 'check-for-pregenerated-files check-for-pregenerated-files)
(add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
(define* (cargo-build #:key inputs (phases %standard-phases)
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:14 GMT) Full text and rfc822 format available.Message #296 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 11/22] build/cargo: Set default value of arguments for build phases. Date: Tue, 29 Apr 2025 00:23:08 +0800
This will make it easier to use these phases in other bulid systems.
* guix/build/cargo-build-system.scm (unpack-rust-crates, configure, build)
(package, install): Set default value for arguments specific to this build
system.
Change-Id: I1dde1b063d8eee57967903abd2fce94574211a0a
---
guix/build/cargo-build-system.scm | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index be96ab309f..d8433e1acd 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -77,7 +77,8 @@ (define (crate-src? path)
" | cut -d/ -f2"
" | grep -q '^Cargo.toml$'")))))
-(define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
+(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
+ #:allow-other-keys)
(define (inputs->rust-inputs inputs)
"Filter using the label part from INPUTS."
(filter (lambda (input)
@@ -141,7 +142,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(define* (configure #:key inputs
target system
- cargo-target
+ (cargo-target #f)
(vendor-dir "guix-vendor")
#:allow-other-keys)
"Vendor Cargo.toml dependencies as guix inputs."
@@ -260,7 +261,7 @@ (define* (patch-cargo-checksums #:key
(define* (build #:key
parallel-build?
- skip-build?
+ (skip-build? #f)
(features '())
(cargo-build-flags '("--release"))
#:allow-other-keys)
@@ -300,8 +301,8 @@ (define* (check #:key
(define* (package #:key
source
- skip-build?
- install-source?
+ (skip-build? #f)
+ (install-source? #t)
(cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
(vendor-dir "guix-vendor")
@@ -376,9 +377,9 @@ (define* (package #:key
(define* (install #:key
inputs
outputs
- skip-build?
- install-source?
- features
+ (skip-build? #f)
+ (install-source? #t)
+ (features '())
(cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:14 GMT) Full text and rfc822 format available.Message #299 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 09/22] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 29 Apr 2025 00:23:06 +0800
* guix/build-system/cargo.scm (cargo-build, cargo-cross-build)
[#:cargo-package-crates]: New argument.
* guix/build/cargo-build-system.scm (package): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
---
doc/guix.texi | 7 +++++++
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 21 +++++++++++++++++++--
3 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 7b418a4089..771547fafb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9493,6 +9493,13 @@ Build Systems
the binaries defined by the crate. Unless @code{install-source? #f} is
defined it will also install a source crate repository of itself and unpacked
sources, to ease in future hacking on rust packages.
+
+This build system supports cargo workspaces. Parameter
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+library crates to package in the @code{package} phase. Specified crates are
+packaged from left to right, in case there's dependency among them. For
+example, specifying @code{''("pcre2-sys" "pcre2")} will package
+@code{"pcre2-sys"} first and then @code{"pcre2"}.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index f2ffc67879..2c47c1feaf 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -98,6 +98,7 @@ (define* (cargo-build name inputs
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(features ''())
(skip-build? #f)
@@ -126,6 +127,7 @@ (define* (cargo-build name inputs
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
@@ -158,6 +160,7 @@ (define* (cargo-cross-build name
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(cargo-target (cargo-triplet (or target system)))
(features ''())
@@ -189,6 +192,7 @@ (define* (cargo-cross-build name
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index c37fd70418..85745d4d91 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -302,11 +302,16 @@ (define* (package #:key
source
skip-build?
install-source?
+ (cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
+ (vendor-dir "guix-vendor")
#:allow-other-keys)
"Run 'cargo-package' for a given Cargo package."
(if install-source?
- (if skip-build?
+ ;; NOTE: Cargo workspace packaging support:
+ ;; #:install-source? #t + #:cargo-package-crates.
+ (if (and (null? cargo-package-crates)
+ skip-build?)
(begin
(install-file source "target/package")
(with-directory-excursion "target/package"
@@ -324,7 +329,19 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+
+ (if (null? cargo-package-crates)
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (find-files "target/package" "\\.crate$"))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:15 GMT) Full text and rfc822 format available.Message #302 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 10/22] build-system: cargo: Support installing Cargo workspace. Date: Tue, 29 Apr 2025 00:23:07 +0800
*guix/build-system/cargo.scm (cargo-build, cargo-cross-build)
[#:cargo-install-paths]: New argument.
* guix/build/cargo-build-system.scm (install): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
---
doc/guix.texi | 5 ++++-
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 17 +++++++++++++----
3 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 771547fafb..6388f7b28f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9499,7 +9499,10 @@ Build Systems
library crates to package in the @code{package} phase. Specified crates are
packaged from left to right, in case there's dependency among them. For
example, specifying @code{''("pcre2-sys" "pcre2")} will package
-@code{"pcre2-sys"} first and then @code{"pcre2"}.
+@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
+binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
+for example.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 2c47c1feaf..7a07003262 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -100,6 +100,7 @@ (define* (cargo-build name inputs
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(features ''())
(skip-build? #f)
(parallel-build? #t)
@@ -129,6 +130,7 @@ (define* (cargo-build name inputs
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
@@ -162,6 +164,7 @@ (define* (cargo-cross-build name
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(cargo-target (cargo-triplet (or target system)))
(features ''())
(skip-build? #f)
@@ -194,6 +197,7 @@ (define* (cargo-cross-build name
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 85745d4d91..be96ab309f 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -379,6 +379,7 @@ (define* (install #:key
skip-build?
install-source?
features
+ (cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
(let* ((out (assoc-ref outputs "out"))
@@ -393,10 +394,18 @@ (define* (install #:key
;; Only install crates which include binary targets,
;; otherwise cargo will raise an error.
(or skip-build?
- (not (has-executable-target?))
- (invoke "cargo" "install" "--offline" "--no-track"
- "--path" "." "--root" out
- "--features" (string-join features)))
+ ;; NOTE: Cargo workspace installation support:
+ ;; #:skip-build? #f + #:cargo-install-paths.
+ (and (null? cargo-install-paths)
+ (not (has-executable-target?)))
+ (for-each
+ (lambda (path)
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" path "--root" out
+ "--features" (string-join features)))
+ (if (null? cargo-install-paths)
+ '(".")
+ cargo-install-paths)))
(when install-source?
;; Install crate tarballs and unpacked sources for later use.
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:15 GMT) Full text and rfc822 format available.Message #305 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH v4 rust-team 12/22] build/cargo: Don't try to unpack sanity-check.py. Date: Tue, 29 Apr 2025 00:23:09 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (crate-src?): Also don't try to
check python scripts for Cargo.toml.
Change-Id: I001a89b83d2e472706b1263007be45d1153c140f
---
guix/build/cargo-build-system.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index d8433e1acd..7b519607f2 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -65,6 +65,7 @@ (define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
(and (not (directory-exists? path)) ; not a tarball
+ (not (string-suffix? "py" path)) ; sanity-check.py
;; First we print out all file names within the tarball to see if it
;; looks like the source of a crate. However, the tarball will include
;; an extra path component which we would like to ignore (since we're
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:16 GMT) Full text and rfc822 format available.Message #308 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 13/22] build/cargo: Support non-workspace directory source inputs. Date: Tue, 29 Apr 2025 00:23:10 +0800
* guix/build/cargo-build-system.scm (single-crate?): New procedure.
(crate-src?): Support non-workspace directory source inputs.
(configure): Likewise.
Change-Id: If2bd318b44ed765baec69309f25d320edceee116
---
guix/build/cargo-build-system.scm | 60 ++++++++++++++++++++-----------
1 file changed, 39 insertions(+), 21 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 7b519607f2..d1020bab52 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -34,6 +34,7 @@ (define-module (guix build cargo-build-system)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 textual-ports)
#:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -61,22 +62,35 @@ (define (has-executable-target?)
(bin-dep? (lambda (dep) (find bin? (get-kinds dep)))))
(find bin-dep? (manifest-targets))))
+(define (single-crate? dir)
+ "Check if directory DIR contains 'Cargo.toml' and is not a workspace."
+ (let ((manifest-file (in-vicinity dir "Cargo.toml")))
+ (and (file-exists? manifest-file)
+ (not (string-contains
+ (call-with-input-file manifest-file get-string-all)
+ "[workspace]")))))
+
(define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
- (and (not (directory-exists? path)) ; not a tarball
- (not (string-suffix? "py" path)) ; sanity-check.py
- ;; First we print out all file names within the tarball to see if it
- ;; looks like the source of a crate. However, the tarball will include
- ;; an extra path component which we would like to ignore (since we're
- ;; interested in checking if a Cargo.toml exists at the root of the
- ;; archive, but not nested anywhere else). We do this by cutting up
- ;; each output line and only looking at the second component. We then
- ;; check if it matches Cargo.toml exactly and short circuit if it does.
- (apply invoke (list "sh" "-c"
- (string-append "tar -tf " path
- " | cut -d/ -f2"
- " | grep -q '^Cargo.toml$'")))))
+ (if (directory-exists? path)
+ ;; The build system only handles sources containing single crate.
+ ;; Workspaces should be packaged into crates (via 'package phase)
+ ;; and used in inputs.
+ (single-crate? path)
+ (and (not (string-suffix? "py" path)) ;sanity-check.py
+ ;; First we print out all file names within the tarball to see
+ ;; if it looks like the source of a crate. However, the tarball
+ ;; will include an extra path component which we would like to
+ ;; ignore (since we're interested in checking if a Cargo.toml
+ ;; exists at the root of the archive, but not nested anywhere
+ ;; else). We do this by cutting up each output line and only
+ ;; looking at the second component. We then check if it matches
+ ;; Cargo.toml exactly and short circuit if it does.
+ (invoke "sh" "-c"
+ (string-append "tar -tf " path
+ " | cut -d/ -f2"
+ " | grep -q '^Cargo.toml$'")))))
(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
#:allow-other-keys)
@@ -161,14 +175,18 @@ (define* (configure #:key inputs
(and (crate-src? path)
;; Gracefully handle duplicate inputs
(not (file-exists? crate-dir))
- (mkdir-p crate-dir)
- ;; Cargo crates are simply gzipped tarballs but with a .crate
- ;; extension. We expand the source to a directory name we control
- ;; so that we can generate any cargo checksums.
- ;; The --strip-components argument is needed to prevent creating
- ;; an extra directory within `crate-dir`.
- (format #t "Unpacking ~a~%" name)
- (invoke "tar" "xf" path "-C" crate-dir "--strip-components" "1")))))
+ (if (directory-exists? path)
+ (copy-recursively path crate-dir)
+ (begin
+ (mkdir-p crate-dir)
+ ;; Cargo crates are simply gzipped tarballs but with a
+ ;; .crate extension. We expand the source to a directory
+ ;; name we control so that we can generate any cargo
+ ;; checksums. The --strip-components argument is needed to
+ ;; prevent creating an extra directory within `crate-dir`.
+ (format #t "Unpacking ~a~%" name)
+ (invoke "tar" "xf" path "-C" crate-dir
+ "--strip-components" "1")))))))
inputs)
;; For cross-building
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, lars <at> 6xq.net, liliana.prikler <at> gmail.com, maxim.cournoyer <at> gmail.com, me <at> bonfacemunyoki.com, noelopez <at> free.fr, sharlatanus <at> gmail.com, steve <at> futurile.net, tanguy <at> bioneland.org, vivien <at> planete-kraus.eu, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:17 GMT) Full text and rfc822 format available.Message #311 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 14/22] build/cargo: Don't vendor self. Date: Tue, 29 Apr 2025 00:23:11 +0800
* guix/build/cargo-build-system.scm (crate-src?, configure): Exclude self.
* gnu/packages/gnome.scm (librsvg)[#:phases]: Remove ‘dont-vendor-self’.
* gnu/packages/python-crypto.scm (python-cryptography)[#:arguments]: Likewise.
* gnu/packages/python-xyz.scm (python-libcst)[#:arguments]: Likewise.
* gnu/packages/syndication.scm (newsboat)[#:arguments]: Likewise.
* gnu/packages/antivirus.scm (clamav)[#:arguments]: Likewise.
Change-Id: I5e01721fb705937f5fcf858c21b6d7e27ade6c10
---
gnu/packages/antivirus.scm | 5 +---
gnu/packages/gnome.scm | 5 ----
gnu/packages/python-crypto.scm | 5 ----
gnu/packages/python-xyz.scm | 5 ----
gnu/packages/syndication.scm | 5 ----
guix/build/cargo-build-system.scm | 43 ++++++++++++++++---------------
6 files changed, 23 insertions(+), 45 deletions(-)
diff --git a/gnu/packages/antivirus.scm b/gnu/packages/antivirus.scm
index 4a01115abd..d2060fa385 100644
--- a/gnu/packages/antivirus.scm
+++ b/gnu/packages/antivirus.scm
@@ -116,10 +116,7 @@ (define-public clamav
(add-after 'configure 'fix-cargo-inputs-vendoring
(lambda _
;; Reproduce the original layout, fails with config.
- (rename-file ".cargo/config" ".cargo/config.toml")
- ;; Wrongly placed by configure
- (delete-file-recursively
- (string-append ".cargo/vendor/clamav-" #$version ".tar.zst"))))
+ (rename-file ".cargo/config" ".cargo/config.toml")))
(add-after 'patch-cargo-checksums 'cmake-configure
(lambda* (#:key outputs #:allow-other-keys)
((assoc-ref cmake:%standard-phases 'configure)
diff --git a/gnu/packages/gnome.scm b/gnu/packages/gnome.scm
index 26416df155..3de0dfb293 100644
--- a/gnu/packages/gnome.scm
+++ b/gnu/packages/gnome.scm
@@ -3780,11 +3780,6 @@ (define-public librsvg
"--host=" #$(%current-target-system)))
#~("--enable-vala")))
args)))
- (add-after 'configure 'dont-vendor-self
- (lambda* (#:key vendor-dir #:allow-other-keys)
- ;; Don't keep the whole tarball in the vendor directory
- (delete-file-recursively
- (string-append vendor-dir "/" #$name "-" #$version ".tar.xz"))))
(replace 'build
(assoc-ref gnu:%standard-phases 'build))
(replace 'check
diff --git a/gnu/packages/python-crypto.scm b/gnu/packages/python-crypto.scm
index d004f6a69e..f7b6609315 100644
--- a/gnu/packages/python-crypto.scm
+++ b/gnu/packages/python-crypto.scm
@@ -561,11 +561,6 @@ (define-public python-cryptography
#:install-source? #false
#:phases
#~(modify-phases %standard-phases
- (add-after 'configure 'dont-vendor-self
- (lambda* (#:key vendor-dir #:allow-other-keys)
- ;; Don't keep the whole tarball in the vendor directory
- (delete-file-recursively
- (string-append vendor-dir "/cryptography-" #$version ".tar.zst"))))
(replace 'build
(assoc-ref py:%standard-phases 'build))
(delete 'check)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index c63a0a7708..b56d15f202 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -23694,11 +23694,6 @@ (define-public python-libcst
(add-after 'unpack 'prepare-source
(lambda _
(delete-file "native/Cargo.lock")))
- (add-after 'configure 'dont-vendor-self
- (lambda* (#:key vendor-dir #:allow-other-keys)
- ;; Don't keep the whole tarball in the vendor directory
- (delete-file-recursively
- (string-append vendor-dir "/libcst-" #$version ".tar.zst"))))
(replace 'build
(assoc-ref py:%standard-phases 'build))
(add-after 'install 'wrap
diff --git a/gnu/packages/syndication.scm b/gnu/packages/syndication.scm
index 61220bb9fe..b597bf9004 100644
--- a/gnu/packages/syndication.scm
+++ b/gnu/packages/syndication.scm
@@ -311,11 +311,6 @@ (define-public newsboat
(("if curl-config")
(string-append
"if " (search-input-file inputs "/bin/curl-config"))))))
- (add-after 'configure 'dont-vendor-self
- (lambda* (#:key vendor-dir #:allow-other-keys)
- ;; Don't keep the whole tarball in the vendor directory
- (delete-file-recursively
- (string-append vendor-dir "/" #$name "-" #$version ".tar.xz"))))
(add-after 'unpack 'patch-source
(lambda* (#:key outputs #:allow-other-keys)
(substitute* "Makefile"
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index d1020bab52..c6865e6e81 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -70,27 +70,28 @@ (define (single-crate? dir)
(call-with-input-file manifest-file get-string-all)
"[workspace]")))))
-(define (crate-src? path)
+(define* (crate-src? path #:key source)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
- (if (directory-exists? path)
- ;; The build system only handles sources containing single crate.
- ;; Workspaces should be packaged into crates (via 'package phase)
- ;; and used in inputs.
- (single-crate? path)
- (and (not (string-suffix? "py" path)) ;sanity-check.py
- ;; First we print out all file names within the tarball to see
- ;; if it looks like the source of a crate. However, the tarball
- ;; will include an extra path component which we would like to
- ;; ignore (since we're interested in checking if a Cargo.toml
- ;; exists at the root of the archive, but not nested anywhere
- ;; else). We do this by cutting up each output line and only
- ;; looking at the second component. We then check if it matches
- ;; Cargo.toml exactly and short circuit if it does.
- (invoke "sh" "-c"
- (string-append "tar -tf " path
- " | cut -d/ -f2"
- " | grep -q '^Cargo.toml$'")))))
+ (and (not (string=? path source)) ;Exclude self.
+ (if (directory-exists? path)
+ ;; The build system only handles sources containing single crate.
+ ;; Workspaces should be packaged into crates (via 'package phase)
+ ;; and used in inputs.
+ (single-crate? path)
+ (and (not (string-suffix? "py" path)) ;sanity-check.py
+ ;; First we print out all file names within the tarball to see
+ ;; if it looks like the source of a crate. However, the tarball
+ ;; will include an extra path component which we would like to
+ ;; ignore (since we're interested in checking if a Cargo.toml
+ ;; exists at the root of the archive, but not nested anywhere
+ ;; else). We do this by cutting up each output line and only
+ ;; looking at the second component. We then check if it matches
+ ;; Cargo.toml exactly and short circuit if it does.
+ (invoke "sh" "-c"
+ (string-append "tar -tf " path
+ " | cut -d/ -f2"
+ " | grep -q '^Cargo.toml$'"))))))
(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
#:allow-other-keys)
@@ -155,7 +156,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(format #t "error: Possible pre-generated file found: ~a~%" file)))
(find-files "." (negate empty-file?)))))
-(define* (configure #:key inputs
+(define* (configure #:key source inputs
target system
(cargo-target #f)
(vendor-dir "guix-vendor")
@@ -172,7 +173,7 @@ (define* (configure #:key inputs
((name . path)
(let* ((basepath (strip-store-file-name path))
(crate-dir (string-append vendor-dir "/" basepath)))
- (and (crate-src? path)
+ (and (crate-src? path #:source source)
;; Gracefully handle duplicate inputs
(not (file-exists? crate-dir))
(if (directory-exists? path)
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:17 GMT) Full text and rfc822 format available.Message #314 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 15/22] build/cargo: Use system libsqlite3. Date: Tue, 29 Apr 2025 00:23:12 +0800
* guix/build/cargo-build-system.scm (configure): Set
LIBSQLITE3_SYS_USE_PKG_CONFIG.
Change-Id: I9a8a5818992ee687ec67abadffef67ce1533c6d3
---
guix/build/cargo-build-system.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index c6865e6e81..c25c26fa76 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -248,6 +248,7 @@ (define* (configure #:key source inputs
(setenv "CC" (string-append (assoc-ref inputs "gcc") "/bin/gcc")))
(setenv "LIBGIT2_SYS_USE_PKG_CONFIG" "1")
+ (setenv "LIBSQLITE3_SYS_USE_PKG_CONFIG" "1")
(setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
(setenv "SODIUM_USE_PKG_CONFIG" "1")
(setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:18 GMT) Full text and rfc822 format available.Message #317 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 16/22] build/cargo: Use sytsem oniguruma. Date: Tue, 29 Apr 2025 00:23:13 +0800
* guix/build/cargo-build-system.scm (configure): Set RUSTONIG_SYSTEM_LIBONIG. Change-Id: I02624d98ade15d65965f46555e5a210a7630dd5d --- guix/build/cargo-build-system.scm | 1 + 1 file changed, 1 insertion(+) diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm index c25c26fa76..39a9487841 100644 --- a/guix/build/cargo-build-system.scm +++ b/guix/build/cargo-build-system.scm @@ -250,6 +250,7 @@ (define* (configure #:key source inputs (setenv "LIBGIT2_SYS_USE_PKG_CONFIG" "1") (setenv "LIBSQLITE3_SYS_USE_PKG_CONFIG" "1") (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1") + (setenv "RUSTONIG_SYSTEM_LIBONIG" "1") (setenv "SODIUM_USE_PKG_CONFIG" "1") (setenv "ZSTD_SYS_USE_PKG_CONFIG" "1") (when (assoc-ref inputs "openssl") -- 2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:18 GMT) Full text and rfc822 format available.Message #320 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 17/22] build/cargo: Unconditionally use system gettext. Date: Tue, 29 Apr 2025 00:23:14 +0800
No special value is required for this environment variable.
* guix/build/cargo-build-system.scm (configure): Set GETTEXT_SYSTEM
unconditionally.
Change-Id: Id40fecaf75fa76c7b7f1404db226fa1ca6a45a5e
---
guix/build/cargo-build-system.scm | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 39a9487841..edc4e0d836 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -247,6 +247,7 @@ (define* (configure #:key source inputs
(setenv "TARGET_PKG_CONFIG" (string-append target "-pkg-config")))
(setenv "CC" (string-append (assoc-ref inputs "gcc") "/bin/gcc")))
+ (setenv "GETTEXT_SYSTEM" "1")
(setenv "LIBGIT2_SYS_USE_PKG_CONFIG" "1")
(setenv "LIBSQLITE3_SYS_USE_PKG_CONFIG" "1")
(setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
@@ -255,8 +256,6 @@ (define* (configure #:key source inputs
(setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
(when (assoc-ref inputs "openssl")
(setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
- (when (assoc-ref inputs "gettext")
- (setenv "GETTEXT_SYSTEM" (assoc-ref inputs "gettext")))
(when (assoc-ref inputs "clang")
(setenv "LIBCLANG_PATH"
(string-append (assoc-ref inputs "clang") "/lib")))
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:21 GMT) Full text and rfc822 format available.Message #323 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 19/22] build-system: cargo: Export ‘cargo-triplet’. Date: Tue, 29 Apr 2025 00:23:16 +0800
This will make Rust cross compilation support in other build systems easier.
* guix/build-system/cargo.scm (cargo-triplet): Export.
Make ‘target’ argument optional.
Change-Id: I226bf4a7d8065eb7911da148d0cdf3352a47bde6
---
guix/build-system/cargo.scm | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 659ad95a35..045809874c 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -51,7 +51,8 @@ (define-module (guix build-system cargo)
crate-name->package-name
crate-source
define-cargo-inputs
- cargo-inputs))
+ cargo-inputs
+ cargo-triplet))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -118,7 +119,9 @@ (define (default-rust-sysroot target)
(let ((module (resolve-interface '(gnu packages rust))))
(module-ref module 'make-rust-sysroot)))
-(define (cargo-triplet target)
+(define* (cargo-triplet #:optional
+ (target (or (%current-target-system)
+ (%current-system))))
(false-if-exception
(platform-rust-target
(lookup-platform-by-target-or-system target))))
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:24 GMT) Full text and rfc822 format available.Message #326 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 18/22] build-system: cargo: Add ‘cargo-inputs’. Date: Tue, 29 Apr 2025 00:23:15 +0800
* gnu/packages/rust-crates.scm: New file.
* gnu/packages/rust-sources.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them.
* guix/build-system/cargo.scm (define-cargo-inputs): New macro.
(crate-source, cargo-inputs): New procedures.
* guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo).
* etc/teams/rust/audit-rust-crates: New file.
* etc/teams/rust/cleanup-crates.sh: New file.
* etc/teams/rust/rust-crates.tmpl: New file.
* etc/teams/rust/unpack-new-crates.sh: New file.
Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3
---
etc/teams/rust/audit-rust-crates | 70 +++++++++++++++++++++++++++++
etc/teams/rust/cleanup-crates.sh | 37 +++++++++++++++
etc/teams/rust/rust-crates.tmpl | 44 ++++++++++++++++++
etc/teams/rust/unpack-new-crates.sh | 45 +++++++++++++++++++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 44 ++++++++++++++++++
gnu/packages/rust-sources.scm | 29 ++++++++++++
guix/build-system/cargo.scm | 46 ++++++++++++++++++-
guix/import/crate.scm | 2 +-
9 files changed, 317 insertions(+), 2 deletions(-)
create mode 100755 etc/teams/rust/audit-rust-crates
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100755 etc/teams/rust/unpack-new-crates.sh
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
diff --git a/etc/teams/rust/audit-rust-crates b/etc/teams/rust/audit-rust-crates
new file mode 100755
index 0000000000..d5546fd1e1
--- /dev/null
+++ b/etc/teams/rust/audit-rust-crates
@@ -0,0 +1,70 @@
+#!/usr/bin/env -S gawk -f
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Efraim Flashner <efraim <at> flashner.co.il>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+# To run:
+# ./etc/teams/rust/audit-rust-crates ./path/to/file.scm
+# Prints the output of cargo-audit to the shell.
+
+# Make sure we have cargo-audit in our PATH
+BEGIN {
+ if (system("which cargo-audit 1> /dev/null"))
+ exit 1;
+ # Parse a record at a time.
+ RS = "\n\n"
+ cargoAudit = "cargo-audit audit --file -"
+}
+
+# Check the crate-source origin-only inputs
+/crate-source/ {
+ for(i=3; i <= NF-2; i++) {
+ if($i == "(crate-source") {
+ cargoLock = cargoLock "[[package]]\nname = " $(i+1) "\nversion = " $(i+2) "\n"
+ next
+ }
+ }
+}
+
+# Check the crates packaged from crates.io tarballs
+/crate-uri/ {
+ for(i=3; i <= NF; i++) {
+ if($i == "(version")
+ crateVersion = $(i+1)
+ if($i == "(crate-uri")
+ crateName = $(i+1)
+ }
+ gsub(/)/, "", crateVersion)
+ cargoLock = cargoLock "[[package]]\nname = " crateName "\nversion = " crateVersion "\n"
+}
+
+# The xxxx-cargo-input variables have a set style
+# TODO: Replace the last dash between the name and the version with a space!
+# This doesn't take into account swapping between "-" and "_" so we skip it.
+#( $2 ~ /-cargo-inputs/ ) {
+# sub(/-cargo-inputs/, "", $2)
+# gsub(/)/, "", $0)
+# gsub(/rust-/, "", $0)
+# #gensub(/([[:alpha:]])-([[:digit:]]+)/, "\\1 \\2", "g", $i)
+# print "[[package]]\nname = \"" $2 "\"\nversion = \"1.0.0\"\ndependencies = ["
+# for (i = 4; i <= NF; i++) {
+# print "\"" $i "\","
+# }
+# print "]"
+#}
+
+END { print cargoLock | cargoAudit }
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
new file mode 100755
index 0000000000..eca37ca00c
--- /dev/null
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -0,0 +1,37 @@
+#!/bin/sh
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+FILE=gnu/packages/rust-crates.scm
+PATTERN='^(define rust-'
+
+grep "$PATTERN" $FILE | cut -d' ' -f2 | while IFS= read -r crate
+do
+ if [ "$(grep -wc "$crate" $FILE)" -eq 1 ]; then
+ echo "\
+(begin
+ (use-modules (guix utils))
+ (let ((source-properties
+ (find-definition-location \"$FILE\" '$crate #:define-prefix 'define)))
+ (and=> source-properties delete-expression)))" |
+ guix repl -t machine
+ fi
+done
+
+# Delete extra newlines.
+sed --in-place ':a;N;$!ba;s/\n\n\+/\n\n/g' $FILE
diff --git a/etc/teams/rust/rust-crates.tmpl b/etc/teams/rust/rust-crates.tmpl
new file mode 100644
index 0000000000..a2c7fcd27d
--- /dev/null
+++ b/etc/teams/rust/rust-crates.tmpl
@@ -0,0 +1,44 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo)
+ #:use-module (gnu packages rust-sources)
+ #:export (lookup-cargo-inputs))
+
+;;;
+;;; This file is managed by ‘guix import’. Do NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust libraries fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/etc/teams/rust/unpack-new-crates.sh b/etc/teams/rust/unpack-new-crates.sh
new file mode 100755
index 0000000000..7ff1080655
--- /dev/null
+++ b/etc/teams/rust/unpack-new-crates.sh
@@ -0,0 +1,45 @@
+#!/bin/sh
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+# Usage: ./etc/teams/rust/unpack-new-crates.sh <DIR>
+# Then inspect DIR with your tool of choice.
+#
+# For example:
+# ./etc/teams/rust/unpack-new-crates.sh tmp-rust-crates &&
+# emacs --eval '(find-dired "tmp-rust-crates" "")'
+#
+# Note this uses ‘git diff’, only unstaged one are unpacked.
+
+UNPACK_DIR="$1"
+CHANGES="$(git diff gnu/packages/rust-crates.scm | grep -E '^\+\(define rust-' | cut -f2 -d' ')"
+
+rm -rf "$UNPACK_DIR"
+mkdir -p "$UNPACK_DIR"
+
+for crate in $CHANGES
+do
+ built="$(./pre-inst-env guix build -e "(@@ (gnu packages rust-crates) "$crate")" -v0 2>/dev/null)"
+ if [[ -n $built ]]; then
+ if [[ -d $built ]]; then
+ cp -r "$built" "$UNPACK_DIR"
+ else
+ tar xf "$built" -C "$UNPACK_DIR"
+ fi
+ fi
+done
diff --git a/gnu/local.mk b/gnu/local.mk
index 4854e2fe92..c60c7f1fc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -616,6 +616,8 @@ GNU_SYSTEM_MODULES = \
%D%/packages/rush.scm \
%D%/packages/rust.scm \
%D%/packages/rust-apps.scm \
+ %D%/packages/rust-crates.scm \
+ %D%/packages/rust-sources.scm \
%D%/packages/samba.scm \
%D%/packages/sagemath.scm \
%D%/packages/sawfish.scm \
diff --git a/gnu/packages/rust-crates.scm b/gnu/packages/rust-crates.scm
new file mode 100644
index 0000000000..a2c7fcd27d
--- /dev/null
+++ b/gnu/packages/rust-crates.scm
@@ -0,0 +1,44 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo)
+ #:use-module (gnu packages rust-sources)
+ #:export (lookup-cargo-inputs))
+
+;;;
+;;; This file is managed by ‘guix import’. Do NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust libraries fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/packages/rust-sources.scm b/gnu/packages/rust-sources.scm
new file mode 100644
index 0000000000..f8d25bbc2c
--- /dev/null
+++ b/gnu/packages/rust-sources.scm
@@ -0,0 +1,29 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-sources)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; Cargo workspaces and Rust libraries requiring external inputs to unbundle.
+;;; These packages are hidden, as they are not interesting to users.
+;;;
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 7a07003262..659ad95a35 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -25,12 +25,15 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix build-system cargo)
+ #:use-module (guix diagnostics)
+ #:use-module (guix i18n)
#:use-module (guix search-paths)
#:use-module (guix store)
#:use-module (guix utils)
#:use-module (guix gexp)
#:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix download)
#:use-module (guix platform)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
@@ -45,7 +48,10 @@ (define-module (guix build-system cargo)
crate-url
crate-url?
crate-uri
- crate-name->package-name))
+ crate-name->package-name
+ crate-source
+ define-cargo-inputs
+ cargo-inputs))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -62,6 +68,44 @@ (define (crate-uri name version)
(define (crate-name->package-name name)
(downstream-package-name "rust-" name))
+;; NOTE: Only use this procedure in (gnu packages rust-crates).
+(define* (crate-source name version hash #:key (patches '()) (snippet #f))
+ (origin
+ (method url-fetch)
+ (uri (crate-uri name version))
+ (file-name
+ (string-append (crate-name->package-name name) "-" version ".tar.gz"))
+ (sha256 (base32 hash))
+ (modules '((guix build utils)))
+ (patches patches)
+ (snippet snippet)))
+
+(define-syntax define-cargo-inputs
+ (syntax-rules (=>)
+ ((_ lookup inputs ...)
+ (define lookup
+ (let ((table (make-hash-table)))
+ (letrec-syntax ((record
+ (syntax-rules (=>)
+ ((_) #t)
+ ((_ (name => lst) rest (... ...))
+ (begin
+ (hashq-set! table 'name (filter identity lst))
+ (record rest (... ...)))))))
+ (record inputs ...)
+ (lambda (name)
+ "Return the inputs for NAME."
+ (hashq-ref table name))))))))
+
+(define* (cargo-inputs name #:key (module '(gnu packages rust-crates)))
+ "Lookup Cargo inputs for NAME defined in MODULE, return an empty list if
+unavailable."
+ (let ((lookup (module-ref (resolve-interface module) 'lookup-cargo-inputs)))
+ (or (lookup name)
+ (begin
+ (warning (G_ "no Cargo inputs available for '~a'~%") name)
+ '()))))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 14e6e28c5b..a6f247bbae 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -27,7 +27,7 @@
(define-module (guix import crate)
#:use-module (guix base32)
- #:use-module (guix build-system cargo)
+ #:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:32:26 GMT) Full text and rfc822 format available.Message #329 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 20/22] build/cargo: Accept rust-sysroot from native-inputs. Date: Tue, 29 Apr 2025 00:23:17 +0800
This is a workaround to support Rust cross-compilation in build systems other
than cargo-build-system. Since there's no interface for target-inputs, using
native-inputs makes most sense.
* guix/build-system/cargo.scm (lower)[bag]<target-inputs>: Label rust-sysroot
with its package name.
* guix/build/cargo-build-system.scm (configure): Use it.
When cross-compiling, accept rust-sysroot from native-inputs if there's none
in inputs.
Change-Id: I5cb63c8c815d3a2c9f0ece01857254f6d0090dd1
---
guix/build-system/cargo.scm | 3 ++-
guix/build/cargo-build-system.scm | 15 +++++++++++----
2 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 045809874c..d2d9a075b9 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -430,7 +430,8 @@ (define* (lower name
;; This provides a separate sysroot for the regular rustc
,@(if target
- `(("rust-sysroot" ,(rust-sysroot target)))
+ `((,(string-append "rust-sysroot-for-" target)
+ ,(rust-sysroot target)))
'())))
(outputs outputs)
(build (if target cargo-cross-build cargo-build))
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index edc4e0d836..45b17df6ff 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -156,7 +156,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(format #t "error: Possible pre-generated file found: ~a~%" file)))
(find-files "." (negate empty-file?)))))
-(define* (configure #:key source inputs
+(define* (configure #:key source inputs native-inputs
target system
(cargo-target #f)
(vendor-dir "guix-vendor")
@@ -193,9 +193,16 @@ (define* (configure #:key source inputs
;; For cross-building
(when target
(setenv "CARGO_BUILD_TARGET" cargo-target)
- (setenv "RUSTFLAGS" (string-append
- (or (getenv "RUSTFLAGS") "")
- " --sysroot " (assoc-ref inputs "rust-sysroot")))
+ (setenv "RUSTFLAGS"
+ (string-append
+ (or (getenv "RUSTFLAGS") "")
+ " --sysroot "
+ (assoc-ref
+ (append inputs
+ ;; Workaround for other build systems, as no interface
+ ;; is available for target-inputs.
+ (or native-inputs '()))
+ (string-append "rust-sysroot-for-" target))))
(setenv "PKG_CONFIG" (string-append target "-pkg-config"))
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:33:02 GMT) Full text and rfc822 format available.Message #332 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Murilo <murilo <at> disroot.org>, Hilton Chain <hako <at> ultrarare.space>, Luis Guilherme Coelho <lgcoelho <at> disroot.org> Subject: [PATCH v4 rust-team 21/22] import: crate: Add Cargo.lock parser. Date: Tue, 29 Apr 2025 00:23:18 +0800
* guix/import/crate/cargo-lock.scm: New file.
* Makefile.am (MODULES): Regisiter it.
* etc/teams.scm (rust)[#:scope]: Add it.
* guix/import/crate.scm (cargo-lock->expressions): New procedure.
* tests/crate.scm (temp-file): New variable.
("crate-lockfile-import"): New test.
Co-authored-by: Murilo <murilo <at> disroot.org>
Co-authored-by: Luis Guilherme Coelho <lgcoelho <at> disroot.org>
Change-Id: I95421e9e2ba11a671b4bc4e1323c6d31a1b012c5
---
Makefile.am | 1 +
etc/teams.scm | 1 +
guix/import/crate.scm | 78 +++++++++++++++++++++++
guix/import/crate/cargo-lock.scm | 105 +++++++++++++++++++++++++++++++
tests/crate.scm | 87 +++++++++++++++++++++++++
5 files changed, 272 insertions(+)
create mode 100644 guix/import/crate/cargo-lock.scm
diff --git a/Makefile.am b/Makefile.am
index 0c12754050..8a32be3ef9 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -295,6 +295,7 @@ MODULES = \
guix/import/cpan.scm \
guix/import/cran.scm \
guix/import/crate.scm \
+ guix/import/crate/cargo-lock.scm \
guix/import/egg.scm \
guix/import/elm.scm \
guix/import/elpa.scm \
diff --git a/etc/teams.scm b/etc/teams.scm
index 6f10941c72..4fe121e63f 100755
--- a/etc/teams.scm
+++ b/etc/teams.scm
@@ -390,6 +390,7 @@ (define-team rust
"guix/build/cargo-utils.scm"
"guix/build-system/cargo.scm"
"guix/import/crate.scm"
+ "guix/import/crate/cargo-lock.scm"
"guix/scripts/import/crate.scm"
"tests/crate.scm")))
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index a6f247bbae..39da867805 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -9,6 +9,9 @@
;;; Copyright © 2023, 2024 David Elsing <david.elsing <at> posteo.net>
;;; Copyright © 2024 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;; Copyright © 2025 Herman Rimm <herman <at> rimm.ee>
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024-2025 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -26,12 +29,14 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix import crate)
+ #:use-module (guix base16)
#:use-module (guix base32)
#:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
#:use-module (guix i18n)
+ #:use-module (guix import crate cargo-lock)
#:use-module (guix import json)
#:use-module (guix import utils)
#:use-module (guix memoization)
@@ -39,9 +44,11 @@ (define-module (guix import crate)
#:use-module (guix read-print)
#:use-module (guix upstream)
#:use-module (guix utils)
+ #:use-module (guix scripts download)
#:use-module (gnu packages)
#:use-module (ice-9 match)
#:use-module (ice-9 regex)
+ #:use-module (ice-9 textual-ports)
#:use-module (json)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
@@ -52,6 +59,7 @@ (define-module (guix import crate)
guix-package->crate-name
string->license
crate-recursive-import
+ cargo-lock->expressions
%crate-updater))
@@ -481,6 +489,76 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
+
+;;;
+;;; Convert ‘Cargo.lock’ to Guix sources.
+;;;
+
+(define (cargo-lock->expressions lockfile package-name)
+ "Given LOCKFILE, a 'Cargo.lock' file, import its content as source
+expressions. Return a source list and a Cargo inputs entry for PACKAGE-NAME
+referencing all imported sources."
+ (define (crate->guix-source crate)
+ (match crate
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source _)
+ ('crate-checksum checksum))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,@(if (or (string-suffix? "src" name)
+ (string-suffix? "sys" name))
+ (list (comment ";; TODO: Check bundled sources.\n" #f))
+ '())
+ (crate-source ,name ,version
+ ,(bytevector->nix-base32-string
+ (base16-string->bytevector checksum)))))
+ ;; Git snapshot.
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source source))
+ (begin
+ (let* ((src (string-split source (char-set #\+ #\? #\#)))
+ (url (second src))
+ (commit (last src))
+ (version (string-append version "." (string-take commit 7)))
+ (checksum
+ (second
+ (string-split
+ (with-output-to-string
+ (lambda _
+ (guix-download "-g" url
+ (string-append "--commit=" commit))))
+ #\newline))))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,(comment
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url ,url)
+ (commit ,commit)))
+ (file-name
+ (git-file-name ,(crate-name->package-name name) ,version))
+ (sha256 (base32 ,checksum)))))))
+ ;; Cargo workspace member.
+ (else #f)))
+
+ (let* ((source-expressions
+ (filter-map crate->guix-source
+ (cargo-lock-string->scm
+ (call-with-input-file lockfile get-string-all))))
+ (cargo-inputs-entry
+ `(,(string->symbol package-name) =>
+ (list ,@(map second source-expressions)))))
+ (values source-expressions cargo-inputs-entry)))
+
;;;
;;; Updater
diff --git a/guix/import/crate/cargo-lock.scm b/guix/import/crate/cargo-lock.scm
new file mode 100644
index 0000000000..b8a6ab20c0
--- /dev/null
+++ b/guix/import/crate/cargo-lock.scm
@@ -0,0 +1,105 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import crate cargo-lock)
+ #:use-module (ice-9 peg)
+ #:export (cargo-lock-string->scm
+
+ crate-name
+ crate-version
+ crate-source
+ crate-checksum
+ crate-dependencies
+ cargo-lock))
+
+;;;
+;;; PEG parser for ‘Cargo.lock’.
+;;;
+
+(define (cargo-lock-string->scm str)
+ (peg:tree (search-for-pattern cargo-lock str)))
+
+;; Auxiliar peg patterns
+(define-peg-pattern numeric-char body
+ (range #\0 #\9))
+
+(define-peg-pattern lowercase-char body
+ (range #\a #\z))
+
+(define-peg-pattern uppercase-char body
+ (range #\A #\Z))
+
+(define-peg-pattern alphabetic-char body
+ (or lowercase-char uppercase-char))
+
+(define-peg-pattern alphanumeric-char body
+ (or alphabetic-char numeric-char))
+
+;; name
+(define-peg-pattern crate-name all
+ (+ (or "-" alphabetic-char
+ "_" numeric-char)))
+
+;; version
+(define-peg-pattern non-negative-integer body
+ (+ numeric-char))
+
+(define-peg-pattern crate-version all
+ (and non-negative-integer "."
+ non-negative-integer "."
+ non-negative-integer
+ (? (+ (or "-" lowercase-char
+ "." uppercase-char
+ "+" numeric-char "_")))))
+
+;; source
+(define-peg-pattern crate-source all
+ (and (or "registry" "git")
+ "+https://"
+ (+ (or "/" "." "?" "=" "-" "#" "_"
+ alphanumeric-char))))
+
+;; checksum
+(define-peg-pattern crate-checksum all
+ (+ (or lowercase-char numeric-char)))
+
+;; dependency specification
+(define-peg-pattern dependency-specification all
+ (and crate-name (? (and (ignore " ") crate-version))))
+
+;; dependencies
+(define-peg-pattern crate-dependencies all
+ (and (ignore "[\n")
+ (+ (and (ignore " \"")
+ (capture dependency-specification)
+ (ignore "\",\n")))
+ (ignore "]")))
+
+;; crates
+(define-peg-pattern crate all
+ (and (ignore "[[package]]\n")
+ (ignore "name = \"") (capture crate-name) (ignore "\"\n")
+ (ignore "version = \"") (capture crate-version) (ignore "\"\n")
+ (? (and (ignore "source = \"") (capture crate-source) (ignore "\"\n")))
+ (? (and (ignore "checksum = \"") (capture crate-checksum) (ignore "\"\n")))
+ (? (ignore (and "dependencies = " crate-dependencies "\n")))))
+
+;; Cargo.lock
+(define-peg-pattern cargo-lock all
+ (+ (and (ignore "\n") crate)))
diff --git a/tests/crate.scm b/tests/crate.scm
index d0dc779cd2..8c1f6e738d 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -34,6 +34,7 @@ (define-module (test-crate)
#:use-module (gnu packages)
#:use-module (ice-9 iconv)
#:use-module (ice-9 match)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-64))
@@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
(description #f)
(license #f)))
+(define temp-file
+ (string-append "t-crate-" (number->string (getpid))))
+
(test-begin "crate")
@@ -1178,4 +1182,87 @@ (define rust-leaf-bob-3.0.2-yanked
(x
(pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
+
+(test-assert "crate-lockfile-import"
+ (begin
+ (call-with-output-file temp-file
+ (lambda (port)
+ (display "\
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = \"adler2\"
+version = \"2.0.0\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
+
+[[package]]
+name = \"aho-corasick\"
+version = \"1.1.3\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
+dependencies = [
+ \"memchr\",
+]
+
+[[package]]
+name = \"smithay\"
+version = \"0.4.0\"
+source = \"git+https://github.com/Smithay/smithay.git?rev=\
+0cd3345c59f7cb139521f267956a1a4e33248393#\
+0cd3345c59f7cb139521f267956a1a4e33248393\"
+dependencies = [
+ \"appendlist\",
+]
+
+[[package]]
+name = \"test\"
+version = \"25.2.0\"\n" port)))
+ (mock
+ ((guix scripts download) guix-download
+ (lambda _
+ (format #t "~a~%~a~%"
+ "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
+ (let-values
+ (((source-expressions cargo-inputs-entry)
+ (cargo-lock->expressions temp-file "test")))
+ (and
+ (match source-expressions
+ (`((define rust-adler2-2.0.0
+ (crate-source
+ "adler2" "2.0.0"
+ "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
+ (define rust-aho-corasick-1.1.3
+ (crate-source
+ "aho-corasick" "1.1.3"
+ "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
+ (define rust-smithay-0.4.0.0cd3345
+ ,($ <comment>
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/Smithay/smithay.git")
+ (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
+ (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
+ (sha256
+ (base32
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
+ #t)
+ (x
+ (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
+ (match cargo-inputs-entry
+ (`(test => (list rust-adler2-2.0.0
+ rust-aho-corasick-1.1.3
+ rust-smithay-0.4.0.0cd3345))
+ #t)
+ (x
+ (pk 'fail x #f))))))))
+
(test-end "crate")
+
+(false-if-exception (delete-file temp-file))
--
2.49.0
guix <at> cbaines.net, divya <at> subvertising.org, efraim <at> flashner.co.il, hako <at> ultrarare.space, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, maxim.cournoyer <at> gmail.com, zimon.toutoune <at> gmail.com, steve <at> futurile.net, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Mon, 28 Apr 2025 16:33:02 GMT) Full text and rfc822 format available.Message #335 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH v4 rust-team 22/22] import: crate: Add ‘--lockfile’ option. Date: Tue, 29 Apr 2025 00:23:19 +0800
* guix/import/crate.scm (cargo-inputs-from-lockfile)
find-cargo-inputs-location, extract-cargo-inputs): New procedures.
* guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
(show-help): Add it.
(guix-import-crate): Use it.
* doc/guix.texi (Invoking guix import): Document it.
Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
---
doc/guix.texi | 5 +++
guix/import/crate.scm | 47 +++++++++++++++++++++++++++++
guix/scripts/import/crate.scm | 57 +++++++++++++++++++++++++++++------
3 files changed, 99 insertions(+), 10 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 6388f7b28f..7aa36b347a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14833,6 +14833,11 @@ Invoking guix import
If a crate dependency is not (yet) packaged, make the corresponding
input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
a comment.
+@item --lockfile=@var{file}
+@itemx -f @var{file}
+When @option{--lockfile} is specified, the importer will ignore other options
+and won't output package expressions, instead importing source expressions
+from @var{file}, a @file{Cargo.lock} file.
@end table
@item elm
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 39da867805..d425e07b48 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -52,6 +52,7 @@ (define-module (guix import crate)
#:use-module (json)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
#:use-module (srfi srfi-69)
#:use-module (srfi srfi-71)
@@ -60,6 +61,9 @@ (define-module (guix import crate)
string->license
crate-recursive-import
cargo-lock->expressions
+ cargo-inputs-from-lockfile
+ find-cargo-inputs-location
+ extract-cargo-inputs
%crate-updater))
@@ -559,6 +563,49 @@ (define (cargo-lock->expressions lockfile package-name)
(list ,@(map second source-expressions)))))
(values source-expressions cargo-inputs-entry)))
+(define* (cargo-inputs-from-lockfile #:optional (lockfile "Cargo.lock"))
+ "Given LOCKFILE (default to \"Cargo.lock\" in current directory), return a
+source list imported from it, to be used as package inputs. This procedure
+can be used for adding a manifest file within the source tree of a Rust
+application."
+ (let-values
+ (((source-expressions cargo-inputs-entry)
+ (cargo-lock->expressions lockfile "cargo-inputs-temporary")))
+ (eval-string
+ (call-with-output-string
+ (lambda (port)
+ (for-each
+ (cut pretty-print-with-comments port <>)
+ `((use-modules (guix build-system cargo))
+ ,@source-expressions
+ (define-cargo-inputs lookup-cargo-inputs ,cargo-inputs-entry)
+ (lookup-cargo-inputs 'cargo-inputs-temporary))))))))
+
+(define (find-cargo-inputs-location file)
+ "Search in FILE for a top-level definition of Cargo inputs. Return the
+location if found, or #f otherwise."
+ (find-definition-location file 'lookup-cargo-inputs
+ #:define-prefix 'define-cargo-inputs))
+
+(define* (extract-cargo-inputs file #:key exclude)
+ "Search in FILE for a top-level definition of Cargo inputs. If found,
+return its entries excluding EXCLUDE, or an empty list otherwise."
+ (call-with-input-file file
+ (lambda (port)
+ (do ((syntax (read-syntax port)
+ (read-syntax port)))
+ ((match (syntax->datum syntax)
+ (('define-cargo-inputs 'lookup-cargo-inputs _ ...) #t)
+ ((? eof-object?) #t)
+ (_ #f))
+ (or (and (not (eof-object? syntax))
+ (match (syntax->datum syntax)
+ (('define-cargo-inputs 'lookup-cargo-inputs inputs ...)
+ (remove (lambda (cargo-input-entry)
+ (eq? exclude (first cargo-input-entry)))
+ inputs))))
+ '()))))))
+
;;;
;;; Updater
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 723cbb3665..a251cd9538 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -25,11 +25,13 @@
(define-module (guix scripts import crate)
#:use-module (guix ui)
#:use-module (guix utils)
+ #:use-module (guix read-print)
#:use-module (guix scripts)
#:use-module (guix import crate)
#:use-module (guix scripts import)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-26)
#:use-module (srfi srfi-37)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
@@ -60,6 +62,9 @@ (define (show-help)
sufficient package exists for it"))
(newline)
(display (G_ "
+ -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
+ (newline)
+ (display (G_ "
-h, --help display this help and exit"))
(display (G_ "
-V, --version display version information and exit"))
@@ -87,6 +92,9 @@ (define %options
(option '("mark-missing") #f #f
(lambda (opt name arg result)
(alist-cons 'mark-missing #t result)))
+ (option '(#\f "lockfile") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'lockfile arg result)))
%standard-import-options))
@@ -101,6 +109,8 @@ (define (guix-import-crate . args)
#:build-options? #f))
(let* ((opts (parse-options))
+ (lockfile (assoc-ref opts 'lockfile))
+ (file-to-insert (assoc-ref opts 'file-to-insert))
(args (filter-map (match-lambda
(('argument . value)
value)
@@ -111,16 +121,43 @@ (define (guix-import-crate . args)
(define-values (name version)
(package-name->name+version spec))
- (match (if (assoc-ref opts 'recursive)
- (crate-recursive-import
- name #:version version
- #:recursive-dev-dependencies?
- (assoc-ref opts 'recursive-dev-dependencies)
- #:allow-yanked? (assoc-ref opts 'allow-yanked))
- (crate->guix-package
- name #:version version #:include-dev-deps? #t
- #:allow-yanked? (assoc-ref opts 'allow-yanked)
- #:mark-missing? (assoc-ref opts 'mark-missing)))
+ (match (cond
+ ((and=> lockfile
+ (lambda (file)
+ (or (file-exists? file)
+ (leave (G_ "file '~a' does not exist~%") file))))
+ (let-values (((source-expressions cargo-inputs-entry)
+ (cargo-lock->expressions lockfile name)))
+ (when file-to-insert
+ (let* ((term (first cargo-inputs-entry))
+ (cargo-inputs
+ `(define-cargo-inputs lookup-cargo-inputs
+ ,@(sort
+ (cons cargo-inputs-entry
+ (extract-cargo-inputs
+ file-to-insert #:exclude term))
+ (lambda (a b)
+ (string< (symbol->string (first a))
+ (symbol->string (first b)))))))
+ (_
+ (and=> (find-cargo-inputs-location file-to-insert)
+ delete-expression))
+ (port (open-file file-to-insert "a")))
+ (pretty-print-with-comments port cargo-inputs)
+ (newline port)
+ (close-port port)))
+ source-expressions))
+ ((assoc-ref opts 'recursive)
+ (crate-recursive-import
+ name #:version version
+ #:recursive-dev-dependencies?
+ (assoc-ref opts 'recursive-dev-dependencies)
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)))
+ (else
+ (crate->guix-package
+ name #:version version #:include-dev-deps? #t
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)
+ #:mark-missing? (assoc-ref opts 'mark-missing))))
((or #f '())
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
--
2.49.0
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 06 May 2025 12:57:01 GMT) Full text and rfc822 format available.Message #338 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Josselin Poiret <dev <at> jpoiret.xyz>, Steve George <steve <at> futurile.net>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Efraim Flashner <efraim <at> flashner.co.il>, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH v4 rust-team 00/22] Cargo.lock importer and build system changes. Date: Tue, 06 May 2025 20:55:00 +0800
Hilton Chain <hako <at> ultrarare.space> writes: > Main changes since v3: > * ‘define-cargo-inputs’ macro suggested by Ludovic. > * ‘cargo-inputs-from-lockfile’ suggested by Zheng Junjie, which can be used in > the guix.scm of a Rust project. > * Rephrase docstring. > > Some changes have been made on rust-team since v3, this revision also sends > all of them out for review. > > I'll change the terms Rust program & dependency to application & library, btw. > > Since there're too many patches in this issue, I think it would be better to > open a new one for documentation. Updated and reworded the documentation, submitted as #78280. I'll prepare deprecation and the blog post next.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sat, 31 May 2025 16:28:03 GMT) Full text and rfc822 format available.Message #341 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Josselin Poiret <dev <at> jpoiret.xyz>, Steve George <steve <at> futurile.net>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Efraim Flashner <efraim <at> flashner.co.il>, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH v4 rust-team 00/22] Cargo.lock importer and build system changes. Date: Sun, 01 Jun 2025 00:23:28 +0800
Hilton Chain <hako <at> ultrarare.space> writes: > Hilton Chain <hako <at> ultrarare.space> writes: > >> Main changes since v3: >> * ‘define-cargo-inputs’ macro suggested by Ludovic. >> * ‘cargo-inputs-from-lockfile’ suggested by Zheng Junjie, which can be used in >> the guix.scm of a Rust project. >> * Rephrase docstring. >> >> Some changes have been made on rust-team since v3, this revision also sends >> all of them out for review. >> >> I'll change the terms Rust program & dependency to application & library, btw. >> >> Since there're too many patches in this issue, I think it would be better to >> open a new one for documentation. > > Updated and reworded the documentation, submitted as #78280. > > I'll prepare deprecation and the blog post next. Blog post sumbitted as: https://codeberg.org/guix/artwork/pulls/1
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.