Package: guix-patches;
Reported by: Troy Figiel <troy <at> troyfigiel.com>
Date: Thu, 25 Jan 2024 21:53:02 UTC
Severity: normal
Tags: patch
Done: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 68721 in the body.
You can then email your comments to 68721 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
guix-patches <at> gnu.org:bug#68721; Package guix-patches.
(Thu, 25 Jan 2024 21:53:02 GMT) Full text and rfc822 format available.Troy Figiel <troy <at> troyfigiel.com>:guix-patches <at> gnu.org.
(Thu, 25 Jan 2024 21:53:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Troy Figiel <troy <at> troyfigiel.com> To: guix-patches <at> gnu.org Subject: [PATCH 0/2] gnu: Add python-fastparquet. Date: Thu, 25 Jan 2024 22:41:02 +0100
This patch series adds python-fastparquet and is a bit unconvential, because it depends on changes currently in the rust-team branch (through python-cramjam, see #68665). Before we can merge these changes, the rust-team branch needs to be merged into master. The base-commit is a commit on the rust-team branch, so be aware that you might need to rebuild some other packages like python-pandas locally. I decided to send these patches to guix-patches anyway, so they are at least out in the open. Troy Figiel (2): gnu: Add python-cramjam. gnu: Add python-fastparquet. gnu/packages/databases.scm | 67 ++++++++++++++++++++++++ gnu/packages/python-compression.scm | 80 ++++++++++++++++++++++++++++- 2 files changed, 146 insertions(+), 1 deletion(-) base-commit: 3228c8176783db5627fd79d87652c67bfeb86512 -- 2.42.0
guix-patches <at> gnu.org:bug#68721; Package guix-patches.
(Thu, 25 Jan 2024 21:55:01 GMT) Full text and rfc822 format available.Message #8 received at 68721 <at> debbugs.gnu.org (full text, mbox):
From: Troy Figiel <troy <at> troyfigiel.com> To: 68721 <at> debbugs.gnu.org Subject: [PATCH 1/2] gnu: Add python-cramjam. Date: Thu, 25 Jan 2024 21:39:50 +0100
* gnu/packages/python-compression.scm (python-cramjam): New variable.
---
gnu/packages/python-compression.scm | 80 ++++++++++++++++++++++++++++-
1 file changed, 79 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/python-compression.scm b/gnu/packages/python-compression.scm
index 6f249dd3af..691fbd3065 100644
--- a/gnu/packages/python-compression.scm
+++ b/gnu/packages/python-compression.scm
@@ -7,7 +7,7 @@
;;; Copyright © 2020 Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
;;; Copyright © 2020, 2022, 2023 Marius Bakke <marius <at> gnu.org>
;;; Copyright © 2021 Brendan Tildesley <mail <at> brendan.scot>
-;;; Copyright © 2023 Troy Figiel <troy <at> troyfigiel.com>
+;;; Copyright © 2023, 2024 Troy Figiel <troy <at> troyfigiel.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -30,12 +30,14 @@ (define-module (gnu packages python-compression)
#:use-module (guix packages)
#:use-module (guix download)
#:use-module (guix gexp)
+ #:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix build-system python)
#:use-module (guix build-system pyproject)
#:use-module (gnu packages)
#:use-module (gnu packages libffi)
#:use-module (gnu packages compression)
+ #:use-module (gnu packages crates-io)
#:use-module (gnu packages check)
#:use-module (gnu packages maths)
#:use-module (gnu packages pkg-config)
@@ -44,6 +46,7 @@ (define-module (gnu packages python-compression)
#:use-module (gnu packages python-check)
#:use-module (gnu packages python-crypto)
#:use-module (gnu packages python-xyz)
+ #:use-module (gnu packages rust-apps)
#:use-module (gnu packages sphinx))
(define-public python-multivolumefile
@@ -73,6 +76,81 @@ (define-public python-multivolumefile
were a single file.")
(license license:lgpl2.1+)))
+(define-public python-cramjam
+ (package
+ (name "python-cramjam")
+ (version "2.7.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "cramjam" version))
+ (sha256
+ (base32 "1b69qlr0q7q3spa7zy55xc1dr5pjgsdavxx8ijhv2j60xqjbg7sp"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list
+ #:imported-modules `(,@%cargo-build-system-modules
+ ,@%pyproject-build-system-modules)
+ #:modules '((guix build cargo-build-system)
+ ((guix build pyproject-build-system)
+ #:prefix py:)
+ (guix build utils))
+ #:phases #~(modify-phases %standard-phases
+ ;; We use Maturin to build the project.
+ (replace 'build
+ (assoc-ref py:%standard-phases
+ 'build))
+ ;; Before being able to run Python tests, we need to
+ ;; install the module and add it to PYTHONPATH.
+ (delete 'install)
+ (add-after 'build 'install
+ (assoc-ref py:%standard-phases
+ 'install))
+ (add-after 'install 'add-install-to-pythonpath
+ (assoc-ref py:%standard-phases
+ 'add-install-to-pythonpath))
+ ;; Finally run the tests. Only Python tests are provided.
+ (replace 'check
+ (lambda* (#:key tests? inputs outputs #:allow-other-keys)
+ (when tests?
+ ;; Without the CI variable, tests are run in "local"
+ ;; mode, which sets a deadline for hypothesis. For a
+ ;; deterministic build, we need to set CI.
+ (setenv "CI" "1")
+ (invoke "pytest" "-vv" "tests")))))
+ #:cargo-inputs `(("rust-brotli" ,rust-brotli-3)
+ ("rust-bzip2" ,rust-bzip2-0.4)
+ ("rust-flate2" ,rust-flate2-1)
+ ("rust-lz4" ,rust-lz4-1)
+ ("rust-pyo3" ,rust-pyo3-0.18)
+ ("rust-snap" ,rust-snap-1)
+ ("rust-zstd" ,rust-zstd-0.11))
+ #:install-source? #f))
+ (native-inputs (list maturin
+ python-pytest
+ python-pytest-xdist
+ python-numpy
+ python-hypothesis
+ python-wrapper))
+ (home-page "https://github.com/milesgranger/cramjam")
+ (synopsis "Python bindings to compression algorithms in Rust")
+ (description
+ "This package provides thin Python bindings to compression and
+decomporession algorithms implemented in Rust. This allows for using
+algorithms such as Snappy without additional system dependencies. The
+following algorithms are available:
+
+@itemize
+@item Snappy
+@item Brotli
+@item Bzip2
+@item LZ4
+@item Gzip
+@item Deflate
+@item Zstd
+@end itemize")
+ (license license:expat)))
+
(define-public python-pybcj
(package
(name "python-pybcj")
--
2.42.0
guix-patches <at> gnu.org:bug#68721; Package guix-patches.
(Thu, 25 Jan 2024 21:56:02 GMT) Full text and rfc822 format available.Message #11 received at 68721 <at> debbugs.gnu.org (full text, mbox):
From: Troy Figiel <troy <at> troyfigiel.com> To: 68721 <at> debbugs.gnu.org Subject: [PATCH 2/2] gnu: Add python-fastparquet. Date: Thu, 25 Jan 2024 21:45:20 +0100
* gnu/packages/databases.scm (python-fastparquet): New variable.
---
gnu/packages/databases.scm | 67 ++++++++++++++++++++++++++++++++++++++
1 file changed, 67 insertions(+)
diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index b56767d311..66f7f7d951 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -62,6 +62,7 @@
;;; Copyright © 2023 Felix Gruber <felgru <at> posteo.ne
;;; Copyright © 2023 Munyoki Kilyungi <me <at> bonfacemunyoki.com>
;;; Copyright © 2023 Giacomo Leidi <goodoldpaul <at> autistici.org>
+;;; Copyright © 2024 Troy Figiel <troy <at> troyfigiel.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -142,6 +143,7 @@ (define-module (gnu packages databases)
#:use-module (gnu packages python)
#:use-module (gnu packages python-build)
#:use-module (gnu packages python-check)
+ #:use-module (gnu packages python-compression)
#:use-module (gnu packages python-crypto)
#:use-module (gnu packages python-science)
#:use-module (gnu packages python-web)
@@ -4896,6 +4898,71 @@ (define-public python-pyarrow-0.16
other traditional Python scientific computing packages.")
(license license:asl2.0)))
+(define-public python-fastparquet
+ (package
+ (name "python-fastparquet")
+ (version "2023.10.1")
+ (source
+ (origin
+ ;; Fastparquet uses setuptools-scm to find the current version. This
+ ;; only works when we use the PyPI tarball, which does not contain
+ ;; tests. Instead, we use the git-fetch method and add the version back
+ ;; ourselves.
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/dask/fastparquet")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "0msc2n4sjbq3h5pq6l94rfx27v0aqrk5cxbpg3yssr74gwx26h4r"))
+ (modules '((guix build utils)))
+ (snippet '(substitute* "setup.py"
+ ;; Remove dependencies on git and setuptools-scm.
+ (("^.*\"git\", \"status\".*$")
+ "")
+ (("'setuptools-scm>1.5.4',")
+ "")
+ ;; Guix is only compatible with a single version of numpy
+ ;; at a time. We can safely remove this dependency.
+ (("'oldest-supported-numpy'")
+ "")))))
+ (build-system pyproject-build-system)
+ (arguments
+ (list
+ #:phases #~(modify-phases %standard-phases
+ ;; Make sure to add back the missing version information
+ ;; the build phase.
+ (add-after 'unpack 'set-version
+ (lambda _
+ (call-with-output-file "fastparquet/_version.py"
+ (lambda (port)
+ (format port "__version__ = ~a"
+ (string-append "\""
+ #$version "\""))))))
+ ;; Cython extensions need to be built for the check phase.
+ (add-before 'check 'build-cython-extensions
+ (lambda _
+ (invoke "python" "setup.py" "build_ext" "--inplace"))))))
+ (propagated-inputs (list python-cramjam python-fsspec python-numpy
+ python-packaging python-pandas))
+ (native-inputs (list python-cython python-pytest-runner))
+ (home-page "https://github.com/dask/fastparquet")
+ (synopsis "Python implementation of the Parquet file format")
+ (description
+ "@code{fastparquet} is a Python implementation of the Parquet file
+format. @code{fastparquet} is used implicitly by @code{dask}, @code{pandas}
+and @code{intake-parquet}. It supports the following compression algorithms:
+
+@itemize
+@item Gzip
+@item Snappy
+@item Brotli
+@item LZ4
+@item Zstd
+@item LZO (optionally)
+@end itemize")
+ (license license:asl2.0)))
+
(define-public python-crate
(package
(name "python-crate")
--
2.42.0
guix-patches <at> gnu.org:bug#68721; Package guix-patches.
(Fri, 01 Mar 2024 08:02:01 GMT) Full text and rfc822 format available.Message #14 received at 68721 <at> debbugs.gnu.org (full text, mbox):
From: Sharlatan Hellseher <sharlatanus <at> gmail.com> To: 68721 <at> debbugs.gnu.org Subject: [PATCH 0/2] gnu: Add python-fastparquet. Date: Fri, 1 Mar 2024 08:00:04 +0000
[Message part 1 (text/plain, inline)]
Hi, As rust-team is merged, I prioritising this patch series to be reviewed. Thanks, Oleg
[Message part 2 (text/html, inline)]
guix-patches <at> gnu.org:bug#68721; Package guix-patches.
(Sat, 02 Mar 2024 18:49:02 GMT) Full text and rfc822 format available.Message #17 received at 68721 <at> debbugs.gnu.org (full text, mbox):
From: Troy Figiel <troy <at> troyfigiel.com> To: 68721 <at> debbugs.gnu.org Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com> Subject: [PATCH v2 0/2] gnu: Add python-fastparquet. Date: Sat, 2 Mar 2024 19:41:14 +0100
Hi Oleg, I recently learned snippets should be used more conservatively. It makes sense from the perspective of hacking on the source code locally. Therefore, I moved the python-fastparquet snippet to a phase. I also updated python-fastparquet to its most recent version. Best wishes, Troy Troy Figiel (2): gnu: Add python-cramjam. gnu: Add python-fastparquet. gnu/packages/databases.scm | 68 ++++++++++++++++++++++++ gnu/packages/python-compression.scm | 80 ++++++++++++++++++++++++++++- 2 files changed, 147 insertions(+), 1 deletion(-) base-commit: 6f5ea7ac1acb3d1c53baf7620cca66cc87fe5a73 -- 2.42.0
guix-patches <at> gnu.org:bug#68721; Package guix-patches.
(Sat, 02 Mar 2024 18:50:02 GMT) Full text and rfc822 format available.Message #20 received at 68721 <at> debbugs.gnu.org (full text, mbox):
From: Troy Figiel <troy <at> troyfigiel.com> To: 68721 <at> debbugs.gnu.org Subject: [PATCH 1/2] gnu: Add python-cramjam. Date: Sat, 2 Mar 2024 19:40:21 +0100
* gnu/packages/python-compression.scm (python-cramjam): New variable.
---
gnu/packages/python-compression.scm | 80 ++++++++++++++++++++++++++++-
1 file changed, 79 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/python-compression.scm b/gnu/packages/python-compression.scm
index 6f249dd3af..691fbd3065 100644
--- a/gnu/packages/python-compression.scm
+++ b/gnu/packages/python-compression.scm
@@ -7,7 +7,7 @@
;;; Copyright © 2020 Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
;;; Copyright © 2020, 2022, 2023 Marius Bakke <marius <at> gnu.org>
;;; Copyright © 2021 Brendan Tildesley <mail <at> brendan.scot>
-;;; Copyright © 2023 Troy Figiel <troy <at> troyfigiel.com>
+;;; Copyright © 2023, 2024 Troy Figiel <troy <at> troyfigiel.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -30,12 +30,14 @@ (define-module (gnu packages python-compression)
#:use-module (guix packages)
#:use-module (guix download)
#:use-module (guix gexp)
+ #:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix build-system python)
#:use-module (guix build-system pyproject)
#:use-module (gnu packages)
#:use-module (gnu packages libffi)
#:use-module (gnu packages compression)
+ #:use-module (gnu packages crates-io)
#:use-module (gnu packages check)
#:use-module (gnu packages maths)
#:use-module (gnu packages pkg-config)
@@ -44,6 +46,7 @@ (define-module (gnu packages python-compression)
#:use-module (gnu packages python-check)
#:use-module (gnu packages python-crypto)
#:use-module (gnu packages python-xyz)
+ #:use-module (gnu packages rust-apps)
#:use-module (gnu packages sphinx))
(define-public python-multivolumefile
@@ -73,6 +76,81 @@ (define-public python-multivolumefile
were a single file.")
(license license:lgpl2.1+)))
+(define-public python-cramjam
+ (package
+ (name "python-cramjam")
+ (version "2.7.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "cramjam" version))
+ (sha256
+ (base32 "1b69qlr0q7q3spa7zy55xc1dr5pjgsdavxx8ijhv2j60xqjbg7sp"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list
+ #:imported-modules `(,@%cargo-build-system-modules
+ ,@%pyproject-build-system-modules)
+ #:modules '((guix build cargo-build-system)
+ ((guix build pyproject-build-system)
+ #:prefix py:)
+ (guix build utils))
+ #:phases #~(modify-phases %standard-phases
+ ;; We use Maturin to build the project.
+ (replace 'build
+ (assoc-ref py:%standard-phases
+ 'build))
+ ;; Before being able to run Python tests, we need to
+ ;; install the module and add it to PYTHONPATH.
+ (delete 'install)
+ (add-after 'build 'install
+ (assoc-ref py:%standard-phases
+ 'install))
+ (add-after 'install 'add-install-to-pythonpath
+ (assoc-ref py:%standard-phases
+ 'add-install-to-pythonpath))
+ ;; Finally run the tests. Only Python tests are provided.
+ (replace 'check
+ (lambda* (#:key tests? inputs outputs #:allow-other-keys)
+ (when tests?
+ ;; Without the CI variable, tests are run in "local"
+ ;; mode, which sets a deadline for hypothesis. For a
+ ;; deterministic build, we need to set CI.
+ (setenv "CI" "1")
+ (invoke "pytest" "-vv" "tests")))))
+ #:cargo-inputs `(("rust-brotli" ,rust-brotli-3)
+ ("rust-bzip2" ,rust-bzip2-0.4)
+ ("rust-flate2" ,rust-flate2-1)
+ ("rust-lz4" ,rust-lz4-1)
+ ("rust-pyo3" ,rust-pyo3-0.18)
+ ("rust-snap" ,rust-snap-1)
+ ("rust-zstd" ,rust-zstd-0.11))
+ #:install-source? #f))
+ (native-inputs (list maturin
+ python-pytest
+ python-pytest-xdist
+ python-numpy
+ python-hypothesis
+ python-wrapper))
+ (home-page "https://github.com/milesgranger/cramjam")
+ (synopsis "Python bindings to compression algorithms in Rust")
+ (description
+ "This package provides thin Python bindings to compression and
+decomporession algorithms implemented in Rust. This allows for using
+algorithms such as Snappy without additional system dependencies. The
+following algorithms are available:
+
+@itemize
+@item Snappy
+@item Brotli
+@item Bzip2
+@item LZ4
+@item Gzip
+@item Deflate
+@item Zstd
+@end itemize")
+ (license license:expat)))
+
(define-public python-pybcj
(package
(name "python-pybcj")
--
2.42.0
guix-patches <at> gnu.org:bug#68721; Package guix-patches.
(Sat, 02 Mar 2024 18:54:02 GMT) Full text and rfc822 format available.Message #23 received at 68721 <at> debbugs.gnu.org (full text, mbox):
From: Troy Figiel <troy <at> troyfigiel.com> To: 68721 <at> debbugs.gnu.org Subject: [PATCH 2/2] gnu: Add python-fastparquet. Date: Sat, 2 Mar 2024 19:40:31 +0100
* gnu/packages/databases.scm (python-fastparquet): New variable.
---
gnu/packages/databases.scm | 68 ++++++++++++++++++++++++++++++++++++++
1 file changed, 68 insertions(+)
diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index db4fe0b447..5ec0234ef1 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -62,6 +62,7 @@
;;; Copyright © 2023 Felix Gruber <felgru <at> posteo.ne
;;; Copyright © 2023 Munyoki Kilyungi <me <at> bonfacemunyoki.com>
;;; Copyright © 2023 Giacomo Leidi <goodoldpaul <at> autistici.org>
+;;; Copyright © 2024 Troy Figiel <troy <at> troyfigiel.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -143,6 +144,7 @@ (define-module (gnu packages databases)
#:use-module (gnu packages python)
#:use-module (gnu packages python-build)
#:use-module (gnu packages python-check)
+ #:use-module (gnu packages python-compression)
#:use-module (gnu packages python-crypto)
#:use-module (gnu packages python-science)
#:use-module (gnu packages python-web)
@@ -4984,6 +4986,72 @@ (define-public python-pyarrow-0.16
other traditional Python scientific computing packages.")
(license license:asl2.0)))
+(define-public python-fastparquet
+ (package
+ (name "python-fastparquet")
+ (version "2024.2.0")
+ (source
+ (origin
+ ;; Fastparquet uses setuptools-scm to find the current version. This
+ ;; only works when we use the PyPI tarball, which does not contain
+ ;; tests. Instead, we use the git-fetch method and add the version back
+ ;; ourselves.
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/dask/fastparquet")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "0f32dj1xvd11l0siznqd33dpjlhg9siylcjcfkcdlqfcy45jfj3v"))))
+ (build-system pyproject-build-system)
+ (arguments
+ (list
+ #:phases #~(modify-phases %standard-phases
+ ;; Make sure to add back the missing version information
+ ;; the build phase.
+ (add-after 'unpack 'set-version
+ (lambda _
+ (call-with-output-file "fastparquet/_version.py"
+ (lambda (port)
+ (format port "__version__ = ~a"
+ (string-append "\""
+ #$version "\""))))))
+ (add-after 'unpack 'relax-dependencies
+ (lambda _
+ (substitute* "setup.py"
+ ;; Remove dependencies on git and setuptools-scm.
+ (("^.*\"git\", \"status\".*$")
+ "")
+ (("'setuptools-scm>1.5.4',")
+ "")
+ ;; Guix is only compatible with a single version of numpy
+ ;; at a time. We can safely remove this dependency.
+ (("'oldest-supported-numpy'")
+ ""))))
+ ;; Cython extensions need to be built for the check phase.
+ (add-before 'check 'build-cython-extensions
+ (lambda _
+ (invoke "python" "setup.py" "build_ext" "--inplace"))))))
+ (propagated-inputs (list python-cramjam python-fsspec python-numpy
+ python-packaging python-pandas))
+ (native-inputs (list python-cython python-pytest-runner))
+ (home-page "https://github.com/dask/fastparquet")
+ (synopsis "Python implementation of the Parquet file format")
+ (description
+ "@code{fastparquet} is a Python implementation of the Parquet file
+format. @code{fastparquet} is used implicitly by @code{dask}, @code{pandas}
+and @code{intake-parquet}. It supports the following compression algorithms:
+
+@itemize
+@item Gzip
+@item Snappy
+@item Brotli
+@item LZ4
+@item Zstd
+@item LZO (optionally)
+@end itemize")
+ (license license:asl2.0)))
+
(define-public python-crate
(package
(name "python-crate")
--
2.42.0
Sharlatan Hellseher <sharlatanus <at> gmail.com>:Troy Figiel <troy <at> troyfigiel.com>:Message #28 received at 68721-done <at> debbugs.gnu.org (full text, mbox):
From: Sharlatan Hellseher <sharlatanus <at> gmail.com> To: 68721-done <at> debbugs.gnu.org Subject: [PATCH 0/2] gnu: Add python-fastparquet. Date: Sun, 03 Mar 2024 08:16:09 +0000
[Message part 1 (text/plain, inline)]
Hi, Applied modifications [5/5] - [X] Rename phase 'relax-dependencies -> 'relax-requirements for consistency with other python packages - [X] Use 'pretend-version to set module version from git checkout - [X] Adjust indentation style - [X] Add optional python-lzo - [X] Parallize tests Pushed as bdecacafec..f00f56514d to master. -- Oleg
[signature.asc (application/pgp-signature, inline)]
Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Sun, 31 Mar 2024 11:24:06 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.