GNU bug report logs - #75801
[PATCH python-team 0/2] NumPy 1+ and 2+.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 75801 in the body.
You can then email your comments to 75801 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: guix-patches <at> gnu.org
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Subject: [PATCH python-team 0/2] NumPy 1+ and 2+.
Date: Fri, 24 Jan 2025 08:03:58 +0000

Hi Guix,

After following Ludovic advise in <https://issues.guix.gnu.org/73439#45>,
NumPy could be upgraded to 1.26.2 which is the latest for 1+, and I've packed
a draft for NumPy 2+.  All tests fail on 2+ which I suspect, due to
vendored-meson.

Thanks,
Oleg

Sharlatan Hellseher (2):
  gnu: python-numpy: Update to 1.26.2.
  gnu: Add python-numpy-2.

 gnu/packages/python-xyz.scm | 100 +++++++++++++++++++++++++++++++++---
 1 file changed, 93 insertions(+), 7 deletions(-)


base-commit: 19e7200286989a7afc90adecb415cf09093cc4dd
prerequisite-patch-id: 9a787f62c93874a8179725e88ed72baa85097a39
prerequisite-patch-id: eaecc55fe1e4de4db84315cebbc117eac7474917
prerequisite-patch-id: 74caf7ddc411a581b6c81e7257c3d2adaeb0500d
prerequisite-patch-id: 16adea1e84245bb59d5678762a4b22e4471c0936
prerequisite-patch-id: 8ea73dcc8758afe588280b9aabda55ab7808e60f
prerequisite-patch-id: 6bf1595fa5d8d63ed61caf8d8b3f1da1c081b01d
prerequisite-patch-id: 25926ed4b9f3e7d574d9bd5baeea056a5abcc1e1
prerequisite-patch-id: cd742dfc2154de56d4c0849deb7926fb3749778f
prerequisite-patch-id: 88b414ed6a3dfb894636da048909994e6f40f999
prerequisite-patch-id: aff81069395caf3ad600a4c3985a6051797c47ab
-- 
2.47.1

Message #8 received at 75801 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 75801 <at> debbugs.gnu.org
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Subject: [PATCH python-team 1/2] gnu: python-numpy: Update to 1.26.2.
Date: Fri, 24 Jan 2025 08:05:33 +0000

* gnu/packages/python-xyz.scm (python-numpy): Update to 1.26.2.
[arguments] <phases>: Add 'hide-gfortran.
[native-inputs]: Remove python-cython.

Change-Id: I1f02d45dd5c5bc7ad248dbbdc63a8e6a7a21ca9f
---
 gnu/packages/python-xyz.scm | 26 +++++++++++++++++++-------
 1 file changed, 19 insertions(+), 7 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 4b311cc9b1..88e1796143 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -9224,11 +9224,12 @@ (define-public python-cython-3
 (define-public python-numpy
   (package
     (name "python-numpy")
-    ;; XXX: Any other versions up to 1.26.4 failed to build with error similar
-    ;; to: 'fenv_t' has not been declared in '::' 58 | using ::fenv_t;
-    ;; See <https://github.com/numpy/numpy/issues/21075#issuecomment-1047976197>,
-    ;; <https://github.com/numpy/numpy/issues/24318>.
-    (version "1.24.4")
+    ;; XXX: Shipped with vendored-meson which is needed for SIMD and
+    ;; BLAS/LAPACK features that are not yet available in upstream Meson.
+    ;; See <https://raw.githubusercontent.com/numpy/numpy/
+    ;; 2f3549c9d7c5048621568e431c86bc7530742723/doc/source/building/
+    ;; understanding_meson.rst>
+    (version "1.26.2")
     (source
      (origin
        (method url-fetch)
@@ -9237,7 +9238,7 @@ (define-public python-numpy
              version "/numpy-" version ".tar.gz"))
        (sha256
         (base32
-         "0qwldmkq5bns561ppkz7psphc4jqfj5j1x4dhq0i8r4qwjjf7xc0"))))
+         "1snknqb4hmv6b720nsaz21g7h6z1ikdvnsqyy5vmgavnfr23hmzn"))))
     (build-system pyproject-build-system)
     (arguments
      (list
@@ -9246,6 +9247,18 @@ (define-public python-numpy
                   (ice-9 format))
       #:phases
       #~(modify-phases %standard-phases
+          ;; XXX: istl/test/matrixtest.cc includes <fenv.h> and fails to find
+          ;; the stdlib types when the gfortran header is used.  Remove
+          ;; gfortran from CPLUS_INCLUDE_PATH as a workaround.
+          ;; Taken from <https://issues.guix.gnu.org/73439#45>.
+          (add-after 'set-paths 'hide-gfortran
+            (lambda* (#:key inputs #:allow-other-keys)
+              (let ((gfortran (assoc-ref inputs "gfortran")))
+                (setenv "CPLUS_INCLUDE_PATH"
+                        (string-join
+                         (delete (string-append gfortran "/include/c++")
+                                 (string-split (getenv "CPLUS_INCLUDE_PATH") #\:))
+                         ":")))))
           (add-before 'build 'parallelize-build
             (lambda _
               (setenv "NPY_NUM_BUILD_JOBS"
@@ -9328,7 +9341,6 @@ (define-public python-numpy
      (list gfortran
            meson-python
            pkg-config
-           python-cython ;; overwrite Cython from meson-python
            python-hypothesis
            python-mypy
            python-pytest
-- 
2.47.1

Message #11 received at 75801 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 75801 <at> debbugs.gnu.org
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Subject: [PATCH python-team 2/2] gnu: Add python-numpy-2.
Date: Fri, 24 Jan 2025 08:05:34 +0000

* gnu/packages/python-xyz.scm (python-numpy-2): New variable.

Change-Id: I07e7aac8c1bf7e686b1c543a11f9ecbf682d6ad5
---
 gnu/packages/python-xyz.scm | 74 +++++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 88e1796143..d6bffb6f77 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -9360,6 +9360,80 @@ (define-public python-numpy
      '((upstream-name . "numpy")))
     (license license:bsd-3)))
 
+(define-public python-numpy-2
+  (package
+    (inherit python-numpy)
+    (name "python-numpy")
+    (version "2.2.2")
+    ;; XXX: Shipped with vendored-meson which is needed for SIMD and
+    ;; BLAS/LAPACK features that are not yet available in upstream Meson.
+    ;; See <https://raw.githubusercontent.com/numpy/numpy/
+    ;; 2f3549c9d7c5048621568e431c86bc7530742723/doc/source/building/
+    ;; understanding_meson.rst>
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+             "https://github.com/numpy/numpy/releases/download/v"
+             version "/numpy-" version ".tar.gz"))
+       (sha256
+        (base32
+         "13sdvwiqn85vw1dn1k1nd5ihadv82zhqm615imrqgmil33v0csgd"))))
+    (arguments
+     (list
+      ;; TODO: Tests fail on setup, there is some issue with vendored-meson.
+      #:tests? #f 
+      #:modules '((guix build utils)
+                  (guix build pyproject-build-system)
+                  (ice-9 format))
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'fix-executable-paths
+            (lambda _
+              (substitute* "numpy/distutils/exec_command.py"
+                (("'/bin/sh'")
+                 (format #f "~s" (which "bash"))))))
+          (add-before 'build 'parallelize-build
+            (lambda _
+              (setenv "OMP_NUM_THREAD"
+                      (number->string (parallel-job-count)))
+              (setenv "NPY_NUM_BUILD_JOBS"
+                      (number->string (parallel-job-count)))))
+          ;; XXX: istl/test/matrixtest.cc includes <fenv.h> and fails to find
+          ;; the stdlib types when the gfortran header is used.  Remove
+          ;; gfortran from CPLUS_INCLUDE_PATH as a workaround.
+          ;; Taken from <https://issues.guix.gnu.org/73439#45>.
+          (add-after 'set-paths 'hide-gfortran
+            (lambda* (#:key inputs #:allow-other-keys)
+              (let ((gfortran (assoc-ref inputs "gfortran")))
+                (setenv "CPLUS_INCLUDE_PATH"
+                        (string-join
+                         (delete (string-append gfortran "/include/c++")
+                                 (string-split (getenv "CPLUS_INCLUDE_PATH") #\:))
+                         ":")))))
+          (add-before 'build 'configure-blas
+            (lambda* (#:key inputs #:allow-other-keys)
+              (call-with-output-file "site.cfg"
+                (lambda (port)
+                  (format port
+                          "[openblas]
+libraries = openblas
+library_dirs = ~a/lib
+include_dirs = ~:*~a/include~%" #$(this-package-input "openblas")))))))))
+    (native-inputs
+     (list gfortran
+           meson-python
+           ninja
+           pkg-config
+           python-hypothesis
+           python-mypy
+           python-pytest
+           python-pytest-xdist
+           python-setuptools
+           python-setuptools
+           python-typing-extensions
+           python-wheel))))
+
 (define-public python-numpy-documentation
   (package
     (inherit python-numpy)
-- 
2.47.1

Message #14 received at 75801 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 75801 <at> debbugs.gnu.org
Cc: Ricardo Wurmus <rekado <at> elephly.net>
Subject: [PATCH python-team 0/2] NumPy 1+ and 2+.
Date: Mon, 27 Jan 2025 22:08:51 +0000

[Message part 1 (text/plain, inline)]

Hi Ricardo,

These two patches bring the latest 1+ version of NumPy and 2+ but
without tests which I could not fix, maybe you have stripes on your belt
to enable them ;-)

--
Thanks,
Oleg

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 75801 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Cc: Tanguy Le Carrour <tanguy <at> bioneland.org>,
 Munyoki Kilyungi <me <at> bonfacemunyoki.com>, Lars-Dominik Braun <lars <at> 6xq.net>,
 jgart <jgart <at> dismail.de>, 75801 <at> debbugs.gnu.org, Marius Bakke <marius <at> gnu.org>
Subject: Re: [bug#75801] [PATCH python-team 2/2] gnu: Add python-numpy-2.
Date: Tue, 04 Feb 2025 14:03:42 +0100

Hello,

Sharlatan Hellseher <sharlatanus <at> gmail.com> skribis:

> * gnu/packages/python-xyz.scm (python-numpy-2): New variable.
>
> Change-Id: I07e7aac8c1bf7e686b1c543a11f9ecbf682d6ad5

I’m no Python expert so I can only comment on the surface of things and
encourage you to keep going, because I hear people would love to have
newer versions of NumPy and this looks like great work.

> +    (version "2.2.2")
> +    ;; XXX: Shipped with vendored-meson which is needed for SIMD and
> +    ;; BLAS/LAPACK features that are not yet available in upstream Meson.
> +    ;; See <https://raw.githubusercontent.com/numpy/numpy/
> +    ;; 2f3549c9d7c5048621568e431c86bc7530742723/doc/source/building/
> +    ;; understanding_meson.rst>

(Let me suggest not breaking URLs on multiple lines. :-))

The file also says “As of Dec'23 […]”.  Do you think current versions of
Meson would work?  That would be an improvement.

Also, not sure what they mean by SIMD, but in Guix we build generic code
and set the ‘tunable?’ property if there’s anything to be gained.

As for BLAS/LAPACK, I suspect it’s just about detecting libraries, so
hopefully not something big in Meson.

> +          ;; XXX: istl/test/matrixtest.cc includes <fenv.h> and fails to find
                     ^
Too much copy/paste.  :-)

Thanks for working on this!

Ludo’.

Message #20 received at 75801 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Cc: Tanguy Le Carrour <tanguy <at> bioneland.org>,
 Munyoki Kilyungi <me <at> bonfacemunyoki.com>, Lars-Dominik Braun <lars <at> 6xq.net>,
 jgart <jgart <at> dismail.de>, 75801 <at> debbugs.gnu.org, Marius Bakke <marius <at> gnu.org>
Subject: Re: [bug#75801] [PATCH python-team 1/2] gnu: python-numpy: Update
 to 1.26.2.
Date: Tue, 04 Feb 2025 14:04:51 +0100

Sharlatan Hellseher <sharlatanus <at> gmail.com> skribis:

> +    ;; XXX: Shipped with vendored-meson which is needed for SIMD and
> +    ;; BLAS/LAPACK features that are not yet available in upstream Meson.
> +    ;; See <https://raw.githubusercontent.com/numpy/numpy/
> +    ;; 2f3549c9d7c5048621568e431c86bc7530742723/doc/source/building/
> +    ;; understanding_meson.rst>
> +    (version "1.26.2")

Does the Meson thing also apply to 1.x?

Message #23 received at 75801 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 75801 <at> debbugs.gnu.org
Cc: rekado <at> elephly.net, ludo <at> gnu.org
Subject: [PATCH python-team 0/2] NumPy 1+ and 2+.
Date: Tue, 04 Feb 2025 13:42:48 +0000

[Message part 1 (text/plain, inline)]

Hi,

Thank you for review points. I'll apply them on v2, I would think to add
python-numpy-2 directly to master which will help with some other leaf
package updates (e.g. in Astronomy).

> Does the Meson thing also apply to 1.x?
As far as I see in GitHub it does for v1.26.5 and not for packaged in Guix v1.24.4.

https://github.com/numpy/numpy/tree/v1.26.5/vendored-meson -> 200
https://github.com/numpy/numpy/tree/v1.24.4/vendored-meson -> 404

> The file also says “As of Dec'23 […]”.  Do you think current versions of
> Meson would work?  That would be an improvement.
They do not upstream changes for some reason and pyproject.toml hard
requires it:

https://github.com/numpy/numpy/blob/v2.2.2/pyproject.toml
--8<---------------cut here---------------start------------->8---
[tool.meson-python]
meson = 'vendored-meson/meson/meson.py'
--8<---------------cut here---------------end--------------->8---

I'll poke around to see if may patch that.

--
Oleg

[signature.asc (application/pgp-signature, inline)]

Message #26 received at 75801 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 75801 <at> debbugs.gnu.org
Subject: [PATCH python-team 0/2] NumPy 1+ and 2+.
Date: Mon, 10 Feb 2025 16:00:58 +0000

[Message part 1 (text/plain, inline)]

Hi,

I've pushed update for 1+ to python-team with minor adjustment to
comments.

--
Oleg

[signature.asc (application/pgp-signature, inline)]

Message #31 received at 75801-done <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 75801-done <at> debbugs.gnu.org
Subject: [PATCH python-team 0/2] NumPy 1+ and 2+.
Date: Wed, 12 Feb 2025 17:53:57 +0000

[Message part 1 (text/plain, inline)]

Hi,

Pushed python-numpy-2 to master as
c9abfacd9f58f7365668b67a8125640b7497932a.

Someone or me later may fix check phase to make it work.

--
Thanks,
Oleg

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.