GNU bug report logs - #43855
gnu: python-shapely fails to build

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 43855 in the body.
You can then email your comments to 43855 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Malte Frank Gerdes <malte.f.gerdes <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: gnu: python-shapely fails to build
Date: Wed, 07 Oct 2020 23:32:01 +0200

Hi,

python-shapely fails to build, because it can't find the c libary,
the package definition was not complete (patch at the end).

Now the package finds a C libary but the Test Suite fails with the
follwoing error:

--8<---------------cut here---------------start------------->8---
======================================================================
FAIL: test_operations (tests.test_operations.OperationsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/guix-build-python-shapely-1.6.4.post2.drv-0/Shapely-1.6.4.post2/tests/test_operations.py", line 25, in test_operations
    self.assertIsInstance(point.intersection(Point(-1, -1)),
AssertionError: <shapely.geometry.point.Point object at 0x7fffe1544f70> is not an instance of <class 'shapely.geometry.collection.GeometryCollection'>
--8<---------------cut here---------------end--------------->8---

This package was last successfully built on 13th September.


Malte

--8<---------------cut here---------------start------------->8---
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 459526941b..02744bfc6d 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -533,13 +533,16 @@ pidof, tty, taskset, pmap.")
              (let ((geos (assoc-ref inputs "geos"))
                    (glibc (assoc-ref inputs ,(if (%current-target-system)
                                                  "cross-libc" "libc"))))
-               (substitute* "shapely/geos.py"
-                 (("_lgeos = load_dll\\('geos_c', fallbacks=.*\\)")
-                  (string-append "_lgeos = load_dll('geos_c', fallbacks=['"
-                                 geos "/lib/libgeos_c.so'])"))
-                 (("free = load_dll\\('c'\\)\\.free")
-                  (string-append "free = load_dll('c', fallbacks=['"
-                                 glibc "/lib/libc.so.6']).free"))))
+               (map
+                (lambda (file)
+                  (substitute* file
+                    (("_lgeos = load_dll\\('geos_c', fallbacks=.*\\)")
+                     (string-append "_lgeos = load_dll('geos_c', fallbacks=['"
+                                    geos "/lib/libgeos_c.so'])"))
+                    (("free = load_dll\\('c'\\)\\.free")
+                     (string-append "free = load_dll('c', fallbacks=['"
+                                    glibc "/lib/libc.so.6']).free"))))
+                '("shapely/geos.py" "shapely/_buildcfg.py")))
              #t)))))
     (home-page "https://github.com/Toblerity/Shapely")
     (synopsis "Library for the manipulation and analysis of geometric objects")
--8<---------------cut here---------------end--------------->8---

Message #8 received at 43855 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Malte Frank Gerdes <malte.f.gerdes <at> gmail.com>
Cc: 43855 <at> debbugs.gnu.org
Subject: Re: bug#43855: gnu: python-shapely fails to build
Date: Thu, 8 Oct 2020 09:44:27 +0300

[Message part 1 (text/plain, inline)]

On Wed, Oct 07, 2020 at 11:32:01PM +0200, Malte Frank Gerdes wrote:
> Hi,
> 
> python-shapely fails to build, because it can't find the c libary,
> the package definition was not complete (patch at the end).
> 
> Now the package finds a C libary but the Test Suite fails with the
> follwoing error:
> 
> --8<---------------cut here---------------start------------->8---
> ======================================================================
> FAIL: test_operations (tests.test_operations.OperationsTestCase)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/tmp/guix-build-python-shapely-1.6.4.post2.drv-0/Shapely-1.6.4.post2/tests/test_operations.py", line 25, in test_operations
>     self.assertIsInstance(point.intersection(Point(-1, -1)),
> AssertionError: <shapely.geometry.point.Point object at 0x7fffe1544f70> is not an instance of <class 'shapely.geometry.collection.GeometryCollection'>
> --8<---------------cut here---------------end--------------->8---
> 
> This package was last successfully built on 13th September.
> 

It turns out this is due to a change in geos with 3.8
https://github.com/Toblerity/Shapely/issues/799

If I cherry-pick the patch to fix it then it passes the test suite. If I
upgrade python-shapely to 1.7.1 the fix is already included but there's
a new build error:

gcc -pthread -shared -Wl,-rpath=/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib build/temp.linux-x86_64-3.8/shapely/vectorized/_vectorized.o -L/gnu/store/pinvpk9w6izxcwci4ghiwgvykmgsfy2c-geos-3.8.1/lib -L/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib -lgeos_c -o /tmp/guix-build-python-shapely-1.7.1.drv-0/Shapely-1.7.1/shapely/vectorized/_vectorized.cpython-38-x86_64-linux-gnu.so
warning: no library file corresponding to '/gnu/store/603imkkh2bqs01z8ik0b2ndgpdz0jghk-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/core/include' found (skipping)
warning: no library file corresponding to '/gnu/store/603imkkh2bqs01z8ik0b2ndgpdz0jghk-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/core/include' found (skipping)
error: Could not find lib c or load any of its variants ['libc.musl-x86_64.so.1'].
command "python" "-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test" failed with status 1
builder for `/gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv' failed with exit code 1
build of /gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv failed
View build log at '/var/log/guix/drvs/2y/z1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv.bz2'.
guix build: error: build of `/gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv' failed

Do you want to take a stab at updating python-shapely to 1.7.1?

> 
> Malte
> 
> --8<---------------cut here---------------start------------->8---
> diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
> index 459526941b..02744bfc6d 100644
> --- a/gnu/packages/python-xyz.scm
> +++ b/gnu/packages/python-xyz.scm
> @@ -533,13 +533,16 @@ pidof, tty, taskset, pmap.")
>               (let ((geos (assoc-ref inputs "geos"))
>                     (glibc (assoc-ref inputs ,(if (%current-target-system)
>                                                   "cross-libc" "libc"))))
> -               (substitute* "shapely/geos.py"
> -                 (("_lgeos = load_dll\\('geos_c', fallbacks=.*\\)")
> -                  (string-append "_lgeos = load_dll('geos_c', fallbacks=['"
> -                                 geos "/lib/libgeos_c.so'])"))
> -                 (("free = load_dll\\('c'\\)\\.free")
> -                  (string-append "free = load_dll('c', fallbacks=['"
> -                                 glibc "/lib/libc.so.6']).free"))))
> +               (map
> +                (lambda (file)
> +                  (substitute* file
> +                    (("_lgeos = load_dll\\('geos_c', fallbacks=.*\\)")
> +                     (string-append "_lgeos = load_dll('geos_c', fallbacks=['"
> +                                    geos "/lib/libgeos_c.so'])"))
> +                    (("free = load_dll\\('c'\\)\\.free")
> +                     (string-append "free = load_dll('c', fallbacks=['"
> +                                    glibc "/lib/libc.so.6']).free"))))
> +                '("shapely/geos.py" "shapely/_buildcfg.py")))
>               #t)))))
>      (home-page "https://github.com/Toblerity/Shapely")
>      (synopsis "Library for the manipulation and analysis of geometric objects")
> --8<---------------cut here---------------end--------------->8---
> 

substitute* can take a list, so it would be simpler to write this as:
substitute* '("shapely/geos.py"
              "shapely/_buildcfg.py")

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 43855 <at> debbugs.gnu.org (full text, mbox):

From: Malte Frank Gerdes <malte.f.gerdes <at> gmail.com>
To: 43855 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: bug#43855: gnu: python-shapely fails to build
Date: Thu, 08 Oct 2020 14:05:50 +0200

Efraim Flashner <efraim <at> flashner.co.il> writes:

> It turns out this is due to a change in geos with 3.8
> https://github.com/Toblerity/Shapely/issues/799
>
> If I cherry-pick the patch to fix it then it passes the test suite. If I
> upgrade python-shapely to 1.7.1 the fix is already included but there's
> a new build error:
>
> gcc -pthread -shared
> -Wl,-rpath=/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib
> build/temp.linux-x86_64-3.8/shapely/vectorized/_vectorized.o
> -L/gnu/store/pinvpk9w6izxcwci4ghiwgvykmgsfy2c-geos-3.8.1/lib
> -L/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib -lgeos_c -o
> /tmp/guix-build-python-shapely-1.7.1.drv-0/Shapely-1.7.1/shapely/vectorized/_vectorized.cpython-38-x86_64-linux-gnu.so
> warning: no library file corresponding to
> '/gnu/store/603imkkh2bqs01z8ik0b2ndgpdz0jghk-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/core/include'
> found (skipping)
> warning: no library file corresponding to
> '/gnu/store/603imkkh2bqs01z8ik0b2ndgpdz0jghk-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/core/include'
> found (skipping)
> error: Could not find lib c or load any of its variants ['libc.musl-x86_64.so.1'].
> command "python" "-c" "import setuptools,
> tokenize;__file__='setup.py';f=getattr(tokenize, 'open',
> open)(__file__);code=f.read().replace('\\r\\n',
> '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test" failed with
> status 1
> builder for `/gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv' failed with exit code 1
> build of /gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv failed
> View build log at '/var/log/guix/drvs/2y/z1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv.bz2'.
> guix build: error: build of `/gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv' failed
>
> Do you want to take a stab at updating python-shapely to 1.7.1?

I can certainly try to fix this. I will look into it in the evening.
BTW: Are you using musl libc as your primary libc? If so, that's cool,
how did you do that? :)

> substitute* can take a list, so it would be simpler to write this as:
> substitute* '("shapely/geos.py"
>               "shapely/_buildcfg.py")

One more thing i've learned :)

Malte

Message #14 received at 43855 <at> debbugs.gnu.org (full text, mbox):

From: Malte Frank Gerdes <malte.f.gerdes <at> gmail.com>
To: 43855 <at> debbugs.gnu.org
Subject: [PATCH] gnu: python-shapely: Update to 1.7.1.
Date: Thu, 08 Oct 2020 20:25:50 +0200

[Message part 1 (text/plain, inline)]

It seems python-shapely assumes musl libc, because the store path of
glibc was missing in one place. Now the Test suite runs without failure.

Malte

[0001-gnu-python-shapely-Update-to-1.7.1.patch (text/x-patch, inline)]

From a320ab014d60f5401f409eb37ccb16aa23c32cf1 Mon Sep 17 00:00:00 2001
From: Malte Frank Gerdes <malte.f.gerdes <at> gmail.com>
Date: Thu, 8 Oct 2020 20:20:12 +0200
Subject: [PATCH] gnu: python-shapely: Update to 1.7.1.

* gnu/packages/python-xyz.scm (python-shapely): Update to 1.7.1.
---
 gnu/packages/python-xyz.scm | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 459526941b..fa4a4f4954 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -507,14 +507,14 @@ pidof, tty, taskset, pmap.")
 (define-public python-shapely
   (package
     (name "python-shapely")
-    (version "1.6.4.post2")
+    (version "1.7.1")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "Shapely" version))
        (sha256
         (base32
-         "03r42fmd9alp6r3q95ad6rldq2f7n1wimrw53zy5kpn33yv7pf64"))))
+         "0adiz4jwmwxk7k1awqifb1a9bj5x4nx4gglb5dz9liam21674h8n"))))
     (build-system python-build-system)
     (native-inputs
      `(("python-cython" ,python-cython)
@@ -533,11 +533,14 @@ pidof, tty, taskset, pmap.")
              (let ((geos (assoc-ref inputs "geos"))
                    (glibc (assoc-ref inputs ,(if (%current-target-system)
                                                  "cross-libc" "libc"))))
-               (substitute* "shapely/geos.py"
+               (substitute* '("shapely/geos.py" "shapely/_buildcfg.py")
                  (("_lgeos = load_dll\\('geos_c', fallbacks=.*\\)")
                   (string-append "_lgeos = load_dll('geos_c', fallbacks=['"
                                  geos "/lib/libgeos_c.so'])"))
                  (("free = load_dll\\('c'\\)\\.free")
+                  (string-append "free = load_dll('c', fallbacks=['"
+                                 glibc "/lib/libc.so.6']).free"))
+                 (("free = load_dll\\('c', fallbacks=.*\\)\\.free")
                   (string-append "free = load_dll('c', fallbacks=['"
                                  glibc "/lib/libc.so.6']).free"))))
              #t)))))
-- 
2.28.0

Message #17 received at 43855 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Malte Frank Gerdes <malte.f.gerdes <at> gmail.com>
Cc: 43855 <at> debbugs.gnu.org
Subject: Re: bug#43855: gnu: python-shapely fails to build
Date: Fri, 9 Oct 2020 07:59:32 +0300

[Message part 1 (text/plain, inline)]

On Thu, Oct 08, 2020 at 02:05:50PM +0200, Malte Frank Gerdes wrote:
> Efraim Flashner <efraim <at> flashner.co.il> writes:
> 
> > It turns out this is due to a change in geos with 3.8
> > https://github.com/Toblerity/Shapely/issues/799
> >
> > If I cherry-pick the patch to fix it then it passes the test suite. If I
> > upgrade python-shapely to 1.7.1 the fix is already included but there's
> > a new build error:
> >
> > gcc -pthread -shared
> > -Wl,-rpath=/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib
> > build/temp.linux-x86_64-3.8/shapely/vectorized/_vectorized.o
> > -L/gnu/store/pinvpk9w6izxcwci4ghiwgvykmgsfy2c-geos-3.8.1/lib
> > -L/gnu/store/q9rm8h9imazsq2c4qiv2yjpvlvliywqb-python-3.8.2/lib -lgeos_c -o
> > /tmp/guix-build-python-shapely-1.7.1.drv-0/Shapely-1.7.1/shapely/vectorized/_vectorized.cpython-38-x86_64-linux-gnu.so
> > warning: no library file corresponding to
> > '/gnu/store/603imkkh2bqs01z8ik0b2ndgpdz0jghk-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/core/include'
> > found (skipping)
> > warning: no library file corresponding to
> > '/gnu/store/603imkkh2bqs01z8ik0b2ndgpdz0jghk-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/core/include'
> > found (skipping)
> > error: Could not find lib c or load any of its variants ['libc.musl-x86_64.so.1'].
> > command "python" "-c" "import setuptools,
> > tokenize;__file__='setup.py';f=getattr(tokenize, 'open',
> > open)(__file__);code=f.read().replace('\\r\\n',
> > '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test" failed with
> > status 1
> > builder for `/gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv' failed with exit code 1
> > build of /gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv failed
> > View build log at '/var/log/guix/drvs/2y/z1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv.bz2'.
> > guix build: error: build of `/gnu/store/2yz1ybl0156canhfz4vhay0g473c5nwv-python-shapely-1.7.1.drv' failed
> >
> > Do you want to take a stab at updating python-shapely to 1.7.1?
> 
> I can certainly try to fix this. I will look into it in the evening.
> BTW: Are you using musl libc as your primary libc? If so, that's cool,
> how did you do that? :)

Ah, no. It comes from the code. I suppose it would be possible to tell
Guix to use musl instead of glibc but I don't think the build system is
set up for that.

> > substitute* can take a list, so it would be simpler to write this as:
> > substitute* '("shapely/geos.py"
> >               "shapely/_buildcfg.py")
> 
> One more thing i've learned :)
> 
> Malte

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #22 received at 43855-done <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Malte Frank Gerdes <malte.f.gerdes <at> gmail.com>
Cc: 43855-done <at> debbugs.gnu.org
Subject: Re: bug#43855: [PATCH] gnu: python-shapely: Update to 1.7.1.
Date: Fri, 9 Oct 2020 08:01:27 +0300

[Message part 1 (text/plain, inline)]

On Thu, Oct 08, 2020 at 08:25:50PM +0200, Malte Frank Gerdes wrote:
> It seems python-shapely assumes musl libc, because the store path of
> glibc was missing in one place. Now the Test suite runs without failure.
> 
> Malte
> 

Looks good! Certainly a smaller patch than I was expecting.

I added a copyright line for you and pushed the patch.


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.