GNU bug report logs - #36026
[PATCH 0/4] Update Conda

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36026 in the body.
You can then email your comments to 36026 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: guix-patches <at> gnu.org
Subject: [PATCH 0/4] Update Conda
Date: Fri, 31 May 2019 15:57:29 +0200

Recent conda needs a separate python-conda-package-handling library.
This library requires a libarchive built with zstd support.
So far so good, except that python-libarchive-c does not seem to support zstd in its source code.

I've reported the issue upstream
(https://github.com/conda/conda-package-handling/issues/14) but
according to them python-libarchive-c is fine.

I don't have enough Python knowledge to understand whether upstream is right
	or if there is indeed something fishy with python-libarchive-c.
Pierre Neidhardt (4):
  gnu: libarchive: Add zstd support.
  gnu: python-libarchive-c: Explicitly rely on latest libarchive.
  gnu: Add python-conda-package-handling.
  gnu: python-conda: Update to 4.7.0.

 gnu/packages/backup.scm             | 10 ++--
 gnu/packages/package-management.scm | 81 ++++++++++++++++-------------
 gnu/packages/python-xyz.scm         |  2 +-
 3 files changed, 54 insertions(+), 39 deletions(-)

-- 
2.21.0

Message #8 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: 36026 <at> debbugs.gnu.org
Subject: [PATCH 1/4] gnu: libarchive: Add zstd support.
Date: Fri, 31 May 2019 16:04:41 +0200

* gnu/packages/backup.scm (libarchive)[inputs]: Include zstd.
[arguments]: Build against zstd.
---
 gnu/packages/backup.scm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/backup.scm b/gnu/packages/backup.scm
index b15c15ad46..da128a0901 100644
--- a/gnu/packages/backup.scm
+++ b/gnu/packages/backup.scm
@@ -216,7 +216,8 @@ backups (called chunks) to allow easy burning to CD/DVD.")
        ("lzo" ,lzo)
        ("bzip2" ,bzip2)
        ("libxml2" ,libxml2)
-       ("xz" ,xz)))
+       ("xz" ,xz)
+       ("zstd" ,zstd)))
     (arguments
      `(#:phases
        (modify-phases %standard-phases
@@ -245,7 +246,8 @@ backups (called chunks) to allow easy burning to CD/DVD.")
                     (libxml2 (assoc-ref inputs "libxml2"))
                     (xz      (assoc-ref inputs "xz"))
                     (zlib    (assoc-ref inputs "zlib"))
-                    (bzip2   (assoc-ref inputs "bzip2")))
+                    (bzip2   (assoc-ref inputs "bzip2"))
+                    (zstd   (assoc-ref inputs "zstd")))
                (substitute* (string-append lib "/pkgconfig/libarchive.pc")
                  (("-lnettle")
                   (string-append "-L" nettle "/lib -lnettle"))
@@ -256,7 +258,9 @@ backups (called chunks) to allow easy burning to CD/DVD.")
                  (("-lz")
                   (string-append "-L" zlib "/lib -lz"))
                  (("-lbz2")
-                  (string-append "-L" bzip2 "/lib -lbz2")))
+                  (string-append "-L" bzip2 "/lib -lbz2"))
+                 (("-lzstd")
+                  (string-append "-L" zstd "/lib -lzstd")))
                #t))))
 
        ;; libarchive/test/test_write_format_gnutar_filenames.c needs to be
-- 
2.21.0

Message #11 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: 36026 <at> debbugs.gnu.org
Subject: [PATCH 2/4] gnu: python-libarchive-c: Explicitly rely on latest
 libarchive.
Date: Fri, 31 May 2019 16:04:42 +0200

* gnu/packages/python-xyz.scm (python-libarchive-c)[inputs]: Do it.
---
 gnu/packages/python-xyz.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 5322b70759..9c9da1d678 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -7191,7 +7191,7 @@ a hash value.")
      `(("python-mock" ,python-mock)
        ("python-pytest" ,python-pytest)))
     (inputs
-     `(("libarchive" ,libarchive)))
+     `(("libarchive" ,libarchive-3.3.3)))
     (home-page "https://github.com/Changaco/python-libarchive-c")
     (synopsis "Python interface to libarchive")
     (description
-- 
2.21.0

Message #14 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: 36026 <at> debbugs.gnu.org
Subject: [PATCH 3/4] gnu: Add python-conda-package-handling.
Date: Fri, 31 May 2019 16:04:43 +0200

* gnu/packages/package-management.scm (python-conda-package-handling): New variable.
---
 gnu/packages/package-management.scm | 36 +++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index b4e1e077fe..28e23fd1d3 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -716,6 +716,42 @@ environments.")
 (define-public python2-anaconda-client
   (package-with-python2 python-anaconda-client))
 
+(define-public python-conda-package-handling
+  (package
+    (name "python-conda-package-handling")
+    (version "1.1.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/conda/conda-package-handling/")
+             (commit version)))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0qjraqb10bnnr329n68s97lkxh8mxchd6dmsynd0n6i9h49hhfgq"))))
+    (build-system python-build-system)
+    (inputs
+     `(("python-six" ,python-six)
+       ("python-libarchive-c" ,python-libarchive-c)
+       ("python-tqdm" ,python-tqdm)))
+    (native-inputs
+     `(("python-pytest" ,python-pytest)
+       ("python-pytest-cov" ,python-pytest-cov)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'check
+           (lambda _
+             ;; TODO: Tests fail.
+             (invoke "pytest" "-vv" "tests"))))))
+    (home-page "https://conda.io")
+    (synopsis "Create and extract conda packages of various formats")
+    (description
+     "This library is an abstraction of conda package handling and a tool for
+extracting, creating, and converting between formats.")
+    (license license:bsd-3)))
+
 (define-public python-conda
   (package
     (name "python-conda")
-- 
2.21.0

Message #17 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: 36026 <at> debbugs.gnu.org
Subject: [PATCH 4/4] gnu: python-conda: Update to 4.7.0.
Date: Fri, 31 May 2019 16:04:44 +0200

* gnu/packages/package-management.scm (python-conda): Update to 4.7.0.
---
 gnu/packages/package-management.scm | 45 +++++++----------------------
 1 file changed, 10 insertions(+), 35 deletions(-)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 28e23fd1d3..fb84f9abfc 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -755,16 +755,17 @@ extracting, creating, and converting between formats.")
 (define-public python-conda
   (package
     (name "python-conda")
-    (version "4.3.16")
+    (version "4.7.0")
     (source
      (origin
-       (method url-fetch)
-       (uri (string-append "https://github.com/conda/conda/archive/"
-                           version ".tar.gz"))
-       (file-name (string-append name "-" version ".tar.gz"))
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/conda/conda/")
+             (commit version)))
+       (file-name (git-file-name name version))
        (sha256
         (base32
-         "1jq8hyrc5npb5sf4vw6s6by4602yj8f79vzpbwdfgpkn02nfk1dv"))))
+         "0dzplykxrlri9a5c65bc48sdjpg2i50qzjf1qf9448qkghvmnmpr"))))
     (build-system python-build-system)
     (arguments
      `(#:phases
@@ -774,33 +775,6 @@ extracting, creating, and converting between formats.")
              (with-output-to-file "conda/.version"
                (lambda () (display ,version)))
              #t))
-         (add-before 'check 'remove-failing-tests
-           (lambda _
-             ;; These tests require internet/network access
-             (let ((network-tests '("test_cli.py"
-                                    "test_create.py"
-                                    "test_export.py"
-                                    "test_fetch.py"
-                                    "test_history.py"
-                                    "test_info.py"
-                                    "test_install.py"
-                                    "test_priority.py"
-                                    "conda_env/test_cli.py"
-                                    "conda_env/test_create.py"
-                                    "conda_env/specs/test_notebook.py"
-                                    "conda_env/utils/test_notebooks.py"
-                                    "core/test_index.py"
-                                    "core/test_repodata.py")))
-               (with-directory-excursion "tests"
-                 (for-each delete-file network-tests)
-
-                 ;; FIXME: This test creates a file, then deletes it and tests
-                 ;; that the file was deleted.  For some reason it fails when
-                 ;; building with guix, but does not when you run it in the
-                 ;; directory left when you build with the --keep-failed
-                 ;; option
-                 (delete-file "gateways/disk/test_delete.py")
-                 #t))))
          (replace 'check
            (lambda _
              (setenv "HOME" "/tmp")
@@ -813,8 +787,9 @@ extracting, creating, and converting between formats.")
        ("python-pytest" ,python-pytest)
        ("python-responses" ,python-responses)
        ("python-pyyaml" ,python-pyyaml)
-       ("python-anaconda-client" ,python-anaconda-client)))
-    (home-page "https://github.com/conda/conda")
+       ("python-anaconda-client" ,python-anaconda-client)
+       ("python-conda-package-handling" ,python-conda-package-handling)))
+    (home-page "https://conda.io")
     (synopsis "Cross-platform, OS-agnostic, system-level binary package manager")
     (description
      "Conda is a cross-platform, Python-agnostic binary package manager.  It
-- 
2.21.0

Message #20 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 36026 <at> debbugs.gnu.org
Subject: Re: [bug#36026] [PATCH 0/4] Update Conda
Date: Thu, 13 Jun 2019 10:45:03 +0200

Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

>   gnu: libarchive: Add zstd support.
>   gnu: python-libarchive-c: Explicitly rely on latest libarchive.
>   gnu: Add python-conda-package-handling.
>   gnu: python-conda: Update to 4.7.0.

Apart from the rebuild issue with patch #1 that needs to be addressed,
it LGTM!

Thanks,
Ludo’.

Message #23 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 36026 <at> debbugs.gnu.org
Subject: Re: [bug#36026] [PATCH 1/4] gnu: libarchive: Add zstd support.
Date: Thu, 13 Jun 2019 10:44:07 +0200

Hi Pierre,

Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> * gnu/packages/backup.scm (libarchive)[inputs]: Include zstd.
> [arguments]: Build against zstd.

Here you’re modifying ‘libarchive’, which has tons of dependents.

If you instead modify ‘libarchive-3.3.3’, you’ll be able to apply it
directly on master, and follow up with a patch to apply the changes on
‘libarchive’ itself in ‘core-updates’.

WDYT?

Thanks,
Ludo’.

Message #26 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 36026 <at> debbugs.gnu.org
Subject: Re: [bug#36026] [PATCH 0/4] Update Conda
Date: Thu, 13 Jun 2019 11:08:25 +0200

[Message part 1 (text/plain, inline)]

Thanks for the clarifications around the libarchive business.

The problem is that this patch still does not work because
python-libarchive-c does not support zstd, which
python-conda-package-handling needs.  I don't understand how upstream
does this.  I've reported it upstream:

https://github.com/conda/conda-package-handling/issues/14

My knowledge of python is not enough to understand if upstream is wrong
or if I'm using python-libarchive-c incorrectly.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #29 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 36026 <at> debbugs.gnu.org
Subject: Re: [bug#36026] [PATCH 0/4] Update Conda
Date: Thu, 13 Jun 2019 20:57:17 +0200

Hi Pierre,

> Recent conda needs a separate python-conda-package-handling library.
> This library requires a libarchive built with zstd support.
> So far so good, except that python-libarchive-c does not seem to support zstd in its source code.
>
> I've reported the issue upstream
> (https://github.com/conda/conda-package-handling/issues/14) but
> according to them python-libarchive-c is fine.
>
> I don't have enough Python knowledge to understand whether upstream is right
> 	or if there is indeed something fishy with python-libarchive-c.
> Pierre Neidhardt (4):
>   gnu: libarchive: Add zstd support.
>   gnu: python-libarchive-c: Explicitly rely on latest libarchive.
>   gnu: Add python-conda-package-handling.
>   gnu: python-conda: Update to 4.7.0.

Does this actually work?  If it does: congratulations!

I tried to upgrade conda a while ago, but I got stuck:

   https://github.com/conda/conda/issues/8151

How did you overcome these problems?  I had to patch a lot of things and
even then couldn’t make it work.  Did I miss something obvious?

I’m glad you took care of this and I didn’t have to become the Conda
guy… :)

--
Ricardo

Message #32 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 36026 <at> debbugs.gnu.org
Subject: Re: [bug#36026] [PATCH 4/4] gnu: python-conda: Update to 4.7.0.
Date: Thu, 13 Jun 2019 20:59:15 +0200

Pierre Neidhardt <mail <at> ambrevar.xyz> writes:

> * gnu/packages/package-management.scm (python-conda): Update to 4.7.0.
[…]
> -         (add-before 'check 'remove-failing-tests
> -           (lambda _
> -             ;; These tests require internet/network access
> -             (let ((network-tests '("test_cli.py"
> -                                    "test_create.py"
> -                                    "test_export.py"
> -                                    "test_fetch.py"
> -                                    "test_history.py"
> -                                    "test_info.py"
> -                                    "test_install.py"
> -                                    "test_priority.py"
> -                                    "conda_env/test_cli.py"
> -                                    "conda_env/test_create.py"
> -                                    "conda_env/specs/test_notebook.py"
> -                                    "conda_env/utils/test_notebooks.py"
> -                                    "core/test_index.py"
> -                                    "core/test_repodata.py")))
> -               (with-directory-excursion "tests"
> -                 (for-each delete-file network-tests)
> -
> -                 ;; FIXME: This test creates a file, then deletes it and tests
> -                 ;; that the file was deleted.  For some reason it fails when
> -                 ;; building with guix, but does not when you run it in the
> -                 ;; directory left when you build with the --keep-failed
> -                 ;; option
> -                 (delete-file "gateways/disk/test_delete.py")
> -                 #t))))

Do none of the tests now require network access?  Are the tests actually
still run?

--
Ricardo

Message #35 received at 36026 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 36026 <at> debbugs.gnu.org
Subject: Re: [bug#36026] [PATCH 0/4] Update Conda
Date: Thu, 13 Jun 2019 21:06:45 +0200

[Message part 1 (text/plain, inline)]

Hi Ricardo,

Sorry to kill your enthusiasm, but nope, it does not work :/
The is an issue building the conda-package-handling dependency.  See the
prior discussion for the details.

So I haven't even reached to the issues you've hinted at on GitHub.
Don't know if they still apply with my patch, we can only figure it out
one step at a time :p

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #40 received at 36026-done <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: 36026-done <at> debbugs.gnu.org
Cc: Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Update Conda
Date: Thu, 13 Aug 2020 00:07:29 +0200

Commit 638ef1e81d upgrades Conda to 4.8.3.  This makes these patches
obsolete.

What a nightmare this was!

-- 
Ricardo

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.