GNU bug report logs - #68332
[PATCH 0/6] Add curl-full

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 68332 in the body.
You can then email your comments to 68332 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: guix-patches <at> gnu.org
Cc: Tomas Volf <~@wolfsden.cz>
Subject: [PATCH 0/6] Add curl-full
Date: Mon,  8 Jan 2024 20:19:36 +0100

Current curl package does not support websockets, does not support HTTP3 and
does not support Public Suffix List.  The last one could even be considered a
security issue.

This commit adds new curl-full package, that enables everything that can be
enabled.

This opportunity was also used to update ngtcp2 and do some small cleaning.

Tomas Volf (6):
  gnu: web: Delete trailing whitespace.
  gnu: Add nghttp3.
  gnu: ngtcp2: Update to 1.1.0.
  gnu: ngtcp2: Build libngtcp2_crypto_gnutls.
  gnu: Add curl-full.
  gnu: curl-full: Update homepage.

 gnu/local.mk                                  |  1 +
 gnu/packages/curl.scm                         | 71 ++++++++++++++++-
 gnu/packages/networking.scm                   | 13 ++-
 .../patches/curl-8.5-use-ssl-cert-env.patch   | 79 +++++++++++++++++++
 gnu/packages/web.scm                          | 38 ++++++++-
 5 files changed, 197 insertions(+), 5 deletions(-)
 create mode 100644 gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch


base-commit: b212e6934643e085f168a5364cb593f61aa616ba
--
2.41.0

Message #8 received at 68332 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: 68332 <at> debbugs.gnu.org
Cc: Tomas Volf <~@wolfsden.cz>
Subject: [PATCH 3/6] gnu: ngtcp2: Update to 1.1.0.
Date: Mon,  8 Jan 2024 20:38:15 +0100

* gnu/packages/networking.scm (ngtcp2): Update to 1.1.0.

Change-Id: I7f6b08952ba508207c8dedbd68c7c48dd17dd0ff
---
 gnu/packages/networking.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index 9844430656..c738716792 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -4385,7 +4385,7 @@ (define-public nbd
 (define-public ngtcp2
   (package
     (name "ngtcp2")
-    (version "1.0.1")
+    (version "1.1.0")
     (source
      (origin
        (method url-fetch)
@@ -4393,7 +4393,7 @@ (define-public ngtcp2
                            "releases/download/v" version "/"
                            "ngtcp2-" version ".tar.xz"))
        (sha256
-        (base32 "0l84hnj9n4bfxjizgmqsqbz71jx7m00a7l1z43fg5ls3apx9ij11"))))
+        (base32 "1pppl6s25hz91w6321g1q7dqvfy4vccz9mmc5r8sfdvdc95fngl0"))))
     (build-system gnu-build-system)
     (native-inputs (list cunit))
     (home-page "https://nghttp2.org/ngtcp2/")
-- 
2.41.0

Message #11 received at 68332 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: 68332 <at> debbugs.gnu.org
Cc: Tomas Volf <~@wolfsden.cz>
Subject: [PATCH 1/6] gnu: web: Delete trailing whitespace.
Date: Mon,  8 Jan 2024 20:38:13 +0100

* gnu/packages/web.scm (krona-tools): Delete trailing whitespace.

Change-Id: Ide601b011d3e2c3c736bf041571f9ab00b5b5b1d
---
 gnu/packages/web.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 13ae53aa8c..e0f8bf4547 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -1481,7 +1481,7 @@ (define-public krona-tools
                   "ImportTaxonomy"
                   "ImportText"
                   "ImportXML"))
-               (for-each 
+               (for-each
                 (lambda (directory)
                   (copy-recursively directory
                                     (string-append perl "/../" directory)))
-- 
2.41.0

Message #14 received at 68332 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: 68332 <at> debbugs.gnu.org
Cc: Tomas Volf <~@wolfsden.cz>
Subject: [PATCH 6/6] gnu: curl-full: Update homepage.
Date: Mon,  8 Jan 2024 20:38:18 +0100

Resolves following lint warning:

    permanent redirect from https://curl.haxx.se/ to https://curl.se/

* gnu/packages/curl.scm (curl-full)[homepage]: Update.

Change-Id: I88c899c0a79b80e1ce86add8227c43c12d8b00ce
---
 gnu/packages/curl.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index 0f0336cb6a..f233eeca7a 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -173,7 +173,7 @@ (define-public curl
 tunneling, and so on.")
     (license (license:non-copyleft "file://COPYING"
                                    "See COPYING in the distribution."))
-    (home-page "https://curl.haxx.se/")))
+    (home-page "https://curl.se/")))
 
 (define-public curl-full
   (package/inherit curl
-- 
2.41.0

Message #17 received at 68332 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: 68332 <at> debbugs.gnu.org
Cc: Tomas Volf <~@wolfsden.cz>
Subject: [PATCH 5/6] gnu: Add curl-full.
Date: Mon,  8 Jan 2024 20:38:17 +0100

New variable providing curl compiled with all available features is added.

* gnu/packages/curl.scm (curl-full): New variable.
* gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.

Change-Id: Ic3c9b2b9bd0a81d6840dcdccd21d52a9d0b311e2
---
 gnu/local.mk                                  |  1 +
 gnu/packages/curl.scm                         | 69 ++++++++++++++++
 .../patches/curl-8.5-use-ssl-cert-env.patch   | 79 +++++++++++++++++++
 3 files changed, 149 insertions(+)
 create mode 100644 gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index f804f4ef5b..73eab7cf93 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1060,6 +1060,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/clucene-contribs-lib.patch               \
   %D%/packages/patches/cube-nocheck.patch			\
   %D%/packages/patches/curl-use-ssl-cert-env.patch		\
+  %D%/packages/patches/curl-8.5-use-ssl-cert-env.patch		\
   %D%/packages/patches/curlftpfs-fix-error-closing-file.patch	\
   %D%/packages/patches/curlftpfs-fix-file-names.patch		\
   %D%/packages/patches/curlftpfs-fix-memory-leak.patch		\
diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index b33f4d36d4..0f0336cb6a 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -16,6 +16,7 @@
 ;;; Copyright © 2021 Felix Gruber <felgru <at> posteo.net>
 ;;; Copyright © 2023 Sharlatan Hellseher <sharlatanus <at> gmail.com>
 ;;; Copyright © 2023 John Kehayias <john.kehayias <at> protonmail.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -46,14 +47,19 @@ (define-module (gnu packages curl)
   #:use-module (guix build-system meson)
   #:use-module ((guix search-paths) #:select ($SSL_CERT_DIR $SSL_CERT_FILE))
   #:use-module (gnu packages)
+  #:use-module (gnu packages certs)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
+  #:use-module (gnu packages gsasl)
+  #:use-module (gnu packages groff)
   #:use-module (gnu packages guile)
   #:use-module (gnu packages kerberos)
   #:use-module (gnu packages logging)
   #:use-module (gnu packages libevent)
   #:use-module (gnu packages libidn)
+  #:use-module (gnu packages openldap)
+  #:use-module (gnu packages networking)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
@@ -169,6 +175,69 @@ (define-public curl
                                    "See COPYING in the distribution."))
     (home-page "https://curl.haxx.se/")))
 
+(define-public curl-full
+  (package/inherit curl
+    (name "curl-full")
+    (version "8.5.0")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://curl.se/download/curl-"
+                                  version ".tar.gz"))
+              (sha256
+               (base32
+                "1pas8z1vijp08za5xgh2vvsajwi1p2208vlhl0vs94xp4pzigz05"))
+              (patches (search-patches "curl-8.5-use-ssl-cert-env.patch"))))
+    (arguments
+     (substitute-keyword-arguments (package-arguments curl)
+       ((#:configure-flags flags)
+        #~(cons* "--enable-manual"
+                 ;; Not yet available: https://github.com/curl/curl/pull/11922
+                 ;; "--enable-ech"
+                 "--enable-websockets"
+                 "--with-brotli"
+                 "--with-ldap"
+                 "--with-ldaps"
+                 "--with-librtmp"
+                 "--with-libssh2"
+                 "--with-nghttp3"
+                 "--with-ngtcp2"
+                 "--with-zstd"
+                 #$flags))
+       ((#:phases phases)
+        #~(modify-phases #$phases
+            (add-after 'unpack 'skip-tests
+              (lambda _
+                (let ((port (open-file "tests/data/DISABLED" "a")))
+                  ;; I do not understand why curl package ignores this test
+                  ;; just on some architectures.  I had it fail even on amd64,
+                  ;; it is just too flaky.
+                  (display "1474\n" port)
+                  ;; Required files are missing, should be fixed in >8.5.0:
+                  ;;   https://github.com/curl/curl/issues/12462
+                  (display "1477\n" port)
+                  (close port))))))))
+    (propagated-inputs
+     (list nss-certs))
+    (native-inputs
+     (modify-inputs (package-native-inputs curl)
+       (prepend groff)))
+    (inputs
+     (modify-inputs (package-inputs curl)
+       (prepend brotli
+                gsasl
+                libpsl
+                libssh2
+                nghttp3
+                ngtcp2
+                openldap
+                rtmpdump
+                `(,zstd "lib"))))
+    (description (string-append (package-description curl)
+                                "\n\n"
+                                "Compared to the regular curl package, this
+one enables everything that can be enabled, bringing in additional features
+and security."))))
+
 (define-public curl-ssh
   (package/inherit curl
     (arguments
diff --git a/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch b/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch
new file mode 100644
index 0000000000..70f703a999
--- /dev/null
+++ b/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch
@@ -0,0 +1,79 @@
+From 0a942e9f2f5255793e99408ca3da0940d52dfdad Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba <at> kadziolka.net>
+Date: Mon, 8 Jan 2024 15:07:38 +0100
+Subject: [PATCH] libcurl: Respect SSL_CERT_DIR and SSL_CERT_FILE.
+
+Make libcurl respect the SSL_CERT_{DIR,FILE} variables by default. The variables
+are fetched during initialization to preserve thread-safety (curl_global_init(3)
+must be called when no other threads exist).
+
+This fixes network functionality in rust:cargo, and probably removes the need
+for other future workarounds.
+---
+ lib/easy.c |  9 +++++++++
+ lib/url.c  | 15 +++++++++++++++
+ 2 files changed, 24 insertions(+)
+
+diff --git a/lib/easy.c b/lib/easy.c
+index 322d1a41b..2a235a19c 100644
+--- a/lib/easy.c
++++ b/lib/easy.c
+@@ -137,6 +137,9 @@ curl_wcsdup_callback Curl_cwcsdup = Curl_wcsdup;
+ static char *leakpointer;
+ #endif
+ 
++char * Curl_ssl_cert_dir = NULL;
++char * Curl_ssl_cert_file = NULL;
++
+ /**
+  * curl_global_init() globally initializes curl given a bitwise set of the
+  * different features of what to initialize.
+@@ -163,6 +166,9 @@ static CURLcode global_init(long flags, bool memoryfuncs)
+     goto fail;
+   }
+ 
++  Curl_ssl_cert_dir = curl_getenv("SSL_CERT_DIR");
++  Curl_ssl_cert_file = curl_getenv("SSL_CERT_FILE");
++
+   if(!Curl_ssl_init()) {
+     DEBUGF(fprintf(stderr, "Error: Curl_ssl_init failed\n"));
+     goto fail;
+@@ -287,6 +293,9 @@ void curl_global_cleanup(void)
+   Curl_ssl_cleanup();
+   Curl_resolver_global_cleanup();
+ 
++  free(Curl_ssl_cert_dir);
++  free(Curl_ssl_cert_file);
++
+ #ifdef _WIN32
+   Curl_win32_cleanup(easy_init_flags);
+ #endif
+diff --git a/lib/url.c b/lib/url.c
+index b81785fe2..d607e74bb 100644
+--- a/lib/url.c
++++ b/lib/url.c
+@@ -454,6 +454,21 @@ CURLcode Curl_init_userdefined(struct Curl_easy *data)
+     if(result)
+       return result;
+ #endif
++    extern char * Curl_ssl_cert_dir;
++    extern char * Curl_ssl_cert_file;
++    if(Curl_ssl_cert_dir) {
++        if(result = Curl_setstropt(&set->str[STRING_SSL_CAPATH], Curl_ssl_cert_dir))
++            return result;
++        if(result = Curl_setstropt(&set->str[STRING_SSL_CAPATH_PROXY], Curl_ssl_cert_dir))
++            return result;
++    }
++
++    if(Curl_ssl_cert_file) {
++        if(result = Curl_setstropt(&set->str[STRING_SSL_CAFILE], Curl_ssl_cert_file))
++            return result;
++        if(result = Curl_setstropt(&set->str[STRING_SSL_CAFILE_PROXY], Curl_ssl_cert_file))
++            return result;
++    }
+   }
+ 
+ #ifndef CURL_DISABLE_FTP
+-- 
+2.41.0
+
-- 
2.41.0

Message #20 received at 68332 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: 68332 <at> debbugs.gnu.org
Cc: Tomas Volf <~@wolfsden.cz>
Subject: [PATCH 2/6] gnu: Add nghttp3.
Date: Mon,  8 Jan 2024 20:38:14 +0100

* gnu/packages/web.scm (nghttp3): New variable.

Change-Id: I4ed318506f6d3bdd0c47a367b1dcc1b4454c09b2
---
 gnu/packages/web.scm | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index e0f8bf4547..3c7ff49fb9 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -65,6 +65,7 @@
 ;;; Copyright © 2023 Christopher Howard <christopher <at> librehacker.com>
 ;;; Copyright © 2023 Felix Lechner <felix.lechner <at> lease-up.com>
 ;;; Copyright © 2023 Evgeny Pisemsky <evgeny <at> pisemsky.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -8137,6 +8138,41 @@ (define-public nghttp2-for-node
                    (("print \\(ver >= '3\\.8'\\)")
                     "print (tuple(map(int, ver.split('.'))) >= (3,8))")))))))))))
 
+(define-public nghttp3
+  (package
+    (name "nghttp3")
+    (version "1.1.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/ngtcp2/nghttp3/"
+                           "releases/download/v" version "/"
+                           "nghttp3-" version ".tar.gz"))
+       (sha256
+        (base32
+         "1fzvadnwb03jlm180313gg5m4fg09qdcc67fwcfrv9zs22anaa55"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     (list pkg-config
+           ;; Required by tests.
+           cunit))
+    (arguments
+     (list
+      #:configure-flags
+      #~(list "--enable-lib-only")))
+    (home-page "https://nghttp2.org/nghttp3/")
+    (synopsis "HTTP/3 protocol library")
+    (description
+     "nghttp3 is an implementation of RFC 9114 HTTP/3 mapping over QUIC and
+RFC 9204 QPACK in C.  It does not depend on any particular QUIC transport
+implementation.
+
+It implements extensions specified in RFC 9218 and RFC 9220.  It supports
+SETTINGS_H3_DATAGRAM from RFC 9297.
+
+It does not support server push.")
+    (license license:expat)))
+
 (define-public hpcguix-web
   (package
     (name "hpcguix-web")
-- 
2.41.0

Message #23 received at 68332 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: 68332 <at> debbugs.gnu.org
Cc: Tomas Volf <~@wolfsden.cz>
Subject: [PATCH 4/6] gnu: ngtcp2: Build libngtcp2_crypto_gnutls.
Date: Mon,  8 Jan 2024 20:38:16 +0100

* gnu/packages/networking.scm (ngtcp2)[arguments]<#:configure-flags>: Add
--with-gnutls.
[native-inputs]: Add pkg-config.
[inputs]: Add gnutls.

Change-Id: Id3a9210992d203a57a9b791f3c3752f2ebd3e634
---
 gnu/packages/networking.scm | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index c738716792..40155480c9 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -61,6 +61,7 @@
 ;;; Copyright © 2023 Yovan Naumovski <yovan <at> gorski.stream>
 ;;; Copyright © 2023 Zheng Junjie <873216071 <at> qq.com>
 ;;; Copyright © 2023 Artyom V. Poptsov <poptsov.artyom <at> gmail.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -4395,7 +4396,13 @@ (define-public ngtcp2
        (sha256
         (base32 "1pppl6s25hz91w6321g1q7dqvfy4vccz9mmc5r8sfdvdc95fngl0"))))
     (build-system gnu-build-system)
-    (native-inputs (list cunit))
+    (arguments
+     (list
+      #:configure-flags
+      ;; openssl package does not support QUIC interface, so just gnutls
+      #~(list "--with-gnutls")))
+    (native-inputs (list cunit pkg-config))
+    (inputs (list gnutls))
     (home-page "https://nghttp2.org/ngtcp2/")
     (synopsis "QUIC protocol implementation")
     (description
-- 
2.41.0

Message #28 received at 68332 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Tomas Volf <~@wolfsden.cz>
Cc: 68332 <at> debbugs.gnu.org
Subject: Re: [bug#68332] [PATCH 0/6] Add curl-full
Date: Wed, 17 Jan 2024 18:22:51 +0100

Hello Tomas,

>   gnu: web: Delete trailing whitespace.
>   gnu: Add nghttp3.
>   gnu: ngtcp2: Update to 1.1.0.
>   gnu: ngtcp2: Build libngtcp2_crypto_gnutls.

I took the liberty of applying those 4 even if you closed this issue, as
discussed on #guix.

Thanks,

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.