GNU bug report logs - #25975
Use HTTPS in `guix pull`

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 25975 in the body.
You can then email your comments to 25975 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: guix-patches <at> gnu.org
Subject: Use HTTPS in `guix pull`
Date: Sun, 05 Mar 2017 15:59:16 +0100

[Message part 1 (text/plain, inline)]

I've tried a number of times to send this through `git send-email`, but
it seems to get caught in a spam filter or similar.

Trying as attachment now.

Note that this uses 'nss-certs' for easy testing, but is intended to use
'le-certs' from this thread:

https://lists.gnu.org/archive/html/guix-devel/2017-02/msg01146.html

[0001-pull-Default-to-HTTPS.patch (text/x-patch, inline)]

From 6667ea5a2ec3a26dd5c4fb5f792485eeb941a969 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke <at> fastmail.com>
Date: Wed, 1 Mar 2017 22:11:02 +0100
Subject: [PATCH] pull: Default to HTTPS.

* guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
(guix-pull): Add GNUTLS and NSS-CERTS to inputs when appropriate.
---
 guix/scripts/pull.scm | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
index a4824e4fd..4031f1d32 100644
--- a/guix/scripts/pull.scm
+++ b/guix/scripts/pull.scm
@@ -29,12 +29,16 @@
   #:use-module (guix monads)
   #:use-module ((guix build utils)
                 #:select (with-directory-excursion delete-file-recursively))
+  #:use-module ((guix build download)
+                #:select (%x509-certificate-directory))
   #:use-module (gnu packages base)
   #:use-module (gnu packages guile)
   #:use-module ((gnu packages bootstrap)
                 #:select (%bootstrap-guile))
+  #:use-module ((gnu packages certs) #:select (nss-certs))
   #:use-module (gnu packages compression)
   #:use-module (gnu packages gnupg)
+  #:use-module ((gnu packages tls) #:select (gnutls))
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
@@ -45,7 +49,7 @@
 
 (define %snapshot-url
   ;; "http://hydra.gnu.org/job/guix/master/tarball/latest/download"
-  "http://git.savannah.gnu.org/cgit/guix.git/snapshot/master.tar.gz"
+  "https://git.savannah.gnu.org/cgit/guix.git/snapshot/master.tar.gz"
   )
 
 (define-syntax-rule (with-environment-variable variable value body ...)
@@ -221,11 +225,35 @@ contained therein."
                   (leave (_ "~A: unexpected argument~%") arg))
                 %default-options))
 
+  (define (use-gnutls? url)
+    (string-prefix? "https://" url))
+
+  (define (use-le-certs? url)
+    (string-prefix? "https://git.savannah.gnu.org" url))
+
+  (define (fetch-tarball store url)
+    (download-to-store store url "guix-latest.tar.gz"))
+
   (with-error-handling
     (let* ((opts  (parse-options))
            (store (open-connection))
            (url   (assoc-ref opts 'tarball-url)))
-      (let ((tarball (download-to-store store url "guix-latest.tar.gz")))
+      (let ((tarball
+             (if (use-gnutls? url)
+                 (begin
+                   ;; Add GnuTLS to inputs and load path.
+                   (set! %load-path
+                     (cons (string-append (package-output store gnutls)
+                                          "/share/guile/site/"
+                                          (effective-version))
+                           %load-path))
+                   (if (use-le-certs? url)
+                       (parameterize ((%x509-certificate-directory
+                                       (string-append (package-output store nss-certs)
+                                                      "/etc/ssl/certs")))
+                         (fetch-tarball store url))
+                       (fetch-tarball store url)))
+                 (fetch-tarball store url))))
         (unless tarball
           (leave (_ "failed to download up-to-date source, exiting\n")))
         (parameterize ((%guile-for-build
-- 
2.12.0

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sun, 5 Mar 2017 13:44:34 -0500

[Message part 1 (text/plain, inline)]

On Sun, Mar 05, 2017 at 03:59:16PM +0100, Marius Bakke wrote:
> Note that this uses 'nss-certs' for easy testing, but is intended to use
> 'le-certs' from this thread:
> 
> https://lists.gnu.org/archive/html/guix-devel/2017-02/msg01146.html

I am ready to prepare the le-certs package, but I am waiting for one
more Guix project member to reproduce the repository, as requested in
the message linked above. We should not use a custom certificate store
that has not been inspected by several people.

> From 6667ea5a2ec3a26dd5c4fb5f792485eeb941a969 Mon Sep 17 00:00:00 2001
> From: Marius Bakke <mbakke <at> fastmail.com>
> Date: Wed, 1 Mar 2017 22:11:02 +0100
> Subject: [PATCH] pull: Default to HTTPS.
> 
> * guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
> (guix-pull): Add GNUTLS and NSS-CERTS to inputs when appropriate.

It works for me! Like I said before, I'm hoping a stronger Schemer than
me will review it.

And we should think about how it might fail and try to work-around those
issues before anybody hits them in practice.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Kei Kebreau <kei <at> openmailbox.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sun, 05 Mar 2017 14:42:58 -0500

[Message part 1 (text/plain, inline)]

Leo Famulari <leo <at> famulari.name> writes:

> On Sun, Mar 05, 2017 at 03:59:16PM +0100, Marius Bakke wrote:
>> Note that this uses 'nss-certs' for easy testing, but is intended to use
>> 'le-certs' from this thread:
>> 
>> https://lists.gnu.org/archive/html/guix-devel/2017-02/msg01146.html
>
> I am ready to prepare the le-certs package, but I am waiting for one
> more Guix project member to reproduce the repository, as requested in
> the message linked above. We should not use a custom certificate store
> that has not been inspected by several people.
>

Reproduce the repository using

GIT_SSL_CAINFO="/tmp/le-certs/le-certs.pem" git clone --depth=1
https://git.savannah.gnu.org/git/guix.git?

If so, I just did successfully. If not, how can I help?

>> From 6667ea5a2ec3a26dd5c4fb5f792485eeb941a969 Mon Sep 17 00:00:00 2001
>> From: Marius Bakke <mbakke <at> fastmail.com>
>> Date: Wed, 1 Mar 2017 22:11:02 +0100
>> Subject: [PATCH] pull: Default to HTTPS.
>> 
>> * guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
>> (guix-pull): Add GNUTLS and NSS-CERTS to inputs when appropriate.
>
> It works for me! Like I said before, I'm hoping a stronger Schemer than
> me will review it.
>
> And we should think about how it might fail and try to work-around those
> issues before anybody hits them in practice.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Kei Kebreau <kei <at> openmailbox.org>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sun, 5 Mar 2017 15:31:44 -0500

[Message part 1 (text/plain, inline)]

On Sun, Mar 05, 2017 at 02:42:58PM -0500, Kei Kebreau wrote:
> Reproduce the repository using
> 
> GIT_SSL_CAINFO="/tmp/le-certs/le-certs.pem" git clone --depth=1
> https://git.savannah.gnu.org/git/guix.git?
> 
> If so, I just did successfully. If not, how can I help?

What I meant is that I'd like for people to try reproducing the contents
of the le-certs repository.

Basically, download the certificates and check that they match what I am
distributing here:

https://github.com/lfam/le-certs/commit/a2528f9be72aaaceb210d516e93151758108683f

If you try it, please send a signed email to that thread with your
results. For example, Marius did it here:

https://lists.gnu.org/archive/html/guix-devel/2017-02/msg01156.html

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Kei Kebreau <kei <at> openmailbox.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sun, 05 Mar 2017 17:23:19 -0500

[Message part 1 (text/plain, inline)]

Leo Famulari <leo <at> famulari.name> writes:

> On Sun, Mar 05, 2017 at 02:42:58PM -0500, Kei Kebreau wrote:
>> Reproduce the repository using
>> 
>> GIT_SSL_CAINFO="/tmp/le-certs/le-certs.pem" git clone --depth=1
>> https://git.savannah.gnu.org/git/guix.git?
>> 
>> If so, I just did successfully. If not, how can I help?
>
> What I meant is that I'd like for people to try reproducing the contents
> of the le-certs repository.
>
> Basically, download the certificates and check that they match what I am
> distributing here:
>
> https://github.com/lfam/le-certs/commit/a2528f9be72aaaceb210d516e93151758108683f
>
> If you try it, please send a signed email to that thread with your
> results. For example, Marius did it here:
>
> https://lists.gnu.org/archive/html/guix-devel/2017-02/msg01156.html

Here are my SHA256 checksums:

f8a8316dcc1f813774e7d7e2f85d7069d8b387c98a81b6073ef9f415be62410e  letsencryptauthorityx1.pem
3f67c48667781f7a7221320ee5b76c353aa4e0f4b2ed24a8a41113e6638f9724  letsencryptauthorityx2.pem
735a28bd5d93161769dd3a5d1d6337f24d1f2662cfe355930c1cffc38cac6a7d  letsencryptauthorityx3.pem
04f703429322d699af9e4d47e558cb696378fa20073700c9309263c448626d00  letsencryptauthorityx4.pem
6c0a324bb803e9d66b8986ea2085bb9d6bdfe33f5c04a03a3f7024f4aa8e7a2d  lets-encrypt-x1-cross-signed.pem
b5791649cc21518a9757d7e1809bc47c5e60edc45c9dddaaf6c060cbe03bcb1d  lets-encrypt-x2-cross-signed.pem
e446c5e9dbef9d09ac9f7027c034602492437a05ff6c40011d7235fca639c79a  lets-encrypt-x3-cross-signed.pem
f524491d9c2966c01ecec75c7803c7169ff46bc5cfd44c396d418cd7053d8015  lets-encrypt-x4-cross-signed.pem
3e6cf961c196c63a39bd99e5e34ff42c83669e3d7bcc2e4a0f9c7c7df40d0d7e  isrgrootx1.pem
139a5e4a4e0fa505378c72c5f700934ce8333f4e6b1b508886c4b0eb14f4be99  dstrootx3.pem
87db2e52cc9e03ed7fb79080545452f730bc5b8e38ace968f55753a612235d5f  le-certs.pem
b28d49140a5192c210a9d7fdc265807bdf256ae5a13d396477dac18a43f7fd2b  le-certs-dstrootx3.pem

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Kei Kebreau <kei <at> openmailbox.org>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Wed, 8 Mar 2017 15:46:54 -0500

[Message part 1 (text/plain, inline)]

On Sun, Mar 05, 2017 at 05:23:19PM -0500, Kei Kebreau wrote:
> Here are my SHA256 checksums:
> 
> f8a8316dcc1f813774e7d7e2f85d7069d8b387c98a81b6073ef9f415be62410e  letsencryptauthorityx1.pem
> 3f67c48667781f7a7221320ee5b76c353aa4e0f4b2ed24a8a41113e6638f9724  letsencryptauthorityx2.pem
> 735a28bd5d93161769dd3a5d1d6337f24d1f2662cfe355930c1cffc38cac6a7d  letsencryptauthorityx3.pem
> 04f703429322d699af9e4d47e558cb696378fa20073700c9309263c448626d00  letsencryptauthorityx4.pem
> 6c0a324bb803e9d66b8986ea2085bb9d6bdfe33f5c04a03a3f7024f4aa8e7a2d  lets-encrypt-x1-cross-signed.pem
> b5791649cc21518a9757d7e1809bc47c5e60edc45c9dddaaf6c060cbe03bcb1d  lets-encrypt-x2-cross-signed.pem
> e446c5e9dbef9d09ac9f7027c034602492437a05ff6c40011d7235fca639c79a  lets-encrypt-x3-cross-signed.pem
> f524491d9c2966c01ecec75c7803c7169ff46bc5cfd44c396d418cd7053d8015  lets-encrypt-x4-cross-signed.pem
> 3e6cf961c196c63a39bd99e5e34ff42c83669e3d7bcc2e4a0f9c7c7df40d0d7e  isrgrootx1.pem
> 139a5e4a4e0fa505378c72c5f700934ce8333f4e6b1b508886c4b0eb14f4be99  dstrootx3.pem
> 87db2e52cc9e03ed7fb79080545452f730bc5b8e38ace968f55753a612235d5f  le-certs.pem
> b28d49140a5192c210a9d7fdc265807bdf256ae5a13d396477dac18a43f7fd2b  le-certs-dstrootx3.pem

Thanks Kei!

Based on the review of the le-certs repository from Kei and Marius, I've
submitted an le-certs package for review:

http://lists.gnu.org/archive/html/guix-patches/2017-03/msg00180.html

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Wed, 08 Mar 2017 21:51:37 +0100

Hi Marius,

Marius Bakke <mbakke <at> fastmail.com> skribis:

> I've tried a number of times to send this through `git send-email`, but
> it seems to get caught in a spam filter or similar.
>
> Trying as attachment now.
>
> Note that this uses 'nss-certs' for easy testing, but is intended to use
> 'le-certs' from this thread:
>
> https://lists.gnu.org/archive/html/guix-devel/2017-02/msg01146.html

Cool.

> From 6667ea5a2ec3a26dd5c4fb5f792485eeb941a969 Mon Sep 17 00:00:00 2001
> From: Marius Bakke <mbakke <at> fastmail.com>
> Date: Wed, 1 Mar 2017 22:11:02 +0100
> Subject: [PATCH] pull: Default to HTTPS.
>
> * guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
> (guix-pull): Add GNUTLS and NSS-CERTS to inputs when appropriate.

[...]

>    (with-error-handling
>      (let* ((opts  (parse-options))
>             (store (open-connection))
>             (url   (assoc-ref opts 'tarball-url)))
> -      (let ((tarball (download-to-store store url "guix-latest.tar.gz")))
> +      (let ((tarball
> +             (if (use-gnutls? url)
> +                 (begin
> +                   ;; Add GnuTLS to inputs and load path.
> +                   (set! %load-path
> +                     (cons (string-append (package-output store gnutls)
> +                                          "/share/guile/site/"
> +                                          (effective-version))
> +                           %load-path))
> +                   (if (use-le-certs? url)
> +                       (parameterize ((%x509-certificate-directory
> +                                       (string-append (package-output store nss-certs)
> +                                                      "/etc/ssl/certs")))
> +                         (fetch-tarball store url))
> +                       (fetch-tarball store url)))
> +                 (fetch-tarball store url))))

This doesn’t really work, contrary to what you may experience.  ;-)

Namely, ‘package-output’ is risky because it returns the output file
name of a package but doesn’t ensure that the store item actually
exists.  So the above code works as intended when your store already
contains nss-certs and GnuTLS, but it breaks otherwise.

Instead we need to do something like this, though it’s not great either:

  (let* ((drv (package-derivation store nss-certs))
         (certs (string-append (derivation->output-path drv) "/etc/…")))
    (build-derivation store (list drv))  ;ugly: builds something right here
    …)

Another problem is changing ‘%load-path’ for the current process: this
will fail weirdly if GnuTLS is linked against a different libguile or
libc than the Guile executing ‘guix pull’.  We should refrain from doing
that and instead rely on the already install GnuTLS (I think we can
officially make it a hard requirement).

The code checks for ‘use-le-certs?’ but then uses all the NSS certs,
whereas the name implies something LE-specific.  Is that intended?  :-)

It’s also a case where I think we might want to use the
already-installed certificates.

Thoughts?

Thanks for working on it!

Ludo’.

Message #26 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Wed, 8 Mar 2017 16:11:58 -0500

On Wed, Mar 08, 2017 at 09:51:37PM +0100, Ludovic Courtès wrote:
> The code checks for ‘use-le-certs?’ but then uses all the NSS certs,
> whereas the name implies something LE-specific.  Is that intended?  :-)

There wasn't an LE certs package ready yet. I think we should avoid
depending on nss-certs for `guix pull` because nss-certs depends on
perl, python, and openssl.

Message #29 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Wed, 08 Mar 2017 23:49:54 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

>> From 6667ea5a2ec3a26dd5c4fb5f792485eeb941a969 Mon Sep 17 00:00:00 2001
>> From: Marius Bakke <mbakke <at> fastmail.com>
>> Date: Wed, 1 Mar 2017 22:11:02 +0100
>> Subject: [PATCH] pull: Default to HTTPS.
>>
>> * guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
>> (guix-pull): Add GNUTLS and NSS-CERTS to inputs when appropriate.
>
> [...]
>
>>    (with-error-handling
>>      (let* ((opts  (parse-options))
>>             (store (open-connection))
>>             (url   (assoc-ref opts 'tarball-url)))
>> -      (let ((tarball (download-to-store store url "guix-latest.tar.gz")))
>> +      (let ((tarball
>> +             (if (use-gnutls? url)
>> +                 (begin
>> +                   ;; Add GnuTLS to inputs and load path.
>> +                   (set! %load-path
>> +                     (cons (string-append (package-output store gnutls)
>> +                                          "/share/guile/site/"
>> +                                          (effective-version))
>> +                           %load-path))
>> +                   (if (use-le-certs? url)
>> +                       (parameterize ((%x509-certificate-directory
>> +                                       (string-append (package-output store nss-certs)
>> +                                                      "/etc/ssl/certs")))
>> +                         (fetch-tarball store url))
>> +                       (fetch-tarball store url)))
>> +                 (fetch-tarball store url))))
>
> This doesn’t really work, contrary to what you may experience.  ;-)
>
> Namely, ‘package-output’ is risky because it returns the output file
> name of a package but doesn’t ensure that the store item actually
> exists.  So the above code works as intended when your store already
> contains nss-certs and GnuTLS, but it breaks otherwise.

I suspected as much[0], but when I tested it with Leos "le-certs"
package that was not in my store, it actually got built by this code.
Not sure what's up with that.

[0] https://lists.gnu.org/archive/html/guix-devel/2017-02/msg01161.html

> Instead we need to do something like this, though it’s not great either:
>
>   (let* ((drv (package-derivation store nss-certs))
>          (certs (string-append (derivation->output-path drv) "/etc/…")))
>     (build-derivation store (list drv))  ;ugly: builds something right here
>     …)

I'll give this a go, thanks!

> Another problem is changing ‘%load-path’ for the current process: this
> will fail weirdly if GnuTLS is linked against a different libguile or
> libc than the Guile executing ‘guix pull’.  We should refrain from doing
> that and instead rely on the already install GnuTLS (I think we can
> officially make it a hard requirement).

What is the best way to do this? Simply propagate "gnutls" with "guix"?
The %load-path trick was stolen from (guix download), so I assumed it
was safe ;-)

> The code checks for ‘use-le-certs?’ but then uses all the NSS certs,
> whereas the name implies something LE-specific.  Is that intended?  :-)

That was for easier testing/review while waiting for the "le-certs"
package.

> It’s also a case where I think we might want to use the
> already-installed certificates.

If the URL is not from savannah, the GnuTLS defaults will be used (which
should consult SSL_CERT_DIR). Or did you mean instead of "le-certs"?

Thanks for the feedback!

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Thu, 09 Mar 2017 11:48:15 +0100

Hi Marius,

Marius Bakke <mbakke <at> fastmail.com> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>>> From 6667ea5a2ec3a26dd5c4fb5f792485eeb941a969 Mon Sep 17 00:00:00 2001
>>> From: Marius Bakke <mbakke <at> fastmail.com>
>>> Date: Wed, 1 Mar 2017 22:11:02 +0100
>>> Subject: [PATCH] pull: Default to HTTPS.
>>>
>>> * guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
>>> (guix-pull): Add GNUTLS and NSS-CERTS to inputs when appropriate.
>>
>> [...]
>>
>>>    (with-error-handling
>>>      (let* ((opts  (parse-options))
>>>             (store (open-connection))
>>>             (url   (assoc-ref opts 'tarball-url)))
>>> -      (let ((tarball (download-to-store store url "guix-latest.tar.gz")))
>>> +      (let ((tarball
>>> +             (if (use-gnutls? url)
>>> +                 (begin
>>> +                   ;; Add GnuTLS to inputs and load path.
>>> +                   (set! %load-path
>>> +                     (cons (string-append (package-output store gnutls)
>>> +                                          "/share/guile/site/"
>>> +                                          (effective-version))
>>> +                           %load-path))
>>> +                   (if (use-le-certs? url)
>>> +                       (parameterize ((%x509-certificate-directory
>>> +                                       (string-append (package-output store nss-certs)
>>> +                                                      "/etc/ssl/certs")))
>>> +                         (fetch-tarball store url))
>>> +                       (fetch-tarball store url)))
>>> +                 (fetch-tarball store url))))
>>
>> This doesn’t really work, contrary to what you may experience.  ;-)
>>
>> Namely, ‘package-output’ is risky because it returns the output file
>> name of a package but doesn’t ensure that the store item actually
>> exists.  So the above code works as intended when your store already
>> contains nss-certs and GnuTLS, but it breaks otherwise.
>
> I suspected as much[0], but when I tested it with Leos "le-certs"
> package that was not in my store, it actually got built by this code.
> Not sure what's up with that.
>
> [0] https://lists.gnu.org/archive/html/guix-devel/2017-02/msg01161.html

Weird.  ‘package-output’ definitely doesn’t build the thing.

>> Instead we need to do something like this, though it’s not great either:
>>
>>   (let* ((drv (package-derivation store nss-certs))
>>          (certs (string-append (derivation->output-path drv) "/etc/…")))
>>     (build-derivation store (list drv))  ;ugly: builds something right here
>>     …)
>
> I'll give this a go, thanks!
>
>> Another problem is changing ‘%load-path’ for the current process: this
>> will fail weirdly if GnuTLS is linked against a different libguile or
>> libc than the Guile executing ‘guix pull’.  We should refrain from doing
>> that and instead rely on the already install GnuTLS (I think we can
>> officially make it a hard requirement).
>
> What is the best way to do this?

Simply assume that GnuTLS is already available and thus do nothing.  :-)

> Simply propagate "gnutls" with "guix"?  The %load-path trick was
> stolen from (guix download), so I assumed it was safe ;-)

The gexp in (guix download) that does that is a different story: it’s a
situation where we spawn a new process and we know which Guile and which
GnuTLS package is being used, so it’s completely safe.

>> The code checks for ‘use-le-certs?’ but then uses all the NSS certs,
>> whereas the name implies something LE-specific.  Is that intended?  :-)
>
> That was for easier testing/review while waiting for the "le-certs"
> package.

OK.

>> It’s also a case where I think we might want to use the
>> already-installed certificates.
>
> If the URL is not from savannah, the GnuTLS defaults will be used (which
> should consult SSL_CERT_DIR). Or did you mean instead of "le-certs"?

Yes, I was talking about le-certs.

Thanks!

Ludo’.

Message #35 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Thu, 09 Mar 2017 16:46:56 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

>> Simply propagate "gnutls" with "guix"?  The %load-path trick was
>> stolen from (guix download), so I assumed it was safe ;-)
>
> The gexp in (guix download) that does that is a different story: it’s a
> situation where we spawn a new process and we know which Guile and which
> GnuTLS package is being used, so it’s completely safe.

Ok, thanks for the clarification. Can we do the same in "pull"? IMO it
should work without intervention, when the user runs `guix pull` for the
first time.

On foreign distributions, the `guix` executable is often a symlink to
the profile of the "root" user. In such cases, each user would have to
install GnuTLS in their profile before `guix pull` works.

>>> It’s also a case where I think we might want to use the
>>> already-installed certificates.
>>
>> If the URL is not from savannah, the GnuTLS defaults will be used (which
>> should consult SSL_CERT_DIR). Or did you mean instead of "le-certs"?
>
> Yes, I was talking about le-certs.

I have a strong preference for "hard coding" le-certs here. If the user
don't have certs in their profile, they would have to both install some
*and* configure the SSL_CERT_DIR variable before `guix pull` works.

Using "le-certs" instead of one of the "kitchen sink" trust stores such
as "nss-certs" also provides strong guarantees against MITM attacks even
from state-level actors due to the certificate transparency program.

LE's signing certificate expires in 2021 and the root in 2035, so it's
not a huge maintenance overhead.

[signature.asc (application/pgp-signature, inline)]

Message #38 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Thu, 09 Mar 2017 17:11:44 +0100

Marius Bakke <mbakke <at> fastmail.com> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>>> Simply propagate "gnutls" with "guix"?  The %load-path trick was
>>> stolen from (guix download), so I assumed it was safe ;-)
>>
>> The gexp in (guix download) that does that is a different story: it’s a
>> situation where we spawn a new process and we know which Guile and which
>> GnuTLS package is being used, so it’s completely safe.
>
> Ok, thanks for the clarification. Can we do the same in "pull"?

No, but we don’t have to: just assume GnuTLS is available and use the
https URL unconditionally.

We’ll update configure.ac and the manual to mention that GnuTLS is now a
requirement.

Sounds good?

>>>> It’s also a case where I think we might want to use the
>>>> already-installed certificates.
>>>
>>> If the URL is not from savannah, the GnuTLS defaults will be used (which
>>> should consult SSL_CERT_DIR). Or did you mean instead of "le-certs"?
>>
>> Yes, I was talking about le-certs.
>
> I have a strong preference for "hard coding" le-certs here. If the user
> don't have certs in their profile, they would have to both install some
> *and* configure the SSL_CERT_DIR variable before `guix pull` works.
>
> Using "le-certs" instead of one of the "kitchen sink" trust stores such
> as "nss-certs" also provides strong guarantees against MITM attacks even
> from state-level actors due to the certificate transparency program.
>
> LE's signing certificate expires in 2021 and the root in 2035, so it's
> not a huge maintenance overhead.

OK, that makes sense to me.  So you can add a ‘build-derivations’ call
for this package and that should be enough.

Or we could just as well ship the LE certificate instead of having a
package that downloads it etc.?

Thank you!

Ludo’.

Message #41 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Thu, 9 Mar 2017 13:13:21 -0500

[Message part 1 (text/plain, inline)]

On Thu, Mar 09, 2017 at 05:11:44PM +0100, Ludovic Courtès wrote:
> Or we could just as well ship the LE certificate instead of having a
> package that downloads it etc.?

I thought about this a bit yesterday. Only three certificate files are
needed for the Let's Encrypt certificate store: the root certificate,
the active intermediate, and the backup intermediate.*

We know where they can be downloaded from, and we know their SHA256
hash, so we could download them directly instead of using a package.

We could also bundle them with Guix, as you suggest.

What does everyone think?

* Technically we could leave out the backup, but I think we should
include it so that everything is "smooth" whenever it needs to become
active.

[signature.asc (application/pgp-signature, inline)]

Message #44 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Leo Famulari <leo <at> famulari.name>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Fri, 10 Mar 2017 11:33:41 +0100

Leo Famulari <leo <at> famulari.name> skribis:

> On Thu, Mar 09, 2017 at 05:11:44PM +0100, Ludovic Courtès wrote:
>> Or we could just as well ship the LE certificate instead of having a
>> package that downloads it etc.?
>
> I thought about this a bit yesterday. Only three certificate files are
> needed for the Let's Encrypt certificate store: the root certificate,
> the active intermediate, and the backup intermediate.*
>
> We know where they can be downloaded from, and we know their SHA256
> hash, so we could download them directly instead of using a package.
>
> We could also bundle them with Guix, as you suggest.
>
> What does everyone think?

Maybe a trivial-build-system package to download these 3 files and put
them in a directory would do.

Thoughts?

Ludo’.

Message #47 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 25975 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Fri, 10 Mar 2017 13:19:09 +0000

Ludovic Courtès transcribed 0.8K bytes:
> Leo Famulari <leo <at> famulari.name> skribis:
> 
> > On Thu, Mar 09, 2017 at 05:11:44PM +0100, Ludovic Courtès wrote:
> >> Or we could just as well ship the LE certificate instead of having a
> >> package that downloads it etc.?
> >
> > I thought about this a bit yesterday. Only three certificate files are
> > needed for the Let's Encrypt certificate store: the root certificate,
> > the active intermediate, and the backup intermediate.*
> >
> > We know where they can be downloaded from, and we know their SHA256
> > hash, so we could download them directly instead of using a package.
> >
> > We could also bundle them with Guix, as you suggest.
> >
> > What does everyone think?
> 
> Maybe a trivial-build-system package to download these 3 files and put
> them in a directory would do.
> 
> Thoughts?
> 
> Ludo’.
> 
> 

Sounds like a good idea.

Message #50 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sat, 11 Mar 2017 01:42:48 -0500

[Message part 1 (text/plain, inline)]

On Fri, Mar 10, 2017 at 11:33:41AM +0100, Ludovic Courtès wrote:
> > I thought about this a bit yesterday. Only three certificate files are
> > needed for the Let's Encrypt certificate store: the root certificate,
> > the active intermediate, and the backup intermediate.*
> >
> > We know where they can be downloaded from, and we know their SHA256
> > hash, so we could download them directly instead of using a package.
> 
> Maybe a trivial-build-system package to download these 3 files and put
> them in a directory would do.

Here's a patch.

[0001-gnu-Add-le-certs.patch (text/plain, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #53 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Leo Famulari <leo <at> famulari.name>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sat, 11 Mar 2017 11:26:12 +0100

Leo Famulari <leo <at> famulari.name> skribis:

> On Fri, Mar 10, 2017 at 11:33:41AM +0100, Ludovic Courtès wrote:
>> > I thought about this a bit yesterday. Only three certificate files are
>> > needed for the Let's Encrypt certificate store: the root certificate,
>> > the active intermediate, and the backup intermediate.*
>> >
>> > We know where they can be downloaded from, and we know their SHA256
>> > hash, so we could download them directly instead of using a package.
>> 
>> Maybe a trivial-build-system package to download these 3 files and put
>> them in a directory would do.
>
> Here's a patch.
>
> From 6fea91135a625a13d92d6951d150d8dc5eb68dc1 Mon Sep 17 00:00:00 2001
> From: Leo Famulari <leo <at> famulari.name>
> Date: Tue, 28 Feb 2017 20:21:10 -0500
> Subject: [PATCH] gnu: Add le-certs.
>
> * gnu/packages/certs.scm (le-certs): New variable.

Great, that was fast!

> +    (arguments
> +     '(#:modules ((guix build utils))
> +       #:builder
> +       (begin
> +         (use-modules (guix build utils))
> +         (let ((root (assoc-ref %build-inputs "isrgrootx1.pem"))
> +               (intermediate (assoc-ref %build-inputs "letsencryptauthorityx3.pem"))
> +               (backup (assoc-ref %build-inputs "letsencryptauthorityx4.pem"))
> +               (out (string-append (assoc-ref %outputs "out") "/etc/ssl/certs")))
> +           (mkdir-p out)
> +           (map (lambda (cert)
> +                  (copy-file cert (string-append out "/"
> +                                                 (strip-store-file-name cert))))
> +                (list root intermediate backup))))))

‘for-each’ instead of ‘map’, to make it clear that it’s for side
effects.

> +    (license license:public-domain))) ; XXX what license?

It’s not copyrightable so yeah, this is a good approximation.

Thank you!

Ludo’.

Message #56 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sat, 11 Mar 2017 11:57:37 -0500

[Message part 1 (text/plain, inline)]

On Sat, Mar 11, 2017 at 11:26:12AM +0100, Ludovic Courtès wrote:
> > +           (map (lambda (cert)
> > +                  (copy-file cert (string-append out "/"
> > +                                                 (strip-store-file-name cert))))
> > +                (list root intermediate backup))))))
> 
> ‘for-each’ instead of ‘map’, to make it clear that it’s for side
> effects.

Done!

> > +    (license license:public-domain))) ; XXX what license?
> 
> It’s not copyrightable so yeah, this is a good approximation.

Okay!

I also changed the home-page to <https://letsencrypt.org/certificates/>.

Updated patch attached.

[0001-gnu-Add-le-certs.patch (text/plain, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #59 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: Leo Famulari <leo <at> famulari.name>, Ludovic Courtès
 <ludo <at> gnu.org>
Cc: 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sat, 11 Mar 2017 19:40:38 +0100

[Message part 1 (text/plain, inline)]

Leo Famulari <leo <at> famulari.name> writes:

> On Sat, Mar 11, 2017 at 11:26:12AM +0100, Ludovic Courtès wrote:
>> > +           (map (lambda (cert)
>> > +                  (copy-file cert (string-append out "/"
>> > +                                                 (strip-store-file-name cert))))
>> > +                (list root intermediate backup))))))
>> 
>> ‘for-each’ instead of ‘map’, to make it clear that it’s for side
>> effects.
>
> Done!
>
>> > +    (license license:public-domain))) ; XXX what license?
>> 
>> It’s not copyrightable so yeah, this is a good approximation.
>
> Okay!
>
> I also changed the home-page to <https://letsencrypt.org/certificates/>.
>
> Updated patch attached.

Great! Here's a revision of `guix pull` that uses this "le-certs"
package, and assumes GnuTLS is available.

One caveat with this approach is that users will need to install GnuTLS
in their profile and set up GUILE_LOAD_PATH before `guix pull` works.

[0001-pull-Default-to-HTTPS.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #62 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Leo Famulari <leo <at> famulari.name>
Cc: Marius Bakke <mbakke <at> fastmail.com>, 25975 <at> debbugs.gnu.org
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sun, 12 Mar 2017 00:29:31 +0100

Leo Famulari <leo <at> famulari.name> skribis:

> From a02327a3d8f3ccc0c87920870671f0500b13d430 Mon Sep 17 00:00:00 2001
> From: Leo Famulari <leo <at> famulari.name>
> Date: Tue, 28 Feb 2017 20:21:10 -0500
> Subject: [PATCH] gnu: Add le-certs.
>
> * gnu/packages/certs.scm (le-certs): New variable.

Perfect, thanks!

Ludo’.

Message #65 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 25975 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sun, 12 Mar 2017 00:41:35 +0100

Marius Bakke <mbakke <at> fastmail.com> skribis:

> From 61bf52ff461e8a53175546928bd4ee41645bb5ca Mon Sep 17 00:00:00 2001
> From: Marius Bakke <mbakke <at> fastmail.com>
> Date: Wed, 1 Mar 2017 22:11:02 +0100
> Subject: [PATCH] pull: Default to HTTPS.
>
> * guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
> (guix-pull): Authenticate against LE-CERTS when URL is from Savannah.

LGTM!

I changed the configury and doc in
1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b so that GnuTLS is a hard
dependency.

Thanks,
Ludo’.

Message #70 received at 25975-done <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 25975-done <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sun, 12 Mar 2017 19:48:42 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

> Marius Bakke <mbakke <at> fastmail.com> skribis:
>
>> From 61bf52ff461e8a53175546928bd4ee41645bb5ca Mon Sep 17 00:00:00 2001
>> From: Marius Bakke <mbakke <at> fastmail.com>
>> Date: Wed, 1 Mar 2017 22:11:02 +0100
>> Subject: [PATCH] pull: Default to HTTPS.
>>
>> * guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
>> (guix-pull): Authenticate against LE-CERTS when URL is from Savannah.
>
> LGTM!
>
> I changed the configury and doc in
> 1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b so that GnuTLS is a hard
> dependency.

Ok, thanks!

I've pushed this patch. Let the bug reports roll in! :-)

[signature.asc (application/pgp-signature, inline)]

Message #73 received at 25975-done <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 25975-done <at> debbugs.gnu.org,
 Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Sun, 12 Mar 2017 22:35:05 -0400

[Message part 1 (text/plain, inline)]

On Sun, Mar 12, 2017 at 07:48:42PM +0100, Marius Bakke wrote:
> I've pushed this patch. Let the bug reports roll in! :-)

Woo-hoo!

It "just worked" for me on GuixSD and my foreign distros :)

[signature.asc (application/pgp-signature, inline)]

Message #76 received at 25975-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Leo Famulari <leo <at> famulari.name>
Cc: 25975-done <at> debbugs.gnu.org, Marius Bakke <mbakke <at> fastmail.com>
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Mon, 13 Mar 2017 10:23:45 +0100

Leo Famulari <leo <at> famulari.name> skribis:

> On Sun, Mar 12, 2017 at 07:48:42PM +0100, Marius Bakke wrote:
>> I've pushed this patch. Let the bug reports roll in! :-)
>
> Woo-hoo!
>
> It "just worked" for me on GuixSD and my foreign distros :)

Works for me too!  Thanks to both of you for carrying it out!

Ludo’.

Message #79 received at 25975 <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: 25975 <at> debbugs.gnu.org, mbakke <at> fastmail.com
Subject: Re: bug#25975: Use HTTPS in `guix pull`
Date: Mon, 13 Mar 2017 11:07:10 +0000

Marius Bakke transcribed 1.3K bytes:
> Ludovic Courtès <ludo <at> gnu.org> writes:
> 
> > Marius Bakke <mbakke <at> fastmail.com> skribis:
> >
> >> From 61bf52ff461e8a53175546928bd4ee41645bb5ca Mon Sep 17 00:00:00 2001
> >> From: Marius Bakke <mbakke <at> fastmail.com>
> >> Date: Wed, 1 Mar 2017 22:11:02 +0100
> >> Subject: [PATCH] pull: Default to HTTPS.
> >>
> >> * guix/scripts/pull.scm (%snapshot-url): Use HTTPS.
> >> (guix-pull): Authenticate against LE-CERTS when URL is from Savannah.
> >
> > LGTM!
> >
> > I changed the configury and doc in
> > 1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b so that GnuTLS is a hard
> > dependency.
> 
> Ok, thanks!
> 
> I've pushed this patch. Let the bug reports roll in! :-)

Ok, native GuixSD worked just fine.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.