GNU bug report logs - #25831
Expose http_proxy setting on GuixSD

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 25831 in the body.
You can then email your comments to 25831 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: bug-guix <at> gnu.org
Cc: ng0 <contact.ng0 <at> cryptolab.net>
Subject: Expose http_proxy setting on GuixSD
Date: Tue, 21 Feb 2017 12:19:12 -0500

----- Forwarded message from ng0 <contact.ng0 <at> cryptolab.net> -----

Date: Tue, 21 Feb 2017 09:43:35 +0000
From: ng0 <contact.ng0 <at> cryptolab.net>
To: Leo Famulari <leo <at> famulari.name>
Cc: help-guix <at> gnu.org
Subject: Re: Proxy documentation is not clear

On 17-02-20 22:32:40, Leo Famulari wrote:
> On Sun, Feb 19, 2017 at 05:06:23PM +0000, ng0 wrote:
> > Substitutes are downloaded over HTTP or HTTPS. The http_proxy
> > environment variable can be set in the environment of guix-daemon and
> > is honored for downloads of substitutes. Note that the value of
> > http_proxy in the environment where guix build, guix package, and
> > other client commands are run has absolutely no effect. 
> > 
> > None of these pages provides any example or further explanation on how
> > this could be achieved, for example to make Guix use a socks5 proxy
> > systemwide. I know how this can be achieved with iptables etc, but it
> > would be better if the documentation is fixed. I have no idea how in
> > this case as the documentation is not clear.
> 
> It depends on how you start the daemon. For systemd, you can set the
> environment variable from the service file. I'm not sure how to pass
> this variable to the daemon on GuixSD.

Exactly, for systemd I would have been able to find it out, but I wonder
how to achieve this in GuixSD. 

----- End forwarded message -----

Message #8 received at 25831 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Lirzin <mthl <at> gnu.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 25831 <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Tue, 21 Feb 2017 19:17:03 +0100

Hello,

Leo Famulari <leo <at> famulari.name> writes:

> From: ng0 <contact.ng0 <at> cryptolab.net>
> Subject: Re: Proxy documentation is not clear
> To: Leo Famulari <leo <at> famulari.name>
> Cc: help-guix <at> gnu.org
> Date: Tue, 21 Feb 2017 09:43:35 +0000 (8 hours, 8 minutes, 14 seconds ago)
>
> On 17-02-20 22:32:40, Leo Famulari wrote:
>> On Sun, Feb 19, 2017 at 05:06:23PM +0000, ng0 wrote:
>> > Substitutes are downloaded over HTTP or HTTPS. The http_proxy
>> > environment variable can be set in the environment of guix-daemon and
>> > is honored for downloads of substitutes. Note that the value of
>> > http_proxy in the environment where guix build, guix package, and
>> > other client commands are run has absolutely no effect. 
>> > 
>> > None of these pages provides any example or further explanation on how
>> > this could be achieved, for example to make Guix use a socks5 proxy
>> > systemwide. I know how this can be achieved with iptables etc, but it
>> > would be better if the documentation is fixed. I have no idea how in
>> > this case as the documentation is not clear.
>> 
>> It depends on how you start the daemon. For systemd, you can set the
>> environment variable from the service file. I'm not sure how to pass
>> this variable to the daemon on GuixSD.
>
> Exactly, for systemd I would have been able to find it out, but I wonder
> how to achieve this in GuixSD. 
>
> ----------
>

Not sure if this is an appropriate answer, but to customize the guix daemon
service field like described in the example here:

  https://www.gnu.org/software/guix/manual/html_node/Using-the-Configuration-System.html#System-Services

you would have to patch "gnu/services/base.scm" by adding a field to the
'guix-configuration' that would be then be passed to the
'#:environment-variables' option of 'make-forkexec-constructor' in
'guix-shepherd-service'.

HTH.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Message #11 received at 25831 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Mathieu Lirzin <mthl <at> gnu.org>
Cc: 25831 <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Tue, 21 Feb 2017 15:26:54 -0500

[Message part 1 (text/plain, inline)]

On Tue, Feb 21, 2017 at 07:17:03PM +0100, Mathieu Lirzin wrote:
> Not sure if this is an appropriate answer, but to customize the guix daemon
> service field like described in the example here:
> 
>   https://www.gnu.org/software/guix/manual/html_node/Using-the-Configuration-System.html#System-Services
> 
> you would have to patch "gnu/services/base.scm" by adding a field to the
> 'guix-configuration' that would be then be passed to the
> '#:environment-variables' option of 'make-forkexec-constructor' in
> 'guix-shepherd-service'.

Thank you, that helped a lot!

What do the two of you think of the attached patch?

Using the attached OS declaration, I verified that the http_proxy
environment variable is set in the guix-daemon's environment, but I
don't have a proxy to test with.

[0001-services-guix-Support-using-an-HTTP-proxy.patch (text/plain, attachment)]

[os-config-bare-bones.scm (text/plain, attachment)]

Message #14 received at 25831 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Lirzin <mthl <at> gnu.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 25831 <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Wed, 22 Feb 2017 10:51:47 +0100

Hi,

Leo Famulari <leo <at> famulari.name> writes:

> On Tue, Feb 21, 2017 at 07:17:03PM +0100, Mathieu Lirzin wrote:
>> Not sure if this is an appropriate answer, but to customize the guix daemon
>> service field like described in the example here:
>> 
>>   https://www.gnu.org/software/guix/manual/html_node/Using-the-Configuration-System.html#System-Services
>> 
>> you would have to patch "gnu/services/base.scm" by adding a field to the
>> 'guix-configuration' that would be then be passed to the
>> '#:environment-variables' option of 'make-forkexec-constructor' in
>> 'guix-shepherd-service'.
>
> Thank you, that helped a lot!
>
> What do the two of you think of the attached patch?
>
> Using the attached OS declaration, I verified that the http_proxy
> environment variable is set in the guix-daemon's environment, but I
> don't have a proxy to test with.
>
> From 1e6e95ea5b43231d09279b9e54f5c581462486f6 Mon Sep 17 00:00:00 2001
> From: Leo Famulari <leo <at> famulari.name>
> Date: Tue, 21 Feb 2017 14:57:02 -0500
> Subject: [PATCH] services: guix: Support using an HTTP proxy.
>
> * gnu/services/base.scm (<guix-configuration>)[http-proxy]: New field.
> (guix-shepherd-service): Use 'http-proxy' in #:environment-variables.
> * doc/guix.texi ...
> ---
>  doc/guix.texi         |  4 ++++
>  gnu/services/base.scm | 13 +++++++++----
>  2 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 6cdb5e592..19a31c659 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -8611,6 +8611,10 @@ are written.
>  @item @code{lsof} (default: @var{lsof})
>  The lsof package to use.
>  
> +@item @code{http-proxy} (default: @code{#f})
> +The HTTP proxy used for downloading fixed-output derivations and
> +substitutes.
> +

I don't know much about HTTP proxying but when reading
'in-band-download' in "guix/download.scm"  I see a line like this:

  #:leaked-env-vars '("http_proxy" "https_proxy")

which makes me think that maybe we should add an https_proxy toggle in
'guix-configuration' too?

However I see that 'open-connection-for-uri' in "guix/download.scm"
ignores "https_proxy" with a comment from 2015:

  ;; For HTTPS URIs, honor 'https_proxy', not 'http_proxy'.
  ;; FIXME: Proxying is not supported for https.

Do you have any idea if this could be easily fixed?

>  @end table
>  @end deftp
>  
> diff --git a/gnu/services/base.scm b/gnu/services/base.scm
> index 57601eab8..352a90be7 100644
> --- a/gnu/services/base.scm
> +++ b/gnu/services/base.scm
> @@ -3,7 +3,7 @@
>  ;;; Copyright © 2015, 2016 Alex Kost <alezost <at> gmail.com>
>  ;;; Copyright © 2015, 2016 Mark H Weaver <mhw <at> netris.org>
>  ;;; Copyright © 2015 Sou Bunnbu <iyzsong <at> gmail.com>
> -;;; Copyright © 2016 Leo Famulari <leo <at> famulari.name>
> +;;; Copyright © 2016, 2017 Leo Famulari <leo <at> famulari.name>
>  ;;; Copyright © 2016 David Craven <david <at> craven.ch>
>  ;;; Copyright © 2016 Ricardo Wurmus <rekado <at> elephly.net>
>  ;;;
> @@ -1114,7 +1114,9 @@ failed to register hydra.gnu.org public key: ~a~%" status))))))))
>    (log-file         guix-configuration-log-file   ;string
>                      (default "/var/log/guix-daemon.log"))
>    (lsof             guix-configuration-lsof       ;<package>
> -                    (default lsof)))
> +                    (default lsof))
> +  (http-proxy       guix-http-proxy               ;string
> +                    (default #f)))

'#f' is not a string.  So either specify ';string | #f' or make the
default value '""'.

>  
>  (define %default-guix-configuration
>    (guix-configuration))
> @@ -1125,7 +1127,7 @@ failed to register hydra.gnu.org public key: ~a~%" status))))))))
>      (($ <guix-configuration> guix build-group build-accounts
>                               authorize-key? keys
>                               use-substitutes? substitute-urls extra-options
> -                             log-file lsof)
> +                             log-file lsof http-proxy)
>       (list (shepherd-service
>              (documentation "Run the Guix daemon.")
>              (provision '(guix-daemon))
> @@ -1142,7 +1144,10 @@ failed to register hydra.gnu.org public key: ~a~%" status))))))))
>  
>                  ;; Add 'lsof' (for the GC) to the daemon's $PATH.
>                  #:environment-variables
> -                (list (string-append "PATH=" #$lsof "/bin"))
> +                (list (string-append "PATH=" #$lsof "/bin")
> +                      #$@(if http-proxy
> +                           #~((string-append"http_proxy=" #$http-proxy))
> +                           #~()))

I think it would be more readable like this:

  #$@(if http-proxy
         (list (string-append "http_proxy=" http-proxy))
         '())

or if '""' is the default value:

  #$@(if (string-null? http-proxy)
         '()
         (list (string-append "http_proxy=" http-proxy)))

Thanks.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Message #17 received at 25831 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Mathieu Lirzin <mthl <at> gnu.org>
Cc: 25831 <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Wed, 22 Feb 2017 14:22:23 -0500

[Message part 1 (text/plain, inline)]

On Wed, Feb 22, 2017 at 10:51:47AM +0100, Mathieu Lirzin wrote:
> I don't know much about HTTP proxying but when reading
> 'in-band-download' in "guix/download.scm"  I see a line like this:
> 
>   #:leaked-env-vars '("http_proxy" "https_proxy")
> 
> which makes me think that maybe we should add an https_proxy toggle in
> 'guix-configuration' too?
> 
> However I see that 'open-connection-for-uri' in "guix/download.scm"
> ignores "https_proxy" with a comment from 2015:
> 
>   ;; For HTTPS URIs, honor 'https_proxy', not 'http_proxy'.
>   ;; FIXME: Proxying is not supported for https.

That's from 'guix/build/download.scm'.

> Do you have any idea if this could be easily fixed?

I don't know. I guess that it would be required to have a TLS client
(gnutls-guile) and a certificate store available, but I'm not sure how
to implement it, or what exactly is missing.

> I think it would be more readable like this:
> 
>   #$@(if http-proxy
>          (list (string-append "http_proxy=" http-proxy))
>          '())

I think this form of the conditional is a little easier to read than the
other one, so I've used it in my updated patch, which is attached.

[0001-services-guix-Support-using-an-HTTP-proxy.patch (text/plain, attachment)]

Message #20 received at 25831 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Lirzin <mthl <at> gnu.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 25831 <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Wed, 22 Feb 2017 21:15:12 +0100

Leo Famulari <leo <at> famulari.name> writes:

> On Wed, Feb 22, 2017 at 10:51:47AM +0100, Mathieu Lirzin wrote:
>> I don't know much about HTTP proxying but when reading
>> 'in-band-download' in "guix/download.scm"  I see a line like this:
>> 
>>   #:leaked-env-vars '("http_proxy" "https_proxy")
>> 
>> which makes me think that maybe we should add an https_proxy toggle in
>> 'guix-configuration' too?
>> 
>> However I see that 'open-connection-for-uri' in "guix/download.scm"
>> ignores "https_proxy" with a comment from 2015:
>> 
>>   ;; For HTTPS URIs, honor 'https_proxy', not 'http_proxy'.
>>   ;; FIXME: Proxying is not supported for https.
>
> That's from 'guix/build/download.scm'.

Oops sorry about that.

>> Do you have any idea if this could be easily fixed?
>
> I don't know. I guess that it would be required to have a TLS client
> (gnutls-guile) and a certificate store available, but I'm not sure how
> to implement it, or what exactly is missing.

OK, hopefully Ludo will be able to bring more context about the code he
wrote.

>> I think it would be more readable like this:
>> 
>>   #$@(if http-proxy
>>          (list (string-append "http_proxy=" http-proxy))
>>          '())
>
> I think this form of the conditional is a little easier to read than the
> other one, so I've used it in my updated patch, which is attached.
>
> From ae8fd98c4a8f985ba835bfeab7f008c594508b2d Mon Sep 17 00:00:00 2001
> From: Leo Famulari <leo <at> famulari.name>
> Date: Tue, 21 Feb 2017 14:57:02 -0500
> Subject: [PATCH] services: guix: Support using an HTTP proxy.
>
> * gnu/services/base.scm (<guix-configuration>)[http-proxy]: New field.
> (guix-shepherd-service): Use 'http-proxy' in #:environment-variables.
> * doc/guix.texi (Base Services)[guix-configuration]: Document it.
> ---
>  doc/guix.texi         |  4 ++++
>  gnu/services/base.scm | 13 +++++++++----
>  2 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 6cdb5e592..19a31c659 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -8611,6 +8611,10 @@ are written.
>  @item @code{lsof} (default: @var{lsof})
>  The lsof package to use.
>  
> +@item @code{http-proxy} (default: @code{#f})
> +The HTTP proxy used for downloading fixed-output derivations and
> +substitutes.
> +
>  @end table
>  @end deftp
>  
> diff --git a/gnu/services/base.scm b/gnu/services/base.scm
> index 57601eab8..20eb19930 100644
> --- a/gnu/services/base.scm
> +++ b/gnu/services/base.scm
> @@ -3,7 +3,7 @@
>  ;;; Copyright © 2015, 2016 Alex Kost <alezost <at> gmail.com>
>  ;;; Copyright © 2015, 2016 Mark H Weaver <mhw <at> netris.org>
>  ;;; Copyright © 2015 Sou Bunnbu <iyzsong <at> gmail.com>
> -;;; Copyright © 2016 Leo Famulari <leo <at> famulari.name>
> +;;; Copyright © 2016, 2017 Leo Famulari <leo <at> famulari.name>
>  ;;; Copyright © 2016 David Craven <david <at> craven.ch>
>  ;;; Copyright © 2016 Ricardo Wurmus <rekado <at> elephly.net>
>  ;;;
> @@ -1114,7 +1114,9 @@ failed to register hydra.gnu.org public key: ~a~%" status))))))))
>    (log-file         guix-configuration-log-file   ;string
>                      (default "/var/log/guix-daemon.log"))
>    (lsof             guix-configuration-lsof       ;<package>
> -                    (default lsof)))
> +                    (default lsof))
> +  (http-proxy       guix-http-proxy               ;string | #f
> +                    (default #f)))
>  
>  (define %default-guix-configuration
>    (guix-configuration))
> @@ -1125,7 +1127,7 @@ failed to register hydra.gnu.org public key: ~a~%" status))))))))
>      (($ <guix-configuration> guix build-group build-accounts
>                               authorize-key? keys
>                               use-substitutes? substitute-urls extra-options
> -                             log-file lsof)
> +                             log-file lsof http-proxy)
>       (list (shepherd-service
>              (documentation "Run the Guix daemon.")
>              (provision '(guix-daemon))
> @@ -1142,7 +1144,10 @@ failed to register hydra.gnu.org public key: ~a~%" status))))))))
>  
>                  ;; Add 'lsof' (for the GC) to the daemon's $PATH.
>                  #:environment-variables
> -                (list (string-append "PATH=" #$lsof "/bin"))
> +                (list (string-append "PATH=" #$lsof "/bin")
> +                      #$@(if http-proxy
> +                           (list (string-append "http_proxy=" http-proxy))
> +                           '()))
                            ^^
                            missin two spaces in THEN and ELSE part ;)

>                  #:log-file #$log-file))
>              (stop #~(make-kill-destructor)))))))

Otherwise, LGTM.

Thanks.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Message #25 received at 25831-done <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Mathieu Lirzin <mthl <at> gnu.org>
Cc: ng0 <contact.ng0 <at> cryptolab.net>, 25831-done <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Wed, 22 Feb 2017 15:26:31 -0500

On Wed, Feb 22, 2017 at 09:15:12PM +0100, Mathieu Lirzin wrote:
> Leo Famulari <leo <at> famulari.name> writes:
> > +                (list (string-append "PATH=" #$lsof "/bin")
> > +                      #$@(if http-proxy
> > +                           (list (string-append "http_proxy=" http-proxy))
> > +                           '()))
>                             ^^
>                             missin two spaces in THEN and ELSE part ;)

I'm still learning what looks right :)

> Otherwise, LGTM.

Thanks for your advice and review!

I pushed the change as 93d32da9f8bba815b1effe6358904b5e1f9c73ff.

Message #28 received at 25831-done <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: Leo Famulari <leo <at> famulari.name>
Cc: Mathieu Lirzin <mthl <at> gnu.org>, 25831-done <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Thu, 23 Feb 2017 16:20:38 +0000

On 17-02-22 15:26:31, Leo Famulari wrote:
> On Wed, Feb 22, 2017 at 09:15:12PM +0100, Mathieu Lirzin wrote:
> > Leo Famulari <leo <at> famulari.name> writes:
> > > +                (list (string-append "PATH=" #$lsof "/bin")
> > > +                      #$@(if http-proxy
> > > +                           (list (string-append "http_proxy=" http-proxy))
> > > +                           '()))
> >                             ^^
> >                             missin two spaces in THEN and ELSE part ;)
> 
> I'm still learning what looks right :)
> 
> > Otherwise, LGTM.
> 
> Thanks for your advice and review!
> 
> I pushed the change as 93d32da9f8bba815b1effe6358904b5e1f9c73ff.

Thanks!
I was hoping this change (added 127.0.0.1:9050) in a test config would
pick up the .onion of bayfront I have in there, but it didn't happen. In
the comments you mentioned https_proxy is at some point favored over
http_proxy, so currently my only option is firewall rules then (which I
will need anyway).

Message #31 received at 25831-done <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Mathieu Lirzin <mthl <at> gnu.org>, 25831-done <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Thu, 23 Feb 2017 12:05:45 -0500

On Thu, Feb 23, 2017 at 04:20:38PM +0000, ng0 wrote:
> Thanks!
> I was hoping this change (added 127.0.0.1:9050) in a test config would
> pick up the .onion of bayfront I have in there, but it didn't happen.

Does it work on a system besides GuixSD?

> In the comments you mentioned https_proxy is at some point favored
> over http_proxy, so currently my only option is firewall rules then
> (which I will need anyway).

I'm not sure which comment you are referring to. For now, https_proxy is
not supported, but I would like it to be implemented as soon as
possible.

Message #34 received at 25831-done <at> debbugs.gnu.org (full text, mbox):

From: ng0 <contact.ng0 <at> cryptolab.net>
To: Leo Famulari <leo <at> famulari.name>
Cc: Mathieu Lirzin <mthl <at> gnu.org>, 25831-done <at> debbugs.gnu.org
Subject: Re: bug#25831: Expose http_proxy setting on GuixSD
Date: Thu, 23 Feb 2017 17:50:21 +0000

On 17-02-23 12:05:45, Leo Famulari wrote:
> On Thu, Feb 23, 2017 at 04:20:38PM +0000, ng0 wrote:
> > Thanks!
> > I was hoping this change (added 127.0.0.1:9050) in a test config would
> > pick up the .onion of bayfront I have in there, but it didn't happen.
> 
> Does it work on a system besides GuixSD?

I will report once I have the option to test it. This can take some
time.
 
> > In the comments you mentioned https_proxy is at some point favored
> > over http_proxy, so currently my only option is firewall rules then
> > (which I will need anyway).
> 
> I'm not sure which comment you are referring to. For now, https_proxy is
> not supported, but I would like it to be implemented as soon as
> possible.
> 
> 
Sorry, not you but the COMMENT / FIXME / XXX in guix-download.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.