Subject: bug#20402: guix cannot download via an http proxy

From: [email-masked] (Ludovic Courtès)
Date: Fri, 01 May 2015 00:07:59 +0200

Hi,

Sorry for the delay, and thanks for the investigation and patch!

Joshua Randall <[email-masked]> skribis:

> This patch modifies http-fetch (in build/download.scm) such that it calls
> Guile's open-socket-for-uri after fixing up the uri so that it always has a
> port specified. I'm not sure how to test the bootstrapping NSS issue that
> required open-connection-for-uri, but my expectation based on reading the
> source is that this alternative should work for Guile > 2.0.7, and I've
> left the original open-connection-for-uri in there for backwards
> compatibility with Guile < 2.0.7. If someone can test this against a
> situation known to have needed the NSS workaround, that would be great.

To name lookup with the bootstrap Guile, one way is to run this:

  $ guix gc -d $(guix build -S -e '(@@ (gnu packages commencement) glibc-final)')
  $ ./pre-inst-env guix build -S \
     -e '(@@ (gnu packages commencement) glibc-final)' --no-substitutes

The second command here uses the bootstrap Guile.

Another approach is this:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(gnu packages bootstrap)
scheme@(guile-user)> ,enter-store-monad
store-monad@(guile-user) [1]> (mlet %store-monad ((guile (package->derivation %bootstrap-guile)))
				(gexp->derivation "foo"
						  #~(begin
						      (mkdir #$output)
						      (pk (getaddrinfo "www.gnu.org" "http")))
						  #:hash-algo 'sha256
						  #:hash (base32 "0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73")
						  #:guile-for-build guile))
$5 = #<derivation /gnu/store/5prfiblj6ddziavk9nz31bkgy8jzaghx-foo.drv => /gnu/store/xf3404zw9kfx6a1gcfk6lmqcx6a53ad5-foo 2ae7960>
store-monad@(guile-user) [1]> (built-derivations (list $5))
building path(s) `/gnu/store/xf3404zw9kfx6a1gcfk6lmqcx6a53ad5-foo'

;;; ((#(0 2 1 6 #(2 3497454484 80) #f) #(0 2 2 17 #(2 3497454484 80) #f) #(0 10 1 6 #(10 42541952298791455573290623124440612874 80 0 0) #f) #(0 10 2 17 #(10 42541952298791455573290623124440612874 80 0 0) #f)))
--8<---------------cut here---------------end--------------->8---

Here ‘built-derivations’ fails but the build log shows that
‘getaddrinfo’ succeeded.

Lastly, one can extract
gnu/packages/bootstrap/x86_64-linux/guile-2.0.9.tar.xz and run:

  strace -o log ./bin/guile -c '(getaddrinfo "www.gnu.org" "http")'

Here the log shows that /etc/nsswitch.conf, /etc/services, and
/etc/hosts are accessed and things just work; it does not attempt to
connect to the nscd.

A bit of archeology shows the following timeline:

  1. d14ecda introduces the ‘open-connection-for-uri’ hack (Oct. 2012).

  2. d3b5972 changes libc used to make bootstrap tarballs to use static
     NSS modules (Jan. 2013).

  3. 0621349 updates the bootstrap guile-2.0.9.tar.xz tarballs
     (Nov. 2013), meaning that our current bootstrap Guile indeed uses
     static NSS modules and doesn’t attempt to talk to nscd.

In other words, the hack is no longer needed.

Thus, ‘open-connection-for-uri’ is almost (see below) unneeded now,
which simplifies the solution to the problem you raise.

> I've also changed the only other call to open-connection-for-uri, which is
> in the probe-uri function in scripts/lint.scm - my suspicion is that won't
> be an issue because I'm guessing the lint scripts are not used while
> bootstrapping, so the open-socket-for-uri function will probably be fine
> for the purpose of probing whether a URL is valid.

‘open-connection-for-uri’ also handles TLS connections, which are also
useful for ‘guix lint’, so we cannot completely get rid of it.

Commit d17551d simplifies it so that it is just a wrapper around
‘open-socket-for-uri’.  After that, ‘guix download’ honors $http_proxy.

$https_proxy is not handled yet because that requires more work, and I
do not fully understand how that is supposed to work.  Patch welcome,
though.  :-)

Thanks!

Ludo’.