GNU bug report logs - #26811
guix does not resume download on FTP timeouts

Previous Next

To reply to this bug, email your comments to 26811 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sergei Trofimovich <slyfox <at> inbox.ru>
To: bug-guix <at> gnu.org
Subject: guix does not resume download on FTP timeouts
Date: Sun, 7 May 2017 09:31:59 +0100

[Message part 1 (text/plain, inline)]

I have slow internet :)

It looks like ftp://tug.org has a FTP session timeout of 2 hours 11 minutes, or something like that.
Over night I tried to build sbcl and failed. Note how download silently stopped at ~80% [1].

Would be nice if guix had a mechanism to reconnect to FTP and continue download after timeout.

Thanks!

[1]: $ while :; do ./pre-inst-env guix build sbcl --no-substitutes; done

@ build-started /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - x86_64-linux /var/log/guix/drvs/yw//pxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv.bz2

Starting download of /gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz
From ftp://tug.org/historic/systems/texlive/2016/texlive-20160523b-texmf.tar.xz...
 …3b-texmf.tar.xz  1.93GiB       213KiB/s 02:11:32 [################    ]  83.0%
output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `062qbbhlin1ys9lihax34kq503ig1fl55br84j94vwawbk04p6j0'
@ build-failed /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - 1 output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `062qbbhlin1ys9lihax34kq503ig1fl55br84j94vwawbk04p6j0'
@ build-started /gnu/store/in6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv - x86_64-linux /var/log/guix/drvs/in//6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv.bz2
cannot build derivation `/gnu/store/q9s2wdsaqvrllkcsf8w1a8424zd9ybc1-texlive-20160523b-texmf.tar.xz.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/b7abk0aarqv0ylxp2g4s2sf9j75xdy04-texlive-texmf-2016.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/kpjy0c1pl8vzxbkys6p45ij3za6a92gj-texlive-2016.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv' failed
@ build-started /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - x86_64-linux /var/log/guix/drvs/yw//pxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv.bz2

Starting download of /gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz
From ftp://tug.org/historic/systems/texlive/2016/texlive-20160523b-texmf.tar.xz...
 …3b-texmf.tar.xz  1.93GiB       213KiB/s 02:11:46 [################    ]  83.3%
output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `1l296b4ypphffv1zpamp01g6a633kqlmfbf9l4x9411rlblb1hfd'
@ build-failed /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - 1 output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `1l296b4ypphffv1zpamp01g6a633kqlmfbf9l4x9411rlblb1hfd'
@ build-started /gnu/store/in6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv - x86_64-linux /var/log/guix/drvs/in//6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv.bz2
cannot build derivation `/gnu/store/q9s2wdsaqvrllkcsf8w1a8424zd9ybc1-texlive-20160523b-texmf.tar.xz.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/b7abk0aarqv0ylxp2g4s2sf9j75xdy04-texlive-texmf-2016.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/kpjy0c1pl8vzxbkys6p45ij3za6a92gj-texlive-2016.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv' failed
@ build-started /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - x86_64-linux /var/log/guix/drvs/yw//pxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv.bz2

Starting download of /gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz
From ftp://tug.org/historic/systems/texlive/2016/texlive-20160523b-texmf.tar.xz...
 …3b-texmf.tar.xz  1.93GiB       213KiB/s 02:11:31 [################    ]  83.1%
output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `143zan8sh0lw2jjhdpvc7w9yjh7cw75w1ry60qq4g1n3hjmfrwi3'
@ build-failed /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - 1 output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `143zan8sh0lw2jjhdpvc7w9yjh7cw75w1ry60qq4g1n3hjmfrwi3'
@ build-started /gnu/store/in6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv - x86_64-linux /var/log/guix/drvs/in//6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv.bz2
cannot build derivation `/gnu/store/q9s2wdsaqvrllkcsf8w1a8424zd9ybc1-texlive-20160523b-texmf.tar.xz.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/b7abk0aarqv0ylxp2g4s2sf9j75xdy04-texlive-texmf-2016.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/kpjy0c1pl8vzxbkys6p45ij3za6a92gj-texlive-2016.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv' failed
@ build-started /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - x86_64-linux /var/log/guix/drvs/yw//pxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv.bz2

-- 

  Sergei

[Message part 2 (application/pgp-signature, inline)]

Message #8 received at 26811 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Sergei Trofimovich <slyfox <at> inbox.ru>
Cc: 26811 <at> debbugs.gnu.org
Subject: Re: bug#26811: guix does not resume download on FTP timeouts
Date: Sun, 7 May 2017 14:25:43 -0400

[Message part 1 (text/plain, inline)]

On Sun, May 07, 2017 at 09:31:59AM +0100, Sergei Trofimovich wrote:
> I have slow internet :)
> 
> It looks like ftp://tug.org has a FTP session timeout of 2 hours 11 minutes, or something like that.
> Over night I tried to build sbcl and failed. Note how download silently stopped at ~80% [1].
> 
> Would be nice if guix had a mechanism to reconnect to FTP and continue download after timeout.

In the meantime, you can download the files with a resumable tool like
wget, and then use `guix download file:///path/to/file` to work around
this missing feature. Note that both the client *and* the server must
support resuming downloads for this to work.

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 26811 <at> debbugs.gnu.org (full text, mbox):

From: Sergei Trofimovich <slyfox <at> inbox.ru>
To: Leo Famulari <leo <at> famulari.name>
Cc: 26811 <at> debbugs.gnu.org
Subject: Re: bug#26811: guix does not resume download on FTP timeouts
Date: Sun, 7 May 2017 19:49:32 +0100

[Message part 1 (text/plain, inline)]

On Sun, 7 May 2017 14:25:43 -0400
Leo Famulari <leo <at> famulari.name> wrote:

> On Sun, May 07, 2017 at 09:31:59AM +0100, Sergei Trofimovich wrote:
> > I have slow internet :)
> > 
> > It looks like ftp://tug.org has a FTP session timeout of 2 hours 11 minutes, or something like that.
> > Over night I tried to build sbcl and failed. Note how download silently stopped at ~80% [1].
> > 
> > Would be nice if guix had a mechanism to reconnect to FTP and continue download after timeout.  
> 
> In the meantime, you can download the files with a resumable tool like
> wget, and then use `guix download file:///path/to/file` to work around
> this missing feature. Note that both the client *and* the server must
> support resuming downloads for this to work.

The workaround worked.

I've noticed it took guix-daemon 2GB of RAM to process this file locally.
Does guix keep all the file in RAM until download finishes?

Thank you!

-- 

  Sergei

[Message part 2 (application/pgp-signature, inline)]

Message #14 received at 26811 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Sergei Trofimovich <slyfox <at> inbox.ru>
Cc: 26811 <at> debbugs.gnu.org
Subject: Re: bug#26811: guix does not resume download on FTP timeouts
Date: Sun, 7 May 2017 14:58:17 -0400

[Message part 1 (text/plain, inline)]

On Sun, May 07, 2017 at 07:49:32PM +0100, Sergei Trofimovich wrote:
> On Sun, 7 May 2017 14:25:43 -0400
> Leo Famulari <leo <at> famulari.name> wrote:
> 
> > On Sun, May 07, 2017 at 09:31:59AM +0100, Sergei Trofimovich wrote:
> > > I have slow internet :)
> > > 
> > > It looks like ftp://tug.org has a FTP session timeout of 2 hours 11 minutes, or something like that.
> > > Over night I tried to build sbcl and failed. Note how download silently stopped at ~80% [1].
> > > 
> > > Would be nice if guix had a mechanism to reconnect to FTP and continue download after timeout.  
> > 
> > In the meantime, you can download the files with a resumable tool like
> > wget, and then use `guix download file:///path/to/file` to work around
> > this missing feature. Note that both the client *and* the server must
> > support resuming downloads for this to work.
> 
> The workaround worked.
> 
> I've noticed it took guix-daemon 2GB of RAM to process this file locally.
> Does guix keep all the file in RAM until download finishes?

I haven't looked, but that's how I'd expect it be coded. The OS can
handle paging things in and out as needed, right?

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 26811 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Sergei Trofimovich <slyfox <at> inbox.ru>
Cc: 26811 <at> debbugs.gnu.org
Subject: Re: bug#26811: guix does not resume download on FTP timeouts
Date: Mon, 08 May 2017 16:27:06 +0200

Sergei Trofimovich <slyfox <at> inbox.ru> skribis:

> Starting download of /gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz
> From ftp://tug.org/historic/systems/texlive/2016/texlive-20160523b-texmf.tar.xz...

Fundamentally, I think we should just fix our ‘texlive’ package such
that it doesn’t download 4G at once…

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.