GNU bug report logs - #23103
A number of corrupt packages on Hydra?

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 23103 in the body.
You can then email your comments to 23103 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christopher Allan Webber <cwebber <at> dustycloud.org>
To: bug-guix <at> gnu.org
Subject: A number of corrupt packages on Hydra?
Date: Wed, 23 Mar 2016 16:48:50 -0700

For at least boost and hydra I'm getting errors like the following:

  From http://hydra.gnunet.org/nar/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0
  Downloading jaf2s3...-ruby-2.3.0 (41.9MiB installed)...
   http://hydra.gnunet.org/nar/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0 2.0MiB/s 00:02 | 3.4MiB transferredd
  bzip2: Compressed file ends unexpectedly;
  	perhaps it is corrupted?  *Possible* reason follows.
  bzip2: Inappropriate ioctl for device
  	Input file = (stdin), output file = (stdout)
  
  It is possible that the compressed file(s) have become corrupted.
  You can use the -tvv option to test integrity of such files.
  
  You can use the `bzip2recover' program to attempt to recover
  data from undamaged sections of corrupted files.
  
  guix substitute: error: corrupt input while restoring '/gnu/store/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0/lib/ruby/2.3.0/rdoc/generator/template/darkfish/fonts/SourceCodePro-Regular.ttf' from #{read pipe}#
  killing process 8510
  guix package: error: build failed: some substitutes for the outputs of derivation `/gnu/store/l7ffwqbkrvd4vk3h51p597psyl44gv67-ruby-2.3.0.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source

Message #8 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: Pjotr Prins <pjotr.public12 <at> thebird.nl>
To: Christopher Allan Webber <cwebber <at> dustycloud.org>
Cc: 23103 <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Thu, 24 Mar 2016 14:25:59 +0100

I am also seeing such corruption on my substitutes server. Is there a way
we can validate files in a running cache so they can be rebuild? Would
be useful for the mirror.guixsd.org too.

I am glad they get picked up (even so) and that we have the --fallback
option :)

Pj.

On Wed, Mar 23, 2016 at 04:48:50PM -0700, Christopher Allan Webber wrote:
> For at least boost and hydra I'm getting errors like the following:
> 
>   From http://hydra.gnunet.org/nar/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0
>   Downloading jaf2s3...-ruby-2.3.0 (41.9MiB installed)...
>    http://hydra.gnunet.org/nar/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0 2.0MiB/s 00:02 | 3.4MiB transferredd
>   bzip2: Compressed file ends unexpectedly;
>   	perhaps it is corrupted?  *Possible* reason follows.
>   bzip2: Inappropriate ioctl for device
>   	Input file = (stdin), output file = (stdout)
>   
>   It is possible that the compressed file(s) have become corrupted.
>   You can use the -tvv option to test integrity of such files.
>   
>   You can use the `bzip2recover' program to attempt to recover
>   data from undamaged sections of corrupted files.
>   
>   guix substitute: error: corrupt input while restoring '/gnu/store/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0/lib/ruby/2.3.0/rdoc/generator/template/darkfish/fonts/SourceCodePro-Regular.ttf' from #{read pipe}#
>   killing process 8510
>   guix package: error: build failed: some substitutes for the outputs of derivation `/gnu/store/l7ffwqbkrvd4vk3h51p597psyl44gv67-ruby-2.3.0.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source
> 
> 
> 

-- 

Message #11 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: Pjotr Prins <pjotr.public12 <at> thebird.nl>
Cc: Christopher Allan Webber <cwebber <at> dustycloud.org>, 23103 <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Fri, 25 Mar 2016 00:08:48 +0100

 http://mirror.guixsd.org/nar/7v3093adf31b2sg2c46y3z2m24x2cjmi-gs-fonts-8.11 815KiB/s 00:02 | 1.2MiB transferred
bzip2: Compressed file ends unexpectedly;
        perhaps it is corrupted?  *Possible* reason follows.

Message #14 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Pjotr Prins <pjotr.public12 <at> thebird.nl>
Cc: Christopher Allan Webber <cwebber <at> dustycloud.org>, 23103 <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Fri, 25 Mar 2016 14:47:20 +0100

Pjotr Prins <pjotr.public12 <at> thebird.nl> skribis:

>  http://mirror.guixsd.org/nar/7v3093adf31b2sg2c46y3z2m24x2cjmi-gs-fonts-8.11 815KiB/s 00:02 | 1.2MiB transferred
> bzip2: Compressed file ends unexpectedly;
>         perhaps it is corrupted?  *Possible* reason follows.

This one should be fixed now.

I’m considering about removing all the cached items that could
potentially be corrupt (those that were transferred before we forced the
use of chunked transfer encoding, in commit
ea01e13f185b6e02c795b3c162c9a551162e4c9d of guix-maintenance.git, March
22nd) but that would cause 7.3 GiB of transfers.  Needs more thought.

I’ll followup on guix-sysadmin.

Ludo’.

Message #19 received at 23103-done <at> debbugs.gnu.org (full text, mbox):

From: Christopher Allan Webber <cwebber <at> dustycloud.org>
To: Pjotr Prins <pjotr.public12 <at> thebird.nl>
Cc: 23103-done <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Sat, 26 Mar 2016 16:24:01 -0700

Allegedly, Ludo has fixed this, or at least 
https://lists.gnu.org/archive/html/guix-devel/2016-03/msg01039.html
makes it sound like it :)

Message #22 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 23103 <at> debbugs.gnu.org
Cc: cwebber <at> dustycloud.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Sun, 27 Mar 2016 23:22:03 +0200

Christopher Allan Webber <cwebber <at> dustycloud.org> skribis:

> Allegedly, Ludo has fixed this, or at least 
> https://lists.gnu.org/archive/html/guix-devel/2016-03/msg01039.html
> makes it sound like it :)

Right.  :-)  AIUI, in its current config nginx is not support to cache
truncated items, but Mark thinks it’s already done so in the past.

So if that pops up again, do let us know!

Ludo’.

Message #25 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 23103 <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Tue, 29 Mar 2016 17:23:19 -0400

On Sun, Mar 27, 2016 at 11:22:03PM +0200, Ludovic Courtès wrote:
> Christopher Allan Webber <cwebber <at> dustycloud.org> skribis:
> 
> > Allegedly, Ludo has fixed this, or at least 
> > https://lists.gnu.org/archive/html/guix-devel/2016-03/msg01039.html
> > makes it sound like it :)
> 
> Right.  :-)  AIUI, in its current config nginx is not support to cache
> truncated items, but Mark thinks it’s already done so in the past.
> 
> So if that pops up again, do let us know!

IRC user 'ziz15' had problems downloading these archives while
initializing a new system:

/gnu/store/kcc3cxnx9l2hbg7pjhxsa0r5ypq2j2f38-python-2.7.10
/gnu/store/klb2s6r0f6cgpns65gkjwd5m8ygw73wj-wicd-1.7.3.drv

Message #28 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: Pjotr Prins <pjotr.public12 <at> thebird.nl>
To: Leo Famulari <leo <at> famulari.name>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 23103 <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Sun, 3 Apr 2016 10:20:00 +0200

 http://mirror.guixsd.org/nar/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4 1017KiB/s 00:02 | 1.7MiB transferred
bzip2: Compressed file ends unexpectedly;
        perhaps it is corrupted?  *Possible* reason follows.
bzip2: Inappropriate ioctl for device
        Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

guix substitute: error: corrupt input while restoring '/gnu/store/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4/lib/tcl8.6/encoding/cp936.enc' from #{read pipe}#
killing process 55815

Message #31 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: Ben Woodcroft <b.woodcroft <at> uq.edu.au>
To: Pjotr Prins <pjotr.public12 <at> thebird.nl>, Leo Famulari <leo <at> famulari.name>
Cc: 23103 <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Sun, 3 Apr 2016 20:55:59 +1000

On 03/04/16 18:20, Pjotr Prins wrote:
>   http://mirror.guixsd.org/nar/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4 1017KiB/s 00:02 | 1.7MiB transferred
> bzip2: Compressed file ends unexpectedly;
>          perhaps it is corrupted?  *Possible* reason follows.
> bzip2: Inappropriate ioctl for device
>          Input file = (stdin), output file = (stdout)
I think this only is a problem on the mirror, substituting from hydra 
itself works for me. You?

ben

Message #34 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ben Woodcroft <b.woodcroft <at> uq.edu.au>
Cc: Pjotr Prins <pjotr.public12 <at> thebird.nl>, 23103 <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Sun, 3 Apr 2016 14:16:11 -0400

On Sun, Apr 03, 2016 at 08:55:59PM +1000, Ben Woodcroft wrote:
> 
> 
> On 03/04/16 18:20, Pjotr Prins wrote:
> >  http://mirror.guixsd.org/nar/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4 1017KiB/s 00:02 | 1.7MiB transferred
> >bzip2: Compressed file ends unexpectedly;
> >         perhaps it is corrupted?  *Possible* reason follows.
> >bzip2: Inappropriate ioctl for device
> >         Input file = (stdin), output file = (stdout)
> I think this only is a problem on the mirror, substituting from hydra itself
> works for me. You?

Most of these recent corrupt archives problems are due to corruptions in
the mirror's cache — the mirror will only try (and possibly fail) to
pull an archive from hydra.gnu.org if hydra has built it successfully.

So, passing --substitute-urls=https://hydra.gnu.org or --fallback are
both reasonable ways to work around the failure.

Please keep reporting these!

Message #37 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Pjotr Prins <pjotr.public12 <at> thebird.nl>
Cc: 23103 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Sun, 03 Apr 2016 22:42:31 +0200

Pjotr Prins <pjotr.public12 <at> thebird.nl> skribis:

>  http://mirror.guixsd.org/nar/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4 1017KiB/s 00:02 | 1.7MiB transferred

Fixed, thanks for reporting it!

Ludo’.

Message #40 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: Pjotr Prins <pjotr.public12 <at> thebird.nl>
To: Pjotr Prins <pjotr.public12 <at> thebird.nl>
Cc: 23103 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Mon, 4 Apr 2016 17:14:33 +0200

Eeep

 http://mirror.guixsd.org/nar/kcc3cxnx9l2hbg7pjhxsa0r5yq2j2f38-python-2.7.10 1.9MiB/s 00:03 | 5.2MiB transferred
bzip2: Compressed file ends unexpectedly;
        perhaps it is corrupted?  *Possible* reason follows.
bzip2: Inappropriate ioctl for device
        Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

guix substitute: error: corrupt input while restoring '/gnu/store/kcc3cxnx9l2hbg7pjhxsa0r5yq2j2f38-python-2.7.10/lib/python2.7/email/test/test_email_renamed.pyc' from #{read pipe}#
killing process 17595


On Sun, Apr 03, 2016 at 10:20:00AM +0200, Pjotr Prins wrote:
>  http://mirror.guixsd.org/nar/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4 1017KiB/s 00:02 | 1.7MiB transferred
> bzip2: Compressed file ends unexpectedly;
>         perhaps it is corrupted?  *Possible* reason follows.
> bzip2: Inappropriate ioctl for device
>         Input file = (stdin), output file = (stdout)
> 
> It is possible that the compressed file(s) have become corrupted.
> You can use the -tvv option to test integrity of such files.
> 
> You can use the `bzip2recover' program to attempt to recover
> data from undamaged sections of corrupted files.
> 
> guix substitute: error: corrupt input while restoring '/gnu/store/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4/lib/tcl8.6/encoding/cp936.enc' from #{read pipe}#
> killing process 55815
> 
> 
> 
> 

-- 

Message #43 received at 23103 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Pjotr Prins <pjotr.public12 <at> thebird.nl>
Cc: 23103 <at> debbugs.gnu.org
Subject: Re: bug#23103: A number of corrupt packages on Hydra?
Date: Mon, 04 Apr 2016 22:44:10 +0200

Pjotr Prins <pjotr.public12 <at> thebird.nl> skribis:

>  http://mirror.guixsd.org/nar/kcc3cxnx9l2hbg7pjhxsa0r5yq2j2f38-python-2.7.10 1.9MiB/s 00:03 | 5.2MiB transferred
> bzip2: Compressed file ends unexpectedly;

Argh, fixed now, thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.