GNU bug report logs - #19805
Numpy failures

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19805 in the body.
You can then email your comments to 19805 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: bug-guix <at> gnu.org, guix-devel <at> gnu.org
Subject: Numpy failures
Date: Sat, 7 Feb 2015 17:41:08 +0100

python-numpy-bootstrap currently fails its tests on hydra, which entails a
bunch of other failures. On my own x86_64 machines, the build succeeds,
however.

Andreas

Message #8 received at 19805 <at> debbugs.gnu.org (full text, mbox):

From: Federico Beffa <beffa <at> ieee.org>
To: Andreas Enge <andreas <at> enge.fr>
Cc: Guix-devel <guix-devel <at> gnu.org>, bug-guix <at> gnu.org
Subject: Re: Numpy failures
Date: Mon, 9 Feb 2015 09:30:16 +0100

Andreas Enge <andreas <at> enge.fr> writes:

> python-numpy-bootstrap currently fails its tests on hydra, which entails a
> bunch of other failures. On my own x86_64 machines, the build succeeds,
> however.

We believe the reason being the fact that hydra doesn't handle the flag
'#:substitutable?' properly. As a result we have the following
situation:

1. hydra builds a version of ATLAS optimized for its CPU locally.

2. 'python-numpy-bootstrap' and co., on some architectures, probably get
an incompatible version of ATLAS and therefore fail to pass some tests.

You can check in the build log of 'python-numpy-bootstrap' that ATLAS,
despite the flas, is substituted (no local build on the slave).

At some point we should fix the support for '#:substitutable?' on hydra
(or the upcoming 'guix publish').

Regards,
Fede

Message #11 received at 19805 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Federico Beffa <beffa <at> ieee.org>
Cc: guix-devel <at> gnu.org, Andreas Enge <andreas <at> enge.fr>, 19805 <at> debbugs.gnu.org
Subject: Re: bug#19805: Numpy failures
Date: Tue, 10 Feb 2015 00:07:07 +0100

Federico Beffa <beffa <at> ieee.org> skribis:

> Andreas Enge <andreas <at> enge.fr> writes:
>
>> python-numpy-bootstrap currently fails its tests on hydra, which entails a
>> bunch of other failures. On my own x86_64 machines, the build succeeds,
>> however.
>
> We believe the reason being the fact that hydra doesn't handle the flag
> '#:substitutable?' properly. As a result we have the following
> situation:
>
> 1. hydra builds a version of ATLAS optimized for its CPU locally.
>
> 2. 'python-numpy-bootstrap' and co., on some architectures, probably get
> an incompatible version of ATLAS and therefore fail to pass some tests.
>
> You can check in the build log of 'python-numpy-bootstrap' that ATLAS,
> despite the flas, is substituted (no local build on the slave).

Oh.  Actually, slaves do not use substitutes at all, but they receive
missing store items from hydra.gnu.org when builds are offloaded.  So I
suppose they receive something built on hydra.gnu.org, which may be
incompatible.

> At some point we should fix the support for '#:substitutable?' on hydra

That probably means that #:substitutable? should be propagated–i.e.,
that anything depending on ATLAS should not be substituted.

Thanks for investigating!

Ludo’.

Message #14 received at 19805 <at> debbugs.gnu.org (full text, mbox):

From: Federico Beffa <beffa <at> ieee.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Guix-devel <guix-devel <at> gnu.org>, Andreas Enge <andreas <at> enge.fr>,
 19805 <at> debbugs.gnu.org
Subject: Re: bug#19805: Numpy failures
Date: Tue, 10 Feb 2015 14:20:57 +0100

On Tue, Feb 10, 2015 at 12:07 AM, Ludovic Courtès <ludo <at> gnu.org> wrote:
>> At some point we should fix the support for '#:substitutable?' on hydra
>
> That probably means that #:substitutable? should be propagated–i.e.,
> that anything depending on ATLAS should not be substituted.

I guess so. Is there a way to specify this in a package or does it
require changes to the guix code?

Regards,
Fede

Message #17 received at 19805 <at> debbugs.gnu.org (full text, mbox):

From: Mark H Weaver <mhw <at> netris.org>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: guix-devel <at> gnu.org, 19805 <at> debbugs.gnu.org, Federico Beffa <beffa <at> ieee.org>
Subject: Re: bug#19805: Numpy failures
Date: Tue, 10 Feb 2015 08:52:23 -0500

ludo <at> gnu.org (Ludovic Courtès) writes:

> Federico Beffa <beffa <at> ieee.org> skribis:
>
>> At some point we should fix the support for '#:substitutable?' on hydra
>
> That probably means that #:substitutable? should be propagated–i.e.,
> that anything depending on ATLAS should not be substituted.

As I just wrote in the "texlive failure" thread, setting
#:substitutable? #f might make sense for texlive.  However, in that case
we certainly would not want to propagate that setting.

So, instead of propagating it, how about simply arranging to not send
such packages to the build slaves, instead forcing them to build it
locally on their own?

I suppose this would cause the texlive build log to be prepended to some
other build log, which is not so good, but that could also be dealt with
in various ways, e.g. by changing the "send dependency to slave" logic
for non-substitutable packages to build the package on that particular
slave.

What do you think?

      Mark

Message #20 received at 19805 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mark H Weaver <mhw <at> netris.org>
Cc: guix-devel <at> gnu.org, 19805 <at> debbugs.gnu.org, Federico Beffa <beffa <at> ieee.org>
Subject: Re: bug#19805: Numpy failures
Date: Tue, 10 Feb 2015 15:41:20 +0100

Mark H Weaver <mhw <at> netris.org> skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> Federico Beffa <beffa <at> ieee.org> skribis:
>>
>>> At some point we should fix the support for '#:substitutable?' on hydra
>>
>> That probably means that #:substitutable? should be propagated–i.e.,
>> that anything depending on ATLAS should not be substituted.
>
> As I just wrote in the "texlive failure" thread, setting
> #:substitutable? #f might make sense for texlive.  However, in that case
> we certainly would not want to propagate that setting.
>
> So, instead of propagating it, how about simply arranging to not send
> such packages to the build slaves, instead forcing them to build it
> locally on their own?

I think there are two properties of interest: “substitutability”, and
“offloadability”.

What do we want for ATLAS, for NumPy, and for TeX Live?

However, note that we can’t currently distinguish between these two
properties: <http://bugs.gnu.org/18747>.

Thanks,
Ludo’.

Message #27 received at 19805-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Mark H Weaver <mhw <at> netris.org>
Cc: 19805-done <at> debbugs.gnu.org, Federico Beffa <beffa <at> ieee.org>
Subject: Re: bug#19805: Numpy failures
Date: Fri, 11 Sep 2015 21:40:11 +0200

This bug is moot now that NumPy uses OpenBLAS instead of ATLAS (commit
dbdfe515), so closing it.

The suggestion about propagating the #:substitutable? flag at
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=19805#11> remains valid,
but it’s a separate issue.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.