GNU bug report logs - #39387
Builds are *not* offloaded when using the "--keep-failed" or -K option.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 39387 in the body.
You can then email your comments to 39387 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: bug-guix <bug-guix <at> gnu.org>
Subject: Builds are *not* offloaded when using the "--keep-failed" or -K
 option.
Date: Sat, 01 Feb 2020 23:57:18 -0500

Consider the following, where 10.42.0.199 is a machine described in
/etc/guix/machines.scm for offloading:

$ ./pre-inst-env guix build php
The following derivation will be built:
   /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv
process 8241 acquired build slot '/var/guix/offload/10.42.0.199:22/0'
load on machine '10.42.0.199' is 0.04 (normalized: 0.04)
building /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv...
^C

$ ./pre-inst-env guix build php --keep-failed
The following derivation will be built:
   /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv
building /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv...
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' set to
`/gnu/store/niqa6dlvw4r2bfbabn65kq1kgdfwcb4b-pkg-config-0.29.2/bin:/gnu/store/qz06d1dz19wkrmibcr1485wqqia9zzhi-bison-3.5.1/bin:/gnu/store/3fzw8qb3cmrdkr46y55njjrfkvxd53lr-intltool-0.51.0/bin:
[...]

In the second case, the build is started on the local machine instead of
being dispatched to the offload machine.  Presumably, this is done so
that the files are available locally; but I'd prefer if it'd offload and
keep the files on the remote.  An alternative would be to copy the remote build directory
locally after failure.

Message #8 received at 39387 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 39387 <at> debbugs.gnu.org
Subject: Re: bug#39387: Builds are *not* offloaded when using the
 "--keep-failed" or -K option.
Date: Sun, 02 Feb 2020 18:43:50 +0100

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> In the second case, the build is started on the local machine instead of
> being dispatched to the offload machine.  Presumably, this is done so
> that the files are available locally; but I'd prefer if it'd offload and
> keep the files on the remote.  An alternative would be to copy the remote build directory
> locally after failure.

Indeed, it’s a feature!  :-)  It was discussed here:

  https://issues.guix.gnu.org/issue/38312

There we discussed the possibility of printing a warning, which wasn’t
implemented.

I’m closing in the meantime, but let me know what you think!

Thank you,
Ludo’.

Message #15 received at 39387 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 39387 <at> debbugs.gnu.org
Subject: Re: bug#39387: Builds are *not* offloaded when using the
 "--keep-failed" or -K option.
Date: Wed, 05 Feb 2020 23:55:19 -0500

Hello!

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:
>
>> In the second case, the build is started on the local machine instead of
>> being dispatched to the offload machine.  Presumably, this is done so
>> that the files are available locally; but I'd prefer if it'd offload and
>> keep the files on the remote.  An alternative would be to copy the remote build directory
>> locally after failure.
>
> Indeed, it’s a feature!  :-)  It was discussed here:
>
>   https://issues.guix.gnu.org/issue/38312
>
> There we discussed the possibility of printing a warning, which wasn’t
> implemented.

I see!

> I’m closing in the meantime, but let me know what you think!

OK!  I think I now see the technical difficulties in implementing a
useful, offloaded interrupted build: the build artifacts, even if copied
locally, would require syncing all the build time dependencies in order
to allow sourcing the "environment-variables" script and be able to
debug.  That's more the job of 'guix publish', not of 'guix offload'.

Maxim

Message #18 received at 39387 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 39387 <at> debbugs.gnu.org
Subject: Re: bug#39387: Builds are *not* offloaded when using the
 "--keep-failed" or -K option.
Date: Wed, 05 Feb 2020 23:56:05 -0500

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:
>
>> In the second case, the build is started on the local machine instead of
>> being dispatched to the offload machine.  Presumably, this is done so
>> that the files are available locally; but I'd prefer if it'd offload and
>> keep the files on the remote.  An alternative would be to copy the remote build directory
>> locally after failure.
>
> Indeed, it’s a feature!  :-)  It was discussed here:
>
>   https://issues.guix.gnu.org/issue/38312
>
> There we discussed the possibility of printing a warning, which wasn’t
> implemented.
>
> I’m closing in the meantime, but let me know what you think!
>
> Thank you,
> Ludo’.

Still, it'd be nice to have an option to keep failed build on the
*remote* machines.  I don't think that's possible currently?

Maxim

Message #21 received at 39387 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 39387 <at> debbugs.gnu.org
Subject: Re: bug#39387: Builds are *not* offloaded when using the
 "--keep-failed" or -K option.
Date: Thu, 06 Feb 2020 14:54:38 +0100

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:

[...]

>> Indeed, it’s a feature!  :-)  It was discussed here:
>>
>>   https://issues.guix.gnu.org/issue/38312
>>
>> There we discussed the possibility of printing a warning, which wasn’t
>> implemented.
>
> I see!
>
>> I’m closing in the meantime, but let me know what you think!
>
> OK!  I think I now see the technical difficulties in implementing a
> useful, offloaded interrupted build: the build artifacts, even if copied
> locally, would require syncing all the build time dependencies in order
> to allow sourcing the "environment-variables" script and be able to
> debug.  That's more the job of 'guix publish', not of 'guix offload'.

Right.  Ideally the build directory of an offloaded could be copied back
to the host of the caller, but that’s not implemented and a bit tricky
maybe.

Before this change, my main use of ‘--no-build-hook’ would be together
with ‘-K’ for that reason.

Ludo’.

Message #24 received at 39387 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 39387 <at> debbugs.gnu.org
Subject: Re: bug#39387: Builds are *not* offloaded when using the
 "--keep-failed" or -K option.
Date: Thu, 06 Feb 2020 14:55:48 +0100

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> Still, it'd be nice to have an option to keep failed build on the
> *remote* machines.  I don't think that's possible currently?

In some cases it’s not even desirable because you could fill up /tmp
without noticing.

But note that another option is to do:

  GUIX_DAEMON_SOCKET=ssh://remote.example.org guix build foo -K

WDYT?

Ludo’.

Message #27 received at 39387 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 39387 <at> debbugs.gnu.org
Subject: Re: bug#39387: Builds are *not* offloaded when using the
 "--keep-failed" or -K option.
Date: Sat, 08 Feb 2020 12:04:48 -0500

Hello Ludovic!

Ludovic Courtès <ludo <at> gnu.org> writes:

> Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:
>
>> Still, it'd be nice to have an option to keep failed build on the
>> *remote* machines.  I don't think that's possible currently?
>
> In some cases it’s not even desirable because you could fill up /tmp
> without noticing.
>
> But note that another option is to do:
>
>   GUIX_DAEMON_SOCKET=ssh://remote.example.org guix build foo -K
>
> WDYT?
>
> Ludo’.

This fulfill my use case well and appears to be sufficiently documented
already.  We're gold!

Thank you,

Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.