GNU bug report logs - #46402
Cuirass rebuilds the same packae multiple times

Previous Next

Full log

Message #11 received at 46402 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: Julien Lepiller <julien <at> lepiller.eu>, 46402 <at> debbugs.gnu.org
Subject: Re: bug#46402: Cuirass rebuilds the same packae multiple times
Date: Wed, 10 Feb 2021 11:46:51 +0100

Hi,

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

>> here is an example: https://ci.guix.gnu.org/build/287478/details and
>> https://ci.guix.gnu.org/build/287476/details are being built at the
>> same time, and the logs currently show they are both running the test
>> phase of the php package.
>
> Thanks for the report. This problem has been briefly discussed
> yesterday. It has been introduced by the new remote building mechanism
> in Cuirass. Hydra solves it by breaking each build into buildsteps
> corresponding to the derivation inputs.
>
> The buildsteps are then submitted to the workers in a logical order. I
> proposed to introduce a similar mechanism in Cuirass but Ludo expressed
> doubts. Ludo do you think this problem could be solved otherwise?

I’m not sure exactly but I can share my feelings.  :-)

Seems to me that ‘BuildSteps’ is an orthogonal concern that has little
to do with Cuirass’ job and with its data model.  In Hydra I saw that as
a (necessary) kludge.

I like the way the Coordinator does it, and AIUI it’s pretty much the
same as what the daemon is doing: submit build requests in topological
order, such that when a derivation build is submitted, its prerequisites
are known to be built already.

I suppose what makes it more difficult here is that we have this extra
“job” abstraction on top of derivations; everything in Cuirass revolves
around jobs, which leads to this impedance mismatch.

If Cuirass would instead delegate derivation build requests to a
Coordinator/daemon-like thing, it wouldn’t have to worry about those
details.  That would better separate concerns.

This is quite a hand-wavy reply but I hope it’s useful!

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.