GNU bug report logs - #46585
ci: Remove hydra support.

Previous Next

Full log

View this message in rfc822 format

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 46585 <at> debbugs.gnu.org
Subject: [bug#46585] ci: Remove hydra support.
Date: Mon, 22 Feb 2021 10:59:11 +0100

Hi,

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

> This removes hydra support to use Cuirass as the only continuous
> integration system.

Yay!  That part could have been a separate commit, for clarity, but
that’s okay.

> It also simplifies the evaluation process. Here's how it's working
> now:
>
> * Cuirass fetches new commits and calls its "evaluate" process.
>
> * The "evaluate" process calls the "cuirass-jobs" procedure in the
> newly checkouted Guix "build-aux/cuirass/gnu-system.scm" file.
>
> * The "hydra-jobs" procedure in "build-aux/hydra/gnu-system.scm" file
>   starts the evaluation of "hydra-jobs" of (gnu ci) module in an
>   inferior using the new commit.
>
> * This procedure returns the list of all the package derivations at that
>   very commit under Hydra job format.
>
> * This list is converted to the Cuirass job format and written on the
>   stdout port.
>
> * The main Cuirass process reads the "evaluate" output using a pipe, and
>   registers the derivation that needs to be built in database.
>
> This is quite complex and it requires to pass around a huge list of
> jobs, consuming a lot of memory.
>
> Here's the simplified method:
>
> * The first two steps are identical.
>
> * The "cuirass-jobs" procedure starts the evaluation of "cuirass-jobs"
>   of (gnu ci) module in an inferior using the new commit. This procedure
>   is passed a registration callback that directly registers the given
>   jobs in database. It doesn't return anything.
>
> As the "register" procedure is a part of Cuirass, the inversion on
> control caused by the inferior is problematic. I had to proxy the
> registration requests from the inferior to the evaluation process using
> a named pipe.

OK.  Another option, to avoid the callback, would have been to write
jobs to stdout as before, but to write them one by one instead of
building the entire list in memory.  Perhaps that would be slightly
simpler than the callback + named pipe?

> Other than that, the process seems now less obfuscated.

Yeah, thumbs up.  Do you know how this affects memory consumption?

Thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.