Subject: Re: [email-masked] is really hard to build

From: Christopher Baines <[email-masked]>
Date: Tue, 14 May 2024 10:05:28 +0100

Christopher Baines <[email-masked]> writes:

> [email-masked] is really hard to build, it's so hard and so important that
> data.guix.gnu.org is still after two days trying to process [1]. I say
> so important because you have to build [email-masked] to compute the channel
> instance derivations for Guix.
>
> 1: https://data.guix.gnu.org/revision/72308f262c910977e40c2c9f350dc563c0a8437a
>
> Looking at the next revision which has been processed [2], it's been
> built on riscv64-linux as the testsuite is disabled, and it has also
> built on aarch64-linux, but there's no successful build for any other
> architecture.
>
> 2: https://data.guix.gnu.org/revision/9f183c3627a006e8fd3bb9708448bc05a6204e6d/package/nss/3.99.0?locale=en_US.UTF-8
>
> I think there's two issues here, was this spotted before merging, and
> what if anything can be done about this now. Where there's not a
> substitute available for [email-masked], this will affect guix pull/guix
> time-machine, e.g.
>
>   → guix time-machine --commit=72308f262c910977e40c2c9f350dc563c0a8437a -- describe
>   Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
>   substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
>   substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
>   substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
>    nss-3.99.tar.xz  55.2MiB                                                                                             13.7MiB/s 00:04 ▕██████████████████▏ 100.0%
>   building /gnu/store/8379qa0y6s7ssjr8gplm5fyw9r5pnxhn-nss-3.99.0.drv...

So with the changes in #70693 merged, this issue should be fixed going
forward, but the revisions with the broken nss are going to be affected
forever and thus the impact is going to drag on for a while. For
example, data.guix.gnu.org is going to be struggling to process the
revisions with the broken nss for a long while to come.

Before closing this bug, it would be good to understand more about how
this happened and from that try to think if anything can be done to
prevent similar issues in the future?

At least from what I can see on the issues, the problem was introduced
with the update to 3.98.0 [3] and then continued with the update to 3.99
[4]. Given the changes in 70662 were sent to guix-patches and then
merged less than 24 hours later, I'd imagine that wasn't sufficient time
for data.qa.guix.gnu.org to fail attempting to build nss.

3: https://issues.guix.gnu.org/70662
4: https://issues.guix.gnu.org/70618

Had the changes waited for longer, then these failures should have been
spotted by QA, I would guess that the revision might have failed to be
processed, and if it was processed successfully, the nss failures should
have shown up, so maybe we should start requiring [5] that not only are
changes sent to [email-masked], but that QA processes them (to
some extent) before merging?

5: https://guix.gnu.org/manual/devel/en/html_node/Managing-Patches-and-Branches.html#

Thanks,

Chris