GNU bug report logs - #50264
ca-certificate-bundle fails to build

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 50264 in the body.
You can then email your comments to 50264 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Lars-Dominik Braun <lars <at> 6xq.net>
To: bug-guix <at> gnu.org
Subject: ca-certificate-bundle fails to build
Date: Mon, 30 Aug 2021 10:12:29 +0200

[Message part 1 (text/plain, inline)]

Hi,

I’ve seen this multiple times now, most recently with commit
8ef38cd1bf7ec17b8d1bc1c0bcb42ac30ac30406 on a foreign system: Some
operation, here a `guix pack` tries to build ca-certificate-bundle and
it fails the first time, but rerunning the same operation succeeds:

---snip---
-builder for `/gnu/store/iqm2zwq8jrr9gaw5jp4vs42j887aglmk-ca-certificate-bundle.drv' failed with exit code 1
Erstellung von /gnu/store/iqm2zwq8jrr9gaw5jp4vs42j887aglmk-ca-certificate-bundle.drv fehlgeschlagen
Das Erstellungsprotokoll kann unter „/var/log/guix/drvs/iq/m2zwq8jrr9gaw5jp4vs42j887aglmk-ca-certificate-bundle.drv.bz2“ eingesehen werden.
cannot build derivation `/gnu/store/riw7sfkl8a5rapyvf6vvh07cwv3nrwil-profile.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/l8zdf69zk0pg599gifcyxbd80h676yp1-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv': 1 dependencies couldn't be built
guix pack: Fehler: build of `/gnu/store/l8zdf69zk0pg599gifcyxbd80h676yp1-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv' failed
---snap---

The backtrace looks like this:

---snip---
Backtrace:
           2 (primitive-load "/gnu/store/74i539y9b0bvslk3bk0kc27q76p?")
In ice-9/eval.scm:
    619:8  1 (_ #f)
In unknown file:
           0 (setlocale 6 "en_US.utf8")

ERROR: In procedure setlocale:
In procedure setlocale: Invalid argument
---snap---

Curiously the output path mentioned in the .drv
(/gnu/store/zjbpik3bfhmcamqzixzm1892mqbknnj9-ca-certificate-bundle) does
not exist, even after successfully running the command. Manually
building the driver with `guix build
/gnu/store/iqm2zwq8jrr9gaw5jp4vs42j887aglmk-ca-certificate-bundle.drv`
yields the same error as above, so I’m assuming the profile hook is
simply broken, while `guix pack` does not actually depend on it.

Cheers,
Lars

[iqm2zwq8jrr9gaw5jp4vs42j887aglmk-ca-certificate-bundle.drv (text/plain, attachment)]

Message #8 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: 50264 <at> debbugs.gnu.org
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Sat, 04 Sep 2021 18:27:29 +0200

Hi,

Lars-Dominik Braun <lars <at> 6xq.net> skribis:

> Hi,
>
> I’ve seen this multiple times now, most recently with commit
> 8ef38cd1bf7ec17b8d1bc1c0bcb42ac30ac30406 on a foreign system: Some
> operation, here a `guix pack` tries to build ca-certificate-bundle and
> it fails the first time, but rerunning the same operation succeeds:
>
> ---snip---
> -builder for `/gnu/store/iqm2zwq8jrr9gaw5jp4vs42j887aglmk-ca-certificate-bundle.drv' failed with exit code 1
> Erstellung von /gnu/store/iqm2zwq8jrr9gaw5jp4vs42j887aglmk-ca-certificate-bundle.drv fehlgeschlagen
> Das Erstellungsprotokoll kann unter „/var/log/guix/drvs/iq/m2zwq8jrr9gaw5jp4vs42j887aglmk-ca-certificate-bundle.drv.bz2“ eingesehen werden.
> cannot build derivation `/gnu/store/riw7sfkl8a5rapyvf6vvh07cwv3nrwil-profile.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/l8zdf69zk0pg599gifcyxbd80h676yp1-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv': 1 dependencies couldn't be built
> guix pack: Fehler: build of `/gnu/store/l8zdf69zk0pg599gifcyxbd80h676yp1-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv' failed
> ---snap---
>
> The backtrace looks like this:
>
> ---snip---
> Backtrace:
>            2 (primitive-load "/gnu/store/74i539y9b0bvslk3bk0kc27q76p?")
> In ice-9/eval.scm:
>     619:8  1 (_ #f)
> In unknown file:
>            0 (setlocale 6 "en_US.utf8")
>
> ERROR: In procedure setlocale:
> In procedure setlocale: Invalid argument
> ---snap---

Is this the backtrace found in the build log (under /var/log/guix/drvs)?

I tried and failed to reproduce it like this:

--8<---------------cut here---------------start------------->8---
$ guix environment --ad-hoc nss-certs -n
La jena derivo estus konstruata:
   /gnu/store/i5s8jqzl52j38qmwqghjyp0v8p7dnlgd-profile.drv

$ guix gc -R /gnu/store/i5s8jqzl52j38qmwqghjyp0v8p7dnlgd-profile.drv |grep certificate
/gnu/store/n63a6h3dfhwnaas9pg35zk78qjhxbas9-cmake-curl-certificates.patch
/gnu/store/c8czsp9prfd73wvnyh595h0riqcllfqp-ca-certificate-bundle-builder
/gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
$ guix build --rounds=10 /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
The following profile hook will be built:
   /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
building CA certificate bundle...
building CA certificate bundle...
building CA certificate bundle...
building CA certificate bundle...
building CA certificate bundle...
building CA certificate bundle...
building CA certificate bundle...
building CA certificate bundle...
building CA certificate bundle...
building CA certificate bundle...
successfully built /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
The following builds are still in progress:
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
  /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv

/gnu/store/gwgja3hnmlajwxy6rqayf8yd937d8yi5-ca-certificate-bundle
--8<---------------cut here---------------end--------------->8---

Could you find a way to reproduce the issue?

Also, could you run ‘guix build glibc-utf8-locales --check’, so make
sure that store item is not corrupt?

I have:

--8<---------------cut here---------------start------------->8---
$ guix build glibc-utf8-locales 
/gnu/store/rgydar9dfvflqqz2irgh7njj34amaxc6-glibc-utf8-locales-2.31
$ guix hash -r $(guix build glibc-utf8-locales)
012a1vcvmhbrqr5kjbmf7qlgpbw2zv36rgj7rxh400dh8wlj97pi
$ wget -qO - https://ci.guix.gnu.org/rgydar9dfvflqqz2irgh7njj34amaxc6.narinfo |grep NarHash
NarHash: sha256:012a1vcvmhbrqr5kjbmf7qlgpbw2zv36rgj7rxh400dh8wlj97pi
--8<---------------cut here---------------end--------------->8---

TIA,
Ludo’.

Message #11 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Lars-Dominik Braun <lars <at> 6xq.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 50264 <at> debbugs.gnu.org
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Mon, 6 Sep 2021 13:22:53 +0200

Hi Ludo,

thanks for looking into this!

> Is this the backtrace found in the build log (under /var/log/guix/drvs)?
Yes, it is printed on stdout, as well as part of the build log.

> I tried and failed to reproduce it like this:
> 
> --8<---------------cut here---------------start------------->8---
> $ guix environment --ad-hoc nss-certs -n
> La jena derivo estus konstruata:
>    /gnu/store/i5s8jqzl52j38qmwqghjyp0v8p7dnlgd-profile.drv
> 
> $ guix gc -R /gnu/store/i5s8jqzl52j38qmwqghjyp0v8p7dnlgd-profile.drv |grep certificate
> /gnu/store/n63a6h3dfhwnaas9pg35zk78qjhxbas9-cmake-curl-certificates.patch
> /gnu/store/c8czsp9prfd73wvnyh595h0riqcllfqp-ca-certificate-bundle-builder
> /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
> $ guix build --rounds=10 /gnu/store/wdnm4j88pxxkg0m72b58db24fghjizmz-ca-certificate-bundle.drv
> …
> --8<---------------cut here---------------end--------------->8---
> 
> Could you find a way to reproduce the issue?
Alright, let’s see. The command I have been using is

    guix pack -L . -C 'zstd' -f docker -S /bin=bin python-jupyterlab bash coreutils findutils

with . being a checkout of guix-science (same applies to `guix
time-machine` though). The first time it’ll fail, but the second time
it succeeds.  Running `guix gc` makes it fail again the first time.

The docker-pack.tar.zst.drv used for the first build is
different from the second one (different hash prefix). For me it’s
/gnu/store/r096cm3np7hbdn853ih35h1a5l39in4s-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
the first time and
/gnu/store/dywspxjshfjhc07i17hkcyrlq8kn7m07-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
the second time. YMMV.

Looking at ca-certificate-bundle.drv, the first one lacks any form of
glibc-utf8-locales as an input (neither in the .drv, nor via `guix gc
--references <.drv> | grep glibc-`), so it’s clear it must fail. I’m
not quite sure why though, since the actual builder still has a reference
to the locales and sets LOCPATH. So I’m a little puzzled.
 
> --8<---------------cut here---------------start------------->8---
> $ guix build glibc-utf8-locales 
> /gnu/store/rgydar9dfvflqqz2irgh7njj34amaxc6-glibc-utf8-locales-2.31
> $ guix hash -r $(guix build glibc-utf8-locales)
> 012a1vcvmhbrqr5kjbmf7qlgpbw2zv36rgj7rxh400dh8wlj97pi
> $ wget -qO - https://ci.guix.gnu.org/rgydar9dfvflqqz2irgh7njj34amaxc6.narinfo |grep NarHash
> NarHash: sha256:012a1vcvmhbrqr5kjbmf7qlgpbw2zv36rgj7rxh400dh8wlj97pi
> --8<---------------cut here---------------end--------------->8---
Exactly the same for me, so we have the same data at least. `guix gc`
with the repair,contents options also does not show any corrupted
items. I’m thus assuming my store is intact.

Any ideas?

Thanks,
Lars

Message #14 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Kyle Andrews <kyle.c.andrews <at> gmail.com>
To: 50264 <at> debbugs.gnu.org
Subject: Re: ca-certificate-bundle fails to build
Date: Mon, 06 Sep 2021 19:18:59 -0400

For what it's worth, this same thing just happened to me when updating
my profile on my laptop. The first time it failed exactly as described,
and the second time it worked.

Kyle

Message #17 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: "bdju" <bdju <at> tilde.team>
To: "Kyle Andrews" <kyle.c.andrews <at> gmail.com>, <50264 <at> debbugs.gnu.org>
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Mon, 06 Sep 2021 18:29:07 -0500

On Mon Sep 6, 2021 at 6:18 PM CDT, Kyle Andrews wrote:
>
> For what it's worth, this same thing just happened to me when updating
> my profile on my laptop. The first time it failed exactly as described,
> and the second time it worked.
>
> Kyle
I've also hit this twice in the last couple weeks. I just ran the
updates again and they worked both times.

Message #20 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Vladilen Kozin <vladilen.kozin <at> gmail.com>
To: 50264 <at> debbugs.gnu.org
Subject: Re: ca-certificate-bundle fails to build
Date: Fri, 10 Sep 2021 12:24:03 +0100

[Message part 1 (text/plain, inline)]

Exact same issue trying to update my Guix Laptop.
guix pull - runs fine
sudo guix reconfigure - fails

fails to build ca-bundle with the above mentioned "invalid argument in
setlocale" error
-- 
Best regards
Vlad Kozin

[Message part 2 (text/html, inline)]

Message #23 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: 50264 <at> debbugs.gnu.org
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Fri, 10 Sep 2021 18:04:19 +0200

Hi,

Lars-Dominik Braun <lars <at> 6xq.net> skribis:

> Alright, let’s see. The command I have been using is
>
>     guix pack -L . -C 'zstd' -f docker -S /bin=bin python-jupyterlab bash coreutils findutils
>
> with . being a checkout of guix-science (same applies to `guix
> time-machine` though). The first time it’ll fail, but the second time
> it succeeds.  Running `guix gc` makes it fail again the first time.
>
> The docker-pack.tar.zst.drv used for the first build is
> different from the second one (different hash prefix). For me it’s
> /gnu/store/r096cm3np7hbdn853ih35h1a5l39in4s-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
> the first time and
> /gnu/store/dywspxjshfjhc07i17hkcyrlq8kn7m07-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
> the second time. YMMV.

Wait, you run the same command twice and it leads a different .drv?

The expected behavior is that the .drv is always the same:

--8<---------------cut here---------------start------------->8---
$ guix pack -C zstd -f docker bash coreutils findutils -d
/gnu/store/gqq2802zy9r6xgb7l5p5smxkwh886x7b-bash-coreutils-findutils-docker-pack.tar.zst.drv
$ guix pack -C zstd -f docker bash coreutils findutils -d
/gnu/store/gqq2802zy9r6xgb7l5p5smxkwh886x7b-bash-coreutils-findutils-docker-pack.tar.zst.drv
$ guix pack -C zstd -f docker bash coreutils findutils -d
/gnu/store/gqq2802zy9r6xgb7l5p5smxkwh886x7b-bash-coreutils-findutils-docker-pack.tar.zst.drv
$ guix pack -L /data/src/guix-science -C zstd -f docker python-jupyterlab bash coreutils findutils -d --no-grafts
/gnu/store/xzp0qrd54sglhrggwn8myl7fd0mfhwln-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
$ guix pack -L /data/src/guix-science -C zstd -f docker python-jupyterlab bash coreutils findutils -d --no-grafts
/gnu/store/xzp0qrd54sglhrggwn8myl7fd0mfhwln-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
$ guix pack -L /data/src/guix-science -C zstd -f docker python-jupyterlab bash coreutils findutils -d --no-grafts
/gnu/store/xzp0qrd54sglhrggwn8myl7fd0mfhwln-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
--8<---------------cut here---------------end--------------->8---

> Looking at ca-certificate-bundle.drv, the first one lacks any form of
> glibc-utf8-locales as an input (neither in the .drv, nor via `guix gc
> --references <.drv> | grep glibc-`), so it’s clear it must fail. I’m
> not quite sure why though, since the actual builder still has a reference
> to the locales and sets LOCPATH. So I’m a little puzzled.

Could you send the contents of this “broken” ca-certificate-bundle.drv
as well as the ca-certificate-bundle-builder it refers to?

Thanks,
Ludo’.

Message #26 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Jack Hill <jackhill <at> jackhill.us>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 50264 <at> debbugs.gnu.org, Lars-Dominik Braun <lars <at> 6xq.net>
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Fri, 10 Sep 2021 12:17:01 -0400 (EDT)

[Message part 1 (text/plain, inline)]

On Fri, 10 Sep 2021, Ludovic Courtès wrote:

> Lars-Dominik Braun <lars <at> 6xq.net> skribis:
>
>> Alright, let’s see. The command I have been using is
>>
>>     guix pack -L . -C 'zstd' -f docker -S /bin=bin python-jupyterlab bash coreutils findutils
>>
>> with . being a checkout of guix-science (same applies to `guix
>> time-machine` though). The first time it’ll fail, but the second time
>> it succeeds.  Running `guix gc` makes it fail again the first time.
>>
>> The docker-pack.tar.zst.drv used for the first build is
>> different from the second one (different hash prefix). For me it’s
>> /gnu/store/r096cm3np7hbdn853ih35h1a5l39in4s-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
>> the first time and
>> /gnu/store/dywspxjshfjhc07i17hkcyrlq8kn7m07-python-jupyterlab-bash-coreutils-docker-pack.tar.zst.drv
>> the second time. YMMV.
>
> Wait, you run the same command twice and it leads a different .drv?
>
> The expected behavior is that the .drv is always the same:

Just to note that I've seen this two, where I get a different derivation 
the next time that works. Apologies for not reporting it at the time. It 
always seems to happen when I'm busy with something else, and since it's 
not reproducible… If I can get it to happen to me again, I'll collect more 
information as well.

Best,
Jack

Message #29 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Lars-Dominik Braun <lars <at> 6xq.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 50264 <at> debbugs.gnu.org
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Sat, 11 Sep 2021 09:08:04 +0200

[Message part 1 (text/plain, inline)]

Hi Ludo,

> Wait, you run the same command twice and it leads a different .drv?
yes, but only if I run `guix gc` before the first one, i.e. it’ll
build/downloads lots of packages before actually building the .drv.

> Could you send the contents of this “broken” ca-certificate-bundle.drv
> as well as the ca-certificate-bundle-builder it refers to?
This is with commit 8b2b027df9198cb5b22505813ada0c5e199929ed for the same
`guix pack` I mentioned earlier. I’m attaching:

Failure:
/gnu/store/kxkxji6m101ylgpa82jmxvmy5szpvk1k-ca-certificate-bundle.drv
/gnu/store/wahy9p8985vbcibwwjk3p73hxh82f90v-ca-certificate-bundle-builder

Success:
/gnu/store/rx6pqcpgwcyl6spfi8acgjzksrpqhac1-ca-certificate-bundle.drv
/gnu/store/wahy9p8985vbcibwwjk3p73hxh82f90v-ca-certificate-bundle-builder

(Yes, exactly the same builder input hash. I double-checked.)

Meanwhile a colleague of mine has seen the same problem during building a
`guix environment` too.

Cheers,
Lars

[kxkxji6m101ylgpa82jmxvmy5szpvk1k-ca-certificate-bundle.drv (text/plain, attachment)]

[wahy9p8985vbcibwwjk3p73hxh82f90v-ca-certificate-bundle-builder (text/plain, attachment)]

[rx6pqcpgwcyl6spfi8acgjzksrpqhac1-ca-certificate-bundle.drv (text/plain, attachment)]

Message #34 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: 50264 <at> debbugs.gnu.org
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Tue, 14 Sep 2021 15:08:19 +0200

Hi,

Lars-Dominik Braun <lars <at> 6xq.net> skribis:

> Failure:
> /gnu/store/kxkxji6m101ylgpa82jmxvmy5szpvk1k-ca-certificate-bundle.drv
> /gnu/store/wahy9p8985vbcibwwjk3p73hxh82f90v-ca-certificate-bundle-builder
>
> Success:
> /gnu/store/rx6pqcpgwcyl6spfi8acgjzksrpqhac1-ca-certificate-bundle.drv
> /gnu/store/wahy9p8985vbcibwwjk3p73hxh82f90v-ca-certificate-bundle-builder
>
> (Yes, exactly the same builder input hash. I double-checked.)

As discussed on IRC, I stumbled upon this bug while running:

  ./pre-inst-env guix system vm gnu/system/examples/desktop.tmpl

on commit 580984f2417853379d98ea927fd95c0f0fbe2c97.

The first attempt downloaded lots of substitutes and produced, as in
your case, a broken ca-certificate-bundle.drv: the ‘-builder’ script is
correct, but there are only ~80 derivation inputs instead of ~230.

The second run right after that produced the correct
ca-certificate-bundle.drv: same ‘-builder’ script, but all the
derivation inputs were there, including glibc-utf8-locales.drv.

I’ve tried to reproduce it running GC during or before the command, to
no avail so far.  If someone has an easy and reliable way to reproduce
it, please share!

I don’t have a good hypothesis so far as to where things go wrong…

Ludo’.

Message #37 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 50264 <at> debbugs.gnu.org, Lars-Dominik Braun <lars <at> 6xq.net>
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Tue, 14 Sep 2021 12:04:22 -0400

On Tue, Sep 14, 2021 at 03:08:19PM +0200, Ludovic Courtès wrote:
> Hi,
> 
> Lars-Dominik Braun <lars <at> 6xq.net> skribis:
> 
> > Failure:
> > /gnu/store/kxkxji6m101ylgpa82jmxvmy5szpvk1k-ca-certificate-bundle.drv
> > /gnu/store/wahy9p8985vbcibwwjk3p73hxh82f90v-ca-certificate-bundle-builder
> >
> > Success:
> > /gnu/store/rx6pqcpgwcyl6spfi8acgjzksrpqhac1-ca-certificate-bundle.drv
> > /gnu/store/wahy9p8985vbcibwwjk3p73hxh82f90v-ca-certificate-bundle-builder
> >
> > (Yes, exactly the same builder input hash. I double-checked.)
> 
> As discussed on IRC, I stumbled upon this bug while running:
> 
>   ./pre-inst-env guix system vm gnu/system/examples/desktop.tmpl
> 
> on commit 580984f2417853379d98ea927fd95c0f0fbe2c97.
> 
> The first attempt downloaded lots of substitutes and produced, as in
> your case, a broken ca-certificate-bundle.drv: the ‘-builder’ script is
> correct, but there are only ~80 derivation inputs instead of ~230.
> 
> The second run right after that produced the correct
> ca-certificate-bundle.drv: same ‘-builder’ script, but all the
> derivation inputs were there, including glibc-utf8-locales.drv.
> 
> I’ve tried to reproduce it running GC during or before the command, to
> no avail so far.  If someone has an easy and reliable way to reproduce
> it, please share!
> 
> I don’t have a good hypothesis so far as to where things go wrong…

I wonder if the problem began after the introduction of
bordeaux.guix.gnu.org, and if everyone who experiences the bug is using
both substitute servers.

I can't imagine what the actual problem is, but it does sound like the
dependency graph has somehow diverged...

Message #40 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Lars-Dominik Braun <lars <at> 6xq.net>
To: Leo Famulari <leo <at> famulari.name>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 50264 <at> debbugs.gnu.org
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Wed, 15 Sep 2021 08:40:54 +0200

Hi Leo,

> I wonder if the problem began after the introduction of
> bordeaux.guix.gnu.org, and if everyone who experiences the bug is using
> both substitute servers.
I started the Guix daemon with only the CI substitute server enabled
explicitly, disabled local discovery, ran a `guix gc` and tried again. It
still fails with exactly the same issue.

Lars

Message #43 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: 50264 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Wed, 15 Sep 2021 14:34:50 +0200

Hi,

Lars-Dominik Braun <lars <at> 6xq.net> skribis:

> I started the Guix daemon with only the CI substitute server enabled
> explicitly, disabled local discovery, ran a `guix gc` and tried again. It
> still fails with exactly the same issue.

I’m wondering whether this could be due to
fa81971cbae85b39183ccf8f51e8d96ac88fb4ac somehow.

If you have a reliable way to test this hypothesis, that’d be great.

Thanks,
Ludo’.

Message #48 received at 50264-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: 50264-done <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Wed, 15 Sep 2021 18:25:18 +0200

Hi!

Ludovic Courtès <ludo <at> gnu.org> skribis:

> I’m wondering whether this could be due to
> fa81971cbae85b39183ccf8f51e8d96ac88fb4ac somehow.

Confirmed!  In essence, ‘map/accumulate-builds’ would return the tail of
its ‘lst’ argument unprocessed when cutoff was reached (ouch!).  Fixed:

  https://git.savannah.gnu.org/cgit/guix.git/commit/?id=f72f4b48c6777da9465ab17baa6762476d6cb270

Let me know if you still experience fishy behavior!

Ludo’.

Message #51 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: 50264 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Wed, 15 Sep 2021 18:30:04 +0200

[Message part 1 (text/plain, inline)]

And for posterity, here’s the script I used to reproduce the problem:
it’d pick 10 packages at random and call ‘ca-certificate-bundle’ on them.

Since this bug depends on what’s in the store, I’d run it on my laptop,
which only contains a fraction of the 18K packages in Guix, so it would
reproduce the bug after a couple of iterations.

That, together with the inevitable ‘pk’ calls plus a bit of chance, voilà!

Ludo’.

[ca-certificate-bundle.scm (text/plain, inline)]

;; https://issues.guix.gnu.org/50264

(use-modules (gnu) (guix)
             (guix profiles) (guix monads)
             (ice-9 match) (srfi srfi-1))

(define (all-packages)
  "Return the list of all the packages, public or private, omitting only
superseded packages."
  (fold-packages (lambda (package lst)
                   (match (package-replacement package)
                     (#f (cons package lst))
                     (replacement
                      (append (list replacement package) lst))))
                 '()
                 #:select? (negate package-superseded)))

(define (random-seed)
  (logxor (getpid) (car (gettimeofday))))

(define shuffle                                   ;from offload.scm
  (let ((state (seed->random-state (random-seed))))
    (lambda (lst)
      "Return LST shuffled (using the Fisher-Yates algorithm.)"
      (define vec (list->vector lst))
      (let loop ((result '())
                 (i (vector-length vec)))
        (if (zero? i)
            result
            (let* ((j (random i state))
                   (val (vector-ref vec j)))
              (vector-set! vec j (vector-ref vec (- i 1)))
              (loop (cons val result) (- i 1))))))))

(define (test packages)
  (pk 'testing-packages (map package-full-name packages))
  (let ((manifest (packages->manifest packages)))
    (with-store store
      (let ((drv (run-with-store store
                   (ca-certificate-bundle manifest))))
        (pk 'drv drv)
        (unless (find (lambda (input)
                        (let ((drv (derivation-input-derivation input)))
                          (string-prefix? "glibc-utf8-locales"
                                          (derivation-name drv))))
                      (derivation-inputs drv))
          (pk 'drv drv (derivation-inputs drv))
          (display-backtrace (make-stack #t) (current-error-port))
          (error "bah!" drv))
        (newline) (newline)))))

(let loop ((packages (shuffle (all-packages))))
  (test (take packages 10))
  (loop (drop packages 10)))

Message #54 received at 50264 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: 50264 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#50264: ca-certificate-bundle fails to build
Date: Wed, 15 Sep 2021 18:30:37 +0200

[Message part 1 (text/plain, inline)]

And for posterity, here’s the script I used to reproduce the problem:
it’d pick 10 packages at random and call ‘ca-certificate-bundle’ on them.

Since this bug depends on what’s in the store, I’d run it on my laptop,
which only contains a fraction of the 18K packages in Guix, so it would
reproduce the bug after a couple of iterations.

That, together with the inevitable ‘pk’ calls plus a bit of chance, voilà!

Ludo’.

[ca-certificate-bundle.scm (text/plain, inline)]

;; https://issues.guix.gnu.org/50264

(use-modules (gnu) (guix)
             (guix profiles) (guix monads)
             (ice-9 match) (srfi srfi-1))

(define (all-packages)
  "Return the list of all the packages, public or private, omitting only
superseded packages."
  (fold-packages (lambda (package lst)
                   (match (package-replacement package)
                     (#f (cons package lst))
                     (replacement
                      (append (list replacement package) lst))))
                 '()
                 #:select? (negate package-superseded)))

(define (random-seed)
  (logxor (getpid) (car (gettimeofday))))

(define shuffle                                   ;from offload.scm
  (let ((state (seed->random-state (random-seed))))
    (lambda (lst)
      "Return LST shuffled (using the Fisher-Yates algorithm.)"
      (define vec (list->vector lst))
      (let loop ((result '())
                 (i (vector-length vec)))
        (if (zero? i)
            result
            (let* ((j (random i state))
                   (val (vector-ref vec j)))
              (vector-set! vec j (vector-ref vec (- i 1)))
              (loop (cons val result) (- i 1))))))))

(define (test packages)
  (pk 'testing-packages (map package-full-name packages))
  (let ((manifest (packages->manifest packages)))
    (with-store store
      (let ((drv (run-with-store store
                   (ca-certificate-bundle manifest))))
        (pk 'drv drv)
        (unless (find (lambda (input)
                        (let ((drv (derivation-input-derivation input)))
                          (string-prefix? "glibc-utf8-locales"
                                          (derivation-name drv))))
                      (derivation-inputs drv))
          (pk 'drv drv (derivation-inputs drv))
          (display-backtrace (make-stack #t) (current-error-port))
          (error "bah!" drv))
        (newline) (newline)))))

(let loop ((packages (shuffle (all-packages))))
  (test (take packages 10))
  (loop (drop packages 10)))

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.