GNU bug report logs - #42009
package cache build is not deterministic

Previous Next

Full log

View this message in rfc822 format

From: zimoun <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 42009 <at> debbugs.gnu.org, Tobias Geerinckx-Rice <me <at> tobias.gr>, Marinus <marinus.savoritias <at> disroot.org>
Subject: bug#42009: package.cache not deterministic
Date: Mon, 29 Jun 2020 17:39:42 +0200

Hi Ludo,

On Mon, 29 Jun 2020 at 14:34, Ludovic Courtès <ludo <at> gnu.org> wrote:

> I realize I was a bit off-topic: I was commenting on the more general
> issue of .go non-reproducibility.
>
> The problem with the package cache seems to be different.  Sorry for the
> confusion!

Well, the .go non-reproducibility is nonetheless interesting. :-)

> So, surprisingly, it’s not just an ordering issue: the caches do contain
> different pieces of information.

Interestingly, on my machine with 8ea91d0, I have 12 differences but
none of them is Python.  Instead, I have: "rust", "ocaml",
"mingw-w64-i686", "clang-toolchain", "clang-runtime", "linux-libre",
"icedtea", "gcc-objc++", "gcc-objc", "bdb", "rust-lazy-static",
"gcc-toolchain".  All are aliases and python-2 is too but does not
appear...
On the other hand, these 2 aliases do not appear either in your list.

--8<---------------cut here---------------start------------->8---
(define-public clang clang-9)
(define-public clang-toolchain clang-toolchain-9)
--8<---------------cut here---------------end--------------->8---

The question is: do we need to declare public e.g. clang-9?  Declare
clang is it not enough already?

For example, ocaml-4.09 is not used outside gnu/packages/ocalm.scm.
Idem for ghc-8.6, etc..

> This patch solves the ordering issue

The 'sort' will not help for improving the speed of "guix pull". ;-)

> But it’s not quite right because the order in which variables are
> traversed is still non-deterministic, so between two runs of
> ‘generate-package-cache’, caches differ like this:

It depends on the eddy current in the upper atmosphere. :-)
https://xkcd.com/378/

Well, it finds one or the other first and 'expand-cache' stores the
first considering the second as "seen", isn't it?

> The problem has to do with aliases: we don’t always see the same
> variable first.  So we have to sort before calling ‘expand-cache’.

The question is why it is not always the same variable first.  Well,
IMHO, it could comes from:
 - either 'scandir*' in 'scheme-files' because all the other functions
seem "pure" and this one not;
 - either 'fold-module-public-variables*' where 'seen' is one or the other.
Or if the .go files are not deterministic, especially about 'gensym',
it should explain why one symbol appears sometimes first.

Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.