GNU bug report logs - #66268
Guix makes invalid assumptions regarding guile-git guarantees leading to guix pull failing

Previous Next

Full log

Message #39 received at 66268 <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Jack Hill <jackhill <at> jackhill.us>, 66268 <at> debbugs.gnu.org
Subject: Re: bug#66268: Guix makes invalid assumptions regarding guile-git
 guarantees leading to guix pull failing
Date: Sat, 12 Apr 2025 13:07:56 +0200

[Message part 1 (text/plain, inline)]

Hello Luvodic,

thank you for pinging me about this! :)  See more below.

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi Tomas,
>
> Ludovic Courtès <ludo <at> gnu.org> skribis:
>
>>   2. Since Guile-Git has been pretending to provide that eq?-ness
>>      guarantee, I’m tempted to fix the problem in Guile-Git, by having a
>>      per-repository lookup table (a weak-value hash table mapping OIDs
>>      to commits).
>
> I gave it a try: see Guile-Git commit
> cd91dc908ac4b215bc87a97455ff64ed4d89b721.
>
> If it works for you, we can tag a Guile-Git release soonish.

I went ahead and gave f20c79dbed9aff763b1bac4b0e570103516c15ff a try.  I
have used the same reproduction script for checking as last time (for
your convenience, I have attached it to this email).  I have run the
script against the test repository from last time[0].

So, the good news is that it seems to work, the checks at the start of
the script now pass.  Full output from both runs (the current guile-git
0.9.0 and the commit being tested) are attached.

However, now the bad news.  The performance of the new version is
atrocious.  Run from the previous version finished in slightly over 6
minutes (6:13.37), run from the commit above *did not finish* after 16
hours (16:05:27) and I had to kill it.  So we are looking at a slowdown
of *at least* 155.3x, probably significantly worse (see below).

So technically I cannot even fully run the reproduction script, since it
just does not finish in reasonable time now (that is why above I have
stated "checks at the start of the script").

Memory wise, the previous version had peak memory usage (RSS) 2.4 GB,
the new version had 1 GB, but remember it did not even finish, so this
is just a partial result.  I would like to be able to tell the memory
overhead of the new caching layer, but, since the script does not
finish, I cannot.

From the memory comparison, assuming the caching has zero memory impact,
that would mean we got about 42% through the list of commits to process
(in 16h).  So the complete run can be estimated to take roughly ~38
hours (compare to the previous ~6 minutes).

What are your thoughts here?  Did I make any mistakes in this analysis?

Tomas

0: https://git.sr.ht/~graywolf/guix-guile-git-repro

[repro.scm (application/x-guile, attachment)]

[guile-git-0.9.0 (plain/text, attachment)]

[guile-git-f20c79dbed9aff763b1bac4b0e570103516c15ff (plain/text, attachment)]

[Message part 5 (text/plain, inline)]

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.