GNU bug report logs - #39258
Faster guix search using an sqlite cache

Previous Next

Full log

View this message in rfc822 format

From: Ludovic Courtès <ludo <at> gnu.org>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Arun Isaac <arunisaac <at> systemreboot.net>, Pierre Neidhardt <mail <at> ambrevar.xyz>, 39258 <at> debbugs.gnu.org
Subject: [bug#39258] benchmark search: default vs v2 vs v3
Date: Sun, 26 Apr 2020 17:49:06 +0200

Hi,

zimoun <zimon.toutoune <at> gmail.com> skribis:

>  1) around 25 seconds added to "guix pull"... but I am more than often
> waiting around 10 minutes when pulling.
>  2) the speedup is clear: more than 2x.

Nice!

It does seem like Arun’s v3 (or maybe even v2) would work nicely.

> The question is the tradeoff between: the slowdown of pull vs the
> speedup of search. What is acceptable?

That’s only one criterion among others.  I hear the argument that 25s is
“nothing” compared to the rest, but it’s really a tradeoff.  Like, if I
spent a day optimizing ‘guix pull’ and managed to save 25s, I would find
it nice.  :-)

> $ time guix pull -C ~/.config/guix/default-channels.scm

It also depends on what’s in that file, of course.

> Then, if I pull back from 8cf6d15 to '--commit=a357849f5b' then it takes:
>
> real    2m13.693s
> user    1m37.418s
> sys     0m0.666s

For me:

--8<---------------cut here---------------start------------->8---
$ guix describe
Generacio 139   Apr 13 2020 21:50:08    (nuna)
  guix bad368b
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: bad368b0d794689f3a8a11b58f1ea4987938682e
$ time guix pull -p /tmp/test --commit=bad368b0d794689f3a8a11b58f1ea4987938682e
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Building from this channel:
  guix      https://git.savannah.gnu.org/git/guix.git   bad368b

[...]

real    0m57.916s
user    1m1.017s
sys     0m0.609s
--8<---------------cut here---------------end--------------->8---

(On a 2.6 GHz i7 though.)

> Well, let remove the profiles and garbage collect the index files:
>
> rm /tmp/default /tmp/v{2,3}*
> guix gc -D \
>    /gnu/store/g5c08vqsv31nkn2r0hr32dbrkhf3cvd8-guix-package-cache \
>    /gnu/store/8xbzhn81hmshagbgazmnr7xfps1cdsa3-guix-package-search-index \
>    /gnu/store/8j78b5c4ddic21gcx7wpbq2akjn7x7mr-guix-package-metadata-cache

Could you do, for v2 and v3:

  time guix build /gnu/store/…-package-metadata-cache.drv --check

?

That we’ll give us the exact cost of that part.  It’ll be interesting
especially in the Xapian case, which we expected to be higher.

Thanks for the insightful benchmarks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.