GNU bug report logs - #39258
Faster guix search using an sqlite cache

Previous Next

Full log

View this message in rfc822 format

From: zimoun <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Arun Isaac <arunisaac <at> systemreboot.net>, Pierre Neidhardt <mail <at> ambrevar.xyz>, 39258 <at> debbugs.gnu.org
Subject: [bug#39258] benchmark search: default vs v2 vs v3
Date: Thu, 30 Apr 2020 15:10:40 +0200

Hi Ludo,

On Sun, 26 Apr 2020 at 17:49, Ludovic Courtès <ludo <at> gnu.org> wrote:

> That’s only one criterion among others.  I hear the argument that 25s is
> “nothing” compared to the rest, but it’s really a tradeoff.  Like, if I
> spent a day optimizing ‘guix pull’ and managed to save 25s, I would find
> it nice.  :-)

I am not sure to understand all what "guix pull" does.
Does "guix pull" compile all the scheme files under 'gnu/'? Probably
only recompiles the "new" files?

I do not know if it makes sense, but I just note this difference:

 1. Search without compiling of all files under 'gnu/packages/'
 2. Compile all the files under 'gnu/packages/' then search
 3. Search with only the file gnu/packages/emacs-xyz.scm not compiled
(all the other files are compiled)
 4. Compile the file above and then search

3b and 4b with gnu/packages/cobol.scm which is smaller than emacs-xyz.scm.


Results:

1) 1m43.312s
2) 0m1.301s (but 9m51.801s compiling)

3) 0m6.526s
4) 0m1.389s (1m8.670s compiling)

3b) 0m0.921s
4b) 0m0.924s (0m1.884s compiling)

Therefore, an option to reduce the time when pulling should to relax
the "compilation" for 'gnu/packages/' and 'gnu/services'; something
less optimized since the packages and services "just" need to be
transformed into bytecode to improve IO when reading them. Perhaps I
miss a point...

And maybe, it is similar than what Andy Wingo is proposing in [1].

[1] https://lists.gnu.org/archive/html/guix-devel/2020-04/msg00444.html


Cheers,
simon

--8<---------------cut here---------------start------------->8---
find gnu/packages -name "*.scm" -type f -exec touch {} \;
time ./pre-inst-env guix search gmsh | recsel -C -p name

;;; note: source file /home/simon/src/guix/wk/tmp/gnu/packages/abduco.scm
;;;       newer than compiled /home/simon/src/guix/wk/tmp/gnu/packages/abduco.go

[...]

;;; note: source file /home/simon/src/guix/wk/tmp/gnu/packages/zwave.scm
;;;       newer than compiled /home/simon/src/guix/wk/tmp/gnu/packages/zwave.go
name: gmsh

real    1m43.312s
user    2m19.318s
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
find gnu/packages -name "*.scm" -type f -exec touch {} \;
time make -j4 && time ./pre-inst-env guix search gmsh | recsel -C -p name

make  all-recursive
make[1]: Entering directory '/home/simon/src/guix/wk/tmp'
Making all in po/guix
make[2]: Entering directory '/home/simon/src/guix/wk/tmp/po/guix'
make[2]: Leaving directory '/home/simon/src/guix/wk/tmp/po/guix'
Making all in po/packages
make[2]: Entering directory '/home/simon/src/guix/wk/tmp/po/packages'
make[2]: Leaving directory '/home/simon/src/guix/wk/tmp/po/packages'
make[2]: Entering directory '/home/simon/src/guix/wk/tmp'
Compiling Scheme modules...
[  0%] LOAD     gnu/packages/abduco.scm
;;; note: source file ./gnu/packages/abduco.scm
;;;       newer than compiled /home/simon/src/guix/wk/tmp/gnu/packages/abduco.go

[...]

[100%] GUILEC   gnu/packages/zwave.go
make[2]: Leaving directory '/home/simon/src/guix/wk/tmp'
make[1]: Leaving directory '/home/simon/src/guix/wk/tmp'

real    9m51.801s
user    29m18.938s
sys     0m5.822s
name: gmsh

real    0m1.301s
user    0m1.266s
sys     0m0.101s
--8<---------------cut here---------------end--------------->8---

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.