GNU bug report logs - #39258
Faster guix search using an sqlite cache

Previous Next

Full log

View this message in rfc822 format

From: Ludovic Courtès <ludo <at> gnu.org>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: mail <at> ambrevar.xyz, 39258 <at> debbugs.gnu.org, zimon.toutoune <at> gmail.com
Subject: [bug#39258] [PATCH 0/4] Xapian for Guix package search
Date: Thu, 05 Mar 2020 17:46:01 +0100

Hello Arun,

Arun Isaac <arunisaac <at> systemreboot.net> skribis:

> * Speed improvement
>
> Despite search-package-index in gnu/packages.scm taking only around 1.5ms, I
> see an overall speedup in `guix search` of only a factor of 2 -- from around
> 2s to around 1s. I wonder what else in `guix search` is taking up so much
> time.

Note that ‘guix search’ time is largely dominated by I/O.  On my laptop,
I get (first measurement is cold cache, second one is warm cache):

--8<---------------cut here---------------start------------->8---
$ sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
$ time guix search foo >/dev/null

real    0m2.631s
user    0m1.134s
sys     0m0.124s
$ time guix search foo >/dev/null

real    0m0.836s
user    0m1.027s
sys     0m0.053s
--8<---------------cut here---------------end--------------->8---

It’s hard to do better on the warm cache case because at this level,
there may be other things to optimize having little to do with searching
itself.

Note that this is on an SSD; the cold-cache case must be worse on NFS or
on a spinning disk, and there we could gain a lot.

I think we should weigh the pros and cons on all these aspects: speed,
complexity and maintenance cost, search result quality, search features,
etc.

Thanks,
Ludo’.

PS: I have not yet looked at the whole series as I’m just coming back to
    the keyboard.  :-)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.