GNU bug report logs - #39258
Faster guix search using an sqlite cache

Previous Next

Full log

Message #209 received at 39258 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 39258 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>, mail <at> ambrevar.xyz, ludo <at> gnu.org,
 zimon.toutoune <at> gmail.com
Subject: [PATCH v3 0/3] Package metadata cache for guix search
Date: Fri, 27 Mar 2020 21:56:51 +0530

Hi everyone,

This is v3 of my attempt to make guix search faster. In this version, I have
abandoned use of xapian. Instead I build a cache of the metadata of all
packages in a profile hook. Then, I use that cache to search and display
search results. This way, package guile modules are not loaded during guix
search.

Speedup is around 2x. Both measurements below are with a warm cache.

--8<---------------cut here---------------start------------->8---
$ time guix search inkscape

real	0m1.722s
user	0m1.776s
sys	0m0.097s
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
$ time /tmp/test/bin/guix search inkscape

real	0m0.749s
user	0m0.770s
sys	0m0.020s
--8<---------------cut here---------------end--------------->8---

This patchset does not affect the search API nor does it improve the relevance
of search results. If there is interest in this approach, I'll complete this
patchset properly. But, in the long run, I do think we should aim to get
xapian or the like for guix search. WDYT?

Unfortunately, generate-package-metadata-cache takes 43 seconds to build the
cache on my relatively slow computer. Performance should be better on other
people's machines.

Meanwhile, it would still be useful if someone built patchset v2 on their
machine and reported the time it took to build the xapian index.

* How to test this patchset

Apply patches and build as usual. Do a guix pull into a temporary profile.

$ ./pre-inst-env guix pull --url=$PWD --branch=the-name-of-the-branch-you-applied-patches-to -p /tmp/test

Then, run guix search from the built profile

$ /tmp/test/bin/guix search inkscape

Thanks!

Arun Isaac (3):
  guix: Generate package metadata cache.
  guix: Search package metadata cache.
  guix: Use package metadata cache for package search.

 gnu/packages.scm         |  88 +++++++++++++++++++++++++-
 guix/channels.scm        |  34 +++++++++-
 guix/packages.scm        |  32 ++++++++++
 guix/scripts/package.scm |   5 +-
 guix/ui.scm              | 132 ++++++++++++++++++++++++++++++++++++---
 5 files changed, 277 insertions(+), 14 deletions(-)

-- 
2.25.1

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.