GNU bug report logs - #39258
Faster guix search using an sqlite cache

Previous Next

Full log

Message #47 received at 39258 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: 39258 <at> debbugs.gnu.org
Cc: Arun Isaac <arunisaac <at> systemreboot.net>, mail <at> ambrevar.xyz, ludo <at> gnu.org,
 zimon.toutoune <at> gmail.com
Subject: [PATCH 0/4] Xapian for Guix package search
Date: Fri, 28 Feb 2020 02:11:46 +0530

Hi,

I have finally got xapian working for package search. Some comments follow.

* Speed improvement

Despite search-package-index in gnu/packages.scm taking only around 1.5ms, I
see an overall speedup in `guix search` of only a factor of 2 -- from around
2s to around 1s. I wonder what else in `guix search` is taking up so much
time.

* Currently indexing only the package descriptions

In this patchset, I have only indexed the package descriptions. In the next
version of this patchset, I will index all other terms as specified in
%package-metrics of guix/ui.scm.

* Should I add guile-xapian as a propagated input to guix in
  gnu/packages/package-management.scm?

* Drop regexp search support

In this patchset, I have retained the older regexp search support. But, I
think we should drop it and only have xapian search. In cases where the search
index is not authoritative, we can build an in-memory xapian search index on
the fly and use it to search. This will slow down the search, but will ensure
our search results are consistent and do not depend on the authoritativeness
of the search index.

* Commit messages

Except for patch 1, I am not sure what prefixes (build-self, gnu, etc.) to use
in the first line of the commit message. Some advice there would be helpful.

Regards,
Arun.

Arun Isaac (4):
  gnu: Add guile-xapian.
  build-self: Add guile-xapian to Guix dependencies.
  gnu: Generate xapian package search index.
  gnu: Use xapian index for package search.

 build-aux/build-self.scm   | 11 ++++++++
 gnu/packages.scm           | 44 ++++++++++++++++++++++++++++-
 gnu/packages/guile-xyz.scm | 50 ++++++++++++++++++++++++++++++++-
 guix/channels.scm          | 34 ++++++++++++++++++++++-
 guix/scripts/package.scm   | 57 ++++++++++++++++++++++----------------
 guix/self.scm              |  7 ++++-
 6 files changed, 175 insertions(+), 28 deletions(-)

-- 
2.23.0

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.