GNU bug report logs - #39258
Faster guix search using an sqlite cache

Previous Next

Full log

View this message in rfc822 format

From: zimoun <zimon.toutoune <at> gmail.com>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: Ludovic Courtès <ludo <at> gnu.org>, Pierre Neidhardt <mail <at> ambrevar.xyz>, 39258 <at> debbugs.gnu.org
Subject: [bug#39258] [PATCH 0/4] Xapian for Guix package search
Date: Fri, 28 Feb 2020 13:36:06 +0100

Hi Arun,

Really cool! Thank you!


On Thu, 27 Feb 2020 at 21:42, Arun Isaac <arunisaac <at> systemreboot.net> wrote:

> * Speed improvement
>
> Despite search-package-index in gnu/packages.scm taking only around 1.5ms, I
> see an overall speedup in `guix search` of only a factor of 2 -- from around
> 2s to around 1s. I wonder what else in `guix search` is taking up so much
> time.

Interesting... maybe an hidden 'fold-packages'?
Well, I have not yet looked into your code.


> * Currently indexing only the package descriptions
>
> In this patchset, I have only indexed the package descriptions. In the next
> version of this patchset, I will index all other terms as specified in
> %package-metrics of guix/ui.scm.

Yes, it appears to me a detail that should be easy to fix. I mean, it
does not seems blocking.


> * Should I add guile-xapian as a propagated input to guix in
>   gnu/packages/package-management.scm?

IMHO, yes.
I mean, I guess. :-)


> * Drop regexp search support
>
> In this patchset, I have retained the older regexp search support. But, I
> think we should drop it and only have xapian search. In cases where the search
> index is not authoritative, we can build an in-memory xapian search index on
> the fly and use it to search. This will slow down the search, but will ensure
> our search results are consistent and do not depend on the authoritativeness
> of the search index.

I understand why you have turned off the regexp support. It is not
necessary at the first experimentation to see if it is worth the
addition or not.
So, before investigating how some better regexp could be used with
Xapian, let start to benchmark Xapian vs plain 'fold-packages'.


> * Commit messages
>
> Except for patch 1, I am not sure what prefixes (build-self, gnu, etc.) to use
> in the first line of the commit message. Some advice there would be helpful.

I cannot help. )-:


All the best,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.