GNU bug report logs - #39258
Faster guix search using an sqlite cache

Previous Next

Full log

Message #185 received at 39258 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Pierre Neidhardt <mail <at> ambrevar.xyz>, 39258 <at> debbugs.gnu.org
Subject: Re: [PATCH v2 2/3] gnu: Generate Xapian package search index.
Date: Mon, 9 Mar 2020 19:19:27 +0100

On Sat, 7 Mar 2020 at 14:31, Arun Isaac <arunisaac <at> systemreboot.net> wrote:

> diff --git a/gnu/packages.scm b/gnu/packages.scm
> index d22c992bb1..c8e221de68 100644
> --- a/gnu/packages.scm
> +++ b/gnu/packages.scm

[...]

> @@ -426,6 +429,43 @@ reducing the memory footprint."
>                                 #:opts '(#:to-file? #t)))))
>    cache-file)
>
> +(define %package-search-index
> +  ;; Location of the package search-index
> +  "/lib/guix/package-search.index")
> +
> +(define (generate-package-search-index directory)
> +  "Generate under DIRECTORY a Xapian index of all the available packages."
> +  (define db-path
> +    (string-append directory %package-search-index))
> +
> +  (mkdir-p (dirname db-path))
> +  (call-with-writable-database db-path
> +    (lambda (db)
> +      (fold-packages (lambda (package _)
> +                       (let* ((idterm (string-append "Q" (package-name package)))
> +                              (doc (make-document #:data (string-trim-right
> +                                                          (call-with-output-string
> +                                                            (cut package->recutils package <>))
> +                                                          #\newline)
> +                                                  #:terms `((,idterm . 0))))
> +                              (term-generator (make-term-generator #:stem (make-stem "en")
> +                                                                   #:document doc)))
> +                         (for-each (match-lambda
> +                                     ((field . weight)
> +                                      (match (field package)
> +                                        ((? string? str)
> +                                         (index-text! term-generator str
> +                                                      #:wdf-increment weight))
> +                                        ((lst ...)
> +                                         (for-each (cut index-text! term-generator <>
> +                                                        #:wdf-increment weight)
> +                                                   lst)))
> +                                      (replace-document! db idterm doc)))
> +                                   %package-metrics)))
> +                     #f)))
> +
> +  db-path)

If I understand correctly, the index is stored with a weight coming
from '%package-metrics', right? Well, I am not convinced it is the
correct way but I have not tried by myself yet. :-)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.