GNU bug report logs - #77833
Xapian cache/search proof of concept

Previous Next

Full log

View this message in rfc822 format

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Arun Isaac <arunisaac <at> systemreboot.net>, noe <at> xn--no-cja.eu, 77833 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [bug#77833] Xapian cache/search proof of concept
Date: Thu, 17 Apr 2025 12:37:34 +0200

Hi,

On Wed, 16 Apr 2025 at 18:12, Arun Isaac <arunisaac <at> systemreboot.net> wrote:

> If there's some way to hide the indexing step from the user, that would
> be good. For example, one way could be to build the index the first time
> search is run. You'll have to figure out some way to make this a smooth
> user experience.

Well, the design isn’t obvious to me: that’s a typical case of cache
invalidation problem and we all know, there are two hard problems:
naming thing and cache invalidation. ;-)

Somehow, the best would to have the ability to hook “guix pull” (or
“guix time-machine”) with this indexing step.  Currently, I do not think
it is doable, is it?   But that could be helpful: trigger some actions
(hook) registered by some Guix extensions.  Well, it seems out of the
scope at first. :-)


> Remember that the user may jump back and forth between multiple guix
> profiles. So, multiple indices may be necessary.

Yes.  Today, I’m more interested in being able to search in all the
history than in having faster search. :-) I mean, I see faster search as
a collateral “damage” of searching in all the Guix history
revisions. ;-)

Somehow, what I started^W failed – because I’m a procrastinator ;-) –
long ago was to be able to substitute some Xapian database.

For example, the extension was named “guix chase”, IIRC, and I wanted to
have “guix chase pull” which updates some local Xapian database.  Then,
I could run “guix chase search hello” and find various versions of
’hello’ with some associated Guix revisions.  Bah in the middle I
entered in a blackhole. Whatever.

Today, searching across the Guix history is really annoying.  Somehow, I
do:

    $ git -C ~/src/guix/guix log --format="%h %s" | grep 'gnu: bowtie:'
    a47a90b900 gnu: bowtie: Remove reference to %outputs.
    f336cc4fe7 gnu: bowtie: Replace invalid characters.
    e5a26a1f02 gnu: bowtie: Remove trailing #T.
    2ec601580b gnu: bowtie: Use TBB 2020.
    21c837405a gnu: bowtie: Update to 2.3.4.3.
    06e372360e gnu: bowtie: Use 'modify-phases'.
    d6e63cf31c gnu: bowtie: Update to 2.3.2.
    2642231b39 gnu: bowtie: Update to 2.2.9.
    0047d26a22 gnu: bowtie: Update to 2.2.6.
    241e122193 gnu: bowtie: fix build errors

which is not super handy.  Well, it was somehow an idea behind the
Magali’s Outreachy internship implementing some “guix git log”: make it
a bit more handy.

Anyway.

Thanks Noé for working on that.  Let me know where is the Git
repository. :-)

Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.