GNU bug report logs - #56442
gnus-search-run-search: Hits notmuch command line length limits

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 56442 in the body.
You can then email your comments to 56442 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sean Whitton <spwhitton <at> spwhitton.name>
To: bug-gnu-emacs <at> gnu.org
Subject: gnus-search-run-search: Hits notmuch command line length limits
Date: Thu, 07 Jul 2022 09:56:05 -0700

[Message part 1 (text/plain, inline)]

Hello,

I'm running a pretty innocent notmuch query over a fairly small Maildir:

"((List:debian-devel.lists.debian.org) or ... or
(List:debian-haskell.lists.debian.org) or
(List:debconf-discuss.lists.debian.org)) and (not path:annex/**)"

but gnus-search-run-search fails to return any results.  The reason is
that Gnus first runs the query with --output=threads to obtain a list of
thread ids, and then runs another query with --output=files and a query
constructed from the output of the first query: "thread:000000000000d9d0
or thread:000000000000d9e0 or thread:000000000000d268 or ..."

The resulting command fails completely:

    emacs: /usr/bin/notmuch: Argument list too long

Instead of running two searches like this, we can just surround the
whole query like this: "thread:{QUERY}".  The manual says it's exactly
equivalent:

    ... the user should think of the query thread:{<something>} as
    expanding to all of the thread IDs which match <something>; not‐
    much then performs a second search using the expanded query.

This should be faster, too, with only running a single external command.
Here is the patch I'm thinking I'll apply, if anyone has comments.

-- 
Sean Whitton

[0001-Use-notmuch-thread-operator-instead-of-performing-tw.patch (text/x-diff, attachment)]

Message #8 received at 56442 <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Sean Whitton <spwhitton <at> spwhitton.name>
Cc: notmuch <at> notmuchmail.org, 56442 <at> debbugs.gnu.org
Subject: Re: bug#56442: gnus-search-run-search: Hits notmuch command line
 length limits
Date: Sat, 09 Jul 2022 09:11:22 -0700

Sean Whitton <spwhitton <at> spwhitton.name> writes:

> Hello,
>
> I'm running a pretty innocent notmuch query over a fairly small Maildir:
>
> "((List:debian-devel.lists.debian.org) or ... or
> (List:debian-haskell.lists.debian.org) or
> (List:debconf-discuss.lists.debian.org)) and (not path:annex/**)"
>
> but gnus-search-run-search fails to return any results.  The reason is
> that Gnus first runs the query with --output=threads to obtain a list of
> thread ids, and then runs another query with --output=files and a query
> constructed from the output of the first query: "thread:000000000000d9d0
> or thread:000000000000d9e0 or thread:000000000000d268 or ..."
>
> The resulting command fails completely:
>
>     emacs: /usr/bin/notmuch: Argument list too long
>
> Instead of running two searches like this, we can just surround the
> whole query like this: "thread:{QUERY}".  The manual says it's exactly
> equivalent:
>
>     ... the user should think of the query thread:{<something>} as
>     expanding to all of the thread IDs which match <something>; not‐
>     much then performs a second search using the expanded query.
>
> This should be faster, too, with only running a single external command.
> Here is the patch I'm thinking I'll apply, if anyone has comments.

Huh, I tried this a couple months ago with a more complicated query,
like:

thread:{from:bob or from:jane}

and I remember notmuch barking at me about spaces or the "or" or
something -- anyway I got the impression that it couldn't accept
multi-part queries inside the "thread:{}" syntax. But looking at your
patch maybe I just needed to quote differently?

Anyway I'll give this a test this weekend. It would be *very* nice if we
could use this syntax rather than the ugly home-grown one.

Thanks for the report!

Eric

Message #11 received at 56442 <at> debbugs.gnu.org (full text, mbox):

From: Sean Whitton <spwhitton <at> spwhitton.name>
To: Eric Abrahamsen <eric <at> ericabrahamsen.net>
Cc: notmuch <at> notmuchmail.org, 56442 <at> debbugs.gnu.org
Subject: Re: bug#56442: gnus-search-run-search: Hits notmuch command line
 length limits
Date: Sat, 09 Jul 2022 11:01:42 -0700

Hello,

On Sat 09 Jul 2022 at 09:11AM -07, Eric Abrahamsen wrote:

> But looking at your patch maybe I just needed to quote differently?

Right, notmuch has this unusual " -> "" quoting thing.

> Anyway I'll give this a test this weekend. It would be *very* nice if
> we could use this syntax rather than the ugly home-grown one.

Let me know the results of your test, and if it's also continuing to
work for me by then, I'll install the change.  Thanks for taking a look.

-- 
Sean Whitton

Message #16 received at 56442-done <at> debbugs.gnu.org (full text, mbox):

From: Sean Whitton <spwhitton <at> spwhitton.name>
To: 56442-done <at> debbugs.gnu.org, Eric Abrahamsen <eric <at> ericabrahamsen.net>
Subject: Re: bug#56442: gnus-search-run-search: Hits notmuch command line
 length limits
Date: Thu, 14 Jul 2022 11:50:13 -0700

Hello,

On Sat 09 Jul 2022 at 11:01AM -07, Sean Whitton wrote:

> On Sat 09 Jul 2022 at 09:11AM -07, Eric Abrahamsen wrote:
>
>> Anyway I'll give this a test this weekend. It would be *very* nice if
>> we could use this syntax rather than the ugly home-grown one.
>
> Let me know the results of your test, and if it's also continuing to
> work for me by then, I'll install the change.  Thanks for taking a look.

I've been testing it with a variety of complex queries for a week now,
and you indicated you were happy with making a change like this, so I've
gone ahead and pushed to master.

-- 
Sean Whitton

Message #19 received at 56442-done <at> debbugs.gnu.org (full text, mbox):

From: Eric Abrahamsen <eric <at> ericabrahamsen.net>
To: Sean Whitton <spwhitton <at> spwhitton.name>
Cc: 56442-done <at> debbugs.gnu.org
Subject: Re: bug#56442: gnus-search-run-search: Hits notmuch command line
 length limits
Date: Thu, 14 Jul 2022 15:11:02 -0700

On 07/14/22 11:50 AM, Sean Whitton wrote:
> Hello,
>
> On Sat 09 Jul 2022 at 11:01AM -07, Sean Whitton wrote:
>
>> On Sat 09 Jul 2022 at 09:11AM -07, Eric Abrahamsen wrote:
>>
>>> Anyway I'll give this a test this weekend. It would be *very* nice if
>>> we could use this syntax rather than the ugly home-grown one.
>>
>> Let me know the results of your test, and if it's also continuing to
>> work for me by then, I'll install the change.  Thanks for taking a look.
>
> I've been testing it with a variety of complex queries for a week now,
> and you indicated you were happy with making a change like this, so I've
> gone ahead and pushed to master.

Sounds good, thanks!

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.