GNU bug report logs -
#26710
project-find-regexp blocks the UI
Previous Next
Full log
Message #31 received at 26710 <at> debbugs.gnu.org (full text, mbox):
On 01.05.2017 10:20, Eli Zaretskii wrote:
> In my testing, find-grep finishes almost instantaneously. The
> exception is when you have a cold cache, but even then it takes about
> 10% of the total run time, for the Emacs source tree (which yields
> about 100,000 hits in the test case).
This particular example, uses a very frequent term. I get 61000 hits or
so, and it's still a lot, the search never finishes here (probably
because I have more minor modes and customizations enabled).
I don't think this is the common case, but let's try to remove some
unnecessary work in Elisp first.
See commit c99a3b9. Please take a look at
xref--regexp-syntax-dependent-p specifically, and see if any significant
false negatives come to mind.
With this, project-find-regexp for 'emacs' finally completes in ~10
seconds on my machine. That's still more than 10 times longer than the
external process takes, but I'm out of big optimization ideas at this point.
> I thought the request was to allow the user do something in the
> foreground, while this processing runs in the background. If that's
> not what was requested, then I guess I no longer understand the
> request.
If the project is huge, and there are only a few hits, parallelizing the
search and processing will allow the user to do whatever they want in
the foreground. Because processing in Elisp, while slow, will still take
a small fraction of the time.
If the search term returns a lot of hits (compared to the size of the
project), processing might indeed take a lot of time, and the UI might
appear sluggish (not sure how sluggish, though, that should depend on
the scheduling of the main and background threads).
Even if it's sluggish, at least the user will see that the search has
started, and there is some progress. We could even allow them to stop
the search midway, and still do something with the first results.
These are some of the advantages 'M-x rgrep' has over project-find-regexp.
>> What we _can_ manage to run in parallel, in the find-grep process in the
>> background, and the post-processing of the results in Elisp.
>
> Yes, you can -- if you invoke find-grep asynchronously and move the
> processing of the hits to the filter function.
Yes, these parts are necessary either way. What I was describing would
go on top of them, as an abstraction.
> But that doesn't need
> to involve threads, and is being done in many packages/features out
> there, so I'm not sure what did you ask me to do with this.
I imagined that the xref API that allows this kind of asynchronous
results might look better and more readable if it's implemented with
threads underneath.
> IOW, it
> should be "trivial", at least in principle, to make this command work
> in the background, just like, say, "M-x grep".
In Compilation buffers (of which Grep is one example), the sentinel code
has access to the buffer where the results are displayed. And the
process outputs to that buffer as well. And 'M-x rgrep' doesn't have to
abstract over possible way to obtain search results.
None of those are the case with the xref API, or the results rendering
code, which has to work with the values returned by an arbitrary xref
backend, as documented.
Right now, an xref backend implements several methods that are allowed
to return the same type of value: "a list of xref items".
Our task, as I see it, is to generalize that return value type for
asynchronous work, and to do that as sanely as possible.
Threads are not strictly necessary for this (see the last paragraph of
my previous email), but this case seems like it could be a good, limited
in scope, showcase for the threading functionality.
> I'm not sure I understand the need for this complexity, given that
> async subprocesses are available. I'm probably missing something
> because I know too little about the internals of the involved code.
The main thing to understand is the xref API, not the internals of the
package.
This bug report was last modified 8 years and 128 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.