Subject: Re: bug#50733: 28.0.1; project-find-regexp can block Emacs for a long time

From: Dmitry Gutov <[email-masked]>
Date: Fri, 24 Sep 2021 01:18:18 +0300

On 23.09.2021 23:42, Daniel Martín wrote:
> Eli Zaretskii <[email-masked]> writes:
> 
>>>
>>> I usually work on a monorepo with ~67000 tracked files (many of them big
>>> binary files).  Here's what I get when using ripgrep as the xref search
>>> program:
>>>
>>> Elapsed time: 36.087181s (8.067474s in 22 GCs)
>>>
>>> Running the same search with ripgrep from the command line takes around
>>> 6 seconds.
>>
>> How many matches does that report?
> 
> ~70400 matches spread across ~3700 files.  Perhaps it's an extreme case
> (I used a generic search term, "color").  I tried a more specific search
> term ("clang analyzer") which returned 30 matches across 14 files.
> Here's the benchmark:
> 
> Elapsed time: 10.176629s (0.946311s in 2 GCs)

30 matches means parsing and printing the matches won't be a bottleneck. 
Just fetching the file names, sending it to the search process, and the 
search itself.

> I guess the command could signal ongoing progress somehow, to assure the
>   user that Emacs is actually working and not frozen.

I suppose we could add some messages like in the patch below.

It's a simple approach where "done" is printed after the search is done 
and the results are parsed, but before they have been printed.

So there can still be a perceptible delay when your search gets 10s of 
1000s of hits or more.

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 68beedad3d..2b69fb7d57 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -851,9 +851,11 @@ project-or-external-find-regexp
  (defun project--find-regexp-in-files (regexp files)
    (unless files
      (user-error "Empty file list"))
+  (message "Searching...")
    (let ((xrefs (xref-matches-in-files regexp files)))
      (unless xrefs
        (user-error "No matches for: %s" regexp))
+    (message "Searching... done")
      xrefs))

  (defvar project-regexp-history-variable 'grep-regexp-history)