GNU bug report logs - #71094
[PATCH] Prefer to run find and grep in parallel in rgrep

Previous Next

Full log

Message #20 received at 71094 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: sbaugh <at> janestreet.com, 71094 <at> debbugs.gnu.org, rgm <at> gnu.org
Subject: Re: bug#71094: [PATCH] Prefer to run find and grep in parallel in
 rgrep
Date: Wed, 22 May 2024 16:50:57 +0300

> Date: Wed, 22 May 2024 15:34:06 +0300
> Cc: 71094 <at> debbugs.gnu.org, rgm <at> gnu.org
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> On 22/05/2024 14:59, Eli Zaretskii wrote:
> 
> > With how many files did you measure the 40% speedup?  Can you show the
> > performance with much fewer and much more files than what you used?
> 
> FWIW my test indicated that for a smaller project (such as Emacs) the 
> difference is fairly small - the new code is slightly better or the same.
> 
> The directory where I saw significant improvement has 300K files.

That's what I thought.  So we are changing the decade-old defaults to
favor huge directories, which is not necessarily the wisest thing to
do.

> > I
> > suspect that the effect depends on that.  (It also depends on the
> > system limit on the number of files and the length of the command line
> > that xargs can use.)  The argument about 'find' waiting is no longer
> > relevant with 'exec-plus', since in most cases there will be just one
> > invocation of 'grep'.
> 
> If there's just one invocation, wouldn't that mean that it will happen 
> at the end of the full directory scan? Rather than in parallel.

That's true, but what is your mental model of how the pipe with xargs
works in practice?  How many invocations of grep will xargs do, and
when will the first invocation happen?

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.