Subject: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores

From: Spencer Baugh <[email-masked]>
Date: Thu, 20 Jul 2023 13:08:24 -0400

Ihor Radchenko <[email-masked]> writes:
> Dmitry Gutov <[email-masked]> writes:
>
>>>> ... Last I checked, Lisp-native file
>>>> listing was simply slower than 'find'.
>>> 
>>> Could it be changed?
>>> In my tests, I was able to improve performance of the built-in
>>> `directory-files-recursively' simply by disabling
>>> `file-name-handler-alist' around its call.
>>
>> Then it won't work with Tramp, right? I think it's pretty nifty that 
>> project-find-regexp and dired-do-find-regexp work over Tramp.
>
> Sure. It might also be optimized. Without trying to convince find devs
> to do something about regexp handling.

Not to derail too much, but find as a subprocess has one substantial
advantage over find in Lisp: It can run in parallel with Emacs, so that
we actually use multiple CPU cores.

Between that, and the remote support part, I personally much prefer find
to be a subprocess rather than in Lisp.  I don't think optimizing
directory-files-recursively is a great solution.

(Really it's entirely plausible that Emacs could be improved by
*removing* directory-files-recursively, in favor of invoking find as a
subprocess: faster, parallelized execution, and better remote support.)