Subject: Re: bug#69233: 30.0.50; project-files + project-find-file is slow in large repositories

From: Dmitry Gutov <[email-masked]>
Date: Sun, 5 May 2024 03:29:12 +0300

On 29/04/2024 23:27, Spencer Baugh wrote:
> Dmitry Gutov<[email-masked]>  writes:
>> On 13/04/2024 05:34, Dmitry Gutov wrote:
>>> Both options are relatively clunky, and the second one might also
>>> fail to work when DIRS is non-nil (or would have to fall back to
>>> absolute names anyway), so I'm leaning toward the first one. It
>>> might also allow certain code to be written supporting both relative
>>> and absolute names (e.g. a process call both binds default-directory
>>> to root and keeps the file names as-is -- the relative ones would be
>>> interpreted as such, the rest just as they are interpreted now).
>> Here's how that change can look.
>>
>> The patch should demonstrate both the performance improvements for
>> project-find-file and project-find-regexp, and some awkwardness in the
>> implementation, chiefly due to backward compatibility.
>>
>> Guess more tests will be required, at the very least.
> I see almost a 50% performance improvement with this patch in my large
> private repository, once adding support for project-files-relative-names
> in my internal project backend.  Seems great so far.
> 
> My benchmarking:
> 
> (let ((proj (project-current)))
>    (list (benchmark-run 10 (let ((project-files-relative-names t)) (length (project-files proj))))
>          (benchmark-run 10 (let ((project-files-relative-names nil)) (length (project-files proj))))))
> 
> ((17.605295389 28 7.647366087000023)
>   (29.918302167 57 19.246283027999993))

Nice!

Too bad it's still takes ~1.7s to list all the files in the project. 
Well above the comfortable wait time (ideally <100ms or at least <500ms, 
I guess).