GNU bug report logs - #69233
30.0.50; project-files + project-find-file is slow in large repositories

Previous Next

Full log

Message #34 received at 69233 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: 69233 <at> debbugs.gnu.org, 69188 <at> debbugs.gnu.org
Subject: Re: bug#69233: 30.0.50; project-files + project-find-file is slow in
 large repositories
Date: Sun, 5 May 2024 03:29:12 +0300

On 29/04/2024 23:27, Spencer Baugh wrote:
> Dmitry Gutov<dmitry <at> gutov.dev>  writes:
>> On 13/04/2024 05:34, Dmitry Gutov wrote:
>>> Both options are relatively clunky, and the second one might also
>>> fail to work when DIRS is non-nil (or would have to fall back to
>>> absolute names anyway), so I'm leaning toward the first one. It
>>> might also allow certain code to be written supporting both relative
>>> and absolute names (e.g. a process call both binds default-directory
>>> to root and keeps the file names as-is -- the relative ones would be
>>> interpreted as such, the rest just as they are interpreted now).
>> Here's how that change can look.
>>
>> The patch should demonstrate both the performance improvements for
>> project-find-file and project-find-regexp, and some awkwardness in the
>> implementation, chiefly due to backward compatibility.
>>
>> Guess more tests will be required, at the very least.
> I see almost a 50% performance improvement with this patch in my large
> private repository, once adding support for project-files-relative-names
> in my internal project backend.  Seems great so far.
> 
> My benchmarking:
> 
> (let ((proj (project-current)))
>    (list (benchmark-run 10 (let ((project-files-relative-names t)) (length (project-files proj))))
>          (benchmark-run 10 (let ((project-files-relative-names nil)) (length (project-files proj))))))
> 
> ((17.605295389 28 7.647366087000023)
>   (29.918302167 57 19.246283027999993))

Nice!

Too bad it's still takes ~1.7s to list all the files in the project. 
Well above the comfortable wait time (ideally <100ms or at least <500ms, 
I guess).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.