GNU bug report logs - #72715
29.2.50; eglot didChangeWatchedFiles runs project-files in an arbitary directory, which can be slow

Previous Next

Full log

Message #11 received at 72715 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, 72715 <at> debbugs.gnu.org
Subject: Re: bug#72715: 29.2.50; eglot didChangeWatchedFiles runs
 project-files in an arbitary directory, which can be slow
Date: Fri, 24 Jan 2025 10:27:44 +0000

Spencer Baugh <sbaugh <at> janestreet.com> writes:

> 6. The eglot implementation of workspace/didChangeWatchedFiles runs
> project--files-in-directory

Eglot's implementation does not call this, it calls project-files.  The
implementation details are up to project.el

> 7. Emacs hangs for a long time as project--files-in-directory runs "find /big-directory"
>
> In particular I've had users experience this when trying to use eglot on
> files in large directory trees on NFS, which makes find even slower.

The topic of `didChangeWatchedFiles` is somehow contested.  The LSP
authors state that it's more reliable to do in the client (which makes
sense when since there are fewer clients... in fact they poorly disguise
their inclination to a $ingle one...).  But as you have discovered this
solution is more complex, less efficient, and problematic in some ways.
Many servers do this watching themselves.

> I'm not sure how this could be fixed.
>
> If project-files could run asynchronously (e.g. with make-process),
> would Eglot be able to use that here, instead of synchronous
> project-files?

It _is_ run asynchronously to the main events.  It is run as part of the
process filter of the underlying LSP process).  The problem is that
Emacs is single-threaded and you the user have to pay the time (in
minutes) anyway.  Because process-files seems to be a relatively "pure"
operation, with make-process you could escape that indeed (creating a
whole separate process just for another thread which your OS will be
able to parallelize or at least cleverly time-slice parallize).

Speaking of time-slice parallelize, I wonder if Emacs's threads could be
leveraged to do this.

Some of this could probably be prototyped, but that solution shouldn't
live in Eglot.

Here are other solutions:

* Why is project-files slow?  Why isn't it fast?  Minutes to get a file
  list in 2025?  Many operating systems feature file name database that
  are very fast at answering these queries (linux locatedb, fd,
  everything in Windows). Even Git has one of these databases, so if the
  project is Git-backed, it could in theory use that, at least for
  half-decent results.  But I have no idea how project.el is implemented
  these days.

* Why is there no project-directories, which is what Eglot needs in this
  case.  It seems in theory that could be made faster.

> Supporting async project-files is on my todo list anyway.

Seems reasonable and an interesting goal, but look at the other above
two ideas.

João

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.