Subject: Re: bug#72715: 29.2.50; eglot didChangeWatchedFiles runs project-files in an arbitary directory, which can be slow

From: Ship Mints <[email-masked]>
Date: Mon, 19 Aug 2024 12:29:12 -0400

Not sure this is eglot specific, though I understand it manifests in eglot.
The internal project documentation acknowledges that project-files can be
slow with large numbers of files and recommends providing an alternate
implementation overriding the function.

;; `project-files' can be overridden for performance purposes.
;; ... `project-files' is the method to optimize (the
;; default implementation gets slower the more files the directory
;; has, and the longer the list of ignores is).

What behavior would you prefer? You could override, for example, to return
just the root for watching a single glob "**" if that's optimal for your
large directory structure.

On Mon, Aug 19, 2024 at 12:03 PM Spencer Baugh <[email-masked]>
wrote:

>
> Assuming:
> - There is /big-directory which has lots of children, such that
> "find /big-directory" takes a long time, e.g. minutes
> - .ext is a language whose language server uses
> workspace/didChangeWatchedFiles
>
> 1. emacs -Q
>
> 2. C-x C-f /big-directory/file.ext
>
> 3. M-x eglot
>
> 4. eglot runs (eglot--current-project); suppose project-current returns
> nil so eglot--current-project returns:
> `(transient . ,(expand-file-name default-directory))
>
> 5. The language server starts up and sends workspace/didChangeWatchedFiles
> to eglot
>
> 6. The eglot implementation of workspace/didChangeWatchedFiles runs
> project--files-in-directory
>
> 7. Emacs hangs for a long time as project--files-in-directory runs "find
> /big-directory"
>
> In particular I've had users experience this when trying to use eglot on
> files in large directory trees on NFS, which makes find even slower.
>
> I'm not sure how this could be fixed.
>
> If project-files could run asynchronously (e.g. with make-process),
> would Eglot be able to use that here, instead of synchronous
> project-files?  Supporting async project-files is on my todo list
> anyway.
>
>
> In GNU Emacs 29.2.50 (build 9, x86_64-pc-linux-gnu, X toolkit, cairo
>  version 1.15.12, Xaw scroll bars) of 2024-07-30 built on
>  igm-qws-u22796a
> Repository revision: cd9604db959c439c5695cf79f6533b5cbd340851
> Repository branch: emacs-29
> Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
> System Description: Rocky Linux 8.10 (Green Obsidian)
>
> Configured using:
>  'configure --with-x-toolkit=lucid --without-gpm --without-gconf
>  --without-selinux --without-imagemagick --with-modules --with-gif=no
>  --with-cairo --with-rsvg --without-compress-install
>  --with-native-compilation=aot --with-tree-sitter
>
>  PKG_CONFIG_PATH=/usr/local/home/garnish/libtree-sitter/0.22.6-1/lib/pkgconfig/'
>
> Configured features:
> CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
> LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
> SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11
> XDBE XIM XINPUT2 XPM LUCID ZLIB
>
> Important settings:
>   value of $LANG: en_US.UTF-8
>   locale-coding-system: utf-8-unix
>
> Major mode: ELisp/l
>
> Features:
> (shadow sort mail-extr emacsbug package-x rect eglot external-completion
> jsonrpc cus-start find-dired ffap dabbrev org-element org-persist org-id
> org-refile avl-tree generator oc-basic ol-eww eww xdg url-queue mm-url
> ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect gnus-art mm-uu mml2015
> mm-view mml-smime smime gnutls dig gnus-sum shr pixel-fill kinsoku
> url-file svg dom gnus-group gnus-undo gnus-start gnus-dbus dbus xml
> gnus-cloud nnimap nnmail mail-source utf7 nnoo gnus-spec gnus-int
> gnus-range gnus-win ol-docview doc-view jka-compr image-mode exif
> ol-bibtex bibtex ol-bbdb ol-w3m ol-doi org-link-doi shortdoc hl-line
> display-line-numbers pcmpl-unix misc vc-bzr vc-src vc-sccs vc-svn vc-cvs
> vc-rcs sh-script treesit mule-util executable goto-addr log-view vc-fe
> vc-hg vc-dir vc magit-imenu git-rebase face-remap vc-git vc-dispatcher
> bug-reference cl-print help-fns radix-tree misearch multi-isearch
> dired-aux pulse color grep let-alist delsel so-long pixel-scroll
> cua-base jane-fe-read-feature tramp tramp-loaddefs trampver
> tramp-integration files-x tramp-compat parse-time iso8601 skeleton
> page-ext magit-extras magit-bookmark magit-submodule magit-obsolete
> magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull
> magit-fetch magit-clone magit-remote magit-commit magit-sequence
> magit-notes magit-worktree magit-tag magit-merge magit-branch
> magit-reset magit-files magit-refs magit-status magit magit-repos
> magit-apply magit-wip magit-log which-func magit-diff smerge-mode diff
> diff-mode git-commit log-edit message sendmail yank-media puny rfc822
> mml mml-sec epa epg rfc6068 epg-config mm-decode mm-bodies mm-encode
> mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils
> mailheader pcvs-util add-log magit-core magit-autorevert autorevert
> magit-margin magit-transient magit-process magit-mode transient
> magit-git magit-section magit-utils dash gnus nnheader gnus-util
> mail-utils range mm-util mail-prsvr gdb-mi bindat gud vundo
> modus-vivendi-theme modus-themes savehist saveplace site-start
> jane-customization jane-diagnose jane-merlin merlin-imenu merlin-xref
> xref merlin-cap merlin crm jane-async-merlin jane-completion jane-comint
> with-editor org-protocol org org-macro org-pcomplete org-list
> org-footnote org-faces org-entities time-date noutline outline ob-dot
> ob-shell ob ob-tangle org-src ob-ref ob-lob ob-table ob-exp ob-comint
> ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold
> org-fold-core org-keys oc org-loaddefs cal-menu calendar cal-loaddefs
> org-compat org-version org-macs format-spec jane-fe-project
> jane-fe-feature-table imenu jane-fe-menu ecaml_plugin edmacro kmacro
> bookmark dired-x image+ advice image-file image-converter editorconfig
> editorconfig-core editorconfig-core-handle editorconfig-fnmatch
> whitespace help-at-pt jane jane-yasnippet jane-micro-features server
> jane-diff unified-test-mode shell-file dired dired-loaddefs easy-mmode
> jane-sexp jane-python jane-ocaml jane-tuareg-theme jane-aide ert ewoc
> debug backtrace find-func tuareg speedbar ezimage dframe tuareg-compat
> tuareg-opam flymake-proc flymake project thingatpt smie caml-types
> caml-help view find-file compile text-property-search jane-cr jane-align
> shell pcomplete comint ansi-osc ansi-color ring gnu-elpa-keyring-update
> jane-telemetry comp comp-cstr warnings cl-extra help-mode ocp-indent
> jane-eglot yasnippet-autoloads vba-mode-autoloads term-cursor-autoloads
> swiper-autoloads org-project-capture-autoloads htmlize-autoloads
> haskell-mode-autoloads eglot-autoloads editorconfig-conf-mode derived
> conf-mode editorconfig-autoloads contrib-autoloads jane-autoloads
> jane-util pcase filenotify cus-edit pp cus-load icons wid-edit
> adaptive-wrap-autoloads corfu-autoloads csv-mode-autoloads
> cyberpunk-theme-autoloads evil-autoloads exwm-autoloads helm-autoloads
> helm-core-autoloads async-autoloads ivy-autoloads magit-autoloads
> git-commit-autoloads finder-inf magit-section-autoloads popup-autoloads
> url-http-ntlm-autoloads url-auth utop-autoloads tuareg-autoloads rx
> caml-autoloads vc-hgcmd-autoloads vertico-autoloads compat-autoloads
> vundo-autoloads with-editor-autoloads xelb-autoloads
> markdown-mode-autoloads multi-line-autoloads info dash-autoloads
> s-autoloads shut-up-autoloads sml-mode-autoloads package browse-url url
> url-proxy url-privacy url-expand url-methods url-history url-cookie
> generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
> auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
> map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
> iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
> vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
> term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
> tabulated-list replace newcomment text-mode lisp-mode prog-mode register
> page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
> scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
> frame minibuffer nadvice seq simple cl-generic indonesian philippine
> cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
> korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
> european ethiopic indian cyrillic chinese composite emoji-zwj charscript
> charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
> cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
> files window text-properties overlay sha1 md5 base64 format env
> code-pages mule custom widget keymap hashtable-print-readable backquote
> threads dbusbind inotify dynamic-setting system-font-setting
> font-render-setting cairo x-toolkit xinput2 x multi-tty
> make-network-process native-compile emacs)
>
> Memory information:
> ((conses 16 1149062 125299)
>  (symbols 48 58114 7)
>  (strings 32 221941 24168)
>  (string-bytes 1 8404217)
>  (vectors 16 95307)
>  (vector-slots 8 2497373 165939)
>  (floats 8 730 417)
>  (intervals 56 59184 781)
>  (buffers 984 197))
>
>
>
>