From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 19 12:02:21 2024 Received: (at submit) by debbugs.gnu.org; 19 Aug 2024 16:02:21 +0000 Received: from localhost ([127.0.0.1]:58985 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sg4po-0000MR-HQ for submit@debbugs.gnu.org; Mon, 19 Aug 2024 12:02:21 -0400 Received: from lists.gnu.org ([209.51.188.17]:42276) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sg4pl-0000MF-QX for submit@debbugs.gnu.org; Mon, 19 Aug 2024 12:02:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sg4p5-0000RT-Lt for bug-gnu-emacs@gnu.org; Mon, 19 Aug 2024 12:01:35 -0400 Received: from mxout6.mail.janestreet.com ([64.215.233.21]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sg4p2-0006JL-W0 for bug-gnu-emacs@gnu.org; Mon, 19 Aug 2024 12:01:35 -0400 From: Spencer Baugh To: bug-gnu-emacs@gnu.org Subject: 29.2.50; eglot didChangeWatchedFiles runs project-files in an arbitary directory, which can be slow Date: Mon, 19 Aug 2024 12:01:30 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1724083291; bh=KcgKDMln7ZrIFwx75T/r4zSBOURTs/ExhlbL+eitEuo=; h=From:To:Cc:Subject:Date; b=sj8PkjQlht1TvqSI1ANP/Ld35u+rBRJNFyn30KxvtaeEOmL/7eF09hMsH0aoGSzoa MFPxKvHVT4HxYgpOajr3WKQ16Fu4JXS/8+20dLsrp30H4d+m44VUCypp5Q+NvmIxb/ OUSviLj3NU3n+KN4HKIEfIPRnxZAxQw1q56SzTWbhGpc1yZk8KzxofBKHEoBqgffOS /Yv8Cd5R2URzccLloA3WwuNm6zWL9tvGGfrdiMuqS+sSdNsKgyHDmnsRyoQ15oc2k5 AgY3yCfkm5blQymgHa+XmRVR/I6B5ALkboWuIY4sx/Fx4WpdMq2xS8meeYKe6Fonbp cI5eULzmz1YUw== Received-SPF: pass client-ip=64.215.233.21; envelope-from=sbaugh@janestreet.com; helo=mxout6.mail.janestreet.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: Dmitry Gutov , =?utf-8?B?Sm/Do28gVMOhdm9yYQ==?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Assuming: - There is /big-directory which has lots of children, such that "find /big-directory" takes a long time, e.g. minutes - .ext is a language whose language server uses workspace/didChangeWatchedFiles 1. emacs -Q 2. C-x C-f /big-directory/file.ext 3. M-x eglot 4. eglot runs (eglot--current-project); suppose project-current returns nil so eglot--current-project returns: `(transient . ,(expand-file-name default-directory)) 5. The language server starts up and sends workspace/didChangeWatchedFiles to eglot 6. The eglot implementation of workspace/didChangeWatchedFiles runs project--files-in-directory 7. Emacs hangs for a long time as project--files-in-directory runs "find /big-directory" In particular I've had users experience this when trying to use eglot on files in large directory trees on NFS, which makes find even slower. I'm not sure how this could be fixed. If project-files could run asynchronously (e.g. with make-process), would Eglot be able to use that here, instead of synchronous project-files? Supporting async project-files is on my todo list anyway. In GNU Emacs 29.2.50 (build 9, x86_64-pc-linux-gnu, X toolkit, cairo version 1.15.12, Xaw scroll bars) of 2024-07-30 built on igm-qws-u22796a Repository revision: cd9604db959c439c5695cf79f6533b5cbd340851 Repository branch: emacs-29 Windowing system distributor 'The X.Org Foundation', version 11.0.12011000 System Description: Rocky Linux 8.10 (Green Obsidian) Configured using: 'configure --with-x-toolkit=lucid --without-gpm --without-gconf --without-selinux --without-imagemagick --with-modules --with-gif=no --with-cairo --with-rsvg --without-compress-install --with-native-compilation=aot --with-tree-sitter PKG_CONFIG_PATH=/usr/local/home/garnish/libtree-sitter/0.22.6-1/lib/pkgconfig/' Configured features: CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM XINPUT2 XPM LUCID ZLIB Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: ELisp/l Features: (shadow sort mail-extr emacsbug package-x rect eglot external-completion jsonrpc cus-start find-dired ffap dabbrev org-element org-persist org-id org-refile avl-tree generator oc-basic ol-eww eww xdg url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect gnus-art mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum shr pixel-fill kinsoku url-file svg dom gnus-group gnus-undo gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7 nnoo gnus-spec gnus-int gnus-range gnus-win ol-docview doc-view jka-compr image-mode exif ol-bibtex bibtex ol-bbdb ol-w3m ol-doi org-link-doi shortdoc hl-line display-line-numbers pcmpl-unix misc vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs sh-script treesit mule-util executable goto-addr log-view vc-fe vc-hg vc-dir vc magit-imenu git-rebase face-remap vc-git vc-dispatcher bug-reference cl-print help-fns radix-tree misearch multi-isearch dired-aux pulse color grep let-alist delsel so-long pixel-scroll cua-base jane-fe-read-feature tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat parse-time iso8601 skeleton page-ext magit-extras magit-bookmark magit-submodule magit-obsolete magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit magit-sequence magit-notes magit-worktree magit-tag magit-merge magit-branch magit-reset magit-files magit-refs magit-status magit magit-repos magit-apply magit-wip magit-log which-func magit-diff smerge-mode diff diff-mode git-commit log-edit message sendmail yank-media puny rfc822 mml mml-sec epa epg rfc6068 epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils mailheader pcvs-util add-log magit-core magit-autorevert autorevert magit-margin magit-transient magit-process magit-mode transient magit-git magit-section magit-utils dash gnus nnheader gnus-util mail-utils range mm-util mail-prsvr gdb-mi bindat gud vundo modus-vivendi-theme modus-themes savehist saveplace site-start jane-customization jane-diagnose jane-merlin merlin-imenu merlin-xref xref merlin-cap merlin crm jane-async-merlin jane-completion jane-comint with-editor org-protocol org org-macro org-pcomplete org-list org-footnote org-faces org-entities time-date noutline outline ob-dot ob-shell ob ob-tangle org-src ob-ref ob-lob ob-table ob-exp ob-comint ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold org-fold-core org-keys oc org-loaddefs cal-menu calendar cal-loaddefs org-compat org-version org-macs format-spec jane-fe-project jane-fe-feature-table imenu jane-fe-menu ecaml_plugin edmacro kmacro bookmark dired-x image+ advice image-file image-converter editorconfig editorconfig-core editorconfig-core-handle editorconfig-fnmatch whitespace help-at-pt jane jane-yasnippet jane-micro-features server jane-diff unified-test-mode shell-file dired dired-loaddefs easy-mmode jane-sexp jane-python jane-ocaml jane-tuareg-theme jane-aide ert ewoc debug backtrace find-func tuareg speedbar ezimage dframe tuareg-compat tuareg-opam flymake-proc flymake project thingatpt smie caml-types caml-help view find-file compile text-property-search jane-cr jane-align shell pcomplete comint ansi-osc ansi-color ring gnu-elpa-keyring-update jane-telemetry comp comp-cstr warnings cl-extra help-mode ocp-indent jane-eglot yasnippet-autoloads vba-mode-autoloads term-cursor-autoloads swiper-autoloads org-project-capture-autoloads htmlize-autoloads haskell-mode-autoloads eglot-autoloads editorconfig-conf-mode derived conf-mode editorconfig-autoloads contrib-autoloads jane-autoloads jane-util pcase filenotify cus-edit pp cus-load icons wid-edit adaptive-wrap-autoloads corfu-autoloads csv-mode-autoloads cyberpunk-theme-autoloads evil-autoloads exwm-autoloads helm-autoloads helm-core-autoloads async-autoloads ivy-autoloads magit-autoloads git-commit-autoloads finder-inf magit-section-autoloads popup-autoloads url-http-ntlm-autoloads url-auth utop-autoloads tuareg-autoloads rx caml-autoloads vc-hgcmd-autoloads vertico-autoloads compat-autoloads vundo-autoloads with-editor-autoloads xelb-autoloads markdown-mode-autoloads multi-line-autoloads info dash-autoloads s-autoloads shut-up-autoloads sml-mode-autoloads package browse-url url url-proxy url-privacy url-expand url-methods url-history url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic indonesian philippine cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget keymap hashtable-print-readable backquote threads dbusbind inotify dynamic-setting system-font-setting font-render-setting cairo x-toolkit xinput2 x multi-tty make-network-process native-compile emacs) Memory information: ((conses 16 1149062 125299) (symbols 48 58114 7) (strings 32 221941 24168) (string-bytes 1 8404217) (vectors 16 95307) (vector-slots 8 2497373 165939) (floats 8 730 417) (intervals 56 59184 781) (buffers 984 197)) From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 19 12:31:16 2024 Received: (at 72715) by debbugs.gnu.org; 19 Aug 2024 16:31:16 +0000 Received: from localhost ([127.0.0.1]:58996 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sg5Hn-0001Jy-3d for submit@debbugs.gnu.org; Mon, 19 Aug 2024 12:31:16 -0400 Received: from mail-vs1-f52.google.com ([209.85.217.52]:57412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sg5Hj-0001Jg-HR for 72715@debbugs.gnu.org; Mon, 19 Aug 2024 12:31:12 -0400 Received: by mail-vs1-f52.google.com with SMTP id ada2fe7eead31-49291b02e20so1714517137.2 for <72715@debbugs.gnu.org>; Mon, 19 Aug 2024 09:30:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724084964; x=1724689764; darn=debbugs.gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=TPsPCz8FC+DJUIIGDm2ZtdfZN8A1LVSzBM7CIW3yHSo=; b=hZqdnz2UEpt7h7aIPiz8IZw9z9LXG1sK5kK7F1MeXB1Eal9cN0C0SLQw5xZ/MVpIu1 eyXDNXYtQNiyJYHGh2C5Qu/HGMiKPK1jGx+zedXrwAtb0SqbrrPFLRG//9w8DgNzuI00 uHyCtpF8fpZyM6pLNhUijZW+HOh1KZ4sWnpS8tWOV9K954odkus9b6hrePT6LbEcswyr xA5WF1g/tzOKRUAxSOTb8gFPZWJBTp3oL6SLSgrcdVJMOrJQiuzPi0Tn+4RnXSXMtzpQ jryiwvXskDor2c27AQiIDf8r8IGLLCHWsKsdT4FUvvARPsBN+DhFFZfytLswx4fn8a0/ H/5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724084964; x=1724689764; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TPsPCz8FC+DJUIIGDm2ZtdfZN8A1LVSzBM7CIW3yHSo=; b=KNc1GZU1rJik+D6wXvZGzjLIx0QahOkEPED3C49wOffHKf7OzPtSK8SGEGLDAx+SBu Z1URmkREALf9Ck55GMSzE9zEz5fi86PjeitPUn/VMFGR36UKu/KfnO/aEDkWUzc67o5g 0FOMfnnVCAQfGpKIIwzgJLjQCBDBsnAA4i9Ay1j6bbMWPlHdeWT+1UDWzujWuhgNPBKu DNuth0ggKpoezLuxaicmyOB8Pabb3EGf6lJp4yN/yyInkRCEfLFkfVlVjqKZyYbXdDoD smU32STvg2G/eUia4ZtfRVGMmev5zfIrHg5TzhFDg2HDr/RDiFcirNMq28qQy5riR3Sm jv9Q== X-Gm-Message-State: AOJu0Yw64FAmShJSHfJdD/bsQ469O3bkXq0xKajN5Bx0uGx+DDcXJXR0 wuNmg3FQs/absiWK3uwpCKFJfixoJ7+WzNE6hZYbWsECyGuuwFA8pPqH1sKfA2p1cQaHSTg9391 iPFA6cq7qyBxh+9uSVtr4Bp2cNLY= X-Google-Smtp-Source: AGHT+IE96tFoD0zABUlIZaEEGVsgqYV4chz0teTcotQ0odh83gfRtONxwlcXxR00K4vwvb45lK6zzhcjg42LuU+No4o= X-Received: by 2002:a05:6102:b08:b0:492:a74d:1664 with SMTP id ada2fe7eead31-49779941fedmr13848538137.8.1724084963776; Mon, 19 Aug 2024 09:29:23 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ship Mints Date: Mon, 19 Aug 2024 12:29:12 -0400 Message-ID: Subject: Re: bug#72715: 29.2.50; eglot didChangeWatchedFiles runs project-files in an arbitary directory, which can be slow To: Spencer Baugh Content-Type: multipart/alternative; boundary="000000000000d420ca06200bcf86" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 72715 Cc: Dmitry Gutov , =?UTF-8?B?Sm/Do28gVMOhdm9yYQ==?= , 72715@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --000000000000d420ca06200bcf86 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Not sure this is eglot specific, though I understand it manifests in eglot. The internal project documentation acknowledges that project-files can be slow with large numbers of files and recommends providing an alternate implementation overriding the function. ;; `project-files' can be overridden for performance purposes. ;; ... `project-files' is the method to optimize (the ;; default implementation gets slower the more files the directory ;; has, and the longer the list of ignores is). What behavior would you prefer? You could override, for example, to return just the root for watching a single glob "**" if that's optimal for your large directory structure. On Mon, Aug 19, 2024 at 12:03=E2=80=AFPM Spencer Baugh wrote: > > Assuming: > - There is /big-directory which has lots of children, such that > "find /big-directory" takes a long time, e.g. minutes > - .ext is a language whose language server uses > workspace/didChangeWatchedFiles > > 1. emacs -Q > > 2. C-x C-f /big-directory/file.ext > > 3. M-x eglot > > 4. eglot runs (eglot--current-project); suppose project-current returns > nil so eglot--current-project returns: > `(transient . ,(expand-file-name default-directory)) > > 5. The language server starts up and sends workspace/didChangeWatchedFile= s > to eglot > > 6. The eglot implementation of workspace/didChangeWatchedFiles runs > project--files-in-directory > > 7. Emacs hangs for a long time as project--files-in-directory runs "find > /big-directory" > > In particular I've had users experience this when trying to use eglot on > files in large directory trees on NFS, which makes find even slower. > > I'm not sure how this could be fixed. > > If project-files could run asynchronously (e.g. with make-process), > would Eglot be able to use that here, instead of synchronous > project-files? Supporting async project-files is on my todo list > anyway. > > > In GNU Emacs 29.2.50 (build 9, x86_64-pc-linux-gnu, X toolkit, cairo > version 1.15.12, Xaw scroll bars) of 2024-07-30 built on > igm-qws-u22796a > Repository revision: cd9604db959c439c5695cf79f6533b5cbd340851 > Repository branch: emacs-29 > Windowing system distributor 'The X.Org Foundation', version 11.0.1201100= 0 > System Description: Rocky Linux 8.10 (Green Obsidian) > > Configured using: > 'configure --with-x-toolkit=3Dlucid --without-gpm --without-gconf > --without-selinux --without-imagemagick --with-modules --with-gif=3Dno > --with-cairo --with-rsvg --without-compress-install > --with-native-compilation=3Daot --with-tree-sitter > > PKG_CONFIG_PATH=3D/usr/local/home/garnish/libtree-sitter/0.22.6-1/lib/pk= gconfig/' > > Configured features: > CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON > LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG > SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 > XDBE XIM XINPUT2 XPM LUCID ZLIB > > Important settings: > value of $LANG: en_US.UTF-8 > locale-coding-system: utf-8-unix > > Major mode: ELisp/l > > Features: > (shadow sort mail-extr emacsbug package-x rect eglot external-completion > jsonrpc cus-start find-dired ffap dabbrev org-element org-persist org-id > org-refile avl-tree generator oc-basic ol-eww eww xdg url-queue mm-url > ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect gnus-art mm-uu mml2015 > mm-view mml-smime smime gnutls dig gnus-sum shr pixel-fill kinsoku > url-file svg dom gnus-group gnus-undo gnus-start gnus-dbus dbus xml > gnus-cloud nnimap nnmail mail-source utf7 nnoo gnus-spec gnus-int > gnus-range gnus-win ol-docview doc-view jka-compr image-mode exif > ol-bibtex bibtex ol-bbdb ol-w3m ol-doi org-link-doi shortdoc hl-line > display-line-numbers pcmpl-unix misc vc-bzr vc-src vc-sccs vc-svn vc-cvs > vc-rcs sh-script treesit mule-util executable goto-addr log-view vc-fe > vc-hg vc-dir vc magit-imenu git-rebase face-remap vc-git vc-dispatcher > bug-reference cl-print help-fns radix-tree misearch multi-isearch > dired-aux pulse color grep let-alist delsel so-long pixel-scroll > cua-base jane-fe-read-feature tramp tramp-loaddefs trampver > tramp-integration files-x tramp-compat parse-time iso8601 skeleton > page-ext magit-extras magit-bookmark magit-submodule magit-obsolete > magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull > magit-fetch magit-clone magit-remote magit-commit magit-sequence > magit-notes magit-worktree magit-tag magit-merge magit-branch > magit-reset magit-files magit-refs magit-status magit magit-repos > magit-apply magit-wip magit-log which-func magit-diff smerge-mode diff > diff-mode git-commit log-edit message sendmail yank-media puny rfc822 > mml mml-sec epa epg rfc6068 epg-config mm-decode mm-bodies mm-encode > mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils > mailheader pcvs-util add-log magit-core magit-autorevert autorevert > magit-margin magit-transient magit-process magit-mode transient > magit-git magit-section magit-utils dash gnus nnheader gnus-util > mail-utils range mm-util mail-prsvr gdb-mi bindat gud vundo > modus-vivendi-theme modus-themes savehist saveplace site-start > jane-customization jane-diagnose jane-merlin merlin-imenu merlin-xref > xref merlin-cap merlin crm jane-async-merlin jane-completion jane-comint > with-editor org-protocol org org-macro org-pcomplete org-list > org-footnote org-faces org-entities time-date noutline outline ob-dot > ob-shell ob ob-tangle org-src ob-ref ob-lob ob-table ob-exp ob-comint > ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold > org-fold-core org-keys oc org-loaddefs cal-menu calendar cal-loaddefs > org-compat org-version org-macs format-spec jane-fe-project > jane-fe-feature-table imenu jane-fe-menu ecaml_plugin edmacro kmacro > bookmark dired-x image+ advice image-file image-converter editorconfig > editorconfig-core editorconfig-core-handle editorconfig-fnmatch > whitespace help-at-pt jane jane-yasnippet jane-micro-features server > jane-diff unified-test-mode shell-file dired dired-loaddefs easy-mmode > jane-sexp jane-python jane-ocaml jane-tuareg-theme jane-aide ert ewoc > debug backtrace find-func tuareg speedbar ezimage dframe tuareg-compat > tuareg-opam flymake-proc flymake project thingatpt smie caml-types > caml-help view find-file compile text-property-search jane-cr jane-align > shell pcomplete comint ansi-osc ansi-color ring gnu-elpa-keyring-update > jane-telemetry comp comp-cstr warnings cl-extra help-mode ocp-indent > jane-eglot yasnippet-autoloads vba-mode-autoloads term-cursor-autoloads > swiper-autoloads org-project-capture-autoloads htmlize-autoloads > haskell-mode-autoloads eglot-autoloads editorconfig-conf-mode derived > conf-mode editorconfig-autoloads contrib-autoloads jane-autoloads > jane-util pcase filenotify cus-edit pp cus-load icons wid-edit > adaptive-wrap-autoloads corfu-autoloads csv-mode-autoloads > cyberpunk-theme-autoloads evil-autoloads exwm-autoloads helm-autoloads > helm-core-autoloads async-autoloads ivy-autoloads magit-autoloads > git-commit-autoloads finder-inf magit-section-autoloads popup-autoloads > url-http-ntlm-autoloads url-auth utop-autoloads tuareg-autoloads rx > caml-autoloads vc-hgcmd-autoloads vertico-autoloads compat-autoloads > vundo-autoloads with-editor-autoloads xelb-autoloads > markdown-mode-autoloads multi-line-autoloads info dash-autoloads > s-autoloads shut-up-autoloads sml-mode-autoloads package browse-url url > url-proxy url-privacy url-expand url-methods url-history url-cookie > generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse > auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x > map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc > iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook > vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win > term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe > tabulated-list replace newcomment text-mode lisp-mode prog-mode register > page tab-bar menu-bar rfn-eshadow isearch easymenu timer select > scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors > frame minibuffer nadvice seq simple cl-generic indonesian philippine > cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao > korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech > european ethiopic indian cyrillic chinese composite emoji-zwj charscript > charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure > cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp > files window text-properties overlay sha1 md5 base64 format env > code-pages mule custom widget keymap hashtable-print-readable backquote > threads dbusbind inotify dynamic-setting system-font-setting > font-render-setting cairo x-toolkit xinput2 x multi-tty > make-network-process native-compile emacs) > > Memory information: > ((conses 16 1149062 125299) > (symbols 48 58114 7) > (strings 32 221941 24168) > (string-bytes 1 8404217) > (vectors 16 95307) > (vector-slots 8 2497373 165939) > (floats 8 730 417) > (intervals 56 59184 781) > (buffers 984 197)) > > > > --000000000000d420ca06200bcf86 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Not sure this is eglot specific, though I understand it manifests in egl= ot. The internal project documentation acknowledges that project-files can = be slow with large numbers of files and recommends providing an alternate i= mplementation overriding the function.

;; `project-files' can be overridden for per= formance purposes.
;; ... `project-files' is the method to optimize (the
;; d= efault implementation gets slower the more files the directory
;; has, a= nd the longer the list of ignores is).

What behavior would you prefer? You could ove= rride, for example, to return just the root for watching a single glob &quo= t;**" if that's optimal=C2=A0for your large directory structure.

On Mon, Aug 19, 2024 at 12:03=E2=80=AFPM Spencer Baugh <sbaugh@janestreet.com> wrote:
=

Assuming:
- There is /big-directory which has lots of children, such that
"find /big-directory" takes a long time, e.g. minutes
- .ext is a language whose language server uses workspace/didChangeWatchedF= iles

1. emacs -Q

2. C-x C-f /big-directory/file.ext

3. M-x eglot

4. eglot runs (eglot--current-project); suppose project-current returns
nil so eglot--current-project returns:
`(transient . ,(expand-file-name default-directory))

5. The language server starts up and sends workspace/didChangeWatchedFiles = to eglot

6. The eglot implementation of workspace/didChangeWatchedFiles runs project= --files-in-directory

7. Emacs hangs for a long time as project--files-in-directory runs "fi= nd /big-directory"

In particular I've had users experience this when trying to use eglot o= n
files in large directory trees on NFS, which makes find even slower.

I'm not sure how this could be fixed.

If project-files could run asynchronously (e.g. with make-process),
would Eglot be able to use that here, instead of synchronous
project-files?=C2=A0 Supporting async project-files is on my todo list
anyway.


In GNU Emacs 29.2.50 (build 9, x86_64-pc-linux-gnu, X toolkit, cairo
=C2=A0version 1.15.12, Xaw scroll bars) of 2024-07-30 built on
=C2=A0igm-qws-u22796a
Repository revision: cd9604db959c439c5695cf79f6533b5cbd340851
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.1= 2011000
System Description: Rocky Linux 8.10 (Green Obsidian)

Configured using:
=C2=A0'configure --with-x-toolkit=3Dlucid --without-gpm --without-gconf=
=C2=A0--without-selinux --without-imagemagick --with-modules --with-gif=3Dn= o
=C2=A0--with-cairo --with-rsvg --without-compress-install
=C2=A0--with-native-compilation=3Daot --with-tree-sitter
=C2=A0PKG_CONFIG_PATH=3D/usr/local/home/garnish/libtree-sitter/0.22.6-1/lib= /pkgconfig/'

Configured features:
CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11
XDBE XIM XINPUT2 XPM LUCID ZLIB

Important settings:
=C2=A0 value of $LANG: en_US.UTF-8
=C2=A0 locale-coding-system: utf-8-unix

Major mode: ELisp/l

Features:
(shadow sort mail-extr emacsbug package-x rect eglot external-completion jsonrpc cus-start find-dired ffap dabbrev org-element org-persist org-id org-refile avl-tree generator oc-basic ol-eww eww xdg url-queue mm-url
ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect gnus-art mm-uu mml2015
mm-view mml-smime smime gnutls dig gnus-sum shr pixel-fill kinsoku
url-file svg dom gnus-group gnus-undo gnus-start gnus-dbus dbus xml
gnus-cloud nnimap nnmail mail-source utf7 nnoo gnus-spec gnus-int
gnus-range gnus-win ol-docview doc-view jka-compr image-mode exif
ol-bibtex bibtex ol-bbdb ol-w3m ol-doi org-link-doi shortdoc hl-line
display-line-numbers pcmpl-unix misc vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs sh-script treesit mule-util executable goto-addr log-view vc-fe
vc-hg vc-dir vc magit-imenu git-rebase face-remap vc-git vc-dispatcher
bug-reference cl-print help-fns radix-tree misearch multi-isearch
dired-aux pulse color grep let-alist delsel so-long pixel-scroll
cua-base jane-fe-read-feature tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat parse-time iso8601 skeleton
page-ext magit-extras magit-bookmark magit-submodule magit-obsolete
magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull
magit-fetch magit-clone magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit magit-repos
magit-apply magit-wip magit-log which-func magit-diff smerge-mode diff
diff-mode git-commit log-edit message sendmail yank-media puny rfc822
mml mml-sec epa epg rfc6068 epg-config mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils
mailheader pcvs-util add-log magit-core magit-autorevert autorevert
magit-margin magit-transient magit-process magit-mode transient
magit-git magit-section magit-utils dash gnus nnheader gnus-util
mail-utils range mm-util mail-prsvr gdb-mi bindat gud vundo
modus-vivendi-theme modus-themes savehist saveplace site-start
jane-customization jane-diagnose jane-merlin merlin-imenu merlin-xref
xref merlin-cap merlin crm jane-async-merlin jane-completion jane-comint with-editor org-protocol org org-macro org-pcomplete org-list
org-footnote org-faces org-entities time-date noutline outline ob-dot
ob-shell ob ob-tangle org-src ob-ref ob-lob ob-table ob-exp ob-comint
ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold
org-fold-core org-keys oc org-loaddefs cal-menu calendar cal-loaddefs
org-compat org-version org-macs format-spec jane-fe-project
jane-fe-feature-table imenu jane-fe-menu ecaml_plugin edmacro kmacro
bookmark dired-x image+ advice image-file image-converter editorconfig
editorconfig-core editorconfig-core-handle editorconfig-fnmatch
whitespace help-at-pt jane jane-yasnippet jane-micro-features server
jane-diff unified-test-mode shell-file dired dired-loaddefs easy-mmode
jane-sexp jane-python jane-ocaml jane-tuareg-theme jane-aide ert ewoc
debug backtrace find-func tuareg speedbar ezimage dframe tuareg-compat
tuareg-opam flymake-proc flymake project thingatpt smie caml-types
caml-help view find-file compile text-property-search jane-cr jane-align shell pcomplete comint ansi-osc ansi-color ring gnu-elpa-keyring-update
jane-telemetry comp comp-cstr warnings cl-extra help-mode ocp-indent
jane-eglot yasnippet-autoloads vba-mode-autoloads term-cursor-autoloads
swiper-autoloads org-project-capture-autoloads htmlize-autoloads
haskell-mode-autoloads eglot-autoloads editorconfig-conf-mode derived
conf-mode editorconfig-autoloads contrib-autoloads jane-autoloads
jane-util pcase filenotify cus-edit pp cus-load icons wid-edit
adaptive-wrap-autoloads corfu-autoloads csv-mode-autoloads
cyberpunk-theme-autoloads evil-autoloads exwm-autoloads helm-autoloads
helm-core-autoloads async-autoloads ivy-autoloads magit-autoloads
git-commit-autoloads finder-inf magit-section-autoloads popup-autoloads
url-http-ntlm-autoloads url-auth utop-autoloads tuareg-autoloads rx
caml-autoloads vc-hgcmd-autoloads vertico-autoloads compat-autoloads
vundo-autoloads with-editor-autoloads xelb-autoloads
markdown-mode-autoloads multi-line-autoloads info dash-autoloads
s-autoloads shut-up-autoloads sml-mode-autoloads package browse-url url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo x-toolkit xinput2 x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 1149062 125299)
=C2=A0(symbols 48 58114 7)
=C2=A0(strings 32 221941 24168)
=C2=A0(string-bytes 1 8404217)
=C2=A0(vectors 16 95307)
=C2=A0(vector-slots 8 2497373 165939)
=C2=A0(floats 8 730 417)
=C2=A0(intervals 56 59184 781)
=C2=A0(buffers 984 197))



--000000000000d420ca06200bcf86-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 24 05:27:45 2025 Received: (at 72715) by debbugs.gnu.org; 24 Jan 2025 10:27:45 +0000 Received: from localhost ([127.0.0.1]:43758 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbGuf-0005NB-7w for submit@debbugs.gnu.org; Fri, 24 Jan 2025 05:27:45 -0500 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]:54619) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tbGud-0005Mu-5Z for 72715@debbugs.gnu.org; Fri, 24 Jan 2025 05:27:43 -0500 Received: by mail-wr1-x42d.google.com with SMTP id ffacd0b85a97d-3863494591bso1031019f8f.1 for <72715@debbugs.gnu.org>; Fri, 24 Jan 2025 02:27:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1737714456; x=1738319256; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=of7+mubHXo/YjIJ2WUkYhFN7nk2qHQeQzOSfqTCUg+o=; b=ebUOZTN4/B5EYwD54qpHGWRsEwjTqmHlai1aTHJ3XmyVxDRgAb31sVcvWwp7n9NhI8 0LQx3PYDRIdgRoddXncPlfeHqrzj38cUVh0GWOQoNkB1sbXCZbp+fgQLIvoVDSVHtYur LuMw0IUU2qE02c70pYp0Pi0va9tDx9v7y9Cxdsp+svnKRRp6HLDRrfyiZZTRBEnjClrL pK4O4vBVvqkkweI1B3tI7K63wqEAv8v+hPfBhhCbHxxvslS8dJZsRdVDe0cB8uM7c1se 6rAbrOXJtoPRpYQhQ48D+2pjrmzTYiMkQnl9CuHvVVCyU5P8J5rX5xWwY+0E6CYmsEU9 yshg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737714456; x=1738319256; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=of7+mubHXo/YjIJ2WUkYhFN7nk2qHQeQzOSfqTCUg+o=; b=RlJcvmkKYQ6OhjrLRFqiKwIj3/pIIt3HnSoqiJ4ZkeIiqDUwEoQd2ofUeEbAEsWlHi AuSf2o3ru2xAA9NevJs+Qo90l57IiDZohh91lVX5UBAvXpNjcLfHrm6VXBTEyzCL5obq l2EdPbAX7eAERDB041hYFdjytk2Etf3SeYXIbmS994iFD7QiV9D5TV++MKCRkPRLbCcK bVQZzi2JMok8rZs/8A1un/GwtnOjmK9VKb/Gsg2TthjGRhAJCUuKl442anaVM1ip7JpC /8vaHC3qJnG3Ca1tvH8M9bClnhkFBkbs25eN/wnP0WLY3nREwqMvb9dNZ35UVq/W+3Fr GXtQ== X-Gm-Message-State: AOJu0YxDKj4I4tYPtmTL+TwfrkaUM04nC0bjORMAM+tJ5qMV/+AieZcl jUtq6EYu+QqPr+YqbDxnQHJP01Li6jGxT1AvaVjvGwb7LSWgLl9/ X-Gm-Gg: ASbGnctZwT9+y2sNCmlczurNfFmJXSXsjg+x0x3cC/DKTSh1qrrjiItYTdDxqYcfM2K HWhrMkZpKEXkV8b7S8ibUwl0aJ9Md46Kcp2vmYKWOUepqi65BmBl4h7bd/HU8tbHe0I90SnzPYO YwqGUINuZ3kPf5V5h+bzHQBvqdZ2uj2YzNFmgHw3YQ1KYDiUv3YGiYkeioL+gIDES68Tf/FXSD4 r8bCyBz3YvnapCQIaicxBwsy+5Y1oJreul6CquwW1BrT+8kkRkQv72G6tgBYNK9ONZIvzo= X-Google-Smtp-Source: AGHT+IFKNRLYGdaRj4tXJHwlIRwkoDL097909DNedU6xId8LmyPje1AX2JOTr+wj3i/QslSyoycxpw== X-Received: by 2002:a05:6000:186f:b0:38a:9ffb:fe2f with SMTP id ffacd0b85a97d-38bf55be471mr26547179f8f.0.1737714455535; Fri, 24 Jan 2025 02:27:35 -0800 (PST) Received: from krug ([62.28.172.234]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c2a1bb04csm2279059f8f.73.2025.01.24.02.27.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2025 02:27:35 -0800 (PST) From: =?utf-8?B?Sm/Do28gVMOhdm9yYQ==?= To: Spencer Baugh Subject: Re: bug#72715: 29.2.50; eglot didChangeWatchedFiles runs project-files in an arbitary directory, which can be slow In-Reply-To: (Spencer Baugh's message of "Mon, 19 Aug 2024 12:01:30 -0400") References: Date: Fri, 24 Jan 2025 10:27:44 +0000 Message-ID: <871pwsv74v.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 72715 Cc: Dmitry Gutov , 72715@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Spencer Baugh writes: > 6. The eglot implementation of workspace/didChangeWatchedFiles runs > project--files-in-directory Eglot's implementation does not call this, it calls project-files. The implementation details are up to project.el > 7. Emacs hangs for a long time as project--files-in-directory runs "find = /big-directory" > > In particular I've had users experience this when trying to use eglot on > files in large directory trees on NFS, which makes find even slower. The topic of `didChangeWatchedFiles` is somehow contested. The LSP authors state that it's more reliable to do in the client (which makes sense when since there are fewer clients... in fact they poorly disguise their inclination to a $ingle one...). But as you have discovered this solution is more complex, less efficient, and problematic in some ways. Many servers do this watching themselves. > I'm not sure how this could be fixed. > > If project-files could run asynchronously (e.g. with make-process), > would Eglot be able to use that here, instead of synchronous > project-files? It _is_ run asynchronously to the main events. It is run as part of the process filter of the underlying LSP process). The problem is that Emacs is single-threaded and you the user have to pay the time (in minutes) anyway. Because process-files seems to be a relatively "pure" operation, with make-process you could escape that indeed (creating a whole separate process just for another thread which your OS will be able to parallelize or at least cleverly time-slice parallize). Speaking of time-slice parallelize, I wonder if Emacs's threads could be leveraged to do this. Some of this could probably be prototyped, but that solution shouldn't live in Eglot. Here are other solutions: * Why is project-files slow? Why isn't it fast? Minutes to get a file list in 2025? Many operating systems feature file name database that are very fast at answering these queries (linux locatedb, fd, everything in Windows). Even Git has one of these databases, so if the project is Git-backed, it could in theory use that, at least for half-decent results. But I have no idea how project.el is implemented these days. * Why is there no project-directories, which is what Eglot needs in this case. It seems in theory that could be made faster. > Supporting async project-files is on my todo list anyway. Seems reasonable and an interesting goal, but look at the other above two ideas. Jo=C3=A3o