GNU bug report logs - #66117
30.0.50; `find-buffer-visiting' is slow when opening large number of buffers

Previous Next

Full log

Message #157 received at 66117 <at> debbugs.gnu.org (full text, mbox):

From: Ihor Radchenko <yantar92 <at> posteo.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmitry <at> gutov.dev, 66117 <at> debbugs.gnu.org
Subject: Re: bug#66117: 30.0.50; `find-buffer-visiting' is slow when opening
 large number of buffers
Date: Mon, 09 Oct 2023 10:02:42 +0000

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> writes:

>> > My advice is to have a prototype working, then time it on local
>> > filesystems.
>> 
>> See the attached patch.
>
> Thanks.  This still has some unnecessary overhead (I actually thought
> about moving the entire find-buffer-visiting into C, not replacing it
> with Lisp that calls new primitives).  But this already makes
> find-buffer-visiting take an insignificant portion of CPU time, so I
> think it proves that the idea is workable:

I looked into rewriting `find-buffer-visiting' fully in C, but it
appears to be rather tricky because it calls `file-truename' and
`abbreviate-file-name', which are Elisp functions recursively calling
other Elisp functions.

And `abbreviate-file-name' is the slowest function showing up in the
profiler (with my patch applied):

;; Increased the number of files to get more than ~2 sec CPU samples.
(dotimes (i 3000) (with-temp-file (format "/tmp/test/%d.txt" i) (insert "* This is test")))
(dolist (file (directory-files "/tmp/test/" t "txt"))
  (find-file-noselect file))

(cpu-profile-w-patch-3000-files, reverse call tree)

        3130  29%   Automatic GC
        1305  12% + abbreviate-file-name
         845   7% + inhibit-local-variables-p
         842   7% + uniquify-rationalize-file-buffer-names
         719   6% + find-buffer-visiting
         619   5% + locate-dominating-file
         350   3% + dir-locals--all-files
         332   3% + set-auto-mode--apply-alist
         283   2% + hack-local-variables--find-variables
         272   2% + file-truename

I am not sure if we want to go deeper into the rabbit hole, but if we
do, here is also the perf profile:

    25.35%  emacs         emacs                            [.] set_default_internal
    15.73%  emacs         emacs                            [.] process_mark_stack
    10.18%  emacs         emacs                            [.] re_match_2_internal
     8.88%  emacs         emacs                            [.] assq_no_quit
     2.23%  emacs         emacs                            [.] Fstring_equal
     2.01%  emacs         emacs                            [.] pdumper_marked_p_impl

The main contributor is set_default_internal. I _suspect_ (because we
have non-linear scaling with the number of buffers - 3x increase in the
number of files lead to ~6x increase in the run time) that the main
contributor here is let-binding `case-fold-search' - special per-buffer
variable with no default value. In particular, `specbind' has the
following comment:

	    /* If SYMBOL is a per-buffer variable which doesn't have a
	       buffer-local value here, make the `let' change the global
	       value by changing the value of SYMBOL in all buffers not
	       having their own value.  This is consistent with what
	       happens with other buffer-local variables.  */

which implies that the binding will iterate over all the buffers,
leading to non-linear scaling in our scenario.

[cpu-profile-w-patch-3000-files (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.