GNU bug report logs -
#78665
31.0.50; Very slow saves
Previous Next
To reply to this bug, email your comments to 78665 AT debbugs.gnu.org.
There is no need to reopen the bug first.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
monnier <at> iro.umontreal.ca, bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Sun, 01 Jun 2025 20:08:04 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Stefan Monnier <monnier <at> iro.umontreal.ca>:
New bug report received and forwarded. Copy sent to
monnier <at> iro.umontreal.ca, bug-gnu-emacs <at> gnu.org.
(Sun, 01 Jun 2025 20:08:04 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Package: Emacs
Version: 31.0.50
The last few weeks I noticed that saving a buffer was sometimes taking
a very long time (like more than 10s). I finally managed to catch it
in the act while running the profiler, and it seems to have something to
do with `outline--hidden-headings-restore-paths`:
9630 75% - command-execute
9630 75% - call-interactively
8638 67% - funcall-interactively
8630 67% - save-buffer
8630 67% - basic-save-buffer
8607 67% - run-hooks
8607 67% - vc-git-resolve-when-done
8593 67% - vc-resynch-buffer
8593 67% - vc-resynch-window
8593 67% - vc-revert-buffer-internal
8593 67% - revert-buffer
8353 65% - mapc
8353 65% - funcall
8353 65% - #<byte-code-function 1FB>
8353 65% - outline--hidden-headings-restore-paths
8353 65% - outline-map-region
7748 60% - #<byte-code-function 28A>
7732 60% - outline-hide-subtree
7728 60% - outline-flag-subtree
4268 33% + outline-back-to-heading
3393 26% + outline-end-of-subtree
63 0% + outline-flag-region
4 0% + outline-end-of-heading
8 0% + mapcar
4 0% gethash
4 0% pos-bol
605 4% + outline-next-heading
232 1% + run-hook-wrapped
8 0% + revert-buffer--default
14 0% re-search-forward
16 0% + basic-save-buffer-1
4 0% + vc-before-save
3 0% + vc-after-save
8 0% + execute-extended-command
992 7% + byte-code
1855 14% + redisplay_internal (C function)
1166 9% + #<byte-code-function 6A0>
45 0% + timer-event-handler
4 0% + jit-lock--antiblink-post-command
4 0% + reveal-post-command
0 0% ...
This one occurred while saving `transient.el`.
Maybe we could avoid the whole thing by changing VC to not
`revert-buffer` (AFAIK this was done to handle the case where CVS/RCS
would update the "keywords" in the file to reflect its new state, but
I believe this is basically never done nowadays).
But in any case `outline--hidden-headings-restore-paths` just takes too
much time.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Mon, 02 Jun 2025 06:11:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 78665 <at> debbugs.gnu.org (full text, mbox):
> Cc: monnier <at> iro.umontreal.ca
> Date: Sun, 01 Jun 2025 16:06:53 -0400
> From: Stefan Monnier via "Bug reports for GNU Emacs,
> the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
>
> Package: Emacs
> Version: 31.0.50
>
>
> The last few weeks I noticed that saving a buffer was sometimes taking
> a very long time (like more than 10s). I finally managed to catch it
> in the act while running the profiler, and it seems to have something to
> do with `outline--hidden-headings-restore-paths`:
>
> 9630 75% - command-execute
> 9630 75% - call-interactively
> 8638 67% - funcall-interactively
> 8630 67% - save-buffer
> 8630 67% - basic-save-buffer
> 8607 67% - run-hooks
> 8607 67% - vc-git-resolve-when-done
> 8593 67% - vc-resynch-buffer
> 8593 67% - vc-resynch-window
> 8593 67% - vc-revert-buffer-internal
> 8593 67% - revert-buffer
> 8353 65% - mapc
> 8353 65% - funcall
> 8353 65% - #<byte-code-function 1FB>
> 8353 65% - outline--hidden-headings-restore-paths
> 8353 65% - outline-map-region
> 7748 60% - #<byte-code-function 28A>
> 7732 60% - outline-hide-subtree
> 7728 60% - outline-flag-subtree
> 4268 33% + outline-back-to-heading
> 3393 26% + outline-end-of-subtree
> 63 0% + outline-flag-region
> 4 0% + outline-end-of-heading
> 8 0% + mapcar
> 4 0% gethash
> 4 0% pos-bol
> 605 4% + outline-next-heading
> 232 1% + run-hook-wrapped
> 8 0% + revert-buffer--default
> 14 0% re-search-forward
> 16 0% + basic-save-buffer-1
> 4 0% + vc-before-save
> 3 0% + vc-after-save
> 8 0% + execute-extended-command
> 992 7% + byte-code
> 1855 14% + redisplay_internal (C function)
> 1166 9% + #<byte-code-function 6A0>
> 45 0% + timer-event-handler
> 4 0% + jit-lock--antiblink-post-command
> 4 0% + reveal-post-command
> 0 0% ...
>
> This one occurred while saving `transient.el`.
>
> Maybe we could avoid the whole thing by changing VC to not
> `revert-buffer` (AFAIK this was done to handle the case where CVS/RCS
> would update the "keywords" in the file to reflect its new state, but
> I believe this is basically never done nowadays).
>
> But in any case `outline--hidden-headings-restore-paths` just takes too
> much time.
Two questions:
. was this file under Git, and if not, how did
vc-git-resolve-when-done come into play?
. why does vc-revert-buffer-internal ended up calling
outline--hidden-headings-restore-paths? was outline-minor-mode
active in the buffer?
IOW, I'd like to better understand the scope of the use cases where
saving a buffer is now so much slower. (How large was the buffer,
btw?)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Mon, 02 Jun 2025 13:34:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 78665 <at> debbugs.gnu.org (full text, mbox):
> Two questions:
>
> . was this file under Git, and if not, how did
> vc-git-resolve-when-done come into play?
Yes.
> . why does vc-revert-buffer-internal ended up calling
> outline--hidden-headings-restore-paths? was outline-minor-mode
> active in the buffer?
Yes, I enable outline-minor-mode in all my ELisp buffers.
>
> IOW, I'd like to better understand the scope of the use cases where
> saving a buffer is now so much slower. (How large was the buffer,
> btw?)
`transient.el` is about 200kB.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Mon, 02 Jun 2025 15:22:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 78665 <at> debbugs.gnu.org (full text, mbox):
> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: 78665 <at> debbugs.gnu.org
> Date: Mon, 02 Jun 2025 09:33:43 -0400
>
> > Two questions:
> >
> > . was this file under Git, and if not, how did
> > vc-git-resolve-when-done come into play?
>
> Yes.
>
> > . why does vc-revert-buffer-internal ended up calling
> > outline--hidden-headings-restore-paths? was outline-minor-mode
> > active in the buffer?
>
> Yes, I enable outline-minor-mode in all my ELisp buffers.
OK, so I guess this affects the above combination.
> > IOW, I'd like to better understand the scope of the use cases where
> > saving a buffer is now so much slower. (How large was the buffer,
> > btw?)
>
> `transient.el` is about 200kB.
10 sec to save that is awfully slow, I agree.
But I cannot find any change during the recent two months that could
explain that, maybe I missed something?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Mon, 02 Jun 2025 18:50:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 78665 <at> debbugs.gnu.org (full text, mbox):
> 10 sec to save that is awfully slow, I agree.
> But I cannot find any change during the recent two months that could
> explain that, maybe I missed something?
I sent the bug-report because I had finally the backtrace to point the
finger at something but couldn't investigate further at that point.
But now that you mention it, maybe the culprit is:
commit 746a3cb3143194436c4a1a63d26aac890c1a705f
Author: Juri Linkov <juri <at> linkov.net>
Date: Tue Apr 29 19:55:48 2025 +0300
Ignore parens in strings for outline headings in emacs-lisp-mode.
[ Poor Juri: while it's his change, I was the one who asked him to install
it into `master` since it seemed like an obvious improvement (I mean,
it is, but I didn't forsee such impacts on performance). ]
FWIW, I have now a direct way to reproduce the problem:
src/emacs -Q --batch lisp/transient.el -f outline-minor-mode \
--eval "(let (f) \
(hide-sublevels 1000) \
(message \"%S\" (benchmark-call (lambda () \
(setq f (outline-revert-buffer-restore-visibility))))) \
(message \"%S\" (benchmark-call (lambda () \
(syntax-ppss-flush-cache (point-min)) \
(funcall f)))))"
and I see that it takes ~10s in my usual build (with lots of
assertions and largely unoptimized) but ~1.7s in a more normal build,
so it seems part of the problem is in the specific build options I use
which magnify the performance issue.
In the mean time, I got a more detailed profile:
7873 83% - normal-top-level
7873 83% - command-line
7873 83% - command-line-1
7873 83% - eval
7873 83% - funcall
7669 81% - #<byte-code-function F5C>
7669 81% - outline--hidden-headings-restore-paths
7669 81% - outline-map-region
7008 74% - #<byte-code-function F79>
7008 74% - outline-hide-subtree
7008 74% - outline-flag-subtree
3710 39% - outline-back-to-heading
3686 38% + elisp-outline-search
16 0% + outline-on-heading-p
4 0% get-char-property
3234 34% - outline-end-of-subtree
3190 33% - outline-next-heading
3182 33% + elisp-outline-search
28 0% + outline-back-to-heading
12 0% + lisp-outline-level
56 0% + outline-flag-region
8 0% + outline-end-of-heading
661 6% + outline-next-heading
204 2% + outline-revert-buffer-restore-visibility
and I'm surprised by the fact that we spend 39% of the time in
`outline-back-to-heading` but only 0% in `outline-on-heading-p` even
tho, by construction `outline-back-to-heading` is always called from
a BOL looking at the heading of the subtree we want to hide, so it
should just call `outline-on-heading-p` and exit.
AFAICT the problem is that sometimes this heading is already hidden
(because of a preceding subtree covering the current one), so we end up
going back to the preceding/larger subtree and re-hiding it. So for
a subtree like that of `;;; Code:` covering N smaller subtrees that were
previously hidden, we end hiding the `;;; Code:` subtree N times (and
each time, this requires looping through all the covered headings),
which introduces an O(N^2) complexity.
And indeed a search&replace of `^;;; ` with `;;;; ` in `transient.el`
after `;;; Code:` (so that `;;; Code:` covers the whole file and the
quadratic performance is in full display) brings the time to run the
above test to more than 40s, in the normal build!
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Mon, 02 Jun 2025 22:19:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 78665 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
> AFAICT the problem is that sometimes this heading is already hidden
> (because of a preceding subtree covering the current one), so we end up
> going back to the preceding/larger subtree and re-hiding it. So for
> a subtree like that of `;;; Code:` covering N smaller subtrees that were
> previously hidden, we end hiding the `;;; Code:` subtree N times (and
> each time, this requires looping through all the covered headings),
> which introduces an O(N^2) complexity.
The patch below seems to fix it (and bug#78673 as well).
Juri, comments?
Stefan
[outline.patch (text/x-diff, inline)]
diff --git a/lisp/outline.el b/lisp/outline.el
index 9d453881b7e..5e8a3360c63 100644
--- a/lisp/outline.el
+++ b/lisp/outline.el
@@ -685,6 +685,7 @@ outline-next-preface
(goto-char (match-beginning 0))
;; Compensate "\n" from the beginning of regexp
(when (and outline-search-function (not (bobp))) (forward-char -1)))
+ ;; FIXME: Use `outline--end-of-previous'.
(when (and (bolp) (or outline-blank-line (eobp)) (not (bobp)))
(forward-char -1)))
@@ -1287,6 +1288,16 @@ outline-flag-subtree
(progn (outline-end-of-subtree) (point))
flag)))
+(defun outline--end-of-previous ()
+ "Go back from BOH (or EOB) to end of previous element."
+ (if (eobp)
+ (if (bolp) (forward-char -1))
+ ;; Go to end of line before heading
+ (forward-char -1)
+ (if (and outline-blank-line (bolp))
+ ;; leave blank line before heading
+ (forward-char -1))))
+
(defun outline-end-of-subtree ()
"Move to the end of the current subtree."
(outline-back-to-heading)
@@ -1298,12 +1309,7 @@ outline-end-of-subtree
(outline-next-heading))
(if (and (bolp) (not (eolp)))
;; We stopped at a nonempty line (the next heading).
- (progn
- ;; Go to end of line before heading
- (forward-char -1)
- (if (and outline-blank-line (bolp))
- ;; leave blank line before heading
- (forward-char -1))))))
+ (outline--end-of-previous))))
(defun outline-show-branches ()
"Show all subheadings of this heading, but not their bodies."
@@ -1717,8 +1723,8 @@ outline-hide-by-heading-regexp
(run-hooks 'outline-view-change-hook))
(defun outline--hidden-headings-paths ()
- "Return a hash with headings of currently hidden outlines.
-Every hash key is a list whose elements compose a complete path
+ "Return a hash-table with headings of currently hidden outlines.
+Every key is a list whose elements compose a complete path
of headings descending from the top level down to the bottom level.
This is useful to save the hidden outlines and restore them later
after reverting the buffer. Also return the outline where point
@@ -1730,40 +1736,60 @@ outline--hidden-headings-paths
(current-end (when current-heading-p (pos-eol))))
(outline-map-region
(lambda ()
- (let* ((level (funcall outline-level))
- (heading (buffer-substring-no-properties (pos-bol) (pos-eol))))
- (while (and path (>= (cdar path) level))
- (pop path))
- (push (cons heading level) path)
- (when (save-excursion
- (outline-end-of-heading)
- (seq-some (lambda (o) (eq (overlay-get o 'invisible)
- 'outline))
- (overlays-at (point))))
- (setf (gethash (mapcar #'car path) paths) t))
+ (let ((level (funcall outline-level)))
+ (if (outline-invisible-p)
+ ;; Covered by "the" previous heading.
+ (cl-callf (lambda (l) (if (numberp l) (min l level) level))
+ (gethash (mapcar #'car path) paths))
+ (let ((heading (buffer-substring-no-properties (pos-bol) (pos-eol))))
+ (while (and path (>= (cdar path) level))
+ (pop path))
+ (push (cons heading level) path)
+ (when (save-excursion
+ (outline-end-of-heading)
+ (outline-invisible-p))
+ (setf (gethash (mapcar #'car path) paths) t))))
(when (and current-heading-p (<= current-beg (point) current-end))
(setq current-path (mapcar #'car path)))))
(point-min) (point-max))
(list paths current-path)))
(defun outline--hidden-headings-restore-paths (paths current-path)
- "Restore hidden outlines from a hash of hidden headings.
+ "Restore hidden outlines from a hash-table of hidden headings.
This is useful after reverting the buffer to restore the outlines
hidden by `outline--hidden-headings-paths'. Also restore point
on the same outline where point was before reverting the buffer."
- (let (path current-point outline-view-change-hook)
+ (let ((hidelevel nil) (hidestart nil)
+ path current-point outline-view-change-hook)
(outline-map-region
(lambda ()
- (let* ((level (funcall outline-level))
- (heading (buffer-substring (pos-bol) (pos-eol))))
- (while (and path (>= (cdar path) level))
- (pop path))
- (push (cons heading level) path)
- (when (gethash (mapcar #'car path) paths)
- (outline-hide-subtree))
+ (let ((level (funcall outline-level)))
+ (if (and (numberp hidelevel) (<= hidelevel level))
+ nil
+ (when hidestart
+ (outline-flag-region hidestart
+ (save-excursion (outline--end-of-previous)
+ (point))
+ t)
+ (setq hidestart nil))
+ (let* ((heading (buffer-substring-no-properties
+ (pos-bol) (pos-eol))))
+ (while (and path (>= (cdar path) level))
+ (pop path))
+ (push (cons heading level) path)
+ (when (setq hidelevel (gethash (mapcar #'car path) paths))
+ (setq hidestart (save-excursion (outline-end-of-heading)
+ (point))))))
(when (and current-path (equal current-path (mapcar #'car path)))
(setq current-point (point)))))
(point-min) (point-max))
+ (when hidestart
+ (outline-flag-region hidestart
+ (save-excursion
+ (goto-char (point-max))
+ (outline--end-of-previous)
+ (point))
+ t))
(when current-point (goto-char current-point))))
(defun outline-revert-buffer-restore-visibility ()
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Tue, 03 Jun 2025 16:43:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 78665 <at> debbugs.gnu.org (full text, mbox):
> I sent the bug-report because I had finally the backtrace to point the
> finger at something but couldn't investigate further at that point.
> But now that you mention it, maybe the culprit is:
>
> commit 746a3cb3143194436c4a1a63d26aac890c1a705f
> Author: Juri Linkov <juri <at> linkov.net>
> Date: Tue Apr 29 19:55:48 2025 +0300
>
> Ignore parens in strings for outline headings in emacs-lisp-mode.
>
> [ Poor Juri: while it's his change, I was the one who asked him to install
> it into `master` since it seemed like an obvious improvement (I mean,
> it is, but I didn't forsee such impacts on performance). ]
So Jonas was right? `syntax-ppss' is too slow
when called too many times? Maybe it's possible
to reduce the number of calls?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Tue, 03 Jun 2025 16:49:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 78665 <at> debbugs.gnu.org (full text, mbox):
>> AFAICT the problem is that sometimes this heading is already hidden
>> (because of a preceding subtree covering the current one), so we end up
>> going back to the preceding/larger subtree and re-hiding it. So for
>> a subtree like that of `;;; Code:` covering N smaller subtrees that were
>> previously hidden, we end hiding the `;;; Code:` subtree N times (and
>> each time, this requires looping through all the covered headings),
>> which introduces an O(N^2) complexity.
>
> The patch below seems to fix it (and bug#78673 as well).
> Juri, comments?
I'm testing it.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#78665; Package
emacs.
(Wed, 04 Jun 2025 17:18:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 78665 <at> debbugs.gnu.org (full text, mbox):
>>> AFAICT the problem is that sometimes this heading is already hidden
>>> (because of a preceding subtree covering the current one), so we end up
>>> going back to the preceding/larger subtree and re-hiding it. So for
>>> a subtree like that of `;;; Code:` covering N smaller subtrees that were
>>> previously hidden, we end hiding the `;;; Code:` subtree N times (and
>>> each time, this requires looping through all the covered headings),
>>> which introduces an O(N^2) complexity.
>>
>> The patch below seems to fix it (and bug#78673 as well).
>> Juri, comments?
>
> I'm testing it.
I've completed testing your patch and found no problems.
Reply sent
to
Stefan Monnier <monnier <at> iro.umontreal.ca>:
You have taken responsibility.
(Wed, 04 Jun 2025 20:41:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Stefan Monnier <monnier <at> iro.umontreal.ca>:
bug acknowledged by developer.
(Wed, 04 Jun 2025 20:41:02 GMT)
Full text and
rfc822 format available.
Message #34 received at 78665-done <at> debbugs.gnu.org (full text, mbox):
> I've completed testing your patch and found no problems.
Thanks, pushed to `master`,
Stefan
This bug report was last modified 10 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.