GNU bug report logs - #16830
[Bug] 24.3.50; massive slow down in forward-line

Previous Next

Package: emacs;

Reported by: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>

Date: Fri, 21 Feb 2014 12:18:01 UTC

Severity: important

Done: Eli Zaretskii <eliz <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 16830 in the body.
You can then email your comments to 16830 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Fri, 21 Feb 2014 12:18:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Fri, 21 Feb 2014 12:18:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
To: bug-gnu-emacs <at> gnu.org
Subject: [Bug] 24.3.50; massive slow down in forward-line
Date: Fri, 21 Feb 2014 13:16:07 +0100
Good day,

I have measured a massive slow down for emacs24 on windows. I struggled
while using org-mode columnview, which takes for a short table several
minutes compared to a second with an older versino of emacs.

I tried to dig it down:

Running on an Core I7 Laptop with Windows 7/64 bit,
starting "emacs -q"

I got the  following times (elp measurement) for the org-mode function
org-goto-line:

Emacs 24.3.1 is ok, on Emacs 24.3.50 (I tested
https://www.dropbox.com/sh/3pgcb3iiy8s9irl/77pwM1O-5K/emacs-20140217-r116465-bin-i386.zip
) got a slow down of factor 10!!

org-goto-line (24.3.1)
test-org-goto-line  1           0.2           0.2
forward-line        10000       0.0640000000  6.40...e-006
widen               10000       0.003         3e-007
goto-char           10001       0.002         1.99...e-007

org-forward-line with narrow (24.3.1)
test-forward-line  1           0.042         0.042
forward-line       10000       0.003         3e-007
goto-char          1           0.0           0.0

org-forward-line without narrow (24.3.1)
test-forward-line  1           0.046         0.046
forward-line       10000       0.007         7e-007
goto-char          1           0.0           0.0


org-goto-line (24.3.50.1)
test-org-goto-line  1           2.141         2.141
forward-line        10000       1.8479999999  0.0001847999
goto-char           10001       0.006         5.99...e-007
widen               10000       0.005         5e-007

org-forward-line with narrow (24.3.50.1)
test-forward-line  1           0.102         0.102
forward-line       10000       0.0160000000  1.60...e-006
goto-char          1           0.0           0.0

org-forward-line without narrow (24.3.50.1)
test-forward-line  1           0.125         0.125
forward-line       10000       0.005         5e-007
goto-char          1           0.0           0.0

Problem seemss to be forward-line.

Tested with following test case:

#+BEGIN_SRC elisp
(defsubst my-org-goto-line (N)
  (save-restriction
    (widen)
    (goto-char (point-min))
    (forward-line (1- N))))

(defun test-org-goto-line ()
  (with-temp-buffer
    (open-line 10000)
    (goto-char (point-min))
    (narrow-to-region (point-min) (point-max))
    (dotimes (i 10000)
      (my-org-goto-line i)
      )
    )
  )

(defun test-forward-line (with-narrow)
  (with-temp-buffer
    (open-line 10000)
    (goto-char (point-min))
    (if with-narrow
        (narrow-to-region (point-min) (point-max)))
    (dotimes (i 10000)
      (forward-line 1)
      )
    )
  )

(defvar test-buffer nil)
(defun test-performance/org-goto-line ()
  (interactive)
  (elp-instrument-list '(test-org-goto-line forward-line goto-char widen))
  (test-org-goto-line)
  (let ((elp-recycle-buffers-p nil)
        rc)
    (elp-results)
    (setq rc (buffer-string))
    (kill-buffer)
    (with-current-buffer test-buffer
      (insert "org-goto-line ("
              emacs-version
              ")\n")
      (insert rc "\n"))
    )
  (elp-restore-all)
)

(defun test-performance/forward-line (with-narrow)
  (interactive)
  (elp-instrument-list '(test-forward-line forward-line goto-char widen))
  (test-forward-line with-narrow)
  (let ((elp-recycle-buffers-p nil)
        rc)
    (elp-results)
    (setq rc (buffer-string))
    (kill-buffer)
    (with-current-buffer test-buffer
      (insert (format "org-forward-line %s (" (if with-narrow "with narrow" "without narrow"))
              emacs-version
              ")\n")
      (insert rc "\n")))
  (elp-restore-all)
)

(defun test-performance ()
  (interactive)
  (setq test-buffer (generate-new-buffer "*Performance*"))
  (test-performance/org-goto-line)
  (test-performance/forward-line t)
  (test-performance/forward-line nil)
  (pop-to-buffer test-buffer))
#+END:

With kind regards,
Stefan

-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Fri, 21 Feb 2014 12:33:02 GMT) Full text and rfc822 format available.

Message #8 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Fri, 21 Feb 2014 14:32:30 +0200
> Date: Fri, 21 Feb 2014 13:16:07 +0100
> From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
> 
> I have measured a massive slow down for emacs24 on windows. I struggled
> while using org-mode columnview, which takes for a short table several
> minutes compared to a second with an older versino of emacs.
> 
> I tried to dig it down:
> 
> Running on an Core I7 Laptop with Windows 7/64 bit,
> starting "emacs -q"
> 
> I got the  following times (elp measurement) for the org-mode function
> org-goto-line:
> 
> Emacs 24.3.1 is ok, on Emacs 24.3.50 (I tested
> https://www.dropbox.com/sh/3pgcb3iiy8s9irl/77pwM1O-5K/emacs-20140217-r116465-bin-i386.zip
> ) got a slow down of factor 10!!

Does this happen with any Org file, even small ones?

If not, perhaps you could show a file with which you see this.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Fri, 21 Feb 2014 15:52:02 GMT) Full text and rfc822 format available.

Message #11 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Fri, 21 Feb 2014 16:51:20 +0100
Mail von Eli Zaretskii, Fri, 21 Feb 2014 at 14:32:30 +0200:

Hello,

> > Emacs 24.3.1 is ok, on Emacs 24.3.50 (I tested
> > https://www.dropbox.com/sh/3pgcb3iiy8s9irl/77pwM1O-5K/emacs-20140217-r116465-bin-i386.zip
> > ) got a slow down of factor 10!!
> 
> Does this happen with any Org file, even small ones?

No.

My org-file is around 1.5 MiB. I make a columnview of a subtree, which
is around 28 KiB. If I put this subtree in a standalone file it's faster.

But it is a change between the versions.

> If not, perhaps you could show a file with which you see this.

No, sorry.

Complete measurements:

The org-mode file is in sum 1.5 MiB, the subtree in which I produce
a table columnview is around 28 KiB.

Noticable is that in case of big file org-goto-line is called 54
times with extremly different runtime:

emacs 24.3.1:
org-goto-line                                                 54          0.0240000000  0.0004444444

emacs 24.3.50.1
org-goto-line                                                 54          110.9         2.0537037037

In contrast this function is not called when rebuilding the table
in a small file case.

#+BEGIN_QUOTE
C-c C-c on subtree in big file.

GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
 of 2013-03-17 on MARVIN
Org-mode version 8.2.5h (release_8.2.5h-633-gec0998 @ d:/home/hs/.emacs-lisp/org-mode/)

elp-instrument-packages org-columns

org-columns-capture-view              1           0.423         0.423
org-columns                           1           0.168         0.168
org-columns-display-here              54          0.0150000000  0.0002777777
org-columns-cleanup-item              54          0.003         5.55...e-005
org-columns-new-overlay               216         0.003         1.38...e-005
org-columns-remove-overlays           2           0.001         0.0005
org-columns-compute-all               1           0.001         0.001
org-columns-quit                      1           0.001         0.001
org-columns-goto-top-level            1           0.0           0.0
org-columns-get-format                1           0.0           0.0
org-columns-compact-links             54          0.0           0.0
org-columns-add-ellipses              162         0.0           0.0
org-columns-get-autowidth-alist       1           0.0           0.0
org-columns-compile-format            1           0.0           0.0
org-columns-display-here-title        1           0.0           0.0


elp-instrument-packages org

org-ctrl-c-ctrl-c                                             1           0.718         0.718
org-update-dblock                                             1           0.71          0.71
org-dblock-write:columnview                                   1           0.67          0.67
org-columns-capture-view                                      1           0.428         0.428
org-columns                                                   1           0.222         0.222
org-entry-properties                                          54          0.1480000000  0.0027407407
org-element--cache-sync                                       2           0.0740000000  0.0370000000
org-get-category                                              54          0.051         0.0009444444
org-refresh-category-properties                               1           0.051         0.051
org-get-tags-at                                               54          0.0390000000  0.0007222222
org-element--cache-compare                                    7894        0.039         4.94...e-006
org-up-heading-safe                                           251         0.0350000000  0.0001394422
org-end-of-subtree                                            13          0.0260000000  0.0020000000
org-goto-line                                                 54          0.0240000000  0.0004444444
org-element--cache-record-change                              81          0.0230000000  0.0002839506
org-get-property-block                                        110         0.0200000000  0.0001818181
org-columns-display-here                                      54          0.0160000000  0.0002962962
org-entry-get                                                 58          0.0150000000  0.0002586206
org-entry-blocked-p                                           54          0.0130000000  0.0002407407
org-element--cache-before-change                              82          0.008         9.75...e-005
org-back-to-heading                                           906         0.008         8.83...e-006
org-babel-execute-src-block-maybe                             1           0.007         0.007
org-babel-where-is-src-block-head                             1           0.007         0.007
org-babel-execute-maybe                                       1           0.007         0.007
org-babel-execute-safely-maybe                                1           0.007         0.007
org-columns-new-overlay                                       216         0.006         2.77...e-005
org-table-align                                               1           0.006         0.006
org-outline-level                                             461         0.004         8.67...e-006
org-before-first-heading-p                                    56          0.003         5.35...e-005
org-prepare-dblock                                            1           0.003         0.003
org-find-entry-with-id                                        1           0.003         0.003
org-activate-footnote-links                                   2           0.002         0.001
org-columns-quit                                              1           0.002         0.002
org-footnote-next-reference-or-definition                     2           0.002         0.001
org-columns-goto-top-level                                    1           0.001         0.001
org-get-tags-string                                           54          0.001         1.85...e-005
org-element-context                                           1           0.001         0.001
org-columns-remove-overlays                                   2           0.001         0.0005
org-element--current-element                                  2           0.001         0.0005
org-element-at-point                                          1           0.001         0.001
org-columns-get-autowidth-alist                               1           0.001         0.001
org-split-string                                              165         0.001         6.06...e-006
org-remove-uninherited-tags                                   161         0.001         6.21...e-006
org-entry-get-with-inheritance                                2           0.001         0.0005
org-font-lock-add-tag-faces                                   2           0.0           0.0
org-fontify-entities                                          2           0.0           0.0
org-columns-cleanup-item                                      54          0.0           0.0
org-table-begin                                               1           0.0           0.0
org-columns-get-format                                        1           0.0           0.0
org-reduced-level                                             54          0.0           0.0
org-quote-vert                                                162         0.0           0.0
org-element-cache-get                                         3           0.0           0.0
org-table-goto-column                                         1           0.0           0.0
org-columns-compact-links                                     54          0.0           0.0
org-activate-code                                             2           0.0           0.0
org-babel-get-inline-src-block-matches                        1           0.0           0.0
org-string-width                                              165         0.0           0.0
org-table-end                                                 1           0.0           0.0
org-babel-lob-execute-maybe                                   1           0.0           0.0
org-element--cache-mapc                                       2           0             0.0
org-listtable-to-string                                       1           0.0           0.0
org-columns-add-ellipses                                      162         0.0           0.0
org-string-match-p                                            2           0.0           0.0
org-delete-all                                                161         0.0           0.0
org-do-latex-and-related                                      2           0.0           0.0
org-fontify-meta-lines-and-blocks-1                           2           0.0           0.0
org-verify-version                                            1           0.0           0.0
org-set-local                                                 6           0.0           0.0
org-font-lock-hook                                            2           0.0           0.0
org-fontify-macros                                            2           0.0           0.0
org-element--collect-affiliated-keywords                      2           0.0           0.0
org-raise-scripts                                             2           0.0           0.0
org-element-property                                          1           0.0           0.0
org-before-change-function                                    82          0.0           0.0
org-font-lock-add-priority-faces                              2           0.0           0.0
org-activate-angle-links                                      2           0.0           0.0
org-get-limited-outline-regexp                                168         0.0           0.0
org-remove-tabs                                               54          0.0           0.0
org-string-nw-p                                               2           0.0           0.0
org-trim                                                      110         0.0           0.0
org-update-property-plist                                     2           0.0           0.0
org-columns-compute-all                                       1           0.0           0.0
org-element-paragraph-parser                                  1           0.0           0.0
org-add-prop-inherited                                        323         0.0           0.0
org-at-table-p                                                1           0.0           0.0
org-remove-if                                                 2           0.0           0.0
org-activate-target-links                                     2           0.0           0.0
org-item-re                                                   1           0.0           0.0
org-activate-bracket-links                                    2           0.0           0.0
org-hide-wide-columns                                         2           0.0           0.0
org-element-dynamic-block-parser                              1           0.0           0.0
org-get-level-face                                            54          0.0           0.0
org-activate-dates                                            2           0.0           0.0
org-babel-hash-at-point                                       1           0.0           0.0
org-add-props                                                 56          0.0           0.0
org-remove-font-lock-display-properties                       2           0.0           0.0
org-unfontify-region                                          2           0.0           0.0
org-at-heading-p                                              59          0.0           0.0
org-not-nil                                                   2           0.0           0.0
org-element-cache-put                                         2           0.0           0.0
org-columns-compile-format                                    1           0.0           0.0
org-table-current-column                                      1           0.0           0.0
org-overlay-display                                           216         0.0           0.0
org-fontify-drawers                                           2           0.0           0.0
org-beginning-of-dblock                                       1           0.0           0.0
org-babel-lob-get-info                                        1           0.0           0.0
org-activate-tags                                             2           0.0           0.0
org-element-type                                              1           0.0           0.0
org-activate-plain-links                                      2           0.0           0.0
org-do-emphasis-faces                                         2           0.0           0.0
org-fontify-meta-lines-and-blocks                             2           0.0           0.0
org-columns-display-here-title                                1           0.0           0.0
org-add-hook                                                  1           0.0           0.0
#+END_QUOTE

#+BEGIN_QUOTE
C-c C-c on subtree in big file.

GNU Emacs 24.3.50.1 (i686-pc-mingw32)
 of 2014-01-19 on LEG570
Org-mode version 8.2.5h (release_8.2.5h-633-gec0998 @ d:/home/hs/.emacs-lisp/org-mode/)
emacs-20140217-r116465-bin-i386.zip

org-columns                           1           215.017       215.017
org-columns-capture-view              1           1.512         1.512
org-columns-display-here              54          0.0420000000  0.0007777777
org-columns-remove-overlays           2           0.003         0.0015
org-columns-new-overlay               216         0.003         1.38...e-005
org-columns-quit                      1           0.002         0.002
org-columns-cleanup-item              54          0.002         3.70...e-005
org-columns-display-here-title        1           0.001         0.001
org-columns-compile-format            1           0.0           0.0
org-columns-compute-all               1           0.0           0.0
org-columns-number-to-string          1           0.0           0.0
org-columns-get-autowidth-alist       1           0.0           0.0
org-columns-add-ellipses              162         0.0           0.0
org-columns-compact-links             54          0.0           0.0
org-columns-get-format                1           0.0           0.0
org-columns-goto-top-level            1           0.0           0.0


elp-instrument-packages org
org-ctrl-c-ctrl-c                                             1           228.703       228.703
org-update-dblock                                             1           228.701       228.701
org-dblock-write:columnview                                   1           226.705       226.705
org-columns                                                   1           216.804       216.804
org-goto-line                                                 54          110.9         2.0537037037
org-table-align                                               1           8.276         8.276
org-columns-capture-view                                      1           1.603         1.603
org-entry-properties                                          54          0.2940000000  0.0054444444
org-get-tags-at                                               54          0.1210000000  0.0022407407
org-entry-get                                                 58          0.1210000000  0.0020862068
org-entry-blocked-p                                           54          0.1190000000  0.0022037037
org-up-heading-safe                                           251         0.0940000000  0.0003745019
org-back-to-heading                                           870         0.0590000000  6.78...e-005
org-outline-level                                             449         0.0490000000  0.0001091314
org-columns-display-here                                      54          0.0460000000  0.0008518518
org-element--cache-record-change                              81          0.0370000000  0.0004567901
org-get-property-block                                        110         0.0310000000  0.0002818181
org-element--cache-before-change                              81          0.0290000000  0.0003580246
org-element--cache-sync                                       2           0.012         0.006
org-before-first-heading-p                                    56          0.0090000000  0.0001607142
org-columns-new-overlay                                       216         0.006         2.77...e-005
org-split-string                                              165         0.005         3.03...e-005
org-find-entry-with-id                                        1           0.004         0.004
org-prepare-dblock                                            1           0.004         0.004
org-get-limited-outline-regexp                                169         0.004         2.36...e-005
org-columns-cleanup-item                                      54          0.004         7.40...e-005
org-do-emphasis-faces                                         3           0.003         0.001
org-at-heading-p                                              63          0.002         3.17...e-005
org-end-of-subtree                                            1           0.002         0.002
org-columns-quit                                              1           0.002         0.002
org-element-at-point                                          1           0.002         0.002
org-columns-remove-overlays                                   2           0.002         0.001
org-element-context                                           1           0.002         0.002
org-get-tags-string                                           54          0.002         3.70...e-005
org-fontify-meta-lines-and-blocks                             3           0.001         0.0003333333
org-clock-update-mode-line                                    4           0.001         0.00025
org-beginning-of-dblock                                       1           0.001         0.001
org-get-category                                              54          0.001         1.85...e-005
org-overlay-display                                           216         0.001         4.62...e-006
org-entry-get-with-inheritance                                2           0.001         0.0005
org-remove-uninherited-tags                                   161         0.001         6.21...e-006
org-element-paragraph-parser                                  1           0.001         0.001
org-remove-tabs                                               54          0.001         1.85...e-005
org-element--current-element                                  4           0.001         0.00025
org-minutes-to-clocksum-string                                5           0.001         0.0002
org-fontify-meta-lines-and-blocks-1                           3           0.001         0.0003333333
org-listtable-to-string                                       1           0.001         0.001
org-string-width                                              165         0.001         6.06...e-006
org-clock-get-clock-string                                    4           0.001         0.00025
org-columns-goto-top-level                                    1           0.001         0.001
org-add-hook                                                  1           0.0           0.0
org-columns-display-here-title                                1           0.0           0.0
org-activate-plain-links                                      3           0.0           0.0
org-element-type                                              1           0.0           0.0
org-activate-tags                                             3           0.0           0.0
org-babel-lob-get-info                                        1           0.0           0.0
org-fontify-drawers                                           3           0.0           0.0
org-babel-execute-safely-maybe                                1           0.0           0.0
org-babel-execute-maybe                                       1           0.0           0.0
org-hours-to-clocksum-string                                  1           0.0           0.0
org-table-current-column                                      1           0.0           0.0
org-columns-compile-format                                    1           0.0           0.0
org-element-cache-put                                         4           0.0           0.0
org-not-nil                                                   2           0.0           0.0
org-unfontify-region                                          3           0.0           0.0
org-remove-font-lock-display-properties                       3           0.0           0.0
org-add-props                                                 56          0.0           0.0
org-babel-hash-at-point                                       1           0.0           0.0
org-activate-dates                                            3           0.0           0.0
org-footnote-next-reference-or-definition                     3           0.0           0.0
org-get-level-face                                            54          0.0           0.0
org-element-dynamic-block-parser                              1           0.0           0.0
org-hide-wide-columns                                         3           0.0           0.0
org-activate-bracket-links                                    3           0.0           0.0
org-item-re                                                   1           0.0           0.0
org-activate-target-links                                     3           0.0           0.0
org-remove-if                                                 2           0.0           0.0
org-at-table-p                                                1           0.0           0.0
org-add-prop-inherited                                        323         0.0           0.0
org-columns-compute-all                                       1           0.0           0.0
org-update-property-plist                                     2           0.0           0.0
org-trim                                                      98          0.0           0.0
org-columns-number-to-string                                  1           0.0           0.0
org-propertize                                                12          0.0           0.0
org-string-nw-p                                               3           0.0           0.0
org-activate-footnote-links                                   3           0.0           0.0
org-babel-where-is-src-block-head                             1           0.0           0.0
org-activate-angle-links                                      3           0.0           0.0
org-font-lock-add-priority-faces                              3           0.0           0.0
org-before-change-function                                    81          0.0           0.0
org-element-property                                          1           0.0           0.0
org-columns-get-autowidth-alist                               1           0.0           0.0
org-raise-scripts                                             3           0.0           0.0
org-element--collect-affiliated-keywords                      4           0.0           0.0
org-element-drawer-parser                                     1           0.0           0.0
org-fontify-macros                                            3           0.0           0.0
org-font-lock-hook                                            3           0.0           0.0
org-verify-version                                            1           0.0           0.0
org-set-local                                                 5           0.0           0.0
org-element--cache-compare                                    91          0.0           0.0
org-do-latex-and-related                                      3           0.0           0.0
org-delete-all                                                161         0.0           0.0
org-resolve-clocks-if-idle                                    4           0.0           0.0
org-string-match-p                                            3           0.0           0.0
org-columns-add-ellipses                                      162         0.0           0.0
org-babel-execute-src-block-maybe                             1           0.0           0.0
org-element--cache-mapc                                       2           0             0.0
org-clock-get-clocked-time                                    4           0.0           0.0
org-float-time                                                8           0.0           0.0
org-element-property-drawer-parser                            1           0.0           0.0
org-babel-lob-execute-maybe                                   1           0.0           0.0
org-table-end                                                 1           0.0           0.0
org-babel-get-inline-src-block-matches                        1           0.0           0.0
org-activate-code                                             3           0.0           0.0
org-columns-compact-links                                     54          0.0           0.0
org-table-goto-column                                         1           0.0           0.0
org-element-cache-get                                         3           0.0           0.0
org-quote-vert                                                162         0.0           0.0
org-reduced-level                                             54          0.0           0.0
org-columns-get-format                                        1           0.0           0.0
org-table-begin                                               1           0.0           0.0
org-fontify-entities                                          3           0.0           0.0
org-font-lock-add-tag-faces                                   3           0.0           0.0
#+END_QUOTE

Same subtree standalone in org-mode file (28 KiB):

#+BEGIN_QUOTE
C-c C-c on same subtree in small file.

GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
 of 2013-03-17 on MARVIN

org-ctrl-c-ctrl-c                                             1           0.109         0.109
org-update-dblock                                             1           0.109         0.109
org-columns                                                   1           0.094         0.094
org-dblock-write:columnview                                   1           0.094         0.094
org-columns-add-ellipses                                      252         0.063         0.00025
org-columns-display-here                                      63          0.063         0.001
org-entry-properties                                          63          0.031         0.0004920634
org-entry-get                                                 67          0.015         0.0002238805
org-entry-blocked-p                                           63          0.015         0.0002380952
org-get-property-block                                        128         0.015         0.0001171875
org-remove-flyspell-overlays-in                               1           0.0           0.0
org-columns-goto-top-level                                    1           0.0           0.0
org-font-lock-add-tag-faces                                   8           0.0           0.0
org-fontify-entities                                          8           0.0           0.0
org-columns-cleanup-item                                      63          0.0           0.0
org-table-begin                                               1           0.0           0.0
org-columns-get-format                                        1           0.0           0.0
org-reduced-level                                             63          0.0           0.0
org-quote-vert                                                252         0.0           0.0
org-element-cache-get                                         3           0.0           0.0
org-table-goto-column                                         1           0.0           0.0
org-get-tags-string                                           63          0.0           0.0
org-columns-compact-links                                     63          0.0           0.0
org-activate-code                                             8           0.0           0.0
org-columns-capture-view                                      1           0.0           0.0
org-babel-get-inline-src-block-matches                        1           0.0           0.0
org-element-context                                           1           0.0           0.0
org-string-width                                              256         0.0           0.0
org-element--cache-before-change                              97          0.0           0.0
org-table-end                                                 1           0.0           0.0
org-babel-lob-execute-maybe                                   1           0.0           0.0
org-element-property-drawer-parser                            1           0.0           0.0
org-columns-new-overlay                                       315         0.0           0.0
org-babel-execute-src-block-maybe                             1           0.0           0.0
org-listtable-to-string                                       1           0.0           0.0
org-string-match-p                                            8           0.0           0.0
org-delete-all                                                125         0.0           0.0
org-do-latex-and-related                                      8           0.0           0.0
org-element--cache-compare                                    20          0.0           0.0
org-fontify-meta-lines-and-blocks-1                           8           0.0           0.0
org-verify-version                                            1           0.0           0.0
org-minutes-to-clocksum-string                                1           0.0           0.0
org-columns-remove-overlays                                   2           0.0           0.0
org-font-lock-hook                                            8           0.0           0.0
org-element--current-element                                  4           0.0           0.0
org-fontify-macros                                            8           0.0           0.0
org-element-drawer-parser                                     1           0.0           0.0
org-element--collect-affiliated-keywords                      4           0.0           0.0
org-raise-scripts                                             8           0.0           0.0
org-element-at-point                                          1           0.0           0.0
org-columns-get-autowidth-alist                               1           0.0           0.0
org-element-property                                          1           0.0           0.0
org-before-change-function                                    97          0.0           0.0
org-font-lock-add-priority-faces                              8           0.0           0.0
org-activate-angle-links                                      8           0.0           0.0
org-babel-where-is-src-block-head                             1           0.0           0.0
org-element--cache-record-change                              96          0.0           0.0
org-split-string                                              129         0.0           0.0
org-activate-footnote-links                                   8           0.0           0.0
org-back-to-heading                                           630         0.0           0.0
org-before-first-heading-p                                    65          0.0           0.0
org-get-limited-outline-regexp                                200         0.0           0.0
org-remove-tabs                                               63          0.0           0.0
org-string-nw-p                                               8           0.0           0.0
org-element--cache-sync                                       2           0.0           0.0
org-rear-nonsticky-at                                         1           0.0           0.0
org-columns-number-to-string                                  1           0.0           0.0
org-trim                                                      162         0.0           0.0
org-update-property-plist                                     2           0.0           0.0
org-columns-compute-all                                       1           0.0           0.0
org-element-paragraph-parser                                  1           0.0           0.0
org-add-prop-inherited                                        62          0.0           0.0
org-at-table-p                                                1           0.0           0.0
org-remove-if                                                 2           0.0           0.0
org-activate-target-links                                     8           0.0           0.0
org-item-re                                                   1           0.0           0.0
org-columns-quit                                              1           0.0           0.0
org-activate-bracket-links                                    8           0.0           0.0
org-end-of-subtree                                            1           0.0           0.0
org-up-heading-safe                                           165         0.0           0.0
org-hide-wide-columns                                         8           0.0           0.0
org-element-dynamic-block-parser                              1           0.0           0.0
org-prepare-dblock                                            1           0.0           0.0
org-get-level-face                                            63          0.0           0.0
org-footnote-next-reference-or-definition                     8           0.0           0.0
org-activate-dates                                            8           0.0           0.0
org-remove-uninherited-tags                                   125         0.0           0.0
org-babel-hash-at-point                                       1           0.0           0.0
org-add-props                                                 65          0.0           0.0
org-remove-font-lock-display-properties                       8           0.0           0.0
org-unfontify-region                                          8           0.0           0.0
org-at-heading-p                                              73          0.0           0.0
org-not-nil                                                   2           0.0           0.0
org-get-tags-at                                               63          0.0           0.0
org-element-cache-put                                         4           0.0           0.0
org-find-entry-with-id                                        1           0.0           0.0
org-columns-compile-format                                    1           0.0           0.0
org-table-current-column                                      1           0.0           0.0
org-entry-get-with-inheritance                                2           0.0           0.0
org-hours-to-clocksum-string                                  1           0.0           0.0
org-babel-execute-maybe                                       1           0.0           0.0
org-overlay-display                                           315         0.0           0.0
org-babel-execute-safely-maybe                                1           0.0           0.0
org-fontify-drawers                                           8           0.0           0.0
org-get-category                                              63          0.0           0.0
org-beginning-of-dblock                                       1           0.0           0.0
org-babel-lob-get-info                                        1           0.0           0.0
org-outline-level                                             268         0.0           0.0
org-activate-tags                                             8           0.0           0.0
org-element-type                                              1           0.0           0.0
org-activate-plain-links                                      8           0.0           0.0
org-do-emphasis-faces                                         8           0.0           0.0
org-fontify-meta-lines-and-blocks                             8           0.0           0.0
org-table-align                                               1           0.0           0.0
org-columns-display-here-title                                1           0.0           0.0
org-add-hook                                                  1           0.0           0.0
#+END_QUOTE

#+BEGIN_QUOTE
C-c C-c on same subtree in small file.

GNU Emacs 24.3.50.1 (i686-pc-mingw32)
 of 2014-02-17 on ODIEONE

org-ctrl-c-ctrl-c                                   1           0.858         0.858
org-update-dblock                                   1           0.733         0.733
org-entry-properties                                63          0.139         0.0022063492
org-babel-execute-safely-maybe                      1           0.125         0.125
org-babel-execute-maybe                             1           0.125         0.125
org-babel-execute-src-block-maybe                   1           0.125         0.125
org-babel-get-inline-src-block-matches              1           0.125         0.125
org-get-tags-at                                     63          0.077         0.0012222222
org-get-property-block                              128         0.047         0.0003671875
org-element--cache-before-change                    96          0.047         0.0004895833
org-entry-blocked-p                                 63          0.047         0.0007460317
org-entry-get                                       67          0.047         0.0007014925
org-element--cache-record-change                    96          0.046         0.0004791666
org-remove-uninherited-tags                         125         0.031         0.000248
org-before-first-heading-p                          65          0.016         0.0002461538
org-back-to-heading                                 630         0.016         2.53...e-005
org-split-string                                    129         0.016         0.0001240310
org-up-heading-safe                                 165         0.015         9.09...e-005
org-add-hook                                        1           0.0           0.0
org-refresh-category-properties                     1           0.0           0.0
org-fontify-meta-lines-and-blocks                   8           0.0           0.0
org-do-emphasis-faces                               8           0.0           0.0
org-activate-plain-links                            8           0.0           0.0
org-element-type                                    1           0.0           0.0
org-activate-tags                                   8           0.0           0.0
org-outline-level                                   268         0.0           0.0
org-babel-lob-get-info                              1           0.0           0.0
org-beginning-of-dblock                             1           0.0           0.0
org-get-category                                    63          0.0           0.0
org-fontify-drawers                                 8           0.0           0.0
org-overlay-display                                 315         0.0           0.0
org-entry-get-with-inheritance                      2           0.0           0.0
org-element-cache-put                               2           0.0           0.0
org-find-entry-with-id                              1           0.0           0.0
org-not-nil                                         2           0.0           0.0
org-at-heading-p                                    68          0.0           0.0
org-unfontify-region                                8           0.0           0.0
org-remove-font-lock-display-properties             8           0.0           0.0
org-add-props                                       65          0.0           0.0
org-find-if                                         58          0.0           0.0
org-babel-hash-at-point                             1           0.0           0.0
org-activate-dates                                  8           0.0           0.0
org-footnote-next-reference-or-definition           8           0.0           0.0
org-get-level-face                                  63          0.0           0.0
org-prepare-dblock                                  1           0.0           0.0
org-element-dynamic-block-parser                    1           0.0           0.0
org-hide-wide-columns                               8           0.0           0.0
org-end-of-subtree                                  1           0.0           0.0
org-activate-bracket-links                          8           0.0           0.0
org-item-re                                         1           0.0           0.0
org-activate-target-links                           8           0.0           0.0
org-remove-if                                       2           0.0           0.0
org-at-table-p                                      1           0.0           0.0
org-add-prop-inherited                              62          0.0           0.0
org-element-paragraph-parser                        1           0.0           0.0
org-update-property-plist                           2           0.0           0.0
org-trim                                            162         0.0           0.0
org-element--cache-sync                             2           0.0           0.0
org-rear-nonsticky-at                               1           0.0           0.0
org-string-nw-p                                     8           0.0           0.0
org-remove-tabs                                     63          0.0           0.0
org-get-limited-outline-regexp                      197         0.0           0.0
org-activate-footnote-links                         8           0.0           0.0
org-babel-where-is-src-block-head                   1           0.0           0.0
org-activate-angle-links                            8           0.0           0.0
org-font-lock-add-priority-faces                    8           0.0           0.0
org-before-change-function                          96          0.0           0.0
org-defkey                                          58          0.0           0.0
org-element-property                                1           0.0           0.0
org-element-at-point                                1           0.0           0.0
org-raise-scripts                                   8           0.0           0.0
org-element--collect-affiliated-keywords            2           0.0           0.0
org-fontify-macros                                  8           0.0           0.0
org-element--current-element                        2           0.0           0.0
org-font-lock-hook                                  8           0.0           0.0
org-fontify-meta-lines-and-blocks-1                 8           0.0           0.0
org-element--cache-compare                          287         0.0           0.0
org-do-latex-and-related                            8           0.0           0.0
org-string-match-p                                  8           0.0           0.0
org-element--cache-mapc                             1           0.0           0.0
org-babel-lob-execute-maybe                         1           0.0           0.0
org-string-width                                    256         0.0           0.0
org-element-context                                 1           0.0           0.0
org-activate-code                                   8           0.0           0.0
org-get-tags-string                                 63          0.0           0.0
org-key                                             58          0.0           0.0
org-element-cache-get                               3           0.0           0.0
org-quote-vert                                      252         0.0           0.0
org-reduced-level                                   63          0.0           0.0
org-fontify-entities                                8           0.0           0.0
org-font-lock-add-tag-faces                         8           0.0           0.0
org-remove-flyspell-overlays-in                     1           0.0           0.0
#+END_QUOTE

With kind regards,
Stefan

-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Fri, 21 Feb 2014 17:51:02 GMT) Full text and rfc822 format available.

Message #14 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Fri, 21 Feb 2014 12:50:10 -0500
>> If not, perhaps you could show a file with which you see this.
> No, sorry.

Could you maybe "neuter" the file?  E.g pass it through

   sed 's/[[:alnum:]]/a/'

so that it doesn't contain any important information any more (while
still triggering the problem)?


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 08:39:01 GMT) Full text and rfc822 format available.

Message #17 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 09:38:42 +0100
Mail von Stefan Monnier, Fri, 21 Feb 2014 at 12:50:10 -0500:

Good morning,

> >> If not, perhaps you could show a file with which you see this.
> > No, sorry.
> 
> Could you maybe "neuter" the file?  E.g pass it through
> 
>    sed 's/[[:alnum:]]/a/'
> 
> so that it doesn't contain any important information any more (while
> still triggering the problem)?

I have 'neutered' the file as you suggested and rewritten the item
in which the or-structure must be preserved.

If you search for #### you will find the item, which shows the problem.
If you then type C-c C-c on the #+BEGIN line you will get it.

I observed that it is not as slow as in the original file. Perhaps this
has to do with to fontifying, because org-mode now has much less to fontify.
It also makes no difference if the items after this item are inside the
file.

But anyway it is compared to an older emacs where it takes less then a
second to update the table.

Hopes this will help.

(I send the file just to you two.)

With kind regards,
Stefan


-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 09:18:02 GMT) Full text and rfc822 format available.

Message #20 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
Cc: monnier <at> iro.umontreal.ca, 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 11:18:08 +0200
> Date: Sat, 22 Feb 2014 09:39:27 +0100
> From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
> Cc: Eli Zaretskii <eliz <at> gnu.org>
> 
> If you search for #### you will find the item, which shows the problem.
> If you then type C-c C-c on the #+BEGIN line you will get it.
> 
> I observed that it is not as slow as in the original file. Perhaps this
> has to do with to fontifying, because org-mode now has much less to fontify.
> It also makes no difference if the items after this item are inside the
> file.
> 
> But anyway it is compared to an older emacs where it takes less then a
> second to update the table.

OK, I can confirm the slow-down.  The profile points to forward-line,
so I disabled cache-long-scans, and -- lo and behold -- the slowdown
went away.  So this should be the workaround for the time being.

It looks like the overhead of caching newlines, which is supposed to
be small, is somehow very large in this case.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 11:10:02 GMT) Full text and rfc822 format available.

Message #23 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 12:08:58 +0100
Mail von Eli Zaretskii, Sat, 22 Feb 2014 at 11:18:08 +0200:

Hello,

> > But anyway it is compared to an older emacs where it takes less then a
> > second to update the table.
> 
> OK, I can confirm the slow-down.  The profile points to forward-line,
> so I disabled cache-long-scans, and -- lo and behold -- the slowdown
> went away.  So this should be the workaround for the time being.

Sounds good. Is it right that I can set this variable globally only by
installing a function setting it in find-file-hook?

I tested it that way with my original file.
(elp-instrument-package org; elp-result for org-ctrl-c-ctrl-c)

emacs 24.3.50.1    2.083 sec (cache-long-scans nil, set by file-find-hook)
emacs 24.3.1       0.758 sec (cache-long-line-scan nil; as it was)

So it is better, but not so good as in the old emacs.

With kind regards,
Stefan


-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 11:30:02 GMT) Full text and rfc822 format available.

Message #26 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 13:29:48 +0200
> Date: Sat, 22 Feb 2014 12:08:58 +0100
> From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
> Cc: 16830 <at> debbugs.gnu.org
> 
> > OK, I can confirm the slow-down.  The profile points to forward-line,
> > so I disabled cache-long-scans, and -- lo and behold -- the slowdown
> > went away.  So this should be the workaround for the time being.
> 
> Sounds good. Is it right that I can set this variable globally only by
> installing a function setting it in find-file-hook?

Doesn't sound like a good idea to me: that would disable the caching
for much more than this Org file, or even just Org files, right?  I
would suggest to selectively disable caching only for those files that
are actually affected in perceptible way, because the caching in
general does speed up things.

> I tested it that way with my original file.
> (elp-instrument-package org; elp-result for org-ctrl-c-ctrl-c)
> 
> emacs 24.3.50.1    2.083 sec (cache-long-scans nil, set by file-find-hook)
> emacs 24.3.1       0.758 sec (cache-long-line-scan nil; as it was)
> 
> So it is better, but not so good as in the old emacs.

I see no difference in the code that could justify this, and on my
system "C-c C-c" in both versions completes in the same time after
disabling the caching, certainly much faster than 2 sec.  So I don't
see how such a 2.5 factor could be explained, except perhaps by some
additional local customizations on your system (I tested in "emacs -Q").




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 11:37:01 GMT) Full text and rfc822 format available.

Message #29 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: stefan.hahn <at> s-hahn.de
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 13:36:17 +0200
> Date: Sat, 22 Feb 2014 13:29:48 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 16830 <at> debbugs.gnu.org
> 
> > emacs 24.3.50.1    2.083 sec (cache-long-scans nil, set by file-find-hook)
> > emacs 24.3.1       0.758 sec (cache-long-line-scan nil; as it was)
> > 
> > So it is better, but not so good as in the old emacs.
> 
> I see no difference in the code that could justify this, and on my
> system "C-c C-c" in both versions completes in the same time after
> disabling the caching, certainly much faster than 2 sec.  So I don't
> see how such a 2.5 factor could be explained, except perhaps by some
> additional local customizations on your system (I tested in "emacs -Q").

Actually, I _can_ think of one possible reason for this difference:
compiler optimizations.  If the 24.3.50.1 binary was compiled without
optimizations, it can definitely be about 2 times slower than an
optimized build of 24.3.1.  What is the value of system-configuration
and system-configuration-options in both builds?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 12:29:02 GMT) Full text and rfc822 format available.

Message #32 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 13:27:47 +0100
Mail von Eli Zaretskii, Sat, 22 Feb 2014 at 13:29:48 +0200:

Hello,

> Doesn't sound like a good idea to me: that would disable the caching
> for much more than this Org file, or even just Org files, right?  I
> would suggest to selectively disable caching only for those files that

Ok.

> > I tested it that way with my original file.
> > (elp-instrument-package org; elp-result for org-ctrl-c-ctrl-c)
> > 
> > emacs 24.3.50.1    2.083 sec (cache-long-scans nil, set by file-find-hook)
> > emacs 24.3.1       0.758 sec (cache-long-line-scan nil; as it was)
> > 
> > So it is better, but not so good as in the old emacs.
> 
> I see no difference in the code that could justify this, and on my
> system "C-c C-c" in both versions completes in the same time after
> disabling the caching, certainly much faster than 2 sec.  So I don't
> see how such a 2.5 factor could be explained, except perhaps by some
> additional local customizations on your system (I tested in "emacs -Q").

Argh... you're right, here the measurement with "emacs -Q":

(elp-instrument-package org; elp-result for org-ctrl-c-ctrl-c)

my original org-mode file:
emacs 24.3.50.1    2.065 sec (cache-long-scans nil, set local)
emacs 24.3.1       0.722 sec (cache-long-line-scan nil; as it was)

test-neuter.org:
emacs 24.3.50.1    0.925 sec (cache-long-scans nil, set local)
emacs 24.3.1       0.381 sec (cache-long-line-scan nil; as it was)

With kind regards,
Stefan


-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 12:34:02 GMT) Full text and rfc822 format available.

Message #35 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 13:33:14 +0100
Mail von Eli Zaretskii, Sat, 22 Feb 2014 at 13:36:17 +0200:

Hello,

> Actually, I _can_ think of one possible reason for this difference:
> compiler optimizations.  If the 24.3.50.1 binary was compiled without
> optimizations, it can definitely be about 2 times slower than an
> optimized build of 24.3.1.  What is the value of system-configuration
> and system-configuration-options in both builds?

How can I retrieve them?

Both are precompiled version I  downloaded; the 24.3.50.1 from dropbox:
https://www.dropbox.com/sh/3pgcb3iiy8s9irl/c171Xhsd99

I'm not sure where I got the older one.

(I wasn't able to compile it myself on windows; I tried it, but I wasn't
able to find all prerequisites. This is really easier on linux.)

With kind reagrds,
Stefan

-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 12:57:01 GMT) Full text and rfc822 format available.

Message #38 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Juanma Barranquero <lekktu <at> gmail.com>
To: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 13:55:53 +0100
[Message part 1 (text/plain, inline)]
On Feb 22, 2014 1:34 PM, "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de> wrote:

> the 24.3.50.1 from dropbox:
> https://www.dropbox.com/sh/3pgcb3iiy8s9irl/c171Xhsd99

Definitely a non-optimized, checking=yes,glyphs build.

Just out of curiosity, where did you get that Dropbox link from?

   J
[Message part 2 (text/html, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 13:06:01 GMT) Full text and rfc822 format available.

Message #41 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 15:05:26 +0200
> Date: Sat, 22 Feb 2014 13:33:14 +0100
> From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
> Cc: 16830 <at> debbugs.gnu.org
> 
> > Actually, I _can_ think of one possible reason for this difference:
> > compiler optimizations.  If the 24.3.50.1 binary was compiled without
> > optimizations, it can definitely be about 2 times slower than an
> > optimized build of 24.3.1.  What is the value of system-configuration
> > and system-configuration-options in both builds?
> 
> How can I retrieve them?

You need to evaluate them inside Emacs:

  M-: system-configuration RET
  M-: system-configuration-options RET

> Both are precompiled version I  downloaded; the 24.3.50.1 from dropbox:
> https://www.dropbox.com/sh/3pgcb3iiy8s9irl/c171Xhsd99

I'm quite sure the release binary is optimized, while the snapshot one
is not.

> (I wasn't able to compile it myself on windows; I tried it, but I wasn't
> able to find all prerequisites. This is really easier on linux.)

Latest versions compile like on GNU/Linux, you just need to create a
suitable environment first, using the advice in nt/INSTALL.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sat, 22 Feb 2014 15:07:02 GMT) Full text and rfc822 format available.

Message #44 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
To: Juanma Barranquero <lekktu <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sat, 22 Feb 2014 16:06:44 +0100
Mail von Juanma Barranquero, Sat, 22 Feb 2014 at 13:55:53 +0100:

Hello,

> > the 24.3.50.1 from dropbox:
> > https://www.dropbox.com/sh/3pgcb3iiy8s9irl/c171Xhsd99
> 
> Definitely a non-optimized, checking=yes,glyphs build.
> 
> Just out of curiosity, where did you get that Dropbox link from?

From Eli.

With kind regards,
Stefan

-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sun, 23 Feb 2014 09:36:02 GMT) Full text and rfc822 format available.

Message #47 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Fri, 21 Feb 2014 08:43:09 +0100
[Message part 1 (text/plain, inline)]
Mail von Eli Zaretskii, Sat, 22 Feb 2014 at 15:05:26 +0200:

Good morning,

> > (I wasn't able to compile it myself on windows; I tried it, but I wasn't
> > able to find all prerequisites. This is really easier on linux.)
> 
> Latest versions compile like on GNU/Linux, you just need to create a
> suitable environment first, using the advice in nt/INSTALL.

(I'm on commit 02825295f from 2014-02-22.)

I tried to install MinGW/MSYS as described in nt/INSTALL. I started msys.bat
and configured with "./configure --prefix=/d/bin/emacs-mingw" .It configured well
but didn't compile:

Configured for `i686-pc-mingw32'.

  Where should the build process find the source code?    .
  What compiler should emacs be built with?               gcc  -std=gnu99 -g3 -O2 -gdwarf-2
  Should Emacs use the GNU version of malloc?             yes
  Should Emacs use a relocating allocator for buffers?    yes
  Should Emacs use mmap(2) for buffer allocation?         no
  What window system should Emacs use?                    w32
  What toolkit should Emacs use?                          none
  Where do we find X Windows header files?                NONE
  Where do we find X Windows libraries?                   NONE
  Does Emacs use -lXaw3d?                                 no
  Does Emacs use -lXpm?                                   no
  Does Emacs use -ljpeg?                                  no
  Does Emacs use -ltiff?                                  no
  Does Emacs use a gif library?                           no
  Does Emacs use -lpng?                                   no
  Does Emacs use -lrsvg-2?                                yes
  Does Emacs use imagemagick?                             yes
  Does Emacs support sound?                               yes
  Does Emacs use -lgpm?                                   no
  Does Emacs use -ldbus?                                  yes
  Does Emacs use -lgconf?                                 no
  Does Emacs use GSettings?                               no
  Does Emacs use a file notification library?             yes -lgio (gfile)
  Does Emacs use access control lists?                    yes
  Does Emacs use -lselinux?                               no
  Does Emacs use -lgnutls?                                yes
  Does Emacs use -lxml2?                                  yes
  Does Emacs use -lfreetype?                              no
  Does Emacs use -lm17n-flt?                              no
  Does Emacs use -lotf?                                   no
  Does Emacs use -lxft?                                   no
  Does Emacs directly use zlib?                           no
  Does Emacs use toolkit scroll bars?                     yes


$ make
[ -r "src/config.in" ] || ( cd . && autoheader )
cd nt && make all                          \
          CC='gcc  -std=gnu99' CFLAGS='-g3 -O2 -gdwarf-2' CPPFLAGS='-mtune=pentium4   -DUSE_CRT_DLL=1 -I /d/home/hs/src/emacs.git/nt
/inc' \
          LDFLAGS='' MAKE='make'
make[1]: Entering directory `/d/home/hs/src/emacs.git/nt'
windres -I . -O coff -o emacs.res ./emacs.rc
gcc  -std=gnu99 -mtune=pentium4    -I. -I.   -mtune=pentium4   -DUSE_CRT_DLL=1 -I /d/home/hs/src/emacs.git/nt/inc -g3 -O2 -gdwarf-2
./runemacs.c emacs.res -mwindows \
         -o runemacs.exe
./runemacs.c: In function `WinMain':
./runemacs.c:74: warning: implicit declaration of function `alloca'
gcc  -std=gnu99 -mtune=pentium4    -I. -I.   -mtune=pentium4   -DUSE_CRT_DLL=1 -I /d/home/hs/src/emacs.git/nt/inc -g3 -O2 -gdwarf-2
./addpm.c -lole32 -luuid -o addpm.exe
./addpm.c: In function `add_registry':
./addpm.c:134: warning: implicit declaration of function `alloca'
gcc  -std=gnu99 -mtune=pentium4    -I. -I.   -mtune=pentium4   -DUSE_CRT_DLL=1 -I /d/home/hs/src/emacs.git/nt/inc -g3 -O2 -gdwarf-2
./cmdproxy.c -o cmdproxy.exe
./cmdproxy.c: In function `make_absolute':
./cmdproxy.c:288: warning: implicit declaration of function `alloca'
gcc  -std=gnu99 -mtune=pentium4    -I. -I.   -mtune=pentium4   -DUSE_CRT_DLL=1 -I /d/home/hs/src/emacs.git/nt/inc -g3 -O2 -gdwarf-2
./ddeclient.c -o ddeclient.exe
gcc  -std=gnu99 -mtune=pentium4    -I. -I.   -mtune=pentium4   -DUSE_CRT_DLL=1 -I /d/home/hs/src/emacs.git/nt/inc -g3 -O2 -gdwarf-2
./addsection.c -o addsection.exe
In file included from /usr/include/sys/fcntl.h:164,
                 from /usr/include/fcntl.h:14,
                 from ./addsection.c:26:
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:90: error: parse error before "st_ino"
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:96: error: parse error before "st_size"
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:103: error: parse error before '}' token
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:108: error: parse error before "_dev_t"
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:110: error: parse error before "st_mode"
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:110: error: conflicting types for 'st_mode'
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:92: error: previous declaration of 'st_mode' was here
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:112: error: conflicting types for 'st_uid'
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:94: error: previous declaration of 'st_uid' was here
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:113: error: conflicting types for 'st_gid'
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:95: error: previous declaration of 'st_gid' was here
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:114: error: parse error before "st_rdev"
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:114: error: conflicting types for 'st_rdev'
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:97: error: previous declaration of 'st_rdev' was here
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:115: error: conflicting types for 'st_size'
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:96: error: previous declaration of 'st_size' was here
/d/home/hs/src/emacs.git/nt/inc/sys/stat.h:120: error: parse error before '}' token
make[1]: *** [addsection.exe] Error 1
make[1]: Leaving directory `/d/home/hs/src/emacs.git/nt'
make: *** [nt] Error 2

gcc --version gives: gcc (GCC) 3.4.4 (msys special).

I attach the whole configure process data.

With kind regards,
Stefan

-- 
Stefan-W. Hahn                          It is easy to make things.
                                        It is hard to make things simple.
[errors.txt (text/plain, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sun, 23 Feb 2014 18:06:02 GMT) Full text and rfc822 format available.

Message #50 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sun, 23 Feb 2014 20:04:52 +0200
> Date: Fri, 21 Feb 2014 08:43:09 +0100
> From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
> Cc: 16830 <at> debbugs.gnu.org

[This doesn't really belong to this bug report.]

> I tried to install MinGW/MSYS as described in nt/INSTALL. I started msys.bat
> and configured with "./configure --prefix=/d/bin/emacs-mingw" .It configured well

No, it didn't configure well.  One can see the signs of trouble
already here:

> Configured for `i686-pc-mingw32'.
> 
>   Where should the build process find the source code?    .
>   What compiler should emacs be built with?               gcc  -std=gnu99 -g3 -O2 -gdwarf-2
>   Should Emacs use the GNU version of malloc?             yes
>   Should Emacs use a relocating allocator for buffers?    yes
>   Should Emacs use mmap(2) for buffer allocation?         no
>   What window system should Emacs use?                    w32
>   What toolkit should Emacs use?                          none
>   Where do we find X Windows header files?                NONE
>   Where do we find X Windows libraries?                   NONE
>   Does Emacs use -lXaw3d?                                 no
>   Does Emacs use -lXpm?                                   no
>   Does Emacs use -ljpeg?                                  no
>   Does Emacs use -ltiff?                                  no
>   Does Emacs use a gif library?                           no
>   Does Emacs use -lpng?                                   no
>   Does Emacs use -lrsvg-2?                                yes
>   Does Emacs use imagemagick?                             yes
                                                            ^^^
>   Does Emacs support sound?                               yes
>   Does Emacs use -lgpm?                                   no
>   Does Emacs use -ldbus?                                  yes
                                                            ^^^
>   Does Emacs use -lgconf?                                 no
>   Does Emacs use GSettings?                               no
>   Does Emacs use a file notification library?             yes -lgio (gfile)
                                                            ^^^^^^^^^^^^^^^^^
>   Does Emacs use access control lists?                    yes
>   Does Emacs use -lselinux?                               no
>   Does Emacs use -lgnutls?                                yes
>   Does Emacs use -lxml2?                                  yes
>   Does Emacs use -lfreetype?                              no
>   Does Emacs use -lm17n-flt?                              no
>   Does Emacs use -lotf?                                   no
>   Does Emacs use -lxft?                                   no
>   Does Emacs directly use zlib?                           no
>   Does Emacs use toolkit scroll bars?                     yes
> gcc  -std=gnu99 -mtune=pentium4    -I. -I.   -mtune=pentium4   -DUSE_CRT_DLL=1 -I /d/home/hs/src/emacs.git/nt/inc -g3 -O2 -gdwarf-2
> ./addsection.c -o addsection.exe
> In file included from /usr/include/sys/fcntl.h:164,
>                  from /usr/include/fcntl.h:14,
>                  from ./addsection.c:26:
> /d/home/hs/src/emacs.git/nt/inc/sys/stat.h:90: error: parse error before "st_ino"

Looks like you don't have MinGW Windows headers, at least some of
them.  Did you install the w32 API headers?

And here's the evidence that you probably didn't:

> checking windows.h usability... no
> checking windows.h presence... yes
> configure: WARNING: windows.h: present but cannot be compiled
> configure: WARNING: windows.h:     check for missing prerequisite headers?
> configure: WARNING: windows.h: see the Autoconf documentation
> configure: WARNING: windows.h:     section "Present But Cannot Be Compiled"
> configure: WARNING: windows.h: proceeding with the compiler's result
> checking for windows.h... no

Likewise this:

> checking netinet/in.h usability... no
> checking netinet/in.h presence... yes
> configure: WARNING: netinet/in.h: present but cannot be compiled
> configure: WARNING: netinet/in.h:     check for missing prerequisite headers?
> configure: WARNING: netinet/in.h: see the Autoconf documentation
> configure: WARNING: netinet/in.h:     section "Present But Cannot Be Compiled"
> configure: WARNING: netinet/in.h: proceeding with the compiler's result
> checking for netinet/in.h... no

Look in config.log, it should probably tell you right away what is
wrong with your MinGW/MSYS installation.

Also, this:

> checking for gnutls >= 3.0.0... yes CFLAGS='-I/usr/include/p11-kit-1  ' LIBS='-L/usr/bin -lgnutls -lnettle -lhogweed -lgmp -ltasn1 -lp11-kit -lz  '

looks wrong: the file names without drive letters seem to suggest that
you installed the MinGW headers into the same tree as MSYS?  I don't
recommend that.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Mon, 10 Mar 2014 18:59:02 GMT) Full text and rfc822 format available.

Message #53 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Mon, 10 Mar 2014 20:58:22 +0200
> Date: Sat, 22 Feb 2014 13:27:47 +0100
> From: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>
> Cc: 16830 <at> debbugs.gnu.org
> 
> my original org-mode file:
> emacs 24.3.50.1    2.065 sec (cache-long-scans nil, set local)
> emacs 24.3.1       0.722 sec (cache-long-line-scan nil; as it was)
> 
> test-neuter.org:
> emacs 24.3.50.1    0.925 sec (cache-long-scans nil, set local)
> emacs 24.3.1       0.381 sec (cache-long-line-scan nil; as it was)

So I think it's a very Good Thing that we have this bug report,
because looking into this issue produced a surprising (for me)
discovery: turning off the newline cache makes forward-line
significantly (5 to 20 times, depending on the file) faster than when
the cache is turned on, for files with relatively short lines.

It looks like the GCC implementation of memchr, used by the "dumb
loop" in find_newline, is so efficient that it easily outperforms the
"smart loop" which uses the cache, unless lines are very long, like at
least 400 characters, at which point the code performs with and
without the cache at the same speed.

This striking difference in speed goes mostly unnoticed, because
typically Emacs seldom if ever calls forward-line too much (see below
for how much should "too much" for this to become visible).  However,
typing "C-c C-c" in the OP's Org file does just that -- it causes
forward-line be invoked a huge number of times, because it walks the
(29K line) Org file one line at a time with this function:

  (defsubst org-goto-line (N)
    (save-restriction
      (widen)
      (goto-char (point-min))
      (forward-line (1- N))))

IOW, to move from line N to line N+1, this goes back to the beginning,
and then traverses all the N lines again, plus one more line.  Not a
very efficient way, to put it mildly.  So I suggest that Org
developers look into making this use case more efficient, no matter
whether the changes suggested below are or aren't installed.

Inspired by the above function, I profiled find_newline, which is the
workhorse of forward-line, using xdisp.c as the test file and this
silly program:

      (let ((n 1))
       (while (not (eobp))
        (goto-char 1) 
	(forward-line n) 
	(setq n (1+ n))))

Running this program on xdisp.c calls find_newline 30K times and
exercises its inner loop, which looks for the next newline, 450
million times.  On my machine and in an optimized build, this takes
about 1 min 14 sec with the cache turned on, and only 12 sec with it
turned off, a factor of 6.

By careful optimization of find_newline, I have succeeded to slash the
time of the above loop by a factor of 2, see the proposed patch below.
The "C-c C-c" command in the OP's Org file runs 3 times faster with
those changes, and takes only 2 sec instead of 6.5.  This still leaves
the no-cache operation faster by a factor of about 3, though, in both
these test cases.

Again, this is for files whose average line length is 30 to 50
characters.  For files whose lines are at least 10 times longer,
the times with and without cache become almost identical, and for
longer lines the cache starts to win.

It would be nice to be able to turn the cache on and off dynamically,
depending on the actual line length of the buffer.  I tried to
implement this, but my naive implementation didn't work well, because
sampling of the lines tends to be extremely un-representative.  If
someone can come up with a smarter implementation, please show it.

Until we can dynamically estimate the line length and turn the cache
on only for long lines, I suggest to leave the default ON, and install
the patches below.  My reasoning is that in most situations the
slow-down is negligible, while for very long lines the speedup can be
significant.

For the record, here are the measurements I made, before and after the
changes, with 2 test cases: xdisp.c scanning with the above program,
and the "neutered" Org file posted by Stefan-W. Hahn:

   Test     Code   Optimized?    Cache ON    Cache OFF
   ---------------------------------------------------
   Org      old    NO              11.3s      0.8s
   Org      new    NO               3.6s      0.8s
   Org      old    YES              6.5s      0.3s
   Org      new    YES              2.0s      0.25s
   xdisp.c  old    NO            2m11.8s     14.7s
   xdisp.c  new    NO            1m03.3s     14.8s
   xdisp.c  old    YES           1m14.4s     12.0s
   xdisp.c  new    YES             32.5s     11.8s

And here are the patches I propose.  (Note that I only handled the
forward scan; the backward scan is used much less, so I left it alone,
but if someone thinks the asymmetry might be confusing, I can do the
same surgery with backward scan.)

Any objections to committing this?

--- src/search.c.~2~	2014-01-02 07:07:04.000000000 +0200
+++ src/search.c	2014-03-10 19:40:08.607562800 +0200
@@ -715,18 +715,61 @@ find_newline (ptrdiff_t start, ptrdiff_t
            examine.  */
 	ptrdiff_t tem, ceiling_byte = end_byte - 1;
 
-        /* If we're looking for a newline, consult the newline cache
-           to see where we can avoid some scanning.  */
+        /* If we're using the newline cache, consult it to see whether
+           we can avoid some scanning.  */
         if (newline_cache)
           {
             ptrdiff_t next_change;
+	    int result = 1;
+
             immediate_quit = 0;
-            while (region_cache_forward
-                   (cache_buffer, newline_cache, start, &next_change))
-              start = next_change;
-            immediate_quit = allow_quit;
+            while (start < end && result)
+	      {
+		ptrdiff_t lim1;
 
-	    start_byte = CHAR_TO_BYTE (start);
+		result = region_cache_forward (cache_buffer, newline_cache,
+					       start, &next_change);
+		if (result)
+		  {
+		    start = next_change;
+		    lim1 = next_change = end;
+		  }
+		else
+		  lim1 = min (next_change, end);
+
+		/* The cache returned zero for this region; see if
+		   this is because the region is known and includes
+		   only newlines.  While at that, count any newlines
+		   we bump into, and exit if we found enough off them.  */
+		start_byte = CHAR_TO_BYTE (start);
+		while (start < lim1
+		       && FETCH_BYTE (start_byte) == '\n')
+		  {
+		    start_byte++;
+		    start++;
+		    if (--count == 0)
+		      {
+			if (bytepos)
+			  *bytepos = start_byte;
+			return start;
+		      }
+		  }
+		/* If we found a non-newline character before hitting
+		   position where the cache will again return non-zero
+		   (i.e. no newlines beyond that position), it means
+		   this region is not yet known to the cache, and we
+		   must resort to the "dumb loop" method.  */
+		if (start < next_change && !result)
+		  break;
+		result = 1;
+	      }
+	    if (start >= end)
+	      {
+		start = end;
+		start_byte = end_byte;
+		break;
+	      }
+            immediate_quit = allow_quit;
 
             /* START should never be after END.  */
             if (start_byte > ceiling_byte)
@@ -762,9 +805,9 @@ find_newline (ptrdiff_t start, ptrdiff_t
 	      unsigned char *nl = memchr (lim_addr + cursor, '\n', - cursor);
 	      next = nl ? nl - lim_addr : 0;
 
-              /* If we're looking for newlines, cache the fact that
-                 this line's region is free of them. */
-              if (newline_cache)
+              /* If we're using the newline cache, cache the fact that
+                 the region we just traversed is free of newlines. */
+              if (newline_cache && cursor != next)
 		{
 		  know_region_cache (cache_buffer, newline_cache,
 				     BYTE_TO_CHAR (lim_byte + cursor),
@@ -840,7 +883,7 @@ find_newline (ptrdiff_t start, ptrdiff_t
 
               /* If we're looking for newlines, cache the fact that
                  this line's region is free of them. */
-              if (newline_cache)
+              if (newline_cache && cursor != prev + 1)
 		{
 		  know_region_cache (cache_buffer, newline_cache,
 				     BYTE_TO_CHAR (ceiling_byte + prev + 1),




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Tue, 11 Mar 2014 08:10:01 GMT) Full text and rfc822 format available.

Message #56 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: martin rudalics <rudalics <at> gmx.at>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 16830 <at> debbugs.gnu.org, "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Tue, 11 Mar 2014 09:08:55 +0100
> Until we can dynamically estimate the line length and turn the cache
> on only for long lines, I suggest to leave the default ON, and install
> the patches below.  My reasoning is that in most situations the
> slow-down is negligible, while for very long lines the speedup can be
> significant.

In general I inspect long lines only in bug reports.  Is that sufficient
reason to not follow the advice

   There is no reason to set this to nil except for debugging purposes.

after your patch is applied?

> And here are the patches I propose.  (Note that I only handled the
> forward scan; the backward scan is used much less, so I left it alone,
> but if someone thinks the asymmetry might be confusing, I can do the
> same surgery with backward scan.)

Thanks for the work.  If it's not difficult, I think the backward scan
should be fixed too.  And I strongly favor to install this for 24.4.

martin




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Tue, 11 Mar 2014 17:04:02 GMT) Full text and rfc822 format available.

Message #59 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: martin rudalics <rudalics <at> gmx.at>
Cc: 16830 <at> debbugs.gnu.org, stefan.hahn <at> s-hahn.de, monnier <at> iro.umontreal.ca
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Tue, 11 Mar 2014 19:03:19 +0200
> Date: Tue, 11 Mar 2014 09:08:55 +0100
> From: martin rudalics <rudalics <at> gmx.at>
> CC: "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>, 
>  Stefan Monnier <monnier <at> iro.umontreal.ca>,
>  16830 <at> debbugs.gnu.org
> 
>  > Until we can dynamically estimate the line length and turn the cache
>  > on only for long lines, I suggest to leave the default ON, and install
>  > the patches below.  My reasoning is that in most situations the
>  > slow-down is negligible, while for very long lines the speedup can be
>  > significant.
> 
> In general I inspect long lines only in bug reports.  Is that sufficient
> reason to not follow the advice
> 
>     There is no reason to set this to nil except for debugging purposes.
> 
> after your patch is applied?

Actually, I suggest to only change the default if you ever see a
tangible difference with and without the cache.

If you review the timings I posted, you will realize that a single
call to find_newline takes a fraction of a microsecond on a reasonably
modern machine, so unless you use code that calls forward-line with a
very large argument, like hundreds of thousands, you will never see
the difference.

Also, turning off cache-long-scans disables not only the newline
cache, but also 2 other caches, at least one of which (the bidi
paragraph start cache) might be important for redisplay speed, and
doesn't suffer from the slowdown I discovered with the newline cache,
because the way we use that cache is very different.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Wed, 12 Mar 2014 14:13:01 GMT) Full text and rfc822 format available.

Message #62 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: martin rudalics <rudalics <at> gmx.at>
Cc: Eli Zaretskii <eliz <at> gnu.org>, "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>,
 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Wed, 12 Mar 2014 10:12:49 -0400
> Thanks for the work.  If it's not difficult, I think the backward scan
> should be fixed too.  And I strongly favor to install this for 24.4.

What he said,


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Sun, 16 Mar 2014 16:33:03 GMT) Full text and rfc822 format available.

Message #65 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: rudalics <at> gmx.at, stefan.hahn <at> s-hahn.de, 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sun, 16 Mar 2014 18:32:24 +0200
> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>,  16830 <at> debbugs.gnu.org
> Date: Wed, 12 Mar 2014 10:12:49 -0400
> 
> > Thanks for the work.  If it's not difficult, I think the backward scan
> > should be fixed too.  And I strongly favor to install this for 24.4.
> 
> What he said,

Done as trunk revision 116775.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Mon, 17 Mar 2014 15:06:02 GMT) Full text and rfc822 format available.

Message #68 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Antipov <antipov <at> dev.rtsoft.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Mon, 17 Mar 2014 19:05:27 +0400
[Message part 1 (text/plain, inline)]
On 03/10/2014 10:58 PM, Eli Zaretskii wrote:

> It would be nice to be able to turn the cache on and off dynamically,
> depending on the actual line length of the buffer.  I tried to
> implement this, but my naive implementation didn't work well, because
> sampling of the lines tends to be extremely un-representative.  If
> someone can come up with a smarter implementation, please show it.

What if we just maintain the '\n' counter per each buffer text?
With that, finding an average line length is straightforward,
and the very basic implementation looks fairly simple; this
should be helpful in maintaining newline_cache as well.

Dmitry

[nl.patch (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#16830; Package emacs. (Mon, 17 Mar 2014 16:40:03 GMT) Full text and rfc822 format available.

Message #71 received at 16830 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Antipov <antipov <at> dev.rtsoft.ru>
Cc: 16830 <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Mon, 17 Mar 2014 18:39:33 +0200
> Date: Mon, 17 Mar 2014 19:05:27 +0400
> From: Dmitry Antipov <antipov <at> dev.rtsoft.ru>
> CC: 16830 <at> debbugs.gnu.org
> 
> On 03/10/2014 10:58 PM, Eli Zaretskii wrote:
> 
> > It would be nice to be able to turn the cache on and off dynamically,
> > depending on the actual line length of the buffer.  I tried to
> > implement this, but my naive implementation didn't work well, because
> > sampling of the lines tends to be extremely un-representative.  If
> > someone can come up with a smarter implementation, please show it.
> 
> What if we just maintain the '\n' counter per each buffer text?

How would you know how many newlines are there in the buffer?
Counting them is an overhead in itself that we currently avoid (see
line-number-display-limit).

But anyway, feel free to implement something and test it.  As I've
written above, I tried (for admittedly short time), but didn't get
good results.




Reply sent to Eli Zaretskii <eliz <at> gnu.org>:
You have taken responsibility. (Sun, 22 Jun 2014 16:52:01 GMT) Full text and rfc822 format available.

Notification sent to "Stefan-W. Hahn" <stefan.hahn <at> s-hahn.de>:
bug acknowledged by developer. (Sun, 22 Jun 2014 16:52:02 GMT) Full text and rfc822 format available.

Message #76 received at 16830-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: 16830-done <at> debbugs.gnu.org
Subject: Re: bug#16830: [Bug] 24.3.50; massive slow down in forward-line
Date: Sun, 22 Jun 2014 19:50:52 +0300
> Date: Mon, 17 Mar 2014 18:39:33 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 16830 <at> debbugs.gnu.org
> 
> > Date: Mon, 17 Mar 2014 19:05:27 +0400
> > From: Dmitry Antipov <antipov <at> dev.rtsoft.ru>
> > CC: 16830 <at> debbugs.gnu.org
> > 
> > On 03/10/2014 10:58 PM, Eli Zaretskii wrote:
> > 
> > > It would be nice to be able to turn the cache on and off dynamically,
> > > depending on the actual line length of the buffer.  I tried to
> > > implement this, but my naive implementation didn't work well, because
> > > sampling of the lines tends to be extremely un-representative.  If
> > > someone can come up with a smarter implementation, please show it.
> > 
> > What if we just maintain the '\n' counter per each buffer text?
> 
> How would you know how many newlines are there in the buffer?
> Counting them is an overhead in itself that we currently avoid (see
> line-number-display-limit).
> 
> But anyway, feel free to implement something and test it.  As I've
> written above, I tried (for admittedly short time), but didn't get
> good results.

No further comments, so I'm closing this bug, as I see no further
place for improving the performance of the cache.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 21 Jul 2014 11:24:03 GMT) Full text and rfc822 format available.

This bug report was last modified 10 years and 336 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.