GNU bug report logs - #67536
29.1; Calc mode's math-read-preprocess-string conses unnecessarily

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Mattias Engdegård <mattias.engdegard <at> gmail.com>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#67536: closed (29.1; Calc mode's math-read-preprocess-string
 conses unnecessarily)
Date: Tue, 19 Dec 2023 18:16:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Tue, 19 Dec 2023 19:15:38 +0100
with message-id <1AE03DA1-3CE0-4897-A45D-59C5C5151A25 <at> gmail.com>
and subject line Re: bug#67536: 29.1; Calc mode's math-read-preprocess-string conses unnecessarily
has caused the debbugs.gnu.org bug report #67536,
regarding 29.1; Calc mode's math-read-preprocess-string conses unnecessarily
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
67536: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=67536
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Raffael Stocker <r.stocker <at> mnet-mail.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.1; Calc mode's math-read-preprocess-string conses unnecessarily
Date: Wed, 29 Nov 2023 22:29:38 +0100

Org table re-calculation is very slow, partly due to
math-read-preprocess-string of calc mode consing unnecessarily.  For
example, in one (large) table, I get the following memory usage from the
profiler:

...
     60,252,646  96%   - org-ctrl-c-ctrl-c
     60,248,166  96%    - org-table-calc-current-TBLFM
     60,216,431  96%     - funcall-interactively
     60,205,119  96%      - org-table-recalculate
     49,094,651  78%       - org-table-eval-formula
     32,624,631  52%        - calc-eval
     32,624,631  52%         - calc-do-calc-eval
     32,620,487  52%          - calc-do-calc-eval
     32,611,151  52%           - math-read-exprs
     29,388,838  47%            + math-read-preprocess-string
      2,343,257   3%            + math-read-expr-list
...

The reason for the slow-down seems to be that
math-read-preprocess-string conses a lot, keeping the GC busy.  This is
due to heavy use of replace-regexp-in-string in this function:

(defun math-read-preprocess-string (str)
  "Replace some substrings of STR by Calc equivalents."
  (setq str
        (replace-regexp-in-string (concat "[" math-read-superscripts "]+")
                                  "^(\\&)" str))
  (setq str
        (replace-regexp-in-string (concat "[" math-read-subscripts "]+")
                                  "_(\\&)" str))
  (let ((rep-list math-read-replacement-list))
    (while rep-list
      ;; consing like a mad-man here:
      (setq str
            (replace-regexp-in-string (nth 0 (car rep-list))
                                      (nth 1 (car rep-list)) str))
      (setq rep-list (cdr rep-list))))
  str)

I would like to propose using a temp buffer instead of kneading the
string into submission:

(defun math-read-preprocess-string (str)
  "Replace some substrings of STR by Calc equivalents."
  (with-temp-buffer
    (insert str)
    (goto-char 0)
    (while (re-search-forward (concat "[" math-read-superscripts "]+") nil t)
      (replace-match "^(\\&)"))
    (goto-char 0)
    (while (re-search-forward (concat "[" math-read-subscripts "]+") nil t)
      (replace-match "_(\\&)"))
    (goto-char 0)
    (let ((rep-list math-read-replacement-list))
      (while rep-list
        (while (re-search-forward (nth 0 (car rep-list)) nil t)
          (replace-match (nth 1 (car rep-list))))
        (goto-char 0)
        (setq rep-list (cdr rep-list))))
    (buffer-string)))

With this replacement, the profiler shows much less memory usage on the
same org table:

...
     30,411,804  91%   - org-ctrl-c-ctrl-c
     30,407,324  91%    - org-table-calc-current-TBLFM
     30,363,932  91%     - funcall-interactively
     30,331,900  91%      - org-table-recalculate
     20,430,223  61%       - org-table-eval-formula
      6,751,852  20%        + org-table-justify-field-maybe
      4,598,523  13%        - calc-eval
      4,586,091  13%         - calc-do-calc-eval
      4,569,619  13%          - calc-do-calc-eval
      4,547,971  13%           - math-read-exprs
      2,297,453   6%            + math-read-expr-list
      1,347,377   4%            + math-read-preprocess-string
          7,296   0%              math-read-token
...

Only I am not sure the replacement is correct in all possible edge cases
(or whether it uses so much less memory because I have overlooked
something).

What do you think? Can a calc-mode expert weigh in here?

Regards,
Raffael

[Message part 3 (message/rfc822, inline)]

From: Mattias Engdegård <mattias.engdegard <at> gmail.com>
To: Raffael Stocker <r.stocker <at> mnet-mail.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 67536-done <at> debbugs.gnu.org
Subject: Re: bug#67536: 29.1; Calc mode's math-read-preprocess-string conses
 unnecessarily
Date: Tue, 19 Dec 2023 19:15:38 +0100

19 dec. 2023 kl. 18.09 skrev Raffael Stocker <r.stocker <at> mnet-mail.de>:

> Yes, I think this will be fine.  Thanks!

Closing.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.