Subject: bug#40704: 28.0.50; Improve and speed up (Gnus) registry saving

From: Michael Heerdegen <[email-masked]>
Date: Sun, 19 Apr 2020 04:14:00 +0200

Hello,

Saving the Gnus registry is quite slow currently.  I profiled a bit and
for now suggest to do something like in the attached patch.  In detail:

(1) We need to bind inhibit-modification-hooks -> t, this offers a good
speedup (~ 4 or so).

(2) Printing the registry which basically consists of huge hash tables,
causes a lot of garbage.  Most of that garbage seems to be unavoidable
(is it created by the printing primitives?).  Anyway, seems we should
temporarily increase `gc-cons-threshold' drastically, this offers
another speedup of 25% or so.  The patch attached uses the value that
works well for me and the size of my registry, and I bind it in
`gnus-registry-save', because I assume other registries outside of Gnus
can be smaller.  What would be a good value of `gc-cons-threshold', or
should it even scale with `gnus-registry-max-entries' instead of being
constant?

(3) I also decided to change `eieio-override-prin1' to print hash tables
"by hand" from Lisp.  The eieio-persistent requires to modify how
elements in the hash tables are printed, and the current way of doing
this (make a copy of the complete table, change the elements, prin1 and
re-read the result) is not only hackish but also inefficient (it does
this recursively for nested tables).

Any comments on the suggested changes?

TIA,

Michael.

From 618362e3496df18d0f60143b71185ad57501fdcb Mon Sep 17 00:00:00 2001
From: Michael Heerdegen <[email-masked]>
Date: Mon, 13 Apr 2020 23:33:39 +0200
Subject: [PATCH] WIP: Try to improve gnus registry saving

---
 lisp/emacs-lisp/eieio-base.el |  3 ++-
 lisp/emacs-lisp/eieio.el      | 33 +++++++++++++++++++++++++--------
 lisp/gnus/gnus-registry.el    |  1 +
 3 files changed, 28 insertions(+), 9 deletions(-)

diff --git a/lisp/emacs-lisp/eieio-base.el b/lisp/emacs-lisp/eieio-base.el
index 2cb1f614ce..010a2b673e 100644
--- a/lisp/emacs-lisp/eieio-base.el
+++ b/lisp/emacs-lisp/eieio-base.el
@@ -473,7 +473,8 @@ eieio-persistent-save
     (let* ((cfn (or file (oref this file)))
            (default-directory (file-name-directory cfn)))
       (cl-letf ((standard-output (current-buffer))
-                ((oref this file)       ;FIXME: Why change it?
+                (inhibit-modification-hooks t)
+                ((oref this file) ;FIXME: Why change it?
                  (if file
                      ;; FIXME: Makes a name relative to (oref this file),
                      ;; whereas I think it should be relative to cfn.
diff --git a/lisp/emacs-lisp/eieio.el b/lisp/emacs-lisp/eieio.el
index 9f8b639e52..43d515e1fa 100644
--- a/lisp/emacs-lisp/eieio.el
+++ b/lisp/emacs-lisp/eieio.el
@@ -934,15 +934,32 @@ eieio-override-prin1
 	((consp thing)
 	 (eieio-list-prin1 thing))
 	((hash-table-p thing)
-         (let ((copy (copy-hash-table thing)))
-	   (maphash
+         (princ "#s(hash-table size ")
+         (prin1 (hash-table-size thing))
+         (princ " test ")
+         (prin1 (hash-table-test thing))
+         (princ " weakness ")
+         (prin1 (hash-table-weakness thing))
+         (princ " rehash-size ")
+         (prin1 (hash-table-rehash-size thing))
+         (princ " rehash-threshold ")
+         (prin1 (hash-table-rehash-threshold thing))
+         (princ " data (")
+         (let* ((eieio-print-depth (if eieio-print-indentation
+                                       (1+ eieio-print-depth)
+                                     eieio-print-depth))
+                (do-indent (if eieio-print-indentation
+                               (lambda () (princ (make-string (* eieio-print-depth 2) ? )))
+                             #'ignore)))
+           (maphash
 	    (lambda (key val)
-	      (setf (gethash key copy)
-		    (read
-		     (with-output-to-string
-		       (eieio-override-prin1 val)))))
-	    copy)
-	   (prin1 copy)))
+              (princ "\n")
+              (funcall do-indent)
+              (prin1 key)
+              (princ " ")
+              (eieio-override-prin1 val))
+            thing))
+         (princ "))"))
 	((vectorp thing)
          (let ((copy (copy-sequence thing)))
 	  (dotimes (i (length copy))
diff --git a/lisp/gnus/gnus-registry.el b/lisp/gnus/gnus-registry.el
index 480ed80ef8..4ac3c84a80 100644
--- a/lisp/gnus/gnus-registry.el
+++ b/lisp/gnus/gnus-registry.el
@@ -398,6 +398,7 @@ gnus-registry-save
   (interactive)
   (let* ((file (or file gnus-registry-cache-file))
          (db (or db gnus-registry-db))
+         (gc-cons-threshold (max gc-cons-threshold (* 800000 500)))
 	 (clone (clone db)))
     (gnus-message 5 "Saving Gnus registry (%d entries) to %s..."
                   (registry-size db) file)
--
2.25.1