GNU bug report logs - #32899
27.0.50; wdired-do-renames: Speed up for long Emacs sessions

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 32899 in the body.
You can then email your comments to 32899 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tino Calancha <tino.calancha <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.50; wdired-do-renames: Speed up for long Emacs sessions
Date: Tue, 2 Oct 2018 02:22:51 +0900

Severity: wishlist

In a long lived Emacs session with many buffers, we can
get a significant improvement with `wdired-do-renames' if we
drop an unnecessary step, i.e., if `dired-rename-file'
calls `dired-rename-subdir' iif FILE is a directory.

I observe gains as high as a factor 15.

--8<-----------------------------cut here---------------start------------->8---
commit 8e742e6c84bc2f992058a03274e60c294e29ee41
Author: Tino Calancha <tino.calancha <at> gmail.com>
Date:   Tue Oct 2 02:00:17 2018 +0900

    wdired-do-renames: Speed up for long Emacs sessions

    `dired-rename-file' calls unconditionally `dired-rename-subdir'.
    The second function performs performs a loop on all the Emacs
    buffers; this step is only needed if FILE is a directory.

    In a long lived Emacs session, this can make a difference
    when renaming a bunch of files with `wdired'.
    For instance, in my 40 days old Emacs session, with ~ 700 buffers,
    this patch increases the speed to rename 2000 files a factor ~ 15.

    * lisp/dired-aux.el (dired-rename-file): Call `dired-rename-subdir'
    iif FILE is a directory.  Add docstring.
    (dired-rename-subdir, dired-remove-entry)
    (dired-remove-file): Add docstring.

    (dired-remove-entry): Move definition into `dired.el'.

    * lisp/wdired.el (wdired-do-renames): Use a progress-reporter.

    * lisp/dired.el (dired-delete-entry):
    Use `dired-remove-entry'.  Add docstring.

    (dired-buffers-for-dir, dired-fun-in-all-buffers):
    Change comment into docstring.
    (dired-fun-in-all-buffers): Prefer `when' and `push' here.

diff --git a/lisp/dired-aux.el b/lisp/dired-aux.el
index 1f13204b7c..9ec97cf3ce 100644
--- a/lisp/dired-aux.el
+++ b/lisp/dired-aux.el
@@ -1485,17 +1485,13 @@ dired-after-subdir-garbage

 ;;;###autoload
 (defun dired-remove-file (file)
+  "Remove entry FILE on each dired buffer.
+Note this doesn't delete FILE in the file system.
+See `dired-delete-file' in case you wish that."
   (dired-fun-in-all-buffers
    (file-name-directory file) (file-name-nondirectory file)
    #'dired-remove-entry file))

-(defun dired-remove-entry (file)
-  (save-excursion
-    (and (dired-goto-file file)
-	 (let (buffer-read-only)
-	   (delete-region (progn (beginning-of-line) (point))
-			  (line-beginning-position 2))))))
-
 ;;;###autoload
 (defun dired-relist-file (file)
   "Create or update the line for FILE in all Dired buffers it would belong in."
@@ -1600,6 +1596,9 @@ dired-copy-file-recursive

 ;;;###autoload
 (defun dired-rename-file (file newname ok-if-already-exists)
+  "Rename FILE to NEWNAME.
+Signal a `file-already-exists' error if a file NEWNAME already exists
+unless OK-IF-ALREADY-EXISTS is non-nil."
   (dired-handle-overwrite newname)
   (dired-maybe-create-dirs (file-name-directory newname))
   (rename-file file newname ok-if-already-exists) ; error is caught
in -create-files
@@ -1609,9 +1608,12 @@ dired-rename-file
 	 (set-visited-file-name newname nil t)))
   (dired-remove-file file)
   ;; See if it's an inserted subdir, and rename that, too.
-  (dired-rename-subdir file newname))
+  (when (file-directory-p file)
+    (dired-rename-subdir file newname)))

 (defun dired-rename-subdir (from-dir to-dir)
+  "Rename subdir FROM-DIR to TO-DIR.
+This updates the name of all buffers visiting files under FROM-DIR."
   (setq from-dir (file-name-as-directory from-dir)
 	to-dir (file-name-as-directory to-dir))
   (dired-fun-in-all-buffers from-dir nil
diff --git a/lisp/dired.el b/lisp/dired.el
index 5c7bb9599c..52399d2623 100644
--- a/lisp/dired.el
+++ b/lisp/dired.el
@@ -2607,12 +2607,12 @@ dired-copy-filename-as-kill
 ;; Keeping Dired buffers in sync with the filesystem and with each other

 (defun dired-buffers-for-dir (dir &optional file)
-;; Return a list of buffers for DIR (top level or in-situ subdir).
-;; If FILE is non-nil, include only those whose wildcard pattern (if any)
-;; matches FILE.
-;; The list is in reverse order of buffer creation, most recent last.
-;; As a side effect, killed dired buffers for DIR are removed from
-;; dired-buffers.
+  "Return a list of buffers for DIR (top level or in-situ subdir).
+If FILE is non-nil, include only those whose wildcard pattern (if any)
+matches FILE.
+The list is in reverse order of buffer creation, most recent last.
+As a side effect, killed dired buffers for DIR are removed from
+dired-buffers."
   (setq dir (file-name-as-directory dir))
   (let (result buf)
     (dolist (elt dired-buffers)
@@ -3149,26 +3149,36 @@ dired-internal-do-deletions
   (dired-move-to-filename))

 (defun dired-fun-in-all-buffers (directory file fun &rest args)
-  ;; In all buffers dired'ing DIRECTORY, run FUN with ARGS.
-  ;; If the buffer has a wildcard pattern, check that it matches FILE.
-  ;; (FILE does not include a directory component.)
-  ;; FILE may be nil, in which case ignore it.
-  ;; Return list of buffers where FUN succeeded (i.e., returned non-nil).
+  "In all buffers dired'ing DIRECTORY, run FUN with ARGS.
+If the buffer has a wildcard pattern, check that it matches FILE.
+\(FILE does not include a directory component).
+FILE may be nil, in which case ignore it.
+Return list of buffers where FUN succeeded (i.e., returned non-nil)."
   (let (success-list)
     (dolist (buf (dired-buffers-for-dir (expand-file-name directory)
 					file))
       (with-current-buffer buf
-	(if (apply fun args)
-	    (setq success-list (cons (buffer-name buf) success-list)))))
+	(when (apply fun args)
+	  (push (buffer-name buf) success-list))))
     success-list))

 ;; Delete the entry for FILE from
-(defun dired-delete-entry (file)
+(defun dired-remove-entry (file)
+  "Remove entry FILE in the current dired buffer.
+Note this doesn't delete FILE in the file system.
+See `dired-delete-file' in case you wish that."
   (save-excursion
     (and (dired-goto-file file)
-	 (let ((inhibit-read-only t))
+	 (let (buffer-read-only)
 	   (delete-region (progn (beginning-of-line) (point))
-			  (save-excursion (forward-line 1) (point))))))
+			  (line-beginning-position 2))))))
+
+(defun dired-delete-entry (file)
+  "Remove entry FILE in the current dired buffer.
+Like `dired-remove-entry' followed by `dired-clean-up-after-deletion'.
+Note this doesn't delete FILE in the file system.
+See `dired-delete-file' in case you wish that."
+  (dired-remove-entry file)
   (dired-clean-up-after-deletion file))

 (defvar dired-clean-up-buffers-too)
diff --git a/lisp/wdired.el b/lisp/wdired.el
index 3157e887d7..8852806dd6 100644
--- a/lisp/wdired.el
+++ b/lisp/wdired.el
@@ -459,10 +459,12 @@ wdired-finish-edit

 (defun wdired-do-renames (renames)
   "Perform RENAMES in parallel."
-  (let ((residue ())
-        (progress nil)
-        (errors 0)
-        (overwrite (or (not wdired-confirm-overwrite) 1)))
+  (let* ((residue ())
+         (progress nil)
+         (errors 0)
+         (total (1- (length renames)))
+         (prep (make-progress-reporter "Renaming" 0 total))
+         (overwrite (or (not wdired-confirm-overwrite) 1)))
     (while (or renames
                ;; We've done one round through the renames, we have found
                ;; some residue, but we also made some progress, so maybe
@@ -470,6 +472,7 @@ wdired-do-renames
                (prog1 (setq renames residue)
                  (setq progress nil)
                  (setq residue nil)))
+      (progress-reporter-update prep (- total (length renames)))
       (let* ((rename (pop renames))
              (file-new (cdr rename)))
         (cond
@@ -517,6 +520,7 @@ wdired-do-renames
                  (dired-log "Rename `%s' to `%s' failed:\n%s\n"
                             file-ori file-new
                             err)))))))))
+    (progress-reporter-done prep)
     errors))

 (defun wdired-create-parentdirs (file-new)

--8<-----------------------------cut here---------------end--------------->8---
In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.11)
Repository revision: 6217746dd64b43a2a2b3b66ab50cfbbfc984f36c

Message #8 received at 32899 <at> debbugs.gnu.org (full text, mbox):

From: Brett Gilio <brettg <at> posteo.net>
To: Tino Calancha <tino.calancha <at> gmail.com>
Cc: 32899 <at> debbugs.gnu.org
Subject: Re: bug#32899: 27.0.50;
 wdired-do-renames: Speed up for long Emacs sessions
Date: Mon, 01 Oct 2018 13:05:46 -0500

Tino Calancha writes:

> Severity: wishlist
>
> In a long lived Emacs session with many buffers, we can
> get a significant improvement with `wdired-do-renames' if we
> drop an unnecessary step, i.e., if `dired-rename-file'
> calls `dired-rename-subdir' iif FILE is a directory.
>
> I observe gains as high as a factor 15.

I might be interested in looking into this further. Though, I 
wonder if
anybody else has any concerns as to why `dired-rename-file' was
implemented here as a step in the first place.


-- 
Brett M. Gilio
Free Software Foundation, Member
https://gnu.org/software/guix | https://emacs.org

Message #11 received at 32899 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Tino Calancha <tino.calancha <at> gmail.com>
Cc: 32899 <at> debbugs.gnu.org
Subject: Re: bug#32899: 27.0.50; wdired-do-renames: Speed up for long Emacs
 sessions
Date: Fri, 04 Sep 2020 06:10:47 +0200

Tino Calancha <tino.calancha <at> gmail.com> writes:

> In a long lived Emacs session with many buffers, we can
> get a significant improvement with `wdired-do-renames' if we
> drop an unnecessary step, i.e., if `dired-rename-file'
> calls `dired-rename-subdir' iif FILE is a directory.
>
> I observe gains as high as a factor 15.

This was apparently never applied at the time, and it no longer applied
cleanly, so I respun the patch and did some testing.  Everything seems
to work for me, and I think the patch makes sense, so I've applied it to
Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.