GNU bug report logs - #53758
28.0.91; Recursive edit during dired-do-find-regexp-and-replace breaks isearch

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 53758 in the body.
You can then email your comments to 53758 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: sbaugh <at> catern.com
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.91; Recursive edit during dired-do-find-regexp-and-replace
 breaks isearch
Date: Thu, 03 Feb 2022 19:11:11 +0000 (UTC)

isearch-forward (C-s) while in a recursive edit (C-r) triggered from
dired-do-find-regexp-and-replace (Q in a dired buffer) always fails to
find any matches for any string, even if there are matches in the
buffer.

Steps to reproduce:
1. Open a dired buffer in a directory containing at least one file which
contains some text (e.g. "Hello world")
2. Run dired-do-find-regexp-and-replace to replace "Hello" with
"Goodbye" ("Q Hello RET Goodbye RET"); this will switch buffers
to the first matching file.   
3. Type C-r to enter a recursive edit (I'm guessing this runs
(recursive-edit)?)
4. At the start of the buffer, run isearch-forward searching for "world"
("C-s world RET")
5. Note that the isearch fails despite "world" being in the buffer.

For what it's worth, this interestingly doesn't happen with
project-query-regexp-replace (which also does multi-file query-replace).


In GNU Emacs 28.0.91 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, cairo version 1.16.0)
Repository revision: 525dc6e5c428185b62c72d7958cd4fe17937f126
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: NixOS 21.05 (Okapi)

Configured using:
 'configure
 --prefix=/nix/store/rkpz8vsk15zlphra3hhngf96irwci7r3-emacs-git-20220115.0
 --disable-build-details --with-modules --with-x-toolkit=gtk3 --with-xft
 --with-cairo'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE
XIM XPM GTK3 ZLIB

Important settings:
  value of $EMACSLOADPATH: 
  value of $EMACSNATIVELOADPATH: /nix/store/fs7slsl0rz28h6dq8rnhgk4ddkk8dh0w-emacs-packages-deps/share/emacs/native-lisp:/nix/store/b854vn0hskfs7d2j4pr1vp71fhf4ynvc-emacs-packages-deps/share/emacs/native-lisp:::
  value of $LANG: C.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Dired by name

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/sbaugh/.nix-profile/share/emacs/site-lisp/site-start hides /nix/store/b854vn0hskfs7d2j4pr1vp71fhf4ynvc-emacs-packages-deps/share/emacs/site-lisp/site-start
/home/sbaugh/.nix-profile/share/emacs/site-lisp/site-start hides /nix/store/rkpz8vsk15zlphra3hhngf96irwci7r3-emacs-git-20220115.0/share/emacs/site-lisp/site-start
/nix/store/b854vn0hskfs7d2j4pr1vp71fhf4ynvc-emacs-packages-deps/share/emacs/site-lisp/elpa/transient-20220112.1305/transient hides /nix/store/rkpz8vsk15zlphra3hhngf96irwci7r3-emacs-git-20220115.0/share/emacs/28.0.91/lisp/transient
/nix/store/b854vn0hskfs7d2j4pr1vp71fhf4ynvc-emacs-packages-deps/share/emacs/site-lisp/elpa/let-alist-1.0.6/let-alist hides /nix/store/rkpz8vsk15zlphra3hhngf96irwci7r3-emacs-git-20220115.0/share/emacs/28.0.91/lisp/emacs-lisp/let-alist

Features:
(shadow sort mail-extr eieio-opt cl-extra speedbar ezimage dframe
find-func shortdoc help-fns radix-tree help-mode emacsbug message rmc
puny rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util
rmail rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
misearch multi-isearch bug-reference pulse color vc-mtn vc-hg vc-git
diff-mode easy-mmode vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs vc
vc-dispatcher find-dired grep compile text-property-search comint
ansi-color xref project ring thingatpt dired-aux cl-loaddefs cl-lib
dired dired-loaddefs seq byte-opt gv bytecomp byte-compile cconv
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 588633 721804)
 (symbols 48 9814 41)
 (strings 32 58972 20078)
 (string-bytes 1 2344778)
 (vectors 16 49128)
 (vector-slots 8 391332 666289)
 (floats 8 169 491)
 (intervals 56 104017 11468)
 (buffers 992 22))

Message #10 received at 53758 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: sbaugh <at> catern.com, 53758 <at> debbugs.gnu.org
Subject: Re: bug#53758: 28.0.91; Recursive edit during
 dired-do-find-regexp-and-replace breaks isearch
Date: Sun, 06 Feb 2022 19:44:34 +0200

> isearch-forward (C-s) while in a recursive edit (C-r) triggered from
> dired-do-find-regexp-and-replace (Q in a dired buffer) always fails to
> find any matches for any string, even if there are matches in the
> buffer.
>
> Steps to reproduce:
> 1. Open a dired buffer in a directory containing at least one file which
> contains some text (e.g. "Hello world")
> 2. Run dired-do-find-regexp-and-replace to replace "Hello" with
> "Goodbye" ("Q Hello RET Goodbye RET"); this will switch buffers
> to the first matching file.
> 3. Type C-r to enter a recursive edit (I'm guessing this runs
> (recursive-edit)?)
> 4. At the start of the buffer, run isearch-forward searching for "world"
> ("C-s world RET")
> 5. Note that the isearch fails despite "world" being in the buffer.
>
> For what it's worth, this interestingly doesn't happen with
> project-query-regexp-replace (which also does multi-file query-replace).

This is because of these lines in xref--query-replace-1:

         ;; Counteract the "do the next match now" hack in
         ;; `perform-replace'.  And still, it'll report that those
         ;; matches were "filtered out" at the end.
         (isearch-filter-predicate
          (lambda (beg end)
            (and current-beg
                 (>= beg current-beg)
                 (<= end current-end))))

Dmitry, could you please explain the comment above.
What I don't understand is where this
"the next match now" hack is in `perform-replace'?

Message #13 received at 53758 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Juri Linkov <juri <at> linkov.net>
Cc: sbaugh <at> catern.com, 53758 <at> debbugs.gnu.org
Subject: Re: bug#53758: 28.0.91; Recursive edit during
 dired-do-find-regexp-and-replace breaks isearch
Date: Mon, 7 Feb 2022 05:03:38 +0200

Hi Juri,

On 06.02.2022 19:44, Juri Linkov wrote:
>> isearch-forward (C-s) while in a recursive edit (C-r) triggered from
>> dired-do-find-regexp-and-replace (Q in a dired buffer) always fails to
>> find any matches for any string, even if there are matches in the
>> buffer.
>>
>> Steps to reproduce:
>> 1. Open a dired buffer in a directory containing at least one file which
>> contains some text (e.g. "Hello world")
>> 2. Run dired-do-find-regexp-and-replace to replace "Hello" with
>> "Goodbye" ("Q Hello RET Goodbye RET"); this will switch buffers
>> to the first matching file.
>> 3. Type C-r to enter a recursive edit (I'm guessing this runs
>> (recursive-edit)?)
>> 4. At the start of the buffer, run isearch-forward searching for "world"
>> ("C-s world RET")
>> 5. Note that the isearch fails despite "world" being in the buffer.
>>
>> For what it's worth, this interestingly doesn't happen with
>> project-query-regexp-replace (which also does multi-file query-replace).
> This is because of these lines in xref--query-replace-1:
> 
>           ;; Counteract the "do the next match now" hack in
>           ;; `perform-replace'.  And still, it'll report that those
>           ;; matches were "filtered out" at the end.
>           (isearch-filter-predicate
>            (lambda (beg end)
>              (and current-beg
>                   (>= beg current-beg)
>                   (<= end current-end))))
> 
> Dmitry, could you please explain the comment above.
> What I don't understand is where this
> "the next match now" hack is in `perform-replace'?

It's referring to the comment on lines starting with 2938 and the 
subsequent code which uses 'looking-at' instead of 
replace-re-search-function.

Here's that comment in full:

	  ;; Otherwise, if matching a regular expression, do the next
	  ;; match now, since the replacement for this match may
	  ;; affect whether the next match is adjacent to this one.
	  ;; If that match is empty, don't use it.

I'm not sure why the result would be different between 
dired-do-find-regexp-and-replace and project-query-regexp-replace, though.

Message #16 received at 53758 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: sbaugh <at> catern.com, 53758 <at> debbugs.gnu.org
Subject: Re: bug#53758: 28.0.91; Recursive edit during
 dired-do-find-regexp-and-replace breaks isearch
Date: Mon, 07 Feb 2022 21:27:07 +0200

[Message part 1 (text/plain, inline)]

> It's referring to the comment on lines starting with 2938 and the
> subsequent code which uses 'looking-at' instead of
> replace-re-search-function.
>
> Here's that comment in full:
>
> 	  ;; Otherwise, if matching a regular expression, do the next
> 	  ;; match now, since the replacement for this match may
> 	  ;; affect whether the next match is adjacent to this one.
> 	  ;; If that match is empty, don't use it.

Thanks, now everything is clear.  Then it can be simplified by this patch:

[xref--query-replace-1.patch (text/x-diff, inline)]

diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index 4efa652084..f334710a28 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -843,25 +843,7 @@ xref--query-replace-1
          (continue t)
          did-it-once buf-pairs pairs
          current-beg current-end
-         ;; Counteract the "do the next match now" hack in
-         ;; `perform-replace'.  And still, it'll report that those
-         ;; matches were "filtered out" at the end.
-         (isearch-filter-predicate
-          (lambda (beg end)
-            (and current-beg
-                 (>= beg current-beg)
-                 (<= end current-end))))
-         (replace-re-search-function
-          (lambda (from &optional _bound noerror)
-            (let (found pair)
-              (while (and (not found) pairs)
-                (setq pair (pop pairs)
-                      current-beg (car pair)
-                      current-end (cdr pair))
-                (goto-char current-beg)
-                (when (re-search-forward from current-end noerror)
-                  (setq found t)))
-              found))))
+         (region-extract-function (lambda (_) pairs)))
     (while (and continue (setq buf-pairs (funcall iter :next)))
       (if did-it-once
           ;; Reuse the same window for subsequent buffers.
@@ -870,8 +852,9 @@ xref--query-replace-1
          (pop-to-buffer (car buf-pairs)))
         (setq did-it-once t))
       (setq pairs (cdr buf-pairs))
+      (goto-char (point-min))
       (setq continue
-            (perform-replace from to t t nil nil multi-query-replace-map)))
+            (perform-replace from to t t nil nil multi-query-replace-map nil nil nil t)))
     (unless did-it-once (user-error "No suitable matches here"))
     (when (and continue (not buf-pairs))
       (message "All results processed"))))

[Message part 3 (text/plain, inline)]

> I'm not sure why the result would be different between
> dired-do-find-regexp-and-replace and project-query-regexp-replace, though.

project-query-regexp-replace doesn't use xref--query-replace-1.

Actually, the reported problem is not specific to xref.
Performing replacements on a rectangular region doesn't allow
searching outside the region on recursive edit too.
So here is a general fix that covers all cases:

[isearch-filter-predicate.patch (text/x-diff, inline)]

diff --git a/lisp/replace.el b/lisp/replace.el
index 23e6483809..5add576d6b 100644
--- a/lisp/replace.el
+++ b/lisp/replace.el
@@ -3233,7 +3233,10 @@ perform-replace
 			       (last-command 'recenter-top-bottom))
 			   (recenter-top-bottom)))
 			((eq def 'edit)
-			 (let ((opos (point-marker)))
+			 (let ((opos (point-marker))
+			       (isearch-filter-predicate isearch-filter-predicate))
+			   (when region-filter
+			     (remove-function isearch-filter-predicate region-filter))
 			   (setq real-match-data (replace-match-data
 						  nil real-match-data
 						  real-match-data))

Message #19 received at 53758 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Juri Linkov <juri <at> linkov.net>
Cc: sbaugh <at> catern.com, 53758 <at> debbugs.gnu.org
Subject: Re: bug#53758: 28.0.91; Recursive edit during
 dired-do-find-regexp-and-replace breaks isearch
Date: Mon, 7 Feb 2022 23:12:55 +0200

On 07.02.2022 21:27, Juri Linkov wrote:
> Thanks, now everything is clear.  Then it can be simplified by this patch:

Thanks!

Are these changes independent? That is, will the new version of xref 
continue to work with Emacs 26.1?

Message #22 received at 53758 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: sbaugh <at> catern.com, 53758 <at> debbugs.gnu.org
Subject: Re: bug#53758: 28.0.91; Recursive edit during
 dired-do-find-regexp-and-replace breaks isearch
Date: Tue, 08 Feb 2022 09:43:39 +0200

>> Thanks, now everything is clear.  Then it can be simplified by this patch:
>
> Thanks!
>
> Are these changes independent? That is, will the new version of xref
> continue to work with Emacs 26.1?

It should continue to work.  Could you suggest how this
independent change in xref could be tested?  I see that
dired-do-find-regexp-and-replace replaces all matches,
but xref--query-replace-1 tries to skip some matches.
What test could emulate the case where some matches
should be skipped?

Message #25 received at 53758 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Juri Linkov <juri <at> linkov.net>
Cc: sbaugh <at> catern.com, 53758 <at> debbugs.gnu.org
Subject: Re: bug#53758: 28.0.91; Recursive edit during
 dired-do-find-regexp-and-replace breaks isearch
Date: Tue, 8 Feb 2022 18:12:01 +0200

On 08.02.2022 09:43, Juri Linkov wrote:
> I see that
> dired-do-find-regexp-and-replace replaces all matches,
> but xref--query-replace-1 tries to skip some matches.
> What test could emulate the case where some matches
> should be skipped?

Perhaps try xref-find-references-and-replace?

Search for some symbol which can also be included as part of the other 
symbols' name.

E.g. search for 'dired-do-find-regexp' and verify that 
'dired-do-find-regexp-and-replace' is not affected.

Message #28 received at 53758 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: sbaugh <at> catern.com, 53758 <at> debbugs.gnu.org
Subject: Re: bug#53758: 28.0.91; Recursive edit during
 dired-do-find-regexp-and-replace breaks isearch
Date: Tue, 08 Feb 2022 21:35:43 +0200

close 53758 29.0.50
thanks

>> I see that
>> dired-do-find-regexp-and-replace replaces all matches,
>> but xref--query-replace-1 tries to skip some matches.
>> What test could emulate the case where some matches
>> should be skipped?
>
> Perhaps try xref-find-references-and-replace?
>
> Search for some symbol which can also be included as part of the other
> symbols' name.
>
> E.g. search for 'dired-do-find-regexp' and verify that
> 'dired-do-find-regexp-and-replace' is not affected.

I see that these custom functions are needed in xref
to search/replace the specific regexp ".*".
Then this is the same problem like in bug#14013.

So I fixed perform-replace to allow using isearch in a recursive edit,
and closed this report.  Other problems could be discussed in bug#14013.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.