From unknown Sun Jun 15 07:56:16 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#54733 <54733@debbugs.gnu.org> To: bug#54733 <54733@debbugs.gnu.org> Subject: Status: Match again in perform-replace Reply-To: bug#54733 <54733@debbugs.gnu.org> Date: Sun, 15 Jun 2025 14:56:16 +0000 retitle 54733 Match again in perform-replace reassign 54733 emacs submitter 54733 Juri Linkov severity 54733 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 05 13:40:26 2022 Received: (at submit) by debbugs.gnu.org; 5 Apr 2022 17:40:26 +0000 Received: from localhost ([127.0.0.1]:56062 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nbnAI-00026g-9p for submit@debbugs.gnu.org; Tue, 05 Apr 2022 13:40:26 -0400 Received: from lists.gnu.org ([209.51.188.17]:51866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nbnAG-00026V-1G for submit@debbugs.gnu.org; Tue, 05 Apr 2022 13:40:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55106) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nbnAF-0001R0-7h for bug-gnu-emacs@gnu.org; Tue, 05 Apr 2022 13:40:23 -0400 Received: from relay7-d.mail.gandi.net ([217.70.183.200]:60379) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nbnAD-0005Q6-0y for bug-gnu-emacs@gnu.org; Tue, 05 Apr 2022 13:40:22 -0400 Received: (Authenticated sender: juri@linkov.net) by mail.gandi.net (Postfix) with ESMTPSA id C3C7B20008 for ; Tue, 5 Apr 2022 17:40:13 +0000 (UTC) From: Juri Linkov To: bug-gnu-emacs@gnu.org Subject: Match again in perform-replace Organization: LINKOV.NET Date: Tue, 05 Apr 2022 20:16:19 +0300 Message-ID: <865ynnl9f0.fsf_-_@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=217.70.183.200; envelope-from=juri@linkov.net; helo=relay7-d.mail.gandi.net X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) [This is a spin-off from bug#14013 and bug#53758] There is a long-standing bug in query-replace. The optimization that uses `match-again` and `looking-at` ignores search-related variables such as isearch-search-fun-function, replace-re-search-function, etc. Here's is a test case that demonstrates the problem in current master with the recent addition of dired-isearch-search-filenames: 1. cd /tmp; touch file1; ln -s file1 file2 2. enter Wdired and move point to the beginning of file2 3. C-M-% .* RET foo RET 4. answer `n` when asked to replace `file2` After that the remaining part of the same line is highlighted, i.e. the part after "file2" (that is a symbolic link) in: file2 -> file1 This is because the `match-again` optimization uses `(looking-at ".*")` after the previous replacement "file2" to ask about the next replacement of " -> file1" that ignores isearch-search-fun-function. Also in bug#53758 Dmitry explained that xref--query-replace-1 needed such a hack to let-bind isearch-filter-predicate because of this problem in perform-replace that uses `looking-at` instead of `replace-re-search-function`. So now we have two cases that require fixing perform-replace. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 05 13:54:02 2022 Received: (at 54733) by debbugs.gnu.org; 5 Apr 2022 17:54:02 +0000 Received: from localhost ([127.0.0.1]:56079 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nbnNS-0002Wf-Gm for submit@debbugs.gnu.org; Tue, 05 Apr 2022 13:54:02 -0400 Received: from relay6-d.mail.gandi.net ([217.70.183.198]:46423) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nbnNP-0002WG-Rp for 54733@debbugs.gnu.org; Tue, 05 Apr 2022 13:54:00 -0400 Received: (Authenticated sender: juri@linkov.net) by mail.gandi.net (Postfix) with ESMTPSA id DB526C0004 for <54733@debbugs.gnu.org>; Tue, 5 Apr 2022 17:53:52 +0000 (UTC) From: Juri Linkov To: 54733@debbugs.gnu.org Subject: Re: bug#54733: Match again in perform-replace Organization: LINKOV.NET References: <865ynnl9f0.fsf_-_@mail.linkov.net> Date: Tue, 05 Apr 2022 20:53:15 +0300 In-Reply-To: <865ynnl9f0.fsf_-_@mail.linkov.net> (Juri Linkov's message of "Tue, 05 Apr 2022 20:16:19 +0300") Message-ID: <86pmlvju2c.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 54733 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.5 (-) > So now we have two cases that require fixing perform-replace. To be able to redesign the match-again part of perform-replace, there is a need to have a test suite that will confirm nothing is broken after redesign. So I pushed a new test in replace-tests.el. From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 29 13:42:32 2022 Received: (at 54733) by debbugs.gnu.org; 29 Apr 2022 17:42:32 +0000 Received: from localhost ([127.0.0.1]:55299 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nkUdU-00012s-Ii for submit@debbugs.gnu.org; Fri, 29 Apr 2022 13:42:32 -0400 Received: from relay6-d.mail.gandi.net ([217.70.183.198]:60987) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nkUdT-00012O-Ep for 54733@debbugs.gnu.org; Fri, 29 Apr 2022 13:42:31 -0400 Received: (Authenticated sender: juri@linkov.net) by mail.gandi.net (Postfix) with ESMTPSA id 77B70C0004 for <54733@debbugs.gnu.org>; Fri, 29 Apr 2022 17:42:24 +0000 (UTC) From: Juri Linkov To: 54733@debbugs.gnu.org Subject: Re: bug#54733: Match again in perform-replace Organization: LINKOV.NET References: <865ynnl9f0.fsf_-_@mail.linkov.net> <86pmlvju2c.fsf@mail.linkov.net> Date: Fri, 29 Apr 2022 20:41:42 +0300 In-Reply-To: <86pmlvju2c.fsf@mail.linkov.net> (Juri Linkov's message of "Tue, 05 Apr 2022 20:53:15 +0300") Message-ID: <86o80ju6qx.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 54733 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.5 (-) >> So now we have two cases that require fixing perform-replace. > > To be able to redesign the match-again part of perform-replace, > there is a need to have a test suite that will confirm nothing > is broken after redesign. So I pushed a new test in replace-tests.el. The need to have `looking-at` in `perform-replace` is explained in the commit message of 5632eb272c7. So now added it to replace-tests. From debbugs-submit-bounces@debbugs.gnu.org Tue May 03 03:11:11 2022 Received: (at 54733) by debbugs.gnu.org; 3 May 2022 07:11:11 +0000 Received: from localhost ([127.0.0.1]:38565 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nlmgg-0006Rs-Lo for submit@debbugs.gnu.org; Tue, 03 May 2022 03:11:10 -0400 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:32909) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nlmga-0006RI-MD for 54733@debbugs.gnu.org; Tue, 03 May 2022 03:11:09 -0400 Received: (Authenticated sender: juri@linkov.net) by mail.gandi.net (Postfix) with ESMTPSA id 291761BF20F for <54733@debbugs.gnu.org>; Tue, 3 May 2022 07:10:57 +0000 (UTC) From: Juri Linkov To: 54733@debbugs.gnu.org Subject: Re: bug#54733: Match again in perform-replace Organization: LINKOV.NET References: <865ynnl9f0.fsf_-_@mail.linkov.net> <86pmlvju2c.fsf@mail.linkov.net> <86o80ju6qx.fsf@mail.linkov.net> Date: Tue, 03 May 2022 10:10:10 +0300 In-Reply-To: <86o80ju6qx.fsf@mail.linkov.net> (Juri Linkov's message of "Fri, 29 Apr 2022 20:41:42 +0300") Message-ID: <861qxb6qvh.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 54733 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.5 (-) --=-=-= Content-Type: text/plain >>> So now we have two cases that require fixing perform-replace. >> >> To be able to redesign the match-again part of perform-replace, >> there is a need to have a test suite that will confirm nothing >> is broken after redesign. So I pushed a new test in replace-tests.el. > > The need to have `looking-at` in `perform-replace` is explained > in the commit message of 5632eb272c7. So now added it to replace-tests. The only way to fix all reported problems is to always use search functions in perform-replace: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=perform-replace-match-again.patch diff --git a/lisp/replace.el b/lisp/replace.el index 81282deb14..7fbaa93ead 100644 --- a/lisp/replace.el +++ b/lisp/replace.el @@ -3013,9 +3013,10 @@ perform-replace (setq match-again (and nonempty-match (or (not regexp-flag) - (and (if backward - (looking-back search-string nil) - (looking-at search-string)) + (and (save-excursion + (replace-search search-string limit + regexp-flag delimited-flag + case-fold-search backward)) (let ((match (match-data))) (and (/= (nth 0 match) (nth 1 match)) match)))))) @@ -3298,8 +3299,12 @@ perform-replace ;; decide whether the search string ;; can match again just after this match. (if (and regexp-flag nonempty-match) - (setq match-again (and (looking-at search-string) - (match-data))))) + (setq match-again + (and (save-window-excursion + (replace-search search-string limit + regexp-flag delimited-flag + case-fold-search backward)) + (match-data))))) ;; Edit replacement. ((eq def 'edit-replacement) (setq real-match-data (replace-match-data diff --git a/test/lisp/replace-tests.el b/test/lisp/replace-tests.el index ef1e5c3eaf..f7a2e043ff 100644 --- a/test/lisp/replace-tests.el +++ b/test/lisp/replace-tests.el @@ -472,8 +472,7 @@ query-replace-search-function-tests found))) (tests '( - ;; FIXME: this test should pass after fixing bug#54733: - ;; ("aaaa" "C-M-% .* RET 1 RET !" "1a1a") + ("aaaa" "C-M-% .* RET 1 RET !" "1a1a") ))) (query-replace--run-tests tests))) @@ -485,8 +484,7 @@ perform-replace-tests ;; Test case from commit 5632eb272c7 ("a a a " "\\ba " "c" nil t nil nil nil nil nil nil nil "ccc") ; not "ca c" ;; The same with region inside the second match - ;; FIXME: this test should pass after fixing bug#54733: - ;; ("a a a " "\\ba " "c" nil t nil nil nil 1 4 nil nil "ca a ") + ("a a a " "\\ba " "c" nil t nil nil nil 1 4 nil nil "ca a ") )) (defun perform-replace--run-tests (tests) diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el index 6e763eef01..981f51c30a 100644 --- a/lisp/progmodes/xref.el +++ b/lisp/progmodes/xref.el @@ -856,14 +856,6 @@ xref--query-replace-1 (continue t) did-it-once buf-pairs pairs current-beg current-end - ;; Counteract the "do the next match now" hack in - ;; `perform-replace'. And still, it'll report that those - ;; matches were "filtered out" at the end. - (isearch-filter-predicate - (lambda (beg end) - (and current-beg - (>= beg current-beg) - (<= end current-end)))) (replace-re-search-function (lambda (from &optional _bound noerror) (let (found pair) --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 20 19:59:35 2022 Received: (at 54733) by debbugs.gnu.org; 20 Jun 2022 23:59:35 +0000 Received: from localhost ([127.0.0.1]:58616 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o3RIp-0002Tp-UK for submit@debbugs.gnu.org; Mon, 20 Jun 2022 19:59:35 -0400 Received: from mail-wr1-f43.google.com ([209.85.221.43]:46929) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o3RIZ-0002TN-C3 for 54733@debbugs.gnu.org; Mon, 20 Jun 2022 19:59:29 -0400 Received: by mail-wr1-f43.google.com with SMTP id e25so12766026wrc.13 for <54733@debbugs.gnu.org>; Mon, 20 Jun 2022 16:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:message-id:date:mime-version:user-agent:subject :content-language:to:references:from:in-reply-to :content-transfer-encoding; bh=9BxnzP025Scji40UHx7Gy6xlaDrtzuksdv4lSAp2y5U=; b=ZXwnrjN1AgqaqtcwSYYyMZi5btOey+XJWwZXuxlDT25ACgI2VDFesqlHds+APW0QYz Iy8HyrnvKpMToo7TUBlaR44C5dJvn45Xd4RNAEjILKBOyclFGMmBIWz7Zi0nZDtdMUBP 6HEpbOHoTFfFzdFEYjw/3sK9QMuuXGAyPRqNeB0R97B2IU2WP851WwU+s1rjmArsKAjq yk1WpmPz5j39Q3OE3fdTv9OAbHaZ/FQssvBSRt9alaR3Zd7YIu9pHDuqtyI+FPpsQUMO Vksb81ZxqEzfvgwg6QoXFQtKfeYAHoUAVTEkoncxmkV3CdDTcBARN0iDuh0a8o+M9wwA KzLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:message-id:date:mime-version:user-agent :subject:content-language:to:references:from:in-reply-to :content-transfer-encoding; bh=9BxnzP025Scji40UHx7Gy6xlaDrtzuksdv4lSAp2y5U=; b=YkrLon9YZStgU/gulIY0XVLm/ruZ9IKSut51QCbZt9onN4XdLV7CLxxM8rVwYlOR35 +3/ugF+lfgUfpEsWnDRKZHtdQ336JnC9wOdd2cvBW7gj3ogH82/iEtVjvOH7fViRUa+P nKHFG8BCM9RFA7mdZWJ5weIKkeflWhSamiyuWOsI8A8JVI7PmPyDOAueEa1IlJQ0dNzI sCTjYTUdG5FvuP6M0y6hNcdJhhK7RJYuGOdJmDeLeSqpgeJ53J0a8nxrlNAfJ+tlOEEc n0ZkY47kNJuLfW34Bj82TJ1JgC5Ppg1e6JrYEFpoBDU9iZ0627zUHQhGeGYq3Y96nVdJ MNDA== X-Gm-Message-State: AJIora9iE3A7BCIorDScxvnJC97TnzFooA2oDkAyMuAHydAlIyj2xhje cA9FCvlaP390+31esF0kw4Q= X-Google-Smtp-Source: AGRyM1vF5qxGSuotFXHWyXIrVREMRcYgpM9PDQW6OUM1gYEGFXiTtBQQufCOzZwMaldCAc5u+HOjMw== X-Received: by 2002:a5d:59a7:0:b0:218:5584:e3b1 with SMTP id p7-20020a5d59a7000000b002185584e3b1mr25383004wrr.421.1655769549083; Mon, 20 Jun 2022 16:59:09 -0700 (PDT) Received: from [192.168.0.6] ([46.251.119.176]) by smtp.googlemail.com with ESMTPSA id u2-20020a5d4342000000b0020fe61acd09sm14754161wrr.12.2022.06.20.16.59.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 20 Jun 2022 16:59:08 -0700 (PDT) Message-ID: <9d54db84-77ba-6453-dcfa-864a7c05e692@yandex.ru> Date: Tue, 21 Jun 2022 02:59:06 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: bug#54733: Match again in perform-replace Content-Language: en-US To: Juri Linkov , 54733@debbugs.gnu.org References: <865ynnl9f0.fsf_-_@mail.linkov.net> <86pmlvju2c.fsf@mail.linkov.net> <86o80ju6qx.fsf@mail.linkov.net> <861qxb6qvh.fsf@mail.linkov.net> From: Dmitry Gutov In-Reply-To: <861qxb6qvh.fsf@mail.linkov.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 54733 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) Hi Juri, Thanks for working on this. On 03.05.2022 10:10, Juri Linkov wrote: > + (replace-search search-string limit > + regexp-flag delimited-flag > + case-fold-search backward)) I don't know this code too well, but perhaps SEARCH_STRING here should be anchored with something like "\\=" at the beginning? Otherwise the search can succeed here even if the next match is not here. Not sure how important that is, though. > - ;; Counteract the "do the next match now" hack in > - ;; `perform-replace'. And still, it'll report that those > - ;; matches were "filtered out" at the end. > - (isearch-filter-predicate > - (lambda (beg end) > - (and current-beg > - (>= beg current-beg) > - (<= end current-end)))) Please note that we'll likely have to keep this code here for a number of Emacs releases. So the patch should be tested with both versions: with this code present and with it removed, to ensure present and future compatibility. From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 21 14:10:19 2022 Received: (at 54733) by debbugs.gnu.org; 21 Jun 2022 18:10:20 +0000 Received: from localhost ([127.0.0.1]:33236 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o3iKR-0006rU-L5 for submit@debbugs.gnu.org; Tue, 21 Jun 2022 14:10:19 -0400 Received: from relay11.mail.gandi.net ([217.70.178.231]:53835) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o3iKO-0006qo-Km for 54733@debbugs.gnu.org; Tue, 21 Jun 2022 14:10:17 -0400 Received: (Authenticated sender: juri@linkov.net) by mail.gandi.net (Postfix) with ESMTPSA id BD840100003; Tue, 21 Jun 2022 18:10:09 +0000 (UTC) From: Juri Linkov To: Dmitry Gutov Subject: Re: bug#54733: Match again in perform-replace Organization: LINKOV.NET References: <865ynnl9f0.fsf_-_@mail.linkov.net> <86pmlvju2c.fsf@mail.linkov.net> <86o80ju6qx.fsf@mail.linkov.net> <861qxb6qvh.fsf@mail.linkov.net> <9d54db84-77ba-6453-dcfa-864a7c05e692@yandex.ru> Date: Tue, 21 Jun 2022 20:55:19 +0300 In-Reply-To: <9d54db84-77ba-6453-dcfa-864a7c05e692@yandex.ru> (Dmitry Gutov's message of "Tue, 21 Jun 2022 02:59:06 +0300") Message-ID: <861qvhj3j4.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 54733 Cc: 54733@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.5 (-) >> + (replace-search search-string limit >> + regexp-flag delimited-flag >> + case-fold-search backward)) > > I don't know this code too well, but perhaps SEARCH_STRING here should be > anchored with something like "\\=" at the beginning? > > Otherwise the search can succeed here even if the next match is not > here. Not sure how important that is, though. I'm afraid that prepending "\\=" might break a complex regexp somehow. So maybe using replace-search is not the best thing to do. I'm inclined to leave looking-at, but only when the default search function is used. Currently I'm writing a new function to search in rectangular regions in bug#14013, but there are many corner cases, so it will take more time. Then this new function could be used in xref as well since the rectangular region uses the same pairs of points as xref--query-replace-1 does. Then perform-replace could detect if this search function is used, and not to do "the next match now" hack. >> - ;; Counteract the "do the next match now" hack in >> - ;; `perform-replace'. And still, it'll report that those >> - ;; matches were "filtered out" at the end. >> - (isearch-filter-predicate >> - (lambda (beg end) >> - (and current-beg >> - (>= beg current-beg) >> - (<= end current-end)))) > > Please note that we'll likely have to keep this code here for a number of > Emacs releases. So the patch should be tested with both versions: with this > code present and with it removed, to ensure present and future > compatibility. Keeping compatibility should be concerned here definitely. I hope to post the complete patch in a few days. From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 22 03:52:20 2022 Received: (at 54733) by debbugs.gnu.org; 22 Jun 2022 07:52:20 +0000 Received: from localhost ([127.0.0.1]:33881 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o3v9v-0008OM-RG for submit@debbugs.gnu.org; Wed, 22 Jun 2022 03:52:20 -0400 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:36005) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o3v9s-0008Nl-CN for 54733@debbugs.gnu.org; Wed, 22 Jun 2022 03:52:17 -0400 Received: (Authenticated sender: juri@linkov.net) by mail.gandi.net (Postfix) with ESMTPSA id 483611BF209; Wed, 22 Jun 2022 07:52:08 +0000 (UTC) From: Juri Linkov To: Dmitry Gutov Subject: Re: bug#54733: Match again in perform-replace Organization: LINKOV.NET References: <865ynnl9f0.fsf_-_@mail.linkov.net> <86pmlvju2c.fsf@mail.linkov.net> <86o80ju6qx.fsf@mail.linkov.net> <861qxb6qvh.fsf@mail.linkov.net> <9d54db84-77ba-6453-dcfa-864a7c05e692@yandex.ru> <861qvhj3j4.fsf@mail.linkov.net> Date: Wed, 22 Jun 2022 10:36:39 +0300 In-Reply-To: <861qvhj3j4.fsf@mail.linkov.net> (Juri Linkov's message of "Tue, 21 Jun 2022 20:55:19 +0300") Message-ID: <86r13h6syo.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 54733 Cc: 54733@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.5 (-) >>> + (replace-search search-string limit >>> + regexp-flag delimited-flag >>> + case-fold-search backward)) >> >> I don't know this code too well, but perhaps SEARCH_STRING here should be >> anchored with something like "\\=" at the beginning? >> >> Otherwise the search can succeed here even if the next match is not >> here. Not sure how important that is, though. > > I'm afraid that prepending "\\=" might break a complex regexp somehow. Another variant without modifying the original regexp is still to call replace-search, but afterwards check if it stayed at the old position with something like (let ((old-pos (point))) (and (replace-search ...) (eq (match-beginning 0) old-pos))) Less efficient, but looking-at is a real problem since it's incompatible with search functions. This is one of the problems faced in bug#14013 where isearch matches `C-M-r ^' outside of positions handled by the search function because `isearch-search-and-update' uses a hack with looking-at. This is a long-standing flaw in isearch that needs to be fixed. Any uses of looking-at in search/replace should be substituted with an equivalent code that relies on the search function. From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 24 13:25:50 2022 Received: (at 54733) by debbugs.gnu.org; 24 Jun 2022 17:25:50 +0000 Received: from localhost ([127.0.0.1]:43411 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o4n42-0004VA-9O for submit@debbugs.gnu.org; Fri, 24 Jun 2022 13:25:50 -0400 Received: from relay12.mail.gandi.net ([217.70.178.232]:41479) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o4n3z-0004Us-8G for 54733@debbugs.gnu.org; Fri, 24 Jun 2022 13:25:49 -0400 Received: (Authenticated sender: juri@linkov.net) by mail.gandi.net (Postfix) with ESMTPSA id C87FA200006; Fri, 24 Jun 2022 17:25:39 +0000 (UTC) From: Juri Linkov To: Dmitry Gutov Subject: Re: bug#54733: Match again in perform-replace Organization: LINKOV.NET References: <865ynnl9f0.fsf_-_@mail.linkov.net> <86pmlvju2c.fsf@mail.linkov.net> <86o80ju6qx.fsf@mail.linkov.net> <861qxb6qvh.fsf@mail.linkov.net> <9d54db84-77ba-6453-dcfa-864a7c05e692@yandex.ru> <861qvhj3j4.fsf@mail.linkov.net> <86r13h6syo.fsf@mail.linkov.net> Date: Fri, 24 Jun 2022 20:25:05 +0300 In-Reply-To: <86r13h6syo.fsf@mail.linkov.net> (Juri Linkov's message of "Wed, 22 Jun 2022 10:36:39 +0300") Message-ID: <86bkuiotxq.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 54733 Cc: 54733@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.5 (-) >>>> + (replace-search search-string limit >>>> + regexp-flag delimited-flag >>>> + case-fold-search backward)) >>> >>> I don't know this code too well, but perhaps SEARCH_STRING here should be >>> anchored with something like "\\=" at the beginning? >>> >>> Otherwise the search can succeed here even if the next match is not >>> here. Not sure how important that is, though. >> >> I'm afraid that prepending "\\=" might break a complex regexp somehow. > > Another variant without modifying the original regexp is still to call > replace-search, but afterwards check if it stayed at the old position A third variant is to add a new variable `looking-at-function' that is like `isearch-search-fun-function', so everyone who needs to redefine the search function, will also need to redefine the looking-at function. From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 30 14:02:33 2022 Received: (at 54733) by debbugs.gnu.org; 30 Jun 2022 18:02:33 +0000 Received: from localhost ([127.0.0.1]:35565 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o6yUq-0002Mg-Pf for submit@debbugs.gnu.org; Thu, 30 Jun 2022 14:02:33 -0400 Received: from relay10.mail.gandi.net ([217.70.178.230]:49017) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o6yUi-0002M9-Lx for 54733@debbugs.gnu.org; Thu, 30 Jun 2022 14:02:26 -0400 Received: (Authenticated sender: juri@linkov.net) by mail.gandi.net (Postfix) with ESMTPSA id 9749D240005; Thu, 30 Jun 2022 18:02:17 +0000 (UTC) From: Juri Linkov To: Dmitry Gutov Subject: Re: bug#54733: Match again in perform-replace In-Reply-To: <9d54db84-77ba-6453-dcfa-864a7c05e692@yandex.ru> (Dmitry Gutov's message of "Tue, 21 Jun 2022 02:59:06 +0300") Organization: LINKOV.NET References: <865ynnl9f0.fsf_-_@mail.linkov.net> <86pmlvju2c.fsf@mail.linkov.net> <86o80ju6qx.fsf@mail.linkov.net> <861qxb6qvh.fsf@mail.linkov.net> <9d54db84-77ba-6453-dcfa-864a7c05e692@yandex.ru> Date: Thu, 30 Jun 2022 20:52:10 +0300 Message-ID: <86mtdut4xh.fsf@mail.linkov.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.5 (/) X-Debbugs-Envelope-To: 54733 Cc: 54733@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.5 (-) --=-=-= Content-Type: text/plain Hi Dmitry, > Thanks for working on this. > >> + (replace-search search-string limit >> + regexp-flag delimited-flag >> + case-fold-search backward)) > > I don't know this code too well, but perhaps SEARCH_STRING here should be > anchored with something like "\\=" at the beginning? > > Otherwise the search can succeed here even if the next match is not > here. Not sure how important that is, though. Stefan confirmed that "\\=" is reliable. Thanks for the suggestion. Now everything is ready. Please try the latest patch in bug#14013 together with the patch for xref.el below. I've tested with your test cases from bug#53758, and everything works well. >> - ;; Counteract the "do the next match now" hack in >> - ;; `perform-replace'. And still, it'll report that those >> - ;; matches were "filtered out" at the end. >> - (isearch-filter-predicate >> - (lambda (beg end) >> - (and current-beg >> - (>= beg current-beg) >> - (<= end current-end)))) > > Please note that we'll likely have to keep this code here for a number of > Emacs releases. So the patch should be tested with both versions: with this > code present and with it removed, to ensure present and future > compatibility. I guess this might need more conditionals like (>= emacs-major-version 29). --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=xref--query-replace-1.patch diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el index 0213ab3cc5..9b4adffa41 100644 --- a/lisp/progmodes/xref.el +++ b/lisp/progmodes/xref.el @@ -858,29 +858,9 @@ xref--outdated-p ;; FIXME: Write a nicer UI. (defun xref--query-replace-1 (from to iter) - (let* ((query-replace-lazy-highlight nil) - (continue t) + (let* ((continue t) did-it-once buf-pairs pairs - current-beg current-end - ;; Counteract the "do the next match now" hack in - ;; `perform-replace'. And still, it'll report that those - ;; matches were "filtered out" at the end. - (isearch-filter-predicate - (lambda (beg end) - (and current-beg - (>= beg current-beg) - (<= end current-end)))) - (replace-re-search-function - (lambda (from &optional _bound noerror) - (let (found pair) - (while (and (not found) pairs) - (setq pair (pop pairs) - current-beg (car pair) - current-end (cdr pair)) - (goto-char current-beg) - (when (re-search-forward from current-end noerror) - (setq found t))) - found)))) + (region-extract-function (lambda (_) pairs))) (while (and continue (setq buf-pairs (funcall iter :next))) (if did-it-once ;; Reuse the same window for subsequent buffers. @@ -889,8 +869,10 @@ xref--query-replace-1 (pop-to-buffer (car buf-pairs))) (setq did-it-once t)) (setq pairs (cdr buf-pairs)) + (goto-char (point-min)) (setq continue - (perform-replace from to t t nil nil multi-query-replace-map))) + (perform-replace from to t t nil nil multi-query-replace-map + nil nil nil t))) (unless did-it-once (user-error "No suitable matches here")) (when (and continue (not buf-pairs)) (message "All results processed")))) --=-=-=--