GNU bug report logs - #78520
31.0.50; Performance issue in dired+isearch with dired-isearch-filenames

Previous Next

Package: emacs;

Reported by: Ergus <spacibba <at> aol.com>

Date: Tue, 20 May 2025 23:34:02 UTC

Severity: normal

Tags: fixed

Found in version 31.0.50

Done: Eli Zaretskii <eliz <at> gnu.org>

To reply to this bug, email your comments to 78520 AT debbugs.gnu.org.
There is no need to reopen the bug first.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Tue, 20 May 2025 23:34:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Ergus <spacibba <at> aol.com>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Tue, 20 May 2025 23:34:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ergus <spacibba <at> aol.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Wed, 21 May 2025 01:33:19 +0200

Hi:

I have been using dired and isearch in a directory with ~8000 files and
emacs became totally non-responsive. It freezed with every letter for
~10 seconds.

I checked my config and it seems that the problem is
`dired-isearch-filenames`. Any non-nil value produces this issue.

I ran with the profiler and it showed that all the time is going in
`next-single-property-change`


       86696  91% - command-execute
       86696  91%  - call-interactively
       86695  91%   - funcall-interactively
       59531  62%    - isearch-printing-char
       59531  62%     - isearch-process-search-char
       59531  62%      - isearch-process-search-string
       59531  62%       - isearch-search-and-update
       45907  48%        - isearch-update
       45890  48%         - isearch-lazy-highlight-new-loop
       45890  48%          - redisplay
       45854  48%           - timer-event-handler
       45854  48%            - apply
       45854  48%             - isearch-lazy-highlight-buffer-update
       45853  48%              - isearch-lazy-highlight-search
       45852  48%               - isearch-search-string
       45852  48%                - #<byte-code-function C84>
       45852  48%                 - apply
       45852  48%                  - search-within-boundaries
       45778  48%                   - #<byte-code-function C9F>
       45726  48%                    - mapcar
       45722  48%                     - #<byte-code-function C7B>
       45721  48%                        next-single-property-change
          35   0%                    + seq-min
           9   0%                      make-closure
           6   0%                      delq
          40   0%                   + #<byte-code-function 990>
           2   0%                   + #<byte-code-function F75>
           1   0%              + isearch-filter-visible
          36   0%             redisplay_internal (C function)
          14   0%         + pos-visible-in-window-group-p
           2   0%         + isearch-message
           1   0%         + window-max-chars-per-line
       13620  14%        + isearch-search
           4   0%        + isearch-message
       27143  28%    - isearch-del-char
       13624  14%     - isearch-update
       13624  14%      - isearch-lazy-highlight-new-loop
       13624  14%       - redisplay
       13622  14%        - timer-event-handler
       13622  14%         - apply
       13622  14%          - isearch-lazy-highlight-buffer-update
       13622  14%           - isearch-lazy-highlight-search
       13622  14%            - isearch-search-string
       13622  14%             - #<byte-code-function 511>
       13622  14%              - apply
       13622  14%               - search-within-boundaries
       13602  14%                - #<byte-code-function 500>
       13584  14%                 - mapcar
       13581  14%                  - #<byte-code-function 55A>
       13578  14%                     next-single-property-change
          10   0%                 + seq-min
           4   0%                   make-closure
           2   0%                   delq
          13   0%                + #<byte-code-function 990>
           2   0%          redisplay_internal (C function)
       13519  14%     - isearch-search
       13519  14%      - isearch-search-string
       13519  14%       - #<byte-code-function 415>
       13519  14%        - apply
       13519  14%         - search-within-boundaries
       13498  14%          - #<byte-code-function 42C>
       13486  14%           - mapcar
       13485  14%            - #<byte-code-function C2A>
       13482  14%               next-single-property-change
           5   0%             make-closure
           3   0%           + seq-min
           2   0%             delq
          12   0%          + #<byte-code-function 990>
          11   0%    + isearch-forward
           6   0%    + isearch-abort
           3   0%    + dired-next-line
           1   0%    + dired-previous-line
        7498   7% - timer-event-handler
        7498   7%  - apply
        6849   7%   - isearch-lazy-highlight-buffer-update
        6837   7%    - isearch-lazy-highlight-search
        6831   7%     - isearch-search-string
        6810   7%      - #<byte-code-function 492>
        6810   7%       - apply
        6810   7%        - search-within-boundaries
        6802   7%         - #<byte-code-function 49D>
        6793   7%          - mapcar
        6793   7%           - #<byte-code-function 7A7>
        6792   7%              next-single-property-change
           3   0%          + seq-min
           3   0%            delq
           1   0%            make-closure
           5   0%         + #<byte-code-function 990>
          15   0%      + #<byte-code-function 990>
           1   0%        char-table-p
           1   0%      + isearch-search-fun
           2   0%     + isearch-filter-visible
           9   0%    + isearch-filter-visible
           1   0%    + internal--before-save-selected-window
         638   0%   + isearch-lazy-highlight-start
           5   0%   + jit-lock-stealth-fontify
           3   0%   + show-paren-function
           3   0%   + jit-lock-context--update
         362   0% + redisplay_internal (C function)
          62   0%   Automatic GC
           8   0% + tooltip-hide
           2   0% + winner-save-old-configurations
           2   0% + undo-auto--add-boundary
           0   0%   ...





In GNU Emacs 31.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version
 3.24.49, cairo version 1.18.4) of 2025-05-19 built on RTX
Repository revision: b499898a5e6e27ecacfb6a60b22a2289afc589dd
Repository branch: project
System Description: Arch Linux

Configured using:
 'configure --prefix=/home/ergo/.local/ --with-mailutils --with-pgtk
 --with-modules --with-cairo --with-harfbuzz
 --with-native-compilation=aot
 '--program-transform-name=s/^ctags$/ctags.emacs/''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Elisp/l

Minor modes in effect:
  windmove-mode: t
  global-auto-revert-mode: t
  recentf-mode: t
  electric-pair-mode: t
  whitespace-mode: t
  flyspell-mode: t
  completion-preview-mode: t
  diff-hl-margin-local-mode: t
  diff-hl-margin-mode: t
  diff-hl-mode: t
  global-corfu-mode: t
  corfu-mode: t
  fancy-compilation-mode: t
  winner-mode: t
  project-multi-mode: t
  gtags-mode: t
  repeat-mode: t
  xterm-mouse-mode: t
  tty-tip-mode: t
  xclip-mode: t
  override-global-mode: t
  save-place-mode: t
  delete-selection-mode: t
  savehist-mode: t
  global-display-fill-column-indicator-mode: t
  display-fill-column-indicator-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  which-key-mode: t
  tooltip-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/mnt/casa/gits/emacs_clones/gtags-mode/gtags-mode hides /home/ergo/.config/emacs/elpa/gtags-mode-1.8.6/gtags-mode
/home/ergo/.config/emacs/elpa/transient-20250516.1031/transient hides /home/ergo/.local/share/emacs/31.0.50/lisp/transient

Features:
(shadow sort mail-extr emacsbug lisp-mnt windmove jka-compr find-func
cl-print dired-subtree dired-hacks-utils dired-aux dash help-fns
radix-tree mc-edit-lines mc-hide-unmatched-lines-mode mc-mark-more
sgml-mode facemenu dom mc-cycle-cursors multiple-cursors-core advice
comp comp-cstr warnings rect autorevert filenotify recentf tree-widget
ffap thingatpt misearch multi-isearch tramp-cache time-stamp tramp-sh
tramp trampver tramp-integration tramp-message tramp-compat shell
pcomplete parse-time iso8601 format-spec tramp-loaddefs vc-git elec-pair
whitespace flyspell-correct flyspell ispell completion-preview
diff-hl-margin diff-hl-dired diff-hl log-view log-edit message sendmail
yank-media puny dired-x dired dired-loaddefs rfc822 mml mml-sec epa
derived epg rfc6068 epg-config gnus-util time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums
mail-prsvr mailabbrev mail-utils gmm-utils mailheader add-log pcvs-util
vc-dir ewoc vc vc-dispatcher diff-mode track-changes corfu
fancy-compilation compile text-property-search comint ansi-osc
ansi-color comp-run comp-common winner cus-edit pp cus-start cus-load
wid-edit project-multi-mode gtags-mode files-x xref project ring
term/tmux term/xterm xterm init rx repeat cape compat use-package-ensure
package browse-url xdg url url-proxy url-privacy url-expand url-methods
url-history url-cookie generate-lisp-file url-domsuf url-util mailcap
url-handlers url-parse auth-source eieio eieio-core cl-macs icons
password-cache json subr-x map url-vars use-package-diminish xt-mouse
tty-tip xclip edmacro kmacro byte-opt use-package-bind-key bind-key
cl-extra help-mode simple-16-theme saveplace delsel savehist easy-mmode
display-fill-column-indicator display-line-numbers diminish which-key
cl-seq use-package-core cl-loaddefs cl-lib bytecomp byte-compile gv
disp-table info arduino-cli-mode-autoloads auctex-autoloads tex-site
avy-zap-autoloads avy-autoloads benchmark-init-autoloads
bufferlo-autoloads caml-autoloads cape-autoloads clang-format-autoloads
cobol-mode-autoloads compile-multi-autoloads compiler-explorer-autoloads
corfu-autoloads crdt-autoloads csv-mode-autoloads d-mode-autoloads
dape-autoloads deadgrep-autoloads debbugs-autoloads diff-hl-autoloads
diminish-autoloads dired-sidebar-autoloads dired-subtree-autoloads
dired-hacks-utils-autoloads dumb-jump-autoloads e2ansi-autoloads
eglot-booster-autoloads emamux-autoloads eshell-toggle-autoloads
esup-autoloads evil-collection-autoloads annalist-autoloads
evil-leader-autoloads evil-autoloads face-explorer-autoloads
fancy-compilation-autoloads flx-autoloads flymake-nasm-autoloads
flymake-quickdef-autoloads flyspell-correct-autoloads
git-commit-ts-mode-autoloads git-modes-autoloads
git-timemachine-autoloads gnuplot-autoloads goto-chg-autoloads
groovy-mode-autoloads gtags-mode-autoloads haskell-mode-autoloads
highlight-indent-guides-autoloads i3wm-config-mode-autoloads
ibuffer-sidebar-autoloads iedit-autoloads imenu-list-autoloads
julia-ts-mode-autoloads julia-mode-autoloads languagetool-autoloads
lice-autoloads lorem-ipsum-autoloads lua-mode-autoloads magit-autoloads
magit-section-autoloads llama-autoloads move-dup-autoloads
multiple-cursors-autoloads mutt-mode-autoloads nasm-mode-autoloads
neotree-autoloads nftables-mode-autoloads nginx-mode-autoloads
notmuch-autoloads objed-autoloads phi-search-autoloads
pkgbuild-mode-autoloads plz-autoloads popup-autoloads
protobuf-ts-mode-autoloads scopeline-autoloads shell-command+-autoloads
slime-autoloads macrostep-autoloads sphinx-mode-autoloads f-autoloads
s-autoloads dash-autoloads spinner-autoloads ssh-config-mode-autoloads
string-inflection-autoloads sudo-edit-autoloads systemd-autoloads
tmux-mode-autoloads transient-autoloads urgrep-autoloads vdiff-autoloads
hydra-autoloads lv-autoloads vterm-toggle-autoloads vterm-autoloads
vundo-autoloads with-editor-autoloads xclip-autoloads
yasnippet-snippets-autoloads yasnippet-autoloads early-init rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win
term/common-win touch-screen pgtk-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty move-toolbar make-network-process tty-child-frames
native-compile emacs)

Memory information:
((conses 16 348255 236056) (symbols 48 21674 0)
 (strings 32 85000 11286) (string-bytes 1 2720636) (vectors 16 38169)
 (vector-slots 8 1104566 168128) (floats 8 143 137)
 (intervals 56 14651 3188) (buffers 1064 19))




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 21 May 2025 06:35:01 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Ergus via "Bug reports for GNU Emacs, the Swiss army knife of text
 editors" <bug-gnu-emacs <at> gnu.org>
Cc: Ergus <spacibba <at> aol.com>, 78520 <at> debbugs.gnu.org
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Wed, 21 May 2025 09:20:43 +0300
> I have been using dired and isearch in a directory with ~8000 files and
> emacs became totally non-responsive. It freezed with every letter for
> ~10 seconds.
>
> I checked my config and it seems that the problem is
> `dired-isearch-filenames`. Any non-nil value produces this issue.

When you customize `dired-isearch-filenames` to non-nil, it uses
`next-single-property-change` to restrict matches to filenames.

> I ran with the profiler and it showed that all the time is going in
> `next-single-property-change`

You can make it twice as quick by removing the property
'dired-symlink-filename' from 'dired-isearch-search-filenames'.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 21 May 2025 06:35:03 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 21 May 2025 12:40:02 GMT) Full text and rfc822 format available.

Message #14 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org
Subject: Re: bug#78520: 31.0.50;
 Performance issue in dired+isearch with dired-isearch-filenames
Date: Wed, 21 May 2025 15:38:59 +0300
> Cc: spacibba <at> aol.com
> From: Juri Linkov <juri <at> linkov.net>
> Date: Wed, 21 May 2025 09:20:43 +0300
> 
> > I have been using dired and isearch in a directory with ~8000 files and
> > emacs became totally non-responsive. It freezed with every letter for
> > ~10 seconds.
> >
> > I checked my config and it seems that the problem is
> > `dired-isearch-filenames`. Any non-nil value produces this issue.
> 
> When you customize `dired-isearch-filenames` to non-nil, it uses
> `next-single-property-change` to restrict matches to filenames.

I guess this could be slow in a buffer with a lot of properties?

Would it be possible to speed this up by searching as usual, but then
rejecting matches whose positions don't have the 'filename' property?
Or was this tried and found to be not faster?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Thu, 22 May 2025 06:47:03 GMT) Full text and rfc822 format available.

Message #17 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Thu, 22 May 2025 09:33:20 +0300
>> > I have been using dired and isearch in a directory with ~8000 files and
>> > emacs became totally non-responsive. It freezed with every letter for
>> > ~10 seconds.
>> >
>> > I checked my config and it seems that the problem is
>> > `dired-isearch-filenames`. Any non-nil value produces this issue.
>> 
>> When you customize `dired-isearch-filenames` to non-nil, it uses
>> `next-single-property-change` to restrict matches to filenames.
>
> I guess this could be slow in a buffer with a lot of properties?

In a Dired buffer the property 'dired-filename' is almost on every line.

> Would it be possible to speed this up by searching as usual, but then
> rejecting matches whose positions don't have the 'filename' property?
> Or was this tried and found to be not faster?

The currently implementation was intended to be quite fast,
and indeed when I try it on a dir with thousands of files,
isearch-lazy-highlight takes only 1 sec, even with thousands of matches.
But apparently on slower hardware it's more slow.  So unless someone wants
to make an effort to optimize the implementation more, IMHO this could be closed.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Thu, 22 May 2025 07:42:08 GMT) Full text and rfc822 format available.

Message #20 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Thu, 22 May 2025 10:41:21 +0300
> From: Juri Linkov <juri <at> linkov.net>
> Cc: 78520 <at> debbugs.gnu.org,  spacibba <at> aol.com
> Date: Thu, 22 May 2025 09:33:20 +0300
> 
> >> When you customize `dired-isearch-filenames` to non-nil, it uses
> >> `next-single-property-change` to restrict matches to filenames.
> >
> > I guess this could be slow in a buffer with a lot of properties?
> 
> In a Dired buffer the property 'dired-filename' is almost on every line.
> 
> > Would it be possible to speed this up by searching as usual, but then
> > rejecting matches whose positions don't have the 'filename' property?
> > Or was this tried and found to be not faster?
> 
> The currently implementation was intended to be quite fast,
> and indeed when I try it on a dir with thousands of files,
> isearch-lazy-highlight takes only 1 sec, even with thousands of matches.
> But apparently on slower hardware it's more slow.

I have fast hardware, but C-s for a match near the end of a Dired
buffer showing 5K files takes about 9 sec.  This is in an unoptimized
build; an optimized build still takes 2.25 sec.

I'm not sure why you are talking about isearch-lazy-highlight, that's
not what the original report is about.  C-s is slow even if I turn off
isearch-lazy-highlight, and the profile below in that case still
points to next-single-property-change as the hot spot.

> So unless someone wants to make an effort to optimize the
> implementation more, IMHO this could be closed.

Does isearch.el have some infrastructure for examining a match and
rejecting it if it doesn't meet some criteria?  If so, can you point
me to that infrastructure?

Here's the profile I collected after turning off
isearch-lazy-highlight:

         621  87% - ...
         618  87%  - isearch-search
         618  87%   - condition-case
         618  87%    - let
         618  87%     - while
         618  87%      - setq
         618  87%       - isearch-search-string
         618  87%        - let*
         618  87%         - save-excursion
         618  87%          - funcall
         618  87%           - #<byte-code-function 21A>
         618  87%            - apply
         618  87%             - search-within-boundaries
         618  87%              - let*
         618  87%               - while
         311  43%                - setq
         311  43%                 - funcall
         311  43%                  - #<interpreted-function E6E>
         311  43%                   - let
         311  43%                    - if
         311  43%                     - mapcar
         308  43%                      - #<interpreted-function D88>
         308  43%                         next-single-property-change
           3   0%                      - function
           3   0%                       - cconv-make-interpreted-closure
           2   0%                          cconv-fv
           1   0%                          macroexpand-all
         307  43%                - if
         303  42%                 - setq
         302  42%                  - funcall
         299  42%                   - #<interpreted-function E6E>
         299  42%                    - let
         298  42%                     - if
         298  42%                      - mapcar
         289  40%                       - #<interpreted-function 740>
         289  40%                          next-single-property-change
           8   1%                       - function
           8   1%                        - cconv-make-interpreted-closure
           3   0%                         - macroexpand-all
           3   0%                          - macroexp--expand-all
           3   0%                           - macroexp--all-forms
           2   0%                            - macroexp--expand-all
           2   0%                             - #<byte-code-function 83C>
           2   0%                              - macroexp--all-forms
           2   0%                               - macroexp--expand-all
           2   0%                                  macroexp-macroexpand
           3   0%                         - cconv-fv
           2   0%                            mapcar
           1   0%                          - cconv-analyze-form
           1   0%                           - cconv--analyze-function
           1   0%                            - mapcar
           1   0%                               #<byte-code-function F28>
           3   0%                   - #<interpreted-function FDE>
           2   0%                    - let
           1   0%                     - funcall
           1   0%                        re-search-forward
           1   0%                       cond
           4   0%                 - if
           3   0%                  - progn
           3   0%                   - if
           2   0%                      goto-char
           2   0%  - completion-try-completion
           2   0%   - completion--nth-completion
           2   0%    - seq-some
           2   0%     - seq-do
           2   0%      - mapc
           2   0%       - #<byte-code-function B6E>
           2   0%        - #<byte-code-function BF2>
           2   0%         - eval
           2   0%          - let
           2   0%           - funcall
           2   0%            - #<byte-code-function C44>
           1   0%             - completion-pcm-try-completion
           1   0%              - completion-pcm--find-all-completions
           1   0%               - completion-pcm--all-completions
           1   0%                - all-completions
           1   0%                 - #<byte-code-function AA0>
           1   0%                  - complete-with-action
           1   0%                     all-completions
           1   0%             - completion-basic-try-completion
           1   0%              - try-completion
           1   0%               - #<byte-code-function AA0>
           1   0%                - complete-with-action
           1   0%                   try-completion
           1   0%  - funcall-interactively
           1   0%   - isearch-printing-char
           1   0%    - let
           1   0%     - if
           1   0%      - isearch-process-search-char
           1   0%       - let*
           1   0%        - isearch-process-search-string
           1   0%         - isearch-search-and-update
           1   0%          - isearch-update
           1   0%           - if
           1   0%            - progn
           1   0%             - if
           1   0%              - let
           1   0%               - setq
           1   0%                - pos-visible-in-window-group-p
           1   0%                   pos-visible-in-window-p
          68   9%   Automatic GC
          14   1% - command-execute
          14   1%  - call-interactively
          14   1%   - byte-code
          14   1%    - read-extended-command
          14   1%     - read-extended-command-1
          14   1%      - completing-read
          14   1%       - completing-read-default
          14   1%        - read-from-minibuffer
           2   0%           redisplay_internal (C function)
           6   0%   redisplay_internal (C function)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Thu, 22 May 2025 17:18:03 GMT) Full text and rfc822 format available.

Message #23 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Thu, 22 May 2025 19:58:07 +0300
>> The currently implementation was intended to be quite fast,
>> and indeed when I try it on a dir with thousands of files,
>> isearch-lazy-highlight takes only 1 sec, even with thousands of matches.
>> But apparently on slower hardware it's more slow.
>
> I have fast hardware, but C-s for a match near the end of a Dired
> buffer showing 5K files takes about 9 sec.  This is in an unoptimized
> build; an optimized build still takes 2.25 sec.

I see the same in an optimized build: ~2 sec until isearch-lazy-count
shows the number of matches (~5000).

> I'm not sure why you are talking about isearch-lazy-highlight, that's
> not what the original report is about.  C-s is slow even if I turn off
> isearch-lazy-highlight, and the profile below in that case still
> points to next-single-property-change as the hot spot.

I see no delay when isearch-lazy-highlight is disabled.

>> So unless someone wants to make an effort to optimize the
>> implementation more, IMHO this could be closed.
>
> Does isearch.el have some infrastructure for examining a match and
> rejecting it if it doesn't meet some criteria?  If so, can you point
> me to that infrastructure?

Everything is in 'search-within-boundaries' where 'next-fun'
is a lambda from 'isearch-search-fun-in-text-property'
that uses 'next-single-property-change'.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Fri, 23 May 2025 07:09:01 GMT) Full text and rfc822 format available.

Message #26 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Fri, 23 May 2025 10:08:33 +0300
> From: Juri Linkov <juri <at> linkov.net>
> Cc: 78520 <at> debbugs.gnu.org,  spacibba <at> aol.com
> Date: Thu, 22 May 2025 19:58:07 +0300
> 
> >> So unless someone wants to make an effort to optimize the
> >> implementation more, IMHO this could be closed.
> >
> > Does isearch.el have some infrastructure for examining a match and
> > rejecting it if it doesn't meet some criteria?  If so, can you point
> > me to that infrastructure?
> 
> Everything is in 'search-within-boundaries' where 'next-fun'
> is a lambda from 'isearch-search-fun-in-text-property'
> that uses 'next-single-property-change'.

Thanks, but what I meant was whether the "normal" search that searches
the entire text has a facility to examine and reject potential
matches.  isearch-search-fun-in-text-property looks only inside text
that has a specified property, and that's not what I meant.  I meant
this idea:

> Would it be possible to speed this up by searching as usual, but then
> rejecting matches whose positions don't have the 'filename' property?
> Or was this tried and found to be not faster?

Here, "searching as usual" means searching the entire buffer text, not
just its chunks that have a specific property.

Do we have such infrastructure in isearch.el?





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Fri, 23 May 2025 18:29:03 GMT) Full text and rfc822 format available.

Message #29 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Fri, 23 May 2025 21:19:27 +0300
>> >> So unless someone wants to make an effort to optimize the
>> >> implementation more, IMHO this could be closed.
>> >
>> > Does isearch.el have some infrastructure for examining a match and
>> > rejecting it if it doesn't meet some criteria?  If so, can you point
>> > me to that infrastructure?
>> 
>> Everything is in 'search-within-boundaries' where 'next-fun'
>> is a lambda from 'isearch-search-fun-in-text-property'
>> that uses 'next-single-property-change'.
>
> Thanks, but what I meant was whether the "normal" search that searches
> the entire text has a facility to examine and reject potential
> matches.  isearch-search-fun-in-text-property looks only inside text
> that has a specified property, and that's not what I meant.  I meant
> this idea:
>
>> Would it be possible to speed this up by searching as usual, but then
>> rejecting matches whose positions don't have the 'filename' property?
>> Or was this tried and found to be not faster?
>
> Here, "searching as usual" means searching the entire buffer text, not
> just its chunks that have a specific property.
>
> Do we have such infrastructure in isearch.el?

To search the entire buffer text is possible by leaving uncustomized
the default value nil of 'dired-isearch-filenames'.

Or do you mean adding a new value to 'dired-isearch-filenames'
that will use 'isearch-filter-predicate' removed in the
commit 935cc4279568?

Then one value will use the current implementation
with 'isearch-search-fun-function'.  And a new value will use
the faster implementation with 'isearch-filter-predicate'.

But I have no idea how to explain this difference in documentation.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Fri, 23 May 2025 21:08:02 GMT) Full text and rfc822 format available.

Message #32 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Fri, 23 May 2025 23:09:25 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

> > Would it be possible to speed this up by searching as usual, but then
> > rejecting matches whose positions don't have the 'filename' property?
> > Or was this tried and found to be not faster?

IIRC (and understand correctly): It had been tried and was faster, but:
we then had dismissed this idea.  One reason was that we wanted to make
^ and $ match the beginning and end of the file names when using regexp
file name isearch.  There were other reasons - lazy highlight, I don't
recall.  There were a few problems, you find it somewhere.  The decision
was not taken lightly - the result was just not convincing, and the
problems not fixable in a sensible way.


Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Sat, 24 May 2025 01:42:02 GMT) Full text and rfc822 format available.

Message #35 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Michael Heerdegen <michael_heerdegen <at> web.de>, Eli Zaretskii <eliz <at> gnu.org>
Cc: "spacibba <at> aol.com" <spacibba <at> aol.com>,
 "78520 <at> debbugs.gnu.org" <78520 <at> debbugs.gnu.org>, Juri Linkov <juri <at> linkov.net>
Subject: RE: [External] : bug#78520: 31.0.50; Performance issue in
 dired+isearch with dired-isearch-filenames
Date: Sat, 24 May 2025 01:41:38 +0000
> > > Would it be possible to speed this up by searching as usual, but then
> > > rejecting matches whose positions don't have the 'filename' property?
> > > Or was this tried and found to be not faster?
> 
> IIRC (and understand correctly): It had been tried and was faster, but:
> we then had dismissed this idea.  One reason was that we wanted to make
> ^ and $ match the beginning and end of the file names when using regexp
> file name isearch.  There were other reasons - lazy highlight, I don't
> recall.  There were a few problems, you find it somewhere.  The decision
> was not taken lightly - the result was just not convincing, and the
> problems not fixable in a sensible way.

Caveat: I rarely use `dired-isearch-filenames', and I've
nothing to say about improving performance of that command.

But wrt searching filenames in Dired it might sometimes
be appropriate to divide the search space (listing(s)
in the buffer), by using narrowings or `occur', before
using `dired-isearch-filenames' (or even plain Isearch).




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Sat, 24 May 2025 07:57:02 GMT) Full text and rfc822 format available.

Message #38 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Sat, 24 May 2025 10:56:13 +0300
tags 78520 wontfix
close 78520
thanks

> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: Juri Linkov <juri <at> linkov.net>,  spacibba <at> aol.com,  78520 <at> debbugs.gnu.org
> Date: Fri, 23 May 2025 23:09:25 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > > Would it be possible to speed this up by searching as usual, but then
> > > rejecting matches whose positions don't have the 'filename' property?
> > > Or was this tried and found to be not faster?
> 
> IIRC (and understand correctly): It had been tried and was faster, but:
> we then had dismissed this idea.  One reason was that we wanted to make
> ^ and $ match the beginning and end of the file names when using regexp
> file name isearch.  There were other reasons - lazy highlight, I don't
> recall.  There were a few problems, you find it somewhere.  The decision
> was not taken lightly - the result was just not convincing, and the
> problems not fixable in a sensible way.

Thanks.  I guess this means users of this feature will have to live
with the slowdown.

I'm therefore closing this bug.




Added tag(s) wontfix. Request was from Eli Zaretskii <eliz <at> gnu.org> to control <at> debbugs.gnu.org. (Sat, 24 May 2025 07:57:03 GMT) Full text and rfc822 format available.

bug closed, send any further explanations to 78520 <at> debbugs.gnu.org and Ergus <spacibba <at> aol.com> Request was from Eli Zaretskii <eliz <at> gnu.org> to control <at> debbugs.gnu.org. (Sat, 24 May 2025 07:57:03 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Sat, 24 May 2025 17:37:02 GMT) Full text and rfc822 format available.

Message #45 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Drew Adams <drew.adams <at> oracle.com>, Michael Heerdegen
 <michael_heerdegen <at> web.de>, Eli Zaretskii <eliz <at> gnu.org>
Cc: "spacibba <at> aol.com" <spacibba <at> aol.com>,
 "78520 <at> debbugs.gnu.org" <78520 <at> debbugs.gnu.org>, Juri Linkov <juri <at> linkov.net>
Subject: RE: [External] : bug#78520: 31.0.50; Performance issue in
 dired+isearch with dired-isearch-filenames
Date: Sat, 24 May 2025 17:36:28 +0000
> But wrt searching filenames in Dired it might sometimes
> be appropriate to divide the search space (listing(s)
> in the buffer), by using narrowings or `occur', before
> using `dired-isearch-filenames' (or even plain Isearch).

Should have mentioned that another way to divide
up the search space for separate searches is to
use wildcards (*) to create subset Dired buffers.
That can already do some of the filename 
"searching".

Mentioning these things (which may be obvious)
for the OP, especially since the bug has now
been closed.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Sun, 25 May 2025 20:37:01 GMT) Full text and rfc822 format available.

Message #48 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: spacibba <at> aol.com, 78520 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Sun, 25 May 2025 22:38:03 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

> Thanks.  I guess this means users of this feature will have to live
> with the slowdown.

Probably.  We could, of course, try to improve things a bit.

I don't recall all cases where this slow kind of doing things is
necessary.  Do you, Juri?

Maybe it would be possible to use a faster, but equivalent, algorithm
when possible, e.g. whenever using a non-regexp filename search.  I have
been experimenting with this approach locally, but I don't know if I'm
breaking any use cases.  It might be worth trying at least - the
performance in large buffers can be a pain.


Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Tue, 27 May 2025 06:56:02 GMT) Full text and rfc822 format available.

Message #51 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Tue, 27 May 2025 09:35:29 +0300
>> Thanks.  I guess this means users of this feature will have to live
>> with the slowdown.
>
> Probably.  We could, of course, try to improve things a bit.
>
> I don't recall all cases where this slow kind of doing things is
> necessary.  Do you, Juri?
>
> Maybe it would be possible to use a faster, but equivalent, algorithm
> when possible, e.g. whenever using a non-regexp filename search.  I have
> been experimenting with this approach locally, but I don't know if I'm
> breaking any use cases.  It might be worth trying at least - the
> performance in large buffers can be a pain.

Unfortunately, I don't remember if this slower implementation is
required for non-regexp case.  Maybe let's revert it for non-regexp.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Tue, 27 May 2025 15:29:02 GMT) Full text and rfc822 format available.

Message #54 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Tue, 27 May 2025 17:30:24 +0200
Juri Linkov <juri <at> linkov.net> writes:

> Unfortunately, I don't remember if this slower implementation is
> required for non-regexp case.  Maybe let's revert it for non-regexp.

A big part of the discussion was in Bug#14013.  I've read most of the
discussion again but found no clue.  So hopefully: no, not required.

I'm now looking at `search-within-boundaries'.  Do you think we could
fall back to a more efficient algorithm even when `subregexp' is nil
(that seems to work for me)?

For reference, I had been using something like this:

#+begin_src emacs-lisp
(defun my-search-within-boundaries--before-while-ad
    (f search-fun get-fun next-fun string &optional bound noerror count)
  "Search more efficiently when possible."
  (cl-flet ((search-with-slow-algorithm ()
              (funcall f search-fun get-fun next-fun string bound noerror count)))
    (if (and isearch-regexp
             ;; do we match the beginning or end of the line (a region)?
             (save-match-data
               (let ((subregexp (make-symbol "subregexp")))
                 (catch subregexp
                   (let ((i 0))
                     (while (string-match "\\^\\|\\$" string i)
                       (setq i (match-end 0))
                       (when (subregexp-context-p string (match-beginning 0))
                         ;; The ^/$ is not inside a char-range or escaped.
                         (throw subregexp t))))))))
        ;; we need to fall back to the slow procedure
        (search-with-slow-algorithm)
      (let ((old (point))
            (search-result nil))
        (unwind-protect
            (setq search-result
                  (and (save-match-data
                         (when (funcall (or search-fun
                                            (isearch-search-fun-default))
                                        string bound 'noeror)
                           (goto-char (if isearch-forward (match-beginning 0) (match-end 0)))
                           t))
                       (search-with-slow-algorithm)))
          (unless search-result (goto-char old)))))))

(advice-add 'search-within-boundaries :around #'my-search-within-boundaries--before-while-ad)
#+end_src

and didn't see any downside so far.


Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Tue, 27 May 2025 18:11:01 GMT) Full text and rfc822 format available.

Message #57 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Tue, 27 May 2025 20:59:03 +0300
> I'm now looking at `search-within-boundaries'.  Do you think we could
> fall back to a more efficient algorithm even when `subregexp' is nil
> (that seems to work for me)?

So you propose the optimization that moves point to the next search match before
fall back to old algorithm.  I expected that optimization like this is needed.
But it will take more time to adapt it into the current implementation.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 28 May 2025 17:18:02 GMT) Full text and rfc822 format available.

Message #60 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Wed, 28 May 2025 20:15:56 +0300
tags 78520 = fixed
thanks

>> Unfortunately, I don't remember if this slower implementation is
>> required for non-regexp case.  Maybe let's revert it for non-regexp.
>
> A big part of the discussion was in Bug#14013.  I've read most of the
> discussion again but found no clue.  So hopefully: no, not required.
>
> I'm now looking at `search-within-boundaries'.  Do you think we could
> fall back to a more efficient algorithm even when `subregexp' is nil
> (that seems to work for me)?

Thanks for the idea of optimization.  Now it's pushed to master.
Please check if everything is correct.

Also thanks Ergus for the request.




Added tag(s) fixed; removed tag(s) wontfix. Request was from Juri Linkov <juri <at> linkov.net> to control <at> debbugs.gnu.org. (Wed, 28 May 2025 17:18:03 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Sun, 01 Jun 2025 23:09:02 GMT) Full text and rfc822 format available.

Message #65 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Mon, 02 Jun 2025 01:10:18 +0200
Juri Linkov <juri <at> linkov.net> writes:

> Thanks for the idea of optimization.  Now it's pushed to master.
> Please check if everything is correct.

Thanks for the patch.

I did not face errors so far but... is it now really faster than before?
I tried M-s f C-s in a huge dired buffer in emacs -Q, and it was
horribly slow.  Slower than with the hack I had been using.  I did not
yet try to have a closer look at your code I must admit.  Did you see a
speed improvement?


Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Tue, 03 Jun 2025 16:47:02 GMT) Full text and rfc822 format available.

Message #68 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Tue, 03 Jun 2025 18:25:33 +0300
> I did not face errors so far but... is it now really faster than before?
> I tried M-s f C-s in a huge dired buffer in emacs -Q, and it was
> horribly slow.  Slower than with the hack I had been using.  I did not
> yet try to have a closer look at your code I must admit.  Did you see a
> speed improvement?

I'm using C-s in large Dired buffers all the time, and before the patch
the search lags were too long (~2 sec on every match).  But now with
the patch the search is instantaneous even on very large Dired buffers.

The search might be still slow only on one very rare case:
when the search string also occurs outside of file names.
For example, when the Dired contains permissions "-rw-r--r--",
then searching for "r" or "rw" will be slow.

Do we need more optimization for such rare cases?

This is possible to do by adding the same code that you suggested
not only before the 'while' loop, but also at the end of every
iteration inside the 'while' loop.  But this will overcomplicate
the function 'search-within-boundaries'.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 04 Jun 2025 01:44:02 GMT) Full text and rfc822 format available.

Message #71 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Wed, 04 Jun 2025 03:44:44 +0200
Juri Linkov <juri <at> linkov.net> writes:

> I'm using C-s in large Dired buffers all the time, and before the patch
> the search lags were too long (~2 sec on every match).  But now with
> the patch the search is instantaneous even on very large Dired
> buffers.

I do have a very large buffer, and in emacs -Q the first match is not
found before at least 10 or 15 seconds have passed.  I have a somewhat
older laptop, but definitely not that slow.

> The search might be still slow only on one very rare case:
> when the search string also occurs outside of file names.
> For example, when the Dired contains permissions "-rw-r--r--",
> then searching for "r" or "rw" will be slow.

I know.  That's not what I did.  I had started the search using
M-s f C-s, not C-s, however.  But let me check for my local changes to
the repo first.  Maybe there is a culprit.

> Do we need more optimization for such rare cases?

Not really worth it I think.

One small thing I noticed (but that does not have a large impact) was
that `search-within-boundaries' is called repeatedly, and the binding

#+begin_src emacs-lisp
(subregexp
  (and isearch-regexp
       (save-match-data
         (catch 'subregexp
           (while (string-match "\\^\\|\\$" string i)
             (setq i (match-end 0))
             (when (subregexp-context-p string (match-beginning 0))
               ;; The ^/$ is not inside a char-range or escaped.
               (throw 'subregexp t)))))))
#+end_src

is computed every time with the same bindings in effect.  This could
maybe be factored out (i.e., the computation could maybe be moved
upwards in the call tree).


Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 04 Jun 2025 02:06:02 GMT) Full text and rfc822 format available.

Message #74 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Wed, 04 Jun 2025 04:06:28 +0200
Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> But let me check for my local changes to the repo first.  Maybe there
> is a culprit.

Ok - good news: it was just that.  I had a pending fix for Bug#73018
that caused the bad performance.  Without it I see a very good performance.


> > Do we need more optimization for such rare cases?
> Not really worth it I think.

But if you want to give it a try, don't hesitate either.  If the user
happens to search for something like the user or group name in
file names, the behavior is annoying.  And a confusing surprise.
So even if it's a corner case, I think it could be worth trying to
improve this, given you have the time.


Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 04 Jun 2025 06:16:02 GMT) Full text and rfc822 format available.

Message #77 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Wed, 04 Jun 2025 09:12:18 +0300
>> But let me check for my local changes to the repo first.  Maybe there
>> is a culprit.
>
> Ok - good news: it was just that.

Good.

> I had a pending fix for Bug#73018 that caused the bad performance.
> Without it I see a very good performance.

Does the problem reported in bug#73018 still exist?

>> > Do we need more optimization for such rare cases?
>> Not really worth it I think.
>
> But if you want to give it a try, don't hesitate either.  If the user
> happens to search for something like the user or group name in
> file names, the behavior is annoying.  And a confusing surprise.
> So even if it's a corner case, I think it could be worth trying to
> improve this, given you have the time.

Searching the user name in file names is too rare use case,
and still can be fast when starting the search outside of filenames,
i.e. using the default search function.

> One small thing I noticed (but that does not have a large impact) was
> that `search-within-boundaries' is called repeatedly, and the binding
> #+begin_src emacs-lisp
> (subregexp
>   (and isearch-regexp
>        (save-match-data
>          (catch 'subregexp
>            (while (string-match "\\^\\|\\$" string i)
>              (setq i (match-end 0))
>              (when (subregexp-context-p string (match-beginning 0))
>                ;; The ^/$ is not inside a char-range or escaped.
>                (throw 'subregexp t)))))))
> #+end_src
> is computed every time with the same bindings in effect.  This could
> maybe be factored out (i.e., the computation could maybe be moved
> upwards in the call tree).

This would require adding a new buffer-local variable
with an ad-hoc name.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 04 Jun 2025 23:40:01 GMT) Full text and rfc822 format available.

Message #80 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Thu, 05 Jun 2025 01:41:26 +0200
Juri Linkov <juri <at> linkov.net> writes:

> > I had a pending fix for Bug#73018 that caused the bad performance.
> > Without it I see a very good performance.
>
> Does the problem reported in bug#73018 still exist?

No, the problem discussed there seems to be fixed.  Good :-)


> > One small thing I noticed (but that does not have a large impact) was
> > that `search-within-boundaries' is called repeatedly, and the binding
> > #+begin_src emacs-lisp
> > (subregexp
> >   (and isearch-regexp
> >        (save-match-data
> >          (catch 'subregexp
> >            (while (string-match "\\^\\|\\$" string i)
> >              (setq i (match-end 0))
> >              (when (subregexp-context-p string (match-beginning 0))
> >                ;; The ^/$ is not inside a char-range or escaped.
> >                (throw 'subregexp t)))))))
> > #+end_src
> > is computed every time with the same bindings in effect.  This could
> > maybe be factored out (i.e., the computation could maybe be moved
> > upwards in the call tree).
>
> This would require adding a new buffer-local variable
> with an ad-hoc name.

Or some kind of reorganisation of the whole logic of the involved
functions, but that's just not worth the trouble.  (buffer-local
variable... would we not want to support multi isearch?)


But let me please come back to wdired, I have some questions about that
before I continue to experiment:

1.  In `wdired-change-to-wdired-mode', we do this:

| (when wdired-search-replace-filenames
|   (add-function :around (local 'isearch-search-fun-function)
|                 #'dired-isearch-search-filenames
|                 '((isearch-message-prefix . "filename ")))
|   (setq-local replace-search-function
|               (setq-local replace-re-search-function
|                           (funcall isearch-search-fun-function)))
|   ;; Original dired hook removes dired-isearch-search-filenames that
|   ;; is needed outside isearch for lazy-highlighting in query-replace.
|   (remove-hook 'isearch-mode-hook #'dired-isearch-filenames-setup t))

Does that mean that after switching to wdired there is no way to switch
between filename-only and normal search and replace?  Is the behavior
determined when switching to wdired, or can it still be change while in
wdired-mode?

2.  How was that - was it possible to control whether to search and
replace in the symlink targets?


TIA,

Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Thu, 05 Jun 2025 06:40:01 GMT) Full text and rfc822 format available.

Message #83 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Thu, 05 Jun 2025 09:35:07 +0300
> But let me please come back to wdired, I have some questions about that
> before I continue to experiment:
>
> 1.  In `wdired-change-to-wdired-mode', we do this:
>
> | (when wdired-search-replace-filenames
> |   (add-function :around (local 'isearch-search-fun-function)
> |                 #'dired-isearch-search-filenames
> |                 '((isearch-message-prefix . "filename ")))
> |   (setq-local replace-search-function
> |               (setq-local replace-re-search-function
> |                           (funcall isearch-search-fun-function)))
> |   ;; Original dired hook removes dired-isearch-search-filenames that
> |   ;; is needed outside isearch for lazy-highlighting in query-replace.
> |   (remove-hook 'isearch-mode-hook #'dired-isearch-filenames-setup t))
>
> Does that mean that after switching to wdired there is no way to switch
> between filename-only and normal search and replace?

Correct.

> Is the behavior determined when switching to wdired, or can it still
> be change while in wdired-mode?

Since wdired is intended for editing filenames, we restrict
search/replace to filenames only.

> 2.  How was that - was it possible to control whether to search and
> replace in the symlink targets?

To search/replace in the symlink, 'dired-isearch-search-filenames'
uses the text property 'dired-symlink-filename'.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Mon, 09 Jun 2025 03:14:01 GMT) Full text and rfc822 format available.

Message #86 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Mon, 09 Jun 2025 05:14:31 +0200
Juri Linkov <juri <at> linkov.net> writes:

> > 1.  In `wdired-change-to-wdired-mode', we do this:
> >
> > | (when wdired-search-replace-filenames
> > |   (add-function :around (local 'isearch-search-fun-function)
> > |                 #'dired-isearch-search-filenames
> > |                 '((isearch-message-prefix . "filename ")))
> > |   (setq-local replace-search-function
> > |               (setq-local replace-re-search-function
> > |                           (funcall isearch-search-fun-function)))
> > |   ;; Original dired hook removes dired-isearch-search-filenames that
> > |   ;; is needed outside isearch for lazy-highlighting in query-replace.
> > |   (remove-hook 'isearch-mode-hook #'dired-isearch-filenames-setup t))
> >
> > Does that mean that after switching to wdired there is no way to switch
> > between filename-only and normal search and replace?
>
> Correct.
>
> > Is the behavior determined when switching to wdired, or can it still
> > be change while in wdired-mode?
>
> Since wdired is intended for editing filenames, we restrict
> search/replace to filenames only.

It's also possible to change permissions.  Anyway...


> > 2.  How was that - was it possible to control whether to search and
> > replace in the symlink targets?
>
> To search/replace in the symlink, 'dired-isearch-search-filenames'
> uses the text property 'dired-symlink-filename'.

Ah - right, that was it.  Thanks.


Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Mon, 09 Jun 2025 06:38:03 GMT) Full text and rfc822 format available.

Message #89 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Mon, 09 Jun 2025 09:32:23 +0300
>> > 1.  In `wdired-change-to-wdired-mode', we do this:
>> >
>> > | (when wdired-search-replace-filenames
>> > |   (add-function :around (local 'isearch-search-fun-function)
>> > |                 #'dired-isearch-search-filenames
>> > |                 '((isearch-message-prefix . "filename ")))
>> > |   (setq-local replace-search-function
>> > |               (setq-local replace-re-search-function
>> > |                           (funcall isearch-search-fun-function)))
>> > |   ;; Original dired hook removes dired-isearch-search-filenames that
>> > |   ;; is needed outside isearch for lazy-highlighting in query-replace.
>> > |   (remove-hook 'isearch-mode-hook #'dired-isearch-filenames-setup t))
>> >
>> > Does that mean that after switching to wdired there is no way to switch
>> > between filename-only and normal search and replace?
>>
>> Correct.
>>
>> > Is the behavior determined when switching to wdired, or can it still
>> > be change while in wdired-mode?
>>
>> Since wdired is intended for editing filenames, we restrict
>> search/replace to filenames only.
>
> It's also possible to change permissions.  Anyway...

We have the option 'wdired-search-replace-filenames'
that can be customized to nil.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Tue, 10 Jun 2025 02:29:04 GMT) Full text and rfc822 format available.

Message #92 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Tue, 10 Jun 2025 04:29:45 +0200
Juri Linkov <juri <at> linkov.net> writes:

> > It's also possible to change permissions.  Anyway...
>
> We have the option 'wdired-search-replace-filenames'
> that can be customized to nil.

That's what I wanted to note: different types of isearch behavior can
potentially be useful in wdired, but it can be controlled _only_ via
customization, in contrast to dired.  It would be nice if toggling
`dired-isearch-filenames-mode' would still do something.  That it
does nothing can be unexpected.  Not the end of the world but
something that I personally don't like that much.


Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Tue, 10 Jun 2025 06:42:02 GMT) Full text and rfc822 format available.

Message #95 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Tue, 10 Jun 2025 09:28:34 +0300
>> We have the option 'wdired-search-replace-filenames'
>> that can be customized to nil.
>
> That's what I wanted to note: different types of isearch behavior can
> potentially be useful in wdired, but it can be controlled _only_ via
> customization, in contrast to dired.  It would be nice if toggling
> `dired-isearch-filenames-mode' would still do something.  That it
> does nothing can be unexpected.  Not the end of the world but
> something that I personally don't like that much.

So you propose to duplicate all related functions from dired to wdired?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Tue, 10 Jun 2025 13:34:03 GMT) Full text and rfc822 format available.

Message #98 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Tue, 10 Jun 2025 15:34:27 +0200
Juri Linkov <juri <at> linkov.net> writes:

> >> We have the option 'wdired-search-replace-filenames'
> >> that can be customized to nil.
> >
> > That's what I wanted to note: different types of isearch behavior can
> > potentially be useful in wdired, but it can be controlled _only_ via
> > customization, in contrast to dired.  It would be nice if toggling
> > `dired-isearch-filenames-mode' would still do something.  That it
> > does nothing can be unexpected.  Not the end of the world but
> > something that I personally don't like that much.
>
> So you propose to duplicate all related functions from dired to wdired?

Do we have to duplicate them?  Can't we just use the same set of
functions?

Looks like that's more or less already the, but with an additional
artificial different treatment of how the search and replace commands
work by default.  Or maybe I'm missing something.

Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Wed, 11 Jun 2025 17:19:02 GMT) Full text and rfc822 format available.

Message #101 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Wed, 11 Jun 2025 20:06:30 +0300
>> >> We have the option 'wdired-search-replace-filenames'
>> >> that can be customized to nil.
>> >
>> > That's what I wanted to note: different types of isearch behavior can
>> > potentially be useful in wdired, but it can be controlled _only_ via
>> > customization, in contrast to dired.  It would be nice if toggling
>> > `dired-isearch-filenames-mode' would still do something.  That it
>> > does nothing can be unexpected.  Not the end of the world but
>> > something that I personally don't like that much.
>>
>> So you propose to duplicate all related functions from dired to wdired?
>
> Do we have to duplicate them?  Can't we just use the same set of
> functions?
>
> Looks like that's more or less already the, but with an additional
> artificial different treatment of how the search and replace commands
> work by default.  Or maybe I'm missing something.

`wdired-search-replace-filenames` is required anyway
since it also sets `replace-search-function`.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#78520; Package emacs. (Thu, 12 Jun 2025 14:37:02 GMT) Full text and rfc822 format available.

Message #104 received at 78520 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 78520 <at> debbugs.gnu.org, spacibba <at> aol.com
Subject: Re: bug#78520: 31.0.50; Performance issue in dired+isearch with
 dired-isearch-filenames
Date: Thu, 12 Jun 2025 16:37:31 +0200
Juri Linkov <juri <at> linkov.net> writes:

> `wdired-search-replace-filenames` is required anyway
> since it also sets `replace-search-function`.

Hmm - but we could also try to do this when query-replace is invoked.
Similar to what we do when invoking isearch.

Do we handle wdired differently only because of this problem?

|  ;; Original dired hook removes dired-isearch-search-filenames that
|  ;; is needed outside isearch for lazy-highlighting in query-replace.
|  (remove-hook 'isearch-mode-hook #'dired-isearch-filenames-setup t)

Maybe we could solve this in some other way?  Then, isearch and
query-replace would just look whether filename only matching is enabled
or not, and do the according setup when any of them is called by the
user.


Michael.




This bug report was last modified 2 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.