GNU bug report logs - #59935
29.0.60; project-list-buffers is slow

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 59935 in the body.
You can then email your comments to 59935 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 03:49:47 +0200

X-Debbugs-Cc: juri <at> linkov.net

When you invoke 'C-x p C-b', there's a noticeable pause before the
buffer list is displayed. It's ~400ms over here when there are 46
non-hidden buffers in the running session.

What I think is happening, the predicate is called 46, which in turn 
calls (project-buffers pr) 46 times, and that ends up being 46 times 
slower than one might have anticipated.

Not sure what's the best fix here (especially in time for the release),
but if the FILTER-PREDICATE arg to list-buffers-noselect turned into a
factory function (e.g. FILTER-PREDICATE-MAKER), that would be one
solution.

Another would be to only leave the legacy codepath: it's not affected,
project-buffers is only called once there.

Curiously, though, it shows a different list of buffers. It also
includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
into that either way.

And the patch is this:

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 016dfdd5b4..ad49df0423 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -1342,16 +1342,13 @@ project-list-buffers
   (interactive "P")
   (let ((pr (project-current t)))
     (display-buffer
-     (if (version< emacs-version "29.0.50")
-         (let ((buf (list-buffers-noselect arg (project-buffers pr))))
-           (with-current-buffer buf
-             (setq-local revert-buffer-function
-                         (lambda (&rest _ignored)
-                           (list-buffers--refresh (project-buffers pr))
-                           (tabulated-list-print t))))
-           buf)
-       (list-buffers-noselect
-        arg nil (lambda (buf) (memq buf (project-buffers pr))))))))
+     (let ((buf (list-buffers-noselect arg (project-buffers pr))))
+       (with-current-buffer buf
+         (setq-local revert-buffer-function
+                     (lambda (&rest _ignored)
+                       (list-buffers--refresh (project-buffers pr))
+                       (tabulated-list-print t))))
+       buf))))
  (defcustom project-kill-buffer-conditions
   '(buffer-file-name    ; All file-visiting buffers are included.

Message #8 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: 59935 <at> debbugs.gnu.org
Cc: juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 04:03:11 +0200

On 10/12/2022 03:49, Dmitry Gutov wrote:
> Curiously, though, it shows a different list of buffers. It also
> includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
> into that either way.

The combined fix for both can be this:

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 016dfdd5b4..835ab07e50 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -1340,18 +1340,21 @@ project-list-buffers
 start with a space (which are for internal use).  With prefix argument
 ARG, show only buffers that are visiting files."
   (interactive "P")
-  (let ((pr (project-current t)))
+  (let* ((pr (project-current t))
+         (fetcher (lambda ()
+                    (cl-delete-if-not
+                     (lambda (b)
+                       (or (buffer-file-name b)
+                           (string-match-p "\\`[^ ]" (buffer-name b))))
+                     (project-buffers pr)))))
     (display-buffer
-     (if (version< emacs-version "29.0.50")
-         (let ((buf (list-buffers-noselect arg (project-buffers pr))))
-           (with-current-buffer buf
-             (setq-local revert-buffer-function
-                         (lambda (&rest _ignored)
-                           (list-buffers--refresh (project-buffers pr))
-                           (tabulated-list-print t))))
-           buf)
-       (list-buffers-noselect
-        arg nil (lambda (buf) (memq buf (project-buffers pr))))))))
+     (let ((buf (list-buffers-noselect arg (funcall fetcher))))
+       (with-current-buffer buf
+         (setq-local revert-buffer-function
+                     (lambda (&rest _ignored)
+                       (list-buffers--refresh (funcall fetcher))
+                       (tabulated-list-print t))))
+       buf))))

 (defcustom project-kill-buffer-conditions
   '(buffer-file-name    ; All file-visiting buffers are included.

Message #11 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 10:11:15 +0200

> Cc: juri <at> linkov.net
> Date: Sat, 10 Dec 2022 03:49:47 +0200
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> When you invoke 'C-x p C-b', there's a noticeable pause before the
> buffer list is displayed. It's ~400ms over here when there are 46
> non-hidden buffers in the running session.
> 
> What I think is happening, the predicate is called 46, which in turn 
> calls (project-buffers pr) 46 times, and that ends up being 46 times 
> slower than one might have anticipated.
> 
> Not sure what's the best fix here (especially in time for the release),
> but if the FILTER-PREDICATE arg to list-buffers-noselect turned into a
> factory function (e.g. FILTER-PREDICATE-MAKER), that would be one
> solution.
> 
> Another would be to only leave the legacy codepath: it's not affected,
> project-buffers is only called once there.
> 
> Curiously, though, it shows a different list of buffers. It also
> includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
> into that either way.

I'm not following.  Are you saying that the problem is that the
FILTER-PREDICATE argument to list-buffers-noselect calls
project-buffers for each buffer?  If so, I don't understand why a
trivial change to project-list-buffers could not call project-buffers
just once, and then reuse the value in the call to
list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
in the FILTER-PREDICATE argument, using memq.

If this is not the problem, then what is?

> And the patch is this:

Consequently, I don't understand the rationale for this patch, nor for
the combined one you posted later.  Please elaborate.

P.S. Btw, if you are not sure the multiple calls to project-buffers is
the reason for the slow operation, how about profiling it to be sure?
We may be barking up the wrong tree.

Message #14 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 12:47:38 +0200

On 10/12/2022 10:11, Eli Zaretskii wrote:
> I'm not following.  Are you saying that the problem is that the
> FILTER-PREDICATE argument to list-buffers-noselect calls
> project-buffers for each buffer?

Yes.

> If so, I don't understand why a
> trivial change to project-list-buffers could not call project-buffers
> just once, and then reuse the value in the call to
> list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
> in the FILTER-PREDICATE argument, using memq.

If we just do that, pressing 'g' will only show the buffers that 
belonged to the project when project-list-buffers was called. And not 
necessarily the *current* list of project buffers.

The previously attached patches solved that problem.

Message #17 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 13:12:14 +0200

> Date: Sat, 10 Dec 2022 12:47:38 +0200
> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> > If so, I don't understand why a
> > trivial change to project-list-buffers could not call project-buffers
> > just once, and then reuse the value in the call to
> > list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
> > in the FILTER-PREDICATE argument, using memq.
> 
> If we just do that, pressing 'g' will only show the buffers that 
> belonged to the project when project-list-buffers was called. And not 
> necessarily the *current* list of project buffers.
> 
> The previously attached patches solved that problem.

I don't understand: how is 'g' relevant?  You were talking about
'C-x p C-b' in your original message.

Message #20 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 13:37:37 +0200

On 10/12/2022 13:12, Eli Zaretskii wrote:
>> Date: Sat, 10 Dec 2022 12:47:38 +0200
>> Cc:59935 <at> debbugs.gnu.org,juri <at> linkov.net
>> From: Dmitry Gutov<dgutov <at> yandex.ru>
>>
>>> If so, I don't understand why a
>>> trivial change to project-list-buffers could not call project-buffers
>>> just once, and then reuse the value in the call to
>>> list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
>>> in the FILTER-PREDICATE argument, using memq.
>> If we just do that, pressing 'g' will only show the buffers that
>> belonged to the project when project-list-buffers was called. And not
>> necessarily the*current*  list of project buffers.
>>
>> The previously attached patches solved that problem.
> I don't understand: how is 'g' relevant?  You were talking about
> 'C-x p C-b' in your original message.

'C-x p C-b' creates a buffer which lists buffers. It also sets up 
revert-buffer-function in that buffer.

And pressing 'g' in that new buffer should supposedly refresh that list 
of buffers appropriately.

Message #23 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 16:33:59 +0200

> Date: Sat, 10 Dec 2022 13:37:37 +0200
> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> On 10/12/2022 13:12, Eli Zaretskii wrote:
> >> Date: Sat, 10 Dec 2022 12:47:38 +0200
> >> Cc:59935 <at> debbugs.gnu.org,juri <at> linkov.net
> >> From: Dmitry Gutov<dgutov <at> yandex.ru>
> >>
> >>> If so, I don't understand why a
> >>> trivial change to project-list-buffers could not call project-buffers
> >>> just once, and then reuse the value in the call to
> >>> list-buffers-noselect both as the BUFFER-LIST argument and (if needed)
> >>> in the FILTER-PREDICATE argument, using memq.
> >> If we just do that, pressing 'g' will only show the buffers that
> >> belonged to the project when project-list-buffers was called. And not
> >> necessarily the*current*  list of project buffers.
> >>
> >> The previously attached patches solved that problem.
> > I don't understand: how is 'g' relevant?  You were talking about
> > 'C-x p C-b' in your original message.
> 
> 'C-x p C-b' creates a buffer which lists buffers. It also sets up 
> revert-buffer-function in that buffer.
> 
> And pressing 'g' in that new buffer should supposedly refresh that list 
> of buffers appropriately.

OK, but I didn't suggest making any changes to revert-buffer-function
set up by "C-x p C-b".  You can still use process-buffers there, and
it shouldn't slow down "C-x p C-b" itself, since the call to
process-buffers that is part of revert-buffer-function is not supposed
to be evaluated when you set revert-buffer-function.  Right?  Or what
else am I missing?

Message #26 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 19:45:09 +0200

> Not sure what's the best fix here (especially in time for the release),
> but if the FILTER-PREDICATE arg to list-buffers-noselect turned into a
> factory function (e.g. FILTER-PREDICATE-MAKER), that would be one
> solution.

I see only one solution: to replace the argument FILTER-PREDICATE
with BUFFER-LIST-FUNCTION.

>> Curiously, though, it shows a different list of buffers. It also
>> includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
>> into that either way.
> +                    (cl-delete-if-not
> +                     (lambda (b)
> +                       (or (buffer-file-name b)
> +                           (string-match-p "\\`[^ ]" (buffer-name b))))
> +                     (project-buffers pr)))))

Please try to copy the exact logic from list-buffers--refresh:

			 (and (or (not (string= (substring name 0 1) " "))
                                  buffer-file-name)
			      (not (eq buffer (current-buffer)))
			      (or file (not Buffer-menu-files-only))

Message #29 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 21:19:30 +0200

On 10/12/2022 16:33, Eli Zaretskii wrote:
> OK, but I didn't suggest making any changes to revert-buffer-function
> set up by "C-x p C-b".  You can still use process-buffers there, and
> it shouldn't slow down "C-x p C-b" itself, since the call to
> process-buffers that is part of revert-buffer-function is not supposed
> to be evaluated when you set revert-buffer-function.  Right?  Or what
> else am I missing?

Then only the 'g' binding will remain slow. An improvement, of course.

But if we go this route, then we don't use the "new" 
list-buffers-noselect convention either, so we might as well do the full 
fix.

Message #32 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 21:22:20 +0200

On 10/12/2022 19:45, Juri Linkov wrote:
>> Not sure what's the best fix here (especially in time for the release),
>> but if the FILTER-PREDICATE arg to list-buffers-noselect turned into a
>> factory function (e.g. FILTER-PREDICATE-MAKER), that would be one
>> solution.
> 
> I see only one solution: to replace the argument FILTER-PREDICATE
> with BUFFER-LIST-FUNCTION.

Ah yeah, that should also work.

>>> Curiously, though, it shows a different list of buffers. It also
>>> includes "hidden" buffers - diff-syntax, Echo Area, etc. We should look
>>> into that either way.
>> +                    (cl-delete-if-not
>> +                     (lambda (b)
>> +                       (or (buffer-file-name b)
>> +                           (string-match-p "\\`[^ ]" (buffer-name b))))
>> +                     (project-buffers pr)))))
> 
> Please try to copy the exact logic from list-buffers--refresh:
> 
> 			 (and (or (not (string= (substring name 0 1) " "))
>                                    buffer-file-name)
> 			      (not (eq buffer (current-buffer)))
> 			      (or file (not Buffer-menu-files-only))

Do you have the time to finish the fix yourself? It was your feature, 
after all.

Message #35 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 21:42:20 +0200

> Date: Sat, 10 Dec 2022 21:19:30 +0200
> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> On 10/12/2022 16:33, Eli Zaretskii wrote:
> > OK, but I didn't suggest making any changes to revert-buffer-function
> > set up by "C-x p C-b".  You can still use process-buffers there, and
> > it shouldn't slow down "C-x p C-b" itself, since the call to
> > process-buffers that is part of revert-buffer-function is not supposed
> > to be evaluated when you set revert-buffer-function.  Right?  Or what
> > else am I missing?
> 
> Then only the 'g' binding will remain slow. An improvement, of course.

Why slow? it calls project-buffers only once.

Message #38 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 21:53:28 +0200

On 10/12/2022 21:42, Eli Zaretskii wrote:
>> Date: Sat, 10 Dec 2022 21:19:30 +0200
>> Cc:59935 <at> debbugs.gnu.org,juri <at> linkov.net
>> From: Dmitry Gutov<dgutov <at> yandex.ru>
>>
>> On 10/12/2022 16:33, Eli Zaretskii wrote:
>>> OK, but I didn't suggest making any changes to revert-buffer-function
>>> set up by "C-x p C-b".  You can still use process-buffers there, and
>>> it shouldn't slow down "C-x p C-b" itself, since the call to
>>> process-buffers that is part of revert-buffer-function is not supposed
>>> to be evaluated when you set revert-buffer-function.  Right?  Or what
>>> else am I missing?
>> Then only the 'g' binding will remain slow. An improvement, of course.
> Why slow? it calls project-buffers only once.

You're looking at the "legacy" code path. Where emacs-version is < 29.

In the "new" one revert-buffer-function uses the filter-predicate 
argument we pass to list-buffers-noselect. Which is

  (lambda (buf) (memq buf (project-buffers pr)))

Message #41 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 22:11:13 +0200

> Date: Sat, 10 Dec 2022 21:53:28 +0200
> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> On 10/12/2022 21:42, Eli Zaretskii wrote:
> >> Date: Sat, 10 Dec 2022 21:19:30 +0200
> >> Cc:59935 <at> debbugs.gnu.org,juri <at> linkov.net
> >> From: Dmitry Gutov<dgutov <at> yandex.ru>
> >>
> >> On 10/12/2022 16:33, Eli Zaretskii wrote:
> >>> OK, but I didn't suggest making any changes to revert-buffer-function
> >>> set up by "C-x p C-b".  You can still use process-buffers there, and
> >>> it shouldn't slow down "C-x p C-b" itself, since the call to
> >>> process-buffers that is part of revert-buffer-function is not supposed
> >>> to be evaluated when you set revert-buffer-function.  Right?  Or what
> >>> else am I missing?
> >> Then only the 'g' binding will remain slow. An improvement, of course.
> > Why slow? it calls project-buffers only once.
> 
> You're looking at the "legacy" code path. Where emacs-version is < 29.

I'm looking at the code you posted as the original one, which is
identical to what I see on the current emacs-29 branch.  That means
Emacs 29, unless I'm seriously confused.

> In the "new" one revert-buffer-function uses the filter-predicate 
> argument we pass to list-buffers-noselect. Which is
> 
>    (lambda (buf) (memq buf (project-buffers pr)))

I see this in emacs-29:

             (setq-local revert-buffer-function
                         (lambda (&rest _ignored)
                           (list-buffers--refresh (project-buffers pr))
                           (tabulated-list-print t))))

So I still don't understand your fears.

Message #44 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sat, 10 Dec 2022 22:23:01 +0200

On 10/12/2022 22:11, Eli Zaretskii wrote:
> I see this in emacs-29:
> 
>               (setq-local revert-buffer-function
>                           (lambda (&rest _ignored)
>                             (list-buffers--refresh (project-buffers pr))
>                             (tabulated-list-print t))))
> 
> So I still don't understand your fears.

That's the (version< emacs-version "29.0.50") branch.

Message #47 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 08:19:05 +0200

> Date: Sat, 10 Dec 2022 22:23:01 +0200
> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> On 10/12/2022 22:11, Eli Zaretskii wrote:
> > I see this in emacs-29:
> > 
> >               (setq-local revert-buffer-function
> >                           (lambda (&rest _ignored)
> >                             (list-buffers--refresh (project-buffers pr))
> >                             (tabulated-list-print t))))
> > 
> > So I still don't understand your fears.
> 
> That's the (version< emacs-version "29.0.50") branch.

Still unclear.  It almost looks like you don't _want_ me to
understand.  A detailed explanation with code fragments would be much
more effective.

Anyway, I only entered this discussion because you asked about making
some non-trivial changes on the release branch.  If that is no longer
an issue, I will gladly bow out of this.

Message #50 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 12:23:13 +0200

On 11/12/2022 08:19, Eli Zaretskii wrote:
>> Date: Sat, 10 Dec 2022 22:23:01 +0200
>> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
>> From: Dmitry Gutov <dgutov <at> yandex.ru>
>>
>> On 10/12/2022 22:11, Eli Zaretskii wrote:
>>> I see this in emacs-29:
>>>
>>>                (setq-local revert-buffer-function
>>>                            (lambda (&rest _ignored)
>>>                              (list-buffers--refresh (project-buffers pr))
>>>                              (tabulated-list-print t))))
>>>
>>> So I still don't understand your fears.
>>
>> That's the (version< emacs-version "29.0.50") branch.
> 
> Still unclear.  It almost looks like you don't _want_ me to
> understand.  A detailed explanation with code fragments would be much
> more effective.

I think I've explained everything, and we are now going in circles. Why 
not look at the definition yourself? It's 12 lines.

The function has two execution branches: one for the latest Emacs (at 
the moment), and one (the largest one) for older Emacsen.

The main problematic behavior (low performance) is exhibited by the 
"latest" branch, which looks like this:

  (let ((pr (project-current t)))
      (display-buffer
       (if (version< emacs-version "29.0.50")
           ;; ...
         (list-buffers-noselect
          arg nil (lambda (buf) (memq buf (project-buffers pr)))))))

> Anyway, I only entered this discussion because you asked about making
> some non-trivial changes on the release branch.  If that is no longer
> an issue, I will gladly bow out of this.

I expect Juri is going to argue for that later. For non-trivial changes 
in buff-menu.el, no less.

Message #53 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 12:54:48 +0200

> Date: Sun, 11 Dec 2022 12:23:13 +0200
> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> The main problematic behavior (low performance) is exhibited by the 
> "latest" branch, which looks like this:
> 
>    (let ((pr (project-current t)))
>        (display-buffer
>         (if (version< emacs-version "29.0.50")
>             ;; ...
>           (list-buffers-noselect
>            arg nil (lambda (buf) (memq buf (project-buffers pr)))))))

Yes, I see that.  I still don't understand: this can easily be changed
to call project-buffers just once.

> > Anyway, I only entered this discussion because you asked about making
> > some non-trivial changes on the release branch.  If that is no longer
> > an issue, I will gladly bow out of this.
> 
> I expect Juri is going to argue for that later. For non-trivial changes 
> in buff-menu.el, no less.

I will wait for his comments, then.

Message #56 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 18:32:22 +0200

On 11/12/2022 12:54, Eli Zaretskii wrote:
>> Date: Sun, 11 Dec 2022 12:23:13 +0200
>> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
>> From: Dmitry Gutov <dgutov <at> yandex.ru>
>>
>> The main problematic behavior (low performance) is exhibited by the
>> "latest" branch, which looks like this:
>>
>>     (let ((pr (project-current t)))
>>         (display-buffer
>>          (if (version< emacs-version "29.0.50")
>>              ;; ...
>>            (list-buffers-noselect
>>             arg nil (lambda (buf) (memq buf (project-buffers pr)))))))
> 
> Yes, I see that.  I still don't understand: this can easily be changed
> to call project-buffers just once.

Cue my very first response:

  If we just do that, pressing 'g' will only show the buffers that
  belonged to the project when project-list-buffers was called. And not
  necessarily the *current* list of project buffers.

Though I guess there might be ways to avoid this, like creating a cache 
which will be invalidated in post-command-hook, something like that.

>>> Anyway, I only entered this discussion because you asked about making
>>> some non-trivial changes on the release branch.  If that is no longer
>>> an issue, I will gladly bow out of this.
>>
>> I expect Juri is going to argue for that later. For non-trivial changes
>> in buff-menu.el, no less.
> 
> I will wait for his comments, then.

Let's.

Message #59 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 19:07:27 +0200

[Message part 1 (text/plain, inline)]

> Do you have the time to finish the fix yourself? It was your feature, after
> all.

Here is a complete patch tested for all possible cases:
- emacs 28/29
- with/without file arg
- with/without reverting

[Buffer-menu-buffer-list-function.patch (text/x-diff, inline)]

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 38d4fdad5fc..64af86558e4 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -1321,18 +1321,31 @@ project-list-buffers
 start with a space (which are for internal use).  With prefix argument
 ARG, show only buffers that are visiting files."
   (interactive "P")
-  (let ((pr (project-current t)))
+  (let* ((pr (project-current t))
+         (buffer-list-function
+          (lambda ()
+            (seq-filter
+             (lambda (buffer)
+               (let ((name (buffer-name buffer))
+                     (file (buffer-file-name buffer)))
+                 (and (or (not (string= (substring name 0 1) " "))
+                          file)
+                      (not (eq buffer (current-buffer)))
+                      (or file (not Buffer-menu-files-only)))))
+             (project-buffers pr)))))
     (display-buffer
      (if (version< emacs-version "29.0.50")
-         (let ((buf (list-buffers-noselect arg (project-buffers pr))))
+         (let ((buf (list-buffers-noselect
+                     arg (let ((Buffer-menu-files-only arg))
+                           (funcall buffer-list-function)))))
            (with-current-buffer buf
              (setq-local revert-buffer-function
                          (lambda (&rest _ignored)
-                           (list-buffers--refresh (project-buffers pr))
+                           (list-buffers--refresh
+                            (funcall buffer-list-function))
                            (tabulated-list-print t))))
            buf)
-       (list-buffers-noselect
-        arg nil (lambda (buf) (memq buf (project-buffers pr))))))))
+       (list-buffers-noselect arg buffer-list-function)))))
 
 (defcustom project-kill-buffer-conditions
   '(buffer-file-name    ; All file-visiting buffers are included.
diff --git a/lisp/buff-menu.el b/lisp/buff-menu.el
index 588fe599a46..cabf7cb8fc4 100644
--- a/lisp/buff-menu.el
+++ b/lisp/buff-menu.el
@@ -100,12 +100,8 @@ Buffer-menu-files-only
 This is set by the prefix argument to `buffer-menu' and related
 commands.")
 
-(defvar-local Buffer-menu-filter-predicate nil
-  "Function to filter out buffers in the buffer list.
-Buffers that don't satisfy the predicate will be skipped.
-The value should be a function of one argument; it will be
-called with the buffer.  If this function returns non-nil,
-then the buffer will be displayed in the buffer list.")
+(defvar-local Buffer-menu-buffer-list-function nil
+  "Function to return buffers for the buffer list.")
 
 (defvar-keymap Buffer-menu-mode-map
   :doc "Local keymap for `Buffer-menu-mode' buffers."
@@ -623,23 +619,23 @@ Buffer-menu-view-other-window
 ;;; Functions for populating the Buffer Menu.
 
 ;;;###autoload
-(defun list-buffers-noselect (&optional files-only buffer-list filter-predicate)
+(defun list-buffers-noselect (&optional files-only buffer-list)
   "Create and return a Buffer Menu buffer.
 This is called by `buffer-menu' and others as a subroutine.
 
 If FILES-ONLY is non-nil, show only file-visiting buffers.
-If BUFFER-LIST is non-nil, it should be a list of buffers; it
-means list those buffers and no others.
-If FILTER-PREDICATE is non-nil, it should be a function
-that filters out buffers from the list of buffers.
-See more at `Buffer-menu-filter-predicate'."
+If BUFFER-LIST is non-nil, it should be either a list of buffers
+or a function that returns a list of buffers; it means
+list those buffers and no others.
+See more at `Buffer-menu-buffer-list-function'."
   (let ((old-buffer (current-buffer))
 	(buffer (get-buffer-create "*Buffer List*")))
     (with-current-buffer buffer
       (Buffer-menu-mode)
       (setq Buffer-menu-files-only
 	    (and files-only (>= (prefix-numeric-value files-only) 0)))
-      (setq Buffer-menu-filter-predicate filter-predicate)
+      (when (functionp buffer-list)
+        (setq Buffer-menu-buffer-list-function buffer-list))
       (list-buffers--refresh buffer-list old-buffer)
       (tabulated-list-print))
     buffer))
@@ -661,13 +657,17 @@ list-buffers--refresh
         (marked-buffers (Buffer-menu-marked-buffers))
         (buffer-menu-buffer (current-buffer))
 	(show-non-file (not Buffer-menu-files-only))
-	(filter-predicate (and (functionp Buffer-menu-filter-predicate)
-			       Buffer-menu-filter-predicate))
 	entries name-width)
     ;; Collect info for each buffer we're interested in.
-    (dolist (buffer (or buffer-list
-			(buffer-list (if Buffer-menu-use-frame-buffer-list
-					 (selected-frame)))))
+    (dolist (buffer (cond
+                     ((functionp buffer-list)
+                      (funcall buffer-list))
+                     (buffer-list)
+                     (Buffer-menu-buffer-list-function
+                      (funcall Buffer-menu-buffer-list-function))
+                     (t (buffer-list
+                         (if Buffer-menu-use-frame-buffer-list
+                             (selected-frame))))))
       (with-current-buffer buffer
 	(let* ((name (buffer-name))
 	       (file buffer-file-name))
@@ -676,9 +676,7 @@ list-buffers--refresh
 			 (and (or (not (string= (substring name 0 1) " "))
                                   file)
 			      (not (eq buffer buffer-menu-buffer))
-			      (or file show-non-file)
-			      (or (not filter-predicate)
-				  (funcall filter-predicate buffer)))))
+			      (or file show-non-file))))
 	    (push (list buffer
 			(vector (cond
                                  ((eq buffer old-buffer) ".")

Message #62 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 19:49:09 +0200

> Cc: 59935 <at> debbugs.gnu.org
> From: Juri Linkov <juri <at> linkov.net>
> Date: Sun, 11 Dec 2022 19:07:27 +0200
> 
> > Do you have the time to finish the fix yourself? It was your feature, after
> > all.
> 
> Here is a complete patch tested for all possible cases:
> - emacs 28/29
> - with/without file arg
> - with/without reverting

Thanks, but I hope you don't intend to ask to install this on the
release branch.  And changing back the signature of
list-buffers-noselect is extremely problematic, even though we changed
it only for Emacs 29.

Please try to find a safer, more compatible solution.

Message #65 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 19:56:24 +0200

>> > Do you have the time to finish the fix yourself? It was your feature, after
>> > all.
>> 
>> Here is a complete patch tested for all possible cases:
>> - emacs 28/29
>> - with/without file arg
>> - with/without reverting
>
> Thanks, but I hope you don't intend to ask to install this on the
> release branch.  And changing back the signature of
> list-buffers-noselect is extremely problematic, even though we changed
> it only for Emacs 29.
>
> Please try to find a safer, more compatible solution.

This is the safest solution for the emacs-29 branch.

Message #68 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 20:08:59 +0200

> From: Juri Linkov <juri <at> linkov.net>
> Cc: dgutov <at> yandex.ru,  59935 <at> debbugs.gnu.org
> Date: Sun, 11 Dec 2022 19:56:24 +0200
> 
> >> > Do you have the time to finish the fix yourself? It was your feature, after
> >> > all.
> >> 
> >> Here is a complete patch tested for all possible cases:
> >> - emacs 28/29
> >> - with/without file arg
> >> - with/without reverting
> >
> > Thanks, but I hope you don't intend to ask to install this on the
> > release branch.  And changing back the signature of
> > list-buffers-noselect is extremely problematic, even though we changed
> > it only for Emacs 29.
> >
> > Please try to find a safer, more compatible solution.
> 
> This is the safest solution for the emacs-29 branch.

I don't see how it could be the safest.  For example, an almost
identical changeset that doesn't change the signature of
list-buffers-noselect is definitely safer.  And likewise with the
change of list-buffers--refresh -- why do we have to do that?

Message #71 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>, Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 20:12:25 +0200

On 11/12/2022 19:49, Eli Zaretskii wrote:
> Thanks, but I hope you don't intend to ask to install this on the
> release branch.  And changing back the signature of
> list-buffers-noselect is extremely problematic, even though we changed
> it only for Emacs 29.

The problem with *not* changing it, is we don't seem to be able to use 
the new calling convention to implement project-list-buffers working 
without this performance problem.

We could just use the old (private) interface, but then the new calling 
convention will be left without any users at all. We extended it in 
order to implement this feature anyway.

Message #74 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>, Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 20:13:25 +0200

On 11/12/2022 20:08, Eli Zaretskii wrote:
> I don't see how it could be the safest.  For example, an almost
> identical changeset that doesn't change the signature of
> list-buffers-noselect is definitely safer.  And likewise with the
> change of list-buffers--refresh -- why do we have to do that?

How do we avoid changing the signature?

Message #77 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 20:17:30 +0200

> Date: Sun, 11 Dec 2022 20:12:25 +0200
> Cc: 59935 <at> debbugs.gnu.org
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> On 11/12/2022 19:49, Eli Zaretskii wrote:
> > Thanks, but I hope you don't intend to ask to install this on the
> > release branch.  And changing back the signature of
> > list-buffers-noselect is extremely problematic, even though we changed
> > it only for Emacs 29.
> 
> The problem with *not* changing it, is we don't seem to be able to use 
> the new calling convention to implement project-list-buffers working 
> without this performance problem.
> 
> We could just use the old (private) interface, but then the new calling 
> convention will be left without any users at all. We extended it in 
> order to implement this feature anyway.

I don't think I follow.  Why not leave alone that additional PREDICATE
argument of list-buffers-noselect, and add a feature where the
BUFFER-LIST argument could be a function?  That's be a compatible
change.  Just keep the PREDICATE argument and the code which supports
it, and document that if PREDICATE is a function, BUFFER-LIST cannot
be a function.

Message #80 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 20:35:24 +0200

On 11/12/2022 20:17, Eli Zaretskii wrote:
> Why not leave alone that additional PREDICATE
> argument of list-buffers-noselect, and add a feature where the
> BUFFER-LIST argument could be a function?  That's be a compatible
> change.  Just keep the PREDICATE argument and the code which supports
> it, and document that if PREDICATE is a function, BUFFER-LIST cannot
> be a function.

While that should work, that change would *also* change the signature. 
Though in a safer way, yes.

Are you worried about some out-of-tree clients that started using the 
new feature in the meantime?

Otherwise, this change could be split into two steps:

- Rollback that signature to pre-commit 125b5684c (i.e. revert its part 
in buff-menu.el). This is something we usually considered to be safe.

- Add new change that allows BUFFER-LIST to be a function.

Message #83 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 20:37:51 +0200

On 11/12/2022 19:07, Juri Linkov wrote:
> +(defvar-local Buffer-menu-buffer-list-function nil
> +  "Function to return buffers for the buffer list.")
> 
>         (setq Buffer-menu-files-only
>   	    (and files-only (>= (prefix-numeric-value files-only) 0)))
> -      (setq Buffer-menu-filter-predicate filter-predicate)
> +      (when (functionp buffer-list)
> +        (setq Buffer-menu-buffer-list-function buffer-list))

Here's an idea: when 'list-buffers-noselect' received a plain list of 
buffers in its BUFFER-LIST argument, it doesn't save that anywhere.

That seems like a bug, doesn't it? That hitting 'g' in such a 
buffer-list buffer resets its contents to all buffer (except hidden, etc).

So it probably makes sense to save it as well.

Long story short, I suggest to name the new variable 
Buffer-menu-buffer-list, and save the value of the BUFFER-LIST argument 
to it no matter what. And, likewise, use it. But when the value is a 
function, call it to obtain the actual list.

One side-effect of this, though, is that the BUFFER-LIST argument to 
list-buffers--refresh will have no purpose anymore.

Message #86 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 21:00:25 +0200

> Date: Sun, 11 Dec 2022 20:35:24 +0200
> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> On 11/12/2022 20:17, Eli Zaretskii wrote:
> > Why not leave alone that additional PREDICATE
> > argument of list-buffers-noselect, and add a feature where the
> > BUFFER-LIST argument could be a function?  That's be a compatible
> > change.  Just keep the PREDICATE argument and the code which supports
> > it, and document that if PREDICATE is a function, BUFFER-LIST cannot
> > be a function.
> 
> While that should work, that change would *also* change the signature. 
> Though in a safer way, yes.

Indeed.

> Are you worried about some out-of-tree clients that started using the 
> new feature in the meantime?

Yes.  In particular, we never change signatures after the branch point
of a release branch -- the branch point is a de-facto code-freeze
point for the release branch.

> Otherwise, this change could be split into two steps:
> 
> - Rollback that signature to pre-commit 125b5684c (i.e. revert its part 
> in buff-menu.el). This is something we usually considered to be safe.
> 
> - Add new change that allows BUFFER-LIST to be a function.

We can do it safer, so I'd prefer that.

Message #89 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 21:41:40 +0200

On 11/12/2022 21:00, Eli Zaretskii wrote:
>> Otherwise, this change could be split into two steps:
>>
>> - Rollback that signature to pre-commit 125b5684c (i.e. revert its part
>> in buff-menu.el). This is something we usually considered to be safe.
>>
>> - Add new change that allows BUFFER-LIST to be a function.
> We can do it safer, so I'd prefer that.

I don't like that idea because we'll leave an unproven new feature in 
the codebase, without any known callers or requestors. That's just extra 
complexity, with all associated long-term costs.

But you're the boss. It's better to fix this than not, obviously.

Message #92 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Sun, 11 Dec 2022 22:42:08 +0200

> Date: Sun, 11 Dec 2022 21:41:40 +0200
> Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> 
> On 11/12/2022 21:00, Eli Zaretskii wrote:
> >> Otherwise, this change could be split into two steps:
> >>
> >> - Rollback that signature to pre-commit 125b5684c (i.e. revert its part
> >> in buff-menu.el). This is something we usually considered to be safe.
> >>
> >> - Add new change that allows BUFFER-LIST to be a function.
> > We can do it safer, so I'd prefer that.
> 
> I don't like that idea because we'll leave an unproven new feature in 
> the codebase, without any known callers or requestors.

Neither do I.  But this is a price to pay for coming this late with
changes in interfaces that are needed to make commands faster, and we
cannot wait.  If this came up half a year ago, we wouldn't be having
this conversation.

The schedule for releasing Emacs 29.1 is very ambitious, from where I
stand, and I don't want any risks we can avoid.

Message #95 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Jean Louis <bugs <at> gnu.support>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: juri <at> linkov.net, 59935 <at> debbugs.gnu.org, Dmitry Gutov <dgutov <at> yandex.ru>
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 13:36:48 +0300

If I may add to this, that project-list-buffers also incidentally
invokes re-connection of any Tramp buffers.

--
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Message #98 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Jean Louis <bugs <at> gnu.support>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 59935 <at> debbugs.gnu.org,
 Dmitry Gutov <dgutov <at> yandex.ru>
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 19:12:17 +0200

> If I may add to this, that project-list-buffers also incidentally
> invokes re-connection of any Tramp buffers.

Does list-buffers invoke Tramp re-connection as well?

Message #101 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, Dmitry Gutov <dgutov <at> yandex.ru>
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 19:16:22 +0200

>> I don't like that idea because we'll leave an unproven new feature in
>> the codebase, without any known callers or requestors.
>
> Neither do I.  But this is a price to pay for coming this late with
> changes in interfaces that are needed to make commands faster, and we
> cannot wait.  If this came up half a year ago, we wouldn't be having
> this conversation.

Ok, then here is the patch for emacs-29:

diff --git a/etc/NEWS b/etc/NEWS
index 233ef3f5729..2e5bb40c972 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -3437,6 +3437,10 @@ The following generalized variables have been made obsolete:
 
 * Lisp Changes in Emacs 29.1
 
+---
+** The argument FILTER-PREDICATE of 'list-buffers-noselect' is obsolete now.
+It will be removed in next versions.
+
 +++
 ** Interpreted closures are "safe for space".
 As was already the case for byte-compiled closures, instead of capturing

Message #104 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 19:27:57 +0200

> From: Juri Linkov <juri <at> linkov.net>
> Cc: Dmitry Gutov <dgutov <at> yandex.ru>,  59935 <at> debbugs.gnu.org
> Date: Mon, 12 Dec 2022 19:16:22 +0200
> 
> >> I don't like that idea because we'll leave an unproven new feature in
> >> the codebase, without any known callers or requestors.
> >
> > Neither do I.  But this is a price to pay for coming this late with
> > changes in interfaces that are needed to make commands faster, and we
> > cannot wait.  If this came up half a year ago, we wouldn't be having
> > this conversation.
> 
> Ok, then here is the patch for emacs-29:
> 
> diff --git a/etc/NEWS b/etc/NEWS
> index 233ef3f5729..2e5bb40c972 100644
> --- a/etc/NEWS
> +++ b/etc/NEWS
> @@ -3437,6 +3437,10 @@ The following generalized variables have been made obsolete:
>  
>  * Lisp Changes in Emacs 29.1
>  
> +---
> +** The argument FILTER-PREDICATE of 'list-buffers-noselect' is obsolete now.
> +It will be removed in next versions.
> +
>  +++
>  ** Interpreted closures are "safe for space".
>  As was already the case for byte-compiled closures, instead of capturing

I thought we wanted to make the command faster in Emacs 29.  I very
much doubt that deprecation of an argument will have that effect.

I'm also not sure I agree with obsoleting and removing that argument.
We don't have enough justification for that, not yet.  That it was
introduced for a particular use case, and that use case no longer uses
it, doesn't mean there won't be others.  After all, we don't introduce
additional arguments unless we think it will be useful in more than
one case.

Message #107 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 19:51:48 +0200

>> +** The argument FILTER-PREDICATE of 'list-buffers-noselect' is obsolete now.
>> +It will be removed in next versions.
>
> I thought we wanted to make the command faster in Emacs 29.  I very
> much doubt that deprecation of an argument will have that effect.
>
> I'm also not sure I agree with obsoleting and removing that argument.
> We don't have enough justification for that, not yet.  That it was
> introduced for a particular use case, and that use case no longer uses
> it, doesn't mean there won't be others.  After all, we don't introduce
> additional arguments unless we think it will be useful in more than
> one case.

Having both FILTER-PREDICATE and BUFFER-LIST-FUNCTION is too problematic.
Which one should take priority over another when both specified?

Message #110 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 20:10:32 +0200

> From: Juri Linkov <juri <at> linkov.net>
> Cc: dgutov <at> yandex.ru,  59935 <at> debbugs.gnu.org
> Date: Mon, 12 Dec 2022 19:51:48 +0200
> 
> >> +** The argument FILTER-PREDICATE of 'list-buffers-noselect' is obsolete now.
> >> +It will be removed in next versions.
> >
> > I thought we wanted to make the command faster in Emacs 29.  I very
> > much doubt that deprecation of an argument will have that effect.
> >
> > I'm also not sure I agree with obsoleting and removing that argument.
> > We don't have enough justification for that, not yet.  That it was
> > introduced for a particular use case, and that use case no longer uses
> > it, doesn't mean there won't be others.  After all, we don't introduce
> > additional arguments unless we think it will be useful in more than
> > one case.
> 
> Having both FILTER-PREDICATE and BUFFER-LIST-FUNCTION is too problematic.
> Which one should take priority over another when both specified?

I thought I answered that in my original proposal:

> Why not leave alone that additional PREDICATE argument of
> list-buffers-noselect, and add a feature where the BUFFER-LIST
> argument could be a function?  That's be a compatible change.  Just
> keep the PREDICATE argument and the code which supports it, and
> document that if PREDICATE is a function, BUFFER-LIST cannot be a
> function.

Note the last sentence.  Doesn't it resolve the difficulty in a
reasonable way?

Message #113 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 20:14:46 +0200

>> Why not leave alone that additional PREDICATE argument of
>> list-buffers-noselect, and add a feature where the BUFFER-LIST
>> argument could be a function?  That's be a compatible change.  Just
>> keep the PREDICATE argument and the code which supports it, and
>> document that if PREDICATE is a function, BUFFER-LIST cannot be a
>> function.
>
> Note the last sentence.  Doesn't it resolve the difficulty in a
> reasonable way?

Then support for a function in BUFFER-LIST should be implemented
in emacs-29?  (while keeping the recently added PREDICATE argument)

Message #116 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 20:22:28 +0200

> From: Juri Linkov <juri <at> linkov.net>
> Cc: dgutov <at> yandex.ru,  59935 <at> debbugs.gnu.org
> Date: Mon, 12 Dec 2022 20:14:46 +0200
> 
> >> Why not leave alone that additional PREDICATE argument of
> >> list-buffers-noselect, and add a feature where the BUFFER-LIST
> >> argument could be a function?  That's be a compatible change.  Just
> >> keep the PREDICATE argument and the code which supports it, and
> >> document that if PREDICATE is a function, BUFFER-LIST cannot be a
> >> function.
> >
> > Note the last sentence.  Doesn't it resolve the difficulty in a
> > reasonable way?
> 
> Then support for a function in BUFFER-LIST should be implemented
> in emacs-29?  (while keeping the recently added PREDICATE argument)

Yes.  Why not?  It cannot possibly hurt any existing callers, because
they won't use a function.  Right?

Message #119 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Jean Louis <bugs <at> gnu.support>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Mon, 12 Dec 2022 21:58:46 +0200

On 12/12/2022 12:36, Jean Louis wrote:
> If I may add to this, that project-list-buffers also incidentally
> invokes re-connection of any Tramp buffers.

Which part of it does? The project-buffers call, or some other step?

Message #122 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Jean Louis <bugs <at> gnu.support>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Tue, 13 Dec 2022 06:10:16 +0300

* Dmitry Gutov <dgutov <at> yandex.ru> [2022-12-12 22:59]:
> On 12/12/2022 12:36, Jean Louis wrote:
> > If I may add to this, that project-list-buffers also incidentally
> > invokes re-connection of any Tramp buffers.
> 
> Which part of it does? The project-buffers call, or some other step?

I don't know which part. All I can see is message below:

Tramp: Opening connection nil for root <at> my.localdomain.localhost using sudo...done

Because I was in sudo buffer (my.localdomain.localhost has been
changed)


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Message #125 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Jean Louis <bugs <at> gnu.support>
To: Juri Linkov <juri <at> linkov.net>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 59935 <at> debbugs.gnu.org,
 Dmitry Gutov <dgutov <at> yandex.ru>
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Tue, 13 Dec 2022 06:10:41 +0300

* Juri Linkov <juri <at> linkov.net> [2022-12-12 20:20]:
> > If I may add to this, that project-list-buffers also incidentally
> > invokes re-connection of any Tramp buffers.
> 
> Does list-buffers invoke Tramp re-connection as well?

I have never observed it.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Message #128 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Jean Louis <bugs <at> gnu.support>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Tue, 13 Dec 2022 17:29:50 +0200

On 13/12/2022 05:10, Jean Louis wrote:
> * Dmitry Gutov<dgutov <at> yandex.ru>  [2022-12-12 22:59]:
>> On 12/12/2022 12:36, Jean Louis wrote:
>>> If I may add to this, that project-list-buffers also incidentally
>>> invokes re-connection of any Tramp buffers.
>> Which part of it does? The project-buffers call, or some other step?
> I don't know which part. All I can see is message below:

You can find out by stepping through project-list-buffers with edebug.

Message #131 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 59935 <at> debbugs.gnu.org, dgutov <at> yandex.ru
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Tue, 13 Dec 2022 19:49:39 +0200

close 59935 29.0.60
thanks

>> Then support for a function in BUFFER-LIST should be implemented
>> in emacs-29?  (while keeping the recently added PREDICATE argument)
>
> Yes.  Why not?  It cannot possibly hurt any existing callers, because
> they won't use a function.  Right?

So now pushed to emacs-29 and closed.

Message #134 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 59935 <at> debbugs.gnu.org
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Tue, 13 Dec 2022 19:49:47 +0200

> Here's an idea: when 'list-buffers-noselect' received a plain list of
> buffers in its BUFFER-LIST argument, it doesn't save that anywhere.
>
> That seems like a bug, doesn't it? That hitting 'g' in such a buffer-list
> buffer resets its contents to all buffer (except hidden, etc).
>
> So it probably makes sense to save it as well.
>
> Long story short, I suggest to name the new variable
> Buffer-menu-buffer-list, and save the value of the BUFFER-LIST argument to
> it no matter what. And, likewise, use it. But when the value is a function,
> call it to obtain the actual list.

Thanks for the idea.  Implemented in the commit a99d0e7e6c9.

> One side-effect of this, though, is that the BUFFER-LIST argument to
> list-buffers--refresh will have no purpose anymore.

I'm not sure about removing the old argument BUFFER-LIST from
list-buffers--refresh, even though it's an internal function.

It also has another argument OLD-BUFFER that could be turned later
into buffer-local, to keep the current buffer marker after revert.

Message #139 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Juri Linkov <juri <at> linkov.net>
Cc: 59935 <at> debbugs.gnu.org
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Tue, 13 Dec 2022 20:51:30 +0200

On 13/12/2022 19:49, Juri Linkov wrote:
>> Here's an idea: when 'list-buffers-noselect' received a plain list of
>> buffers in its BUFFER-LIST argument, it doesn't save that anywhere.
>>
>> That seems like a bug, doesn't it? That hitting 'g' in such a buffer-list
>> buffer resets its contents to all buffer (except hidden, etc).
>>
>> So it probably makes sense to save it as well.
>>
>> Long story short, I suggest to name the new variable
>> Buffer-menu-buffer-list, and save the value of the BUFFER-LIST argument to
>> it no matter what. And, likewise, use it. But when the value is a function,
>> call it to obtain the actual list.
> 
> Thanks for the idea.  Implemented in the commit a99d0e7e6c9.

Thank you.

>> One side-effect of this, though, is that the BUFFER-LIST argument to
>> list-buffers--refresh will have no purpose anymore.
> 
> I'm not sure about removing the old argument BUFFER-LIST from
> list-buffers--refresh, even though it's an internal function.

Maybe sometime later on master.

> It also has another argument OLD-BUFFER that could be turned later
> into buffer-local, to keep the current buffer marker after revert.

Why not. Although at that point which of the buffers was current 
previously doesn't seem too important.

Message #142 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Jean Louis <bugs <at> gnu.support>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Tue, 13 Dec 2022 22:30:59 +0300

* Dmitry Gutov <dgutov <at> yandex.ru> [2022-12-13 18:30]:
> On 13/12/2022 05:10, Jean Louis wrote:
> > * Dmitry Gutov<dgutov <at> yandex.ru>  [2022-12-12 22:59]:
> > > On 12/12/2022 12:36, Jean Louis wrote:
> > > > If I may add to this, that project-list-buffers also incidentally
> > > > invokes re-connection of any Tramp buffers.
> > > Which part of it does? The project-buffers call, or some other step?
> > I don't know which part. All I can see is message below:
> 
> You can find out by stepping through project-list-buffers with edebug.

It iterates over results of list-buffers-noselect and among many
buffers finds those Tramp buffers. But why those Tramp buffers start
re-connecting I do not know.

For me this description below is not what that function does, as that
function seem not to select properly. What are project buffers? Are
they not files which are in directories specified as project? It seems
that function is iterating over buffers not necessary to iterate.

project-list-buffers is an autoloaded interactive Lisp closure in
‘project.el’.

It is bound to C-x p C-b.

(project-list-buffers &optional ARG)

Display a list of project buffers.
The list is displayed in a buffer named "*Buffer List*".

By default, all project buffers are listed except those whose names
start with a space (which are for internal use).  With prefix argument
ARG, show only buffers that are visiting files.



-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Message #145 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Jean Louis <bugs <at> gnu.support>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Tue, 13 Dec 2022 22:31:44 +0200

On 13/12/2022 21:30, Jean Louis wrote:
> * Dmitry Gutov <dgutov <at> yandex.ru> [2022-12-13 18:30]:
>> On 13/12/2022 05:10, Jean Louis wrote:
>>> * Dmitry Gutov<dgutov <at> yandex.ru>  [2022-12-12 22:59]:
>>>> On 12/12/2022 12:36, Jean Louis wrote:
>>>>> If I may add to this, that project-list-buffers also incidentally
>>>>> invokes re-connection of any Tramp buffers.
>>>> Which part of it does? The project-buffers call, or some other step?
>>> I don't know which part. All I can see is message below:
>>
>> You can find out by stepping through project-list-buffers with edebug.
> 
> It iterates over results of list-buffers-noselect and among many
> buffers finds those Tramp buffers. But why those Tramp buffers start
> re-connecting I do not know.

So 'M-x list-buffers' or 'C-x C-b' have the same effect? This is not 
specific to project-list-buffers?

> For me this description below is not what that function does, as that
> function seem not to select properly. What are project buffers? Are
> they not files which are in directories specified as project? It seems
> that function is iterating over buffers not necessary to iterate.

The answer to what buffers are project buffers is encoded in each 
project backend's implementation of 'project-buffers'.

The default considers non-file buffers as well.

Message #148 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Jean Louis <bugs <at> gnu.support>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Thu, 15 Dec 2022 17:58:06 +0300

* Dmitry Gutov <dgutov <at> yandex.ru> [2022-12-13 23:32]:
> > It iterates over results of list-buffers-noselect and among many
> > buffers finds those Tramp buffers. But why those Tramp buffers start
> > re-connecting I do not know.
> 
> So 'M-x list-buffers' or 'C-x C-b' have the same effect? This is not
> specific to project-list-buffers?

Not, it is specific to project-list-buffers and not to list-buffers.

> > For me this description below is not what that function does, as that
> > function seem not to select properly. What are project buffers? Are
> > they not files which are in directories specified as project? It seems
> > that function is iterating over buffers not necessary to iterate.
> 
> The answer to what buffers are project buffers is encoded in each project
> backend's implementation of 'project-buffers'.
> 
> The default considers non-file buffers as well.

How do I define non-default?

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Message #151 received at 59935 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Jean Louis <bugs <at> gnu.support>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 59935 <at> debbugs.gnu.org, juri <at> linkov.net
Subject: Re: bug#59935: 29.0.60; project-list-buffers is slow
Date: Thu, 15 Dec 2022 17:12:30 +0200

On 15/12/2022 16:58, Jean Louis wrote:
> * Dmitry Gutov <dgutov <at> yandex.ru> [2022-12-13 23:32]:
>>> It iterates over results of list-buffers-noselect and among many
>>> buffers finds those Tramp buffers. But why those Tramp buffers start
>>> re-connecting I do not know.
>>
>> So 'M-x list-buffers' or 'C-x C-b' have the same effect? This is not
>> specific to project-list-buffers?
> 
> Not, it is specific to project-list-buffers and not to list-buffers.

Then could you explain it in more detail? When you say "It iterates over 
results of list-buffers-noselect", what is "it"? Surely not 
project-list-buffers itself: list-buffers-noselect returns a buffer, not 
a list anyway.

>>> For me this description below is not what that function does, as that
>>> function seem not to select properly. What are project buffers? Are
>>> they not files which are in directories specified as project? It seems
>>> that function is iterating over buffers not necessary to iterate.
>>
>> The answer to what buffers are project buffers is encoded in each project
>> backend's implementation of 'project-buffers'.
>>
>> The default considers non-file buffers as well.
> 
> How do I define non-default?

By either redefinining project-vc's project-buffers method, or writing 
your own entire backend.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.