GNU bug report logs - #73117
30.0.90; Imenu missing entries when flattening by group

Previous Next

To reply to this bug, email your comments to 73117 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.90; Imenu missing entries when flattening by group
Date: Sat, 7 Sep 2024 21:41:43 -0400

This issue appears to be similar to the issue reported in 70846, but
this is specifically regarding when Imenu is configured to flatten
into "groups" (as opposed to "annotation" as was reported there).
When "imenu-flatten" is set to "group", I see an issue where nested
entries, with the same name but belonging to different parents, aren't
all displayed.

I've included an example below (based on the example menu
configuration described in 70846).  This example cycles through
flattening based on "index", "group" and "annotation" with the example
menu configuration.

For "prefix" and "annotation" configurations, it appears to work
correctly, as pressing "M-<down>" when the menu prompt is displayed, I
can see both entries identified in the "*Completions*" buffer.

However, when I do this with "imenu-flatten" set to "group" and press
"M-<down>" to display the completions window, the window indicates "2
possible completions" but only one is actually displayed and
selectable (i.e., the one under "Bar").  The menu entry "Foo" under
"Baz" is not displayed at all and it appears there is no way to select
it.

```
;; begin
(progn
  (require 'imenu)
  (dolist (flatten '(prefix group annotation))
    (setq imenu-flatten flatten)

    (imenu-choose-buffer-index (format "(%s) Index item: " flatten)
                               `(("Bar" . (("Foo" . ,(point-min-marker))))
                                 ("Baz" . (("Foo" . ,(point-max-marker))))))))
;; end
```

Message #8 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Troy Brown <brownts <at> troybrown.dev>
Cc: 73117 <at> debbugs.gnu.org
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sun, 08 Sep 2024 09:53:16 +0300

> This issue appears to be similar to the issue reported in 70846, but
> this is specifically regarding when Imenu is configured to flatten
> into "groups" (as opposed to "annotation" as was reported there).
> When "imenu-flatten" is set to "group", I see an issue where nested
> entries, with the same name but belonging to different parents, aren't
> all displayed.
>
> I've included an example below (based on the example menu
> configuration described in 70846).  This example cycles through
> flattening based on "index", "group" and "annotation" with the example
> menu configuration.
>
> For "prefix" and "annotation" configurations, it appears to work
> correctly, as pressing "M-<down>" when the menu prompt is displayed, I
> can see both entries identified in the "*Completions*" buffer.
>
> However, when I do this with "imenu-flatten" set to "group" and press
> "M-<down>" to display the completions window, the window indicates "2
> possible completions" but only one is actually displayed and
> selectable (i.e., the one under "Bar").  The menu entry "Foo" under
> "Baz" is not displayed at all and it appears there is no way to select
> it.
>
> ```
> ;; begin
> (progn
>   (require 'imenu)
>   (dolist (flatten '(prefix group annotation))
>     (setq imenu-flatten flatten)
>
>     (imenu-choose-buffer-index (format "(%s) Index item: " flatten)
>                                `(("Bar" . (("Foo" . ,(point-min-marker))))
>                                  ("Baz" . (("Foo" . ,(point-max-marker))))))))
> ;; end
> ```

Sorry for leaving out of documentation an unapparent mention
of `completions-group`.  We are discussing this currently at
https://mail.gnu.org/archive/html/emacs-devel/2024-08/msg00241.html
So a prerequisite would be to use `(setopt completions-group t)`.
But currently this should be mentioned in the docstring.

Also in the same discussion we came to conclusion that
`M-<down>` can't be used to select imenu items for
`annotation` and `group`.  So this limitation was
documented recently in the docstring of `imenu-flatten`:

  @@ -158,6 +158,9 @@ imenu-flatten
   with a suffix that is the section name to which it belongs.
   If the value is `group', split completion candidates into groups
   according to the sections.
  +Since the values `annotation' and `group' rely on text properties,
  +you can use them only by selecting candidates from the completions
  +buffer, not by typing in the minibuffer.

Otherwise, `group` should work nicely when using `<down>`
with `minibuffer-visible-completions`.

Message #11 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: Juri Linkov <juri <at> linkov.net>
Cc: 73117 <at> debbugs.gnu.org
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sun, 8 Sep 2024 10:19:26 -0400

On Sun, Sep 8, 2024 at 2:57 AM Juri Linkov <juri <at> linkov.net> wrote:
>
> > ```
> > ;; begin
> > (progn
> >   (require 'imenu)
> >   (dolist (flatten '(prefix group annotation))
> >     (setq imenu-flatten flatten)
> >
> >     (imenu-choose-buffer-index (format "(%s) Index item: " flatten)
> >                                `(("Bar" . (("Foo" . ,(point-min-marker))))
> >                                  ("Baz" . (("Foo" . ,(point-max-marker))))))))
> > ;; end
> > ```
>
> Sorry for leaving out of documentation an unapparent mention
> of `completions-group`.  We are discussing this currently at
> https://mail.gnu.org/archive/html/emacs-devel/2024-08/msg00241.html
> So a prerequisite would be to use `(setopt completions-group t)`.
> But currently this should be mentioned in the docstring.
>
> Also in the same discussion we came to conclusion that
> `M-<down>` can't be used to select imenu items for
> `annotation` and `group`.  So this limitation was
> documented recently in the docstring of `imenu-flatten`:
>
>   @@ -158,6 +158,9 @@ imenu-flatten
>    with a suffix that is the section name to which it belongs.
>    If the value is `group', split completion candidates into groups
>    according to the sections.
>   +Since the values `annotation' and `group' rely on text properties,
>   +you can use them only by selecting candidates from the completions
>   +buffer, not by typing in the minibuffer.
>
> Otherwise, `group` should work nicely when using `<down>`
> with `minibuffer-visible-completions`.

Hi Juri,

I did see the text in the diff above.  I assumed that was indicating
that you need to use the completions buffer to distinguish between the
identically named items (although after re-reading it, maybe it
affects the selection too).  I was using M-<down> to pop-up the
completions buffer so that I could see those distinctions.  Maybe
using `(setopt completion-auto-help t)` and hitting TAB is a better
method.  Either way, I'm just trying to display the completions
buffer.  This does work for the "annotations" setting as I see both
entries with the suffix annotation in the completions buffer.  Maybe
there is a different issue using M-<down> for the selection, but my
issue is that I can't get the completions buffer to even display all
of the items.

Just to be clear I'm running "emacs -Q" on the latest in the emacs-30
branch as of yesterday.  I ran the code snippet I posted in the
original email by running `eval-last-sexp` in the scratch buffer.

I did try adding your suggested settings to the above code snippet,
but they did not seem to improve anything with regards to what shows
up in the completions buffer:
```
(setopt completions-group t)
(setopt minibuffer-visible-completions t)
```

The following is what I see in the "*Completions*" buffer for all
three settings of `imenu-flatten` (i.e., "prefix", "group" and
"annotation").  Notice that the second output (which is the "group"
setting) only shows a single entry, the one for Bar/Foo.  The "Bar"
group is displayed and the "Foo" entry under it is also displayed, but
the "Baz" group is completely missing (even though it says "2 possible
completions").  This is the issue I'm having.

--8<---------------cut here---------------start------------->8---
Click or type RET on a completion to select it.
Type <right>, <left>, <down>, <up> to move point between completions.

2 possible completions:
Bar:Foo
Baz:Foo
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
Click or type RET on a completion to select it.
Type <right>, <left>, <down>, <up> to move point between completions.

2 possible completions:
     Bar
Foo
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
Click or type RET on a completion to select it.
Type <right>, <left>, <down>, <up> to move point between completions.

2 possible completions:
Foo (Bar)
Foo (Baz)
--8<---------------cut here---------------end--------------->8---

Message #14 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Troy Brown <brownts <at> troybrown.dev>
Cc: 73117 <at> debbugs.gnu.org, Daniel Mendler <mail <at> daniel-mendler.de>
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sun, 08 Sep 2024 19:35:09 +0300

> Maybe using `(setopt completion-auto-help t)`
> and hitting TAB is a better method.

Or just using `(setopt imenu-eager-completion-buffer nil)`
because otherwise with its default value you need to hit
'?' (minibuffer-completion-help) instead of TAB
to show the completions buffer with two identical
completions in different groups.

> I did try adding your suggested settings to the above code snippet,
> but they did not seem to improve anything with regards to what shows
> up in the completions buffer:
>
> ```
> (setopt completions-group t)
> (setopt minibuffer-visible-completions t)
> ```

I'm so sorry, I forgot to check with `emacs -Q`, and there is
an additional important setting that affects the groups:

```
(setopt completions-group t)
(setopt completions-format 'vertical)
```

The default value of 'completions-format' is 'horizontal'
that doesn't support groups, and I don't know why.

Maybe Daniel (Cc:ed) could help us understand
why 'completion--insert-horizontal' doesn't display groups.

Message #17 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: Juri Linkov <juri <at> linkov.net>
Cc: 73117 <at> debbugs.gnu.org, Daniel Mendler <mail <at> daniel-mendler.de>
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sun, 8 Sep 2024 13:31:36 -0400

On Sun, Sep 8, 2024 at 12:42 PM Juri Linkov <juri <at> linkov.net> wrote:
>
> > Maybe using `(setopt completion-auto-help t)`
> > and hitting TAB is a better method.
>
> Or just using `(setopt imenu-eager-completion-buffer nil)`
> because otherwise with its default value you need to hit
> '?' (minibuffer-completion-help) instead of TAB
> to show the completions buffer with two identical
> completions in different groups.
>

Ah, thanks!  Yes, I'm used to using Vertico so I'm not very familiar
with the default settings/bindings.  In fact that's how I first ran
into this issue using Vertico and Imenu with `imenu-flatten` set to
"group".

> I'm so sorry, I forgot to check with `emacs -Q`, and there is
> an additional important setting that affects the groups:
>
> ```
> (setopt completions-group t)
> (setopt completions-format 'vertical)
> ```
>
> The default value of 'completions-format' is 'horizontal'
> that doesn't support groups, and I don't know why.
>
> Maybe Daniel (Cc:ed) could help us understand
> why 'completion--insert-horizontal' doesn't display groups.

No worries, at least I know I'm not going crazy, lol.

Indeed, when I set `completions-format` to "vertical" it does show all
entries.  I then expanded this example to see what happens with 3
entries with both "horizontal" and "vertical" values for
`completions-format`.  With a three section menu (see example below),
and format set to "horizontal", it does show multiple groups
vertically.  However, even under this condition, the groups which had
identical entries only show up once (which seems like a bug).

The following example configures `imenu-flatten" to "group" and then
shows the menu, first with `completions-format` set to "horizontal"
and then "vertical".

```elisp
(progn
  (setopt completions-group t)
  (setopt imenu-eager-completion-buffer nil)
  (setopt minibuffer-visible-completions t)
  (setopt imenu-flatten 'group)

  (require 'imenu)

  (dolist (format '(horizontal vertical))
    (setopt completions-format format)

    (imenu-choose-buffer-index (format "(%s) Index item: " format)
                               `(("XYZ" . (("123" . ,(point-min-marker))))
                                 ("Bar" . (("Foo" . ,(point-min-marker))))
                                 ("Baz" . (("Foo" . ,(point-max-marker))))))))
```

--8<---------------cut here---------------start------------->8---
Click or type RET on a completion to select it.
Type <right>, <left>, <down>, <up> to move point between completions.

3 possible completions:
     XYZ
123
     Bar
Foo
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
Click or type RET on a completion to select it.
Type <right>, <left>, <down>, <up> to move point between completions.

3 possible completions:
     XYZ
123
     Bar
Foo
     Baz
Foo
--8<---------------cut here---------------end--------------->8---

Message #20 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Daniel Mendler <mail <at> daniel-mendler.de>
To: Juri Linkov <juri <at> linkov.net>
Cc: 73117 <at> debbugs.gnu.org, Troy Brown <brownts <at> troybrown.dev>
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sun, 08 Sep 2024 20:28:07 +0200

Juri Linkov <juri <at> linkov.net> writes:

> Maybe Daniel (Cc:ed) could help us understand
> why 'completion--insert-horizontal' doesn't display groups.

If I remember correctly, I didn't add group titles to the horizontal
formatting in order to keep the display more dense, with the goal of
fitting as many candidates on the screen as possible. In my Vertico
package, I am using a similar strategy, where group titles are not shown
for the flat and grid display modes. In any case, there shouldn't be a
technical reason which prevents us from showing the titles also for the
horizontal display.

Daniel

Message #23 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: Daniel Mendler <mail <at> daniel-mendler.de>
Cc: 73117 <at> debbugs.gnu.org, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sun, 8 Sep 2024 15:25:45 -0400

On Sun, Sep 8, 2024 at 2:28 PM Daniel Mendler <mail <at> daniel-mendler.de> wrote:
>
> Juri Linkov <juri <at> linkov.net> writes:
>
> > Maybe Daniel (Cc:ed) could help us understand
> > why 'completion--insert-horizontal' doesn't display groups.
>
> If I remember correctly, I didn't add group titles to the horizontal
> formatting in order to keep the display more dense, with the goal of
> fitting as many candidates on the screen as possible. In my Vertico
> package, I am using a similar strategy, where group titles are not shown
> for the flat and grid display modes. In any case, there shouldn't be a
> technical reason which prevents us from showing the titles also for the
> horizontal display.

It appears to be this line in `completion--insert-horizontal` which is
causing all the trouble.  When I comment that out, I get the output
I'm expecting.  Looking at the difference between
`completion--insert-horizontal` and `completion--insert-vertical`,
that check happens after the check for `group-fun`.  Maybe
`completion--insert-horizontal` should be restructured similarly?

```elisp
      (unless (equal last-string str) ; Remove (consecutive) duplicates.
```

Message #26 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: Daniel Mendler <mail <at> daniel-mendler.de>
Cc: 73117 <at> debbugs.gnu.org, Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sun, 15 Sep 2024 11:58:53 -0400

On Sun, Sep 8, 2024 at 3:25 PM Troy Brown <brownts <at> troybrown.dev> wrote:
>
> It appears to be this line in `completion--insert-horizontal` which is
> causing all the trouble.  When I comment that out, I get the output
> I'm expecting.  Looking at the difference between
> `completion--insert-horizontal` and `completion--insert-vertical`,
> that check happens after the check for `group-fun`.  Maybe
> `completion--insert-horizontal` should be restructured similarly?
>
> ```elisp
>       (unless (equal last-string str) ; Remove (consecutive) duplicates.
> ```

Daniel, any thoughts on my comment above?

It appears completion--insert-horizontal incorrectly filters
consecutive strings when they belong to different groups.  From what I
can tell, it appears completion--insert-horizontal does support
groups, but just displays them vertically.  However, it does contain
the above problem.  I also notice this issue coming through in my use
of consult-imenu.

Message #29 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Troy Brown <brownts <at> troybrown.dev>
Cc: 73117 <at> debbugs.gnu.org, Daniel Mendler <mail <at> daniel-mendler.de>
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Fri, 20 Sep 2024 09:58:24 +0300

> The following example configures `imenu-flatten" to "group" and then
> shows the menu, first with `completions-format` set to "horizontal"
> and then "vertical".
>
> ```elisp
> (progn
>   (setopt completions-group t)
>   (setopt imenu-eager-completion-buffer nil)
>   (setopt minibuffer-visible-completions t)
>   (setopt imenu-flatten 'group)
>
>   (require 'imenu)
>
>   (dolist (format '(horizontal vertical))
>     (setopt completions-format format)

Any fixes for completion--insert-horizontal should be pushed
only to master, not to the release branch.

But for the release branch we should push the documentation fixes
that will mention all prerequisites for this feature to work
correctly:

diff --git a/lisp/imenu.el b/lisp/imenu.el
index 8f1b1f22a67..13440cfb626 100644
--- a/lisp/imenu.el
+++ b/lisp/imenu.el
@@ -158,9 +158,18 @@ imenu-flatten
 with a suffix that is the section name to which it belongs.
 If the value is `group', split completion candidates into groups
 according to the sections.
+
 Since the values `annotation' and `group' rely on text properties,
 you can use them only by selecting candidates from the completions
-buffer, not by typing in the minibuffer.
+buffer, not by typing in the minibuffer.  This also means that
+if you use `minibuffer-next-completion' (`M-<down>') to select
+a completion while point stays in the minibuffer, you need
+to customize `minibuffer-completion-auto-choose' to nil that
+doesn't insert completion candidates to the minibuffer.
+Also note that for using the value `group' you need to customize
+`completions-group' to the value t, and `completions-format'
+to the value `vertical'.
+
 Any other value is treated as `prefix'.
 
 The value of `imenu-level-separator', a string, is used to separate

Message #32 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 73117 <at> debbugs.gnu.org, mail <at> daniel-mendler.de, brownts <at> troybrown.dev
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sat, 28 Sep 2024 12:20:29 +0300

> Cc: 73117 <at> debbugs.gnu.org, Daniel Mendler <mail <at> daniel-mendler.de>
> From: Juri Linkov <juri <at> linkov.net>
> Date: Fri, 20 Sep 2024 09:58:24 +0300
> 
> > The following example configures `imenu-flatten" to "group" and then
> > shows the menu, first with `completions-format` set to "horizontal"
> > and then "vertical".
> >
> > ```elisp
> > (progn
> >   (setopt completions-group t)
> >   (setopt imenu-eager-completion-buffer nil)
> >   (setopt minibuffer-visible-completions t)
> >   (setopt imenu-flatten 'group)
> >
> >   (require 'imenu)
> >
> >   (dolist (format '(horizontal vertical))
> >     (setopt completions-format format)
> 
> Any fixes for completion--insert-horizontal should be pushed
> only to master, not to the release branch.
> 
> But for the release branch we should push the documentation fixes
> that will mention all prerequisites for this feature to work
> correctly:

Feel free to install on the release branch, and thanks.

Message #35 received at 73117 <at> debbugs.gnu.org (full text, mbox):

From: Troy Brown <brownts <at> troybrown.dev>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 73117 <at> debbugs.gnu.org, mail <at> daniel-mendler.de,
 Juri Linkov <juri <at> linkov.net>
Subject: Re: bug#73117: 30.0.90; Imenu missing entries when flattening by group
Date: Sun, 5 Jan 2025 19:15:47 -0500

On Sat, Sep 28, 2024 at 5:20 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> > Cc: 73117 <at> debbugs.gnu.org, Daniel Mendler <mail <at> daniel-mendler.de>
> > From: Juri Linkov <juri <at> linkov.net>
> > Date: Fri, 20 Sep 2024 09:58:24 +0300
> >
> > > The following example configures `imenu-flatten" to "group" and then
> > > shows the menu, first with `completions-format` set to "horizontal"
> > > and then "vertical".
> > >
> > > ```elisp
> > > (progn
> > >   (setopt completions-group t)
> > >   (setopt imenu-eager-completion-buffer nil)
> > >   (setopt minibuffer-visible-completions t)
> > >   (setopt imenu-flatten 'group)
> > >
> > >   (require 'imenu)
> > >
> > >   (dolist (format '(horizontal vertical))
> > >     (setopt completions-format format)
> >
> > Any fixes for completion--insert-horizontal should be pushed
> > only to master, not to the release branch.
> >
> > But for the release branch we should push the documentation fixes
> > that will mention all prerequisites for this feature to work
> > correctly:
>
> Feel free to install on the release branch, and thanks.

While discussing a similar issue for Vertico here
(https://github.com/minad/vertico/discussions/553), Daniel discovered
that icomplete-mode also won't display all of the grouped candidates.
The following is a reproducer which can be used to demonstrate this
issue.  In the reproducer, the first time through the loop,
icomplete-mode is not enabled and the completion buffer shows all four
possible candidates along with their corresponding groups, however the
next time through the loop, icomplete-vertical-mode is enabled and
only 3 of the 4 candidates are displayed.  See his comments in the
provided link for his take on the issue as well as his discovery of
yet more places that groups don't seem to work correctly (e.g.,
completion--insert-column).

```elisp
(progn
  (setopt completions-group t)
  (setopt completions-format 'vertical)
  (setopt imenu-flatten 'group)
  (setopt icomplete-show-matches-on-no-input t)

  (require 'imenu)

  (dolist (arg '(-1 +1))
    (icomplete-mode arg)
    (icomplete-vertical-mode arg)
    (setopt imenu-eager-completion-buffer icomplete-vertical-mode)
    (imenu-choose-buffer-index (format "(icomplete-vertical-mode = %s)
Index item: " icomplete-vertical-mode)
                               `(("XYZ" . (("123" . ,(point-min-marker))))
                                 ("ABC" . (("124" . ,(point-min-marker))))
                                 ("Bar" . (("Foo" . ,(point-min-marker))))
                                 ("Baz" . (("Foo" . ,(point-max-marker))))))))
```

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.