GNU bug report logs - #77054
Completion highlighting applied outside completion-lazy-hilit-fn

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 77054 in the body.
You can then email your comments to 77054 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: João Guerra <joca.bt <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Completion highlighting applied outside completion-lazy-hilit-fn
Date: Sun, 16 Mar 2025 16:27:00 +0100

TL;DR: Emacs applies additional completion highlighting even when
completion-lazy-hilit-fn is set, leading to incorrect behaviour with
alternative completion styles and frontends.

I'm implementing a custom fuzzy completion style and encountering an
issue where Emacs highlights parts of completions that it shouldn't.

For example, given the following directory structure:

```
<dir>/
  ...
  set-editing.el
  ...
```

and the input pattern `set`, my completion style produces three
possible matches for `set-editing.el`:

```
1 set-editing.el
  ^^^
2 set-editing.el
  ^^     ^
3 set-editing.el
  ^   ^  ^
```

Each match has a different score. In this case, match #3 has the
highest score and should be the one being highlighted.

Completion styles can bind completion-lazy-hilit-fn to control
highlighting. However, Emacs is still applying additional highlighting
outside completion-lazy-hilit-fn, resulting in the following
highlights:

```
set-editing.el
^^^ ^   ^
```

When completing files and buffers, but not commands, Emacs is
highlighting prefixes in
[completion--twq-all](https://github.com/emacs-mirror/emacs/blob/8ac894e2246f25d2a2a97d866b10e6e0b0fede5a/lisp/minibuffer.el#L677).
This becomes more evident if I make my completion-lazy-hilit-fn not do
any highlighting: completions are still highlighted when they
shouldn't at all.

Expected behavior: when lazy highlighting is enabled, Emacs shouldn't
apply any additional highlighting. While the current behavior works
for the default completion style (i.e. completions buffer), it easily
break as soon as users employ alternative styles or frontends.

Steps to reproduce (Emacs 30.1):
- emacs -Q
- Load:
    - https://github.com/minad/vertico/
    - https://github.com/lewang/flx/
    - https://gist.github.com/joca-bt/356ba1201c4fe842144e19ee1c723c28/
- Eval:
   - (vertico-mode t)
   - (setopt completion-styles '(flx))

Message #8 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: João Guerra <joca.bt <at> gmail.com>, João Távora <joaotavora <at> gmail.com>
Cc: 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Sun, 16 Mar 2025 17:52:44 +0200

> From: João Guerra <joca.bt <at> gmail.com>
> Date: Sun, 16 Mar 2025 16:27:00 +0100
> 
> TL;DR: Emacs applies additional completion highlighting even when
> completion-lazy-hilit-fn is set, leading to incorrect behaviour with
> alternative completion styles and frontends.
> 
> I'm implementing a custom fuzzy completion style and encountering an
> issue where Emacs highlights parts of completions that it shouldn't.
> 
> For example, given the following directory structure:
> 
> ```
> <dir>/
>   ...
>   set-editing.el
>   ...
> ```
> 
> and the input pattern `set`, my completion style produces three
> possible matches for `set-editing.el`:
> 
> ```
> 1 set-editing.el
>   ^^^
> 2 set-editing.el
>   ^^     ^
> 3 set-editing.el
>   ^   ^  ^
> ```
> 
> Each match has a different score. In this case, match #3 has the
> highest score and should be the one being highlighted.
> 
> Completion styles can bind completion-lazy-hilit-fn to control
> highlighting. However, Emacs is still applying additional highlighting
> outside completion-lazy-hilit-fn, resulting in the following
> highlights:
> 
> ```
> set-editing.el
> ^^^ ^   ^
> ```
> 
> When completing files and buffers, but not commands, Emacs is
> highlighting prefixes in
> [completion--twq-all](https://github.com/emacs-mirror/emacs/blob/8ac894e2246f25d2a2a97d866b10e6e0b0fede5a/lisp/minibuffer.el#L677).
> This becomes more evident if I make my completion-lazy-hilit-fn not do
> any highlighting: completions are still highlighted when they
> shouldn't at all.
> 
> Expected behavior: when lazy highlighting is enabled, Emacs shouldn't
> apply any additional highlighting. While the current behavior works
> for the default completion style (i.e. completions buffer), it easily
> break as soon as users employ alternative styles or frontends.
> 
> Steps to reproduce (Emacs 30.1):
> - emacs -Q
> - Load:
>     - https://github.com/minad/vertico/
>     - https://github.com/lewang/flx/
>     - https://gist.github.com/joca-bt/356ba1201c4fe842144e19ee1c723c28/
> - Eval:
>    - (vertico-mode t)
>    - (setopt completion-styles '(flx))

Thank you for your report.

Adding João Távora, the original author of this feature, to the
discussion.

Message #11 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: João Guerra <joca.bt <at> gmail.com>, 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Tue, 18 Mar 2025 22:32:47 +0000

On Sun, Mar 16, 2025 at 3:52 PM Eli Zaretskii <eliz <at> gnu.org> wrote:

> > Steps to reproduce (Emacs 30.1):
> > - emacs -Q
> > - Load:
> >     - https://github.com/minad/vertico/
> >     - https://github.com/lewang/flx/
> >     - https://gist.github.com/joca-bt/356ba1201c4fe842144e19ee1c723c28/
> > - Eval:
> >    - (vertico-mode t)
> >    - (setopt completion-styles '(flx))
>
> Thank you for your report.
>
> Adding João Távora, the original author of this feature, to the
> discussion.

Hi (and hi João! nice to hear from you and know you're still an Emacs
user)

Anyway, this was about 1,5 years agora and I'm afraid I don't have
the  details in my mental cache nor do I have the time to test this
complex setup.  AFAIK this feature has been in Emacs at least for
a full major version cycle (29) and had it failed in its basic mission
we would have heard about it.  I also remember testing it thoroughly.

It's important to note that completion-lazy-highlight-fn is not the only
variable involved in this scheme.  There is another:
completion-lazy-hilit.  I don't see it mentioned in this report. Both
need to be set for lazy highlighting to take place and actually save work.

completion-lazy-hilit is to be set by the frontend: (that would be vertico?)
while c-lazy-hilit-fn is set by the style.  At least according to the
docstrings
I wrote at the time, and which I suggest reading.

HTH,
João

Message #14 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: João Guerra <joca.bt <at> gmail.com>
To: João Távora <joaotavora <at> gmail.com>, 
 Eli Zaretskii <eliz <at> gnu.org>
Cc: 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Wed, 19 Mar 2025 16:12:02 +0100

[Message part 1 (text/plain, inline)]

> AFAIK this feature has been in Emacs at least for a full major version cycle (29) and had it failed in its basic mission we would have heard about it. I also remember testing it thoroughly.

It feels a bit disappointing when the answer to a possible bug report
is something like "nobody has reported it so it must be working fine"
or "it worked on my machine".

Anyway, here's a way to reproduce it in a less "complex" setup, only
using Emacs builtins.

The flex style sets `completion-lazy-hilit-fn` in the middle of
`completion-pcm--hilit-commonality` to `completion--hilit-from-re`.
Let's override that just for demonstration purposes in a *hacky way*.

```
(setopt completion-styles '(flex)
        file-name-shadow-properties `(invisible t
,@file-name-shadow-properties))
(setq completion-lazy-hilit t)
(defun completion--hilit-from-re (string regexp &optional point-idx)
  "Bogus fontification that only fontifies the last character of STRING."
  (message "lazy %s" completion-lazy-hilit)
  (let ((last (1- (length string))))
    (add-face-text-property last (1+ last) 'completions-common-part t string))
  string)
(icomplete-mode t)
```

See the attached picture for the outcome (if that doesn't work here's
an alternative url
https://drive.proton.me/urls/ZMMW1D9S2R#EXEwlTnWeCHy). You'll notice
that my fontification function has run, fontifying the last character.
However, additional characters are fontified, meaning there's
fontification happening elsewhere (the culprit indicated in the
original bug report). It could be the case that the frontend decided
to do additional fontification, but that's not the case (except for
the first candidate).

My expectation as a user or package maintainer is that no
fontification will be performed by Emacs if I am using lazy
highlighting. However, that's not the case in *specific* situations as
I am trying to show in this bug report.

> At least according to the docstrings I wrote at the time, and which I suggest reading.

Neither docstring directly discusses whether additional fontification
can be performed by Emacs when using lazy highlighting, so my
assumption would be no.

If it helps, I can reframe this bug report as: if lazy highlighting is
enabled, can Emacs perform additional fontification (outside the
completion frontend and style)?

If the answer is yes, can we consider changing that behaviour? As I
believe it constraints what packages can do.

[77054.png (image/png, attachment)]

Message #17 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: João Guerra <joca.bt <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 77054 <at> debbugs.gnu.org
Subject: Fwd: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Wed, 19 Mar 2025 16:18:18 +0100

Eli, is this how we want Emacs maintainers to behave?

---------- Forwarded message ---------
From: João Távora <joaotavora <at> gmail.com>
Date: Wed, 19 Mar 2025 at 16:14
Subject: Re: bug#77054: Completion highlighting applied outside
completion-lazy-hilit-fn
To: João Guerra <joca.bt <at> gmail.com>

So, sorry to disappoint you João, and best of luck!

João

On Wed, Mar 19, 2025, 15:12 João Guerra <joca.bt <at> gmail.com> wrote:
>
> > AFAIK this feature has been in Emacs at least for a full major version cycle (29) and had it failed in its basic mission we would have heard about it. I also remember testing it thoroughly.
>
> It feels a bit disappointing when the answer to a possible bug report
> is something like "nobody has reported it so it must be working fine"
> or "it worked on my machine".
>
> Anyway, here's a way to reproduce it in a less "complex" setup, only
> using Emacs builtins.
>
> The flex style sets `completion-lazy-hilit-fn` in the middle of
> `completion-pcm--hilit-commonality` to `completion--hilit-from-re`.
> Let's override that just for demonstration purposes in a *hacky way*.
>
> ```
> (setopt completion-styles '(flex)
>         file-name-shadow-properties `(invisible t
> ,@file-name-shadow-properties))
> (setq completion-lazy-hilit t)
> (defun completion--hilit-from-re (string regexp &optional point-idx)
>   "Bogus fontification that only fontifies the last character of STRING."
>   (message "lazy %s" completion-lazy-hilit)
>   (let ((last (1- (length string))))
>     (add-face-text-property last (1+ last) 'completions-common-part t string))
>   string)
> (icomplete-mode t)
> ```
>
> See the attached picture for the outcome (if that doesn't work here's
> an alternative url
> https://drive.proton.me/urls/ZMMW1D9S2R#EXEwlTnWeCHy). You'll notice
> that my fontification function has run, fontifying the last character.
> However, additional characters are fontified, meaning there's
> fontification happening elsewhere (the culprit indicated in the
> original bug report). It could be the case that the frontend decided
> to do additional fontification, but that's not the case (except for
> the first candidate).
>
> My expectation as a user or package maintainer is that no
> fontification will be performed by Emacs if I am using lazy
> highlighting. However, that's not the case in *specific* situations as
> I am trying to show in this bug report.
>
> > At least according to the docstrings I wrote at the time, and which I suggest reading.
>
> Neither docstring directly discusses whether additional fontification
> can be performed by Emacs when using lazy highlighting, so my
> assumption would be no.
>
> If it helps, I can reframe this bug report as: if lazy highlighting is
> enabled, can Emacs perform additional fontification (outside the
> completion frontend and style)?
>
> If the answer is yes, can we consider changing that behaviour? As I
> believe it constraints what packages can do.

Message #20 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: João Guerra <joca.bt <at> gmail.com>, 
 Eli Zaretskii <eliz <at> gnu.org>, 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Wed, 19 Mar 2025 20:41:32 +0000

Forgot to CC the bug tracker.  So just wanted to say again I'm sorry
to disappoint, and to clarify a possible misunderstanding.  "lazy" has
nothing to do with "what" is highlighted but merely "when". As far
as I can tell,that part is working fine,so whatever your problem is
And again, lazy highlighting only concerns a collaboration between
style and frontend.

toodle-oo!
João



On Wed, Mar 19, 2025 at 3:14 PM João Távora <joaotavora <at> gmail.com> wrote:
>
> So, sorry to disappoint you João, and best of luck!
>
> João
>
> On Wed, Mar 19, 2025, 15:12 João Guerra <joca.bt <at> gmail.com> wrote:
>>
>> > AFAIK this feature has been in Emacs at least for a full major version cycle (29) and had it failed in its basic mission we would have heard about it. I also remember testing it thoroughly.
>>
>> It feels a bit disappointing when the answer to a possible bug report
>> is something like "nobody has reported it so it must be working fine"
>> or "it worked on my machine".
>>
>> Anyway, here's a way to reproduce it in a less "complex" setup, only
>> using Emacs builtins.
>>
>> The flex style sets `completion-lazy-hilit-fn` in the middle of
>> `completion-pcm--hilit-commonality` to `completion--hilit-from-re`.
>> Let's override that just for demonstration purposes in a *hacky way*.
>>
>> ```
>> (setopt completion-styles '(flex)
>>         file-name-shadow-properties `(invisible t
>> ,@file-name-shadow-properties))
>> (setq completion-lazy-hilit t)
>> (defun completion--hilit-from-re (string regexp &optional point-idx)
>>   "Bogus fontification that only fontifies the last character of STRING."
>>   (message "lazy %s" completion-lazy-hilit)
>>   (let ((last (1- (length string))))
>>     (add-face-text-property last (1+ last) 'completions-common-part t string))
>>   string)
>> (icomplete-mode t)
>> ```
>>
>> See the attached picture for the outcome (if that doesn't work here's
>> an alternative url
>> https://drive.proton.me/urls/ZMMW1D9S2R#EXEwlTnWeCHy). You'll notice
>> that my fontification function has run, fontifying the last character.
>> However, additional characters are fontified, meaning there's
>> fontification happening elsewhere (the culprit indicated in the
>> original bug report). It could be the case that the frontend decided
>> to do additional fontification, but that's not the case (except for
>> the first candidate).
>>
>> My expectation as a user or package maintainer is that no
>> fontification will be performed by Emacs if I am using lazy
>> highlighting. However, that's not the case in *specific* situations as
>> I am trying to show in this bug report.
>>
>> > At least according to the docstrings I wrote at the time, and which I suggest reading.
>>
>> Neither docstring directly discusses whether additional fontification
>> can be performed by Emacs when using lazy highlighting, so my
>> assumption would be no.
>>
>> If it helps, I can reframe this bug report as: if lazy highlighting is
>> enabled, can Emacs perform additional fontification (outside the
>> completion frontend and style)?
>>
>> If the answer is yes, can we consider changing that behaviour? As I
>> believe it constraints what packages can do.



--
João Távora

Message #23 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: João Guerra <joca.bt <at> gmail.com>
Cc: joaotavora <at> gmail.com, 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Thu, 20 Mar 2025 14:19:54 +0200

> From: João Guerra <joca.bt <at> gmail.com>
> Date: Wed, 19 Mar 2025 16:12:02 +0100
> Cc: 77054 <at> debbugs.gnu.org
> 
> ```
> (setopt completion-styles '(flex)
>         file-name-shadow-properties `(invisible t
> ,@file-name-shadow-properties))
> (setq completion-lazy-hilit t)
> (defun completion--hilit-from-re (string regexp &optional point-idx)
>   "Bogus fontification that only fontifies the last character of STRING."
>   (message "lazy %s" completion-lazy-hilit)
>   (let ((last (1- (length string))))
>     (add-face-text-property last (1+ last) 'completions-common-part t string))
>   string)
> (icomplete-mode t)
> ```
> 
> See the attached picture for the outcome (if that doesn't work here's
> an alternative url
> https://drive.proton.me/urls/ZMMW1D9S2R#EXEwlTnWeCHy). You'll notice
> that my fontification function has run, fontifying the last character.
> However, additional characters are fontified, meaning there's
> fontification happening elsewhere (the culprit indicated in the
> original bug report). It could be the case that the frontend decided
> to do additional fontification, but that's not the case (except for
> the first candidate).
> 
> My expectation as a user or package maintainer is that no
> fontification will be performed by Emacs if I am using lazy
> highlighting. However, that's not the case in *specific* situations as
> I am trying to show in this bug report.
> 
> > At least according to the docstrings I wrote at the time, and which I suggest reading.
> 
> Neither docstring directly discusses whether additional fontification
> can be performed by Emacs when using lazy highlighting, so my
> assumption would be no.

The doc string says (in Emacs 30.1):

  completion-lazy-hilit-fn is a variable defined in ‘minibuffer.el’.

  Its value is nil

  Fontification function set by lazy-highlighting completions styles.
  When a given style wants to enable support for ‘completion-lazy-hilit’
  (which see), that style should set this variable to a function of one
  argument.  It will be called with each completion candidate, a string, to
  be displayed to the user, and should destructively propertize these
  strings with the ‘face’ property.

The "should destructively propertize" part tells me that the function
should take care of removing the face properties present in the input
string where it doesn't want those fontifications.  By contrast, your
function calls add-text-properties, which only adds the face
properties to a single character, but does nothing with the face
properties of the rest of the string.

> If it helps, I can reframe this bug report as: if lazy highlighting is
> enabled, can Emacs perform additional fontification (outside the
> completion frontend and style)?

I believe the answer is YES; if you would like to _replace_ the
fontifications of the candidate, your function should remove the face
text properties from all the characters of the string, before adding
your fontifications.

> If the answer is yes, can we consider changing that behaviour? As I
> believe it constraints what packages can do.

Actually, I think the existing behavior leaves the packages more
freedom, because they can retain any of the existing fontifications as
they see fit.  Don't you agree?

Or what am I missing?

Message #26 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: João Guerra <joca.bt <at> gmail.com>, 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Thu, 20 Mar 2025 17:11:36 +0000

[Message part 1 (text/plain, inline)]

On Thu, Mar 20, 2025, 12:20 Eli Zaretskii <eliz <at> gnu.org> wrote:

> >
> > If it helps, I can reframe this bug report as: if lazy highlighting is
> > enabled, can Emacs perform additional fontification (outside the
> > completion frontend and style)?
>
> I believe the answer is YES; if you would like to _replace_ the
> fontifications of the candidate, your function should remove the face
> text properties from all the characters of the string, before adding
> your fontifications.
>

The function may do this and it's passed a fresh string to destroy at will
I think, but it may not matter if later parts of the machinery add these
properties. It could come from the table's sorting for example. There's
little the style can do about that.

Furthermore, as I've been trying to explain, lazy highlighting as I
implemented is completely orthogonal. It's just a mechanism to allow front
ends to delay fontification to the last possible moment when they actually
display stuff to the user. Styles are not supposed to set it to anything
that fontifies differently/specially because styles don't know in general
what frontend is driving them. IOW, whatever problem the OP is experiencing
would likely be there if lazy highlighting were turned off (by the frontend
or the style, or both).

João

[Message part 2 (text/html, inline)]

Message #29 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: João Távora <joaotavora <at> gmail.com>,
 Eli Zaretskii <eliz <at> gnu.org>
Cc: João Guerra <joca.bt <at> gmail.com>, 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Thu, 20 Mar 2025 19:26:55 +0200

On 20/03/2025 19:11, João Távora wrote:
> The function may do this and it's passed a fresh string to destroy at 
> will I think, but it may not matter if later parts of the machinery add 
> these properties. It could come from the table's sorting for example. 
> There's little the style can do about that.
> 
> Furthermore, as I've been trying to explain, lazy highlighting as I 
> implemented is completely orthogonal. It's just a mechanism to allow 
> front ends to delay fontification to the last possible moment when they 
> actually display stuff to the user. Styles are not supposed to set it to 
> anything that fontifies differently/specially because styles don't know 
> in general what frontend is driving them. IOW, whatever problem the OP 
> is experiencing would likely be there if lazy highlighting were turned 
> off (by the frontend or the style, or both).

Styles are indeed not supposed to alter it - they're supposed to obey it 
instead.

But completion-lazy-hilit does have something to do with not altering 
the original string: that's the whole definition of 
completion-lazy-hilit: if on, do copy-sequence first.

Completion tables might alter the strings inside their sorting, but they 
own those string so that seems fine. Completion style's sorting should 
avoid that. This was a large part of the discussion we had at the time, 
and one result is that the sorter inside 
completion--flex-adjust-metadata does not alter the strings.

Message #32 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: João Guerra <joca.bt <at> gmail.com>, 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Thu, 20 Mar 2025 19:33:58 +0200

Hi!

On 16/03/2025 17:27, João Guerra wrote:
> TL;DR: Emacs applies additional completion highlighting even when
> completion-lazy-hilit-fn is set, leading to incorrect behaviour with
> alternative completion styles and frontends.
> 
> I'm implementing a custom fuzzy completion style and encountering an
> issue where Emacs highlights parts of completions that it shouldn't.

To clarify: this doesn't happen with any of the built-in styles, right?

That might mean that the new style violates some of the assumptions - 
perhaps ones that are not too well documented.

> When completing files and buffers, but not commands, Emacs is
> highlighting prefixes in
> [completion--twq-all](https://github.com/emacs-mirror/emacs/blob/8ac894e2246f25d2a2a97d866b10e6e0b0fede5a/lisp/minibuffer.el#L677).
> This becomes more evident if I make my completion-lazy-hilit-fn not do
> any highlighting: completions are still highlighted when they
> shouldn't at all.

To look at this another way - like Joao said, if this happens it might 
be a problem even without lazy highlighting. Does it occur then?

If so, might be unrelated to your style as well.

Message #35 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Eli Zaretskii <eliz <at> gnu.org>,
 João Guerra <joca.bt <at> gmail.com>, 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Thu, 20 Mar 2025 19:02:47 +0000

[Message part 1 (text/plain, inline)]

On Thu, Mar 20, 2025, 17:26 Dmitry Gutov <dmitry <at> gutov.dev> wrote:

> On 20/03/2025 19:11, João Távora wrote:
> > The function may do this and it's passed a fresh string to destroy at
> > will I think, but it may not matter if later parts of the machinery add
> > these properties. It could come from the table's sorting for example.
> > There's little the style can do about that.
> >
> > Furthermore, as I've been trying to explain, lazy highlighting as I
> > implemented is completely orthogonal. It's just a mechanism to allow
> > front ends to delay fontification to the last possible moment when they
> > actually display stuff to the user. Styles are not supposed to set it to
> > anything that fontifies differently/specially because styles don't know
> > in general what frontend is driving them. IOW, whatever problem the OP
> > is experiencing would likely be there if lazy highlighting were turned
> > off (by the frontend or the style, or both).
>
> Styles are indeed not supposed to alter it - they're supposed to obey it
> instead.
>

Exactly. Well, they're _encouraged_ to obey it by setting the fn. But they
don't have to. Afair many styles don't, likely because that wouldn't bring
much speed. Flex does it because by its nature it can match a lot of stuff.

But completion-lazy-hilit does have something to do with not altering
> the original string: that's the whole definition of
> completion-lazy-hilit: if on, do copy-sequence first.
>

Yup.

Completion tables might alter the strings inside their sorting, but they
> own those string so that seems fine. Completion style's sorting should
> avoid that. This was a large part of the discussion we had at the time,
> and one result is that the sorter inside
> completion--flex-adjust-metadata does not alter the strings.
>

All true!

João

[Message part 2 (text/html, inline)]

Message #38 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: João Guerra <joca.bt <at> gmail.com>
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, Eli Zaretskii <eliz <at> gnu.org>,
 João Távora <joaotavora <at> gmail.com>, 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Fri, 21 Mar 2025 12:09:31 +0100

> The "should destructively propertize" part tells me that the function should take care of removing the face properties present in the input string where it doesn't want those fontifications.  By contrast, your function calls add-text-properties, which only adds the face properties to a single character, but does nothing with the face properties of the rest of the string.

I read "should destructively propertize" as "mutate the string to
avoid using more memory / efficiency reasons".

While I understand this argument, I could see this behaviour being
problematic in certain situations. For example, a frontend may decide
to do some fontification on top of completion-lazy-hilit-fn (e.g. to
highlight the current candidate differently). While there's an obvious
workaround - the frontend doing their fontification after calling
completion-lazy-hilit-fn - I'm not sure whether it could cause
restrictions in more complex scenarios. Having the completion style
remove all prior fontification sounds a bit extreme for an interface.
For me, it makes more sense to be additive, but I don't know what
Emacs' vision is on this topic.

> can Emacs perform additional fontification (outside the completion frontend and style)?
> I believe the answer is YES

I believe this is problematic. I would like to understand in which
situations does it make sense for Emacs to do fontification outside
the completion frontend and completion style. These are the two main
drivers when completing, and are the ones responsible for the user
interface. Having other parts of the code chime in on fontification,
which is basically user interface, seems to break that division.

`completion--twq-all` is doing fontification. This function doesn't
look like it's part of the completion frontend nor of the completion
style. Why is it doing fontification? Should it be doing
fontification?

> To clarify: this doesn't happen with any of the built-in styles, right?
> To look at this another way - like Joao said, if this happens it might be a problem even without lazy highlighting. Does it occur then?

The reported issue is caused by the fontification applied by
`completion--twq-all` and it happens even without lazy highlighting
(it's just easier to spot when using lazy highlighting because the
expectation is that no fontification will happen prior to it). I would
say that the reason it hasn't been noticed before is that its
behaviour mostly matches the behaviour of the builtin completion
styles, so it won't be noticeable because they're fontifiying the same
characters with the same faces.

Message #41 received at 77054 <at> debbugs.gnu.org (full text, mbox):

From: João Távora <joaotavora <at> gmail.com>
To: João Guerra <joca.bt <at> gmail.com>
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, Eli Zaretskii <eliz <at> gnu.org>,
 77054 <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Fri, 21 Mar 2025 12:41:55 +0000

[Message part 1 (text/plain, inline)]

On Fri, Mar 21, 2025, 11:10 João Guerra <joca.bt <at> gmail.com> wrote:

> . Having the completion style
> remove all prior fontification sounds a bit extreme for an interface.
> For me, it makes more sense to be additive, but I don't know what
> Emacs' vision is on this topic.
>

It's up to the style. If you like it additive keep it additive, else do
what Eli suggests.

`completion--twq-all` is doing fontification. This function doesn't
> look like it's part of the completion frontend nor of the completion
> style. Why is it doing fontification? Should it be doing
> fontification?
>

It's part of the table. It does what it has done probably for many years.
If you want to change that behavior for everyone, go ahead, but IME you'll
eventually bother lots of people using simpler setups. If your style is
bothered by it, remove it there just like Eli suggests. In lazy and in
non-lazy situations. IOW the lazy optimization has nothing to do with this.

And if for some reason you're worrying about performance, don't. The lazy
optimization is largely about avoiding premature upfront string
allocations, not propertization. And it more than likely doesn't matter for
file tables anyway which are small compared to symbol obarrays, for
instance.

[Message part 2 (text/html, inline)]

Message #46 received at 77054-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: João Távora <joaotavora <at> gmail.com>
Cc: dmitry <at> gutov.dev, joca.bt <at> gmail.com, 77054-done <at> debbugs.gnu.org
Subject: Re: bug#77054: Completion highlighting applied outside
 completion-lazy-hilit-fn
Date: Sat, 05 Apr 2025 11:35:22 +0300

> From: João Távora <joaotavora <at> gmail.com>
> Date: Fri, 21 Mar 2025 12:41:55 +0000
> Cc: Dmitry Gutov <dmitry <at> gutov.dev>, Eli Zaretskii <eliz <at> gnu.org>, 77054 <at> debbugs.gnu.org
> 
> On Fri, Mar 21, 2025, 11:10 João Guerra <joca.bt <at> gmail.com> wrote:
> 
>  . Having the completion style
>  remove all prior fontification sounds a bit extreme for an interface.
>  For me, it makes more sense to be additive, but I don't know what
>  Emacs' vision is on this topic.
> 
> It's up to the style. If you like it additive keep it additive, else do what Eli suggests.
> 
>  `completion--twq-all` is doing fontification. This function doesn't
>  look like it's part of the completion frontend nor of the completion
>  style. Why is it doing fontification? Should it be doing
>  fontification?
> 
> It's part of the table. It does what it has done probably for many years. If you want to change that behavior for
> everyone, go ahead, but IME you'll eventually bother lots of people using simpler setups. If your style is
> bothered by it, remove it there just like Eli suggests. In lazy and in non-lazy situations. IOW the lazy
> optimization has nothing to do with this. 
> 
> And if for some reason you're worrying about performance, don't. The lazy optimization is largely about
> avoiding premature upfront string allocations, not propertization. And it more than likely doesn't matter for file
> tables anyway which are small compared to symbol obarrays, for instance.

No further comments within 2 weeks, so I presume the problem is
solved, and I'm closing this bug.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.