Subject: bug#76658: Emacs 27.2; Prioritized invisible overlay is NOT taking precedence over non-prioritized display overlay

From: "David's Coding Lounge" <[email-masked]>
Date: Tue, 4 Mar 2025 23:28:55 -0500

Thanks for the response, Eli. Your understanding of the problem is correct.
Your response both makes sense as well and aligns with what I have
encountered in testing. Apologies for the extra information in my posts,
but I clearly was confused in my understanding of how the properties of
overlays interacted with each other and overlays between themselves.

After re-reading the documentation for an Overlay's `priority` property
several times before I asked my question, I had a suspicion that it was
only relevant on clashing properties that were declared across multiple
overlays (i.e. Overlay Priority is a tie-breaker at the individual property
level between the same properties, not at the overlay level between
overlays: first line of the second paragraph). I was unsure though and
thought the behavior I was experiencing was wrong/my understanding of the
documentation was incorrect. The clarification and confirmation is
appreciated.

The behavior around `display` taking precedence over `invisible` is
surprising ... somewhat. I am not sure if that is documented anywhere or
called out explicitly, I probably missed it if it was. However, stopping to
think about the context, it seems to make sense and is logically what would
be expected. There is still some doubt I have for certain scenarios and
what makes intuitive sense but that is more about me needing to more
clearly understand the display engine at a finer level than I currently do
(`xdisp.c` here I come).

The cases presented thus far make sense, the first overlay processed
essentially "wins" and if the starting positioning of an overlay is the
same as another with each specifying either `invisible` or `display`, the
`display` property value of whichever overlay "wins" (since the underlying
text is _essentially_ invisible for all intents and purposes based on
what/how `display` works, correct?). In other words, it could be thought of
as `display` is `invisible` with benefits: `hide but show something else
instead` OR the other way around, `invisible` is functionally equivalent to
just `display ""`?

To solve my issue, I did end up specifying a `display` property on the
`invisible` overlay with a condition similar to the condition applied for
`invisible`: `display (if XYZ "")`. It worked, however, at the time it just
felt "wrong" or "doing more than it should need to do for the effect
desired" (if that makes sense).

Thanks for the explanation, clarification, and confirmations Eli!

Much appreciated,
David

On Sat, Mar 1, 2025 at 5:35 AM Eli Zaretskii <[email-masked]> wrote:

> tags 76658 notabug
> thanks
>
> > From: "David's Coding Lounge" <[email-masked]>
> > Date: Fri, 28 Feb 2025 18:23:41 -0500
> >
> > This bug report is essentially taken from my Emacs Stack Exchange post:
> >
> https://emacs.stackexchange.com/questions/83169/why-doesnt-a-prioritized-invisible-overlay-take-precedence-over-non-prioritized
>
> It is not easy to understand what exactly is the issue which you are
> reporting and for which you are asking for explanations.  Both this
> report and the linked postings have a lot of stuff, most of which is
> AFAIU irrelevant.
>
> My understanding is that the issue you are asking about is the effect
> on display of having both 'display' and 'invisible' property start at
> the same buffer position.  If this is not the issue you are asking
> about, what's below may or may not make sense.
>
> When both 'display' and 'invisible' properties start at the same
> buffer position, and the 'display' property is a "replacing" property
> (i.e. it instructs Emacs to show something else instead of buffer
> text), then the 'display' property "wins", in the sense that the
> invisible property is effectively ignored.  This happens due to how
> the Emacs display engine processes properties:
>
>   . it processes 'display' properties before 'invisible'
>   . when either a replacing 'display' property or 'invisible' property
>     is found, the display engine completely skips the text covered by
>     the property, so any other properties in the same text are not
>     processed
>
> This should explain everything that you see.  In particular, overlay
> priorities have nothing to do with this, since (as the ELisp manual
> says) the priorities are only examined when two or more overlays have
> the same property for the same buffer position, which is not the case
> here.  Also, it explains why, if the 'invisible' property starts
> before 'display', it makes all the text invisible, including the
> overlay that specifies a 'display' property which shows an image.
>
> If you sometimes need to have the image (defined via the 'display'
> property) to vanish from display, simply remove the 'display'
> property, or use the conditional 'display' property of the form
> '(when CONDITION . DISPLAY-SPEC)' and change Lisp variables that
> affect CONDITION.
>
> Bottom line: this is not a bug.
>