GNU bug report logs - #76658
Emacs 27.2; Prioritized invisible overlay is NOT taking precedence over non-prioritized display overlay

Previous Next

Full log

View this message in rfc822 format

From: "David's Coding Lounge" <davidcodinglounge <at> gmail.com>
To: 76658 <at> debbugs.gnu.org
Subject: bug#76658: Emacs 27.2; Prioritized invisible overlay is NOT taking precedence over non-prioritized display overlay
Date: Fri, 28 Feb 2025 18:23:41 -0500

[Message part 1 (text/plain, inline)]

Hello,

This bug report is essentially taken from my Emacs Stack Exchange post:
https://emacs.stackexchange.com/questions/83169/why-doesnt-a-prioritized-invisible-overlay-take-precedence-over-non-prioritized

## Details

I have several overlays being applied to a buffer to replace the `display`
property of certain characters. There is also another overlay being applied
(conditionally) for making the nested items/text `invisible` in _addition_
to changing the `display` property. I set the overlay with the `invisible`
property to a higher priority (i.e. actually having a priority), but it is
not working as expected.

The character is _still_ being displayed with the alternative `display`
character while _also_ having the `invisible` property set to `t` from the
higher priority overlay.

The only thing that I could find that would _seem_ to explain this behavior
was an obscure Reddit thread:
https://www.reddit.com/r/emacs/comments/17nxc9s/problem_making_overlays_invisible/
whereby the `display` property apparently takes precedence over
`invisible`. This makes sense for non-prioritized overlays, however I
wouldn't imagine this would be correct when there is a `priority` property
applied.

## Reproducible Code/Data

Example code (using `ov` library) to demonstrate the problem:

```lisp
(ov 1 2 'display "•")
(ov 10 12 'display "★")
(ov 19 21 'display "★")
(ov 28 30 'display "★")
(ov 10 37 'invisible t 'priority 10)
```

Example data to show the issue (NOTE: - Org mode is irrelevant, it occurs
by executing the code against the same data in a scratch buffer):

```text
* Root A
** Sub A
** Sub B
** Sub C
* Root B
** Sub 1
** Sub 2
** Sub 3
** Sub 4
* Root C
** Sub I
** Sub II
* Root D
```

The even more confusing issue is that the effect _would_ seem to work as
expected when I change the `invisible` overlay start index by 1 less (e.g.
`(ov 9 37 'invisible t 'priority 10)`). This doesn't seem to logically be
consistent.

### Image of Example Code/Data Reproducing the Issue:
https://i.sstatic.net/8cAP29TK.png

*The character at index position 10 is visible despite having the
`invisible` property set to `t` and the `invisible` overlay having the
highest priority.*

## Investigation Notes

Even after reviewing the documentation for `overlays` and the `priority`
property (
https://www.gnu.org/software/emacs/manual/html_node/elisp/Overlay-Properties.html),
it _still_ does not make sense. The documentation says (emphasis mine):

> This property’s value determines the priority of the overlay. If you want
to specify a priority value, use either nil (or zero), or a positive
integer, or a cons of two values. Any other value triggers undefined
behavior.
>
> **The priority matters when two or more overlays cover the same character
and both specify the same property with different values; the one whose
priority value is higher overrides the other.** (For the face property, the
higher priority overlay’s value does not completely override the other
value; instead, its individual face attributes override the corresponding
face attributes of the face property whose priority is lower.) If two
overlays have the same priority value, and one is “nested” in the other
(i.e., covers fewer buffer or string positions), then the inner one will
prevail over the outer one. If neither is nested in the other then you
should not make assumptions about which overlay will prevail.
>
> **When a Lisp program puts overlays with defined priorities on text that
might have overlays without priorities, this could cause undesirable
results, because any overlay with a positive priority value will override
all the overlays without a priority.** Since most Emacs features that use
overlays don’t specify priorities for their overlays, integer priorities
should be used with care. Instead of using integer priorities and risk
overriding other overlays, you can use priority values of the form (primary
. secondary), where the primary value is used as described above, and
secondary is the fallback value used when primary and the nesting
considerations fail to resolve the precedence between overlays. In
particular, priority value (nil . n), with n a positive integer, enables
you to have the overlays ordered by priority when necessary without
completely overriding other overlays.
>
> Currently, all overlays take priority over text properties.

Even when reading the first sentence of the third paragraph in the
documentation, the issue _still_ occurs even if I defined a lower priority
for the `display` overlays.

NOTE: - The `buffer-invisibility-spec` is correctly set up to support
invisibility too, just in case anyone thought that might be the issue. It
is not. I am certain of it.

## Question

Can anyone explain why this issue only seems to occur when the overlay `(ov
10 37 'invisible t 'priority 10)` is applied but not when `(ov 9 37
'invisible t 'priority 10)` is applied (i.e. why does including the
preceding new line make the effect "work")? Is there a finer point of
nuance for overlays and the `priority`, `display`, and `invisible`
properties?

An ancillary note: I _can_ achieve the proper effect, with the proper
indexes, by explicitly setting the `display` property to an empty string on
the `invisible` overlay: `(ov 10 37 'invisible t 'priority 10 'display "")`.

However, despite this, I still want to properly understand things, why the
issue occurs/doesn't seem to make sense, etc. It's not an XY problem.

#### Standard environment disclosure:

- OS: MacOS
- Chip: Intel
- Emacs Version: 27.2.1
- Port: Mitsuharu Yamamoto / Railwaycat
- Other Notes: - No special distributions / etc.
- Relevant Common Packages Used:
    - ov
    - Standard internal Emacs packages

Thanks,
David

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.