Package: emacs;
Reported by: JD Smith <jdtsmith <at> gmail.com>
Date: Wed, 20 Aug 2025 01:40:02 UTC
Severity: normal
Found in version 30.2.50
Done: Eli Zaretskii <eliz <at> gnu.org>
To reply to this bug, email your comments to 79275 AT debbugs.gnu.org.
There is no need to reopen the bug first.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-gnu-emacs <at> gnu.org
:bug#79275
; Package emacs
.
(Wed, 20 Aug 2025 01:40:02 GMT) Full text and rfc822 format available.JD Smith <jdtsmith <at> gmail.com>
:bug-gnu-emacs <at> gnu.org
.
(Wed, 20 Aug 2025 01:40:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: JD Smith <jdtsmith <at> gmail.com> To: bug-gnu-emacs <at> gnu.org Subject: 30.2.50; overlay line-prefix display property fighting with text display property Date: Tue, 19 Aug 2025 21:39:38 -0400
An overlay which applies a line-prefix (e.g. to set the fringe) across several lines conflicts with underlying text which has a replacing display property set. Evaluate the following in the *scratch* buffer, with at least 3 blank lines at the top of the buffer. (progn (delete-all-overlays) (let ((ov (make-overlay 1 4))) (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display '(left-fringe right-triangle success)))) (put-text-property 2 3 'display ">testing fringe display")) The result is very strange: the SHOULDNOTSEETHIS prefix string appears (sans fringe display), but then vanishes on the next redisplay. Some (well-balanced) fight between overlay line-prefix display and normal display properties seems to be occurring.
bug-gnu-emacs <at> gnu.org
:bug#79275
; Package emacs
.
(Wed, 20 Aug 2025 13:36:02 GMT) Full text and rfc822 format available.Message #8 received at 79275 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: JD Smith <jdtsmith <at> gmail.com> Cc: 79275 <at> debbugs.gnu.org Subject: Re: bug#79275: 30.2.50; overlay line-prefix display property fighting with text display property Date: Wed, 20 Aug 2025 16:35:20 +0300
> From: JD Smith <jdtsmith <at> gmail.com> > Date: Tue, 19 Aug 2025 21:39:38 -0400 > > An overlay which applies a line-prefix (e.g. to set the fringe) across several lines conflicts with underlying text which has a replacing display property set. Evaluate the following in the *scratch* buffer, with at least 3 blank lines at the top of the buffer. > > (progn > (delete-all-overlays) > (let ((ov (make-overlay 1 4))) > (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display > '(left-fringe right-triangle success)))) > (put-text-property 2 3 'display ">testing fringe display")) > > The result is very strange: the SHOULDNOTSEETHIS prefix string appears (sans fringe display), but then vanishes on the next redisplay. Some (well-balanced) fight between overlay line-prefix display and normal display properties seems to be occurring. Thanks. One of the redisplay optimizations we use couldn't cope with this tricky situation (a line-prefix immediately followed by a display string at the beginning of a line), and needs to be disabled in this case. Does the patch below give good results? diff --git a/src/xdisp.c b/src/xdisp.c index 2691296..6026038 100644 --- a/src/xdisp.c +++ b/src/xdisp.c @@ -7261,6 +7261,8 @@ push_it (struct it *it, struct text_pos *position) p->from_disp_prop_p = it->from_disp_prop_p; ++it->sp; + it->string_from_prefix_prop_p = false; + /* Save the state of the bidi iterator as well. */ if (it->bidi_p) bidi_push_it (&it->bidi_it); @@ -22657,8 +22659,23 @@ #define GIVE_UP(X) return 0 /* Give up if the row starts with a display property that draws on the fringes, since that could prevent correct display of line-prefix and wrap-prefix. */ - if (it.sp > 1 - && it.method == GET_FROM_IMAGE && it.image_id == -1) + if ((it.sp > 1 + && it.method == GET_FROM_IMAGE && it.image_id == -1) + /* Give up if there's a line/wrap-prefix property on buffer + text, and the row begins with a display or overlay string. + This is because in that case the iterator state produced by + init_to_row_end is already set to the display/overlay + string, and thus cannot be used to display the prefix + before the display/overlay string. */ + || (it.sp == 1 + && it.method == GET_FROM_STRING + && !it.string_from_prefix_prop_p + && (!NILP (Fget_char_property (make_fixnum (IT_CHARPOS (it)), + Qline_prefix, + it.w->contents)) + || !NILP (Fget_char_property (make_fixnum (IT_CHARPOS (it)), + Qwrap_prefix, + it.w->contents))))) GIVE_UP (26); start_pos = it.current.pos;
bug-gnu-emacs <at> gnu.org
:bug#79275
; Package emacs
.
(Wed, 20 Aug 2025 15:07:02 GMT) Full text and rfc822 format available.Message #11 received at 79275 <at> debbugs.gnu.org (full text, mbox):
From: "J.D. Smith" <jdtsmith <at> gmail.com> To: Eli Zaretskii <eliz <at> gnu.org> Cc: 79275 <at> debbugs.gnu.org Subject: Re: bug#79275: 30.2.50; overlay line-prefix display property fighting with text display property Date: Wed, 20 Aug 2025 11:06:20 -0400
Eli Zaretskii <eliz <at> gnu.org> writes: >> From: JD Smith <jdtsmith <at> gmail.com> >> Date: Tue, 19 Aug 2025 21:39:38 -0400 >> >> An overlay which applies a line-prefix (e.g. to set the fringe) across several lines conflicts with underlying text which has a replacing display property set. Evaluate the following in the *scratch* buffer, with at least 3 blank lines at the top of the buffer. >> >> (progn >> (delete-all-overlays) >> (let ((ov (make-overlay 1 4))) >> (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display >> '(left-fringe right-triangle success)))) >> (put-text-property 2 3 'display ">testing fringe display")) >> >> The result is very strange: the SHOULDNOTSEETHIS prefix string appears (sans fringe display), but then vanishes on the next redisplay. Some (well-balanced) fight between overlay line-prefix display and normal display properties seems to be occurring. > > Thanks. > > One of the redisplay optimizations we use couldn't cope with this > tricky situation (a line-prefix immediately followed by a display > string at the beginning of a line), and needs to be disabled in this > case. > > Does the patch below give good results? Yes it does, thanks. There still appears to be another corner case, however. If the conflict occurs at the /beginning/ of the overlay: (progn (delete-all-overlays) (let ((ov (make-overlay 1 4))) (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display '(left-fringe right-triangle success)))) (put-text-property 1 2 'display ">testing fringe display")) ;; changed from 2->3 to 1->2 SHOULDNOTSEETHIS still appears (and remains). That situation is what actually motivated the report.
bug-gnu-emacs <at> gnu.org
:bug#79275
; Package emacs
.
(Wed, 20 Aug 2025 16:14:02 GMT) Full text and rfc822 format available.Message #14 received at 79275 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: "J.D. Smith" <jdtsmith <at> gmail.com> Cc: 79275 <at> debbugs.gnu.org Subject: Re: bug#79275: 30.2.50; overlay line-prefix display property fighting with text display property Date: Wed, 20 Aug 2025 19:13:30 +0300
> From: "J.D. Smith" <jdtsmith <at> gmail.com> > Cc: 79275 <at> debbugs.gnu.org > Date: Wed, 20 Aug 2025 11:06:20 -0400 > > Eli Zaretskii <eliz <at> gnu.org> writes: > > >> From: JD Smith <jdtsmith <at> gmail.com> > >> Date: Tue, 19 Aug 2025 21:39:38 -0400 > >> > >> An overlay which applies a line-prefix (e.g. to set the fringe) across several lines conflicts with underlying text which has a replacing display property set. Evaluate the following in the *scratch* buffer, with at least 3 blank lines at the top of the buffer. > >> > >> (progn > >> (delete-all-overlays) > >> (let ((ov (make-overlay 1 4))) > >> (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display > >> '(left-fringe right-triangle success)))) > >> (put-text-property 2 3 'display ">testing fringe display")) > >> > >> The result is very strange: the SHOULDNOTSEETHIS prefix string appears (sans fringe display), but then vanishes on the next redisplay. Some (well-balanced) fight between overlay line-prefix display and normal display properties seems to be occurring. > > > > Thanks. > > > > One of the redisplay optimizations we use couldn't cope with this > > tricky situation (a line-prefix immediately followed by a display > > string at the beginning of a line), and needs to be disabled in this > > case. > > > > Does the patch below give good results? > > Yes it does, thanks. There still appears to be another corner case, > however. If the conflict occurs at the /beginning/ of the overlay: > > (progn > (delete-all-overlays) > (let ((ov (make-overlay 1 4))) > (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display > '(left-fringe right-triangle success)))) > (put-text-property 1 2 'display ">testing fringe display")) ;; changed from 2->3 to 1->2 > > SHOULDNOTSEETHIS still appears (and remains). That situation is what > actually motivated the report. That's a completely different problem, and I've run out of free time for debugging this stuff today. So this will have to wait till I have time again. Or maybe someone else will beat me to it. In the future, please make an effort to describe all of the situations you found that produce unexpected results, especially if some situation was the original motivation for the bug report. It is not easy for me to find enough time to set up a debugging session and step through this tricky code, so I prefer not to invest that overhead more than just once for a given bug report. Please keep that in mind when you decide what to tell and what not to tell, and please don't assume that everything you see is the consequence of the same problem in the code. TIA
bug-gnu-emacs <at> gnu.org
:bug#79275
; Package emacs
.
(Thu, 21 Aug 2025 18:15:01 GMT) Full text and rfc822 format available.Message #17 received at 79275 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: "J.D. Smith" <jdtsmith <at> gmail.com> Cc: 79275 <at> debbugs.gnu.org Subject: Re: bug#79275: 30.2.50; overlay line-prefix display property fighting with text display property Date: Thu, 21 Aug 2025 21:13:58 +0300
> From: "J.D. Smith" <jdtsmith <at> gmail.com> > Cc: 79275 <at> debbugs.gnu.org > Date: Wed, 20 Aug 2025 11:06:20 -0400 > > Eli Zaretskii <eliz <at> gnu.org> writes: > > >> From: JD Smith <jdtsmith <at> gmail.com> > >> Date: Tue, 19 Aug 2025 21:39:38 -0400 > >> > >> An overlay which applies a line-prefix (e.g. to set the fringe) across several lines conflicts with underlying text which has a replacing display property set. Evaluate the following in the *scratch* buffer, with at least 3 blank lines at the top of the buffer. > >> > >> (progn > >> (delete-all-overlays) > >> (let ((ov (make-overlay 1 4))) > >> (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display > >> '(left-fringe right-triangle success)))) > >> (put-text-property 2 3 'display ">testing fringe display")) > >> > >> The result is very strange: the SHOULDNOTSEETHIS prefix string appears (sans fringe display), but then vanishes on the next redisplay. Some (well-balanced) fight between overlay line-prefix display and normal display properties seems to be occurring. > > > > Thanks. > > > > One of the redisplay optimizations we use couldn't cope with this > > tricky situation (a line-prefix immediately followed by a display > > string at the beginning of a line), and needs to be disabled in this > > case. > > > > Does the patch below give good results? > > Yes it does, thanks. There still appears to be another corner case, > however. If the conflict occurs at the /beginning/ of the overlay: > > (progn > (delete-all-overlays) > (let ((ov (make-overlay 1 4))) > (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display > '(left-fringe right-triangle success)))) > (put-text-property 1 2 'display ">testing fringe display")) ;; changed from 2->3 to 1->2 > > SHOULDNOTSEETHIS still appears (and remains). That situation is what > actually motivated the report. Please try the patch below, I hope it fixes both of the situations you described. diff --git a/src/xdisp.c b/src/xdisp.c index 2691296..b8088f6 100644 --- a/src/xdisp.c +++ b/src/xdisp.c @@ -7261,6 +7261,8 @@ push_it (struct it *it, struct text_pos *position) p->from_disp_prop_p = it->from_disp_prop_p; ++it->sp; + it->string_from_prefix_prop_p = false; + /* Save the state of the bidi iterator as well. */ if (it->bidi_p) bidi_push_it (&it->bidi_it); @@ -22657,8 +22659,23 @@ #define GIVE_UP(X) return 0 /* Give up if the row starts with a display property that draws on the fringes, since that could prevent correct display of line-prefix and wrap-prefix. */ - if (it.sp > 1 + if ((it.sp > 1 && it.method == GET_FROM_IMAGE && it.image_id == -1) + /* Give up if there's a line/wrap-prefix property on buffer + text, and the row begins with a display or overlay string. + This is because in that case the iterator state produced by + init_to_row_end is already set to the display/overlay + string, and thus cannot be used to display the prefix + before the display/overlay string. */ + || (it.sp == 1 + && it.method == GET_FROM_STRING + && !it.string_from_prefix_prop_p + && (!NILP (Fget_char_property (make_fixnum (IT_CHARPOS (it)), + Qline_prefix, + it.w->contents)) + || !NILP (Fget_char_property (make_fixnum (IT_CHARPOS (it)), + Qwrap_prefix, + it.w->contents))))) GIVE_UP (26); start_pos = it.current.pos; @@ -24710,15 +24727,29 @@ cursor_row_p (struct glyph_row *row) /* Push the property PROP so that it will be rendered at the current - position in IT. Return true if PROP was successfully pushed, false - otherwise. Called from handle_line_prefix to handle the - `line-prefix' and `wrap-prefix' properties. */ + position in IT. FROM_BUFFER non-zero means the property was found on + buffer text, even though IT is set to iterate a string. + Return true if PROP was successfully pushed, false otherwise. + Called from handle_line_prefix to handle the `line-prefix' and + `wrap-prefix' properties. */ static bool -push_prefix_prop (struct it *it, Lisp_Object prop) +push_prefix_prop (struct it *it, Lisp_Object prop, int from_buffer) { - struct text_pos pos = - STRINGP (it->string) ? it->current.string_pos : it->current.pos; + struct text_pos pos; + + if (STRINGP (it->string)) + { + if (from_buffer) /* a string, but prefix property from buffer */ + pos = it->current.string_pos; + else /* a string and prefix property from string */ + pos.charpos = pos.bytepos = 0; /* we have yet to iterate that string */ + } + else /* a buffer and prefix property from buffer */ + pos = it->current.pos; + + bool phoney_display_string = + from_buffer && STRINGP (it->string) && it->string_from_display_prop_p; eassert (it->method == GET_FROM_BUFFER || it->method == GET_FROM_DISPLAY_VECTOR @@ -24737,6 +24768,13 @@ push_prefix_prop (struct it *it, Lisp_Object prop) it->position not yet set when this function is called. */ push_it (it, &pos); + /* Reset this flag, since it is not relevant (comes from a display + string that follows iterator position). If we don't do that, any + display properties on the prefix string will be ignored. The call + to pop_it when we are done with the prefix will restore the flag. */ + if (phoney_display_string) + it->string_from_display_prop_p = false; + if (STRINGP (prop)) { if (SCHARS (prop) == 0) @@ -24794,7 +24832,7 @@ push_prefix_prop (struct it *it, Lisp_Object prop) #endif /* HAVE_WINDOW_SYSTEM */ else { - pop_it (it); /* bogus display property, give up */ + pop_it (it); /* bogus prefix property, give up */ return false; } @@ -24806,11 +24844,14 @@ push_prefix_prop (struct it *it, Lisp_Object prop) static Lisp_Object get_it_property (struct it *it, Lisp_Object prop) { - Lisp_Object position, object = it->object; + Lisp_Object position, object; - if (STRINGP (object)) - position = make_fixnum (IT_STRING_CHARPOS (*it)); - else if (BUFFERP (object)) + if (STRINGP (it->string)) + { + position = make_fixnum (IT_STRING_CHARPOS (*it)); + object = it->string; + } + else if (BUFFERP (it->object)) { position = make_fixnum (IT_CHARPOS (*it)); object = it->window; @@ -24825,15 +24866,21 @@ get_it_property (struct it *it, Lisp_Object prop) current IT->OBJECT and the underlying buffer text. */ static Lisp_Object -get_line_prefix_it_property (struct it *it, Lisp_Object prop) +get_line_prefix_it_property (struct it *it, Lisp_Object prop, + int *from_buffer) { Lisp_Object prefix = get_it_property (it, prop); + *from_buffer = false; + /* If we are looking at a display or overlay string, check also the underlying buffer text. */ - if (NILP (prefix) && it->sp > 0 && STRINGP (it->object)) - return Fget_char_property (make_fixnum (IT_CHARPOS (*it)), prop, - it->w->contents); + if (NILP (prefix) && it->sp > 0 && STRINGP (it->string)) + { + *from_buffer = true; + return Fget_char_property (make_fixnum (IT_CHARPOS (*it)), prop, + it->w->contents); + } return prefix; } @@ -24844,21 +24891,22 @@ handle_line_prefix (struct it *it) { Lisp_Object prefix; bool wrap_prop = false; + int from_buffer; if (it->continuation_lines_width > 0) { - prefix = get_line_prefix_it_property (it, Qwrap_prefix); + prefix = get_line_prefix_it_property (it, Qwrap_prefix, &from_buffer); if (NILP (prefix)) prefix = Vwrap_prefix; wrap_prop = true; } else { - prefix = get_line_prefix_it_property (it, Qline_prefix); + prefix = get_line_prefix_it_property (it, Qline_prefix, &from_buffer); if (NILP (prefix)) prefix = Vline_prefix; } - if (! NILP (prefix) && push_prefix_prop (it, prefix)) + if (! NILP (prefix) && push_prefix_prop (it, prefix, from_buffer)) { /* If the prefix is wider than the window, and we try to wrap it, it would acquire its own wrap prefix, and so on till the
bug-gnu-emacs <at> gnu.org
:bug#79275
; Package emacs
.
(Thu, 21 Aug 2025 19:47:02 GMT) Full text and rfc822 format available.Message #20 received at 79275 <at> debbugs.gnu.org (full text, mbox):
From: "J.D. Smith" <jdtsmith <at> gmail.com> To: Eli Zaretskii <eliz <at> gnu.org> Cc: 79275 <at> debbugs.gnu.org Subject: Re: bug#79275: 30.2.50; overlay line-prefix display property fighting with text display property Date: Thu, 21 Aug 2025 15:46:03 -0400
Eli Zaretskii <eliz <at> gnu.org> writes: > Please try the patch below, I hope it fixes both of the situations you > described. Your latest patch indeed fixes this display conflict between overlays and text properties at both the front of a fringe-displaying overlay and in its middle. The code I used to test: ;; Insert at least 4 blank lines above this and evaluate (progn (delete-all-overlays) (let ((ov (make-overlay 1 5))) (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display '(left-fringe right-triangle success)))) (put-text-property 1 2 'display ">testing front-fringe display\n") (put-text-property 3 4 'display ">testing mid-fringe display\n")) Thanks very much for your work on this. I know how challenging it can be to hunt down subtle bugs in the redisplay code. I appreciate all the time and energy you volunteer to continuously improve Emacs.
bug-gnu-emacs <at> gnu.org
:bug#79275
; Package emacs
.
(Fri, 22 Aug 2025 06:50:02 GMT) Full text and rfc822 format available.Message #23 received at 79275 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: "J.D. Smith" <jdtsmith <at> gmail.com> Cc: 79275 <at> debbugs.gnu.org Subject: Re: bug#79275: 30.2.50; overlay line-prefix display property fighting with text display property Date: Fri, 22 Aug 2025 09:49:04 +0300
> From: "J.D. Smith" <jdtsmith <at> gmail.com> > Cc: 79275 <at> debbugs.gnu.org > Date: Thu, 21 Aug 2025 15:46:03 -0400 > > Eli Zaretskii <eliz <at> gnu.org> writes: > > > Please try the patch below, I hope it fixes both of the situations you > > described. > > Your latest patch indeed fixes this display conflict between overlays > and text properties at both the front of a fringe-displaying overlay and > in its middle. The code I used to test: > > ;; Insert at least 4 blank lines above this and evaluate > (progn > (delete-all-overlays) > (let ((ov (make-overlay 1 5))) > (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display > '(left-fringe right-triangle success)))) > (put-text-property 1 2 'display ">testing front-fringe display\n") > (put-text-property 3 4 'display ">testing mid-fringe display\n")) > > Thanks very much for your work on this. I know how challenging it can > be to hunt down subtle bugs in the redisplay code. I appreciate all the > time and energy you volunteer to continuously improve Emacs. Thanks for testing. I will run a few more tests before installing this: as you could see, this bug revealed a couple of serious design blunders in how line/wrap-prefix was implemented, so I'd like to make sure the changes didn't break anything. What astonished me the most was the use of it->object to decide whether we iterate a buffer or a string, something that I learned long ago (and forgot) to be a very bad idea, see this comment in dispextern.h: Do NOT use !BUFFERP (it.object) as a test whether we are iterating over a string; use STRINGP (it.string) instead.
Eli Zaretskii <eliz <at> gnu.org>
:JD Smith <jdtsmith <at> gmail.com>
:Message #28 received at 79275-done <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: jdtsmith <at> gmail.com Cc: 79275-done <at> debbugs.gnu.org Subject: Re: bug#79275: 30.2.50; overlay line-prefix display property fighting with text display property Date: Sat, 23 Aug 2025 11:38:56 +0300
> Cc: 79275 <at> debbugs.gnu.org > Date: Fri, 22 Aug 2025 09:49:04 +0300 > From: Eli Zaretskii <eliz <at> gnu.org> > > > From: "J.D. Smith" <jdtsmith <at> gmail.com> > > Cc: 79275 <at> debbugs.gnu.org > > Date: Thu, 21 Aug 2025 15:46:03 -0400 > > > > Eli Zaretskii <eliz <at> gnu.org> writes: > > > > > Please try the patch below, I hope it fixes both of the situations you > > > described. > > > > Your latest patch indeed fixes this display conflict between overlays > > and text properties at both the front of a fringe-displaying overlay and > > in its middle. The code I used to test: > > > > ;; Insert at least 4 blank lines above this and evaluate > > (progn > > (delete-all-overlays) > > (let ((ov (make-overlay 1 5))) > > (overlay-put ov 'line-prefix (propertize "SHOULDNOTSEETHIS" 'display > > '(left-fringe right-triangle success)))) > > (put-text-property 1 2 'display ">testing front-fringe display\n") > > (put-text-property 3 4 'display ">testing mid-fringe display\n")) > > > > Thanks very much for your work on this. I know how challenging it can > > be to hunt down subtle bugs in the redisplay code. I appreciate all the > > time and energy you volunteer to continuously improve Emacs. > > Thanks for testing. I will run a few more tests before installing > this: as you could see, this bug revealed a couple of serious design > blunders in how line/wrap-prefix was implemented, so I'd like to make > sure the changes didn't break anything. > > What astonished me the most was the use of it->object to decide > whether we iterate a buffer or a string, something that I learned long > ago (and forgot) to be a very bad idea, see this comment in > dispextern.h: > > Do NOT use !BUFFERP (it.object) as a test whether we are > iterating over a string; use STRINGP (it.string) instead. > The fix is now installed on the master branch. Closing the bug.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.