GNU bug report logs - #65105
Reusing the same string as 'display on consecutive characters evades display

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 65105 in the body.
You can then email your comments to 65105 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Reusing the same string as 'display on consecutive characters evades
 display
Date: Sat, 5 Aug 2023 14:35:23 -0400

Evaluate:

(let ((s1 "test1")
      (s2 "test2"))
  (insert "\n"
          (propertize " " 'display s1)
          (propertize " " 'display s1)
          (propertize " " 'display s2)
          (propertize " " 'display s1)))


The first space display does not take effect, since the s1 string is used for two consecutive characters.  This has a practical impact for font-lock backends that use the ‘display text-property and would like to minimize string allocation.  

Tested Emacs 27/28/29.

Message #8 received at 65105 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: JD Smith <jdtsmith <at> gmail.com>
Cc: 65105 <at> debbugs.gnu.org
Subject: Re: bug#65105: Reusing the same string as 'display on consecutive
 characters evades display
Date: Sat, 05 Aug 2023 22:03:35 +0300

tags 65105 notabug
thanks

> From: JD Smith <jdtsmith <at> gmail.com>
> Date: Sat, 5 Aug 2023 14:35:23 -0400
> 
> Evaluate:
> 
> (let ((s1 "test1")
>       (s2 "test2"))
>   (insert "\n"
>           (propertize " " 'display s1)
>           (propertize " " 'display s1)
>           (propertize " " 'display s2)
>           (propertize " " 'display s1)))
> 
> 
> The first space display does not take effect, since the s1 string is used for two consecutive characters.  This has a practical impact for font-lock backends that use the ‘display text-property and would like to minimize string allocation.  

Emacs cannot distinguish between two consecutive characters having
each a text property with the same value, and two characters having
the same property.  If you think about this for a moment, you will
understand why: we use intervals for text properties, so two adjacent
intervals with the identical property values and one interval with
that same value are indistinguishable (and in fact Emacs optimizes
this during GC by making just one interval from these two).

This is not a bug.

Message #13 received at 65105 <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 65105 <at> debbugs.gnu.org
Subject: Re: bug#65105: Reusing the same string as 'display on consecutive
 characters evades display
Date: Sat, 5 Aug 2023 16:49:33 -0400

[Message part 1 (text/plain, inline)]

> On Aug 5, 2023, at 3:03 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
> tags 65105 notabug
> thanks
> 
>> From: JD Smith <jdtsmith <at> gmail.com>
>> Date: Sat, 5 Aug 2023 14:35:23 -0400
>> 
>> Evaluate:
>> 
>> (let ((s1 "test1")
>>      (s2 "test2"))
>>  (insert "\n"
>>          (propertize " " 'display s1)
>>          (propertize " " 'display s1)
>>          (propertize " " 'display s2)
>>          (propertize " " 'display s1)))
>> 
>> 
>> The first space display does not take effect, since the s1 string is used for two consecutive characters.  This has a practical impact for font-lock backends that use the ‘display text-property and would like to minimize string allocation.  
> 
> Emacs cannot distinguish between two consecutive characters having
> each a text property with the same value, and two characters having
> the same property.  If you think about this for a moment, you will
> understand why: we use intervals for text properties, so two adjacent
> intervals with the identical property values and one interval with
> that same value are indistinguishable (and in fact Emacs optimizes
> this during GC by making just one interval from these two).
> 
> This is not a bug.

Aha, thanks.  It does make sense from an optimization standpoint to “gang” properties in this manner.  Are you aware of any approach that allow re-using a string for ‘display, but permits consecutive intervals to remain distinct?

[Message part 2 (text/html, inline)]

Message #16 received at 65105 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: JD Smith <jdtsmith <at> gmail.com>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 65105 <at> debbugs.gnu.org
Subject: Re: bug#65105: Reusing the same string as 'display on consecutive
 characters evades display
Date: Sun, 6 Aug 2023 01:46:49 +0300

On 05/08/2023 23:49, JD Smith wrote:
> Aha, thanks.  It does make sense from an optimization standpoint to 
> “gang” properties in this manner.  Are you aware of any approach that 
> allow re-using a string for ‘display, but permits consecutive intervals 
> to remain distinct?

Overlays? But they have certain performance problems when their number 
grows.

Or you could try resolving this manually: when the previous character is 
assigned with the same line, do a copy-sequence, and when it's a 
different one, you can use the "singleton" one.

Message #19 received at 65105 <at> debbugs.gnu.org (full text, mbox):

From: JD Smith <jdtsmith <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 65105 <at> debbugs.gnu.org
Subject: Re: bug#65105: Reusing the same string as 'display on consecutive
 characters evades display
Date: Sat, 5 Aug 2023 18:49:11 -0400

That’s probably the right approach.  Or even simpler, allocate two identical strings and alternate.  Thanks.

> On Aug 5, 2023, at 6:46 PM, Dmitry Gutov <dmitry <at> gutov.dev> wrote:
> 
> On 05/08/2023 23:49, JD Smith wrote:
>> Aha, thanks.  It does make sense from an optimization standpoint to “gang” properties in this manner.  Are you aware of any approach that allow re-using a string for ‘display, but permits consecutive intervals to remain distinct?
> 
> Overlays? But they have certain performance problems when their number grows.
> 
> Or you could try resolving this manually: when the previous character is assigned with the same line, do a copy-sequence, and when it's a different one, you can use the "singleton" one.

Message #22 received at 65105 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 65105 <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#65105: Reusing the same string as 'display on consecutive
 characters evades display
Date: Sun, 06 Aug 2023 07:54:47 +0300

> Date: Sun, 6 Aug 2023 01:46:49 +0300
> Cc: 65105 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> On 05/08/2023 23:49, JD Smith wrote:
> > Aha, thanks.  It does make sense from an optimization standpoint to 
> > “gang” properties in this manner.  Are you aware of any approach that 
> > allow re-using a string for ‘display, but permits consecutive intervals 
> > to remain distinct?
> 
> Overlays? But they have certain performance problems when their number 
> grows.

Not in Emacs 29 and later, right?

Message #27 received at 65105-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: JD Smith <jdtsmith <at> gmail.com>
Cc: dmitry <at> gutov.dev, 65105-done <at> debbugs.gnu.org
Subject: Re: bug#65105: Reusing the same string as 'display on consecutive
 characters evades display
Date: Sun, 06 Aug 2023 07:55:25 +0300

> From: JD Smith <jdtsmith <at> gmail.com>
> Date: Sat, 5 Aug 2023 18:49:11 -0400
> Cc: Eli Zaretskii <eliz <at> gnu.org>,
>  65105 <at> debbugs.gnu.org
> 
> That’s probably the right approach.  Or even simpler, allocate two identical strings and alternate.  Thanks.

And with that, I'm closing this bug.

Message #30 received at 65105 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 65105 <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#65105: Reusing the same string as 'display on consecutive
 characters evades display
Date: Sun, 6 Aug 2023 20:48:49 +0300

On 06/08/2023 07:54, Eli Zaretskii wrote:
>> Date: Sun, 6 Aug 2023 01:46:49 +0300
>> Cc:65105 <at> debbugs.gnu.org
>> From: Dmitry Gutov<dmitry <at> gutov.dev>
>>
>> On 05/08/2023 23:49, JD Smith wrote:
>>> Aha, thanks.  It does make sense from an optimization standpoint to
>>> “gang” properties in this manner.  Are you aware of any approach that
>>> allow re-using a string for ‘display, but permits consecutive intervals
>>> to remain distinct?
>> Overlays? But they have certain performance problems when their number
>> grows.
> Not in Emacs 29 and later, right?

Hopefully not -- I haven't benchmarked any particular problem cases 
recently.

But I'm guessing the package JD is working on aims to support previous 
versions anyway.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.