Subject: bug#77924: 31.0.50; [Feature branch] Change marker implementation

From: Eli Zaretskii <[email-masked]>
Date: Thu, 24 Apr 2025 08:01:10 +0300

> From: Stefan Monnier <[email-masked]>
> Cc: [email-masked],  [email-masked],
> 	[email-masked], Ihor Radchenko <[email-masked]>
> Date: Wed, 23 Apr 2025 11:41:43 -0400
> 
> > Btw, did I miss some benchmarks?  I presume that the main motivation
> > for this branch is to speed up markers and charpos-to-bytepos
> > conversions, is that right?  If so, the only benchmark I saw shows only
> > a very mild speed-up, but maybe I missed something?
> 
> AFAICT we have tuned the current charpos<->bytepos conversion code such
> that its worst case virtually never shows up in practice.
> 
> Supposedly Ihor has a use case where that conversion still can make
> a noticeable difference, but I don't know if he tried the `text-index`
> branch to confirm that it makes a measurable difference on his case.
> 
> Gerd and I played with some micro benchmarks to confirm that the
> pathological behaviors are indeed solved, but other than that, the focus
> was on making sure the branch does not make the normal case slower
> (contrary to my sorted-array-of-markers-with-gap).

Still, for performance-oriented changes, it would be nice to see some
benchmarks.  Scrolling through some large file with many non-ASCII
characters comes to mind (the Unicode UCD has quite a few of such
files).

Also, I believe there were cases where a lot of conversions
created gobs of markers which slowed down things; it would be good to
run those cases to see how they are affected.

And finally, how about running the whole benchmark suite we have, in
case it uncovers some unexpected results?

Thanks.