Subject: Re: bug#71644: 30.0.50; Severe slowdown in larger files with markers beginning in emacs 29+

From: Stefan Monnier <[email-masked]>
Date: Wed, 26 Jun 2024 09:30:25 -0400

>> - The markers histogram is weirdly flat.
>> - The markers histogram seems to have a threshold around 800
>>   I can't explain.  It seems directly related to the buffer size, tho:
>>   with a file half the size, I get half as many markers (40k) and
>>   a threshold in the histogram around 400.
>> - The number of times we consider exactly 1 marker is probably "too
>>   high" because the C code happens to look at the first marker before
>>   checking if it's necessary.
>
> The fact that it's related to buffer size is probably because your
> test buffer is basically N >> 1 repetitions of the same pattern, so
> whatever Org does it does that N times, where N is directly
> proportional to the buffer size.

Yes, definitely.  The threshold is at about N/100 where N is the number
of markers/headings.  But why 100?

> How many markers are there in this buffer during the experiment where
> you produced the histogram you've posted?  My guess is 48400 or
> thereabouts?

It's the first number in the list (the one before `:markers`): 80044

> Also, what were the buffer positions where you tried your editing
> operations?  In particular, where those positions distributed evenly
> over the buffer?

I focused on edits on the first copy and on the last copy.

> And questions to Ihor: what are the reasons Org creates these markers?
> In particular, is there any way to know or guess whether these markers
> will be distributed more-or-less evenly over the buffer text, or
> clustered near some specific positions?  Also, does Org move these
> markers frequently, or do they stay put once created?

They're created by `consult-outline` rather than by Org.
There's one marker per heading, AFAICT, and so they're very
evenly spread (and presumably ordered last-to-first in the `markers`
list).

        Stefan