Subject: bug#56682: Fix the long lines font locking related slowdowns

From: Dmitry Gutov <[email-masked]>
Date: Sat, 13 Aug 2022 22:19:26 +0300

On 13.08.2022 20:43, Gregory Heytings wrote:
> 
>>> I already did, and told you that these multi-second delays were still 
>>> present in larger JSON files (and therefore also in smaller files 
>>> with slower CPUs).
>>
>> Delays or "a delay"? Just the first time you scroll to EOB? Or did you 
>> do the whole dance with editing near BOB, then goto EOB, and repeat?
>>
> 
> Delays.  When scrolling to EOB and when using isearch, for example.
> 
>>>> Then the goal failed because we don't disable bidi hpa? It has a 
>>>> much more noticeable effect on editing that font-lock.
>>>
>>> That's not correct, no.  The BPA algorithm makes C-n and C-p slower 
>>> at the beginning of large files only in a very specific case: when 
>>> the beginning of the buffer contains opening brackets whose matching 
>>> closing bracket is at the end of the buffer.
>>
>> To be fair, that's pretty much all JSON files.
>>
> 
> Again, the aim of this effort has _nothing_ to do with JSON files.

What kind of files are your main target?

>> The branch:
>>
>> 1) Allows everyone interested to evaluate the performance of 
>> unrestricted font-lock even in large files (single-line or not) and 
>> see how big on a problem the delays caused by syntax-ppss actually are 
>> in their experience. It's an important question.
>>
>> 2) Figure out the file sizes where syntax-ppss's performance really 
>> does become a problem. That can give us data for better defaults later.
>>
> 
> Buffers with long lines are rare enough.  It wouldn't make sense (and 
> would be a nightmare in terms of long term maintenance) to add 25 
> defcustoms to allow anyone to fine-tune what Emacs does in such buffers.

I have regularly encountered redisplay slowdown caused by long lines in 
my work. One doesn't need to have a 18 MB files to see them.

Having a 5000-10000 character line is enough to see redisplay starting 
to stutter. And, of course, there's nothing font-lock related in those 
stutters.

>> 3) Play around with two easy solutions that we discussed previously: 
>> narrowing during font-lock (one of the values for 
>> font-lock-large-files pretty much matches the current behavior on 
>> master), or fontifying only the first X characters (e.g. 1000000) of 
>> the buffer, and skipping on the rest.
>>
>> 4) It should be plain to see that implementing additional approaches 
>> should be easy enough. For instance, a hybrid like "fontify the first 
>> 1 MB correctly, and the rest - on best-effort basis". Although the 
>> value '(narrow . 1000000) should provide behavior a very similar 
>> behavior already. Maybe ever a better one: the boundaries are stable. 
>> Maybe sure to try it together with (setq bidi-inhibit-hpa t): the 
>> result looks very fast to me.
>>
> 
> These 3) and 4) assume that the major (and minor) modes are 
> well-behaving, and will obey the restrictions.  Which isn't true, and 
> was the reason why locked narrowing was introduced.

What are your examples of misbehaving major/minor modes?