Subject: bug#56682: Fix the long lines font locking related slowdowns

From: Dmitry Gutov <[email-masked]>
Date: Sat, 6 Aug 2022 13:50:41 +0300

On 06.08.2022 09:07, Eli Zaretskii wrote:
>> Date: Fri, 5 Aug 2022 23:23:25 +0300
>> Cc: [email-masked], [email-masked], [email-masked]
>> From: Dmitry Gutov <[email-masked]>
>>
>>> We need to look at more than just M->.  C-n/C-p, C-v/M-v, C-l are also
>>> important, as are the time it takes from typing M-x or M-: until you
>>> see the prompt in the minibuffer, and the time to update the display
>>> after inserting or deleting a single character.
>>
>> I'm not seeing any particular sluggishness in these operations when
>> visiting dictionary.json.
> 
> Numbers, please.  You have a very fast machine, so what doesn't look
> sluggish on your system could very well be so on others.

How do you measure these operations including the redisplay lag?

Anyway, all of these look instant.

>>> Thanks, so I guess we may have a solution for JSON files, if disabling
>>> syntax-propertize-function doesn't have any downsides.  What about
>>> other modes that we see in files with long lines, like XML?
>>
>> Someone will need to test it with some typical large file. xml-mode
>> (alias to nxml-mode) does have a syntax-propertize-function, but it's
>> probably faster than js-syntax-propertize.
>>
>>> And how scalable is the solution you propose, i.e. how it behaves in
>>> JSON files with a much longer lines?
>>
>> parse-partial-sexp is O(length of text span)
> 
> Isn't that the part you proposed to remove?  I was asking about the
> remaining parts.

No, removing syntax-propertize-function just made parse-partial-sexp 5x 
faster here, resulting in much faster fontification near EOB.

But in a large enough buffer, it's still going to be the slowest part.

>> Meaning, it scales linearly. You'll see a 10x delay in a JSON file that
>> is 10x as large.
> 
> Linear scaling is less optimal than O(0), which is what the current
> solution produces.

Correct syntax highlighting requires parsing the buffer from the 
beginning. Otherwise we get random results, essentially.

Just like depicted on my latest screenshot for downloadify.js: the 
beginning of the narrowed region ended up inside a string, and as a 
result several screenfuls were entirely mis-fontified, with strings and 
non-strings inverted.