Subject: bug#56682: Fix the long lines font locking related slowdowns

From: Dmitry Gutov <[email-masked]>
Date: Tue, 2 Aug 2022 04:05:57 +0300

On 01.08.2022 15:08, Eli Zaretskii wrote:
>> Date: Mon, 1 Aug 2022 04:23:21 +0300
>> Cc: [email-masked], Eli Zaretskii <[email-masked]>,
>>   Stefan Monnier <[email-masked]>
>> From: Dmitry Gutov <[email-masked]>
>>
>> IIUC this state of affairs is caused by your chosen approach to speeding
>> up font-lock (hard narrowing while it is called), which makes the
>> initial call to syntax-ppss happen inside that narrowing as well.
>>
>> The alternative being that font-lock would call syntax-ppss right away
>> with no restriction, but then only apply highlighting to limited parts
>> of the buffer.
> 
> AFAIU, this seems to assume that highlighting is much faster than
> syntax-ppss.  Is that a given?  If not, I don't think I understand how
> this could help.

I don't have the concrete numbers at hand, but from experience I'd say:

- syntax-ppss over the whole buffer is fast-ish. But it takes O(N) time 
of course, and the bigger the buffer is, the longer it'll take. Not much 
we can do about it.
- font-lock has to do more work, so over the whole buffer it will take 
an order of a magnitude more time than syntax-ppss.

Further:

- syntax-ppss is also important for correctness: for commands to 
understand whether the point is inside a string, comments, etc. So it's 
better to avoid applying narrowing when calling it. Unless you're in a 
multiple-major-modes situation.
- font-lock calls syntax-ppss.

So ideally font-lock is either called with undo-able narrowing, or is 
simply passed a range of positions, and shouldn't fontify too far from them.

The latter seems to be the case already (if you open xdisp.c and press 
M->, only top and bottom of the buffer are fontified), with the caveat 
that font-lock always tries to backtrack to BOL when fontifying the 
current hunk. Which makes sense, of course, but could be tweaked for 
long lines to avoid re-fontifying the whole buffer again and again.

IOW, IIUC the fix for font-lock performance could be better implemented 
inside font-lock itself, as long as all the info about whether the 
current line is "long" is available to Lisp.