Subject: Re: bug#56682: Fix the long lines font locking related slowdowns

From: Dmitry Gutov <[email-masked]>
Date: Tue, 2 Aug 2022 17:47:12 +0300

On 02.08.2022 15:35, Eli Zaretskii wrote:

>> - syntax-ppss is also important for correctness: for commands to
>> understand whether the point is inside a string, comments, etc. So it's
>> better to avoid applying narrowing when calling it. Unless you're in a
>> multiple-major-modes situation.
>> - font-lock calls syntax-ppss.
> 
> I believe I was talking about syntax-ppss being called from font-lock,
> indeed.  Before Gregory's changes, if you visit a large file with very
> long lines, and interrupt Emacs while it is non-responsive, you will
> in many/most cases find yourself in syntax-propertize or its
> subroutines, and you will see that it is almost always called to
> traverse the entire long line.

Interrupt it right after pressing 'M->'? Or at any time during editing 
the buffer later?

The latter really shouldn't happen. If it does, perhaps it's the result 
of narrowing during redisplay, which might blow syntax-ppss's caches.

In any case, if you could point to a scenario and the revision to test 
it on, I'll be sure to take a look.

>> So ideally font-lock is either called with undo-able narrowing, or is
>> simply passed a range of positions, and shouldn't fontify too far from them.
> 
> Many major-modes do widen the buffer, though.

Whether they do or not, font-lock widens by default, see 
font-lock-dont-widen.

>> The latter seems to be the case already (if you open xdisp.c and press
>> M->, only top and bottom of the buffer are fontified)
> 
> It is not enough to look for faces in order to realize how much of the
> buffer was scanned.

I evaluated (next-single-property-change 1 'fontified), when near BOB 
and when near EOB.

>> with the caveat that font-lock always tries to backtrack to BOL when
>> fontifying the current hunk. Which makes sense, of course, but could
>> be tweaked for long lines to avoid re-fontifying the whole buffer
>> again and again.
> 
> "Tweaked" how?

15b2138719b34083 is one example.

>> IOW, IIUC the fix for font-lock performance could be better implemented
>> inside font-lock itself, as long as all the info about whether the
>> current line is "long" is available to Lisp.
> 
> No one will object to making font-lock faster.  But the experts who
> can do that are few and far in-between, and seem to have other itches
> to scratch, since these issues are known for a long time, and several
> times were even discussed at length.

The fact that we have +1 contributor to the C part of Emacs (the display 
engine, etc), and a successful one at that, does nothing about the fact 
that Lisp is easier to write and debug.

If we're able to demonstrate that the remaining bottlenecks are inside 
font-lock, it should be easier to improve there.