Subject: Re: bug#56682: Fix the long lines font locking related slowdowns

From: Dmitry Gutov <[email-masked]>
Date: Tue, 16 Aug 2022 00:18:35 +0300

On 15.08.2022 21:28, Eli Zaretskii wrote:
>> Date: Mon, 15 Aug 2022 21:17:23 +0300
>> Cc: [email-masked], [email-masked], [email-masked]
>> From: Dmitry Gutov <[email-masked]>
>>
>>>> I'm pretty sure that doing all that in Lisp is a good thing, and would
>>>> be conducive to whatever improvements we might want to add later...
>>>
>>> No, because quite a few packages use jit-lock-register to run stuff
>>> that is not font-lock at all, and I don't want that to slow down
>>> redisplay when long lines are involved.  And trying various values of
>>> a variable exposed to Lisp is easy enough.
>>
>> It should also be possible to move the whole invocation of the
>> fontification-functions up to Lisp
> 
> How do you do that, given that these functions should be called from
> redisplay with the region of buffer text only redisplay knows about?

Something like in the attached patch.

>> Another reason I chose font-lock-fontify-region, though, is that it's
>> called from different places. It's the common entry point for font-lock.
> 
> I have no problem with doing something similar in font-lock, I'm just
> saying that it cannot be the only place.

If we apply narrowing in handle_fontified_prop (or like in the 
attachment), and also in font-lock-fontify-region, it seems like every 
significant (*) execution path will go through two repetitions of the 
same logic.

That's not ideal.

(*) Significant by the metric of having impact on Emacs's performance, 
in large files especially.

>> I've never seen fontification-functions contain anything but
>> jit-lock-function, but jit-lock-functions is used by nlinum, for
>> instance. And bug-reference-mode, and goto-address-mode. Perhaps we
>> should ask Stefan's opinion on this.
> 
> Stefan was and is on the CC list for the entire discussion.
> 
>> And nlinum might not appreciate being narrowed. It seems to be working
>> okay without that in my 20 MB XML file. And in 200 MB one too.
> 
> Why is it important what nlinum does, when we have native line numbers
> nowadays?

Just an example of a mode which uses jit-lock-functions. With an 
implication that other, less known modes, might have similar needs and 
behavior. There aren't too many known examples.

diff --git a/lisp/font-core.el b/lisp/font-core.el
index f70c42bb03..09e55bcfb1 100644
--- a/lisp/font-core.el
+++ b/lisp/font-core.el
@@ -266,6 +266,12 @@ turn-on-font-lock-if-desired
     (let (inhibit-quit)
       (turn-on-font-lock))))
 
+(defun run-fontification-functions (pos)
+  ;; XXX: Apply narrowing around here.
+  ;; (pcase-exhaustive font-lock-large-files
+  ;;   ...)
+  (run-hook-with-args 'fontification-functions pos))
+
 (define-globalized-minor-mode global-font-lock-mode
   font-lock-mode turn-on-font-lock-if-desired
   ;; What was this :extra-args thingy for?  --Stef
diff --git a/src/xdisp.c b/src/xdisp.c
index 1e8f70b2db..ce90aeefbc 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -4381,53 +4381,18 @@ handle_fontified_prop (struct it *it)
 	  NILP (prop) && IT_CHARPOS (*it) < Z))
     {
       specpdl_ref count = SPECPDL_INDEX ();
-      Lisp_Object val;
       struct buffer *obuf = current_buffer;
       ptrdiff_t begv = BEGV, zv = ZV;
       bool old_clip_changed = current_buffer->clip_changed;
       bool saved_inhibit_flag = it->f->inhibit_clear_image_cache;
 
-      val = Vfontification_functions;
-      specbind (Qfontification_functions, Qnil);
-
       eassert (it->end_charpos == ZV);
 
       /* Don't allow Lisp that runs from 'fontification-functions'
 	 clear our face and image caches behind our back.  */
       it->f->inhibit_clear_image_cache = true;
 
-      if (!CONSP (val) || EQ (XCAR (val), Qlambda))
-	safe_call1 (val, pos);
-      else
-	{
-	  Lisp_Object fns, fn;
-
-	  fns = Qnil;
-
-	  for (; CONSP (val); val = XCDR (val))
-	    {
-	      fn = XCAR (val);
-
-	      if (EQ (fn, Qt))
-		{
-		  /* A value of t indicates this hook has a local
-		     binding; it means to run the global binding too.
-		     In a global value, t should not occur.  If it
-		     does, we must ignore it to avoid an endless
-		     loop.  */
-		  for (fns = Fdefault_value (Qfontification_functions);
-		       CONSP (fns);
-		       fns = XCDR (fns))
-		    {
-		      fn = XCAR (fns);
-		      if (!EQ (fn, Qt))
-			safe_call1 (fn, pos);
-		    }
-		}
-	      else
-		safe_call1 (fn, pos);
-	    }
-	}
+      safe_call1 (intern ("run-fontification-functions"), pos);
 
       it->f->inhibit_clear_image_cache = saved_inhibit_flag;
       unbind_to (count, Qnil);