Subject: bug#77620: closed (Re: bug#77620: thingatpt can be incredibly slow in python-mode buffers)

From: [email-masked] (GNU bug Tracking System)
Date: Sat, 26 Apr 2025 11:24:02 +0000

Your bug report

#77620: thingatpt can be incredibly slow in python-mode buffers

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
77620: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=77620
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#77620: thingatpt can be incredibly slow in python-mode buffers
From: Eli Zaretskii <[email-masked]>
To: kobarity <[email-masked]>
Cc: [email-masked], [email-masked]
Date: Sat, 26 Apr 2025 14:22:59 +0300
List-Id: <debbugs-submit.debbugs.gnu.org>

> Date: Mon, 21 Apr 2025 23:22:14 +0900
> From: kobarity <[email-masked]>
> Cc: [email-masked]
> 
> kobarity wrote:
> > In the process of looking into #77620, I noticed bugs in the
> > end-of-block detection functions.  So I submitted a patch to #77941.
> > Please note that the patch attached to this mail assumes the #77941
> > patch.
> > 
> > In my environment, the next test shows about 2 seconds on _axes.py's
> > L8817.
> > 
> > > (/ (car (benchmark-run 10 (thing-at-point 'sexp))) 10)
> > 
> > But there are worse cases.  Let's add the next two lines to the end of
> > _axes.py with 4-character indent.
> > 
> >     a = 1
> >     b = 2
> > 
> > The same measurement at the end of line "a = 1" gives 6 seconds; at
> > the end of line "b = 2" it gives 7 seconds.  This is because it calls
> > time-consuming functions many times.
> > 
> > `python-nav-beginning-of-block' and `python-nav-end-of-block' are very
> > time consuming if the block is large.  In the case of _axes.py, "class
> > Axes" is a large block with over 8000 lines.
> > 
> > However, `python-nav-forward-sexp' that (thing-at-point 'sexp) calls
> > does not necessarily have to perform these functions.  They are just
> > called by `python-info-statement-ends-block-p': (Before #77941 patch)
> > 
> > (defun python-info-statement-ends-block-p ()
> >   "Return non-nil if point is at end of block."
> >   (let ((end-of-block-pos (save-excursion
> >                             (python-nav-end-of-block)))
> >         (end-of-statement-pos (save-excursion
> >                                 (python-nav-end-of-statement))))
> >     (and end-of-block-pos end-of-statement-pos
> >          (= end-of-block-pos end-of-statement-pos))))
> > 
> > The logic is that if the end of the current statement is the end of
> > the current block, then it is the last line of the block.  This is not
> > wrong, but the problem is that it takes time to move to the end of the
> > block.
> > 
> > Fortunately, in many cases it is much easier to determine that it is
> > not the end of the block.  Specifically, if the indentation of the
> > next statement is equal to or greater than the indentation of the
> > current statement, then the current statement cannot be the end of the
> > block.  The attached patch
> > 0001-Performance-optimization-of-python-info-statement-en.patch takes
> > advantage of this to improve the performance of
> > `python-info-statement-ends-block-p'.  Please remember that it assumes
> > the #77941 patch is applied.
> > 
> > 0001-Performance-optimization-of-python-info-statement-en.patch solves
> > the performance problem with _axes.py, but does not have much effect
> > on the problem with lines "a = 1" and "b = 2" added above.  The
> > attached patch
> > 0002-Add-cache-to-Python-block-navigation-functions.patch is an
> > attempt to introduce a cache to alleviate this problem.
> > 
> > I measured the following to disable the cache for each benchmark.
> > 
> > (/ (car (benchmark-run 10 (progn (thing-at-point 'sexp) (setq python-nav-cache nil)))) 10)
> > 
> > It shows about 1.6 seconds for both "a = 1" and "b = 2" lines.
> > 
> > I believe 0002-Add-cache-to-Python-block-navigation-functions.patch is
> > a PoC level patch, so I welcome opinions on what you think of this
> > direction.
> 
> I revised 0002-Add-cache-to-Python-block-navigation-functions.patch.
> There was no need to use a macro.

Thanks, installed, and closing the bug.

Subject: thingatpt can be incredibly slow in python-mode buffers
From: JD Smith <[email-masked]>
To: [email-masked]
Date: Mon, 7 Apr 2025 17:48:49 -0400
List-Id: <debbugs-submit.debbugs.gnu.org>

As discussed in Bug#77588:

> (/ (car (benchmark-run 10 (thing-at-point 'sexp))) 10)

Try this test at the start of L8817 in this file (no eglot needed; either python-mode or python-ts-mode is fine, as both show the same issue):

https://raw.githubusercontent.com/matplotlib/matplotlib/refs/heads/main/lib/matplotlib/axes/_axes.py
https://raw.githubusercontent.com/matplotlib/matplotlib/refs/heads/main/lib/matplotlib/axes/_axes.py￼
_axes
Text Document · 353 KB

In #77588 this was shown to lead to long intermittent pauses with eglot in large python buffers, but the issue is more general for any packages using thingatpt in python buffers.