GNU bug report logs - #77620
thingatpt can be incredibly slow in python-mode buffers

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#77620: closed (thingatpt can be incredibly slow in
 python-mode buffers)
Date: Sat, 26 Apr 2025 11:24:01 +0000

[Message part 1 (text/plain, inline)]

Your message dated Sat, 26 Apr 2025 14:22:59 +0300
with message-id <86ikmrupyk.fsf <at> gnu.org>
and subject line Re: bug#77620: thingatpt can be incredibly slow in python-mode buffers
has caused the debbugs.gnu.org bug report #77620,
regarding thingatpt can be incredibly slow in python-mode buffers
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
77620: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=77620
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: JD Smith <jdtsmith <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: thingatpt can be incredibly slow in python-mode buffers
Date: Mon, 7 Apr 2025 17:48:49 -0400

[Message part 3 (text/plain, inline)]

As discussed in Bug#77588:

> (/ (car (benchmark-run 10 (thing-at-point 'sexp))) 10)


Try this test at the start of L8817 in this file (no eglot needed; either python-mode or python-ts-mode is fine, as both show the same issue):

https://raw.githubusercontent.com/matplotlib/matplotlib/refs/heads/main/lib/matplotlib/axes/_axes.py
https://raw.githubusercontent.com/matplotlib/matplotlib/refs/heads/main/lib/matplotlib/axes/_axes.py
_axes
Text Document · 353 KB

In #77588 this was shown to lead to long intermittent pauses with eglot in large python buffers, but the issue is more general for any packages using thingatpt in python buffers.

[Message part 4 (text/html, inline)]

[preview.png (image/png, inline)]

[Message part 6 (message/rfc822, inline)]

From: Eli Zaretskii <eliz <at> gnu.org>
To: kobarity <kobarity <at> gmail.com>
Cc: 77620-done <at> debbugs.gnu.org, jdtsmith <at> gmail.com
Subject: Re: bug#77620: thingatpt can be incredibly slow in python-mode buffers
Date: Sat, 26 Apr 2025 14:22:59 +0300

> Date: Mon, 21 Apr 2025 23:22:14 +0900
> From: kobarity <kobarity <at> gmail.com>
> Cc: 77620 <at> debbugs.gnu.org
> 
> kobarity wrote:
> > In the process of looking into #77620, I noticed bugs in the
> > end-of-block detection functions.  So I submitted a patch to #77941.
> > Please note that the patch attached to this mail assumes the #77941
> > patch.
> > 
> > In my environment, the next test shows about 2 seconds on _axes.py's
> > L8817.
> > 
> > > (/ (car (benchmark-run 10 (thing-at-point 'sexp))) 10)
> > 
> > But there are worse cases.  Let's add the next two lines to the end of
> > _axes.py with 4-character indent.
> > 
> >     a = 1
> >     b = 2
> > 
> > The same measurement at the end of line "a = 1" gives 6 seconds; at
> > the end of line "b = 2" it gives 7 seconds.  This is because it calls
> > time-consuming functions many times.
> > 
> > `python-nav-beginning-of-block' and `python-nav-end-of-block' are very
> > time consuming if the block is large.  In the case of _axes.py, "class
> > Axes" is a large block with over 8000 lines.
> > 
> > However, `python-nav-forward-sexp' that (thing-at-point 'sexp) calls
> > does not necessarily have to perform these functions.  They are just
> > called by `python-info-statement-ends-block-p': (Before #77941 patch)
> > 
> > (defun python-info-statement-ends-block-p ()
> >   "Return non-nil if point is at end of block."
> >   (let ((end-of-block-pos (save-excursion
> >                             (python-nav-end-of-block)))
> >         (end-of-statement-pos (save-excursion
> >                                 (python-nav-end-of-statement))))
> >     (and end-of-block-pos end-of-statement-pos
> >          (= end-of-block-pos end-of-statement-pos))))
> > 
> > The logic is that if the end of the current statement is the end of
> > the current block, then it is the last line of the block.  This is not
> > wrong, but the problem is that it takes time to move to the end of the
> > block.
> > 
> > Fortunately, in many cases it is much easier to determine that it is
> > not the end of the block.  Specifically, if the indentation of the
> > next statement is equal to or greater than the indentation of the
> > current statement, then the current statement cannot be the end of the
> > block.  The attached patch
> > 0001-Performance-optimization-of-python-info-statement-en.patch takes
> > advantage of this to improve the performance of
> > `python-info-statement-ends-block-p'.  Please remember that it assumes
> > the #77941 patch is applied.
> > 
> > 0001-Performance-optimization-of-python-info-statement-en.patch solves
> > the performance problem with _axes.py, but does not have much effect
> > on the problem with lines "a = 1" and "b = 2" added above.  The
> > attached patch
> > 0002-Add-cache-to-Python-block-navigation-functions.patch is an
> > attempt to introduce a cache to alleviate this problem.
> > 
> > I measured the following to disable the cache for each benchmark.
> > 
> > (/ (car (benchmark-run 10 (progn (thing-at-point 'sexp) (setq python-nav-cache nil)))) 10)
> > 
> > It shows about 1.6 seconds for both "a = 1" and "b = 2" lines.
> > 
> > I believe 0002-Add-cache-to-Python-block-navigation-functions.patch is
> > a PoC level patch, so I welcome opinions on what you think of this
> > direction.
> 
> I revised 0002-Add-cache-to-Python-block-navigation-functions.patch.
> There was no need to use a macro.

Thanks, installed, and closing the bug.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.