Subject: bug#7343: closed (Re: bug#7343: Making flyspell incredibly fast when checking whole files)

From: [email-masked] (GNU bug Tracking System)
Date: Wed, 10 Nov 2010 14:40:05 +0000

Your bug report

#7343: Making flyspell incredibly fast when checking whole files

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
7343: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=7343
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#7343: Making flyspell incredibly fast when checking whole files
From: Agustin Martin <[email-masked]>
To: Brandon Craig Rhodes <[email-masked]>, [email-masked]
Date: Wed, 10 Nov 2010 15:44:14 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

On Sat, Nov 06, 2010 at 10:03:52AM -0400, Brandon Craig Rhodes wrote:
> A patch is attached that fixes this problem, and - here on my laptop, at
> least - makes flyspell blazing fast at even large files.  The mechanism
> is simple: I have added a second optional argument to flyspell-word,
> named "known-misspelling", that tells flyspell-word that the word has
> already been checked and is a misspelling and does not need to be
> checked again.  Then, down in the function, I simply placed the entire
> interactive session with ispell/aspell/hunspell inside of an "if".

Installed in the Emacs bzr repo. Closing bug report.

Thanks a lot for your patch,

-- 
Agustin

Subject: Making flyspell incredibly fast when checking whole files
From: Brandon Craig Rhodes <[email-masked]>
To: [email-masked]
Date: Sat, 06 Nov 2010 10:03:52 -0400
List-Id: <debbugs-submit.debbugs.gnu.org>

Spell-checking programs like "aspell" and "hunspell" are blazingly fast
when simply asked to check words with their "-l" option, but become much
slower (the difference is often one or more orders of magnitude,
depending on the dictionary size) when asked about words interactively
because in that case they generate helpful near-misses.

(Actually, "aspell" can turn this off even in interactive mode, which
might become the basis of a further patch; but right now I will confine
myself to submitting this one, since it has gotten my Emacs running fast
enough again that I am happy.)

Anyway, flyspell does try to take advantage of the above behavior by
checking whether a region is larger than flyspell-large-region
characters, and if so then it runs the spell checker as a separate
process with "-l".  But then it does something that, in many cases, is
rather ruinous: it takes every misspelling so identified, and passes it
*back* through the normal interactive spell-checking logic!  This is
because all of the real logic of what to do with a misspelling - how to
highlight it, how to search for nearby instances of the same word, how
to cache spellings, and so forth - is bound up in flyspell-word, so the
flyspell-external-point-words function, which processes the actual
misspellings discovered by flyspell-large-region, really has no other
choice but to call flyspell-word for each misspelling.

So to let flyspell-large-region enjoy the speed that it really should,
we need to tell it never to re-check its words against the live *spell
process attached to Emacs, because that is (a) redundant and (b) very
expensive since, this second time, the spell checker will pause to
generate near-misses.

A patch is attached that fixes this problem, and - here on my laptop, at
least - makes flyspell blazing fast at even large files.  The mechanism
is simple: I have added a second optional argument to flyspell-word,
named "known-misspelling", that tells flyspell-word that the word has
already been checked and is a misspelling and does not need to be
checked again.  Then, down in the function, I simply placed the entire
interactive session with ispell/aspell/hunspell inside of an "if".

I apologize in advance that this diff is constructed against the Ubuntu
version of flyspell, which has who-knows-how-many differences with the
official Emacs one.  If the patch is too difficult to apply, let me
know, and I will find the time to check out Emacs from trunk myself and
reapply the patch there.

Thanks!

diff -r 0de00de3360c -r 06af33083844 site-lisp/flyspell.el
--- a/site-lisp/flyspell.el	Sat Nov 06 08:10:07 2010 -0400
+++ b/site-lisp/flyspell.el	Sat Nov 06 08:23:58 2010 -0400
@@ -1009,7 +1009,7 @@
 ;;*---------------------------------------------------------------------*/
 ;;*    flyspell-word ...                                                */
 ;;*---------------------------------------------------------------------*/
-(defun flyspell-word (&optional following)
+(defun flyspell-word (&optional following known-misspelling)
   "Spell check a word."
   (interactive (list ispell-following-word))
   (ispell-set-spellchecker-params)    ; Initialize variables and dicts alists
@@ -1071,29 +1071,35 @@
 	    (setq flyspell-word-cache-end end)
 	    (setq flyspell-word-cache-word word)
 	    ;; now check spelling of word.
-	    (ispell-send-string "%\n")
-	    ;; put in verbose mode
-	    (ispell-send-string (concat "^" word "\n"))
-	    ;; we mark the ispell process so it can be killed
-	    ;; when emacs is exited without query
-	    (set-process-query-on-exit-flag ispell-process nil)
-	    ;; Wait until ispell has processed word.  Since this code is often
-            ;; executed from post-command-hook but the ispell process may not
-            ;; be responsive, it's important to make sure we re-enable C-g.
-	    (with-local-quit
-	      (while (progn
-		       (accept-process-output ispell-process)
-		       (not (string= "" (car ispell-filter))))))
-	    ;; (ispell-send-string "!\n")
-	    ;; back to terse mode.
-	    ;; Remove leading empty element
-	    (setq ispell-filter (cdr ispell-filter))
-	    ;; ispell process should return something after word is sent.
-	    ;; Tag word as valid (i.e., skip) otherwise
-	    (or ispell-filter
-		(setq ispell-filter '(*)))
-	    (if (consp ispell-filter)
-		(setq poss (ispell-parse-output (car ispell-filter))))
+            (if (not known-misspelling)
+                (progn
+                  (ispell-send-string "%\n")
+                  ;; put in verbose mode
+                  (ispell-send-string (concat "^" word "\n"))
+                  ;; we mark the ispell process so it can be killed
+                  ;; when emacs is exited without query
+                  (set-process-query-on-exit-flag ispell-process nil)
+                  ;; Wait until ispell has processed word.  Since this
+                  ;; code is often executed from post-command-hook but
+                  ;; the ispell process may not be responsive, it's
+                  ;; important to make sure we re-enable C-g.
+                  (with-local-quit
+                    (while (progn
+                             (accept-process-output ispell-process)
+                             (not (string= "" (car ispell-filter))))))
+                  ;; (ispell-send-string "!\n")
+                  ;; back to terse mode.
+                  ;; Remove leading empty element
+                  (setq ispell-filter (cdr ispell-filter))
+                  ;; ispell process should return something after word is sent.
+                  ;; Tag word as valid (i.e., skip) otherwise
+                  (or ispell-filter
+                      (setq ispell-filter '(*)))
+                  (if (consp ispell-filter)
+                      (setq poss (ispell-parse-output (car ispell-filter)))))
+              ;; Else, this was a known misspelling to begin with, and
+              ;; we should forge an ispell return value.
+              (setq poss (list word 0 '() '())))
 	    (let ((res (cond ((eq poss t)
 			      ;; correct
 			      (setq flyspell-word-cache-result t)
@@ -1424,7 +1430,7 @@
 					t
 				      nil))))
 			(setq keep nil)
-			(flyspell-word)
+			(flyspell-word nil t)
 			;; Search for next misspelled word will begin from
 			;; end of last validated match.
 			(setq buffer-scan-pos (point))))

-- 
Brandon Craig Rhodes   [email-masked]   http://rhodesmill.org/brandon