GNU bug report logs - #496
23.0.60; ispell-word becomes increasingly slower

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 496 in the body.
You can then email your comments to 496 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Markus Triska <markus.triska <at> gmx.at>
To: emacs-pretest-bug <at> gnu.org
Subject: 23.0.60; ispell-word becomes increasingly slower
Date: Sat, 28 Jun 2008 02:26:55 +0200 (CEST)

When I save the following form to test.el:

(let ((n 0))
  (with-temp-buffer
    (insert "test")
    (while t
      (setq n (1+ n))
      (when (= (mod n 100) 0)
        (message "n: %s -- %s" n (emacs-uptime)))
      (ispell-word nil t))))

and then do "emacs -Q --script test.el", I get:

   Starting new Ispell process [default] ...
   n: 100 -- 0 seconds
   n: 200 -- 0 seconds
   n: 300 -- 0 seconds
   n: 400 -- 0 seconds
   n: 500 -- 1 second
   n: 600 -- 1 second
   n: 700 -- 1 second
   n: 800 -- 1 second
   n: 900 -- 1 second
   n: 1000 -- 1 second
   n: 1100 -- 1 second
   n: 1200 -- 1 second
   n: 1300 -- 1 second
   n: 1400 -- 1 second
   n: 1500 -- 1 second
   n: 1600 -- 1 second
   n: 1700 -- 1 second
   n: 1800 -- 1 second
   n: 1900 -- 1 second
   n: 2000 -- 1 second
   ...
   n: 52400 -- 3 minutes, 12 seconds
   n: 52500 -- 3 minutes, 13 seconds
   n: 52600 -- 3 minutes, 14 seconds
   n: 52700 -- 3 minutes, 15 seconds
   ..
   n: 70500 -- 8 minutes, 30 seconds
   n: 70600 -- 8 minutes, 31 seconds
   n: 70700 -- 8 minutes, 33 seconds
   n: 70800 -- 8 minutes, 35 seconds
   n: 70900 -- 8 minutes, 36 seconds
   n: 71000 -- 8 minutes, 38 seconds
   ...

i.e., the program becomes increasingly slower, from many iterations
per second to several seconds per iteration. With "top", I also see
that aspell uses increasingly more memory. I can reproduce this on
Fedora Core 9 and OSX 10.4 with Aspell 0.60.5. In a simplified test to
check for memory leaks in aspell, I did:

   $ while true; do echo "test"; done | aspell -a

and in this case, aspell's memory usage seems to stay constant.


In GNU Emacs 23.0.60.1 (i386-apple-darwin8.11.1, GTK+ Version 2.12.9)
 of 2008-06-15 on mt-computer.local
Windowing system distributor `The XFree86 Project, Inc', version 11.0.40400000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default-enable-multibyte-characters: t

Message #10 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Markus Triska <markus.triska <at> gmx.at>, 496 <at> debbugs.gnu.org
Cc: emacs-pretest-bug <at> gnu.org, bug-submit-list <at> donarmstrong.com,
        bug-gnu-emacs <at> gnu.org
Subject: Re: bug#496: 23.0.60; ispell-word becomes increasingly slower
Date: Sat, 28 Jun 2008 10:20:00 +0300

> Resent-From: Markus Triska <markus.triska <at> gmx.at>
> Resent-To: bug-submit-list <at> donarmstrong.com
> Resent-CC: Emacs Bugs <bug-gnu-emacs <at> gnu.org>
> Resent-Sender: don <at> donarmstrong.com
> From: Markus Triska <markus.triska <at> gmx.at>
> Date: Sat, 28 Jun 2008 02:26:55 +0200 (CEST)
> Cc: 
> 
> 
> When I save the following form to test.el:
> 
> (let ((n 0))
>   (with-temp-buffer
>     (insert "test")
>     (while t
>       (setq n (1+ n))
>       (when (= (mod n 100) 0)
>         (message "n: %s -- %s" n (emacs-uptime)))
>       (ispell-word nil t))))
> 
> and then do "emacs -Q --script test.el", I get:
> 
>    Starting new Ispell process [default] ...
>    n: 100 -- 0 seconds
>    n: 200 -- 0 seconds
>    n: 300 -- 0 seconds
>    n: 400 -- 0 seconds
>    n: 500 -- 1 second
>    n: 600 -- 1 second
>    n: 700 -- 1 second
>    n: 800 -- 1 second
>    n: 900 -- 1 second
>    n: 1000 -- 1 second
>    n: 1100 -- 1 second
>    n: 1200 -- 1 second
>    n: 1300 -- 1 second
>    n: 1400 -- 1 second
>    n: 1500 -- 1 second
>    n: 1600 -- 1 second
>    n: 1700 -- 1 second
>    n: 1800 -- 1 second
>    n: 1900 -- 1 second
>    n: 2000 -- 1 second
>    ...
>    n: 52400 -- 3 minutes, 12 seconds
>    n: 52500 -- 3 minutes, 13 seconds
>    n: 52600 -- 3 minutes, 14 seconds
>    n: 52700 -- 3 minutes, 15 seconds
>    ..
>    n: 70500 -- 8 minutes, 30 seconds
>    n: 70600 -- 8 minutes, 31 seconds
>    n: 70700 -- 8 minutes, 33 seconds
>    n: 70800 -- 8 minutes, 35 seconds
>    n: 70900 -- 8 minutes, 36 seconds
>    n: 71000 -- 8 minutes, 38 seconds
>    ...
> 
> i.e., the program becomes increasingly slower, from many iterations
> per second to several seconds per iteration. With "top", I also see
> that aspell uses increasingly more memory. I can reproduce this on
> Fedora Core 9 and OSX 10.4 with Aspell 0.60.5. In a simplified test to
> check for memory leaks in aspell, I did:
> 
>    $ while true; do echo "test"; done | aspell -a
> 
> and in this case, aspell's memory usage seems to stay constant.

Garbage collection?

Message #25 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Markus Triska <markus.triska <at> gmx.at>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 496 <at> debbugs.gnu.org, emacs-pretest-bug <at> gnu.org,
        bug-submit-list <at> donarmstrong.com, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#496: 23.0.60; ispell-word becomes increasingly slower
Date: Sat, 28 Jun 2008 11:25:48 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> check for memory leaks in aspell, I did:
>> 
>>    $ while true; do echo "test"; done | aspell -a
>> 
>> and in this case, aspell's memory usage seems to stay constant.
>
> Garbage collection?

I've received a better test case from the Aspell maintainer: 

  ( while true; do echo '!'; echo "-"; echo "%"; echo "^test"; done ) | aspell -a

According to him, it is the constant switching of modes that is causing
a problem. He notes:

   "I have narrowed down the problem code. Technically it is not a leak
   as the code is still reachable (according to valgrind), which makes
   it harder to fix. I will let you know."

It is thus more an Aspell problem, but if it cannot be fixed in Aspell,
I will look whether the Emacs side of communication can be improved.

Message #40 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Markus Triska <markus.triska <at> gmx.at>
Cc: 496 <at> debbugs.gnu.org, emacs-pretest-bug <at> gnu.org,
        bug-submit-list <at> donarmstrong.com, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#496: 23.0.60; ispell-word becomes increasingly slower
Date: Sat, 28 Jun 2008 18:04:02 +0300

> From: Markus Triska <markus.triska <at> gmx.at>
> Cc: 496 <at> emacsbugs.donarmstrong.com, emacs-pretest-bug <at> gnu.org,
> 	bug-submit-list <at> donarmstrong.com, bug-gnu-emacs <at> gnu.org
> Date: Sat, 28 Jun 2008 11:25:48 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> check for memory leaks in aspell, I did:
> >> 
> >>    $ while true; do echo "test"; done | aspell -a
> >> 
> >> and in this case, aspell's memory usage seems to stay constant.
> >
> > Garbage collection?
> 
> I've received a better test case from the Aspell maintainer: 
> 
>   ( while true; do echo '!'; echo "-"; echo "%"; echo "^test"; done ) | aspell -a
> 
> According to him, it is the constant switching of modes that is causing
> a problem. He notes:
> 
>    "I have narrowed down the problem code. Technically it is not a leak
>    as the code is still reachable (according to valgrind), which makes
>    it harder to fix. I will let you know."
> 
> It is thus more an Aspell problem, but if it cannot be fixed in Aspell,
> I will look whether the Emacs side of communication can be improved.

But your test.el shows a similar problem with Ispell.  So either
Aspell and Ispell share the same problem, or Emacs still has something
to do with the slowdown.

Message #55 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Markus Triska <markus.triska <at> gmx.at>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 496 <at> debbugs.gnu.org, emacs-pretest-bug <at> gnu.org,
        bug-submit-list <at> donarmstrong.com, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#496: 23.0.60; ispell-word becomes increasingly slower
Date: Sat, 28 Jun 2008 17:21:52 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> your test.el shows a similar problem with Ispell. So either Aspell and
> Ispell share the same problem, or Emacs still has something to do with
> the slowdown.

Aspell's maintainer Kevin Atkinson has generously sent me the following
patch for Aspell that fixes the problem for me with the latest CVS
version of Aspell. He says it will be in Aspell 0.60.7. Here's a simpler
test case for the problem in versions up to 0.60.6:

  while true; do echo "-"; done | aspell -a

Index: common/config.cpp
===================================================================
RCS file: /sources/aspell/aspell/common/config.cpp,v
retrieving revision 1.79
diff -u -r1.79 config.cpp
--- common/config.cpp	12 Oct 2007 00:33:17 -0000	1.79
+++ common/config.cpp	28 Jun 2008 10:42:10 -0000
@@ -1000,7 +1000,7 @@
       }
       
       assert(ki->def != 0); // if null this key should never have values
-      // directly added to it
+                            // directly added to it
       String value(entry->action == Reset ? get_default(ki) : entry->value);
       
       switch (ki->type) {
@@ -1027,11 +1027,13 @@
           goto error;
         }
 
+        clear_others(entry);
         NOTIFY_ALL(item_updated(ki, val));
         break;
         
       } case KeyInfoString:
         
+        clear_others(entry);
         NOTIFY_ALL(item_updated(ki, value));
         break;
         
@@ -1040,6 +1042,7 @@
         int num;
         
         if (sscanf(value.str(), "%i", &num) == 1 && num >= 0) {
+          clear_others(entry);
           NOTIFY_ALL(item_updated(ki, num));
         } else {
           pe = make_err(bad_value, entry->key, value, _("a positive integer"));
@@ -1049,6 +1052,9 @@
         break;
       }
       case KeyInfoList:
+
+        if (entry->action == ListSet || entry->action == ListClear)
+          clear_others(entry);
         
         NOTIFY_ALL(list_updated(ki));
         break;
@@ -1064,6 +1070,26 @@
       return (PosibErrBase &)pe;
   }
 
+  // Delete all previous entries with the same key
+  void Config::clear_others(Entry * entry) {
+
+    String & key = entry->key;
+    //printf("Clear Others For %s\n", key.c_str());
+    Entry * * cur = &first_;
+    while (*cur != entry) {
+      assert(*cur != *insert_point_);
+      if ((*cur)->key == key) {
+        //printf("JAP \"%s\"!\n", (*cur)->value.c_str());
+        Entry * tmp = *cur;
+        *cur = (*cur)->next;
+        delete tmp;
+      } else {
+        cur = &(*cur)->next;
+      }
+    }
+    
+  }
+
 #undef NOTIFY_ALL
 
 
Index: common/config.hpp
===================================================================
RCS file: /sources/aspell/aspell/common/config.hpp,v
retrieving revision 1.27
diff -u -r1.27 config.hpp
--- common/config.hpp	3 May 2005 05:08:19 -0000	1.27
+++ common/config.hpp	28 Jun 2008 10:42:10 -0000
@@ -132,6 +132,8 @@
 
     PosibErr<int> commit(Entry * entry, Conv * conv = 0);
 
+    void clear_others(Entry * entry);
+
     bool settings_read_in_;
 
   public:

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.