GNU bug report logs - #1024
Large C++ files load slowly, regardless of font-lock-maximum-size

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 1024 in the body.
You can then email your comments to 1024 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: John W <jw_spambox <at> yahoo.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Large C++ files load slowly, regardless of font-lock-maximum-size
Date: Wed, 24 Sep 2008 23:07:19 -0700 (PDT)

To reproduce:
(1) Generate a 3M C++ file.
(2) Load file.
(3) Wait for emacs. <-- bug

I'm using jit-lock, but I tested lazy-lock, and the behavior is the same.

font-lock-maximum-size is the default.  I.e.
Its value is 256000

I used edebug to see what emacs was doing, and it gave me a stack like:

  c-literal-limits()
  c-neutralize-syntax-in-CPP(1 3527391 3527390)
  funcall(c-neutralize-syntax-in-CPP 1 3527391 3527390)
  (if nil c-before-font-lock-function (funcall c-before-font-lock-function (point-min) (point-max) (- ... ...)))
  (save-excursion (if c-get-state-before-change-function (funcall c-get-state-before-change-function ... ...)) (if nil c-before-font-lock-function (funcall c-before-font-lock-function ... ... ...)))
  (save-restriction (widen) (save-excursion (if c-get-state-before-change-function ...) (if nil c-before-font-lock-function ...)))
  c-common-init(c++-mode)
  c++-mode()
  set-auto-mode-0(c++-mode nil)
  set-auto-mode()
  normal-mode(t)
  after-find-file(nil t)

I blame c-before-font-lock-function .

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
/usr/local/share/emacs/22.2/etc/DEBUG for instructions.


In GNU Emacs 22.2.1 (i686-pc-linux-gnu)
 of 2008-08-27 on augustine
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.utf8
  locale-coding-system: utf-8
  default-enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  encoded-kbd-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
[ 6 ~ ESC [ 6 ~ ESC [ 6 ~ ESC [ 6 ~ ESC [ 6 ~ ESC [ 
6 ~ ESC [ 5 ~ ESC [ 5 ~ ESC [ 5 ~ ESC [ 5 ~ ESC [ 5 
~ ESC [ 5 ~ ESC [ 5 ~ ESC [ 5 ~ ESC [ 5 ~ ESC [ 5 ~ 
ESC [ 5 ~ ESC [ 5 ~ C-x k RET C-x C-e C-x k RET ESC 
[ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A 
ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ A ESC 
[ A ESC [ A ESC [ A ESC [ A ESC [ A ESC [ B ESC [ B 
ESC [ B ESC [ A ESC f ESC f ESC f ESC f ESC f ESC b 
C-k C-_ ESC [ 5 ~ ESC [ 6 ~ ESC [ 5 ~ ESC [ 6 ~ ESC 
[ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B ESC [ B 
ESC [ B ESC [ B ESC [ B ESC [ A C-@ ESC [ B ESC [ B 
ESC [ B C-x b C-g C-x o C-g ESC [ A ESC [ A ESC [ A 
ESC f ESC f ESC b ESC f ESC f ESC f ESC f ESC b C-k 
C-_ C-x b RET C-x b RET ESC x e m a c s - b DEL d e 
TAB DEL DEL TAB DEL DEL DEL DEL DEL DEL b u TAB DEL 
DEL g n TAB DEL DEL DEL DEL r e p o TAB r t TAB RE
T

Recent messages:
#<buffer Connect.i>
Auto-saving...done
Undo!
Mark set
Entering debugger...
Quit
Undo!
Auto-saving...done
Making completion list... [3 times]
Loading emacsbug...done



      

Message #12 received at 1024 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: martin rudalics <rudalics <at> gmx.at>
To: jw_spambox <at> yahoo.com, 1024 <at> debbugs.gnu.org
Subject: Re: bug#1024: Large C++ files load slowly, regardless of font-lock-maximum-size
Date: Thu, 25 Sep 2008 14:12:22 +0200

> I used edebug to see what emacs was doing, and it gave me a stack like:
>
>   c-literal-limits()
>   c-neutralize-syntax-in-CPP(1 3527391 3527390)
>   funcall(c-neutralize-syntax-in-CPP 1 3527391 3527390)
>   (if nil c-before-font-lock-function (funcall c-before-font-lock-function (point-min) (point-max) (- ... ...)))
>   (save-excursion (if c-get-state-before-change-function (funcall c-get-state-before-change-function ... ...)) (if nil c-before-font-lock-function (funcall c-before-font-lock-function ... ... ...)))
>   (save-restriction (widen) (save-excursion (if c-get-state-before-change-function ...) (if nil c-before-font-lock-function ...)))
>   c-common-init(c++-mode)
>   c++-mode()
>   set-auto-mode-0(c++-mode nil)
>   set-auto-mode()
>   normal-mode(t)
>   after-find-file(nil t)
>
> I blame c-before-font-lock-function .

Could you please have a look at the description of bug#851

  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=851

and try the recipe proposed there by Chong Yidong?

martin

Message #17 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Alan Mackenzie <acm <at> colin2.muc.de>
To: gnu-emacs-bug <at> moderators.isc.org
Subject: Re: bug#1024: Large C++ files load slowly,	regardless of font-lock-maximum-size
Date: 25 Sep 2008 14:53:03 +0200

Hi, John.

John W <jw_spambox <at> yahoo.com> wrote:
> To reproduce:
> (1) Generate a 3M C++ file.
> (2) Load file.
> (3) Wait for emacs. <-- bug

> I'm using jit-lock, but I tested lazy-lock, and the behavior is the same.

> font-lock-maximum-size is the default.  I.e.
> Its value is 256000

> I used edebug to see what emacs was doing, and it gave me a stack like:

> c-literal-limits()
>  c-neutralize-syntax-in-CPP(1 3527391 3527390)
>  funcall(c-neutralize-syntax-in-CPP 1 3527391 3527390)
>  (if nil c-before-font-lock-function (funcall c-before-font-lock-function (point-min) (point-max) (- ... ...)))
>  (save-excursion (if c-get-state-before-change-function (funcall c-get-state-before-change-function ... ...)) (if nil c-before-font-lock-function (funcall c-before-font-lock-function ... ... ...)))
>  (save-restriction (widen) (save-excursion (if c-get-state-before-change-function ...) (if nil c-before-font-lock-function ...)))
>  c-common-init(c++-mode)
>  c++-mode()
>  set-auto-mode-0(c++-mode nil)
>  set-auto-mode()
>  normal-mode(t)
>  after-find-file(nil t)

Thanks for taking the trouble to generate this stack dump; it identifies
the problem completely.

This bug was caused by a previous bug fix which introduced a complete
scan over the entire buffer when it's loaded.  One function,
`c-neutralize-syntax-in-CPP' had been coded very inefficiently.  It was
recoded for speed on 2008-05-24, committed in
.../emacs/lisp/progmodes/cc-mode.el version 1.58.2.12 (Emacs 22 branch)
and version 1.75 (trunk) at savannah.


To get the fixed version, either:
(i) Upgrade to Emacs 22.3; or
(ii) If you're using the Emacs 23 CVS, update your version.

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #24 received at 1024 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: John W <jw_spambox <at> yahoo.com>
To: martin rudalics <rudalics <at> gmx.at>
Cc: 1024 <at> debbugs.gnu.org
Subject: Re: bug#1024: Large C++ files load slowly, regardless of font-lock-maximum-size
Date: Thu, 25 Sep 2008 20:22:34 -0700 (PDT)

> Could you please have a look at the description of bug#851
>
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=851
>
> and try the recipe proposed there by Chong Yidong?

Sure.  I tried this patch

+++ cc-engine.el	2008/04/01 21:41:21	1.56.2.11

but it made no difference; it is still slow, and when I examine the
stack, it is the same stack trace.  The file loads in 2 seconds in
fundamental mode, but takes about 2 minutes in C++ mode.  I should
have mentioned that this is a preprocessed C++ file, with many
compiler directies, such as #line 1 "somefile.h", and I now think the
problem is provoked by the compiler directives.

If I globally replace the #line stuff like so:

  #line 1 "some_file.h"
 -->
  s = "some_file.h"

the file loads in about 7 seconds.

- John


      

Message #31 received at 1024 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: jw_spambox <at> yahoo.com, 1024 <at> debbugs.gnu.org
Cc: martin rudalics <rudalics <at> gmx.at>
Subject: Re: bug#1024: Large C++ files load slowly,
	regardless of font-lock-maximum-size
Date: Fri, 26 Sep 2008 07:53:19 +0000

Hi, John,

On Thu, Sep 25, 2008 at 08:22:34PM -0700, John W wrote:
> > Could you please have a look at the description of bug#851

> > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=851

> > and try the recipe proposed there by Chong Yidong?

> Sure.  I tried this patch

> +++ cc-engine.el	2008/04/01 21:41:21	1.56.2.11

This slowdown happened because of an earlier bug fix.  Part of that fix
is scanning the entire buffer, "fixing" things which could derail the
syntax and font locking, e.g. preprocessor lines like this:

#warning this isn't fixed yet!
                 ^
, where the apostrophe was being treated like a string opener.

That fix was not coded with a view to speed, unfortunately.  It was later
fixed in cc-mode.el version 1.75 (CVS trunk at savannah) and version
1.58.2.12 (in the Emacs 22 branch) on 2008-05-24.

The fix was incorporated into Emacs 22.3, released a few days ago.

Also, thanks for generating and posting the stack dump, which made it
trivially easy to track down the problem.

> - John

-- 
Alan Mackenzie (Nuremberg, Germany).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.