GNU bug report logs - #18749
24.3.94; CC Mode 5.32.5 (C/l); cc-mode state cache failure leading to bad indentation

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18749 in the body.
You can then email your comments to 18749 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Michael Welsh Duggan <mwd <at> md5i.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3.94; CC Mode 5.32.5 (C/l);
 cc-mode state cache failure leading to bad indentation
Date: Thu, 16 Oct 2014 17:12:25 -0400

[Message part 1 (text/plain, inline)]

A consistent c-mode indentation error that causes bad indentation, much
cursing, and gnashing of teeth.  This occurs in the latest emacs-24
branch (and trunk).

From emacs -Q, using the attached file:

[rwuniqsetup.c.before.2 (text/plain, attachment)]

[Message part 3 (text/plain, inline)]

    M-x c-toggle-parse-state-debug
    C-x C-f rwuniqsetup.c.before.2
    C-s ipaddr_rec_to_bin RET
    C-v

At this point you should see the following in *Messages*:

    c-parse-state inconsistency at 46940: using cache: nil, from scratch: ((43406 . 45002))
    Old state:
    (setq c-state-cache nil  c-state-cache-good-pos 45713  c-state-nonlit-pos-cache '(76939 73939 70939 67939 64939 61939 58939 55939 52939 49939 46939 43114 39291 36291 33291 30291 27178 24178 21178 18178 15145 12046 9013 6002 3002)  c-state-nonlit-pos-cache-limit 76939  c-state-semi-nonlit-pos-cache '(45291 42291 39291 36291 33291 30291 27094 24058 21058 18058 15058 12046 9013 6001 3001)  c-state-semi-nonlit-pos-cache-limit 45291  c-state-brace-pair-desert '(1 . 46940)  c-state-point-min 1  c-state-point-min-lit-type nil  c-state-point-min-lit-start nil  c-state-min-scan-pos 1  c-state-old-cpp-beg nil  c-state-old-cpp-end nil  c-parse-state-point 46940)

At this point, indentation is having problems.  Try hitting TAB on line
1371, for example.  Or line 1374 for a complete failure to indent at
all.


Emacs  : GNU Emacs 24.3.94.1 (x86_64-unknown-linux-gnu, X toolkit)
 of 2014-10-16 on maru2
Package: CC Mode 5.32.5 (C/l)
Buffer Style: gnu
c-emacs-features: (pps-extended-state col-0-paren posix-char-classes gen-string-delim gen-comment-delim syntax-properties 1-bit)

current state:
==============
(setq
 c-basic-offset 4
 c-comment-only-line-offset '(0 . 0)
 c-indent-comment-alist '((anchored-comment column . 0) (end-block space . 1)
			  (cpp-end-block space . 2))
 c-indent-comments-syntactically-p nil
 c-block-comment-prefix ""
 c-comment-prefix-regexp '((pike-mode . "//+!?\\|\\**") (awk-mode . "#+")
			   (other . "//+\\|\\**"))
 c-doc-comment-style '((java-mode . javadoc) (pike-mode . autodoc)
		       (c-mode . gtkdoc))
 c-cleanup-list '(scope-operator)
 c-hanging-braces-alist '((substatement-open before after)
			  (arglist-cont-nonempty))
 c-hanging-colons-alist nil
 c-hanging-semi&comma-criteria '(c-semi&comma-inside-parenlist)
 c-backslash-column 48
 c-backslash-max-column 72
 c-special-indent-hook '(c-gnu-impose-minimum)
 c-label-minimum-indentation 1
 c-offsets-alist '((inexpr-class . +)
		   (inexpr-statement . +)
		   (lambda-intro-cont . +)
		   (inlambda . c-lineup-inexpr-block)
		   (template-args-cont c-lineup-template-args +)
		   (incomposition . +)
		   (inmodule . +)
		   (innamespace . +)
		   (inextern-lang . +)
		   (composition-close . 0)
		   (module-close . 0)
		   (namespace-close . 0)
		   (extern-lang-close . 0)
		   (composition-open . 0)
		   (module-open . 0)
		   (namespace-open . 0)
		   (extern-lang-open . 0)
		   (objc-method-call-cont
		    c-lineup-ObjC-method-call-colons
		    c-lineup-ObjC-method-call
		    +
		    )
		   (objc-method-args-cont . c-lineup-ObjC-method-args)
		   (objc-method-intro . [0])
		   (friend . 0)
		   (cpp-define-intro c-lineup-cpp-define +)
		   (cpp-macro-cont . +)
		   (cpp-macro . [0])
		   (inclass . +)
		   (stream-op . c-lineup-streamop)
		   (arglist-cont-nonempty
		    c-lineup-gcc-asm-reg
		    c-lineup-arglist
		    )
		   (arglist-cont c-lineup-gcc-asm-reg 0)
		   (comment-intro
		    c-lineup-knr-region-comment
		    c-lineup-comment
		    )
		   (catch-clause . 0)
		   (else-clause . 0)
		   (do-while-closure . 0)
		   (access-label . -)
		   (case-label . 0)
		   (substatement . +)
		   (statement-case-intro . +)
		   (statement . 0)
		   (brace-entry-open . 0)
		   (brace-list-entry . 0)
		   (brace-list-intro . +)
		   (brace-list-close . 0)
		   (block-close . 0)
		   (block-open . 0)
		   (inher-cont . c-lineup-multi-inher)
		   (inher-intro . +)
		   (member-init-cont . c-lineup-multi-inher)
		   (member-init-intro . +)
		   (annotation-var-cont . +)
		   (annotation-top-cont . 0)
		   (topmost-intro . 0)
		   (knr-argdecl . 0)
		   (func-decl-cont . +)
		   (inline-close . 0)
		   (class-close . 0)
		   (class-open . 0)
		   (defun-block-intro . +)
		   (defun-close . 0)
		   (defun-open . 0)
		   (c . c-lineup-C-comments)
		   (string . c-lineup-dont-change)
		   (topmost-intro-cont
		    first
		    c-lineup-topmost-intro-cont
		    c-lineup-gnu-DEFUN-intro-cont
		    )
		   (brace-list-open . +)
		   (inline-open . 0)
		   (arglist-close . c-lineup-arglist)
		   (arglist-intro . c-lineup-arglist-intro-after-paren)
		   (statement-cont . +)
		   (statement-case-open . +)
		   (label . 0)
		   (substatement-label . 0)
		   (substatement-open . +)
		   (knr-argdecl-intro . 5)
		   (statement-block-intro . +)
		   )
 c-buffer-is-cc-mode 'c-mode
 c-tab-always-indent t
 c-syntactic-indentation t
 c-syntactic-indentation-in-macros t
 c-ignore-auto-fill '(string cpp code)
 c-auto-align-backslashes t
 c-backspace-function 'backward-delete-char-untabify
 c-delete-function 'delete-char
 c-electric-pound-behavior nil
 c-default-style '((java-mode . "java") (awk-mode . "awk") (other . "gnu"))
 c-enable-xemacs-performance-kludge-p nil
 c-old-style-variable-behavior nil
 defun-prompt-regexp nil
 tab-width 8
 comment-column 32
 parse-sexp-ignore-comments t
 parse-sexp-lookup-properties t
 auto-fill-function nil
 comment-multi-line t
 comment-start-skip "\\(//+\\|/\\*+\\)\\s *"
 fill-prefix nil
 fill-column 70
 paragraph-start "[ 	]*\\(//+\\|\\**\\)[ 	]*$\\|^\f"
 adaptive-fill-mode t
 adaptive-fill-regexp "[ 	]*\\(//+\\|\\**\\)[ 	]*\\([ 	]*\\([-–!|#%;>*·•‣⁃◦]+[ 	]*\\)*\\)"
 )

In GNU Emacs 24.3.94.1 (x86_64-unknown-linux-gnu, X toolkit)
 of 2014-10-16 on maru2
Repository revision: 117589 rgm <at> gnu.org-20141016014710-zqpn29rkvl69dg14
Windowing system distributor `The X.Org Foundation', version 11.0.11601000
System Description:	Debian GNU/Linux unstable (sid)

Configured using:
 `configure --without-toolkit-scroll-bars --with-x-toolkit=lucid
 'CFLAGS=-Og -ggdb' --with-wide-int'

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Message

Minor modes in effect:
  mml-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  auto-fill-function: message-do-auto-fill
  transient-mark-mode: t
  abbrev-mode: t

Recent input:
C-x C-f ~ / t m <tab> <tab> b <tab> <return> C-s _ 
C O M A P <backspace> <backspace> P C-g C-g C-x k <return> 
M-x c - t o <tab> p <tab> <return> C-x C-f M-p <return> 
C-s _ C O M P A R E <return> C-v C-v C-v C-v <help-echo> 
M-x c - r e p <tab> <return> y S t a t e SPC c a c 
h e SPC f a i l u r e SPC l e a d i n g SPC t o SPC 
b a d SPC i n d e n t a t i o n <return> C-n C-n C-n 
C-p C-p C-x 1 C-p C-p C-p C-p C-e M-b M-b <M-backspace> 
C-n C-n C-p C-p C-p C-a C-v C-v C-v C-v <escape> < 
M-x r e p o r t - e m <tab> <return>

Recent messages:
Old state:
(setq c-state-cache '(47851 47758 47502 (47151 . 47403))  c-state-cache-good-pos 47852  c-state-nonlit-pos-cache '(76939 73939 70939 67939 64939 61939 58939 55939 52939 49939 46939 43114 39291 36291 33291 30291 27178 24178 21178 18178 15145 12046 9013 6002 3002)  c-state-nonlit-pos-cache-limit 76939  c-state-semi-nonlit-pos-cache '(45291 42291 39291 36291 33291 30291 27094 24058 21058 18058 15058 12046 9013 6001 3001)  c-state-semi-nonlit-pos-cache-limit 45291  c-state-brace-pair-desert nil  c-state-point-min 1  c-state-point-min-lit-type nil  c-state-point-min-lit-start nil  c-state-min-scan-pos 1  c-state-old-cpp-beg nil  c-state-old-cpp-end nil  c-parse-state-point 47852)
c-parse-state inconsistency at 47350: using cache: ((47287 . 47348) 47151 (47051 . 47131)), from scratch: ((43406 . 45002))
Old state:
(setq c-state-cache '((47287 . 47348) 47151 (47051 . 47131))  c-state-cache-good-pos 47348  c-state-nonlit-pos-cache '(76939 73939 70939 67939 64939 61939 58939 55939 52939 49939 46939 43114 39291 36291 33291 30291 27178 24178 21178 18178 15145 12046 9013 6002 3002)  c-state-nonlit-pos-cache-limit 76939  c-state-semi-nonlit-pos-cache '(45291 42291 39291 36291 33291 30291 27094 24058 21058 18058 15058 12046 9013 6001 3001)  c-state-semi-nonlit-pos-cache-limit 45291  c-state-brace-pair-desert nil  c-state-point-min 1  c-state-point-min-lit-type nil  c-state-point-min-lit-start nil  c-state-min-scan-pos 1  c-state-old-cpp-beg nil  c-state-old-cpp-end nil  c-parse-state-point 47350)
Do you want to submit a report on CC Mode? (y or n) y
Parsing /home/md5i/.mailrc... done
Mark set
Formatting bug report buffer.......
Please enter your report.  Type C-c C-c to send, C-c C-k to abort.

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail gnus-util message format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 ietf-drums mm-util help-fns mail-prsvr mailabbrev
mail-utils gmm-utils mailheader reporter misearch multi-isearch cc-langs
cl-loaddefs cl-lib cc-mode cc-fonts easymenu cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs time-date tooltip electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting x-toolkit x multi-tty emacs)

Memory information:
((conses 16 110171 9329)
 (symbols 48 20183 0)
 (miscs 40 172 211)
 (strings 32 17591 3830)
 (string-bytes 1 597205)
 (vectors 16 12254)
 (vector-slots 8 419544 7579)
 (floats 8 67 354)
 (intervals 56 1339 33)
 (buffers 960 15)
 (heap 1024 19484 929))



-- 
Michael Welsh Duggan
(md5i <at> md5i.com)

Message #8 received at 18749 <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Michael Welsh Duggan <mwd <at> md5i.com>
Cc: 18749 <at> debbugs.gnu.org
Subject: Re: bug#18749: 24.3.94; CC Mode 5.32.5 (C/l);
 cc-mode state cache failure leading to bad indentation
Date: 17 Oct 2014 18:19:50 -0000

Hello, Michael.

It's been a fun day.  ;-)

In article <87h9z367nq.fsf <at> maru2.md5i.com> you wrote:
> [-- text/plain, encoding 7bit, charset: US-ASCII, 6 lines --]

> A consistent c-mode indentation error that causes bad indentation, much
> cursing, and gnashing of teeth.  This occurs in the latest emacs-24
> branch (and trunk).

> From emacs -Q, using the attached file:

> [-- text/plain, encoding 7bit, charset: US-ASCII, 2434 lines, name: rwuniqsetup.c.before.2 --]

[ .... ]

> [-- text/plain, encoding quoted-printable, charset: utf-8, 291 lines --]


>     M-x c-toggle-parse-state-debug
>     C-x C-f rwuniqsetup.c.before.2
>     C-s ipaddr_rec_to_bin RET
>     C-v

> At this point you should see the following in *Messages*:

[ The following has been altered from the original to show the correct
  first inconsistency which occurred.]
> Mark saved where search started
> c-parse-state inconsistency at 46719: using cache: ((46217 . 46572) 45454 (43406 . 45002)), from scratch: (45454 (43406 . 45002))
> Old state:
> (setq c-state-cache '((46217 . 46572) 45454 (43406 . 45002))  c-state-cache-good-pos 46747  c-state-nonlit-pos-cache '(76939 73939 70939 67939 64939 61939 58939 55939 52939 49939 46939 43114 39291 36291 33291 30291 27178 24178 21178 18178 15145 12046 9013 6002 3002)  c-state-nonlit-pos-cache-limit 76939  c-state-semi-nonlit-pos-cache '(45291 42291 39291 36291 33291 30291 27094 24058 21058 18058 15058 12046 9013 6001 3001)  c-state-semi-nonlit-pos-cache-limit 45291 c-state-brace-pair-desert '(1 . 45935)  c-state-point-min 1  c-state-point-min-lit-type nil  c-state-point-min-lit-start nil  c-state-min-scan-pos 1  c-state-old-cpp-beg #<marker at 45006 in rwuniqsetup.c.before.2>  c-state-old-cpp-end #<marker at 46938 in rwuniqsetup.c.before.2>  c-parse-state-point 46793)

> At this point, indentation is having problems.  Try hitting TAB on line
> 1371, for example.  Or line 1374 for a complete failure to indent at
> all.

OK, for once c-parse-state seems to be blameless.

What seems to be happening is that during the display fontification,
operations are done at buffer offset 46793.  This is the spot after the
indentation on L1360.

CC Mode puts a backwards search limit on many operations, and this is
often implemented by temporarily narrowing the buffer (narrow-to-region
(- (point) n) (point-max)), where n is sometimes 1000.

1000 bytes before 46793 is 45793.  This spot is in the middle of the "##"
operator on L1346.  Thus the buffer is getting narrowed such that the
first character, ostensibly at BOL (to the Emacs regexp matcher), is "#".

A `c-beginning-of-macro' operation is thus spuriously finding point
45793, in the middle of this operator.  This spot is then getting
permanently marked as a beginning-of-CPP.  (Technically, this is done
with the 'category property, giving it a value 'c-cpp-delimiter.)

When c-parse-state later tries to scan this part of the code, it trips up
on this spurious beginning-of-CPP, terminating its scanning.  If you look
at the two inconsistent values for c-parse-state above, the "from
scratch" value is actually the false one, where it fails to find the
brace pair (46217 . 46572) which is after 45793.

I think what I'm going to do is insert an extra check into
c-beginning-of-macro, such that if there seems to be a match at
(point-min) of a narrowed buffer (or possibly even on the top line of
one), it will be widened and tested again.

Look forward to a patch soon!

> -- 
> Michael Welsh Duggan
> (md5i <at> md5i.com)

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #11 received at 18749 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Alan Mackenzie <acm <at> muc.de>
Cc: Michael Welsh Duggan <mwd <at> md5i.com>, 18749 <at> debbugs.gnu.org
Subject: Re: bug#18749: 24.3.94; CC Mode 5.32.5 (C/l);
 cc-mode state cache failure leading to bad indentation
Date: Fri, 17 Oct 2014 15:35:22 -0400

> 1000 bytes before 46793 is 45793.  This spot is in the middle of the "##"
> operator on L1346.  Thus the buffer is getting narrowed such that the
> first character, ostensibly at BOL (to the Emacs regexp matcher), is "#".

For these kinds of reasons (along with various related ones), I have
learned to stay away from narrowing.


        Stefan

Message #14 received at 18749 <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Michael Welsh Duggan <mwd <at> md5i.com>
Cc: 18749 <at> debbugs.gnu.org
Subject: Re: bug#18749: 24.3.94; CC Mode 5.32.5 (C/l); cc-mode state cache
 failure leading to bad indentation
Date: Fri, 17 Oct 2014 20:12:05 +0000

Hello, Michael.

On Fri, Oct 17, 2014 at 06:19:50PM -0000, Alan Mackenzie wrote:

> > A consistent c-mode indentation error that causes bad indentation, much
> > cursing, and gnashing of teeth.  This occurs in the latest emacs-24
> > branch (and trunk).

[ .... ]

> >     M-x c-toggle-parse-state-debug
> >     C-x C-f rwuniqsetup.c.before.2
> >     C-s ipaddr_rec_to_bin RET
> >     C-v

> > At this point, indentation is having problems.  Try hitting TAB on line
> > 1371, for example.  Or line 1374 for a complete failure to indent at
> > all.

[ .... ]

> Look forward to a patch soon!

OK, Could you try this, please, and get back to me ASAP.  Thanks!



diff -r a785658eaedb cc-engine.el
--- a/cc-engine.el	Thu Oct 02 20:17:43 2014 +0000
+++ b/cc-engine.el	Fri Oct 17 19:58:19 2014 +0000
@@ -254,6 +254,24 @@
     (setq c-macro-cache-start-pos beg
 	  c-macro-cache-syntactic nil))))
 
+(defun c-macro-is-genuine-p ()
+  ;; Check that the ostensible CPP construct at point is a real one.  In
+  ;; particular, if point is on the first line of a narrowed buffer, make sure
+  ;; that the "#" isn't, say, the second character of a "##" operator.  Return
+  ;; t when the macro is real, nil otherwise.
+  (let ((here (point)))
+    (beginning-of-line)
+    (prog1
+	(if (and (eq (point) (point-min))
+		 (/= (point) 1))
+	    (save-restriction
+	      (widen)
+	      (beginning-of-line)
+	      (and (looking-at c-anchored-cpp-prefix)
+		   (eq (match-beginning 1) here)))
+	  t)
+      (goto-char here))))
+
 (defun c-beginning-of-macro (&optional lim)
   "Go to the beginning of a preprocessor directive.
 Leave point at the beginning of the directive and return t if in one,
@@ -284,7 +302,8 @@
 	    (forward-line -1))
 	  (back-to-indentation)
 	  (if (and (<= (point) here)
-		   (looking-at c-opt-cpp-start))
+		   (looking-at c-opt-cpp-start)
+		   (c-macro-is-genuine-p))
 	      (progn
 		(setq c-macro-cache (cons (point) nil)
 		      c-macro-cache-start-pos here)


> > -- 
> > Michael Welsh Duggan
> > (md5i <at> md5i.com)

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #17 received at 18749 <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Michael Welsh Duggan <mwd <at> md5i.com>, 18749 <at> debbugs.gnu.org
Subject: Re: bug#18749: 24.3.94; CC Mode 5.32.5 (C/l); cc-mode state cache
 failure leading to bad indentation
Date: Sat, 18 Oct 2014 08:48:28 +0000

Hi, Stefan.

On Fri, Oct 17, 2014 at 03:35:22PM -0400, Stefan Monnier wrote:
> > 1000 bytes before 46793 is 45793.  This spot is in the middle of the "##"
> > operator on L1346.  Thus the buffer is getting narrowed such that the
> > first character, ostensibly at BOL (to the Emacs regexp matcher), is "#".

> For these kinds of reasons (along with various related ones), I have
> learned to stay away from narrowing.

That would be all very well if all movement and scanning primitives
(like `scan-lists' and the things that use it) actually had limit
parameters.  They don't.  What alternative, then, is there to narrowing?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #20 received at 18749 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Alan Mackenzie <acm <at> muc.de>
Cc: Michael Welsh Duggan <mwd <at> md5i.com>, 18749 <at> debbugs.gnu.org
Subject: Re: bug#18749: 24.3.94; CC Mode 5.32.5 (C/l);
 cc-mode state cache failure leading to bad indentation
Date: Sat, 18 Oct 2014 15:11:49 -0400

> That would be all very well if all movement and scanning primitives
> (like `scan-lists' and the things that use it) actually had limit
> parameters.  They don't.  What alternative, then, is there to narrowing?

In the worst case you can narrow just around the call to scan-lists.
Or you live with the fact that it might move further than you intended.
I've changed a fair bit of code to avoid narrowing, and haven't bumped
into any real show stopper so far.


        Stefan

Message #25 received at 18749-done <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: 18749-done <at> debbugs.gnu.org
Subject: Re: bug#18749: 24.3.94; CC Mode 5.32.5 (C/l);
 cc-mode state cache failure leading to bad indentation
Date: 24 Dec 2015 19:41:05 -0000

Closing the bug, which was fixed in revision #118147 on the trunk, and
revision #117673 on the Emacs-24 branch on, respectively, 2014-10-18 and
2014-11-05.

-- 
Alan Mackenzie (Nuremberg, Germany).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.