Subject: bug#32204: CC Mode 5.33.1 (Dart//l); cc-mode without fontification

From: Alan Mackenzie <[email-masked]>
Date: Sat, 21 Jul 2018 21:08:48 +0000

Hello, Brady.

On Wed, Jul 18, 2018 at 12:55:25 -0700, Brady Trainor wrote:
> Should it be possible for a language mode author to use only the
> indentation functions from cc-mode, and write the font-lock-defaults
> independently of cc-mode?

There is currently no provision in CC Mode for this strategy.  "Should"
is a difficult, philosophical word.  ;-)

I'd normally ask at this point why you'd want to do this, but you
explain lower down.

> I attempted to do this with by using c-basic-common-init, instead of
> c-common-init.

> Here is the contents of my dart-mode.el

> --8<---------------cut here---------------start------------->8---
> ;;; dart-mode.el --- Major mode for editing Dart files -*- lexical-binding: t; -*-

> ;;; Code:

> (require 'cc-mode)
> (eval-when-compile
>   (require 'cc-langs)
>   (require 'cc-fonts))

> (eval-and-compile (c-add-language 'dart-mode 'java-mode))

> (c-lang-defconst c-multiline-string-start-char
>   dart ?@)

> (defconst dart-c-style
>   '("java")
>   "The default Dart styles.")

> (c-add-style "dart" dart-c-style)

> (defvar dart-mode-map (c-make-inherited-keymap)
>   "Keymap used in dart-mode buffers.")

> ;;;###autoload
> (add-to-list 'auto-mode-alist '("\\.dart\\'" . dart-mode))

> ;;;###autoload
> (define-derived-mode dart-mode prog-mode "Dart"
>   "Major mode for editing Dart files.

> The hook `c-mode-common-hook' is run with no args at mode
> initialization, then `dart-mode-hook'.

> Key bindings:
> \\{dart-mode-map}"
>   (c-initialize-cc-mode t)
>   (c-init-language-vars dart-mode)
>   (c-basic-common-init 'dart-mode '((dart-mode . "dart")))
>   (setq font-lock-defaults '(nil)))

> (provide 'dart-mode)

> ;;; dart-mode.el ends here
> --8<---------------cut here---------------end--------------->8---

> I then open emacs with:

> emacs -Q tmp.dart -l dart-mode.el --eval "(progn (toggle-debug-on-error) (dart-mode))"

> However, I get errors when for example tmp.dart has the contents:

> class SpaceCraft {

> I put point after opening curly, hit RET, and type the closing curly, I
> get the following stack trace:

> --8<---------------cut here---------------end--------------->8---
> Debugger entered--Lisp error: (args-out-of-range 20 40)
>   put-text-property(20 40 fontified nil)
>   c-extend-after-change-region(20 20 4)
>   font-lock-extend-jit-lock-region-after-change(20 20 4)
>   run-hook-with-args(font-lock-extend-jit-lock-region-after-change 20 20 4)
>   jit-lock-after-change(20 20 4)
>   c-shift-line-indentation(-4)
>   c-indent-line(((class-close 1)))
>   c-electric-brace(nil)
>   funcall-interactively(c-electric-brace nil)
>   call-interactively(c-electric-brace nil nil)
>   command-execute(c-electric-brace)
> --8<---------------cut here---------------end--------------->8---

I loaded dart-mode.el and tried it out.  Just before typing that closing
brace, after-change-functions looks like this:

    (jit-lock-after-change c-after-change t)

.  For CC Mode to work, it would need to look like this:

    (c-after-change jit-lock-after-change t)

, with CC Mode's after change hook executing before jit-lock's, so that
c-after-change can adjust certain position variables which later get
used by c-extend-after-change-region.

Normally this rearrangement of after-change-functions is done by the
function c-after-font-lock-init, which is placed on font-lock-mode-hook
by c-font-lock-init.

c-font-lock-init is called from c-common-init, which dart-mode isn't
calling.  It's worth emphasising that c-font-lock-init is initialising
CC Mode for font-lock, just as much as it is initialising font-lock for
CC Mode.

> If I remove the following expression, then the issue is resolved.

> (c-lang-defconst c-multiline-string-start-char
>   dart ?@)

I think it's just coincidence that that "works".  I also think that
you'll run into a whole sequence of bugs like this while attempting to
remove CC Mode's fontification routines.

> But there are many c-lang-defconst expressions in current dart-mode
> master, and I have not understood cc-mode enough to understand why each
> should be there, and which should not.

All the c-lang-defconsts are needed; if you don't specify a value, you
will inherit the value from the base language (here, Java Mode).  The
c-lang-defconts are compile time variables.  From them are built the
c-lang-defvars, which are normal runtime variables (constants, really)
which take on different values for different languages.  In a sense they
_define_ the different CC Mode languages.

> If I remove the above line, I can continue to find such errors, so it
> seems more about having removed the cc-mode fontification from the
> major mode.

Yes.  Bear in mind CC Mode is very old code indeed, it's development
extending back more that 30 years.  In that time it's become somewhat
twisted and inelegant (for which I'm partly to blame).

> (Why would I want to remove only the cc fontification? dart-mode may
> become easier to maintain and deal with edge cases once we remove
> cc-mode framework from its implementation. I would like to do this
> piecemeal. It may take some time to create an indentation function from
> scratch that is strictly as good as the one implemented via cc-mode
> framework.

This depends on whether you definitely want to build a better mode, or
whether you merely want to have a better mode.  Indentation functions
for C-like languages are difficult.  Correct fontification is not much
easier.  As a matter of interest, Stefan Monnier has done some work
creating a modern C Mode; I'm not entirely sure how far he's got, but
it's certainly at least basically working.  You can reach him on the
Emacs developers' mailing list, [email-masked].

> But there are several issues on issue tracker related to
> fontification, which are quite tractable to solve simply using the
> font-lock framework standing alone.

Fontification (and indeed indentation) errors tend to be like bubbles of
paste underneath freshly hung wallpaper; you think you've managed to
eliminate an unsightly bubble, but it's just popped up somewhere else.

> So I would like to add my fontification improvements to the master
> branch, leaving cc-mode indentation intact, so that I am not creating
> regressions on master branch.

Do make sure they really are improvements.  You might well be interested
in the CC Mode unit tests which test indentation and fontification.
They're in directory "tests" in the standalone CC Mode repository at
SourceForge.

> In fact, inspired by haskell-mode, I'd like to offer multiple choices
> in indentation functions, so that potentially, contributors can
> improve their choice of either the cc-mode indentation or the "from
> scratch" indentation function.

CC Mode's indentation is already very configurable, see the CC Mode
manual.  There is no single "CC Mode indentation".

> I have more ideas for choices in indentation, like being able to call
> `dartfmt` on regions, or have it only look at region between point and
> point-min (to avoid dartfmt's predilection to choke and spew errors).)

> Thank you for reading!

Happy hacking!

> --
> Brady

[ CC Mode configuration appreciated, but snipped. ]

-- 
Alan Mackenzie (Nuremberg, Germany).