GNU bug report logs - #21875
24.5; Invalid tokens in smie indentation rules

Previous Next

To reply to this bug, email your comments to 21875 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: mail <at> ap4y.me
To: bug-gnu-emacs <at> gnu.org
Subject: 24.5; Invalid tokens in smie indentation rules
Date: Wed, 11 Nov 2015 13:59:30 +1300

For the simple grammar like this:

(id)
(inst ("if" exp "begin" insts "end")
      (exp))
(insts (insts ";" insts) (inst))
(exp (exp "." id)
     (id ":" exp)
     (exp "=" exp))
(exps (exps "," exps) (exp))

And lexer rule that overrides braces with begin/end tokens, for example
like this:

((looking-at "{") (forward-char 1) "begin")
((looking-at "}") (forward-char 1) "end")

When trying to indent such construction:

if true {
    bar
|}

I'm recieveing indentation request for :close-all '}', but no such
token was returned by lexer, full log:

forward: 19 -> 20 = end
backward: 20 -> 19 = end
backward: 19 -> 18 = ;
backward: 18 -> 15 = bar
backward: 15 -> 9 = begin
backward: 9 -> 4 = true
backward: 4 -> 1 = if
:close-all '}'; sibling-p:t parent:(nil 1 if) hanging:nil == nil
backward: 20 -> 19 = end
backward: 19 -> 18 = ;
backward: 18 -> 15 = bar
backward: 15 -> 9 = begin
backward: 9 -> 4 = true
backward: 4 -> 1 = if



In GNU Emacs 24.5.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.16.6)
 of 2015-09-10 on foutrelis
Windowing system distributor `The X.Org Foundation', version 11.0.11702000
System Description:	Arch Linux

Configured using:
 `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 --param=ssp-buffer-size=4' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'

Important settings:
  value of $EMACSLOADPATH: /home/ap4y/github/swift-mode/.cask/24.5.1/elpa/flycheck-cask-20150920.453:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/flycheck-20151027.755:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/dash-20151021.113:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/pkg-info-20150517.443:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/epl-20150517.433:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/let-alist-1.0.4:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/s-20150924.406:/usr/share/emacs/24.5/lisp:/usr/share/emacs/24.5/lisp/vc:/usr/share/emacs/24.5/lisp/url:/usr/share/emacs/24.5/lisp/textmodes:/usr/share/emacs/24.5/lisp/progmodes:/usr/share/emacs/24.5/lisp/play:/usr/share/emacs/24.5/lisp/org:/usr/share/emacs/24.5/lisp/nxml:/usr/share/emacs/24.5/lisp/net:/usr/share/emacs/24.5/lisp/mh-e:/usr/share/emacs/24.5/lisp/mail:/usr/share/emacs/24.5/lisp/leim:/usr/share/emacs/24.5/lisp/language:/usr/share/emacs/24.5/lisp/international:/usr/share/emacs/24.5/lisp/gnus:/usr/share/emacs/24.5/lisp/eshell:/usr/share/emacs/24.5/lisp/erc:/usr/share/emacs/24.5/lisp/emulation:/usr/share/emacs/24.5/lisp/emacs-parallel:/usr/share/emacs/24.5/lisp/emacs-lisp:/usr/share/emacs/24.5/lisp/cedet:/usr/share/emacs/24.5/lisp/calendar:/usr/share/emacs/24.5/lisp/calc:/usr/share/emacs/24.5/lisp/obsolete
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Swift

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
File mode specification error: (wrong-type-argument stringp func-args)

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
help-fns mail-prsvr mail-utils vc-git swift-mode edmacro kmacro easymenu
derived pcase smie cl-loaddefs cl-lib comint ansi-color ring rx
time-date tooltip electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment lisp-mode prog-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind gfilenotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

Memory information:
((conses 16 81358 6097)
 (symbols 48 18856 0)
 (miscs 40 38 135)
 (strings 32 13410 4787)
 (string-bytes 1 381215)
 (vectors 16 10445)
 (vector-slots 8 396568 6410)
 (floats 8 66 65)
 (intervals 56 205 0)
 (buffers 960 12)
 (heap 1024 36497 1152))

Message #8 received at 21875 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 21875 <at> debbugs.gnu.org, mail <at> ap4y.me
Subject: Re: bug#21875: 24.5; Invalid tokens in smie indentation rules
Date: Wed, 11 Nov 2015 14:02:58 -0500

Stefan, FYI:


mail <at> ap4y.me wrote:

> For the simple grammar like this:
>
> (id)
> (inst ("if" exp "begin" insts "end")
>       (exp))
> (insts (insts ";" insts) (inst))
> (exp (exp "." id)
>      (id ":" exp)
>      (exp "=" exp))
> (exps (exps "," exps) (exp))
>
> And lexer rule that overrides braces with begin/end tokens, for example
> like this:
>
> ((looking-at "{") (forward-char 1) "begin")
> ((looking-at "}") (forward-char 1) "end")
>
> When trying to indent such construction:
>
> if true {
>     bar
> |}
>
> I'm recieveing indentation request for :close-all '}', but no such
> token was returned by lexer, full log:
>
> forward: 19 -> 20 = end
> backward: 20 -> 19 = end
> backward: 19 -> 18 = ;
> backward: 18 -> 15 = bar
> backward: 15 -> 9 = begin
> backward: 9 -> 4 = true
> backward: 4 -> 1 = if
> :close-all '}'; sibling-p:t parent:(nil 1 if) hanging:nil == nil
> backward: 20 -> 19 = end
> backward: 19 -> 18 = ;
> backward: 18 -> 15 = bar
> backward: 15 -> 9 = begin
> backward: 9 -> 4 = true
> backward: 4 -> 1 = if
>
>
>
> In GNU Emacs 24.5.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.16.6)
>  of 2015-09-10 on foutrelis
> Windowing system distributor `The X.Org Foundation', version 11.0.11702000
> System Description:	Arch Linux
>
> Configured using:
>  `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
>  --localstatedir=/var --with-x-toolkit=gtk3 --with-xft
>  'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
>  --param=ssp-buffer-size=4' CPPFLAGS=-D_FORTIFY_SOURCE=2
>  LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'
>
> Important settings:
>   value of $EMACSLOADPATH: /home/ap4y/github/swift-mode/.cask/24.5.1/elpa/flycheck-cask-20150920.453:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/flycheck-20151027.755:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/dash-20151021.113:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/pkg-info-20150517.443:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/epl-20150517.433:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/let-alist-1.0.4:/home/ap4y/github/swift-mode/.cask/24.5.1/elpa/s-20150924.406:/usr/share/emacs/24.5/lisp:/usr/share/emacs/24.5/lisp/vc:/usr/share/emacs/24.5/lisp/url:/usr/share/emacs/24.5/lisp/textmodes:/usr/share/emacs/24.5/lisp/progmodes:/usr/share/emacs/24.5/lisp/play:/usr/share/emacs/24.5/lisp/org:/usr/share/emacs/24.5/lisp/nxml:/usr/share/emacs/24.5/lisp/net:/usr/share/emacs/24.5/lisp/mh-e:/usr/share/emacs/24.5/lisp/mail:/usr/share/emacs/24.5/lisp/leim:/usr/share/emacs/24.5/lisp/language:/usr/share/emacs/24.5/lisp/international:/usr/share/emacs/24.5/lisp/gnus:/usr/share/emacs/24.5/lisp/eshell:/usr/share/emacs/24.5/lisp/erc:/usr/share/emacs/24.5/lisp/emulation:/usr/share/emacs/24.5/lisp/emacs-parallel:/usr/share/emacs/24.5/lisp/emacs-lisp:/usr/share/emacs/24.5/lisp/cedet:/usr/share/emacs/24.5/lisp/calendar:/usr/share/emacs/24.5/lisp/calc:/usr/share/emacs/24.5/lisp/obsolete
>   value of $LANG: en_US.UTF-8
>   locale-coding-system: utf-8-unix
>
> Major mode: Swift
>
> Minor modes in effect:
>   tooltip-mode: t
>   electric-indent-mode: t
>   mouse-wheel-mode: t
>   tool-bar-mode: t
>   menu-bar-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>   line-number-mode: t
>   transient-mark-mode: t
>
> Recent messages:
> For information about GNU Emacs and the GNU system, type C-h C-a.
> File mode specification error: (wrong-type-argument stringp func-args)
>
> Load-path shadows:
> None found.

Message #11 received at 21875 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: 21875 <at> debbugs.gnu.org
Subject: Re: bug#21875: 24.5; Invalid tokens in smie indentation rules
Date: Wed, 11 Nov 2015 15:51:32 -0500

>> And lexer rule that overrides braces with begin/end tokens, for example
>> like this:
>>
>> ((looking-at "{") (forward-char 1) "begin")
>> ((looking-at "}") (forward-char 1) "end")

This is not enough info: we need the complete code so we can run it on
our side and single-step through it to see what's going on.


        Stefan

Message #14 received at 21875 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: mail <at> ap4y.me
Cc: 21875 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#21875: 24.5; Invalid tokens in smie indentation rules
Date: Wed, 11 Nov 2015 16:57:26 -0500

[resending including the OP in the list of recipients]

Stefan Monnier wrote:

>>> And lexer rule that overrides braces with begin/end tokens, for example
>>> like this:
>>>
>>> ((looking-at "{") (forward-char 1) "begin")
>>> ((looking-at "}") (forward-char 1) "end")
>
> This is not enough info: we need the complete code so we can run it on
> our side and single-step through it to see what's going on.

Message #17 received at 21875 <at> debbugs.gnu.org (full text, mbox):

From: Arthur Evstifeev <mail <at> ap4y.me>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 21875 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#21875: 24.5; Invalid tokens in smie indentation rules
Date: Fri, 13 Nov 2015 15:00:26 +1300

[Message part 1 (text/plain, inline)]

Sorry for the delay. I have attached full code of the swift-mode with
minimal smie implementation. I didn't clean other pieces of the mode
like syntax table and highlighting code, hope this won't be too
inconvenient to work with.

[swift-mode.el (application/emacs-lisp, attachment)]

[Message part 3 (text/plain, inline)]

Glenn Morris writes:

> [resending including the OP in the list of recipients]
>
> Stefan Monnier wrote:
>
>>>> And lexer rule that overrides braces with begin/end tokens, for example
>>>> like this:
>>>>
>>>> ((looking-at "{") (forward-char 1) "begin")
>>>> ((looking-at "}") (forward-char 1) "end")
>>
>> This is not enough info: we need the complete code so we can run it on
>> our side and single-step through it to see what's going on.

--
Sent with my mu4e

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.