GNU bug report logs - #19455
24.4; sh-mode highlighting breaks with ZSH glob modifiers

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 19455 in the body.
You can then email your comments to 19455 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jonathan H <pythonnut <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.4; sh-mode highlighting breaks with ZSH glob modifiers
Date: Sat, 27 Dec 2014 19:03:56 -0800

[Message part 1 (text/plain, inline)]

In ZSH globs can have optional modifiers before globs (as well as
after). Usually, the format is (#<letter here>)<glob>. The leading
pound symbol confuses sh-mode syntax highlighting, making it appear to
be a comment. As these modifiers usually appear in a conditional
expression (they are used for group capture), they also break
indentation.

I've attached a file that demonstrates this.

In GNU Emacs 24.4.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.14.3)
 of 2014-10-21 on bitzer.hoetzel.info
Windowing system distributor `The Cygwin/X Project', version 11.0.11602000
Configured using:
 `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 --param=ssp-buffer-size=4' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: λ

Minor modes in effect:
  ido-vertical-mode: t
  flx-ido-mode: t
  minibuffer-depth-indicate-mode: t
  magit-auto-revert-mode: t
  diff-auto-refine-mode: t
  shell-dirtrack-mode: t
  helm-match-plugin-mode: t
  helm-occur-match-plugin-mode: t
  aggressive-indent-mode: t
  global-flycheck-mode: t
  flyspell-mode: t
  ido-ubiquitous-mode: t
  auto-indent-global-mode: t
  auto-indent-mode: t
  global-auto-complete-mode: t
  auto-complete-mode: t
  cua-mode: t
  adaptive-wrap-prefix-mode: t
  show-paren-mode: t
  rainbow-delimiters-mode: t
  smartparens-global-mode: t
  smartparens-mode: t
  linum-mode: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  evil-leader-mode: t
  evil-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  evil-local-mode: t
  savehist-mode: t
  global-hl-line-mode: t
  delete-selection-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  global-visual-line-mode: t
  visual-line-mode: t
  transient-mark-mode: t

Recent input:
<help-echo> <select-window> <help-echo> <select-window>
<help-echo> <select-window> <select-window> <select-window>
<help-echo> <down-mouse-1> <mouse-movement> <mouse-1>
M-x r e p o r t - e m a <return>

Recent messages:
Loading /home/pythonnut/.emacs.d/.mc-lists.el (source)...done
idle-require: require windmove
idle-require: require framemove
Loading easy-kill...done
idle-require: require flx-isearch
idle-require: require whole-line-or-region
Loading smex...done
idle-require: require psvn
idle-require finished
(No files need saving)

Load-path shadows:
~/.emacs.d/personal/ws-butler hides
/home/pythonnut/.emacs.d/elpa/ws-butler-20140816.1448/ws-butler
~/.emacs.d/personal/edmacro hides /usr/share/emacs/24.4/lisp/edmacro

Features:
(shadow sort mail-extr emacsbug sendmail ido-vertical-mode flx-ido
mb-depth psvn smex whole-line-or-region flx-isearch flx easy-kill
framemove multiple-cursors mc-hide-unmatched-lines-mode
mc-separate-operations rectangular-region-mode mc-mark-pop mc-mark-more
mc-cycle-cursors mc-edit-lines multiple-cursors-core magit-filenotify
icicles icicles-mode icicles-cmd2 icicles-cmd1 icicles-mcmd icicles-fn
icicles-var icicles-opt cus-theme cus-edit wid-edit hexrgb icicles-face
helm-ag helm-semantic helm-imenu imenu semantic/util-modes semantic/util
semantic semantic/tag semantic/lex semantic/fw mode-local cedet
helm-projectile helm-config async-bytecomp async helm-aliases
magit-key-mode magit view epa derived epg diff-git diff-mode autorevert
filenotify git-rebase-mode git-commit-mode server log-edit message idna
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 ietf-drums mailabbrev mail-utils gmm-utils mailheader
pcvs-util add-log projectile ibuf-ext ibuffer pkg-info lisp-mnt epl f s
ucs-normalize helm-ring helm-elisp helm-eval eldoc helm-files
image-dired tramp tramp-compat tramp-loaddefs trampver shell dired-x
dired-aux ffap helm-buffers helm-elscreen helm-tags helm-bookmark
helm-adaptive helm-info helm-net browse-url xml url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util url-parse auth-source gnus-util mm-util mail-prsvr
password-cache url-vars mailcap bookmark pp helm-locate helm-help
helm-org org org-macro org-footnote org-pcomplete pcomplete org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint
ob-core ob-eval org-compat org-macs org-loaddefs format-spec cal-menu
calendar cal-loaddefs helm-match-plugin helm-grep helm-regexp
helm-plugin grep helm-external helm-utils ls-lisp dired helm helm-source
eieio eieio-core idle-require flymake compile comint ansi-color
aggressive-indent easy-mmode names edebug flycheck find-func help-mode
rx pcase subr-x flyspell ispell load-dir config-solarized
solarized-dark-theme solarized config-intel config-minibuffer
config-icicles config-helm config-ido ido-ubiquitous warnings
auto-indent-mode auto-complete-config auto-complete popup ido config-vcs
config-auto-complete config-paste cua-base config-whitespace
adaptive-wrap config-indent config-ui paren config-rainbow-delimiters
rainbow-delimiters config-smartparens smartparens-config smartparens
dash config-undo diminish config-ace-easymotion evil-easymotion noflet
cl-indent cl-macs ace-jump-mode cl gv config-line-numbers linum-relative
linum config-scroll config-evil config-evil-textobjects
config-evil-modules evil-surround evil-leader evil evil-integration
undo-tree diff evil-maps evil-commands evil-command-window evil-types
evil-search evil-ex evil-macros evil-repeat evil-states evil-core
evil-common windmove thingatpt rect evil-digraphs evil-vars ring
config-safety config-desktop savehist config-modes key-chord hl-line
config-package finder-inf edmacro kmacro cl-loaddefs cl-lib info
easymenu package epg-config config-setq delsel cus-start cus-load
byte-opt bytecomp byte-compile cconv advice help-fns time-date tooltip
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 16 782052 178836)
  (symbols 48 58485 0)
  (miscs 40 120 268)
  (strings 32 135655 141330)
  (string-bytes 1 4259469)
  (vectors 16 110096)
  (vector-slots 8 2260594 96673)
  (floats 8 10670 428)
  (intervals 56 550 248)
  (buffers 960 12)
  (heap 1024 63413 7100))

[zsh_testcase.zsh.txt (text/plain, attachment)]

Message #8 received at 19455 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan H <pythonnut <at> gmail.com>
To: 19455 <at> debbugs.gnu.org
Subject: Preliminary Patch
Date: Sat, 20 Feb 2016 18:07:56 -0800

[Message part 1 (text/plain, inline)]

Hi all!

I've produced a patch that should fix this issue. As brought up in this
Emacs.SE answer <http://emacs.stackexchange.com/a/20406/2642>, the main
challenge was distinguishing the qualifiers from subshells. To do this, I
disallowed leading octothorpes wrapped in parentheses where the parentheses
are not separated from other tokens by metacharacters. For qualifiers
(which appear after a glob), the fix is trivial. For flags (which appear
before the glob), I have to rely on the fact that flags may not contain
parentheses themselves to avoid needing more sophisticated parsing.

I've tested this against a fairly comprehensive test file, which I've also
attached.

Thanks,
Jonathan

[Message part 2 (text/html, inline)]

[0001-Highlight-zsh-glob-flags-and-qualifiers-in-sh-mode.patch (application/octet-stream, attachment)]

[sh_comment_tests.sh (application/x-sh, attachment)]

Message #11 received at 19455 <at> debbugs.gnu.org (full text, mbox):

From: Alex <agrambot <at> gmail.com>
To: Jonathan H <pythonnut <at> gmail.com>
Cc: 19455 <at> debbugs.gnu.org
Subject: Re: bug#19455: 24.4;
 sh-mode highlighting breaks with ZSH glob modifiers
Date: Sat, 13 Aug 2016 22:26:04 -0600

tags 19455 patch
found 19455 25.1.50
quit

I was just about to report a similar issue. My issue was with a
zsh-specific substitution, such as:

echo ${$(some_function)#*.}

The patch provided fixed that for me.

Message #18 received at 19455 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan H <pythonnut <at> gmail.com>
To: 19455 <at> debbugs.gnu.org
Subject: Bump
Date: Tue, 20 Dec 2016 08:40:46 -0800

[Message part 1 (text/plain, inline)]

Since misinterpreting glob modifiers often breaks the matching of
paired-delimiters (which in turn breaks indentation) this particular
problem can be quite crippling.

What do I need to do to get my (or another, should something be wrong with
it) patch applied?

[Message part 2 (text/html, inline)]

Message #21 received at 19455 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jonathan H <pythonnut <at> gmail.com>
Cc: 19455 <at> debbugs.gnu.org
Subject: Re: bug#19455: Bump
Date: Tue, 20 Dec 2016 19:09:58 +0200

> From: Jonathan H <pythonnut <at> gmail.com>
> Date: Tue, 20 Dec 2016 08:40:46 -0800
> 
> What do I need to do to get my (or another, should something be wrong with it) patch applied? 

Someone who knows zsh and sh-script.el to review the patch and okay
it.

Thanks.

Message #24 received at 19455 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Jonathan H <pythonnut <at> gmail.com>
Cc: 19455 <at> debbugs.gnu.org
Subject: Re: bug#19455: Preliminary Patch
Date: Tue, 25 Jun 2019 23:00:59 +0200

Jonathan H <pythonnut <at> gmail.com> writes:

> I've tested this against a fairly comprehensive test file, which I've
> also attached.

Thanks; I've tested the change with the test file, and things seem to
work as promised.  I've now applied the patch to the Emacs trunk.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.