Subject: 27.1; python mode font-lock confused by string concatenation
From: Tom Tromey <[email-masked]>
To: [email-masked]
Date: Fri, 15 Jan 2021 11:03:52 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

Consider this 2-line Python file:

x = "hello"""
y = "confused"

If you put this into a .py file, Emacs will font-lock the second line
incorrectly.  It appears that Emacs thinks the `"""' on the first line
is an opening triple quote.  However, it is actually string
concatenation, with the second string being empty.

I tripped over a case like this in some real code.

In GNU Emacs 27.1 (build 1, x86_64-redhat-linux-gnu, GTK+ Version 3.24.21, cairo version 1.16.0)
 of 2020-08-20 built on buildvm-x86-24.iad2.fedoraproject.org
Windowing system distributor 'Fedora Project', version 11.0.12010000
System Description: Fedora 32 (Workstation Edition)

Recent messages:
Wrote /home/tromey/.newsrc.eld
Saving /home/tromey/.newsrc.eld...done
(No changes need to be saved)
(New file)
Can’t guess python-indent-offset, using defaults: 4
EOL while scanning string literal [3 times]
Saving file /tmp/q.py...
Wrote /tmp/q.py
Quit
Undo

Configured using:
 'configure --build=x86_64-redhat-linux-gnu
 --host=x86_64-redhat-linux-gnu --program-prefix=
 --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr
 --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc
 --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64
 --libexecdir=/usr/libexec --localstatedir=/var
 --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --with-dbus --with-gif --with-jpeg --with-png
 --with-rsvg --with-tiff --with-xft --with-xpm --with-x-toolkit=gtk3
 --with-gpm=no --with-xwidgets --with-modules --with-harfbuzz
 --with-cairo --with-json build_alias=x86_64-redhat-linux-gnu
 host_alias=x86_64-redhat-linux-gnu 'CFLAGS=-DMAIL_USE_LOCKF -O2 -g
 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong
 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
 -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection'
 LDFLAGS=-Wl,-z,relro
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS
LIBSYSTEMD JSON PDUMPER GMP

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Python

Minor modes in effect:
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-pcomplete-mode: t
  erc-netsplit-mode: t
  erc-spelling-mode: t
  erc-truncate-mode: t
  display-fill-column-indicator-mode: t
  flyspell-mode: t
  shell-dirtrack-mode: t
  which-function-mode: t
  erc-track-mode: t
  erc-track-minor-mode: t
  erc-notify-mode: t
  erc-notifications-mode: t
  erc-match-mode: t
  erc-services-mode: t
  erc-networks-mode: t
  erc-hl-nicks-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  flycheck-mode: t
  savehist-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  auto-fill-function: python-do-auto-fill
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow emacsbug tramp-cmds yaml-mode gnus-draft vc-annotate tramp-cache
thai-util thai-word goto-addr log-edit sh-script smie executable cc-mode
cc-fonts cc-guess cc-menus cc-cmds tabify man find-dired rst ffap grep
webjump log-view pcvs-util dabbrev mule-util ggtags etags fileloop
generator bug-reference tcl jka-compr gnus-html url-queue help-fns
radix-tree url-cache mm-url flow-fill misearch multi-isearch shr-color
supercite regi smerge-mode diff python tramp-sh ada-mode ada-imenu align
ada-skel wisi-skel skeleton ada-process wisi-process-parse
ada-indent-user-options ada-core wisi-prj wisi xref wisi-fringe
wisi-parse-common semantic/lex semantic/fw mode-local uniquify-files
project find-file compile erc-list erc-menu erc-join erc-ring
erc-pcomplete erc-netsplit erc-spelling erc-truncate cl-extra smiley
mm-archive gnus-bcklg qp gnus-async gnus-ml disp-table sort gnus-cite
vc-mtn vc-hg mailalias mail-hist nnir gnus-topic nndraft nnmh nnfolder
utf-7 bbdb-gnus bbdb-mua bbdb-com crm gnutls network-stream nsm
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art
mm-uu mml2015 mm-view mml-smime smime dig nntp gnus-cache gnus-sum url
url-proxy url-privacy url-expand url-methods url-history mailcap shr
url-cookie url-domsuf svg dom gnus-group gnus-undo smtpmail gnus-start
gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int
gnus-range gnus-win gnus nnheader bbdb-message sendmail mail-extr
message rmc puny rfc822 mml mml-sec epa derived epg epg-config gnus-util
rmail rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader rfc2368 copyright add-log
make-mode vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs vc-git diff-mode
easy-mmode term/xterm xterm display-fill-column-indicator flyspell
ispell diminish appt diary-lib diary-loaddefs cal-menu calendar
cal-loaddefs tramp tramp-loaddefs trampver tramp-integration files-x
tramp-compat shell pcomplete parse-time iso8601 time-date ls-lisp
which-func imenu autorevert filenotify desktop frameset cus-start
cus-load git-link url-util erc-track erc-notify
erc-desktop-notifications erc-match erc-services erc-networks
notifications dbus erc-hl-nicks color erc-button erc-fill erc-stamp
wid-edit erc-goodies erc erc-backend erc-compat format-spec thingatpt pp
erc-loaddefs dired-aux dired-x dired dired-loaddefs warnings advice
vc-dir ewoc vc vc-dispatcher flycheck find-func help-mode rx dash
cc-styles cc-align cc-engine cc-vars cc-defs bbdb bbdb-site timezone
ange-ftp comint ansi-color ring server savehist finder-inf clang-rename
clang-include-fixer let-alist clang-format xml info package easymenu
browse-url url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache json subr-x map url-vars seq
byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting xwidget-internal cairo move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 1294698 143374)
 (symbols 48 40600 90)
 (strings 32 416876 28749)
 (string-bytes 1 9986293)
 (vectors 16 135630)
 (vector-slots 8 2470110 287678)
 (floats 8 541 727)
 (intervals 56 112191 2695)
 (buffers 1000 395))