GNU bug report logs - #54131
29.0.50; Flyspell incorrectly reports first word in Python f-string

Previous Next

To reply to this bug, email your comments to 54131 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.50; Flyspell incorrectly reports first word in Python f-string
Date: Wed, 23 Feb 2022 19:16:11 +0100

Given this Python file:

$ cat /tmp/a.py 
print(f'hello world')

Visit it and enable Flyspell:

$ emacs -Q /tmp/a.py -f flyspell-prog-mode

Flyspell then marks the string "f'hello" as incorrect, thinking it's a
misspelling of "hello".  But it shouldn't cross the string boundary.


In GNU Emacs 29.0.50 (build 59, x86_64-pc-linux-gnu, GTK+ Version 3.24.31, cairo version 1.16.0)
 of 2022-02-23
Repository revision: 85ad8616007e286c237bb2906d1928bb551462e7
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12014000
System Description: Debian GNU/Linux rodete

Configured using:
 'configure --enable-gcc-warnings=warn-only
 --enable-gtk-deprecation-warnings --without-pop --with-mailutils
 --enable-checking=all --enable-check-lisp-object-type --with-modules
 'CFLAGS=-O0 -ggdb3''

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP
SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LC_TIME: en_DK.utf8
  value of $LANG: en_US.utf8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail phst skeleton pcase ffap
thingatpt url url-proxy url-privacy url-expand url-methods url-history
url-cookie url-domsuf url-util url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs json map url-vars rx message mailcap
yank-media rmc dired dired-loaddefs rfc822 mml mml-sec password-cache
epa derived epg rfc6068 epg-config gnus-util time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader gnutls
puny elp dbus xml seq gv subr-x byte-opt bytecomp byte-compile cconv
compile text-property-search comint ansi-color ring cl-loaddefs cl-lib
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget keymap hashtable-print-readable backquote threads
dbusbind inotify dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 67449 9159)
 (symbols 48 8204 1)
 (strings 32 23683 2078)
 (string-bytes 1 760952)
 (vectors 16 15596)
 (vector-slots 8 208549 49248)
 (floats 8 28 30)
 (intervals 56 230 0)
 (buffers 992 11))

-- 
Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Geschäftsführer: Paul Manicle, Liana Sebastian
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich.  Falls Sie diese fälschlicherweise erhalten haben
sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie
alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail
an die falsche Person gesendet wurde.

This e-mail is confidential.  If you received this communication by mistake,
please don’t forward it to anyone else, please erase all copies and
attachments, and please let me know that it has gone to the wrong person.

Message #8 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50; Flyspell incorrectly reports first word in
 Python f-string
Date: Wed, 23 Feb 2022 21:13:27 +0100

Philipp Stephani <p.stephani2 <at> gmail.com> writes:

> Given this Python file:
>
> $ cat /tmp/a.py 
> print(f'hello world')
>
> Visit it and enable Flyspell:
>
> $ emacs -Q /tmp/a.py -f flyspell-prog-mode
>
> Flyspell then marks the string "f'hello" as incorrect, thinking it's a
> misspelling of "hello".  But it shouldn't cross the string boundary.

Hm, yes.  In this case, the mode knows that f isn't part of the
expression, but I guess we have no way of communicating that to ispell?

Skimming ispell-get-word, it looks like it uses a regexp to determine
what the word at point is, so we'd need to make some sort of framework
to allow modes to say where a string begins and ends?  Or if we want to
just do something hackish, we could make that function check
the face for font-lock-string-face and then limit based on that.  (Which
sounds simple enough.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #11 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50; Flyspell incorrectly reports first word in
 Python f-string
Date: Wed, 23 Feb 2022 21:27:04 +0100

Am Mi., 23. Feb. 2022 um 21:13 Uhr schrieb Lars Ingebrigtsen <larsi <at> gnus.org>:
>
> Philipp Stephani <p.stephani2 <at> gmail.com> writes:
>
> > Given this Python file:
> >
> > $ cat /tmp/a.py
> > print(f'hello world')
> >
> > Visit it and enable Flyspell:
> >
> > $ emacs -Q /tmp/a.py -f flyspell-prog-mode
> >
> > Flyspell then marks the string "f'hello" as incorrect, thinking it's a
> > misspelling of "hello".  But it shouldn't cross the string boundary.
>
> Hm, yes.  In this case, the mode knows that f isn't part of the
> expression, but I guess we have no way of communicating that to ispell?
>
> Skimming ispell-get-word, it looks like it uses a regexp to determine
> what the word at point is, so we'd need to make some sort of framework
> to allow modes to say where a string begins and ends?

Like the syntax table? (nth 8 (syntax-ppss)) gives you the beginning
of the string, and that seems to give correct results even for
f-strings.

Message #14 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: p.stephani2 <at> gmail.com, 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50;
 Flyspell incorrectly reports first word in Python f-string
Date: Thu, 24 Feb 2022 08:27:49 +0200

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Date: Wed, 23 Feb 2022 21:13:27 +0100
> Cc: 54131 <at> debbugs.gnu.org
> 
> Skimming ispell-get-word, it looks like it uses a regexp to determine
> what the word at point is, so we'd need to make some sort of framework
> to allow modes to say where a string begins and ends?  Or if we want to
> just do something hackish, we could make that function check
> the face for font-lock-string-face and then limit based on that.  (Which
> sounds simple enough.)

Our spell-checking modes aren't supposed to work on code, so they
don't pay any attention to the buffer's syntax table, and instead use
regular expressions specific to the proof-reading language to know
what can and cannot be in a word.

For program sources, flyspell-prog-mode relies on font-lock faces to
tell it where strings and comments are, so perhaps this part doesn't
work for some reason (and perhaps the root cause is in font-lock of
python-mode, not in flyspell.el per se).

Message #17 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: larsi <at> gnus.org, 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50;
 Flyspell incorrectly reports first word in Python f-string
Date: Thu, 24 Feb 2022 08:33:24 +0200

> Resent-From: Philipp Stephani <p.stephani2 <at> gmail.com>
> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
> Resent-CC: bug-gnu-emacs <at> gnu.org
> Resent-Sender: help-debbugs <at> gnu.org
> From: Philipp Stephani <p.stephani2 <at> gmail.com>
> Date: Wed, 23 Feb 2022 21:27:04 +0100
> Cc: 54131 <at> debbugs.gnu.org
> 
> Am Mi., 23. Feb. 2022 um 21:13 Uhr schrieb Lars Ingebrigtsen <larsi <at> gnus.org>:
> >
> > Philipp Stephani <p.stephani2 <at> gmail.com> writes:
> >
> > > Given this Python file:
> > >
> > > $ cat /tmp/a.py
> > > print(f'hello world')
> > >
> > > Visit it and enable Flyspell:
> > >
> > > $ emacs -Q /tmp/a.py -f flyspell-prog-mode
> > >
> > > Flyspell then marks the string "f'hello" as incorrect, thinking it's a
> > > misspelling of "hello".  But it shouldn't cross the string boundary.
> >
> > Hm, yes.  In this case, the mode knows that f isn't part of the
> > expression, but I guess we have no way of communicating that to ispell?
> >
> > Skimming ispell-get-word, it looks like it uses a regexp to determine
> > what the word at point is, so we'd need to make some sort of framework
> > to allow modes to say where a string begins and ends?
> 
> Like the syntax table? (nth 8 (syntax-ppss)) gives you the beginning
> of the string, and that seems to give correct results even for
> f-strings.

That's not how flyspell-prog-mode works, see my other message.  I
guess it doesn't want to run syntax analysis functions on the fly, as
that could be too slow (flyspell being on post-command-hook), but
instead relies on font lock that is run by the display engine anyway.

Message #20 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: p.stephani2 <at> gmail.com, 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50; Flyspell incorrectly reports first word in
 Python f-string
Date: Thu, 24 Feb 2022 10:12:58 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

> For program sources, flyspell-prog-mode relies on font-lock faces to
> tell it where strings and comments are, so perhaps this part doesn't
> work for some reason (and perhaps the root cause is in font-lock of
> python-mode, not in flyspell.el per se).

I didn't know about flyspell-prog-mode, but it does indeed do the right
thing here.  (I've now mentioned the mode in the flyspell-mode doc
string.)

But `M-$' does not do the right thing -- if you `M-$' on f'hello, it
tries to correct the whole thing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #23 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: martin rudalics <rudalics <at> gmx.at>
To: Lars Ingebrigtsen <larsi <at> gnus.org>, Eli Zaretskii <eliz <at> gnu.org>
Cc: p.stephani2 <at> gmail.com, 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50; Flyspell incorrectly reports first word in
 Python f-string
Date: Thu, 24 Feb 2022 10:26:29 +0100

> But `M-$' does not do the right thing -- if you `M-$' on f'hello, it
> tries to correct the whole thing.

Try with 'ispell-comment-or-string-at-point'.

martin

Message #26 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: p.stephani2 <at> gmail.com, 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50; Flyspell incorrectly reports first word in
 Python f-string
Date: Thu, 24 Feb 2022 12:19:59 +0200

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: p.stephani2 <at> gmail.com,  54131 <at> debbugs.gnu.org
> Date: Thu, 24 Feb 2022 10:12:58 +0100
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > For program sources, flyspell-prog-mode relies on font-lock faces to
> > tell it where strings and comments are, so perhaps this part doesn't
> > work for some reason (and perhaps the root cause is in font-lock of
> > python-mode, not in flyspell.el per se).
> 
> I didn't know about flyspell-prog-mode, but it does indeed do the right
> thing here.  (I've now mentioned the mode in the flyspell-mode doc
> string.)

That's strange, because the OP explicitly invoked flyspell-prog-mode,
so how come it didn't work for Philipp, but did for you?

> But `M-$' does not do the right thing -- if you `M-$' on f'hello, it
> tries to correct the whole thing.

Maybe we should have a key binding for
ispell-comment-or-string-at-point, and/or maybe
ispell-comments-and-strings and flyspell-prog-mode should rebind M-$
to ispell-comment-or-string-at-point.

Message #29 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: Visuwesh <visuweshm <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Lars Ingebrigtsen <larsi <at> gnus.org>, p.stephani2 <at> gmail.com,
 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50; Flyspell incorrectly reports first word in
 Python f-string
Date: Thu, 24 Feb 2022 16:53:48 +0530

[வியாழன், பிப்ரவரி 24 2022] Eli Zaretskii wrote:

>> From: Lars Ingebrigtsen <larsi <at> gnus.org>
>> Cc: p.stephani2 <at> gmail.com,  54131 <at> debbugs.gnu.org
>> Date: Thu, 24 Feb 2022 10:12:58 +0100
>> 
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>> > For program sources, flyspell-prog-mode relies on font-lock faces to
>> > tell it where strings and comments are, so perhaps this part doesn't
>> > work for some reason (and perhaps the root cause is in font-lock of
>> > python-mode, not in flyspell.el per se).
>> 
>> I didn't know about flyspell-prog-mode, but it does indeed do the right
>> thing here.  (I've now mentioned the mode in the flyspell-mode doc
>> string.)
>
> That's strange, because the OP explicitly invoked flyspell-prog-mode,
> so how come it didn't work for Philipp, but did for you?
>
>> But `M-$' does not do the right thing -- if you `M-$' on f'hello, it
>> tries to correct the whole thing.
>
> Maybe we should have a key binding for
> ispell-comment-or-string-at-point, and/or maybe
> ispell-comments-and-strings and flyspell-prog-mode should rebind M-$
> to ispell-comment-or-string-at-point.

I misspell parts of a symbol quite often so rebinding M-$ to a command
that NOOPs when the point is not over a comment/string would end up
annoying, IMO.  I am not sure how common this usecase is though.

Message #32 received at 54131 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Visuwesh <visuweshm <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, p.stephani2 <at> gmail.com, 54131 <at> debbugs.gnu.org
Subject: Re: bug#54131: 29.0.50; Flyspell incorrectly reports first word in
 Python f-string
Date: Thu, 24 Feb 2022 15:36:46 +0100

Visuwesh <visuweshm <at> gmail.com> writes:

> I misspell parts of a symbol quite often so rebinding M-$ to a command
> that NOOPs when the point is not over a comment/string would end up
> annoying, IMO.  I am not sure how common this usecase is though.

I think it's pretty common.  I think a new command that does what
ispell-comment-or-string-at-point does if we're in a comment/string and
what ispell-word does otherwise might be nice.

And flyspell-prog-mode could rebind `M-$', as Eli suggests, but to that
new command.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.