GNU bug report logs - #2967
smart quotes in .emacs

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 2967 in the body.
You can then email your comments to 2967 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: "Kit O'Connell" <vulpine <at> pobox.com>
To: bug-gnu-emacs <at> gnu.org
Subject: smart quotes in .emacs
Date: Sun, 12 Apr 2009 07:53:37 -0500

Hi,

I was troubleshooting a friend's .emacs issues with the help of some
internet assistance. When I copy/pasted people's suggested .emacs
entries, some smart quotes got into the file, causing error messages.
You can see the whole process of how this happened here:
http://dreamcafe.com/words/2009/04/11/help-us-dr-internet-emacs-edition/#comment-5222

I think it would be good if emacs looked for smart quotes in .emacs
files and gave a warning or notice if it detected them. This would
help troubleshooting.

Kit

Message #12 received at 2967 <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: "Kit O'Connell" <vulpine <at> pobox.com>
Cc: 2967 <at> debbugs.gnu.org
Subject: Re: bug#2967: smart quotes in .emacs
Date: Mon, 13 Apr 2009 08:23:06 -0400

> I was troubleshooting a friend's .emacs issues with the help of some
> internet assistance.  When I copy/pasted people's suggested .emacs
> entries, some smart quotes got into the file, causing error messages.

Please explain what you mean by "smart quotes" and describe the error
messages you got (and when you got them, and if they seemed to have any
actual effect).


        Stefan

Message #15 received at 2967 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 2967 <at> debbugs.gnu.org
Subject: Re: bug#2967: smart quotes in .emacs
Date: Fri, 10 Aug 2012 15:59:01 -0400

Stefan Monnier wrote:

>> I was troubleshooting a friend's .emacs issues with the help of some
>> internet assistance.  When I copy/pasted people's suggested .emacs
>> entries, some smart quotes got into the file, causing error messages.
>
> Please explain what you mean by "smart quotes" and describe the error
> messages you got (and when you got them, and if they seemed to have any
> actual effect).

I assume the OP means something like:

(set 'foo t)

where the quote character is actually U+2019 "RIGHT SINGLE QUOTATION MARK"
rather than a plain old-ASCII apostrophe.

This leads to errors like:

  Symbol's value as variable is void: 'foo

Recent example:

http://lists.gnu.org/archive/html/help-gnu-emacs/2012-08/msg00117.html

Message #18 received at 2967 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Glenn Morris <rgm <at> gnu.org>
Cc: 2967 <at> debbugs.gnu.org, Alan Mackenzie <acm <at> muc.de>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#2967: smart quotes in .emacs
Date: Tue, 06 Jun 2017 23:46:48 -0400

Glenn Morris <rgm <at> gnu.org> writes:

> I assume the OP means something like:
>
> (set 'foo t)
>
> where the quote character is actually U+2019 "RIGHT SINGLE QUOTATION MARK"
> rather than a plain old-ASCII apostrophe.
>
> This leads to errors like:
>
>   Symbol's value as variable is void: 'foo

Adding Alan to Cc, I think this is relevant also to the scenario in
Bug#23425.

I propose making these funny quotes give a read syntax error in symbol
name contexts.  Theoretically this breaks backwards compability with
code that uses variables containing them, but probably no such code
exists.

Something like this (not sure if this is the best way to test for them):

--- i/src/lread.c
+++ w/src/lread.c
@@ -3349,6 +3349,15 @@ read1 (Lisp_Object readcharfun, int *pch, bool first_in_list)
 	     ? multibyte_chars_in_text ((unsigned char *) read_buffer,
 					nbytes)
 	     : nbytes);
+        if (!quoted && multibyte)
+          {
+            switch (STRING_CHAR ((unsigned char *) read_buffer))
+              {
+              case 0x2018:      /* LEFT SINGLE QUOTATION MARK */
+              case 0x2019:      /* RIGHT SINGLE QUOTATION MARK */
+                invalid_syntax ("strange quote");
+              }
+          }
 	Lisp_Object name = ((uninterned_symbol && ! NILP (Vpurify_flag)
 			     ? make_pure_string : make_specified_string)
 			    (read_buffer, nchars, nbytes, multibyte));

We then get the following error from evaluating (read (format-message "'foo")):

Debugger entered--Lisp error: (invalid-read-syntax "strange quote")
  read("’foo")
  eval((read (format-message "'foo")) nil)

Message #21 received at 2967 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: npostavs <at> users.sourceforge.net
Cc: 2967 <at> debbugs.gnu.org, rgm <at> gnu.org, monnier <at> iro.umontreal.ca, acm <at> muc.de
Subject: Re: bug#2967: smart quotes in .emacs
Date: Wed, 07 Jun 2017 08:09:30 +0300

> From: npostavs <at> users.sourceforge.net
> Date: Tue, 06 Jun 2017 23:46:48 -0400
> Cc: 2967 <at> debbugs.gnu.org, Alan Mackenzie <acm <at> muc.de>,
> 	Stefan Monnier <monnier <at> iro.umontreal.ca>
> 
> --- i/src/lread.c
> +++ w/src/lread.c
> @@ -3349,6 +3349,15 @@ read1 (Lisp_Object readcharfun, int *pch, bool first_in_list)
>  	     ? multibyte_chars_in_text ((unsigned char *) read_buffer,
>  					nbytes)
>  	     : nbytes);
> +        if (!quoted && multibyte)
> +          {
> +            switch (STRING_CHAR ((unsigned char *) read_buffer))
> +              {
> +              case 0x2018:      /* LEFT SINGLE QUOTATION MARK */
> +              case 0x2019:      /* RIGHT SINGLE QUOTATION MARK */
> +                invalid_syntax ("strange quote");
> +              }
> +          }
>  	Lisp_Object name = ((uninterned_symbol && ! NILP (Vpurify_flag)
>  			     ? make_pure_string : make_specified_string)
>  			    (read_buffer, nchars, nbytes, multibyte));
> 
> We then get the following error from evaluating (read (format-message "'foo")):
> 
> Debugger entered--Lisp error: (invalid-read-syntax "strange quote")
>   read("’foo")
>   eval((read (format-message "'foo")) nil)

Please include the character itself in the error message text.

Also, are we sure these two are the only ones we want to catch in this
manner?

Finally, if this is accepted, let's have a test for this, and maybe
also some documentation, at least in NEWS.

Thanks.

Message #24 received at 2967 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 2967 <at> debbugs.gnu.org, rgm <at> gnu.org, monnier <at> iro.umontreal.ca, acm <at> muc.de
Subject: Re: bug#2967: smart quotes in .emacs
Date: Sun, 16 Jul 2017 20:49:39 -0400

[Message part 1 (text/plain, inline)]

tags 2967 + patch
quit

Eli Zaretskii <eliz <at> gnu.org> writes:

> Please include the character itself in the error message text.
>
> Also, are we sure these two are the only ones we want to catch in this
> manner?

I added some more likely looking ones from
https://en.wikipedia.org/wiki/Quotation_mark_glyphs.

> Finally, if this is accepted, let's have a test for this, and maybe
> also some documentation, at least in NEWS.

[v2-0001-Signal-error-for-symbol-names-with-strange-quotes.patch (text/x-diff, inline)]

From 438030f42698f347e38a9dea1561eedadd7e2edc Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs <at> gmail.com>
Date: Wed, 7 Jun 2017 19:59:09 -0400
Subject: [PATCH v2] Signal error for symbol names with strange quotes
 (Bug#2967)

* src/lread.c (read1): Signal an error when a symbol starts with a
non-escaped quote-like character.
* test/src/lread-tests.el (lread-tests--funny-quote-symbols): New
test.
* etc/NEWS: Announce change.
---
 etc/NEWS                |  4 ++++
 src/lread.c             | 18 ++++++++++++++++++
 test/src/lread-tests.el | 17 +++++++++++++++++
 3 files changed, 39 insertions(+)

diff --git a/etc/NEWS b/etc/NEWS
index edb71118ef..d297d049d1 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1110,6 +1110,10 @@ instead of its first.
 renamed to 'lread--old-style-backquotes'.  No user code should use
 this variable.
 
+** To avoid confusion caused by "smart quotes", the reader no longer
+accepts Lisp symbols which begin with the following quotation
+characters: ‘’‛“”‟〞＂＇, unless they are escaped with backslash.
+
 +++
 ** Module functions are now implemented slightly differently; in
 particular, the function 'internal--module-call' has been removed.
diff --git a/src/lread.c b/src/lread.c
index 901e40b348..dbaadce4b4 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -3479,6 +3479,24 @@ read1 (Lisp_Object readcharfun, int *pch, bool first_in_list)
 	    if (! NILP (result))
 	      return unbind_to (count, result);
 	  }
+        if (!quoted && multibyte)
+          {
+            int ch = STRING_CHAR ((unsigned char *) read_buffer);
+            switch (ch)
+              {
+              case 0x2018: /* LEFT SINGLE QUOTATION MARK */
+              case 0x2019: /* RIGHT SINGLE QUOTATION MARK */
+              case 0x201B: /* SINGLE HIGH-REVERSED-9 QUOTATION MARK */
+              case 0x201C: /* LEFT DOUBLE QUOTATION MARK */
+              case 0x201D: /* RIGHT DOUBLE QUOTATION MARK */
+              case 0x201F: /* DOUBLE HIGH-REVERSED-9 QUOTATION MARK */
+              case 0x301E: /* DOUBLE PRIME QUOTATION MARK */
+              case 0xFF02: /* FULLWIDTH QUOTATION MARK */
+              case 0xFF07: /* FULLWIDTH APOSTROPHE */
+                xsignal2 (Qinvalid_read_syntax, build_string ("strange quote"),
+                          CALLN (Fstring, make_number (ch)));
+              }
+          }
 	{
 	  Lisp_Object result;
 	  ptrdiff_t nbytes = p - read_buffer;
diff --git a/test/src/lread-tests.el b/test/src/lread-tests.el
index a0a317feee..dd5a2003b4 100644
--- a/test/src/lread-tests.el
+++ b/test/src/lread-tests.el
@@ -142,6 +142,23 @@ (ert-deftest lread-tests--unescaped-char-literals ()
                            "unescaped character literals "
                            "`?\"', `?(', `?)', `?;', `?[', `?]' detected!")))))
 
+(ert-deftest lread-tests--funny-quote-symbols ()
+  "Check that 'smart quotes' or similar trigger errors in symbol names."
+  (dolist (quote-char
+           '(#x2018 ;; LEFT SINGLE QUOTATION MARK
+             #x2019 ;; RIGHT SINGLE QUOTATION MARK
+             #x201B ;; SINGLE HIGH-REVERSED-9 QUOTATION MARK
+             #x201C ;; LEFT DOUBLE QUOTATION MARK
+             #x201D ;; RIGHT DOUBLE QUOTATION MARK
+             #x201F ;; DOUBLE HIGH-REVERSED-9 QUOTATION MARK
+             #x301E ;; DOUBLE PRIME QUOTATION MARK
+             #xFF02 ;; FULLWIDTH QUOTATION MARK
+             #xFF07 ;; FULLWIDTH APOSTROPHE
+             ))
+    (let ((str (format "%cfoo" quote-char)))
+     (should-error (read str) :type 'invalid-read-syntax)
+     (should (eq (read (concat "\\" str)) (intern str))))))
+
 (ert-deftest lread-test-bug26837 ()
   "Test for http://debbugs.gnu.org/26837 ."
   (let ((load-path (cons
-- 
2.11.1

Message #29 received at 2967 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: npostavs <at> users.sourceforge.net
Cc: 2967 <at> debbugs.gnu.org, rgm <at> gnu.org, monnier <at> iro.umontreal.ca, acm <at> muc.de
Subject: Re: bug#2967: smart quotes in .emacs
Date: Sat, 22 Jul 2017 12:20:59 +0300

> From: npostavs <at> users.sourceforge.net
> Cc: 2967 <at> debbugs.gnu.org,  rgm <at> gnu.org,  monnier <at> iro.umontreal.ca,  acm <at> muc.de
> Date: Sun, 16 Jul 2017 20:49:39 -0400
> 
> > Please include the character itself in the error message text.
> >
> > Also, are we sure these two are the only ones we want to catch in this
> > manner?
> 
> I added some more likely looking ones from
> https://en.wikipedia.org/wiki/Quotation_mark_glyphs.
> 
> > Finally, if this is accepted, let's have a test for this, and maybe
> > also some documentation, at least in NEWS.
> 
> 
> >From 438030f42698f347e38a9dea1561eedadd7e2edc Mon Sep 17 00:00:00 2001
> From: Noam Postavsky <npostavs <at> gmail.com>
> Date: Wed, 7 Jun 2017 19:59:09 -0400
> Subject: [PATCH v2] Signal error for symbol names with strange quotes
>  (Bug#2967)
> 
> * src/lread.c (read1): Signal an error when a symbol starts with a
> non-escaped quote-like character.
> * test/src/lread-tests.el (lread-tests--funny-quote-symbols): New
> test.
> * etc/NEWS: Announce change.

Thanks, this LGTM.  Please push.

Message #32 received at 2967 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 2967 <at> debbugs.gnu.org, rgm <at> gnu.org, monnier <at> iro.umontreal.ca, acm <at> muc.de
Subject: Re: bug#2967: smart quotes in .emacs
Date: Sat, 22 Jul 2017 10:41:22 -0400

tags 2967 fixed
close 2967 26.1
quit

Eli Zaretskii <eliz <at> gnu.org> writes:

> Thanks, this LGTM.  Please push.

Done.

[1: 8b18911a5c]: 2017-07-22 08:47:31 -0400
  Signal error for symbol names with strange quotes (Bug#2967)
  http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=8b18911a5c7c6c8a15b3cff12a4376ba68205e1c

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.