GNU bug report logs - #5757
23.1; String literal parse problem in ruby-mode

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 5757 in the body.
You can then email your comments to 5757 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Rhett Sutphin <r-sutphin <at> northwestern.edu>
To: bug-gnu-emacs <at> gnu.org
Subject: 23.1; String literal parse problem in ruby-mode
Date: Tue, 23 Mar 2010 11:21:10 -0500

ruby-mode will misunderstand a ruby double-quoted string literal which
contains a single quote and ends with a question mark.  It thinks that
the string literal is unterminated, which contaminates the syntax
highlighting for the remainder of the buffer.

Example ruby code which will demonstrate the problem:

["Is 'this' a string?"], [:something, :else]

If there's anything between the question mark and the terminating
double-quote, the string will be correctly interpreted.

Thanks,
Rhett Sutphin
rhett <at> detailedbalance.net


In GNU Emacs 23.1.1 (i386-apple-darwin9.8.0, NS apple-appkit-949.54)
 of 2009-08-16 on black.local
Windowing system distributor `Apple', version 10.3.949
configured using `configure  '--with-ns''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default-enable-multibyte-characters: t

Major mode: Ruby

Minor modes in effect:
  yas/global-mode: t
  yas/minor-mode: t
  auto-fill-function: do-auto-fill
  global-auto-revert-mode: t
  hi-lock-mode: t
  shell-dirtrack-mode: t
  hl-line-mode: t
  global-linum-mode: t
  linum-mode: t
  show-paren-mode: t
  nxhtml-global-minor-mode: t
  recentf-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<right> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <left> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <up> <up> <up>
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up>
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <left>
<left> <right> <right> ' <right> <left> " <left> <left>
<left> <left> <right> <left> <right> <right> <left>
<left> <right> <right> <right> <left> <left> <left>
<left> <right> <right> <right> <right> <right> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> <right>
<right> <right> <right> <right> <right> <left> <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <left> <left> <left>
<left> <left> <left> <left> <left> <left> <left> <left>
<left> <backspace> <left> <left> <backspace> ' <right>
<right> <right> <right> <right> <right> <right> <right>
<right> <right> <right> <right> <right> <right> , SPC
[ : a ] <left> <left> <left> <left> <left> <left> <left>
<left> <left> <left> <left> <left> <left> <left> <left>
<left> <left> <left> <left> <left> <backspace> <right>
<right> <right> <right> <right> ' <right> <right> <right>
<right> <right> <right> ' <backspace> <left> <left>
<left> <left> <left> <left> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> ' '
' ' ' ' ' ' ' ' ' ' ' <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> " <right> <right> <right> <right> <right>
<right> ? <backspace> <left> <left> <left> ? <down-mouse-1>
<mouse-1> M-x r e p o r t - e m <tab> <return>

Recent messages:
Auto-saving...done
Quit
Mark set
Undo! [6 times]
Quit
Auto-saving...done
Making completion list...
Quit
ruby-indent-to: invalid nest [2 times]
call-interactively: End of buffer

Message #8 received at 5757 <at> debbugs.gnu.org (full text, mbox):

From: Pål de Vibe <pauldevibe <at> yahoo.no>
To: 5757 <at> debbugs.gnu.org
Subject: The line which causes the bug is 1185
Date: Wed, 30 Mar 2011 23:08:33 +0100 (BST)

[Message part 1 (text/plain, inline)]

Line 1185:
          ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[?$]\\([#\"'`]\\)" 3 (1 . nil))

A workaround (with, to me, uknown consequences) is to remove the question mark 
from the line, like this:

          ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[$]\\([#\"'`]\\)" 3 (1 . nil))

[Message part 2 (text/html, inline)]

Message #11 received at 5757 <at> debbugs.gnu.org (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Nobuyoshi Nakada <nobu <at> ruby-lang.org>
Cc: Pål de Vibe <pauldevibe <at> yahoo.no>, 5757 <at> debbugs.gnu.org
Subject: Re: bug#5757: String literal parse problem in ruby-mode
Date: Sat, 02 Apr 2011 20:50:11 -0400

Hi Nakada-san,

Could you help review Pål de Vibe's proposed fix to the following
problem in Emacs ruby-mode?  Thanks.


Pål de Vibe <pauldevibe <at> yahoo.no> writes:

>> ruby-mode will misunderstand a ruby double-quoted string literal which
>> contains a single quote and ends with a question mark.  It thinks that
>> the string literal is unterminated, which contaminates the syntax
>> highlighting for the remainder of the buffer.
>>
>> Example ruby code which will demonstrate the problem:
>>
>> ["Is 'this' a string?"], [:something, :else]
>>
>> If there's anything between the question mark and the terminating
>> double-quote, the string will be correctly interpreted.
>
> Line 1185:
> ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[?$]\\([#\"'`]\\)" 3 (1 . nil))
>
> A workaround (with, to me, uknown consequences) is to remove the
> question mark from the line, like this:
>
> ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[$]\\([#\"'`]\\)" 3 (1 . nil))

Message #14 received at 5757 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Chong Yidong <cyd <at> stupidchicken.com>
Cc: Pål de Vibe <pauldevibe <at> yahoo.no>,
	Nobuyoshi Nakada <nobu <at> ruby-lang.org>, 5757 <at> debbugs.gnu.org
Subject: Re: bug#5757: String literal parse problem in ruby-mode
Date: Mon, 04 Apr 2011 09:53:45 -0400

> Could you help review Pål de Vibe's proposed fix to the following
> problem in Emacs ruby-mode?  Thanks.

His proposed fix is not correct: in ruby (like in Elisp) ?<char> is used
for chars (including ?' and ?") and $' and $" are also special vars, so
his fix just disables the special treatment of ?.
For Emacs-24, we use a different chunk of code which doesn't suffer from
this problem (mostly calling syntax-ppss to determine if we're inside
a string).


        Stefan


> Pål de Vibe <pauldevibe <at> yahoo.no> writes:

>>> ruby-mode will misunderstand a ruby double-quoted string literal which
>>> contains a single quote and ends with a question mark.  It thinks that
>>> the string literal is unterminated, which contaminates the syntax
>>> highlighting for the remainder of the buffer.
>>> 
>>> Example ruby code which will demonstrate the problem:
>>> 
>>> ["Is 'this' a string?"], [:something, :else]
>>> 
>>> If there's anything between the question mark and the terminating
>>> double-quote, the string will be correctly interpreted.
>> 
>> Line 1185:
>> ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[?$]\\([#\"'`]\\)" 3 (1 . nil))
>> 
>> A workaround (with, to me, uknown consequences) is to remove the
>> question mark from the line, like this:
>> 
>> ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[$]\\([#\"'`]\\)" 3 (1 . nil))

Message #17 received at 5757 <at> debbugs.gnu.org (full text, mbox):

From: Pål de Vibe <pauldevibe <at> yahoo.no>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Chong Yidong <cyd <at> stupidchicken.com>, Nobuyoshi Nakada <nobu <at> ruby-lang.org>,
	5757 <at> debbugs.gnu.org
Subject: Re: bug#5757: String literal parse problem in ruby-mode
Date: Mon, 4 Apr 2011 09:25:25 -0430

[Message part 1 (text/plain, inline)]

Hi. This wasn't really a proposed fix but more a hint to help debugging. I
haven't had time to really study the code. Thanks for the feedback.

On Mon, Apr 4, 2011 at 9:23 AM, Stefan Monnier <monnier <at> iro.umontreal.ca>wrote:

> > Could you help review Pål de Vibe's proposed fix to the following
> > problem in Emacs ruby-mode?  Thanks.
>
> His proposed fix is not correct: in ruby (like in Elisp) ?<char> is used
> for chars (including ?' and ?") and $' and $" are also special vars, so
> his fix just disables the special treatment of ?.
> For Emacs-24, we use a different chunk of code which doesn't suffer from
> this problem (mostly calling syntax-ppss to determine if we're inside
> a string).
>
>
>        Stefan
>
>
> > Pål de Vibe <pauldevibe <at> yahoo.no> writes:
>
> >>> ruby-mode will misunderstand a ruby double-quoted string literal which
> >>> contains a single quote and ends with a question mark.  It thinks that
> >>> the string literal is unterminated, which contaminates the syntax
> >>> highlighting for the remainder of the buffer.
> >>>
> >>> Example ruby code which will demonstrate the problem:
> >>>
> >>> ["Is 'this' a string?"], [:something, :else]
> >>>
> >>> If there's anything between the question mark and the terminating
> >>> double-quote, the string will be correctly interpreted.
> >>
> >> Line 1185:
> >> ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[?$]\\([#\"'`]\\)" 3 (1 . nil))
> >>
> >> A workaround (with, to me, uknown consequences) is to remove the
> >> question mark from the line, like this:
> >>
> >> ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[$]\\([#\"'`]\\)" 3 (1 . nil))
>
>
>
>

[Message part 2 (text/html, inline)]

Message #20 received at 5757 <at> debbugs.gnu.org (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Pål de Vibe <pauldevibe <at> yahoo.no>,
	Nobuyoshi Nakada <nobu <at> ruby-lang.org>, 5757 <at> debbugs.gnu.org
Subject: Re: bug#5757: String literal parse problem in ruby-mode
Date: Sat, 09 Apr 2011 16:34:10 -0400

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> His proposed fix is not correct: in ruby (like in Elisp) ?<char> is used
> for chars (including ?' and ?") and $' and $" are also special vars, so
> his fix just disables the special treatment of ?.
> For Emacs-24, we use a different chunk of code which doesn't suffer from
> this problem (mostly calling syntax-ppss to determine if we're inside
> a string).

Is this worth backporting to Emacs 23?  If not, let's close this bug.

Message #23 received at 5757 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Chong Yidong <cyd <at> stupidchicken.com>
Cc: Pål de Vibe <pauldevibe <at> yahoo.no>,
	Nobuyoshi Nakada <nobu <at> ruby-lang.org>, 5757 <at> debbugs.gnu.org
Subject: Re: bug#5757: String literal parse problem in ruby-mode
Date: Tue, 12 Apr 2011 00:16:38 -0300

>> His proposed fix is not correct: in ruby (like in Elisp) ?<char> is used
>> for chars (including ?' and ?") and $' and $" are also special vars, so
>> his fix just disables the special treatment of ?.
>> For Emacs-24, we use a different chunk of code which doesn't suffer from
>> this problem (mostly calling syntax-ppss to determine if we're inside
>> a string).
> Is this worth backporting to Emacs 23?  If not, let's close this bug.

I think it's more important to keep in touch with the upstream
maintainers.  E.g. I still haven't heard anything from them regarding
the changes I've made for syntax-propertize (which happen to fix the
OP's problem).  I'm worried we might not stay in sync.


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.