GNU bug report logs - #21343
24.5; parse-partial-sexp mistakes string for a comment

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21343 in the body.
You can then email your comments to 21343 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Oleh Krehel <ohwoeowho <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.5; parse-partial-sexp mistakes string for a comment
Date: Tue, 25 Aug 2015 15:11:18 +0200

To reproduce, paste this code into *scratch*, "|" is the point:

("|foo"
 ";; (bar)")

M-x indent-sexp will result in "    ;" being inserted after the sexp.

This is because `parse-partial-sexp' for the second line detects a
comment at one stage.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andreas Röhler <andreas.roehler <at> easy-emacs.de>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#21343: 24.5; parse-partial-sexp mistakes string for a comment
Date: Tue, 25 Aug 2015 19:45:43 +0200

Am 25.08.2015 um 15:11 schrieb Oleh Krehel:
> To reproduce, paste this code into *scratch*, "|" is the point:
>
> ("|foo"
>   ";; (bar)")
>
> M-x indent-sexp will result in "    ;" being inserted after the sexp.
>
> This is because `parse-partial-sexp' for the second line detects a
> comment at one stage.
>
>
>
>
>

Can't reproduce with GNU Emacs 25.0.50.1 (i686-pc-linux-gnu, GTK+ 
Version 2.24.23) of 2015-08-10

TAB indents nicely both lines, detects inside string when cursor at bar.

Message #11 received at 21343 <at> debbugs.gnu.org (full text, mbox):

From: Oleh Krehel <ohwoeowho <at> gmail.com>
To: Andreas Röhler <andreas.roehler <at> easy-emacs.de>
Cc: 21343 <at> debbugs.gnu.org
Subject: Re: bug#21343: 24.5; parse-partial-sexp mistakes string for a comment
Date: Wed, 26 Aug 2015 13:27:26 +0200

Andreas Röhler <andreas.roehler <at> easy-emacs.de> writes:

> Am 25.08.2015 um 15:11 schrieb Oleh Krehel:
>> To reproduce, paste this code into *scratch*, "|" is the point:
>>
>> ("|foo"
>>   ";; (bar)")
>>
>> M-x indent-sexp will result in "    ;" being inserted after the sexp.
>>
>> This is because `parse-partial-sexp' for the second line detects a
>> comment at one stage.

> Can't reproduce with GNU Emacs 25.0.50.1 (i686-pc-linux-gnu, GTK+
> Version 2.24.23) of 2015-08-10
>
> TAB indents nicely both lines, detects inside string when cursor at
> bar.

This is reproducible with "emacs -Q" on both 25 and 24.5.2.  You need to
not press "TAB", but "M-x indent-sexp" from the specified point
position.

`parse-partial-sexp' will detect comment on line 2 only when called in a
sequence that `indent-sexp' calls it, i.e. with the previous pps data
passed to the second pps call.

Message #14 received at 21343 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Röhler <andreas.roehler <at> easy-emacs.de>
To: Oleh Krehel <ohwoeowho <at> gmail.com>
Cc: 21343 <at> debbugs.gnu.org
Subject: Re: bug#21343: 24.5; parse-partial-sexp mistakes string for a comment
Date: Wed, 26 Aug 2015 16:52:41 +0200

Am 26.08.2015 um 13:27 schrieb Oleh Krehel:
> Andreas Röhler <andreas.roehler <at> easy-emacs.de> writes:
>
>> Am 25.08.2015 um 15:11 schrieb Oleh Krehel:
>>> To reproduce, paste this code into *scratch*, "|" is the point:
>>>
>>> ("|foo"
>>>    ";; (bar)")
>>>
>>> M-x indent-sexp will result in "    ;" being inserted after the sexp.
>>>
>>> This is because `parse-partial-sexp' for the second line detects a
>>> comment at one stage.
>> Can't reproduce with GNU Emacs 25.0.50.1 (i686-pc-linux-gnu, GTK+
>> Version 2.24.23) of 2015-08-10
>>
>> TAB indents nicely both lines, detects inside string when cursor at
>> bar.
> This is reproducible with "emacs -Q" on both 25 and 24.5.2.  You need to
> not press "TAB", but "M-x indent-sexp" from the specified point
> position.
>
> `parse-partial-sexp' will detect comment on line 2 only when called in a
> sequence that `indent-sexp' calls it, i.e. with the previous pps data
> passed to the second pps call.

Okay, see the bug now, thanks.

Message #17 received at 21343 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Oleh Krehel <ohwoeowho <at> gmail.com>
Cc: Andreas Röhler <andreas.roehler <at> easy-emacs.de>,
 21343 <at> debbugs.gnu.org
Subject: Re: bug#21343: 24.5; parse-partial-sexp mistakes string for a comment
Date: Sat, 02 Jul 2016 22:40:56 -0400

tags 21343 confirmed
quit

Oleh Krehel <ohwoeowho <at> gmail.com> writes:

>> Am 25.08.2015 um 15:11 schrieb Oleh Krehel:
>>> To reproduce, paste this code into *scratch*, "|" is the point:
>>>
>>> ("|foo"
>>>   ";; (bar)")
>>>
>>> M-x indent-sexp will result in "    ;" being inserted after the sexp.
>
> `parse-partial-sexp' will detect comment on line 2 only when called in a
> sequence that `indent-sexp' calls it, i.e. with the previous pps data
> passed to the second pps call.

In fact, the problem already occurs on the 1st line: calling
`parse-partial-sexp' from within a string gives wrong results, since it
doesn't look at the starting quote.  You can reproduce the bug with just

"|;"

A solution could be to use syntax-ppss to go to start of string before
parsing (though I wonder if syntax-ppss should be used more for the
parsing itself in that case, it seems like there's a lot of work going
that probably duplicates what's already being done):

diff --git c/lisp/emacs-lisp/lisp-mode.el i/lisp/emacs-lisp/lisp-mode.el
index a277d7a..8b5efa3 100644
--- c/lisp/emacs-lisp/lisp-mode.el
+++ i/lisp/emacs-lisp/lisp-mode.el
@@ -1070,6 +1070,10 @@ indent-sexp
 	;; Get error now if we don't have a complete sexp after point.
 	(save-excursion (forward-sexp 1)))
     (save-excursion
+      (let ((syn-start (nth 8 (syntax-ppss))))
+        (when syn-start
+          (unless endpos (setq starting-point syn-start))
+          (goto-char syn-start)))
       (setq outer-loop-done nil)
       (while (if endpos (< (point) endpos)
 	       (not outer-loop-done))

Message #22 received at 21343 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Oleh Krehel <ohwoeowho <at> gmail.com>
Cc: Andreas Röhler <andreas.roehler <at> easy-emacs.de>,
 21343 <at> debbugs.gnu.org
Subject: Re: bug#21343: 24.5; parse-partial-sexp mistakes string for a comment
Date: Sun, 05 Mar 2017 01:12:45 -0500

[Message part 1 (text/plain, inline)]

tags 21343 patch
quit

npostavs <at> users.sourceforge.net writes:

> A solution could be to use syntax-ppss to go to start of string before
> parsing (though I wonder if syntax-ppss should be used more for the
> parsing itself in that case, it seems like there's a lot of work going
> that probably duplicates what's already being done):

Actually, it turns out using syntax-ppss in the loop is slower, at least
in the case of calling `pp' on a big list (which is a pessimal case for
a cache based parser).

Here is a patch (the first one just simplifies the code).

[v1-0001-lisp-emacs-lisp-lisp-mode.el-indent-sexp-Simplify.patch (text/plain, attachment)]

[v1-0002-Fix-indent-sexp-when-called-from-inside-a-string-.patch (text/x-diff, inline)]

From b318e8c0bdd0f08b49f7f8c635bf886d967bc742 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs <at> gmail.com>
Date: Sun, 5 Mar 2017 00:53:58 -0500
Subject: [PATCH v1 2/2] Fix indent-sexp when called from inside a string
 (Bug#21343)

* lisp/emacs-lisp/lisp-mode.el (indent-sexp): Get initial syntax parse
state from `syntax-ppss'.
---
 lisp/emacs-lisp/lisp-mode.el | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el
index de0e66039a..ce61c69f3c 100644
--- a/lisp/emacs-lisp/lisp-mode.el
+++ b/lisp/emacs-lisp/lisp-mode.el
@@ -1062,10 +1062,14 @@ indent-sexp
          ;; since every line we indent is more deeply nested than point is.
          (starting-point (save-excursion (if endpos (beginning-of-defun))
                                          (point)))
-         (state nil)
-         (init-depth 0)
-         (next-depth 0)
-         (last-depth 0)
+         ;; Use `syntax-ppss' to get initial state so we don't get
+         ;; confused by starting inside a string.  We don't use
+         ;; `syntax-ppss' in the loop, because this is measurably
+         ;; slower when we're called on a long list.
+         (state (syntax-ppss))
+         (init-depth (car state))
+         (next-depth init-depth)
+         (last-depth init-depth)
          (last-syntax-point (point)))
     (unless endpos
       ;; Get error now if we don't have a complete sexp after point.
-- 
2.11.1

Message #27 received at 21343 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Oleh Krehel <ohwoeowho <at> gmail.com>
Cc: Andreas Röhler <andreas.roehler <at> easy-emacs.de>,
 21343 <at> debbugs.gnu.org
Subject: Re: bug#21343: 24.5; parse-partial-sexp mistakes string for a comment
Date: Sun, 05 Mar 2017 08:59:23 -0500

[Message part 1 (text/plain, inline)]

npostavs <at> users.sourceforge.net writes:

> Here is a patch (the first one just simplifies the code).

I had a mistake in the first patch, here's a corrected version.

[v2-0001-lisp-emacs-lisp-lisp-mode.el-indent-sexp-Simplify.patch (text/plain, attachment)]

Message #30 received at 21343 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Oleh Krehel <ohwoeowho <at> gmail.com>
Cc: Andreas Röhler <andreas.roehler <at> easy-emacs.de>,
 21343 <at> debbugs.gnu.org
Subject: Re: bug#21343: 24.5; parse-partial-sexp mistakes string for a comment
Date: Sun, 12 Mar 2017 20:15:47 -0400

tags 21343 fixed
close 21343 26.1
quit

npostavs <at> users.sourceforge.net writes:

> npostavs <at> users.sourceforge.net writes:
>
>> Here is a patch (the first one just simplifies the code).
>
> I had a mistake in the first patch, here's a corrected version.

Pushed to master [1: cf670b49a7].

1: 2017-03-12 20:08:32 -0400 cf670b49a7704d63575863f832426d32bf6a8c3c
  Fix indent-sexp when called from inside a string (Bug#21343)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.