GNU bug report logs - #69168
[BUG] 30.0.50; pp-emacs-lisp-code produces invalid elisp for backquoted forms

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 69168 in the body.
You can then email your comments to 69168 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: No Wayman <iarchivedmywholelife <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [BUG] 30.0.50; pp-emacs-lisp-code produces invalid elisp for
 backquoted forms
Date: Thu, 15 Feb 2024 14:19:19 -0500

Observe the output of the following two expressions:


(eval (read (with-temp-buffer
             (pp-emacs-lisp-code '`(1+ ,(1+ 1)))
             (buffer-string)))
     t)

;; => eval: Invalid function: `(1+ (,(1+ 1)))

(eval
(read (with-temp-buffer
        (insert (pp-to-string '`(1+ ,(1+ 1))))
        (buffer-string)))
t)

;; => (1+ 2)


The form returned by pp-emacs-lisp-code is:

(` (1+ (, (1+ 1))))

The backqoute and comma are not escaped, resulting in the error.
It looks like the logic in pp--insert-lisp would need to be 
extended to account for backquoting constructs.

In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
3.24.41, cairo version 1.18.0) of 2024-02-03 built on laptop
Repository revision: 4ebded3f5ee8617ac6b1debaa01706cd78206f39
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 
11.0.12101011
System Description: Arch Linux

Message #8 received at 69168 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: No Wayman <iarchivedmywholelife <at> gmail.com>
Cc: 69168 <at> debbugs.gnu.org
Subject: Re: bug#69168: [BUG] 30.0.50; pp-emacs-lisp-code produces invalid
 elisp for backquoted forms
Date: Fri, 16 Feb 2024 03:11:55 +0100

No Wayman <iarchivedmywholelife <at> gmail.com> writes:

> The backqoute and comma are not escaped, resulting in the error.
> It looks like the logic in pp--insert-lisp would need to be extended
> to account for backquoting constructs.

Indeed - the function formats the expression as a list.

The second problem is that the recursive calls of `pp--insert-lisp'
print symbols using `princ' (i.e., without quoting), so even the list
representation is wrong.

Michael.

Message #11 received at 69168 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: No Wayman <iarchivedmywholelife <at> gmail.com>
Cc: 69168 <at> debbugs.gnu.org
Subject: Re: bug#69168: [BUG] 30.0.50; pp-emacs-lisp-code produces invalid
 elisp for backquoted forms
Date: Sun, 25 Feb 2024 06:00:38 +0100

[Message part 1 (text/plain, inline)]

Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> The second problem is that the recursive calls of `pp--insert-lisp'
> print symbols using `princ' (i.e., without quoting), so even the list
> representation is wrong.

I think we should print any symbol using `prin1', not `princ' - printing
symbols that need quoting without quoting is not useful:

[0001-Fix-pp-emacs-lisp-code-printing-of-symbols.patch (text/x-diff, inline)]

From f493708a456608f59b29c3f2308ecc51177667ec Mon Sep 17 00:00:00 2001
From: Michael Heerdegen <michael_heerdegen <at> web.de>
Date: Sun, 18 Feb 2024 02:48:15 +0100
Subject: [PATCH] Fix pp-emacs-lisp-code printing of symbols

* lisp/emacs-lisp/pp.el (pp--insert-lisp): Print symbols
readably (bug#69168).
---
 lisp/emacs-lisp/pp.el | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lisp/emacs-lisp/pp.el b/lisp/emacs-lisp/pp.el
index 1d722051406..944dd750839 100644
--- a/lisp/emacs-lisp/pp.el
+++ b/lisp/emacs-lisp/pp.el
@@ -458,6 +458,8 @@ pp--insert-lisp
     (string
      (let ((print-escape-newlines t))
        (prin1 sexp (current-buffer))))
+    (symbol
+     (prin1 sexp (current-buffer)))
     (otherwise (princ sexp (current-buffer)))))

 (defun pp--format-vector (sexp)
--
2.39.2

[Message part 3 (text/plain, inline)]

Michael.

Message #14 received at 69168 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: iarchivedmywholelife <at> gmail.com, 69168 <at> debbugs.gnu.org
Subject: Re: bug#69168: [BUG] 30.0.50;
 pp-emacs-lisp-code produces invalid elisp for backquoted forms
Date: Sun, 25 Feb 2024 08:39:34 +0200

> Cc: 69168 <at> debbugs.gnu.org
> Date: Sun, 25 Feb 2024 06:00:38 +0100
> From:  Michael Heerdegen via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> 
> Michael Heerdegen <michael_heerdegen <at> web.de> writes:
> 
> > The second problem is that the recursive calls of `pp--insert-lisp'
> > print symbols using `princ' (i.e., without quoting), so even the list
> > representation is wrong.
> 
> I think we should print any symbol using `prin1', not `princ' - printing
> symbols that need quoting without quoting is not useful:
> 
> 
> From f493708a456608f59b29c3f2308ecc51177667ec Mon Sep 17 00:00:00 2001
> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Date: Sun, 18 Feb 2024 02:48:15 +0100
> Subject: [PATCH] Fix pp-emacs-lisp-code printing of symbols
> 
> * lisp/emacs-lisp/pp.el (pp--insert-lisp): Print symbols
> readably (bug#69168).
> ---
>  lisp/emacs-lisp/pp.el | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/lisp/emacs-lisp/pp.el b/lisp/emacs-lisp/pp.el
> index 1d722051406..944dd750839 100644
> --- a/lisp/emacs-lisp/pp.el
> +++ b/lisp/emacs-lisp/pp.el
> @@ -458,6 +458,8 @@ pp--insert-lisp
>      (string
>       (let ((print-escape-newlines t))
>         (prin1 sexp (current-buffer))))
> +    (symbol
> +     (prin1 sexp (current-buffer)))
>      (otherwise (princ sexp (current-buffer)))))
> 
>  (defun pp--format-vector (sexp)
> --
> 2.39.2

Thanks.  Stefan, any comments?

Message #17 received at 69168 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 69168 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 iarchivedmywholelife <at> gmail.com
Subject: Re: bug#69168: [BUG] 30.0.50; pp-emacs-lisp-code produces invalid
 elisp for backquoted forms
Date: Sun, 25 Feb 2024 09:02:05 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

> Thanks.  Stefan, any comments?

One from me: I'm working on a second patch for pp that will make
printing of backquote forms using the nicer `X syntax.  But this small
fix is independent of such improvements.

Michael.

Message #20 received at 69168 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>,
 iarchivedmywholelife <at> gmail.com, 69168 <at> debbugs.gnu.org
Subject: Re: bug#69168: [BUG] 30.0.50; pp-emacs-lisp-code produces invalid
 elisp for backquoted forms
Date: Sun, 25 Feb 2024 10:08:45 -0500

> Thanks.  Stefan, any comments?

Just that I fully agree with:

>> I think we should print any symbol using `prin1', not `princ' - printing
>> symbols that need quoting without quoting is not useful:

For that same reason, I've been changing several %s into %S in `format`
and `message` arguments.


        Stefan

Message #23 received at 69168 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 69168 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>,
 iarchivedmywholelife <at> gmail.com
Subject: Re: bug#69168: [BUG] 30.0.50; pp-emacs-lisp-code produces invalid
 elisp for backquoted forms
Date: Thu, 29 Feb 2024 03:30:12 +0100

[Message part 1 (text/plain, inline)]

Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> One from me: I'm working on a second patch for pp that will make
> printing of backquote forms using the nicer `X syntax.

Here is the additional patch.  No Wayman, maybe you can try if it gives
good results?

This change seems quite straightforward, and I tried to avoid to make
the code less efficient.  For me, it seems to give good results.
Please (Eli), if names and comments are bad English, please feel free to
suggest a better wording.

[0001-WIP-Improve-pp-emacs-lisp-code-backquote-form-printi.patch (text/x-diff, inline)]

From a6ad2e056b7ac136ac766b1fcdb5f59f2f505e15 Mon Sep 17 00:00:00 2001
From: Michael Heerdegen <michael_heerdegen <at> web.de>
Date: Sun, 18 Feb 2024 01:55:54 +0100
Subject: [PATCH] WIP: Improve pp-emacs-lisp-code backquote form printing

* lisp/emacs-lisp/pp.el (pp--quoted-or-unquoted-form-p): New helper
function.
(pp--insert-lisp): Take care of quoted, backquoted and
unquoted expressions; print using an recursive call.
(pp--format-list): Exclude more cases from printing as a function call
by default.  Print lists whose second-last element is an (un)quoting
symbol using dotted list syntax; e.g. (a b . ,c) instead of (a b \, c).
---
 lisp/emacs-lisp/pp.el | 56 ++++++++++++++++++++++++++++++-------------
 1 file changed, 40 insertions(+), 16 deletions(-)

diff --git a/lisp/emacs-lisp/pp.el b/lisp/emacs-lisp/pp.el
index 944dd750839..045cc171eaa 100644
--- a/lisp/emacs-lisp/pp.el
+++ b/lisp/emacs-lisp/pp.el
@@ -430,23 +430,33 @@ pp-emacs-lisp-code
           (replace-match ""))
         (insert-into-buffer obuf)))))

+(defvar pp--quoting-syntaxes
+  `((quote    . "'")
+    (function . "#'")
+    (,backquote-backquote-symbol . "`")
+    (,backquote-unquote-symbol   . ",")
+    (,backquote-splice-symbol    . ",@")))
+
+(defun pp--quoted-or-unquoted-form-p (cons)
+  ;; Return non-nil when CONS has one of the forms 'X, `X, ,X or ,@X
+  (let ((head (car cons)))
+    (and (symbolp head)
+         (assq head pp--quoting-syntaxes)
+         (let ((rest (cdr cons)))
+           (and (consp rest) (null (cdr rest)))))))
+
 (defun pp--insert-lisp (sexp)
   (cl-case (type-of sexp)
     (vector (pp--format-vector sexp))
     (cons (cond
            ((consp (cdr sexp))
-            (if (and (length= sexp 2)
-                     (memq (car sexp) '(quote function)))
-                (cond
-                 ((symbolp (cadr sexp))
-                  (let ((print-quoted t))
-                    (prin1 sexp (current-buffer))))
-                 ((consp (cadr sexp))
-                  (insert (if (eq (car sexp) 'quote)
-                              "'" "#'"))
-                  (pp--format-list (cadr sexp)
-                                   (set-marker (make-marker) (1- (point))))))
-              (pp--format-list sexp)))
+            (let ((head (car sexp)))
+              (if-let (((null (cddr sexp)))
+                       (syntax-entry (assq head pp--quoting-syntaxes)))
+                  (progn
+                    (insert (cdr syntax-entry))
+                    (pp--insert-lisp (cadr sexp)))
+                (pp--format-list sexp))))
            (t
             (prin1 sexp (current-buffer)))))
     ;; Print some of the smaller integers as characters, perhaps?
@@ -470,15 +480,29 @@ pp--format-vector
   (insert "]"))

 (defun pp--format-list (sexp &optional start)
-  (if (and (symbolp (car sexp))
-           (not pp--inhibit-function-formatting)
-           (not (keywordp (car sexp))))
+  (if (not (let ((head (car sexp)))
+             (or pp--inhibit-function-formatting
+                 (not (symbolp head))
+                 (keywordp head)
+                 (let ((l sexp))
+                   (catch 'not-funcall
+                     (while l
+                       (when (or
+                              (not (consp l)) ; SEXP is a dotted list
+                              ;; Is SEXP is of a form like (ELT... . ,X) ?
+                              (pp--quoted-or-unquoted-form-p l))
+                         (throw 'not-funcall t))
+                       (setq l (cdr l)))
+                     nil)))))
       (pp--format-function sexp)
     (insert "(")
     (pp--insert start (pop sexp))
     (while sexp
       (if (consp sexp)
-          (pp--insert " " (pop sexp))
+          (if (not (pp--quoted-or-unquoted-form-p sexp))
+              (pp--insert " " (pop sexp))
+            (pp--insert " . " sexp)
+            (setq sexp nil))
         (pp--insert " . " sexp)
         (setq sexp nil)))
     (insert ")")))
--
2.39.2

[Message part 3 (text/plain, inline)]

Michael.

Message #26 received at 69168 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: 69168 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca,
 iarchivedmywholelife <at> gmail.com
Subject: Re: bug#69168: [BUG] 30.0.50; pp-emacs-lisp-code produces invalid
 elisp for backquoted forms
Date: Sat, 09 Mar 2024 10:49:15 +0200

> From: Michael Heerdegen <michael_heerdegen <at> web.de>
> Cc: Stefan Monnier <monnier <at> iro.umontreal.ca>,
>   iarchivedmywholelife <at> gmail.com,  69168 <at> debbugs.gnu.org
> Date: Thu, 29 Feb 2024 03:30:12 +0100
> 
> Michael Heerdegen <michael_heerdegen <at> web.de> writes:
> 
> > One from me: I'm working on a second patch for pp that will make
> > printing of backquote forms using the nicer `X syntax.
> 
> Here is the additional patch.  No Wayman, maybe you can try if it gives
> good results?
> 
> This change seems quite straightforward, and I tried to avoid to make
> the code less efficient.  For me, it seems to give good results.
> Please (Eli), if names and comments are bad English, please feel free to
> suggest a better wording.

LGTM, with one minor nit:

> +                              (not (consp l)) ; SEXP is a dotted list
> +                              ;; Is SEXP is of a form like (ELT... . ,X) ?

The "Is SEXP is" part seems to have one "is" too many...

I suggest

  Does SEXP have a form like (ELT... . ,X) ?

Thanks.

Message #29 received at 69168 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: "iarchivedmywholelife <at> gmail.com" <iarchivedmywholelife <at> gmail.com>,
 "69168 <at> debbugs.gnu.org" <69168 <at> debbugs.gnu.org>,
 "monnier <at> iro.umontreal.ca" <monnier <at> iro.umontreal.ca>
Subject: RE: [External] : bug#69168: [BUG] 30.0.50; pp-emacs-lisp-code
 produces invalid elisp for backquoted forms
Date: Sat, 9 Mar 2024 15:34:57 +0000

> > (not (consp l))

Aka (atom l).

Message #34 received at 69168-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: iarchivedmywholelife <at> gmail.com, monnier <at> iro.umontreal.ca,
 Drew Adams <drew.adams <at> oracle.com>, 69168-done <at> debbugs.gnu.org
Subject: Re: bug#69168: [BUG] 30.0.50; pp-emacs-lisp-code produces invalid
 elisp for backquoted forms
Date: Mon, 11 Mar 2024 05:48:30 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

> I suggest
>
>   Does SEXP have a form like (ELT... . ,X) ?

Done, thanks.  Along with Drew's correction.

I have pushed both commits to master and close this report.


Michael.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.