GNU bug report logs - #76132
Clojure-style auto-gensyms for macros

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 76132 in the body.
You can then email your comments to 76132 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: bug-gnu-emacs <at> gnu.org
Subject: Clojure-style auto-gensyms for macros
Date: Fri, 07 Feb 2025 22:12:14 +0100

[Message part 1 (text/plain, inline)]

Tags: patch

Hi all,

in a recent bug report the topic macro hygiene came up, i.e., that a
macro which introduces local bindings in its expansion better uses
uninterned symbols for those in order not to clash with code passed as
macro arguments which are spliced into the expansion.

Clojure has a very convenient feature to make that easy.  While you can
write such macros traditionally like

(defmacro foo [x y]
  (let [xv (gensym "x")
        yv (gensym "y")]
    `(let [,xv ,x
           ,yv ,y]
       (do-stuff (* ,xv ,xv) (* ,yv ,yv)))))

you can also write much more concise and convenient

(defmacro foo [x y]
  `(let [xv# ,x
         yv# ,y]
     (do-stuff (* xv# xv#) (* yv# yv#))))

where each symbol ending in # will be replaced by a unique gensymed
symbol (per name).  The expansion of the two macros is the same.

Would there be interest in adding something like that to Elisp?

I've attached a proof-of-concept implementation where the feature is
provided by a macro with-uninterned-symbols [1] which you simply wrap
around your backquoted form.

[1] In Clojure, it's a feature of the reader triggered by backquote.

In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.48, cairo version 1.18.2) of 2025-02-07 built on thinkpad-t440p
Repository revision: 1751739152149608d28853782ce53b0b9a749bb2
Repository branch: master
System Description: Arch Linux

Configured using:
 'configure --without-native-compilation --with-modules --with-pgtk'

[clojure-macro.el (text/patch, attachment)]

Message #8 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Tassilo Horn <tsdh <at> gnu.org>, "76132 <at> debbugs.gnu.org"
 <76132 <at> debbugs.gnu.org>
Subject: RE: [External] : bug#76132: Clojure-style auto-gensyms for macros
Date: Fri, 7 Feb 2025 22:25:10 +0000

> Clojure has a very convenient feature to make that easy.  While you can
> write such macros traditionally like
> 
> (defmacro foo [x y]
>   (let [xv (gensym "x")
>         yv (gensym "y")]
>     `(let [,xv ,x
>            ,yv ,y]
>        (do-stuff (* ,xv ,xv) (* ,yv ,yv)))))
> 
> you can also write much more concise and convenient
> > you can also write much more concise and convenient
> 
> (defmacro foo [x y]
>   `(let [xv# ,x
>          yv# ,y]
>      (do-stuff (* xv# xv#) (* yv# yv#))))
> 
> where each symbol ending in # will be replaced
> by a unique gensymed symbol (per name).
> Would there be interest in adding something
> like that to Elisp?

So you could no longer let-bind a variable
whose name ends in `#', to get a normal let
binding?  (Admittedly, you need to write that
as `\#' in the source code.)

Doesn't sound like an improvement, to me.
A priori, I'm not in favor of limiting the
names you can use for variables.

That would be especially pernicious with let
bindings of dynamic ("special") variables.
If it were limited to lexical variables it
wouldn't be so bad.  We should be able to
bind dynamic vars whose names end with `#'.
(Again though, admittedly the `#' chars need
to be escaped in source code: `\#'.)

The cliché of handling this kind of thing in
macros with gensym is pretty standard (it may
even be the main use of gensym).  Yes, it can
be error prone, but it's not really harder to
use gensym than it is to add `#' to var names.

And the body is IMO clearer without the added
`#'s and subtracted commas.

There might be some other way to provide such
a shortcut, without removing the _general_
possibility of naming let variables with `#'
at the end.  A new variety (yet another?) of
`let' perhaps, that does what you suggest.

I'll note too that Common Lisp doesn't bother
with such things either.  And people have been
writing Lisp macros with Common Lisp and its
ancestors for many, many moon.

Message #11 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Thierry Volpiatto <thievol <at> posteo.net>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sat, 08 Feb 2025 03:27:57 +0000

Tassilo Horn <tsdh <at> gnu.org> writes:

> Tags: patch
>
> Hi all,
>
> in a recent bug report the topic macro hygiene came up, i.e., that a
> macro which introduces local bindings in its expansion better uses
> uninterned symbols for those in order not to clash with code passed as
> macro arguments which are spliced into the expansion.
>
> Clojure has a very convenient feature to make that easy.  While you can
> write such macros traditionally like
>
> (defmacro foo [x y]
>   (let [xv (gensym "x")
>         yv (gensym "y")]
>     `(let [,xv ,x
>            ,yv ,y]
>        (do-stuff (* ,xv ,xv) (* ,yv ,yv)))))
>
> you can also write much more concise and convenient
>
> (defmacro foo [x y]
>   `(let [xv# ,x
>          yv# ,y]
>      (do-stuff (* xv# xv#) (* yv# yv#))))

Don't we have cl-with-gensyms which is very convenient as well?
In Helm we have helm-with-gensyms which is the same (Thanks Michael ;-))

> where each symbol ending in # will be replaced by a unique gensymed
> symbol (per name).  The expansion of the two macros is the same.
>
> Would there be interest in adding something like that to Elisp?
>
> I've attached a proof-of-concept implementation where the feature is
> provided by a macro with-uninterned-symbols [1] which you simply wrap
> around your backquoted form.
>
> [1] In Clojure, it's a feature of the reader triggered by backquote.
>
> In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
>  3.24.48, cairo version 1.18.2) of 2025-02-07 built on thinkpad-t440p
> Repository revision: 1751739152149608d28853782ce53b0b9a749bb2
> Repository branch: master
> System Description: Arch Linux
>
> Configured using:
>  'configure --without-native-compilation --with-modules --with-pgtk'
>
>

-- 
Thierry

Message #14 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: "76132 <at> debbugs.gnu.org" <76132 <at> debbugs.gnu.org>
Subject: Re: [External] : bug#76132: Clojure-style auto-gensyms for macros
Date: Sat, 08 Feb 2025 08:07:46 +0100

Drew Adams <drew.adams <at> oracle.com> writes:

>> (defmacro foo [x y]
>>   `(let [xv# ,x
>>          yv# ,y]
>>      (do-stuff (* xv# xv#) (* yv# yv#))))
>> 
>> where each symbol ending in # will be replaced
>> by a unique gensymed symbol (per name).
>> Would there be interest in adding something
>> like that to Elisp?
>
> So you could no longer let-bind a variable
> whose name ends in `#', to get a normal let
> binding?  (Admittedly, you need to write that
> as `\#' in the source code.)

No, have you looked at the file I attached?  You will see the feature is
implemented by a macro itself, so if you don't like it, don't use it.

> Doesn't sound like an improvement, to me.
> A priori, I'm not in favor of limiting the
> names you can use for variables.
>
> That would be especially pernicious with let
> bindings of dynamic ("special") variables.

How many special variables are there ending in # (or actually $ which I
use in my example)?

Bye,
Tassilo

Message #17 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Thierry Volpiatto <thievol <at> posteo.net>
Cc: 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sat, 08 Feb 2025 08:15:55 +0100

Thierry Volpiatto <thievol <at> posteo.net> writes:

Hi Thierry,

>> you can also write much more concise and convenient
>>
>> (defmacro foo [x y]
>>   `(let [xv# ,x
>>          yv# ,y]
>>      (do-stuff (* xv# xv#) (* yv# yv#))))
>
> Don't we have cl-with-gensyms which is very convenient as well?

Ah, I didn't know.  Well, cl-with-gensyms is basically just a let which
binds the given names to new gensyms.  Then you have to use them as you
did before, e.g., splice them in the expansion `(... ,v1 ,v2).  My
variant infers and replaces the symbols itself by a naming convention
("ending with $" in my example file).

Bye,
Tassilo

Message #20 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Tassilo Horn <tsdh <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sat, 08 Feb 2025 09:54:59 +0200

> From: Tassilo Horn <tsdh <at> gnu.org>
> Date: Fri, 07 Feb 2025 22:12:14 +0100
> 
> in a recent bug report the topic macro hygiene came up, i.e., that a
> macro which introduces local bindings in its expansion better uses
> uninterned symbols for those in order not to clash with code passed as
> macro arguments which are spliced into the expansion.
> 
> Clojure has a very convenient feature to make that easy.  While you can
> write such macros traditionally like
> 
> (defmacro foo [x y]
>   (let [xv (gensym "x")
>         yv (gensym "y")]
>     `(let [,xv ,x
>            ,yv ,y]
>        (do-stuff (* ,xv ,xv) (* ,yv ,yv)))))
> 
> you can also write much more concise and convenient
> 
> (defmacro foo [x y]
>   `(let [xv# ,x
>          yv# ,y]
>      (do-stuff (* xv# xv#) (* yv# yv#))))
> 
> where each symbol ending in # will be replaced by a unique gensymed
> symbol (per name).  The expansion of the two macros is the same.
> 
> Would there be interest in adding something like that to Elisp?

I'm very hesitant to extend the Emacs Lisp language with such
features, when this can be had for a price of a simple function call.
We have enough magic names and punctuation characters already, and
they get in the way of code readability.

Stefan, WDYT?

Message #23 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sat, 08 Feb 2025 09:29:04 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Would there be interest in adding something like that to Elisp?
>
> I'm very hesitant to extend the Emacs Lisp language with such
> features, when this can be had for a price of a simple function call.
> We have enough magic names and punctuation characters already, and
> they get in the way of code readability.

In my opinion, readability is the key point of the feature.  With the
regular "declare the locals you need to introduce beforehand and then
splice them into the expansion" approach, the distinction between locals
and spliced-in macro args gets lost.  With the suggested
with-uninterned-symbols macro, it's clear that foo$ is a local defined
in the expansion while ,foo is something from "the outside".

I'm not booked on the $-suffix, though.  It's just easy to type, stands
out a bit and usually isn't used in the wild.

Bye,
Tassilo

Message #26 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 76132 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sat, 8 Feb 2025 08:02:18 -0500

[Message part 1 (text/plain, inline)]

I've seen many unhygienic macros (found the hard way) and written some
myself out of laziness. It's a nice idea to encourage safer lazy macro
writing.

I prefer a hat ^ prefix as it is easier to read, rather than a dollar $
suffix which seems muddled to my eye.

I found no evidence of symbols with a ^ prefix in the Emacs code base or in
the elpa packages I use so would risk less conflict than $ which I have
seen around. I'd highlight these hat-prefixed symbols in some nice new
font-lock face.

(defmacro sm/test (x y)
  (with-uninterned-symbols
   `(let ((^foo ,x)
          (^bar ,y))
      (list :args `(,^foo . ,^bar)
            :add   (+ ^foo ^bar)
            :sub   (- ^foo ^bar)
            :mul   (* ^foo ^bar)
            :div   (/ ^foo ^bar)))))

-Stephane

On Sat, Feb 8, 2025 at 3:30 AM Tassilo Horn <tsdh <at> gnu.org> wrote:

> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> Would there be interest in adding something like that to Elisp?
> >
> > I'm very hesitant to extend the Emacs Lisp language with such
> > features, when this can be had for a price of a simple function call.
> > We have enough magic names and punctuation characters already, and
> > they get in the way of code readability.
>
> In my opinion, readability is the key point of the feature.  With the
> regular "declare the locals you need to introduce beforehand and then
> splice them into the expansion" approach, the distinction between locals
> and spliced-in macro args gets lost.  With the suggested
> with-uninterned-symbols macro, it's clear that foo$ is a local defined
> in the expansion while ,foo is something from "the outside".
>
> I'm not booked on the $-suffix, though.  It's just easy to type, stands
> out a bit and usually isn't used in the wild.
>
> Bye,
> Tassilo
>
>
>
>

[Message part 2 (text/html, inline)]

Message #29 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sat, 08 Feb 2025 10:29:07 -0500

> Clojure has a very convenient feature to make that easy.  While you can
> write such macros traditionally like
>
> (defmacro foo [x y]
>   (let [xv (gensym "x")
>         yv (gensym "y")]
>     `(let [,xv ,x
>            ,yv ,y]
>        (do-stuff (* ,xv ,xv) (* ,yv ,yv)))))
>
> you can also write much more concise and convenient
>
> (defmacro foo [x y]
>   `(let [xv# ,x
>          yv# ,y]
>      (do-stuff (* xv# xv#) (* yv# yv#))))

IIUC the two versions above aren't quite equivalent.
The Clojure version seems to behave similarly to the macro you propose,
which behaves more like:

    (let [xv (gensym "x")
          yv (gensym "y")]
      (defmacro foo [x y]
        `(let [,xv ,x
               ,yv ,y]
           (do-stuff (* ,xv ,xv) (* ,yv ,yv)))))

in the sense that the same uninterned symbols will be used for every
expansion.  This is usually fine, but can still result in name capture
in some weird corner cases (tho I must admit I can't even remember what
those corner cases are).

> I'm very hesitant to extend the Emacs Lisp language with such
> features, when this can be had for a price of a simple function call.
> We have enough magic names and punctuation characters already, and
> they get in the way of code readability.

To the extent that it's all implemented within a normal macro
(i.e. doesn't touch things like the reader or the `macroexp.el` code),
I'm not too bothered.  It could even live in a separate package if we
don't want it in core.


        Stefan

Message #32 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sat, 08 Feb 2025 17:12:00 +0100

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> Clojure has a very convenient feature to make that easy.  While you
>> can write such macros traditionally like
>>
>> (defmacro foo [x y]
>>   (let [xv (gensym "x")
>>         yv (gensym "y")]
>>     `(let [,xv ,x
>>            ,yv ,y]
>>        (do-stuff (* ,xv ,xv) (* ,yv ,yv)))))
>>
>> you can also write much more concise and convenient
>>
>> (defmacro foo [x y]
>>   `(let [xv# ,x
>>          yv# ,y]
>>      (do-stuff (* xv# xv#) (* yv# yv#))))
>
> IIUC the two versions above aren't quite equivalent.
> The Clojure version seems to behave similarly to the macro you
> propose, which behaves more like:
>
>     (let [xv (gensym "x")
>           yv (gensym "y")]
>       (defmacro foo [x y]
>         `(let [,xv ,x
>                ,yv ,y]
>            (do-stuff (* ,xv ,xv) (* ,yv ,yv)))))
>
> in the sense that the same uninterned symbols will be used for every
> expansion.  This is usually fine, but can still result in name capture
> in some weird corner cases (tho I must admit I can't even remember
> what those corner cases are).

Why? I would expect that it's one set of uninterned symbols per
expansion.  I first tried implementing it with an alist instead of a
hash-table and didn't think about the fact that my alist function arg is
just a pointer to the head of the list so that when popping back from a
recursive call where I pushed onto the list (that is, to its head), the
new entry got lost.  With that failing approach, every foo$ occurrence
was substituted with a separate (make-symbol "foo"), so although the
exansion looked correct, every #:foo in there was distinct from all
other #:foo-s.  Doesn't that contradict your statement?

Oh, and obviously you made me curious about those weird corner cases.
Please elaborate! ;-)

Bye,
Tassilo

Message #35 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Ship Mints <shipmints <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 76132 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sun, 09 Feb 2025 09:32:54 +0100

Ship Mints <shipmints <at> gmail.com> writes:

> I prefer a hat ^ prefix as it is easier to read, rather than a dollar
> $ suffix which seems muddled to my eye.

I don't mind what character to use (the macro could even take it as an
argument) but I'd prefer using a suffix simply because there's already
the special prefix _ indicating to the byte-compiler that this variable
is not used intentionally.  You might want to use that capability when
your macro expands to a function which needs to have a certain signature
but your implementation doesn't use all arguments, for example.

Bye,
Tassilo

Message #38 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Ship Mints <shipmints <at> gmail.com>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 76132 <at> debbugs.gnu.org,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sun, 9 Feb 2025 04:22:05 -0500

[Message part 1 (text/plain, inline)]

Good point on _. A suffix works. ^ suggests the mnemonic "it comes from
above."

(defmacro sm/test (x y)
  (with-uninterned-symbols
   `(let ((foo^ ,x)
          (bar^ ,y))
      (list :args `(,foo^ . ,bar^)
            :add   (+ foo^ bar^)
            :sub   (- foo^ bar^)
            :mul   (* foo^ bar^)
            :div   (/ foo^ bar^)))))

On Sun, Feb 9, 2025 at 3:33 AM Tassilo Horn <tsdh <at> gnu.org> wrote:

> Ship Mints <shipmints <at> gmail.com> writes:
>
> > I prefer a hat ^ prefix as it is easier to read, rather than a dollar
> > $ suffix which seems muddled to my eye.
>
> I don't mind what character to use (the macro could even take it as an
> argument) but I'd prefer using a suffix simply because there's already
> the special prefix _ indicating to the byte-compiler that this variable
> is not used intentionally.  You might want to use that capability when
> your macro expands to a function which needs to have a certain signature
> but your implementation doesn't use all arguments, for example.
>
> Bye,
> Tassilo
>

[Message part 2 (text/html, inline)]

Message #41 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Sean Whitton <spwhitton <at> spwhitton.name>
To: Eli Zaretskii <eliz <at> gnu.org>, Tassilo Horn <tsdh <at> gnu.org>,  Stefan
 Monnier <monnier <at> iro.umontreal.ca>,  76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sun, 09 Feb 2025 09:51:38 +0000

Hello,

On Sat 08 Feb 2025 at 09:54am +02, Eli Zaretskii wrote:

> I'm very hesitant to extend the Emacs Lisp language with such
> features, when this can be had for a price of a simple function call.
> We have enough magic names and punctuation characters already, and
> they get in the way of code readability.

Yeah.  I think we should not add this for the similar reasons to how we
don't have reader macros: Lisp is an almost syntax-free language, and
there are advantages to that.

Any step away from that cuts deeply into the advantages, and the
corresonding benefits of the new syntax are often very minor.

cl-with-gensyms is only slightly more typing, and is more trad Lisp
style.

-- 
Sean Whitton

Message #44 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Sean Whitton <spwhitton <at> spwhitton.name>, Eli Zaretskii <eliz <at> gnu.org>,
 Tassilo Horn <tsdh <at> gnu.org>, 
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sun, 9 Feb 2025 07:51:39 -0500

Sean Whitton <spwhitton <at> spwhitton.name> writes:

> On Sat 08 Feb 2025 at 09:54am +02, Eli Zaretskii wrote:
>
>> I'm very hesitant to extend the Emacs Lisp language with such
>> features, when this can be had for a price of a simple function call.
>> We have enough magic names and punctuation characters already, and
>> they get in the way of code readability.
>
> Yeah.  I think we should not add this for the similar reasons to how we
> don't have reader macros: Lisp is an almost syntax-free language, and
> there are advantages to that.
>
> Any step away from that cuts deeply into the advantages, and the
> corresonding benefits of the new syntax are often very minor.
>
> cl-with-gensyms is only slightly more typing, and is more trad Lisp
> style.

I'd tend to agree, FWIW.

Message #47 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sun, 09 Feb 2025 18:34:55 -0500

> Why?

I'd ask the author of the code.

> I would expect that it's one set of uninterned symbols per expansion.

Try:

    (defmacro my-foo (exp)
      (with-uninterned-symbols
       `(let ((x$ 6))
          (+ x$ ,exp))))

and then check (equal (macroexpand '(my-foo r)) (macroexpand '(my-foo r)))
[ Admittedly, this all depends on when `with-uninterned-symbols` is
  macro-expanded, but in most cases it will be macro-expanded once and
  for all when the macro is defined.  ]

> Doesn't that contradict your statement?

Apparently not.  🙂

> Oh, and obviously you made me curious about those weird corner cases.
> Please elaborate! ;-)

I wish I could.  I'm starting to wonder if I dreamed it.
The best I could come up with is:

    (defmacro my-countref-let (var form &rest body)
      (with-uninterned-symbols
       `(let ((x$ ,form)
              (count$ 0))
          (cl-symbol-macrolet ((,var (progn (cl-incf count$) x$)))
            (unwind-protect
                (progn ,@body)
              (message "%S ref'd %d times" ',var count$))))))

where (my-countref-let v1 67 (my-countref-let v2 89 (+ v1 v2))) will say
that `v2` was referenced twice and `v1` zero times (at least if that
macro is byte-compiled).


        Stefan

Message #50 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Mon, 10 Feb 2025 08:36:54 +0100

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

Hi Stefan,

>> Why?
>
> I'd ask the author of the code.

;-)

>> I would expect that it's one set of uninterned symbols per expansion.
>
> Try:
>
>     (defmacro my-foo (exp)
>       (with-uninterned-symbols
>        `(let ((x$ 6))
>           (+ x$ ,exp))))
>
> and then check (equal (macroexpand '(my-foo r)) (macroexpand '(my-foo r)))

That's shocking! :-|

> [ Admittedly, this all depends on when `with-uninterned-symbols` is
>   macro-expanded, but in most cases it will be macro-expanded once and
>   for all when the macro is defined.  ]

Well, my naive assumption about macro expansion is that it's done lazily
from inside-out.  In that case, I would assume the macro works as I
intended.  Also nesting should be no problem, because when the outer
with-uninterned-symbols is expanded, any inner with-uninterned-symbols
has already been expanded so its own xs $ have been replaced with #:xs.

I guess you are talking about eager macro expansion, right?  I have no
clue how that works and the docs don't tell either.  Does it mean the
macro is expanded once its defmacro form is reached and the resulting
code is just inserted at its usages?  But if that's the case, how are
the macro arguments injected in the code at the right places?

And furthermore, why does

(defmacro th/my-foo (exp)
  (let ((x (make-symbol "x")))
    `(let ((,x 6))
       (+ ,x ,exp))))

not exhibit the exactly same problem?  I guess it's because "the
machine" knows that every ,foo needs to be once-per-expansion, right?
Or maybe this dependency disables eager macro expansion for this macro
altoghether?

Is there a way to fix my macro?  Maybe to define it so that it would
result in something like

  (let-alist (collect-$-vars exp)
    ;; Now build a replacement for exp which uses ,.x accesses in place
    ;; of x$, e.g., this:
    `(let ((,.x 6))
       (+ ,.x ,exp)))

such that there's a "dependency" of the expansion to an expression
that's evaluated at expansion time which in turn depends on exp?

>> Oh, and obviously you made me curious about those weird corner cases.
>> Please elaborate! ;-)
>
> I wish I could.  I'm starting to wonder if I dreamed it.
> The best I could come up with is:
>
>     (defmacro my-countref-let (var form &rest body)
>       (with-uninterned-symbols
>        `(let ((x$ ,form)
>               (count$ 0))
>           (cl-symbol-macrolet ((,var (progn (cl-incf count$) x$)))
>             (unwind-protect
>                 (progn ,@body)
>               (message "%S ref'd %d times" ',var count$))))))
>
> where (my-countref-let v1 67 (my-countref-let v2 89 (+ v1 v2))) will
> say that `v2` was referenced twice and `v1` zero times (at least if
> that macro is byte-compiled).

Jesus Christ! ;-)

Bye,
Tassilo

Message #53 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Mon, 10 Feb 2025 09:21:03 -0500

>> [ Admittedly, this all depends on when `with-uninterned-symbols` is
>>   macro-expanded, but in most cases it will be macro-expanded once and
>>   for all when the macro is defined.  ]
> Well, my naive assumption about macro expansion is that it's done lazily
> from inside-out.

Interesting: that's impossible.
Think about a case like:

    (dolist (push pushes) (message "%S" `(pop ,push)))

how can the macro expander decide whether or not `(push pushes)` and
`(pop ,push)` are calls to macros `push` and `pop` without first
macro-expanding `dolist` and backquote?

Similarly, we can't compile the above code without first expanding all
the macros.

> And furthermore, why does
>
> (defmacro th/my-foo (exp)
>   (let ((x (make-symbol "x")))
>     `(let ((,x 6))
>        (+ ,x ,exp))))
>
> not exhibit the exactly same problem?

Because the above code says explicitly that `make-symbol` is called
every time a call to `th/my-foo` is expanded.  Whereas in your version,
the code says that `make-symbol` is called every time a call to
`with-uninterned-symbols` is expanded, but that can happen either when
we define `my-foo` or when "use" my-foo (depending on whether the
macroexpansion is done lazily or eagerly, and it can't be done lazily
if we compile the macro).

> Is there a way to fix my macro?

I don't think so, in general.  E.g. if you want to generate fresh new
uninterned symbols every time the code is executed, then you can't
compile something like

      (with-uninterned-symbols
        (let ((x$ 6))
          (+ x$ exp)))

since the name of the variable is not known until runtime (and will be
different each time).

I think to fix it, you have to fuse your macro with backquote, so users
would write

    (defmacro my-other-foo (exp)
      (uninterned-backquote
       (let ((x$ 6))
         (+ x$ ,exp))))

but of course you could still support the syntax

    (defmacro my-foo (exp)
      (with-uninterned-symbols
       `(let ((x$ 6))
          (+ x$ ,exp))))

and simply have `with-uninterned-symbols` check that its argument is of
the form `(...) and treat it as a use of `uninterned-backquote`.


        Stefan

Message #58 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76132 <at> debbugs.gnu.org, tsdh <at> gnu.org
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Wed, 12 Feb 2025 05:41:25 -0500

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

      > (defmacro my-foo (exp)
      >   (with-uninterned-symbols
      >    `(let ((x$ 6))
      >       (+ x$ ,exp))))

with-uninterned-symbols does not seem to exist in my checkout.
Is it new?  Proposed?

I can guess from the example what it does.  It is terribly un-Lispy.
Let's use this syntax instead:

   (with-uninterned-symbols (x)
     `(let ((,x 6))
        (+ ,x ,exp)))

It adds just one list of variables to the overall syntactic complexity
of the construct when used, and it adds nothing to the syntax of Lipp.

We could use the name `with-gensyms', a shorter name.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Message #61 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Richard Stallman <rms <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Wed, 12 Feb 2025 11:53:29 +0100

Richard Stallman <rms <at> gnu.org> writes:

Hi Richard,

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>       > (defmacro my-foo (exp)
>       >   (with-uninterned-symbols
>       >    `(let ((x$ 6))
>       >       (+ x$ ,exp))))
>
> with-uninterned-symbols does not seem to exist in my checkout.
> Is it new?  Proposed?

Yes, I've proposed it but as it stands, it's already declined. That's
totally acceptable for me given the details of eager macro expansion
Stefan M. explained to me.

> I can guess from the example what it does.  It is terribly un-Lispy.
> Let's use this syntax instead:
>
>    (with-uninterned-symbols (x)
>      `(let ((,x 6))
>         (+ ,x ,exp)))
>
> It adds just one list of variables to the overall syntactic complexity
> of the construct when used, and it adds nothing to the syntax of Lipp.
>
> We could use the name `with-gensyms', a shorter name.

That's already available, named cl-with-gensyms and defined in
cl-macs.el.  But truth to be told, that exhibits the very same problem
that my proposed macro also has, i.e., that

  (equal (macroexpand '(cl-with-gensyms (x)
                         `(+ ,x ,x)))
         (macroexpand '(cl-with-gensyms (x)
                         `(+ ,x ,x))))
  ;;=> t

meaning that every expansion uses the very same uninterned symbol x, not
one unique symbol x per expansion.  As Stefan explained, that can lead
to problems in certain (honestly quite uncommon) corner-cases.

Bye,
Tassilo

Message #64 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org, Richard Stallman <rms <at> gnu.org>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Wed, 12 Feb 2025 08:55:51 -0500

>   (equal (macroexpand '(cl-with-gensyms (x)
>                          `(+ ,x ,x)))
>          (macroexpand '(cl-with-gensyms (x)
>                          `(+ ,x ,x))))
>   ;;=> t

That doesn't test what you think it does because

    (macroexpand '(cl-with-gensyms (x) `(+ ,x ,x)))

does not return any uninterned symbols.  It returns the code which *when
executed* will generate a new uninterned symbol.  Try

    (equal (eval (macroexpand '(cl-with-gensyms (x)
                                 `(+ ,x ,x))))
           (eval (macroexpand '(cl-with-gensyms (x)
                                 `(+ ,x ,x)))))


- Stefan

Message #67 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org, Richard Stallman <rms <at> gnu.org>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Wed, 12 Feb 2025 09:02:14 -0500

> does not return any uninterned symbols.  It returns the code which *when
> executed* will generate a new uninterned symbol.  Try
>
>     (equal (eval (macroexpand '(cl-with-gensyms (x)
>                                  `(+ ,x ,x))))
>            (eval (macroexpand '(cl-with-gensyms (x)
>                                  `(+ ,x ,x)))))

Actually a more precise test would be:

    (let ((codegen (macroexpand-all '(cl-with-gensyms (x)
                                       `(+ ,x ,x)))))
      (equal (eval codegen t) (eval codegen t)))


- Stefan

Message #70 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 76132 <at> debbugs.gnu.org, Richard Stallman <rms <at> gnu.org>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Wed, 12 Feb 2025 15:19:27 +0100

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>>   (equal (macroexpand '(cl-with-gensyms (x)
>>                          `(+ ,x ,x)))
>>          (macroexpand '(cl-with-gensyms (x)
>>                          `(+ ,x ,x))))
>>   ;;=> t
>
> That doesn't test what you think it does because
>
>     (macroexpand '(cl-with-gensyms (x) `(+ ,x ,x)))
>
> does not return any uninterned symbols.  It returns the code which
> *when executed* will generate a new uninterned symbol.  Try
>
>     (equal (eval (macroexpand '(cl-with-gensyms (x)
>                                  `(+ ,x ,x))))
>            (eval (macroexpand '(cl-with-gensyms (x)
>                                  `(+ ,x ,x)))))

Argh, indeed.

Thanks,
Tassilo

Message #73 received at 76132 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: 76132 <at> debbugs.gnu.org, Richard Stallman <rms <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sun, 23 Feb 2025 05:40:26 +0000

Tassilo Horn <tsdh <at> gnu.org> writes:

> Richard Stallman <rms <at> gnu.org> writes:
>
> Hi Richard,
>
>> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>> [[[ whether defending the US Constitution against all enemies,     ]]]
>> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>>
>>       > (defmacro my-foo (exp)
>>       >   (with-uninterned-symbols
>>       >    `(let ((x$ 6))
>>       >       (+ x$ ,exp))))
>>
>> with-uninterned-symbols does not seem to exist in my checkout.
>> Is it new?  Proposed?
>
> Yes, I've proposed it but as it stands, it's already declined. That's
> totally acceptable for me given the details of eager macro expansion
> Stefan M. explained to me.
>
>> I can guess from the example what it does.  It is terribly un-Lispy.
>> Let's use this syntax instead:
>>
>>    (with-uninterned-symbols (x)
>>      `(let ((,x 6))
>>         (+ ,x ,exp)))
>>
>> It adds just one list of variables to the overall syntactic complexity
>> of the construct when used, and it adds nothing to the syntax of Lipp.
>>
>> We could use the name `with-gensyms', a shorter name.
>
> That's already available, named cl-with-gensyms and defined in
> cl-macs.el.  But truth to be told, that exhibits the very same problem
> that my proposed macro also has, i.e., that
>
>   (equal (macroexpand '(cl-with-gensyms (x)
>                          `(+ ,x ,x)))
>          (macroexpand '(cl-with-gensyms (x)
>                          `(+ ,x ,x))))
>   ;;=> t
>
> meaning that every expansion uses the very same uninterned symbol x, not
> one unique symbol x per expansion.  As Stefan explained, that can lead
> to problems in certain (honestly quite uncommon) corner-cases.
>
> Bye,
> Tassilo

So what's the conclusion here?  Should this bug be closed, or is there
more left to discuss?

Message #78 received at 76132-done <at> debbugs.gnu.org (full text, mbox):

From: Tassilo Horn <tsdh <at> gnu.org>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 76132-done <at> debbugs.gnu.org, Richard Stallman <rms <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sun, 23 Feb 2025 10:11:08 +0100

Stefan Kangas <stefankangas <at> gmail.com> writes:

> So what's the conclusion here?  

Macros (in the presence of eager macro expansion) are hard. :-)

> Should this bug be closed, or is there more left to discuss?

No, I'm closing it now.

Bye,
Tassilo

Message #81 received at 76132-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Tassilo Horn <tsdh <at> gnu.org>
Cc: 76132-done <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>,
 Richard Stallman <rms <at> gnu.org>
Subject: Re: bug#76132: Clojure-style auto-gensyms for macros
Date: Sun, 23 Feb 2025 11:19:53 -0500

Tassilo Horn [2025-02-23 10:11:08] wrote:
> Stefan Kangas <stefankangas <at> gmail.com> writes:
>> So what's the conclusion here?  
> Macros (in the presence of eager macro expansion) are hard. :-)

Historical notes about "eager macro expansion":

- Macros have been expanded eagerly in Emacs since the introduction of
  a byte-compiler, i.e. before even Emacs-18.
- Macros are designed specifically to allow "eager macro expansion".
  In early Lisp there was another mechanism which supported only lazy
  expansion, called [fexpr](https://en.wikipedia.org/wiki/Fexpr).
  They died because of their fundamental incompatibility with compilation.


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.