Subject: bug#78898: Make read/readevalloop reentrant

From: Stefan Monnier <[email-masked]>
Date: Sat, 26 Jul 2025 12:00:08 -0400

>> BTW memory that's not returned to the OS but that Emacs can reuse
>> later is generally not considered as a leak.  E.g., most `malloc`
>> implementations have the property of (virtually) never bothering to
>> return freed memory to the OS.
> Terminology aside, the memory used for the stack is never freed for
> use by other parts of Emacs.

Yup.  Still not considered a leak.  🙁
It may be a waste of memory, of course, but we use similar approaches in
other places without worrying very much about it.  For example, we grow
hashtables on-demand in `puthash`, but we never shrink them back in
`remhash`.

That doesn't mean I'm opposed to changing the read code to be more
careful with its use of memory, but I don't consider it a serious
concern, so code clarity, speed, reentrancy, and things like that are
more important.

>> Yup: the \N{CHARNAME} notation cannot occur too early in the build.
>> And we know it won't occur at all in the (non-bootstrap) dump because
>> the \N{CHARNAME} notation is never used in the `.elc` or `.eln` files
>> we generate (because `print.c` never generates it).
>
> I'll put this in the "very unlikely" category, at least in configurations
> supported by Emacs maintainers.

🙂

>> I'd suggest you simply concatenate a bunch of `.elc` files, until you
>> get something "large enough" to make the timing easy to measure.
>> Then measure the time to read that file (put the content inside
>> a buffer, wrap it inside a pair of `(...)`, then (benchmark-run
>> (read))`).
>
> So the circular expression syntax supports creating such nested
> expressions?

Yes:

    ELISP> '(#1=4 #1# #1=5 #1#)
    (4 4 5 5)

    ELISP> 

> I didn't notice read0 recurring to the internal start (to get
> fresh hash tables for the object map).  I don't see how such an artificial
> list of byte code could be guaranteed to be read correctly if those hash
> tables are not cleared between top level expressions in the file.

The #N# references in a `.elc` file would fail when loaded if they refer
to a #N= that's not in the same file.  After concatenation, since the
#N# refers to the "most recent" #N= it should always refer to the
"right" one.

        Stefan