On Sat, Jul 26, 2025 at 12:00 PM Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>
> >> BTW memory that's not returned to the OS but that Emacs can reuse
> >> later is generally not considered as a leak.  E.g., most `malloc`
> >> implementations have the property of (virtually) never bothering to
> >> return freed memory to the OS.
> > Terminology aside, the memory used for the stack is never freed for
> > use by other parts of Emacs.
>
> Yup.  Still not considered a leak.  🙁
> It may be a waste of memory, of course, but we use similar approaches in
> other places without worrying very much about it.  For example, we grow
> hashtables on-demand in `puthash`, but we never shrink them back in
> `remhash`.
>
> That doesn't mean I'm opposed to changing the read code to be more
> careful with its use of memory, but I don't consider it a serious
> concern, so code clarity, speed, reentrancy, and things like that are
> more important.

Ok, the attached patch bootstraps with -O0 without native compilation.
I haven't done any benchmarking or tuning, or tested with optimization
turned on, but it shows the technique.  The changes don't seem to have
leaked beyond the obviously required scope.  The patch does eliminate
one unwind_protect (per entry to read0) which is made unnecessary by
using alloca.

If this approach works in the reader, the same technique could be
applied to making tail recursion safe for space in the bytecode VM.

> >> I'd suggest you simply concatenate a bunch of `.elc` files, until you
> >> get something "large enough" to make the timing easy to measure.
> >> Then measure the time to read that file (put the content inside
> >> a buffer, wrap it inside a pair of `(...)`, then (benchmark-run
> >> (read))`).
> >
> > So the circular expression syntax supports creating such nested
> > expressions?
>
> Yes:
>
>     ELISP> '(#1=4 #1# #1=5 #1#)
>     (4 4 5 5)
>
>     ELISP>
>
> > I didn't notice read0 recurring to the internal start (to get
> > fresh hash tables for the object map).  I don't see how such an artificial
> > list of byte code could be guaranteed to be read correctly if those hash
> > tables are not cleared between top level expressions in the file.
>
> The #N# references in a `.elc` file would fail when loaded if they refer
> to a #N= that's not in the same file.  After concatenation, since the
> #N# refers to the "most recent" #N= it should always refer to the
> "right" one.

I need to go back and check some elc files, because it seemed like
some of the byte-code had #N= definitions after #N# uses, which could
happen if the enumeration is performed breadth-first rather than
depth-first.

In any case, for benchmarking purposes:
* the cost of allocating the structure being read should count toward
the cost of read, but the cost of growing those hash tables repeatedly
should not.
* none of the constructed expressions are released, so the cost of
expanding the heap size (from the OS) is included.

One quibble I would have is that there is some difference in the
details of reading a character from a buffer and reading from a file.
It just seems weird because ELC files are never loaded into a buffer,
and buffers are not exactly simple.  An alternative would be to memory
map elc files that fstat indicates are regular.  That might be a
useful change in itself.

Lynn