Hi, Stefan,
I've attached an updated patch that removes (I think) the
modifications that aren't actually for re-entrancy of
read/readevalloop.
I think I got all of the unintentional whitespace changes backed out as well.
I also applied one of Pip's changes, and the fix that lets it compile
and start without a hitch, setting file streams to load as multibyte,
since the utf-8-emacs decoding was always in the readchar function.
I ran "make check", and attached a short elisp file I used to load the
ones with failures as well as the results of "M-x ert t".
It looks like eval-buffer doesn't restore the buffer state correctly
at least some of the time.

Also, the rx-tests fail because (load "<...>/rx-tests.elc") has a
failure in read0 - something is screwy that seems to be related to the
#@10 near the beginning, since readevalloop successfully evaluates the
first exec-byte-code form (the whole thing is available as "val" in
the frame when I look through the backtrace), but read0 starts the
next expression 10 characters back at a "]".  Which, I don't know how
it goes back by 10 characters when it's just reading a stdio stream
one character at a time.  I'll have to investigate further.


On Sun, Jun 29, 2025 at 11:55 PM Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>
> >>     temacs ... '(progn (blabla))'
> >>
> >> I still can't see why you'd need a reentrant `read` for that.
> >
> > Basically the latter, because I was looking at using it Makefiles and
> > didn't want to deal with generating little files to load.  If I was
> > willing to limit the strings to single expressions, you'd be right,
> > but why should the form a program is input (string, file, buffer,
> > whatever) determine what's allowed?
>
> I don't understand the connection.
> Hmm... oh, is it because when you use `read` on a string it can read
> only the first sexp?  If so, I kinda see your point, but since you know
> you're starting from a string, you can use `read-from-string` which
> returns the info needed for the loop that reads all the sexps.
>
> > I did try to get by with just save/restore on the string variables,
> > but it failed when evaluating '(load "loadup.el") (print "Finished")".
>
> I still can't imagine what that "just save/restore on the string
> variables" looked like.
>
> > The failure certainly seems like an issue caused by some subtlety in
> > the way recursive use of readevalloop is implemented, but I can't
>
> From which I gather the above "just save/restore on the string
> variables" somehow involved `readevalloop`, but I don't know how.
>
> > bring myself to try and figure out a workaround for the specific
> > brokenness when I could just rework it to be sure its reentrant, then
> > see if that fixes the issue.
>
> 🙂
> Seems like a big hammer for that little problem.
> But if it can get you to improve our code, who are we to judge, eh?
>
> > Or maybe the problem I'm solving is my ignorance of how the Emacs
> > runtime works in practice?  A lack of amusing pasttimes?  A stubborn
> > refusal to accept things that do not make sense to me?
>
> You're at the right place.

Also, this kind of exercise is just table stakes if I ever want to try
my hand at some of the ideas I've been bouncing around emacs-devel for
years now.  Like encapsulating the lisp machine so a single process
can have multiple lisps running, then work out distributed data
structures, etc.  Having a gc that supports concurrency will remove
the biggest blocker for that.
>
> >> Any hope you can give us an actual concrete scenario?
> > I keep giving you one, you just consider it unnecessary.
>
> No, I just can't imagine how you get from the problem you describe to an
> attempted solution that uses `readevalloop` (from there I see how you
> got to the re-entrancy thingy).
>
Maybe it will make more sense once I submit the crazy feature I've
implemented that this started with.  I needed a basic trampoline in
place of command-loop, since Frecursive_edit requires things that
aren't available without loadup.el loaded.

> >> >> How/when&why do you expect/want `read` to be called recursively?
> >> > It can definitely be entered multiple times if a stream being read
> >> > blocks and the command loop executes another function that calls read.
> >> > That can clearly happen when reading from user-supplied procedures or
> >> > a raw stdio stream (i.e. from "load").
> >>
> >> A concrete scenario would help, here as well.
> >
> > Ok, there are calls to maybe quit in readbyte_from_stdio where the
> > file returns EOF due to an interrupt.  So, if one function is reading
> > a lisp expression when this occurs, then the user type "M-: (read
> > <something>)", you will have a recursive re-entry to read.
>
> Hmm... IIRC `maybe_quit` is not supposed to run ELisp code.
> The more likely path to the problem would seem to be via a stream
> that's a function.
>
> In any case, if you don't know of a recipe to reproduce the problem,
> don't bother looking for one, I was asking because I thought you had
> bumped into one, so you could just exhibit it, I didn't mean to ask you
> to reverse-engineer it from the code.
>
> [ FWIW, I definitely believe you that it's possible.  But I expect we
>   have *many* potential problems like these in our code base, so unless
>   we have a clear way to solve a whole class of them I'd rather focus on
>   the ones that are known to occur in existing code.  ]
>
> > It's not really intended to be a procedure, just a static initializer.
>
> I understand, but functions are just so much better behaved than macros
> that it's worth avoiding macros when a function can be used just as well.
>
> > I haven't regularly programmed in C since the decade I learned the
> > language from the recently published K&R 2nd edition, so my style
> > discipline is pretty low.
>
> That was my assumption.  Since then, compilers have gotten much better
> at inlining functions, so you can do
>
>     init_inbuffer (&x);
>
> and they'll generate the same code as it does with your
>
>     struct inbuffer x = INIT_INBUFFER ();
>
> [ Well, admittedly, not with `-O0`.  ]
>
> Note also that
>
>     struct inbuffer x = { .buffer = buf };
>
> should work just as well, so maybe "nothing" is an even better option
> than either.

I noticed that, now, I was only using those macros in one place each,
so I've just done away with the macros and written them out (in the
attached patch).

>
> > It took me forever to work out a "missing braces" message was
> > referring to a specpdl_ref initializer in the INIT_LREAD_FRAME.
>
> 🙂
>
> >> Also, why do you need the "uninit" case?
> > That's my unfamiliarity with C99 and zero-initializing a union
> > and being certain it zeros the entire thing and not just the
> > first alternative.
>
> Ah I see.  AFAIK `memset` is the standard solution.
> `grep` shows it's used extensively in our code for that very purpose.
> [ Current C compilers routinely replace calls to `memset` and `memcpy`
>   with open-coded machine code when the size is known at compile-time.  ]

I don't have a style discipline (at least not once GNU's style is
chosen - my preferences run to more compact presentations).  But I do
have a kind of aesthetic preference for writing constant values as
literals (compound literals in this case).  I mean, that is definitely
a LISP heritage that's shown up in C and other imperative languages.

I may follow up on some of the other points later, but maybe I should
give you a chance to look at the cleaned up patch.

Lynn