Package: emacs;
Reported by: Lynn Winebarger <owinebar <at> gmail.com>
Date: Wed, 25 Jun 2025 22:01:05 UTC
Severity: normal
Message #56 received at 78898 <at> debbugs.gnu.org (full text, mbox):
From: Lynn Winebarger <owinebar <at> gmail.com> To: Stefan Monnier <monnier <at> iro.umontreal.ca> Cc: 78898 <at> debbugs.gnu.org Subject: Re: bug#78898: Make read/readevalloop reentrant Date: Tue, 1 Jul 2025 02:43:04 -0400
[Message part 1 (text/plain, inline)]
Hi, Stefan,
I've attached an updated patch that removes (I think) the
modifications that aren't actually for re-entrancy of
read/readevalloop.
I think I got all of the unintentional whitespace changes backed out as well.
I also applied one of Pip's changes, and the fix that lets it compile
and start without a hitch, setting file streams to load as multibyte,
since the utf-8-emacs decoding was always in the readchar function.
I ran "make check", and attached a short elisp file I used to load the
ones with failures as well as the results of "M-x ert t".
It looks like eval-buffer doesn't restore the buffer state correctly
at least some of the time.
Also, the rx-tests fail because (load "<...>/rx-tests.elc") has a
failure in read0 - something is screwy that seems to be related to the
#@10 near the beginning, since readevalloop successfully evaluates the
first exec-byte-code form (the whole thing is available as "val" in
the frame when I look through the backtrace), but read0 starts the
next expression 10 characters back at a "]". Which, I don't know how
it goes back by 10 characters when it's just reading a stdio stream
one character at a time. I'll have to investigate further.
On Sun, Jun 29, 2025 at 11:55 PM Stefan Monnier
<monnier <at> iro.umontreal.ca> wrote:
>
> >> temacs ... '(progn (blabla))'
> >>
> >> I still can't see why you'd need a reentrant `read` for that.
> >
> > Basically the latter, because I was looking at using it Makefiles and
> > didn't want to deal with generating little files to load. If I was
> > willing to limit the strings to single expressions, you'd be right,
> > but why should the form a program is input (string, file, buffer,
> > whatever) determine what's allowed?
>
> I don't understand the connection.
> Hmm... oh, is it because when you use `read` on a string it can read
> only the first sexp? If so, I kinda see your point, but since you know
> you're starting from a string, you can use `read-from-string` which
> returns the info needed for the loop that reads all the sexps.
>
> > I did try to get by with just save/restore on the string variables,
> > but it failed when evaluating '(load "loadup.el") (print "Finished")".
>
> I still can't imagine what that "just save/restore on the string
> variables" looked like.
>
> > The failure certainly seems like an issue caused by some subtlety in
> > the way recursive use of readevalloop is implemented, but I can't
>
> From which I gather the above "just save/restore on the string
> variables" somehow involved `readevalloop`, but I don't know how.
>
> > bring myself to try and figure out a workaround for the specific
> > brokenness when I could just rework it to be sure its reentrant, then
> > see if that fixes the issue.
>
> 🙂
> Seems like a big hammer for that little problem.
> But if it can get you to improve our code, who are we to judge, eh?
>
> > Or maybe the problem I'm solving is my ignorance of how the Emacs
> > runtime works in practice? A lack of amusing pasttimes? A stubborn
> > refusal to accept things that do not make sense to me?
>
> You're at the right place.
Also, this kind of exercise is just table stakes if I ever want to try
my hand at some of the ideas I've been bouncing around emacs-devel for
years now. Like encapsulating the lisp machine so a single process
can have multiple lisps running, then work out distributed data
structures, etc. Having a gc that supports concurrency will remove
the biggest blocker for that.
>
> >> Any hope you can give us an actual concrete scenario?
> > I keep giving you one, you just consider it unnecessary.
>
> No, I just can't imagine how you get from the problem you describe to an
> attempted solution that uses `readevalloop` (from there I see how you
> got to the re-entrancy thingy).
>
Maybe it will make more sense once I submit the crazy feature I've
implemented that this started with. I needed a basic trampoline in
place of command-loop, since Frecursive_edit requires things that
aren't available without loadup.el loaded.
> >> >> How/when&why do you expect/want `read` to be called recursively?
> >> > It can definitely be entered multiple times if a stream being read
> >> > blocks and the command loop executes another function that calls read.
> >> > That can clearly happen when reading from user-supplied procedures or
> >> > a raw stdio stream (i.e. from "load").
> >>
> >> A concrete scenario would help, here as well.
> >
> > Ok, there are calls to maybe quit in readbyte_from_stdio where the
> > file returns EOF due to an interrupt. So, if one function is reading
> > a lisp expression when this occurs, then the user type "M-: (read
> > <something>)", you will have a recursive re-entry to read.
>
> Hmm... IIRC `maybe_quit` is not supposed to run ELisp code.
> The more likely path to the problem would seem to be via a stream
> that's a function.
>
> In any case, if you don't know of a recipe to reproduce the problem,
> don't bother looking for one, I was asking because I thought you had
> bumped into one, so you could just exhibit it, I didn't mean to ask you
> to reverse-engineer it from the code.
>
> [ FWIW, I definitely believe you that it's possible. But I expect we
> have *many* potential problems like these in our code base, so unless
> we have a clear way to solve a whole class of them I'd rather focus on
> the ones that are known to occur in existing code. ]
>
> > It's not really intended to be a procedure, just a static initializer.
>
> I understand, but functions are just so much better behaved than macros
> that it's worth avoiding macros when a function can be used just as well.
>
> > I haven't regularly programmed in C since the decade I learned the
> > language from the recently published K&R 2nd edition, so my style
> > discipline is pretty low.
>
> That was my assumption. Since then, compilers have gotten much better
> at inlining functions, so you can do
>
> init_inbuffer (&x);
>
> and they'll generate the same code as it does with your
>
> struct inbuffer x = INIT_INBUFFER ();
>
> [ Well, admittedly, not with `-O0`. ]
>
> Note also that
>
> struct inbuffer x = { .buffer = buf };
>
> should work just as well, so maybe "nothing" is an even better option
> than either.
I noticed that, now, I was only using those macros in one place each,
so I've just done away with the macros and written them out (in the
attached patch).
>
> > It took me forever to work out a "missing braces" message was
> > referring to a specpdl_ref initializer in the INIT_LREAD_FRAME.
>
> 🙂
>
> >> Also, why do you need the "uninit" case?
> > That's my unfamiliarity with C99 and zero-initializing a union
> > and being certain it zeros the entire thing and not just the
> > first alternative.
>
> Ah I see. AFAIK `memset` is the standard solution.
> `grep` shows it's used extensively in our code for that very purpose.
> [ Current C compilers routinely replace calls to `memset` and `memcpy`
> with open-coded machine code when the size is known at compile-time. ]
I don't have a style discipline (at least not once GNU's style is
chosen - my preferences run to more compact presentations). But I do
have a kind of aesthetic preference for writing constant values as
literals (compound literals in this case). I mean, that is definitely
a LISP heritage that's shown up in C and other imperative languages.
I may follow up on some of the other points later, but maybe I should
give you a chance to look at the cleaned up patch.
Lynn
[0001-Changes-to-make-entry-to-read0-and-readevalloop-recu.patch (text/x-patch, attachment)]
[bad-tests.el (text/x-emacs-lisp, attachment)]
[ert-failures.el (text/x-emacs-lisp, attachment)]
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.