Subject: bug#79387: 30.1.90; If a thread exits, other threads in wait_reading_process_output aren't woken up

From: Eli Zaretskii <[email-masked]>
Date: Sat, 06 Sep 2025 09:36:46 +0300

> From: Spencer Baugh <[email-masked]>
> Cc: [email-masked],  [email-masked]
> Date: Fri, 05 Sep 2025 11:57:35 -0400
> 
> Eli Zaretskii <[email-masked]> writes:
> > I'm not yet convinced we should try to fix this.  It is certainly not
> > as serious a problem as the one with deleting processes locked to
> > other threads, something we need to fix ASAP, and might affect what
> > happens in the situation you describe here.
> 
> I agree it's unlikely to be a serious problem in real-world programs.
> But nevertheless it's one I would like to fix.
> 
> Do you think the "associate a pipe with each thread to use for wakeups,
> and write to it when a thread exits" approach sounds reasonable?

I'd prefer not to, because a self-pipe is not portable enough.

Moreover, when a thread exits, it releases the global lock, so some
other thread will start running, and Emacs will eventually watch the
processes that are still alive when it becomes idle.  So we already
have a mechanism for recovering from these situations, to some extent.

> BTW, if we added such a pipe it would also be useful for other purposes;
> if we reimplemented condition-wait by using this pipe, that would
> resolve my concerns about locked processes, so we'd be able to finally
> drop that topic and move forward with the "processes are locked by
> default" behavior.

Let's not introduce infrastructure that complicates the design and the
implementation, just because we might need them for some future issue
that did not yet happen.

> >> Sorry to open up yet another thread bug, but while working on fixes for
> >> the other ones, I ran into this.  And I think this one is a fairly
> >> fundamental problem which will require more substantial changes, so
> >> let's fix this one first.
> >
> > Yes, let's please fix the other ones first.
> 
> I'm working on both.  There's no rush.  I discovered these bugs and all
> the other thread bugs I've posted recently, and I will fix them in
> whatever order I prefer.  I will fix them all eventually, I promise.

The problems with deleting processes locked to other threads is quite
serious, and might well affect other issues, since all of the
scenarios we are discussing use sub-processes.  So I think we should
expedite the fix for that problem, and then revisit the others and see
whether they still exist and how did the behavior change there.

> So I will follow my own software development approach, and hopefully you
> will not block that.

What is that approach?

> > But I fail to understand why this is a problem.  A single-threaded
> > Emacs is documented to process output from subprocesses only when it
> > is idle, so the exact timing of the process output being made
> > available to Lisp programs is already not deterministic.  In the case
> > of several threads, the timing also depends on what other threads do
> > when the process produces its output.  IOW, a program that expects the
> > output to appear after exactly 1 sec just because of "sleep 1" has
> > wrong expectations.
> 
> That's all true.  However, a program can reasonably expect that its
> process output will appear eventually, and this bug means that the
> process output might *never* appear, if other threads aren't
> experiencing timeouts or events.

It's unlikely, because Emacs always processes these events when it is
idle.  A situation where Emacs never becomes idle means some grave
bug, because it means there will be no response for keyboard input,
either.  In any case, if this is the result of a bug, buggy behavior
is to be expected.  We don't have to fix bugs of Lisp programs.

> > Especially since the thread exits without waiting for the output from
> > its process and without shutting down the process, which is basically
> > not clean and arguably a bug.
> 
> A thread might exit due to a bug and leave a process object lying
> around, indeed.  But if that results in that process object becoming
> stuck, and never running its filter and sentinel again even if the OS
> process is writing output or has exited, that seems bad.

I don't see yet how a process could become stuck forever.  But even if
it does, a buggy Lisp program can cause such bugs, even without
threads, and there's nothing here we should care too much to fix in
the infrastructure, definitely not by complicating what is already
quite complex.