GNU bug report logs - #79387
30.1.90; If a thread exits, other threads in wait_reading_process_output aren't woken up

Previous Next

Full log

View this message in rfc822 format

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmitry <at> gutov.dev, 79387 <at> debbugs.gnu.org
Subject: bug#79387: 30.1.90; If a thread exits, other threads in wait_reading_process_output aren't woken up
Date: Fri, 05 Sep 2025 11:57:35 -0400

Eli Zaretskii <eliz <at> gnu.org> writes:
>> Cc: Dmitry Gutov <dmitry <at> gutov.dev>, Eli Zaretskii <eliz <at> gnu.org>
>> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> Date: Thu, 04 Sep 2025 15:57:18 -0400
>> To fix this, we need to wake up threads which are blocked in
>> wait_reading_process_output when another thread exits.  This will
>> require adding a new mechanism to support this.  I think a good way is
>> for thread creation to open a pipe specific to that thread, used for
>> this purpose; then when a thread is blocked in
>> wait_reading_process_output, it should monitor this pipe along with
>> other file descriptors.  Then an exiting thread can write to this pipe
>> to wake up a thread which is currently blocked in
>> wait_reading_process_output.
>
> I'm not yet convinced we should try to fix this.  It is certainly not
> as serious a problem as the one with deleting processes locked to
> other threads, something we need to fix ASAP, and might affect what
> happens in the situation you describe here.

I agree it's unlikely to be a serious problem in real-world programs.
But nevertheless it's one I would like to fix.

Do you think the "associate a pipe with each thread to use for wakeups,
and write to it when a thread exits" approach sounds reasonable?

BTW, if we added such a pipe it would also be useful for other purposes;
if we reimplemented condition-wait by using this pipe, that would
resolve my concerns about locked processes, so we'd be able to finally
drop that topic and move forward with the "processes are locked by
default" behavior.

>> Sorry to open up yet another thread bug, but while working on fixes for
>> the other ones, I ran into this.  And I think this one is a fairly
>> fundamental problem which will require more substantial changes, so
>> let's fix this one first.
>
> Yes, let's please fix the other ones first.

I'm working on both.  There's no rush.  I discovered these bugs and all
the other thread bugs I've posted recently, and I will fix them in
whatever order I prefer.  I will fix them all eventually, I promise.

I keep discovering additional bugs as I work on fixing the bugs I've
already discovered.  This is good: the more bugs I discover, the more I
understand what is wrong about the system.  It may be that these
individual bugs are a symptom of a single deeper problem, and I can fix
all of them with a change to fix that problem, once I understand enough.

So I will follow my own software development approach, and hopefully you
will not block that.  Again, there's no rush.  The bugs I've reported
have been in Emacs for many years.

>> You can reproduce this bug by running this code with "emacs -Q --batch":
>> ;; -*- lexical-binding: t; -*-
>> (defun my-filter (proc out)
>>   (message "%s %s" proc out))
>> (make-thread
>>  (lambda ()
>>    (message "starting thread")
>>    (sit-for 1)
>>    (message "making process")
>>    (make-process
>>     :name "true"
>>     :filter #'my-filter
>>     :command '("sh" "-c" "sleep 1 && echo hello && sleep inf"))
>>    (message "thread exiting")))
>> (message "main thread sleeping")
>> (sit-for 10)
>> (message "main thread sleeping again")
>> (sit-for 10)
>> 
>> The process output will not be printed until the first sit-for is
>> complete.
>
> First, the results I see are not deterministic, and also
> system-dependent.  On MS-Windows, the process output is consistently
> printed right after "thread exiting".  On GNU/Linux, this also
> happens, but very rarely; in most runs indeed the process output
> appears after several seconds.

Yes, that sounds likely.  My fix will make the behavior deterministic
and not system-dependent.

> But I fail to understand why this is a problem.  A single-threaded
> Emacs is documented to process output from subprocesses only when it
> is idle, so the exact timing of the process output being made
> available to Lisp programs is already not deterministic.  In the case
> of several threads, the timing also depends on what other threads do
> when the process produces its output.  IOW, a program that expects the
> output to appear after exactly 1 sec just because of "sleep 1" has
> wrong expectations.

That's all true.  However, a program can reasonably expect that its
process output will appear eventually, and this bug means that the
process output might *never* appear, if other threads aren't
experiencing timeouts or events.

> Especially since the thread exits without waiting for the output from
> its process and without shutting down the process, which is basically
> not clean and arguably a bug.

A thread might exit due to a bug and leave a process object lying
around, indeed.  But if that results in that process object becoming
stuck, and never running its filter and sentinel again even if the OS
process is writing output or has exited, that seems bad.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.