GNU bug report logs - #79333
31.0.50; Processes (still) aren't actually locked to threads

Previous Next

Full log

View this message in rfc822 format

From: Eli Zaretskii <eliz <at> gnu.org>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: dmitry <at> gutov.dev, 79333 <at> debbugs.gnu.org
Subject: bug#79333: 31.0.50; Processes (still) aren't actually locked to threads
Date: Fri, 29 Aug 2025 16:19:30 +0300

> From: Spencer Baugh <sbaugh <at> janestreet.com>
> Cc: dmitry <at> gutov.dev,  79333 <at> debbugs.gnu.org
> Date: Fri, 29 Aug 2025 08:55:41 -0400
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > Making sentinels run in the context of the thread that started the
> > process would require to redesign this part of Emacs.  Until then, we
> > will need to document this subtlety.
> 
> Makes sense.
> 
> >> So even after c93be71e45, processes aren't actually locked to threads.
> >
> > Only the sentinel is not locked.
> 
> The filter can also be run in a different thread, because it's run for
> any remaining output at the time the process terminates.
> 
> This code demonstrates that:

In the discussion of bug#79334, I suggested to make a change in
status_notify which should prevent that, as long as the thread to
which the process is locked is alive.

> >> (Once again, I think we should take the opportunity here to delete the
> >> code for locking processes to threads, since IMO it is not useful, and
> >> it is still broken)
> >
> > I did explain why it is useful, and you haven't brought up any
> > arguments to the contrary.  And if "broken" means that the sentinel
> > can run in the context of a random thread, then how come you are
> > asking to leave it in this broken state?
> 
> I was working on an example to demonstrate how process locking can cause
> problems for unrelated Lisp code when I found this bug.
> 
> Here's a finished example of the problem.
> 
> Suppose I have the following Lisp program which doesn't use threads:
> 
> (run-at-time .1 nil #'async-shell-command "sleep 1 && echo foobar && sleep inf")
> (with-current-buffer (get-buffer-create shell-command-buffer-name-async)
>   (while (string-empty-p (buffer-string))
>     (message "waiting for some process output")
>     (sit-for 1))
>   (message "buffer contents: %s" (buffer-string)))
> 
> This is intended to represent some arbitrary package which calls
> make-process in a timer or a hook.  The command "sleep 1 && echo foobar
> && sleep inf" is chosen to represent some interactive executable like a
> shell or REPL.
> 
> This code runs just fine, with the output appearing in the buffer as
> expected.
> 
> Now suppose I have some other unrelated package which runs the following
> code using threads:
> 
>   (make-thread
>    (lambda ()
>      (accept-process-output nil 1)
>      (thread-join (make-thread (lambda () (while t (message "doing work") (sit-for 10)))))))
> 
> This is intended to represent thread 1 waiting for some other thread 2
> to complete some long-running task.  Crucially, thread 1 is blocked in
> thread-join, which doesn't run wait_reading_process_output, so thread 1
> won't read output from any locked processes.
> 
> Running the second piece of code hangs the first piece of code, even
> though neither of them are buggy, and they aren't visibly interacting,
> and they can be written by totally different authors.
> 
> Specifically: The timer can be run in the thread created by the second
> piece of code, and then the process will be created locked to that
> thread.  Then the process's output will never be read by thread 1.  So
> the first piece of code will hang.

I don't understand why one thread starts a process, then another
thread waits for its output, and the program which arranges for that
doesn't unlock the process so the other thread could do its job.  This
is what set-process-thread is for, and in this (IMO rather unusual)
arrangement, calling it with a nil THREAD argument is exactly what
should be done.

> As far as I can tell, the only possible fix for this problem is to not
> lock processes to threads.

No, the fix is for the program to unlock the process using
set-process-thread.  Did you try that, and if so, did it help?

> (Especially because, as the original bug demonstrates, we aren't fully
> locking processes to threads, neither sentinels nor filters, so we
> aren't actually getting the benefits of that locking, only the costs)

If we want programs using threads to be more deterministic and
predictable, we need to beef up the locking, not throw it away.  At
least that is my conclusion from all these discussions, and the above
doesn't contradict it, at least not yet.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.