GNU bug report logs - #79333
31.0.50; Processes (still) aren't actually locked to threads

Previous Next

Full log

Message #26 received at 79333 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmitry <at> gutov.dev, 79333 <at> debbugs.gnu.org
Subject: Re: bug#79333: 31.0.50; Processes (still) aren't actually locked to
 threads
Date: Fri, 29 Aug 2025 11:20:32 -0400

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> Cc: dmitry <at> gutov.dev,  79333 <at> debbugs.gnu.org
>> Date: Fri, 29 Aug 2025 09:29:37 -0400
>> 
>> > I don't understand why one thread starts a process, then another
>> > thread waits for its output, and the program which arranges for that
>> > doesn't unlock the process so the other thread could do its job.
>> 
>> Yes, that's the bug.
>> 
>> It is not intentional that the process is started in a thread.  That is
>> what causes the bug.
>> 
>> > This is what set-process-thread is
>> > for, and in this (IMO rather unusual) arrangement, calling it with a
>> > nil THREAD argument is exactly what should be done.
>> 
>> The point is that these are two independent pieces of code, written by
>> different authors.
>> 
>> If they just happen to interleave in this way, then the process is
>> *accidentally, unintentionally* started in a thread.
>> 
>> How would the author of snippet 1 know to call set-process-thread in
>> this case?
>
> If you are saying that two arbitrary independently-written pieces of
> code can get in trouble if they are lumped together to run by the same
> Lisp program in two separate threads, then I agree.

I guess that's what I'm saying.  But the Lisp program here is just
"Emacs".  This combination of two independent pieces of code just
automatically happens when users is using one package which is using
timers, and another package which is using threads.  Which of course
happens all the time without anyone choosing to do it.

For example, one package might add a find-file-hook which starts a
subprocess, then another package might add a find-file-hook which starts
a thread.  Then when the two hooks run in succession, it would cause
this problem.

> However, having a function that starts a process, but doesn't process
> its output, and another function that doesn't start any processes, but
> does accept output from subprocesses, is an unusual thing to do.

Ah, I guess you're referring to the explicit accept-process-output call.
I think that was a confusing part of my example, because it was not
necessary to cause the issue.

Here's a more refined example:

;; Package 1 (perhaps run in a find-file-hook)
(run-at-time .3 nil #'async-shell-command "sleep 1 && echo foobar && sleep inf")
;; Package 2 (perhaps run in a find-file-hook)
(make-thread
 (lambda ()
   (sit-for 1)
   (thread-join (make-thread (lambda () (while t (sit-for 1)))))))

The shell command started by package 1 will sometimes hang forever
without producing output.

> This could happen as a deliberate design of a program, but then we are
> not talking about two snippets oblivious to one another, because the
> person who brings them together like that in the same program does
> that deliberately, and should understand that for it to work, the
> process should be either unlocked or locked to the thread which wants
> to read and process its output.
>
> IOW, when making such programs where threads are not independent calls
> for some adjustments in the code of each thread.
>
> What I have in mind is a different case, which I think is much more
> common, at least at this stage of using Lisp thread in Emacs.  It's a
> case where one takes a single-threaded Lisp program, and runs it from
> a separate thread so as to avoid blocking the Emacs's main thread.  In
> that case, the same thread will both start the process and expect to
> be able to process its output (because that's how single-threaded Lisp
> programs work), and therefore having the process locked by default
> lets such code work as expected when it is run from a thread.
> Especially if you take several such programs, each with its own
> subprocess, and let them all run from several different threads at the
> same time.

Yes, I definitely want that real-world case to work right.  I agree that
that is a very important case.  But I think it already works right with
processes not locked to threads for any non-buggy program.

(I personally have written or used lots of code like that with threads,
and the fact that processes were not fully locked to threads did not
cause problems.  As one public example, diff-hl-mode's
diff-hl-update-async)

For example, a program like this will work correctly even in a thread:

(let ((proc (make-process ...)))
  (accept-process-output proc))

This would run the filter functions in the same thread, because output
from PROC can't be read by another thread until we do a thread switch,
which will only happen when we call accept-process-output.

If the program was instead something like:

(let ((proc (make-process ...)))
  (sit-for 1)
  (accept-process-output proc))

then the (accept-process-output proc) might block because the sit-for
can thread switch.  But this program is already buggy, since sit-for
runs wait_reading_process_output which could read the output from PROC.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.