Subject: bug#78946: 30.1; accept-process-output not allowed in threads on macOS

From: Robert Pluim <[email-masked]>
Date: Wed, 09 Jul 2025 10:12:17 +0200

>>>>> On Mon, 7 Jul 2025 21:47:09 +0100, Alan Third <[email-masked]> said:

    Alan> OK, the fd_handler thread only exists because we "need" to run the NS
    Alan> run loop and pselect simultaneously. Obviously we can't do that with a
    Alan> single thread, so one has to be put into a separate thread. The
    Alan> fd_handler only communicates with the run loop in as much as it tells
    Alan> it to stop. All it's actual *results* are handed back via the global
    Alan> variables select_readfds, select_writefds, etc.

    Alan> So there's nothing special about running pselect in the main thread
    Alan> here, it's just not done because the run loop is required at the same
    Alan> time.

    Alan> This means that if we decide we don't need to run the run loop (which
    Alan> is something we used to do) then we could just call thread_select. And
    Alan> just to confuse matters we have to run thread_select anyway, as that's
    Alan> how Emacs chooses to run another lisp thread. So you'll see a call
    Alan> near the top of the ns_select function with a zero timeout and we just
    Alan> ignore the return value.

I guess you mean this in ns_select_1, which does a non-blocking
pselect for 0 seconds on no file descriptors:

  else
    {
      struct timespec t = {0, 0};
      thread_select (pselect, 0, NULL, NULL, NULL, &t, sigmask);
    }

If itʼs there for the emacs thread infrastructure, is there a more
explicit way of doing whatever it does?

    Alan> Sub-threads, as you say, can't call the run loop, so for them
    Alan> ns_select should act pretty much just like a call to pselect.

OK. But it would be good if we could have a single code path, rather
than 2.

    Alan> Ultimately, ns_select should look to the caller just like a call to
    Alan> pselect, or whatever. It should return whatever pselect would return
    Alan> in that situation, and shouldn't behave differently, although you and
    Alan> I know it's doing a whole bunch of other stuff behind the caller's
    Alan> back.

    Alan> I scare quoted "need" above because the reality is that we don't
    Alan> actually *need* to run the run loop in ns_select, it's just a
    Alan> convenient place for it.

    Alan> The ideal situation would be for the run loop to run in it's own
    Alan> thread and ns_select could then be replaced with a simple call to
    Alan> thread_select like some of the other terms.

Yes. I was a bit surprised when I discovered that weʼre starting and
stopping the run loop all the time. It makes reasoning about stuff a
lot harder.

    Alan> But implementing that comes with it's own set of nightmares.

Such as? If we put the run loop in its own thread, running forever,
then the main emacs code (and emacs threads) can call
pselect/thread_select without having to communicate with the
fd_handler thread, which we could remove[1]. It would remove a bunch of
code thatʼs just needed to synchronize between the main thread, the
run loop, and the fd_handler thread, which is evidently fragile enough
that we keep futzing with it.

    Alan> Another approach I looked at briefly to simplify this is emulating
    Alan> pselect in the run loop. It can monitor the read fds, but not the
    Alan> write fds, so I gave up on that.

Is that because macOS doesnʼt send events for fds becoming writeable?

Robert

Footnotes:
[1]  It works currently, but based on my traces we end up sending a
     lot of messages back and forth that are ignored.

--