GNU bug report logs - #79228
30.1.90; native--compile-async sentinel can error if threads are running

Previous Next

Package: emacs;

Reported by: Spencer Baugh <sbaugh <at> janestreet.com>

Date: Tue, 12 Aug 2025 21:57:01 UTC

Severity: normal

Found in version 30.1.90

Done: Eli Zaretskii <eliz <at> gnu.org>

To reply to this bug, email your comments to 79228 AT debbugs.gnu.org.
There is no need to reopen the bug first.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to johnw <at> gnu.org, bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Tue, 12 Aug 2025 21:57:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Spencer Baugh <sbaugh <at> janestreet.com>:
New bug report received and forwarded. Copy sent to johnw <at> gnu.org, bug-gnu-emacs <at> gnu.org. (Tue, 12 Aug 2025 21:57:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.1.90; native--compile-async sentinel can error if threads are
 running
Date: Tue, 12 Aug 2025 17:56:29 -0400
Other threads can run pending_funcalls if they call
accept-process-output.  This should probably not be allowed.

A specific bug is that the C core calls native--compile-async using
pending_funcalls.  If that runs on another thread, then the processes
created in native--compile-async are locked to that other thread.  Then
the sentinels for those processes error when they call
accept-process-output on the processes while on yet another thrad.  Or
something like that; the sentinels were definitely erroring with the
"Attempt to accept output from process %s locked to thread %s" error.

This can be reproduced somewhat reliably by causing the C code to start
async native compilation while there are other threads running.  Sorry
for not making a full reproduction, but I think it's fairly
self-explanatory, and having pending_funcalls run on other threads is
clearly wrong.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Wed, 13 Aug 2025 11:43:02 GMT) Full text and rfc822 format available.

Message #8 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Spencer Baugh <sbaugh <at> janestreet.com>,
 Andrea Corallo <acorallo <at> gnu.org>
Cc: johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90;
 native--compile-async sentinel can error if threads are running
Date: Wed, 13 Aug 2025 14:41:46 +0300
> Cc: John Wiegley <johnw <at> gnu.org>
> Date: Tue, 12 Aug 2025 17:56:29 -0400
> From:  Spencer Baugh via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> 
> 
> Other threads can run pending_funcalls if they call
> accept-process-output.  This should probably not be allowed.
> 
> A specific bug is that the C core calls native--compile-async using
> pending_funcalls.  If that runs on another thread, then the processes
> created in native--compile-async are locked to that other thread.  Then
> the sentinels for those processes error when they call
> accept-process-output on the processes while on yet another thrad.  Or
> something like that; the sentinels were definitely erroring with the
> "Attempt to accept output from process %s locked to thread %s" error.
> 
> This can be reproduced somewhat reliably by causing the C code to start
> async native compilation while there are other threads running.  Sorry
> for not making a full reproduction, but I think it's fairly
> self-explanatory, and having pending_funcalls run on other threads is
> clearly wrong.

Can't say I understood the scenario, but then I'm not an expert on
native-compilation's low-level details.  Adding Andrea, in the hope
that he understands better.

(How can native--compile-async run "on another thread"?)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Sat, 30 Aug 2025 07:32:02 GMT) Full text and rfc822 format available.

Message #11 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: sbaugh <at> janestreet.com, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90;
 native--compile-async sentinel can error if threads are running
Date: Sat, 30 Aug 2025 10:31:30 +0300
Ping!  Andrea, could you please look into this?

> Cc: johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
> Date: Wed, 13 Aug 2025 14:41:46 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
> > Cc: John Wiegley <johnw <at> gnu.org>
> > Date: Tue, 12 Aug 2025 17:56:29 -0400
> > From:  Spencer Baugh via "Bug reports for GNU Emacs,
> >  the Swiss army knife of text editors" <bug-gnu-emacs <at> gnu.org>
> > 
> > 
> > Other threads can run pending_funcalls if they call
> > accept-process-output.  This should probably not be allowed.
> > 
> > A specific bug is that the C core calls native--compile-async using
> > pending_funcalls.  If that runs on another thread, then the processes
> > created in native--compile-async are locked to that other thread.  Then
> > the sentinels for those processes error when they call
> > accept-process-output on the processes while on yet another thrad.  Or
> > something like that; the sentinels were definitely erroring with the
> > "Attempt to accept output from process %s locked to thread %s" error.
> > 
> > This can be reproduced somewhat reliably by causing the C code to start
> > async native compilation while there are other threads running.  Sorry
> > for not making a full reproduction, but I think it's fairly
> > self-explanatory, and having pending_funcalls run on other threads is
> > clearly wrong.
> 
> Can't say I understood the scenario, but then I'm not an expert on
> native-compilation's low-level details.  Adding Andrea, in the hope
> that he understands better.
> 
> (How can native--compile-async run "on another thread"?)
> 
> 
> 
> 




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Mon, 01 Sep 2025 07:53:02 GMT) Full text and rfc822 format available.

Message #14 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Andrea Corallo <acorallo <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: sbaugh <at> janestreet.com, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Mon, 01 Sep 2025 03:51:43 -0400
Eli Zaretskii <eliz <at> gnu.org> writes:

Hi Eli,

must say I, like you, still don't understand the issue here.

Spencer which threads are we talking about?  Native async compilation
spawn processes not threads.

Thanks!

  Andrea




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Mon, 01 Sep 2025 12:56:02 GMT) Full text and rfc822 format available.

Message #17 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Andrea Corallo <acorallo <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Mon, 1 Sep 2025 08:54:58 -0400
[Message part 1 (text/plain, inline)]
On Mon, Sep 1, 2025, 3:52 AM Andrea Corallo <acorallo <at> gnu.org> wrote:

> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> Hi Eli,
>
> must say I, like you, still don't understand the issue here.
>
> Spencer which threads are we talking about?  Native async compilation
> spawn processes not threads.
>

Lisp threads - the thread API in Emacs Lisp.

>
[Message part 2 (text/html, inline)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Mon, 01 Sep 2025 15:13:02 GMT) Full text and rfc822 format available.

Message #20 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Mon, 01 Sep 2025 18:11:49 +0300
> From: Spencer Baugh <sbaugh <at> janestreet.com>
> Date: Mon, 1 Sep 2025 08:54:58 -0400
> Cc: Eli Zaretskii <eliz <at> gnu.org>, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
> 
> On Mon, Sep 1, 2025, 3:52 AM Andrea Corallo <acorallo <at> gnu.org> wrote:
> 
>  Eli Zaretskii <eliz <at> gnu.org> writes:
> 
>  Hi Eli,
> 
>  must say I, like you, still don't understand the issue here.
> 
>  Spencer which threads are we talking about?  Native async compilation
>  spawn processes not threads.
> 
> Lisp threads - the thread API in Emacs Lisp.

Are you talking about some 3rd-party package?  If not, how come
threads come into the picture when native-compilation is involved?

IOW, would you please describe the situation from its very beginning,
assuming someone runs something from "emacs -Q"?  Otherwise, it is
very hard to understand what kind of situations could cause async
processes running native compilation to be locked to the wrong
threads.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Mon, 01 Sep 2025 15:22:01 GMT) Full text and rfc822 format available.

Message #23 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Mon, 01 Sep 2025 11:20:58 -0400
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> Date: Mon, 1 Sep 2025 08:54:58 -0400
>> Cc: Eli Zaretskii <eliz <at> gnu.org>, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
>> 
>> On Mon, Sep 1, 2025, 3:52 AM Andrea Corallo <acorallo <at> gnu.org> wrote:
>> 
>>  Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>>  Hi Eli,
>> 
>>  must say I, like you, still don't understand the issue here.
>> 
>>  Spencer which threads are we talking about?  Native async compilation
>>  spawn processes not threads.
>> 
>> Lisp threads - the thread API in Emacs Lisp.
>
> Are you talking about some 3rd-party package?  If not, how come
> threads come into the picture when native-compilation is involved?
>
> IOW, would you please describe the situation from its very beginning,
> assuming someone runs something from "emacs -Q"?  Otherwise, it is
> very hard to understand what kind of situations could cause async
> processes running native compilation to be locked to the wrong
> threads.

1. emacs -Q
2. Run some command which starts a thread
3. Run some command which starts native compilation

Native compilation is done via pending_funcalls.  pending_funcalls can
be run in the non-main thread started by 2.  Thus the native compilation
can happen in a non-main thread.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Mon, 01 Sep 2025 16:00:02 GMT) Full text and rfc822 format available.

Message #26 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Mon, 01 Sep 2025 18:59:39 +0300
> From: Spencer Baugh <sbaugh <at> janestreet.com>
> Cc: acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
> Date: Mon, 01 Sep 2025 11:20:58 -0400
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Spencer Baugh <sbaugh <at> janestreet.com>
> >> Date: Mon, 1 Sep 2025 08:54:58 -0400
> >> Cc: Eli Zaretskii <eliz <at> gnu.org>, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
> >> 
> >> On Mon, Sep 1, 2025, 3:52 AM Andrea Corallo <acorallo <at> gnu.org> wrote:
> >> 
> >>  Eli Zaretskii <eliz <at> gnu.org> writes:
> >> 
> >>  Hi Eli,
> >> 
> >>  must say I, like you, still don't understand the issue here.
> >> 
> >>  Spencer which threads are we talking about?  Native async compilation
> >>  spawn processes not threads.
> >> 
> >> Lisp threads - the thread API in Emacs Lisp.
> >
> > Are you talking about some 3rd-party package?  If not, how come
> > threads come into the picture when native-compilation is involved?
> >
> > IOW, would you please describe the situation from its very beginning,
> > assuming someone runs something from "emacs -Q"?  Otherwise, it is
> > very hard to understand what kind of situations could cause async
> > processes running native compilation to be locked to the wrong
> > threads.
> 
> 1. emacs -Q
> 2. Run some command which starts a thread
> 3. Run some command which starts native compilation
> 
> Native compilation is done via pending_funcalls.  pending_funcalls can
> be run in the non-main thread started by 2.

How?

> Thus the native compilation can happen in a non-main thread.

So step 3 is not necessary?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Mon, 01 Sep 2025 16:04:01 GMT) Full text and rfc822 format available.

Message #29 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Mon, 01 Sep 2025 12:03:21 -0400
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> Cc: acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
>> Date: Mon, 01 Sep 2025 11:20:58 -0400
>> 
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>> >> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> >> Date: Mon, 1 Sep 2025 08:54:58 -0400
>> >> Cc: Eli Zaretskii <eliz <at> gnu.org>, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
>> >> 
>> >> On Mon, Sep 1, 2025, 3:52 AM Andrea Corallo <acorallo <at> gnu.org> wrote:
>> >> 
>> >>  Eli Zaretskii <eliz <at> gnu.org> writes:
>> >> 
>> >>  Hi Eli,
>> >> 
>> >>  must say I, like you, still don't understand the issue here.
>> >> 
>> >>  Spencer which threads are we talking about?  Native async compilation
>> >>  spawn processes not threads.
>> >> 
>> >> Lisp threads - the thread API in Emacs Lisp.
>> >
>> > Are you talking about some 3rd-party package?  If not, how come
>> > threads come into the picture when native-compilation is involved?
>> >
>> > IOW, would you please describe the situation from its very beginning,
>> > assuming someone runs something from "emacs -Q"?  Otherwise, it is
>> > very hard to understand what kind of situations could cause async
>> > processes running native compilation to be locked to the wrong
>> > threads.
>> 
>> 1. emacs -Q
>> 2. Run some command which starts a thread
>> 3. Run some command which starts native compilation
>> 
>> Native compilation is done via pending_funcalls.  pending_funcalls can
>> be run in the non-main thread started by 2.
>
> How?

Because pending_funcalls is run by timer_check_2, which is run by
timer_check, which is run by wait_reading_process_output, which can be
run by non-main threads.

>> Thus the native compilation can happen in a non-main thread.
>
> So step 3 is not necessary?

It's necesary because, of course, if nothing causes native compilation
to happen, native compilation won't happen.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Mon, 01 Sep 2025 19:08:01 GMT) Full text and rfc822 format available.

Message #32 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Mon, 01 Sep 2025 22:07:16 +0300
> From: Spencer Baugh <sbaugh <at> janestreet.com>
> Cc: acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
> Date: Mon, 01 Sep 2025 12:03:21 -0400
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> 1. emacs -Q
> >> 2. Run some command which starts a thread
> >> 3. Run some command which starts native compilation
> >> 
> >> Native compilation is done via pending_funcalls.  pending_funcalls can
> >> be run in the non-main thread started by 2.
> >
> > How?
> 
> Because pending_funcalls is run by timer_check_2, which is run by
> timer_check, which is run by wait_reading_process_output, which can be
> run by non-main threads.

So maybe we should run timers only in the main thread?

> >> Thus the native compilation can happen in a non-main thread.
> >
> > So step 3 is not necessary?
> 
> It's necesary because, of course, if nothing causes native compilation
> to happen, native compilation won't happen.

Native compilation can be triggered by loading a Lisp package which
doesn't yet have a .eln file.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Mon, 01 Sep 2025 19:12:01 GMT) Full text and rfc822 format available.

Message #35 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Dmitry Gutov <dmitry <at> gutov.dev>, acorallo <at> gnu.org, johnw <at> gnu.org,
 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Mon, 01 Sep 2025 15:10:59 -0400
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> Cc: acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
>> Date: Mon, 01 Sep 2025 12:03:21 -0400
>> 
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>> >> 1. emacs -Q
>> >> 2. Run some command which starts a thread
>> >> 3. Run some command which starts native compilation
>> >> 
>> >> Native compilation is done via pending_funcalls.  pending_funcalls can
>> >> be run in the non-main thread started by 2.
>> >
>> > How?
>> 
>> Because pending_funcalls is run by timer_check_2, which is run by
>> timer_check, which is run by wait_reading_process_output, which can be
>> run by non-main threads.
>
> So maybe we should run timers only in the main thread?

That would break, for example, with-timeout.

Alternatively, if we stopped locking processes to threads, this problem
would also be fixed.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Tue, 02 Sep 2025 11:15:02 GMT) Full text and rfc822 format available.

Message #38 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: dmitry <at> gutov.dev, acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Tue, 02 Sep 2025 14:14:15 +0300
> From: Spencer Baugh <sbaugh <at> janestreet.com>
> Cc: acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org, Dmitry Gutov
>   <dmitry <at> gutov.dev>
> Date: Mon, 01 Sep 2025 15:10:59 -0400
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Spencer Baugh <sbaugh <at> janestreet.com>
> >> Cc: acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
> >> Date: Mon, 01 Sep 2025 12:03:21 -0400
> >> 
> >> Eli Zaretskii <eliz <at> gnu.org> writes:
> >> 
> >> >> 1. emacs -Q
> >> >> 2. Run some command which starts a thread
> >> >> 3. Run some command which starts native compilation
> >> >> 
> >> >> Native compilation is done via pending_funcalls.  pending_funcalls can
> >> >> be run in the non-main thread started by 2.
> >> >
> >> > How?
> >> 
> >> Because pending_funcalls is run by timer_check_2, which is run by
> >> timer_check, which is run by wait_reading_process_output, which can be
> >> run by non-main threads.
> >
> > So maybe we should run timers only in the main thread?
> 
> That would break, for example, with-timeout.

So?  I'm not at all surprised that timers and threads, two very
different ways of having the same pseudo-parallel execution in Emacs,
do not live together in peace.

> Alternatively, if we stopped locking processes to threads, this problem
> would also be fixed.

That's too radical, and has too many downsides, as already discussed.
And it's unjustified, since if worse comes to worst, we can simply
unlock the process that performs async compilation.  No need to reach
far-fetched conclusions about processes in general because of this
specific case.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Tue, 02 Sep 2025 13:19:01 GMT) Full text and rfc822 format available.

Message #41 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmitry <at> gutov.dev, acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Tue, 02 Sep 2025 09:18:12 -0400
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> Cc: acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org, Dmitry Gutov
>>   <dmitry <at> gutov.dev>
>> Date: Mon, 01 Sep 2025 15:10:59 -0400
>> 
>> Eli Zaretskii <eliz <at> gnu.org> writes:
>> 
>> >> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> >> Cc: acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
>> >> Date: Mon, 01 Sep 2025 12:03:21 -0400
>> >> 
>> >> Eli Zaretskii <eliz <at> gnu.org> writes:
>> >> 
>> >> >> 1. emacs -Q
>> >> >> 2. Run some command which starts a thread
>> >> >> 3. Run some command which starts native compilation
>> >> >> 
>> >> >> Native compilation is done via pending_funcalls.  pending_funcalls can
>> >> >> be run in the non-main thread started by 2.
>> >> >
>> >> > How?
>> >> 
>> >> Because pending_funcalls is run by timer_check_2, which is run by
>> >> timer_check, which is run by wait_reading_process_output, which can be
>> >> run by non-main threads.
>> >
>> > So maybe we should run timers only in the main thread?
>> 
>> That would break, for example, with-timeout.
>
> So?  I'm not at all surprised that timers and threads, two very
> different ways of having the same pseudo-parallel execution in Emacs,
> do not live together in peace.
>
>> Alternatively, if we stopped locking processes to threads, this problem
>> would also be fixed.
>
> That's too radical, and has too many downsides, as already discussed.
> And it's unjustified, since if worse comes to worst, we can simply
> unlock the process that performs async compilation.  No need to reach
> far-fetched conclusions about processes in general because of this
> specific case.

Okay, then this patch should fix the bug.

[0001-Allow-any-thread-to-run-native-compilation.patch (text/x-patch, inline)]
From d0f0e4b5471d188b483c35eb69086f2de60b5405 Mon Sep 17 00:00:00 2001
From: Spencer Baugh <sbaugh <at> janestreet.com>
Date: Tue, 2 Sep 2025 09:17:06 -0400
Subject: [PATCH] Allow any thread to run native compilation

* lisp/emacs-lisp/comp-run.el (comp--run-async-workers): Don't
lock the native compilation process to a thread (bug#79228)
---
 lisp/emacs-lisp/comp-run.el | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lisp/emacs-lisp/comp-run.el b/lisp/emacs-lisp/comp-run.el
index 1761190ea17..5e33b1bc318 100644
--- a/lisp/emacs-lisp/comp-run.el
+++ b/lisp/emacs-lisp/comp-run.el
@@ -366,6 +366,7 @@ comp--run-async-workers
                                                       (eq load1 'late))))
                                (comp--run-async-workers))
                              :noquery (not native-comp-async-query-on-exit))))
+              (set-process-thread process nil)
               (puthash source-file process comp-async-compilations))
          when (>= (comp--async-runnings) (comp--effective-async-max-jobs))
          do (cl-return)))
-- 
2.43.7


Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Tue, 02 Sep 2025 14:22:02 GMT) Full text and rfc822 format available.

Message #44 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: dmitry <at> gutov.dev, acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Tue, 02 Sep 2025 17:21:16 +0300
> From: Spencer Baugh <sbaugh <at> janestreet.com>
> Cc: dmitry <at> gutov.dev,  acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
> Date: Tue, 02 Sep 2025 09:18:12 -0400
> 
> > And it's unjustified, since if worse comes to worst, we can simply
> > unlock the process that performs async compilation.  No need to reach
> > far-fetched conclusions about processes in general because of this
> > specific case.
> 
> Okay, then this patch should fix the bug.

But does it?  You've brought up the issue, so presumably you have a
way of recreating the problem.  Then please try this patch and tell
whether it indeed solves the problem.

Thanks.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#79228; Package emacs. (Tue, 02 Sep 2025 14:42:02 GMT) Full text and rfc822 format available.

Message #47 received at 79228 <at> debbugs.gnu.org (full text, mbox):

From: Spencer Baugh <sbaugh <at> janestreet.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: dmitry <at> gutov.dev, acorallo <at> gnu.org, johnw <at> gnu.org, 79228 <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Tue, 02 Sep 2025 10:40:53 -0400
Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Spencer Baugh <sbaugh <at> janestreet.com>
>> Cc: dmitry <at> gutov.dev,  acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
>> Date: Tue, 02 Sep 2025 09:18:12 -0400
>> 
>> > And it's unjustified, since if worse comes to worst, we can simply
>> > unlock the process that performs async compilation.  No need to reach
>> > far-fetched conclusions about processes in general because of this
>> > specific case.
>> 
>> Okay, then this patch should fix the bug.
>
> But does it?  You've brought up the issue, so presumably you have a
> way of recreating the problem.

Not easily, but fine, I did some more work to figure out a reliable
reproduction:

1. Delete eln-cache
2. emacs -Q --eval '(dotimes (_ 5) (make-thread (lambda () (while t (sit-for 1)))))'
3. Observe errors in *Messages* from native compilation

> Then please try this patch and tell whether it indeed solves the
> problem.

Yes, it does.




Reply sent to Eli Zaretskii <eliz <at> gnu.org>:
You have taken responsibility. (Tue, 02 Sep 2025 15:09:01 GMT) Full text and rfc822 format available.

Notification sent to Spencer Baugh <sbaugh <at> janestreet.com>:
bug acknowledged by developer. (Tue, 02 Sep 2025 15:09:02 GMT) Full text and rfc822 format available.

Message #52 received at 79228-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Spencer Baugh <sbaugh <at> janestreet.com>
Cc: dmitry <at> gutov.dev, acorallo <at> gnu.org, johnw <at> gnu.org,
 79228-done <at> debbugs.gnu.org
Subject: Re: bug#79228: 30.1.90; native--compile-async sentinel can error if
 threads are running
Date: Tue, 02 Sep 2025 18:07:33 +0300
> From: Spencer Baugh <sbaugh <at> janestreet.com>
> Cc: dmitry <at> gutov.dev,  acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
> Date: Tue, 02 Sep 2025 10:40:53 -0400
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Spencer Baugh <sbaugh <at> janestreet.com>
> >> Cc: dmitry <at> gutov.dev,  acorallo <at> gnu.org,  johnw <at> gnu.org,  79228 <at> debbugs.gnu.org
> >> Date: Tue, 02 Sep 2025 09:18:12 -0400
> >> 
> >> > And it's unjustified, since if worse comes to worst, we can simply
> >> > unlock the process that performs async compilation.  No need to reach
> >> > far-fetched conclusions about processes in general because of this
> >> > specific case.
> >> 
> >> Okay, then this patch should fix the bug.
> >
> > But does it?  You've brought up the issue, so presumably you have a
> > way of recreating the problem.
> 
> Not easily, but fine, I did some more work to figure out a reliable
> reproduction:
> 
> 1. Delete eln-cache
> 2. emacs -Q --eval '(dotimes (_ 5) (make-thread (lambda () (while t (sit-for 1)))))'
> 3. Observe errors in *Messages* from native compilation
> 
> > Then please try this patch and tell whether it indeed solves the
> > problem.
> 
> Yes, it does.

Thanks, installed on master, and closing the bug.




This bug report was last modified 4 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.