Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections

From: Steven Allen <[email-masked]>
Date: Thu, 12 Jun 2025 13:54:01 -0700

Steven Allen <[email-masked]> writes:

> Eli Zaretskii <[email-masked]> writes:
>
>>> From: Steven Allen <[email-masked]>
>>> Cc: [email-masked], [email-masked]
>>> Date: Thu, 12 Jun 2025 11:02:41 -0700
>>>
>>> > Perhaps instrumenting wait_reading_process_output with printf's would
>>> > help to understand the control flow in the case of nil and non-nil
>>> > PROCESS argument?
>>>
>>> I'll do that today. I have a suspicion that fast requests waiting on a
>>> single process exit early here:
>>>
>>> https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5562
>>
>> The condition for that block is
>>
>>       if (wait_proc
>> 	  && ! EQ (wait_proc->status, Qrun)
>> 	  && ! connecting_status (wait_proc->status))
>>
>> And the comment says "Don't wait for output from a non-running
>> process."  Is the case here that the network connection was already
>> closed when we read from the sub-process?  I thought that was not the
>> case.
>
> With my patch, it does hit that case, but on the third loop through so
> it's not exiting early.
>
> I also tried setting the timeout in `accept-process-output' to nil and
> that got me the same speedup as my fix but ended up looping 23 times so
> there's something really strange going on with the timeout here.

Ok, I think I've figured it out. We loop twice and wait on a timeout
(READ_OUTPUT_DELAY_INCREMENT) the second loop because we're already done
reading.

The first time through, we hit the following, recording that we've
gotten some output:

  https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5974

We hit the following, setting our timeout to READ_OUTPUT_DELAY_INCREMENT:

  https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5679

Then we hit the select call and wait the full timeout (nothing to read)
and we finally exit here (because our timeout has passed):

  https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5832

***

I think what we need to do is modify:

  https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5978

from

    		  if (wait_proc == XPROCESS (proc))
		    wait = MINIMUM;

to

    		  if (!wait_proc || wait_proc == XPROCESS (proc))
		    wait = MINIMUM;

So that we set the timeout to 0 here:

  https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5439

And exit early.

>From 0772d96d50734fff14a17b2f207661418f5c814c Mon Sep 17 00:00:00 2001
From: Steven Allen <[email-masked]>
Date: Thu, 12 Jun 2025 13:48:08 -0700
Subject: [PATCH] Exit accept-process-output early when we get output from any
 process

Previously, the caller requested to wait on output from any process (not
a specific process), we'd always end up waiting
10ms (READ_OUTPUT_DELAY_INCREMENT) one final time before returning.

* src/process.c (wait_reading_process_output): abort waiting early if we
either receive output from the process we were waiting on OR we weren't
waiting on a process.
---
 src/process.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/process.c b/src/process.c
index e61ec425f7e..7b8fd1357e3 100644
--- a/src/process.c
+++ b/src/process.c
@@ -5975,7 +5975,7 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
 	      if (nread > 0)
 		{
 		  /* Vacuum up any leftovers without waiting.  */
-		  if (wait_proc == XPROCESS (proc))
+		  if (!wait_proc || wait_proc == XPROCESS (proc))
 		    wait = MINIMUM;
 		  /* Since read_process_output can run a filter,
 		     which can call accept-process-output,
-- 
2.49.0