Subject: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections

From: Eli Zaretskii <[email-masked]>
Date: Sat, 14 Jun 2025 23:01:03 +0300

> From: Steven Allen <[email-masked]>
> Cc: [email-masked], [email-masked], [email-masked]
> Date: Sat, 14 Jun 2025 12:37:38 -0700
> 
> Eli Zaretskii <[email-masked]> writes:
> 
> >> To be clear, calling `accept-process-output' with a nil process is the
> >> exception, not the norm (8 out of 105 call sites).
> >
> > That might be, but in this particular case we changed the call to give
> > it the nil argument for a reason, and it will be hard to convince me
> > to go back on that decision.
> 
> It was changed in 93e1248c2085dfb675d7ed916ec5621e3fe6e2c6 as a blind
> attempt to fix a hard to reproduce bug. The commit message was:
> 
>     * lisp/url/url.el (url-retrieve-synchronously): Use
>     `accept-process-output' on a null process.  That is the safer, more
>     conventional way of achieving non-blocking sleep-for (bug#49897).
> 
> I understand if you want to leave well enough alone, but I also want to
> find SOME way to fix this bug. Which leads us to...
> 
> > So I suggest to explore other ways of solving this first.
> 
> This is what I've been trying to do. There are only two ways to fix
> this:
> 
> 1. Change `accept-process-output' to not unconditionally wait an
> additional 10ms after receiving output when PROCESS is nil "just in
> case" there's some additional data to read.
> 
> 2. Change `url-retrieve-synchronously' pass a non-nil PROCESS to
> `accept-process-output' thereby avoiding the additional 10ms wait.
> 
> I've been trying to go down the first path (I think that *is* the
> correct path) but fixing it will necessarily require
> `accept-process-output` to return immediately after receiving output
> when PROCESS is nil, instead of waiting around in case some additional
> output arrives.

Taking a step back, you never explained how come waiting for 10 msec
causes some operation to take a full second.  Could you please
describe what happens in more details, so that such a significant
effect of such a small delay could be understood without knowing what
emacs-syncthing is or does?

One aspect of this which I'd like to understand is how general is this
problem and what kind of use patterns will bump into it.  Because if
it turns out that just emacs-syncthing is affected, a simple solution
would be for emacs-syncthing to have its own variant of
url-retrieve-synchronously, in which case it can do anything it likes
with its variant of that function.  By contrast, if we want to modify
the code in Emacs, we need at least to understand what use cases need
that and consider their importance vs the other kinds, which might
suffer degradation in performance due to such a change.