Subject: [PATCH] Speedup url-retrieve-synchronously for low-latency connections

From: Steven Allen <[email-masked]>
Date: Wed, 11 Jun 2025 21:08:45 -0700

Tags: patch

**Present context:**

I'm running into an issue in emacs-syncthing [1] where making a few
localhost network requests are taking a full second (blocking Emacs)
when they should be instantaneous. While digging into
`url-retrieve-synchronously', I found that waiting on the correct
network process instead of passing nil to `accept-process-output' made
these network requests instantaneous (although I have no idea why). See
the attached patch.

[1]: https://github.com/KeyWeeUsr/emacs-syncthing

To test this patch, you can:

1. Run a simple web server on localhost. I usually use

    python -m http.server --bind 127.0.0.1 8000

2. Evaluate:  (benchmark 100 '(url-retrieve-synchronously "http://127.0.0.1:8000"))

With this patch, this form is ~16x faster on my machine. I've also
tested this against a remote machine with a ~45ms latency and found a
1.25x speedup.

I've confirmed that this isn't busy-waiting by modifying this code to
print a message each time it loops: it loops the same number of times
with or without my patch.

I've confirmed that this speedup is strictly due to passing the target
process to `accept-process-output' by applying my patch then changing
JUST "proc" to "nil":

                    ;; ms, so split the difference.
                    (accept-process-output proc 0.05))

to

                    ;; ms, so split the difference.
                    (accept-process-output nil 0.05))

With this one change, this code goes back to being as slow as it was before.

**Historical context**

`url-retrieve-synchronously' was changed to wait on "nil" in:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49897,

Motivated by this bug report:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49861

However, the patch in question also changed the rest of
`url-retrieve-synchronously' so I'm hoping the issue lies elsewhere?

In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, cairo version
 1.18.4) of 2025-06-11 built on Laptop
Repository revision: 87244ef1661f9e9d7c08f65047551e8a34cd92b2
Repository branch: makepkg
Windowing system distributor 'The X.Org Foundation', version 11.0.12101016
System Description: Arch Linux

Configured using:
 'configure
 'CPPFLAGS=-I/run/user/1000/build/emacs-git/src/mps-git/build/include '
 'LDFLAGS=-L/run/user/1000/build/emacs-git/src/mps-git/build/lib -Wl,-O1
 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
 -Wl,-z,pack-relative-relocs -flto=auto' --prefix=/usr --sysconfdir=/etc
 --libexecdir=/usr/lib --localstatedir=/var --mandir=/usr/share/man
 --with-gameuser=:games --with-modules --without-m17n-flt
 --without-selinux --without-pop --without-gconf --disable-gc-mark-trace
 --with-mps=yes --enable-link-time-optimization
 --with-native-compilation=yes --with-xinput2 --with-x-toolkit=no
 --without-toolkit-scroll-bars --without-xaw3d --without-gsettings
 --with-cairo-xcb --without-xft --with-sound=no --with-tree-sitter
 --without-gpm --without-compress-install
 '--program-transform-name=s/\([ec]tags\)/\1.emacs/'
 'CFLAGS=-march=native -mtune=native -O3 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection -fomit-frame-pointer
 -fno-math-errno -fno-trapping-math -fno-math-errno -fno-trapping-math
 -flto=auto''

>From 4529f63f3c55b40fd43642592cd4dc7162766c7f Mon Sep 17 00:00:00 2001
From: Steven Allen <[email-masked]>
Date: Wed, 11 Jun 2025 20:19:01 -0700
Subject: [PATCH] Speedup url-retrieve-synchronously for low-latency
 connections

On my machine, this provides a ~16x speedup for localhost request and a
1.28x speedup for a 50ms latency connection.

* lisp/url/url.el (url-retrieve-synchronously): Wait for output from the
target process instead of waiting for output from any process.
---
 lisp/url/url.el | 23 +++++++++++++----------
 1 file changed, 13 insertions(+), 10 deletions(-)

diff --git a/lisp/url/url.el b/lisp/url/url.el
index 090a952cf4c..c8dc002b0ac 100644
--- a/lisp/url/url.el
+++ b/lisp/url/url.el
@@ -270,17 +270,20 @@ url-retrieve-synchronously
                     (kill-buffer proc-buffer))
                   ;; Accommodate hack in commit 55d1d8b.
                   (setq proc-buffer redirect-buffer)))
-              (when-let* ((proc (get-buffer-process proc-buffer)))
-                (when (memq (process-status proc)
+              (if-let* ((proc (get-buffer-process proc-buffer)))
+                  (if (memq (process-status proc)
                             '(closed exit signal failed))
-                  ;; Process sentinel vagaries occasionally cause
-                  ;; url-retrieve to fail calling callback.
-                  (unless data-buffer
-                    (url-debug 'retrieval "Dead process %s" url)
-		    (throw 'done 'exception))))
-              ;; Querying over consumer internet in the US takes 100
-              ;; ms, so split the difference.
-              (accept-process-output nil 0.05)))
+                      ;; Process sentinel vagaries occasionally cause
+                      ;; url-retrieve to fail calling callback.
+                      (unless data-buffer
+                        (url-debug 'retrieval "Dead process %s" url)
+                        (throw 'done 'exception))
+                    ;; Querying over consumer internet in the US takes 100
+                    ;; ms, so split the difference.
+                    (accept-process-output proc 0.05))
+                ;; This case shouldn't happen but this keeps us from
+                ;; spinning and locking up Emacs if it does.
+                (accept-process-output nil 0.05))))
         ;; Kill the process buffer on redirects.
         (when (and data-buffer
                    (not (eq data-buffer proc-buffer)))
-- 
2.49.0