Subject: bug#34763: 27.0.50; url-retrieve-synchronously misbehaves inside eldoc-documentation-function

From: Dmitry Gutov <[email-masked]>
Date: Tue, 9 Apr 2019 03:48:17 +0300

On 08.04.2019 20:13, Eli Zaretskii wrote:

>> url-retrieve-synchronously doesn't bother to reset them, just kills the
>> process. Speaking of, shouldn't that be enough for our scenario?
> 
> I think so, yes.

So I guess the first step would be to figure out why 
url-retrieve-synchronously does that isn't enough.

>> Well, if url-http-debug is never entered, its cleanup logic will never
>> be executed. Shouldn't we consider that a problem as well?
> 
> Depends on how the cleanup code will be written.  It can be written
> such that it works regardless where the interruption happens.

Do you have a "good" approach in mind that would still use url-http-debug?

>> I mean, like, interrupted by a different kind of error. Not a signal
>> that's cleanly handled in Emacs's internals.
> 
> A bug, then?

Possibly. Or I imagine something like a response timeout (in an 
already-established connection) could also signal some kind of error.

> That'd also be a bug, IMO.  We can expect bugs to behave abnormally.

Sometimes it's useful to perform cleanup properly anyway. If feasible.

>> And as for "bug of the code", I'm saying that there must be some code
>> that can hog the CPU (the comment refers to it), and we might want to
>> handle that carefully.
> 
> A simple C-g should theoretically take handle that carefully.

That also speaks for removing the cleanup code from url-http-debug, I think.

 >> Simply waiting for a some amount of time tends to get the problem
 >> "unstuck", though the improvement is gradual and fairly unpredictable.
 >
 > Is it related in any way with the outstanding connections being
 > completed/closed?  What does netstat show?

Feel free to suggest a particular invocation, but here's some results. 
I'm calling it while filtering out some irrelevant programs:

netstat -tp --wide | grep -v qbittorrent | grep -v ...

Output at the beginning, when everything seems okay:

Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address 
State       PID/Program name
tcp        0    300 zappa:53781 
cust-138-33-109-94.dyn.as47377.net:23048 FIN_WAIT1   -
tcp        0      0 zappa:54612 
mrs08s03-in-f10.1e100.net:https TIME_WAIT   -
tcp        0      1 zappa:33575 
ec2-3-83-213-10.compute-1.amazonaws.com:49839 FIN_WAIT1   - 

tcp        0      1 zappa:55515 
svaio-12.dyn.pool1.garodo.net:6881 SYN_SENT    -
tcp        0     69 zappa:48829 
cust-138-33-109-94.dyn.as47377.net:20142 FIN_WAIT1   -

Here's the output right after I repro'd the problem:
(Not all processes could be identified, non-owned process info
  will not be shown, you would have to be root to see it all.)
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address 
State       PID/Program name
tcp        0     69 zappa:44551 
cust-138-33-109-94.dyn.as47377.net:20142 FIN_WAIT1   -
tcp        0      0 zappa:50064 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50066 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55858 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55930 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50076 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55928 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      1 zappa:48910 
ns345059.ip-176-31-225.eu:http LAST_ACK    -
tcp        0      0 zappa:55926 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55856 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:38389 
nz184l178.bb122100.ctm.net:32766 TIME_WAIT   -
tcp        0    516 zappa:46189 
S0106f0f249850973.wp.shawcable.net:62318 LAST_ACK    -
tcp        0      0 zappa:50062 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55920 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50068 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50070 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55922 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55832 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55924 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:34997 
ec2-3-83-213-10.compute-1.amazonaws.com:49839 FIN_WAIT2   - 

tcp        0      0 zappa:50074 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50072 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs

Here it is after a few minutes has passed:

(Not all processes could be identified, non-owned process info
  will not be shown, you would have to be root to see it all.)
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address 
State       PID/Program name
tcp        0     69 zappa:44551 
cust-138-33-109-94.dyn.as47377.net:20142 FIN_WAIT1   -
tcp        0      0 zappa:50064 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50066 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55858 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55930 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50076 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55928 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:54792 
mrs08s03-in-f10.1e100.net:https TIME_WAIT   -
tcp        0      0 zappa:55926 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55856 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50062 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55920 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50068 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50070 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55922 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55832 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:55924 
mrs08s04-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50074 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs
tcp        0      0 zappa:50072 
mrs08s02-in-f4.1e100.net:http ESTABLISHED 4594/src/emacs

The list gets shorter over time.