Subject: bug#19860: closed (Re: bug#19860: 25.0.50; One url-retrieve-synchronously call running concurrently with another can freeze it)

From: [email-masked] (GNU bug Tracking System)
Date: Sat, 25 Apr 2015 23:21:02 +0000

Your bug report

#19860: 25.0.50; One url-retrieve-synchronously call running concurrently with another can freeze it

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
19860: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19860
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#19860: 25.0.50; One url-retrieve-synchronously call running concurrently with another can freeze it
From: Paul Eggert <[email-masked]>
To: Dmitry Gutov <[email-masked]>, [email-masked]
Date: Sat, 25 Apr 2015 16:19:52 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

Dmitry Gutov wrote:

> 'git bisect' points to 12dc5429352223f7ba8314d2e16177036a762733.
>
> Paul, could you look into this?

Thanks for doing all that legwork.  It let me reproduce the problem and come up 
with a simple fix that works for me.  I installed the attached patch.

>From 9ca712e2689f1875d7facd229f2cb1916719cf49 Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Sat, 25 Apr 2015 16:16:58 -0700
Subject: [PATCH] Don't freeze with unreadable processes

Don't freeze if an exiting process can't be read from. (Bug#19860).
This fixes a bug I introduced in
2014-07-08T07:24:07Z@[email-masked]
"* process.c: Add sanity checks for file descriptors."
Dmitry Gutov did most of the legwork in finding the problem.
* src/process.c (wait_reading_process_output):
Treat non-running processes that can't be read from
the same as other non-running processes.
---
 src/process.c | 40 ++++++++++++++++++++++------------------
 1 file changed, 22 insertions(+), 18 deletions(-)

diff --git a/src/process.c b/src/process.c
index c745507..3e04cb7 100644
--- a/src/process.c
+++ b/src/process.c
@@ -4785,37 +4785,41 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
       if (wait_proc && wait_proc->raw_status_new)
 	update_status (wait_proc);
       if (wait_proc
-	  && wait_proc->infd >= 0
 	  && ! EQ (wait_proc->status, Qrun)
 	  && ! EQ (wait_proc->status, Qconnect))
 	{
 	  bool read_some_bytes = false;
 
 	  clear_waiting_for_input ();
-	  XSETPROCESS (proc, wait_proc);
 
-	  /* Read data from the process, until we exhaust it.  */
-	  while (true)
+	  /* If data can be read from the process, do so until exhausted.  */
+	  if (wait_proc->infd >= 0)
 	    {
-	      int nread = read_process_output (proc, wait_proc->infd);
-	      if (nread < 0)
+	      XSETPROCESS (proc, wait_proc);
+
+	      while (true)
 		{
-		  if (errno == EIO || errno == EAGAIN)
-		    break;
+		  int nread = read_process_output (proc, wait_proc->infd);
+		  if (nread < 0)
+		    {
+		    if (errno == EIO || errno == EAGAIN)
+		      break;
 #ifdef EWOULDBLOCK
-		  if (errno == EWOULDBLOCK)
-		    break;
+		    if (errno == EWOULDBLOCK)
+		      break;
 #endif
-		}
-	      else
-		{
-		  if (got_some_input < nread)
-		    got_some_input = nread;
-		  if (nread == 0)
-		    break;
-		  read_some_bytes = true;
+		    }
+		  else
+		    {
+		      if (got_some_input < nread)
+			got_some_input = nread;
+		      if (nread == 0)
+			break;
+		      read_some_bytes = true;
+		    }
 		}
 	    }
+
 	  if (read_some_bytes && do_display)
 	    redisplay_preserve_echo_area (10);
 
-- 
2.1.4

Subject: 25.0.50; One url-retrieve-synchronously call running concurrently with another can freeze it
From: Dmitry Gutov <[email-masked]>
To: [email-masked]
Date: Sat, 14 Feb 2015 02:09:57 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

This applies both to master and emacs-24 (with a caveat).

With sufficiently suitable server, calling long-ops-now or long-ops-idle
can lead to the "outer" request never returning:

--8<---------------cut here---------------start------------->8---
(defun long-ops-idle ()
  (interactive)
  (run-with-idle-timer 0.2 nil #'long-outer-op)
  (run-with-idle-timer 0.5 nil #'long-inner-op))

(defun long-ops-now ()
  (interactive)
  (run-with-timer 0.2 nil #'long-inner-op)
  (long-outer-op))

(defun long-outer-op ()
  (message "retrieving outer")
  (url-retrieve-synchronously "http://localhost:9292")
  (message "outer retrieved"))

(defun long-inner-op ()
  (message "retrieving inner")
  (url-retrieve-synchronously "http://localhost:9292")
  (message "inner retrieved"))
--8<---------------cut here---------------end--------------->8---

The window freezes, the last message in the echo area being "inner
retrieved" (but C-g breaks out of it, and if we're calling
`long-ops-idle', the "outer" request succeeds, too).

It doesn't happen with just any web server, though. I can reproduce it
using both master and emacs-24 using this Ruby server:

--8<---------------cut here---------------start------------->8---
run proc { |env|
  sleep 1
  [200, {'Content-Type' => 'text/plain'}, [""]]
}
--8<---------------cut here---------------end--------------->8---

Save it as test.ru and launch with 'rackup test.ru'.

But only with master using this Python server (an example for those who
don't have Ruby installed):

--8<---------------cut here---------------start------------->8---
from wsgiref.util import setup_testing_defaults
from wsgiref.simple_server import make_server
import time

def simple_app(environ, start_response):
    setup_testing_defaults(environ)

    status = '200 OK'
    headers = [('Content-type', 'text/plain')]

    time.sleep(1)

    start_response(status, headers)

    return ""

httpd = make_server('', 9292, simple_app)
print "Serving on port 9292..."
httpd.serve_forever()
--8<---------------cut here---------------end--------------->8---

Save it as server.py, and launch with 'python server.py'.

The key difference seems to be Keep-Alive (enabled by default by WEBrick
(Ruby); wsgiref doesn't support it at all). Adding "Connection: close"
to the Ruby example server makes emacs-24 behave as expected with it.

And someone who doesn't want to run the server can also try the
following URLs with master: http://xip.io/ and http://ya.ru/. When
calling xip.io, the outer request eventually returns, but much later
than expected; when calling ya.ru, it doesn't.

The present bug report is based on this long-standing issue, which I
haven't been able to reliably reproduce until now:
https://github.com/dgutov/robe/issues/36

In GNU Emacs 25.0.50.5 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
 of 2015-02-13 on axl
Repository revision: 69e38a5b1fdb5ac20440b6ce7ba1fc2cd575f4e6
Windowing system distributor `The X.Org Foundation', version 11.0.11601901
System Description:	Ubuntu 14.10