From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 03 12:59:10 2014 Received: (at submit) by debbugs.gnu.org; 3 Sep 2014 16:59:10 +0000 Received: from localhost ([127.0.0.1]:58009 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPDtl-0008Ef-TU for submit@debbugs.gnu.org; Wed, 03 Sep 2014 12:59:10 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51224) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPDtj-0008EQ-Pp for submit@debbugs.gnu.org; Wed, 03 Sep 2014 12:59:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XPDtZ-0002Qr-0C for submit@debbugs.gnu.org; Wed, 03 Sep 2014 12:59:02 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:55246) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XPDtY-0002Qn-TH for submit@debbugs.gnu.org; Wed, 03 Sep 2014 12:58:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52927) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XPDtT-0007pA-Vt for bug-gnu-emacs@gnu.org; Wed, 03 Sep 2014 12:58:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XPDtL-0002Oa-HV for bug-gnu-emacs@gnu.org; Wed, 03 Sep 2014 12:58:51 -0400 Received: from loki.jorgenschaefer.de ([87.230.15.51]:34201) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XPDtL-0002OC-B2 for bug-gnu-emacs@gnu.org; Wed, 03 Sep 2014 12:58:43 -0400 Received: by loki.jorgenschaefer.de (Postfix, from userid 1000) id 6957A2011D5; Wed, 3 Sep 2014 18:58:11 +0200 (CEST) From: Jorgen Schaefer To: bug-gnu-emacs@gnu.org Subject: 24.3.1; On windows, process-send-string can freeze Emacs Date: Wed, 03 Sep 2014 18:58:11 +0200 Message-ID: <87k35kod3w.fsf@loki.jorgenschaefer.de> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi! A user's bug report[1] on my project Elpy has revealed an apparent bug in Emacs. [1] https://github.com/jorgenschaefer/elpy/issues/212#issuecomment-54015294 Elpy starts a Python process using `start-process' with `process-connection-type' set to nil, `default-directory' set to "/", and an unchanged coding system. It then proceeds to talk with the subprocess using a JSON-RPC-based protocol. After some interactions, this freezes an Emacs under Windows. C-g does not lead to any reaction. Killing the subprocess unfreezes Emacs. The freeze happens more reliably the larger the data sent is. I asked the user to change the function that does the process communication to the following: (defun elpy-rpc--call (method-name params success error) (let ((promise (elpy-promise success error))) (with-current-buffer (elpy-rpc--get-rpc-buffer) (setq elpy-rpc--call-id (1+ elpy-rpc--call-id)) (elpy-rpc--register-callback elpy-rpc--call-id promise) (let ((proc (get-buffer-process (current-buffer))) (text (json-encode `((id . ,elpy-rpc--call-id) (method . ,method-name) (params . ,params))))) (message "Sending %s bytes ..." (length text)) (process-send-string proc text) (process-send-string proc "\n") (message "Sending %s bytes ... done." (length text)))) promise)) This lead to the following output in *Messages*: Sending 978 bytes ... done. Sending 958 bytes ... done. Sending 959 bytes ... done. Sending 960 bytes ... done. Sending 961 bytes ... done. Sending 962 bytes ... done. Sending 958 bytes ... At this point, Emacs froze. Apparently, it did so in the middle of one of the two `process-send-string' calls. Killing the subprocess caused the following output: eldoc error: (file-error writing to process invalid argument elpy-rpc [project:~/ python:pythonw]) Elpy indeed is called from eldoc, and "elpy-rpc [project:~/ python:pythonw]" is the process name. I'm a bit at a loss now as to how to continue debugging this. I do not use Windows myself. I'm sure the user is willing to do some debugging and reproduction cases there, but I wouldn't even know how to start or what to ask them to do (Windows doesn't have strace, does it?). Jorgen From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 03 14:03:01 2014 Received: (at 18396) by debbugs.gnu.org; 3 Sep 2014 18:03:01 +0000 Received: from localhost ([127.0.0.1]:58063 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPEtY-0001OW-Ce for submit@debbugs.gnu.org; Wed, 03 Sep 2014 14:03:00 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:33952) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPEtV-0001OH-30 for 18396@debbugs.gnu.org; Wed, 03 Sep 2014 14:02:58 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0NBC00H00797EY00@a-mtaout20.012.net.il> for 18396@debbugs.gnu.org; Wed, 03 Sep 2014 21:02:50 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NBC00HPN7GP0G60@a-mtaout20.012.net.il>; Wed, 03 Sep 2014 21:02:50 +0300 (IDT) Date: Wed, 03 Sep 2014 21:03:00 +0300 From: Eli Zaretskii Subject: Re: bug#18396: 24.3.1; On windows, process-send-string can freeze Emacs In-reply-to: <87k35kod3w.fsf@loki.jorgenschaefer.de> X-012-Sender: halo1@inter.net.il To: Jorgen Schaefer Message-id: <831trsinu3.fsf@gnu.org> References: <87k35kod3w.fsf@loki.jorgenschaefer.de> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 18396 Cc: 18396@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Jorgen Schaefer > Date: Wed, 03 Sep 2014 18:58:11 +0200 > > A user's bug report[1] on my project Elpy has revealed an apparent bug > in Emacs. I'm not sure it is a bug in Emacs. > Elpy starts a Python process using `start-process' with > `process-connection-type' set to nil, `default-directory' set to "/", > and an unchanged coding system. Using "/" as the default directory on Windows is a bad idea, as that is not a fully-qualified absolute file name. > After some interactions, this freezes an Emacs under Windows. C-g does > not lead to any reaction. On Windows, C-g cannot interrupt a system call. > Killing the subprocess unfreezes Emacs. Probably because it breaks the pipe to the subprocess. Which probably means Emacs is not hung, it waits for something that doesn't happen. > The freeze happens more reliably the larger the data sent is. This is consistent with the pipe not being read hypothesis, see below. > (defun elpy-rpc--call (method-name params success error) > (let ((promise (elpy-promise success error))) > (with-current-buffer (elpy-rpc--get-rpc-buffer) > (setq elpy-rpc--call-id (1+ elpy-rpc--call-id)) > (elpy-rpc--register-callback elpy-rpc--call-id promise) > (let ((proc (get-buffer-process (current-buffer))) > (text (json-encode `((id . ,elpy-rpc--call-id) > (method . ,method-name) > (params . ,params))))) > (message "Sending %s bytes ..." (length text)) > (process-send-string proc text) > (process-send-string proc "\n") > (message "Sending %s bytes ... done." (length text)))) > promise)) > > This lead to the following output in *Messages*: > > Sending 978 bytes ... done. > Sending 958 bytes ... done. > Sending 959 bytes ... done. > Sending 960 bytes ... done. > Sending 961 bytes ... done. > Sending 962 bytes ... done. > Sending 958 bytes ... > > At this point, Emacs froze. Apparently, it did so in the middle of one > of the two `process-send-string' calls. Killing the subprocess caused > the following output: > > eldoc error: (file-error writing to process invalid argument elpy-rpc [project:~/ python:pythonw]) Looks like the write to the pipe never returned. This could be because the pipe is full and is not being read from the other end (Windows pipes have 4K buffers, and you show above more than 6K of data). > I'm a bit at a loss now as to how to continue debugging this. The obvious way: attach a debugger to Emacs and see where it is hung or waiting. It is important to ask the user to produce backtraces from all the threads, because at least 2 threads are involved in interaction with a subprocess on MS-Windows. > (Windows doesn't have strace, does it?). It does, but you don't want to use that, believe me: you would be drowned in a flood of system calls, most of which are undocumented, and even if they were, it is entirely non-obvious how to relate them to what Emacs does. From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 03 14:43:15 2014 Received: (at 18396) by debbugs.gnu.org; 3 Sep 2014 18:43:15 +0000 Received: from localhost ([127.0.0.1]:58080 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPFWU-0002Qx-LC for submit@debbugs.gnu.org; Wed, 03 Sep 2014 14:43:15 -0400 Received: from loki.jorgenschaefer.de ([87.230.15.51]:45425) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPFWR-0002Qk-RW for 18396@debbugs.gnu.org; Wed, 03 Sep 2014 14:43:12 -0400 Received: by loki.jorgenschaefer.de (Postfix, from userid 998) id 87A922011D7; Wed, 3 Sep 2014 20:43:09 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lvps87-230-15-51.dedicated.hosteurope.de X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.2 Received: from forcix (port-52353.pppoe.wtnet.de [46.59.205.39]) by loki.jorgenschaefer.de (Postfix) with ESMTPSA id F29FA2011D2; Wed, 3 Sep 2014 20:43:08 +0200 (CEST) Date: Wed, 3 Sep 2014 20:43:07 +0200 From: Jorgen Schaefer To: Eli Zaretskii Subject: Re: bug#18396: 24.3.1; On windows, process-send-string can freeze Emacs Message-ID: <20140903204307.0bcf515c@forcix> In-Reply-To: <831trsinu3.fsf@gnu.org> References: <87k35kod3w.fsf@loki.jorgenschaefer.de> <831trsinu3.fsf@gnu.org> X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.24; i586-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 18396 Cc: 18396@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) On Wed, 03 Sep 2014 21:03:00 +0300 Eli Zaretskii wrote: > > From: Jorgen Schaefer > > Date: Wed, 03 Sep 2014 18:58:11 +0200 > > > > A user's bug report[1] on my project Elpy has revealed an apparent > > bug in Emacs. > > I'm not sure it is a bug in Emacs. After your explanation, I'm not so sure, either. > > Elpy starts a Python process using `start-process' with > > `process-connection-type' set to nil, `default-directory' set to > > "/", and an unchanged coding system. > > Using "/" as the default directory on Windows is a bad idea, as that > is not a fully-qualified absolute file name. What would be the equivalent for "out of the way and not blocking any mount point" (or equivalent) on Windows? > Looks like the write to the pipe never returned. This could be > because the pipe is full and is not being read from the other end > (Windows pipes have 4K buffers, and you show above more than 6K of > data). That is quite likely the explanation. The Python process does the equivalent of a REPL, reading one RPC call, evaluating it, and writing the response. If in the duration of that evaluation Emacs sends more than 4k of data, it will hang. If the response is larger than 4k, Python in turn will hang. Resulting in a deadlock. Am I missing something? The same would happen on Unix, except the buffer size is much larger, meaning it's a lot less likely. I guess the Python process could use threading to avoid this. Does Emacs have a chance to check for a pipe to be writable before doing so? The whole process blocking like this feels a bit weird. > > I'm a bit at a loss now as to how to continue debugging this. > > The obvious way: attach a debugger to Emacs and see where it is hung > or waiting. It is important to ask the user to produce backtraces > from all the threads, because at least 2 threads are involved in > interaction with a subprocess on MS-Windows. Thanks. I'll ask, though I'm not sure if the user has a debugger available. Jorgen From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 03 15:01:18 2014 Received: (at 18396) by debbugs.gnu.org; 3 Sep 2014 19:01:18 +0000 Received: from localhost ([127.0.0.1]:58092 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPFnx-0002tU-A3 for submit@debbugs.gnu.org; Wed, 03 Sep 2014 15:01:17 -0400 Received: from mtaout26.012.net.il ([80.179.55.182]:59589) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPFnt-0002tC-QJ for 18396@debbugs.gnu.org; Wed, 03 Sep 2014 15:01:15 -0400 Received: from conversion-daemon.mtaout26.012.net.il by mtaout26.012.net.il (HyperSendmail v2007.08) id <0NBC00J009BZHK00@mtaout26.012.net.il> for 18396@debbugs.gnu.org; Wed, 03 Sep 2014 21:59:11 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout26.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NBC00B3KA2NWXA0@mtaout26.012.net.il>; Wed, 03 Sep 2014 21:59:11 +0300 (IDT) Date: Wed, 03 Sep 2014 22:01:18 +0300 From: Eli Zaretskii Subject: Re: bug#18396: 24.3.1; On windows, process-send-string can freeze Emacs In-reply-to: <20140903204307.0bcf515c@forcix> X-012-Sender: halo1@inter.net.il To: Jorgen Schaefer Message-id: <83zjegh6kh.fsf@gnu.org> References: <87k35kod3w.fsf@loki.jorgenschaefer.de> <831trsinu3.fsf@gnu.org> <20140903204307.0bcf515c@forcix> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 18396 Cc: 18396@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Wed, 3 Sep 2014 20:43:07 +0200 > From: Jorgen Schaefer > Cc: 18396@debbugs.gnu.org > > > Using "/" as the default directory on Windows is a bad idea, as that > > is not a fully-qualified absolute file name. > > What would be the equivalent for "out of the way and not blocking any > mount point" (or equivalent) on Windows? I might have a suggestion, if you explain what "out of the way" and "not blocking any mount points" mean. > > Looks like the write to the pipe never returned. This could be > > because the pipe is full and is not being read from the other end > > (Windows pipes have 4K buffers, and you show above more than 6K of > > data). > > That is quite likely the explanation. The Python process does the > equivalent of a REPL, reading one RPC call, evaluating it, and writing > the response. If in the duration of that evaluation Emacs sends more > than 4k of data, it will hang. If the response is larger than 4k, > Python in turn will hang. Resulting in a deadlock. > > Am I missing something? I'd expect Python to continue reading from the pipe once it evaluated one call and sent back the response. It should see that more input is available and continue reading. Could this be an end-of-line format issue? Are you sure the commands used from Emacs side produce Windows-style CRLF EOLs? Or maybe they do, but Python expects Unix-style newline-only EOLs (maybe it's a Cygwin or MSYS Python, for example)? A wrong EOL format might cause Python to fail to realize it was handed a full line of input. > Does Emacs have a chance to check for a pipe to be writable before > doing so? The whole process blocking like this feels a bit weird. I don't know how to do such a check with pipes on Windows. More importantly, how would that help? The pipe will fill up anyway, and the communications with Python will stop. Being able to interrupt with C-g vs killing the subprocess is not such a big win, IMO. > > The obvious way: attach a debugger to Emacs and see where it is hung > > or waiting. It is important to ask the user to produce backtraces > > from all the threads, because at least 2 threads are involved in > > interaction with a subprocess on MS-Windows. > > Thanks. I'll ask, though I'm not sure if the user has a debugger > available. If the user doesn't have GDB, he/she can download one from the MinGW site. I think using a debugger is the only way to understand what happens here. From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 03 15:28:40 2014 Received: (at 18396) by debbugs.gnu.org; 3 Sep 2014 19:28:40 +0000 Received: from localhost ([127.0.0.1]:58116 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPGER-0003YQ-Tb for submit@debbugs.gnu.org; Wed, 03 Sep 2014 15:28:40 -0400 Received: from loki.jorgenschaefer.de ([87.230.15.51]:37409) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPGEP-0003YF-Fn for 18396@debbugs.gnu.org; Wed, 03 Sep 2014 15:28:38 -0400 Received: by loki.jorgenschaefer.de (Postfix, from userid 998) id 5ECD3202AB7; Wed, 3 Sep 2014 21:28:35 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lvps87-230-15-51.dedicated.hosteurope.de X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.2 Received: from forcix (port-52353.pppoe.wtnet.de [46.59.205.39]) by loki.jorgenschaefer.de (Postfix) with ESMTPSA id 96C392011DA; Wed, 3 Sep 2014 21:28:34 +0200 (CEST) Date: Wed, 3 Sep 2014 21:28:33 +0200 From: Jorgen Schaefer To: Eli Zaretskii Subject: Re: bug#18396: 24.3.1; On windows, process-send-string can freeze Emacs Message-ID: <20140903212833.14562b90@forcix> In-Reply-To: <83zjegh6kh.fsf@gnu.org> References: <87k35kod3w.fsf@loki.jorgenschaefer.de> <831trsinu3.fsf@gnu.org> <20140903204307.0bcf515c@forcix> <83zjegh6kh.fsf@gnu.org> X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.24; i586-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 18396 Cc: 18396@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) On Wed, 03 Sep 2014 22:01:18 +0300 Eli Zaretskii wrote: > > Date: Wed, 3 Sep 2014 20:43:07 +0200 > > From: Jorgen Schaefer > > Cc: 18396@debbugs.gnu.org > > > > > Using "/" as the default directory on Windows is a bad idea, as > > > that is not a fully-qualified absolute file name. > > > > What would be the equivalent for "out of the way and not blocking > > any mount point" (or equivalent) on Windows? > > I might have a suggestion, if you explain what "out of the way" and > "not blocking any mount points" mean. The primary reason Elpy starts the process in "/" is to avoid accidental imports of Python modules. As the process is started from a Python buffer, there often are Python files in the current directory, which can accidentally be imported. "/" is unlikely to have Python modules. Also, a current directory of "/" means the process won't accidentally block a mount point, much like why daemons chdir to /. I'm not sure if this concept makes sense in Windows. > > > Looks like the write to the pipe never returned. This could be > > > because the pipe is full and is not being read from the other end > > > (Windows pipes have 4K buffers, and you show above more than 6K of > > > data). > > > > That is quite likely the explanation. The Python process does the > > equivalent of a REPL, reading one RPC call, evaluating it, and > > writing the response. If in the duration of that evaluation Emacs > > sends more than 4k of data, it will hang. If the response is larger > > than 4k, Python in turn will hang. Resulting in a deadlock. > > > > Am I missing something? > > I'd expect Python to continue reading from the pipe once it evaluated > one call and sent back the response. It should see that more input is > available and continue reading. But if the sending of the response runs into the same problem? The response can contain docstrings and can easily be larger than 4k, so it's conceivable that Python sends more than 4k of data as well, which would block the Python process, too? And thus prevent it from reading, which keeps Emacs blocked? > Could this be an end-of-line format issue? Are you sure the commands > used from Emacs side produce Windows-style CRLF EOLs? Or maybe they > do, but Python expects Unix-style newline-only EOLs (maybe it's a > Cygwin or MSYS Python, for example)? A wrong EOL format might cause > Python to fail to realize it was handed a full line of input. Unlikely. The RPC calls work perfectly fine most of the time, so the line ending convention does not cause any confusion. Also, changing the line ending convention on the Emacs side caused an error, so I assume it's working fine as is. > > Does Emacs have a chance to check for a pipe to be writable before > > doing so? The whole process blocking like this feels a bit weird. > > I don't know how to do such a check with pipes on Windows. More > importantly, how would that help? The pipe will fill up anyway, and > the communications with Python will stop. Being able to interrupt > with C-g vs killing the subprocess is not such a big win, IMO. Well, if the deadlock hypothesis is correct, Emacs would check if the pipe is writable, notice that it isn't and keep checking, to notice that the pipe is readable, read data, and thus break the deadlock. That of course requires that the deadlock hypothesis is indeed correct. :-) > > > The obvious way: attach a debugger to Emacs and see where it is > > > hung or waiting. It is important to ask the user to produce > > > backtraces from all the threads, because at least 2 threads are > > > involved in interaction with a subprocess on MS-Windows. > > > > Thanks. I'll ask, though I'm not sure if the user has a debugger > > available. > > If the user doesn't have GDB, he/she can download one from the MinGW > site. I think using a debugger is the only way to understand what > happens here. I'll pass it on, thanks. Jorgen From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 03 22:51:45 2014 Received: (at 18396) by debbugs.gnu.org; 4 Sep 2014 02:51:45 +0000 Received: from localhost ([127.0.0.1]:58275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPN9E-0007AJ-Gg for submit@debbugs.gnu.org; Wed, 03 Sep 2014 22:51:44 -0400 Received: from mtaout29.012.net.il ([80.179.55.185]:54529) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XPN9A-00079z-Q9 for 18396@debbugs.gnu.org; Wed, 03 Sep 2014 22:51:42 -0400 Received: from conversion-daemon.mtaout29.012.net.il by mtaout29.012.net.il (HyperSendmail v2007.08) id <0NBC00500UOB4P00@mtaout29.012.net.il> for 18396@debbugs.gnu.org; Thu, 04 Sep 2014 05:51:22 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout29.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NBC003CVVXMVT40@mtaout29.012.net.il>; Thu, 04 Sep 2014 05:51:22 +0300 (IDT) Date: Thu, 04 Sep 2014 05:51:45 +0300 From: Eli Zaretskii Subject: Re: bug#18396: 24.3.1; On windows, process-send-string can freeze Emacs In-reply-to: <20140903212833.14562b90@forcix> X-012-Sender: halo1@inter.net.il To: Jorgen Schaefer Message-id: <83y4u0gkse.fsf@gnu.org> References: <87k35kod3w.fsf@loki.jorgenschaefer.de> <831trsinu3.fsf@gnu.org> <20140903204307.0bcf515c@forcix> <83zjegh6kh.fsf@gnu.org> <20140903212833.14562b90@forcix> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 18396 Cc: 18396@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Wed, 3 Sep 2014 21:28:33 +0200 > From: Jorgen Schaefer > Cc: 18396@debbugs.gnu.org > > On Wed, 03 Sep 2014 22:01:18 +0300 > Eli Zaretskii wrote: > > > > Date: Wed, 3 Sep 2014 20:43:07 +0200 > > > From: Jorgen Schaefer > > > Cc: 18396@debbugs.gnu.org > > > > > > > Using "/" as the default directory on Windows is a bad idea, as > > > > that is not a fully-qualified absolute file name. > > > > > > What would be the equivalent for "out of the way and not blocking > > > any mount point" (or equivalent) on Windows? > > > > I might have a suggestion, if you explain what "out of the way" and > > "not blocking any mount points" mean. > > The primary reason Elpy starts the process in "/" is to avoid > accidental imports of Python modules. As the process is started from a > Python buffer, there often are Python files in the current directory, > which can accidentally be imported. "/" is unlikely to have Python > modules. Why not use ~/, then? > Also, a current directory of "/" means the process won't > accidentally block a mount point, much like why daemons chdir to /. I'm > not sure if this concept makes sense in Windows. (expand-file-name "/") should do what you want, I think. > > > > Looks like the write to the pipe never returned. This could be > > > > because the pipe is full and is not being read from the other end > > > > (Windows pipes have 4K buffers, and you show above more than 6K of > > > > data). > > > > > > That is quite likely the explanation. The Python process does the > > > equivalent of a REPL, reading one RPC call, evaluating it, and > > > writing the response. If in the duration of that evaluation Emacs > > > sends more than 4k of data, it will hang. If the response is larger > > > than 4k, Python in turn will hang. Resulting in a deadlock. > > > > > > Am I missing something? > > > > I'd expect Python to continue reading from the pipe once it evaluated > > one call and sent back the response. It should see that more input is > > available and continue reading. > > But if the sending of the response runs into the same problem? Each direction of the pipe has its own separate buffering, so this is unlikely. > The response can contain docstrings and can easily be larger than > 4k, so it's conceivable that Python sends more than 4k of data as > well, which would block the Python process, too? And thus prevent it > from reading, which keeps Emacs blocked? Emacs reads in a separate thread, so again, unlikely. > > Could this be an end-of-line format issue? Are you sure the commands > > used from Emacs side produce Windows-style CRLF EOLs? Or maybe they > > do, but Python expects Unix-style newline-only EOLs (maybe it's a > > Cygwin or MSYS Python, for example)? A wrong EOL format might cause > > Python to fail to realize it was handed a full line of input. > > Unlikely. The RPC calls work perfectly fine most of the time Perhaps this user has a different kind of Python than others. > > > Does Emacs have a chance to check for a pipe to be writable before > > > doing so? The whole process blocking like this feels a bit weird. > > > > I don't know how to do such a check with pipes on Windows. More > > importantly, how would that help? The pipe will fill up anyway, and > > the communications with Python will stop. Being able to interrupt > > with C-g vs killing the subprocess is not such a big win, IMO. > > Well, if the deadlock hypothesis is correct, Emacs would check if the > pipe is writable, notice that it isn't and keep checking, to notice > that the pipe is readable, read data, and thus break the deadlock. See above: there's no interconnection between reading and writing, so that's not the problem. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 09 14:45:25 2014 Received: (at 18396) by debbugs.gnu.org; 9 Sep 2014 18:45:25 +0000 Received: from localhost ([127.0.0.1]:35656 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XRQPs-0003ZX-4X for submit@debbugs.gnu.org; Tue, 09 Sep 2014 14:45:24 -0400 Received: from mtaout26.012.net.il ([80.179.55.182]:39153) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XRQPo-0003ZF-3x for 18396@debbugs.gnu.org; Tue, 09 Sep 2014 14:45:21 -0400 Received: from conversion-daemon.mtaout26.012.net.il by mtaout26.012.net.il (HyperSendmail v2007.08) id <0NBN00900DA1G200@mtaout26.012.net.il> for 18396@debbugs.gnu.org; Tue, 09 Sep 2014 21:43:08 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout26.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NBN003FHDBVU350@mtaout26.012.net.il>; Tue, 09 Sep 2014 21:43:08 +0300 (IDT) Date: Tue, 09 Sep 2014 21:45:23 +0300 From: Eli Zaretskii Subject: Re: bug#18396: 24.3.1; On windows, process-send-string can freeze Emacs In-reply-to: <83y4u0gkse.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: forcer@forcix.cx Message-id: <83mwa8ab0c.fsf@gnu.org> References: <87k35kod3w.fsf@loki.jorgenschaefer.de> <831trsinu3.fsf@gnu.org> <20140903204307.0bcf515c@forcix> <83zjegh6kh.fsf@gnu.org> <20140903212833.14562b90@forcix> <83y4u0gkse.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 18396 Cc: 18396@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Thu, 04 Sep 2014 05:51:45 +0300 > From: Eli Zaretskii > Cc: 18396@debbugs.gnu.org > > > > > Does Emacs have a chance to check for a pipe to be writable before > > > > doing so? The whole process blocking like this feels a bit weird. > > > > > > I don't know how to do such a check with pipes on Windows. More > > > importantly, how would that help? The pipe will fill up anyway, and > > > the communications with Python will stop. Being able to interrupt > > > with C-g vs killing the subprocess is not such a big win, IMO. > > > > Well, if the deadlock hypothesis is correct, Emacs would check if the > > pipe is writable, notice that it isn't and keep checking, to notice > > that the pipe is readable, read data, and thus break the deadlock. > > See above: there's no interconnection between reading and writing, so > that's not the problem. I've just posted a tentative patch for the deadlock problem in http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18420#20. If some of your users can build their own Emacs, please ask them to apply the patch and see if it resolves the problem. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sat Nov 01 11:38:07 2014 Received: (at 18396-close) by debbugs.gnu.org; 1 Nov 2014 15:38:07 +0000 Received: from localhost ([127.0.0.1]:44755 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Xkakg-0004et-R2 for submit@debbugs.gnu.org; Sat, 01 Nov 2014 11:38:07 -0400 Received: from loki.jorgenschaefer.de ([87.230.15.51]:36828) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Xkakd-0004ef-Ci for 18396-close@debbugs.gnu.org; Sat, 01 Nov 2014 11:38:04 -0400 Received: by loki.jorgenschaefer.de (Postfix, from userid 998) id E527520458A; Sat, 1 Nov 2014 16:37:58 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lvps87-230-15-51.dedicated.hosteurope.de X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.2 Received: from forcix (port-50001.pppoe.wtnet.de [46.59.195.237]) by loki.jorgenschaefer.de (Postfix) with ESMTPSA id 8F178204588 for <18396-close@debbugs.gnu.org>; Sat, 1 Nov 2014 16:37:58 +0100 (CET) Date: Sat, 1 Nov 2014 16:37:57 +0100 From: Jorgen Schaefer To: 18396-close@debbugs.gnu.org Subject: No further updates Message-ID: <20141101163757.2b23689a@forcix> X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.25; i586-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 18396-close X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) The user did not provide any further feedback after three weeks. Closing this bug report, assuming the problem was fixed. Thank you! From unknown Sat Sep 06 02:03:25 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sun, 30 Nov 2014 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator