From unknown Wed Jun 18 00:28:25 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33839: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Resent-From: Philipp Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 23 Dec 2018 02:30:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 33839 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 33839@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.154553215027743 (code B ref -1); Sun, 23 Dec 2018 02:30:03 +0000 Received: (at submit) by debbugs.gnu.org; 23 Dec 2018 02:29:10 +0000 Received: from localhost ([127.0.0.1]:60376 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gatVu-0007DP-6o for submit@debbugs.gnu.org; Sat, 22 Dec 2018 21:29:10 -0500 Received: from eggs.gnu.org ([208.118.235.92]:41811) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gatVs-0007D6-I4 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 21:29:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gatVm-0000Hx-C0 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 21:29:03 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=BAYES_50, FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:34732) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gatVm-0000Hr-9H for submit@debbugs.gnu.org; Sat, 22 Dec 2018 21:29:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59148) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gatVl-00087g-6R for bug-gnu-emacs@gnu.org; Sat, 22 Dec 2018 21:29:02 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gatVi-0000EO-2m for bug-gnu-emacs@gnu.org; Sat, 22 Dec 2018 21:29:01 -0500 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]:39992) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gatVh-0000Dw-SQ for bug-gnu-emacs@gnu.org; Sat, 22 Dec 2018 21:28:58 -0500 Received: by mail-wm1-x32f.google.com with SMTP id f188so9131229wmf.5 for ; Sat, 22 Dec 2018 18:28:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version; bh=2SZr0IkZI0uQP/44F6FoYXtXPOGyqo8WSMUkPlf+fLE=; b=aWXzW8MDt2jf3/kuH4IS1D2KjE5CNJFvipgQczvsx3XSzZvBCx8vkPNnGCdWQgYObx 2e2qBcWzKORyEHBAoV11xghjqToj5Dexq95WYx4bvdXL3pNDfpjRGb/aTCWDg+U1NPXn yI0wy7ycM4EIqQGYIv9ApuYAN9No7tY1n26V2UdqxV7cWnSHLUyOYRzyDrHfHbKzUfte GfeYkiXtTQ99Gg1s61KHaaeyGl/onWCIoUSdBsPdgPvBGSdQAYc+nZoPpL6gYM7D+W0o 6fyrjMI6MtvDJGmP2Mq/SmV8IbSIHE6YmCtmN00KZ1v8lTbUFNyc6l2P0S6Zr3Iqhfgo 9U8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version; bh=2SZr0IkZI0uQP/44F6FoYXtXPOGyqo8WSMUkPlf+fLE=; b=RkFx13uyuBr/bcUusdNXYNZKpdVoCVglEM32YNTksc1vNkLfwd8mbUUT0T3EmnI9Iy 1mzPPDW0X2APlorcrISlWJDaq0Q7igM8syr9gYNHCI5nPCUu4NdR/m3/UlbssQMzRom7 mdaMNmSwjHfjoknxW9m0El5gk5jb2EhR9TJNZP592mLoVizGjvxYq3xJwNk0H3hBb2aW 0Z/p4awyq4SoKzOouBLBT6pFVd0ZUMjZ0LodReT0ZuSH9T0bqdi/U9cRvfrQM/ObYR6u y3UA0Wzu5nf7ojJCdTYLBPT1F7T1731ZoXGNv8c30/oT09+meWln1DRVdQH0CCfZTUoF f8wQ== X-Gm-Message-State: AA+aEWaMxzHPKrY+337gbzGoNW9CHEodWbcVNfOZ4M++moeq5NcJlJZH 5QgYx4P2xsLSiv9aQdFSDB3l/+nZ X-Google-Smtp-Source: AFSGD/WLLvNAhBOuTGDB9ZqmFxrcMgLjRpRKjQu2w5dUXRO8g6SLQFkq9vTcDFJknu8UqGzCPGGC1Q== X-Received: by 2002:a1c:2547:: with SMTP id l68mr7749679wml.11.1545532136281; Sat, 22 Dec 2018 18:28:56 -0800 (PST) Received: from p (p579979C6.dip0.t-ipconnect.de. [87.153.121.198]) by smtp.gmail.com with ESMTPSA id o17sm31345292wmg.35.2018.12.22.18.28.54 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Dec 2018 18:28:55 -0800 (PST) From: Philipp Date: Sun, 23 Dec 2018 03:28:53 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -3.8 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.8 (----) Add the following code to /tmp/form.el: (require 'cl-lib) (with-temp-buffer (let ((proc (make-process :name "test" :command '("bash" "-c" "echo stdout; echo stderr >&2") :buffer (current-buffer) :connection-type 'pipe :sentinel #'ignore :noquery t :coding '(utf-8-unix . utf-8-unix)))) (when (process-live-p proc) (process-send-eof proc)) (while (process-live-p proc) (accept-process-output proc)) (cl-assert (equal (buffer-string) "stdout\nstderr\n") :show-args))) Then evaluate this form repeatedly. Occasionally the buffer is empty and the assertion triggers. For example: $ for i in {1..100}; do emacs -Q -batch -l /tmp/form.el ; done Assertion failed: (equal (buffer-string) "stdout stderr "), "" Assertion failed: (equal (buffer-string) "stdout stderr "), "" i.e. here the output didn't arrive in 2 of 100 iterations. In GNU Emacs 26.1.90 (build 5, x86_64-apple-darwin18.0.0, NS appkit-1671.10 Version 10.14.1 (Build 18B75)) of 2018-12-22 Repository revision: 24ddea074a2e61f7accde60cdf941ba67b1ce82a Windowing system distributor 'Apple', version 10.3.1671 Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Configured using: 'configure --enable-checking --enable-check-lisp-object-type --enable-gtk-deprecation-warnings --with-modules --without-xml2 --without-pop --with-mailutils --enable-gcc-warnings=warn-only MAKEINFO=/usr/local/opt/texinfo/bin/makeinfo 'CFLAGS=-O0 -ggdb3' LDFLAGS=-O0' Configured features: JPEG NOTIFY ACL GNUTLS ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS LCMS2 Important settings: value of $LANG: de_DE.UTF-8 locale-coding-system: utf-8-unix Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc puny seq dired dired-loaddefs format-spec rfc822 mml easymenu mml-sec epa derived epg epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils phst pcase ffap thingatpt url-parse auth-source cl-seq eieio byte-opt bytecomp byte-compile cconv eieio-core cl-macs gv eieio-loaddefs password-cache url-vars subr-x rx gnutls dbus xml cl-loaddefs cl-lib elec-pair time-date tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads kqueue cocoa ns lcms2 multi-tty make-network-process emacs) Memory information: ((conses 16 216710 8580) (symbols 48 21650 0) (miscs 40 59 186) (strings 32 32914 1496) (string-bytes 1 892039) (vectors 16 37089) (vector-slots 8 751562 17790) (floats 8 56 60) (intervals 56 210 0) (buffers 992 11)) From unknown Wed Jun 18 00:28:25 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33839: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 23 Dec 2018 15:23:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33839 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Philipp Cc: 33839@debbugs.gnu.org Received: via spool by 33839-submit@debbugs.gnu.org id=B33839.154557853812808 (code B ref 33839); Sun, 23 Dec 2018 15:23:02 +0000 Received: (at 33839) by debbugs.gnu.org; 23 Dec 2018 15:22:18 +0000 Received: from localhost ([127.0.0.1]:33895 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb5a5-0003KW-QJ for submit@debbugs.gnu.org; Sun, 23 Dec 2018 10:22:18 -0500 Received: from eggs.gnu.org ([208.118.235.92]:46689) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb5a3-0003KJ-SF for 33839@debbugs.gnu.org; Sun, 23 Dec 2018 10:22:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gb5Zx-0001pK-Qw for 33839@debbugs.gnu.org; Sun, 23 Dec 2018 10:22:10 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54833) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gb5Zx-0001p8-N5; Sun, 23 Dec 2018 10:22:09 -0500 Received: from [176.228.60.248] (port=3954 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gb5Zx-00089w-3q; Sun, 23 Dec 2018 10:22:09 -0500 Date: Sun, 23 Dec 2018 17:21:58 +0200 Message-Id: <835zvk9skp.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Philipp on Sun, 23 Dec 2018 03:28:53 +0100) References: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Philipp > Date: Sun, 23 Dec 2018 03:28:53 +0100 > > (with-temp-buffer > (let ((proc (make-process :name "test" > :command '("bash" "-c" "echo stdout; echo stderr >&2") > :buffer (current-buffer) > :connection-type 'pipe > :sentinel #'ignore > :noquery t > :coding '(utf-8-unix . utf-8-unix)))) > (when (process-live-p proc) > (process-send-eof proc)) > (while (process-live-p proc) > (accept-process-output proc)) > (cl-assert (equal (buffer-string) "stdout\nstderr\n") :show-args))) > > Then evaluate this form repeatedly. Occasionally the buffer is empty > and the assertion triggers. Isn't there an inherent race condition here? From unknown Wed Jun 18 00:28:25 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33839: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Resent-From: Philipp Stephani Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 23 Dec 2018 16:46:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33839 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33839@debbugs.gnu.org Received: via spool by 33839-submit@debbugs.gnu.org id=B33839.154558355020505 (code B ref 33839); Sun, 23 Dec 2018 16:46:01 +0000 Received: (at 33839) by debbugs.gnu.org; 23 Dec 2018 16:45:50 +0000 Received: from localhost ([127.0.0.1]:33943 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb6sw-0005Kf-AF for submit@debbugs.gnu.org; Sun, 23 Dec 2018 11:45:50 -0500 Received: from mail-ot1-f43.google.com ([209.85.210.43]:43310) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb6su-0005KT-NT for 33839@debbugs.gnu.org; Sun, 23 Dec 2018 11:45:49 -0500 Received: by mail-ot1-f43.google.com with SMTP id a11so9310747otr.10 for <33839@debbugs.gnu.org>; Sun, 23 Dec 2018 08:45:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WQYBVcUregZb9RyXLOQefloVKqgnhDx0+3+kSlE/b2o=; b=VduU2igxqPs8I/mVum9rqBw0hA+U/Ra7rIIMIPReowUODxbY+oNi+dc6swjF1v3Kf7 AWEqz+lYlf32b5kj54APD78LgUtTjcGjsw++1+qzDuEGRkcBQpkAMfyiTZFlPuJVf5Po jmYsZSpkRx/yOcE45VcLgszCEL77OUFbWRF0QWA7msbLIC1yj0XdjH+pYTC/0RLe5Snc tm3HQqn/WC9q1DqdDcIbeS/1oZfwK2euGRGJsyW6YEpfULZ6qGfG80322fMuE5Gn3xFB QfePG1LQYqXnqdI36GIrub02AOXoBgxTRzFdKJennpOc48PuSkm4OBQhbWgk46V+zHVt +jCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WQYBVcUregZb9RyXLOQefloVKqgnhDx0+3+kSlE/b2o=; b=fIAfKlF9/GrWCsmJo6RmipwpVoYoA1EzERWon3A3cNraqnr6n012OUhSg8rBDl6kfW zjVnauOTHLtF5HUC2mpm8RjwW/Aj+o7qxfH9mTOZ/H1Nql3S3+0xWsKAzrUq/I5k9Wft vZM6Jp+w8vQ5/cVpi4erzkI6NGHeIFZW+MlMTsf8MnTy35vdfUqFaoJrte4o6Eu7apA4 Qod4F++JR60E+/Qw/N5jFYerC9oy1RIARhIIib9US494Gne9sJR38O9GMNGOfvG6Gxzm qYIv/RihOrn07bc1AuD6QfKnHWI/vm0JVIcA7iAlwKBB/F0NbLO++vvLrNjaIEppTzTD jZHg== X-Gm-Message-State: AJcUukeztGsesPdUvXpO+ZAjtXhQu1Mc2u0A2Eaa426DBC1Xb1PSqtER 5ov34A+5eFf7w2ylxjSK6BfrfM2t0SyNJl2RD8r+2w== X-Google-Smtp-Source: ALg8bN5eWtNL0hpokQyOij3dYxgqz1YhMtzp+iH6AjFhrrjZbck6fnLQDHBwzh52EeWy+v68lTsrDSOkJmdIu3Ndo7Y= X-Received: by 2002:a9d:4549:: with SMTP id p9mr6490189oti.51.1545583542885; Sun, 23 Dec 2018 08:45:42 -0800 (PST) MIME-Version: 1.0 References: <835zvk9skp.fsf@gnu.org> In-Reply-To: <835zvk9skp.fsf@gnu.org> From: Philipp Stephani Date: Sun, 23 Dec 2018 17:45:31 +0100 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.2 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Am So., 23. Dez. 2018 um 16:22 Uhr schrieb Eli Zaretskii : > > > From: Philipp > > Date: Sun, 23 Dec 2018 03:28:53 +0100 > > > > (with-temp-buffer > > (let ((proc (make-process :name "test" > > :command '("bash" "-c" "echo stdout; echo stderr >&2") > > :buffer (current-buffer) > > :connection-type 'pipe > > :sentinel #'ignore > > :noquery t > > :coding '(utf-8-unix . utf-8-unix)))) > > (when (process-live-p proc) > > (process-send-eof proc)) > > (while (process-live-p proc) > > (accept-process-output proc)) > > (cl-assert (equal (buffer-string) "stdout\nstderr\n") :show-args))) > > > > Then evaluate this form repeatedly. Occasionally the buffer is empty > > and the assertion triggers. > > Isn't there an inherent race condition here? Maybe? If so, then it should be documented, with an explanation how to write this in a race-free manner. (It seems waiting for the process sentinel to run works reliably.) From unknown Wed Jun 18 00:28:25 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33839: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 23 Dec 2018 16:55:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33839 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Philipp Stephani Cc: 33839@debbugs.gnu.org Received: via spool by 33839-submit@debbugs.gnu.org id=B33839.154558407821327 (code B ref 33839); Sun, 23 Dec 2018 16:55:01 +0000 Received: (at 33839) by debbugs.gnu.org; 23 Dec 2018 16:54:38 +0000 Received: from localhost ([127.0.0.1]:33959 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb71S-0005Xv-5j for submit@debbugs.gnu.org; Sun, 23 Dec 2018 11:54:38 -0500 Received: from eggs.gnu.org ([208.118.235.92]:32910) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gb71Q-0005Xi-OF for 33839@debbugs.gnu.org; Sun, 23 Dec 2018 11:54:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gb71H-0006t1-Pn for 33839@debbugs.gnu.org; Sun, 23 Dec 2018 11:54:31 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57540) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gb71F-0006pv-RT; Sun, 23 Dec 2018 11:54:27 -0500 Received: from [176.228.60.248] (port=1735 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gb71F-0006D7-DZ; Sun, 23 Dec 2018 11:54:25 -0500 Date: Sun, 23 Dec 2018 18:54:15 +0200 Message-Id: <83o99c89qg.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Philipp Stephani on Sun, 23 Dec 2018 17:45:31 +0100) References: <835zvk9skp.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Philipp Stephani > Date: Sun, 23 Dec 2018 17:45:31 +0100 > Cc: 33839@debbugs.gnu.org > > Am So., 23. Dez. 2018 um 16:22 Uhr schrieb Eli Zaretskii : > > > > > From: Philipp > > > Date: Sun, 23 Dec 2018 03:28:53 +0100 > > > > > > (with-temp-buffer > > > (let ((proc (make-process :name "test" > > > :command '("bash" "-c" "echo stdout; echo stderr >&2") > > > :buffer (current-buffer) > > > :connection-type 'pipe > > > :sentinel #'ignore > > > :noquery t > > > :coding '(utf-8-unix . utf-8-unix)))) > > > (when (process-live-p proc) > > > (process-send-eof proc)) > > > (while (process-live-p proc) > > > (accept-process-output proc)) > > > (cl-assert (equal (buffer-string) "stdout\nstderr\n") :show-args))) > > > > > > Then evaluate this form repeatedly. Occasionally the buffer is empty > > > and the assertion triggers. > > > > Isn't there an inherent race condition here? > > Maybe? If so, then it should be documented, with an explanation how to > write this in a race-free manner. Can you tell why you used the process-live-p condition for calling accept-process-output? What happens if you do that unconditionally? From unknown Wed Jun 18 00:28:25 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33839: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Resent-From: Philipp Stephani Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 25 Dec 2018 16:39:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33839 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33839@debbugs.gnu.org Received: via spool by 33839-submit@debbugs.gnu.org id=B33839.154575593218544 (code B ref 33839); Tue, 25 Dec 2018 16:39:02 +0000 Received: (at 33839) by debbugs.gnu.org; 25 Dec 2018 16:38:52 +0000 Received: from localhost ([127.0.0.1]:36379 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbpjG-0004p1-UM for submit@debbugs.gnu.org; Tue, 25 Dec 2018 11:38:51 -0500 Received: from mail-oi1-f182.google.com ([209.85.167.182]:45494) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbpjE-0004ol-85 for 33839@debbugs.gnu.org; Tue, 25 Dec 2018 11:38:48 -0500 Received: by mail-oi1-f182.google.com with SMTP id y1so11636949oie.12 for <33839@debbugs.gnu.org>; Tue, 25 Dec 2018 08:38:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+kQnVVbUEvYDUiPyxTHPo8mxg7Mh7//Ymj4lGOfoSWw=; b=vLHsf/RQdib3eDMLYwRQZlYmHJd0fx0J6WyWUbgQJEKsVBRHx/Qsc/Ee5g4QxC/0xc bxd7PNzCrVMpaanBBhAWBmHHyUyjOXufiVQRYeqJBG2DQmqkmN7pIr6jcBVzaUPIfk2l gyV3SVfq1biWZANY3T+ewJi46JwUsg6elT4lmPovVRAaX45HFLjQMkD0r1w2EcsHO7pl BncorK8zr2q5yJk+jcu6WECz60PqOIZWuCRb31J36iEnXCgsPN8DcERG0tFXoF2ygJu8 IGuRl5qdMrvICY3W+OlJQJPpseLRF9PNpGr+QLx4z5Uo2OjnGYJLGUfEcyHXvI19Vbqh A05g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+kQnVVbUEvYDUiPyxTHPo8mxg7Mh7//Ymj4lGOfoSWw=; b=HWlcC6nYvlL1QnzE3l8zskhDeHi46DtbUP6gUgNogh19A3olXHrHoavcHdHwg+7qtq R37YmDF5zZAGyruLMs243875jWhVQ6gsF2UMcbAhIzvkLNm42AOM+S7O1UkVMX+9K+zM BKTp8Qq91bBKN05suHm8LzAim/5gc8hhhIzQs8zMlDbs4xhQpcyKPSwJrGesW40JFT7O 2upCfggzgzJ1Er00zczZRoyfvtSuKjBQ+wTq3SOH5jiT/n+zjwPUTx256gNIrfRjhRoq 2VD34ZadvJHnMlwMWeoCNwanJ1YdCFn1u2u9Uaa7tG5Ey3xJtPy8akCL/EroZJ5AWqU0 NkNw== X-Gm-Message-State: AA+aEWZ5AjiNFVO9aBtzRJRB+FmmFO+zCAdaa0PzML9Oej2N7Z5XLBpZ QO8b2xBJ7T4xfxsVaZoIoxV7QffQxPgg1mjOopw= X-Google-Smtp-Source: AFSGD/WzUHTOuacL6biAUbNYvQuy9vlvDFGY+5mtbAPPUn7MKtnQuYBWzyQIsyGilpVhKqQz2V7EjXp28OfRa/VVGEI= X-Received: by 2002:aca:b104:: with SMTP id a4mr10211371oif.133.1545755922537; Tue, 25 Dec 2018 08:38:42 -0800 (PST) MIME-Version: 1.0 References: <835zvk9skp.fsf@gnu.org> In-Reply-To: From: Philipp Stephani Date: Tue, 25 Dec 2018 17:38:31 +0100 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.2 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Am So., 23. Dez. 2018 um 17:45 Uhr schrieb Philipp Stephani : > > Am So., 23. Dez. 2018 um 16:22 Uhr schrieb Eli Zaretskii : > > > > > From: Philipp > > > Date: Sun, 23 Dec 2018 03:28:53 +0100 > > > > > > (with-temp-buffer > > > (let ((proc (make-process :name "test" > > > :command '("bash" "-c" "echo stdout; echo stderr >&2") > > > :buffer (current-buffer) > > > :connection-type 'pipe > > > :sentinel #'ignore > > > :noquery t > > > :coding '(utf-8-unix . utf-8-unix)))) > > > (when (process-live-p proc) > > > (process-send-eof proc)) > > > (while (process-live-p proc) > > > (accept-process-output proc)) > > > (cl-assert (equal (buffer-string) "stdout\nstderr\n") :show-args))) > > > > > > Then evaluate this form repeatedly. Occasionally the buffer is empty > > > and the assertion triggers. > > > > Isn't there an inherent race condition here? > > Maybe? If so, then it should be documented, with an explanation how to > write this in a race-free manner. (It seems waiting for the process > sentinel to run works reliably.) This is also probably closely related to https://debbugs.gnu.org/cgi/bugreport.cgi?bug=31214. From unknown Wed Jun 18 00:28:25 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33839: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Resent-From: Philipp Stephani Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 25 Dec 2018 16:42:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33839 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 33839@debbugs.gnu.org Received: via spool by 33839-submit@debbugs.gnu.org id=B33839.154575609618811 (code B ref 33839); Tue, 25 Dec 2018 16:42:01 +0000 Received: (at 33839) by debbugs.gnu.org; 25 Dec 2018 16:41:36 +0000 Received: from localhost ([127.0.0.1]:36382 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbplw-0004tJ-Br for submit@debbugs.gnu.org; Tue, 25 Dec 2018 11:41:36 -0500 Received: from mail-ot1-f54.google.com ([209.85.210.54]:45040) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbplt-0004t6-Tg for 33839@debbugs.gnu.org; Tue, 25 Dec 2018 11:41:34 -0500 Received: by mail-ot1-f54.google.com with SMTP id f18so12615517otl.11 for <33839@debbugs.gnu.org>; Tue, 25 Dec 2018 08:41:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=aZpSDeNEgya20a8l/24DdEP3eO8GCNBTsp53CwmAkf8=; b=FHuTTFPXxAtkTdXxgk02xn/gK6km4ETXSDF+66TUqVVhgPSDcXR2H6gNkgIYmhZY2D JskuqAzyfZ1YfTxdd3PI+lozTM2jchVjUABLy90lOTLLd8KS6XjHmWOUoSopTLuyk8Ou hkYdnUBgMQD2YjqXo7YcPeWb/a7tn52xgoXSrlaK9JK+0yb2R9SNjLIuuhreO7lNKCQ9 iKFQ1m1ip4cl1XfAHiVEtCtjVe8YCFm5wlKn1AMKUBX17babw+N8AJ05YAFYf5LUSFvB R/lIrkLXMqI4T/FRGiltiRmBWAYOkkNgFgvdyd9HURxwg3P+9UwTQZzzjfQOnTDnNSV8 YEHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=aZpSDeNEgya20a8l/24DdEP3eO8GCNBTsp53CwmAkf8=; b=a8ZgG24vqtzynSs4IsCc8MImg8xOiWJVCwAR3LjErGE9V+hlpqKIjQnoJohOGxZTPJ 0UQj/kDkpJi3MJZlA4DNPpBe9TPGRPz1YZXngN/KJIMRldXiatBAZDzrlLfCbCKPvRGK Ju8zTogH6QNu9I+xsj1O/cul5a+J5YNm+mLavQpUHsdtm6IVcLk9j8xfbT/84+75sP65 qNME53zZQOb+9hWRQCRnrPxe8p5ulKuDjmkwoH+xSY3Lt9ZcqdqL0e3UGf50GI9h9U4r kOx4YP1nj9kkI7Qabr3OKmTUGrVPLTtA5L9wMOIf/9IJw6W27m/VEEnT8FQvjPCBwucc VLSg== X-Gm-Message-State: AJcUukfkTq6Wb9UtxwnRMl+h3w3AOVjXrtSlP4o+5DYmHKwxbUcs7/a1 gsYVdGrF6UvVCaajtNIItMAgoojnWspposifOiW2nQ== X-Google-Smtp-Source: ALg8bN4pmtT7GvMxa5IK+7sbXqEHqz4/ZxO2oThuGQ1Cubb6AlJiQYt7zk8eMtG1SgoewDu0ovFsGqwjm5bheXHGMZ0= X-Received: by 2002:a9d:2926:: with SMTP id d35mr11226876otb.355.1545756088243; Tue, 25 Dec 2018 08:41:28 -0800 (PST) MIME-Version: 1.0 References: <835zvk9skp.fsf@gnu.org> <83o99c89qg.fsf@gnu.org> In-Reply-To: <83o99c89qg.fsf@gnu.org> From: Philipp Stephani Date: Tue, 25 Dec 2018 17:41:17 +0100 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.2 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Am So., 23. Dez. 2018 um 17:54 Uhr schrieb Eli Zaretskii : > > > From: Philipp Stephani > > Date: Sun, 23 Dec 2018 17:45:31 +0100 > > Cc: 33839@debbugs.gnu.org > > > > Am So., 23. Dez. 2018 um 16:22 Uhr schrieb Eli Zaretskii : > > > > > > > From: Philipp > > > > Date: Sun, 23 Dec 2018 03:28:53 +0100 > > > > > > > > (with-temp-buffer > > > > (let ((proc (make-process :name "test" > > > > :command '("bash" "-c" "echo stdout; echo stderr >&2") > > > > :buffer (current-buffer) > > > > :connection-type 'pipe > > > > :sentinel #'ignore > > > > :noquery t > > > > :coding '(utf-8-unix . utf-8-unix)))) > > > > (when (process-live-p proc) > > > > (process-send-eof proc)) > > > > (while (process-live-p proc) > > > > (accept-process-output proc)) > > > > (cl-assert (equal (buffer-string) "stdout\nstderr\n") :show-args))) > > > > > > > > Then evaluate this form repeatedly. Occasionally the buffer is empty > > > > and the assertion triggers. > > > > > > Isn't there an inherent race condition here? > > > > Maybe? If so, then it should be documented, with an explanation how to > > write this in a race-free manner. > > Can you tell why you used the process-live-p condition for calling > accept-process-output? Because it seemed logical to do so: while the process is running, wait for process output. I don't think the manual states that output can arrive after the process has finished, but if that's the case, then it should do so. > What happens if you do that unconditionally? You mean, without a while loop? It works in this specific case, but doesn't work in other cases, e.g. when a timer runs while the process is still running. To make this work reliably, a loop is required because there's no guarantee that accept-process-output returns only when the process has finished. From unknown Wed Jun 18 00:28:25 2025 X-Loop: help-debbugs@gnu.org Subject: bug#33839: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 25 Dec 2018 17:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33839 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Philipp Stephani Cc: 33839@debbugs.gnu.org Received: via spool by 33839-submit@debbugs.gnu.org id=B33839.154575891823042 (code B ref 33839); Tue, 25 Dec 2018 17:29:02 +0000 Received: (at 33839) by debbugs.gnu.org; 25 Dec 2018 17:28:38 +0000 Received: from localhost ([127.0.0.1]:36399 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbqVS-0005zZ-99 for submit@debbugs.gnu.org; Tue, 25 Dec 2018 12:28:38 -0500 Received: from eggs.gnu.org ([208.118.235.92]:53526) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gbqVQ-0005zM-Hz for 33839@debbugs.gnu.org; Tue, 25 Dec 2018 12:28:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gbqVI-0000ZO-7c for 33839@debbugs.gnu.org; Tue, 25 Dec 2018 12:28:31 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:52067) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gbqVI-0000ZG-4Z; Tue, 25 Dec 2018 12:28:28 -0500 Received: from [176.228.60.248] (port=2647 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gbqVH-0002XI-OR; Tue, 25 Dec 2018 12:28:28 -0500 Date: Tue, 25 Dec 2018 19:28:22 +0200 Message-Id: <83bm598qix.fsf@gnu.org> From: Eli Zaretskii In-reply-to: (message from Philipp Stephani on Tue, 25 Dec 2018 17:41:17 +0100) References: <835zvk9skp.fsf@gnu.org> <83o99c89qg.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Philipp Stephani > Date: Tue, 25 Dec 2018 17:41:17 +0100 > Cc: 33839@debbugs.gnu.org > > > What happens if you do that unconditionally? > > You mean, without a while loop? It works in this specific case, but > doesn't work in other cases, e.g. when a timer runs while the process > is still running. > To make this work reliably, a loop is required because there's no > guarantee that accept-process-output returns only when the process has > finished. What I meant was that this condition "refuses" to take output from the subprocess if it has exited. I'm questioning the correctness of this, since it could well be that the last portion of that output is still buffered somewhere, even though we already got SIGCHLD. I'm not sure I understand the example with a timer, but if in that situation your program is willing to accept output even after the subprocess already exited, does it work correctly then? From unknown Wed Jun 18 00:28:25 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Philipp Subject: bug#33839: closed (26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode) Message-ID: References: X-Gnu-PR-Message: they-closed 33839 X-Gnu-PR-Package: emacs Reply-To: 33839@debbugs.gnu.org Date: Thu, 27 Dec 2018 21:08:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1545944882-12597-1" This is a multi-part message in MIME format... ------------=_1545944882-12597-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #33839: 26.1.90; Emacs occasionally fails to receive asynchronous subproces= s output in batch mode which was filed against the emacs package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 33839@debbugs.gnu.org. --=20 33839: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D33839 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1545944882-12597-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 33839-done) by debbugs.gnu.org; 27 Dec 2018 21:07:03 +0000 Received: from localhost ([127.0.0.1]:39567 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gccrv-0003Fv-3y for submit@debbugs.gnu.org; Thu, 27 Dec 2018 16:07:03 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:38290) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gccrt-0003FP-EC for 33839-done@debbugs.gnu.org; Thu, 27 Dec 2018 16:07:01 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 2AB42160D8E; Thu, 27 Dec 2018 13:06:56 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Zg383imGezdj; Thu, 27 Dec 2018 13:06:55 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 3A6B4160D8F; Thu, 27 Dec 2018 13:06:55 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id dZfbABEDzSw5; Thu, 27 Dec 2018 13:06:55 -0800 (PST) Received: from [192.168.1.9] (cpe-23-242-74-103.socal.res.rr.com [23.242.74.103]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 0A2E4160CF0; Thu, 27 Dec 2018 13:06:55 -0800 (PST) To: Philipp From: Paul Eggert Subject: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Organization: UCLA Computer Science Department Message-ID: Date: Thu, 27 Dec 2018 13:06:54 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="------------6E34EAADB5AC26BAB63BD12E" Content-Language: en-US X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 33839-done Cc: 33839-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) This is a multi-part message in MIME format. --------------6E34EAADB5AC26BAB63BD12E Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit [This is a corrected version of email I mistakenly sent elsewhere.] > I don't think the manual states that output can > arrive after the process has finished, but if that's the case, then it > should do so. Good point, and I installed the attached patch into emacs-26 to try to do that. As this bug report seems to stem from a misunderstanding of accept-process-output (quite understandable, as its functionality is obscure) I'm taking the liberty of closing the report. If I'm wrong please feel free to reopen it. --------------6E34EAADB5AC26BAB63BD12E Content-Type: text/x-patch; name="0001-Improve-accept-process-process-doc.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-Improve-accept-process-process-doc.patch" >From c9fdd1b4965ebd02aa408f878320c4955f5e2cc7 Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Thu, 27 Dec 2018 12:52:45 -0800 Subject: [PATCH] Improve accept-process-process doc * doc/lispref/processes.texi (Accepting Output): * src/process.c (Faccept_process_output): Document that (accept-process-output P) can return non-nil even after P has exited, and that it can return nil even if P is still running (Bug#33839). --- doc/lispref/processes.texi | 7 +++++-- src/process.c | 7 ++++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/doc/lispref/processes.texi b/doc/lispref/processes.texi index 623be09cc6..2aca7f82a1 100644 --- a/doc/lispref/processes.texi +++ b/doc/lispref/processes.texi @@ -1795,7 +1795,8 @@ Accepting Output This function allows Emacs to read pending output from processes. The output is given to their filter functions. If @var{process} is non-@code{nil} then this function does not return until some output -has been received from @var{process}. +has been received from @var{process} or @var{process} has closed the +connection. The arguments @var{seconds} and @var{millisec} let you specify timeout periods. The former specifies a period measured in seconds and the @@ -1820,7 +1821,9 @@ Accepting Output The function @code{accept-process-output} returns non-@code{nil} if it got output from @var{process}, or from any process if @var{process} is -@code{nil}. It returns @code{nil} if the timeout expired before output +@code{nil}; this can occur even after a process has exited if the +corresponding connection contains buffered data. The function returns +@code{nil} if the timeout expired or the connection was closed before output arrived. @end defun diff --git a/src/process.c b/src/process.c index e306b2ae9e..4dafee8cbe 100644 --- a/src/process.c +++ b/src/process.c @@ -4581,8 +4581,8 @@ DEFUN ("accept-process-output", Faccept_process_output, Saccept_process_output, 0, 4, 0, doc: /* Allow any pending output from subprocesses to be read by Emacs. It is given to their filter functions. -Optional argument PROCESS means do not return until output has been -received from PROCESS. +Optional argument PROCESS means to return only after output is +received from PROCESS or PROCESS closes the connection. Optional second argument SECONDS and third argument MILLISEC specify a timeout; return after that much time even if there is @@ -4594,7 +4594,8 @@ If optional fourth argument JUST-THIS-ONE is non-nil, accept output from PROCESS only, suspending reading output from other processes. If JUST-THIS-ONE is an integer, don't run any timers either. Return non-nil if we received any output from PROCESS (or, if PROCESS -is nil, from any process) before the timeout expired. */) +is nil, from any process) before the timeout expired or the +corresponding connection was closed. */) (Lisp_Object process, Lisp_Object seconds, Lisp_Object millisec, Lisp_Object just_this_one) { -- 2.17.1 --------------6E34EAADB5AC26BAB63BD12E-- ------------=_1545944882-12597-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 23 Dec 2018 02:29:10 +0000 Received: from localhost ([127.0.0.1]:60376 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gatVu-0007DP-6o for submit@debbugs.gnu.org; Sat, 22 Dec 2018 21:29:10 -0500 Received: from eggs.gnu.org ([208.118.235.92]:41811) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gatVs-0007D6-I4 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 21:29:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gatVm-0000Hx-C0 for submit@debbugs.gnu.org; Sat, 22 Dec 2018 21:29:03 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=BAYES_50, FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:34732) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gatVm-0000Hr-9H for submit@debbugs.gnu.org; Sat, 22 Dec 2018 21:29:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59148) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gatVl-00087g-6R for bug-gnu-emacs@gnu.org; Sat, 22 Dec 2018 21:29:02 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gatVi-0000EO-2m for bug-gnu-emacs@gnu.org; Sat, 22 Dec 2018 21:29:01 -0500 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]:39992) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gatVh-0000Dw-SQ for bug-gnu-emacs@gnu.org; Sat, 22 Dec 2018 21:28:58 -0500 Received: by mail-wm1-x32f.google.com with SMTP id f188so9131229wmf.5 for ; Sat, 22 Dec 2018 18:28:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version; bh=2SZr0IkZI0uQP/44F6FoYXtXPOGyqo8WSMUkPlf+fLE=; b=aWXzW8MDt2jf3/kuH4IS1D2KjE5CNJFvipgQczvsx3XSzZvBCx8vkPNnGCdWQgYObx 2e2qBcWzKORyEHBAoV11xghjqToj5Dexq95WYx4bvdXL3pNDfpjRGb/aTCWDg+U1NPXn yI0wy7ycM4EIqQGYIv9ApuYAN9No7tY1n26V2UdqxV7cWnSHLUyOYRzyDrHfHbKzUfte GfeYkiXtTQ99Gg1s61KHaaeyGl/onWCIoUSdBsPdgPvBGSdQAYc+nZoPpL6gYM7D+W0o 6fyrjMI6MtvDJGmP2Mq/SmV8IbSIHE6YmCtmN00KZ1v8lTbUFNyc6l2P0S6Zr3Iqhfgo 9U8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version; bh=2SZr0IkZI0uQP/44F6FoYXtXPOGyqo8WSMUkPlf+fLE=; b=RkFx13uyuBr/bcUusdNXYNZKpdVoCVglEM32YNTksc1vNkLfwd8mbUUT0T3EmnI9Iy 1mzPPDW0X2APlorcrISlWJDaq0Q7igM8syr9gYNHCI5nPCUu4NdR/m3/UlbssQMzRom7 mdaMNmSwjHfjoknxW9m0El5gk5jb2EhR9TJNZP592mLoVizGjvxYq3xJwNk0H3hBb2aW 0Z/p4awyq4SoKzOouBLBT6pFVd0ZUMjZ0LodReT0ZuSH9T0bqdi/U9cRvfrQM/ObYR6u y3UA0Wzu5nf7ojJCdTYLBPT1F7T1731ZoXGNv8c30/oT09+meWln1DRVdQH0CCfZTUoF f8wQ== X-Gm-Message-State: AA+aEWaMxzHPKrY+337gbzGoNW9CHEodWbcVNfOZ4M++moeq5NcJlJZH 5QgYx4P2xsLSiv9aQdFSDB3l/+nZ X-Google-Smtp-Source: AFSGD/WLLvNAhBOuTGDB9ZqmFxrcMgLjRpRKjQu2w5dUXRO8g6SLQFkq9vTcDFJknu8UqGzCPGGC1Q== X-Received: by 2002:a1c:2547:: with SMTP id l68mr7749679wml.11.1545532136281; Sat, 22 Dec 2018 18:28:56 -0800 (PST) Received: from p (p579979C6.dip0.t-ipconnect.de. [87.153.121.198]) by smtp.gmail.com with ESMTPSA id o17sm31345292wmg.35.2018.12.22.18.28.54 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Dec 2018 18:28:55 -0800 (PST) From: Philipp To: bug-gnu-emacs@gnu.org Subject: 26.1.90; Emacs occasionally fails to receive asynchronous subprocess output in batch mode Date: Sun, 23 Dec 2018 03:28:53 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -3.8 (---) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.8 (----) Add the following code to /tmp/form.el: (require 'cl-lib) (with-temp-buffer (let ((proc (make-process :name "test" :command '("bash" "-c" "echo stdout; echo stderr >&2") :buffer (current-buffer) :connection-type 'pipe :sentinel #'ignore :noquery t :coding '(utf-8-unix . utf-8-unix)))) (when (process-live-p proc) (process-send-eof proc)) (while (process-live-p proc) (accept-process-output proc)) (cl-assert (equal (buffer-string) "stdout\nstderr\n") :show-args))) Then evaluate this form repeatedly. Occasionally the buffer is empty and the assertion triggers. For example: $ for i in {1..100}; do emacs -Q -batch -l /tmp/form.el ; done Assertion failed: (equal (buffer-string) "stdout stderr "), "" Assertion failed: (equal (buffer-string) "stdout stderr "), "" i.e. here the output didn't arrive in 2 of 100 iterations. In GNU Emacs 26.1.90 (build 5, x86_64-apple-darwin18.0.0, NS appkit-1671.10 Version 10.14.1 (Build 18B75)) of 2018-12-22 Repository revision: 24ddea074a2e61f7accde60cdf941ba67b1ce82a Windowing system distributor 'Apple', version 10.3.1671 Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Configured using: 'configure --enable-checking --enable-check-lisp-object-type --enable-gtk-deprecation-warnings --with-modules --without-xml2 --without-pop --with-mailutils --enable-gcc-warnings=warn-only MAKEINFO=/usr/local/opt/texinfo/bin/makeinfo 'CFLAGS=-O0 -ggdb3' LDFLAGS=-O0' Configured features: JPEG NOTIFY ACL GNUTLS ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS LCMS2 Important settings: value of $LANG: de_DE.UTF-8 locale-coding-system: utf-8-unix Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc puny seq dired dired-loaddefs format-spec rfc822 mml easymenu mml-sec epa derived epg epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils phst pcase ffap thingatpt url-parse auth-source cl-seq eieio byte-opt bytecomp byte-compile cconv eieio-core cl-macs gv eieio-loaddefs password-cache url-vars subr-x rx gnutls dbus xml cl-loaddefs cl-lib elec-pair time-date tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads kqueue cocoa ns lcms2 multi-tty make-network-process emacs) Memory information: ((conses 16 216710 8580) (symbols 48 21650 0) (miscs 40 59 186) (strings 32 32914 1496) (string-bytes 1 892039) (vectors 16 37089) (vector-slots 8 751562 17790) (floats 8 56 60) (intervals 56 210 0) (buffers 992 11)) ------------=_1545944882-12597-1--