From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 08 Feb 2021 21:22:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 46388@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.161281927217926 (code B ref -1); Mon, 08 Feb 2021 21:22:02 +0000 Received: (at submit) by debbugs.gnu.org; 8 Feb 2021 21:21:12 +0000 Received: from localhost ([127.0.0.1]:52037 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Dy3-0004f3-HT for submit@debbugs.gnu.org; Mon, 08 Feb 2021 16:21:11 -0500 Received: from lists.gnu.org ([209.51.188.17]:33764) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Dxx-0004es-Rq for submit@debbugs.gnu.org; Mon, 08 Feb 2021 16:21:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:35244) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l9Dxw-0001Ul-FR for bug-gnu-emacs@gnu.org; Mon, 08 Feb 2021 16:21:04 -0500 Received: from mail-ot1-x334.google.com ([2607:f8b0:4864:20::334]:33840) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l9Dxt-0004kG-Hz for bug-gnu-emacs@gnu.org; Mon, 08 Feb 2021 16:21:04 -0500 Received: by mail-ot1-x334.google.com with SMTP id y11so15588147otq.1 for ; Mon, 08 Feb 2021 13:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=1nJFGSGOvcoMKoIwe+VBpx4DDYI4g7XQetIaBigsn5M=; b=bpKP9I22XQyvIAdkEoBI2CmDeG3XPDb30iscC/DDucBha41UJqQqgESkXaKXTsAcOo ScvHZsfyCN3wnTAFw5+o+MlvmAKXBN/QYpt0ubZVaxDM0qJIM6VlYZUeaxg/FF1245J0 VIDAm3tTlt8ZsPOk4IhqTvixhWYLIGdmT8fzfIVrPL7sGEW8Idi7MkqZDv3Sgkgp5xlD YSwgr+VZJ5jozE7chdnaqICKbE1jy5ihRQXjARaytZldhxVHeoUNefu25Jnr9yWxxUyA twNeNxrzUJZKznpERBpDjBVst+OgCTYC+cW2XVOP43spC5hHmKEyczsKJFV8gluGJ4HB Lt4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=1nJFGSGOvcoMKoIwe+VBpx4DDYI4g7XQetIaBigsn5M=; b=XTkSmNChrtH7PngfgGjxGMwoiy+wwtmTpyUmdvRta6bYXjzsGgGdpNSolBCezP1wDB Gdvy9ZdkV+Az/F0J9HeZfl7dY0XhNzZrRrCo0k73gt0Gn1XAOCHI3Pbt/4kUEgV0DIpA Ybwrl7I8DAuC7ospiae4Q9+MIDFLRMTteJuxc4n6jfP3gy64VAg2EMH6gtGqgdpBZ3j7 6IenJD/GEFta9UqpacZfP1wNM+5GPoGFp60iVWJY1Qjhqv++RhJF7oW7DoeSXvM4hEUB 9nIxlySuMFq0A7Jhdc70obMMdYV8tcdzCfNSFSBHsnxm8Fxln7llBu16nzG1pt/hkMkz 13/g== X-Gm-Message-State: AOAM531N66xz0wvKiCah/ODrjHlaQ3lQNavCi/VsTerbLDkqQW8GEitw M4/1pNXK5xtq0P9XjPMvzDyC7q2wgINcPkyg5eanjshYp6akJQ== X-Google-Smtp-Source: ABdhPJwcrSCRhFvb6x4FjjxXLyAxHx3ZTaUnK/ueqRUenKq7KOnqh7TTPHe7+0emewB3/AAAkzZDunb0PAvAqaVHP8Y= X-Received: by 2002:a9d:3b26:: with SMTP id z35mr13613555otb.192.1612819256991; Mon, 08 Feb 2021 13:20:56 -0800 (PST) MIME-Version: 1.0 From: Ioannis Kappas Date: Mon, 8 Feb 2021 21:20:47 +0000 Message-ID: Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::334; envelope-from=ioannis.kappas@gmail.com; helo=mail-ot1-x334.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) There appears to be an issue with the /message/ function when emacs is invoked in -batch mode on windows-nt from outside the windows console (i.e. not directly from the command prompt). Messages are not displayed to the user until emacs exits. Consider the following command from example, which suppose to print the message "hi" first to the user and then exit after 5 seconds: : emacs -Q --batch --eval="(progn (message \"hi\") (sit-for 5))" When this command is invoked from within an emacs eshell on windows, the message is only displayed to the user after about 5 seconds, i.e when emacs is about to exit. Here is a small summary running the above under Linux and windows native: | system | invoked from | terminal output | |---------------------+----------------+--------------------------------------------| | x86_64-pc-linux-gnu | terminal | prints "hi", then after 5 seconds it exits | | x86_64-pc-linux-gnu | emacs eshell | prints "hi", then after 5 seconds it exits | | x86_64-w64-mingw32 | command prompt | prints "hi", then after 5 seconds it exits | | x86_64-w64-mingw32 | emacs eshell | after 5 seconds prints "hi", then exits | The issue on windows is not exclusive to invoking emacs -batch from within emacs (e.g. from eshell), it applies to any invocation that happens outside the command prompt, e.g. from inside the MSYS2 mintty terminal. It appears as if the issue is caused by a different stderr buffering mode used when a program is invoked outside of the command prompt. Analysis to follow. In GNU Emacs 27.1 (build 1, x86_64-w64-mingw32) of 2020-11-19 built on fv-az68-340 Repository revision: ec297125a76481c55390d0b329e541907879d6f3 Repository branch: master Windowing system distributor 'Microsoft Corp.', version 10 From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt References: In-Reply-To: Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 08 Feb 2021 21:43:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161282054827971 (code B ref 46388); Mon, 08 Feb 2021 21:43:01 +0000 Received: (at 46388) by debbugs.gnu.org; 8 Feb 2021 21:42:28 +0000 Received: from localhost ([127.0.0.1]:52064 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9EIe-0007H3-CA for submit@debbugs.gnu.org; Mon, 08 Feb 2021 16:42:28 -0500 Received: from mail-oo1-f48.google.com ([209.85.161.48]:39633) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9EIb-0007Gq-Hn for 46388@debbugs.gnu.org; Mon, 08 Feb 2021 16:42:27 -0500 Received: by mail-oo1-f48.google.com with SMTP id z36so3800938ooi.6 for <46388@debbugs.gnu.org>; Mon, 08 Feb 2021 13:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=yHf2sAFP2LMWhXB8Ra91Ac4AHQI+YetmhHNFrZHr9SM=; b=hlMcI9ajTf58upgoS5DPfL8+QDN/eQdDGo1IEi/c0ANaZ0128dpBtkXCR8HCF1o3dX Z7epxF/O0jFrDhXE54R5MnxPCMbRNMxTIMbkOn5Rny0HJ7DRRTE9YV090TuZ6MJ9tOSo 13z6tephzPQm6QHg9ot+GbfYT4hHaBN3hsuVofDT4emyK5yaIS5t479eZRwnuXha8CZw LHgEs+B3DLGoX0jB9LdJ+ZLm1eSvT4CdIQmRZ7xxW+sUsJfwA67zPed93tjAwyK7rTYI fmRi+UyaouRqf2tyOmjKlsSU0tG0PfAn+LFJnuHP5jQXj9TuungwCoU/lEA0RctuyhI1 8q1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=yHf2sAFP2LMWhXB8Ra91Ac4AHQI+YetmhHNFrZHr9SM=; b=EjV8p5R56TCUnAJ5XMQppfJsCrX3gNgSGA52Fk7+PEdRdeboxjLyMntjWPiTvf95+g BLJcvwdfADu6A/kR4xk1PnErzzIU2J5eWVSNNeI8zkoA0n9Y6up08Aoa+0zTO1yR4+rM YCdqbLjOZIxXzyWWj8jdKqkdV056YI3bKoPVEn94Ndj7cb3RVdmlJbvDUObM05WGw5AD 3osaQT3ZQukmIFGSXaVTHSrKddWCXpaOv30PxGE2EEabgX0HTJu3ZOSWk5U0vFTAt3q2 0/dwmGsEizFug8DVKaHa3NQlQe3k7G5chkSkBMMcNVP8DdnKJlyb6l9L0tkTx8b5CVIU rdOQ== X-Gm-Message-State: AOAM5314V813ibAZ2nqxr7q2M5ay4pd2Y0hGT6CjSgnlk9mevZsI3Njh yNm3920st8zOkiUabx49yJkg4OP02YJTQK1MJ+0dnagCe+z+0A== X-Google-Smtp-Source: ABdhPJwcsgORFcGnfFeXIrxZKN+PghuafeJlxdK1P2Gf+sHPRnau50nyAnjZnO7TaTlqbJVJ8cFSKHCJzDfzpAa8QV0= X-Received: by 2002:a4a:5787:: with SMTP id u129mr5281631ooa.81.1612820539761; Mon, 08 Feb 2021 13:42:19 -0800 (PST) MIME-Version: 1.0 From: Ioannis Kappas Date: Mon, 8 Feb 2021 21:42:09 +0000 Message-ID: Content-Type: multipart/mixed; boundary="0000000000005c30c605bada0b06" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --0000000000005c30c605bada0b06 Content-Type: text/plain; charset="UTF-8" Some analysis follows Looking at the code, it appears that /message/s in -batch mode are delivered to the process' standard error by the [editfnc.c:message()] ->[xdisp.c:message3_nolog()] ->[xdispl.c:message_to_stderr()] function, i.e. messages are written to stderr. The latter function uses the standard fwrite and fputc calls to deliver the message to stderr. The buffering regime used for stderror on windows appears to be different depending whether a program was started from the command prompt or not. When started from the command prompt stderr is unbuffered, while there is a buffer employed otherwise and output is only delivered when that buffer is full. Below is a sample C program that can demonstrate the behaviour, which writes a single character to the standard output and then exits after 2 seconds: #include #include #include int main() { fwrite("t", 1, sizeof("t"), stderr); sleep(2); return 0; } | case | invoked from | result | |------+------------------------+-------------------------------------------| | 1 | command prompt | prints "t", then after 2 seconds exits | | 2 | outside command prompt | after about 2 seconds prints "t", then exits | It appears that the buffer size used in #2 is 2048 bytes (i.e. the message is only immediately displayed when is more than 2048 bytes long). A solution thus to correct this behavior in emacs -batch on windows-nt would be to check if it is connected to the console, and when not, set stderr mode to unbuffered. A) Likely the win32 api provide GetConsoleMode() that can be used to check if a standard HANDLE is attached to the console. Given the STD_ERROR_HANDLE as an argument, it will return non-zero if is attached to the console or a 0 otherwise, setting GetLastError() to "The handle is invalid" message. B) To set stderr to an unbuffered mode, we can use standard C's setvbuf() with _IONBF (no buffering). C) The location where the descriptors are prepared for use in emacs appear to be at w32.c:init_ntproc(). If the above analysis is correct, here is an example patch that puts A/B/C together: --- a/src/w32.c +++ b/src/w32.c @@ -10216,6 +10216,19 @@ init_ntproc (int dumping) else _open ("nul", O_TEXT | O_NOINHERIT | O_WRONLY); _fdopen (2, "w"); + + /* When in -batch mode, windows appears to buffer stderr when it + is invoked outside of the command prompt. This has the + undesired side effect that /message/s are not output unless the + buffer is full or emacs exits */ + DWORD console_mode; + if (noninteractive && + // stderr is not attached to the console + !GetConsoleMode (GetStdHandle(STD_ERROR_HANDLE), &console_mode)) + { + // set stderr to unbuffered + setvbuf (stderr, NULL, _IONBF, 0); + } I have also attached an ert tests to capture the correct behavior. The test invokes an emacs -batch process which prints a /message/ and exits after five seconds. The test check immediately after two seconds if the batch process has output the message as expected. It can be invoked with: : emacs.exe -batch -l ert -l batch-message-test.el -f ert-run-tests-batch-and-exit --0000000000005c30c605bada0b06 Content-Type: application/octet-stream; name="batch-message-test.el" Content-Disposition: attachment; filename="batch-message-test.el" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_kkx3l92a0 Ozs7IC0qLSBsZXhpY2FsLWJpbmRpbmc6IHQ7IC0qLQ0KKHJlcXVpcmUgJ2VydCkNCg0KKGVydC1k ZWZ0ZXN0IGJhdGNoLW1lc3NhZ2UgKCkNCiAgIlRlc3QgdGhhdCB3aGVuIGludm9raW5nIGVtYWNz IGluIGJhdGNoIG1vZGUgYXMgYQ0Kc3VicHJvY2VzcyAoaS5lLiBub3QgZGlyZWN0bHkgZnJvbSBh IHRlcm1pbmFsKSwgL21lc3NhZ2UvcyBhcmUNCmZsdXNoZWQgdG8gb3V0cHV0IGltbWVkaWF0ZWx5 LiAiDQogIChtZXNzYWdlICI6c3lzdGVtICVzIDp2ZXJzaW9uICVzIiBzeXN0ZW0tY29uZmlndXJh dGlvbiBlbWFjcy12ZXJzaW9uKQ0KDQogICh3aXRoLXRlbXAtYnVmZmVyIA0KICAgIChsZXQqICgo cHJvYy1idWYgKGN1cnJlbnQtYnVmZmVyKSkNCg0KCSAgIDs7IHN0YXJ0IGEgbmV3IGVtYWNzIHBy b2Nlc3MgdGhhdCB3aWxsIHByaW50IGEgbWVzc2FnZSwNCgkgICA7OyB3YWl0IGZvciBmaXZlIHNl Y29uZCBhbmQgdGhlbiBleGl0Lg0KCSAgIChjbWQgKGZvcm1hdCAiZW1hY3MgLVEgLS1iYXRjaCAt LWV2YWw9XCIlc1wiIg0KCQkJJyhwcm9nbiAobWVzc2FnZSBcXFwiaGlcXFwiKSANCgkJCQkoc2l0 LWZvciA1KSkpKQ0KCSAgIChwcm9jIChzdGFydC1maWxlLXByb2Nlc3Mtc2hlbGwtY29tbWFuZA0K ICAgICAgICAgICAgICAgICAgInRlc3QvYmF0Y2gtbWVzc2FnZSIgcHJvYy1idWYgY21kKSkNCg0K CSAgIChvdXRwdXRzICcoKSkpDQogICAgICANCiAgICAgIDs7IGNhcHR1cmUgZW1hY3Mgb3V0cHV0 DQogICAgICAoc2V0LXByb2Nlc3MtZmlsdGVyIHByb2MgKGxhbWJkYSAocHJvYyBvdXRwdXQpDQoJ CQkJIChwdXNoIG91dHB1dCBvdXRwdXRzKSkpDQogICAgICANCiAgICAgIDs7IHdhaXQgZm9yIHRo ZSBwcm9jZXNzIHRvIHN0YXJ0DQogICAgICAoc2xlZXAtZm9yIDIpDQogICAgICAoc2hvdWxkIChl cXVhbCAncnVuIChwcm9jZXNzLXN0YXR1cyBwcm9jKSkpDQogICAgICA7OyBwcm9ncmFtIHNob3Vs ZCBoYXZlIHByaW50ZWQgb3V0ICJoaVxuIg0KICAgICAgKHNob3VsZCAoZXF1YWwgJygiaGlcbiIp IG91dHB1dHMpKQ0KDQogICAgICA7OyBraWxsIHByb2Nlc3MgYW5kIHdhaXQgZm9yIGl0IHRvIGRp ZQ0KICAgICAgKGRlbGV0ZS1wcm9jZXNzIHByb2MpDQogICAgICAoc2xlZXAtZm9yIDEpKSkpDQoN Cg== --0000000000005c30c605bada0b06-- From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 09 Feb 2021 03:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ioannis Kappas Cc: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161284193810820 (code B ref 46388); Tue, 09 Feb 2021 03:39:01 +0000 Received: (at 46388) by debbugs.gnu.org; 9 Feb 2021 03:38:58 +0000 Received: from localhost ([127.0.0.1]:52313 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Jrd-0002oS-J3 for submit@debbugs.gnu.org; Mon, 08 Feb 2021 22:38:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48934) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Jra-0002oD-O0 for 46388@debbugs.gnu.org; Mon, 08 Feb 2021 22:38:55 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37634) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9JrV-00085g-Gd; Mon, 08 Feb 2021 22:38:49 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3155 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l9JrU-0006h3-Qk; Mon, 08 Feb 2021 22:38:49 -0500 Date: Tue, 09 Feb 2021 05:38:46 +0200 Message-Id: <835z31lxkp.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (message from Ioannis Kappas on Mon, 8 Feb 2021 21:20:47 +0000) References: X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Ioannis Kappas > Date: Mon, 8 Feb 2021 21:20:47 +0000 > > There appears to be an issue with the /message/ function when > emacs is invoked in -batch mode on windows-nt from outside the > windows console (i.e. not directly from the command > prompt). Messages are not displayed to the user until emacs exits. > > Consider the following command from example, which suppose to > print the message "hi" first to the user and then exit after 5 seconds: > > : emacs -Q --batch --eval="(progn (message \"hi\") (sit-for 5))" > > When this command is invoked from within an emacs eshell on > windows, the message is only displayed to the user after about 5 seconds, > i.e when emacs is about to exit. I believe this is because of buffering: Eshell on MS-Windows invokes subprograms via pipes, which are buffered. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 09 Feb 2021 05:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ioannis Kappas Cc: 46388@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161284900221384 (code B ref 46388); Tue, 09 Feb 2021 05:37:01 +0000 Received: (at 46388) by debbugs.gnu.org; 9 Feb 2021 05:36:42 +0000 Received: from localhost ([127.0.0.1]:52380 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Lha-0005Yp-Cg for submit@debbugs.gnu.org; Tue, 09 Feb 2021 00:36:42 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38118) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9LhY-0005Yd-0H for 46388@debbugs.gnu.org; Tue, 09 Feb 2021 00:36:41 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:39049) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9LhS-0001fZ-QB; Tue, 09 Feb 2021 00:36:34 -0500 Received: from eliz by fencepost.gnu.org with local (Exim 4.82) (envelope-from ) id 1l9LhR-0002lF-EV; Tue, 09 Feb 2021 00:36:33 -0500 From: Eli Zaretskii In-Reply-To: (message from Ioannis Kappas on Mon, 8 Feb 2021 21:42:09 +0000) References: Message-Id: Date: Tue, 09 Feb 2021 00:36:33 -0500 X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Ioannis Kappas > Date: Mon, 8 Feb 2021 21:42:09 +0000 > > A solution thus to correct this behavior in emacs -batch on > windows-nt would be to check if it is connected to the console, > and when not, set stderr mode to unbuffered. Thanks for the analysis and the patch proposal. However, I don't think this is a good idea. For starters, stderr could be connected to a file, in which case we do want it to be buffered. More generally, stderr could be used for something other than outputting urgent messages, in which case making it unbuffered will make I/O less efficient for no good reason. And this would make Emacs on Windows behave differently from Posix systems, which is also a downside. I think a much better way forward in this area is to teach Emacs to use the Pseudo Consoles introduced in recent Windows versions. That would allow us to support subprocess communications via PTYs on MS-Windows, and thus will solve this and other similar issues. Patches to support PTYs on Windows are welcome. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 09 Feb 2021 20:16:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161290175827624 (code B ref 46388); Tue, 09 Feb 2021 20:16:01 +0000 Received: (at 46388) by debbugs.gnu.org; 9 Feb 2021 20:15:58 +0000 Received: from localhost ([127.0.0.1]:54742 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9ZQT-0007As-7C for submit@debbugs.gnu.org; Tue, 09 Feb 2021 15:15:58 -0500 Received: from mail-ot1-f41.google.com ([209.85.210.41]:35652) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9ZQN-00070b-IZ for 46388@debbugs.gnu.org; Tue, 09 Feb 2021 15:15:56 -0500 Received: by mail-ot1-f41.google.com with SMTP id k10so16320600otl.2 for <46388@debbugs.gnu.org>; Tue, 09 Feb 2021 12:15:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=GrjDOgnTSryxhZUfd1hXJZ/5xSTLqcl1eEX7IyGuf+s=; b=NjnEok8TlEdRRf0po7fxh+Q/rmF3wKW9OLTllZ1Ji3o6LEhqUUlJOZsjss7F3wlvjO ST/dKSLPE6iH683P2J6uZOtQRigkameHm3xdDyZ6IDXXt0huT8w29r0ByuK9xU+sSvY4 0MN4SrTFjaU2oeBx4A42CgBmfrqxHCAHAouK1zvYDy4KrwhE+ecX/qxluifn/6TWSmBG 1ma5IYe2ps3xFCFE4DVv6iLUvn35ZmvQU7pTaM4bvs6i/CBRFGnU5YZDE5QXssCV0v9e 6SCxShLrBIToeKCflCShV2eCDaIvaK/bwtcbTqqP0oNvZugzSvvMTIrP9/akEAJz1JVZ EyWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=GrjDOgnTSryxhZUfd1hXJZ/5xSTLqcl1eEX7IyGuf+s=; b=ss2X2sH21v5RWG502XyL90eXZ+nxmEdXVS3m8IsFia64Z4l5PARmxVntet59rk9g/x 6BWkIyD1+zlwtblGZsQi3dh8qASP+TuUP0A0pDLr4XdracX1jV7bi8n6xSsnKWrmJzFB Cg/3HWLmOdW5y5LJxevrksNZ8rjAMN+/TE+crKpiZ2IR9TtwQjlts6USqu66zMfiY8/u mA9EaZ3GRqH/AtEboIY+y1oetjAhvIUWm/NN0uq43TTBiRoNDk0FBAmCozFA0bltZP7/ 7NU8hiFOqJIwahttU1/OIsJfkWkUZxzqS0pkJegzde6471ZhxkF1N7ObRgE6sY33tdhQ mTmg== X-Gm-Message-State: AOAM531X/CCKV/ThgE5TqIWipkWfwMf+oOxQsckPXyXbAz5bkI4eDdPJ zp0YekMuyV57KeANA5wWMhpAmcBGvghzxp4iJdk= X-Google-Smtp-Source: ABdhPJypC17Znnvd5CofAw2lIpT7Fio0yQK55P3BIKonNpC+dYacsff0g1IHgmburT4Sr6xKzxgO2RqHsswlJ9plXwo= X-Received: by 2002:a9d:3b26:: with SMTP id z35mr16731361otb.192.1612901745885; Tue, 09 Feb 2021 12:15:45 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Ioannis Kappas Date: Tue, 9 Feb 2021 20:15:34 +0000 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Eli, thanks for taking the time to look into this bug report! (my apologies for the long reply that follows) I understand there are two concerns: 1. stderr could be connected to a file, in which case we do want it to be buffered for efficiency. 2. If stderr is forced to unbuffered, this will make it behave differently from posix systems. I had a look around #2 in the posix standard. The section under stderr, stdin, stdout at https://pubs.opengroup.org/onlinepubs/9699919799/functions/stdin.html reads: """At program start-up, three streams shall be predefined and need not be opened explicitly: standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). When opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device. """ Thus, at open (I assume this to mean at program invocation), the posix spec suggests that stderr is not "fully buffered". The section under setvbuf under the same spec at https://pubs.opengroup.org/onlinepubs/009695399/functions/setvbuf.html differentiates between the three different buffering regimes: """ {_IOFBF} shall cause input/output to be fully buffered. {_IOLBF} shall cause input/output to be line buffered. {_IONBF} shall cause input/output to be unbuffered. """ So, if stderr is not "fully buffered", it can be either "line buffered" or "unbuffered". (This does make some sense to me, since stderr is usually for urgent messages and thus any messages are needed to be readily available to the recipients. Fully buffered defeats the purpose?). How does Linux conforms to the posix standard? Having a look at stderr(3) in the foot note, it reads: """ Notes The stream stderr is unbuffered. The stream stdout is line-buffered when it points to a terminal. Partial lines will not appear until fflush(3) or exit(3) is called, or a newline is printed. This can produce unexpected results, especially with debugging output. The buffering mode of the standard streams (or any other stream) can be changed using the setbuf(3) or setvbuf(3) call. Note that in case stdin is associated with a terminal, there may also be input buffering in the terminal driver, entirely unrelated to stdio buffering. (Indeed, normally terminal input is line buffered in the kernel.) This kernel input handling can be modified using calls like tcsetattr(3); see also stty(1), and termios(3). """ Which appears to suggests that stderr always starts as "unbuffered" when a program starts, irrespective of how it was invoked? If so, then it conforms to the posix standard of not being "fully buffered" at open, by always being "unbuffered" (this behavior has also been seen in all tests mentioned so far). If this is the case, then the desired buffering efficiency as mentioned in concern #1, does not exist in Linux. I have also conducted an additional test to check Linux behavior when stderr is redirected to a file (using the same c program that I mentioned in the original bug report): : ./fwrite-a-character-to-stderr-and-sleep-for-two-seconds 2> io.txt while at around the same time in another terminal I read from that file: : cat io.txt The result is that cat outputs "t", the character the fwrite program has written to stderr, i.e. the io.txt file, proving that no buffering has being utilized by the fwrite process. The result is the same when I try the same experiment from inside an emacs shell. Back to the windows behavior at hand, the simple test conducted with fwrite as the bug report mentioned, has a 2048 bytes associated with it which I will consider it to be categorized as "fully buffered", which brings it further away to the posix standard and even more further away from Linux being on the other end (always unbuffered). (There is also this thread on stackoverlow quoting the following from comp.lang.c: """ Unix convention is that stdin and stdout are line-buffered when associated with a terminal, and fully-buffered (aka block-buffered) otherwise. stderr is always unbuffered. """ emphasis on *Unix convention ... stderr .. always unbuffered*, though it will be difficult to confirm this claim for all *nices) Thus, if we assume the above interpretation given so far to be correct, then both of the concerns can be addressed as such: For #1, we can use Linux as a benchmark of established behavior. There is no efficiency in Linux with regards to buffering when stderr is redirected to files, thus having emacs batch mode on windows being on par with Linux should be good enough in this respect. For #2, emacs batch stderr behavior when invoked from outside the command line is not compatible with the posix standard, since it uses "fully buffered" mode, which actually makes it behave very differently from Linux (which is posix compliant by always having stderr as being unbuffered). Thus, the suggested patch actually addresses this concern, rather than invalidating it. (I understand ConPTY is the long term solution looking for a hacker, though I still do suppose the current behavior on windows is odd and needs to be addressed until that time). Sincerely On Tue, Feb 9, 2021 at 5:36 AM Eli Zaretskii wrote: > > > From: Ioannis Kappas > > Date: Mon, 8 Feb 2021 21:42:09 +0000 > > > > A solution thus to correct this behavior in emacs -batch on > > windows-nt would be to check if it is connected to the console, > > and when not, set stderr mode to unbuffered. > > Thanks for the analysis and the patch proposal. However, I don't > think this is a good idea. For starters, stderr could be connected to > a file, in which case we do want it to be buffered. More generally, > stderr could be used for something other than outputting urgent > messages, in which case making it unbuffered will make I/O less > efficient for no good reason. And this would make Emacs on Windows > behave differently from Posix systems, which is also a downside. > > I think a much better way forward in this area is to teach Emacs to > use the Pseudo Consoles introduced in recent Windows versions. That > would allow us to support subprocess communications via PTYs on > MS-Windows, and thus will solve this and other similar issues. > > Patches to support PTYs on Windows are welcome. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 09 Feb 2021 20:53:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ioannis Kappas Cc: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.16129039503073 (code B ref 46388); Tue, 09 Feb 2021 20:53:01 +0000 Received: (at 46388) by debbugs.gnu.org; 9 Feb 2021 20:52:30 +0000 Received: from localhost ([127.0.0.1]:54814 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Zzp-0000nV-W9 for submit@debbugs.gnu.org; Tue, 09 Feb 2021 15:52:30 -0500 Received: from eggs.gnu.org ([209.51.188.92]:32828) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Zzo-0000nK-Us for 46388@debbugs.gnu.org; Tue, 09 Feb 2021 15:52:29 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55376) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9Zzh-0002wH-DT; Tue, 09 Feb 2021 15:52:22 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3562 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l9Zzc-0005IT-ND; Tue, 09 Feb 2021 15:52:19 -0500 Date: Tue, 09 Feb 2021 22:52:13 +0200 Message-Id: <83h7mlj75u.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (message from Ioannis Kappas on Tue, 9 Feb 2021 20:15:34 +0000) References: X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Ioannis Kappas > Date: Tue, 9 Feb 2021 20:15:34 +0000 > Cc: 46388@debbugs.gnu.org > > Which appears to suggests that stderr always starts as "unbuffered" > when a program starts, irrespective of how it was invoked? AFAIR, on GNU/Linux stderr is not unbuffered by default. We had a long discussion of related issues in June 2019, I think people who know about the glibc internals told there stderr is not really completely unbuffered (and Posix allows that). On MS-Windows it is by default unbuffered. > Back to the windows behavior at hand, the simple test conducted with > fwrite as the bug report mentioned, has a 2048 bytes associated with > it which I will consider it to be categorized as "fully buffered", > which brings it further away to the posix standard and even more further > away from Linux being on the other end (always unbuffered). I'm not sure your conclusion is correct. I think the buffering is imposed by the pipes which Emacs uses for communications with subprocesses. Or at least this is an additional buffering that is related to the issue at hand. > Unix convention is that stdin and stdout are line-buffered when > associated with a terminal, and fully-buffered (aka block-buffered) > otherwise. stderr is always unbuffered. > > """ > > emphasis on *Unix convention ... stderr .. always unbuffered*, though > it will be difficult to confirm this claim for all *nices) I think Windows behaves the same, at least by default. But GNU/Linux isn't. > For #2, emacs batch stderr behavior when invoked from outside the > command line is not compatible with the posix standard, since it uses > "fully buffered" mode, which actually makes it behave very differently > from Linux (which is posix compliant by always having stderr as being > unbuffered). Thus, the suggested patch actually addresses this > concern, rather than invalidating it. Again, are you sure it isn't the pipe that imposes buffering? > (I understand ConPTY is the long term solution looking for a hacker, > though I still do > suppose the current behavior on windows is odd and needs to be > addressed until that time). I don't see it as odd. The behavior of stdout and stdin could be odd (again, due to pipes being used), but not that of stderr. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 09 Feb 2021 21:16:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: ioannis.kappas@gmail.com Cc: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.16129053055176 (code B ref 46388); Tue, 09 Feb 2021 21:16:01 +0000 Received: (at 46388) by debbugs.gnu.org; 9 Feb 2021 21:15:05 +0000 Received: from localhost ([127.0.0.1]:54846 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9aLg-0001LP-NM for submit@debbugs.gnu.org; Tue, 09 Feb 2021 16:15:04 -0500 Received: from eggs.gnu.org ([209.51.188.92]:41834) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9aLe-0001Ke-Tm for 46388@debbugs.gnu.org; Tue, 09 Feb 2021 16:15:03 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56171) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9aLZ-0004G1-OG; Tue, 09 Feb 2021 16:14:57 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1189 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l9aLY-0003Ak-NU; Tue, 09 Feb 2021 16:14:57 -0500 Date: Tue, 09 Feb 2021 23:14:55 +0200 Message-Id: <83eehpj640.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <83h7mlj75u.fsf@gnu.org> (message from Eli Zaretskii on Tue, 09 Feb 2021 22:52:13 +0200) References: <83h7mlj75u.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Date: Tue, 09 Feb 2021 22:52:13 +0200 > From: Eli Zaretskii > Cc: 46388@debbugs.gnu.org > > > For #2, emacs batch stderr behavior when invoked from outside the > > command line is not compatible with the posix standard, since it uses > > "fully buffered" mode, which actually makes it behave very differently > > from Linux (which is posix compliant by always having stderr as being > > unbuffered). Thus, the suggested patch actually addresses this > > concern, rather than invalidating it. > > Again, are you sure it isn't the pipe that imposes buffering? Moreover, even if we did make the change you propose, it will only affect Emacs as a subprocess, but not when Emacs is the parent process and some other program (e.g., Python) is the subprocess. IOW, the main problem, which is that interactive subprocesses behave on MS-Windows differently from their behavior when invoked from the console -- this problem will remain unsolved as long as Emacs can only use pipes for communications with subprocesses. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 10 Feb 2021 12:49:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii , 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161296133528516 (code B ref 46388); Wed, 10 Feb 2021 12:49:01 +0000 Received: (at 46388) by debbugs.gnu.org; 10 Feb 2021 12:48:55 +0000 Received: from localhost ([127.0.0.1]:55698 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9ovP-0007Pr-2c for submit@debbugs.gnu.org; Wed, 10 Feb 2021 07:48:55 -0500 Received: from mail-oi1-f178.google.com ([209.85.167.178]:45851) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9ovM-0007Pf-V1 for 46388@debbugs.gnu.org; Wed, 10 Feb 2021 07:48:54 -0500 Received: by mail-oi1-f178.google.com with SMTP id m7so1810656oiw.12 for <46388@debbugs.gnu.org>; Wed, 10 Feb 2021 04:48:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=POLmag0GoWnTB2xcL3yMrkOcD/+RPwpzgqXAoXp8nJI=; b=NvsgtXW4uLWwj8eDKrdkgWVjyMI1lxRxQkeZgzehkF84lwZIN7viLgyasB5zEVUIHA Nf9Q3yuEZ2dljUsuITFQ5RU70dBCUwKdkT/yYyVr9cnDEiA0SjgxHQ21OgstyujV7Vq0 bWBKaJ2xF3sA6ocLKM8bHdQA19I8WdeTL+zkPww2aMa6fhJJW/bq92AELH+Y5t8ozsG+ P7rVT8jnJny3QWm+TE3NsbfzhcwdIMkxn1F5SqOrtrYvazxBZKFidhPXjRWcFCL5u8n9 xacszm7NoFliMcaaOEyOC+O7mERR3Ql9zHx+W8g6xfTZulEKdBiGw0+umQtjfioNYyeZ eWpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=POLmag0GoWnTB2xcL3yMrkOcD/+RPwpzgqXAoXp8nJI=; b=X8OmLau0lXA+Swl4xp1iFhEiVZm6CLtGKDODozzbhWnL2yg8MhsEnB3GcbCgrGITRq W23nLW5FmvB2Lh7yf1WfiswRgeBNczDp9TQ2SjgAYsvqUfEwWalCzLk7T7SY2jgI21T7 kXz7PP8BntoRSnVYwF0AdHvi/l3R2kpvMDIDu1PEBwq8RGhKQc9nURAeL8eygAu5XfbV 8ByH4V2TF1xFz692G12Xy1JGXdWlr0KgN3urvsRuAXCouGSVmsluOCGlvJtHWLKS9DNN 4+HIzT+VKGkpEq3ybqEj1WEBRAcHn0qOJQrorsbKf8zWJHNgBmeWErGjKWOaKdz5xpK9 jKuA== X-Gm-Message-State: AOAM533h2gmkELV9P1od3WTKjKK2Sc1A+FVL4pRK7TGgY3i9w0BDwta4 5OLFwixpBd83GUhZiNiwcxi3Z5/1+CQUta0+wfo= X-Google-Smtp-Source: ABdhPJzs2FpJuwWuUrRm0b51BwB7r1WFneXlp23CnKikOYX8RtjTDp2CAFdU40hui8UfdHZqpbKGLPWUpKsfVGFx774= X-Received: by 2002:aca:1a1a:: with SMTP id a26mr1980147oia.78.1612961327124; Wed, 10 Feb 2021 04:48:47 -0800 (PST) MIME-Version: 1.0 References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> In-Reply-To: From: Ioannis Kappas Date: Wed, 10 Feb 2021 12:48:38 +0000 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Eli, the reported issue and the draft patch is only concerned with the behavior of the MESSAGE function in emacs -batch, not with the communication of any emacs subprocess in general. I think I now understand where you are coming from. Let me summarize once more where we are, introducing buffering in the description (assuming MESSAGE length is < 2048): | # | System | emacs -batch invoked from | MESSAGE behavior | |---+------------+---------------------------+----------------------------------------------------------------------------------------------------| | 1 | Linux | bash | any MESSAGE is immediately printed, i.e. stderr is unbuffered | | 2 | Linux | emacs eshell/shell etc | >> | | 3 | Windows 10 | command prompt | >> | | 4 | Windows 10 | mintty | MESSAGEs are only displayed when emacs -batch is about to exit, i.e. stderr appears to be buffered | | 5 | Windows 10 | emacs eshell | >> | The issue here is that if someone invokes emacs -batch with #4 and #5, they won't be getting any MESSAGEs/feedback on their terminal until the program exits. This, I consider to be unacceptable if the caller/user is expecting to get feedback on their terminal from a long run emacs -batch script. And thus the bug report. I think I've just realized what you were saying from the beginning. That the difference in behavior is expected, since it is the parent process which decides the buffering regime to be used for the subprocess. Thus in #5, it is emacs on windows that decided to invoke emacs -batch as a subprocess using pipes, which has resulted in emacs -batch's stderr to be buffered. From my side, seeing that #4 and #5 behaving similarly, I had made the wild assumption that it is was windows that is enforcing the emacs -batch's stderr to be buffered, only based on the fact that the subprocess was not attached to the console alone. And thus the suggested patch to correct this as such. I will need to look into how exactly mintty and emacs invoke a subprocess and confirm indeed that stderr is buffered because they are both using pipes (or similar methods) as you suggested, and is not as I have arbitrarily assumed it to be. If the current behavior is indeed the correct expected behavior, how do I flush text message to stderr (or even stdout) from an emacs -batch script/eval? Hope this clarifies things a bit. On Tue, Feb 9, 2021 at 9:14 PM Eli Zaretskii wrote: > > > Date: Tue, 09 Feb 2021 22:52:13 +0200 > > From: Eli Zaretskii > > Cc: 46388@debbugs.gnu.org > > > > > For #2, emacs batch stderr behavior when invoked from outside the > > > command line is not compatible with the posix standard, since it uses > > > "fully buffered" mode, which actually makes it behave very differently > > > from Linux (which is posix compliant by always having stderr as being > > > unbuffered). Thus, the suggested patch actually addresses this > > > concern, rather than invalidating it. > > > > Again, are you sure it isn't the pipe that imposes buffering? > > Moreover, even if we did make the change you propose, it will only > affect Emacs as a subprocess, but not when Emacs is the parent process > and some other program (e.g., Python) is the subprocess. IOW, the > main problem, which is that interactive subprocesses behave on > MS-Windows differently from their behavior when invoked from the > console -- this problem will remain unsolved as long as Emacs can only > use pipes for communications with subprocesses. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 10 Feb 2021 15:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ioannis Kappas Cc: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161297264422828 (code B ref 46388); Wed, 10 Feb 2021 15:58:02 +0000 Received: (at 46388) by debbugs.gnu.org; 10 Feb 2021 15:57:24 +0000 Received: from localhost ([127.0.0.1]:56953 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9rrn-0005w7-QG for submit@debbugs.gnu.org; Wed, 10 Feb 2021 10:57:24 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42930) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9rrm-0005vu-2K for 46388@debbugs.gnu.org; Wed, 10 Feb 2021 10:57:22 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:45923) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9rrg-00038v-QL; Wed, 10 Feb 2021 10:57:16 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2490 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l9rrf-0003B7-15; Wed, 10 Feb 2021 10:57:15 -0500 Date: Wed, 10 Feb 2021 17:57:15 +0200 Message-Id: <83pn17j4pw.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (message from Ioannis Kappas on Wed, 10 Feb 2021 12:48:38 +0000) References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Ioannis Kappas > Date: Wed, 10 Feb 2021 12:48:38 +0000 > > I think I now understand where you are coming from. > > Let me summarize once more where we are, introducing buffering in > the description (assuming MESSAGE length is < 2048): > > | # | System | emacs -batch invoked from | MESSAGE behavior > > | > |---+------------+---------------------------+----------------------------------------------------------------------------------------------------| > | 1 | Linux | bash | any MESSAGE is > immediately printed, i.e. stderr is unbuffered > | > | 2 | Linux | emacs eshell/shell etc | >> Did you try this 2nd item when the connection type for the subprocess is 'pipe'? Because otherwise we are comparing apples to oranges. > I think I've just realized what you were saying from the > beginning. That the difference in behavior is expected, since > it is the parent process which decides the buffering regime to be > used for the subprocess. Thus in #5, it is emacs on windows that > decided to invoke emacs -batch as a subprocess using pipes, which > has resulted in emacs -batch's stderr to be buffered. That is correct. As we don't support PTYs on Windows, we can only use pipes for communicating with subprocesses there. Btw, did you try to play with the value of w32-pipe-buffer-size, e.g. setting it to a small value? > If the current behavior is indeed the correct expected behavior, how do I > flush text message to stderr (or even stdout) from an emacs > -batch script/eval? My reading of the code is that we already fflush stderr after emitting a message, so this should already happen. See message_to_stderr. If that still doesn't help, then there's some buffering in the OS (for example, in the pipe machinery itself), which we cannot control. There were some changes in this area lately (that's the discussion from 2019 I mentioned before): we now try to make a buffered variant of stderr, and use that for error messages. The reason, in a nutshell, is that when you build Emacs with "make -jN", several copies of the Emacs process can work in parallel, so it was deemed better to have their messages be emitted atomically, instead of being interspersed with one another, which produces an illegible mess. However, that change makes a line-buffered variant of stderr, and on Windows line buffering is the same as full buffering. So maybe we would need more changes in that area, for example some variable to control this behavior instead of making it unconditional. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 11 Feb 2021 08:11:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.16130310527942 (code B ref 46388); Thu, 11 Feb 2021 08:11:03 +0000 Received: (at 46388) by debbugs.gnu.org; 11 Feb 2021 08:10:52 +0000 Received: from localhost ([127.0.0.1]:57648 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lA73s-000242-5s for submit@debbugs.gnu.org; Thu, 11 Feb 2021 03:10:52 -0500 Received: from mail-ot1-f41.google.com ([209.85.210.41]:34448) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lA73q-00023p-Ku for 46388@debbugs.gnu.org; Thu, 11 Feb 2021 03:10:51 -0500 Received: by mail-ot1-f41.google.com with SMTP id y11so4434249otq.1 for <46388@debbugs.gnu.org>; Thu, 11 Feb 2021 00:10:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ykpcj/2Mr1v/Y/rDQ3NeRhAv8rELUE5N/GaeWcd3udo=; b=pL4fR8TJjZXqCRk85z9sIT2+tfWK6Z2E8mO0BOeKL12RnHD+ZOZlFfBwuHFXZnK5wz QlScD3F5vka3577Xkspk5CgM+aCkY91tLMr0PfdZXkWYrfVSgwzRw45vzfUzDrjrAHtr hUbeL2mLD1Hqc4EFw0QLCNbnxZU+Np7bbf6D1HzqkN6dOy+WBZegM8geM9zlkKojmGRM VqISrO/C5hQpqWfMzSNflwRonbeziCmRrzQObHwRA+Y5yy8bu+/zlG+43qrLQLSHN0Wy UkVCoo1AujcctGFoSPnE1zSM5FrkAGXjeThpS53bpf00lVhl2mmGGrjMtbVkGDAc7w18 +n7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ykpcj/2Mr1v/Y/rDQ3NeRhAv8rELUE5N/GaeWcd3udo=; b=GaWnF51Hv4iONdqqMccGvPK6GHISqog0iozkanAbBWRhbhZW17lIjvb3ojzxoe2lZH XVIO5Cv6/mOsZ+vBJAAJqwSbn3EPOKO42fPf67fgX+mUYbi9QJaKKyKW8OtQ/hUcvNHt Z0AsqexrAzug3+rrwCuxQ+xVn4b0rSyyyZN/Z3JFJNzqq8c4r661fH2ZLqc+2q8WbPGO zWU9n6TUZxNtVorXTJ8EhKCIpkMxwG+vhVkg2so1vqks7CMIa8wT6+HCUf96TRW5BeAs IX2oXETyJ0pOF38a908/5N8ZNxyZQgLibu76iUaCJMmDLeRmiUzzjJnBJT023FtozAU4 UoyA== X-Gm-Message-State: AOAM531toiszV9s3ZJWGQIp8P+A9S/JVN03miy0/6PDNhyH3POijS55r ttLWEfwt3PGOVukQjHh7t+nssmNwaq1AtX+JbQP57fpdIuc5TQ== X-Google-Smtp-Source: ABdhPJzhXL5hgZEH6t0OxDNWEtusJij10unciUuExl5N8qe2IloPyRmSLrlmqBimPY+2F/CatIETsct2dQjZe70KI70= X-Received: by 2002:a9d:506:: with SMTP id 6mr5161059otw.5.1613031044953; Thu, 11 Feb 2021 00:10:44 -0800 (PST) MIME-Version: 1.0 References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> In-Reply-To: <83pn17j4pw.fsf@gnu.org> From: Ioannis Kappas Date: Thu, 11 Feb 2021 08:10:34 +0000 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, Feb 10, 2021 at 3:57 PM Eli Zaretskii wrote: > > > > | # | System | emacs -batch invoked from | MESSAGE behavior > > > > | > > |---+------------+---------------------------+----------------------------------------------------------------------------------------------------| > > | 1 | Linux | bash | any MESSAGE is > > immediately printed, i.e. stderr is unbuffered > > | > > | 2 | Linux | emacs eshell/shell etc | >> > > Did you try this 2nd item when the connection type for the subprocess > is 'pipe'? Because otherwise we are comparing apples to oranges. I haven't, how do I do this? I was merely describing the plain user experience of running emacs -batch from a shell. for case #2 for example: 1. On Linux, open emacs 2. M-x eshell 3. at the eshell prompt type the following command: 3.1 emacs -Q --batch --eval="(progn (message \"hi\") (sit-for 5))" 4. observe the result 4.1 "hi" is printed immediately, emacs -batch exits after five seconds. In my tests, the second command was ran in all five cases, without applying any other configuration to the parent command prompt/shell/eshell process. Apologies if this was not clear. > > I think I've just realized what you were saying from the > > beginning. That the difference in behavior is expected, since > > it is the parent process which decides the buffering regime to be > > used for the subprocess. Thus in #5, it is emacs on windows that > > decided to invoke emacs -batch as a subprocess using pipes, which > > has resulted in emacs -batch's stderr to be buffered. > > That is correct. As we don't support PTYs on Windows, we can only use > pipes for communicating with subprocesses there. > > Btw, did you try to play with the value of w32-pipe-buffer-size, > e.g. setting it to a small value? No, I haven't experimented with pipes yet, my next plan is to study how subprocess are invoked from within emacs and mintty (since both demonstrate the same behavior). > > If the current behavior is indeed the correct expected behavior, how do I > > flush text message to stderr (or even stdout) from an emacs > > -batch script/eval? > > My reading of the code is that we already fflush stderr after emitting > a message, so this should already happen. See message_to_stderr. If > that still doesn't help, then there's some buffering in the OS (for > example, in the pipe machinery itself), which we cannot control. the xdisp.c:message_to_stderr() is the first function i studied with gdb when I started the investigation. Unless I've missed something, it does not seem to lead to calling fflush (under windows at least): 1. it will sysdep.c:errwrite() the message 1.1 errwrite() will call sysdep.c:errstream() to get stderr 1.1.1 errstream() *may* call fflush on stderr in some system (via buferr static variable), but this does not happen under windows-nt at least. 1.2 will call fwrite to write the message 2. it may call sysdep.c:errputc() to to append a newline, following exactly the same logic as for errwrite(), but using fputc() instead /* Log the message M to stderr. Log an empty line if M is not a string. */ static void message_to_stderr (Lisp_Object m) { if (noninteractive_need_newline) { noninteractive_need_newline = false; errputc ('\n'); } if (STRINGP (m)) { Lisp_Object coding_system = Vlocale_coding_system; Lisp_Object s; if (!NILP (Vcoding_system_for_write)) coding_system = Vcoding_system_for_write; if (!NILP (coding_system)) s = code_convert_string_norecord (m, coding_system, true); else s = m; errwrite (SDATA (s), SBYTES (s)); } if (STRINGP (m) || !cursor_in_echo_area) errputc ('\n'); } void errputc (int c) { fputc_unlocked (c, errstream ()); } void errwrite (void const *buf, ptrdiff_t nbuf) { fwrite_unlocked (buf, 1, nbuf, errstream ()); } /* Return the error output stream. */ static FILE * errstream (void) { FILE *err = buferr; if (!err) return stderr; fflush_unlocked (stderr); return err; } in sysdep.c:init_standard_fds() ... /* Set buferr if possible on platforms defining _PC_PIPE_BUF, as they support the notion of atomic writes to pipes. */ #ifdef _PC_PIPE_BUF buferr = fdopen (STDERR_FILENO, "w"); if (buferr) setvbuf (buferr, NULL, _IOLBF, 0); #endif ... > There were some changes in this area lately (that's the discussion > from 2019 I mentioned before): we now try to make a buffered variant > of stderr, and use that for error messages. The reason, in a > nutshell, is that when you build Emacs with "make -jN", several copies > of the Emacs process can work in parallel, so it was deemed better to > have their messages be emitted atomically, instead of being > interspersed with one another, which produces an illegible mess. > However, that change makes a line-buffered variant of stderr, and on > Windows line buffering is the same as full buffering. So maybe we > would need more changes in that area, for example some variable to > control this behavior instead of making it unconditional. Noted, thanks again! From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 11 Feb 2021 14:10:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ioannis Kappas , Paul Eggert Cc: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161305257625929 (code B ref 46388); Thu, 11 Feb 2021 14:10:01 +0000 Received: (at 46388) by debbugs.gnu.org; 11 Feb 2021 14:09:36 +0000 Received: from localhost ([127.0.0.1]:58044 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lACf2-0006k8-IW for submit@debbugs.gnu.org; Thu, 11 Feb 2021 09:09:36 -0500 Received: from eggs.gnu.org ([209.51.188.92]:35970) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lACey-0006js-IG for 46388@debbugs.gnu.org; Thu, 11 Feb 2021 09:09:34 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:41835) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lACer-0004LR-Hg; Thu, 11 Feb 2021 09:09:25 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1687 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lACeq-000414-L5; Thu, 11 Feb 2021 09:09:25 -0500 Date: Thu, 11 Feb 2021 16:09:28 +0200 Message-Id: <831rdmitlz.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (message from Ioannis Kappas on Thu, 11 Feb 2021 08:10:34 +0000) References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Ioannis Kappas > Date: Thu, 11 Feb 2021 08:10:34 +0000 > Cc: 46388@debbugs.gnu.org > > > My reading of the code is that we already fflush stderr after emitting > > a message, so this should already happen. See message_to_stderr. If > > that still doesn't help, then there's some buffering in the OS (for > > example, in the pipe machinery itself), which we cannot control. > > the xdisp.c:message_to_stderr() is the first function i studied with > gdb when I started the investigation. Unless I've missed something, > it does not seem to lead to calling fflush (under windows at least): Then maybe this: > /* Return the error output stream. */ > static FILE * > errstream (void) > { > FILE *err = buferr; > if (!err) > return stderr; > fflush_unlocked (stderr); <<<<<<<<<<<<<<<< > return err; > } should be fixed to fflush 'buferr' instead (or in addition to stderr)? Paul, isn't that a bug that we fflush stderr here, and not 'buferr'? From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 11 Feb 2021 19:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 46388@debbugs.gnu.org, Paul Eggert Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161307154414949 (code B ref 46388); Thu, 11 Feb 2021 19:26:02 +0000 Received: (at 46388) by debbugs.gnu.org; 11 Feb 2021 19:25:44 +0000 Received: from localhost ([127.0.0.1]:59293 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAHax-0003t3-QU for submit@debbugs.gnu.org; Thu, 11 Feb 2021 14:25:44 -0500 Received: from mail-wm1-f49.google.com ([209.85.128.49]:52130) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAHat-0003sn-5F for 46388@debbugs.gnu.org; Thu, 11 Feb 2021 14:25:42 -0500 Received: by mail-wm1-f49.google.com with SMTP id u16so2891483wmq.1 for <46388@debbugs.gnu.org>; Thu, 11 Feb 2021 11:25:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=HomaABrx46NaocFe6Tpvz2gUJq0DUsw6l5YP++FSXOg=; b=OBV+A9CR08U1roldGPn0tVZVmq/cG7LiG7C6T0Eld88vhE45kskvo21+7a6qYp8NWe oW6JUgaTkx3ep8uzrLFlp9kcTPc1deLk1QPlDvUJkxpKpGnDYa0ss7aukrC0aPMLrgeo jItSarbeDNBSiVZbGoAkbN/TvA5jHS+xIYdKZlaPSC788U5nv03DpIP8Vv4YlcFJZ6km 1PA43RfiUeiTVRv5N4bIRxIyZuYT3XRgx7DbuWas/46w8GCN8e6jeMkh0I2tAh2gSD+G FKWUEy3IQJCObJ1xUz+fnItz0yrMVX03lFcaNlupl6b28BpnMGreFX0Ipc3jYeYl5oiy kYYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=HomaABrx46NaocFe6Tpvz2gUJq0DUsw6l5YP++FSXOg=; b=tjrhDm3fxWuW34+DXKPeOi/hm+LH7B854P3CJY0/p0cS5tRYtpxCId9uAUU00qI24N fY5djoZ4Z87IwR8E17Ltb1RU4iBrY7QJzw2EYyyfQIi0AA29bD4RQIB97+4jobHZPpLi tHMsolj7UuskyuFT1TRAj0imCcvYxos7IKfBzcgVtfuyUHsg2mM9Z2RuBBw+CuYJC4sH ERBFElMg6dqrvQFLzj+DJgiEl/8U+C2DnskncI4KFicgUCnUJbzQQ+c9phy4DlxIrth4 U3o502bKau+oTMyL9iieZoUL08FFSeCAOwW8VX93pInRlJHjqiZBln7IsLPiCsUeN2Cb mfAA== X-Gm-Message-State: AOAM533xqEAn+MPS3tORfXQpZz1yEYTAaSNYm9SFBAYd+WgQbi/Kiv0n pf7FOGx48DgWpKpbVzMa4Cs= X-Google-Smtp-Source: ABdhPJxaFqbQ01k9XwoWNkzk5f7LomgyW/QAxeuRGcbZf6ZfvcxVLM/BgxJLyjVC6H6EHFoauayFtQ== X-Received: by 2002:a1c:2d8a:: with SMTP id t132mr6326795wmt.119.1613071533210; Thu, 11 Feb 2021 11:25:33 -0800 (PST) Received: from 192.168.1.103 ([188.214.14.133]) by smtp.gmail.com with ESMTPSA id f2sm6270690wrt.7.2021.02.11.11.25.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Feb 2021 11:25:32 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.40.0.2.32\)) From: Ioannis Kappas In-Reply-To: <831rdmitlz.fsf@gnu.org> Date: Thu, 11 Feb 2021 19:25:31 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> X-Mailer: Apple Mail (2.3654.40.0.2.32) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > On 11 Feb 2021, at 14:09, Eli Zaretskii wrote: >=20 >> From: Ioannis Kappas >> Date: Thu, 11 Feb 2021 08:10:34 +0000 >> Cc: 46388@debbugs.gnu.org >>=20 >>> My reading of the code is that we already fflush stderr after = emitting >>> a message, so this should already happen. See message_to_stderr. = If >>> that still doesn't help, then there's some buffering in the OS (for >>> example, in the pipe machinery itself), which we cannot control. >>=20 >> the xdisp.c:message_to_stderr() is the first function i studied with >> gdb when I started the investigation. Unless I've missed something, >> it does not seem to lead to calling fflush (under windows at least): >=20 > Then maybe this: >=20 >> /* Return the error output stream. */ >> static FILE * >> errstream (void) >> { >> FILE *err =3D buferr; >> if (!err) >> return stderr; >> fflush_unlocked (stderr); <<<<<<<<<<<<<<<< >> return err; >> } >=20 > should be fixed to fflush 'buferr' instead (or in addition to stderr)? >=20 > Paul, isn't that a bug that we fflush stderr here, and not 'buferr=E2=80= =99? (just a small note, =E2=80=9Cbuffer" is NULL under windows, the fn thus = returns without flushing anything. Even if buffer was not NULL, the = fflush fn would have flushed the content of what ever has been = accumulated on the stderr buffer so far, but not the message just sent = to message_to_stderr that we want to print out. Although, there would be = this weird effect; message_to_sderr() does an fwrite of the message = followed by an fputc of a newline. This means that if errstream() was to = fflush stderr, it would have flushed only the message written by fwrite, = and not the newline written by fputc. I think that, if we are indeed = considering to explicitly flush the message to stder, the correct place = to do it would be directly inside the message_to_stderr(), thanks) From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 11 Feb 2021 19:56:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ioannis Kappas Cc: 46388@debbugs.gnu.org, eggert@cs.ucla.edu Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161307330717768 (code B ref 46388); Thu, 11 Feb 2021 19:56:01 +0000 Received: (at 46388) by debbugs.gnu.org; 11 Feb 2021 19:55:07 +0000 Received: from localhost ([127.0.0.1]:59342 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAI3P-0004cW-4e for submit@debbugs.gnu.org; Thu, 11 Feb 2021 14:55:07 -0500 Received: from eggs.gnu.org ([209.51.188.92]:41682) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAI3N-0004bw-9v for 46388@debbugs.gnu.org; Thu, 11 Feb 2021 14:55:06 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49506) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAI3H-00016r-GW; Thu, 11 Feb 2021 14:54:59 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3609 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lAI3F-00026L-8P; Thu, 11 Feb 2021 14:54:58 -0500 Date: Thu, 11 Feb 2021 21:55:00 +0200 Message-Id: <83a6sagz1n.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (message from Ioannis Kappas on Thu, 11 Feb 2021 19:25:31 +0000) References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Ioannis Kappas > Date: Thu, 11 Feb 2021 19:25:31 +0000 > Cc: Paul Eggert , > 46388@debbugs.gnu.org > > (just a small note, “buffer" is NULL under windows Which is a Good Thing, since otherwise we'd have stderr always fully buffered on Windows (since _IOLBF on Windows is interpreted the same as _IOFBF, i.e. fully-buffered). >, the fn thus returns without flushing anything. Even if buffer was not NULL, the fflush fn would have flushed the content of what ever has been accumulated on the stderr buffer so far, but not the message just sent to message_to_stderr that we want to print out. Although, there would be this weird effect; message_to_sderr() does an fwrite of the message followed by an fputc of a newline. This means that if errstream() was to fflush stderr, it would have flushed only the message written by fwrite, and not the newline written by fputc. I think that, if we are indeed considering to explicitly flush the message to stder, the correct place to do it would be directly inside the message_to_stderr(), thanks) If buferr is NULL, we are using the original stderr, which is supposed to be unbuffered. But I think you again are looking at the wrong side of the pipe: the parent Emacs process reads from a pipe, which as its own buffering. So whatever we do with Emacs's stderr will only affect subprocesses when the child process is also Emacs, and will not have any effect on other programs being run as subprocesses. The correct solution, one that will seamlessly fix all the aspects of the buffering, is to add pseudo-console support to Emacs on Windows, and use that by default on systems that can support it. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 11 Feb 2021 21:16:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 46388@debbugs.gnu.org, Ioannis Kappas Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161307815325579 (code B ref 46388); Thu, 11 Feb 2021 21:16:01 +0000 Received: (at 46388) by debbugs.gnu.org; 11 Feb 2021 21:15:53 +0000 Received: from localhost ([127.0.0.1]:59434 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAJJY-0006eV-Oh for submit@debbugs.gnu.org; Thu, 11 Feb 2021 16:15:52 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:42562) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAJJW-0006eD-En for 46388@debbugs.gnu.org; Thu, 11 Feb 2021 16:15:51 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 729F51600CC; Thu, 11 Feb 2021 13:15:44 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id i5YvShOkgRfF; Thu, 11 Feb 2021 13:15:43 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id B076D1600EF; Thu, 11 Feb 2021 13:15:43 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id qemXST_l00_u; Thu, 11 Feb 2021 13:15:43 -0800 (PST) Received: from [192.168.1.9] (cpe-23-243-218-95.socal.res.rr.com [23.243.218.95]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 869D01600CC; Thu, 11 Feb 2021 13:15:43 -0800 (PST) References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <13eb72a9-237d-e645-f106-74ecea47742e@cs.ucla.edu> Date: Thu, 11 Feb 2021 13:15:42 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <831rdmitlz.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.8 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.8 (-) On 2/11/21 6:09 AM, Eli Zaretskii wrote: > Then maybe this: >=20 >> /* Return the error output stream. */ >> static FILE * >> errstream (void) >> { >> FILE *err =3D buferr; >> if (!err) >> return stderr; >> fflush_unlocked (stderr); <<<<<<<<<<<<<<<< >> return err; >> } > should be fixed to fflush 'buferr' instead (or in addition to stderr)? >=20 > Paul, isn't that a bug that we fflush stderr here, and not 'buferr'? No, it's intended. The goal of 'errstream' is to flush out all the stuff=20 written to stderr, before outputting anything sent to buferr. There is=20 no need to fflush buferr here, since it's gonna be flushed soon anyway=20 when a newline is output. Functions like message_to_stderr etc. are supposed to do a series of=20 calls to one or more functions like errwrite that end with outputting a=20 newline, without invoking any code that outputs directly to stderr. This=20 is explained in the comment before sysdep.c's errputc function, a=20 comment that could perhaps be improved. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 12 Feb 2021 20:00:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 46388@debbugs.gnu.org, Paul Eggert Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.16131599636887 (code B ref 46388); Fri, 12 Feb 2021 20:00:02 +0000 Received: (at 46388) by debbugs.gnu.org; 12 Feb 2021 19:59:23 +0000 Received: from localhost ([127.0.0.1]:33030 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAeb4-0001n0-R8 for submit@debbugs.gnu.org; Fri, 12 Feb 2021 14:59:23 -0500 Received: from mail-ot1-f44.google.com ([209.85.210.44]:44513) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAeb0-0001ml-RF for 46388@debbugs.gnu.org; Fri, 12 Feb 2021 14:59:20 -0500 Received: by mail-ot1-f44.google.com with SMTP id e5so282252otb.11 for <46388@debbugs.gnu.org>; Fri, 12 Feb 2021 11:59:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=XNtH97FOEdeeNcJ3pWEpUgX9UI/MJQiRhMU/c6AScWc=; b=YGwA9AQZ3svyyKi8dTMw7iW6anM8wtBvMf2M7dgvUP/x1/kZFHuc/h/VkY6JxpnmyV XMTuNAmhHPsI60anhnncA9NFcHouV+Aqxo2BF9wZhiusOww5tuWONUug3O2ZUXn854L8 ONIjrkGmybtD7RO5lhKGN5H/2rVF0EG+uYyez1Zhd6OFoJdwOvsiLJTLRAZq2prCLIU/ 23FajpVEfGXZrhegutvnJaq4m00TEdpywcHj20lD/XLL6cXmuYI1i6tAyUEhgdhyiEK1 Ho1WTrMoxCjhiJBic++yWlufYqeMFJ38O5B+aEO3pHIRG/C9jHW8RCYI1Rx7srToKvJy aDwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=XNtH97FOEdeeNcJ3pWEpUgX9UI/MJQiRhMU/c6AScWc=; b=fa7HgGJtv60B9maf7wC5BRmintVNnCilJJBaDDXk9aMSKBOZi5tUxa7zzL/R4gxbac zBzlveryr6AX10i0KRBvhPwA3sRQ4MHJ0vwhlBflvW0XgyECKtMtMCamMzaR20DgfF/Q s1vsOImne7lAx3GNW+OWoZK6MWotHza/3ztmTEc4+HLtjVgmDf5d/iLRSMflIYnolSGz ML4/C8FQEacz4a9Jtc9+Sn1e5skL6zfJ1BiPxFSTRjMqQ9G++Ni0HQJ5mPptXEN7803S BK1Jol/kLdWPqWQIYEAxq4RT9WGTutaGhMHmpLL+TOljwFtvHG1A0Qi2fCoZhQe5W8mR i7SA== X-Gm-Message-State: AOAM533TLEMMIoywk0TG7GUzwKFIR8mQxxdwNrYx2kGqJJkiZAoyPdOR SYukcEiBf1EF1Bop16Y8csBjSvK9tSaGOgjAFKM= X-Google-Smtp-Source: ABdhPJyegR62uvRnICtbSM4RSq71DYKWtUoZNdcX18Gf6cUgKg29XIK7O3dLLyaOWQGudvOYzw6IEk19JbRMXb6zG5s= X-Received: by 2002:a9d:d72:: with SMTP id 105mr2006803oti.192.1613159953167; Fri, 12 Feb 2021 11:59:13 -0800 (PST) MIME-Version: 1.0 References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> <83a6sagz1n.fsf@gnu.org> In-Reply-To: <83a6sagz1n.fsf@gnu.org> From: Ioannis Kappas Date: Fri, 12 Feb 2021 19:59:02 +0000 Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Eli Just to recap (with a touch of drama this time). The following command emacs -Q --batch --eval=3D"(progn (message \"The bomb will explode in 10 seconds, cut the red cable to disarm\") (sit-for 3600))" will behave differently depending on the architecture/prompt it is invoked = from: When it is invoked from the Linux terminal or from a plain emacs eshell (i.e. M-x eshell) prompt or from a windows command prompt =3D> message is printed out immediately, user cuts the red cable, bomb doesn't explode. When it is invoked from msys64 mintty terminal or from emacs eshell on windows =3D> no message is printed out (until emacs -batch returns), no cables cut, bomb explodes after ten seconds. How do you suggest we flush the message out so that it is always displayed to the user irrespective of the architecture/prompt it is ran on? Is there perhaps an elisp function to flush messages out (e.g. (progn (message "...") (message-flush!) (sit-for ...))) ? So far we said: 1. We don't want to a solution where the emacs -batch's stderrr is forced to unbuffered mode (original patch proposal) 2. message_to_stderr() does not directly or indirectly invoke stderr flush (at least in windows). Thanks On Thu, Feb 11, 2021 at 7:55 PM Eli Zaretskii wrote: > > > From: Ioannis Kappas > > Date: Thu, 11 Feb 2021 19:25:31 +0000 > > Cc: Paul Eggert , > > 46388@debbugs.gnu.org > > > > (just a small note, =E2=80=9Cbuffer" is NULL under windows > > Which is a Good Thing, since otherwise we'd have stderr always fully > buffered on Windows (since _IOLBF on Windows is interpreted the same > as _IOFBF, i.e. fully-buffered). > > >, the fn thus returns without flushing anything. Even if buffer was not = NULL, the fflush fn would have flushed the content of what ever has been ac= cumulated on the stderr buffer so far, but not the message just sent to me= ssage_to_stderr that we want to print out. Although, there would be this we= ird effect; message_to_sderr() does an fwrite of the message followed by a= n fputc of a newline. This means that if errstream() was to fflush stderr, = it would have flushed only the message written by fwrite, and not the newli= ne written by fputc. I think that, if we are indeed considering to explicit= ly flush the message to stder, the correct place to do it would be directly= inside the message_to_stderr(), thanks) > > If buferr is NULL, we are using the original stderr, which is supposed > to be unbuffered. > > But I think you again are looking at the wrong side of the pipe: the > parent Emacs process reads from a pipe, which as its own buffering. > So whatever we do with Emacs's stderr will only affect subprocesses > when the child process is also Emacs, and will not have any effect on > other programs being run as subprocesses. > > The correct solution, one that will seamlessly fix all the aspects of > the buffering, is to add pseudo-console support to Emacs on Windows, > and use that by default on systems that can support it. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 12 Feb 2021 20:04:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ioannis Kappas Cc: 46388@debbugs.gnu.org, eggert@cs.ucla.edu Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.16131602257401 (code B ref 46388); Fri, 12 Feb 2021 20:04:01 +0000 Received: (at 46388) by debbugs.gnu.org; 12 Feb 2021 20:03:45 +0000 Received: from localhost ([127.0.0.1]:33045 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAefJ-0001vJ-A7 for submit@debbugs.gnu.org; Fri, 12 Feb 2021 15:03:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37802) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lAefF-0001v5-LH for 46388@debbugs.gnu.org; Fri, 12 Feb 2021 15:03:44 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59432) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lAef7-0006rL-M3; Fri, 12 Feb 2021 15:03:35 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1746 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lAef6-0006PX-GL; Fri, 12 Feb 2021 15:03:33 -0500 Date: Fri, 12 Feb 2021 22:03:40 +0200 Message-Id: <83tuqhdper.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (message from Ioannis Kappas on Fri, 12 Feb 2021 19:59:02 +0000) References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> <83a6sagz1n.fsf@gnu.org> X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > From: Ioannis Kappas > Date: Fri, 12 Feb 2021 19:59:02 +0000 > Cc: Paul Eggert , 46388@debbugs.gnu.org > > How do you suggest we flush the message out so that it is always > displayed to the user irrespective of the architecture/prompt it is > ran on? Is there perhaps an elisp function to flush messages out (e.g. > (progn (message "...") (message-flush!) (sit-for ...))) ? > > So far we said: > 1. We don't want to a solution where the emacs -batch's stderrr is > forced to unbuffered mode (original patch proposal) > 2. message_to_stderr() does not directly or indirectly invoke stderr > flush (at least in windows). As I said, this cannot be helped, not until we teach Emacs on Windows to use the pseudo-console. This is how the platform behaves when we use pipes. Sorry, there's no way around this. From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 06 Mar 2021 15:01:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 46388@debbugs.gnu.org, Paul Eggert Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161504283622813 (code B ref 46388); Sat, 06 Mar 2021 15:01:01 +0000 Received: (at 46388) by debbugs.gnu.org; 6 Mar 2021 15:00:36 +0000 Received: from localhost ([127.0.0.1]:38087 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIYPz-0005vs-5U for submit@debbugs.gnu.org; Sat, 06 Mar 2021 10:00:35 -0500 Received: from mail-oi1-f171.google.com ([209.85.167.171]:43169) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lIYPx-0005vd-Pi for 46388@debbugs.gnu.org; Sat, 06 Mar 2021 10:00:34 -0500 Received: by mail-oi1-f171.google.com with SMTP id d20so5986041oiw.10 for <46388@debbugs.gnu.org>; Sat, 06 Mar 2021 07:00:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oo76tIhJMVV/kp00FyfVJVNgvP2F1LTm0BljxEqxDOU=; b=VWHfBwnfBcq+G+RXteucGnLRFdEqvPfD+CH0upm0QMGi+D3d/+nGW2gw0UyoP1JmIn 378NkzPybudEHDVVUo9lRZ310VpXng3yjHgi2TJDL25KLIdbnhO5c2TGxkds3sIpJuAn 7lrAYp21Zim8xSna3v9IYYJ5wKkSdHUMtWpwDdzbtHY8IE/aHP6fj4DzVQjAwo7xxaU+ 58BmpFsQBqUkyt66Wq4MzG50nekFVA5JgTH1w3MQABRmMeFaDRGhIxImyvcEITjFg4FX WwxUb1BcwNIOVGMiomdkGjorFfARZyiainU25FO5nfHPm4VrqUXWR69RZkxcSLrDu5F0 Ndvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oo76tIhJMVV/kp00FyfVJVNgvP2F1LTm0BljxEqxDOU=; b=Q51Rmn3ZD/kBvW/7nMY/KGaV/2H83d9ngc39OYC8dFWMK9aok5rjwpdFoFniMOFHEC SifcaJBovsknWSTPy+794vvEFNXI3wjHMmIvr6ZLfL7cb743XP3DPjGL0CTd+NkvUW8q a57JnHgTerGO36wEuC0zKhaCmIQEGlnqgXrEeSOEGb2i7Ewxj0KN2AKgYrGHPQeuizwe wbGGTAMrNL2R0u9sG8wkkAgTT0LlVIX4DEimObi6/AmZ2SC6SRxa6c5vYXfUuWyG41JJ 5QXfm0V+nLIeVoYr+L2STQRCNFlQmLqWTjhbNG9+prTd+BapulIYvXmlWZmFu1K4wrKF f2JA== X-Gm-Message-State: AOAM530EXRY6BiPknliPqQaF8/GoHvE3t2pA7W55t+FopDIWDikGXD7m oVj0LieSIMHdrexmzfw+Wwgxn1nACAYJFZ8WaPyw7QvW1z4= X-Google-Smtp-Source: ABdhPJzk0Xe8sey8CxsREAPuWBds9HI8phRNyx/mmno8taesAw2WcJwjHkyv6dvAqS6H16ofIrsMa4ylca1r7eMA+5U= X-Received: by 2002:a05:6808:6cc:: with SMTP id m12mr10812706oih.129.1615042828117; Sat, 06 Mar 2021 07:00:28 -0800 (PST) MIME-Version: 1.0 References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> <83a6sagz1n.fsf@gnu.org> <83tuqhdper.fsf@gnu.org> In-Reply-To: <83tuqhdper.fsf@gnu.org> From: Ioannis Kappas Date: Sat, 6 Mar 2021 15:00:17 +0000 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Eli, (apologies for the long email) On Fri, Feb 12, 2021 at 8:03 PM Eli Zaretskii wrote: > > > From: Ioannis Kappas > > Date: Fri, 12 Feb 2021 19:59:02 +0000 > > Cc: Paul Eggert , 46388@debbugs.gnu.org > > > > How do you suggest we flush the message out so that it is always > > displayed to the user irrespective of the architecture/prompt it is > > ran on? Is there perhaps an elisp function to flush messages out (e.g. > > (progn (message "...") (message-flush!) (sit-for ...))) ? > > > > As I said, this cannot be helped, not until we teach Emacs on Windows > to use the pseudo-console. This is how the platform behaves when we > use pipes. > > Sorry, there's no way around this. although ConPTY support will hopefully solve all issues with regards to an Emacs parent process interacting with its children, it wont solve this particular issue when the parent process is different than Emacs or Emacs is running on a Windows version less than 10 (ConPTY as I understand is only available on Windows 10). A have created a little C program and written an analysis on the buffering behavior of stderr when redirected to a pipe at https://github.com/ikappaki/standard-streams-test#readme, with the aim to understanding the technology involved, reach your level of understanding and hopefully strengthen my argument for a fix. Just to summarize the results: 1. stderr is unbuffered when attached to the console. 2. A child's stderr stream is fully buffered when redirected to a pipe, with a default buffer size of 4096 bytes long. This buffer is complementary to the pipe's buffer. 3. The pipe's buffer size, configured by the parent process, defaults to a size of 4096 bytes long (the size is controlled by `w32-pipe-buffer-size' in Emacs, of which see). 4. The stderr stream mode can be reset to unbuffered or fully buffered at process start up when no characters are yet written to it. (PARENT PROCESS) READER <= [PIPE BUFFER] <= [STDERR STREAM BUFFER] <= WRITER (CHILD PROCESS) Given the above results, it seems to me like a proper fix to reset the stream to unbuffered when Emacs --batch stderr stream has been redirected to a pipe it, ensuring first of course that we distinguish between pipe, file and socket redirections. I believe one of the arguments against the original patch is that it will affect redirections to files, which we do need to be buffered at the stream level. Please note, that resetting the stream's buffer mode does not reset the pipe's buffer which is still in effect. The child writer (in our case Emacs --batch) will not block on a write, unless of course the write is more than 4096 bytes long and the parent process is unavailable to consume data. Here is the updated patch: @@ -10266,6 +10266,37 @@ init_ntproc (int dumping) else _open ("nul", O_TEXT | O_NOINHERIT | O_WRONLY); _fdopen (2, "w"); + + /* stderr is unbuffered when attached to the console but fully + buffered (4096 bytes) when redirected to a pipe (this buffer is + complementary to the pipe buffer). + + Since Emacs --batch does not, at least on Windows, flush stderr + it means output (such as that generated with `message') will + not reach the parent process, not until at least this process + exits or the stream buffer is full, resulting to a very poor + interaction with the parent process. + + We thus switch to unbuffered mode and rely only on the pipe's + buffer for ensuring this process is not blocked writing data + chunks to stderr (the default pipe's buffer size is 4096 bytes, + but this is only configurable by the parent process). Setting + the stream up to line buffer mode unfortunately does not work + because it is the same as fully buffered on win32. + + Note that the below does not affect stderr redirection to a + file, since this will come back as `FILE_TYPE_DISK'. + */ + HANDLE seh = (HANDLE)_get_osfhandle(_fileno(stderr)); + if (noninteractive + /* is pipe (anonymous, named or socket)*/ + && GetFileType(seh) == FILE_TYPE_PIPE + /* and is definitely not a socket */ + && GetNamedPipeInfo(seh, NULL, NULL, NULL, NULL)) + { + setvbuf(stderr, NULL, _IONBF, 0); + DebPrint((":stderr-mode-set-to-unbuf\n")); + } Please let me know of any cases you might think removing stderr buffer might prove problematic and is not covered by the presence of the pipe's buffer presence alone. An alternative to the above, and perhaps a little more involved, is to provide an interface to `message' to be able to flush a pipe-redirected stderr stream after outputting a message with a newline. This will make it behave as if the `message's output was line buffered, which I believe would have been the ideal outcome for stderr to begin with. Please also note that as mentioned earlier in this thread, there is a precedence of flushing stderr "on platforms defining _PC_PIPE_BUF" (see /sysdep.c:init_standard_fds()/), strengthening the argument of the eligibility of the following fix. In the patch below, I tried to maintain what I thought was the existing writing style and also make it obvious that the decision to flush the buffer was explicitly taken by /xdisp.c:message_to_stderr()/ rather than perhaps hiding it in /sysdep.c:errputc()/, although I am not so sure about scattering such a simple login in three different files: @@ -2789,6 +2789,30 @@ safe_strsignal (int code) /* Output to stderr. */ +/* On windows, stderr is unbuffered when attached to the console but + fully buffered (4096 bytes) when redirected to a pipe (this buffer + is complementary to the pipe buffer). + + Since Emacs --batch, at least on Windows, does not flush stderr it + means output (such as that created with `message') will not reach + the parent process, not until at least this process exits or the + stream buffer is full, resulting to a very poor interaction with + the parent process. + + We thus provide an interface to functions such as `message' to + flush stderr after they output a newline, make them in effect + behave as if stderr was line buffered (unfortunately resetting the + stream to line buffer mode is the same as resetting it to full + buffer mode, so this can't be enforced at the stream level). +*/ +void maybe_flush_stderr_after_newline (void) +{ +#ifdef WINDOWSNT + if (is_stderr_pipe()) + fflush_unlocked(stderr); +#endif /* WINDOWSNT */ +} + /* Return the error output stream. */ static FILE * errstream (void) modified src/sysstdio.h @@ -27,6 +27,7 @@ #define EMACS_SYSSTDIO_H #include "unlocked-io.h" extern FILE *emacs_fopen (char const *, char const *); +extern void maybe_flush_stderr_after_newline (void); extern void errputc (int); extern void errwrite (void const *, ptrdiff_t); extern void close_output_streams (void); modified src/w32.c @@ -10190,6 +10190,12 @@ term_ntproc (int ignored) term_w32select (); } +static bool _is_stderr_pipe = false; +bool is_stderr_pipe (void) +{ + return _is_stderr_pipe; +} + void init_ntproc (int dumping) { @@ -10268,6 +10274,13 @@ init_ntproc (int dumping) _fdopen (2, "w"); } + HANDLE seh = (HANDLE)_get_osfhandle(_fileno(stderr)); + _is_stderr_pipe = + /* is pipe (anonymous, named or socket)*/ + GetFileType(seh) == FILE_TYPE_PIPE + /* and is definitely not a socket */ + && GetNamedPipeInfo(seh, NULL, NULL, NULL, NULL); + /* unfortunately, atexit depends on implementation of malloc */ /* atexit (term_ntproc); */ if (!dumping) modified src/w32.h @@ -170,6 +170,9 @@ #define FILE_SERIAL 0x0800 extern HANDLE maybe_load_unicows_dll (void); extern void globals_of_w32 (void); +/* return whether standard error is redirected to a pipe. */ +extern bool is_stderr_pipe (void); + extern void term_timers (void); extern void init_timers (void); modified src/xdisp.c @@ -10968,6 +10968,7 @@ message_to_stderr (Lisp_Object m) { noninteractive_need_newline = false; errputc ('\n'); + maybe_flush_stderr_after_newline(); } if (STRINGP (m)) { @@ -10984,7 +10985,11 @@ message_to_stderr (Lisp_Object m) errwrite (SDATA (s), SBYTES (s)); } if (STRINGP (m) || !cursor_in_echo_area) - errputc ('\n'); + { + errputc ('\n'); + maybe_flush_stderr_after_newline(); + } + } /* The non-logging version of message3. Thank you! From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 08 Mar 2021 04:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ioannis Kappas , Eli Zaretskii Cc: 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.16151763188072 (code B ref 46388); Mon, 08 Mar 2021 04:06:02 +0000 Received: (at 46388) by debbugs.gnu.org; 8 Mar 2021 04:05:18 +0000 Received: from localhost ([127.0.0.1]:41860 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lJ78v-000267-T0 for submit@debbugs.gnu.org; Sun, 07 Mar 2021 23:05:18 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:55438) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lJ78r-00025n-J4 for 46388@debbugs.gnu.org; Sun, 07 Mar 2021 23:05:16 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 24AA21600BE; Sun, 7 Mar 2021 20:05:08 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id gUZGGMDJpySQ; Sun, 7 Mar 2021 20:05:07 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 68F3E1600B7; Sun, 7 Mar 2021 20:05:07 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id A2S0A3qMktRI; Sun, 7 Mar 2021 20:05:07 -0800 (PST) Received: from [192.168.1.9] (cpe-23-243-218-95.socal.res.rr.com [23.243.218.95]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 3C4101600BE; Sun, 7 Mar 2021 20:05:07 -0800 (PST) References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> <83a6sagz1n.fsf@gnu.org> <83tuqhdper.fsf@gnu.org> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <68caa967-89b2-3c35-0d18-ac490a02383a@cs.ucla.edu> Date: Sun, 7 Mar 2021 20:05:06 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------C3A2F1F011C48963705CECB7" Content-Language: en-US X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) This is a multi-part message in MIME format. --------------C3A2F1F011C48963705CECB7 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Wouldn't the attached be a simpler fix? --------------C3A2F1F011C48963705CECB7 Content-Type: text/x-patch; charset=UTF-8; name="0001-On-MS-Windows-fflush-stderr-after-newline.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-On-MS-Windows-fflush-stderr-after-newline.patch" >From ff81c979a9528af8065169f83a78de599e99177b Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Sun, 7 Mar 2021 20:01:37 -0800 Subject: [PATCH] On MS-Windows, fflush stderr after newline Problem reported by Ioannis Kappas (Bug#46388). * src/sysdep.c (errputc) [WINDOWSNT]: Flush stderr after newline. --- src/sysdep.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/sysdep.c b/src/sysdep.c index 941b4e2fa2..88dd3e71e4 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -2662,6 +2662,13 @@ errstream (void) errputc (int c) { fputc_unlocked (c, errstream ()); + +#ifdef WINDOWSNT + /* Flush stderr after outputting a newline since stderr is fully + buffered when redirected to a pipe, contrary to POSIX. */ + if (c == '\n') + fflush_unlocked (stderr); +#endif } void -- 2.29.2 --------------C3A2F1F011C48963705CECB7-- From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Ioannis Kappas Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 08 Mar 2021 07:57:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Paul Eggert Cc: Eli Zaretskii , 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161519020531109 (code B ref 46388); Mon, 08 Mar 2021 07:57:02 +0000 Received: (at 46388) by debbugs.gnu.org; 8 Mar 2021 07:56:45 +0000 Received: from localhost ([127.0.0.1]:42010 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lJAkv-00085h-0b for submit@debbugs.gnu.org; Mon, 08 Mar 2021 02:56:45 -0500 Received: from mail-ot1-f41.google.com ([209.85.210.41]:38276) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lJAkt-00085T-GF for 46388@debbugs.gnu.org; Mon, 08 Mar 2021 02:56:43 -0500 Received: by mail-ot1-f41.google.com with SMTP id a17so8370959oto.5 for <46388@debbugs.gnu.org>; Sun, 07 Mar 2021 23:56:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=IFgbqdohTO9KMpispQKGnoj79L7eCcldOLS/ZAHluJw=; b=LFVbArCgCLKvo7x7K2Rs2VKYzoIW5xxwYob7IAKfNSxZUA5gbuZrkF7YitWi0HhFr6 HH0lPxUQWz+Ya27oKtc/VDhcMjsXJrHjGPdt3uMu/OqDLjAC8qTQGBAlggfYuoIsJDXc BvuwbNKCcn504TpXw1WpE5WFDdhYVVfknb6JCxTmGy/yXzJkyYwmEEvRrzKdGqA6gDUU mEHViHh+nfQAjU9sg0lXsYPmPLDz6CTLscfdO/NVVjSa8CxZ0dGVEBEO2fe2aYf85e/n w7I/BrNEgq//wDyjdjvJpjFne58u8+OZDXlTrmBsEP6+/ulLfIc9w1kXf9EC0km52URg Qs6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=IFgbqdohTO9KMpispQKGnoj79L7eCcldOLS/ZAHluJw=; b=XhjkIOcMni12ga9Mnu1O7pjMXz1LFkntWDb9UvnesfB8AVZAT6KptDrmwtzVsW2lwX yTamvuPIN8o8kd2qVVrSR9YEi0sIS4ttiQJlJAlYRNvo5pDSZqXKjlQLgXUkXtnOePtc aCZah6UXt0KrSQ0Z8uy9E1Dy2vLDkmNGow/KikoP7eVmFZ2YS+bc+GOm5UPi2sMk8aRm Ialkljjd00r+kN9qS/5gufiHhBIWg6efxmMRLZERORqoTNWuoFmpz+nQ0HvgPoskNQsY qLZI8GlwT9RjeiSpoDH1Wq08+50fmSH1dtEBLj6aEBxMpcqzbTPk6Ugaf/LH1ZZzpnQb +QCw== X-Gm-Message-State: AOAM533f9Eb/yy+uBM5FD3MVgzEIIufpBx8kjbPmle/ijasplHNfxA63 Deg5nyls/nWc8iYCY75dhspgc7gBFb5XioNTFC8= X-Google-Smtp-Source: ABdhPJwuf70QrtoAu832STHNAz5ieCbyPkyrz0ScuJbKudmniLBRhVukH4x1hgcoC0ZPB82cgUh8hJ+DIJXWlnWvt6c= X-Received: by 2002:a9d:5ef:: with SMTP id 102mr8617379otd.192.1615190197669; Sun, 07 Mar 2021 23:56:37 -0800 (PST) MIME-Version: 1.0 References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> <83a6sagz1n.fsf@gnu.org> <83tuqhdper.fsf@gnu.org> <68caa967-89b2-3c35-0d18-ac490a02383a@cs.ucla.edu> In-Reply-To: <68caa967-89b2-3c35-0d18-ac490a02383a@cs.ucla.edu> From: Ioannis Kappas Date: Mon, 8 Mar 2021 07:56:27 +0000 Message-ID: Content-Type: text/plain; charset="UTF-8" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Paul, On Mon, Mar 8, 2021 at 4:05 AM Paul Eggert wrote: > > Wouldn't the attached be a simpler fix? It is fine by me. Thanks From unknown Tue Jun 17 22:29:09 2025 X-Loop: help-debbugs@gnu.org Subject: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 11 Mar 2021 14:28:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46388 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Paul Eggert Cc: ioannis.kappas@gmail.com, 46388@debbugs.gnu.org Received: via spool by 46388-submit@debbugs.gnu.org id=B46388.161547287425880 (code B ref 46388); Thu, 11 Mar 2021 14:28:02 +0000 Received: (at 46388) by debbugs.gnu.org; 11 Mar 2021 14:27:54 +0000 Received: from localhost ([127.0.0.1]:52907 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lKMI6-0006jM-2w for submit@debbugs.gnu.org; Thu, 11 Mar 2021 09:27:54 -0500 Received: from eggs.gnu.org ([209.51.188.92]:46374) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lKMI4-0006j4-8e for 46388@debbugs.gnu.org; Thu, 11 Mar 2021 09:27:52 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40342) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lKMHx-0007JO-W7; Thu, 11 Mar 2021 09:27:46 -0500 Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:1040 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lKMHt-0000B6-Oz; Thu, 11 Mar 2021 09:27:42 -0500 Date: Thu, 11 Mar 2021 16:27:42 +0200 Message-Id: <83im5xn4td.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <68caa967-89b2-3c35-0d18-ac490a02383a@cs.ucla.edu> (message from Paul Eggert on Sun, 7 Mar 2021 20:05:06 -0800) References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> <83a6sagz1n.fsf@gnu.org> <83tuqhdper.fsf@gnu.org> <68caa967-89b2-3c35-0d18-ac490a02383a@cs.ucla.edu> X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) > Cc: 46388@debbugs.gnu.org > From: Paul Eggert > Date: Sun, 7 Mar 2021 20:05:06 -0800 > > Wouldn't the attached be a simpler fix? > > >From ff81c979a9528af8065169f83a78de599e99177b Mon Sep 17 00:00:00 2001 > From: Paul Eggert > Date: Sun, 7 Mar 2021 20:01:37 -0800 > Subject: [PATCH] On MS-Windows, fflush stderr after newline > > Problem reported by Ioannis Kappas (Bug#46388). > * src/sysdep.c (errputc) [WINDOWSNT]: Flush stderr after newline. > --- > src/sysdep.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/src/sysdep.c b/src/sysdep.c > index 941b4e2fa2..88dd3e71e4 100644 > --- a/src/sysdep.c > +++ b/src/sysdep.c > @@ -2662,6 +2662,13 @@ errstream (void) > errputc (int c) > { > fputc_unlocked (c, errstream ()); > + > +#ifdef WINDOWSNT > + /* Flush stderr after outputting a newline since stderr is fully > + buffered when redirected to a pipe, contrary to POSIX. */ > + if (c == '\n') > + fflush_unlocked (stderr); > +#endif > } > This is fine, please install. Thanks (and sorry for a late response). From unknown Tue Jun 17 22:29:09 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Ioannis Kappas Subject: bug#46388: closed (Re: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt) Message-ID: References: <34cf6aaa-ee85-fbdc-3e39-5f4f44eb1ac9@cs.ucla.edu> X-Gnu-PR-Message: they-closed 46388 X-Gnu-PR-Package: emacs Reply-To: 46388@debbugs.gnu.org Date: Thu, 11 Mar 2021 18:44:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1615488242-27289-1" This is a multi-part message in MIME format... ------------=_1615488242-27289-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #46388: 27.1; emacs -batch does not output messages immediately when invoke= d outside of the command prompt which was filed against the emacs package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 46388@debbugs.gnu.org. --=20 46388: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D46388 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1615488242-27289-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 46388-done) by debbugs.gnu.org; 11 Mar 2021 18:43:50 +0000 Received: from localhost ([127.0.0.1]:54854 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lKQHl-00075f-VX for submit@debbugs.gnu.org; Thu, 11 Mar 2021 13:43:50 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:55766) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lKQHk-00075U-M0 for 46388-done@debbugs.gnu.org; Thu, 11 Mar 2021 13:43:48 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 30351160109; Thu, 11 Mar 2021 10:43:42 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id ayFCOdgtwue2; Thu, 11 Mar 2021 10:43:41 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 5AFC916011C; Thu, 11 Mar 2021 10:43:41 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id BUlVJDw1ERu5; Thu, 11 Mar 2021 10:43:41 -0800 (PST) Received: from [192.168.1.9] (cpe-23-243-218-95.socal.res.rr.com [23.243.218.95]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 35FE9160109; Thu, 11 Mar 2021 10:43:41 -0800 (PST) To: Eli Zaretskii References: <83h7mlj75u.fsf@gnu.org> <83eehpj640.fsf@gnu.org> <83pn17j4pw.fsf@gnu.org> <831rdmitlz.fsf@gnu.org> <83a6sagz1n.fsf@gnu.org> <83tuqhdper.fsf@gnu.org> <68caa967-89b2-3c35-0d18-ac490a02383a@cs.ucla.edu> <83im5xn4td.fsf@gnu.org> From: Paul Eggert Organization: UCLA Computer Science Department Subject: Re: bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt Message-ID: <34cf6aaa-ee85-fbdc-3e39-5f4f44eb1ac9@cs.ucla.edu> Date: Thu, 11 Mar 2021 10:43:40 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <83im5xn4td.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 46388-done Cc: ioannis.kappas@gmail.com, 46388-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) On 3/11/21 6:27 AM, Eli Zaretskii wrote: > This is fine, please install. OK, done, and closing the bug report. ------------=_1615488242-27289-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 8 Feb 2021 21:21:12 +0000 Received: from localhost ([127.0.0.1]:52037 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Dy3-0004f3-HT for submit@debbugs.gnu.org; Mon, 08 Feb 2021 16:21:11 -0500 Received: from lists.gnu.org ([209.51.188.17]:33764) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9Dxx-0004es-Rq for submit@debbugs.gnu.org; Mon, 08 Feb 2021 16:21:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:35244) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l9Dxw-0001Ul-FR for bug-gnu-emacs@gnu.org; Mon, 08 Feb 2021 16:21:04 -0500 Received: from mail-ot1-x334.google.com ([2607:f8b0:4864:20::334]:33840) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l9Dxt-0004kG-Hz for bug-gnu-emacs@gnu.org; Mon, 08 Feb 2021 16:21:04 -0500 Received: by mail-ot1-x334.google.com with SMTP id y11so15588147otq.1 for ; Mon, 08 Feb 2021 13:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=1nJFGSGOvcoMKoIwe+VBpx4DDYI4g7XQetIaBigsn5M=; b=bpKP9I22XQyvIAdkEoBI2CmDeG3XPDb30iscC/DDucBha41UJqQqgESkXaKXTsAcOo ScvHZsfyCN3wnTAFw5+o+MlvmAKXBN/QYpt0ubZVaxDM0qJIM6VlYZUeaxg/FF1245J0 VIDAm3tTlt8ZsPOk4IhqTvixhWYLIGdmT8fzfIVrPL7sGEW8Idi7MkqZDv3Sgkgp5xlD YSwgr+VZJ5jozE7chdnaqICKbE1jy5ihRQXjARaytZldhxVHeoUNefu25Jnr9yWxxUyA twNeNxrzUJZKznpERBpDjBVst+OgCTYC+cW2XVOP43spC5hHmKEyczsKJFV8gluGJ4HB Lt4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=1nJFGSGOvcoMKoIwe+VBpx4DDYI4g7XQetIaBigsn5M=; b=XTkSmNChrtH7PngfgGjxGMwoiy+wwtmTpyUmdvRta6bYXjzsGgGdpNSolBCezP1wDB Gdvy9ZdkV+Az/F0J9HeZfl7dY0XhNzZrRrCo0k73gt0Gn1XAOCHI3Pbt/4kUEgV0DIpA Ybwrl7I8DAuC7ospiae4Q9+MIDFLRMTteJuxc4n6jfP3gy64VAg2EMH6gtGqgdpBZ3j7 6IenJD/GEFta9UqpacZfP1wNM+5GPoGFp60iVWJY1Qjhqv++RhJF7oW7DoeSXvM4hEUB 9nIxlySuMFq0A7Jhdc70obMMdYV8tcdzCfNSFSBHsnxm8Fxln7llBu16nzG1pt/hkMkz 13/g== X-Gm-Message-State: AOAM531N66xz0wvKiCah/ODrjHlaQ3lQNavCi/VsTerbLDkqQW8GEitw M4/1pNXK5xtq0P9XjPMvzDyC7q2wgINcPkyg5eanjshYp6akJQ== X-Google-Smtp-Source: ABdhPJwcrSCRhFvb6x4FjjxXLyAxHx3ZTaUnK/ueqRUenKq7KOnqh7TTPHe7+0emewB3/AAAkzZDunb0PAvAqaVHP8Y= X-Received: by 2002:a9d:3b26:: with SMTP id z35mr13613555otb.192.1612819256991; Mon, 08 Feb 2021 13:20:56 -0800 (PST) MIME-Version: 1.0 From: Ioannis Kappas Date: Mon, 8 Feb 2021 21:20:47 +0000 Message-ID: Subject: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt To: bug-gnu-emacs@gnu.org Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::334; envelope-from=ioannis.kappas@gmail.com; helo=mail-ot1-x334.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) There appears to be an issue with the /message/ function when emacs is invoked in -batch mode on windows-nt from outside the windows console (i.e. not directly from the command prompt). Messages are not displayed to the user until emacs exits. Consider the following command from example, which suppose to print the message "hi" first to the user and then exit after 5 seconds: : emacs -Q --batch --eval="(progn (message \"hi\") (sit-for 5))" When this command is invoked from within an emacs eshell on windows, the message is only displayed to the user after about 5 seconds, i.e when emacs is about to exit. Here is a small summary running the above under Linux and windows native: | system | invoked from | terminal output | |---------------------+----------------+--------------------------------------------| | x86_64-pc-linux-gnu | terminal | prints "hi", then after 5 seconds it exits | | x86_64-pc-linux-gnu | emacs eshell | prints "hi", then after 5 seconds it exits | | x86_64-w64-mingw32 | command prompt | prints "hi", then after 5 seconds it exits | | x86_64-w64-mingw32 | emacs eshell | after 5 seconds prints "hi", then exits | The issue on windows is not exclusive to invoking emacs -batch from within emacs (e.g. from eshell), it applies to any invocation that happens outside the command prompt, e.g. from inside the MSYS2 mintty terminal. It appears as if the issue is caused by a different stderr buffering mode used when a program is invoked outside of the command prompt. Analysis to follow. In GNU Emacs 27.1 (build 1, x86_64-w64-mingw32) of 2020-11-19 built on fv-az68-340 Repository revision: ec297125a76481c55390d0b329e541907879d6f3 Repository branch: master Windowing system distributor 'Microsoft Corp.', version 10 ------------=_1615488242-27289-1--