From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: 24.0.50; shell buffer overflow when input longer than 4096 bytes Resent-From: jidanni@jidanni.org Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 10 May 2010 04:17:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 6149@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.12734650113013 (code B ref -1); Mon, 10 May 2010 04:17:01 +0000 Received: (at submit) by debbugs.gnu.org; 10 May 2010 04:16:51 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OBKQF-0000mY-AO for submit@debbugs.gnu.org; Mon, 10 May 2010 00:16:51 -0400 Received: from mx10.gnu.org ([199.232.76.166]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OBKP5-0000m5-HC for submit@debbugs.gnu.org; Mon, 10 May 2010 00:16:50 -0400 Received: from lists.gnu.org ([199.232.76.165]:40976) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1OBKOq-0003iR-Qp for submit@debbugs.gnu.org; Mon, 10 May 2010 00:15:24 -0400 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1OBKOp-0008Ho-PJ for bug-gnu-emacs@gnu.org; Mon, 10 May 2010 00:15:23 -0400 Received: from [140.186.70.92] (port=57086 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OBKOi-0008A7-9y for bug-gnu-emacs@gnu.org; Mon, 10 May 2010 00:15:23 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_MIME_NO_TEXT, T_TVD_MIME_NO_HEADERS autolearn=unavailable version=3.3.1 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OBKOS-0004Dg-FC for bug-gnu-emacs@gnu.org; Mon, 10 May 2010 00:15:05 -0400 Received: from caiajhbdcahe.dreamhost.com ([208.97.132.74]:40434 helo=homiemail-a14.g.dreamhost.com) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OBKOS-0004DP-9J for bug-gnu-emacs@gnu.org; Mon, 10 May 2010 00:15:00 -0400 Received: from jidanni.org (218-163-3-173.dynamic.hinet.net [218.163.3.173]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: jidanni@jidanni.org) by homiemail-a14.g.dreamhost.com (Postfix) with ESMTPSA id 5C2BB8C06A for ; Sun, 9 May 2010 21:14:57 -0700 (PDT) From: jidanni@jidanni.org Date: Mon, 10 May 2010 12:14:54 +0800 Message-ID: <87aas81jgh.fsf@jidanni.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Spam-Score: -5.3 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.3 (-----) --=-=-= This is a serious bug in M-x shell. It is not a bash or dash bug. It is not a readline bug. It does not happen in xterm. It does not happen when using pipes or backticks to get the input. It only happens in M-x shell... when one gives lines longer than ~4096 characters. Actually it is not buffer overflow, but buffer truncation, with NO WARNING to the user. One day the wrong file will get removed via this mess. In GNU Emacs 24.0.50.1 (i486-pc-linux-gnu, GTK+ Version 2.20.0) of 2010-05-01 on elegiac, modified by Debian (emacs-snapshot package, version 1:20100501-1) --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=input_truncation.txt.gz Content-Transfer-Encoding: base64 Content-Description: buffer truncation H4sICIKG50sAA2lucHV0X3RydW5jYXRpb24udHh0AO1YyW7jOBC991f0YeYUkBCpxfLXGJREK5xQ opqkbHdjkG+fKsoLtbg754EPQZCwVNt7tZB/fZf1u/muVT9eiFZVTRp5OnBa0DQl7FNeBmlVJ3sv NM0PKi0L2sgK5Kvz+UzgUB9yWqY5YQeh9e0MpLWpOehJaPrZKs8TliR5UoBYrKMzv/5xaXNgdE85 zT6rUemGk3QmVIu+ktYK0vqP5JBQni203AQ2Dys9Sm+Mf08PGS1Swh+nR2W7s7CSTNEfrZRPI78F d343ykHIENfMjPuhlZcpSQ4paOApZbGl2poeDpJBM8JYEn+JllUnWkmupjkpyuIrCOCXlXB/9vke p5VCe/kR0hTB9XDhbpm/8d0svnfvB+KkPUlLnOoGLSfwE5qlc+zbERhEGE0gFyyfJRxPnQcvOtAy 6LFVvQshJJShOP6ifL9A8P7JH6SE0ztgUg6pXxz1ppPkToQdhJgm4N9CSho1MFCQ0d1nJb1I35qj a2OpBWF60wfOzNI5l5mf1YMyYJwxkj2UVq7pBMh6pd2hBGKxyOJgTT24A/s7FSlUSEn2v+EOJ3mE IGFR5hGTDTTw3yuUGnU8Tt5MZmeJgh8lrs6iR4sq6KpaK+DfgYdPc8o/QxLnZs9XqeT3Yq7RjPLJ IoEcAjaQvIwUc9z8Bw9k45Qna+yn09p0HZTgXSTmq1CalU++hp+78pymm8T6kD+t6ttkotVMRW2s vAGbz9I4fLQQDsvpji76lTHOE2Um0jugIzhwmH4tGuMgOtKZZtTSoS4wPT+/8CIj+zTAmFB36kMX ztieJHThS+j6d3+i7IgT+p8nhzwogeJ6uyviO1CUzDNiOqihDOSSBZi6EQPQNCM4FTLKl6k2tgZt WajvLIjNqGDGAT9mG0l2P91J9Qo4R8uyccf406pqKvykiDA/mt7XULqqJdMv/BD9jSJv8QQ9TYFz C09DvkLzWyXs5gqZQN9wCMfVaHV62E1845t8a4QXD8rFZIXS045jje+xG7IFIcAtFrjAs9nJScM0 hp5MZxN4+OnfTQ9jYTDWV2MLwLE4C9M5B+p2EFUnNJjN4a9innxi6yjUewpB7dSK1rRN2jVhsdVB I9TYViDl+5WbbhzQzxBGeTfzY2iO2FPn9TU62C1G6ENAUEj0XRrBcbVVg3+Gjjgdje2Ez/lXKV/X LJTYJuvFIOqPtD2GZslIOTtF39O180/xwORBY8BM5ewr7nXQOEMBAE3jXIvBKw/NH9JTBL8irz/C FngdAKi+XPLfdkT1Xmo2VfIy5ql2YMRtVo8Xrkfc+ZyKANhkErDdzVooVMtGsTyKOKrepZdWNNL0 z9wEnL0VfbpuVwIGdRMwxQV2uVGCZr6t8nhUOa55dL84gPLDUbBRgsAeAvanRW6LQY9A2ZNIA2Db UOGEsFAGsBPeCy4ik3PvCA+O1XzxpfNN/fZWbLrkzq4WWn51JoR+R6bOtd32zCB7K84sVOsyhtHL bvCw/YUI8nkzwQ3orIHlfUtgRtg6bGN0P61u8ZJbWXdqebQIFPNxPgGR3YDIVrDXBLY6hxBCxydR iOHSQqRzsNEo6JGMLW5Kk2W+YXS95DixueXEcd42zO1IsYU+tjfQFARYrG6wynWcQMnddUFBO7y0 STvXukGJkCVcF9ZJAoUhfdDrCF6BonEakruZ1x8+I5dOH7LQQqEdxWZ76WvhCRIEMgWpKffxcT0M 23qr60JNGlNPq10OaZ3WnzIaqcoZoEQjcZllxXzWomNNNbptz24WtrQvw4MozsZ+bCu6ygBHnp7H HNmgx7QDEuGcAhDD0AMOPiq9ld7LS/g3tNb5Mv8uNewMUJYwavZx9Hce8edEQtdOPTaQAiwub0tI Byu13GSE/zXtOJA0FeNxBi9hs1nd7mSrsbGE9TDBBWG5ODHSSSdIqy8QTLmxyY+TxPoU0LkVfeQI DFnoPMHBhPOMRCXdvr1docrm5Xy5XpAvxrZQUcPoifvZ47CtXcBt5TbcpHG9xzWpfNrZnpTkhjV5 Cnt8wAtTPscKq0joaVlhSJFdvr7R3PfLcj6otGmt8cIjlpDCWDXesYjs4TYvb11neZe6pnMqx0dK Y74NqYZLPr6f7Eu8B9DVM8mtAaKSzd43GGhMFU7LMGhZ8SiBEBeBol1H5mQD2OMWFD15/Pv9XH97 vYq9XsVer2KvV7EwK1+vYo9Z+noVe72KvV7FXq9ir1ex16vYQ/PrVez1KvZ6FXu9iv0PXsW+/Qem /j/+GCMAAA== --=-=-=-- From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: 24.0.50; shell buffer overflow when input longer than 4096 bytes Resent-From: Stefan Monnier Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 01 Jun 2010 01:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: moreinfo unreproducible To: jidanni@jidanni.org Cc: 6149@debbugs.gnu.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.12753570406650 (code B ref 6149); Tue, 01 Jun 2010 01:51:02 +0000 Received: (at 6149) by debbugs.gnu.org; 1 Jun 2010 01:50:40 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OJGcq-0001jD-I0 for submit@debbugs.gnu.org; Mon, 31 May 2010 21:50:40 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.183] helo=ironport2-out.pppoe.ca) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OJGco-0001j6-Vl for 6149@debbugs.gnu.org; Mon, 31 May 2010 21:50:39 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlUFAGIEBExMCpdY/2dsb2JhbACSJIwHcr8BhRYEjEw X-IronPort-AV: E=Sophos;i="4.53,337,1272859200"; d="scan'208";a="66559480" Received: from 76-10-151-88.dsl.teksavvy.com (HELO pastel.home) ([76.10.151.88]) by ironport2-out.pppoe.ca with ESMTP; 31 May 2010 21:50:37 -0400 Received: by pastel.home (Postfix, from userid 20848) id 3B98681F8; Mon, 31 May 2010 21:50:37 -0400 (EDT) From: Stefan Monnier Message-ID: References: <87aas81jgh.fsf@jidanni.org> Date: Mon, 31 May 2010 21:50:37 -0400 In-Reply-To: <87aas81jgh.fsf@jidanni.org> (jidanni@jidanni.org's message of "Mon, 10 May 2010 12:14:54 +0800") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -1.4 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.2 (--) >>>>> "jidanni" == jidanni writes: > This is a serious bug in M-x shell. It is not a bash or dash bug. It is > not a readline bug. It does not happen in xterm. It does not happen when > using pipes or backticks to get the input. It only happens in M-x > shell... when one gives lines longer than ~4096 characters. > Actually it is not buffer overflow, but buffer truncation, with NO > WARNING to the user. One day the wrong file will get removed via this > mess. > In GNU Emacs 24.0.50.1 (i486-pc-linux-gnu, GTK+ Version 2.20.0) > of 2010-05-01 on elegiac, modified by Debian > (emacs-snapshot package, version 1:20100501-1) Thanks for this nice test case. It appears it was a silly mistake (code placed in the wrong side of a #if). I've installed the patch below which should fix it, Stefan === modified file 'src/sysdep.c' --- src/sysdep.c 2010-05-04 07:40:53 +0000 +++ src/sysdep.c 2010-06-01 01:40:00 +0000 @@ -537,15 +537,6 @@ s.main.c_cflag = (s.main.c_cflag & ~CBAUD) | B9600; /* baud rate sanity */ #endif /* AIX */ -#else /* not HAVE_TERMIO */ - - s.main.sg_flags &= ~(ECHO | CRMOD | ANYP | ALLDELAY | RAW | LCASE - | CBREAK | TANDEM); - s.main.sg_flags |= LPASS8; - s.main.sg_erase = 0377; - s.main.sg_kill = 0377; - s.lmode = LLITOUT | s.lmode; /* Don't strip 8th bit */ - /* We used to enable ICANON (and set VEOF to 04), but this leads to problems where process.c wants to send EOFs every once in a while to force the output, which leads to weird effects when the @@ -558,6 +549,15 @@ s.main.c_cc[VMIN] = 1; s.main.c_cc[VTIME] = 0; +#else /* not HAVE_TERMIO */ + + s.main.sg_flags &= ~(ECHO | CRMOD | ANYP | ALLDELAY | RAW | LCASE + | CBREAK | TANDEM); + s.main.sg_flags |= LPASS8; + s.main.sg_erase = 0377; + s.main.sg_kill = 0377; + s.lmode = LLITOUT | s.lmode; /* Don't strip 8th bit */ + #endif /* not HAVE_TERMIO */ EMACS_SET_TTY (out, &s, 0); From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 22 02:30:42 2010 Received: (at control) by debbugs.gnu.org; 22 Jun 2010 06:30:42 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OQx0M-00049u-C5 for submit@debbugs.gnu.org; Tue, 22 Jun 2010 02:30:42 -0400 Received: from fencepost.gnu.org ([140.186.70.10]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OQx0L-00049p-1I for control@debbugs.gnu.org; Tue, 22 Jun 2010 02:30:41 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.69) (envelope-from ) id 1OQx0F-0003N7-O6; Tue, 22 Jun 2010 02:30:35 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19488.22667.377046.876848@fencepost.gnu.org> Date: Tue, 22 Jun 2010 02:30:35 -0400 From: Glenn Morris To: control Subject: control X-Attribution: GM X-Mailer: VM (www.wonderworks.com/vm), GNU Emacs (www.gnu.org/software/emacs) X-Hue: white X-Ran: HrkKm}gAM':F_6|R!ZW,M&C6n#=p;r<^j+?h?$5SU,~m$S(\K6{Xl~s%*=.DlP*ftA{.Tj X-Debbugs-No-Ack: yes X-Spam-Score: -5.1 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.1 (-----) close 6149 close 6428 From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 28 15:47:48 2018 Received: (at control) by debbugs.gnu.org; 28 Sep 2018 19:47:48 +0000 Received: from localhost ([127.0.0.1]:57991 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g5yjs-0001ss-3g for submit@debbugs.gnu.org; Fri, 28 Sep 2018 15:47:48 -0400 Received: from sinyavsky.aurox.ch ([37.35.109.145]:51596) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g5yjp-0001oK-Ur for control@debbugs.gnu.org; Fri, 28 Sep 2018 15:47:46 -0400 Received: from sinyavsky.aurox.ch (sinyavsky.aurox.ch [127.0.0.1]) by sinyavsky.aurox.ch (Postfix) with ESMTP id CCF7B2286F for ; Fri, 28 Sep 2018 19:51:34 +0000 (UTC) Authentication-Results: sinyavsky.aurox.ch (amavisd-new); dkim=pass reason="pass (just generated, assumed good)" header.d=aurox.ch DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=aurox.ch; h= reply-to:subject:subject:to:from:from:message-id:date:date; s= dkim; t=1538164294; x=1539028295; bh=1VFGL9+pztD45gsLf2zvehHX2Iz n170gH+wtr7UjXr4=; b=WbnD/5mnT4T5hAgXZMWbq3Pjt9Zj2G/Muw0GvEwr+uV +UBgrdW96YvzhaOF5OgqalWB0zoEIL38NSuhZdZPDLpxXp32HbityuUX4UbKO1lX XOjTyMIUnK+VCSEu4Dag0xIrhKbO+CAYl6xprFMGdn1BlcKE6RULlUH7zX33hDZs = X-Virus-Scanned: Debian amavisd-new at test.virtualizor.com Received: from sinyavsky.aurox.ch ([127.0.0.1]) by sinyavsky.aurox.ch (sinyavsky.aurox.ch [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Z2U7dMVMQ7xw for ; Fri, 28 Sep 2018 19:51:34 +0000 (UTC) Received: from gray (unknown [IPv6:2a02:1205:c693:2d60:c62c:3ff:fe30:b864]) by sinyavsky.aurox.ch (Postfix) with ESMTPSA id 5415F226F4 for ; Fri, 28 Sep 2018 19:51:34 +0000 (UTC) Date: Fri, 28 Sep 2018 21:50:31 +0200 Message-Id: From: charles@aurox.ch (Charles A. Roelli) To: control@debbugs.gnu.org Subject: unarchive 6149 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: charles@aurox.ch Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) unarchive 6149 From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: 24.0.50; shell buffer overflow when input longer than 4096 bytes Resent-From: charles@aurox.ch (Charles A. Roelli) Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 28 Sep 2018 20:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: jidanni@jidanni.org Cc: 6149@debbugs.gnu.org Reply-To: charles@aurox.ch Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.153816541517616 (code B ref 6149); Fri, 28 Sep 2018 20:11:02 +0000 Received: (at 6149) by debbugs.gnu.org; 28 Sep 2018 20:10:15 +0000 Received: from localhost ([127.0.0.1]:58015 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g5z5b-0004a3-6y for submit@debbugs.gnu.org; Fri, 28 Sep 2018 16:10:15 -0400 Received: from sinyavsky.aurox.ch ([37.35.109.145]:51664) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g5z5a-0004Zp-3i for 6149@debbugs.gnu.org; Fri, 28 Sep 2018 16:10:14 -0400 Received: from sinyavsky.aurox.ch (sinyavsky.aurox.ch [127.0.0.1]) by sinyavsky.aurox.ch (Postfix) with ESMTP id 8F42A2287E for <6149@debbugs.gnu.org>; Fri, 28 Sep 2018 20:14:03 +0000 (UTC) Authentication-Results: sinyavsky.aurox.ch (amavisd-new); dkim=pass reason="pass (just generated, assumed good)" header.d=aurox.ch DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=aurox.ch; h= references:reply-to:subject:subject:in-reply-to:to:from:from :message-id:date:date; s=dkim; t=1538165641; x=1539029642; bh=N5 Qo+caOmQGTJHcBrO6QfqdFN6wjHIh/8A/SV/jZEa8=; b=S9/JBc58W8NzPSefj7 sEoyS6JTKZ0E62ODJHumbogR5rHERZVpwGW8nwm7sXHu1ci2s3IYN1HJg3DMv2/C +LukWJ2GlWt0BITx8AeD4o4dQ0X1A0TSps+0BVs/wcvBiwPs7nqwCaaest/4PYvO HgikOzvWHZs3GbDVokJhg3TH8= X-Virus-Scanned: Debian amavisd-new at test.virtualizor.com Received: from sinyavsky.aurox.ch ([127.0.0.1]) by sinyavsky.aurox.ch (sinyavsky.aurox.ch [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id dk3dxUVI_5eK for <6149@debbugs.gnu.org>; Fri, 28 Sep 2018 20:14:01 +0000 (UTC) Received: from gray (unknown [IPv6:2a02:1205:c693:2d60:c62c:3ff:fe30:b864]) by sinyavsky.aurox.ch (Postfix) with ESMTPSA id 385B9226F4; Fri, 28 Sep 2018 20:14:01 +0000 (UTC) Date: Fri, 28 Sep 2018 22:13:11 +0200 Message-Id: From: charles@aurox.ch (Charles A. Roelli) In-reply-to: <87aas81jgh.fsf@jidanni.org> References: <87aas81jgh.fsf@jidanni.org> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: jidanni@jidanni.org > Date: Mon, 10 May 2010 12:14:54 +0800 > > This is a serious bug in M-x shell. It is not a bash or dash bug. It is > not a readline bug. It does not happen in xterm. It does not happen when > using pipes or backticks to get the input. It only happens in M-x > shell... when one gives lines longer than ~4096 characters. > > Actually it is not buffer overflow, but buffer truncation, with NO > WARNING to the user. One day the wrong file will get removed via this > mess. > > In GNU Emacs 24.0.50.1 (i486-pc-linux-gnu, GTK+ Version 2.20.0) > of 2010-05-01 on elegiac, modified by Debian > (emacs-snapshot package, version 1:20100501-1) > > > [application/octet-stream input_truncation.txt.gz (2kB)] I can still reproduce this bug in 26.1 with the following recipe: M-x shell RET echo SPC C-SPC C-u 5000 a RET C-p C-e M-= On GNU/Linux: Region has 2 lines, 2 words, and 9096 characters. If echo had received all of the input, you would expect around 10000 characters in the region. Instead, there are 5000 + 4096 characters. Back when EOF chars were used to flush output, we had an "fpathconf" check as in: https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=3d082a269ece18058ed82957f8a056822b39789e It might be possible to reinstate this "fpathconf" check to warn the user that he has gone over the PTY limit, or maybe to prevent overlong lines from being sent at all. There is further discussion at: http://lists.gnu.org/archive/html/emacs-devel/2010-08/msg00209.html (Also, repeating this recipe on macOS with Emacs 26.1 results in the behavior pointed out in Bug#32438.) From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 13 16:37:44 2019 Received: (at control) by debbugs.gnu.org; 13 Oct 2019 20:37:44 +0000 Received: from localhost ([127.0.0.1]:37130 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iJkcZ-0001aO-NU for submit@debbugs.gnu.org; Sun, 13 Oct 2019 16:37:43 -0400 Received: from quimby.gnus.org ([80.91.231.51]:38322) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iJkcX-0001aE-HF for control@debbugs.gnu.org; Sun, 13 Oct 2019 16:37:41 -0400 Received: from cm-84.212.202.86.getinternet.no ([84.212.202.86] helo=marnie) by quimby.gnus.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1iJkcU-0005Cx-QS for control@debbugs.gnu.org; Sun, 13 Oct 2019 22:37:40 +0200 Date: Sun, 13 Oct 2019 22:37:38 +0200 Message-Id: <87d0f08jql.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #6149 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: forcemerge 6149 12440 quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) forcemerge 6149 12440 quit From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Spencer Baugh Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 20 Jul 2023 20:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Stefan Monnier Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.168988413631387 (code B ref 6149); Thu, 20 Jul 2023 20:16:02 +0000 Received: (at 6149) by debbugs.gnu.org; 20 Jul 2023 20:15:36 +0000 Received: from localhost ([127.0.0.1]:60014 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMa3j-0008A2-5Q for submit@debbugs.gnu.org; Thu, 20 Jul 2023 16:15:36 -0400 Received: from mxout1.mail.janestreet.com ([38.105.200.78]:55743) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMa3S-000896-09 for 6149@debbugs.gnu.org; Thu, 20 Jul 2023 16:15:19 -0400 From: Spencer Baugh In-Reply-To: (Stefan Monnier's message of "Mon, 31 May 2010 21:50:37 -0400") References: <87aas81jgh.fsf@jidanni.org> Date: Thu, 20 Jul 2023 16:15:11 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain I see that this bug is about 13 years old. I think there's a pretty obvious solution: process-connection-type should default to nil. Otherwise this is a footgun just waiting to happen for anyone writing process-interaction code in Emacs. But if we don't do that, we should at least document it. See my attached patch. Btw, just to feed the fire, here's my own reproducer: (with-temp-buffer (make-process :name "broken" :buffer (current-buffer) :command '("cat")) (process-send-string nil (make-string 10000 ?x)) (process-send-eof) (sit-for 1) (cons (point-min) (point-max))) --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-Include-warning-about-long-line-truncation-in-proces.patch >From dcfd129b3f08273a8b0705f03b6074443a7a33c1 Mon Sep 17 00:00:00 2001 From: Spencer Baugh Date: Thu, 20 Jul 2023 16:13:56 -0400 Subject: [PATCH] Include warning about long line truncation in process-send-string Maybe we can't fix this. But we can at least warn the user about it! To have no warning anywhere about this default behavior which silently discards data, is very user-hostile. * src/process.c (Fprocess_send_string): Include a warning about long line truncation (bug#6149) --- src/process.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/process.c b/src/process.c index 67d1d3e425f..82ace1b3a41 100644 --- a/src/process.c +++ b/src/process.c @@ -6755,6 +6755,8 @@ DEFUN ("process-send-string", Fprocess_send_string, Sprocess_send_string, of which depends on the process connection type and the operating system), it is sent in several bunches. This may happen even for shorter strings. Output from processes can arrive in between bunches. +If the process connection type is `pty', then long lines present in +STRING may be truncated depending on the operating system. If PROCESS is a non-blocking network process that hasn't been fully set up yet, this function will block until socket setup has completed. */) -- 2.39.3 --=-=-=-- From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 20 Jul 2023 21:23:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Spencer Baugh Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.16898881266442 (code B ref 6149); Thu, 20 Jul 2023 21:23:02 +0000 Received: (at 6149) by debbugs.gnu.org; 20 Jul 2023 21:22:06 +0000 Received: from localhost ([127.0.0.1]:60102 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMb65-0001fk-Pb for submit@debbugs.gnu.org; Thu, 20 Jul 2023 17:22:06 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:57396) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMb61-0001f6-My; Thu, 20 Jul 2023 17:22:04 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id D0E60442C30; Thu, 20 Jul 2023 17:21:55 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 656E5442C32; Thu, 20 Jul 2023 17:21:54 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1689888114; bh=uGXvrhhlltnVdVP10M8w4uUOjenJPQgF9nOuSW9zTKA=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=LBltPPGmA6uTRyiMZJhnfpCxZ5qSl0fdxP7MJuX9cnHLDa96F7pxqGdYvOwM7ZHj5 CnNWtdtAjssjjdF9bC6Zr3AXI/d1vCxIoRBQaEjx8Sr+hk+WfmvyUjjLzDceKZ0pWV rISYitQpdYYE+tmcuK6hvFpeaAwOor9EY+PJQOVovgOIV25TNqHiHBDi8T+Tk1hNcA GWxlyyLdXnomOlAjh5kyPvCn2i3GFCQXAgjoSOb7cTrEN/a0nz2JDTcBACImm3W8bt w28a7GQktJZ2xvd1bD26vdTxIKOSdBxA09bNwBqK8smWaiXDZWAoivAQT3kZ8+oxAT 6Bf/zaLt0Ntjw== Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 555E7120321; Thu, 20 Jul 2023 17:21:54 -0400 (EDT) From: Stefan Monnier In-Reply-To: (Spencer Baugh's message of "Thu, 20 Jul 2023 16:15:11 -0400") Message-ID: References: <87aas81jgh.fsf@jidanni.org> Date: Thu, 20 Jul 2023 17:21:53 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.132 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > I see that this bug is about 13 years old. I think there's a pretty > obvious solution: process-connection-type should default to nil. In practice, that can't fly because it'll break existing code. Also I don't think either value of `process-connection-type` is a good option. IOW, I think that the connection type should be a mandatory argument when creating an async process (except maybe for those processes with no input/output). So maybe, the default value of `process-connection-type` should be `unspecified` and the process creation code should emit a warning when creating a process whose connection type is `unspecified` (just a warning, tho: it should then pursue execution as if that value was t, as usual, so as to preserve compatibility). Stefan From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 21 Jul 2023 05:40:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Stefan Monnier Cc: 24531@debbugs.gnu.org, sbaugh@janestreet.com, 6149@debbugs.gnu.org, jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.16899179917312 (code B ref 6149); Fri, 21 Jul 2023 05:40:02 +0000 Received: (at 6149) by debbugs.gnu.org; 21 Jul 2023 05:39:51 +0000 Received: from localhost ([127.0.0.1]:60360 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMirm-0001tr-S0 for submit@debbugs.gnu.org; Fri, 21 Jul 2023 01:39:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47488) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMirh-0001tW-Vf; Fri, 21 Jul 2023 01:39:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qMira-00040D-6a; Fri, 21 Jul 2023 01:39:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=sz40LrDqf2E8EDxiFG/QQ/yAFYZH/VZlTh3SqEsZETw=; b=GlFGdWbA/Ghc kiWCE/SU1GXlx3XAAyb3zfoTQjZM89KoUYKx7UDwdvGK9xssseMwNZhwVuSXtmnXV9nJr54yUc36s zHHIFbMkHYecntBXxkOkiBXpW8Y5DGFEXVYryCjFdhAwDIYf2cWeUZkUVjFCGE741M80vKrIynjAg MBkqZJr9HLb0z/MigmkvDnc6GyTkSPK79AwSjRVMGqeec3Ay1OuXvs1m7wDjVndTFyo1GDPZXNZv9 gZ9XrQumTSwtg4DfH9/F1+5vss2gFawx0Jtp1QhxoOlggcxE3LOVgVEIX04AYRou1no1K7pv0T49t Stc9hNm+VciPu3HmtIY3vA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qMirF-0000B9-CM; Fri, 21 Jul 2023 01:39:31 -0400 Date: Fri, 21 Jul 2023 08:39:52 +0300 Message-Id: <83sf9h257b.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (bug-gnu-emacs@gnu.org) References: <87aas81jgh.fsf@jidanni.org> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, jidanni@jidanni.org > Date: Thu, 20 Jul 2023 17:21:53 -0400 > From: Stefan Monnier via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > > I see that this bug is about 13 years old. I think there's a pretty > > obvious solution: process-connection-type should default to nil. > > In practice, that can't fly because it'll break existing code. Indeed. A large portion (I think the majority, but I'm not sure) of Lisp programs that use bidirectional communications with async subprocesses actually _want_ the PTY interface, because they act as GUI front-ends to other programs. Think "M-x term", "M-x gdb", inferior-python-mode, etc. Even "M-x grep" and the likes need that because they rely on default color output, which only happens if Grep is connected to a terminal device. Some of the Emacs features based on this don't work or don't work well on MS-Windows because Windows only supports pipes. Do we really want such semi-broken behavior on GNU and Unix systems? The number of applications that (a) don't need console-like behavior and (b) need to send larger-than-4KB buffers to sub-processes is quite small. Which is why this issue comes up only very rarely. So making pipes the default will fix a very small fraction of applications, and break the vast majority -- clearly a wrong balance. > Also I don't think either value of `process-connection-type` is a good > option. IOW, I think that the connection type should be a mandatory > argument when creating an async process (except maybe for those > processes with no input/output). If we go that way, we should start by specifying :connection-type for all the uses of make-process and start-process we have in the core. It's a large job, but before it is done we cannot in good faith make such an incompatible transition. > So maybe, the default value of `process-connection-type` should be > `unspecified` and the process creation code should emit a warning when > creating a process whose connection type is `unspecified` (just > a warning, tho: it should then pursue execution as if that value was t, > as usual, so as to preserve compatibility). Something like that, yes. But I'm actually wondering how come modern Linux kernels don't have a way of lifting this restriction, or at least enlarging the limit so it makes the problem even less frequent. Is there some inherent limitation that this must be 4KB and nothing larger? From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Spencer Baugh Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 21 Jul 2023 13:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Eli Zaretskii Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, Stefan Monnier , jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.168994792710517 (code B ref 6149); Fri, 21 Jul 2023 13:59:02 +0000 Received: (at 6149) by debbugs.gnu.org; 21 Jul 2023 13:58:47 +0000 Received: from localhost ([127.0.0.1]:34319 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMqed-0002jY-3V for submit@debbugs.gnu.org; Fri, 21 Jul 2023 09:58:47 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:60791) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMqea-0002j8-52 for 6149@debbugs.gnu.org; Fri, 21 Jul 2023 09:58:45 -0400 From: Spencer Baugh In-Reply-To: <83sf9h257b.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 21 Jul 2023 08:39:52 +0300") References: <87aas81jgh.fsf@jidanni.org> <83sf9h257b.fsf@gnu.org> Date: Fri, 21 Jul 2023 09:58:38 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, jidanni@jidanni.org >> Date: Thu, 20 Jul 2023 17:21:53 -0400 >> From: Stefan Monnier via "Bug reports for GNU Emacs, >> the Swiss army knife of text editors" >> >> > I see that this bug is about 13 years old. I think there's a pretty >> > obvious solution: process-connection-type should default to nil. >> >> In practice, that can't fly because it'll break existing code. > > Indeed. I agree that we probably can't change the default. However... > A large portion (I think the majority, but I'm not sure) of > Lisp programs that use bidirectional communications with async > subprocesses actually _want_ the PTY interface, because they act as > GUI front-ends to other programs. Think "M-x term", "M-x gdb", > inferior-python-mode, etc. Even "M-x grep" and the likes need that > because they rely on default color output, which only happens if Grep > is connected to a terminal device. Some of the Emacs features based > on this don't work or don't work well on MS-Windows because Windows > only supports pipes. Do we really want such semi-broken behavior on > GNU and Unix systems? > > The number of applications that (a) don't need console-like behavior > and (b) need to send larger-than-4KB buffers to sub-processes is quite > small. Which is why this issue comes up only very rarely. So making > pipes the default will fix a very small fraction of applications, and > break the vast majority -- clearly a wrong balance. I see your point, but at the same time, the PTY interface on its own is not sufficient to make these applications work, not at all. Specialized modes are necessary to make M-x term (to implement a terminal) and M-x grep (to parse ANSI color codes) and other such programs work. Running things in a PTY without such specialized code doesn't give you anything, AFAIK, because a PTY alone is far from enough to make the Emacs end behave like a terminal. So such programs need to be aware and careful about such things anyway, and need additional infrastructure on top of make-process. So the default being "pty" gives such programs very little: it doesn't save them any complexity. Programs that just want to do some data processing with a subprocess, on the other hand, work fine with just make-process alone, they need no additional infrastructure, just process-send-string and reading directly from the process buffer. The default being "pipe" would take away a big footgun for such programs, since it's easy to forget that and then have a silently wrong program which will fail once you get large input. >> Also I don't think either value of `process-connection-type` is a good >> option. IOW, I think that the connection type should be a mandatory >> argument when creating an async process (except maybe for those >> processes with no input/output). > > If we go that way, we should start by specifying :connection-type for > all the uses of make-process and start-process we have in the core. > It's a large job, but before it is done we cannot in good faith make > such an incompatible transition. I can do that. However, what about my patch adding a warning about this to process-send-string? I think that is independently valuable. Right now we have no documentation of this problem... >> So maybe, the default value of `process-connection-type` should be >> `unspecified` and the process creation code should emit a warning when >> creating a process whose connection type is `unspecified` (just >> a warning, tho: it should then pursue execution as if that value was t, >> as usual, so as to preserve compatibility). > > Something like that, yes. > > But I'm actually wondering how come modern Linux kernels don't have a > way of lifting this restriction, or at least enlarging the limit so it > makes the problem even less frequent. Is there some inherent > limitation that this must be 4KB and nothing larger? Unfortunately from looking at Linux the limit of 4096 seems to be hardcoded. From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 21 Jul 2023 14:19:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Spencer Baugh Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, monnier@iro.umontreal.ca, jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.168994909012336 (code B ref 6149); Fri, 21 Jul 2023 14:19:02 +0000 Received: (at 6149) by debbugs.gnu.org; 21 Jul 2023 14:18:10 +0000 Received: from localhost ([127.0.0.1]:34332 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMqxO-0003Ct-0F for submit@debbugs.gnu.org; Fri, 21 Jul 2023 10:18:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37356) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMqxL-0003Cc-7P; Fri, 21 Jul 2023 10:18:09 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qMqxE-0003dh-PF; Fri, 21 Jul 2023 10:18:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=t9KuBf7oYux5laraWfRTHJ1OQMMJXSd436pIbHu+gic=; b=Z9cPQNMF47fT PrhLWsCCvqhibPzFItrdAMdJFrPg3BR9GJQo1nQQIVvFgNOFk1cQloj/E1twj8B266RbH1pbbfQ7Q sJF+CNFXoUKqUKnrzY7x2iP8Iclf2EnypplbDDj1ljjXCTtzHey3VyjxOUc7t5MbzT/uA6tgB0ziv z0yvarzrz64nW3ANKjxpiRBwAj/VaDXOCJta4kMrcEQyZwCuufUVTpkVFN7gS2sAKre4wMi9viZqQ WK7dlGcdixOw2GCFb0J1UMLK90Ttv21xkgscYLYW5ZFzN1RTn52O45FhmDqjof6B0APMUCjMbeSrc yzt/9SpHzcYBiAMNFPQwhw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qMqxE-00008z-8s; Fri, 21 Jul 2023 10:18:00 -0400 Date: Fri, 21 Jul 2023 17:18:34 +0300 Message-Id: <83edl1bb5x.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (message from Spencer Baugh on Fri, 21 Jul 2023 09:58:38 -0400) References: <87aas81jgh.fsf@jidanni.org> <83sf9h257b.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Spencer Baugh > Cc: Stefan Monnier , 24531@debbugs.gnu.org, > 6149@debbugs.gnu.org, jidanni@jidanni.org > Date: Fri, 21 Jul 2023 09:58:38 -0400 > > Eli Zaretskii writes: > > > The number of applications that (a) don't need console-like behavior > > and (b) need to send larger-than-4KB buffers to sub-processes is quite > > small. Which is why this issue comes up only very rarely. So making > > pipes the default will fix a very small fraction of applications, and > > break the vast majority -- clearly a wrong balance. > > I see your point, but at the same time, the PTY interface on its own is > not sufficient to make these applications work, not at all. Specialized > modes are necessary to make M-x term (to implement a terminal) and M-x > grep (to parse ANSI color codes) and other such programs work. Running > things in a PTY without such specialized code doesn't give you anything, > AFAIK, because a PTY alone is far from enough to make the Emacs end > behave like a terminal. So such programs need to be aware and careful > about such things anyway, and need additional infrastructure on top of > make-process. So the default being "pty" gives such programs very > little: it doesn't save them any complexity. That Emacs needs to do something doesn't invalidate my point. My point is that communications via a PTY is a necessary (though a sufficient) condition for these features. Basically, you cannot use pipes for any interactive feature, because pipes are buffered. > However, what about my patch adding a warning about this to > process-send-string? I think that is independently valuable. Right now > we have no documentation of this problem... This should be documented in the ELisp manual, and in more detail, not just as a vague warning. From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 21 Jul 2023 15:10:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Eli Zaretskii Cc: 24531@debbugs.gnu.org, sbaugh@janestreet.com, 6149@debbugs.gnu.org, jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.168995216917681 (code B ref 6149); Fri, 21 Jul 2023 15:10:02 +0000 Received: (at 6149) by debbugs.gnu.org; 21 Jul 2023 15:09:29 +0000 Received: from localhost ([127.0.0.1]:34485 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMrl3-0004b2-Dj for submit@debbugs.gnu.org; Fri, 21 Jul 2023 11:09:29 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:50532) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qMrkz-0004ae-0B; Fri, 21 Jul 2023 11:09:27 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 68E5780575; Fri, 21 Jul 2023 11:09:19 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 2BDF980058; Fri, 21 Jul 2023 11:09:18 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1689952158; bh=uF20kHarf1v+hk7anbvAyj0y9cvctV9Za+N7/9SOO30=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=lHOd6BbXFMVc6eJSnynvtBH4YuaqOYkU5DAgBCayKQ9Yz9s0lVkyAjWpdyovfuNVM i0UN4sjYzU/VTOom//n7Fb9wCY3giEeM/mOu8seiLQEJBFOD8+ZTOMLMogbQL1nOm5 gAYoM+EUA8H9BBMp1p7iF2Wl2TT/FKjXiYJPMfXFwSJFvRYPaYtqcI0qj1x4v59q1V B9DlzQVdttbljd0vPRvbBKMDJpEknUG8nt+YKVbPvneWUS/pzY7YtlRNjAIhAO1dWf X7uAbe2roEJCYA5kVEUfSXngs8QOpEl8kfOIdKvTF0IogophFMMYniBXFWhdkxz1yc N6ojRScg/vdrA== Received: from alfajor (unknown [23.233.149.155]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E634B12037A; Fri, 21 Jul 2023 11:09:17 -0400 (EDT) From: Stefan Monnier In-Reply-To: <83sf9h257b.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 21 Jul 2023 08:39:52 +0300") Message-ID: References: <87aas81jgh.fsf@jidanni.org> <83sf9h257b.fsf@gnu.org> Date: Fri, 21 Jul 2023 11:09:17 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.230 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> Also I don't think either value of `process-connection-type` is a good >> option. IOW, I think that the connection type should be a mandatory >> argument when creating an async process (except maybe for those >> processes with no input/output). > If we go that way, we should start by specifying :connection-type for > all the uses of make-process and start-process we have in the core. That would be a good start, yes. Stefan From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 27 Jul 2023 01:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Spencer Baugh , Eli Zaretskii Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, Stefan Monnier , jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.169042250930527 (code B ref 6149); Thu, 27 Jul 2023 01:49:02 +0000 Received: (at 6149) by debbugs.gnu.org; 27 Jul 2023 01:48:29 +0000 Received: from localhost ([127.0.0.1]:40398 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qOq7B-0007wD-CN for submit@debbugs.gnu.org; Wed, 26 Jul 2023 21:48:29 -0400 Received: from wout3-smtp.messagingengine.com ([64.147.123.19]:35445) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qOq79-0007vS-EG; Wed, 26 Jul 2023 21:48:28 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 632A93200917; Wed, 26 Jul 2023 21:48:21 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 26 Jul 2023 21:48:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690422500; x=1690508900; bh=swj++But/B6fBXM5vt/zbu6jN8vlelyDDz3 EQoEm76I=; b=OdzikHWIcaeaq0dRpvISx2zQuHIgIjAtrWzk5uH+D2M3GKklN8i gcsu7TyR0H8KMWXjb41PR+WIDYx3o8ghxg/UrGEDpiSSKtcNrv5wEi7DFixlN/1u FjkQ3aqMxOhOBXeDrtghYTwGJQkACJHl1bTEJCqaEDEsKEFl28W4lI3lMUurvr8Z 7WV7d4iFHWiCyAASovZXxktyM31paKMMJMGOMO6IgPo24w6VRtCyfYPAgcx9rhVV zvhmTPRTiVCeMZiGXqTYnedRhVGZAhOiM+3Eokq/j/kQfAPYXAky32Dm4NFZWNGz XOPec4f9y1kytx8WLSRWBxxzIuDLD/vWZzA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690422500; x=1690508900; bh=swj++But/B6fBXM5vt/zbu6jN8vlelyDDz3 EQoEm76I=; b=iKREnXM+etOM/WdBO3uFpryipiN29uluVOomvR4AAtgMbvOPsLy wTLiz3e4Rri4f7a5+iGw7bd6KJ/M7FkFsxM4nGCrKe+D7TC11eD8PLY6YqC4XEPb eLITkb4oGeptDde/sx417nNnVNY726uzfBguoT7O5u09Lat+HyQqZWnas/sqj9HO cY24H637w73fmgmR/O+pKhfnLjQ3W5dC8lFK64uHWskxyR4CRuVJr0OaoQZvtdyK lKdQOLC6gTvEC9S1d4gVz0FdAHpQ2RBi5EK+TMORCbkGTIPv4k1/YGDghTFQ+sx8 gxPxvG5BAg3dlyLcmmA4sLmoeyJrWB03hpA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrieefgdehtdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 26 Jul 2023 21:48:19 -0400 (EDT) Message-ID: <19e73ab0-19b5-d7f4-8912-20c9e822e3fb@gutov.dev> Date: Thu, 27 Jul 2023 04:48:18 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Content-Language: en-US References: <87aas81jgh.fsf@jidanni.org> <83sf9h257b.fsf@gnu.org> From: Dmitry Gutov In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.8 (-) On 21/07/2023 16:58, Spencer Baugh wrote: >> But I'm actually wondering how come modern Linux kernels don't have a >> way of lifting this restriction, or at least enlarging the limit so it >> makes the problem even less frequent. Is there some inherent >> limitation that this must be 4KB and nothing larger? > Unfortunately from looking at Linux the limit of 4096 seems to be > hardcoded. If some syscall or etc limits the length of a string to 4096, can't we detect this case, split the string and emit said call multiple times? This function's docstring already mentions the case of If STRING is larger than the input buffer of the process, ... it is sent in several bunches From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 27 Jul 2023 05:42:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Dmitry Gutov , Paul Eggert Cc: 24531@debbugs.gnu.org, sbaugh@janestreet.com, 6149@debbugs.gnu.org, monnier@iro.umontreal.ca, jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.169043647421363 (code B ref 6149); Thu, 27 Jul 2023 05:42:02 +0000 Received: (at 6149) by debbugs.gnu.org; 27 Jul 2023 05:41:14 +0000 Received: from localhost ([127.0.0.1]:40535 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qOtkQ-0005YU-4a for submit@debbugs.gnu.org; Thu, 27 Jul 2023 01:41:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50334) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qOtkN-0005YE-Hr; Thu, 27 Jul 2023 01:41:12 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qOtkF-0000Sk-Ue; Thu, 27 Jul 2023 01:41:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=6M3e2VB8lMWVUjSKBa6gPNw2Gm/IWtM1+sK3CBH0R4w=; b=IH3PTrPOEPuB UoROcTfDj3a6/BmC+JOK8xNw1bkOV9CoBo3zwy1eJIss8oMOsFmxi9OQ9jjd06X0atKjo1ZBnAg2B ha6sXFgNLRDbiBUHJVkqy6btKRKzZjwnqlVFKkrlAq222Fb2rKtRJlqAKc57y+IhsHlLRBxCPWJgg WSw7QUuSwCobJMvmsPBrqSoYe0YJyomcXJi1hYvieI7NvWsGj5JzjQiYgkKyXdauuexN7r8n81lA1 NiJGVh0bqnLxz33hvUbK43hdu5b6xstrFbx5dOVRANGQ/+H91P0axnZ07pEOQ5OzpCj0/Mvx+jiCl Sxm6djTNMA1kjdvdy3DIiQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qOtkF-0000ef-Dw; Thu, 27 Jul 2023 01:41:03 -0400 Date: Thu, 27 Jul 2023 08:41:52 +0300 Message-Id: <83r0otnc67.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <19e73ab0-19b5-d7f4-8912-20c9e822e3fb@gutov.dev> (message from Dmitry Gutov on Thu, 27 Jul 2023 04:48:18 +0300) References: <87aas81jgh.fsf@jidanni.org> <83sf9h257b.fsf@gnu.org> <19e73ab0-19b5-d7f4-8912-20c9e822e3fb@gutov.dev> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Thu, 27 Jul 2023 04:48:18 +0300 > Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, > Stefan Monnier , jidanni@jidanni.org > From: Dmitry Gutov > > If some syscall or etc limits the length of a string to 4096, can't we > detect this case, split the string and emit said call multiple times? > > This function's docstring already mentions the case of > > If STRING is larger than the input buffer of the process, ... > it is sent in several bunches AFAIU, that is based on the errno value returned by a 'write' call which attempts to write too many bytes (see the would_block function). I guess writes to PTYs don't do that? Paul, do you know anything about that? From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Spencer Baugh Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 27 Jul 2023 14:01:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Eli Zaretskii Cc: 6149@debbugs.gnu.org, Dmitry Gutov , Paul Eggert , monnier@iro.umontreal.ca, jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.169046640220203 (code B ref 6149); Thu, 27 Jul 2023 14:01:02 +0000 Received: (at 6149) by debbugs.gnu.org; 27 Jul 2023 14:00:02 +0000 Received: from localhost ([127.0.0.1]:42751 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qP1X7-0005FK-Op for submit@debbugs.gnu.org; Thu, 27 Jul 2023 10:00:02 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:44839) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qP1X4-0005F6-R9 for 6149@debbugs.gnu.org; Thu, 27 Jul 2023 10:00:00 -0400 From: Spencer Baugh In-Reply-To: <83r0otnc67.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 27 Jul 2023 08:41:52 +0300") References: <87aas81jgh.fsf@jidanni.org> <83sf9h257b.fsf@gnu.org> <19e73ab0-19b5-d7f4-8912-20c9e822e3fb@gutov.dev> <83r0otnc67.fsf@gnu.org> Date: Thu, 27 Jul 2023 09:59:53 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> Date: Thu, 27 Jul 2023 04:48:18 +0300 >> Cc: 24531@debbugs.gnu.org, 6149@debbugs.gnu.org, >> Stefan Monnier , jidanni@jidanni.org >> From: Dmitry Gutov >> >> If some syscall or etc limits the length of a string to 4096, can't we >> detect this case, split the string and emit said call multiple times? >> >> This function's docstring already mentions the case of >> >> If STRING is larger than the input buffer of the process, ... >> it is sent in several bunches Alas it's far more cursed than that. The length of a *line* is limited to 4096 characters. So regardless of how big or small your buffers for writing are, if you write more than 4095 characters before writing a newline, the remaining characters will be discarded. There is no way to prevent this with ptys. So even if we wrote one character at a time, characters would start getting dropped after writing 4095 non-newline characters. > > AFAIU, that is based on the errno value returned by a 'write' call > which attempts to write too many bytes (see the would_block function). > I guess writes to PTYs don't do that? Writes to PTYs do tell us when the data has been truncated. There's just nothing we can do with that information. From unknown Fri Jun 13 10:47:08 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6149: bug#24531: process-send-string seems to truncate lines over 4096 characters Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 27 Jul 2023 14:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6149 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed To: Spencer Baugh , Eli Zaretskii Cc: 6149@debbugs.gnu.org, Dmitry Gutov , monnier@iro.umontreal.ca, jidanni@jidanni.org Received: via spool by 6149-submit@debbugs.gnu.org id=B6149.169046950625562 (code B ref 6149); Thu, 27 Jul 2023 14:52:02 +0000 Received: (at 6149) by debbugs.gnu.org; 27 Jul 2023 14:51:46 +0000 Received: from localhost ([127.0.0.1]:42851 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qP2L8-0006eA-OQ for submit@debbugs.gnu.org; Thu, 27 Jul 2023 10:51:45 -0400 Received: from mail.cs.ucla.edu ([131.179.128.66]:47242) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qP2L3-0006du-H6 for 6149@debbugs.gnu.org; Thu, 27 Jul 2023 10:51:41 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id F23483C011BD9; Thu, 27 Jul 2023 07:51:31 -0700 (PDT) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 0yw-nbW4GA9T; Thu, 27 Jul 2023 07:51:31 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id A73C63C011BDA; Thu, 27 Jul 2023 07:51:31 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu A73C63C011BDA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1690469491; bh=POTI/VGioPjY2O/ksnzrG2VhCjLxobvZdmrkrrLZ3GY=; h=Message-ID:Date:MIME-Version:To:From; b=BzAbibrf300Qe5Fk0GSXlS4Cvm6Wkf7z3G78ORPIgzX4SWw9ZxMIlXReIEGpTlbJ5 6vzkC9u8YTdeMSDxLZ5ESh8+gxD4M9PUYijWvlrSpjx/hzN3G/Xhi1HkJGsAHP/5dq TZuKTFx6sXt/RJjI8FFuV8RvcSb8Xw89f8wpa/xQ9ssHsXuC1l3XE7mefFGHkilIic tCbzCUO58Bm4awssS9uzieRFvV9Z1CsP6jCV1blk3bdNta+g5oEVlDPSm/3SKte+0Q ZGVkU2Us0Yxo8+5TTqAQsPfZvhc9zyr4mlWbng5N8ksLcMnMTP14pn+oKsurje8NpC 42QG5LikF1/LQ== X-Virus-Scanned: amavisd-new at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id hTym5FBOdbxg; Thu, 27 Jul 2023 07:51:31 -0700 (PDT) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 751FB3C011BD9; Thu, 27 Jul 2023 07:51:31 -0700 (PDT) Message-ID: Date: Thu, 27 Jul 2023 07:51:31 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Content-Language: en-US References: <87aas81jgh.fsf@jidanni.org> <83sf9h257b.fsf@gnu.org> <19e73ab0-19b5-d7f4-8912-20c9e822e3fb@gutov.dev> <83r0otnc67.fsf@gnu.org> From: Paul Eggert Organization: UCLA Computer Science Department In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.1 (-) On 2023-07-27 06:59, Spencer Baugh wrote: >> AFAIU, that is based on the errno value returned by a 'write' call >> which attempts to write too many bytes (see the would_block function). >> I guess writes to PTYs don't do that? > Writes to PTYs do tell us when the data has been truncated. Unfortunately not. Data bytes are silently truncated, at least on Ubuntu 23.04. If I fire up Emacs and type: M-x shell RET cat >out RET C-u 4096 x RET C-d the last RET causes Emacs to write 4097 bytes (4096 'x's followed by a newline) to the pty. This 'write' system call succeeds and returns 4097. However, the two 'read' calls that 'cat' executes see only 4095 'x's followed by '\n' ('read' returns 4096) followed by EOF ('read' returns 0). An 'x' was lost, and Emacs has no way to see this directly. This comes from the canonical mode of Linux's terminal driver, which silently discards non-newline bytes after the 4095th byte of an input line. See: https://github.com/torvalds/linux/blob/v6.4/drivers/tty/n_tty.c#L1648 One possibility is that Emacs could monitor writes to a Linux pty, looking for too many non-newline bytes in a row, and warn the user if that number exceeds 4095. That might be the best it can do in this troublesome environment. (The warning would be irrelevant for ttys operating in non-canonical mode, which have a different set of problems.)