From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 19 21:00:04 2021 Received: (at submit) by debbugs.gnu.org; 20 Aug 2021 01:00:04 +0000 Received: from localhost ([127.0.0.1]:60986 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGst9-0003F7-Uf for submit@debbugs.gnu.org; Thu, 19 Aug 2021 21:00:04 -0400 Received: from lists.gnu.org ([209.51.188.17]:48018) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGst4-0003EB-Nf for submit@debbugs.gnu.org; Thu, 19 Aug 2021 21:00:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49428) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGst4-00051A-FI for bug-grep@gnu.org; Thu, 19 Aug 2021 20:59:58 -0400 Received: from omta001.cacentral1.a.cloudfilter.net ([3.97.99.32]:37329) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGst2-0004Jj-Ip for bug-grep@gnu.org; Thu, 19 Aug 2021 20:59:58 -0400 Received: from shw-obgw-4004a.ext.cloudfilter.net ([10.228.9.227]) by cmsmtp with ESMTP id GnUMmqKDZFRDpGst0m6Nry; Fri, 20 Aug 2021 00:59:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=shaw.ca; s=s20180605; t=1629421194; bh=YCGdy0pnDlz791AoEHkxEdn/u56E+7NV/sYjiUG8dNY=; h=To:Subject:Date:From; b=HaKXVPVXw4+zpsnFNV/WSdrkY0qP5L7CRVhSxfp6qzu6H0BURJ//fmBkdUn8sOvdp i1v0XcD1ZHrEak6MdFkZnrVuhMGhU/B60vcNfaQlasqt/Hy/KE1QrSDw1riAHaOlTb hpmgjkKNoLOX/A+p4lTugVgvSDw+sdNbdvaOC6HS7+PbBLVP/0x2fnZnClrJ/Au8n2 DIgun9toClPJlAQWcyC4vlU7tu5FIX4ULSGKpj72n233nm6FQV8wALBlhl3UIcIk6V osJ54cIqDnmS/3w+9EbQpRz97hrQSY25LDlZvDDNgGtvS9DAF/MVqZ+M3jqQQhYwP3 1jX0l/NAETrUg== Received: from kylheku.com ([70.79.163.252]) by cmsmtp with ESMTPA id Gsszm6h0Z3DJAGsszm55Hv; Fri, 20 Aug 2021 00:59:54 +0000 Authentication-Results: ; auth=pass (PLAIN) smtp.auth=kkylheku@shaw.ca X-Authority-Analysis: v=2.4 cv=FMjee8ks c=1 sm=1 tr=0 ts=611efe8a a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17 a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=MhDmnRu9jo8A:10 a=t56ZrXdRcGisBULz4NgA:9 a=QEXdDO2ut3YA:10 Received: from www-data by kylheku.com with local (Exim 4.72) (envelope-from ) id 1mGssz-0008Fw-5D for bug-grep@gnu.org; Thu, 19 Aug 2021 17:59:53 -0700 To: bug-grep@gnu.org Subject: -f - option doesn't respond to single EOF from TTY. X-PHP-Originating-Script: 501:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Date: Thu, 19 Aug 2021 17:59:53 -0700 From: Kaz Kylheku Message-ID: <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> X-Sender: kaz@kylheku.com User-Agent: Roundcube Webmail/0.9.2 X-CMAE-Envelope: MS4xfPuZSubN9yNA2IMUWL/F2Nf0kiezgDvttt890mVXrojj3l9/a06it24gWZmaU0rVBongzZNgGB29CrPX8NuFNDlufvVYKgQ/7cNKCf4p9HnBmLOun3uw liUZA5R4yosj5pAu5nhWJ1oC+ynIUYiATIWAYC3437kelQ3au+Ybv3j9Ps10uInywia175EBRKMdjA== Received-SPF: pass client-ip=3.97.99.32; envelope-from=kkylheku@shaw.ca; helo=omta001.cacentral1.a.cloudfilter.net X-Spam_score_int: 1 X-Spam_score: 0.1 X-Spam_bar: / X-Spam_report: (0.1 / 5.0 requ) BAYES_20=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.1 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.1 (--) This reproduces on grep 3.1 on Ubuntu. The command: grep -f - should accept a list of patterns from standard input, like this: $ grep -f - pat1 pat2 pat3 [Ctrl-D] Upon receiving the EOF indication (zero byte read), the program should immediately conclude that the list of patterns has ended, and begin processing the input using the patterns. This does not seem to be working. After a single Ctrl-D, grep is still accumulating patterns: It appears that Ctrl-D must be issued twice: $ grep -f - pat1 [Ctrl-D] ;; effectively ignored pat2 ;; can add more patterns pat3 [Ctrl-D] [Ctrl-D] ;; OK, now we are in the matching loop. pat3blahblah pat3blahblah ... Cheers ... From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 19 22:11:00 2021 Received: (at submit) by debbugs.gnu.org; 20 Aug 2021 02:11:00 +0000 Received: from localhost ([127.0.0.1]:32796 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGtzn-0004uj-Pg for submit@debbugs.gnu.org; Thu, 19 Aug 2021 22:11:00 -0400 Received: from lists.gnu.org ([209.51.188.17]:39004) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGtzm-0004ua-2i for submit@debbugs.gnu.org; Thu, 19 Aug 2021 22:10:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58258) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGtzl-00049J-Rn for bug-grep@gnu.org; Thu, 19 Aug 2021 22:10:57 -0400 Received: from wforward1-smtp.messagingengine.com ([64.147.123.30]:46899) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mGtzj-0001nZ-Tr for bug-grep@gnu.org; Thu, 19 Aug 2021 22:10:57 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailforward.west.internal (Postfix) with ESMTP id 24BC21AC018A for ; Thu, 19 Aug 2021 22:10:53 -0400 (EDT) Received: from imap21 ([10.202.2.71]) by compute1.internal (MEProxy); Thu, 19 Aug 2021 22:10:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=XDumFx 0iSwDQJ7x9FEY32F7qTNBCnbhb2YA9eS003Qo=; b=Xc0m1zypEE4aDts2d2+cmt XSri8rugdgHyjXOfYoZ9cBekoBW+Dx6uoMvbWhAGNMad/whp7N7SzEuwL3VVp3GF QF1wkqpDWLWGVSXKIA04Kskgx5DAf9UMymt4v18/L8118U43ZdkaEvhhvTdyYnS3 8JKGkJ2ynFU28N/xj1O8sXHNU04hHVWpvPYsZbn0TJvRF7h/37lOPb+QazuXHRjv YLZNXUshMB+M4nWC0uQZCBA1s3SlSHlrXRllCeSu31aDxZCycdTaYGJQESsxDICi rVzDWXDgnXYv50er0NpqHFBdLOA4edOJvFxu4KIGrLjusm5aV8A15UD/vSrTPgmg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrleekgdehgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkjghffffhvffutgesthdtre dtreertdenucfhrhhomhepfdfrrghulhculfgrtghkshhonhdfuceophhjsehushgrrdhn vghtqeenucggtffrrghtthgvrhhnpefhvedtheejtefhgeejtdeigffftdelgfdvkeeufe ehjeekveevhedufffgudeljeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhep mhgrihhlfhhrohhmpehpjhesuhhsrgdrnhgvth X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 7250051C0060; Thu, 19 Aug 2021 22:10:52 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.5.0-alpha0-1118-g75eff666e5-fm-20210816.002-g75eff666 Mime-Version: 1.0 Message-Id: <6d15dbe7-0b12-4926-b369-e568a210583b@www.fastmail.com> In-Reply-To: <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> References: <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> Date: Thu, 19 Aug 2021 21:10:32 -0500 From: "Paul Jackson" To: bug-grep@gnu.org Subject: Re: bug#50129: -f - option doesn't respond to single EOF from TTY. Content-Type: text/plain Received-SPF: neutral client-ip=64.147.123.30; envelope-from=pj@usa.net; helo=wforward1-smtp.messagingengine.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_NEUTRAL=0.779 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.6 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.6 (--) My fading recollection from long, long ago is that this is a difficult to avoid artifact of using stdio. If no patterns are fed into grep, just a single EOF, then grep exits immediately. But some non-empty input (just a single "pat1" is sufficient) is fed to grep, then it takes two EOF's to get grep to exit. These two cases can be seen in the output of the following two commands: # With an input of "pat1", it takes two reads returning 0 bytes (EOF's) to exit. (sleep 2; echo pat1) | strace -tt -T grep -f - 2>&1 | tail -15 # With no "pat1" input, it only takes one read of 0 bytes to exit. sleep 2 | strace -tt -T grep -f - 2>&1 | tail -15 === === Here's a copy-paste of a terminal session in which I invoke the above two commands. Notice the read() system calls that return 0 bytes. That's how the kernel presents an EOF to a user process on a read. $ (sleep 2; echo pat1) | strace -tt -T grep -f - 2>&1 | tail -15 21:07:12.887450 openat(AT_FDCWD, "/usr/share/locale-langpack/en.utf8/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000013> 21:07:12.887490 openat(AT_FDCWD, "/usr/share/locale-langpack/en/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000013> 21:07:12.887535 rt_sigaction(SIGSEGV, {sa_handler=0x55e0d5c49570, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_NODEFER|SA_RESETHAND|SA_SIGINFO, sa_restorer=0x7f9b26cd3210}, NULL, 8) = 0 <0.000011> 21:07:12.887579 fstat(0, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 <0.000012> 21:07:12.887623 read(0, "pat1\n", 4096) = 5 <1.993486> 21:07:14.881203 read(0, "", 4096) = 0 <0.000020> 21:07:14.881269 fstat(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 <0.000019> 21:07:14.881420 brk(0x55e0d7954000) = 0x55e0d7954000 <0.000023> 21:07:14.881477 fstat(0, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 <0.000012> 21:07:14.881522 lseek(0, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) <0.000012> 21:07:14.881570 read(0, "", 98304) = 0 <0.000012> 21:07:14.881616 close(1) = 0 <0.000012> 21:07:14.881655 close(2) = 0 <0.000011> 21:07:14.881706 exit_group(1) = ? 21:07:14.881936 +++ exited with 1 +++ $ sleep 2 | strace -tt -T grep -f - 2>&1 | tail -15 21:07:25.407841 openat(AT_FDCWD, "/usr/share/locale/en/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000015> 21:07:25.407888 openat(AT_FDCWD, "/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000014> 21:07:25.407933 openat(AT_FDCWD, "/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000014> 21:07:25.407980 openat(AT_FDCWD, "/usr/share/locale-langpack/en_US/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000014> 21:07:25.408026 openat(AT_FDCWD, "/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000014> 21:07:25.408071 openat(AT_FDCWD, "/usr/share/locale-langpack/en.utf8/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000014> 21:07:25.408116 openat(AT_FDCWD, "/usr/share/locale-langpack/en/LC_MESSAGES/grep.mo", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000015> 21:07:25.408165 rt_sigaction(SIGSEGV, {sa_handler=0x55ac5b515570, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_NODEFER|SA_RESETHAND|SA_SIGINFO, sa_restorer=0x7f84afc54210}, NULL, 8) = 0 <0.000012> 21:07:25.408213 fstat(0, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 <0.000012> 21:07:25.408260 read(0, "", 4096) = 0 <1.992492> 21:07:27.400830 fstat(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0 <0.000019> 21:07:27.400921 close(1) = 0 <0.000018> 21:07:27.400979 close(2) = 0 <0.000019> 21:07:27.401046 exit_group(1) = ? 21:07:27.401259 +++ exited with 1 +++ -- Paul Jackson pj@usa.net From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 20 11:25:21 2021 Received: (at 50129-done) by debbugs.gnu.org; 20 Aug 2021 15:25:21 +0000 Received: from localhost ([127.0.0.1]:35355 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mH6OX-00033D-AZ for submit@debbugs.gnu.org; Fri, 20 Aug 2021 11:25:21 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:58376) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mH6OV-000330-F6 for 50129-done@debbugs.gnu.org; Fri, 20 Aug 2021 11:25:20 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 905C51600FB; Fri, 20 Aug 2021 08:25:13 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id EnOmgL_UqDig; Fri, 20 Aug 2021 08:25:13 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id E9244160101; Fri, 20 Aug 2021 08:25:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id tVNg4BWBtHsz; Fri, 20 Aug 2021 08:25:12 -0700 (PDT) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id C82E11600FB; Fri, 20 Aug 2021 08:25:12 -0700 (PDT) To: Kaz Kylheku References: <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> From: Paul Eggert Organization: UCLA Computer Science Department Subject: Re: bug#50129: -f - option doesn't respond to single EOF from TTY. Message-ID: <0ac56bc0-016a-2840-e877-ac9036137bf0@cs.ucla.edu> Date: Fri, 20 Aug 2021 08:25:12 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.4 (--) X-Debbugs-Envelope-To: 50129-done Cc: 50129-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.4 (---) On 8/19/21 5:59 PM, Kaz Kylheku wrote: > This reproduces on grep 3.1 on Ubuntu. That's a pretty old version of grep. I cannot reproduce the problem on=20 Ubuntu 21.04, which has grep 3.6. So it sounds like the bug is fixed,=20 and you can fix your problem by upgrading to a more-recent Ubuntu version= . (Not that anyone would ever want to *use* plain "grep -f -", except=20 perhaps to file bug reports....) From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 20 12:53:46 2021 Received: (at 50129) by debbugs.gnu.org; 20 Aug 2021 16:53:47 +0000 Received: from localhost ([127.0.0.1]:35463 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mH7m6-0001DN-OK for submit@debbugs.gnu.org; Fri, 20 Aug 2021 12:53:46 -0400 Received: from omta002.cacentral1.a.cloudfilter.net ([3.97.99.33]:52990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mH7m5-0001DA-Di for 50129@debbugs.gnu.org; Fri, 20 Aug 2021 12:53:46 -0400 Received: from shw-obgw-4004a.ext.cloudfilter.net ([10.228.9.227]) by cmsmtp with ESMTP id H2HEm3M4K4bInH7lzmA3uq; Fri, 20 Aug 2021 16:53:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=shaw.ca; s=s20180605; t=1629478419; bh=awuaRqoiQ/JwSJxmoXCB7Eqow0imMT2ktOCEK/I2mBA=; h=To:Subject:Date:From:In-Reply-To:References; b=sS988KN3fEVJpiDdILnHg48SXdy/a96ZoZJWHVvMMHzcxtAhznWZpWAHCq0Fm2EJn cPLY4RTh+G3wTJ2EkezGiTuRm4UfmPXxlWlzFGR6Lu62znpjbXgSgtxlKpRguGuKHI ykAu7hn+fSbi+bFPaLpYopfPfsaroYg7/oY3awvCiU+sqZydNMLa9w/H/4NGApPCsY jBvgxKlZmnWM2kmxkEw9pZL7XBs7howjclJ6eZtUbsDrGdu+MYBmw4ZQFG3OZdSTM1 gJRI/p8WOZ5kIVAAofu33AF7Cgo8HdgpXP+7bjDQkwB82X7/QigUURRMFSAzU01Zpa XhdddHtMnoiAA== Received: from kylheku.com ([70.79.163.252]) by cmsmtp with ESMTPA id H7lymCBPv3DJAH7lzm7HCs; Fri, 20 Aug 2021 16:53:39 +0000 Authentication-Results: ; auth=pass (PLAIN) smtp.auth=kkylheku@shaw.ca X-Authority-Analysis: v=2.4 cv=FMjee8ks c=1 sm=1 tr=0 ts=611fde13 a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17 a=yx6ycu90wq5mB401:21 a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=MhDmnRu9jo8A:10 a=oiOWpaCd7sYxOaFaf7cA:9 a=QEXdDO2ut3YA:10 a=WdUQyDgsvWDCkgkgqURO:22 Received: from www-data by kylheku.com with local (Exim 4.72) (envelope-from ) id 1mH7ly-0000H3-Np for 50129@debbugs.gnu.org; Fri, 20 Aug 2021 09:53:38 -0700 To: 50129@debbugs.gnu.org Subject: Re: bug#50129: closed (Re: bug#50129: -f - option doesn't respond to single EOF from TTY.) X-PHP-Originating-Script: 501:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Date: Fri, 20 Aug 2021 09:53:38 -0700 From: Kaz Kylheku In-Reply-To: References: <0ac56bc0-016a-2840-e877-ac9036137bf0@cs.ucla.edu> <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> Message-ID: <350c14c98c922f48bbd2c33bf0f67b9f@mail.kylheku.com> X-Sender: kaz@kylheku.com User-Agent: Roundcube Webmail/0.9.2 X-CMAE-Envelope: MS4xfAIDzlAFNMTq7ZVNOnecQJO8LhWlvfBeZ81CodHVJ0VSoH+sm+9LNaAB4PIO/+/6pv6AdRgiJHOdxeD8FwmvjhUaWW/iFOUyRxZYI2RWEOwmI3IRj2Hu xO39T8z0SaAocyg4hcQKwgf2oPS57rRI0OKWlEnI5aHGVqbAduCleSQCuAwUoktzS7BcphVFQFknVQ== X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 50129 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Please allow me to add a semi-final comment to a closed bug. Looking at grep's small body of code for handling the -f option, I don't see anything substantially different between 3.1 and 3.7. It's the same logic wrapped around the fread function. Though the code has been worked on, there is no difference in how the end of input is detected. From that perspective, this looks like it may in fact be a problem in the libc fread function. However, on the same system, I cannot reproduce the issue with a similar loop around fread, e.g. with this program: #include int main(void) { char buf[128]; setvbuf(stdin, NULL, _IOFBF, 0); while (fread(buf, sizeof buf, 1, stdin) !=3D 0); return 0; } Regardless of buffering mode, the fread loop promptly quits if Ctrl-D is given from the TTY on an empty line. The most plausible explanation at this point is that Debian/Ubuntu had applied some stinky patch to grep. If I have some time, I will drill into that, in which case I will post a final comment here, too. Cheers ... From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 20 13:25:19 2021 Received: (at 50129) by debbugs.gnu.org; 20 Aug 2021 17:25:19 +0000 Received: from localhost ([127.0.0.1]:35489 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mH8Gd-000209-0c for submit@debbugs.gnu.org; Fri, 20 Aug 2021 13:25:19 -0400 Received: from omta001.cacentral1.a.cloudfilter.net ([3.97.99.32]:35521) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mH8Ga-0001zu-Lf for 50129@debbugs.gnu.org; Fri, 20 Aug 2021 13:25:17 -0400 Received: from shw-obgw-4001a.ext.cloudfilter.net ([10.228.9.142]) by cmsmtp with ESMTP id H5fLmr4shFRDpH8GVm8MD1; Fri, 20 Aug 2021 17:25:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=shaw.ca; s=s20180605; t=1629480311; bh=lQpay+UgD+11UPkCrg2VjyWEPVUEYmV9C+4z/JP+tL8=; h=To:Subject:Date:From:In-Reply-To:References; b=H+dqXFp9fo0TBSb26Dl/1wj3fNbkwBBjHmomtsL9vpX4d9IO9508cH1VihQAkyidD Uz5rP+Fn1JEmnZ218f1gkxIzmnPAKjab+4Y59Oa5b/KwR6neLISlpLTpMvF3g7aQ8a qF/VbJbOODv5Odvk0PtbHjJEn33AGVKXB8CMJ+ZvdMuZY7lK4rnd+ZyDP0jAGj4eVh wifhMPgIIdQut7BELmvcFAAdG2Ui0hIKUb7z7bTntEyTbl+NNgSH3Za+cqGlN7dO4H 92OzRZpi3xlGhI79lV0WMJ1zK3mP/W9XV9A2Up1tWUdjpedT9rhGCd+ccH87JOW7Zk fvJMvcOLYZODw== Received: from kylheku.com ([70.79.163.252]) by cmsmtp with ESMTPA id H8GUmfbNQM8olH8GUm9Fr1; Fri, 20 Aug 2021 17:25:11 +0000 Authentication-Results: ; auth=pass (PLAIN) smtp.auth=kkylheku@shaw.ca X-Authority-Analysis: v=2.4 cv=Ua6U9IeN c=1 sm=1 tr=0 ts=611fe577 a=95A0EdhkF1LMGt25d7h1IQ==:117 a=95A0EdhkF1LMGt25d7h1IQ==:17 a=yx6ycu90wq5mB401:21 a=IkcTkHD0fZMA:10 a=SMorJkV_YP8A:10 a=MhDmnRu9jo8A:10 a=SDNVNFhLCPRW-TVzd40A:9 a=QEXdDO2ut3YA:10 a=WdUQyDgsvWDCkgkgqURO:22 Received: from www-data by kylheku.com with local (Exim 4.72) (envelope-from ) id 1mH8GU-0000aO-5k for 50129@debbugs.gnu.org; Fri, 20 Aug 2021 10:25:10 -0700 To: 50129@debbugs.gnu.org Subject: Re: bug#50129: closed (Re: bug#50129: -f - option doesn't respond to single EOF from TTY.) X-PHP-Originating-Script: 501:rcmail.php MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Date: Fri, 20 Aug 2021 10:25:10 -0700 From: Kaz Kylheku In-Reply-To: <350c14c98c922f48bbd2c33bf0f67b9f@mail.kylheku.com> References: <0ac56bc0-016a-2840-e877-ac9036137bf0@cs.ucla.edu> <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> <350c14c98c922f48bbd2c33bf0f67b9f@mail.kylheku.com> Message-ID: X-Sender: kaz@kylheku.com User-Agent: Roundcube Webmail/0.9.2 X-CMAE-Envelope: MS4xfJNajQGjTljxyDF8QfzbOj4jmT59yeYwcABzsASj9GklQeoGlYAP96keZM/eTTtTbf4watGZso+XHpWBPRHchlvb8vLlvrUAxf7FFR8axe6nT59w87Oh oDRIBTbW8lGeQeXxNM8BTT35rb3LQOkqyqJy+lrAa9QIWnnpeJpw+t8Rgcdx8jRnzNsk83pmfobLEw== X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: 50129 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Nope! It looks like a behavior of fread. We have to reverse the middle two arguments of fread, so that it's reading an n-sized array of 1-byte objects, rather than one object of size n. #include int main(void) { char buf[8192]; while (fread(buf, 1, sizeof buf, stdin) !=3D 0); // ^^^^^^^^^^^^^^ reversed these! return 0; } and now, a behavior closely resembling the issue is reproduced. You can see in the strace that two zero-sized reads are needed to bail out: $ strace ./fread read(0, abc "abc\n", 8192) =3D 4 read(0, "", 7168) =3D 0 read(0, "", 8192) =3D 0 exit_group(0) =3D ? +++ exited with 0 +++ what happens is that upon receiving the zero byte read, fread is returning the number of bytes it has accumulated to the application. The application calls fread again. This is expected, because fread didn't fail. In my original example, fread is called only once and returns zero, because it was told to read a 128 byte object, during which it received a short read, at which point it is game over. Still, I don't understand. The grep 3.1 and 3.7 code both have 1 as the second argument of fread, and the size as the third. This argument order is necessary, so that fread doesn't discard partial data at EOF. So there is something more there. I'd have to play with the code to see how the second argument is behaving under the buffer resize logic and whatnot. (E.g. if it happens that the second argument is now always 1, that would explain why the behavior is fixed.) From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 20 14:52:03 2021 Received: (at submit) by debbugs.gnu.org; 20 Aug 2021 18:52:03 +0000 Received: from localhost ([127.0.0.1]:35550 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mH9cV-0006Db-JV for submit@debbugs.gnu.org; Fri, 20 Aug 2021 14:52:03 -0400 Received: from lists.gnu.org ([209.51.188.17]:41040) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mH9cQ-0006DO-NR for submit@debbugs.gnu.org; Fri, 20 Aug 2021 14:51:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33660) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mH9cQ-0004fM-2J for bug-grep@gnu.org; Fri, 20 Aug 2021 14:51:54 -0400 Received: from wforward1-smtp.messagingengine.com ([64.147.123.30]:57215) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mH9cK-0008SQ-FX for bug-grep@gnu.org; Fri, 20 Aug 2021 14:51:53 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailforward.west.internal (Postfix) with ESMTP id 2B5EC1AC0327 for ; Fri, 20 Aug 2021 14:51:46 -0400 (EDT) Received: from imap21 ([10.202.2.71]) by compute1.internal (MEProxy); Fri, 20 Aug 2021 14:51:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=oiulc7 /0BgIxJeBmtgGjNqHXgFRAiFimRqRXGS60Y+o=; b=EUVkc/3K6BVUrludKcYYEG X6X9FCmLjUkQvpPzbVK833i3sV1/D/kP1P4PkpGyXtjGVRIMDK7Jg7/9prxZzNU+ 9J1Z4IzEY1BbIhA7IFYolriOXW2nS7OoA1konRUZKAxt00lJEzhZB1QXxmdWOrsd kN0a5yfZyTaDw127jVK01gSMXcC41SJnZh5wA+2wHClAcrkDCcumvPxBqm57egXQ U4G1xa1cOfo1RRibhAVlEvQfIzLsqTtyOMKSiS+JSpV+mCnzR40uNvXoyT4ypwwL sKllzkx6BCIRBFSfZZpr+pcqkYQ1KdUpUKmulAdq42RaW4MSUm5j9dNbEarx3frg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrleelgdduvdejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderreejnecuhfhrohhmpedfrfgruhhlucflrggtkhhsohhnfdcuoehpjhesuhhsrgdr nhgvtheqnecuggftrfgrthhtvghrnhepfeejudeuvddtteevgedvjeevjeefffehhefhle dtiefhhfethfevfefhueeijedtnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucev lhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpjhesuhhsrg drnhgvth X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 4047F51C0060; Fri, 20 Aug 2021 14:51:45 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.5.0-alpha0-1118-g75eff666e5-fm-20210816.002-g75eff666 Mime-Version: 1.0 Message-Id: In-Reply-To: References: <0ac56bc0-016a-2840-e877-ac9036137bf0@cs.ucla.edu> <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> <350c14c98c922f48bbd2c33bf0f67b9f@mail.kylheku.com> Date: Fri, 20 Aug 2021 13:51:06 -0500 From: "Paul Jackson" To: bug-grep@gnu.org Subject: =?UTF-8?Q?Re:_bug#50129:_closed_(Re:_bug#50129:_-f_-_option_doesn't_resp?= =?UTF-8?Q?ond_to_single_EOF_from_TTY.)?= Content-Type: text/plain Received-SPF: neutral client-ip=64.147.123.30; envelope-from=pj@usa.net; helo=wforward1-smtp.messagingengine.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_NEUTRAL=0.779 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.6 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.6 (--) I'd guess you're seeing some of the design decisions in stdio, that were rather "controversial", back around Version 7 Unix, inside Bell Labs, circa late 1978, when stdio was first introduced. I've avoided using fread(3S) "forever", even to the point of rolling my own i/o buffering library, the most recent incarnation of which can be found at https://github.com/ThePythonicCow/rawscan The fixed input buffer size of my 'rawscan' probably makes it unsuitable for most uses such as GNU grep that expect to handle arbitrarily long input lines. This 'rawscan' scans over and otherwise ignores the tail end of any input line that exceeds the line length you specified when opening the buffer stream. On a very large test case rather carefully chosen to highlight the performance of rawscan, it was over twice as fast searching for a fixed pattern than was grep. See the "Comparative Performance" section of rawscan's github page for these results: https://github.com/ThePythonicCow/rawscan -- Paul Jackson pj@usa.net From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 21 13:50:54 2021 Received: (at 50129) by debbugs.gnu.org; 21 Aug 2021 17:50:54 +0000 Received: from localhost ([127.0.0.1]:37838 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mHV8s-0000Va-Db for submit@debbugs.gnu.org; Sat, 21 Aug 2021 13:50:54 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:40622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mHV8n-0000VL-8p for 50129@debbugs.gnu.org; Sat, 21 Aug 2021 13:50:49 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 37223160060; Sat, 21 Aug 2021 10:50:39 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id Ltvw14FIqKPn; Sat, 21 Aug 2021 10:50:31 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 6AD17160078; Sat, 21 Aug 2021 10:50:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id jkM27_5ypX6M; Sat, 21 Aug 2021 10:50:31 -0700 (PDT) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 30207160060; Sat, 21 Aug 2021 10:50:31 -0700 (PDT) From: Paul Eggert To: Kaz Kylheku References: <11c094a6daf9b9c2fb983647b6e91584@mail.kylheku.com> <0ac56bc0-016a-2840-e877-ac9036137bf0@cs.ucla.edu> Organization: UCLA Computer Science Department Subject: Re: bug#50129: -f - option doesn't respond to single EOF from TTY. Message-ID: <7fbe8399-1a3f-6769-667f-998b68a2edbe@cs.ucla.edu> Date: Sat, 21 Aug 2021 10:50:30 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <0ac56bc0-016a-2840-e877-ac9036137bf0@cs.ucla.edu> Content-Type: multipart/mixed; boundary="------------2F3176566649754A91569420" Content-Language: en-US X-Spam-Score: -2.4 (--) X-Debbugs-Envelope-To: 50129 Cc: 50129@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.4 (---) This is a multi-part message in MIME format. --------------2F3176566649754A91569420 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 8/20/21 8:25 AM, Paul Eggert wrote: > (Not that anyone would ever want to *use* plain "grep -f -", except=20 > perhaps to file bug reports....) I discovered a more-artificial case where grep messes up even on modern=20 platforms, namely 'grep -f - -f -' where grep essentially ignores the=20 second '-f -'. I installed the attached to fix that, along with another=20 bug where grep wasn't reporting fclose errors. I think these bugs are so unlikely and artificial that they're not worth=20 mentioning in NEWS. As a side effect, this patch might fix the bug#50129 problem on your old=20 platform. Hard to say. --------------2F3176566649754A91569420 Content-Type: text/x-patch; charset=UTF-8; name="0001-grep-avoid-sticky-problem-with-f-f.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0001-grep-avoid-sticky-problem-with-f-f.patch" =46rom 5649624925c421427cd4ec0f8a7e332c94532b33 Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Sat, 21 Aug 2021 10:44:17 -0700 Subject: [PATCH] =3D?UTF-8?q?grep:=3D20avoid=3D20sticky=3D20problem=3D20w= ith=3D20?=3D =3D?UTF-8?q?=3DE2=3D80=3D98-f=3D20-=3D20-f=3D20-=3DE2=3D80=3D99?=3D MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit Inspired by bug#50129 even though this is a different bug. * src/grep.c (main): For =E2=80=98-f -=E2=80=99, use clearerr (stdin) aft= er reading, so that =E2=80=98grep -f - -f -=E2=80=99 reads stdin twice even when stdin is a tty. Also, for =E2=80=98-f FILE=E2=80=99, report any I/O error when closing FILE. --- src/grep.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/src/grep.c b/src/grep.c index 7a33686..b2a0566 100644 --- a/src/grep.c +++ b/src/grep.c @@ -2477,7 +2477,6 @@ main (int argc, char **argv) int matcher =3D -1; int opt; int prev_optind, last_recursive; - int fread_errno; intmax_t default_context; FILE *fp; exit_failure =3D EXIT_TROUBLE; @@ -2648,11 +2647,17 @@ main (int argc, char **argv) if (cc =3D=3D 0) break; } - fread_errno =3D errno; - if (ferror (fp)) - die (EXIT_TROUBLE, fread_errno, "%s", optarg); - if (fp !=3D stdin) - fclose (fp); + int err =3D errno; + if (!ferror (fp)) + { + err =3D 0; + if (fp =3D=3D stdin) + clearerr (fp); + else if (fclose (fp) !=3D 0) + err =3D errno; + } + if (err) + die (EXIT_TROUBLE, err, "%s", optarg); /* Append final newline if file ended in non-newline. */ if (newkeycc !=3D keycc && keys[newkeycc - 1] !=3D '\n') keys[newkeycc++] =3D '\n'; --=20 2.30.2 --------------2F3176566649754A91569420-- From unknown Sat Jun 21 10:40:56 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sun, 19 Sep 2021 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator