From unknown Mon Aug 18 17:54:04 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions Resent-From: Fulvio Scapin Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Tue, 08 Mar 2016 16:33:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 22945 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: 22945@debbugs.gnu.org X-Debbugs-Original-To: bug-gzip@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.145745478310392 (code B ref -1); Tue, 08 Mar 2016 16:33:03 +0000 Received: (at submit) by debbugs.gnu.org; 8 Mar 2016 16:33:03 +0000 Received: from localhost ([127.0.0.1]:39909 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1adKZC-0002hR-KB for submit@debbugs.gnu.org; Tue, 08 Mar 2016 11:33:02 -0500 Received: from eggs.gnu.org ([208.118.235.92]:36760) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1adG2C-0000wW-Iz for submit@debbugs.gnu.org; Tue, 08 Mar 2016 06:42:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1adG26-0007Ep-AW for submit@debbugs.gnu.org; Tue, 08 Mar 2016 06:42:35 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:59223) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adG26-0007El-6z for submit@debbugs.gnu.org; Tue, 08 Mar 2016 06:42:34 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55949) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adG25-0004on-2o for bug-gzip@gnu.org; Tue, 08 Mar 2016 06:42:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1adG23-0007Dh-Vp for bug-gzip@gnu.org; Tue, 08 Mar 2016 06:42:32 -0500 Received: from mail-vk0-x230.google.com ([2607:f8b0:400c:c05::230]:35977) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adG23-0007Dd-QD for bug-gzip@gnu.org; Tue, 08 Mar 2016 06:42:31 -0500 Received: by mail-vk0-x230.google.com with SMTP id c3so13000601vkb.3 for ; Tue, 08 Mar 2016 03:42:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to; bh=qJAY0Rk7H61VF0X0bSTqFUISQkE7Be4jfj8hlwn3oXc=; b=LP8eqA5/2Xe5/bVQD4mvrG7d5JzaYMSTTtq0+gS4+Z+HbZzEoPBYulzXM/lmUVJEke EjU1pBzp4t+8aiDzi8N+SIfC6WN4Ptg1BpsyAblGNJpTcJ/uueDtCzsbSIBtzznon+xN Sg1C2mj00GfEg9iFQ344xAlr2cxTUoq1jZdkHL+znb6lrJPVvibo1Cfl8Cg7s9mU+6qF 3Ympx1LgFkjMmqy7KzMkQMXumcXyVL8Ipx5Ed5InN8WmZKcfZOtICDbb3jqF01ArC1Yu d5aoORNdPflsTCDWlwiAX0z/vfiXcwwPtt+PKqVKnfbs9pMeqH5Gsp+PwvkA3ZEIBR9W ZrVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to; bh=qJAY0Rk7H61VF0X0bSTqFUISQkE7Be4jfj8hlwn3oXc=; b=iOajac7EqUbdSVWjmxIrloh2z9MFWgbUwHsv+swCm/HEMaLQDoAmJPwbUzmoH5MbeA 6rmUgvEsrIwwYospEY6agd/uWMLjCzQ4MJaPfGeFVHRDR2Iwb+QHH2RefRYfpig9Ak44 yXCpy4dOFbrnhwQfEvbJtDmrtQ/SvT4pwDRif0grePDnkKCgC8vYXynD39pZoeRP23qE yFhcrAep5kze5MbGvF/pQ/zRPm8bkKXYtdGEeNKimY8ih46Rr+Sfj720uU2hcpCMD7jd cJ92N5JsXA5CDnHXesP0qmfwnuJTY1ZCVa13+wJJLJ1xT5VU8kZgJDg1nn5CutCFuYKE BI0A== X-Gm-Message-State: AD7BkJKbYo4cnB39x583GCfkVA78chOW5sxmJEqVfNIhZIKaiUU4OHN3ASK3jb/UXwyyVeZnkk4cYF/BivvJyA== MIME-Version: 1.0 X-Received: by 10.31.180.85 with SMTP id d82mr21778534vkf.83.1457437350961; Tue, 08 Mar 2016 03:42:30 -0800 (PST) Received: by 10.103.113.197 with HTTP; Tue, 8 Mar 2016 03:42:30 -0800 (PST) Date: Tue, 8 Mar 2016 12:42:30 +0100 Message-ID: From: Fulvio Scapin Content-Type: multipart/alternative; boundary=001a114382ce962d53052d881352 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Mailman-Approved-At: Tue, 08 Mar 2016 11:33:02 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) --001a114382ce962d53052d881352 Content-Type: text/plain; charset=UTF-8 Hello. There is a problem with zgrep whenever the -f option actually reads from the output of a process substition in bash. A willingly trivial example below. $ mkdir /tmp/test $ cd /tmp/test $ cat > first aaa $ cat > second bbb $ cat > third ccc $ cat > fourth ddd $ tail * ==> first <== aaa ==> fourth <== ddd ==> second <== bbb ==> third <== ccc $ gzip -9 * $ ls first.gz fourth.gz second.gz third.gz $ cat > patterns aaa bbb ccc ddd $ tail patterns aaa bbb ccc ddd $ zfgrep -f <( cat patterns ) first.gz fourth.gz second.gz third.gz first.gz:aaa $ zfgrep -f patterns first.gz fourth.gz second.gz third.gz first.gz:aaa fourth.gz:ddd second.gz:bbb third.gz:ccc zfgrep -f <( cat patterns ) first.gz fourth.gz second.gz third.gz translates in zfgrep -f /dev/fd/XX first.gz fourth.gz second.gz third.gz where XX is a number, 63 for instance . The problem, from what I understand, arises since zgrep -f patternfile a.gz b.gz c.gz actually is a succession of gzip -dc a.gz | grep -f patternfile gzip -dc b.gz | grep -f patternfile gzip -dc c.gz | grep -f patternfile Since patternfile in this case is /dev/fd/XX, only the first invocation of grep in the first pipeline actually reads a pattern list, while the second and third invocation get nothing, giving no match for b.gz and c.gz as a result. >From /bin/zgrep (Version 1.6, Ubuntu 15.10) one can read (-f | --file) # The pattern is coming from a file rather than the command-line. # If the file is actually stdin then we need to do a little # magic, since we use stdin to pass the gzip output to grep. # Turn the -f option into an -e option by copying the file's # contents into OPTARG. case $optarg in (" '-'" | " '/dev/stdin'" | " '/dev/fd/0'") option=-e optarg=" '"$(sed "$escape") || exit 2;; esac have_pat=1;; The workaround concerning stdin should (maybe) also apply to situations such as the one in my example? Thanks in advance. Fulvio Scapin --001a114382ce962d53052d881352 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hello.

<= div>There is a problem with zgre= p whenever the -f option actually reads from the output of a process substi= tion in bash.
A= willingly trivial example below.

$ mkdir = /tmp/test=C2=A0

$ cd /tmp/test=C2=A0

$ cat > first
aaa

$ cat &g= t; second
bbb

$ cat &g= t; third
ccc

$ cat &g= t; fourth
ddd

$ tail *
=3D=3D> first <=3D=3D
aaa

=3D=3D> fourth <=3D=3D
ddd

=3D=3D> second <=3D=3D
bbb

=3D=3D> third <=3D=3D
ccc

$ gzip -9 *

$ ls =C2=A0=
first.gz =C2=A0fourth.gz =C2=A0second.gz =C2=A0third.gz

$ cat > = patterns
aaa
bbb
ccc
ddd

$ tail patt= erns
aaa
bbb
ccc
ddd

$ zfgrep -f= <( cat patterns ) first.gz fourth.gz second.gz third.gz =C2=A0
first= .gz:aaa

$ zfgrep -f= patterns first.gz fourth.gz second.gz third.gz
first.gz:aaa
fourth.gz:ddd
second.gz:bbb
third.gz:ccc



zfgrep -f <( cat patterns ) first.gz fourth.gz second.gz thir= d.gz
translates in
zfgrep -f /dev/fd/XX first.gz fourth.gz sec= ond.gz third.gz
where XX is a= number, 63 for instance .

The problem, from what I understan= d, arises since

=
zgrep -f patternfile a.gz b.gz c.gz

actually is a succession of

gzip -dc a.gz | grep -= f patternfile
gzip -= dc b.gz | grep -f patternfile
gzip -dc c.gz | grep -f patte= rnfile


Since patternfile in this case is /dev/fd/XX, only the first invocati= on of grep in the first pipeline actually reads a pattern list, while the s= econd and third invocation get nothing, giving no match for b.gz and c.gz a= s a result.
=C2=A0

From /bin/zgrep (Version 1.6, Ubuntu= 15.10) one can read

=C2=A0 (-f | --file)
=C2=A0=C2=A0=C2=A0# The pattern is coming from a file rather th= an the command-line.
=C2=A0=C2=A0=C2=A0# If the file is actually stdin then we need to do a= little
=C2=A0=C2=A0=C2=A0# magic, since we use stdin to pass the gzip output = to grep.
=C2=A0=C2=A0=C2=A0# Turn the -f option into an -e option by copying th= e file's
=C2=A0=C2=A0=C2=A0# contents into OPTARG.
=C2=A0=C2=A0=C2=A0case $optarg in
=C2=A0=C2=A0=C2=A0(" '-'" | " '/dev/stdin&#= 39;" | " '/dev/fd/0'")
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0option=3D-e
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0optarg=3D" '"$(sed "$= escape") || exit 2;;
=C2=A0=C2=A0=C2=A0esac
=C2=A0=C2=A0=C2=A0have_pat=3D1;;

The workaround = concerning stdin should (maybe) also apply to situations such as the one in= my example?

Thanks in advance.

<= span style=3D"font-family:monospace">Fulvio Scapin

--001a114382ce962d53052d881352-- From unknown Mon Aug 18 17:54:04 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions Resent-From: Jim Meyering Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Wed, 16 Mar 2016 02:36:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22945 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: Fulvio Scapin Cc: 22945@debbugs.gnu.org Received: via spool by 22945-submit@debbugs.gnu.org id=B22945.145809571726144 (code B ref 22945); Wed, 16 Mar 2016 02:36:01 +0000 Received: (at 22945) by debbugs.gnu.org; 16 Mar 2016 02:35:17 +0000 Received: from localhost ([127.0.0.1]:49591 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ag1Iq-0006nc-PM for submit@debbugs.gnu.org; Tue, 15 Mar 2016 22:35:16 -0400 Received: from mail-oi0-f47.google.com ([209.85.218.47]:33066) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ag1Io-0006nP-Q2 for 22945@debbugs.gnu.org; Tue, 15 Mar 2016 22:35:15 -0400 Received: by mail-oi0-f47.google.com with SMTP id d205so28492254oia.0 for <22945@debbugs.gnu.org>; Tue, 15 Mar 2016 19:35:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Qm7AgBAfvYUuFO4oCm0c6K5QasmDWtK2nG6AJAGGJ1w=; b=jKQQPTVjfiH5GRepGATCgZWN/QjxrKvjJG202dEBqrSG52J3iPUj/ZVqX/Ij/K3tK5 1Zc1fP4jn+fT3Y98eBCaRYkBnVXJtFSgHmjogizqYE3XktimcsoYaVuZ3UH16m9JR253 aOxTTdiMTh2/9dirCB6LCXmfx0GgVyPoDxvgDmhaPMJKsvtQK9s+wUYlfs4J1cYe/Yh0 9qvJtCqd3DtDoDR+xR7bKjjOpeCkyc4e1fPL+zt/UBXRqSDydUzR8uun+SvkCgq/c9A0 A/aJqM7aTdk/tGQ0g8ZjfGCSYkZ1zue5R2zSXu9Won20fgOtk9MANumuQTxdzmjvfMuk JMQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Qm7AgBAfvYUuFO4oCm0c6K5QasmDWtK2nG6AJAGGJ1w=; b=Ja2DvUgZrnyr9Lp+JEZkN1x3HfnSwaBXMGMUYZahRFFQYz+iQ1gRmuLLnqBqsgexY6 /xy+EAgHEZcAekihbETio0ZZLdeNh4e2O4zfMZYMnx+wwyERpA9QPJurhSmd6MlDg2Ij /F0HcKI6AjGB/2xjlkDvVDUMzH10dy0QAyy2SJAsDmqrX4by/zl8New89TDJ51P+qisR 3noSSv54WkAaSP2NfEt+5e8doFQ/X+J2PmhR2fkaz1jT/EA0zFJ0ztMEFmwyhnVLnjHO m03Q7expYhKoJI/V3B85+Ip81DiGP6eUGz+Q0JfYvbyHNFeH1uCspGVAAC6tcwHKukre pKbw== X-Gm-Message-State: AD7BkJLGlocgbDod0Jip/FVm/3tLdLxQ5J70nr6mixzyoApnvQbX6HrMT2VdTUQrSVMsxMX4aMb1M7igF5kSsA== X-Received: by 10.202.218.133 with SMTP id r127mr818918oig.36.1458095709265; Tue, 15 Mar 2016 19:35:09 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.44.5 with HTTP; Tue, 15 Mar 2016 19:34:49 -0700 (PDT) In-Reply-To: References: From: Jim Meyering Date: Tue, 15 Mar 2016 19:34:49 -0700 X-Google-Sender-Auth: d5bGDuWArOZWZ618sMH7PIZI_IU Message-ID: Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -0.4 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.4 (/) On Tue, Mar 8, 2016 at 3:42 AM, Fulvio Scapin wrote: > Hello. > > There is a problem with zgrep whenever the -f option actually reads from > the output of a process substition in bash. > A willingly trivial example below. > > $ mkdir /tmp/test ... >>From /bin/zgrep (Version 1.6, Ubuntu 15.10) one can read Thank you for the report. To summarize, with zgrep-1.6, this erroneously prints matches only from the first file: $ zgrep -f <(echo .) <(echo a) <(echo b) /dev/fd/12:a However, with the latest from git (and soon to be gzip-1.7), this now works as desired: $ zgrep -f <(echo .) <(echo a) <(echo b) /dev/fd/12:a /dev/fd/13:b I see there is no NEWS entry for this fix and haven't yet identified the origin of the bug or the commit that fixed it, but will do so. From unknown Mon Aug 18 17:54:04 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Wed, 16 Mar 2016 21:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22945 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: Jim Meyering , Fulvio Scapin Cc: 22945@debbugs.gnu.org Received: via spool by 22945-submit@debbugs.gnu.org id=B22945.145816238112380 (code B ref 22945); Wed, 16 Mar 2016 21:07:01 +0000 Received: (at 22945) by debbugs.gnu.org; 16 Mar 2016 21:06:21 +0000 Received: from localhost ([127.0.0.1]:50480 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1agIe5-0003Db-8O for submit@debbugs.gnu.org; Wed, 16 Mar 2016 17:06:21 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:57035) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1agIe3-0003DO-AT for 22945@debbugs.gnu.org; Wed, 16 Mar 2016 17:06:19 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 7672C160E65; Wed, 16 Mar 2016 14:06:13 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id EpZP7iU1evmS; Wed, 16 Mar 2016 14:06:12 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 70E08160E84; Wed, 16 Mar 2016 14:06:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id c25kRakSop8u; Wed, 16 Mar 2016 14:06:12 -0700 (PDT) Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 56152160E65; Wed, 16 Mar 2016 14:06:12 -0700 (PDT) References: From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <56E9CAC4.3070708@cs.ucla.edu> Date: Wed, 16 Mar 2016 14:06:12 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) On 03/15/2016 07:34 PM, Jim Meyering wrote: > Thank you for the report. > To summarize, with zgrep-1.6, this erroneously prints matches only > from the first file: > > $ zgrep -f <(echo .) <(echo a) <(echo b) > /dev/fd/12:a > > However, with the latest from git (and soon to be gzip-1.7), this now > works as desired: > > $ zgrep -f <(echo .) <(echo a) <(echo b) > /dev/fd/12:a > /dev/fd/13:b > > I see there is no NEWS entry for this fix and haven't yet identified > the origin of the bug or the commit that fixed it, but will do so. Draft gzip 1.7 doesn't work for me (Fedora 23 x86-64). I have worked on a patch but don't have a reliable fix yet, or even a portable test case to illustrate the bug. Perhaps we should just think of it as a known bug for now. From unknown Mon Aug 18 17:54:04 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions Resent-From: Antonio Diaz Diaz Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Thu, 17 Mar 2016 16:33:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22945 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: 22945@debbugs.gnu.org Cc: Fulvio Scapin Received: via spool by 22945-submit@debbugs.gnu.org id=B22945.145823233916787 (code B ref 22945); Thu, 17 Mar 2016 16:33:02 +0000 Received: (at 22945) by debbugs.gnu.org; 17 Mar 2016 16:32:19 +0000 Received: from localhost ([127.0.0.1]:51224 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1agaqR-0004Mh-Jl for submit@debbugs.gnu.org; Thu, 17 Mar 2016 12:32:19 -0400 Received: from eggs.gnu.org ([208.118.235.92]:57312) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1agaqQ-0004MV-5Z for 22945@debbugs.gnu.org; Thu, 17 Mar 2016 12:32:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1agaqE-0006JH-El for 22945@debbugs.gnu.org; Thu, 17 Mar 2016 12:32:12 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34184) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1agaqE-0006JD-BF; Thu, 17 Mar 2016 12:32:06 -0400 Received: from 176.red-217-124-67.dialup.dynamic.ccgg.telefonica.net ([217.124.67.176]:37498) by fencepost.gnu.org with esmtpa (Exim 4.82) (envelope-from ) id 1agaqD-0008OF-B9; Thu, 17 Mar 2016 12:32:06 -0400 Message-ID: <56EADDBC.5030701@gnu.org> Date: Thu, 17 Mar 2016 17:39:24 +0100 From: Antonio Diaz Diaz User-Agent: Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.8.1.4) Gecko/20070601 SeaMonkey/1.1.2 MIME-Version: 1.0 References: <56E9CAC4.3070708@cs.ucla.edu> In-Reply-To: <56E9CAC4.3070708@cs.ucla.edu> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Paul Eggert wrote: > Draft gzip 1.7 doesn't work for me (Fedora 23 x86-64). I have worked on > a patch but don't have a reliable fix yet, or even a portable test case > to illustrate the bug. Perhaps we should just think of it as a known bug > for now. What about using command substitution with '-e' instead of process substitution with '-f'? zgrep -e "$(cat FILE)" file1.lz file2.gz Best regards, Antonio. From unknown Mon Aug 18 17:54:04 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Thu, 17 Mar 2016 20:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22945 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: Jim Meyering , Fulvio Scapin Cc: 22945@debbugs.gnu.org Received: via spool by 22945-submit@debbugs.gnu.org id=B22945.145824562517077 (code B ref 22945); Thu, 17 Mar 2016 20:14:01 +0000 Received: (at 22945) by debbugs.gnu.org; 17 Mar 2016 20:13:45 +0000 Received: from localhost ([127.0.0.1]:51481 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ageIi-0004RN-Q0 for submit@debbugs.gnu.org; Thu, 17 Mar 2016 16:13:44 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:45537) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ageIg-0004R9-Lw for 22945@debbugs.gnu.org; Thu, 17 Mar 2016 16:13:43 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id BC258160255; Thu, 17 Mar 2016 13:13:35 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id kf27DaqXfQYH; Thu, 17 Mar 2016 13:13:34 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id E1918160E82; Thu, 17 Mar 2016 13:13:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id veaeP8Uol_g9; Thu, 17 Mar 2016 13:13:34 -0700 (PDT) Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id C4F73160255; Thu, 17 Mar 2016 13:13:34 -0700 (PDT) References: <56E9CAC4.3070708@cs.ucla.edu> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <56EB0FEE.4010302@cs.ucla.edu> Date: Thu, 17 Mar 2016 13:13:34 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <56E9CAC4.3070708@cs.ucla.edu> Content-Type: multipart/mixed; boundary="------------030709000809080605070200" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) This is a multi-part message in MIME format. --------------030709000809080605070200 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 03/16/2016 02:06 PM, Paul Eggert wrote: > I have worked on a patch but don't have a reliable fix yet, or even a > portable test case to illustrate the bug. On further thought I found a test case and a fix, which I've attached. Normally I would just install this, but we're so close to a release that I'll wait for a word from Jim. --------------030709000809080605070200 Content-Type: application/x-patch; name="0001-zgrep-with-f-SPECIAL-read-SPECIAL-just-once.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="0001-zgrep-with-f-SPECIAL-read-SPECIAL-just-once.patch" RnJvbSAxYjM2ZTUxNDQ2ZmUwODQ2Mjk5YTNmODA0YTBiNzRjMjQwODcxYzViIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBUaHUsIDE3IE1hciAyMDE2IDEzOjA4OjA2IC0wNzAwClN1YmplY3Q6IFtQQVRD SF0gemdyZXA6IHdpdGggLWYgU1BFQ0lBTCwgcmVhZCBTUEVDSUFMIGp1c3Qgb25jZQoKUHJv YmxlbSByZXBvcnRlZCBieSBGdWx2aW8gU2NhcGluIGluOiBodHRwOi8vYnVncy5nbnUub3Jn LzIyOTQ1CiogTkVXUzogRG9jdW1lbnQgdGhpcy4KKiB0ZXN0cy96Z3JlcC1mOiBBZGQgYSB0 ZXN0LgoqIHpncmVwLmluICh3aXRoX2ZpbGVuYW1lKTogV2l0aCAtZiBGSUxFLCBpZiBGSUxF IGlzIG5vdAphIHJlZ3VsYXIgZmlsZSwgcmVhZCBpdCBoZXJlIGFuZCB0dXJuIGl0IGludG8g YW4gLWUgb3B0aW9uLgotLS0KIE5FV1MgICAgICAgICAgfCAzICsrKwogdGVzdHMvemdyZXAt ZiB8IDggKysrKysrKysKIHpncmVwLmluICAgICAgfCA2ICsrKysrKwogMyBmaWxlcyBjaGFu Z2VkLCAxNyBpbnNlcnRpb25zKCspCgpkaWZmIC0tZ2l0IGEvTkVXUyBiL05FV1MKaW5kZXgg NjM2M2Q3MS4uNTQxYWQ5NCAxMDA2NDQKLS0tIGEvTkVXUworKysgYi9ORVdTCkBAIC0zNSw2 ICszNSw5IEBAIEdOVSBnemlwIE5FV1MgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAtKi0gb3V0bGluZSAtKi0KICAgZ3ppcCAtayAtdiBubyBsb25nZXIgcmVwb3J0cyB0 aGF0IGZpbGVzIGFyZSByZXBsYWNlZC4KICAgW2J1ZyBwcmVzZW50IHNpbmNlIHRoZSBiZWdp bm5pbmddCiAKKyAgemdyZXAgLWYgQSBCIEMgbm8gbG9uZ2VyIHJlYWRzIEEgbW9yZSB0aGFu IG9uY2UgaWYgQSBpcyBub3QgYSByZWd1bGFyIGZpbGUuCisgIFRoaXMgYmV0dGVyIHN1cHBv cnRzIGludm9jYXRpb25zIGxpa2UgJ3pncmVwIC1mIDwoQ09NTUFORCkgQiBDJyBpbiBCYXNo LgorICBbYnVnIGludHJvZHVjZWQgaW4gZ3ppcC0xLjJdCiAKICogTm90ZXdvcnRoeSBjaGFu Z2VzIGluIHJlbGVhc2UgMS42ICgyMDEzLTA2LTA5KSBbc3RhYmxlXQogCmRpZmYgLS1naXQg YS90ZXN0cy96Z3JlcC1mIGIvdGVzdHMvemdyZXAtZgppbmRleCBhOGViNzQ2Li45YTg2NTUw IDEwMDc1NQotLS0gYS90ZXN0cy96Z3JlcC1mCisrKyBiL3Rlc3RzL3pncmVwLWYKQEAgLTI5 LDYgKzI5LDE0IEBAIHpncmVwIC1mIC0gaGF5c3RhY2suZ3ogPCBuID4gb3V0IDI+JjEgfHwg ZmFpbD0xCiAKIGNvbXBhcmUgb3V0IG4gfHwgZmFpbD0xCiAKK2lmICR7QkFTSF9WRVJTSU9O Kzp9IGZhbHNlOyB0aGVuCisgIHNldCArbyBwb3NpeAorICAjIFRoaXMgZmFpbGVkIHdpdGgg Z3ppcCAxLjYuCisgIGNhdCBuIG4gPm5uIHx8IGZyYW1ld29ya19mYWlsdXJlXworICBldmFs ICd6Z3JlcCAtaCAtZiA8KGNhdCBuKSBoYXlzdGFjay5neiBoYXlzdGFjay5neicgPm91dCB8 fCBmYWlsPTEKKyAgY29tcGFyZSBvdXQgbm4gfHwgZmFpbD0xCitmaQorCiAjIFRoaXMgZmFp bGVkIHdpdGggZ3ppcCAxLjQuCiBlY2hvIGEtYiB8IHpncmVwIC1lIC0gPiAvZGV2L251bGwg fHwgZmFpbD0xCiAKZGlmZiAtLWdpdCBhL3pncmVwLmluIGIvemdyZXAuaW4KaW5kZXggYzI0 YmU1Ny4uZGEwMzg3MSAxMDA2NDQKLS0tIGEvemdyZXAuaW4KKysrIGIvemdyZXAuaW4KQEAg LTExMywxMiArMTEzLDE4IEBAIHdoaWxlIHRlc3QgJCMgLW5lIDA7IGRvCiAgICAgIyBUaGUg cGF0dGVybiBpcyBjb21pbmcgZnJvbSBhIGZpbGUgcmF0aGVyIHRoYW4gdGhlIGNvbW1hbmQt bGluZS4KICAgICAjIElmIHRoZSBmaWxlIGlzIGFjdHVhbGx5IHN0ZGluIHRoZW4gd2UgbmVl ZCB0byBkbyBhIGxpdHRsZQogICAgICMgbWFnaWMsIHNpbmNlIHdlIHVzZSBzdGRpbiB0byBw YXNzIHRoZSBnemlwIG91dHB1dCB0byBncmVwLgorICAgICMgU2ltaWxhcmx5IGlmIGl0IGlz IG5vdCBhIHJlZ3VsYXIgZmlsZSwgc2luY2UgaXQgbWlnaHQgYmUgcmVhZCByZXBlYXRlZGx5 LgogICAgICMgVHVybiB0aGUgLWYgb3B0aW9uIGludG8gYW4gLWUgb3B0aW9uIGJ5IGNvcHlp bmcgdGhlIGZpbGUncwogICAgICMgY29udGVudHMgaW50byBPUFRBUkcuCiAgICAgY2FzZSAk b3B0YXJnIGluCiAgICAgKCIgJy0nIiB8ICIgJy9kZXYvc3RkaW4nIiB8ICIgJy9kZXYvZmQv MCciKQogICAgICAgb3B0aW9uPS1lCiAgICAgICBvcHRhcmc9IiAnIiQoc2VkICIkZXNjYXBl IikgfHwgZXhpdCAyOzsKKyAgICAoKikKKyAgICAgIGlmIGV2YWwgInRlc3QgISAtZiAkb3B0 YXJnIjsgdGhlbgorICAgICAgICBvcHRpb249LWUKKyAgICAgICAgb3B0YXJnPSIgJyIkKGV2 YWwgJ3NlZCAiJGVzY2FwZSIgPCciJG9wdGFyZyIpIHx8IGV4aXQgMgorICAgICAgZmk7Owog ICAgIGVzYWMKICAgICBoYXZlX3BhdD0xOzsKICAgKC0taCB8IC0taGUgfCAtLWhlbCB8IC0t aGVscCkKLS0gCjIuNS4wCgo= --------------030709000809080605070200-- From unknown Mon Aug 18 17:54:04 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions Resent-From: Jim Meyering Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Fri, 18 Mar 2016 03:59:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22945 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: Paul Eggert Cc: Fulvio Scapin , 22945@debbugs.gnu.org Received: via spool by 22945-submit@debbugs.gnu.org id=B22945.14582735212093 (code B ref 22945); Fri, 18 Mar 2016 03:59:01 +0000 Received: (at 22945) by debbugs.gnu.org; 18 Mar 2016 03:58:41 +0000 Received: from localhost ([127.0.0.1]:51635 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aglYf-0000Xg-Hi for submit@debbugs.gnu.org; Thu, 17 Mar 2016 23:58:41 -0400 Received: from mail-ob0-f181.google.com ([209.85.214.181]:35193) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aglYd-0000XU-NL for 22945@debbugs.gnu.org; Thu, 17 Mar 2016 23:58:40 -0400 Received: by mail-ob0-f181.google.com with SMTP id fp4so104662356obb.2 for <22945@debbugs.gnu.org>; Thu, 17 Mar 2016 20:58:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=8m2L3i7blRviJ/5l8U1N821oziDiAc7Ho7gEFYlDC1c=; b=NGwTQMv0kOb9L3hXut8yGVv/eWQ5PlNqI5Y4trjB1AdRYna7iJl2eDT2Lee4vy+geJ pkrAvMRm0mHINOvH0Gaooy5FxZO3thyeH4tup9M5lQNGIAKhMJkfVXFOsgoJThXIe6Bh 7U5AhL0PdzZrzt9S0q5y2SSbvcT9O+5bQFpek4ngmmWrE+5yc0e54Y70raVHMDCmOHSm N0YYb3Hh2wXFqLM2r1mMIZ4i2Xn/Qj8Tho474ozYynNXdcXI4SBPVPWT1m6X7X949AFx d2XLz0CcUZwimzJsYXGeGe9qIoVcUm+gjNhceZh7gmhd8tu7AkmLT+v9Cj/+rJH1Ei2z iClA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=8m2L3i7blRviJ/5l8U1N821oziDiAc7Ho7gEFYlDC1c=; b=E5RUPHK1CSv82JJiKDNWzUTe1bzoBuH2WPa8lirO6FrJSdjI4VYdHWonpDJcDLVHiI fLrJKo6wlt66vC/iXNBjMHYrHt/Ns+HWI0hmvk3a2B9QsBx7S5ehICzmj/1CqrvHM17H cPywMPwDZF/+YSCIWLLFwS68qPCjxuTOl/Jfe45vMO+GkZe6Cif8vErSqsN3NYogBhSX yoTd0h/4kEGmQwsJwuUS1qL3dFVnlan+g+RMdzXbvW/QZjBjKKZceCgq2XaEExUffswv EidFZYzSkftDhU1HFSUhSTa8VoaCld1XvhOGvpTxGOgDllJ2VEN4emNxuuomQq068m46 Fn8A== X-Gm-Message-State: AD7BkJIa9YjpQrTp3sfj61aZ22Uo3JgFUDz3MKgi4sV5VNbvxKt9Nw17AdNHGuju9AlxPCc+/M1iG/V1CuZfrg== X-Received: by 10.182.242.4 with SMTP id wm4mr8502207obc.85.1458273514095; Thu, 17 Mar 2016 20:58:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.44.5 with HTTP; Thu, 17 Mar 2016 20:58:14 -0700 (PDT) In-Reply-To: <56EB0FEE.4010302@cs.ucla.edu> References: <56E9CAC4.3070708@cs.ucla.edu> <56EB0FEE.4010302@cs.ucla.edu> From: Jim Meyering Date: Thu, 17 Mar 2016 20:58:14 -0700 X-Google-Sender-Auth: NX8_3Y_8BvYGctbEuBM2DFhLlPY Message-ID: Content-Type: text/plain; charset=UTF-8 X-Spam-Score: -0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) On Thu, Mar 17, 2016 at 1:13 PM, Paul Eggert wrote: > On 03/16/2016 02:06 PM, Paul Eggert wrote: >> >> I have worked on a patch but don't have a reliable fix yet, or even a >> portable test case to illustrate the bug. > > On further thought I found a test case and a fix, which I've attached. > Normally I would just install this, but we're so close to a release that > I'll wait for a word from Jim. Thank you for working on that. One nit: perhaps it should continue to work when a search string contains a NUL byte? E.g., this works before your change on OS X yet finds no match with the patch applied: $ zgrep -af <(printf 'b\0\na') <(printf 'b\0') <(echo a) /dev/fd/12:b /dev/fd/13:a Might be tricky to portably transform that NUL byte into something we can embed in a command-line-specified search string. Is there even a notation for that? I don't think so. But NUL problems aside, this also should work, requiring alternation in the regexp derived from input with two or more lines, but then we'll have to escape embedded '|' bytes, too: $ zgrep -f <(printf 'a\nb') <(echo b) <(echo a) /dev/fd/12:b /dev/fd/13:a From unknown Mon Aug 18 17:54:04 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions Resent-From: Paul Eggert Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Fri, 18 Mar 2016 07:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22945 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: Jim Meyering Cc: Fulvio Scapin , 22945@debbugs.gnu.org Received: via spool by 22945-submit@debbugs.gnu.org id=B22945.145828719629963 (code B ref 22945); Fri, 18 Mar 2016 07:47:02 +0000 Received: (at 22945) by debbugs.gnu.org; 18 Mar 2016 07:46:36 +0000 Received: from localhost ([127.0.0.1]:51719 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1agp7D-0007nD-NN for submit@debbugs.gnu.org; Fri, 18 Mar 2016 03:46:35 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:50068) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1agp7A-0007my-UZ for 22945@debbugs.gnu.org; Fri, 18 Mar 2016 03:46:33 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 01791160E42; Fri, 18 Mar 2016 00:46:27 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id jD8jwZnVNUwy; Fri, 18 Mar 2016 00:46:25 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id C292B160F86; Fri, 18 Mar 2016 00:46:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Iznq4rA-PUkH; Fri, 18 Mar 2016 00:46:25 -0700 (PDT) Received: from [192.168.1.9] (pool-100-32-155-148.lsanca.fios.verizon.net [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 96594160E42; Fri, 18 Mar 2016 00:46:25 -0700 (PDT) References: <56E9CAC4.3070708@cs.ucla.edu> <56EB0FEE.4010302@cs.ucla.edu> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <56EBB251.7090302@cs.ucla.edu> Date: Fri, 18 Mar 2016 00:46:25 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------030404050304070603060802" X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) This is a multi-part message in MIME format. --------------030404050304070603060802 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Jim Meyering wrote: > Might be tricky to portably transform that NUL byte into something we > can embed in a command-line-specified search string. Is there even a > notation for that? I don't think so. > > But NUL problems aside, this also should work, requiring alternation > in the regexp derived from input with two or more lines, but then > we'll have to escape embedded '|' bytes, too: How about the attached patch instead? It uses a bigger hammer, which should address both issues. --------------030404050304070603060802 Content-Type: text/x-diff; name="0001-zgrep-with-f-SPECIAL-read-SPECIAL-just-once.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-zgrep-with-f-SPECIAL-read-SPECIAL-just-once.patch" >From a5c927ea71ccc86fbb90ab0ea6083bf3cdcd9472 Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Thu, 17 Mar 2016 13:08:06 -0700 Subject: [PATCH] zgrep: with -f SPECIAL, read SPECIAL just once Problem reported by Fulvio Scapin in: http://bugs.gnu.org/22945 * NEWS: Document this. * tests/zgrep-f: Add a test. * zgrep.in (with_filename): With -f FILE, if FILE is stdin or not a regular file, copy it into a temporary and use the temporary. --- NEWS | 3 +++ tests/zgrep-f | 8 ++++++++ zgrep.in | 45 ++++++++++++++++++++++++++++++++++++++------- 3 files changed, 49 insertions(+), 7 deletions(-) diff --git a/NEWS b/NEWS index 6363d71..541ad94 100644 --- a/NEWS +++ b/NEWS @@ -35,6 +35,9 @@ GNU gzip NEWS -*- outline -*- gzip -k -v no longer reports that files are replaced. [bug present since the beginning] + zgrep -f A B C no longer reads A more than once if A is not a regular file. + This better supports invocations like 'zgrep -f <(COMMAND) B C' in Bash. + [bug introduced in gzip-1.2] * Noteworthy changes in release 1.6 (2013-06-09) [stable] diff --git a/tests/zgrep-f b/tests/zgrep-f index a8eb746..9a86550 100755 --- a/tests/zgrep-f +++ b/tests/zgrep-f @@ -29,6 +29,14 @@ zgrep -f - haystack.gz < n > out 2>&1 || fail=1 compare out n || fail=1 +if ${BASH_VERSION+:} false; then + set +o posix + # This failed with gzip 1.6. + cat n n >nn || framework_failure_ + eval 'zgrep -h -f <(cat n) haystack.gz haystack.gz' >out || fail=1 + compare out nn || fail=1 +fi + # This failed with gzip 1.4. echo a-b | zgrep -e - > /dev/null || fail=1 diff --git a/zgrep.in b/zgrep.in index c24be57..06baf38 100644 --- a/zgrep.in +++ b/zgrep.in @@ -55,6 +55,7 @@ files_with_matches=0 files_without_matches=0 no_filename=0 with_filename=0 +pattmp= while test $# -ne 0; do option=$1 @@ -113,13 +114,34 @@ while test $# -ne 0; do # The pattern is coming from a file rather than the command-line. # If the file is actually stdin then we need to do a little # magic, since we use stdin to pass the gzip output to grep. - # Turn the -f option into an -e option by copying the file's - # contents into OPTARG. - case $optarg in - (" '-'" | " '/dev/stdin'" | " '/dev/fd/0'") - option=-e - optarg=" '"$(sed "$escape") || exit 2;; - esac + # Similarly if it is not a regular file, since it might be read repeatedly. + # In either of these two cases, copy the pattern into a temporary file, + # and use that file instead. The pattern might contain null bytes, + # so we cannot simply switch to -e here. + if case $optarg in + (" '-'" | " '/dev/stdin'" | " '/dev/fd/0'") + :;; + (*) + eval "test ! -f$optarg";; + esac + then + if test -n "$pattmp"; then + eval "cat --$optarg" >>"$pattmp" + continue + fi + trap ' + test -n "$pattmp" && rm -f "$pattmp" + (exit 2); exit 2 + ' HUP INT PIPE TERM 0 + if type mktemp >/dev/null 2>&1; then + pattmp=$(mktemp -t -- "zgrep.XXXXXX") || exit 2 + else + set -C + pattmp=${TMPDIR-/tmp}/zgrep.$$ + fi + eval "cat --$optarg" >"$pattmp" + optarg=' "$pattmp"' + fi have_pat=1;; (--h | --he | --hel | --help) echo "$usage" || exit 2 @@ -232,5 +254,14 @@ do test 126 -le $res && break done +if test -n "$pattmp"; then + rm -f "$pattmp" || { + r=$? + test $r -lt 2 && r=2 + test $res -lt $r && res=$r + } + trap - HUP INT PIPE TERM 0 +fi + test 128 -le $res && kill -$(expr $res % 128) $$ exit $res -- 2.5.0 --------------030404050304070603060802-- From unknown Mon Aug 18 17:54:04 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions Resent-From: Jim Meyering Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Fri, 18 Mar 2016 20:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22945 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: Paul Eggert Cc: Fulvio Scapin , 22945@debbugs.gnu.org Received: via spool by 22945-submit@debbugs.gnu.org id=B22945.14583327239734 (code B ref 22945); Fri, 18 Mar 2016 20:26:01 +0000 Received: (at 22945) by debbugs.gnu.org; 18 Mar 2016 20:25:23 +0000 Received: from localhost ([127.0.0.1]:52971 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ah0xX-0002Ww-Ee for submit@debbugs.gnu.org; Fri, 18 Mar 2016 16:25:23 -0400 Received: from mail-ob0-f178.google.com ([209.85.214.178]:34155) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ah0xV-0002Wg-IV for 22945@debbugs.gnu.org; Fri, 18 Mar 2016 16:25:22 -0400 Received: by mail-ob0-f178.google.com with SMTP id ts10so127170424obc.1 for <22945@debbugs.gnu.org>; Fri, 18 Mar 2016 13:25:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=JXqE67rOUwXK7o2d1hZwtQ8dAy/sZy1TFilm8T6Xitw=; b=t7PUVgNq99zZOXn16Hi6Urcpp0KflLUBs2KR5kxFA/+fva0ZHJjEyN7JdtUk3Z/6e+ MyR5LFUQ6PmxH9B9R+DtHQ0JZOGggqBlJf5oOLJqD+kxhvktubWPJLZbDqqfA2uCZZAA gQH85drpa53UoKoYjoRDNNJwzH5BtGHQWIhfq5F1AGQzfSYfxJ+Zik0PGWr99u5DU7/Q 025Tpl+43ql7vmbYbWHHwvaSbR9A/aqQk2IE500R09SbbTJa1qhhNP4eLzgpEcPePSNF W7a77HoBc5O5oy1muljX6GQbzV5IN/bm/72a1qCi9wyevk3+ry53mLb0Px+XmYageyks D3jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=JXqE67rOUwXK7o2d1hZwtQ8dAy/sZy1TFilm8T6Xitw=; b=QXvBJCiL7DEkMXRkWc2gWPnXxTHeB8n8mFv/MVNXl21Cwf/ikP/Gq1keR3mTx0yhv5 Hit1y+lw4ZrtoD4OvALTJizf9Sezg34YfBJevMpJeX5Ex5M4Ap6g0BJpKTFjyvEf5t7w FMy3tl0is3aZZW5oTsxzsut4ofkTzsqF2ZUeOiaWECt1OW1QOzJcRKJm0rfMmlN/LxP7 YW7ddXbOSqIfHC8gdyq59tjMXkZpcHr3Eeipt6Oy5NGV/QPj/qlZUVO+btCJrbXGem5b 56gtcx7MYmPIkJoMar7DQUH9l4wLJSPWq9SltUnSj8+eemVkVnovUztOuW92YK+YvHtb 27mA== X-Gm-Message-State: AD7BkJIuo8lTLysY+EjpPgcivTWE6+YHUHIbZ2c+LJlx9+XtrbR7W2KoH9C3dYar8tIp6JGikjOXLokCIf58fQ== X-Received: by 10.182.242.4 with SMTP id wm4mr11134306obc.85.1458332715918; Fri, 18 Mar 2016 13:25:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.44.5 with HTTP; Fri, 18 Mar 2016 13:24:55 -0700 (PDT) In-Reply-To: <56EBB251.7090302@cs.ucla.edu> References: <56E9CAC4.3070708@cs.ucla.edu> <56EB0FEE.4010302@cs.ucla.edu> <56EBB251.7090302@cs.ucla.edu> From: Jim Meyering Date: Fri, 18 Mar 2016 13:24:55 -0700 X-Google-Sender-Auth: BkNaUOII1_phzOMamuMkHEROxp4 Message-ID: Content-Type: multipart/mixed; boundary=e89a8ff24ea37fd25b052e588bac X-Spam-Score: -0.5 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) --e89a8ff24ea37fd25b052e588bac Content-Type: text/plain; charset=UTF-8 On Fri, Mar 18, 2016 at 12:46 AM, Paul Eggert wrote: > Jim Meyering wrote: >> >> Might be tricky to portably transform that NUL byte into something we >> can embed in a command-line-specified search string. Is there even a >> notation for that? I don't think so. >> >> But NUL problems aside, this also should work, requiring alternation >> in the regexp derived from input with two or more lines, but then >> we'll have to escape embedded '|' bytes, too: > > > How about the attached patch instead? It uses a bigger hammer, which should > address both issues. Very nice. Thank you very much. You are welcome to push that with changes like the following: - retain the 2-empty-line section separator in NEWS (there's a syntax-check hook to test for that in other packages, but not yet here in gzip) - adjust the test to cover the case of more than one line in -f's input: --e89a8ff24ea37fd25b052e588bac Content-Type: text/x-patch; charset=US-ASCII; name="k.patch" Content-Disposition: attachment; filename="k.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ily5l1s61 ZGlmZiAtLWdpdCBhL05FV1MgYi9ORVdTCmluZGV4IDU0MWFkOTQuLmQ2MGQ1Y2MgMTAwNjQ0Ci0t LSBhL05FV1MKKysrIGIvTkVXUwpAQCAtMzksNiArMzksNyBAQCBHTlUgZ3ppcCBORVdTICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLSotIG91dGxpbmUgLSotCiAgIFRoaXMgYmV0 dGVyIHN1cHBvcnRzIGludm9jYXRpb25zIGxpa2UgJ3pncmVwIC1mIDwoQ09NTUFORCkgQiBDJyBp biBCYXNoLgogICBbYnVnIGludHJvZHVjZWQgaW4gZ3ppcC0xLjJdCgorCiAqIE5vdGV3b3J0aHkg Y2hhbmdlcyBpbiByZWxlYXNlIDEuNiAoMjAxMy0wNi0wOSkgW3N0YWJsZV0KCiAqKiBOZXcgZmVh dHVyZXMKZGlmZiAtLWdpdCBhL3Rlc3RzL3pncmVwLWYgYi90ZXN0cy96Z3JlcC1mCmluZGV4IDlh ODY1NTAuLjFjZThjYzIgMTAwNzU1Ci0tLSBhL3Rlc3RzL3pncmVwLWYKKysrIGIvdGVzdHMvemdy ZXAtZgpAQCAtMjAsOCArMjAsOCBAQAoKIC4gIiR7c3JjZGlyPS59L2luaXQuc2giOyBwYXRoX3By ZXBlbmRfIC4uCgotZWNobyBuZWVkbGUgPiBuIHx8IGZyYW1ld29ya19mYWlsdXJlXwotZWNobyBu ZWVkbGUgPiBoYXlzdGFjayB8fCBmcmFtZXdvcmtfZmFpbHVyZV8KK3ByaW50ZiAnbmVlZGxlXG5u MlxuJyA+IG4gfHwgZnJhbWV3b3JrX2ZhaWx1cmVfCitjcCBuIGhheXN0YWNrIHx8IGZyYW1ld29y a19mYWlsdXJlXwogZ3ppcCBoYXlzdGFjayB8fCBmcmFtZXdvcmtfZmFpbHVyZV8KCiBmYWlsPTAK --e89a8ff24ea37fd25b052e588bac-- From unknown Mon Aug 18 17:54:04 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Fulvio Scapin Subject: bug#22945: closed (Re: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions) Message-ID: References: <56EC8180.4080808@cs.ucla.edu> X-Gnu-PR-Message: they-closed 22945 X-Gnu-PR-Package: gzip Reply-To: 22945@debbugs.gnu.org Date: Fri, 18 Mar 2016 22:31:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1458340262-21625-1" This is a multi-part message in MIME format... ------------=_1458340262-21625-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #22945: Surprising behaviour (bug?) of zgrep in combination with the -f opt= ion and process substitutions which was filed against the gzip package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 22945@debbugs.gnu.org. --=20 22945: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D22945 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1458340262-21625-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 22945-done) by debbugs.gnu.org; 18 Mar 2016 22:30:34 +0000 Received: from localhost ([127.0.0.1]:53049 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ah2ug-0005c8-3s for submit@debbugs.gnu.org; Fri, 18 Mar 2016 18:30:34 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:44360) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ah2ue-0005bv-3p for 22945-done@debbugs.gnu.org; Fri, 18 Mar 2016 18:30:32 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 2102E160E8A; Fri, 18 Mar 2016 15:30:26 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 9VMFJXNV-Bzy; Fri, 18 Mar 2016 15:30:25 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 383B116112A; Fri, 18 Mar 2016 15:30:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id paHuP08il8Qo; Fri, 18 Mar 2016 15:30:25 -0700 (PDT) Received: from penguin.cs.ucla.edu (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 19A8B160E8A; Fri, 18 Mar 2016 15:30:25 -0700 (PDT) Subject: Re: bug#22945: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions To: Jim Meyering References: <56E9CAC4.3070708@cs.ucla.edu> <56EB0FEE.4010302@cs.ucla.edu> <56EBB251.7090302@cs.ucla.edu> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <56EC8180.4080808@cs.ucla.edu> Date: Fri, 18 Mar 2016 15:30:24 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------010302020907080603020600" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 22945-done Cc: Fulvio Scapin , 22945-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) This is a multi-part message in MIME format. --------------010302020907080603020600 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 03/18/2016 01:24 PM, Jim Meyering wrote: > You are welcome to push that with changes like the following: OK, thanks, I pushed the attached patch, which contains those changes, plus one more change: check for errors when writing to the temporary pattern file. Marking this as done. --------------010302020907080603020600 Content-Type: application/x-patch; name="0001-zgrep-with-f-SPECIAL-read-SPECIAL-just-once.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="0001-zgrep-with-f-SPECIAL-read-SPECIAL-just-once.patch" RnJvbSBmMzU5ZTM0YTFlOTY4Y2ViOTQ5ODAyODZhOGVhMDc5YmUyMWIzYjU2IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQYXVsIEVnZ2VydCA8ZWdnZXJ0QGNzLnVjbGEuZWR1 PgpEYXRlOiBGcmksIDE4IE1hciAyMDE2IDE1OjIwOjUzIC0wNzAwClN1YmplY3Q6IFtQQVRD SF0gemdyZXA6IHdpdGggLWYgU1BFQ0lBTCwgcmVhZCBTUEVDSUFMIGp1c3Qgb25jZQoKUHJv YmxlbSByZXBvcnRlZCBieSBGdWx2aW8gU2NhcGluIGluOiBodHRwOi8vYnVncy5nbnUub3Jn LzIyOTQ1CiogTkVXUzogRG9jdW1lbnQgdGhpcy4KKiB0ZXN0cy96Z3JlcC1mOiBBZGQgYSB0 ZXN0LgpBZGp1c3QgYSB0ZXN0IHRvIGNvdmVyIHRoZSBjYXNlIG9mIG1vcmUgdGhhbiBvbmUg bGluZSBpbiAtZidzIGlucHV0LgoqIHpncmVwLmluICh3aXRoX2ZpbGVuYW1lKTogV2l0aCAt ZiBGSUxFLCBpZiBGSUxFIGlzIHN0ZGluIG9yIG5vdAphIHJlZ3VsYXIgZmlsZSwgY29weSBp dCBpbnRvIGEgdGVtcG9yYXJ5IGFuZCB1c2UgdGhlIHRlbXBvcmFyeS4KLS0tCiBORVdTICAg ICAgICAgIHwgIDQgKysrKwogdGVzdHMvemdyZXAtZiB8IDEyICsrKysrKysrKystLQogemdy ZXAuaW4gICAgICB8IDQ1ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr LS0tLS0tLQogMyBmaWxlcyBjaGFuZ2VkLCA1MiBpbnNlcnRpb25zKCspLCA5IGRlbGV0aW9u cygtKQoKZGlmZiAtLWdpdCBhL05FV1MgYi9ORVdTCmluZGV4IDYzNjNkNzEuLmQ2MGQ1Y2Mg MTAwNjQ0Ci0tLSBhL05FV1MKKysrIGIvTkVXUwpAQCAtMzUsNiArMzUsMTAgQEAgR05VIGd6 aXAgTkVXUyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0qLSBvdXRsaW5l IC0qLQogICBnemlwIC1rIC12IG5vIGxvbmdlciByZXBvcnRzIHRoYXQgZmlsZXMgYXJlIHJl cGxhY2VkLgogICBbYnVnIHByZXNlbnQgc2luY2UgdGhlIGJlZ2lubmluZ10KIAorICB6Z3Jl cCAtZiBBIEIgQyBubyBsb25nZXIgcmVhZHMgQSBtb3JlIHRoYW4gb25jZSBpZiBBIGlzIG5v dCBhIHJlZ3VsYXIgZmlsZS4KKyAgVGhpcyBiZXR0ZXIgc3VwcG9ydHMgaW52b2NhdGlvbnMg bGlrZSAnemdyZXAgLWYgPChDT01NQU5EKSBCIEMnIGluIEJhc2guCisgIFtidWcgaW50cm9k dWNlZCBpbiBnemlwLTEuMl0KKwogCiAqIE5vdGV3b3J0aHkgY2hhbmdlcyBpbiByZWxlYXNl IDEuNiAoMjAxMy0wNi0wOSkgW3N0YWJsZV0KIApkaWZmIC0tZ2l0IGEvdGVzdHMvemdyZXAt ZiBiL3Rlc3RzL3pncmVwLWYKaW5kZXggYThlYjc0Ni4uMWNlOGNjMiAxMDA3NTUKLS0tIGEv dGVzdHMvemdyZXAtZgorKysgYi90ZXN0cy96Z3JlcC1mCkBAIC0yMCw4ICsyMCw4IEBACiAK IC4gIiR7c3JjZGlyPS59L2luaXQuc2giOyBwYXRoX3ByZXBlbmRfIC4uCiAKLWVjaG8gbmVl ZGxlID4gbiB8fCBmcmFtZXdvcmtfZmFpbHVyZV8KLWVjaG8gbmVlZGxlID4gaGF5c3RhY2sg fHwgZnJhbWV3b3JrX2ZhaWx1cmVfCitwcmludGYgJ25lZWRsZVxubjJcbicgPiBuIHx8IGZy YW1ld29ya19mYWlsdXJlXworY3AgbiBoYXlzdGFjayB8fCBmcmFtZXdvcmtfZmFpbHVyZV8K IGd6aXAgaGF5c3RhY2sgfHwgZnJhbWV3b3JrX2ZhaWx1cmVfCiAKIGZhaWw9MApAQCAtMjks NiArMjksMTQgQEAgemdyZXAgLWYgLSBoYXlzdGFjay5neiA8IG4gPiBvdXQgMj4mMSB8fCBm YWlsPTEKIAogY29tcGFyZSBvdXQgbiB8fCBmYWlsPTEKIAoraWYgJHtCQVNIX1ZFUlNJT04r On0gZmFsc2U7IHRoZW4KKyAgc2V0ICtvIHBvc2l4CisgICMgVGhpcyBmYWlsZWQgd2l0aCBn emlwIDEuNi4KKyAgY2F0IG4gbiA+bm4gfHwgZnJhbWV3b3JrX2ZhaWx1cmVfCisgIGV2YWwg J3pncmVwIC1oIC1mIDwoY2F0IG4pIGhheXN0YWNrLmd6IGhheXN0YWNrLmd6JyA+b3V0IHx8 IGZhaWw9MQorICBjb21wYXJlIG91dCBubiB8fCBmYWlsPTEKK2ZpCisKICMgVGhpcyBmYWls ZWQgd2l0aCBnemlwIDEuNC4KIGVjaG8gYS1iIHwgemdyZXAgLWUgLSA+IC9kZXYvbnVsbCB8 fCBmYWlsPTEKIApkaWZmIC0tZ2l0IGEvemdyZXAuaW4gYi96Z3JlcC5pbgppbmRleCBjMjRi ZTU3Li45OWFjZTU5IDEwMDY0NAotLS0gYS96Z3JlcC5pbgorKysgYi96Z3JlcC5pbgpAQCAt NTUsNiArNTUsNyBAQCBmaWxlc193aXRoX21hdGNoZXM9MAogZmlsZXNfd2l0aG91dF9tYXRj aGVzPTAKIG5vX2ZpbGVuYW1lPTAKIHdpdGhfZmlsZW5hbWU9MAorcGF0dG1wPQogCiB3aGls ZSB0ZXN0ICQjIC1uZSAwOyBkbwogICBvcHRpb249JDEKQEAgLTExMywxMyArMTE0LDM0IEBA IHdoaWxlIHRlc3QgJCMgLW5lIDA7IGRvCiAgICAgIyBUaGUgcGF0dGVybiBpcyBjb21pbmcg ZnJvbSBhIGZpbGUgcmF0aGVyIHRoYW4gdGhlIGNvbW1hbmQtbGluZS4KICAgICAjIElmIHRo ZSBmaWxlIGlzIGFjdHVhbGx5IHN0ZGluIHRoZW4gd2UgbmVlZCB0byBkbyBhIGxpdHRsZQog ICAgICMgbWFnaWMsIHNpbmNlIHdlIHVzZSBzdGRpbiB0byBwYXNzIHRoZSBnemlwIG91dHB1 dCB0byBncmVwLgotICAgICMgVHVybiB0aGUgLWYgb3B0aW9uIGludG8gYW4gLWUgb3B0aW9u IGJ5IGNvcHlpbmcgdGhlIGZpbGUncwotICAgICMgY29udGVudHMgaW50byBPUFRBUkcuCi0g ICAgY2FzZSAkb3B0YXJnIGluCi0gICAgKCIgJy0nIiB8ICIgJy9kZXYvc3RkaW4nIiB8ICIg Jy9kZXYvZmQvMCciKQotICAgICAgb3B0aW9uPS1lCi0gICAgICBvcHRhcmc9IiAnIiQoc2Vk ICIkZXNjYXBlIikgfHwgZXhpdCAyOzsKLSAgICBlc2FjCisgICAgIyBTaW1pbGFybHkgaWYg aXQgaXMgbm90IGEgcmVndWxhciBmaWxlLCBzaW5jZSBpdCBtaWdodCBiZSByZWFkIHJlcGVh dGVkbHkuCisgICAgIyBJbiBlaXRoZXIgb2YgdGhlc2UgdHdvIGNhc2VzLCBjb3B5IHRoZSBw YXR0ZXJuIGludG8gYSB0ZW1wb3JhcnkgZmlsZSwKKyAgICAjIGFuZCB1c2UgdGhhdCBmaWxl IGluc3RlYWQuICBUaGUgcGF0dGVybiBtaWdodCBjb250YWluIG51bGwgYnl0ZXMsCisgICAg IyBzbyB3ZSBjYW5ub3Qgc2ltcGx5IHN3aXRjaCB0byAtZSBoZXJlLgorICAgIGlmIGNhc2Ug JG9wdGFyZyBpbgorICAgICAgICgiICctJyIgfCAiICcvZGV2L3N0ZGluJyIgfCAiICcvZGV2 L2ZkLzAnIikKKyAgICAgICAgIDo7OworICAgICAgICgqKQorICAgICAgICAgZXZhbCAidGVz dCAhIC1mJG9wdGFyZyI7OworICAgICAgIGVzYWMKKyAgICB0aGVuCisgICAgICBpZiB0ZXN0 IC1uICIkcGF0dG1wIjsgdGhlbgorICAgICAgICBldmFsICJjYXQgLS0kb3B0YXJnIiA+PiIk cGF0dG1wIiB8fCBleGl0IDIKKyAgICAgICAgY29udGludWUKKyAgICAgIGZpCisgICAgICB0 cmFwICcKKyAgICAgICAgdGVzdCAtbiAiJHBhdHRtcCIgJiYgcm0gLWYgIiRwYXR0bXAiCisg ICAgICAgIChleGl0IDIpOyBleGl0IDIKKyAgICAgICcgSFVQIElOVCBQSVBFIFRFUk0gMAor ICAgICAgaWYgdHlwZSBta3RlbXAgPi9kZXYvbnVsbCAyPiYxOyB0aGVuCisgICAgICAgIHBh dHRtcD0kKG1rdGVtcCAtdCAtLSAiemdyZXAuWFhYWFhYIikgfHwgZXhpdCAyCisgICAgICBl bHNlCisgICAgICAgIHNldCAtQworICAgICAgICBwYXR0bXA9JHtUTVBESVItL3RtcH0vemdy ZXAuJCQKKyAgICAgIGZpCisgICAgICBldmFsICJjYXQgLS0kb3B0YXJnIiA+IiRwYXR0bXAi IHx8IGV4aXQgMgorICAgICAgb3B0YXJnPScgIiRwYXR0bXAiJworICAgIGZpCiAgICAgaGF2 ZV9wYXQ9MTs7CiAgICgtLWggfCAtLWhlIHwgLS1oZWwgfCAtLWhlbHApCiAgICAgZWNobyAi JHVzYWdlIiB8fCBleGl0IDIKQEAgLTIzMiw1ICsyNTQsMTQgQEAgZG8KICAgdGVzdCAxMjYg LWxlICRyZXMgJiYgYnJlYWsKIGRvbmUKIAoraWYgdGVzdCAtbiAiJHBhdHRtcCI7IHRoZW4K KyAgcm0gLWYgIiRwYXR0bXAiIHx8IHsKKyAgICByPSQ/CisgICAgdGVzdCAkciAtbHQgMiAm JiByPTIKKyAgICB0ZXN0ICRyZXMgLWx0ICRyICYmIHJlcz0kcgorICB9CisgIHRyYXAgLSBI VVAgSU5UIFBJUEUgVEVSTSAwCitmaQorCiB0ZXN0IDEyOCAtbGUgJHJlcyAmJiBraWxsIC0k KGV4cHIgJHJlcyAlIDEyOCkgJCQKIGV4aXQgJHJlcwotLSAKMi41LjAKCg== --------------010302020907080603020600-- ------------=_1458340262-21625-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 8 Mar 2016 16:33:03 +0000 Received: from localhost ([127.0.0.1]:39909 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1adKZC-0002hR-KB for submit@debbugs.gnu.org; Tue, 08 Mar 2016 11:33:02 -0500 Received: from eggs.gnu.org ([208.118.235.92]:36760) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1adG2C-0000wW-Iz for submit@debbugs.gnu.org; Tue, 08 Mar 2016 06:42:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1adG26-0007Ep-AW for submit@debbugs.gnu.org; Tue, 08 Mar 2016 06:42:35 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:59223) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adG26-0007El-6z for submit@debbugs.gnu.org; Tue, 08 Mar 2016 06:42:34 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55949) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adG25-0004on-2o for bug-gzip@gnu.org; Tue, 08 Mar 2016 06:42:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1adG23-0007Dh-Vp for bug-gzip@gnu.org; Tue, 08 Mar 2016 06:42:32 -0500 Received: from mail-vk0-x230.google.com ([2607:f8b0:400c:c05::230]:35977) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adG23-0007Dd-QD for bug-gzip@gnu.org; Tue, 08 Mar 2016 06:42:31 -0500 Received: by mail-vk0-x230.google.com with SMTP id c3so13000601vkb.3 for ; Tue, 08 Mar 2016 03:42:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to; bh=qJAY0Rk7H61VF0X0bSTqFUISQkE7Be4jfj8hlwn3oXc=; b=LP8eqA5/2Xe5/bVQD4mvrG7d5JzaYMSTTtq0+gS4+Z+HbZzEoPBYulzXM/lmUVJEke EjU1pBzp4t+8aiDzi8N+SIfC6WN4Ptg1BpsyAblGNJpTcJ/uueDtCzsbSIBtzznon+xN Sg1C2mj00GfEg9iFQ344xAlr2cxTUoq1jZdkHL+znb6lrJPVvibo1Cfl8Cg7s9mU+6qF 3Ympx1LgFkjMmqy7KzMkQMXumcXyVL8Ipx5Ed5InN8WmZKcfZOtICDbb3jqF01ArC1Yu d5aoORNdPflsTCDWlwiAX0z/vfiXcwwPtt+PKqVKnfbs9pMeqH5Gsp+PwvkA3ZEIBR9W ZrVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to; bh=qJAY0Rk7H61VF0X0bSTqFUISQkE7Be4jfj8hlwn3oXc=; b=iOajac7EqUbdSVWjmxIrloh2z9MFWgbUwHsv+swCm/HEMaLQDoAmJPwbUzmoH5MbeA 6rmUgvEsrIwwYospEY6agd/uWMLjCzQ4MJaPfGeFVHRDR2Iwb+QHH2RefRYfpig9Ak44 yXCpy4dOFbrnhwQfEvbJtDmrtQ/SvT4pwDRif0grePDnkKCgC8vYXynD39pZoeRP23qE yFhcrAep5kze5MbGvF/pQ/zRPm8bkKXYtdGEeNKimY8ih46Rr+Sfj720uU2hcpCMD7jd cJ92N5JsXA5CDnHXesP0qmfwnuJTY1ZCVa13+wJJLJ1xT5VU8kZgJDg1nn5CutCFuYKE BI0A== X-Gm-Message-State: AD7BkJKbYo4cnB39x583GCfkVA78chOW5sxmJEqVfNIhZIKaiUU4OHN3ASK3jb/UXwyyVeZnkk4cYF/BivvJyA== MIME-Version: 1.0 X-Received: by 10.31.180.85 with SMTP id d82mr21778534vkf.83.1457437350961; Tue, 08 Mar 2016 03:42:30 -0800 (PST) Received: by 10.103.113.197 with HTTP; Tue, 8 Mar 2016 03:42:30 -0800 (PST) Date: Tue, 8 Mar 2016 12:42:30 +0100 Message-ID: Subject: Surprising behaviour (bug?) of zgrep in combination with the -f option and process substitutions From: Fulvio Scapin To: bug-gzip@gnu.org Content-Type: multipart/alternative; boundary=001a114382ce962d53052d881352 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Tue, 08 Mar 2016 11:33:02 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) --001a114382ce962d53052d881352 Content-Type: text/plain; charset=UTF-8 Hello. There is a problem with zgrep whenever the -f option actually reads from the output of a process substition in bash. A willingly trivial example below. $ mkdir /tmp/test $ cd /tmp/test $ cat > first aaa $ cat > second bbb $ cat > third ccc $ cat > fourth ddd $ tail * ==> first <== aaa ==> fourth <== ddd ==> second <== bbb ==> third <== ccc $ gzip -9 * $ ls first.gz fourth.gz second.gz third.gz $ cat > patterns aaa bbb ccc ddd $ tail patterns aaa bbb ccc ddd $ zfgrep -f <( cat patterns ) first.gz fourth.gz second.gz third.gz first.gz:aaa $ zfgrep -f patterns first.gz fourth.gz second.gz third.gz first.gz:aaa fourth.gz:ddd second.gz:bbb third.gz:ccc zfgrep -f <( cat patterns ) first.gz fourth.gz second.gz third.gz translates in zfgrep -f /dev/fd/XX first.gz fourth.gz second.gz third.gz where XX is a number, 63 for instance . The problem, from what I understand, arises since zgrep -f patternfile a.gz b.gz c.gz actually is a succession of gzip -dc a.gz | grep -f patternfile gzip -dc b.gz | grep -f patternfile gzip -dc c.gz | grep -f patternfile Since patternfile in this case is /dev/fd/XX, only the first invocation of grep in the first pipeline actually reads a pattern list, while the second and third invocation get nothing, giving no match for b.gz and c.gz as a result. >From /bin/zgrep (Version 1.6, Ubuntu 15.10) one can read (-f | --file) # The pattern is coming from a file rather than the command-line. # If the file is actually stdin then we need to do a little # magic, since we use stdin to pass the gzip output to grep. # Turn the -f option into an -e option by copying the file's # contents into OPTARG. case $optarg in (" '-'" | " '/dev/stdin'" | " '/dev/fd/0'") option=-e optarg=" '"$(sed "$escape") || exit 2;; esac have_pat=1;; The workaround concerning stdin should (maybe) also apply to situations such as the one in my example? Thanks in advance. Fulvio Scapin --001a114382ce962d53052d881352 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hello.

<= div>There is a problem with zgre= p whenever the -f option actually reads from the output of a process substi= tion in bash.
A= willingly trivial example below.

$ mkdir = /tmp/test=C2=A0

$ cd /tmp/test=C2=A0

$ cat > first
aaa

$ cat &g= t; second
bbb

$ cat &g= t; third
ccc

$ cat &g= t; fourth
ddd

$ tail *
=3D=3D> first <=3D=3D
aaa

=3D=3D> fourth <=3D=3D
ddd

=3D=3D> second <=3D=3D
bbb

=3D=3D> third <=3D=3D
ccc

$ gzip -9 *

$ ls =C2=A0=
first.gz =C2=A0fourth.gz =C2=A0second.gz =C2=A0third.gz

$ cat > = patterns
aaa
bbb
ccc
ddd

$ tail patt= erns
aaa
bbb
ccc
ddd

$ zfgrep -f= <( cat patterns ) first.gz fourth.gz second.gz third.gz =C2=A0
first= .gz:aaa

$ zfgrep -f= patterns first.gz fourth.gz second.gz third.gz
first.gz:aaa
fourth.gz:ddd
second.gz:bbb
third.gz:ccc



zfgrep -f <( cat patterns ) first.gz fourth.gz second.gz thir= d.gz
translates in
zfgrep -f /dev/fd/XX first.gz fourth.gz sec= ond.gz third.gz
where XX is a= number, 63 for instance .

The problem, from what I understan= d, arises since

=
zgrep -f patternfile a.gz b.gz c.gz

actually is a succession of

gzip -dc a.gz | grep -= f patternfile
gzip -= dc b.gz | grep -f patternfile
gzip -dc c.gz | grep -f patte= rnfile


Since patternfile in this case is /dev/fd/XX, only the first invocati= on of grep in the first pipeline actually reads a pattern list, while the s= econd and third invocation get nothing, giving no match for b.gz and c.gz a= s a result.
=C2=A0

From /bin/zgrep (Version 1.6, Ubuntu= 15.10) one can read

=C2=A0 (-f | --file)
=C2=A0=C2=A0=C2=A0# The pattern is coming from a file rather th= an the command-line.
=C2=A0=C2=A0=C2=A0# If the file is actually stdin then we need to do a= little
=C2=A0=C2=A0=C2=A0# magic, since we use stdin to pass the gzip output = to grep.
=C2=A0=C2=A0=C2=A0# Turn the -f option into an -e option by copying th= e file's
=C2=A0=C2=A0=C2=A0# contents into OPTARG.
=C2=A0=C2=A0=C2=A0case $optarg in
=C2=A0=C2=A0=C2=A0(" '-'" | " '/dev/stdin&#= 39;" | " '/dev/fd/0'")
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0option=3D-e
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0optarg=3D" '"$(sed "$= escape") || exit 2;;
=C2=A0=C2=A0=C2=A0esac
=C2=A0=C2=A0=C2=A0have_pat=3D1;;

The workaround = concerning stdin should (maybe) also apply to situations such as the one in= my example?

Thanks in advance.

<= span style=3D"font-family:monospace">Fulvio Scapin

--001a114382ce962d53052d881352-- ------------=_1458340262-21625-1--