GNU bug report logs - #23837
grep 2.7 EXIT STATUS inconsistency

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 23837 in the body.
You can then email your comments to 23837 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "O'Brien, Michael" <michael.o-brien <at> hpe.com>
To: "bug-grep <at> gnu.org" <bug-grep <at> gnu.org>
Cc: "Desostoa, Chuck \(ES\)" <chuck.desostoa <at> hpe.com>, "Lenahan,
 Mike" <mike.lenahan <at> hpe.com>
Subject: grep 2.7 EXIT STATUS inconsistency
Date: Fri, 24 Jun 2016 02:27:29 +0000

[Message part 1 (text/plain, inline)]

GNU grep team,

I've found some inconsistency with the EXIT STATUS processing of grep 2.7.   All data used in the test is shown below, along with the command line arguments to grep.   Everything needed to repeat my test cases is described in this email.  Below are the details.

Version Information:

> grep --version
grep (GNU grep) 2.7
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
>

Details:

From the Manual page of grep, the text of the EXIT STATUS follows:

EXIT STATUS
       Normally, the exit status is 0 if selected lines are found and 1 otherwise.  But the exit status is 2 if an  error  occurred,  unless
       the  -q  or  --quiet  or --silent option is used and a selected line is found.  Note, however, that POSIX only mandates, for programs
       such as grep, cmp, and diff, that the exit status in case of error be greater than 1; it is therefore  advisable,  for  the  sake  of
       portability, to use logic that tests for this general condition instead of strict equality with 2.

Below are two test cases that show the EXIT STATUS from grep is not necessarily described as "Normally, the exit status is 0 if selected lines are found and 1 otherwise".

*         Note1: $GREP_EXCLUDE used below is simply a shell variable pointing to a file containing the lines to exclude from the output, and set with a command similar too: GREP_EXCLUDE=./testing.grep_exclude

*         Note2: During the testing below, the *only change* was to the contents of the $GREP_EXCLUDE file.

*         Note3: The input data remained the same between test cases.

*         Note4: The input data is ascii text, with each line terminated with a newline.   Each line in the input file begins with the pattern: "n,n," and continues to the EOL, or newline character.  There are 7 lines in the input file.

Test case one will show the return code from grep when the -ivEF "exclude" removes *all* data records from the input.   Notice, the input file is fed to grep, and there's no output -- in other words, all records were (or "selected to be") excluded.   The return code (or EXIT STATUS) is 1.

Thu Jun 23 22:12:46 UTC 2016
Test Case 1: grep exclude file with filters that will exclude all data from input
hpehrshdev:~/tmp> cat mjodata.txt | grep -ivEf $GREP_EXCLUDE
hpehrshdev:~/tmp> echo $?
1
hpehrshdev:~/tmp> cat mjodata.txt
0,5,"2016-06-17 16:30:54.404000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 16:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=1, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 16:30:53.0000000."
0,5,"2016-06-17 15:30:54.158000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 15:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 15:30:53.0000000."
0,5,"2016-06-17 14:30:53.780000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 14:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 14:30:53.0000000."
0,5,"2016-06-17 13:30:54.906000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 13:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 13:30:53.0000000."
0,5,"2016-06-17 12:30:54.665000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 12:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 12:30:53.0000000."
77,4,"2016-06-17 12:30:56.582000000","The disk usage of this database has exceeded 7498 GB. See system view M_DISK_USAGE for more details."
69,4,"2016-06-17 12:33:20.228000000","Automatic log backup is disabled."
hpehrshdev:~/tmp>
hpehrshdev:~/tmp>
hpehrshdev:~/tmp> cat $GREP_EXCLUDE
^69,4,.*Automatic log backup is disabled.*$
^77,4,.*The disk usage of this database has exceeded.*See system view M_DISK_USAGE for more details.*$
^0,5,.*301 unique constraint violated: TrexUpdate failed on table
hpehrshdev:~/tmp>

Test case two will show the return code from grep when the -ivEF "exclude" removes *only some* of the data records from the input.   Notice, the input file is fed to grep, and output is returned -- in other words, some records were (selected to be) excluded, but some of the records did not get excluded (because there was no filter for them).   The return code (or EXIT STATUS) is 0.

Thu Jun 23 22:15:35 UTC 2016
Test Case 2: grep exclude file with filters that will exclude some of the data from input
hpehrshdev:~/tmp> cat mjodata.txt | grep -ivEf $GREP_EXCLUDE
0,5,"2016-06-17 16:30:54.404000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 16:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=1, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 16:30:53.0000000."
0,5,"2016-06-17 15:30:54.158000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 15:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 15:30:53.0000000."
0,5,"2016-06-17 14:30:53.780000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 14:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 14:30:53.0000000."
0,5,"2016-06-17 13:30:54.906000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 13:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 13:30:53.0000000."
0,5,"2016-06-17 12:30:54.665000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 12:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 12:30:53.0000000."
hpehrshdev:~/tmp> echo $?
0
hpehrshdev:~/tmp> cat mjodata.txt
0,5,"2016-06-17 16:30:54.404000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 16:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=1, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 16:30:53.0000000."
0,5,"2016-06-17 15:30:54.158000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 15:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 15:30:53.0000000."
0,5,"2016-06-17 14:30:53.780000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 14:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 14:30:53.0000000."
0,5,"2016-06-17 13:30:54.906000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 13:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 13:30:53.0000000."
0,5,"2016-06-17 12:30:54.665000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 12:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 12:30:53.0000000."
77,4,"2016-06-17 12:30:56.582000000","The disk usage of this database has exceeded 7498 GB. See system view M_DISK_USAGE for more details."
69,4,"2016-06-17 12:33:20.228000000","Automatic log backup is disabled."
hpehrshdev:~/tmp> cat $GREP_EXCLUDE
^69,4,.*Automatic log backup is disabled.*$
^77,4,.*The disk usage of this database has exceeded.*See system view M_DISK_USAGE for more details.*$
hpehrshdev:~/tmp>

Issue:

In scripting, it is normal to process the EXIT STATUS of grep in such a way that an EXIT STATUS of 0 (zero) indicates that matches to the grep pattern were found.   In *both of* the above test cases matches *were* found, yet the EXIT STATUS was not consistently 0.  In test case 1, *all* records were matched, yet the EXIT STATUS was 1.  With this behaviour of grep, it is * not possible to correctly* perform the following simple logic (bash syntax):

typeset -i RC=0

cat mjodata.txt | grep -ivEf $GREP_EXCLUDE
RC=$?

if [ $RC -eq 0 ]; then
     #Grep found matches
     #Do something...
else
     #No match found, or possibly some error -- needs more checking
fi


Michael J. O'Brien
Consultant Architect, AMS ABS Global Practice
michael.o-brien <at> hpe.com<mailto:michael.o-brien <at> hpe.com>

+1 650 258 2628  Office
+1 916 335 3010  Mobile

US/Pacific time zone
hpe.com


[HPE logo]<http://www.hpe.com/>

[Message part 2 (text/html, inline)]

Message #10 received at 23837-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: "O'Brien, Michael" <michael.o-brien <at> hpe.com>, 23837-done <at> debbugs.gnu.org
Cc: "Desostoa, Chuck \(ES\)" <chuck.desostoa <at> hpe.com>, "Lenahan,
 Mike" <mike.lenahan <at> hpe.com>
Subject: Re: bug#23837: grep 2.7 EXIT STATUS inconsistency
Date: Fri, 24 Jun 2016 10:36:46 +0200

On 06/24/2016 04:27 AM, O'Brien, Michael wrote:
> Notice, the input file is fed to grep, and there's no output -- in other words, all records were (or "selected to be") excluded.

No, -v inverts the sense of matching, so that non-matching lines are 
selected. So there is no bug here.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.