GNU grep team, I've found some inconsistency with the EXIT STATUS processing of grep 2.7. All data used in the test is shown below, along with the command line arguments to grep. Everything needed to repeat my test cases is described in this email. Below are the details.
Version Information: > grep --version grep (GNU grep) 2.7 Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>. > Details: >From the Manual page of grep, the text of the EXIT STATUS follows: EXIT STATUS Normally, the exit status is 0 if selected lines are found and 1 otherwise. But the exit status is 2 if an error occurred, unless the -q or --quiet or --silent option is used and a selected line is found. Note, however, that POSIX only mandates, for programs such as grep, cmp, and diff, that the exit status in case of error be greater than 1; it is therefore advisable, for the sake of portability, to use logic that tests for this general condition instead of strict equality with 2. Below are two test cases that show the EXIT STATUS from grep is not necessarily described as "Normally, the exit status is 0 if selected lines are found and 1 otherwise". * Note1: $GREP_EXCLUDE used below is simply a shell variable pointing to a file containing the lines to exclude from the output, and set with a command similar too: GREP_EXCLUDE=./testing.grep_exclude * Note2: During the testing below, the *only change* was to the contents of the $GREP_EXCLUDE file. * Note3: The input data remained the same between test cases. * Note4: The input data is ascii text, with each line terminated with a newline. Each line in the input file begins with the pattern: "n,n," and continues to the EOL, or newline character. There are 7 lines in the input file. Test case one will show the return code from grep when the -ivEF "exclude" removes *all* data records from the input. Notice, the input file is fed to grep, and there's no output -- in other words, all records were (or "selected to be") excluded. The return code (or EXIT STATUS) is 1. Thu Jun 23 22:12:46 UTC 2016 Test Case 1: grep exclude file with filters that will exclude all data from input hpehrshdev:~/tmp> cat mjodata.txt | grep -ivEf $GREP_EXCLUDE hpehrshdev:~/tmp> echo $? 1 hpehrshdev:~/tmp> cat mjodata.txt 0,5,"2016-06-17 16:30:54.404000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 16:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=1, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 16:30:53.0000000." 0,5,"2016-06-17 15:30:54.158000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 15:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 15:30:53.0000000." 0,5,"2016-06-17 14:30:53.780000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 14:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 14:30:53.0000000." 0,5,"2016-06-17 13:30:54.906000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 13:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 13:30:53.0000000." 0,5,"2016-06-17 12:30:54.665000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 12:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 12:30:53.0000000." 77,4,"2016-06-17 12:30:56.582000000","The disk usage of this database has exceeded 7498 GB. See system view M_DISK_USAGE for more details." 69,4,"2016-06-17 12:33:20.228000000","Automatic log backup is disabled." hpehrshdev:~/tmp> hpehrshdev:~/tmp> hpehrshdev:~/tmp> cat $GREP_EXCLUDE ^69,4,.*Automatic log backup is disabled.*$ ^77,4,.*The disk usage of this database has exceeded.*See system view M_DISK_USAGE for more details.*$ ^0,5,.*301 unique constraint violated: TrexUpdate failed on table hpehrshdev:~/tmp> Test case two will show the return code from grep when the -ivEF "exclude" removes *only some* of the data records from the input. Notice, the input file is fed to grep, and output is returned -- in other words, some records were (selected to be) excluded, but some of the records did not get excluded (because there was no filter for them). The return code (or EXIT STATUS) is 0. Thu Jun 23 22:15:35 UTC 2016 Test Case 2: grep exclude file with filters that will exclude some of the data from input hpehrshdev:~/tmp> cat mjodata.txt | grep -ivEf $GREP_EXCLUDE 0,5,"2016-06-17 16:30:54.404000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 16:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=1, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 16:30:53.0000000." 0,5,"2016-06-17 15:30:54.158000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 15:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 15:30:53.0000000." 0,5,"2016-06-17 14:30:53.780000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 14:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 14:30:53.0000000." 0,5,"2016-06-17 13:30:54.906000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 13:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 13:30:53.0000000." 0,5,"2016-06-17 12:30:54.665000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 12:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 12:30:53.0000000." hpehrshdev:~/tmp> echo $? 0 hpehrshdev:~/tmp> cat mjodata.txt 0,5,"2016-06-17 16:30:54.404000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 16:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=1, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 16:30:53.0000000." 0,5,"2016-06-17 15:30:54.158000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 15:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 15:30:53.0000000." 0,5,"2016-06-17 14:30:53.780000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 14:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=3, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 14:30:53.0000000." 0,5,"2016-06-17 13:30:54.906000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 13:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 13:30:53.0000000." 0,5,"2016-06-17 12:30:54.665000000","[29] 301 unique constraint violated: TrexUpdate failed on table '_SYS_STATISTICS:STATISTICS_ALERTS_BASE' with error: unique constraint violation in self check for table _SYS_STATISTICS:STATISTICS_ALERTS_BASEen, constraint='$trexexternalkey$', udiv='2016-06-17 12:30:53;29;27,PBM_ECC_DB.ZFE_FDSTAGDETAIL', pos=4, indexname=_SYS_TREE_CS_#145806_#0_#P0, rc=55 SNAPSHOT_ID: 2016-06-17 12:30:53.0000000." 77,4,"2016-06-17 12:30:56.582000000","The disk usage of this database has exceeded 7498 GB. See system view M_DISK_USAGE for more details." 69,4,"2016-06-17 12:33:20.228000000","Automatic log backup is disabled." hpehrshdev:~/tmp> cat $GREP_EXCLUDE ^69,4,.*Automatic log backup is disabled.*$ ^77,4,.*The disk usage of this database has exceeded.*See system view M_DISK_USAGE for more details.*$ hpehrshdev:~/tmp> Issue: In scripting, it is normal to process the EXIT STATUS of grep in such a way that an EXIT STATUS of 0 (zero) indicates that matches to the grep pattern were found. In *both of* the above test cases matches *were* found, yet the EXIT STATUS was not consistently 0. In test case 1, *all* records were matched, yet the EXIT STATUS was 1. With this behaviour of grep, it is * not possible to correctly* perform the following simple logic (bash syntax): typeset -i RC=0 cat mjodata.txt | grep -ivEf $GREP_EXCLUDE RC=$? if [ $RC -eq 0 ]; then #Grep found matches #Do something... else #No match found, or possibly some error -- needs more checking fi Michael J. O'Brien Consultant Architect, AMS ABS Global Practice michael.o-br...@hpe.com<mailto:michael.o-br...@hpe.com> +1 650 258 2628 Office +1 916 335 3010 Mobile US/Pacific time zone hpe.com [HPE logo]<http://www.hpe.com/>
