Subject: bug#23185: closed (Re: bug#23185: GNU grep matching discrepancy between -a/--text and not.)

From: [email-masked] (GNU bug Tracking System)
Date: Wed, 06 Apr 2016 06:57:02 +0000

Your bug report

#23185: GNU grep matching discrepancy between -a/--text and not.

which was filed against the grep package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
23185: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=23185
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#23185: GNU grep matching discrepancy between -a/--text and not.
From: Paul Eggert <[email-masked]>
To: Shlomi Fish <[email-masked]>, [email-masked]
Date: Tue, 5 Apr 2016 23:56:22 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

Thanks for pointing out the seeming inconsistency. The documentation mentions 
the issue but is perhaps not clear enough, so I installed the attached patch.

The input file contains NUL bytes and so is treated as binary data, and the grep 
documentation (secton "File and Directory Selection", option "--binary-files") 
says "When processing binary data, ‘grep’ may treat non-text bytes as line 
terminators". This behavior was added to GNU grep in release 2.21 dated 2014, 
partly for performance reasons.

There are two instances in riddle.he of a space followed by a NUL byte, so

   grep -P '[ \t]\r?$' riddles.he

finds a match when the $ matches just before the NUL byte.

-a is one way to get the behavior you evidently expected. Another (perhaps 
better) way is -z. The command:

   grep -zP '[ \t]\r?\n' riddles.he

outputs nothing and exits with status 1.
>From 7cfd9d20773e1a67cb085a14206fd33274c64387 Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Tue, 5 Apr 2016 23:53:30 -0700
Subject: [PATCH] Give another example of binary file processing

Problem reported by Shlomi Fish
* doc/grep.texi (File and Directory Selection):
Document that 'q$' might match 'q' followed by a NUL
if --binary-files=binary is in effect.
---
 doc/grep.texi | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/doc/grep.texi b/doc/grep.texi
index 074113b..1d3d5cb 100644
--- a/doc/grep.texi
+++ b/doc/grep.texi
@@ -607,10 +607,6 @@ By default, @var{type} is @samp{binary}, and when @command{grep}
 discovers that a file is binary it suppresses any further output, and
 instead outputs either a one-line message saying that a binary file
 matches, or no message if there is no match.
-When processing binary data, @command{grep} may treat non-text bytes
-as line terminators; for example, the pattern @samp{.} (period) might
-not match a null byte, as the null byte might be treated as a line
-terminator even without the @option{-z} (@option{--null-data}) option.
 
 If @var{type} is @samp{without-match},
 when @command{grep} discovers that a file is binary
@@ -621,6 +617,16 @@ If @var{type} is @samp{text},
 @command{grep} processes a binary file as if it were text;
 this is equivalent to the @option{-a} option.
 
+When @var{type} is @samp{binary}, @command{grep} may treat non-text
+bytes as line terminators even without the @option{-z}
+(@option{--null-data}) option.  This means choosing @samp{binary}
+versus @samp{text} can affect whether a pattern matches a file.  For
+example, when @var{type} is @samp{binary} the pattern @samp{q$} might
+match @samp{q} immediately followed by a null byte, even though this
+is not matched when @var{type} is @samp{text}.  Conversely, when
+@var{type} is @samp{binary} the pattern @samp{.} (period) might not
+match a null byte.
+
 @emph{Warning:} @samp{--binary-files=text} might output binary garbage,
 which can have nasty side effects
 if the output is a terminal and
-- 
2.5.5

Subject: GNU grep matching discrepancy between -a/--text and not.
From: Shlomi Fish <[email-masked]>
To: [email-masked]
Date: Sat, 2 Apr 2016 15:00:12 +0300
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi all,

as can be seen in this repository:

https://github.com/shlomif/gnu-grep-trailing-space-and-CR-on-riddles.he-false-match

GNU grep says a document it suspects to be binary matches without -a/--text and
doesn't match it or return results with that flag applied. perl sides with the
latter.

I'm on Mageia linux x86-64 v6 and have built GNU grep from the latest git
commit ( c767ed70eca9a82d76f07dcdbcaafa21ec7f86d6 ) to test.

Regards,

	Shlomi Fish

P.S: it seems the build system uses gperf but configure does not verify that it
exists in the path.

-- 
-----------------------------------------------------------------
Shlomi Fish       http://www.shlomifish.org/
Interview with Ben Collins-Sussman - http://shlom.in/sussman

Can I SCO now? Sue who you wanna sue, it doesn't matter anyhoo, it's time to
litigate.
    — http://www.shlomifish.org/humour/bits/Can-I-SCO-Now/

Please reply to list if it's a mailing list post - http://shlom.in/reply .