Subject: Re: bug#20974: Weird newline matching behaviour in --null-data mode

From: Norihiro Tanaka <[email-masked]>
Date: Sat, 04 Jul 2015 13:40:55 +0900

On Fri, 3 Jul 2015 20:10:08 -0700
Jim Meyering <[email-masked]> wrote:

> Oh, nice! I see that Paul Eggert has just fixed this with
> the following patch:
>   http://git.sv.gnu.org/cgit/grep.git/commit/?id=0e8fda0d880cccd0
> 
> So I'm closing this ticket.
> 

Paul's fix is very nice, I could not found it.

However, following case is not fixed yet.  Not only '.' but also hat
list (e.g. [^a]) should match newline with -z.  So we need clear
RE_HAT_LISTS_NOT_NEWLINE bit.

$ seq 2 | LC_ALL=C grep --null-data '[1-2][^a][1-2]'
1
2
$ seq 2 | LC_ALL=en_US.iso88591 grep --null-data '[1-2][^a][1-2]'

From f991043875dc4c2bc658ab22548e720faf02e380 Mon Sep 17 00:00:00 2001
From: Norihiro Tanaka <[email-masked]>
Date: Fri, 3 Jul 2015 03:43:15 +0900
Subject: [PATCH] grep: -z '[^a]' now consistently matches newline

* NEWS: Document this.
* tests/utf8-bracket: Add test case for this bug.
* src/grep.c (Gcompile, Ecompile): Clear RE_HAT_LISTS_NOT_NEWLINE bit.
---
 NEWS               | 2 +-
 src/grep.c         | 8 +++++---
 tests/utf8-bracket | 2 +-
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/NEWS b/NEWS
index 88ed0f4..262ebfc 100644
--- a/NEWS
+++ b/NEWS
@@ -12,7 +12,7 @@ GNU grep NEWS                                    -*- outline -*-
   grep no longer reads from uninitialized memory or from beyond the end
   of the heap-allocated input buffer.  This fix addressed CVE-2015-1345.
 
-  With -z, '.' in a pattern now consistently matches newline.
+  With -z, '.' and '[^a]' in a pattern now consistently matches newline.
   Previously, it sometimes matched newline, and sometimes did not.
   [bug introduced in grep-2.4]
 
diff --git a/src/grep.c b/src/grep.c
index ed54dc2..ea4b33d 100644
--- a/src/grep.c
+++ b/src/grep.c
@@ -1862,15 +1862,17 @@ static void
 Gcompile (char const *pattern, size_t size)
 {
   GEAcompile (pattern, size,
-              RE_SYNTAX_GREP | RE_DOT_NEWLINE | RE_NO_EMPTY_RANGES);
+              ((RE_SYNTAX_GREP & ~RE_HAT_LISTS_NOT_NEWLINE)
+               | RE_DOT_NEWLINE | RE_NO_EMPTY_RANGES));
 }
 
 static void
 Ecompile (char const *pattern, size_t size)
 {
   GEAcompile (pattern, size,
-              (RE_SYNTAX_POSIX_EGREP | RE_DOT_NEWLINE
-               | RE_NO_EMPTY_RANGES | RE_UNMATCHED_RIGHT_PAREN_ORD));
+              ((RE_SYNTAX_POSIX_EGREP & ~RE_HAT_LISTS_NOT_NEWLINE)
+               | RE_DOT_NEWLINE | RE_NO_EMPTY_RANGES
+               | RE_UNMATCHED_RIGHT_PAREN_ORD));
 }
 
 static void
diff --git a/tests/utf8-bracket b/tests/utf8-bracket
index f5c4a60..a60a9e4 100755
--- a/tests/utf8-bracket
+++ b/tests/utf8-bracket
@@ -24,7 +24,7 @@ printf '1\n2\n' >in || framework_failure_
 fail=0
 
 for locale in C en_US.UTF-8; do
-  for pattern in '1.2' '[12].2' '[1-2].2'; do
+  for pattern in '1.2' '[12].2' '[1-2].2' '[1-2][^a][1-2]'; do
     for suffix in '' '\(\)\1'; do
       LC_ALL=$locale grep --null-data --quiet "$pattern$suffix" in || fail=1
     done
-- 
2.2.0