Subject: bug#23892: closed (Re: bug#23892: grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.)

From: [email-masked] (GNU bug Tracking System)
Date: Mon, 04 Jul 2016 20:06:01 +0000

Your bug report

#23892: grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.

which was filed against the grep package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
23892: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=23892
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#23892: grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.
From: Jim Meyering <[email-masked]>
To: Pascal <[email-masked]>
Cc: [email-masked]
Date: Mon, 4 Jul 2016 13:05:25 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

tags 23892 notabug
thanks

[I've re-added the bug-tracking address to record that this was not a
bug and that the issue auto-created by your email is closed. ]

On Mon, Jul 4, 2016 at 11:56 AM, Pascal <[email-masked]> wrote:
> that's right, with LANG=C before grep : all lines are processed :-)

Use LC_ALL=C, not LANG=C. The latter is not portable, while the former is.

> but why it was good with grep 2.22 ?

We discovered bugs -- triggered by e.g., invalid multibyte characters --
that could cause a segfault or an infinite loop that were present in 2.22,
and to fix them, we had to make grep more strict.

Subject: grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.
From: Pascal <[email-masked]>
To: [email-masked]
Date: Mon, 4 Jul 2016 15:57:18 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

hi,

I've a big (3.3Go) gzipped file which comes from nsrl with fields separated
by one tabulation :

$ zcat nsrlfiletxt.gz | head -2
sha-1    md5    crc32    filename    filesize    productcode
opsystemcode    specialcode
000000206738748edd92c4e3d2e823896700f849
392126e756571ebf112cb1c1cdedf926    ebd105a0    i05002t2.pfb    98865
3095    win

I've a file with fixed patterns (windows only from field 7 opsystemcode) :

$ cat win.os
2000 sp 4
2ksp3
dos
...
xp sp2
xphomeedw/sp2
xpprofessw/sp2

my os is :

$ uname -a
Linux arch 4.4.14-1-lts #1 SMP Fri Jun 24 21:35:25 CEST 2016 x86_64
GNU/Linux

and grep is :

$ grep --version
grep (GNU grep) 2.25
...

$ pacman -Q grep
grep 2.25-2

when I try this :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed
's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,4k 0:00:00 [ 776k/s] [ <=> ]

only 59.4k lines are processed, with no error :-( !
(sed is used on win.os to match only on field and pipe view is used to show
progess)

I downgrade to grep 2.24 :

# pacman -U /var/cache/pacman/pkg/grep-2.24-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed
's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,4k 0:00:00 [ 863k/s] [ <=> ]

again, only 59.4k lines are processed, with no error :-( !

I downgrade to grep 2.23 :

# pacman -U /var/cache/pacman/pkg/grep-2.23-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed
's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,1k 0:00:00 [ 823k/s] [ <=> ]

only 59.1k lines are processed, with no error :-( !

I downgrade to grep 2.22 :

# pacman -U /var/cache/pacman/pkg/grep-2.22-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed
's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
 157M 0:04:36 [ 567k/s] [ <=> ]

all the 157M of lines are well processed :-) !

so I think there's a bug introduced with grep 2.23...

regards.