GNU bug report logs - #23892
grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Jim Meyering <jim <at> meyering.net>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#23892: closed (grep is not "grepping" from grep-2.23-1
 (archlinux) with external fixed patterns file.)
Date: Mon, 04 Jul 2016 20:06:01 +0000

[Message part 1 (text/plain, inline)]

Your message dated Mon, 4 Jul 2016 13:05:25 -0700
with message-id <CA+8g5KEewfJPn+h=B3U4sQtoa4FNHDrxKsRrBj8HFHJwUESwMg <at> mail.gmail.com>
and subject line Re: bug#23892: grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.
has caused the debbugs.gnu.org bug report #23892,
regarding grep is not "grepping" from grep-2.23-1 (archlinux) with external fixed patterns file.
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
23892: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=23892
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Pascal <patatetom <at> gmail.com>
To: bug-grep <at> gnu.org
Subject: grep is not "grepping" from grep-2.23-1 (archlinux) with external
 fixed patterns file.
Date: Mon, 4 Jul 2016 15:57:18 +0200

[Message part 3 (text/plain, inline)]

hi,

I've a big (3.3Go) gzipped file which comes from nsrl with fields separated
by one tabulation :

$ zcat nsrlfiletxt.gz | head -2
sha-1    md5    crc32    filename    filesize    productcode
opsystemcode    specialcode
000000206738748edd92c4e3d2e823896700f849
392126e756571ebf112cb1c1cdedf926    ebd105a0    i05002t2.pfb    98865
3095    win

I've a file with fixed patterns (windows only from field 7 opsystemcode) :

$ cat win.os
2000 sp 4
2ksp3
dos
...
xp sp2
xphomeedw/sp2
xpprofessw/sp2

my os is :

$ uname -a
Linux arch 4.4.14-1-lts #1 SMP Fri Jun 24 21:35:25 CEST 2016 x86_64
GNU/Linux

and grep is :

$ grep --version
grep (GNU grep) 2.25
...

$ pacman -Q grep
grep 2.25-2

when I try this :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed
's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,4k 0:00:00 [ 776k/s] [ <=> ]

only 59.4k lines are processed, with no error :-( !
(sed is used on win.os to match only on field and pipe view is used to show
progess)

I downgrade to grep 2.24 :

# pacman -U /var/cache/pacman/pkg/grep-2.24-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed
's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,4k 0:00:00 [ 863k/s] [ <=> ]

again, only 59.4k lines are processed, with no error :-( !

I downgrade to grep 2.23 :

# pacman -U /var/cache/pacman/pkg/grep-2.23-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed
's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
59,1k 0:00:00 [ 823k/s] [ <=> ]

only 59.1k lines are processed, with no error :-( !

I downgrade to grep 2.22 :

# pacman -U /var/cache/pacman/pkg/grep-2.22-1-x86_64.pkg.tar.xz
...

and retry this (the same) :

$ zcat nsrlfiletxt.gz | pv -l | grep --fixed-strings --file=<( sed
's;^.*$;\t&\t;' win.os ) > /opt/nsrl.windows
 157M 0:04:36 [ 567k/s] [ <=> ]

all the 157M of lines are well processed :-) !

so I think there's a bug introduced with grep 2.23...

regards.

[Message part 4 (text/html, inline)]

[Message part 5 (message/rfc822, inline)]

From: Jim Meyering <jim <at> meyering.net>
To: Pascal <patatetom <at> gmail.com>
Cc: 23892-done <at> debbugs.gnu.org
Subject: Re: bug#23892: grep is not "grepping" from grep-2.23-1 (archlinux)
 with external fixed patterns file.
Date: Mon, 4 Jul 2016 13:05:25 -0700

tags 23892 notabug
thanks

[I've re-added the bug-tracking address to record that this was not a
bug and that the issue auto-created by your email is closed. ]

On Mon, Jul 4, 2016 at 11:56 AM, Pascal <patatetom <at> gmail.com> wrote:
> that's right, with LANG=C before grep : all lines are processed :-)

Use LC_ALL=C, not LANG=C. The latter is not portable, while the former is.

> but why it was good with grep 2.22 ?

We discovered bugs -- triggered by e.g., invalid multibyte characters --
that could cause a segfault or an infinite loop that were present in 2.22,
and to fix them, we had to make grep more strict.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.