GNU bug report logs - #26832
bug on grep 3.0

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 26832 in the body.
You can then email your comments to 26832 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: 川崎 正貴 <masataka-kawasaki <at> koeitecmo.co.jp>
To: bug-grep <at> gnu.org
Subject: bug on grep 3.0
Date: Mon, 8 May 2017 16:56:31 +0900

I found a bug on grep 3.0 on 64bit cygwin.
It seems that '\/' before '$' causes probrems.

grep 2.25(correct)
>echo rr/| grep '^.*\/$'
rr/
>echo rr/| gawk '/^.*\/$/'
rr/
>echo rr/| sed -ne '/^.*\/$/p'
rr/

grep 3.0(incorrect)
>echo rr/| grep '^.*\/$'
>echo rr/| gawk '/^.*\/$/'
>echo rr/| sed -ne '/^.*\/$/p'

thanks.

Message #8 received at 26832 <at> debbugs.gnu.org (full text, mbox):

From: Norihiro Tanaka <noritnk <at> kcn.ne.jp>
To: <masataka-kawasaki <at> koeitecmo.co.jp>
Cc: 26832 <at> debbugs.gnu.org
Subject: Re: bug#26832: bug on grep 3.0
Date: Tue, 09 May 2017 08:19:18 +0900

On Mon, 8 May 2017 16:56:31 +0900
Masataka Kawasaki <masataka-kawasaki <at> koeitecmo.co.jp> wrote:

> I found a bug on grep 3.0 on 64bit cygwin.
> It seems that '\/' before '$' causes probrems.
> 
> grep 2.25(correct)
> >echo rr/| grep '^.*\/$'
> rr/
> >echo rr/| gawk '/^.*\/$/'
> rr/
> >echo rr/| sed -ne '/^.*\/$/p'
> rr/
> 
> grep 3.0(incorrect)
> >echo rr/| grep '^.*\/$'
> >echo rr/| gawk '/^.*\/$/'
> >echo rr/| sed -ne '/^.*\/$/p'
> 
> thanks.

Hi,

$ env LC_ALL=C src/grep --version
grep (GNU grep) 3.0
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
$ echo rr/ | env LC_ALL=C src/grep '^.*\/$'
rr/
$ echo rr/ | env LC_ALL=ja_JP.utf8 src/grep '^.*\/$'
rr/
$ echo rr/ | env LC_ALL=ja_JP.eucjp src/grep '^.*\/$'
rr/

I think that it is not a bug.

Thanks,
Norihiro

Message #11 received at 26832 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 川崎 正貴 <masataka-kawasaki <at> koeitecmo.co.jp>
Cc: 26832 <at> debbugs.gnu.org, Norihiro Tanaka <noritnk <at> kcn.ne.jp>
Subject: Re: bug#26832: bug on grep 3.0
Date: Mon, 8 May 2017 16:46:39 -0700

This bug has the feel of Bug#25707, which was fixed as described in:

http://bugs.gnu.org/25707

Can you try a grep with that fix? Here's one way to build and test grep 
with the fix, assuming you have sufficient developer tools installed:

git clone https://git.savannah.gnu.org/git/grep.git
cd grep
./bootstrap
./configure
make
echo rr/| src/grep '^.*\/$'

Message #14 received at 26832 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 川崎 正貴 <masataka-kawasaki <at> koeitecmo.co.jp>
Cc: 26832 <at> debbugs.gnu.org, Norihiro Tanaka <noritnk <at> kcn.ne.jp>
Subject: Re: bug#26832: bug on grep 3.0
Date: Wed, 10 May 2017 01:28:14 -0700

川崎 正貴 wrote:
> bootstrap modified shebang '#!/bin/sh -x' outputs:
> --
> + git submodule update -- gnulib
> Cloning into '/home/username/grep/gnulib'...
> fatal: repository 'https://git.sv.gnu.org/gnulib.git/' not found

I'm afraid I don't know offhand what sort of problem would cause this. (Perhaps 
it's the same problem that is affecting grep.) I don't observe it on my 
GNU/Linux host. Perhaps you can try the recipe on a non-MS-Windows host; if it 
works there, you can do a 'make dist' to generate a tarball, and then extract 
the tarball under MS-Windows and build it there.

Message #17 received at 26832 <at> debbugs.gnu.org (full text, mbox):

From: 川崎 正貴 <masataka-kawasaki <at> koeitecmo.co.jp>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 26832 <at> debbugs.gnu.org, Norihiro Tanaka <noritnk <at> kcn.ne.jp>
Subject: Re: bug#26832: bug on grep 3.0
Date: Wed, 10 May 2017 17:02:25 +0900

On 2017/05/09 8:46, Paul Eggert wrote:
> This bug has the feel of Bug#25707, which was fixed as described in:
> 
> http://bugs.gnu.org/25707
> 
> Can you try a grep with that fix? Here's one way to build and test grep
> with the fix, assuming you have sufficient developer tools installed:
> 
> git clone https://git.savannah.gnu.org/git/grep.git
> cd grep
> ./bootstrap
> ./configure
> make
> echo rr/| src/grep '^.*\/$'

bootstrap failed.
bootstrap modified shebang '#!/bin/sh -x' outputs:
--
+ git submodule update -- gnulib
Cloning into '/home/username/grep/gnulib'...
fatal: repository 'https://git.sv.gnu.org/gnulib.git/' not found
--
Accessing to the url by web browser,it is redirected
'https://git.savannah.gnu.org/gnulib.git/'
and displays 404 page.

BTW, I realized follows.
Incorrect outputs of grep are output only on command prompt of Windows.
Correct results are output on console of cygwin.
Both grep path are /usr/bin/grep and the version is Cygwin (3.0-2).

If use 'sh -c' on command prompt of Windows, I can get correct output.
>sh -c "echo rr/| grep '^.*\/$'"
rr/

thanks.

Message #24 received at 26832-done <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: 川崎 正貴 <masataka-kawasaki <at> koeitecmo.co.jp>,
 Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 26832-done <at> debbugs.gnu.org
Subject: Re: bug#26832: bug on grep 3.0
Date: Wed, 10 May 2017 12:38:58 -0500

[Message part 1 (text/plain, inline)]

tag 26832 notabug
thanks

On 05/10/2017 03:02 AM, 川崎 正貴 wrote:

> 
> BTW, I realized follows.
> Incorrect outputs of grep are output only on command prompt of Windows.
> Correct results are output on console of cygwin.

Then this is an issue in how cygwin programs handle their arguments when
invoked from a native windows program, and/or a factor of the (weird)
quoting rules in use by the Windows native command shell (which are NOT
AT ALL like normal POSIX shell quoting rules), and nothing to do with
upstream grep. You'll probably have better results posting your question
on the cygwin mailing list.

As such, I'm taking the liberty to close this in the upstream database,
as there's nothing we can do here to change behavior.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #27 received at 26832 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: 26832 <at> debbugs.gnu.org, masataka-kawasaki <at> koeitecmo.co.jp
Subject: Re: bug#26832: bug on grep 3.0
Date: Thu, 11 May 2017 13:06:58 -0500

[Message part 1 (text/plain, inline)]

On 05/10/2017 12:38 PM, Eric Blake wrote:
>> BTW, I realized follows.
>> Incorrect outputs of grep are output only on command prompt of Windows.
>> Correct results are output on console of cygwin.
> 
> Then this is an issue in how cygwin programs handle their arguments when
> invoked from a native windows program, and/or a factor of the (weird)
> quoting rules in use by the Windows native command shell (which are NOT
> AT ALL like normal POSIX shell quoting rules), and nothing to do with
> upstream grep. You'll probably have better results posting your question
> on the cygwin mailing list.

In fact, the cygwin list pointed out that the change was intentional -
when you are using the builtin 'echo' of a Windows command prompt, you
are feeding CRLF data into grep, but the cygwin build of grep no longer
forcefully treats binary files in text mode, which means the pattern
"/$" no longer matches on the input string "/\r\n", but only on "/\n".
When you use a Cygwin shell, the builtin 'echo' does not produce \r in
the first place.

https://cygwin.com/ml/cygwin-announce/2017-02/msg00035.html
shows that the change was coordinated between grep, sed, and awk; we
can't help you if you fail to read release notes from your distribution.

The solution is the same as you would do on Linux: preprocess your data
to strip out \r (or write patterns that specifically accommodate for
\r).  Or, for the case where your data resides on the file system, you
can use the cygwin-specific approach of making that directory a text
mount point rather than the default binary mount point, so that cygwin
will strip the \r on your behalf.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #30 received at 26832 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>,
 川崎 正貴 <masataka-kawasaki <at> koeitecmo.co.jp>
Cc: 26832 <at> debbugs.gnu.org
Subject: Re: bug#26832: bug on grep 3.0
Date: Thu, 11 May 2017 13:09:08 -0500

[Message part 1 (text/plain, inline)]

On 05/08/2017 06:46 PM, Paul Eggert wrote:
> This bug has the feel of Bug#25707, which was fixed as described in:
> 
> http://bugs.gnu.org/25707

In fact, that bug was filed upstream _because_ of cygwin's coordinated
move to have grep, sed, and awk no longer mangle binary files.

> 
> Can you try a grep with that fix? Here's one way to build and test grep
> with the fix, assuming you have sufficient developer tools installed:

In fact, the pre-built grep shipping in cygwin DOES have that fix
already installed.  The bug is therefore not in grep, but in usage
expectations of an environment feeding \r\n to grep without accounting
for the \r no longer being silently ignored.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.