GNU bug report logs - #39368
Possible visualization bug in GNU grep 3.1 (Ubuntu 18.04 x86_64)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 39368 in the body.
You can then email your comments to 39368 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andrea Ranieri <andrea.ranieri <at> cnr.it>
To: bug-grep <at> gnu.org
Subject: Possible visualization bug in GNU grep 3.1 (Ubuntu 18.04 x86_64)
Date: Fri, 31 Jan 2020 11:38:15 +0100

[Message part 1 (text/plain, inline)]

Dear grep maintainers,

I think I've found a file that makes grep display wrong things, namely
endlines when grep is run "alone", piping just two greps together vs. text
when the output is piped e.g. to more or hd (hexdump).

You can find this file attached, but the file is public and can be found in
this zipfile:
https://www-intuidoc.irisa.fr/files/2017/07/ComplementaryInfo_LMDHG.zip
(part of the Leap Motion Dynamic Hand Gesture (LMDHG) database:
https://www-intuidoc.irisa.fr/en/english-leap-motion-dynamic-hand-gesture-lmdhg-database/
).

The offending command is this one:

#> grep -v "Position\|hand type\|^\t\t$" ArticulationsOrder.txt | grep
'[^A-Z]' # outputs just endlines

While:

#> grep -v "Position\|hand type\|^\t\t$" ArticulationsOrder.txt | grep
'[^A-Z]' | more # outputs this lines
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"

METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"
METACARPAL   ="(63.1641, 111.558, 103.153)"
PROXIMAL     ="(63.1641, 111.558, 103.153)"
        INTERMEDIATE ="(32.0933, 130.062, 69.1525)"
DISTAL"      ="(21.6896, 137.526, 37.7594)"

I can reproduce this also on my Raspberry Pi 4 (with Ubuntu 18.04 and grep
v3.1 as well) and another x86_64 node with Xeon Silver belonging to a
research cluster (grep v2.20).

If you have any questions, do not hesitate to contact me. Thanks for all
the work you do.

Best regards
Andrea Ranieri

#> grep --version
grep (GNU grep) 3.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others, see <
http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

#> uname -a
Linux zapp-brannigan 5.3.0-24-generic #26~18.04.2-Ubuntu SMP Tue Nov 26
12:34:22 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

#> alias | grep grep
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'


-- 
Andrea Ranieri, Ph.D.

Researcher
*National Research Council of Italy*
Via De Marini 6, 16149 Genova, Italy
Email: andrea.ranieri <at> cnr.it

[Message part 2 (text/html, inline)]

[ArticulationsOrder.txt (text/plain, attachment)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Paul Jackson" <pj <at> usa.net>
To: bug-grep <at> gnu.org
Subject: Re: bug#39368: Possible visualization bug in GNU grep 3.1 (Ubuntu 18.04 x86_64)
Date: Fri, 31 Jan 2020 07:54:03 -0600

You wrote:

==========
#> grep -v "Position\|hand type\|^\t\t$" ArticulationsOrder.txt | grep
'[^A-Z]' # outputs just endlines
==========

When I downloaded your attached file, and converted the above
command to the following single, non-wrapped line (as I presume
you intended):

grep -v "Position\|hand type\|^\t\t$" ArticulationsOrder.txt | grep '[^A-Z]'

then I do NOT get just endlines. 

Rather I get the same useful output that you reported getting when
piping this output through "more".

_However_, in either case, each output line ends in a carriage
return '\r', which I suspect is the key to the problem you report.

Try the following:

    echo 'abc\r' 
    echo 'xyz\r' | more

When I do that, I see both the "abc" and the "xyz"

I'm guessing you'll see just a blank line and the "xyz".

Then, for your ArticulationsOrder.txt data, try the following command,
which may (I'm guessing wildly) work better for your purposes:

grep -E -v 'hand type|Position' ArticulationsOrder.txt | tr -d '\r' 

That second grep, the "grep '[^A-Z]'", does nothing that I can see,
on your data, since every line has at least one character that is
not an [A-Z] upper case letter, so every line matches that pattern.

I would suggest that using "grep -E" (or "egrep") is clearer
than using "grep" and escaping the '|' symbols.

-- 
                Paul Jackson
                pj <at> usa.net

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Paul Jackson" <pj <at> usa.net>
To: bug-grep <at> gnu.org
Subject: Re: bug#39368: Possible visualization bug in GNU grep 3.1 (Ubuntu 18.04 x86_64)
Date: Fri, 31 Jan 2020 08:06:28 -0600

oh - and the exclusion of lines with the pattern "^\t\t$"
makes no difference on your dataset, since that dataset
has no lines consisting solely and exactly of two tabs.

-- 
                Paul Jackson
                pj <at> usa.net

Message #16 received at 39368-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Paul Jackson <pj <at> usa.net>
Cc: 39368-done <at> debbugs.gnu.org, Andrea Ranieri <andrea.ranieri <at> cnr.it>
Subject: Re: bug#39368: Possible visualization bug in GNU grep 3.1 (Ubuntu
 18.04 x86_64)
Date: Fri, 31 Jan 2020 12:20:44 -0800

On 1/31/20 5:54 AM, Paul Jackson wrote:
> I'm guessing you'll see just a blank line and the "xyz".

Yes, this appears to be an issue where the user's terminal emulator 
clears the line after \r\n is output. So the user will see similar 
output using (say) 'cat', and the problem has nothing to do with 'grep' 
per se. Closing the bug report here, though it's quite possible a bug 
report should be filed against the user's terminal emulator (whatever it 
is).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.