GNU bug report logs - #56843
grep mangling lines

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 56843 in the body.
You can then email your comments to 56843 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "David G. Pickett" <dgpickett <at> aol.com>
To: "bug-grep <at> gnu.org" <bug-grep <at> gnu.org>
Subject: grep mangling lines
Date: Sat, 30 Jul 2022 15:57:39 +0000 (UTC)

[Message part 1 (text/plain, inline)]

I was stunned to see the grep command remove bytes:

$ crontab -l|grep clam30 23 * * * /usr/bin/freshclam --datadir=/home/dgp/.clamtk/db --log=/home/dgp/.clamtk/db/freshclam.$(/bin/date +\%b-\%d-\%Y).log >/home/dgp/.clamtk/db/freshcla_cron.log 2>&1 ; /usr/local/bin/clamdscan --config-file=/etc/clamav/clamd.conf --reload --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" >>.clamtk/history/clamscan.log 2>&1 ; /usr/local/bin/clamdscan --config-file=/etc/clamav/clad.conf --quiet --multiscan --fdpass --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" $HOME >>.clamtk/history/clamscan.log 2>&1 # freshclam update andclam-scan
from this line:
30 23 * * * /usr/bin/freshclam --datadir=/home/dgp/.clamtk/db --log=/home/dgp/.clamtk/db/freshclam.$(/bin/date +\%b-\%d-\%Y).log >/home/dgp/.clamtk/db/freshclam_cron.log 2>&1 ; /usr/local/bin/clamdscan --config-file=/etc/clamav/clamd.conf --reload --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" >>.clamtk/history/clamscan.log 2>&1 ; /usr/local/bin/clamdscan --config-file=/etc/clamav/clamd.conf --quiet --multiscan --fdpass --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" $HOME >>.clamtk/history/clamscan.log 2>&1 # freshclam update and clam-scan
On my 80x25 xterm, the missing byte should have been at the end of the line, and is highlighted in red.  Highlighting is ok but accurate data is nicer?

[Message part 2 (text/html, inline)]

Message #8 received at 56843 <at> debbugs.gnu.org (full text, mbox):

From: Dimitry Andric <dimitry <at> andric.com>
To: "David G. Pickett" <dgpickett <at> aol.com>
Cc: 56843 <at> debbugs.gnu.org
Subject: Re: bug#56843: grep mangling lines
Date: Sat, 30 Jul 2022 19:15:49 +0200

On 30 Jul 2022, at 17:57, David G. Pickett via Bug reports for GNU grep <bug-grep <at> gnu.org> wrote:
> 
> I was stunned to see the grep command remove bytes:
> 
> $ crontab -l|grep clam30 23 * * * /usr/bin/freshclam --datadir=/home/dgp/.clamtk/db --log=/home/dgp/.clamtk/db/freshclam.$(/bin/date +\%b-\%d-\%Y).log >/home/dgp/.clamtk/db/freshcla_cron.log 2>&1 ; /usr/local/bin/clamdscan --config-file=/etc/clamav/clamd.conf --reload --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" >>.clamtk/history/clamscan.log 2>&1 ; /usr/local/bin/clamdscan --config-file=/etc/clamav/clad.conf --quiet --multiscan --fdpass --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" $HOME >>.clamtk/history/clamscan.log 2>&1 # freshclam update andclam-scan
> from this line:
> 30 23 * * * /usr/bin/freshclam --datadir=/home/dgp/.clamtk/db --log=/home/dgp/.clamtk/db/freshclam.$(/bin/date +\%b-\%d-\%Y).log >/home/dgp/.clamtk/db/freshclam_cron.log 2>&1 ; /usr/local/bin/clamdscan --config-file=/etc/clamav/clamd.conf --reload --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" >>.clamtk/history/clamscan.log 2>&1 ; /usr/local/bin/clamdscan --config-file=/etc/clamav/clamd.conf --quiet --multiscan --fdpass --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" $HOME >>.clamtk/history/clamscan.log 2>&1 # freshclam update and clam-scan
> On my 80x25 xterm, the missing byte should have been at the end of the line, and is highlighted in red.  Highlighting is ok but accurate data is nicer?

I'm sorry, but the formatting in your mail seems completely mangled. ;)

It's very hard to make out what you think is wrong, so could you make
that more explicit?

-Dimitry

Message #11 received at 56843 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: "David G. Pickett" <dgpickett <at> aol.com>
Cc: 56843 <at> debbugs.gnu.org
Subject: Re: bug#56843: grep mangling lines
Date: Sat, 30 Jul 2022 11:17:43 -0700

I'm not seeing the problem on Ubuntu 22.04 LTS, using grep 3.7 and GNOME 
Terminal 3.44.0 using VTE 0.68.0 +BIDI +GNUTLS +ICU +SYSTEMD. The output 
isn't even colored unless I pass something like '--color=always' to 
'grep', which leads me to wonder whether you're using an alias for 
'grep' instead of the real thing.

Here's how I tried to reproduce the problem. What happens when you try 
the same thing on your platform? and what is your platform?

$ cat >x <<'EOF'
> 30 23 * * * /usr/bin/freshclam --datadir=/home/dgp/.clamtk/db 
--log=/home/dgp/.clamtk/db/freshclam.$(/bin/date +\%b-\%d-\%Y).log 
>/home/dgp/.clamtk/db/freshclam_cron.log 2>&1 ; 
/usr/local/bin/clamdscan --config-file=/etc/clamav/clamd.conf --reload 
--log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" 
>>.clamtk/history/clamscan.log 2>&1 ; /usr/local/bin/clamdscan 
--config-file=/etc/clamav/clamd.conf --quiet --multiscan --fdpass 
--log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" $HOME 
>>.clamtk/history/clamscan.log 2>&1 # freshclam update and clam-scan
> EOF
$ cat x | grep clam | diff x -
$ e=$(printf '\033')
$ cat x | grep --color=always clam | sed "s/$e\\[[0-9;]*.//g" | diff x -
$ which grep
/usr/bin/grep

Message #14 received at 56843 <at> debbugs.gnu.org (full text, mbox):

From: "David G. Pickett" <dgpickett <at> aol.com>
To: "eggert <at> cs.ucla.edu" <eggert <at> cs.ucla.edu>
Cc: "56843 <at> debbugs.gnu.org" <56843 <at> debbugs.gnu.org>
Subject: Re: bug#56843: grep mangling lines
Date: Sun, 31 Jul 2022 00:42:05 +0000 (UTC)

[Message part 1 (text/plain, inline)]

That was a bit funny and not an exact reproduction, as <<EOF expands shell things.
I did this to see if grep was at fault.  Apparently it is the terminal!  Any clues on how I got an xterm to mess up?  (My crontab continues to morph.  My trussxx is a strace with options and vi wrapper, evolved since i left Solaris and its 'truss'.):


$ crontab -l|(sleep 2 ; grep clam)& trussxx -p $![1] 23978strace: Process 23978 attachedstrace: Process 23985 attached30 23 * * * /usr/bin/freshclam --datadir=/home/dgp/.clamtk/db --log=/home/dgp/.clamtk/db/freshclam.$(/bin/date +\%b-\%d-\%Y).log >/home/dgp/.clamtk/db/freshcla_cron.log 2>&1 ; /usr/local/bin/clamdscan --reload --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" >>.clamtk/history/clamscan.log 2>&1 ; if [ `fuser .clamtk/history/clamscan.log 2>/dev/null` = "" ]; then  echo Still running >>.clamtk/history/clamscan.log ; else  /usr/local/bin/clamdscan --quiet --multiscan --fdpass --log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" / >>.clamtk/history/clamscan.log 2>&1 ; fi # freshclam update and clam-scan
===================================
23985 write(1, "30 23 * * * /usr/bin/fresh\33[01;31m\33[Kclam\33[m\33[K --datadir=/home/dgp/.\33[01;31m\33[Kclam\33[m\33[Ktk/db --log=/home/dgp/.\33[01;31m\33[Kclam\33[m\33[Ktk/db/fresh\33[01;31m\33[Kclam\33[m\33[K.$(/bin/date +\\%b-\\%d-\\%Y).log >/home/dgp/.\33[01;31m\33[Kclam\33[m\33[Ktk/db/fresh\33[01;31m\33[Kclam\33[m\33[K_cron.log 2>&1 ; /usr/local/bin/\33[01;31m\33[Kclam\33[m\33[Kdscan --reload --log=\"$HOME/.\33[01;31m\33[Kclam\33[m\33[Ktk/history/$(/bin/date +\\%b-\\%d-\\%Y).log\" >>.\33[01;31m\33[Kclam\33[m\33[Ktk/history/\33[01;31m\33[Kclam\33[m\33[Kscan.log 2>&1 ; if [ `fuser .\33[01;31m\33[Kclam\33[m\33[Ktk/history/\33[01;31m\33[Kclam\33[m\33[Kscan.log 2>/dev/null` = \"\" ]; then  echo Still running >>.\33[01;31m\33[Kclam\33[m\33[Ktk/history/\33[01;31m\33[Kclam\33[m\33[Kscan.log ; else  /usr/local/bin/\33[01;31m\33[Kclam\33[m\33[Kdscan --quiet --multiscan --fdpass --log=\"$HOME/.\33[01;31m\33[Kclam\33[m\33[Ktk/history/$(/bin/date +\\%b-\\%d-\\%Y).log\" / >>.\33[01;31m\33[Kclam\33[m\33[Ktk/history/\33[01;31m\33[Kclam\33[m\33[Kscan.log 2>&1 ; fi # fresh\33[01;31m\33[Kclam\33[m\33[K update and \33[01;31m\33[Kclam\33[m\33[K-scan\n", 965) = 965
========================dgp <at> dgp-p6803w:~$ stty -aspeed 38400 baud; rows 25; columns 80; line = 0;intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;werase = ^W; lnext = ^V; discard = ^O; min = 1; time = 0;-parenb -parodd -cmspar cs8 -hupcl -cstopb cread -clocal -crtscts-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff-iuclc -ixany -imaxbel iutf8opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprtechoctl echoke -flusho -extprocdgp <at> dgp-p6803w:~$ echo $TERMxtermdgp <at> dgp-p6803w:~$ pse xtermUID          PID    PPID  C STIME TTY          TIME CMDdgp         3841    1862  0 12:38 ?        00:00:03 xterm -T x3831 -n x3831 -geometry 80x25 -fn 12x24 -sb -vb -cn -sl 9999 -lsdgp         3877    1862  0 12:38 ?        00:00:01 xterm -T x3867 -n x3867 -geometry 80x25 -fn 12x24 -sb -vb -cn -sl 9999 -lsdgp         3913    1862  0 12:38 ?        00:00:01 xterm -T x3903 -n x3903 -geometry 80x25 -fn 12x24 -sb -vb -cn -sl 9999 -lsdgp         3949    1862  0 12:38 ?        00:00:01 xterm -T x3939 -n x3939 -geometry 80x25 -fn 12x24 -sb -vb -cn -sl 9999 -lsdgp <at> dgp-p6803w:~$ 
-----Original Message-----
From: Paul Eggert <eggert <at> cs.ucla.edu>
To: David G. Pickett <dgpickett <at> aol.com>
Cc: 56843 <at> debbugs.gnu.org
Sent: Sat, Jul 30, 2022 2:17 pm
Subject: Re: bug#56843: grep mangling lines

I'm not seeing the problem on Ubuntu 22.04 LTS, using grep 3.7 and GNOME 
Terminal 3.44.0 using VTE 0.68.0 +BIDI +GNUTLS +ICU +SYSTEMD. The output 
isn't even colored unless I pass something like '--color=always' to 
'grep', which leads me to wonder whether you're using an alias for 
'grep' instead of the real thing.

Here's how I tried to reproduce the problem. What happens when you try 
the same thing on your platform? and what is your platform?

$ cat >x <<'EOF'
 > 30 23 * * * /usr/bin/freshclam --datadir=/home/dgp/.clamtk/db 
--log=/home/dgp/.clamtk/db/freshclam.$(/bin/date +\%b-\%d-\%Y).log 
 >/home/dgp/.clamtk/db/freshclam_cron.log 2>&1 ; 
/usr/local/bin/clamdscan --config-file=/etc/clamav/clamd.conf --reload 
--log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" 
 >>.clamtk/history/clamscan.log 2>&1 ; /usr/local/bin/clamdscan 
--config-file=/etc/clamav/clamd.conf --quiet --multiscan --fdpass 
--log="$HOME/.clamtk/history/$(/bin/date +\%b-\%d-\%Y).log" $HOME 
 >>.clamtk/history/clamscan.log 2>&1 # freshclam update and clam-scan
 > EOF
$ cat x | grep clam | diff x -
$ e=$(printf '\033')
$ cat x | grep --color=always clam | sed "s/$e\\[[0-9;]*.//g" | diff x -
$ which grep
/usr/bin/grep

[Message part 2 (text/html, inline)]

Message #19 received at 56843-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: "David G. Pickett" <dgpickett <at> aol.com>
Cc: 56843-done <at> debbugs.gnu.org
Subject: Re: bug#56843: grep mangling lines
Date: Sat, 30 Jul 2022 21:50:06 -0700

On 7/30/22 17:42, David G. Pickett wrote:
> Any clues on how I got an xterm to mess up?

Presumably by sending that sequence of escape characters to xterm. You 
should be able to reproduce the problem by using grep --color=always and 
sending the output to a file F, and then by typing 'cat F'.

As this is not a grep bug, I'll close the grep bug report.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.