GNU bug report logs - #30430
[column] accounting for ansi color codes

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 30430 in the body.
You can then email your comments to 30430 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Boruch Baum <boruch_baum <at> gmx.com>
To: bug-coreutils <at> gnu.org,
	P�draig Brady <P <at> draigBrady.com>
Subject: [column] accounting for ansi color codes
Date: Mon, 12 Feb 2018 01:07:15 -0500

The `column -t' command doesn't align input properly when that input
includes ansi color codes.

An argument that it should do so, is that `column' serves to make input
more readable, and easier to find and compare fields of input data - the
same goal as colorization.

I encountered this when trying to columnate log output that had been fed
through the colorization program `ccze'.

Example: Compare the output of the following two one-liners:

 awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
 /var/log/dpkg.log | column -t

 awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
 /var/log/dpkg.log | ccze -m ansi -o noscroll >&1 | column -t

-- 
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1  7286 0036 9E45 1595 8BC0

Message #8 received at 30430 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Boruch Baum <boruch_baum <at> gmx.com>, 30430 <at> debbugs.gnu.org, P <at> draigBrady.com
Subject: Re: bug#30430: [column] accounting for ansi color codes
Date: Mon, 12 Feb 2018 08:23:29 +0100

On 02/12/2018 07:07 AM, Boruch Baum wrote:
> The `column -t' command doesn't align input properly when that input
> includes ansi color codes.
> 
> An argument that it should do so, is that `column' serves to make input
> more readable, and easier to find and compare fields of input data - the
> same goal as colorization.
> 
> I encountered this when trying to columnate log output that had been fed
> through the colorization program `ccze'.
> 
> Example: Compare the output of the following two one-liners:
> 
>   awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
>   /var/log/dpkg.log | column -t
> 
>   awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
>   /var/log/dpkg.log | ccze -m ansi -o noscroll >&1 | column -t

Unfortunately, you did not show us neither the input - and I don't
have dpkg -, nor the output, so I'd have to guess what you think
might be going wrong.

Would you provide a simple reproducer, please?

In the following example, column(1) produces correct output:

  $ printf "%s\n"                \
      "this_is_a_long_string 99" \
      "short abcdefghijklmnop"   \
      | ccze -m ansi -o noscroll \
      | column -t
  this_is_a_long_string  99
  short                  abcdefghijklmnop

FWIW: what happens if you switch the two formatting commands, i.e., use

  $ ... | column -t | ccze ...

instead of

  $ ... | ccze .... | column -t

?
At least when the input uses a different separator, then you'd have to
do that anyway - otherwise the separation of the columns is messed;
compare:

  $ head -n3 /etc/passwd | ccze -m ansi -o noscroll | column -t -s:

  $ head -n3 /etc/passwd | column -t -s: | ccze -m ansi -o noscroll

Is that the same what's happening to you?

Have a nice day,
Berny

Message #11 received at 30430 <at> debbugs.gnu.org (full text, mbox):

From: Boruch Baum <boruch_baum <at> gmx.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>
Cc: P <at> draigBrady.com, 30430 <at> debbugs.gnu.org
Subject: Re: bug#30430: [column] accounting for ansi color codes
Date: Mon, 12 Feb 2018 07:22:22 -0500

[Message part 1 (text/plain, inline)]

On 2018-02-12 08:23, Bernhard Voelker wrote:
> On 02/12/2018 07:07 AM, Boruch Baum wrote:
>> ...

> Unfortunately, you did not show us neither the input

Easy... A sample input would be:

2018-02-09 11:10:27 status unpacked libefivar0:amd64 0.15-3
2018-02-09 11:10:28 status half-configured libefivar0:amd64 0.15-3
2018-02-09 11:10:28 status installed libefivar0:amd64 0.15-3
2018-02-09 11:10:28 trigproc man-db:amd64 2.7.0.2-5 <none>
2018-02-09 11:10:28 status half-configured man-db:amd64 2.7.0.2-5
2018-02-09 11:10:29 status installed man-db:amd64 2.7.0.2-5
2018-02-09 11:10:29 configure efibootmgr:amd64 0.11.0-3 <none>
2018-02-09 11:10:29 status unpacked efibootmgr:amd64 0.11.0-3
2018-02-09 11:10:29 status half-configured efibootmgr:amd64 0.11.0-3
2018-02-09 11:10:29 status installed efibootmgr:amd64 0.11.0-3

> - and I don't have dpkg -, nor the output,

Also easy. First, an easily readable sample output, without piping
colorization into `column' would be, :

>>   awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
>>   /var/log/dpkg.log | column -t

2018-02-09  11:10:27  status     unpacked         libefivar0:amd64
0.15-3
2018-02-09  11:10:28  status     half-configured  libefivar0:amd64
0.15-3
2018-02-09  11:10:28  status     installed        libefivar0:amd64
0.15-3
2018-02-09  11:10:28  trigproc   _                man-db:amd64
2.7.0.2-5  <none>
2018-02-09  11:10:28  status     half-configured  man-db:amd64
2.7.0.2-5
2018-02-09  11:10:29  status     installed        man-db:amd64
2.7.0.2-5
2018-02-09  11:10:29  configure  _                efibootmgr:amd64
0.11.0-3   <none>
2018-02-09  11:10:29  status     unpacked         efibootmgr:amd64
0.11.0-3
2018-02-09  11:10:29  status     half-configured  efibootmgr:amd64
0.11.0-3
2018-02-09  11:10:29  status     installed        efibootmgr:amd64  0.11.0-3


For the colorized output, ie. the actual raw output and the subject of
the bug report, here are two versions. First, in-line, with the
colorization codes stripped out so that it might be easily readable by
all, and; second, in raw format, as an mail attachment file, to avoid
having the control sequences somehow possibly messing with some hapless
reader's TTY.

>>   awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
>>   /var/log/dpkg.log | ccze -m ansi -o noscroll >&1 | column -t

2018-02-09  11:10:27  status  unpacked         libefivar0:amd64  0.15-3
2018-02-09  11:10:28  status  half-configured  libefivar0:amd64  0.15-3
2018-02-09  11:10:28  status  installed        libefivar0:amd64  0.15-3
2018-02-09  11:10:28  trigproc   _                man-db:amd64
2.7.0.2-5  <none>
2018-02-09  11:10:28  status  half-configured  man-db:amd64
2.7.0.2-5
2018-02-09  11:10:29  status  installed        man-db:amd64
2.7.0.2-5
2018-02-09  11:10:29  configure  _                efibootmgr:amd64
0.11.0-3   <none>
2018-02-09  11:10:29  status  unpacked         efibootmgr:amd64
0.11.0-3
2018-02-09  11:10:29  status  half-configured  efibootmgr:amd64
0.11.0-3
2018-02-09  11:10:29  status  installed        efibootmgr:amd64
0.11.0-3

> FWIW: what happens if you switch the two formatting commands, i.e., use
>
>   $ ... | column -t | ccze ...
>
> instead of
>
>   $ ... | ccze .... | column -t
>
> ?

! The colorization is not usefully applied, ie. the entire output is
monochrome, but in a color different that the input or normal STDOUT.

> At least when the input uses a different separator,

That would be a misunderstanding in what `ccze' does. It is specifically
intended for log files only. See, for example, man (1) ccze.

> then you'd have to do that anyway - otherwise the separation of the
> columns is messed;
> compare:
>
>   $ head -n3 /etc/passwd | ccze -m ansi -o noscroll | column -t -s:

As above, the colorization is not usefully applied, ie. the entire
output is monochrome, but in a color different that the input or normal
STDOUT.

>   $ head -n3 /etc/passwd | column -t -s: | ccze -m ansi -o noscroll

A reasonably colorized output is applied.

-- 
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1  7286 0036 9E45 1595 8BC0

[warning_-_columnized_ccze_output_including_raw_ansi_color_codes.txt (text/plain, attachment)]

Message #14 received at 30430 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Boruch Baum <boruch_baum <at> gmx.com>, 30430 <at> debbugs.gnu.org
Subject: Re: bug#30430: [column] accounting for ansi color codes
Date: Sat, 17 Feb 2018 13:33:48 -0800

On 11/02/18 22:07, Boruch Baum wrote:
> The `column -t' command doesn't align input properly when that input
> includes ansi color codes.
> 
> An argument that it should do so, is that `column' serves to make input
> more readable, and easier to find and compare fields of input data - the
> same goal as colorization.
> 
> I encountered this when trying to columnate log output that had been fed
> through the colorization program `ccze'.
> 
> Example: Compare the output of the following two one-liners:
> 
>  awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
>  /var/log/dpkg.log | column -t
> 
>  awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
>  /var/log/dpkg.log | ccze -m ansi -o noscroll >&1 | column -t
> 

Note column(1) is part of the util-linux project.
We've similar considerations though with wc -L
which we've at least mentioned at:
http://www.pixelbeat.org/docs/coreutils-gotchas.html#wc

Message #17 received at 30430 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: 30430 <at> debbugs.gnu.org
Subject: Re: bug#30430: [column] accounting for ansi color codes
Date: Mon, 29 Oct 2018 20:45:17 -0600

tags 30430 notabug
close 30430
stop

(triaging old bugs)

On 2018-02-17 2:33 p.m., Pádraig Brady wrote:
> On 11/02/18 22:07, Boruch Baum wrote:
>> The `column -t' command doesn't align input properly when that input
>> includes ansi color codes.
>>
> 
> Note column(1) is part of the util-linux project.
> We've similar considerations though with wc -L
> which we've at least mentioned at:
> http://www.pixelbeat.org/docs/coreutils-gotchas.html#wc
> 

Given the above, I'm closing this bug.

-assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.