GNU bug report logs - #44704
uniq: replace repeated lines with a message about how many repeated lines

Previous Next

Full log

Message #12 received at control <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: "Brian J. Murrell" <brian <at> interlinx.bc.ca>, 44704 <at> debbugs.gnu.org
Subject: Re: bug#44704: uniq: replace repeated lines with a message about how
 many repeated lines
Date: Tue, 17 Nov 2020 08:05:46 -0700

tag 44704 notabug
severity 44704 wishlist
stop

Hello,

On 2020-11-17 6:32 a.m., Brian J. Murrell wrote:
> It would be a useful enhancement to uniq to replace all lines
> considered non-uniq (i.e. those that would be removed from the output)
> with a message about how many times the previous line was repeated.
> 
> I.e.
> 
> $ cat <<EOF | uniq --replace-with-message '[previous line repeated %d times]'
[...]

uniq supports the "--group" option, which adds a blank line after each
group of identical lines - this can be used down-stream to process
groups in any way you want.

Example:
  $ cat <<EOF > in
  first line
  second line
  repeated line
  repeated line
  repeated line
  repeated line
  repeated line
  third line
  EOF

  $ cat in | uniq --group=append
  first line

  second line

  repeated line
  repeated line
  repeated line
  repeated line
  repeated line

  third line


  $ cat in | uniq --group=append \
      | awk '$0=="" { print "do something after group" ; next } ;
             1 { print }'
  first line
  do something after group
  second line
  do something after group
  repeated line
  repeated line
  repeated line
  repeated line
  repeated line
  do something after group
  third line
  do something after group

And with counting:

$ cat in | uniq --group=append \
     | awk 'BEGIN { c = 0 } ;
            $0=="" { print "Group has " c " lines" ; c=0 ; next } ;
            1 { print ; c++ }'
  first line
  Group has 1 lines
  second line
  Group has 1 lines
  repeated line
  repeated line
  repeated line
  repeated line
  repeated line
  Group has 5 lines
  third line
  Group has 1 lines


Hope this helps.
More information about "uniq --group=X" is here:

https://www.gnu.org/software/coreutils/manual/html_node/uniq-invocation.html

I'm marking this as "notabug/wishlist", but will likely close soon as
"wontfix" unless we come up with convincing argument why "--group"
is not sufficient for your use case.

Regardless of the status, discussion can continue by replying to this 
thread.

regards,
 - assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.