GNU bug report logs - #24371
Bad value of size with --list

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24371 in the body.
You can then email your comments to 24371 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: messages JB <messages.jb <at> free.fr>
To: bug-gzip <at> gnu.org
Subject: Bad value of size with --list
Date: Mon, 5 Sep 2016 11:39:48 +0200

[Message part 1 (text/plain, inline)]

Hi!

The size of the file restored by the option "--list" is incorrect if the 
file is more big than 4GB.

   [/mnt/DATA/TEST]ll
   total 8184100
   -rw-r--r-- 1 jacques jacques *8380511744* févr. 16  2016 file.bin

   [/mnt/DATA/TEST]gzip file.bin

   [/mnt/DATA/TEST]gzip -l file.bin.gz
             compressed        uncompressed  ratio uncompressed_name
             2490092407 *4085544448*  39.1% file.bin

   [/mnt/DATA/TEST]gzip -V
   gzip 1.6
   Copyright (C) 2007, 2010, 2011 Free Software Foundation, Inc.
   Copyright (C) 1993 Jean-loup Gailly.
   This is free software.  You may redistribute copies of it under the
   terms of
   the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
   There is NO WARRANTY, to the extent permitted by law.

   Written by Jean-loup Gailly.

   [/mnt/DATA/TEST]


best regards

[Message part 2 (text/html, inline)]

Message #10 received at 24371-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: messages JB <messages.jb <at> free.fr>
Cc: 24371-done <at> debbugs.gnu.org
Subject: Re: bug#24371: Bad value of size with --list
Date: Mon, 5 Sep 2016 18:45:34 -0700

On Mon, Sep 5, 2016 at 2:39 AM, messages JB <messages.jb <at> free.fr> wrote:
> Hi!
>
> The size of the file restored by the option "--list" is incorrect if the
> file is more big than 4GB.
>
>    [/mnt/DATA/TEST]ll
>    total 8184100
>    -rw-r--r-- 1 jacques jacques *8380511744* févr. 16  2016 file.bin
>
>    [/mnt/DATA/TEST]gzip file.bin
>
>    [/mnt/DATA/TEST]gzip -l file.bin.gz
>              compressed        uncompressed  ratio uncompressed_name
>              2490092407 *4085544448*  39.1% file.bin
>
>    [/mnt/DATA/TEST]gzip -V
>    gzip 1.6

Thanks for the report.

However, this is a known and documented limitation in the format, so
cannot be fixed in gzip. Consider using a tool with a more modern
format, like xz.

Here is the relevant part of the documentation from "info gzip":

     The ‘gzip’ format represents the input size modulo 2^32, so the
     uncompressed size and compression ratio are listed incorrectly for
     uncompressed files 4 GiB and larger.  To work around this problem,
     you can use the following command to discover a large uncompressed
     file’s true size:

          zcat file.gz | wc -c

Since there is nothing in gzip that can be changed to address this,
I'm closing the ticket.

Message #13 received at 24371-done <at> debbugs.gnu.org (full text, mbox):

From: Mark Adler <madler <at> alumni.caltech.edu>
To: Jim Meyering <jim <at> meyering.net>
Cc: messages JB <messages.jb <at> free.fr>, 24371-done <at> debbugs.gnu.org
Subject: Re: bug#24371: Bad value of size with --list
Date: Mon, 5 Sep 2016 22:57:07 -0600

On Sep 5, 2016, at 7:45 PM, Jim Meyering <jim <at> meyering.net> wrote:
> Since there is nothing in gzip that can be changed to address this,
> I'm closing the ticket.

Well, technically there is, but it could be argued that the fix is not in the spirit of -l, i.e. not O(1) in time. The fix is that gzip could decompress the entire file without writing it, a la -t, and count the number of uncompressed bytes, which would be O(n).

pigz -l does what gzip -l does, O(1), but pigz -lt guarantees the correct uncompressed size by reading and decompressing the entire file, O(n).

Mark

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.