GNU bug report logs - #67022
Gzip decompression can be 60% faster using zlib's CRC32

Previous Next

Package: gzip;

Reported by: Young Mo Kang <kym327 <at> gmail.com>

Date: Thu, 9 Nov 2023 17:42:01 UTC

Severity: normal

Done: Paul Eggert <eggert <at> cs.ucla.edu>

Bug is archived. No further changes may be made.

Full log

Message #20 received at 67022 <at> debbugs.gnu.org (full text, mbox):

From: "Adler, Mark" <madler <at> alumni.caltech.edu>
To: wrotycz <wrotycz <at> wir.pl>
Cc: "67022 <at> debbugs.gnu.org" <67022 <at> debbugs.gnu.org>
Subject: Re: bug#67022: Gzip decompression can be 60% faster using zlib's CRC32
Date: Thu, 14 Mar 2024 22:57:23 +0000

On Mar 14, 2024, at 3:46 PM, wrotycz <wrotycz <at> not.really> wrote:
> 
> 
> Despite that the question is how do I use zlib crc32()? It doesn't give me correct result.
> 
> My 'rig' is this:
> 
> ~~~
> crc = -1
> while (buffer, length = read_data()):
>     {
>     crc = crcfunc(crc, buffer, length)
>     }
> crc = ~crc
> ~~~
> 
> This doesn't work with `crc32_z();’

You would need to read the documentation in zlib.h in order to be able to use its crc32() or crc32_z() correctly. From zlib.h:

   Usage example:

     uLong crc = crc32(0L, Z_NULL, 0);

     while (read_buffer(buffer, length) != EOF) {
       crc = crc32(crc, buffer, length);
     }
     if (crc != original_crc) error();

The initial value returned by crc32(0L, Z_NULL, 0) is 0, not -1. And there is no post inversion. To get your test code to work, it would need to be:

> crc = 0
> while (buffer, length = read_data()):
>     {
>     crc = crc32_z(crc, buffer, length)
>     }
This bug report was last modified 100 days ago.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.