GNU bug report logs - #23113
parallel gzip processes trash hard disks, need larger buffers

Previous Next

Full log

View this message in rfc822 format

From: Mark Adler <madler <at> alumni.caltech.edu>
To: "Chevreux, Bastien" <bastien.chevreux <at> dsm.com>
Cc: Jim Meyering <jim <at> meyering.net>, "23113 <at> debbugs.gnu.org" <23113 <at> debbugs.gnu.org>
Subject: bug#23113: parallel gzip processes trash hard disks, need larger buffers
Date: Tue, 12 Apr 2016 10:18:08 -0700

Bastien,

On Apr 12, 2016, at 9:55 AM, Chevreux, Bastien <bastien.chevreux <at> dsm.com> wrote:
> Questions: how stable / error proof is pigz compared to gzip? I always shied away from it as gzip is so much tried and tested that errors are unlikely ... and the zlib.net homepage does not make an "official" statement like "you should all now move to pigz, it's good and tested enough."

Certainly with -p 1, it is nothing more than a wrapper around zlib, which itself is extensively tested. With -p > 1 it uses threads, which has been tested on many systems successfully. Though I'd wonder about how portable it really is. Unfortunately I have no way to know how widely deployed and used pigz is. (Nor do I know how widely deployed and used gzip is, but pretty widely.)

> Additional question: is there a pigzlib planned? :-)

I have been toying with ideas about how to provide parallel support in zlib. At this point, I'm not sure what the interface should be.

Mark

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.