GNU bug report logs - #15380
Bug in gzip.c buffering, and two feature requests

Previous Next

Full log

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Martin Langhoff <martin.langhoff <at> gmail.com>
To: bug-gzip <at> gnu.org, Jim Meyering <jim <at> meyering.net>,
 Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Bug in gzip.c buffering, and two feature requests
Date: Sat, 14 Sep 2013 20:57:14 -0400

Hi gzip maintainers!

In the course of trying to use gzip as a stream-compressor with
Apache's "piped logs" feature I hit what I think is a bug in gzip.
Additionally, using gzip as a stream log compressor triggered two
related "feature requests".

For the time being, I have rolled my own mini-reimplementation of
gzip.c (in python, I was in a rush...) and the tool I will describe
below. However, I think these improvements belong in gzip proper.

Bug: gzip.c discards its buffers when it gets SIGTERM, with no effort
made to flush them to disk. Logging via gzip adds buffering, when
apache is stopped (or restarted, usually daily to rotate logs), apache
sends SIGTERM before it closes the fh; gzip discards everything it has
buffered.

[ Apache logs are highly compressible, so the loss is quite significant! ]

Feature requests:

 - for debugging / diagnostics of live services, gzip (when in
"stream" mode) should flush its buffers to disk when it receives a
SIGUSR1

 - zcat needs to support a "followtail" option, where it reads to the
end of an open gzipped file and then "follows" it (as in tail -f).
This tool would be one user of the "flush on SIGUSR1" feature.

Working implementations of "compresslog" (a gzip-stream-compressor)
and "ztail", implemented in Python, are available here:

   http://repo.or.cz/w/compresslog.git

Is there any interest in this? I am not familiar with gzip.c sources;
and hesitant to throw a lot of effort without hearing from the
maintainers.

cheers,



m
-- 
 martin.langhoff <at> gmail.com
 -  ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 ~ http://docs.moodle.org/en/User:Martin_Langhoff

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.