From unknown Sun Jun 22 11:42:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#37665: Concatenation of gzip compressed bytearray Resent-From: Zhu Li Original-Sender: "Debbugs-submit" Resent-CC: bug-gzip@gnu.org Resent-Date: Tue, 08 Oct 2019 15:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 37665 X-GNU-PR-Package: gzip X-GNU-PR-Keywords: To: 37665@debbugs.gnu.org X-Debbugs-Original-To: bug-gzip@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.157054989814202 (code B ref -1); Tue, 08 Oct 2019 15:52:02 +0000 Received: (at submit) by debbugs.gnu.org; 8 Oct 2019 15:51:38 +0000 Received: from localhost ([127.0.0.1]:51272 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iHrlx-0003gz-Mw for submit@debbugs.gnu.org; Tue, 08 Oct 2019 11:51:38 -0400 Received: from lists.gnu.org ([209.51.188.17]:48439) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iHrQ8-00037Z-KI for submit@debbugs.gnu.org; Tue, 08 Oct 2019 11:29:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59728) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHrQ7-0008Eu-D5 for bug-gzip@gnu.org; Tue, 08 Oct 2019 11:29:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iHrQ6-0007gj-2M for bug-gzip@gnu.org; Tue, 08 Oct 2019 11:29:03 -0400 Received: from mail-io1-xd2e.google.com ([2607:f8b0:4864:20::d2e]:43912) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iHrQ5-0007gK-Tt for bug-gzip@gnu.org; Tue, 08 Oct 2019 11:29:02 -0400 Received: by mail-io1-xd2e.google.com with SMTP id v2so37332839iob.10 for ; Tue, 08 Oct 2019 08:29:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Z/zTjEHrKHtRyg0yraM+SyTYXw4rLfN58bgidmpAC8s=; b=BKdruXA0V0nhvM6SEu+/PFCdd3ldD2zcdDTpwx9e1PuIrgKGJ6MLurDwTtRBHADrqg L97H/+XUMNClJ0gCI/5soY17tkMjr9lalMuzfIiaK82jIcqjvVAsR2sA+EbKD4Vs1jNL 42CGPDFaz3QlJKxQGFEqpMnkhqWJgJJfpL6hF643XHnQOd2wS8WDQ+vJGLHXlll6r4iw zUUCjn9W8qI5fkmWrmCikDWsfSUZgTOHmEJOnE32Ego53Cr8VKLt3OKPHtGSWgVIkMMF PyR5NDTs0uPZsswaJLQ900QCvNYdAm10gWMb+xTtTRiYCCfVej1O/Df0+8uYei2bFMNR Gb0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Z/zTjEHrKHtRyg0yraM+SyTYXw4rLfN58bgidmpAC8s=; b=UjOg6ggEULSpSsFqenVGxynH27TWw7FJmzY/6B/QDlAJq1wD+xik+xheUjdnZFQQqL xr59kXLF0PDK8SDkuv6OSM6nxFf0ggytx1ZXUpJB0YmJiLzom5NuZcp0Kurw21Nm8CVb F85BYFHtvuqn9WHTLRPi6z/xpLLZI17K6QYzpOZKBHGpImQhmOOWJdbBGFXKSRNfiIz0 0jWIVJBBDBJc47lMvpaA5ePKV3CpeIvN494gmrc/XI0JxASNywzfIrPYKp9hnk0UZr44 x5FIUtxS6zBrjTzvabR4LdTpnuAOS2vicWkXGgHixYO56am9jNItB5pNh8I8N4b0SJvD mtkw== X-Gm-Message-State: APjAAAWLaBEhwlyqLyHLNJTpCDC85LCL6W8r4Dc/j7/JfiE8GEHeZSUQ t4k1IRLam7pXORrr6iCYQqEMSsFj/cMtWR3M1rEyKM3NNeg= X-Google-Smtp-Source: APXvYqwj4W/e2iKpDpNrw90/s+x3s0VHHlppPl0fYWEwLfbFWswm6gsE95LErrKQrpXWfUAzjHK0tzxBEhjuVMLbLKw= X-Received: by 2002:a6b:2b91:: with SMTP id r139mr31120266ior.293.1570548540428; Tue, 08 Oct 2019 08:29:00 -0700 (PDT) MIME-Version: 1.0 From: Zhu Li Date: Tue, 8 Oct 2019 10:28:49 -0500 Message-ID: Content-Type: multipart/alternative; boundary="000000000000dadc93059467d339" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::d2e X-Spam-Score: 0.7 (/) X-Mailman-Approved-At: Tue, 08 Oct 2019 11:51:36 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --000000000000dadc93059467d339 Content-Type: text/plain; charset="UTF-8" Hello there, Our team has a use case where we want to concatenate gzip-compressed byte arrays. We do compression with GZIPOutputStream.java with Java 8 and decompression with GZIPInputStream.java. In a simple test case I compressed several instances of byte[] with GZIPOutputStream and output the compressed results to several instances of byte[], and then put all of them into a ByteBuffer. Then I tried to decompress the concatenated data in the final ByteBuffer with GZIPInputStream. And I found that sometimes this works as expected but other times the last few instances of input byte[] are ignored by GZIPInputStream(it seems to get EOF at the end of a compressed byte[], which is not the last compressed byte[] in the ByteBuffer) I was testing with the default setting. So does this mean that GZIPInputStream and GZIPOutputStream don't support such operation by design or is there something we can configure/change for this to work? Regards, Zhu --000000000000dadc93059467d339 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello there,

Our team has a use case wh= ere we want to concatenate gzip-compressed byte arrays. We do compression w= ith=C2=A0GZIPOutputStream.java with Java 8 and decompression with GZIPInput= Stream.java.=C2=A0
In a simple test case I compressed several ins= tances of byte[] with GZIPOutputStream and output the compressed results to= several instances of byte[], and then put all of them into a ByteBuffer. T= hen I tried to decompress the concatenated data in the final ByteBuffer wit= h GZIPInputStream.
And I found that sometimes this works as expec= ted but other times the last few instances of input byte[] are ignored by= =C2=A0GZIPInputStream(it seems to get EOF at the end of a compressed byte[]= , which is not the last compressed byte[] in the ByteBuffer) I was testing = with the default setting.
So does this mean that GZIPInputStream = and=C2=A0GZIPOutputStream don't support such operation by design or is = there something we can configure/change for this to work?

Regards,
Zhu

--000000000000dadc93059467d339-- From unknown Sun Jun 22 11:42:43 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Zhu Li Subject: bug#37665: closed (Re: bug#37665: Concatenation of gzip compressed bytearray) Message-ID: References: <427f9954-ace6-c8f1-d6ae-fa9cae019bd4@cs.ucla.edu> X-Gnu-PR-Message: they-closed 37665 X-Gnu-PR-Package: gzip Reply-To: 37665@debbugs.gnu.org Date: Tue, 08 Oct 2019 16:53:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1570553582-4043-1" This is a multi-part message in MIME format... ------------=_1570553582-4043-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #37665: Concatenation of gzip compressed bytearray which was filed against the gzip package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 37665@debbugs.gnu.org. --=20 37665: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D37665 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1570553582-4043-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 37665-done) by debbugs.gnu.org; 8 Oct 2019 16:52:48 +0000 Received: from localhost ([127.0.0.1]:51399 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iHsjA-00012m-FP for submit@debbugs.gnu.org; Tue, 08 Oct 2019 12:52:48 -0400 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:53268) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iHsj8-00012Q-6A for 37665-done@debbugs.gnu.org; Tue, 08 Oct 2019 12:52:47 -0400 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id D3750160615; Tue, 8 Oct 2019 09:52:39 -0700 (PDT) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id sBCLOqCRVSsM; Tue, 8 Oct 2019 09:52:39 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id F0F9B160612; Tue, 8 Oct 2019 09:52:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id FBAzk1zKffQL; Tue, 8 Oct 2019 09:52:38 -0700 (PDT) Received: from Penguin.CS.UCLA.EDU (Penguin.CS.UCLA.EDU [131.179.64.200]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id D7C2216060F; Tue, 8 Oct 2019 09:52:38 -0700 (PDT) Subject: Re: bug#37665: Concatenation of gzip compressed bytearray To: Zhu Li References: From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <427f9954-ace6-c8f1-d6ae-fa9cae019bd4@cs.ucla.edu> Date: Tue, 8 Oct 2019 09:52:31 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 37665-done Cc: 37665-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 10/8/19 8:28 AM, Zhu Li wrote: > does this mean that GZIPInputStream and GZIPOutputStream don't support > such operation by design Sorry, but you seem to be writing about some gzip-compatible software, not about gzip itself. Please write to whoever maintains that other software, instead of to bug-gzip. ------------=_1570553582-4043-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 8 Oct 2019 15:51:38 +0000 Received: from localhost ([127.0.0.1]:51272 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iHrlx-0003gz-Mw for submit@debbugs.gnu.org; Tue, 08 Oct 2019 11:51:38 -0400 Received: from lists.gnu.org ([209.51.188.17]:48439) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iHrQ8-00037Z-KI for submit@debbugs.gnu.org; Tue, 08 Oct 2019 11:29:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59728) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHrQ7-0008Eu-D5 for bug-gzip@gnu.org; Tue, 08 Oct 2019 11:29:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iHrQ6-0007gj-2M for bug-gzip@gnu.org; Tue, 08 Oct 2019 11:29:03 -0400 Received: from mail-io1-xd2e.google.com ([2607:f8b0:4864:20::d2e]:43912) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iHrQ5-0007gK-Tt for bug-gzip@gnu.org; Tue, 08 Oct 2019 11:29:02 -0400 Received: by mail-io1-xd2e.google.com with SMTP id v2so37332839iob.10 for ; Tue, 08 Oct 2019 08:29:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Z/zTjEHrKHtRyg0yraM+SyTYXw4rLfN58bgidmpAC8s=; b=BKdruXA0V0nhvM6SEu+/PFCdd3ldD2zcdDTpwx9e1PuIrgKGJ6MLurDwTtRBHADrqg L97H/+XUMNClJ0gCI/5soY17tkMjr9lalMuzfIiaK82jIcqjvVAsR2sA+EbKD4Vs1jNL 42CGPDFaz3QlJKxQGFEqpMnkhqWJgJJfpL6hF643XHnQOd2wS8WDQ+vJGLHXlll6r4iw zUUCjn9W8qI5fkmWrmCikDWsfSUZgTOHmEJOnE32Ego53Cr8VKLt3OKPHtGSWgVIkMMF PyR5NDTs0uPZsswaJLQ900QCvNYdAm10gWMb+xTtTRiYCCfVej1O/Df0+8uYei2bFMNR Gb0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Z/zTjEHrKHtRyg0yraM+SyTYXw4rLfN58bgidmpAC8s=; b=UjOg6ggEULSpSsFqenVGxynH27TWw7FJmzY/6B/QDlAJq1wD+xik+xheUjdnZFQQqL xr59kXLF0PDK8SDkuv6OSM6nxFf0ggytx1ZXUpJB0YmJiLzom5NuZcp0Kurw21Nm8CVb F85BYFHtvuqn9WHTLRPi6z/xpLLZI17K6QYzpOZKBHGpImQhmOOWJdbBGFXKSRNfiIz0 0jWIVJBBDBJc47lMvpaA5ePKV3CpeIvN494gmrc/XI0JxASNywzfIrPYKp9hnk0UZr44 x5FIUtxS6zBrjTzvabR4LdTpnuAOS2vicWkXGgHixYO56am9jNItB5pNh8I8N4b0SJvD mtkw== X-Gm-Message-State: APjAAAWLaBEhwlyqLyHLNJTpCDC85LCL6W8r4Dc/j7/JfiE8GEHeZSUQ t4k1IRLam7pXORrr6iCYQqEMSsFj/cMtWR3M1rEyKM3NNeg= X-Google-Smtp-Source: APXvYqwj4W/e2iKpDpNrw90/s+x3s0VHHlppPl0fYWEwLfbFWswm6gsE95LErrKQrpXWfUAzjHK0tzxBEhjuVMLbLKw= X-Received: by 2002:a6b:2b91:: with SMTP id r139mr31120266ior.293.1570548540428; Tue, 08 Oct 2019 08:29:00 -0700 (PDT) MIME-Version: 1.0 From: Zhu Li Date: Tue, 8 Oct 2019 10:28:49 -0500 Message-ID: Subject: Concatenation of gzip compressed bytearray To: bug-gzip@gnu.org Content-Type: multipart/alternative; boundary="000000000000dadc93059467d339" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::d2e X-Spam-Score: 0.7 (/) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Tue, 08 Oct 2019 11:51:36 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --000000000000dadc93059467d339 Content-Type: text/plain; charset="UTF-8" Hello there, Our team has a use case where we want to concatenate gzip-compressed byte arrays. We do compression with GZIPOutputStream.java with Java 8 and decompression with GZIPInputStream.java. In a simple test case I compressed several instances of byte[] with GZIPOutputStream and output the compressed results to several instances of byte[], and then put all of them into a ByteBuffer. Then I tried to decompress the concatenated data in the final ByteBuffer with GZIPInputStream. And I found that sometimes this works as expected but other times the last few instances of input byte[] are ignored by GZIPInputStream(it seems to get EOF at the end of a compressed byte[], which is not the last compressed byte[] in the ByteBuffer) I was testing with the default setting. So does this mean that GZIPInputStream and GZIPOutputStream don't support such operation by design or is there something we can configure/change for this to work? Regards, Zhu --000000000000dadc93059467d339 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello there,

Our team has a use case wh= ere we want to concatenate gzip-compressed byte arrays. We do compression w= ith=C2=A0GZIPOutputStream.java with Java 8 and decompression with GZIPInput= Stream.java.=C2=A0
In a simple test case I compressed several ins= tances of byte[] with GZIPOutputStream and output the compressed results to= several instances of byte[], and then put all of them into a ByteBuffer. T= hen I tried to decompress the concatenated data in the final ByteBuffer wit= h GZIPInputStream.
And I found that sometimes this works as expec= ted but other times the last few instances of input byte[] are ignored by= =C2=A0GZIPInputStream(it seems to get EOF at the end of a compressed byte[]= , which is not the last compressed byte[] in the ByteBuffer) I was testing = with the default setting.
So does this mean that GZIPInputStream = and=C2=A0GZIPOutputStream don't support such operation by design or is = there something we can configure/change for this to work?

Regards,
Zhu

--000000000000dadc93059467d339-- ------------=_1570553582-4043-1--