From unknown Fri Jun 20 07:17:29 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#33133 <33133@debbugs.gnu.org> To: bug#33133 <33133@debbugs.gnu.org> Subject: Status: 26.1.50; zlib-decompress-region too rigid Reply-To: bug#33133 <33133@debbugs.gnu.org> Date: Fri, 20 Jun 2025 14:17:29 +0000 retitle 33133 26.1.50; zlib-decompress-region too rigid reassign 33133 emacs submitter 33133 Katsumi Yamaoka severity 33133 normal tag 33133 fixed patch thanks From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 23 19:09:09 2018 Received: (at submit) by debbugs.gnu.org; 23 Oct 2018 23:09:10 +0000 Received: from localhost ([127.0.0.1]:38900 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gF5nR-0007yN-MY for submit@debbugs.gnu.org; Tue, 23 Oct 2018 19:09:09 -0400 Received: from eggs.gnu.org ([208.118.235.92]:52424) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gF5nO-0007xv-U6 for submit@debbugs.gnu.org; Tue, 23 Oct 2018 19:09:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gF5nE-0006ja-HT for submit@debbugs.gnu.org; Tue, 23 Oct 2018 19:09:01 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50, T_TVD_MIME_NO_HEADERS autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:58596) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gF5nE-0006is-1w for submit@debbugs.gnu.org; Tue, 23 Oct 2018 19:08:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33664) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gF5nC-00023f-IN for bug-gnu-emacs@gnu.org; Tue, 23 Oct 2018 19:08:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gF5n3-0006Vz-UR for bug-gnu-emacs@gnu.org; Tue, 23 Oct 2018 19:08:52 -0400 Received: from coral.maple.relay.mailchannels.net ([23.83.214.39]:42673) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gF5mx-0005e2-FS for bug-gnu-emacs@gnu.org; Tue, 23 Oct 2018 19:08:41 -0400 X-Sender-Id: tih5qno0ow|x-authuser|yamaoka@hampton.hostforweb.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 24D99282822; Tue, 23 Oct 2018 23:07:56 +0000 (UTC) Received: from hampton.hostforweb.net (unknown [100.96.19.78]) (Authenticated sender: tih5qno0ow) by relay.mailchannels.net (Postfix) with ESMTPA id 90CB9282C63; Tue, 23 Oct 2018 23:07:55 +0000 (UTC) X-Sender-Id: tih5qno0ow|x-authuser|yamaoka@hampton.hostforweb.net Received: from hampton.hostforweb.net (hampton.hostforweb.net [172.245.115.217]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.16.2); Tue, 23 Oct 2018 23:07:56 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: tih5qno0ow|x-authuser|yamaoka@hampton.hostforweb.net X-MailChannels-Auth-Id: tih5qno0ow X-Vacuous-Abaft: 111069091fa92dfb_1540336075988_2192594722 X-MC-Loop-Signature: 1540336075987:3337611917 X-MC-Ingress-Time: 1540336075987 Received: from s70.gtokyofl21.vectant.ne.jp ([202.215.75.70]:60000 helo=localhost) by hampton.hostforweb.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89_1) (envelope-from ) id 1gF5mC-009Pi0-41; Tue, 23 Oct 2018 18:07:53 -0500 Date: Wed, 24 Oct 2018 08:07:50 +0900 Message-ID: From: Katsumi Yamaoka To: bug-gnu-emacs@gnu.org Subject: 26.1.50; zlib-decompress-region too rigid Organization: Emacsen advocacy group User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1.50 (x86_64-unknown-cygwin) Cancel-Lock: sha1:cQKpMnMykhq6rg+1wFwf2GKc/GU= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-OutGoing-Spam-Status: No, score=-0.2 X-AuthUser: yamaoka@hampton.hostforweb.net X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit Cc: Kevin Ryde X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.1 (-----) --=-=-= Hi, Whereas `gzip -d' does, zlib-decompress-region doesn't decompress corrupted data of a certain kind. For instance, visiting https://www.gutenberg.org/no-such-page-exists using eww shows raw gzipped data. The data extracted is attached. As for `gzip -d', it says "unexpected end of file" in stderr. Here is a recipe to reproduce zlib-decompress-region not working: (let ((buffer (get-buffer-create "*testing*")) (coding-system-for-read 'binary) (cw (selected-window)) jka-compr-compression-info-list format-alist) (switch-to-buffer-other-window buffer) (erase-buffer) (set-buffer-multibyte nil) (insert-file-contents "/TEMP/corrupted-data.gz") (sit-for 1) (prog1 (zlib-decompress-region (point-min) (point-max)) (select-window cw))) If there is no prospect to improve zlib-decompress-region, how about this workaround? --- url-http.el~ 2018-09-12 07:48:16.110765500 +0000 +++ url-http.el 2018-10-23 23:04:48.060829900 +0000 @@ -951,7 +951,12 @@ (widen) (goto-char (point-min)) (when (search-forward "\n\n") - (zlib-decompress-region (point) (point-max))))))) + (or (zlib-decompress-region (point) (point-max)) + (let ((coding-system-for-write 'binary) + (coding-system-for-read 'binary) + (default-process-coding-system (cons 'binary 'binary))) + (zerop (call-process-region (point) (point-max) "gzip" + t '(t nil) nil "-d"))))))))) ;; Miscellaneous (defun url-http-activate-callback () In GNU Emacs 26.1.50 (build 1, x86_64-unknown-cygwin, GTK+ Version 3.22.28) of 2018-10-22 built on localhost Windowing system distributor 'The Cygwin/X Project', version 11.0.12001000 --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=corrupted-data.gz Content-Transfer-Encoding: base64 H4sIAAAAAAAAA8xYW2/bNhR+rn/FCYelLVpbtpMNaWNraJO0C9CmQZqh6FNAS5TEmhI1krLrDfvv OyRlWbKdZEBfFqANRZ7Lx3NnJgfnn85uv15fwO+3Hz/A9R9vP1yeAekHwZejsyA4vz33B8eD4SgI Lq4IkMyY8nUQLJfLwfJoIFUa3N4EmcnFcaCN4pEZxCYmYa83sZsgaJFOCStwB2CSMRrbBS5zZihY YX32Z8UXU3ImC8MK079dlYxA5L+mxLDvxsk/hSijSjMzvfz8qX9y8sur/oi0hRU0Z1MyZ6ulVLEm sP3TiGQzKef6JbjfL6H+8h+JYuwlyELwAn/TKuZyj5KY6Ujx0nBZkHuUHB0Nh8MXTp5XpGupe8RF gmrNEx7RbYmNuLKaCR6Rx4z32awE22vCSGvrFcfuwYNBsimhZSlqzcE3uqD+sNYEwBN4ZmQJB1PQ TCTP4e/eGhxuD4T0rAPFSkEjBs+etiIkrRDBjKnUBsrT56cNKxVMGaS9VvIbiwy8XxMC10Dh3c3F BapTC44Sl9xkcPUJcpYjhc54CcpeW7F4AJcJrGQFJeUxaJmjpeMVMKEZooOUGciYQkfmdI47Gcsh 5QvmWPCfglwWbAUzGs0PNvD+8VYKvCVCqK2GzpujZjEl2lpZZ4wZUhuxMTFkiiVTYtdBmfaHw/HA WX5bQiaViSoDPPIOr9kSurA7A/xv7WvDjWDhNU0ZXEkD72RVxNCHHctNAk9p8yzwiWaX1iC1pJgv wIXalOSsqGZUNV52ZzyeEiFTSdZU7iNsnDahYOiMFzH7PiUj0uzbOG0xtPfray35nAcfKS/u7D06 FA70lLyX1mGWBBxJOOF52qZ7QAn+aBWhmpJHzuqWwpm+LNJtyiWPTYbwx6/QV4ynGSbIyXCbigrc RhABbSwUoInqBNq1ZdtKW0f9VMmqhM2yL8u6HrYMWzNkI9Ky8fGQAI0ipjVWtSkZk661MJ53w2Bj QwySNw3zmNzji7Ch95ed6JIWbfyalSScHqbmFDMCz0LoAG9FxKiDNtmD9krCmdQGpIJ3WBhjmf+G qbsFNWmgJp6GhHv4HkLb7z8GddyBKv6TYT9gMSqwsOwAFg1g4UlIWNP+GMijDsj5HpDvKx4z21Q0 JGgbW194ke4inDcIM7k0sm8JLUhPfoPFIv9BrMcdrOpxrJW2qikoOcOqtoNYNYgdQUDCG0fYxroG 4jPz4fz73+bbvkt0P9alOZYFNWy7OD/p+OHVCYHeE6/Z9mFdN+KSrkoqBpHMg+8RRuk8mFkHINip F/vz8bDTrA9pXp5yw/I7N6KcWyJs8y+MfFGb4EVjAl+s4d5+8GRTdn8db6ru0cgftWu3h5nyZF+P cCAYFHJpvbHrCZOhp9MMjbu6psILd4W8ZrztnoddB5ThrsQCa432k0JcW+AA2v2w7eOG7XUt6K7Z ubuygu6+oqC7tSkR3zq6cEzJpWLAi0QCnUmcC9bq9N6rIm/40XJcIoe9xiQoHw6eDMsni+/p7Jt8 2O6tm0iqA4nPOA6i0oZIm66+yd5xzquG2Qpq5p3mfo/yVmDUnH0vC3v7sevt7ZhbN/bRsBVi4w7J vV29tdwZleoxWndnpXWBkfaxM3GcGEEuD+qjREiK82H7EugsD3uN7xjTtcZtlx7f2pjC0sCsEjM/ JYa9cxxy7eg6dymQUYPl2ihZpCHuToJ6DRFW8IyJEmsslErGVbR+gvTQCbgjk77COmoL8LfKttWi 7moUYrr6refK3etNfLf8X6ZxOSiYLcjn3D74ZpV17rUVamXigN7E4yM1bV2CHVd7gMpGW+MuzrOj R+XYQde9O8vws1RqhS86zCNbs93VsOVYEzj7IWzEjInli70N0qrApw8XdCbYoOfQW0Ff0CSYZami cUWFWEFVxpiWtnEJAeytf9YJxG/dUcBPo+EAn30voUdxSMc679QrabXgI0QuLKt7hljlWF6WkHBh N/UKAzsfNJp7WCucIzdFxrsQDa8ZVVHmuih1l7IWH6A5ehZvKTGP+F/MnVv9vMDoWbCCsyJiSNa7 Rahz34UXXHN3n53MxccvXiHD1+EK6mA52K0ye/yQSGm6/uwkU7lSNq67/bhsfwF8YTNAXAwacji0 y1MYD4dH/fFwNNw7n6FeqlbwBq1jX3kudlwVhcM8pjo7hTfothsrUOMkYR+Y+IZsAwnKx1tys5wE /nE1cX+fCHv/AgAA//8= --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 23 20:27:08 2018 Received: (at 33133) by debbugs.gnu.org; 24 Oct 2018 00:27:08 +0000 Received: from localhost ([127.0.0.1]:38945 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gF70u-0005PH-H7 for submit@debbugs.gnu.org; Tue, 23 Oct 2018 20:27:08 -0400 Received: from mail-it1-f176.google.com ([209.85.166.176]:34023) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gF70t-0005Os-2T for 33133@debbugs.gnu.org; Tue, 23 Oct 2018 20:27:07 -0400 Received: by mail-it1-f176.google.com with SMTP id e81-v6so2882668itc.1 for <33133@debbugs.gnu.org>; Tue, 23 Oct 2018 17:27:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=EEzXwbSIFbD+aSL5HevF0ZWUzEItGnHYg8rIAkpruyE=; b=POhnoupInDzkTCkhpHkSFmoFV8CGU6v8VcgZB4nXYZypUkJZ2qzBInhysqX7e+flbb BRCO4qAKXReT+fhmwQsW3UnBNItMXucranB4N6adr4jm9DDiv+vhEDDI7VdQ7UNu91T6 /kxxZVpjV7J88V/go5ORgDttcmSgtiRk60BAgI1ZzhSbI26CRWXHAu14RBrM0cW3eoXo 5vaSrrOZE2PRrw5lA4VhpI7g1r9doT9FCOv/80jPZQkizBT1OJuKYahNhMcn7j0je9DX 2i4YjmE7RHr8OYEp4SKQXhZfxroXLVxHZK4KSz3Ah25TFrQ8h7QUsbB3REdJV3viX9HL 6Vjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=EEzXwbSIFbD+aSL5HevF0ZWUzEItGnHYg8rIAkpruyE=; b=pcBMKU55S5kWDKeUWj/Fq47ZAkX5RPZTNH3T6Udgdh6Bw0P+zFA+tpXPUrCag35MKw O+neVNYGhg+2Iv8Ac74s8zO6XTcQf1agQsMqcRdV/dBV3XjiwxGbxg/5fcLm1HuRe3/e SpFBW1536qSuF9mfqI9lxlkZ1d/vsyhLn0dsXb9SaUk95+AxQy3Gn55OSrhuyTE6L7Us zXQrfCIoC0+V+lewuDHLrC7zoJvtq1Fa36C3r+WXmJbYty9aeUsiSyDxFjPGLXHBWWtA 9WoqEkdtUynLJHofOoVMDpDgojNOYEtnymDRgOgQWROSm1htx1qstoNIGCsry3gqMFZm YTrA== X-Gm-Message-State: AGRZ1gLRnMxADytR1FFICb4SDtXqWiGUZ8JLQXbyxQdvqbkrGZhJVQLm 7snTHl8EdajIDXNo3baTKZI= X-Google-Smtp-Source: AJdET5dQ45avk9OaawuuS0GgQaDfkOpfptgsYcPezBfE9/v+b54DrM84MkGUCVZawr35HiYLv9pK3w== X-Received: by 2002:a24:a10b:: with SMTP id y11-v6mr260903ite.148.1540340821571; Tue, 23 Oct 2018 17:27:01 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id 197-v6sm1102757iou.87.2018.10.23.17.27.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 Oct 2018 17:27:00 -0700 (PDT) From: Noam Postavsky To: Katsumi Yamaoka Subject: Re: bug#33133: 26.1.50; zlib-decompress-region too rigid References: Date: Tue, 23 Oct 2018 20:26:59 -0400 In-Reply-To: (Katsumi Yamaoka's message of "Wed, 24 Oct 2018 08:07:50 +0900") Message-ID: <87a7n4mbos.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33133 Cc: Kevin Ryde , 33133@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Katsumi Yamaoka writes: > Whereas `gzip -d' does, zlib-decompress-region doesn't decompress > corrupted data of a certain kind. For instance, visiting > > https://www.gutenberg.org/no-such-page-exists > > using eww shows raw gzipped data. The data extracted is attached. > As for `gzip -d', it says "unexpected end of file" in stderr. > If there is no prospect to improve zlib-decompress-region, how > about this workaround? We could have zlib-decompress-region ignore unexpected eof as well, e.g., the below (though it should obviously be enhanced to depend on a new parameter): --- i/src/decompress.c +++ w/src/decompress.c @@ -206,7 +206,7 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, } while (inflate_status == Z_OK); - if (inflate_status != Z_STREAM_END) + if (inflate_status != Z_STREAM_END && inflate_status != Z_BUF_ERROR) return unbind_to (count, Qnil); unwind_data.start = 0; From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 23 21:16:28 2018 Received: (at 33133) by debbugs.gnu.org; 24 Oct 2018 01:16:28 +0000 Received: from localhost ([127.0.0.1]:38979 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gF7me-0006Yf-3U for submit@debbugs.gnu.org; Tue, 23 Oct 2018 21:16:28 -0400 Received: from bonobo.maple.relay.mailchannels.net ([23.83.214.22]:4624) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gF7mb-0006YU-Dv for 33133@debbugs.gnu.org; Tue, 23 Oct 2018 21:16:26 -0400 X-Sender-Id: tih5qno0ow|x-authuser|yamaoka@hampton.hostforweb.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id D7D94502121; Wed, 24 Oct 2018 01:16:20 +0000 (UTC) Received: from hampton.hostforweb.net (unknown [100.96.26.166]) (Authenticated sender: tih5qno0ow) by relay.mailchannels.net (Postfix) with ESMTPA id 53C70501839; Wed, 24 Oct 2018 01:16:20 +0000 (UTC) X-Sender-Id: tih5qno0ow|x-authuser|yamaoka@hampton.hostforweb.net Received: from hampton.hostforweb.net (hampton.hostforweb.net [172.245.115.217]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.16.2); Wed, 24 Oct 2018 01:16:20 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: tih5qno0ow|x-authuser|yamaoka@hampton.hostforweb.net X-MailChannels-Auth-Id: tih5qno0ow X-Ruddy-Ruddy: 45b2524842cb0e12_1540343780732_1419938232 X-MC-Loop-Signature: 1540343780732:102784458 X-MC-Ingress-Time: 1540343780731 Received: from s70.gtokyofl21.vectant.ne.jp ([202.215.75.70]:60000 helo=localhost) by hampton.hostforweb.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89_1) (envelope-from ) id 1gF7mT-009umR-50; Tue, 23 Oct 2018 20:16:18 -0500 Date: Wed, 24 Oct 2018 10:16:16 +0900 Message-ID: From: Katsumi Yamaoka To: Noam Postavsky Subject: Re: bug#33133: 26.1.50; zlib-decompress-region too rigid References: <87a7n4mbos.fsf@gmail.com> Organization: Emacsen advocacy group X-Face: #kKnN,xUnmKia.'[pp`; Omh}odZK)?7wQSl"4o04=EixTF+V[""w~iNbM9ZL+.b*_CxUmFk B#Fu[*?MZZH@IkN:!"\w%I_zt>[$nm7nQosZ<3eu; B:$Q_:p!',P.c0-_Cy[dz4oIpw0ESA^D*1Lw= L&i*6&( User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (x86_64-unknown-cygwin) Cancel-Lock: sha1:Qo2Rzd+w62/tHdHM1fjtafiG3KE= MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-OutGoing-Spam-Status: No, score=-0.2 X-AuthUser: yamaoka@hampton.hostforweb.net X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33133 Cc: Kevin Ryde , 33133@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Tue, 23 Oct 2018 20:26:59 -0400, Noam Postavsky wrote: > We could have zlib-decompress-region ignore unexpected eof as well, > e.g., the below (though it should obviously be enhanced to depend on a > new parameter): > --- i/src/decompress.c > +++ w/src/decompress.c > @@ -206,7 +206,7 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, > while (inflate_status == Z_OK); > - if (inflate_status != Z_STREAM_END) > + if (inflate_status != Z_STREAM_END && inflate_status != Z_BUF_ERROR) > return unbind_to (count, Qnil); > unwind_data.start = 0; I confirmed that it makes it work for the corrupted web site in question. Thank you! From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 27 17:48:36 2018 Received: (at 33133) by debbugs.gnu.org; 27 Oct 2018 21:48:36 +0000 Received: from localhost ([127.0.0.1]:45869 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gGWRf-0004Fe-Tm for submit@debbugs.gnu.org; Sat, 27 Oct 2018 17:48:36 -0400 Received: from mail-it1-f180.google.com ([209.85.166.180]:55239) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gGWRe-0004FM-08; Sat, 27 Oct 2018 17:48:34 -0400 Received: by mail-it1-f180.google.com with SMTP id l191-v6so5460625ita.4; Sat, 27 Oct 2018 14:48:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=QCHdACLKeS8jzzr8o1vAVC4Bn4t/JajO+czw+Mm5V/8=; b=M24ashCPLlIc7mhvtcSck0NouLi1e358wZLglW/wWnLrKuoE3TGOl62k7mLvtwTmqJ /pu7fUFLgmGyWgoCsgVQLzoRE12IQGLlxW5KVv8RtV4udAV6HlBkvR4SONv78bCjwufB Ic8N2pjPjgi9m0RgLuU/ySufmMtCJ4kscKSXwpRynXZk/3aEATfIGBqaNDNpXFAv5GDb GcZuk3+cZHelO8iZe1K/CykmVRSDHXb3/MLM/NRWmknRfYe2ZelZogRXFi2fFVffXXrz rS3cGdtxj9k9nyL+enI6fAfvaLOkwm/nfTn96DEVQXNfIhOQBLArpY7XJzJwOJKVPkWu 29BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=QCHdACLKeS8jzzr8o1vAVC4Bn4t/JajO+czw+Mm5V/8=; b=ons+D0oalNnVQ8T3r7go9gCa+JZE49aAPBgNuXEgMtRrxVxXfuha/H8SW2AUV8mz5i lQkEyLRbDchws++u+VOZBhC5CFGjZYldOTnUeWZ9l1+LIrTkcndUbcEpH4tAKpDHsDjA hgK1eYRkDq3WGWRFTSpur5eE7jxSxJglvoH6Q/2TaT314g3azedc0jMHSYGlUnhmiizb Abx0W5PZOGhKhusCVx+3JrnRntfMMU9fTTQwmh+pEoqjo0CFC1LNie8SXbsk/pA5KEl5 dLDeGrCbAq+MR9WB8K97TbwHEJEcjdU/eySuRSjJpGez3zKde2Qo/MWxTWqfvRcyUnVe mc7Q== X-Gm-Message-State: AGRZ1gLhUGInG1KWaKdFGkJfLCQi12ueHyEnfPp2KaosQ+ZpyFr2leNO KAVF1vmhlfcxYbCnoKMMi5lFutK7 X-Google-Smtp-Source: AJdET5exkDXcHLoyvuEhC7FACp0w6QmcIjdVj2m3EM8WTgKIqmEM7Ll59NueDo2On+vO/YBitmV4mA== X-Received: by 2002:a02:c18:: with SMTP id g24-v6mr6457762jad.131.1540676908126; Sat, 27 Oct 2018 14:48:28 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id u68-v6sm3208552itd.1.2018.10.27.14.48.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 27 Oct 2018 14:48:27 -0700 (PDT) From: Noam Postavsky To: Katsumi Yamaoka Subject: Re: bug#33133: 26.1.50; zlib-decompress-region too rigid References: <87a7n4mbos.fsf@gmail.com> Date: Sat, 27 Oct 2018 17:48:26 -0400 In-Reply-To: (Katsumi Yamaoka's message of "Wed, 24 Oct 2018 10:16:16 +0900") Message-ID: <87efcbjc2d.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33133 Cc: Kevin Ryde , 33133@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain tags 33133 + patch quit Katsumi Yamaoka writes: > On Tue, 23 Oct 2018 20:26:59 -0400, Noam Postavsky wrote: >> --- i/src/decompress.c >> +++ w/src/decompress.c >> @@ -206,7 +206,7 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, > >> while (inflate_status == Z_OK); > >> - if (inflate_status != Z_STREAM_END) >> + if (inflate_status != Z_STREAM_END && inflate_status != Z_BUF_ERROR) >> return unbind_to (count, Qnil); > >> unwind_data.start = 0; > > I confirmed that it makes it work for the corrupted web site in > question. Thank you! Here's a proper patch. --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=v1-0001-Allow-partial-decompression-Bug-33133.patch Content-Description: patch >From 430ebd936b0bc41bd3e33e171938161846597196 Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Sat, 27 Oct 2018 17:45:52 -0400 Subject: [PATCH v1] Allow partial decompression (Bug#33133) * src/decompress.c (Fzlib_decompress_region): Add optional ALLOW-PARTIAL parameter. * lisp/url/url-http.el (url-handle-content-transfer-encoding): Use it. * doc/lispref/text.texi (Decompression): Document it. * etc/NEWS: Announce it. --- doc/lispref/text.texi | 10 ++++++---- etc/NEWS | 6 ++++++ lisp/url/url-http.el | 5 +++-- src/decompress.c | 22 +++++++++++++++++----- 4 files changed, 32 insertions(+), 11 deletions(-) diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi index 6c38d8eed0..e39ba6a192 100644 --- a/doc/lispref/text.texi +++ b/doc/lispref/text.texi @@ -4462,14 +4462,16 @@ Decompression available. @end defun -@defun zlib-decompress-region start end +@defun zlib-decompress-region start end &optional allow-partial This function decompresses the region between @var{start} and @var{end}, using built-in zlib decompression. The region should contain data that were compressed with gzip or zlib. On success, the function replaces the contents of the region with the decompressed -data. On failure, the function leaves the region unchanged and -returns @code{nil}. This function can be called only in unibyte -buffers. +data. If @var{allow-partial} is @code{nil}, on failure, the function +leaves the region unchanged and returns @code{nil}. Otherwise, it +returns the number of bytes that were not decompressed and replaces +the region text by whatever data was successfully decompressed. This +function can be called only in unibyte buffers. @end defun diff --git a/etc/NEWS b/etc/NEWS index 3f86195695..395169253d 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1159,6 +1159,12 @@ to mean that it is not known whether DST is in effect. 'json-insert', 'json-parse-string', and 'json-parse-buffer'. These are implemented in C using the Jansson library. ++++ +** 'zlib-decompress-region' can partially decompress corrupted data. +If the new optional ALLOW-PARTIAL argument is passed, then the data +that was decompressed successfully before failing will be inserted +into the buffer. + ** Mailcap --- diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 6b5749e1bc..94ac660fcf 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -939,7 +939,8 @@ url-http-parse-headers (goto-char (point-min)) success)) -(declare-function zlib-decompress-region "decompress.c" (start end)) +(declare-function zlib-decompress-region "decompress.c" + (start end &optional allow-partial)) (defun url-handle-content-transfer-encoding () (let ((encoding (mail-fetch-field "content-encoding"))) @@ -951,7 +952,7 @@ url-handle-content-transfer-encoding (widen) (goto-char (point-min)) (when (search-forward "\n\n") - (zlib-decompress-region (point) (point-max))))))) + (zlib-decompress-region (point) (point-max) t)))))) ;; Miscellaneous (defun url-http-activate-callback () diff --git a/src/decompress.c b/src/decompress.c index 2836338216..3872014739 100644 --- a/src/decompress.c +++ b/src/decompress.c @@ -120,12 +120,18 @@ DEFUN ("zlib-available-p", Fzlib_available_p, Szlib_available_p, 0, 0, 0, DEFUN ("zlib-decompress-region", Fzlib_decompress_region, Szlib_decompress_region, - 2, 2, 0, + 2, 3, 0, doc: /* Decompress a gzip- or zlib-compressed region. Replace the text in the region by the decompressed data. -On failure, return nil and leave the data in place. + +If optional parameter ALLOW-PARTIAL is nil or omitted, on failure, +return nil and leave the data in place. Otherwise, return the number +of bytes that were not decompressed and replace the region text by +whatever data was successfully decompressed. If decompression is +completely successful return t. + This function can be called only in unibyte buffers. */) - (Lisp_Object start, Lisp_Object end) + (Lisp_Object start, Lisp_Object end, Lisp_Object allow_partial) { ptrdiff_t istart, iend, pos_byte; z_stream stream; @@ -206,8 +212,14 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, } while (inflate_status == Z_OK); + Lisp_Object ret = Qt; if (inflate_status != Z_STREAM_END) - return unbind_to (count, Qnil); + { + if (!NILP (allow_partial)) + ret = make_int (iend - pos_byte); + else + return unbind_to (count, Qnil); + } unwind_data.start = 0; @@ -218,7 +230,7 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, signal_after_change (istart, iend - istart, unwind_data.nbytes); update_compositions (istart, istart, CHECK_HEAD); - return unbind_to (count, Qt); + return unbind_to (count, ret); } -- 2.11.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 28 11:41:27 2018 Received: (at 33133) by debbugs.gnu.org; 28 Oct 2018 15:41:27 +0000 Received: from localhost ([127.0.0.1]:49183 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gGnBv-0007tN-10 for submit@debbugs.gnu.org; Sun, 28 Oct 2018 11:41:27 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47210) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gGnBs-0007tA-Qq for 33133@debbugs.gnu.org; Sun, 28 Oct 2018 11:41:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gGnBi-0007rZ-32 for 33133@debbugs.gnu.org; Sun, 28 Oct 2018 11:41:18 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:49005) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gGnBe-0007pk-MR; Sun, 28 Oct 2018 11:41:12 -0400 Received: from [176.228.60.248] (port=3009 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gGnBe-00084l-8s; Sun, 28 Oct 2018 11:41:10 -0400 Date: Sun, 28 Oct 2018 17:41:17 +0200 Message-Id: <837ei2m63m.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <87efcbjc2d.fsf@gmail.com> (message from Noam Postavsky on Sat, 27 Oct 2018 17:48:26 -0400) Subject: Re: bug#33133: 26.1.50; zlib-decompress-region too rigid References: <87a7n4mbos.fsf@gmail.com> <87efcbjc2d.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33133 Cc: user42_kevin@yahoo.com.au, yamaoka@jpl.org, 33133@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Noam Postavsky > Date: Sat, 27 Oct 2018 17:48:26 -0400 > Cc: Kevin Ryde , 33133@debbugs.gnu.org > > Here's a proper patch. Thanks. I have a few comments: > +data. If @var{allow-partial} is @code{nil}, on failure, the function We usually say "nil or omitted" for optional arguments. Also, I'd say "then on failure, ...", otherwise this could be misinterpreted as if "on failure" qualifies the "is nil" part. Same comment regarding the doc string of the function. > +leaves the region unchanged and returns @code{nil}. Otherwise, it > +returns the number of bytes that were not decompressed and replaces > +the region text by whatever data was successfully decompressed. This > +function can be called only in unibyte buffers. Maybe it would make sense here to say that this emulates what 'gzip' does? > + Lisp_Object ret = Qt; > if (inflate_status != Z_STREAM_END) > - return unbind_to (count, Qnil); > + { > + if (!NILP (allow_partial)) > + ret = make_int (iend - pos_byte); > + else > + return unbind_to (count, Qnil); > + } Hmm... should we display a warning message, like gzip does? From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 30 20:25:19 2018 Received: (at 33133) by debbugs.gnu.org; 31 Oct 2018 00:25:19 +0000 Received: from localhost ([127.0.0.1]:56040 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gHeJy-0006iO-To for submit@debbugs.gnu.org; Tue, 30 Oct 2018 20:25:19 -0400 Received: from mail-it1-f170.google.com ([209.85.166.170]:55424) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gHeJx-0006iB-O4 for 33133@debbugs.gnu.org; Tue, 30 Oct 2018 20:25:18 -0400 Received: by mail-it1-f170.google.com with SMTP id e17so14363720itk.5 for <33133@debbugs.gnu.org>; Tue, 30 Oct 2018 17:25:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=2utDccJOQCkPTv2V4W6CZBGuav3alhfZv+4+EQAX5oc=; b=F4C87piCOzFvz042smDizjkxU5PNWX6LQgHjMTRwYYIIz0HeK6jnVH1am/iUUpRLQ8 IyrTYzqUJbRUY/PjzoZjFkDbrhdY628E9uU8eO0FcxVFvrq3iZxp/pUF37U+DgEHWVBc I+a9IpJHsbSmYwBNMNJcVqIZPwZBALj81jIrJtRxCS1jG2961feN7alSSx09rJfLNfJR ueTcvwLs1RveeDe25qd2hleEbrn5T0fpiNBoqr3sC+Fi1vNQ6dxmBXCEkOibvcgOxA8w qtiGG2y8Cy8Nurz6Ll6FKaqbc4Gv1kHQ583v64BJiuEOSjhaFWLvo1ZbtwGo63bUgZ3y WbxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=2utDccJOQCkPTv2V4W6CZBGuav3alhfZv+4+EQAX5oc=; b=E2wDLLAjhMu8BtX3WPyooyY6Fuu4Z77ypZ+zo5+/jGnWEr2191dhkmuKHYeWu8eNiY s0Wpr3pIKkaY6054At/jym8nwASRBxAEZNmGWwp5TTjRcU0CtP+F23kQQWLn4UrhIitx mElQCU/TAmrfVT7e+gZW1YqZV1k9TFCi68r0V98ItlSZmDQuIr25igfNj2sHgwpYl2zT tZ3UR//f/gGhGV2PBiEuBkoQxOHKron2wcZjckV6OdtQoyMfncj+bb7PVPjv5mlFFNnT gLmP8mJ1DBsVVNL1UUB/0yPZ5g2RfrJbAFzQ/apn0vcy3C4DgQ82rpZLpK+dlX3SZgoQ 31ew== X-Gm-Message-State: AGRZ1gI7P1IScYlXl5LqpZOhGiiimt/wfuwlTTkcpRZ6yMm5DXqtP/5n 94zKeKdE6R2gc1XamLs6swV4OQJO X-Google-Smtp-Source: AJdET5cGCDJZwMMuRmGCm1vJMjF9Z6vPhWyxu+z4PKXFr3Fp1tBjRCg3X5UrEQBu+OFCQJUThOv2uQ== X-Received: by 2002:a02:958a:: with SMTP id b10-v6mr727908jai.130.1540945512048; Tue, 30 Oct 2018 17:25:12 -0700 (PDT) Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id i17-v6sm8757700iog.56.2018.10.30.17.25.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 30 Oct 2018 17:25:11 -0700 (PDT) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#33133: 26.1.50; zlib-decompress-region too rigid References: <87a7n4mbos.fsf@gmail.com> <87efcbjc2d.fsf@gmail.com> <837ei2m63m.fsf@gnu.org> Date: Tue, 30 Oct 2018 20:25:10 -0400 In-Reply-To: <837ei2m63m.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 28 Oct 2018 17:41:17 +0200") Message-ID: <87pnvrhsih.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33133 Cc: user42_kevin@yahoo.com.au, yamaoka@jpl.org, 33133@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Eli Zaretskii writes: >> +data. If @var{allow-partial} is @code{nil}, on failure, the function > > We usually say "nil or omitted" for optional arguments. Also, I'd say > "then on failure, ...", otherwise this could be misinterpreted as if > "on failure" qualifies the "is nil" part. > > Same comment regarding the doc string of the function. Makes sense, done. >> +leaves the region unchanged and returns @code{nil}. Otherwise, it >> +returns the number of bytes that were not decompressed and replaces >> +the region text by whatever data was successfully decompressed. This >> +function can be called only in unibyte buffers. > > Maybe it would make sense here to say that this emulates what 'gzip' > does? Hmm, maybe. I've added a mention of this, not sure if it actually helps. >> + Lisp_Object ret = Qt; >> if (inflate_status != Z_STREAM_END) >> + { >> + if (!NILP (allow_partial)) >> + ret = make_int (iend - pos_byte); > Hmm... should we display a warning message, like gzip does? Not unconditionally, I'd say. In the example which prompted this bug thread, a warning would just be a nuisance. We could just leave it up to the caller to print a warning message if they want, e.g.: (unless (eq t (zlib-decompress-region START END t)) (message "Incomplete decompression")) Or perhaps instead of returning the byte count, return an error indicator which the caller could use to contruct a warning message (this could allow for a slightly more specific message)? Or maybe it's easier if the ALLOW-PARTIAL parameter could have another possible value to control display of the warning message? --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=v2-0001-Allow-partial-decompression-Bug-33133.patch Content-Description: patch >From 43a912181a4a30d826b2c016ca05b5c9d8daf3f4 Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Sat, 27 Oct 2018 17:45:52 -0400 Subject: [PATCH v2] Allow partial decompression (Bug#33133) * src/decompress.c (Fzlib_decompress_region): Add optional ALLOW-PARTIAL parameter. * lisp/url/url-http.el (url-handle-content-transfer-encoding): Use it. * doc/lispref/text.texi (Decompression): Document it. * etc/NEWS: Announce it. --- doc/lispref/text.texi | 11 +++++++---- etc/NEWS | 6 ++++++ lisp/url/url-http.el | 5 +++-- src/decompress.c | 22 +++++++++++++++++----- 4 files changed, 33 insertions(+), 11 deletions(-) diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi index 6c38d8eed0..9f241a6c3f 100644 --- a/doc/lispref/text.texi +++ b/doc/lispref/text.texi @@ -4462,14 +4462,17 @@ Decompression available. @end defun -@defun zlib-decompress-region start end +@defun zlib-decompress-region start end &optional allow-partial This function decompresses the region between @var{start} and @var{end}, using built-in zlib decompression. The region should contain data that were compressed with gzip or zlib. On success, the function replaces the contents of the region with the decompressed -data. On failure, the function leaves the region unchanged and -returns @code{nil}. This function can be called only in unibyte -buffers. +data. If @var{allow-partial} is @code{nil} or omitted, then on +failure, the function leaves the region unchanged and returns +@code{nil}. Otherwise, it returns the number of bytes that were not +decompressed and replaces the region text by whatever data was +successfully decompressed. This function can be called only in +unibyte buffers. @end defun diff --git a/etc/NEWS b/etc/NEWS index 3f86195695..395169253d 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1159,6 +1159,12 @@ to mean that it is not known whether DST is in effect. 'json-insert', 'json-parse-string', and 'json-parse-buffer'. These are implemented in C using the Jansson library. ++++ +** 'zlib-decompress-region' can partially decompress corrupted data. +If the new optional ALLOW-PARTIAL argument is passed, then the data +that was decompressed successfully before failing will be inserted +into the buffer. + ** Mailcap --- diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 6b5749e1bc..94ac660fcf 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -939,7 +939,8 @@ url-http-parse-headers (goto-char (point-min)) success)) -(declare-function zlib-decompress-region "decompress.c" (start end)) +(declare-function zlib-decompress-region "decompress.c" + (start end &optional allow-partial)) (defun url-handle-content-transfer-encoding () (let ((encoding (mail-fetch-field "content-encoding"))) @@ -951,7 +952,7 @@ url-handle-content-transfer-encoding (widen) (goto-char (point-min)) (when (search-forward "\n\n") - (zlib-decompress-region (point) (point-max))))))) + (zlib-decompress-region (point) (point-max) t)))))) ;; Miscellaneous (defun url-http-activate-callback () diff --git a/src/decompress.c b/src/decompress.c index 2836338216..dd55fd68e8 100644 --- a/src/decompress.c +++ b/src/decompress.c @@ -120,12 +120,18 @@ DEFUN ("zlib-available-p", Fzlib_available_p, Szlib_available_p, 0, 0, 0, DEFUN ("zlib-decompress-region", Fzlib_decompress_region, Szlib_decompress_region, - 2, 2, 0, + 2, 3, 0, doc: /* Decompress a gzip- or zlib-compressed region. Replace the text in the region by the decompressed data. -On failure, return nil and leave the data in place. + +If optional parameter ALLOW-PARTIAL is nil or omitted, then on +failure, return nil and leave the data in place. Otherwise, return +the number of bytes that were not decompressed and replace the region +text by whatever data was successfully decompressed (similar to gzip). +If decompression is completely successful return t. + This function can be called only in unibyte buffers. */) - (Lisp_Object start, Lisp_Object end) + (Lisp_Object start, Lisp_Object end, Lisp_Object allow_partial) { ptrdiff_t istart, iend, pos_byte; z_stream stream; @@ -206,8 +212,14 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, } while (inflate_status == Z_OK); + Lisp_Object ret = Qt; if (inflate_status != Z_STREAM_END) - return unbind_to (count, Qnil); + { + if (!NILP (allow_partial)) + ret = make_int (iend - pos_byte); + else + return unbind_to (count, Qnil); + } unwind_data.start = 0; @@ -218,7 +230,7 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, signal_after_change (istart, iend - istart, unwind_data.nbytes); update_compositions (istart, istart, CHECK_HEAD); - return unbind_to (count, Qt); + return unbind_to (count, ret); } -- 2.11.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 31 12:07:50 2018 Received: (at 33133) by debbugs.gnu.org; 31 Oct 2018 16:07:50 +0000 Received: from localhost ([127.0.0.1]:57404 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gHt25-0004G3-U1 for submit@debbugs.gnu.org; Wed, 31 Oct 2018 12:07:50 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56346) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gHt25-0004Fp-23 for 33133@debbugs.gnu.org; Wed, 31 Oct 2018 12:07:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gHt1v-00009D-LC for 33133@debbugs.gnu.org; Wed, 31 Oct 2018 12:07:43 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37653) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gHt1u-00008g-9w; Wed, 31 Oct 2018 12:07:38 -0400 Received: from [176.228.60.248] (port=1687 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gHt1t-0006V1-CC; Wed, 31 Oct 2018 12:07:37 -0400 Date: Wed, 31 Oct 2018 18:07:33 +0200 Message-Id: <83r2g6hzga.fsf@gnu.org> From: Eli Zaretskii To: Noam Postavsky In-reply-to: <87pnvrhsih.fsf@gmail.com> (message from Noam Postavsky on Tue, 30 Oct 2018 20:25:10 -0400) Subject: Re: bug#33133: 26.1.50; zlib-decompress-region too rigid References: <87a7n4mbos.fsf@gmail.com> <87efcbjc2d.fsf@gmail.com> <837ei2m63m.fsf@gnu.org> <87pnvrhsih.fsf@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33133 Cc: user42_kevin@yahoo.com.au, yamaoka@jpl.org, 33133@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) > From: Noam Postavsky > Cc: user42_kevin@yahoo.com.au, yamaoka@jpl.org, 33133@debbugs.gnu.org > Date: Tue, 30 Oct 2018 20:25:10 -0400 > > >From 43a912181a4a30d826b2c016ca05b5c9d8daf3f4 Mon Sep 17 00:00:00 2001 > From: Noam Postavsky > Date: Sat, 27 Oct 2018 17:45:52 -0400 > Subject: [PATCH v2] Allow partial decompression (Bug#33133) > > * src/decompress.c (Fzlib_decompress_region): Add optional > ALLOW-PARTIAL parameter. > * lisp/url/url-http.el (url-handle-content-transfer-encoding): Use it. > * doc/lispref/text.texi (Decompression): Document it. > * etc/NEWS: Announce it. Thanks, this LGTM. From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 02 22:09:34 2019 Received: (at 33133) by debbugs.gnu.org; 3 Apr 2019 02:09:34 +0000 Received: from localhost ([127.0.0.1]:41931 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hBVLK-0006pO-FZ for submit@debbugs.gnu.org; Tue, 02 Apr 2019 22:09:34 -0400 Received: from mail-qt1-f182.google.com ([209.85.160.182]:43547) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hBVLI-0006p5-0v; Tue, 02 Apr 2019 22:09:32 -0400 Received: by mail-qt1-f182.google.com with SMTP id v32so17482810qtc.10; Tue, 02 Apr 2019 19:09:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=Hqh2aBN9C893Tl73KBa0vefFDew12WtDd2c6RQx/fyI=; b=pF3snyYEA9HEo2+PHasR43YhGR8LPvvU5QvN7Isa6HjQtZ//i1+VRDofxE31Z09WEV wFSYNEXxF2TpJ/DT6J97IpMSAR0YhpEzonfSJ1Eu/xDCWq47Kl+QA7/uaNCJnWlrbj/e toMlsHkZmIMMtZvcRJWuIjC0iJkf6PpvXZ3jF6+qcvvNlNhSoBhqwgdZP+lxNzPCapAu awPgm1gVNy/JGLZUI1XJIfMs92sx++uBKl/DR9iEO7B42UE6LR5FnbijeuHJno8rYRdM WP5ydSYNC0g71ucdWr4/f7J9nNA5PiB9Dvk1h3ncy+IdH+sM/mNeKlXOHp4dvCLIkPAe FDHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=Hqh2aBN9C893Tl73KBa0vefFDew12WtDd2c6RQx/fyI=; b=SSGsoLtCnDEZSxQ+/HTj/5+9tG2eq7IVRhmH4gkVzuP4dtAVl6xmctT0pH0evG3Sir 7d8HJ+hjXPFUKIa9w9RrkuvVAlxwazLJ4UUEVn3Q1c1LPAEeNrZ1P+L+UOw6jdAefJBi ynjzPrQ/w2kMt+aqEKuyFB8gsnBHcKfEHS9pe8xF6O2Na/5pO0u1Mxrm4pGN2BmxvapS xixHVkOQBx1hVlh4N0xLIgSjAjPYsX7EtrAfv7PnOf7KtG621i5ottgrHhZFuWlzPXxN yns5YSDy6+b4f/QqY18uLfkOmdHQuQD8LwXKpGiYXdrgxAGS4MvhXTurmA5pVYPZ16mg 0+Lg== X-Gm-Message-State: APjAAAWYZan+txI8d+x6yZdJbctoP/GjnszVouVQOqrz8Z4Mr/ntlj2u 8E/rehf5pDCCZco16/RTQBSZzcaL X-Google-Smtp-Source: APXvYqwvxv0XAUFaZBMvLSY6MuzIKHpu8qRjwunrMqlGl+KdC9EsX1hlD/kVl4zikCE72fhUXUozrg== X-Received: by 2002:aed:3b9c:: with SMTP id r28mr63533445qte.22.1554257366431; Tue, 02 Apr 2019 19:09:26 -0700 (PDT) Received: from minid (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id p130sm7714980qka.96.2019.04.02.19.09.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Apr 2019 19:09:25 -0700 (PDT) From: Noam Postavsky To: Eli Zaretskii Subject: Re: bug#33133: 26.1.50; zlib-decompress-region too rigid References: <87a7n4mbos.fsf@gmail.com> <87efcbjc2d.fsf@gmail.com> <837ei2m63m.fsf@gnu.org> <87pnvrhsih.fsf@gmail.com> <83r2g6hzga.fsf@gnu.org> Date: Tue, 02 Apr 2019 22:09:24 -0400 In-Reply-To: <83r2g6hzga.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 31 Oct 2018 18:07:33 +0200") Message-ID: <87o95ng7qj.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33133 Cc: user42_kevin@yahoo.com.au, yamaoka@jpl.org, 33133@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) tags 33133 fixed close 33133 27.1 quit Eli Zaretskii writes: >> Subject: [PATCH v2] Allow partial decompression (Bug#33133) >> >> * src/decompress.c (Fzlib_decompress_region): Add optional >> ALLOW-PARTIAL parameter. >> * lisp/url/url-http.el (url-handle-content-transfer-encoding): Use it. >> * doc/lispref/text.texi (Decompression): Document it. >> * etc/NEWS: Announce it. > > Thanks, this LGTM. Pushed. [1: b36913d803]: 2019-04-02 22:02:32 -0400 Allow partial decompression (Bug#33133) https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=b36913d803ee22a314f2e0a27523fbadeb60dd2c From unknown Fri Jun 20 07:17:29 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 01 May 2019 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator