From unknown Thu Jun 19 14:05:55 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#47157 <47157@debbugs.gnu.org> To: bug#47157 <47157@debbugs.gnu.org> Subject: Status: =?UTF-8?Q?=E2=80=9CBad?= Read-Header-Line header: =?UTF-8?Q?#=E2=80=9D?= while substituting Reply-To: bug#47157 <47157@debbugs.gnu.org> Date: Thu, 19 Jun 2025 21:05:55 +0000 retitle 47157 =E2=80=9CBad Read-Header-Line header: #=E2=80=9D while s= ubstituting reassign 47157 guix submitter 47157 Ludovic Court=C3=A8s severity 47157 serious thanks From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 15 10:26:55 2021 Received: (at submit) by debbugs.gnu.org; 15 Mar 2021 14:26:55 +0000 Received: from localhost ([127.0.0.1]:36313 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLoBL-0002bK-5r for submit@debbugs.gnu.org; Mon, 15 Mar 2021 10:26:55 -0400 Received: from lists.gnu.org ([209.51.188.17]:47450) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLoBJ-0002b7-0t for submit@debbugs.gnu.org; Mon, 15 Mar 2021 10:26:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54890) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLoBI-0000cE-Pn for bug-guix@gnu.org; Mon, 15 Mar 2021 10:26:52 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:23639) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLoBE-0005m1-DO for bug-guix@gnu.org; Mon, 15 Mar 2021 10:26:52 -0400 X-IronPort-AV: E=Sophos;i="5.81,249,1610406000"; d="scan'208";a="498019878" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Mar 2021 15:26:43 +0100 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Subject: =?utf-8?Q?=E2=80=9CBad?= Read-Header-Line header: =?utf-8?Q?#=3Ce?= =?utf-8?Q?of=3E=E2=80=9D?= while substituting X-Debbugs-Cc: Christopher Baines X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 25 =?utf-8?Q?Vent=C3=B4se?= an 229 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 15 Mar 2021 15:26:42 +0100 Message-ID: <87eeggh4rh.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.83; envelope-from=ludovic.courtes@inria.fr; helo=mail2-relais-roc.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) As reported by a few people on IRC, =E2=80=98guix substitute=E2=80=99 somet= imes fails in a way that I just experienced (from 8154beffd8c121e953a7c4cd75c3eebfcc073a9a): --8<---------------cut here---------------start------------->8--- downloading from https://ci.guix.gnu.org/nar/gzip/0bji0q5n59595xaqkqrp2gv52= lbz55xz-libpng-1.6.37 . libpng-1.6.37 275KiB 11.0MiB/s 00:00 [###= ###############] 100.0% downloading from https://ci.guix.gnu.org/nar/lzip/h3a5ygxxh4gakhnl53mq7z9b4= 3l8z05g-python-minimal. python-minimal-wrapper-3.8.2 351B 293KiB/s 00:00 [###= ###############] 100.0% downloading from https://ci.guix.gnu.org/nar/lzip/h8j09yb5d8dh3jffvpzawxsli= g9bwhdr-freetype-2.10.. freetype-2.10.4 600KiB 3.0MiB/s 00:00 [###= ###############] 100.0% building /gnu/store/2wfzazqz9g5xizi4vq4pv75nkh1m24bp-perl-5.30.2.drv... Backtrace: In guix/ui.scm: 2164:12 19 (run-guix-command _ . _) In guix/scripts/substitute.scm: 691:2 18 (guix-substitute . _) In unknown file: 17 (with-continuation-barrier #) In ice-9/boot-9.scm: 1736:10 16 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) In unknown file: 15 (apply-smob/0 #) In ice-9/boot-9.scm: 1736:10 14 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) 1736:10 13 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) 1731:15 12 (with-exception-handler # _) In guix/scripts/substitute.scm: 740:17 11 (_) 434:7 10 (process-substitution _ "/gnu/store/ns00dyapjbq9037dwrxa7hc31d= vir00n-grub-minimal-2.) In ice-9/boot-9.scm: 1736:10 9 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) In guix/scripts/substitute.scm: 443:9 8 (_) In ice-9/boot-9.scm: 1731:15 7 (with-exception-handler # _) 1669:16 6 (raise-exception _ #:continuable? _) 1667:16 5 (raise-exception _ #:continuable? _) 1669:16 4 (raise-exception _ #:continuable? _) 1764:13 3 (_ #<&compound-exception components: (#<&error> #<&irritants i= rritants: (read-header) 1669:16 2 (raise-exception _ #:continuable? _) 1667:16 1 (raise-exception _ #:continuable? _) 1669:16 0 (raise-exception _ #:continuable? _) ice-9/boot-9.scm:1669:16: In procedure raise-exception: Bad Read-Header-Line header: # --8<---------------cut here---------------end--------------->8--- This is the kind of issue that =E2=80=98with-cached-connection=E2=80=99 as = it can be seen in 9158020d7853b6e7925802e0d0a082801c680e8f avoided: --8<---------------cut here---------------start------------->8--- (define* (call-with-cached-connection uri proc #:optional (open-connection open-connection-for-uri/cached)) (let ((port (open-connection uri))) (catch #t (lambda () (proc port)) (lambda (key . args) ;; If PORT was cached and the server closed the connection in the ;; meantime, we get EPIPE. In that case, open a fresh connection a= nd ;; retry. We might also get 'bad-response or a similar exception f= rom ;; (web response) later on, once we've sent the request, or a ;; ERROR/INVALID-SESSION from GnuTLS. (if (or (and (eq? key 'system-error) (=3D EPIPE (system-error-errno `(,key ,@args)))) (and (eq? key 'gnutls-error) (eq? (first args) error/invalid-session)) (memq key '(bad-response bad-header bad-header-component))) (proc (open-connection uri #:fresh? #t)) (apply throw key args)))))) --8<---------------cut here---------------end--------------->8--- I think 7b812f7c84c43455cdd68a0e51b6ded018afcc8e and subsequent commits may have caused this regression. In particular, in 20c08a8a45d0f137ead7c05e720456b2aea44402, =E2=80=98call-with-connection-error-handling=E2=80=99 is now used, but that= one doesn=E2=80=99t catch the exceptions mentioned above, in this case =E2=80=98bad-header=E2= =80=99. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 15 10:32:24 2021 Received: (at control) by debbugs.gnu.org; 15 Mar 2021 14:32:24 +0000 Received: from localhost ([127.0.0.1]:36320 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLoGd-0002lJ-UM for submit@debbugs.gnu.org; Mon, 15 Mar 2021 10:32:24 -0400 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:25413) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLoGc-0002l5-BG for control@debbugs.gnu.org; Mon, 15 Mar 2021 10:32:22 -0400 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AMx9AI66X1+cO059C2wPXwB/XdLJzesId70hD?= =?us-ascii?q?6mlaTxtJfsuE0/20lPMA2hPuzBoXUncsmdePUZPhfVr385lp7Y4NeaqzRQWOgg?= =?us-ascii?q?qVBaxr8IeK+VbdMgLk8Oo178tdWoxfLPG1ElRgl8b952CDfeoI5NWc6qiniaP/?= =?us-ascii?q?wh5WJz1CUK1r4wdnBgvzKCQfeCB9GZE7GJCAj/ApmxOcfx0sA/iGOg=3D=3D?= X-IronPort-AV: E=Sophos;i="5.81,249,1610406000"; d="scan'208";a="498021765" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Mar 2021 15:31:44 +0100 Date: Mon, 15 Mar 2021 15:31:44 +0100 Message-Id: <87czw0h4j3.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #47157 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) severity 47157 serious quit From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 15 13:02:40 2021 Received: (at 47157) by debbugs.gnu.org; 15 Mar 2021 17:02:40 +0000 Received: from localhost ([127.0.0.1]:36657 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLqc3-0000dR-W2 for submit@debbugs.gnu.org; Mon, 15 Mar 2021 13:02:40 -0400 Received: from mira.cbaines.net ([212.71.252.8]:33530) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLqc2-0000dJ-5o for 47157@debbugs.gnu.org; Mon, 15 Mar 2021 13:02:38 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 5AC1327BC52; Mon, 15 Mar 2021 17:02:37 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 7cf3fda7; Mon, 15 Mar 2021 17:02:36 +0000 (UTC) References: <87eeggh4rh.fsf@inria.fr> User-agent: mu4e 1.4.15; emacs 27.1 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#47157: =?utf-8?Q?=E2=80=9CBad?= Read-Header-Line header: =?utf-8?Q?#=3Ceof=3E=E2=80=9D?= while substituting In-reply-to: <87eeggh4rh.fsf@inria.fr> Date: Mon, 15 Mar 2021 17:02:34 +0000 Message-ID: <871rcgfiz9.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 47157 Cc: 47157@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > As reported by a few people on IRC, =E2=80=98guix substitute=E2=80=99 som= etimes fails in > a way that I just experienced (from > 8154beffd8c121e953a7c4cd75c3eebfcc073a9a): > > --8<---------------cut here---------------start------------->8--- > downloading from https://ci.guix.gnu.org/nar/gzip/0bji0q5n59595xaqkqrp2gv= 52lbz55xz-libpng-1.6.37 . > libpng-1.6.37 275KiB 11.0MiB/s 00:00 [#= #################] 100.0% > > downloading from https://ci.guix.gnu.org/nar/lzip/h3a5ygxxh4gakhnl53mq7z9= b43l8z05g-python-minimal. > python-minimal-wrapper-3.8.2 351B 293KiB/s 00:00 [#= #################] 100.0% > > downloading from https://ci.guix.gnu.org/nar/lzip/h8j09yb5d8dh3jffvpzawxs= lig9bwhdr-freetype-2.10.. > freetype-2.10.4 600KiB 3.0MiB/s 00:00 [#= #################] 100.0% > > building /gnu/store/2wfzazqz9g5xizi4vq4pv75nkh1m24bp-perl-5.30.2.drv... > Backtrace: > In guix/ui.scm: > 2164:12 19 (run-guix-command _ . _) > In guix/scripts/substitute.scm: > 691:2 18 (guix-substitute . _) > In unknown file: > 17 (with-continuation-barrier #) > In ice-9/boot-9.scm: > 1736:10 16 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) > In unknown file: > 15 (apply-smob/0 #) > In ice-9/boot-9.scm: > 1736:10 14 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) > 1736:10 13 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) > 1731:15 12 (with-exception-handler # _) > In guix/scripts/substitute.scm: > 740:17 11 (_) > 434:7 10 (process-substitution _ "/gnu/store/ns00dyapjbq9037dwrxa7hc3= 1dvir00n-grub-minimal-2.) > In ice-9/boot-9.scm: > 1736:10 9 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) > In guix/scripts/substitute.scm: > 443:9 8 (_) > In ice-9/boot-9.scm: > 1731:15 7 (with-exception-handler # _) > 1669:16 6 (raise-exception _ #:continuable? _) > 1667:16 5 (raise-exception _ #:continuable? _) > 1669:16 4 (raise-exception _ #:continuable? _) > 1764:13 3 (_ #<&compound-exception components: (#<&error> #<&irritants= irritants: (read-header) > 1669:16 2 (raise-exception _ #:continuable? _) > 1667:16 1 (raise-exception _ #:continuable? _) > 1669:16 0 (raise-exception _ #:continuable? _) > > ice-9/boot-9.scm:1669:16: In procedure raise-exception: > Bad Read-Header-Line header: # > --8<---------------cut here---------------end--------------->8--- > > This is the kind of issue that =E2=80=98with-cached-connection=E2=80=99 a= s it can be > seen in 9158020d7853b6e7925802e0d0a082801c680e8f avoided: > > --8<---------------cut here---------------start------------->8--- > (define* (call-with-cached-connection uri proc > #:optional > (open-connection > open-connection-for-uri/cached)) > (let ((port (open-connection uri))) > (catch #t > (lambda () > (proc port)) > (lambda (key . args) > ;; If PORT was cached and the server closed the connection in the > ;; meantime, we get EPIPE. In that case, open a fresh connection= and > ;; retry. We might also get 'bad-response or a similar exception= from > ;; (web response) later on, once we've sent the request, or a > ;; ERROR/INVALID-SESSION from GnuTLS. > (if (or (and (eq? key 'system-error) > (=3D EPIPE (system-error-errno `(,key ,@args)))) > (and (eq? key 'gnutls-error) > (eq? (first args) error/invalid-session)) > (memq key '(bad-response bad-header bad-header-component)= )) > (proc (open-connection uri #:fresh? #t)) > (apply throw key args)))))) > --8<---------------cut here---------------end--------------->8--- > > I think 7b812f7c84c43455cdd68a0e51b6ded018afcc8e and subsequent commits > may have caused this regression. In particular, in > 20c08a8a45d0f137ead7c05e720456b2aea44402, > =E2=80=98call-with-connection-error-handling=E2=80=99 is now used, but th= at one doesn=E2=80=99t > catch the exceptions mentioned above, in this case =E2=80=98bad-header=E2= =80=99. I think the behaviour changed unintentionally with [1], however, thinking about the connection reuse in process-substitution compared with http-multiple-get, there's no attempt here to look at if the server has specified whether the connection should be closed. 1: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3Df50f5751fff4cfc6= d5abba9681054569694b7a5c Just like http-multiple-get, it's probably worth trying to check the headers of the response, look at whether the server has indicated that the connection should be closed, and if so, close the connection, forcing a new one to be established for future requests. I haven't tested this theory, but maybe if that happened, then some occurrences of trying to read a response, and not being able to would be prevented. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmBPkypfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XcL/hAAlXF3sB5OW9jZtO5eGKwDnRqIxQfmSDXa WNEDujhBYdKQCy8rsab5FBhE2A4WoTpHCbDWrRH/eXlPTCSd1Z+FO80zsShvD7O5 qtM4vBXZsOCI5Fze+jUEx5P7MCDiszOQ3BSfR2yp43TAD33c6W/Ax20uXklDuSff 4CQXNUnYtwItAB1i+LYO1Cde1jIXcUwwx9QJZY77HMaWqdvOkMaLWXYPA/eR/D9k 34Er/E0AFbuoLKj0keVs52pVfgwEvVcL5RZDiU9hEB3vPEAyFrnjwY1kjv55UmZl Uehu21hK4HVBzuaaJCNENAdUjzYkuaMlMAgjmHMFv2uyBGChq6EIje2XsNN/NGs1 gpiC0naNCWxit1FhnymQxfcxlSwnE1WbNLEGulaR4k0BJwC+K4jLW/yuo9iFF64g U7DKxgqeqCXkWID9V6jkoxd3/N0zsrdoqPPx7/n4wHwRpSP/8bJwp5/NHtTmSN42 d5OAqOL8dJzBOmxOqkrqJAuiO9AX3IGwnT0fa38mTH0zGemfkHSB+7UFUhET4eRv eKz7QIvZeDeCpSRJ485qBbdgdZrr4gEoh5TaM5eZJdiEjrOub1cJVf80OtAn+75Y biJ0O/AxTZg6Ya7bYO/TcZbIzNE6gSTLmLX20f01AjGVXpPklCXmStDTEyMyaDtW OzLyw1b6q1w= =w5/u -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 15 15:54:02 2021 Received: (at 47157) by debbugs.gnu.org; 15 Mar 2021 19:54:02 +0000 Received: from localhost ([127.0.0.1]:37245 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLtHt-0007jt-Oz for submit@debbugs.gnu.org; Mon, 15 Mar 2021 15:54:02 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36268) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLtHr-0007jk-7M for 47157@debbugs.gnu.org; Mon, 15 Mar 2021 15:54:00 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 488A427BC52; Mon, 15 Mar 2021 19:53:58 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 0b9d6fe2; Mon, 15 Mar 2021 19:53:57 +0000 (UTC) References: <87eeggh4rh.fsf@inria.fr> <871rcgfiz9.fsf@cbaines.net> User-agent: mu4e 1.4.15; emacs 27.1 From: Christopher Baines To: 47157@debbugs.gnu.org Subject: Re: bug#47157: =?utf-8?Q?=E2=80=9CBad?= Read-Header-Line header: =?utf-8?Q?#=3Ceof=3E=E2=80=9D?= while substituting In-reply-to: <871rcgfiz9.fsf@cbaines.net> Date: Mon, 15 Mar 2021 19:53:54 +0000 Message-ID: <87v99sdwh9.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 47157 Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Christopher Baines writes: > Ludovic Court=C3=A8s writes: > >> As reported by a few people on IRC, =E2=80=98guix substitute=E2=80=99 so= metimes fails in >> a way that I just experienced (from >> 8154beffd8c121e953a7c4cd75c3eebfcc073a9a): >> >> --8<---------------cut here---------------start------------->8--- >> downloading from https://ci.guix.gnu.org/nar/gzip/0bji0q5n59595xaqkqrp2g= v52lbz55xz-libpng-1.6.37 . >> libpng-1.6.37 275KiB 11.0MiB/s 00:00 [= ##################] 100.0% >> >> downloading from https://ci.guix.gnu.org/nar/lzip/h3a5ygxxh4gakhnl53mq7z= 9b43l8z05g-python-minimal. >> python-minimal-wrapper-3.8.2 351B 293KiB/s 00:00 [= ##################] 100.0% >> >> downloading from https://ci.guix.gnu.org/nar/lzip/h8j09yb5d8dh3jffvpzawx= slig9bwhdr-freetype-2.10.. >> freetype-2.10.4 600KiB 3.0MiB/s 00:00 [= ##################] 100.0% >> >> building /gnu/store/2wfzazqz9g5xizi4vq4pv75nkh1m24bp-perl-5.30.2.drv... >> Backtrace: >> In guix/ui.scm: >> 2164:12 19 (run-guix-command _ . _) >> In guix/scripts/substitute.scm: >> 691:2 18 (guix-substitute . _) >> In unknown file: >> 17 (with-continuation-barrier #) >> In ice-9/boot-9.scm: >> 1736:10 16 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) >> In unknown file: >> 15 (apply-smob/0 #) >> In ice-9/boot-9.scm: >> 1736:10 14 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) >> 1736:10 13 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) >> 1731:15 12 (with-exception-handler # _) >> In guix/scripts/substitute.scm: >> 740:17 11 (_) >> 434:7 10 (process-substitution _ "/gnu/store/ns00dyapjbq9037dwrxa7hc= 31dvir00n-grub-minimal-2.) >> In ice-9/boot-9.scm: >> 1736:10 9 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) >> In guix/scripts/substitute.scm: >> 443:9 8 (_) >> In ice-9/boot-9.scm: >> 1731:15 7 (with-exception-handler # _) >> 1669:16 6 (raise-exception _ #:continuable? _) >> 1667:16 5 (raise-exception _ #:continuable? _) >> 1669:16 4 (raise-exception _ #:continuable? _) >> 1764:13 3 (_ #<&compound-exception components: (#<&error> #<&irritant= s irritants: (read-header) >> 1669:16 2 (raise-exception _ #:continuable? _) >> 1667:16 1 (raise-exception _ #:continuable? _) >> 1669:16 0 (raise-exception _ #:continuable? _) >> >> ice-9/boot-9.scm:1669:16: In procedure raise-exception: >> Bad Read-Header-Line header: # >> --8<---------------cut here---------------end--------------->8--- >> >> This is the kind of issue that =E2=80=98with-cached-connection=E2=80=99 = as it can be >> seen in 9158020d7853b6e7925802e0d0a082801c680e8f avoided: >> >> --8<---------------cut here---------------start------------->8--- >> (define* (call-with-cached-connection uri proc >> #:optional >> (open-connection >> open-connection-for-uri/cached)) >> (let ((port (open-connection uri))) >> (catch #t >> (lambda () >> (proc port)) >> (lambda (key . args) >> ;; If PORT was cached and the server closed the connection in the >> ;; meantime, we get EPIPE. In that case, open a fresh connectio= n and >> ;; retry. We might also get 'bad-response or a similar exceptio= n from >> ;; (web response) later on, once we've sent the request, or a >> ;; ERROR/INVALID-SESSION from GnuTLS. >> (if (or (and (eq? key 'system-error) >> (=3D EPIPE (system-error-errno `(,key ,@args)))) >> (and (eq? key 'gnutls-error) >> (eq? (first args) error/invalid-session)) >> (memq key '(bad-response bad-header bad-header-component= ))) >> (proc (open-connection uri #:fresh? #t)) >> (apply throw key args)))))) >> --8<---------------cut here---------------end--------------->8--- >> >> I think 7b812f7c84c43455cdd68a0e51b6ded018afcc8e and subsequent commits >> may have caused this regression. In particular, in >> 20c08a8a45d0f137ead7c05e720456b2aea44402, >> =E2=80=98call-with-connection-error-handling=E2=80=99 is now used, but t= hat one doesn=E2=80=99t >> catch the exceptions mentioned above, in this case =E2=80=98bad-header= =E2=80=99. > > I think the behaviour changed unintentionally with [1], however, > thinking about the connection reuse in process-substitution compared > with http-multiple-get, there's no attempt here to look at if the server > has specified whether the connection should be closed. > > 1: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3Df50f5751fff4cf= c6d5abba9681054569694b7a5c > > Just like http-multiple-get, it's probably worth trying to check the > headers of the response, look at whether the server has indicated that > the connection should be closed, and if so, close the connection, > forcing a new one to be established for future requests. > > I haven't tested this theory, but maybe if that happened, then some > occurrences of trying to read a response, and not being able to would be > prevented. I've now actually got around to testing this, I'm no expert at running the substitute script manually without the guix-daemon, but I gave it a go, using a local NGinx instance which just allowed two requests per connection. With these changes [2], connections were closed when appropriate and a new one is established when the next nar is fetched. 2: https://issues.guix.gnu.org/47174 When testing using the same approach with master, I get this exception [3], so either I'm doing something wrong, or this isn't a circumstance under which there can be a bad-header issue. Regardless, I'm not sure what's going on with this exception below. 3: ice-9/boot-9.scm:1669:16: In procedure raise-exception: Wrong number of values returned to continuation (expected 2) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmBPu1JfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xfh+w/9EbWSWeht2vYbWh/5lPi5itYR2PABmcKH JTe26bliASrUcDCuYEpL531rahykntPMAwQinJXf7Lx5gfWJquVuw74u+XhWOee9 bOcqxsN+UPLuXXOVa6I+ExN9pdUG4csuRMbJDHM9TGCZsMC/rvFv+olVRUBWO7Ai xMJcsJklo9w6pMhlaI6ar5ytGC7aOxbUZP57EGuwqofcKMt4Yv0MvmpeYdMPupy2 lf/SnCQ4guojZ/4JQqCJtm0164StGKbILdvTAmD0SNSv9P6f1lFwqIiHHHoFuEUC fIxBRt2blDd7LAqbWzvLzK3hUtGGXCKGNVXVOCO8EjDtxU9sUxchSvUb9yhms+dO gfBO07UA7s1AeqpJGzx88K6Rpsk4/esy7IACS1k8RPt38Dxp98VVj+TwAB7pg18u g985OhK/GLSRMz/efpAcRw9g1PCnmodN0Yco0+ekaYnAT0iPWmK+XyNCU3JALFmu aks+qVDWJIuFUtSVLOeoF2H6lZoxkHE0JLFW2Z2WbZipooNZQgIPLhzovOWf0yNv w1zXSnjVqKQb3FxA0k8YygoWVfLVJvxKFam9+S1QbWaGR0MpcjmlzhkYEueMp3Ut kVsOqrA2+TDN589LBVSnmTEfxN2nyHVPb57iz1mVTPWeLY76h7D/Rsf3STEiZCxp fSkzfjyZWhY= =Fb25 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 15 16:30:26 2021 Received: (at 47157) by debbugs.gnu.org; 15 Mar 2021 20:30:27 +0000 Received: from localhost ([127.0.0.1]:37302 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLtr8-0000Cb-KR for submit@debbugs.gnu.org; Mon, 15 Mar 2021 16:30:26 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33666) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLtr6-0000CP-Jz for 47157@debbugs.gnu.org; Mon, 15 Mar 2021 16:30:25 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51567) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtr1-0006Go-9b; Mon, 15 Mar 2021 16:30:19 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=46664 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lLtqy-00017f-Lw; Mon, 15 Mar 2021 16:30:18 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: bug#47157: =?utf-8?Q?=E2=80=9CBad?= Read-Header-Line header: =?utf-8?Q?#=3Ceof=3E=E2=80=9D?= while substituting References: <87eeggh4rh.fsf@inria.fr> <871rcgfiz9.fsf@cbaines.net> Date: Mon, 15 Mar 2021 21:30:15 +0100 In-Reply-To: <871rcgfiz9.fsf@cbaines.net> (Christopher Baines's message of "Mon, 15 Mar 2021 17:02:34 +0000") Message-ID: <87a6r4cg88.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 47157 Cc: 47157@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Christopher Baines skribis: >> I think 7b812f7c84c43455cdd68a0e51b6ded018afcc8e and subsequent commits >> may have caused this regression. In particular, in >> 20c08a8a45d0f137ead7c05e720456b2aea44402, >> =E2=80=98call-with-connection-error-handling=E2=80=99 is now used, but t= hat one doesn=E2=80=99t >> catch the exceptions mentioned above, in this case =E2=80=98bad-header= =E2=80=99. > > I think the behaviour changed unintentionally with [1], however, > thinking about the connection reuse in process-substitution compared > with http-multiple-get, there's no attempt here to look at if the server > has specified whether the connection should be closed. > > 1: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3Df50f5751fff4cf= c6d5abba9681054569694b7a5c > > Just like http-multiple-get, it's probably worth trying to check the > headers of the response, look at whether the server has indicated that > the connection should be closed, and if so, close the connection, > forcing a new one to be established for future requests. I think that=E2=80=99s not enough because we can=E2=80=99t rely on the serv= er=E2=80=99s state intent here. For example, you have a keep-alive connection that you keep in cache. Minutes later, you come back and send a request over that port. If the server dropped the connection in the meantime, that can manifest in any of the ways we=E2=80=99ve seen: 'bad-response when attempting to read the response, some 'gnutls-error, 'system-error and EPIPE, etc. There=E2=80=99= s no way to determine in advance whether the socket is fine. That=E2=80=99s why the initial approach was to wrap all the call sites were= the socket was known to be possibly =E2=80=9Ctainted=E2=80=9D in =E2=80=98with-= cached-connection=E2=80=99. > I've now actually got around to testing this, I'm no expert at running > the substitute script manually without the guix-daemon, but I gave it a > go, using a local NGinx instance which just allowed two requests per > connection. I believe in this case =E2=80=98port-closed?=E2=80=99 returns true because = the socket/TLS record port got closed right at the end of the response, so it=E2=80=99s the =E2=80=9Ceasy=E2=80=9D case; I don=E2=80=99t think it capt= ures the situation I described above where an error comes up later while trying to write to/read from the port. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 15 16:37:40 2021 Received: (at 47157) by debbugs.gnu.org; 15 Mar 2021 20:37:40 +0000 Received: from localhost ([127.0.0.1]:37314 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLty7-0000NQ-V5 for submit@debbugs.gnu.org; Mon, 15 Mar 2021 16:37:40 -0400 Received: from mira.cbaines.net ([212.71.252.8]:36884) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lLty6-0000NJ-L3 for 47157@debbugs.gnu.org; Mon, 15 Mar 2021 16:37:39 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 9180527BC52; Mon, 15 Mar 2021 20:37:37 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 8faf2117; Mon, 15 Mar 2021 20:37:36 +0000 (UTC) References: <87eeggh4rh.fsf@inria.fr> <871rcgfiz9.fsf@cbaines.net> <87a6r4cg88.fsf@gnu.org> User-agent: mu4e 1.4.15; emacs 27.1 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#47157: =?utf-8?Q?=E2=80=9CBad?= Read-Header-Line header: =?utf-8?Q?#=3Ceof=3E=E2=80=9D?= while substituting In-reply-to: <87a6r4cg88.fsf@gnu.org> Date: Mon, 15 Mar 2021 20:37:35 +0000 Message-ID: <87sg4wdugg.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 47157 Cc: 47157@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Christopher Baines skribis: > >>> I think 7b812f7c84c43455cdd68a0e51b6ded018afcc8e and subsequent commits >>> may have caused this regression. In particular, in >>> 20c08a8a45d0f137ead7c05e720456b2aea44402, >>> =E2=80=98call-with-connection-error-handling=E2=80=99 is now used, but = that one doesn=E2=80=99t >>> catch the exceptions mentioned above, in this case =E2=80=98bad-header= =E2=80=99. >> >> I think the behaviour changed unintentionally with [1], however, >> thinking about the connection reuse in process-substitution compared >> with http-multiple-get, there's no attempt here to look at if the server >> has specified whether the connection should be closed. >> >> 1: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3Df50f5751fff4c= fc6d5abba9681054569694b7a5c >> >> Just like http-multiple-get, it's probably worth trying to check the >> headers of the response, look at whether the server has indicated that >> the connection should be closed, and if so, close the connection, >> forcing a new one to be established for future requests. > > I think that=E2=80=99s not enough because we can=E2=80=99t rely on the se= rver=E2=80=99s state > intent here. > > For example, you have a keep-alive connection that you keep in cache. > Minutes later, you come back and send a request over that port. If the > server dropped the connection in the meantime, that can manifest in any > of the ways we=E2=80=99ve seen: 'bad-response when attempting to read the > response, some 'gnutls-error, 'system-error and EPIPE, etc. There=E2=80= =99s no > way to determine in advance whether the socket is fine. > > That=E2=80=99s why the initial approach was to wrap all the call sites we= re the > socket was known to be possibly =E2=80=9Ctainted=E2=80=9D in =E2=80=98wit= h-cached-connection=E2=80=99. > >> I've now actually got around to testing this, I'm no expert at running >> the substitute script manually without the guix-daemon, but I gave it a >> go, using a local NGinx instance which just allowed two requests per >> connection. > > I believe in this case =E2=80=98port-closed?=E2=80=99 returns true becaus= e the > socket/TLS record port got closed right at the end of the response, so > it=E2=80=99s the =E2=80=9Ceasy=E2=80=9D case; I don=E2=80=99t think it ca= ptures the situation I > described above where an error comes up later while trying to write > to/read from the port. Yeah, of course, I think error handling is needed as well, it just occurred to me when looking at this issue and the relevant code. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmBPxZBfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XeivA//WjkDgeggXZKfZyzacELl6b7tcx0BS2Ql 7ZaubFvK0Y2rFwEfBUsmCAhTtOsfh/+Q+PsaekXSMxrhUV7greS7LLesrFCbvidr n+sWmQ5M6sj/+pp5bh/Xk3VKJiy3bC2bjGzskxt2LJq8c/Nxw9hZpg8qsGwZSqI3 TU0r/nmvh8B+7OuQ4zjBtLGM3uF3H1GTdqnfQyQr4B+K5HmYlk3YECegdRcyJy0n OedztwAIBrJrXFYqsQTS9GmQfg8EnMb+T7c+jXUptFDniKJPLhGldfV2NrBNqhjY w+sRsKwX1XEwgzhii7AxUPNI2xUeu2RorZgHjIwYRlMh82GNEMz5VP0mYx3kbVrC dBhcHvqlELuaK+21uv4g6mg+VPUTvjCdPuDWimdZ2MJX3GaACQk7ed+8erb2zn+0 YH91i+yLxlCFwoF9YHIhNsrmFLc7IvxZzsl5FSDQyfS56xsHvbreqWW8d0OfJ6GY XFaf0oNY8TMVC2yy28mOBKvJQTeGxdaJAORmfNOrA4Rl52oVgNuiWnGzpkg/yRdm HYdbXTVzHYzf2byvlwOwot7JBaSPxHGPzbl4KglOCb2RxwpM0luEtce2CwK3uVpr vt1afnrW4kUPJeGXnkVmyKbinioMuQ9AV9VBZfDrO7VrHurcQd3xDJ/CcLnF8nUX v9WqaGi8iCE= =A6Tc -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 18 12:04:39 2021 Received: (at 47157-done) by debbugs.gnu.org; 18 Mar 2021 16:04:39 +0000 Received: from localhost ([127.0.0.1]:47845 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lMv8Z-0001yb-9s for submit@debbugs.gnu.org; Thu, 18 Mar 2021 12:04:39 -0400 Received: from eggs.gnu.org ([209.51.188.92]:60420) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lMv8X-0001yJ-2U for 47157-done@debbugs.gnu.org; Thu, 18 Mar 2021 12:04:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58245) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lMv8R-0002Zh-Qu; Thu, 18 Mar 2021 12:04:31 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=53756 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lMv8R-0005bo-AT; Thu, 18 Mar 2021 12:04:31 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: bug#47157: =?utf-8?Q?=E2=80=9CBad?= Read-Header-Line header: =?utf-8?Q?#=3Ceof=3E=E2=80=9D?= while substituting References: <87eeggh4rh.fsf@inria.fr> <871rcgfiz9.fsf@cbaines.net> <87a6r4cg88.fsf@gnu.org> <87sg4wdugg.fsf@cbaines.net> Date: Thu, 18 Mar 2021 17:04:29 +0100 In-Reply-To: <87sg4wdugg.fsf@cbaines.net> (Christopher Baines's message of "Mon, 15 Mar 2021 20:37:35 +0000") Message-ID: <8735wsmos2.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 47157-done Cc: 47157-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) This should now be fixed by c37e3b92ad0334ba2fe7ee4e98631f0a4edeee21. Thanks, Chris! Ludo=E2=80=99. From unknown Thu Jun 19 14:05:55 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 16 Apr 2021 11:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator