From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 02 16:03:08 2020 Received: (at submit) by debbugs.gnu.org; 2 Mar 2020 21:03:08 +0000 Received: from localhost ([127.0.0.1]:38411 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sDT-0001n4-7L for submit@debbugs.gnu.org; Mon, 02 Mar 2020 16:03:08 -0500 Received: from lists.gnu.org ([209.51.188.17]:48765) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sDR-0001mw-BS for submit@debbugs.gnu.org; Mon, 02 Mar 2020 16:03:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42600) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sDQ-0004a7-EP for guix-patches@gnu.org; Mon, 02 Mar 2020 16:03:05 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,UNPARSEABLE_RELAY autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j8sDP-0000kc-4y for guix-patches@gnu.org; Mon, 02 Mar 2020 16:03:04 -0500 Received: from mira.cbaines.net ([212.71.252.8]:47046) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1j8sDO-0000jz-Ub for guix-patches@gnu.org; Mon, 02 Mar 2020 16:03:03 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id DBD3C27BBE4 for ; Mon, 2 Mar 2020 21:03:01 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 516b0a8d for ; Mon, 2 Mar 2020 21:02:59 +0000 (UTC) User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines To: guix-patches@gnu.org Subject: [PATCH 0/3] http-multiple-get enhancements Date: Mon, 02 Mar 2020 21:02:57 +0000 Message-ID: <87imjmeagu.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 212.71.252.8 X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --=-=-= Content-Type: text/plain These are a few changes I made to the copy of http-multiple-get in the Guix Data Service. Christopher Baines (3): substitute: Use the same port for multiple request batches. substitute: Make http-multiple-get batch size configurable. substitute: Close port at the end of http-multiple-get. guix/scripts/substitute.scm | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl5ddIFfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XecExAAhLXSlDod7wFD7C1Hpf6N1vk/YYPpjhe+xFVWZtGZ069ZZ0JCZ45R1qKF F3C3SOPpMSeiTHepRqUP0eoNkDugH1nOMZRYymFdeUq4iSTuntYTw5juLeBFFeP9 AUerXp3qMEX0rkV2/uwvIrSHT3FRmVHRdsP7RVEcI+eDVvV80QxzMucQJqQmm5bB GSU7M9Vwp/RL8+SZfEZpbGwCJ734O1BIOd3cExISG81/JPPD0W+GFK9fZKwt3wbL dsXq4B6kH+S/bK0mTQhVSiQNS3YSjRyMtQPPkdgvw3Y1MH5P2yrIXufWq8MH7aiZ KFmSlu87SZh+o9FXrq4esmDszC6XC/Jegexvm1eCTCUmW+g65h092+4DTkrfTjaL hARBDyKMB6w77uy69sryM2K6c6Tlddz+7LPRLwQ1r8plLbNqd17V5ySbjuUISoPs uezJcNN9lgztlkKHxLh371lvYgiKnye1ky+IzJ8UKlXMW/Hz0/aNJR4mvk0SXeuQ w1tpM166aVFj4ui03r03eenx0Vaeq2uE7Ug/ebxCojjZ7BUJDYIGrPNs/qRpSCNr zE3uVcCsdre2pWsQPrW3HaK+gnHyD3DxvrAMOn9ZIA03e68wKgwXmlVW6HKn7W79 A0BW7l5EJVfK13Eyx5UR2MYYboUql9fLwBPWD8IsY5T/u8bWmdg= =MglG -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 02 16:07:40 2020 Received: (at 39873) by debbugs.gnu.org; 2 Mar 2020 21:07:40 +0000 Received: from localhost ([127.0.0.1]:38418 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHs-0001u8-HV for submit@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:40 -0500 Received: from mira.cbaines.net ([212.71.252.8]:48634) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHq-0001to-G5 for 39873@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:38 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id E2AD027BBE4 for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:37 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 6b8fc5cb for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:35 +0000 (UTC) From: Christopher Baines To: 39873@debbugs.gnu.org Subject: [PATCH 1/3] substitute: Use the same port for multiple request batches. Date: Mon, 2 Mar 2020 21:07:33 +0000 Message-Id: <20200302210735.13337-1-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <87imjmeagu.fsf@cbaines.net> References: <87imjmeagu.fsf@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 39873 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) In http-multiple-get. * guix/scripts/substitute.scm (http-multiple-get): Switch port to p in one occurrence. --- guix/scripts/substitute.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index dfb975a24a..5ed43d36c9 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -545,7 +545,7 @@ initial connection on which HTTP requests are sent." (() (reverse result)) (remainder - (connect port remainder result)))) + (connect p remainder result)))) ((head tail ...) (let* ((resp (read-response p)) (body (response-body-port resp)) -- 2.25.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 02 16:07:40 2020 Received: (at 39873) by debbugs.gnu.org; 2 Mar 2020 21:07:41 +0000 Received: from localhost ([127.0.0.1]:38420 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHs-0001uA-Nx for submit@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:40 -0500 Received: from mira.cbaines.net ([212.71.252.8]:48636) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHq-0001tn-G2 for 39873@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:38 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id 002AF27BBEA for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:37 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 8974fb65 for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:35 +0000 (UTC) From: Christopher Baines To: 39873@debbugs.gnu.org Subject: [PATCH 2/3] substitute: Make http-multiple-get batch size configurable. Date: Mon, 2 Mar 2020 21:07:34 +0000 Message-Id: <20200302210735.13337-2-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200302210735.13337-1-mail@cbaines.net> References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 39873 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * guix/scripts/substitute.scm (http-multiple-get): Add batch-size parameter. --- guix/scripts/substitute.scm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 5ed43d36c9..a88cb5bcfe 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -501,7 +501,8 @@ MAX-LENGTH first elements." (loop (+ 1 len) tail (cons head result))))))) (define* (http-multiple-get base-uri proc seed requests - #:key port (verify-certificate? #t)) + #:key port (verify-certificate? #t) + (batch-size 1000)) "Send all of REQUESTS to the server at BASE-URI. Call PROC for each response, passing it the request object, the response, a port from which to read the response body, and the previous result, starting with SEED, à la @@ -511,7 +512,7 @@ initial connection on which HTTP requests are sent." (requests requests) (result seed)) (define batch - (at-most 1000 requests)) + (at-most batch-size requests)) ;; (format (current-error-port) "connecting (~a requests left)..." ;; (length requests)) -- 2.25.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 02 16:07:41 2020 Received: (at 39873) by debbugs.gnu.org; 2 Mar 2020 21:07:41 +0000 Received: from localhost ([127.0.0.1]:38422 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHs-0001uH-Ua for submit@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:41 -0500 Received: from mira.cbaines.net ([212.71.252.8]:48638) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHq-0001tp-GG for 39873@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:39 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id 0FDAB27BBEB for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:38 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 570a5acb for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:35 +0000 (UTC) From: Christopher Baines To: 39873@debbugs.gnu.org Subject: [PATCH 3/3] substitute: Close port at the end of http-multiple-get. Date: Mon, 2 Mar 2020 21:07:35 +0000 Message-Id: <20200302210735.13337-3-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200302210735.13337-1-mail@cbaines.net> References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 39873 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * guix/scripts/substitute.scm (http-multiple-get): Add close-port call. --- guix/scripts/substitute.scm | 1 + 1 file changed, 1 insertion(+) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index a88cb5bcfe..e3f5837a8e 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -544,6 +544,7 @@ initial connection on which HTTP requests are sent." (() (match (drop requests processed) (() + (close-port p) (reverse result)) (remainder (connect p remainder result)))) -- 2.25.0 From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 07 16:13:09 2020 Received: (at 39873) by debbugs.gnu.org; 7 Mar 2020 21:13:09 +0000 Received: from localhost ([127.0.0.1]:47646 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jAgku-00018j-Rl for submit@debbugs.gnu.org; Sat, 07 Mar 2020 16:13:09 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37608) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jAgkt-00018K-5C for 39873@debbugs.gnu.org; Sat, 07 Mar 2020 16:13:07 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:43771) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1jAgkn-000080-R5; Sat, 07 Mar 2020 16:13:01 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=34296 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jAgkm-0003We-VR; Sat, 07 Mar 2020 16:13:01 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#39873] [PATCH 1/3] substitute: Use the same port for multiple request batches. References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> Date: Sat, 07 Mar 2020 22:12:58 +0100 In-Reply-To: <20200302210735.13337-1-mail@cbaines.net> (Christopher Baines's message of "Mon, 2 Mar 2020 21:07:33 +0000") Message-ID: <875zffg97p.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 39873 Cc: 39873@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Christopher! Christopher Baines skribis: > In http-multiple-get. Leftover? :-) > * guix/scripts/substitute.scm (http-multiple-get): Switch port to p in one > occurrence. > --- > guix/scripts/substitute.scm | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm > index dfb975a24a..5ed43d36c9 100755 > --- a/guix/scripts/substitute.scm > +++ b/guix/scripts/substitute.scm > @@ -545,7 +545,7 @@ initial connection on which HTTP requests are sent." > (() > (reverse result)) > (remainder > - (connect port remainder result)))) > + (connect p remainder result)))) LGTM! Did you notice an occurrence of this bug somewhere? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 07 16:13:19 2020 Received: (at 39873) by debbugs.gnu.org; 7 Mar 2020 21:13:19 +0000 Received: from localhost ([127.0.0.1]:47649 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jAgl5-000197-9B for submit@debbugs.gnu.org; Sat, 07 Mar 2020 16:13:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37621) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jAgl3-00018t-CM for 39873@debbugs.gnu.org; Sat, 07 Mar 2020 16:13:17 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:43772) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1jAgky-0000V0-9d; Sat, 07 Mar 2020 16:13:12 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=34298 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jAgkv-0003XT-Rq; Sat, 07 Mar 2020 16:13:11 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#39873] [PATCH 2/3] substitute: Make http-multiple-get batch size configurable. References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> <20200302210735.13337-2-mail@cbaines.net> Date: Sat, 07 Mar 2020 22:13:08 +0100 In-Reply-To: <20200302210735.13337-2-mail@cbaines.net> (Christopher Baines's message of "Mon, 2 Mar 2020 21:07:34 +0000") Message-ID: <871rq3g97f.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 39873 Cc: 39873@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Christopher Baines skribis: > * guix/scripts/substitute.scm (http-multiple-get): Add batch-size parameter. LGTM! From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 07 16:14:11 2020 Received: (at 39873) by debbugs.gnu.org; 7 Mar 2020 21:14:11 +0000 Received: from localhost ([127.0.0.1]:47654 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jAglv-0001An-JW for submit@debbugs.gnu.org; Sat, 07 Mar 2020 16:14:11 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37803) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jAglu-0001Ab-HW for 39873@debbugs.gnu.org; Sat, 07 Mar 2020 16:14:10 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:43803) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1jAglp-0001bX-F1; Sat, 07 Mar 2020 16:14:05 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=34300 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jAglo-0003aV-VS; Sat, 07 Mar 2020 16:14:05 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Christopher Baines Subject: Re: [bug#39873] [PATCH 3/3] substitute: Close port at the end of http-multiple-get. References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> <20200302210735.13337-3-mail@cbaines.net> Date: Sat, 07 Mar 2020 22:14:03 +0100 In-Reply-To: <20200302210735.13337-3-mail@cbaines.net> (Christopher Baines's message of "Mon, 2 Mar 2020 21:07:35 +0000") Message-ID: <87wo7veulg.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 39873 Cc: 39873@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Christopher Baines scribes: > * guix/scripts/substitute.scm (http-multiple-get): Add close-port call. > --- > guix/scripts/substitute.scm | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm > index a88cb5bcfe..e3f5837a8e 100755 > --- a/guix/scripts/substitute.scm > +++ b/guix/scripts/substitute.scm > @@ -544,6 +544,7 @@ initial connection on which HTTP requests are sent." > (() > (match (drop requests processed) > (() > + (close-port p) LGTM! Did you notice a file descriptor leak somewhere? Thank you! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 08 19:58:55 2020 Received: (at 39873) by debbugs.gnu.org; 8 Mar 2020 23:58:55 +0000 Received: from localhost ([127.0.0.1]:49502 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jB5ot-0001HD-Cm for submit@debbugs.gnu.org; Sun, 08 Mar 2020 19:58:55 -0400 Received: from mira.cbaines.net ([212.71.252.8]:55422) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jB5os-0001H6-3y for 39873@debbugs.gnu.org; Sun, 08 Mar 2020 19:58:54 -0400 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id 80CDC27BBEB; Sun, 8 Mar 2020 23:58:53 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id faae7e33; Sun, 8 Mar 2020 23:58:51 +0000 (UTC) References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> <20200302210735.13337-3-mail@cbaines.net> <87wo7veulg.fsf@gnu.org> User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#39873] [PATCH 3/3] substitute: Close port at the end of http-multiple-get. In-reply-to: <87wo7veulg.fsf@gnu.org> Date: Sun, 08 Mar 2020 23:58:51 +0000 Message-ID: <877dzue6v8.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 39873 Cc: 39873@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Christopher Baines scribes: > >> * guix/scripts/substitute.scm (http-multiple-get): Add close-port call. >> --- >> guix/scripts/substitute.scm | 1 + >> 1 file changed, 1 insertion(+) >> >> diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm >> index a88cb5bcfe..e3f5837a8e 100755 >> --- a/guix/scripts/substitute.scm >> +++ b/guix/scripts/substitute.scm >> @@ -544,6 +544,7 @@ initial connection on which HTTP requests are sent." >> (() >> (match (drop requests processed) >> (() >> + (close-port p) > > LGTM! > > Did you notice a file descriptor leak somewhere? No, I was looking in to some wierd TLS related errors I had when using http-multiple-get to query Curiass from the Guix Data Service, and I noticed that maybe the port wasn't being closed when it should be. I still haven't done a lot of testing, but I think some of these changes have helped (although probably not this one). --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl5lhrtfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XemRA//XN+hsKKOfddNAzOdb8xe6VzkqEIVjhgensZmBQuskx+iXqyMzXGy9YFG Q+PdhmXQZQzREgy/w8Ua0c8APMRzbulKpBloXnxGt+vnu3HPuDEcdvcTynmt1YHB yGXesfb0izj9JTbg9pyQDx4tukvSlrnat/NWy+C5EtS6HXc0BZeCBz/jMpoj7rv/ z47YxnrJ0MGt2KR0hEkyd53w7QBJUDrS7KuqCii3rfkJg6uKAs+5OAw98ZYC0Emb uri/nb1y8EnQaVzJHrKS5nHlh11szj3dMJ06JN/lLynUcX0EQzlmd9tdVRPMP4nN D/Lo3p8Vmc/etL1CE8M/EfYPSPa8jxe1KvWzLovpny2Z1l0NQZumgEprr6M+XH6L k9Holo3EegcdMun9aggaXTqA8SAKSaR+Nls/sjb1nPV6iZV4MQKsQ7P1Co7h/OZt s5qmwrzotdAO6GuVDTL6wLHQvAGOKHKiHgbTWAjatKXRbsAjNXnyPG41L7Dr/2Ep wYYwXUE6tpeYq1f056m7R2LzZVQS8gpw70azF83/LoYYSOV5rV4unnBeqoaWmXTJ 9OCgr+HbEOquAIdvtWb36Yb2eNImrxoplwlN6DyUpn1IZcCcRXAEoEaF2pX5OEDw e6tOB5WY1XADBCzAnCXff0YAPeSOgsUAB49SQBa+Ul7bzznhFpA= =eviw -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 08 20:07:56 2020 Received: (at 39873) by debbugs.gnu.org; 9 Mar 2020 00:07:56 +0000 Received: from localhost ([127.0.0.1]:49514 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jB5xb-0001Vi-Qr for submit@debbugs.gnu.org; Sun, 08 Mar 2020 20:07:56 -0400 Received: from mira.cbaines.net ([212.71.252.8]:55432) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jB5xZ-0001Va-WA for 39873@debbugs.gnu.org; Sun, 08 Mar 2020 20:07:55 -0400 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id 58B9B27BBEE; Mon, 9 Mar 2020 00:07:53 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 0f905faf; Mon, 9 Mar 2020 00:07:51 +0000 (UTC) References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> <875zffg97p.fsf@gnu.org> User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#39873] [PATCH 1/3] substitute: Use the same port for multiple request batches. In-reply-to: <875zffg97p.fsf@gnu.org> Date: Mon, 09 Mar 2020 00:07:46 +0000 Message-ID: <875zfee6gd.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 39873 Cc: 39873@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi Christopher! > > Christopher Baines skribis: > >> In http-multiple-get. > > Leftover? :-) No, just clarifying what function is being referred to. Although the line below reveals that also. >> * guix/scripts/substitute.scm (http-multiple-get): Switch port to p in o= ne >> occurrence. >> --- >> guix/scripts/substitute.scm | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm >> index dfb975a24a..5ed43d36c9 100755 >> --- a/guix/scripts/substitute.scm >> +++ b/guix/scripts/substitute.scm >> @@ -545,7 +545,7 @@ initial connection on which HTTP requests are sent." >> (() >> (reverse result)) >> (remainder >> - (connect port remainder result)))) >> + (connect p remainder result)))) > > LGTM! > > Did you notice an occurrence of this bug somewhere? I was looking for something that could be going wrong here as I was seeing some kind of error talking to Cuirass using this in the Guix Data Service, and happened to notice this in the code. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl5liNJfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XeJvg/+Pb3lzcFhNPDt83Fs6nDeLv9DGRCD9eRHyk6AHj0c1MtXwY43ezcF4ZJm gAQAxgfTYVJMJMes7rXmXtzxPZroneHjGymFFCpR8iqM2umC1cwR8LLvHufSU6tq /ACB9oM5TK7WCrQodffua8mOq0Y4uLWk8g0bJycQo+w53Grsp2FSPjLyDcGtyxLW bm0iM5cz82QeFH/1N/Vmrr2YSRMN1K45Jd9UE311qEEMp9pkNQI3TF58XBuCpCyw 5vaztgtZDLg1zEdIVFokJjkFrmd/Tm7BEqhXO9U6wulL+Q0j+eo4bOGFMMoG/Y80 8utWzb+9sQqIpiHX242fo5+p1iwofuJbb5/Ef9CVwD37sK2QoeiAezz34U2s8XJF lVhd4JRxY6vDP106HNWN9htPQFR1VtGmu2tF7TlJi3O3vBVNq1+B4+CCbfhXhJm5 GHCvJHyRVLV1sLdFDQgc6SKwfAUtMVchKrNs2j7IrreDPDOX9DofD5iqqXNF1kta ZCeuLdU8Ytvx41jp8+PQw3heK/LJAhm0lmYbVlcRYRmARZEGzvaknbH/v7H84PC+ zt+kWPqC47Xva6GneyYfpD6iuaF3uvfYBdyPqZPGxZVVF4evxWHARRrJFvokhpcd TD02u5iLv67LpgzSUDvbATTQIO8NJcTPwrQKrHpWdnl3rFXvhEs= =fWO2 -----END PGP SIGNATURE----- --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 26 14:40:40 2020 Received: (at 39873-done) by debbugs.gnu.org; 26 Apr 2020 18:40:40 +0000 Received: from localhost ([127.0.0.1]:34892 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jSmCm-0001uX-4l for submit@debbugs.gnu.org; Sun, 26 Apr 2020 14:40:40 -0400 Received: from mira.cbaines.net ([212.71.252.8]:44384) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jSmCk-0001uO-Vy for 39873-done@debbugs.gnu.org; Sun, 26 Apr 2020 14:40:39 -0400 Received: from localhost (unknown [46.237.175.119]) by mira.cbaines.net (Postfix) with ESMTPSA id 0C34927BBE1 for <39873-done@debbugs.gnu.org>; Sun, 26 Apr 2020 19:40:38 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id fd8d63ae for <39873-done@debbugs.gnu.org>; Sun, 26 Apr 2020 18:40:35 +0000 (UTC) References: <87imjmeagu.fsf@cbaines.net> User-agent: mu4e 1.2.0; emacs 26.3 From: Christopher Baines To: 39873-done@debbugs.gnu.org Subject: Re: [bug#39873] [PATCH 0/3] http-multiple-get enhancements In-reply-to: <87imjmeagu.fsf@cbaines.net> Date: Sun, 26 Apr 2020 19:40:35 +0100 Message-ID: <874kt6nl98.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 39873-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Christopher Baines writes: > These are a few changes I made to the copy of http-multiple-get in the > Guix Data Service. > > > Christopher Baines (3): > substitute: Use the same port for multiple request batches. > substitute: Make http-multiple-get batch size configurable. > substitute: Close port at the end of http-multiple-get. > > guix/scripts/substitute.scm | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) I've finally got around to pushing these as 928dc1bb1c1e96e6dfbe03dac2185ecf41a7b4f5 now. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl6l1aNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xf5hRAAif5j8hWtfIfr8x02rZ5U4sNAHh2yR9cpY9v1Z88gQtxcbKk77YaSnEic YSW+gDo7lshUeiyusexjLFmjUepZSeo7HmmE/agt/Bopgdm1jXN0jkr4toQddWuX nwBTW07RnH+M+gt0sdoMDfBEEIxVj8zCEUK+DlK5X0Ta38aClm317wN0Q2XcLvcK z3i0mrO/4NQHf335jqjldn9mswhuybjA2HS1eEf+zSfmtLU1pr/qtoMnR6T5L63k pCXqH7gLx0RyMdv8rV+fuUAqBQy7RDZ3EjnUSzWuDn/Ej+jzxNjNb1kZgs0Lxoau 9vfK5LyzIWD9wRCXzGd6GhclJx863XRFYsff93wsTR2n6wk2xVIWVT8hx2+HPISr kUeTimQC0O862dNGFOTs4eLIVeiBJveeLeyW3tMjIy8wNK7kwTrJS7LtuXllGaXl Ea7dxUPYFeZZc93rS/wmyRbZC6ES+z2yvLfzhIggD5HjbFLHDvT+V2is68hyudR8 3/16go//7EfzVJARMB5XqGx7zhG9IeApE66Ux4Jhu4j9CVsqtYD0z6DMg5E9LjPH upVCqBW41QO4LWxZdNUdLINBgAk1DojzSKZj/8xlcjy609R0pg/wUz4nDV9GRqlK 7/kB+Cbo6qQu39qM967aXjeDLoLx3CwEqRj4fNh21lTb+mrr0wo= =E2dk -----END PGP SIGNATURE----- --=-=-=-- From unknown Sun Jun 22 07:30:10 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 25 May 2020 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator