From unknown Sat Jun 21 10:21:22 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#44906 <44906@debbugs.gnu.org> To: bug#44906 <44906@debbugs.gnu.org> Subject: Status: Substitute requests fail if URL has trailing slash Reply-To: bug#44906 <44906@debbugs.gnu.org> Date: Sat, 21 Jun 2025 17:21:22 +0000 retitle 44906 Substitute requests fail if URL has trailing slash reassign 44906 guix submitter 44906 Hartmut Goebel severity 44906 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 27 16:20:03 2020 Received: (at submit) by debbugs.gnu.org; 27 Nov 2020 21:20:03 +0000 Received: from localhost ([127.0.0.1]:46343 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kil9u-0006hc-Ez for submit@debbugs.gnu.org; Fri, 27 Nov 2020 16:20:02 -0500 Received: from lists.gnu.org ([209.51.188.17]:38510) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kil9r-0006hG-F0 for submit@debbugs.gnu.org; Fri, 27 Nov 2020 16:20:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:54554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kil9r-0005iY-9j for bug-guix@gnu.org; Fri, 27 Nov 2020 16:19:59 -0500 Received: from mail-out.m-online.net ([2001:a60:0:28:0:1:25:1]:46240) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kil9m-0003QX-Ou for bug-guix@gnu.org; Fri, 27 Nov 2020 16:19:59 -0500 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4CjSFL3jPWz1rwml for ; Fri, 27 Nov 2020 22:19:50 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4CjSFL3c4Cz1qtmp for ; Fri, 27 Nov 2020 22:19:50 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id uvbkKENM71Wv for ; Fri, 27 Nov 2020 22:19:48 +0100 (CET) Received: from hermia.goebel-consult.de (ppp-188-174-58-171.dynamic.mnet-online.de [188.174.58.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS for ; Fri, 27 Nov 2020 22:19:48 +0100 (CET) Received: from thisbe.goebel-consult.de (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id 667F86028B for ; Fri, 27 Nov 2020 22:19:49 +0100 (CET) To: bug-guix From: Hartmut Goebel Organization: crazy-compilers.com Subject: Substitute requests fail if URL has trailing slash Message-ID: <3848e5d4-3694-e7f4-cb42-f97a51bde5b4@crazy-compilers.com> Date: Fri, 27 Nov 2020 22:19:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US Received-SPF: none client-ip=2001:a60:0:28:0:1:25:1; envelope-from=h.goebel@crazy-compilers.com; helo=mail-out.m-online.net X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) If the substitute-URL ends with a slash, api requests fail. Expected behavior: Substitute-URLs with and without trailing slash should behave the same. This is especially true for substitute-URLs with empty path ("http://server") and path "/" (http://server/") - for which the RFCs explicitly state to be equivalent. According to RFC 7230, sec 2.7.3 "http and https URI Normalization and Comparison" [1]: […] an empty path component is equivalent to an absolute path of "/", so the normal form is to provide a path of "/" instead. [1] https://tools.ietf.org/html/rfc7230#section-2.7.3 How to reproduce: no trailing slash: $ guix weather --substitute-urls="https://ci.guix.gnu.org" gcc-toolchain … https://ci.guix.gnu.org   100.0% substitutes available (3 out of 3) … Trailing slash: $ guix weather --substitute-urls="https://ci.guix.gnu.org/" gcc-toolchain … https://ci.guix.gnu.org/   0.0% substitutes available (0 out of 3) …   'https://ci.guix.gnu.org//api/queue?nr=1000' returned 400 ("Bad Request") -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 27 18:44:48 2020 Received: (at 44906) by debbugs.gnu.org; 27 Nov 2020 23:44:48 +0000 Received: from localhost ([127.0.0.1]:46624 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kinPz-00065J-Nt for submit@debbugs.gnu.org; Fri, 27 Nov 2020 18:44:48 -0500 Received: from mail-wr1-f50.google.com ([209.85.221.50]:46815) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kinPw-000655-VO for 44906@debbugs.gnu.org; Fri, 27 Nov 2020 18:44:46 -0500 Received: by mail-wr1-f50.google.com with SMTP id g14so7108553wrm.13 for <44906@debbugs.gnu.org>; Fri, 27 Nov 2020 15:44:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:in-reply-to:references:date:message-id:mime-version :content-transfer-encoding; bh=2XbmVIoSLdH5ZVJ6kOSW44NjjG8+xuQIOfP64ToGNJs=; b=qtI54843jwXtP+iP8XeiKLSIP7RMpwuDJ+zKMCoLVxmTocla6Otzsk6nB3xENVJZLn SUimodHhzFAoU+E8pccFrn2lzyY453jJgLjlCG9gh6YRUwbu0Vn+vmM++nAhL9pUFTdo Lo491gOBHinOH1ijAKHCOzs7U2rHs/T5O9RQYz5q9pMRGcJ9rUIW6EstlMof3wIt01Cx 3qF3jaPVh70KG9sMaq/qKAniz/XvGZbYyFgdeSaXbQdMABZRvYWxSx5iY9LCNy+b1TFa P/zSQ5jlN3cTvAxRTkAm9gTnlsk7BilK4xxot7xTYfNMqfO4UrebHdXaDusIxLoz5Qi1 S9QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=2XbmVIoSLdH5ZVJ6kOSW44NjjG8+xuQIOfP64ToGNJs=; b=Fp7NyL1WQ4paL50EA3qwSZ1T/2mjfcvtD/CvQ8HoHVZ3893JR2aspdiSfaj3L7Kahg Gf34QqaUiGoRf+88BMKeiVN9+zxffu7xsIV7rtB/ilU0SyCBCo4mhnWc1JysiGO7azn+ JcND+DTAy/rrjNA8SHX23bZRZ2QyllsKdhmsRtEby6Zzx7ebczH85ik8Op1OgTj20rqO MUWnnz9fEA8m0nVYJRoL7GrVa59vicXKWOPpLztWA647baxR7o4A0VmJ2HrTx4tJ3kW3 pQbf6sAGj4jW6mYUfYWtcKGqzF79j5nUrtI9p6r5ayDzQs+M0mESzfVtvzxQy1Kmymb5 mwxg== X-Gm-Message-State: AOAM532VxpMigYj+WqLaE7Gb0SO2x6H8171xwo+sBcdiqZcnrk2EMeO8 o2WsCAikxGeEtPhwGFwPTjRxwBKk9YF4rQ== X-Google-Smtp-Source: ABdhPJx0ZsrOZ/4HpsTRxPktW0nCSjsOmaL6MejxFl9LJYqKAH+h5AvISmz9wg0WtquTrda3WkDFMA== X-Received: by 2002:adf:eb91:: with SMTP id t17mr13665240wrn.330.1606520678697; Fri, 27 Nov 2020 15:44:38 -0800 (PST) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id e1sm14653913wmd.16.2020.11.27.15.44.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Nov 2020 15:44:38 -0800 (PST) From: zimoun To: Hartmut Goebel , 44906@debbugs.gnu.org Subject: Re: bug#44906: Substitute requests fail if URL has trailing slash In-Reply-To: <3848e5d4-3694-e7f4-cb42-f97a51bde5b4@crazy-compilers.com> References: <3848e5d4-3694-e7f4-cb42-f97a51bde5b4@crazy-compilers.com> Date: Sat, 28 Nov 2020 00:37:22 +0100 Message-ID: <86a6v2qslp.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 44906 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Dear, Thank you for the report. Tweaking the function such as: --8<---------------cut here---------------start------------->8--- (define (narinfo-request cache-url path) "Return an HTTP request for the narinfo of PATH at CACHE-URL." (let ((url (string-append cache-url "/" (store-path-hash-part path) ".narinfo")) (headers '((User-Agent . "GNU Guile")))) (format #t "~%Narinfo request: ~a~%~%" url) (build-request (string->uri url) #:method 'GET #:headers headers))) --8<---------------cut here---------------end--------------->8--- and removing the cache adequately, then running: --8<---------------cut here---------------start------------->8--- ./pre-inst-env guix weather \ --substitute-urls=3D"https://ci.guix.gnu.org/ https://ci.guix.gnu.org"= \ hello computing 1 package derivations for x86_64-linux... looking for 1 store items on https://ci.guix.gnu.org/... Narinfo request: https://ci.guix.gnu.org//a462kby1q51ndvxdv3b6p0rsixxrgx1h.= narinfo updating substitutes from 'https://ci.guix.gnu.org/'... 100.0% https://ci.guix.gnu.org/ 0.0% substitutes available (0 out of 1) [...] 'https://ci.guix.gnu.org//api/queue?nr=3D1000' returned 400 ("Bad Request= ") looking for 1 store items on https://ci.guix.gnu.org... Narinfo request: https://ci.guix.gnu.org/a462kby1q51ndvxdv3b6p0rsixxrgx1h.n= arinfo updating substitutes from 'https://ci.guix.gnu.org'... 100.0% https://ci.guix.gnu.org 100.0% substitutes available (1 out of 1) [...] at least 1,000 queued builds [...] build rate: 36.89 builds per hour [...] --8<---------------cut here---------------end--------------->8--- On Fri, 27 Nov 2020 at 22:19, Hartmut Goebel = wrote: > According to RFC 7230, sec 2.7.3 "http and https URI Normalization and > Comparison" [1]: > > [=E2=80=A6] an empty > path component is equivalent to an absolute path of "/", so the > normal form is to provide a path of "/" instead. > > [1] https://tools.ietf.org/html/rfc7230#section-2.7.3 Now, the question is where should the fix go? =E2=80=9Cguix publish=E2=80= =9D exposing the narinfos or =E2=80=9Cguix weather=E2=80=9C? Or both? >From my understanding, one fix should go to =E2=80=98guix publish=E2=80=99 = exposing the narinfos since: https://ci.guix.gnu.org//a462kby1q51ndvxdv3b6p0rsixxrgx1h.narinfo should be a valid URL and return the narinfo file. However, taking this road, it means that the cache folder will not be the same: ~/.cache/guix/substitute/x2wcz6gz3evwlqcrz3fqstmezkfcfnpfb5kfyxbz7kjikc7upk= iq/ ~/.cache/guix/substitute/4refhwxbjmeua2kwg2nmzhv4dg4d3dorpjefq7kiciw2pfhaf2= 6a/ https://ci.guix.gnu.org/ resp. https://ci.guix.gnu.org Therefore, =E2=80= =98guix weather=E2=80=99 should be fixed too. WDYT? All the best, simon From debbugs-submit-bounces@debbugs.gnu.org Sat Nov 28 04:47:40 2020 Received: (at 44906) by debbugs.gnu.org; 28 Nov 2020 09:47:40 +0000 Received: from localhost ([127.0.0.1]:46969 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kiwpQ-0005x7-76 for submit@debbugs.gnu.org; Sat, 28 Nov 2020 04:47:40 -0500 Received: from mail-out.m-online.net ([212.18.0.9]:53999) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kiwpN-0005wy-4X for 44906@debbugs.gnu.org; Sat, 28 Nov 2020 04:47:38 -0500 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4Cjmr81nN6z1qs0P; Sat, 28 Nov 2020 10:47:36 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4Cjmr8078lz1qv4t; Sat, 28 Nov 2020 10:47:35 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id xFaehybHDt0Z; Sat, 28 Nov 2020 10:47:34 +0100 (CET) Received: from hermia.goebel-consult.de (ppp-188-174-58-171.dynamic.mnet-online.de [188.174.58.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Sat, 28 Nov 2020 10:47:34 +0100 (CET) Received: from lenashee.goebel-consult.de (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id 6CF78602A2; Sat, 28 Nov 2020 10:48:14 +0100 (CET) Subject: Re: bug#44906: Substitute requests fail if URL has trailing slash To: zimoun , 44906@debbugs.gnu.org References: <3848e5d4-3694-e7f4-cb42-f97a51bde5b4@crazy-compilers.com> <86a6v2qslp.fsf@gmail.com> From: Hartmut Goebel Organization: crazy-compilers.com Message-ID: <7b367712-d759-ba3d-3ffc-d3323cb859c7@crazy-compilers.com> Date: Sat, 28 Nov 2020 10:47:31 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: <86a6v2qslp.fsf@gmail.com> Content-Type: multipart/alternative; boundary="------------68AA154E086793323F917DC4" Content-Language: en-US X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 44906 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) This is a multi-part message in MIME format. --------------68AA154E086793323F917DC4 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Am 28.11.20 um 00:37 schrieb zimoun: > Now, the question is where should the fix go? “guix publish” exposing > the narinfos or “guix weather“? Or both? I propose fixing all places where string-append is used to join URLs, since joining URLs is not the same as string concatenation. We might restrict our algorithm to only joining a path. shows the complete algorithm, where this is the relevant part for only joining a path (R.path) to a base URL's path (T.path). if (R.path starts-with "/") then T.path = remove_dot_segments(R.path); else T.path = merge(Base.path, R.path); T.path = remove_dot_segments(T.path); (Side-node: guile module (web uri) seems to lack respective, easy to use functions.) -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | --------------68AA154E086793323F917DC4 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit
Am 28.11.20 um 00:37 schrieb zimoun:
Now, the question is where should the fix go?  “guix publish” exposing
the narinfos or “guix weather“?  Or both?

I propose fixing all places where string-append is used to join URLs, since joining URLs is not the same as string concatenation. We might restrict our algorithm to only joining a path. <https://tools.ietf.org/html/rfc3986#section-5.2.2> shows the complete algorithm, where this is the relevant part for only joining a path (R.path) to a base URL's path (T.path).

               if (R.path starts-with "/") then
                  T.path = remove_dot_segments(R.path);
               else
                  T.path = merge(Base.path, R.path);
                  T.path = remove_dot_segments(T.path);

(Side-node: guile module (web uri) <https://www.gnu.org/software/guile/manual/html_node/URIs.html> seems to lack respective, easy to use functions.)

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |
--------------68AA154E086793323F917DC4-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 03 12:01:18 2020 Received: (at 44906) by debbugs.gnu.org; 3 Dec 2020 17:01:18 +0000 Received: from localhost ([127.0.0.1]:41568 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kkryo-0003rO-EB for submit@debbugs.gnu.org; Thu, 03 Dec 2020 12:01:18 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59044) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kkryl-0003l3-FH for 44906@debbugs.gnu.org; Thu, 03 Dec 2020 12:01:17 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47122) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kkryf-0003dT-FZ; Thu, 03 Dec 2020 12:01:09 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51424 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kkryc-0006FX-Gp; Thu, 03 Dec 2020 12:01:09 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Hartmut Goebel Subject: Re: bug#44906: Substitute requests fail if URL has trailing slash References: <3848e5d4-3694-e7f4-cb42-f97a51bde5b4@crazy-compilers.com> <86a6v2qslp.fsf@gmail.com> <7b367712-d759-ba3d-3ffc-d3323cb859c7@crazy-compilers.com> Date: Thu, 03 Dec 2020 18:01:05 +0100 In-Reply-To: <7b367712-d759-ba3d-3ffc-d3323cb859c7@crazy-compilers.com> (Hartmut Goebel's message of "Sat, 28 Nov 2020 10:47:31 +0100") Message-ID: <87im9iddta.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44906 Cc: 44906@debbugs.gnu.org, zimoun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Hartmut Goebel skribis: > I propose fixing all places where string-append is used to join URLs, > since joining URLs is not the same as string concatenation. We might=20 > restrict our algorithm to only joining a > path. shows the > complete algorithm, where this is the relevant part for only joining a > path (R.path) to a base URL's path (T.path). > > if (R.path starts-with "/") then > T.path =3D remove_dot_segments(R.path); > else > T.path =3D merge(Base.path, R.path); > T.path =3D remove_dot_segments(T.path); To begin with, we could define =E2=80=98url-append=E2=80=99 in (guix http-c= lient), say, and use it in (guix scripts substitute). Eventually it would be nice to have that in (web uri). Thoughts? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 03 23:16:46 2020 Received: (at 44906) by debbugs.gnu.org; 4 Dec 2020 04:16:46 +0000 Received: from localhost ([127.0.0.1]:42374 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kl2WU-0002CT-Ck for submit@debbugs.gnu.org; Thu, 03 Dec 2020 23:16:46 -0500 Received: from world.peace.net ([64.112.178.59]:59926) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kl2WR-0002CF-VC for 44906@debbugs.gnu.org; Thu, 03 Dec 2020 23:16:45 -0500 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kl2WK-0005Fe-Sh; Thu, 03 Dec 2020 23:16:36 -0500 From: Mark H Weaver To: Ludovic =?utf-8?Q?Court=C3=A8s?= , Hartmut Goebel Subject: Re: bug#44906: Substitute requests fail if URL has trailing slash In-Reply-To: <87im9iddta.fsf@gnu.org> References: <3848e5d4-3694-e7f4-cb42-f97a51bde5b4@crazy-compilers.com> <86a6v2qslp.fsf@gmail.com> <7b367712-d759-ba3d-3ffc-d3323cb859c7@crazy-compilers.com> <87im9iddta.fsf@gnu.org> Date: Thu, 03 Dec 2020 23:15:45 -0500 Message-ID: <87zh2udx4j.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 44906 Cc: 44906@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, Ludovic Court=C3=A8s writes: > Hartmut Goebel skribis: > >> I propose fixing all places where string-append is used to join URLs, >> since joining URLs is not the same as string concatenation. We might=20 >> restrict our algorithm to only joining a >> path. shows the >> complete algorithm, where this is the relevant part for only joining a >> path (R.path) to a base URL's path (T.path). >> >> if (R.path starts-with "/") then >> T.path =3D remove_dot_segments(R.path); >> else >> T.path =3D merge(Base.path, R.path); >> T.path =3D remove_dot_segments(T.path); > > To begin with, we could define =E2=80=98url-append=E2=80=99 in (guix http= -client), say, > and use it in (guix scripts substitute). > > Eventually it would be nice to have that in (web uri). Note that 'resolve-uri-reference' in (guix build download) implements the algorithm specified in RFC 3986 section 5.2.2, for purposes of supporting HTTP redirects. Perhaps some of that code will be useful. Regards, Mark From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 09 04:38:50 2021 Received: (at 44906) by debbugs.gnu.org; 9 Jul 2021 08:38:50 +0000 Received: from localhost ([127.0.0.1]:58309 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1m26-0000fd-LB for submit@debbugs.gnu.org; Fri, 09 Jul 2021 04:38:50 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:57425) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1m23-0000fM-5n for 44906@debbugs.gnu.org; Fri, 09 Jul 2021 04:38:49 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4GLmln62Xyz1qtds; Fri, 9 Jul 2021 10:38:45 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4GLmln4Fh8z1qr4T; Fri, 9 Jul 2021 10:38:45 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id 2YGhrtEX1Sal; Fri, 9 Jul 2021 10:38:42 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-58-132.dynamic.mnet-online.de [188.174.58.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Fri, 9 Jul 2021 10:38:42 +0200 (CEST) Received: from thisbe.goebel-consult.de (hermia.goebel-consult.de [192.168.110.7]) by hermia.goebel-consult.de (Postfix) with ESMTP id E5ABF601A5; Fri, 9 Jul 2021 10:38:43 +0200 (CEST) From: Hartmut Goebel To: 44906@debbugs.gnu.org, Mark H Weaver , zimoun , Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: [PATCH 0/3] Properly construct URLs if base-url has trailing slash Date: Fri, 9 Jul 2021 10:38:38 +0200 Message-Id: X-Mailer: git-send-email 2.30.2 In-Reply-To: <87zh2udx4j.fsf@netris.org> References: <3848e5d4-3694-e7f4-cb42-f97a51bde5b4@crazy-compilers.com> <86a6v2qslp.fsf@gmail.com> <7b367712-d759-ba3d-3ffc-d3323cb859c7@crazy-compilers.com> <87im9iddta.fsf@gnu.org> <87zh2udx4j.fsf@netris.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 44906 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Here is now an attempt to solve the issue. It had to be fixed in guix/substitutes.scm and guix/ci.scm only. In guix/scripts/publish.scm I did not spot any place where wrong URLs are constructed. Many thanks to Mark for pointing to 'resolve-uri-reference'. Regarding CI: I did some tests, so these should work. Anyhow, I did not find a tests-suite for fully testing this part. Hartmut Goebel (3): substitute: Fix handling of short option "-h". substitutes: Properly construct URLs. ci: Properly construct URLs. guix/ci.scm | 79 +++++++++++++++++++++---------------- guix/scripts/substitute.scm | 2 +- guix/substitutes.scm | 13 +++--- 3 files changed, 55 insertions(+), 39 deletions(-) -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 09 04:38:51 2021 Received: (at 44906) by debbugs.gnu.org; 9 Jul 2021 08:38:51 +0000 Received: from localhost ([127.0.0.1]:58311 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1m26-0000ff-Qh for submit@debbugs.gnu.org; Fri, 09 Jul 2021 04:38:51 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:40990) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1m25-0000fQ-8j for 44906@debbugs.gnu.org; Fri, 09 Jul 2021 04:38:49 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4GLmlr5WPpz1qtd6; Fri, 9 Jul 2021 10:38:48 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4GLmlr5Ktxz1qr4T; Fri, 9 Jul 2021 10:38:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id 52_kw36rZa-e; Fri, 9 Jul 2021 10:38:48 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-58-132.dynamic.mnet-online.de [188.174.58.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Fri, 9 Jul 2021 10:38:48 +0200 (CEST) Received: from thisbe.goebel-consult.de (hermia.goebel-consult.de [192.168.110.7]) by hermia.goebel-consult.de (Postfix) with ESMTP id D5BE2601BF; Fri, 9 Jul 2021 10:38:45 +0200 (CEST) From: Hartmut Goebel To: 44906@debbugs.gnu.org, guix-patches@gnu.org Subject: [PATCH 1/3] substitute: Fix handling of short option "-h". Date: Fri, 9 Jul 2021 10:38:38 +0200 Message-Id: X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 44906 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) The short option was listed in the help-text, but not recognized. --- guix/scripts/substitute.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 03115ffe44..c044e1d47a 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -777,7 +777,7 @@ default value." (loop)))))) ((or ("-V") ("--version")) (show-version-and-exit "guix substitute")) - (("--help") + ((or ("-h") ("--help")) (show-help)) (opts (leave (G_ "~a: unrecognized options~%") opts)))))) -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 09 04:38:56 2021 Received: (at 44906) by debbugs.gnu.org; 9 Jul 2021 08:38:56 +0000 Received: from localhost ([127.0.0.1]:58318 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1m2C-0000gM-8R for submit@debbugs.gnu.org; Fri, 09 Jul 2021 04:38:56 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:33256) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1m28-0000fu-Gb for 44906@debbugs.gnu.org; Fri, 09 Jul 2021 04:38:52 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4GLmlw09CHz1qtd6; Fri, 9 Jul 2021 10:38:52 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4GLmlw04Ftz1qr4R; Fri, 9 Jul 2021 10:38:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id 23ZFgpiZUpLp; Fri, 9 Jul 2021 10:38:51 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-58-132.dynamic.mnet-online.de [188.174.58.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Fri, 9 Jul 2021 10:38:51 +0200 (CEST) Received: from thisbe.goebel-consult.de (hermia.goebel-consult.de [192.168.110.7]) by hermia.goebel-consult.de (Postfix) with ESMTP id 67A3A601D6; Fri, 9 Jul 2021 10:38:46 +0200 (CEST) From: Hartmut Goebel To: 44906@debbugs.gnu.org, guix-patches@gnu.org Subject: [PATCH 2/3] substitutes: Properly construct URLs. Date: Fri, 9 Jul 2021 10:38:39 +0200 Message-Id: <26c8bb31c4d468e770e35cd743aca736f5ccd093.1625819848.git.h.goebel@crazy-compilers.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 44906 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Use relative URIs and "resolve-uri-reference" (which implements the algorithm specified in RFC 3986 section 5.2.2) for building the URL, instead of just appending strings. This avoids issued if the cache-url ends with a slash. * guix/substitutes.scm (narinfo-request): Use resolve-uri-reference for constructing the url. --- guix/substitutes.scm | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/guix/substitutes.scm b/guix/substitutes.scm index 4987cda165..a5c554acff 100644 --- a/guix/substitutes.scm +++ b/guix/substitutes.scm @@ -37,7 +37,8 @@ #:use-module ((guix build utils) #:select (mkdir-p dump-port)) #:use-module ((guix build download) #:select ((open-connection-for-uri - . guix:open-connection-for-uri))) + . guix:open-connection-for-uri) + resolve-uri-reference)) #:use-module (guix progress) #:use-module (ice-9 rdelim) #:use-module (ice-9 regex) @@ -155,10 +156,12 @@ indicates that PATH is unavailable at CACHE-URL." (define (narinfo-request cache-url path) "Return an HTTP request for the narinfo of PATH at CACHE-URL." - (let ((url (string-append cache-url "/" (store-path-hash-part path) - ".narinfo")) - (headers '((User-Agent . "GNU Guile")))) - (build-request (string->uri url) #:method 'GET #:headers headers))) + (let* ((base (string->uri cache-url)) + (ref (build-relative-ref + #:path (string-append (store-path-hash-part path) ".narinfo"))) + (url (resolve-uri-reference ref base)) + (headers '((User-Agent . "GNU Guile")))) + (build-request url #:method 'GET #:headers headers))) (define (narinfo-from-file file url) "Attempt to read a narinfo from FILE, using URL as the cache URL. Return #f -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 09 04:38:59 2021 Received: (at 44906) by debbugs.gnu.org; 9 Jul 2021 08:38:59 +0000 Received: from localhost ([127.0.0.1]:58323 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1m2E-0000ge-QQ for submit@debbugs.gnu.org; Fri, 09 Jul 2021 04:38:59 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:48169) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1m2C-0000gN-Tg for 44906@debbugs.gnu.org; Fri, 09 Jul 2021 04:38:57 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4GLmlz6lsXz1s8Nq; Fri, 9 Jul 2021 10:38:55 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4GLmlz3wjJz1qr4T; Fri, 9 Jul 2021 10:38:55 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id uTiNp5X4guTE; Fri, 9 Jul 2021 10:38:54 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-58-132.dynamic.mnet-online.de [188.174.58.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Fri, 9 Jul 2021 10:38:54 +0200 (CEST) Received: from thisbe.goebel-consult.de (hermia.goebel-consult.de [192.168.110.7]) by hermia.goebel-consult.de (Postfix) with ESMTP id F1CB56022C; Fri, 9 Jul 2021 10:38:46 +0200 (CEST) From: Hartmut Goebel To: 44906@debbugs.gnu.org, guix-patches@gnu.org Subject: [PATCH 3/3] ci: Properly construct URLs. Date: Fri, 9 Jul 2021 10:38:40 +0200 Message-Id: X-Mailer: git-send-email 2.30.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 44906 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Implement a new function "api-url", which constructs URLs using relative URI and "resolve-uri-reference" (which implements the algorithm specified in RFC 3986 section 5.2.2) for building the URL, instead of just appending strings. This avoids issued if the server-url ends with a slash. Since "api-url" uses URI-objects, it makes sense to also construct the query-part of the URL here. For this "api-url" accepts optional key-value-pairs. New function "json-api-fetch" is a wrapper using "api-url". * guix/ci.scm (api-url): New function. (build): Use it. (json-api-fetch): New function. (queued-builds, latest-builds, evaluation, latest-evaluations, evaluation-jobs: Use it. --- guix/ci.scm | 79 +++++++++++++++++++++++++++++++---------------------- 1 file changed, 46 insertions(+), 33 deletions(-) diff --git a/guix/ci.scm b/guix/ci.scm index dde93bbd53..cf39744567 100644 --- a/guix/ci.scm +++ b/guix/ci.scm @@ -20,9 +20,12 @@ (define-module (guix ci) #:use-module (guix http-client) #:use-module (guix utils) + #:use-module ((guix build download) + #:select (resolve-uri-reference)) #:use-module (json) #:use-module (srfi srfi-1) #:use-module (ice-9 match) + #:use-module (web uri) #:use-module (guix i18n) #:use-module (guix diagnostics) #:autoload (guix channels) (channel) @@ -146,16 +149,41 @@ ;; Max number of builds requested in queries. 1000) +(define* (api-url base-url path #:rest query) + "Build a proper API url, taking into account BASE_URL's trailing slashes." + + (define (build-query-string query) + (let lp ((query (or (reverse query) '())) (acc '())) + (match query + (() (string-concatenate acc)) + (((_ #f) . rest) (lp rest acc)) + (((name val) . rest) + (lp rest (cons* + name "=" + (if (string? val) (uri-encode val) (number->string val)) + (if (null? acc) "" "&") + acc)))))) + + (let* ((query-string (build-query-string query)) + (base (string->uri base-url)) + (ref (build-relative-ref #:path path #:query query-string))) + (resolve-uri-reference ref base))) + + (define (json-fetch url) (let* ((port (http-fetch url)) (json (json->scm port))) (close-port port) json)) +(define* (json-api-fetch base-url path #:rest query) + (json-fetch (apply api-url base-url path query))) + + (define* (queued-builds url #:optional (limit %query-limit)) "Return the list of queued derivations on URL." - (let ((queue (json-fetch (string-append url "/api/queue?nr=" - (number->string limit))))) + (let ((queue + (json-api-fetch url "/api/queue" `("nr" ,limit)))) (map json->build (vector->list queue)))) (define* (latest-builds url #:optional (limit %query-limit) @@ -163,28 +191,21 @@ "Return the latest builds performed by the CI server at URL. If EVALUATION is an integer, restrict to builds of EVALUATION. If SYSTEM is true (a system string such as \"x86_64-linux\"), restrict to builds for SYSTEM." - (define* (option name value #:optional (->string identity)) - (if value - (string-append "&" name "=" (->string value)) - "")) - - (let ((latest (json-fetch (string-append url "/api/latestbuilds?nr=" - (number->string limit) - (option "evaluation" evaluation - number->string) - (option "system" system) - (option "job" job) - (option "status" status - number->string))))) + (let ((latest (json-api-fetch + url "/api/latestbuilds" + `("nr" ,limit) + `("evaluation" ,evaluation) + `("system" ,system) + `("job" ,job) + `("status" ,status)))) ;; Note: Hydra does not provide a "derivation" field for entries in ;; 'latestbuilds', but Cuirass does. (map json->build (vector->list latest)))) (define (evaluation url evaluation) "Return the given EVALUATION performed by the CI server at URL." - (let ((evaluation (json-fetch - (string-append url "/api/evaluation?id=" - (number->string evaluation))))) + (let ((evaluation + (json-api-fetch url "/api/evaluation" `("id" ,evaluation)))) (json->evaluation evaluation))) (define* (latest-evaluations url @@ -192,16 +213,10 @@ string such as \"x86_64-linux\"), restrict to builds for SYSTEM." #:key spec) "Return the latest evaluations performed by the CI server at URL. If SPEC is passed, only consider the evaluations for the given SPEC specification." - (let ((spec (if spec - (format #f "&spec=~a" spec) - ""))) - (map json->evaluation - (vector->list - (json->scm - (http-fetch - (string-append url "/api/evaluations?nr=" - (number->string limit) - spec))))))) + (map json->evaluation + (vector->list + (json-api-fetch + url "/api/evaluations" `("nr" ,limit) `("spec" ,spec))))) (define* (evaluations-for-commit url commit #:optional (limit %query-limit)) "Return the evaluations among the latest LIMIT evaluations that have COMMIT @@ -216,16 +231,14 @@ as one of their inputs." "Return the list of jobs of evaluation EVALUATION-ID." (map json->job (vector->list - (json->scm (http-fetch - (string-append url "/api/jobs?evaluation=" - (number->string evaluation-id))))))) + (json-api-fetch url "/api/jobs" `("evaluation" ,evaluation-id))))) (define (build url id) "Look up build ID at URL and return it. Raise &http-get-error if it is not found (404)." (json->build - (http-fetch (string-append url "/build/" ;note: no "/api" here - (number->string id))))) + (http-fetch (api-url url (string-append "/build/" ;note: no "/api" here + (number->string id)))))) (define (job-build url job) "Return the build associated with JOB." -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 15 03:35:45 2021 Received: (at 44906) by debbugs.gnu.org; 15 Jul 2021 07:35:45 +0000 Received: from localhost ([127.0.0.1]:46677 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m3vuK-0006zR-Rv for submit@debbugs.gnu.org; Thu, 15 Jul 2021 03:35:45 -0400 Received: from eggs.gnu.org ([209.51.188.92]:39534) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m3vuJ-0006zA-8m; Thu, 15 Jul 2021 03:35:43 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:39612) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m3vuD-0002PG-CE; Thu, 15 Jul 2021 03:35:37 -0400 Received: from [2a01:e0a:19b:d9a0:f2f7:a404:c3d3:f8b4] (port=34772 helo=meije) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m3vu8-0005j1-Vf; Thu, 15 Jul 2021 03:35:36 -0400 From: Mathieu Othacehe To: Hartmut Goebel Subject: Re: [bug#49482] [PATCH 3/3] ci: Properly construct URLs. References: Date: Thu, 15 Jul 2021 09:35:31 +0200 In-Reply-To: (Hartmut Goebel's message of "Fri, 9 Jul 2021 10:38:40 +0200") Message-ID: <871r80do70.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 44906 Cc: 44906@debbugs.gnu.org, 49482@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Hartmut, Thanks for this patchset! > +(define* (api-url base-url path #:rest query) > + "Build a proper API url, taking into account BASE_URL's trailing slashes." s/BASE_URL/BASE-URL/ You could also indicate what is the expect format for query: '("name" "value") lists. > + (((_ #f) . rest) (lp rest acc)) > + (((name val) . rest) > + (lp rest (cons* > + name "=" > + (if (string? val) (uri-encode val) (number->string val)) What about booleans? False is filtered above but true will throw an exception. > + (resolve-uri-reference ref base))) > + > + There's an extra new line here. > +(define* (json-api-fetch base-url path #:rest query) > + (json-fetch (apply api-url base-url path query))) > + > + Here also. Otherwise, it looks nice :) Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 16 13:55:50 2021 Received: (at 44906) by debbugs.gnu.org; 16 Jul 2021 17:55:51 +0000 Received: from localhost ([127.0.0.1]:52275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m4S3y-0005vQ-Mz for submit@debbugs.gnu.org; Fri, 16 Jul 2021 13:55:50 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:52330) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m4S3u-0005vB-M3; Fri, 16 Jul 2021 13:55:49 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4GRJnF2mFPz1qv97; Fri, 16 Jul 2021 19:55:45 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4GRJnF0ymMz1qqkH; Fri, 16 Jul 2021 19:55:45 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id NLmtX6VXw9Ko; Fri, 16 Jul 2021 19:55:44 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-58-97.dynamic.mnet-online.de [188.174.58.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Fri, 16 Jul 2021 19:55:44 +0200 (CEST) Received: from thisbe.goebel-consult.de (hermia.goebel-consult.de [192.168.110.7]) by hermia.goebel-consult.de (Postfix) with ESMTP id 0E1B26014D; Fri, 16 Jul 2021 19:55:56 +0200 (CEST) Subject: Re: [bug#49482] [PATCH 3/3] ci: Properly construct URLs. To: Mathieu Othacehe References: <871r80do70.fsf@gnu.org> From: Hartmut Goebel Organization: crazy-compilers.com Message-ID: <2d5afdf0-5e6b-b949-057e-2a1cb24b9703@crazy-compilers.com> Date: Fri, 16 Jul 2021 19:55:41 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <871r80do70.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 44906 Cc: 44906@debbugs.gnu.org, 49482@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Mathieu, thanks for the review. I updated the doc-string, fixed the other parts and pushed as 3ee0f170c8bd883728d8abb2c2e00f445c13f17d. > What about booleans? False is filtered above but true will throw an > exception. False is used to omit elements from the query-string. Booleans and other types are not handled, since this low-level function doesn't know how to convert them into a string to be put into the query. #t could be "1", "t", "true", depending on the API used. -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 16 13:58:46 2021 Received: (at control) by debbugs.gnu.org; 16 Jul 2021 17:58:46 +0000 Received: from localhost ([127.0.0.1]:52289 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m4S6o-000603-Fk for submit@debbugs.gnu.org; Fri, 16 Jul 2021 13:58:46 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:49935) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m4S6m-0005zu-9y for control@debbugs.gnu.org; Fri, 16 Jul 2021 13:58:45 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4GRJrg47mKz1sF4d for ; Fri, 16 Jul 2021 19:58:43 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4GRJrg1JnTz1qqkH for ; Fri, 16 Jul 2021 19:58:43 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id iAk6yx_Bh6l0 for ; Fri, 16 Jul 2021 19:58:42 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-58-97.dynamic.mnet-online.de [188.174.58.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS for ; Fri, 16 Jul 2021 19:58:42 +0200 (CEST) Received: from thisbe.goebel-consult.de (hermia.goebel-consult.de [192.168.110.7]) by hermia.goebel-consult.de (Postfix) with ESMTP id 817E26014D for ; Fri, 16 Jul 2021 19:58:54 +0200 (CEST) To: control From: Hartmut Goebel Subject: close Organization: Goebel Consult Message-ID: Date: Fri, 16 Jul 2021 19:58:40 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: de-DE X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) close 44906 49483 49482 From unknown Sat Jun 21 10:21:22 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 14 Aug 2021 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator