From unknown Mon Jun 23 04:11:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#36976] [PATCH 1/1] download: Map file-name characters not allowed in store. Resent-From: Hartmut Goebel Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 08 Aug 2019 14:46:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 36976 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 36976@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.156527550710800 (code B ref -1); Thu, 08 Aug 2019 14:46:02 +0000 Received: (at submit) by debbugs.gnu.org; 8 Aug 2019 14:45:07 +0000 Received: from localhost ([127.0.0.1]:41571 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hvjf4-0002nk-1X for submit@debbugs.gnu.org; Thu, 08 Aug 2019 10:45:06 -0400 Received: from lists.gnu.org ([209.51.188.17]:46512) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hvjf1-0002nJ-Db for submit@debbugs.gnu.org; Thu, 08 Aug 2019 10:44:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58355) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hvjey-0002Ik-IR for guix-patches@gnu.org; Thu, 08 Aug 2019 10:44:59 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=BAYES_40,RCVD_IN_DNSWL_LOW, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hvjev-0002F9-68 for guix-patches@gnu.org; Thu, 08 Aug 2019 10:44:56 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:41662) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hvjeu-00029h-SG for guix-patches@gnu.org; Thu, 08 Aug 2019 10:44:53 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 464B3l0ZXVz1rKXK for ; Thu, 8 Aug 2019 16:44:50 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 464B3k6yHDz1qqkD for ; Thu, 8 Aug 2019 16:44:50 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id A9H_pyzXdR0Y for ; Thu, 8 Aug 2019 16:44:49 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-62-181.dynamic.mnet-online.de [188.174.62.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS for ; Thu, 8 Aug 2019 16:44:49 +0200 (CEST) Received: from lenashee.goebel-consult.de (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id 916B4601CC for ; Thu, 8 Aug 2019 16:45:43 +0200 (CEST) From: Hartmut Goebel Date: Thu, 8 Aug 2019 16:44:48 +0200 Message-Id: <20190808144448.25147-1-h.goebel@crazy-compilers.com> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.18.0.9 X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) In the file-name to be used for storing into the store, replace any character not allowed in the store-file-name by an underscore. This is only done when NAME is not given and thus defaults to toe URL's basename. If NAME is given, this is used unchanged, allowing for more control by other functions. Fixes * guix/download.scm (safe-name): New function. (download-to-store): NAME defaults to the "safe" basename of URL. --- guix/download.scm | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/guix/download.scm b/guix/download.scm index b24aaa0a86..249f612237 100644 --- a/guix/download.scm +++ b/guix/download.scm @@ -6,6 +6,7 @@ ;;; Copyright =C2=A9 2016 David Craven ;;; Copyright =C2=A9 2017 Tobias Geerinckx-Rice ;;; Copyright =C2=A9 2019 Guy Fleury Iteriteka +;;; Copyright =C2=A9 2019 Hartmut Goeel ;;; ;;; This file is part of GNU Guix. ;;; @@ -563,13 +564,27 @@ own. This helper makes it easier to deal with \"zi= p bombs\"." #:graft? #f #:local-build? #t))) =20 -(define* (download-to-store store url #:optional (name (basename url)) +(define (safe-name name) + "Replace any character not allowed in a stroe name by an underscore." + + (define valid-characters + ;; according to nix/libstore/store-api.cc + (string->list (string-append "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "abcdefghijklmnopqrstuvwxyz" + "0123456789" "+-._?=3D"))) + (string-map (lambda (c) + (if (member c valid-characters) c #\_)) + name)) + +(define* (download-to-store store url + #:optional (name (safe-name (basename url))) #:key (log (current-error-port)) recursive? (verify-certificate? #t)) - "Download from URL to STORE, either under NAME or URL's basename if -omitted. Write progress reports to LOG. RECURSIVE? has the same effect= as -the same-named parameter of 'add-to-store'. VERIFY-CERTIFICATE? determi= nes -whether or not to validate HTTPS server certificates." + "Download from URL to STORE, either under NAME. If NAME is omitted, UR= L's +basename with invalid characters replaced is used. Write progress repor= ts to +LOG. RECURSIVE? has the same effect as the same-named parameter of +'add-to-store'. VERIFY-CERTIFICATE? determines whether or not to valida= te +HTTPS server certificates." (define uri (string->uri url)) =20 --=20 2.21.0 From unknown Mon Jun 23 04:11:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#36976] [PATCH 1/1] download: Map file-name characters not allowed in store. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 23 Aug 2019 21:09:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36976 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Hartmut Goebel Cc: 36976@debbugs.gnu.org Received: via spool by 36976-submit@debbugs.gnu.org id=B36976.156659451912787 (code B ref 36976); Fri, 23 Aug 2019 21:09:01 +0000 Received: (at 36976) by debbugs.gnu.org; 23 Aug 2019 21:08:39 +0000 Received: from localhost ([127.0.0.1]:41312 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i1GnW-0003KA-R6 for submit@debbugs.gnu.org; Fri, 23 Aug 2019 17:08:39 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52682) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i1GnV-0003Jw-7b for 36976@debbugs.gnu.org; Fri, 23 Aug 2019 17:08:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51142) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i1GnP-0004vh-PC; Fri, 23 Aug 2019 17:08:31 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52766 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i1GnP-0000i2-BV; Fri, 23 Aug 2019 17:08:31 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20190808144448.25147-1-h.goebel@crazy-compilers.com> Date: Fri, 23 Aug 2019 23:08:29 +0200 In-Reply-To: <20190808144448.25147-1-h.goebel@crazy-compilers.com> (Hartmut Goebel's message of "Thu, 8 Aug 2019 16:44:48 +0200") Message-ID: <874l271tf6.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, Hartmut Goebel skribis: > In the file-name to be used for storing into the store, replace any > character not allowed in the store-file-name by an underscore. > This is only done when NAME is not given and thus defaults to > toe URL's basename. If NAME is given, this is used unchanged, > allowing for more control by other functions. > > Fixes > > * guix/download.scm (safe-name): New function. > (download-to-store): NAME defaults to the "safe" basename of URL. What about moving this automatic renaming feature to (guix scripts download)? I=E2=80=99d rather not do it in the core APIs. > +(define (safe-name name) > + "Replace any character not allowed in a stroe name by an underscore." ^^ Typo. I=E2=80=99d call it =E2=80=98ensure-valid-store-file-name=E2=80=99 or simil= ar, WDYT? > + (define valid-characters > + ;; according to nix/libstore/store-api.cc > + (string->list (string-append "ABCDEFGHIJKLMNOPQRSTUVWXYZ" > + "abcdefghijklmnopqrstuvwxyz" > + "0123456789" "+-._?=3D"))) > + (string-map (lambda (c) > + (if (member c valid-characters) c #\_)) > + name)) Instead of a list, please use a SRFI-14 =E2=80=9Ccharacter set=E2=80=9D, li= ke this: (define valid (string->char-set =E2=80=A6)) (string-map (lambda (c) (if (char-set-contains? valid c) =E2=80=A6)) name) Thanks, Ludo=E2=80=99. From unknown Mon Jun 23 04:11:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#36976] [PATCH 1/1] download: Map file-name characters not allowed in store. Resent-From: Hartmut Goebel Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 27 Aug 2019 07:54:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36976 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 36976@debbugs.gnu.org Received: via spool by 36976-submit@debbugs.gnu.org id=B36976.156689239319604 (code B ref 36976); Tue, 27 Aug 2019 07:54:02 +0000 Received: (at 36976) by debbugs.gnu.org; 27 Aug 2019 07:53:13 +0000 Received: from localhost ([127.0.0.1]:47619 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i2WHx-000568-GV for submit@debbugs.gnu.org; Tue, 27 Aug 2019 03:53:13 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:35122) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i2WHv-00055y-6B for 36976@debbugs.gnu.org; Tue, 27 Aug 2019 03:53:11 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 46Hh1y1CSKz1rK4R; Tue, 27 Aug 2019 09:53:10 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 46Hh1y0GHDz1qqkt; Tue, 27 Aug 2019 09:53:10 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id 57jhjvjE_S79; Tue, 27 Aug 2019 09:53:09 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-55-68.dynamic.mnet-online.de [188.174.55.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Tue, 27 Aug 2019 09:53:09 +0200 (CEST) Received: from [192.168.110.2] (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id 600986018D; Tue, 27 Aug 2019 09:54:45 +0200 (CEST) References: <20190808144448.25147-1-h.goebel@crazy-compilers.com> <874l271tf6.fsf@gnu.org> From: Hartmut Goebel Openpgp: preference=signencrypt Organization: crazy-compilers.com Message-ID: Date: Tue, 27 Aug 2019 09:53:08 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <874l271tf6.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Spam-Score: 0.8 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.2 (/) Hi, Thanks for the review and the coding suggestions.. Am 23.08.19 um 23:08 schrieb Ludovic Courtès: >> * guix/download.scm (safe-name): New function. >> (download-to-store): NAME defaults to the "safe" basename of URL. > What about moving this automatic renaming feature to (guix scripts > download)? I’d rather not do it in the core APIs. `download-to-store store` was defined as:    (define* (download-to-store store url #:optional (name (basename url)) … When developing this patch, I decided to put it into the core since users of this function would expect to be allowed to just pass any url and don't need to take care about valid characters. If not doing the automatic renaming here, users would need to perform the conversion prior to calling this function in any case (except when 100% sure only valid characters are used). -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | From unknown Mon Jun 23 04:11:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#36976] [PATCH 1/1] download: Map file-name characters not allowed in store. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 01 Sep 2019 19:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36976 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Hartmut Goebel Cc: 36976@debbugs.gnu.org Received: via spool by 36976-submit@debbugs.gnu.org id=B36976.156736679112438 (code B ref 36976); Sun, 01 Sep 2019 19:40:01 +0000 Received: (at 36976) by debbugs.gnu.org; 1 Sep 2019 19:39:51 +0000 Received: from localhost ([127.0.0.1]:57250 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4VhX-0003EY-2J for submit@debbugs.gnu.org; Sun, 01 Sep 2019 15:39:51 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43196) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4VhU-0003EK-2j for 36976@debbugs.gnu.org; Sun, 01 Sep 2019 15:39:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51422) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i4VhO-0007LV-2q; Sun, 01 Sep 2019 15:39:42 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40588 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i4VhN-00030U-Iv; Sun, 01 Sep 2019 15:39:41 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20190808144448.25147-1-h.goebel@crazy-compilers.com> <874l271tf6.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 15 Fructidor an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 01 Sep 2019 21:39:39 +0200 In-Reply-To: (Hartmut Goebel's message of "Tue, 27 Aug 2019 09:53:08 +0200") Message-ID: <87woerajr8.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, Hartmut Goebel skribis: > Thanks for the review and the coding suggestions.. > > Am 23.08.19 um 23:08 schrieb Ludovic Court=C3=A8s: >>> * guix/download.scm (safe-name): New function. >>> (download-to-store): NAME defaults to the "safe" basename of URL. >> What about moving this automatic renaming feature to (guix scripts >> download)? I=E2=80=99d rather not do it in the core APIs. > `download-to-store store` was defined as: > > =C2=A0=C2=A0 (define* (download-to-store store url #:optional (name (base= name url)) =E2=80=A6 > > When developing this patch, I decided to put it into the core since > users of this function would expect to be allowed to just pass any url > and don't need to take care about valid characters. If not doing the > automatic renaming here, users would need to perform the conversion > prior to calling this function in any case (except when 100% sure only > valid characters are used). Yes, but that=E2=80=99s OK to me: IMO, procedures have to focused on one th= ing; users can perform additional processing beforehand if they need it. Conversely, commands have to do the right thing by default, which is why I agree that =E2=80=98guix download=E2=80=99 should rename automatically wh= en needed. How does that sound? Ludo=E2=80=99. From unknown Mon Jun 23 04:11:26 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#36976] [Patch v2] guix download: Ensure destination file-name is valid in the store. References: <20190808144448.25147-1-h.goebel@crazy-compilers.com> In-Reply-To: <20190808144448.25147-1-h.goebel@crazy-compilers.com> Resent-From: Hartmut Goebel Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 03 Sep 2019 16:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36976 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 36976@debbugs.gnu.org Received: via spool by 36976-submit@debbugs.gnu.org id=B36976.15675276352568 (code B ref 36976); Tue, 03 Sep 2019 16:21:02 +0000 Received: (at 36976) by debbugs.gnu.org; 3 Sep 2019 16:20:35 +0000 Received: from localhost ([127.0.0.1]:60503 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i5BXn-0000fM-0s for submit@debbugs.gnu.org; Tue, 03 Sep 2019 12:20:35 -0400 Received: from mail-out.m-online.net ([212.18.0.10]:46348) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i5BXj-0000f8-TO for 36976@debbugs.gnu.org; Tue, 03 Sep 2019 12:20:33 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 46NBy64f3Bz1rf76 for <36976@debbugs.gnu.org>; Tue, 3 Sep 2019 18:20:30 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 46NBy63pC7z1qqkB for <36976@debbugs.gnu.org>; Tue, 3 Sep 2019 18:20:30 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id NIXqPboy2Dsr for <36976@debbugs.gnu.org>; Tue, 3 Sep 2019 18:20:29 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-49-62.dynamic.mnet-online.de [188.174.49.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS for <36976@debbugs.gnu.org>; Tue, 3 Sep 2019 18:20:29 +0200 (CEST) Received: from lenashee.goebel-consult.de (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id A606F60113 for <36976@debbugs.gnu.org>; Tue, 3 Sep 2019 18:22:21 +0200 (CEST) From: Hartmut Goebel Date: Tue, 3 Sep 2019 18:20:28 +0200 Message-Id: <20190903162028.5373-1-h.goebel@crazy-compilers.com> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.8 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.2 (/) Avoid invalid store-file-name by explicitly passing the destination name, replacing any character not allowed in the store-file-name by an underscore. Fixes * guix/scripts/download.scm (safe-naensure-valid-store-file-nameme): New function. (download-to-store*): Use it to generate a "safe" basename of URL. --- guix/scripts/download.scm | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/guix/scripts/download.scm b/guix/scripts/download.scm index d8fe71ce12..22cd75ea0b 100644 --- a/guix/scripts/download.scm +++ b/guix/scripts/download.scm @@ -33,6 +33,7 @@ #:use-module (web uri) #:use-module (ice-9 match) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-14) #:use-module (srfi srfi-26) #:use-module (srfi srfi-37) #:use-module (rnrs bytevectors) @@ -54,9 +55,23 @@ (url-fetch url file #:mirrors %mirrors))) file)) +(define (ensure-valid-store-file-name name) + "Replace any character not allowed in a stror name by an underscore." + + (define valid + ;; according to nix/libstore/store-api.cc + (string->char-set (string-append "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "abcdefghijklmnopqrstuvwxyz" + "0123456789" "+-._?="))) + (string-map (lambda (c) + (if (char-set-contains? valid c) c #\_)) + name)) + + (define* (download-to-store* url #:key (verify-certificate? #t)) (with-store store (download-to-store store url + (ensure-valid-store-file-name (basename url)) #:verify-certificate? verify-certificate?))) (define %default-options -- 2.21.0 From unknown Mon Jun 23 04:11:26 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Hartmut Goebel Subject: bug#36976: closed (Re: [bug#36976] [PATCH 1/1] download: Map file-name characters not allowed in store.) Message-ID: References: <20190808144448.25147-1-h.goebel@crazy-compilers.com> X-Gnu-PR-Message: they-closed 36976 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 36976@debbugs.gnu.org Date: Thu, 26 Sep 2019 15:52:04 +0000 Content-Type: multipart/mixed; boundary="----------=_1569513124-27800-1" This is a multi-part message in MIME format... ------------=_1569513124-27800-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #36976: [PATCH 1/1] download: Map file-name characters not allowed in store. which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 36976@debbugs.gnu.org. --=20 36976: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D36976 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1569513124-27800-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 36976-close) by debbugs.gnu.org; 26 Sep 2019 15:51:07 +0000 Received: from localhost ([127.0.0.1]:44405 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iDW2t-0007DA-BX for submit@debbugs.gnu.org; Thu, 26 Sep 2019 11:51:07 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:55528) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iDW2p-0007CZ-1m for 36976-close@debbugs.gnu.org; Thu, 26 Sep 2019 11:51:06 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 46fKCR5XG5z1rKJp; Thu, 26 Sep 2019 17:50:59 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 46fKCR5C1lz1qqkL; Thu, 26 Sep 2019 17:50:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id 0CgMzZSxbYYA; Thu, 26 Sep 2019 17:50:59 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-55-104.dynamic.mnet-online.de [188.174.55.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Thu, 26 Sep 2019 17:50:59 +0200 (CEST) Received: from thisbe.goebel-consult.de (hermia.goebel-consult.de [192.168.110.7]) by hermia.goebel-consult.de (Postfix) with ESMTP id 549DE60084; Thu, 26 Sep 2019 17:53:42 +0200 (CEST) Subject: Re: [bug#36976] [PATCH 1/1] download: Map file-name characters not allowed in store. References: <20190808144448.25147-1-h.goebel@crazy-compilers.com> <874l271tf6.fsf@gnu.org> <87woerajr8.fsf@gnu.org> To: 36976-close@debbugs.gnu.org From: Hartmut Goebel Openpgp: preference=signencrypt Organization: crazy-compilers.com Message-ID: Date: Thu, 26 Sep 2019 17:50:57 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <87woerajr8.fsf@gnu.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 36976-close Cc: =?UTF-8?Q?Ludovic_Court=c3=a8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Committed as dec845606d2d184da31065fa26cd951b84b3ce2d Thank for the review. -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible | ------------=_1569513124-27800-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 8 Aug 2019 14:45:07 +0000 Received: from localhost ([127.0.0.1]:41571 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hvjf4-0002nk-1X for submit@debbugs.gnu.org; Thu, 08 Aug 2019 10:45:06 -0400 Received: from lists.gnu.org ([209.51.188.17]:46512) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hvjf1-0002nJ-Db for submit@debbugs.gnu.org; Thu, 08 Aug 2019 10:44:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58355) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hvjey-0002Ik-IR for guix-patches@gnu.org; Thu, 08 Aug 2019 10:44:59 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=BAYES_40,RCVD_IN_DNSWL_LOW, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hvjev-0002F9-68 for guix-patches@gnu.org; Thu, 08 Aug 2019 10:44:56 -0400 Received: from mail-out.m-online.net ([212.18.0.9]:41662) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hvjeu-00029h-SG for guix-patches@gnu.org; Thu, 08 Aug 2019 10:44:53 -0400 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 464B3l0ZXVz1rKXK for ; Thu, 8 Aug 2019 16:44:50 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 464B3k6yHDz1qqkD for ; Thu, 8 Aug 2019 16:44:50 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id A9H_pyzXdR0Y for ; Thu, 8 Aug 2019 16:44:49 +0200 (CEST) Received: from hermia.goebel-consult.de (ppp-188-174-62-181.dynamic.mnet-online.de [188.174.62.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS for ; Thu, 8 Aug 2019 16:44:49 +0200 (CEST) Received: from lenashee.goebel-consult.de (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id 916B4601CC for ; Thu, 8 Aug 2019 16:45:43 +0200 (CEST) From: Hartmut Goebel To: guix-patches@gnu.org Subject: [PATCH 1/1] download: Map file-name characters not allowed in store. Date: Thu, 8 Aug 2019 16:44:48 +0200 Message-Id: <20190808144448.25147-1-h.goebel@crazy-compilers.com> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.18.0.9 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) In the file-name to be used for storing into the store, replace any character not allowed in the store-file-name by an underscore. This is only done when NAME is not given and thus defaults to toe URL's basename. If NAME is given, this is used unchanged, allowing for more control by other functions. Fixes * guix/download.scm (safe-name): New function. (download-to-store): NAME defaults to the "safe" basename of URL. --- guix/download.scm | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/guix/download.scm b/guix/download.scm index b24aaa0a86..249f612237 100644 --- a/guix/download.scm +++ b/guix/download.scm @@ -6,6 +6,7 @@ ;;; Copyright =C2=A9 2016 David Craven ;;; Copyright =C2=A9 2017 Tobias Geerinckx-Rice ;;; Copyright =C2=A9 2019 Guy Fleury Iteriteka +;;; Copyright =C2=A9 2019 Hartmut Goeel ;;; ;;; This file is part of GNU Guix. ;;; @@ -563,13 +564,27 @@ own. This helper makes it easier to deal with \"zi= p bombs\"." #:graft? #f #:local-build? #t))) =20 -(define* (download-to-store store url #:optional (name (basename url)) +(define (safe-name name) + "Replace any character not allowed in a stroe name by an underscore." + + (define valid-characters + ;; according to nix/libstore/store-api.cc + (string->list (string-append "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "abcdefghijklmnopqrstuvwxyz" + "0123456789" "+-._?=3D"))) + (string-map (lambda (c) + (if (member c valid-characters) c #\_)) + name)) + +(define* (download-to-store store url + #:optional (name (safe-name (basename url))) #:key (log (current-error-port)) recursive? (verify-certificate? #t)) - "Download from URL to STORE, either under NAME or URL's basename if -omitted. Write progress reports to LOG. RECURSIVE? has the same effect= as -the same-named parameter of 'add-to-store'. VERIFY-CERTIFICATE? determi= nes -whether or not to validate HTTPS server certificates." + "Download from URL to STORE, either under NAME. If NAME is omitted, UR= L's +basename with invalid characters replaced is used. Write progress repor= ts to +LOG. RECURSIVE? has the same effect as the same-named parameter of +'add-to-store'. VERIFY-CERTIFICATE? determines whether or not to valida= te +HTTPS server certificates." (define uri (string->uri url)) =20 --=20 2.21.0 ------------=_1569513124-27800-1--