From unknown Wed Jun 18 23:04:34 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#49196 <49196@debbugs.gnu.org> To: bug#49196 <49196@debbugs.gnu.org> Subject: Status: [PATCH] "guix import go" Improve error handling Reply-To: bug#49196 <49196@debbugs.gnu.org> Date: Thu, 19 Jun 2025 06:04:34 +0000 retitle 49196 [PATCH] "guix import go" Improve error handling reassign 49196 guix-patches submitter 49196 Sarah Morgensen severity 49196 normal tag 49196 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 23 16:49:29 2021 Received: (at submit) by debbugs.gnu.org; 23 Jun 2021 20:49:29 +0000 Received: from localhost ([127.0.0.1]:41920 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lw9oI-0005GV-JF for submit@debbugs.gnu.org; Wed, 23 Jun 2021 16:49:29 -0400 Received: from lists.gnu.org ([209.51.188.17]:38158) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lw9oF-0005GM-Pz for submit@debbugs.gnu.org; Wed, 23 Jun 2021 16:49:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51132) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lw9oE-00012h-2n for guix-patches@gnu.org; Wed, 23 Jun 2021 16:49:18 -0400 Received: from out0.migadu.com ([94.23.1.103]:41681) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lw9o9-0002S5-8I for guix-patches@gnu.org; Wed, 23 Jun 2021 16:49:17 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1624481348; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=GIqMS4yx7U+W6hvM9xbuYm1hHoQwWWIqtyxX/iDNpsA=; b=eojOVO7SEM39PfXQZC0DWvOKwBhvhqlo+Mwz44Hq/AySf9/71AMvlVdLsRahuoo75XDN6d YZThzBqwePkOTbpmkTjWj45b4+j589tHUNx4jGCunA+a9a5fMTijUJuTEmOiD/nz0VMPwL DnBU2L1fLxQha7pyhgU4023rPYNq1hA= From: Sarah Morgensen To: guix-patches@gnu.org Subject: [PATCH] import: utils: 'recursive-import' skips unfound packages Date: Wed, 23 Jun 2021 13:49:05 -0700 Message-Id: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev Received-SPF: pass client-ip=94.23.1.103; envelope-from=iskarian@mgsn.dev; helo=out0.migadu.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.6 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.9 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/import/utils.scm (recursive-import): Skip packages when the package returned by repo->guix-package is false. * guix/import/go.scm (go-module-recursive-import): Explicitly return false in 'repo- [...] Content analysis details: (1.9 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.9 SPF_FAIL SPF: sender does not match SPF record (fail) [SPF failed: Please see http://www.openspf.org/Why?s=mfrom;id=iskarian%40mgsn.dev;ip=209.51.188.17;r=debbugs.gnu.org] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager * guix/import/utils.scm (recursive-import): Skip packages when the package returned by repo->guix-package is false. * guix/import/go.scm (go-module-recursive-import): Explicitly return false in 'repo->guix-package' when packages are not found. * tests/import-utils.scm ("recursive-import: skip false packages"): New test. --- Hello Guix, While trying to recursively import a go package I encountered the following error when it was unable to find one of the dependencies: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix import go -r git.sr.ht/~sircmpwn/aerc guix import: warning: Failed to import package "github.com/brunnre8/go.notmuch". reason: "https://pkg.go.dev/github.com/brunnre8/go.notmuch" could not be fetched: HTTP error 404 ("Not Found"). This package and its dependencies won't be imported. following redirection to `https://github.com/ProtonMail/go-crypto?go-get=1'... following redirection to `https://github.com/jtolio/gls?go-get=1'... Backtrace: In srfi/srfi-1.scm: 586:29 19 (map1 ((package (name "go-github-com-rivo-uniseg") …) …)) 586:29 18 (map1 ((package (name "go-github-com-mattn-go-r…") …) …)) 586:29 17 (map1 ((package (name "go-github-com-miolini-da…") …) …)) 586:29 16 (map1 ((package (name "go-github-com-riywo-logi…") …) …)) 586:29 15 (map1 ((package (name "go-github-com-neelance-a…") …) …)) 586:29 14 (map1 ((package (name "go-github-com-neelance-s…") …) …)) 586:29 13 (map1 ((package (name "go-github-com-shurcool-go") …) …)) 586:29 12 (map1 ((package (name "go-github-com-shurcool-h…") …) …)) 586:29 11 (map1 ((package (name "go-github-com-shurcool-v…") …) …)) 586:29 10 (map1 ((package (name "go-github-com-gopherjs-g…") …) …)) 586:29 9 (map1 ((package (name "go-github-com-jtolds-gls") # …) …)) 586:29 8 (map1 ((package (name "go-github-com-smartystre…") …) …)) 586:29 7 (map1 ((package (name "go-github-com-smartystre…") …) …)) 586:29 6 (map1 ((package (name "go-github-com-google-go-…") …) …)) 586:29 5 (map1 ((package (name "go-google-golang-org-pro…") …) …)) 586:29 4 (map1 ((package (name "go-github-com-golang-pro…") …) …)) 586:29 3 (map1 ((package (name "go-google-golang-org-app…") …) …)) 586:29 2 (map1 ((package (name "go-github-com-protonmail…") …) …)) 586:17 1 (map1 (() (package (name "go-git-sr-ht-~sircmpwn…") …))) In guix/import/utils.scm: 280:2 0 (package->definition _ _) guix/import/utils.scm:280:2: In procedure package->definition: Throw to key `match-error' with args `("match" "no matching pattern" ())'. --8<---------------cut here---------------end--------------->8--- It seems like several importers (gem, egg, go) expect to be able to return #f or '() in #:repo->guix-package when a package is not found (while printing a warning to the user) but recursive-import doesn't look like it handles it. I suppose this worked before but was silently broken at some point. This patch (re-)enables this behavior. I added a test for this as well. Hope this helps, Sarah guix/import/go.scm | 3 ++- guix/import/utils.scm | 14 ++++++++------ tests/import-utils.scm | 18 ++++++++++++++++++ 3 files changed, 28 insertions(+), 7 deletions(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index d110954664..c859098492 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2021 Maxim Cournoyer ;;; Copyright © 2021 Ludovic Courtès ;;; Copyright © 2021 Xinglu Chen +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -651,7 +652,7 @@ This package and its dependencies won't be imported.~%") (uri->string (http-get-error-uri c)) (http-get-error-code c) (http-get-error-reason c)) - (values '() '()))) + (values #f '()))) (receive (package-sexp dependencies) (go-module->guix-package* name #:goproxy goproxy #:version version diff --git a/guix/import/utils.scm b/guix/import/utils.scm index d817318a91..49f38cfa2a 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com> ;;; Copyright © 2020 Martin Becze ;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -469,16 +470,17 @@ to obtain the Guix package name corresponding to the upstream name." (normalized-deps (map (match-lambda ((name version) (list name version)) (name (list name #f))) dependencies))) - (make-node name version package normalized-deps))) + (and package + (make-node name version package normalized-deps)))) (map node-package (topological-sort (list (lookup-node package-name version)) (lambda (node) - (map (lambda (name-version) - (apply lookup-node name-version)) - (remove (lambda (name-version) - (apply exists? name-version)) - (node-dependencies node)))) + (filter-map (lambda (name-version) + (apply lookup-node name-version)) + (remove (lambda (name-version) + (apply exists? name-version)) + (node-dependencies node)))) (lambda (node) (string-append (node-name node) diff --git a/tests/import-utils.scm b/tests/import-utils.scm index 874816442e..1b728104e0 100644 --- a/tests/import-utils.scm +++ b/tests/import-utils.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2017 Ricardo Wurmus ;;; Copyright © 2016 Ben Woodcroft ;;; Copyright © 2020 Martin Becze +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -64,6 +65,23 @@ '()))) #:guix-name identity)) +(test-equal "recursive-import: skip false packages" + '((package + (name "foo") + (inputs `(("bar" ,bar))))) + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values '(package + (name "foo") + (inputs `(("bar" ,bar)))) + '("bar"))) + (("bar" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + (test-assert "alist->package with simple source" (let* ((meta '(("name" . "hello") ("version" . "2.10") base-commit: acfa55a581ca4e688ee4b8f860fe365b1f153ef9 -- 2.31.1 From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 24 08:22:41 2021 Received: (at 49196) by debbugs.gnu.org; 24 Jun 2021 12:22:41 +0000 Received: from localhost ([127.0.0.1]:42591 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwONN-0008HK-EI for submit@debbugs.gnu.org; Thu, 24 Jun 2021 08:22:41 -0400 Received: from mail-wm1-f43.google.com ([209.85.128.43]:54257) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwONL-0008H4-JH for 49196@debbugs.gnu.org; Thu, 24 Jun 2021 08:22:32 -0400 Received: by mail-wm1-f43.google.com with SMTP id w13so3833069wmc.3 for <49196@debbugs.gnu.org>; Thu, 24 Jun 2021 05:22:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=lbGWsfwxQyBkXcUFnLP4I5PV6YgDveXoE7i4ODBN1IY=; b=bZQhLBJDABwcw3cK1oo5kX8vyu3txaWEduG46MUgM0Vyoh6XM7jGmDkz1tXNbxWR7h ad1asPVVb4uPHxMImVU3JZuon3HFDAOAI0FME/OnSSG2VXm9PkrtqxCar9vewceKeUXz rJN9KywAWRj2KgUSwT1YZxQLIRGb0y3pxscQC4Uy2D1+bnu2UElRMQsLBbqe5sA32p83 CwgWB9HLlqM64Fjhud0wpucuBnXcIvE7vbrQgvjB+VPr109j6ctjQ11bcD5RC5aR/gNw h2vfFg9pGtH5VzgmBddXbhHS0c7miaHs4WHcnz5Pp9W95YwZZscmPSi7mbG4zg9zGIbW vkcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=lbGWsfwxQyBkXcUFnLP4I5PV6YgDveXoE7i4ODBN1IY=; b=mrzdNS0Q7a/Lab/UerJXuA8rBOaINtOX0PQTBshCw2Kijz95PonXiZJ4fCwnEeOGcF 9Z9gJCeH1ClR5AwgfhjnxOi6SQOOo59t1PnBGjLXJg6al/3z5IQTJkNPOtDFPkym7BpO 62skuRKEO2hXhay1cJp0vzUWSnf265WrOe9/PbRcGIjOam+PpUJlBjHTr1CN6/hBo8dY d4+15/bQNPqZmTsdprQwl8I/O0htiRAA4ZI7OxfHBGWpTs1Yre+/FL2ZIjzcUpzMT0ED d1mnofP4U0i9Q2XgpWCztg3Ldxx9MxQwKZAQSjyUEhB8yniGm2VFiu6ezR5QZ4/+4IvS yBlg== X-Gm-Message-State: AOAM532gMWlt7wz9c1fa77cPYWzbSwLQZq4WxAGxMi/P0/VtLfi3mA/z zAKlEtPT/m9eq8Fe4KiGR+Vlpk3lxZ8= X-Google-Smtp-Source: ABdhPJzMpSumQomh9Idt/4QIw/04eYjEP0Vl3IKeN59RtUWWg+9VsZ0yFSWbUPASg1+HA0V/Jyq71g== X-Received: by 2002:a05:600c:4417:: with SMTP id u23mr4004646wmn.26.1624537345524; Thu, 24 Jun 2021 05:22:25 -0700 (PDT) Received: from bioinfomeary01-Precision-7820-Tower ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id x25sm9162493wmj.23.2021.06.24.05.22.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 05:22:25 -0700 (PDT) From: zimoun To: Sarah Morgensen Subject: Re: bug#49196: [PATCH] import: utils: 'recursive-import' skips unfound packages References: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> Date: Thu, 24 Jun 2021 14:21:37 +0200 In-Reply-To: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> (Sarah Morgensen's message of "Wed, 23 Jun 2021 13:49:05 -0700") Message-ID: <854kdna1wu.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, Thanks for the patch. On mer., 23 juin 2021 at 13:49, Sarah Morgensen wrote: Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.128.43 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (zimon.toutoune[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.128.43 listed in list.dnswl.org] X-Debbugs-Envelope-To: 49196 Cc: 49196@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) Hi, Thanks for the patch. On mer., 23 juin 2021 at 13:49, Sarah Morgensen wrote: > It seems like several importers (gem, egg, go) expect to be able to retur= n #f or > '() in #:repo->guix-package when a package is not found (while printing a > warning to the user) but recursive-import doesn't look like it handles it= . I > suppose this worked before but was silently broken at some point. This pa= tch > (re-)enables this behavior. I added a test for this as well. Indeed, there is an inconsistency betweem all the recursive importers. An attempt to fix this is done by [1]. 1: > diff --git a/guix/import/go.scm b/guix/import/go.scm > index d110954664..c859098492 100644 > --- a/guix/import/go.scm > +++ b/guix/import/go.scm > @@ -5,6 +5,7 @@ > ;;; Copyright =C2=A9 2021 Maxim Cournoyer > ;;; Copyright =C2=A9 2021 Ludovic Court=C3=A8s > ;;; Copyright =C2=A9 2021 Xinglu Chen > +;;; Copyright =C2=A9 2021 Sarah Morgensen > ;;; > ;;; This file is part of GNU Guix. > ;;; > @@ -651,7 +652,7 @@ This package and its dependencies won't be imported.~= %") > (uri->string (http-get-error-uri c)) > (http-get-error-code c) > (http-get-error-reason c)) > - (values '() '()))) > + (values #f '()))) Yes, there is an inconsistency... > (receive (package-sexp dependencies) > (go-module->guix-package* name #:goproxy goproxy > #:version version > diff --git a/guix/import/utils.scm b/guix/import/utils.scm > index d817318a91..49f38cfa2a 100644 > --- a/guix/import/utils.scm > +++ b/guix/import/utils.scm > @@ -8,6 +8,7 @@ > ;;; Copyright =C2=A9 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com> > ;;; Copyright =C2=A9 2020 Martin Becze > ;;; Copyright =C2=A9 2021 Maxim Cournoyer > +;;; Copyright =C2=A9 2021 Sarah Morgensen > ;;; > ;;; This file is part of GNU Guix. > ;;; > @@ -469,16 +470,17 @@ to obtain the Guix package name corresponding to th= e upstream name." > (normalized-deps (map (match-lambda > ((name version) (list name version)) > (name (list name #f))) dependencies))) > - (make-node name version package normalized-deps))) > + (and package > + (make-node name version package normalized-deps)))) > > (map node-package > (topological-sort (list (lookup-node package-name version)) > (lambda (node) > - (map (lambda (name-version) > - (apply lookup-node name-version)) > - (remove (lambda (name-version) > - (apply exists? name-version)) > - (node-dependencies node)))) > + (filter-map (lambda (name-version) > + (apply lookup-node name-version= )) > + (remove (lambda (name-version) > + (apply exists? name-ver= sion)) > + (node-dependencies node))= )) ...however, I am not convinced this fixes the issue. Compare: --8<---------------cut here---------------start------------->8--- $ guix import go do-not-exist -r guix import: warning: Failed to import package "do-not-exist". reason: "https://proxy.golang.org/do-not-exist/@v/list" could not be fetche= d: HTTP error 410 ("Gone"). This package and its dependencies won't be imported. Backtrace: 4 (primitive-load "/home/sitour/.config/guix/current/bin/guix") In guix/ui.scm: 2147:12 3 (run-guix-command _ . _) In guix/scripts/import.scm: 120:11 2 (guix-import . _) In srfi/srfi-1.scm: 586:17 1 (map1 (())) In guix/import/utils.scm: 280:2 0 (package->definition _ _) guix/import/utils.scm:280:2: In procedure package->definition: Throw to key `match-error' with args `("match" "no matching pattern" ())'. --8<---------------cut here---------------end--------------->8--- with: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix import go do-not-exist -r guix import: warning: Failed to import package "do-not-exist". reason: "https://proxy.golang.org/do-not-exist/@v/list" could not be fetche= d: HTTP error 410 ("Gone"). This package and its dependencies won't be imported. Backtrace: In ice-9/boot-9.scm: 1752:10 8 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) In unknown file: 7 (apply-smob/0 #) In ice-9/boot-9.scm: 724:2 6 (call-with-prompt _ _ #) In ice-9/eval.scm: 619:8 5 (_ #(#(#))) In guix/ui.scm: 2147:12 4 (run-guix-command _ . _) In guix/scripts/import.scm: 120:11 3 (guix-import . _) In guix/scripts/import/go.scm: 116:20 2 (guix-import-go . _) In guix/import/utils.scm: 440:34 1 (recursive-import _ #:repo->guix-package _ #:guix-name _ #:ver= sion _ #:repo _) 486:28 0 (_ #f) guix/import/utils.scm:486:28: In procedure struct-vtable: Wrong type argume= nt in position 1 (expecting struct): #f --8<---------------cut here---------------end--------------->8--- Then, the patch introduces issues against other importers, for instance: --8<---------------cut here---------------start------------->8--- r@bioinfomeary01-Precision-7820-Tower$ guix import gem do-not-exist -r #f $ ./pre-inst-env guix import gem do-not-exist -r Backtrace: In ice-9/boot-9.scm: 1752:10 8 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) In unknown file: 7 (apply-smob/0 #) In ice-9/boot-9.scm: 724:2 6 (call-with-prompt _ _ #) In ice-9/eval.scm: 619:8 5 (_ #(#(#))) In guix/ui.scm: 2147:12 4 (run-guix-command _ . _) In guix/scripts/import.scm: 120:11 3 (guix-import . _) In guix/scripts/import/gem.scm: 97:16 2 (guix-import-gem . _) In guix/import/utils.scm: 440:34 1 (recursive-import _ #:repo->guix-package _ #:guix-name _ #:ver= sion _ #:repo _) 486:28 0 (_ #f) guix/import/utils.scm:486:28: In procedure struct-vtable: Wrong type argume= nt in position 1 (expecting struct): #f --8<---------------cut here---------------end--------------->8--- Well, it is not worse for most of the importers. And perhaps this patch is worth. As explained in [1], all the recursive importers should be unified and the errors correctly handled, IMHO. With jeko, we are pair-programming to work on it... but we are really slow. ;-) > diff --git a/tests/import-utils.scm b/tests/import-utils.scm > index 874816442e..1b728104e0 100644 > --- a/tests/import-utils.scm > +++ b/tests/import-utils.scm Thanks for adding a test. :-) Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 25 00:22:49 2021 Received: (at 49196) by debbugs.gnu.org; 25 Jun 2021 04:22:49 +0000 Received: from localhost ([127.0.0.1]:44478 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwdMf-0008IK-0v for submit@debbugs.gnu.org; Fri, 25 Jun 2021 00:22:49 -0400 Received: from out1.migadu.com ([91.121.223.63]:23291) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwdMZ-0008I4-20 for 49196@debbugs.gnu.org; Fri, 25 Jun 2021 00:22:47 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1624594961; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=h+SwKwhNGKPGl0RG8G3FHXrAvwHPOjvpJzbcj7TB3v8=; b=PR759fVecNiVP9wlPGubExKmtHnss2CDY1bHcPlzy9DOlJJ8z3Jf2oFDAVx/R5wB+rrn7g q/pr2AB1UMPZTwcwjZ3qOEXB1j8VUmwKTYqSRXEELolhq6+Ad8BwRHY8Kks9M2ZlU0YZgB OVuasVCBMajWLwr4twp5PccHFYc5LfE= From: Sarah Morgensen To: zimoun Subject: Re: bug#49196: [PATCH] import: utils: 'recursive-import' skips unfound packages References: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> <854kdna1wu.fsf@gmail.com> Date: Thu, 24 Jun 2021 21:22:36 -0700 In-Reply-To: <854kdna1wu.fsf@gmail.com> (zimoun's message of "Thu, 24 Jun 2021 14:21:37 +0200") Message-ID: <86lf6y7eur.fsf_-_@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 49196 Cc: 49196@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hello, Thanks for the review! zimoun writes: > Indeed, there is an inconsistency betweem all the recursive importers. > An attempt to fix this is done by [1]. > > 1: Thanks, that was a good read. With context, I see where you're coming from. I agree that the direction to take with these importers is to unify and standardize. The goal of this patch is just to allow recursive import to provide a usable result despite some failures, when the importer supports it. I'd rather hunt down one package than 20+ :) This may make reporting errors more difficult, but I think the use-case is worth it. > ...however, I am not convinced this fixes the issue. Compare: > > $ guix import go do-not-exist -r > > with: > > $ ./pre-inst-env guix import go do-not-exist -r Good catch. I did not think to handle the toplevel package not being found! This actually leads to making this a much simpler patch... --8<---------------cut here---------------start------------->8--- - (map node-package + (filter-map node-package (topological-sort (list (lookup-node package-name version)) --8<---------------cut here---------------end--------------->8--- ...which also works for other importers which return (values #f ...): --8<---------------cut here---------------start------------->8--- ~/guix$ for importer in stackage elpa gem cran go ; do printf "\n### $importer\n" ; ./pre-inst-env guix import $importer really-does-not-exist -r ;done ### stackage guix import: error: really-does-not-exist: Stackage package not found ### elpa guix import: error: couldn't find meta-data for ELPA package `really-does-not-exist'. ### gem ### cran error: failed to retrieve package information from "https://cran.r-project.org/web/packages/really-does-not-exist/DESCRIPTION": 404 ("Not Found") guix import: error: couldn't find meta-data for R package ### go guix import: warning: Failed to import package "really-does-not-exist". reason: "https://proxy.golang.org/really-does-not-exist/@v/list" could not be fetched: HTTP error 410 ("Gone"). This package and its dependencies won't be imported. --8<---------------cut here---------------end--------------->8--- > Well, it is not worse for most of the importers. And perhaps this patch > is worth. As explained in [1], all the recursive importers should be > unified and the errors correctly handled, IMHO. With jeko, we are > pair-programming to work on it... but we are really slow. ;-) Yes, this is very much just a stopgap. In your words (from #45984): > Well, this patch set are trivial changes that quickly fix the current > broken situation without a deep revamp. I will follow up with an updated patch. Regards, Sarah From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 25 00:48:37 2021 Received: (at 49196) by debbugs.gnu.org; 25 Jun 2021 04:48:37 +0000 Received: from localhost ([127.0.0.1]:44594 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwdlY-0002nr-QD for submit@debbugs.gnu.org; Fri, 25 Jun 2021 00:48:37 -0400 Received: from out1.migadu.com ([91.121.223.63]:56262) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwdlW-0002nb-9x for 49196@debbugs.gnu.org; Fri, 25 Jun 2021 00:48:32 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1624596508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lF4oF1r5xTcLpwOTDamAzc8hyfLU7Mb8QAtxHMswXes=; b=HSr6o2jAy591p6K/8gAONbDOrDXMprzWq5yNuPrsBmDk1ZdmsDBnaEF01DhD/sI8DXBM3M ftL8NXweqWoEqHzi+vkENqNUE/x8hIUHjSTFwMc/rkUBWd2g7FgYrqYvlYMQTIeyJYMYdy 320bVTsbCCnlDrmTOx99gJEqeQ1F53M= From: Sarah Morgensen To: 49196@debbugs.gnu.org Subject: [PATCH v2] import: utils: 'recursive-import' skips unfound packages Date: Thu, 24 Jun 2021 21:48:26 -0700 Message-Id: <770a40d86bb1b67d628b23ef3e4ae7668b7e522f.1624595313.git.iskarian@mgsn.dev> In-Reply-To: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> References: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/import/utils.scm (recursive-import): Skip packages when the package returned by repo->guix-package is false. * guix/import/go.scm (go-module-recursive-import): Explicitly return false in 'repo- [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [91.121.223.63 listed in list.dnswl.org] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-Debbugs-Envelope-To: 49196 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) * guix/import/utils.scm (recursive-import): Skip packages when the package returned by repo->guix-package is false. * guix/import/go.scm (go-module-recursive-import): Explicitly return false in 'repo->guix-package' when packages are not found. * tests/import-utils.scm ("recursive-import: skip false packages"): New test. --- guix/import/go.scm | 3 ++- guix/import/utils.scm | 26 ++++++++++++++------------ tests/import-utils.scm | 28 ++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+), 13 deletions(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index d110954664..c859098492 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2021 Maxim Cournoyer ;;; Copyright © 2021 Ludovic Courtès ;;; Copyright © 2021 Xinglu Chen +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -651,7 +652,7 @@ This package and its dependencies won't be imported.~%") (uri->string (http-get-error-uri c)) (http-get-error-code c) (http-get-error-reason c)) - (values '() '()))) + (values #f '()))) (receive (package-sexp dependencies) (go-module->guix-package* name #:goproxy goproxy #:version version diff --git a/guix/import/utils.scm b/guix/import/utils.scm index d817318a91..7f2e7ecb46 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com> ;;; Copyright © 2020 Martin Becze ;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -471,15 +472,16 @@ to obtain the Guix package name corresponding to the upstream name." (name (list name #f))) dependencies))) (make-node name version package normalized-deps))) - (map node-package - (topological-sort (list (lookup-node package-name version)) - (lambda (node) - (map (lambda (name-version) - (apply lookup-node name-version)) - (remove (lambda (name-version) - (apply exists? name-version)) - (node-dependencies node)))) - (lambda (node) - (string-append - (node-name node) - (or (node-version node) "")))))) + (filter-map + node-package + (topological-sort (list (lookup-node package-name version)) + (lambda (node) + (map (lambda (name-version) + (apply lookup-node name-version)) + (remove (lambda (name-version) + (apply exists? name-version)) + (node-dependencies node)))) + (lambda (node) + (string-append + (node-name node) + (or (node-version node) "")))))) diff --git a/tests/import-utils.scm b/tests/import-utils.scm index 874816442e..7c6c782917 100644 --- a/tests/import-utils.scm +++ b/tests/import-utils.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2017 Ricardo Wurmus ;;; Copyright © 2016 Ben Woodcroft ;;; Copyright © 2020 Martin Becze +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -64,6 +65,33 @@ '()))) #:guix-name identity)) +(test-equal "recursive-import: skip false packages (toplevel)" + '() + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + +(test-equal "recursive-import: skip false packages (dependency)" + '((package + (name "foo") + (inputs `(("bar" ,bar))))) + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values '(package + (name "foo") + (inputs `(("bar" ,bar)))) + '("bar"))) + (("bar" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + (test-assert "alist->package with simple source" (let* ((meta '(("name" . "hello") ("version" . "2.10") base-commit: c7804cd97b28ef012acc20c1d861904e9592382b -- 2.31.1 From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 25 02:53:56 2021 Received: (at 49196) by debbugs.gnu.org; 25 Jun 2021 06:53:56 +0000 Received: from localhost ([127.0.0.1]:44691 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwfiu-0004QY-9A for submit@debbugs.gnu.org; Fri, 25 Jun 2021 02:53:56 -0400 Received: from mail-qk1-f173.google.com ([209.85.222.173]:38593) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwfis-0004QI-Ls for 49196@debbugs.gnu.org; Fri, 25 Jun 2021 02:53:55 -0400 Received: by mail-qk1-f173.google.com with SMTP id 65so683733qko.5 for <49196@debbugs.gnu.org>; Thu, 24 Jun 2021 23:53:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mk5hMWLDVhpLNTou9jwUaiMQ6V7P04aUsFC3lhRnROI=; b=sJsZ6t+mP5Q94lWwX6UbUmsdBOcM57QbITIpNhZbddbwwGFVlfbm/LdmEABqN8K1RN A7Xt2DHh1PCATzFocsni/7h4OZ+v2QReUHdKBGdH6oUct2er3+nTAjUaLX9QAual7h9B ZVmn0NwNgGyvecLUolOCJF6pUM31KhPN3qxkPfn8BC7lCSsNsbrlYN7BSjJ4LPIyDjPj Xl25JrW+SLcOtRmAhMq8vc/PqHyC4M8npN/7FKRBwEzBtHs0NdBHkKia+ikfKWfC0ud2 yCEOqsi3qBJo0nC/1ij+HF88YAjpIyQYJDu+AH9doPAnYfMYZ8hpH5L6wVMyNxgoJvXJ LrcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mk5hMWLDVhpLNTou9jwUaiMQ6V7P04aUsFC3lhRnROI=; b=KN509slY0GmlD3qr0gTFIWFcW+EmMNpYHFCktTRt0wTtnglaBwIQGWu36pdAieArvW FojqvvLebeVzokgepeREl1lkAW0WocTleNnhsiygcPajvrAInfza2gQIOnZGqJob0H4W Uca8s8/PUm/B0uI36a4efBMhh06jhqpeQ+whjXx6OkXVFhcmP1unmMGhDtKVN6zHwgZ1 IyApW65vFas/AHkMn9dn1830pm8lFBLAw2BUTk8BXO0Kh8qtgRWfaK2e+w8Q3Vsr6rH5 VPKsjMvYlvBdw5CTnTJskTClJwxv/yNXtMOg9RsqomZUGTdsPVlsD8phTV3upnSjIB70 T/pQ== X-Gm-Message-State: AOAM531zg5yC8GARwL3Kwl9CPjwyAovQVWQVH0b1ND08/F7pyKLkESP6 HS4IeGHlbWuyC/yZ35CMUN+tWGvBqiFr0ZetZy0eHJuM X-Google-Smtp-Source: ABdhPJypy5leE/p3trlsK+bzNyrDM6i4UYTqwFu3O8Fr+wyMK2R0lFXJbnuHYJU50E3hOsZEzSB0pClZ5WB9hJ6aLGE= X-Received: by 2002:a05:620a:543:: with SMTP id o3mr10094989qko.80.1624604028927; Thu, 24 Jun 2021 23:53:48 -0700 (PDT) MIME-Version: 1.0 References: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> <854kdna1wu.fsf@gmail.com> <86lf6y7eur.fsf_-_@mgsn.dev> In-Reply-To: <86lf6y7eur.fsf_-_@mgsn.dev> From: zimoun Date: Fri, 25 Jun 2021 08:53:37 +0200 Message-ID: Subject: Re: bug#49196: [PATCH] import: utils: 'recursive-import' skips unfound packages To: Sarah Morgensen Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 49196 Cc: 49196@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Fri, 25 Jun 2021 at 06:22, Sarah Morgensen wrote: > The goal of this patch is just to allow recursive import to provide a > usable result despite some failures, when the importer supports it. I'd > rather hunt down one package than 20+ :) This may make reporting errors > more difficult, but I think the use-case is worth it. I agree. > Good catch. I did not think to handle the toplevel package not being > found! This actually leads to making this a much simpler patch... > > --8<---------------cut here---------------start------------->8--- > - (map node-package > + (filter-map node-package > (topological-sort (list (lookup-node package-name version)) > --8<---------------cut here---------------end--------------->8--- Cool! > Yes, this is very much just a stopgap. In your words (from #45984): > > > Well, this patch set are trivial changes that quickly fix the current > > broken situation without a deep revamp. I agree. Heh! I am consistent with my words. ;-) > I will follow up with an updated patch. Cool, thanks! Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 25 12:38:09 2021 Received: (at 49196) by debbugs.gnu.org; 25 Jun 2021 16:38:09 +0000 Received: from localhost ([127.0.0.1]:46694 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwoqG-0001It-Um for submit@debbugs.gnu.org; Fri, 25 Jun 2021 12:38:09 -0400 Received: from mail-wm1-f49.google.com ([209.85.128.49]:34616) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwoqE-0001I3-Dh for 49196@debbugs.gnu.org; Fri, 25 Jun 2021 12:38:07 -0400 Received: by mail-wm1-f49.google.com with SMTP id u5-20020a7bc0450000b02901480e40338bso6703651wmc.1 for <49196@debbugs.gnu.org>; Fri, 25 Jun 2021 09:38:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rty0ZqD+74SEHhUaLtCsSlBbmFKBoQTnN3k5Vftk+7k=; b=Rjf3HBatfg67aqeE6yJDHorQVPkXdusIozmloghMJeLmr/e1uptvjvpo27ZsQWqtj6 RjMVAIEZJA1Up1foYNztqBxpUlXxrhbBnmYBedZzQU9fNxTmhqMJAaZcwEEuYCVXWOXK Jf4ZX8NdM+yWZkwWQUpS/eaS4fmFS+JfnDOi9bJaVFC8Khyd3hDR8xuinsU3Ek3V4T9u kcgpBr1/lWJOIAcSzkZ6YvdkY2JV2RRdXg8NbNJPtLNY1V77Jhl2N/7yY36BVDlBI5++ HThowZgUc9kYkPdEooxkLxuCBa+GzrAjSsz2EoIWKHBzQFtpteurCEPYW+l4rdn+e+1K /0BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rty0ZqD+74SEHhUaLtCsSlBbmFKBoQTnN3k5Vftk+7k=; b=uf3NQa0gbYKPqn6hb+l/ycd8JliPgb7aObnjNFrZ9jBUkwVM7w08+CPpH9J5EWFa6R xVBrv2mM4UZy1V7rBDsTX3I+QA2is9VaTC/VkA3FM/0wro3TwVldos19UgC0wyabc4/y T7nOP3dTNCaa3cCM7xAKkdOBcVLdvaIeeGTZyGzV26O/OTE1ccnnIzZt85vjz6fc4eA6 8EzfYBKHMf0Unf5gBZf5wEaPYL2g/Q8NUviB//PWoLWOUTncxi+gNZ7y0A8LKmP5SG9k YQslHh5LfqBJXWUSlm/a5nimLbR3C05hUlbxbtSlg4hNggHj03qILVFTmRWN4Kn3N3+W B37A== X-Gm-Message-State: AOAM532/FwHBhdaXGVQBAHTacryoWbNvag+l4tU3hR2AjZka7j9v2cKQ AMT6czAElti9igW1ITGyCBxF+iDTJIY= X-Google-Smtp-Source: ABdhPJz5wSw43PANWqV8eQ4eZ0A47XTTYUbTjXth+kL0EaU9DgPZhoxbO3QhtbY9ds3221IDbpZMvg== X-Received: by 2002:a7b:c052:: with SMTP id u18mr11808733wmc.105.1624639080843; Fri, 25 Jun 2021 09:38:00 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id 2sm11642170wmk.24.2021.06.25.09.38.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:38:00 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v3 2/3] import: utils: Skip not found packages. Date: Fri, 25 Jun 2021 18:37:48 +0200 Message-Id: <20210625163749.65196-2-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210625163749.65196-1-zimon.toutoune@gmail.com> References: <20210625163749.65196-1-zimon.toutoune@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Sarah Morgensen * guix/import/utils.scm (recursive-import): Skip packages when the package returned by 'repo->guix-package' is false. * tests/import-utils.scm: New tests. --- guix/import/utils.scm | 26 ++++++++++++++------------ tests/import-utils.scm | 28 ++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 12 deletions(-) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index d817318a91..7f2e7ecb46 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com> ;;; Copyright © 2020 Martin Becze ;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -471,15 +472,16 @@ to obtain the Guix package name corresponding to the upstream name." (name (list name #f))) dependencies))) (make-node name version package normalized-deps))) - (map node-package - (topological-sort (list (lookup-node package-name version)) - (lambda (node) - (map (lambda (name-version) - (apply lookup-node name-version)) - (remove (lambda (name-version) - (apply exists? name-version)) - (node-dependencies node)))) - (lambda (node) - (string-append - (node-name node) - (or (node-version node) "")))))) + (filter-map + node-package + (topological-sort (list (lookup-node package-name version)) + (lambda (node) + (map (lambda (name-version) + (apply lookup-node name-version)) + (remove (lambda (name-version) + (apply exists? name-version)) + (node-dependencies node)))) + (lambda (node) + (string-append + (node-name node) + (or (node-version node) "")))))) diff --git a/tests/import-utils.scm b/tests/import-utils.scm index 874816442e..7c6c782917 100644 --- a/tests/import-utils.scm +++ b/tests/import-utils.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2017 Ricardo Wurmus ;;; Copyright © 2016 Ben Woodcroft ;;; Copyright © 2020 Martin Becze +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -64,6 +65,33 @@ '()))) #:guix-name identity)) +(test-equal "recursive-import: skip false packages (toplevel)" + '() + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + +(test-equal "recursive-import: skip false packages (dependency)" + '((package + (name "foo") + (inputs `(("bar" ,bar))))) + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values '(package + (name "foo") + (inputs `(("bar" ,bar)))) + '("bar"))) + (("bar" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + (test-assert "alist->package with simple source" (let* ((meta '(("name" . "hello") ("version" . "2.10") -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 25 12:38:13 2021 Received: (at 49196) by debbugs.gnu.org; 25 Jun 2021 16:38:13 +0000 Received: from localhost ([127.0.0.1]:46696 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwoqK-0001J6-E9 for submit@debbugs.gnu.org; Fri, 25 Jun 2021 12:38:12 -0400 Received: from mail-wr1-f47.google.com ([209.85.221.47]:34577) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwoqF-0001I9-46 for 49196@debbugs.gnu.org; Fri, 25 Jun 2021 12:38:08 -0400 Received: by mail-wr1-f47.google.com with SMTP id e22so11269317wrc.1 for <49196@debbugs.gnu.org>; Fri, 25 Jun 2021 09:38:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i1UQbryFVBMTxsNb3xibfTlsptO7S3Oy039jdx7j4nw=; b=IuaLey39yLNp04yAI132qRrUdOfzXpdHNfBKgpNcgYm5iqGLIVDxr9eUEXMi8IbIo+ 2ofzLXWpW+MaTx0dtfWdD01lf4dG0MZIWxlH8TFcVzXuJQVkAsN9i6Yf1ZoiNNsehLDt R+9FUE2iUsnOESw58iyX8caV3lms/LH1FOrNWdPmAG0eb+QZWfpzcbSRG1Fpnwkwgv0S TZnDZsX+M3UnGoBbwLpIJvhl5Yn/5/MSk3HAB6diSBGAW+LwTUul4lEfSO80g2wNpLGq PLR3jHqQ+ovMvEeWBhEb+0rN0/U/J9ZSSv/cKwbJQxUV2/Wm8Cgz1xYCBl1Ws3f7aa9V 0YlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i1UQbryFVBMTxsNb3xibfTlsptO7S3Oy039jdx7j4nw=; b=pcA1mkZycVNu2vV8I4vRGqt2ViP2Pp8+YWeSukTbZt8fzxrF69dhk6cf2/tY+A39Mt WqkzhcQqqNj+qvMxP008oE40DzfzkjLOLPEnAHduN9Os8cs50KXLOAMHK+t03+b+/BlW 8+EZRWKpRjGsuq1B5wLvpx9U8TKYxSKLPSI9nt27MrSQua73ysNAcMoHxf/fsFRK0386 /YYJFXtxeKb8u+exskzzgtn9HS960Eb+f6oOTCGN8rI7C3FiIWkqODK9Nmf3Bk/hNcq6 deJCEUZQpPSBX5iLd6p0RHI5dCLeXL542A9c10qnNIKf2oa/U2eCPbEeA7auKuhEgeo9 b7ew== X-Gm-Message-State: AOAM5328SEqsPq4oETIphdluW+n0czacVBVd6KEEVOvZ4tGafQzTGLJm xe2GzJqObTQ+mWXzDCQ2CcpEJWSpPSM= X-Google-Smtp-Source: ABdhPJw/qWKGdnl/NqK/r2AIwCfZ1Tl+sSUvmA+UKwZhirrhIVtRfsid4s8XTs8gRhu406aYDUwsiQ== X-Received: by 2002:adf:9bd0:: with SMTP id e16mr12101162wrc.392.1624639081291; Fri, 25 Jun 2021 09:38:01 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id 2sm11642170wmk.24.2021.06.25.09.38.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:38:01 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v3 3/3] import: go: Improve error handling. Date: Fri, 25 Jun 2021 18:37:49 +0200 Message-Id: <20210625163749.65196-3-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210625163749.65196-1-zimon.toutoune@gmail.com> References: <20210625163749.65196-1-zimon.toutoune@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/import/go.scm (go-module->guix-package*): Handle errors. (go-module-recursive-import): Remove 'guard'. * guix/scripts/import/go.scm (guix-import-go): Adjust. * tests/go.scm: Adjust. --- guix/im [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (zimon.toutoune[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.221.47 listed in wl.mailspike.net] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.221.47 listed in list.dnswl.org] X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev, zimoun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) * guix/import/go.scm (go-module->guix-package*): Handle errors. (go-module-recursive-import): Remove 'guard'. * guix/scripts/import/go.scm (guix-import-go): Adjust. * tests/go.scm: Adjust. --- guix/import/go.scm | 43 ++++++++++++++++++++++---------------- guix/scripts/import/go.scm | 6 +++--- tests/go.scm | 2 +- 3 files changed, 29 insertions(+), 22 deletions(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index c859098492..c559f02e5b 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -6,6 +6,7 @@ ;;; Copyright © 2021 Ludovic Courtès ;;; Copyright © 2021 Xinglu Chen ;;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2021 Simon Tournier ;;; ;;; This file is part of GNU Guix. ;;; @@ -61,7 +62,7 @@ #:use-module (web response) #:use-module (web uri) - #:export (go-module->guix-package + #:export (go-module->guix-package* go-module-recursive-import)) ;;; Commentary: @@ -630,17 +631,9 @@ hint: use one of the following available versions ~a\n" dependencies+versions dependencies)))) -(define go-module->guix-package* (memoize go-module->guix-package)) - -(define* (go-module-recursive-import package-name - #:key (goproxy "https://proxy.golang.org") - version - pin-versions?) - - (recursive-import - package-name - #:repo->guix-package - (lambda* (name #:key version repo) +(define go-module->guix-package* + (memoize + (lambda args ;; Disable output buffering so that the following warning gets printed ;; consistently. (setvbuf (current-error-port) 'none) @@ -648,15 +641,29 @@ hint: use one of the following available versions ~a\n" (warning (G_ "Failed to import package ~s. reason: ~s could not be fetched: HTTP error ~a (~s). This package and its dependencies won't be imported.~%") - name + (match args ((name _ ...) name)) (uri->string (http-get-error-uri c)) (http-get-error-code c) (http-get-error-reason c)) + (values #f '())) + (else + (warning (G_ "Something went wrong with ~s.~%") args) (values #f '()))) - (receive (package-sexp dependencies) - (go-module->guix-package* name #:goproxy goproxy - #:version version - #:pin-versions? pin-versions?) - (values package-sexp dependencies)))) + (apply go-module->guix-package args))))) + +(define* (go-module-recursive-import package-name + #:key (goproxy "https://proxy.golang.org") + version + pin-versions?) + + (recursive-import + package-name + #:repo->guix-package + (lambda* (name #:key version repo) + (receive (package-sexp dependencies) + (go-module->guix-package* name #:goproxy goproxy + #:version version + #:pin-versions? pin-versions?) + (values package-sexp dependencies))) #:guix-name go-module->guix-package-name #:version version)) diff --git a/guix/scripts/import/go.scm b/guix/scripts/import/go.scm index 74e8e60cce..071ecdb2ef 100644 --- a/guix/scripts/import/go.scm +++ b/guix/scripts/import/go.scm @@ -115,10 +115,10 @@ that are not yet in Guix")) (map package->definition* (apply go-module-recursive-import arguments)) ;; Single import. - (let ((sexp (apply go-module->guix-package arguments))) + (let ((sexp (apply go-module->guix-package* arguments))) (unless sexp - (leave (G_ "failed to download meta-data for module '~a'~%") - module-name)) + (leave (G_ "failed to download meta-data for module '~a'.~%") + name)) (package->definition* sexp)))))) (() (leave (G_ "too few arguments~%"))) diff --git a/tests/go.scm b/tests/go.scm index b088ab50d2..929a8b39d1 100644 --- a/tests/go.scm +++ b/tests/go.scm @@ -286,6 +286,6 @@ package.") (nix-base32-string->bytevector "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5") #f))) - (go-module->guix-package "github.com/go-check/check"))))))) + (go-module->guix-package* "github.com/go-check/check"))))))) (test-end "go") -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 25 12:38:13 2021 Received: (at 49196) by debbugs.gnu.org; 25 Jun 2021 16:38:13 +0000 Received: from localhost ([127.0.0.1]:46698 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwoqK-0001JA-VD for submit@debbugs.gnu.org; Fri, 25 Jun 2021 12:38:13 -0400 Received: from mail-wm1-f45.google.com ([209.85.128.45]:40622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwoqE-0001I1-7B for 49196@debbugs.gnu.org; Fri, 25 Jun 2021 12:38:08 -0400 Received: by mail-wm1-f45.google.com with SMTP id o33-20020a05600c5121b02901e360c98c08so7104035wms.5 for <49196@debbugs.gnu.org>; Fri, 25 Jun 2021 09:38:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=XxiTGtRPeTke2y/VJ6teh0xYTNnIwzmo2KerftKyvvw=; b=lWzBzphHecAAcRCX+LUUigDsxZsboC+1weC1aQutUajxSKTcmlEY8L1oxERVkH6Izt 3Lsgs5FazGSWz8tKLJG26Y61usyyD8VIXLBtsztwJ+VHwAjvPlvPlUQYhdQqRG+XlK78 K599MAJzTnDAwAfEytd9t3+NeMFf4X2lVBZBsgT4ogmm1PGo/FGRaT9qVyWtzH6DHmOK 9Lsc7bzcPblsBgFDTg1JJFOlJ/ragB9MeKfm2689RQQd289HoPrX10BbAVI2eUqRilJG 7KI2+IEu42af0VlcojkUqHHhyDXWk8qDY7EgnKGwiTM6+OIFs5gKiBhc9P7kGKRRANqt 9/Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=XxiTGtRPeTke2y/VJ6teh0xYTNnIwzmo2KerftKyvvw=; b=OTs+jZgRnTR6kHajEzPJLKL/nuV3qQzTZ0UxlWsn1a3p9+e7dRT0NsdxXFbR0KxR+2 hh6WJaI04nH7ylatSdNld1RO73rzbLNhO/eTN9VHsFVnORmd3fOon76saQXzRx33dj9H JaUYorsfra3hytc+EkjDIb2xOb2DfCJqrX+JMFASbAkw4s2m1bq1q9Lnm4OtdEi4tEe5 osJBfJexBM/IY213z5eaHip93WiVANcnAtw6DTrS3wp2aYs1sQjBIRp8KZY8ISoSEBW9 Bv4PvhYQ0TxzsQPU+2kMoZppF+yhPazUmCcHtMlKsq20iUuMlIQ7Z1qkQyAuaqojWECb WJ7Q== X-Gm-Message-State: AOAM531rKVbnN4dYDRJv0QN+GoAkKUvdx0Z9tj5Vdi8wRgNveDFmW6wn AojeXE9+MNFLhijpmAu+IyYd4+pAhAU= X-Google-Smtp-Source: ABdhPJyQ/7aX/qU3KlNC1Z+L6ySaCo1zxoJr4+vYG1wGOuC70XiMWkMG69W63cS0IB6021VUnqSyGw== X-Received: by 2002:a7b:c8c1:: with SMTP id f1mr11651962wml.90.1624639080316; Fri, 25 Jun 2021 09:38:00 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id 2sm11642170wmk.24.2021.06.25.09.37.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:37:59 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v3 1/3] import: go: Return false for package not found. Date: Fri, 25 Jun 2021 18:37:47 +0200 Message-Id: <20210625163749.65196-1-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Sarah Morgensen * guix/import/go.scm (go-module-recursive-import): Explicitly return false when packages are not found. --- guix/import/go.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (zimon.toutoune[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.128.45 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.128.45 listed in wl.mailspike.net] X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) From: Sarah Morgensen * guix/import/go.scm (go-module-recursive-import): Explicitly return false when packages are not found. --- guix/import/go.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index d110954664..c859098492 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2021 Maxim Cournoyer ;;; Copyright © 2021 Ludovic Courtès ;;; Copyright © 2021 Xinglu Chen +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -651,7 +652,7 @@ This package and its dependencies won't be imported.~%") (uri->string (http-get-error-uri c)) (http-get-error-code c) (http-get-error-reason c)) - (values '() '()))) + (values #f '()))) (receive (package-sexp dependencies) (go-module->guix-package* name #:goproxy goproxy #:version version base-commit: 88c7c739740b56cab132cf1a3f16392c434408f7 -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 25 12:48:04 2021 Received: (at 49196) by debbugs.gnu.org; 25 Jun 2021 16:48:04 +0000 Received: from localhost ([127.0.0.1]:46712 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwozs-0001aR-Fn for submit@debbugs.gnu.org; Fri, 25 Jun 2021 12:48:04 -0400 Received: from mail-wm1-f54.google.com ([209.85.128.54]:43939) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwozj-0001Zf-QR for 49196@debbugs.gnu.org; Fri, 25 Jun 2021 12:47:56 -0400 Received: by mail-wm1-f54.google.com with SMTP id p8-20020a7bcc880000b02901dbb595a9f1so6253451wma.2 for <49196@debbugs.gnu.org>; Fri, 25 Jun 2021 09:47:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=UkPd/sRmmdnmzmWROqOQXNgu7WKtaClM5CwL7J1hPoA=; b=j70L/uMd8BLEzJd/Fa02EAyRyCf2yOgvd/VHXk1H0nlzLo7aJJrYDRYalbKuyfXAbq D/BTSnSjQoZPKUHE9bVS9NcBG6wszYEYj+wXsLD4cDGD7UFmR2kgjT3nv7r8XXnJ9gsX HmHJ0a3G2tKu1OtmUtpsnbeuUvzD9Wr2DmVW1HmaD5SVjuzkBgS7TJt97IdkYBF5NMyh +DqKt6CxvFKA9JmkXyc/KFLC6TaeSCYgWNlaAA8/rihfBgsEPkAzBGXR4IeDh5OJbRyw FQDTQ/vb08YK/JwvRYR2YPxuqFwPk4uKr1Vh/zs25vnjt02K4xqs+MIGLXroGRCB7cjK pLAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=UkPd/sRmmdnmzmWROqOQXNgu7WKtaClM5CwL7J1hPoA=; b=L9vfdIK6BHU6ufDHe82++loTI8FTi6TMJWSBuKCZjOQfpD3y0DJH5T2PQGBH9RnjoD daRnfhC5Yzkyhdh/QuDmW4gu02R6ULorD1R4FNUgG2sTb04oMA3BVxGhz8ul+m+wmvo3 ER981Ps+MLOnuq8LrBoPJmux5hJlss8ig0+Ebv6Ri2aJbvY9EtJ2ly6pb6rq20ECNNAx zCrasXcADzoywHnaFQ1pWCN3WiYqgd9SunkG6KEJJkA6dYwJ7JUpe/pi8XB4w/xkCFdh OTyvh8iFlabfZMJV80+tsBn5xsuo9KAjI/1U72Nf+CFWXMgPx62ORQQuZ/P5U62T2swD vXqA== X-Gm-Message-State: AOAM533EIaj5o6G8a12xRMwA8AtiJpscatiIfe0iH48thjfnyKZONTDC FzF6TNchq3zSynObBlosL1FgO9jHj74= X-Google-Smtp-Source: ABdhPJzrigGcxg//A+xWcN7TrzOJnbyG9gYgDxxbz1vDjM8YDUkzqMHz170IWLBV5yop43r9pKbS0Q== X-Received: by 2002:a7b:c089:: with SMTP id r9mr11694421wmh.167.1624639669897; Fri, 25 Jun 2021 09:47:49 -0700 (PDT) Received: from bioinfomeary01-Precision-7820-Tower ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id u10sm6086823wmm.21.2021.06.25.09.47.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 09:47:49 -0700 (PDT) From: zimoun To: Sarah Morgensen Subject: Re: bug#49196: [PATCH] import: utils: 'recursive-import' skips unfound packages References: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> <854kdna1wu.fsf@gmail.com> <86lf6y7eur.fsf_-_@mgsn.dev> Date: Fri, 25 Jun 2021 18:47:23 +0200 In-Reply-To: <86lf6y7eur.fsf_-_@mgsn.dev> (Sarah Morgensen's message of "Thu, 24 Jun 2021 21:22:36 -0700") Message-ID: <85v96199ic.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 49196 Cc: 49196@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Sarah, I sent a v3 where your v2 is split into 2 parts. Then I move the guard. Now, your initial example returns: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix import go git.sr.ht/~sircmpwn/aerc -r following redirection to `https://pkg.go.dev/github.com/zenhack/go.notmuch'... following redirection to `https://pkg.go.dev/github.com/zenhack/go.notmuch'... following redirection to `https://github.com/ProtonMail/go-crypto?go-get=1'... following redirection to `https://github.com/jtolio/gls?go-get=1'... guix import: warning: Something went wrong with ("github.com/neelance/sourcemap" #:goproxy "https://proxy.golang.org" #:version #f #:pin-versions? #f). guix import: warning: Something went wrong with ("github.com/neelance/astrewrite" #:goproxy "https://proxy.golang.org" #:version #f #:pin-versions? #f). (define-public go-git-sr-ht-~sircmpwn-getopt (package (name "go-git-sr-ht-~sircmpwn-getopt") (version "0.0.0-20201218204720-9961a9c6298f") (source [...] ("go-github-com-creack-pty" ,go-github-com-creack-pty) ("go-git-sr-ht-~sircmpwn-getopt" ,go-git-sr-ht-~sircmpwn-getopt))) (home-page "https://git.sr.ht/~sircmpwn/aerc") (synopsis "aerc") (description "aerc is an email client for your terminal.") (license license:expat))) --8<---------------cut here---------------end--------------->8--- WDYT? Well, I do not know if it is good idea to catch the 'else' case because it hiddes what is wrong by the importer. On the other hand, it seems a bit friendler for the end-user. :-) Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 27 00:46:18 2021 Received: (at 49196) by debbugs.gnu.org; 27 Jun 2021 04:46:18 +0000 Received: from localhost ([127.0.0.1]:48848 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxMgU-0007bn-Ip for submit@debbugs.gnu.org; Sun, 27 Jun 2021 00:46:18 -0400 Received: from out0.migadu.com ([94.23.1.103]:15466) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxMgQ-0007ba-W2 for 49196@debbugs.gnu.org; Sun, 27 Jun 2021 00:46:16 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1624769173; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nW+ClB8sH82z4vMSs1ALt9u7zG0xgSl8hvUzNP4e5hg=; b=dJOsjli+48Lz8WAaJV9lnYLoLO3WKPHwyATVd+4dm8yVrSaRmRANHxy4K+I8yXJlMfWMhN MenBpBu9LdxTxJq0I3POZZ+lF5C5VPk9+eDW4QvEPtyp9B8isSBswOJRRPECh8d76Hq22M FzA6Eg76rELDkV1QfsaNfYxKtEw28os= From: Sarah Morgensen To: zimoun Subject: Re: bug#49196: [PATCH] import: utils: 'recursive-import' skips unfound packages References: <20210625163749.65196-1-zimon.toutoune@gmail.com> <20210625163749.65196-3-zimon.toutoune@gmail.com> Date: Sat, 26 Jun 2021 21:46:10 -0700 In-Reply-To: <20210625163749.65196-3-zimon.toutoune@gmail.com> (zimoun's message of "Fri, 25 Jun 2021 18:37:49 +0200") Message-ID: <868s2v7w4t.fsf_-_@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 49196 Cc: 49196@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi, Apologies for the delay. Thanks for the work. zimoun writes: > - #:export (go-module->guix-package > + #:export (go-module->guix-package* Would it be better to export both, in case a callee wants access to the actual errors? > +(define go-module->guix-package* > + (memoize > + (lambda args I would remove the memoize from this method (to put it back in later on), because multiple invocations with errors would only yield one error reported. I do not think this makes sense if another tool is using this. > + (else > + (warning (G_ "Something went wrong with ~s.~%") args) A catch-all is fine, but we should at least report the actual error, even if it's not pretty: (warning (G_ "Failed to import package ~s.~% reason: ~s") package-name (exception-args c)) However, if we want to move in the direction of proper error handling like guix/packages.scm and guix/ui.scm, we can do something like... --8<---------------cut here---------------start------------->8--- (define (report-import-error package-name error) (cond ((http-get-error? error) [...] (else [...])))) (define* (go-module->guix-package* module-path #:key (on-error report-import-error) #:allow-other-keys #:rest args) (with-exception-handler (lambda (error) (on-error module-path error) (values #f '())) (lambda () (apply go-module->guix-package module-path args)) #:unwind? #t)) (define* (go-module-recursive-import package-name #:key (goproxy "https://proxy.golang.org") version pin-versions?) (recursive-import package-name #:repo->guix-package (memoize (lambda* (name #:key version repo) (go-module->guix-package* name #:goproxy goproxy #:version version #:pin-versions? pin-versions?))) #:guix-name go-module->guix-package-name #:version version)) --8<---------------cut here---------------end--------------->8--- Looks like I got a little carried away :) But breaking out the error reporting gives us the future option of "plugging in" other error reporting strategies, such as collating them and returning them alongside a programmatic recursive import, or being able to provide a list of packages which failed to import at the end. This will be much more useful once we have a proper set of import error conditions. Too much, perhaps? Sarah From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 28 07:42:46 2021 Received: (at 49196) by debbugs.gnu.org; 28 Jun 2021 11:42:46 +0000 Received: from localhost ([127.0.0.1]:51179 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxpf4-0004TN-Dy for submit@debbugs.gnu.org; Mon, 28 Jun 2021 07:42:46 -0400 Received: from mail-qk1-f172.google.com ([209.85.222.172]:35488) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxpf1-0004T5-8y for 49196@debbugs.gnu.org; Mon, 28 Jun 2021 07:42:45 -0400 Received: by mail-qk1-f172.google.com with SMTP id q190so27050725qkd.2 for <49196@debbugs.gnu.org>; Mon, 28 Jun 2021 04:42:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=PDsX5SYmlU78jiHwO2tVkXni3x1yyyD50t036g50Eoo=; b=W/0HfCS+VUUsLB4IOSp6SAFlTTsiv3Q98czvaZEPdmhbiKLQbBCQOhnXWn0Oe90arR mBXyGUQoqJIdgH/mM2K0yUhYUuYvIrY6rW7YLUN99iwRngS96A2ody/JGNbyp4jISknc y6csfMoLsCNqKmkxkXNsDtVsJlJS5NgLZ3BUNFuNzFBsdWclLA7YFbop+zFZVam2tLJs GJYWnKbg4MnQXPv08yGjxZ6qUVFxcHj/SNsEC+CJ7/58b3uq0fs+9u4z98ToYhHFIxTv uD+AGguYM7AboJwLd3l1G/vDK2nlzjLQApDVCCkmW7gFXpxb3EyTOll/qF7JxFfwqzXT qivg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=PDsX5SYmlU78jiHwO2tVkXni3x1yyyD50t036g50Eoo=; b=afkuuAUVSb5g/rxXi8555QmQ5JRs8rJZzUpnIjco/XmNZTpCrRNSGYcaZtoDQY9Vbd /fL+N5VaTzPo2l1e7ht4gruJEQA0rbYSvVzz5DiS9dWwatDRsP6//r1piRFDgAUToDZ4 h/5M0Mk6WGOmRU9t2zrcfPzFCcqOn2p4mQUE5tvyPYGqNTIBQ+atP0RxhOGZn+VxRZ4E KvhKwDOzIPNYXVCh+oebRBTA2o6bxSH1Nz53hi0vbWLpoR39RSboff6vW0J061i3uLmo Fy/+UZbf5al9B7yBnsJ8rQnc2rlHirAlHMMV3AFl3dnwW9qJbA19ZkJataBRxqeYENyr ucdQ== X-Gm-Message-State: AOAM532pGmhCCzG/EKiPeRTuf8NR+IusNAKppr2rqYlGJBjCLuYsYc6T Z06L/v5jbzJafdIQFspqJEBKQOpSSN8pEr+rTNI= X-Google-Smtp-Source: ABdhPJwU9cMGTgYpotVuRdfuMKTKUmQu/bI8OXSLWiTzBdXCqPsxZ+lfOhZm0XYQfWwovT4xWburpFfD4TugkrOWDyQ= X-Received: by 2002:a05:620a:543:: with SMTP id o3mr25129954qko.80.1624880557680; Mon, 28 Jun 2021 04:42:37 -0700 (PDT) MIME-Version: 1.0 References: <20210625163749.65196-1-zimon.toutoune@gmail.com> <20210625163749.65196-3-zimon.toutoune@gmail.com> <868s2v7w4t.fsf_-_@mgsn.dev> In-Reply-To: <868s2v7w4t.fsf_-_@mgsn.dev> From: zimoun Date: Mon, 28 Jun 2021 13:42:26 +0200 Message-ID: Subject: Re: bug#49196: [PATCH] import: utils: 'recursive-import' skips unfound packages To: Sarah Morgensen Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 49196 Cc: 49196@debbugs.gnu.org, =?UTF-8?Q?J=C3=A9r=C3=A9my_Korwin=2DZmijowski?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Sun, 27 Jun 2021 at 06:46, Sarah Morgensen wrote: > A catch-all is fine, but we should at least report the actual error, > even if it's not pretty: > > (warning (G_ "Failed to import package ~s.~% reason: ~s") > package-name (exception-args c)) Well, why not, even if I am not convinced the reason is meaningful because it is mostly an incorrect parsing which returns: reason: ("match" "no matching pattern" #f). and I think it is better to display the complete 'args' instead of extract the URL (package-name) from 'args'. > However, if we want to move in the direction of proper error handling > like guix/packages.scm and guix/ui.scm, we can do something like... Thanks for the idea. As explained patch#45984 [1], all the UI messages must be in guix/scripts/import and not in guix/import and therefore, indeed, error reporting needs to be unified between all the importers and raised accordingly; that's what we are working on with jeko (J=C3=A9r=C3=A9my Korwin-Zmijowski) as pair-programming exercise. :-) 1: > --8<---------------cut here---------------start------------->8--- > (define (report-import-error package-name error) > (cond > ((http-get-error? error) > [...] > (else > [...])))) > > (define* (go-module->guix-package* module-path > #:key (on-error report-import-error) > #:allow-other-keys #:rest args) > (with-exception-handler > (lambda (error) > (on-error module-path error) > (values #f '())) > (lambda () (apply go-module->guix-package module-path args)) > #:unwind? #t)) > > (define* (go-module-recursive-import package-name > #:key (goproxy "https://proxy.golang= .org") > version > pin-versions?) > (recursive-import > package-name > #:repo->guix-package > (memoize > (lambda* (name #:key version repo) > (go-module->guix-package* name #:goproxy goproxy > #:version version > #:pin-versions? pin-versions?))) > #:guix-name go-module->guix-package-name > #:version version)) > --8<---------------cut here---------------end--------------->8--- > > Looks like I got a little carried away :) But breaking out the error > reporting gives us the future option of "plugging in" other error > reporting strategies, such as collating them and returning them > alongside a programmatic recursive import, or being able to provide a > list of packages which failed to import at the end. This will be much > more useful once we have a proper set of import error conditions. Back to the initial patch, I think it is better to simply fix with the minor improvements of v3 your proposed and let this last proposal for #45984; feel free to comment there. ;-) Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 28 12:20:35 2021 Received: (at 49196) by debbugs.gnu.org; 28 Jun 2021 16:20:35 +0000 Received: from localhost ([127.0.0.1]:52329 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxtzr-0005Un-UK for submit@debbugs.gnu.org; Mon, 28 Jun 2021 12:20:35 -0400 Received: from mail-wr1-f49.google.com ([209.85.221.49]:40769) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxtzp-0005UU-Hd for 49196@debbugs.gnu.org; Mon, 28 Jun 2021 12:20:29 -0400 Received: by mail-wr1-f49.google.com with SMTP id g7so17360637wri.7 for <49196@debbugs.gnu.org>; Mon, 28 Jun 2021 09:20:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=xyfzeGHjgYipDP017Y3vpgziAEjuDkbsOhIFiawVpP0=; b=crmwLD4l1byqZsAm5RleM2fxg6pYz7kvI6Bn1EmMKab5b28hJyrzPRNtUIVuo/CiN7 On9qEqq4j3I9COG8B/aCcvshtt0T0dRum9dsT0QhGEqI3LXMBPnT/NxMbZxEjb/WEuL2 DEWjFwGhVX4tJfhY5EhotMHov5HpjdAcCTvrEFPDZvj16kV7vHLAHuR6r8nan6oI8/gO QXzTcZrGYNhi/0OhQrE162wT15YTLtMm3tHFMnlwco8q94fnfG2qCKP86IzPJpe4sol1 w2b6wwNsZwMISxm0j7OFpbBRQmGy54xrS09tS0nbbXUIfknQ1J2hHMQiAtYzMmRtsmJ2 +cFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=xyfzeGHjgYipDP017Y3vpgziAEjuDkbsOhIFiawVpP0=; b=lefwR+8LdUbE+Hto+avQTMOkVvPz4URnSMXH2N9QRG9+hJUwJApNn+oOwNhGspL8iD tdaMH+xJqWtQTPpgjNHKfVoRmC/xqEKmEpDtwRamGrAuGZ/O/qb2mZAVdHdmcl/Oy2Sr pEOqFh8QVqbo7DGDa8/jiRX0etyNjTT52RmDjl2BgxrTi/+Q2Tj7MbHZs7EFFiiT5/tz ql2l/R5zkR0jvLMZAxQvUuZWJPUBk/mKBOdQrGRQMGHuPfvUfSRW+nbhw2fd0sHUhg5k +s4homCv9hdnwpIlm7x2GxtaDIBpf3xE/AusvwS8WBxspv5d6BFwQAl0kqvRlZ1ZXgqt aetg== X-Gm-Message-State: AOAM533rM1SWqT8yK87BGzlFpmsNhh1NEItCVa2ojHycmuUpntIMgUrq DTcaJlwBLS5BpRPXxJHvLre8n3GoUAo= X-Google-Smtp-Source: ABdhPJwWL623Zy1dk36NXncqxW+uYn0lekxLyBz7TJwM/sU0yeJqSDSReld+fAsC9hJqmXcD0iqzDw== X-Received: by 2002:a5d:6992:: with SMTP id g18mr27909451wru.73.1624897223495; Mon, 28 Jun 2021 09:20:23 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id v18sm17050567wrv.24.2021.06.28.09.20.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Jun 2021 09:20:23 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v4 1/3] import: go: Return false for package not found. Date: Mon, 28 Jun 2021 18:20:12 +0200 Message-Id: <20210628162014.15319-1-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Sarah Morgensen * guix/import/go.scm (go-module-recursive-import): Explicitly return false when packages are not found. --- guix/import/go.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index 5e23d6a2b3..a2863c79ad 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -653,7 +653,7 @@ This package and its dependencies won't be imported.~%") (uri->string (http-get-error-uri c)) (http-get-error-code c) (http-get-error-reason c)) - (values '() '()))) + (values #f '()))) (receive (package-sexp dependencies) (go-module->guix-package* name #:goproxy goproxy #:version version base-commit: 77c9c5c103ed2d09a43709bb78bc0b13a399e797 -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 28 12:20:37 2021 Received: (at 49196) by debbugs.gnu.org; 28 Jun 2021 16:20:37 +0000 Received: from localhost ([127.0.0.1]:52331 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxtzw-0005V4-Cr for submit@debbugs.gnu.org; Mon, 28 Jun 2021 12:20:37 -0400 Received: from mail-wr1-f46.google.com ([209.85.221.46]:38776) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxtzp-0005UV-Jp for 49196@debbugs.gnu.org; Mon, 28 Jun 2021 12:20:35 -0400 Received: by mail-wr1-f46.google.com with SMTP id r8so8204686wrx.5 for <49196@debbugs.gnu.org>; Mon, 28 Jun 2021 09:20:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rty0ZqD+74SEHhUaLtCsSlBbmFKBoQTnN3k5Vftk+7k=; b=QGnuDnBzuU6Yo0rhapoQIUuzDIFTOtIzfTIzNGlYw8txbE8rnayWpl47QpplG4yFob Fsb95I4xBKe64ul7bZFJFLy5nvkkxM4zlyAfp1A3NQN+HV/k+XKUIfCoWDNLun8/345b VCg4CYdk9qktsCa6RgGmg8DJsvkCfsPDwNX9OLbs7hnFAGQB+j0pnp2Sg0O7cz/X7pZs aarou4FD9RCJWR3tizLmmLDBticLhpTFTn0yiDjPWNzaAJDxh4Mpm3+ZyHFzK0ZWcKso pW408AhSD4MZTCSyWXO7pBwxm5yrYh1xy1LN2FB0ryxNzjycwD/7yLsk434tgYPCofuq xZJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rty0ZqD+74SEHhUaLtCsSlBbmFKBoQTnN3k5Vftk+7k=; b=p5HOY1s5qyJSwoIuolaqfCqW8bdSEqBQJ8Yk17w0Rmu7dQitdht9sAkX0A5VLV3iMa gJy9DuKaHFOoeZR4o9nsbxAUnmkgmEbbH8MWNvtbcmh8Cu2NlS4VCGy9FbqY3dpJ+gqc VXoGmP48HOzuWtiiYGfpFNM2PKMmauoJruo7oAjl2Q/t+RmFWaiSIgdUy9IFgWhmiMyr cgOzc524j3K0bTvP23ppgHoqPcWNQhzniFbg8LL8MNbrxIU1msBj17ExwncMDmLBZqFu 97NYUDeqAhQt9AxxOC85W6mm4Sbn3Jp7V8xvltCJGTqy4KggiSgL/Bs0roNbK6lVYbQN RK0Q== X-Gm-Message-State: AOAM5334BBnsJrKXo7uaDRIP39EY5tfgxq8+9S9KvWEZ4z7TdyPX5Uf3 3dNIEtcG/wfAM5CtCeMw+B/L3zAxNmI= X-Google-Smtp-Source: ABdhPJz+s04ntMiKNgglNM3GXUWKW1YRF7NXCNBP+gfQTAGMCKbxCL6H7HbaVMgnQkmIMZblbIJFYg== X-Received: by 2002:adf:f690:: with SMTP id v16mr12259239wrp.411.1624897224033; Mon, 28 Jun 2021 09:20:24 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id v18sm17050567wrv.24.2021.06.28.09.20.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Jun 2021 09:20:23 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v4 2/3] import: utils: Skip not found packages. Date: Mon, 28 Jun 2021 18:20:13 +0200 Message-Id: <20210628162014.15319-2-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210628162014.15319-1-zimon.toutoune@gmail.com> References: <20210628162014.15319-1-zimon.toutoune@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Sarah Morgensen * guix/import/utils.scm (recursive-import): Skip packages when the package returned by 'repo->guix-package' is false. * tests/import-utils.scm: New tests. --- guix/import/utils.scm | 26 ++++++++++++++------------ tests/import-utils.scm | 28 ++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 12 deletions(-) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index d817318a91..7f2e7ecb46 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com> ;;; Copyright © 2020 Martin Becze ;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -471,15 +472,16 @@ to obtain the Guix package name corresponding to the upstream name." (name (list name #f))) dependencies))) (make-node name version package normalized-deps))) - (map node-package - (topological-sort (list (lookup-node package-name version)) - (lambda (node) - (map (lambda (name-version) - (apply lookup-node name-version)) - (remove (lambda (name-version) - (apply exists? name-version)) - (node-dependencies node)))) - (lambda (node) - (string-append - (node-name node) - (or (node-version node) "")))))) + (filter-map + node-package + (topological-sort (list (lookup-node package-name version)) + (lambda (node) + (map (lambda (name-version) + (apply lookup-node name-version)) + (remove (lambda (name-version) + (apply exists? name-version)) + (node-dependencies node)))) + (lambda (node) + (string-append + (node-name node) + (or (node-version node) "")))))) diff --git a/tests/import-utils.scm b/tests/import-utils.scm index 874816442e..7c6c782917 100644 --- a/tests/import-utils.scm +++ b/tests/import-utils.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2017 Ricardo Wurmus ;;; Copyright © 2016 Ben Woodcroft ;;; Copyright © 2020 Martin Becze +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -64,6 +65,33 @@ '()))) #:guix-name identity)) +(test-equal "recursive-import: skip false packages (toplevel)" + '() + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + +(test-equal "recursive-import: skip false packages (dependency)" + '((package + (name "foo") + (inputs `(("bar" ,bar))))) + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values '(package + (name "foo") + (inputs `(("bar" ,bar)))) + '("bar"))) + (("bar" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + (test-assert "alist->package with simple source" (let* ((meta '(("name" . "hello") ("version" . "2.10") -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 28 12:20:48 2021 Received: (at 49196) by debbugs.gnu.org; 28 Jun 2021 16:20:48 +0000 Received: from localhost ([127.0.0.1]:52333 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxu06-0005Va-5Q for submit@debbugs.gnu.org; Mon, 28 Jun 2021 12:20:47 -0400 Received: from mail-wr1-f49.google.com ([209.85.221.49]:41861) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxtzq-0005UW-1l for 49196@debbugs.gnu.org; Mon, 28 Jun 2021 12:20:45 -0400 Received: by mail-wr1-f49.google.com with SMTP id u8so8683847wrq.8 for <49196@debbugs.gnu.org>; Mon, 28 Jun 2021 09:20:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l/x0W3/XbvXoKmOKYcgxtiOksxk+Xm5/lB61rm36/pI=; b=LmRo6e6t2eSJVDdktPPfz16VcqydcYm8g4+IRRALgZRkKAPjRkMgte7zvBo8t9cDzN TWhHgMGC3zyErmmkPLFybwzfCwVQzIYlVoU3sFrtR4paqGMZisWP7mv/l0/ylPWZneBW cQo8YrRv0QOkDrnQzKBPIfm6p2M3mz1QNsuCpQpsrcX6SoADqC5b1wkBpP8txrYJzHVW M5FpC+XDrSQ35ne6n3/EIID6B0kvz21u+v9DUsWhcj/en3gpAgNLB/EcrVCHeVpC4KTR YOtxJRfIM1npKM75ZEJgvyQoFiUbBXR6Kp9IVn8rGjynsaSwcd0cqzrInngE9B7HQYvC Wg1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l/x0W3/XbvXoKmOKYcgxtiOksxk+Xm5/lB61rm36/pI=; b=knUtJtT6U+xnr8BpDaQWgsxj0Se/uBkoHj0kmnoRZGaDQX7jA1C9BAXPCRrxmsD4Ur bUB2+1oZB2r1hJ9vjCedEDu9xjKqtdHEUXSctOkPznqqJFmXfLK8kspJkdturx0VOZDN 0Eyb8Jrlf064SFJuawXnPToz3Cl+j7m5gWnLSWz1RN6XsW3ApcCLfeJNITJNfh/qJQJ0 ZDxDIBbX1P797HHUY+z/cOBsdO5BAQbSOVm+Tx4pDTAkVNktYY5KyTpnt03Dhs23uxOd H7zwC81AGFNJpAdNKWfbiRtPBor/zoacUdT4JT/yvHG1z5oh4k7oLl2xEuSq+QWrCWpw Ox2w== X-Gm-Message-State: AOAM530lJLEbv0D1jUA/bcLy/uCLtgXkjXH/04MTqli0Us7xckMiuvPx /E3wrtYNN+wT0z0AYqSeJCgLYIz8NpM= X-Google-Smtp-Source: ABdhPJz6cI9aA9S/z303Uq1vln1W+FEsOWPkm0u/WW6MjDBZTuTSbmg2Bm3tWdYQKrZJCvFMRNZVKg== X-Received: by 2002:a05:6000:1251:: with SMTP id j17mr28543979wrx.122.1624897224606; Mon, 28 Jun 2021 09:20:24 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id v18sm17050567wrv.24.2021.06.28.09.20.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Jun 2021 09:20:24 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v4 3/3] import: go: Improve error handling. Date: Mon, 28 Jun 2021 18:20:14 +0200 Message-Id: <20210628162014.15319-3-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210628162014.15319-1-zimon.toutoune@gmail.com> References: <20210628162014.15319-1-zimon.toutoune@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/import/go.scm (go-module->guix-package*): Handle errors. (go-module-recursive-import): Remove 'guard'. * guix/scripts/import/go.scm (guix-import-go): Adjust. * tests/go.scm: Adjust. --- guix/im [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (zimon.toutoune[at]gmail.com) X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev, zimoun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) * guix/import/go.scm (go-module->guix-package*): Handle errors. (go-module-recursive-import): Remove 'guard'. * guix/scripts/import/go.scm (guix-import-go): Adjust. * tests/go.scm: Adjust. --- guix/import/go.scm | 49 +++++++++++++++++++++++--------------- guix/scripts/import/go.scm | 6 ++--- tests/go.scm | 2 +- 3 files changed, 34 insertions(+), 23 deletions(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index a2863c79ad..e7d8bc3959 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -5,7 +5,8 @@ ;;; Copyright © 2021 Maxim Cournoyer ;;; Copyright © 2021 Ludovic Courtès ;;; Copyright © 2021 Xinglu Chen -;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2021 Simon Tournier ;;; ;;; This file is part of GNU Guix. ;;; @@ -62,6 +63,7 @@ #:use-module (web uri) #:export (go-module->guix-package + go-module->guix-package* go-module-recursive-import)) ;;; Commentary: @@ -631,7 +633,28 @@ hint: use one of the following available versions ~a\n" dependencies+versions dependencies)))) -(define go-module->guix-package* (memoize go-module->guix-package)) +(define go-module->guix-package* + (lambda args + ;; Disable output buffering so that the following warning gets printed + ;; consistently. + (setvbuf (current-error-port) 'none) + (let ((package-name (match args ((name _ ...) name)))) + (guard (c ((http-get-error? c) + (warning (G_ "Failed to import package ~s. +reason: ~s could not be fetched: HTTP error ~a (~s). +This package and its dependencies won't be imported.~%") + package-name + (uri->string (http-get-error-uri c)) + (http-get-error-code c) + (http-get-error-reason c)) + (values #f '())) + (else + (warning (G_ "Failed to import package ~s. +reason: ~s.~%") + package-name + (exception-args c)) + (values #f '()))) + (apply go-module->guix-package args))))) (define* (go-module-recursive-import package-name #:key (goproxy "https://proxy.golang.org") @@ -642,22 +665,10 @@ hint: use one of the following available versions ~a\n" package-name #:repo->guix-package (lambda* (name #:key version repo) - ;; Disable output buffering so that the following warning gets printed - ;; consistently. - (setvbuf (current-error-port) 'none) - (guard (c ((http-get-error? c) - (warning (G_ "Failed to import package ~s. -reason: ~s could not be fetched: HTTP error ~a (~s). -This package and its dependencies won't be imported.~%") - name - (uri->string (http-get-error-uri c)) - (http-get-error-code c) - (http-get-error-reason c)) - (values #f '()))) - (receive (package-sexp dependencies) - (go-module->guix-package* name #:goproxy goproxy - #:version version - #:pin-versions? pin-versions?) - (values package-sexp dependencies)))) + (receive (package-sexp dependencies) + (go-module->guix-package* name #:goproxy goproxy + #:version version + #:pin-versions? pin-versions?) + (values package-sexp dependencies))) #:guix-name go-module->guix-package-name #:version version)) diff --git a/guix/scripts/import/go.scm b/guix/scripts/import/go.scm index 74e8e60cce..071ecdb2ef 100644 --- a/guix/scripts/import/go.scm +++ b/guix/scripts/import/go.scm @@ -115,10 +115,10 @@ that are not yet in Guix")) (map package->definition* (apply go-module-recursive-import arguments)) ;; Single import. - (let ((sexp (apply go-module->guix-package arguments))) + (let ((sexp (apply go-module->guix-package* arguments))) (unless sexp - (leave (G_ "failed to download meta-data for module '~a'~%") - module-name)) + (leave (G_ "failed to download meta-data for module '~a'.~%") + name)) (package->definition* sexp)))))) (() (leave (G_ "too few arguments~%"))) diff --git a/tests/go.scm b/tests/go.scm index b088ab50d2..929a8b39d1 100644 --- a/tests/go.scm +++ b/tests/go.scm @@ -286,6 +286,6 @@ package.") (nix-base32-string->bytevector "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5") #f))) - (go-module->guix-package "github.com/go-check/check"))))))) + (go-module->guix-package* "github.com/go-check/check"))))))) (test-end "go") -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 28 13:13:27 2021 Received: (at 49196) by debbugs.gnu.org; 28 Jun 2021 17:13:27 +0000 Received: from localhost ([127.0.0.1]:52402 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxup5-000702-IN for submit@debbugs.gnu.org; Mon, 28 Jun 2021 13:13:27 -0400 Received: from out0.migadu.com ([94.23.1.103]:59513) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lxup1-0006zs-VV for 49196@debbugs.gnu.org; Mon, 28 Jun 2021 13:13:26 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1624900402; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xp7OjEw7UWjO6YuhmV5GU6Ccz8MkYx8O/KgyKDQ1mKc=; b=P7/hrmyGHxiCewTIbpLJxf7ms+MajW6AaPMnV0qkhF0M2gK5NWzfjEUpuyMyXndslqIl32 9+dbyTfiwlDVUFWsn7PLougWJhCc2KXiVfXKLSuY+hrE4THCZk8BbvBuTRX4KlP44EDjo1 vloaVXpOABudAtxf3jBW0FI0P04zFho= From: Sarah Morgensen To: zimoun Subject: Re: bug#49196: [PATCH] import: utils: 'recursive-import' skips unfound packages In-Reply-To: (zimoun's message of "Mon, 28 Jun 2021 13:42:26 +0200") References: <20210625163749.65196-1-zimon.toutoune@gmail.com> <20210625163749.65196-3-zimon.toutoune@gmail.com> <868s2v7w4t.fsf_-_@mgsn.dev> Date: Mon, 28 Jun 2021 10:13:19 -0700 Message-ID: <86lf6t7w0g.fsf_-_@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 49196 Cc: 49196@debbugs.gnu.org, =?utf-8?B?SsOpcsOpbXk=?= Korwin-Zmijowski X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) zimoun writes: Hello, Thanks for the v4. > Hi, > > On Sun, 27 Jun 2021 at 06:46, Sarah Morgensen wrote: > >> A catch-all is fine, but we should at least report the actual error, >> even if it's not pretty: >> >> (warning (G_ "Failed to import package ~s.~% reason: ~s") >> package-name (exception-args c)) > > Well, why not, even if I am not convinced the reason is meaningful > because it is mostly an incorrect parsing which returns: > > reason: ("match" "no matching pattern" #f). > Yes, it is a less than ideal compromise... I could not quickly figure out how to properly format it without a lot of complexity (like guix/ui.scm does in 'call-with-error-handling'). I found it hard to read the full exception object, but I would not object strongly to printing the full exception object either. As you say, your patch will fix it anyway ;) > and I think it is better to display the complete 'args' instead of > extract the URL (package-name) from 'args'. You're not wrong; I was just trying to keep it somewhat consistent with the other error message. >> However, if we want to move in the direction of proper error handling >> like guix/packages.scm and guix/ui.scm, we can do something like... > > Thanks for the idea. As explained patch#45984 [1], all the UI > messages must be in guix/scripts/import and not in guix/import and Yes, this was my secret trick: having separated out the error reporting, it could be easily be moved to scripts/import later. =20 > therefore, indeed, error reporting needs to be unified between all the > importers and raised accordingly; that's what we are working on with > jeko (J=C3=A9r=C3=A9my Korwin-Zmijowski) as pair-programming exercise. :-) I look forward to the results! > Back to the initial patch, I think it is better to simply fix with the > minor improvements of v3 your proposed and let this last proposal for > #45984; feel free to comment there. ;-) I agree. Your v4 looks good to me, except... > #:repo->guix-package > (lambda* (name #:key version repo) I apologize for not being clear earlier; by "put [memoize] back in later on" I meant "later on in the call chain," e.g. #:repo->guix-package + (memoize (lambda* (name #:key version repo) That's my only nit this time! ;) Thanks for bearing with me. Regards, Sarah From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 29 06:53:03 2021 Received: (at 49196) by debbugs.gnu.org; 29 Jun 2021 10:53:03 +0000 Received: from localhost ([127.0.0.1]:53684 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyBMV-0007on-8A for submit@debbugs.gnu.org; Tue, 29 Jun 2021 06:53:03 -0400 Received: from mail-wm1-f45.google.com ([209.85.128.45]:36399) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyBMR-0007o4-Lr for 49196@debbugs.gnu.org; Tue, 29 Jun 2021 06:53:01 -0400 Received: by mail-wm1-f45.google.com with SMTP id m41-20020a05600c3b29b02901dcd3733f24so1489708wms.1 for <49196@debbugs.gnu.org>; Tue, 29 Jun 2021 03:52:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rty0ZqD+74SEHhUaLtCsSlBbmFKBoQTnN3k5Vftk+7k=; b=JIUDhdtpSNdHo/yIp4CRlrP1/NzvCPfls5BNZ1z2ujyAlvDwnSkSIotH3MkCvthUAI AN40K86HQigL7XfNtISCD+swjlgCpyETDS4p8Om85dP+f84im9upuB9scC/rv4ia/g4A rMlX0oF22ZRoygMBlfY7KZeG9Ii209eP5c3qExvOkSR8FEyNUjQQIP5JBbBJKarsi5Qr nmjNvqetRiZwC2tLzg+gvnh5uukjd5/0fBjj/iM8eLmvgftgsN9DpoXmXxju71Rl9D2C RfRxGyXiqmZev3q2nwEVE1IQKVJqsI7tv828+jI1HPXaLPj7q0iLAfdMtTCU91eOM6Ku vdqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rty0ZqD+74SEHhUaLtCsSlBbmFKBoQTnN3k5Vftk+7k=; b=mfoZ3UC6nKXuJx2F/EhMKgX0R2nGOvTngbOmXHIJTLgwySIcciaOApTPV4CIo+bLck gJdIVpcS9ouseHbYxtlM21okASYoHa4s9OImzBkFh+IrHNokpnPwjfXAt2ImdXE9WyXs cys5ZBicoM2mskemn4WUHN2GwoARr09QW3tCBKDu236CEgCiD1mzJjXyvM9YaYnl1/0g jEV4iljor1d+veCjSkf+QQyThAWcdXK5Ih9ZLhvZbUZuPsA2gD/0huckq3r75r6SRicO MeJZX0ZQ/Eh+zdtuzjnR6pS6/JEEspSGKgvRhGAgnNKphIxJt+yTPT8zSxUv7NVp91U5 iH0Q== X-Gm-Message-State: AOAM531IRlaOuDshZTPmItZH4ZsrvF6zBWoOjwU+M+YCx8cnDWzok8uv 4KVF8Wxr87bVqmtF/Aa2nNA/stUPCkM= X-Google-Smtp-Source: ABdhPJybMDv02phLmZWqzOyBiOCuPeBCr3pEno4J+dEpdOYv4HU5grz3W4PcuhYXN85eFAODZM8Lyw== X-Received: by 2002:a05:600c:3b8b:: with SMTP id n11mr31150510wms.159.1624963973995; Tue, 29 Jun 2021 03:52:53 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id c7sm17784794wrs.23.2021.06.29.03.52.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Jun 2021 03:52:53 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v5 2/3] import: utils: Skip not found packages. Date: Tue, 29 Jun 2021 12:52:36 +0200 Message-Id: <20210629105237.45517-2-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210629105237.45517-1-zimon.toutoune@gmail.com> References: <20210629105237.45517-1-zimon.toutoune@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Sarah Morgensen * guix/import/utils.scm (recursive-import): Skip packages when the package returned by 'repo->guix-package' is false. * tests/import-utils.scm: New tests. --- guix/import/utils.scm | 26 ++++++++++++++------------ tests/import-utils.scm | 28 ++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 12 deletions(-) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index d817318a91..7f2e7ecb46 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com> ;;; Copyright © 2020 Martin Becze ;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -471,15 +472,16 @@ to obtain the Guix package name corresponding to the upstream name." (name (list name #f))) dependencies))) (make-node name version package normalized-deps))) - (map node-package - (topological-sort (list (lookup-node package-name version)) - (lambda (node) - (map (lambda (name-version) - (apply lookup-node name-version)) - (remove (lambda (name-version) - (apply exists? name-version)) - (node-dependencies node)))) - (lambda (node) - (string-append - (node-name node) - (or (node-version node) "")))))) + (filter-map + node-package + (topological-sort (list (lookup-node package-name version)) + (lambda (node) + (map (lambda (name-version) + (apply lookup-node name-version)) + (remove (lambda (name-version) + (apply exists? name-version)) + (node-dependencies node)))) + (lambda (node) + (string-append + (node-name node) + (or (node-version node) "")))))) diff --git a/tests/import-utils.scm b/tests/import-utils.scm index 874816442e..7c6c782917 100644 --- a/tests/import-utils.scm +++ b/tests/import-utils.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2017 Ricardo Wurmus ;;; Copyright © 2016 Ben Woodcroft ;;; Copyright © 2020 Martin Becze +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -64,6 +65,33 @@ '()))) #:guix-name identity)) +(test-equal "recursive-import: skip false packages (toplevel)" + '() + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + +(test-equal "recursive-import: skip false packages (dependency)" + '((package + (name "foo") + (inputs `(("bar" ,bar))))) + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values '(package + (name "foo") + (inputs `(("bar" ,bar)))) + '("bar"))) + (("bar" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + (test-assert "alist->package with simple source" (let* ((meta '(("name" . "hello") ("version" . "2.10") -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 29 06:53:03 2021 Received: (at 49196) by debbugs.gnu.org; 29 Jun 2021 10:53:03 +0000 Received: from localhost ([127.0.0.1]:53686 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyBMV-0007op-KJ for submit@debbugs.gnu.org; Tue, 29 Jun 2021 06:53:03 -0400 Received: from mail-wm1-f42.google.com ([209.85.128.42]:41586) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyBMR-0007o3-D1 for 49196@debbugs.gnu.org; Tue, 29 Jun 2021 06:53:02 -0400 Received: by mail-wm1-f42.google.com with SMTP id a5-20020a7bc1c50000b02901e3bbe0939bso2133375wmj.0 for <49196@debbugs.gnu.org>; Tue, 29 Jun 2021 03:52:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=IGA4ZSM54I/5q0++oHK7QuQzQUGYQVY/2a8hg+KRNY4=; b=HTmj9VQvb+Rf9MLvmrgNgVPIO92WcMUcsIQpt+YAImH7ExxdyBBFXy/oi3xzXcrmlL AY21wKaylfWzRJh2MFF0H1ydbAlJ8fZRp9cz3vgABQCFMJmsPT4/MFT61NNK9lKpQaSm 6+MVCYAd8LpbYkzpxRLq22LyI+s+ktfkgPLHwectjteyE+yquGh/M1X244Dlw4gDIOIN jnnHEZAUpkH9amCDz0pAaXzdTdxSNeffsgF2cyuZFIJiMDbvwCLtKyiS4dawI/2rWGm7 yq5OXxrhezkuQ1eHoK6F6kNOzIA8iJmXDUzQ8Jig5p4Z28wI2ggjwwyaCZUt6C8Kgh4j hlSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=IGA4ZSM54I/5q0++oHK7QuQzQUGYQVY/2a8hg+KRNY4=; b=ahx1Z1dXCvDJzKiDMrtgnYqjXTrHh7SZ8dgoQbhesmuMWcmOf8cSuGAI2EHp7vYP2j LibiSV0ojfKRO7IjEolgccLuA1umKy8D2oyJns73aAziH4CDu1QjZ9uvh+Yr9p2pPv+1 t9S+AmIUMDuIoV4oQmadDkXkyDqYlKndumgDNzwuJdCoo16rX6BvBctjs5n5wlJnjWHt 676MM+YkSRAPJoupDJ4oRZL0aQgmcak8RVrBAGwQbqoOShLUSJIqZywpfldaeJfXOrHm NiHCkd0y+QMBigKMx2VjMGw5DrDD1dKouZZS88B/VroZ6oIwdXMvsCzU39H292XMnisa /lwg== X-Gm-Message-State: AOAM530h3NEa2/BSj+QI8g1lVJkzXkEXJXTtbXZJh5qBh2DXr17W0RBO 99bc9eV3ZJLIctocGAi5Bpu2VHMfMRA= X-Google-Smtp-Source: ABdhPJwM127EDdjo09GbxEDQBtINIm2mT8Jr/uT/asoXfE5HuRvKA7R/xVSZZSxXgnjaFuvnIq8pWA== X-Received: by 2002:a1c:4646:: with SMTP id t67mr31558167wma.70.1624963973513; Tue, 29 Jun 2021 03:52:53 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id c7sm17784794wrs.23.2021.06.29.03.52.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Jun 2021 03:52:53 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v5 1/3] import: go: Return false for package not found. Date: Tue, 29 Jun 2021 12:52:35 +0200 Message-Id: <20210629105237.45517-1-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Sarah Morgensen * guix/import/go.scm (go-module-recursive-import): Explicitly return false when packages are not found. --- guix/import/go.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index 5e23d6a2b3..a2863c79ad 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -653,7 +653,7 @@ This package and its dependencies won't be imported.~%") (uri->string (http-get-error-uri c)) (http-get-error-code c) (http-get-error-reason c)) - (values '() '()))) + (values #f '()))) (receive (package-sexp dependencies) (go-module->guix-package* name #:goproxy goproxy #:version version base-commit: 5ed105a8bb1a812975496dc3a091596355a0234c -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 29 06:53:08 2021 Received: (at 49196) by debbugs.gnu.org; 29 Jun 2021 10:53:08 +0000 Received: from localhost ([127.0.0.1]:53688 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyBMZ-0007p6-TA for submit@debbugs.gnu.org; Tue, 29 Jun 2021 06:53:08 -0400 Received: from mail-wm1-f44.google.com ([209.85.128.44]:44920) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyBMS-0007o5-69 for 49196@debbugs.gnu.org; Tue, 29 Jun 2021 06:53:02 -0400 Received: by mail-wm1-f44.google.com with SMTP id m9-20020a05600c3b09b02901f246b43bbeso62521wms.3 for <49196@debbugs.gnu.org>; Tue, 29 Jun 2021 03:53:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6GnBx654lagXlXKu1Fv1bgbXFV7lglLDoD95AU3q25o=; b=Jxk7jmw7Ttio7H3lfp7HCYx4Ie+kmScR8UWwt+VX7N7PGmWed9/s7fb9gW0IcyyRcz Ah0NFFbxWnZNZWtgr+loBBROW0nRmIsGoQ+qlB5xz9oloaNzex0fGS/Jsc2wKcG0rlzM r2cMza1Ij2OM1+DP5vAVC1csaraDOrTly1hw6oK8G/dP+lhg5yV+VDGKd71u7NhFbNOn XIxD/CZEYQkxdWjsfoHvvRT0l7jRCYhs3AwNr+pIOEgGDEiVmZQREd7K1HKbuCiD2H6x 4Th+qXX4NmmPOqQ1e0/fIxqWjf6oTLdShSvewGzFWwkJ23w+20Vp4Rtht2WPIh6TebLA /IDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6GnBx654lagXlXKu1Fv1bgbXFV7lglLDoD95AU3q25o=; b=Z0bkbc93v0REoHHKKkxU455TJTwvJuTWRFsCCJbt0kxljF6GExsFS8dd5iBklom/ao 2hC9wJC8H4h7mYfDPxaN0TFj9ZRkSicJao10lz19Bjb4dGWs7PQwo5l90jSADABrIm8f uEwBloLB1FnaNTfQXMZcikrI3KCyqwBd4dB4O/3LUBvQ12tZ7i8rnzF7m+UQeiXACiEd laqCTKTGt2H4uzZffxXjDrYh8iqPNsalpauccHeLe3Ukbs7B+G+HSIxlDIEBeRjdGxbj iHovlww5HeH5C6MayUHyGGELM0iDoQabELMYAZTOktCfFQpjrX0A/fdUNYs8EUZ1oFxA biGQ== X-Gm-Message-State: AOAM532UeCM+bSPMu3unYIEhovaDnQ71hRDwMlLVZTXiTHyh0dW/AwWx sgVI7xdyjgrteN31FiQx9aWaERs2BdY= X-Google-Smtp-Source: ABdhPJyrxx5G4F+mj5EcSTZJ/fOoMOzIn6eTl02c8MdXQ9obfTtaj5cH1CC1gEfdlANwXORC8cicrQ== X-Received: by 2002:a05:600c:4ecd:: with SMTP id g13mr32295835wmq.174.1624963974492; Tue, 29 Jun 2021 03:52:54 -0700 (PDT) Received: from localhost.localdomain ([193.48.40.110]) by smtp.gmail.com with ESMTPSA id c7sm17784794wrs.23.2021.06.29.03.52.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Jun 2021 03:52:54 -0700 (PDT) From: zimoun To: 49196@debbugs.gnu.org Subject: [PATCH v5 3/3] import: go: Improve error handling. Date: Tue, 29 Jun 2021 12:52:37 +0200 Message-Id: <20210629105237.45517-3-zimon.toutoune@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210629105237.45517-1-zimon.toutoune@gmail.com> References: <20210629105237.45517-1-zimon.toutoune@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: * guix/import/go.scm (go-module->guix-package*): Handle errors, remove memoize. (go-module-recursive-import): Remove 'guard', add memoize. * guix/scripts/import/go.scm (guix-import-go): Adjust. * test [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (zimon.toutoune[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.128.44 listed in wl.mailspike.net] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.128.44 listed in list.dnswl.org] X-Debbugs-Envelope-To: 49196 Cc: iskarian@mgsn.dev, zimoun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) * guix/import/go.scm (go-module->guix-package*): Handle errors, remove memoize. (go-module-recursive-import): Remove 'guard', add memoize. * guix/scripts/import/go.scm (guix-import-go): Adjust. * tests/go.scm: Adjust. --- guix/import/go.scm | 52 +++++++++++++++++++++++--------------- guix/scripts/import/go.scm | 6 ++--- tests/go.scm | 2 +- 3 files changed, 36 insertions(+), 24 deletions(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index a2863c79ad..5bec31201d 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -5,7 +5,8 @@ ;;; Copyright © 2021 Maxim Cournoyer ;;; Copyright © 2021 Ludovic Courtès ;;; Copyright © 2021 Xinglu Chen -;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2021 Simon Tournier ;;; ;;; This file is part of GNU Guix. ;;; @@ -62,6 +63,7 @@ #:use-module (web uri) #:export (go-module->guix-package + go-module->guix-package* go-module-recursive-import)) ;;; Commentary: @@ -631,7 +633,28 @@ hint: use one of the following available versions ~a\n" dependencies+versions dependencies)))) -(define go-module->guix-package* (memoize go-module->guix-package)) +(define go-module->guix-package* + (lambda args + ;; Disable output buffering so that the following warning gets printed + ;; consistently. + (setvbuf (current-error-port) 'none) + (let ((package-name (match args ((name _ ...) name)))) + (guard (c ((http-get-error? c) + (warning (G_ "Failed to import package ~s. +reason: ~s could not be fetched: HTTP error ~a (~s). +This package and its dependencies won't be imported.~%") + package-name + (uri->string (http-get-error-uri c)) + (http-get-error-code c) + (http-get-error-reason c)) + (values #f '())) + (else + (warning (G_ "Failed to import package ~s. +reason: ~s.~%") + package-name + (exception-args c)) + (values #f '()))) + (apply go-module->guix-package args))))) (define* (go-module-recursive-import package-name #:key (goproxy "https://proxy.golang.org") @@ -641,23 +664,12 @@ hint: use one of the following available versions ~a\n" (recursive-import package-name #:repo->guix-package - (lambda* (name #:key version repo) - ;; Disable output buffering so that the following warning gets printed - ;; consistently. - (setvbuf (current-error-port) 'none) - (guard (c ((http-get-error? c) - (warning (G_ "Failed to import package ~s. -reason: ~s could not be fetched: HTTP error ~a (~s). -This package and its dependencies won't be imported.~%") - name - (uri->string (http-get-error-uri c)) - (http-get-error-code c) - (http-get-error-reason c)) - (values #f '()))) - (receive (package-sexp dependencies) - (go-module->guix-package* name #:goproxy goproxy - #:version version - #:pin-versions? pin-versions?) - (values package-sexp dependencies)))) + (memoize + (lambda* (name #:key version repo) + (receive (package-sexp dependencies) + (go-module->guix-package* name #:goproxy goproxy + #:version version + #:pin-versions? pin-versions?) + (values package-sexp dependencies)))) #:guix-name go-module->guix-package-name #:version version)) diff --git a/guix/scripts/import/go.scm b/guix/scripts/import/go.scm index 74e8e60cce..071ecdb2ef 100644 --- a/guix/scripts/import/go.scm +++ b/guix/scripts/import/go.scm @@ -115,10 +115,10 @@ that are not yet in Guix")) (map package->definition* (apply go-module-recursive-import arguments)) ;; Single import. - (let ((sexp (apply go-module->guix-package arguments))) + (let ((sexp (apply go-module->guix-package* arguments))) (unless sexp - (leave (G_ "failed to download meta-data for module '~a'~%") - module-name)) + (leave (G_ "failed to download meta-data for module '~a'.~%") + name)) (package->definition* sexp)))))) (() (leave (G_ "too few arguments~%"))) diff --git a/tests/go.scm b/tests/go.scm index b088ab50d2..929a8b39d1 100644 --- a/tests/go.scm +++ b/tests/go.scm @@ -286,6 +286,6 @@ package.") (nix-base32-string->bytevector "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5") #f))) - (go-module->guix-package "github.com/go-check/check"))))))) + (go-module->guix-package* "github.com/go-check/check"))))))) (test-end "go") -- 2.32.0 From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 02 12:33:36 2021 Received: (at control) by debbugs.gnu.org; 2 Jul 2021 16:33:36 +0000 Received: from localhost ([127.0.0.1]:37001 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lzM6h-0007Jp-M7 for submit@debbugs.gnu.org; Fri, 02 Jul 2021 12:33:36 -0400 Received: from mail-wm1-f51.google.com ([209.85.128.51]:34652) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lzM6e-0007JG-8b for control@debbugs.gnu.org; Fri, 02 Jul 2021 12:33:33 -0400 Received: by mail-wm1-f51.google.com with SMTP id u5-20020a7bc0450000b02901480e40338bso6916118wmc.1 for ; Fri, 02 Jul 2021 09:33:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:to:from:subject; bh=Zr12CG9nb3Dp2MV4TsfrKwkMi84bZET1B1zeWZT498M=; b=MOxjWHYWSZ6Mm+UUkx6WtPeA40Vcpd9B2qE+d/nXYf3FUneKCH9SDt4s4do/6aoNji 8ybcs9sUjOiUNfW3RRx2yuYN0mkeKEzDdJ1YA7HQ44nQe5NvP5Oy+4ZriV6ujwCb/SOA SM61SHpK8aH9sqXWQuDahK+6mR4XBkE1r5xdiuIGBJ1UV3MDTfsriiMwcuiTV9zmyGga fS51/y699vmRL/+HtJHHPaRnbpBNEfywIky4NOM8+VHEE19YpwK3cZbxFIcM7BCouocl VYQ1yK0zVZgAE+TxUkds0KDdpmxFSR7G0yFPVjODa3kprplz/TYIoPs78WgBCaUKKUBi oTHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:to:from:subject; bh=Zr12CG9nb3Dp2MV4TsfrKwkMi84bZET1B1zeWZT498M=; b=azVBj7iJJI67AQib32o2o2xK+6+e7WPASi771p00HwUrWmYv6ZxepVgbxjXKKWKqFW YC4xsUemQnkBJ5I4jEByIC5kleBtDupgRPzOxeAeGK7h35OOXg2QqxLl2Po+CS4EcD1U uKWgshmjOcSdNWGy7Bq7udV2UFeGABrIDG+r6jQQZ9mOYHDUHEUmPmgqQI5iHbmOzIvB 5cxPIjxP8Nd7+xFzi2e3HRWwsu+GPXzum7Vzg1huQpJz1JD7BxHCq5hsDBK64JD9wroF 7g4Lj49Te7/duz9goXLzFLlo9KWBLNFa3FV5c01+YdAEbhQEdIDZosj1UuTqp8LhEciL q7WA== X-Gm-Message-State: AOAM5324S7N8kgPE97NNcXx42vi28V0Wx2QMqhzeROZUiFPSMmTgQOsn UcDLR9HerlImhkhUEuGv7zWTY18i39w= X-Google-Smtp-Source: ABdhPJxaQniuAep99Ty438lVe0iBYJsqNv/3L/ddi4bgFFSHVxzS4DeBe8CcsvlANF3aLpZj/D8UeA== X-Received: by 2002:a1c:9d8f:: with SMTP id g137mr652384wme.13.1625243606624; Fri, 02 Jul 2021 09:33:26 -0700 (PDT) Received: from pfiuh02 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id o17sm13041342wmh.19.2021.07.02.09.33.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jul 2021 09:33:26 -0700 (PDT) Date: Fri, 02 Jul 2021 18:26:42 +0200 Message-Id: <87lf6on0l9.fsf@gmail.com> To: control@debbugs.gnu.org From: zimoun Subject: control message for bug #49196 X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) retitle 49196 [PATCH] "guix import go" Improve error handling quit From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 06 14:04:52 2021 Received: (at 49196) by debbugs.gnu.org; 6 Aug 2021 18:04:52 +0000 Received: from localhost ([127.0.0.1]:51305 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mC4DE-0000ih-GL for submit@debbugs.gnu.org; Fri, 06 Aug 2021 14:04:52 -0400 Received: from out0.migadu.com ([94.23.1.103]:24251) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mC4DA-0000iV-LF for 49196@debbugs.gnu.org; Fri, 06 Aug 2021 14:04:51 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1628273086; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LSvG4vl+cLybwy07fs8/J6fAqnP30aqum/ZtlOfrAJk=; b=JpOuHr8bra2apJ71fjqLTsLNqFcvFucAXmXS6RdUmNmwh2MDg/ZKz7ITL3sjy6Ia+TP99t dE1qurmWghMzK9Lly6hsPYP9CV9uKRUmCuA/Yd1tmWB2UQGKj7lQEqCYH9yoHijZUMq8ey 0es8ijDu4FJ8OyVw/qGqw3PUwviSe1g= From: Sarah Morgensen To: 49196@debbugs.gnu.org Subject: [PATCH v6 1/3] import: go: Return false for package not found. Date: Fri, 6 Aug 2021 11:04:43 -0700 Message-Id: <5d11fc04400d26de06efba709cf2e1b14a1d5949.1628272957.git.iskarian@mgsn.dev> In-Reply-To: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> References: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 49196 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) * guix/import/go.scm (go-module-recursive-import): Explicitly return false when packages are not found. --- Hello Guix, I took the liberty of rebasing this patchset since it no longer applies on master. Is there any one else who would like to comment? If not, I believe this has reached a final state. -- Sarah guix/import/go.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index 617a0d0e23..a4775f973f 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -668,7 +668,7 @@ This package and its dependencies won't be imported.~%") (uri->string (http-get-error-uri c)) (http-get-error-code c) (http-get-error-reason c)) - (values '() '()))) + (values #f '()))) (receive (package-sexp dependencies) (go-module->guix-package* name #:goproxy goproxy #:version version base-commit: c8e2be3b32fe784a9db52d8a1a12902ab12ae7cb -- 2.31.1 From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 06 14:05:24 2021 Received: (at 49196) by debbugs.gnu.org; 6 Aug 2021 18:05:24 +0000 Received: from localhost ([127.0.0.1]:51310 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mC4Dj-0000jx-Nn for submit@debbugs.gnu.org; Fri, 06 Aug 2021 14:05:24 -0400 Received: from out1.migadu.com ([91.121.223.63]:51350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mC4Dg-0000jm-WC for 49196@debbugs.gnu.org; Fri, 06 Aug 2021 14:05:22 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1628273119; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GY+tWV8ZnrRX0Txct26gOjpBxfXEfckdpKmstWzlp14=; b=Dc1DxY9FZr9q8xMy0Bx47+tQQFTbwsQfV8Yjxk/Bbn0fFObNGr91A39b4cXMm2jmefAjyr UKPxrm3n60oFAdXsSkAsetuRjZ1y5s9FohLoHzO1QUGzaI1suheuNStmfYBDecCxSWcZJZ sCmYHsauzFKQbangNEUfK+ZdIOA2lR0= From: Sarah Morgensen To: 49196@debbugs.gnu.org Subject: [PATCH v6 2/3] import: utils: Skip not found packages. Date: Fri, 6 Aug 2021 11:05:16 -0700 Message-Id: In-Reply-To: <5d11fc04400d26de06efba709cf2e1b14a1d5949.1628272957.git.iskarian@mgsn.dev> References: <5d11fc04400d26de06efba709cf2e1b14a1d5949.1628272957.git.iskarian@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 49196 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) * guix/import/utils.scm (recursive-import): Skip packages when the package returned by 'repo->guix-package' is false. * tests/import-utils.scm: New tests. --- guix/import/utils.scm | 26 ++++++++++++++------------ tests/import-utils.scm | 28 ++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 12 deletions(-) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index d817318a91..7f2e7ecb46 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com> ;;; Copyright © 2020 Martin Becze ;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -471,15 +472,16 @@ to obtain the Guix package name corresponding to the upstream name." (name (list name #f))) dependencies))) (make-node name version package normalized-deps))) - (map node-package - (topological-sort (list (lookup-node package-name version)) - (lambda (node) - (map (lambda (name-version) - (apply lookup-node name-version)) - (remove (lambda (name-version) - (apply exists? name-version)) - (node-dependencies node)))) - (lambda (node) - (string-append - (node-name node) - (or (node-version node) "")))))) + (filter-map + node-package + (topological-sort (list (lookup-node package-name version)) + (lambda (node) + (map (lambda (name-version) + (apply lookup-node name-version)) + (remove (lambda (name-version) + (apply exists? name-version)) + (node-dependencies node)))) + (lambda (node) + (string-append + (node-name node) + (or (node-version node) "")))))) diff --git a/tests/import-utils.scm b/tests/import-utils.scm index 874816442e..7c6c782917 100644 --- a/tests/import-utils.scm +++ b/tests/import-utils.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2017 Ricardo Wurmus ;;; Copyright © 2016 Ben Woodcroft ;;; Copyright © 2020 Martin Becze +;;; Copyright © 2021 Sarah Morgensen ;;; ;;; This file is part of GNU Guix. ;;; @@ -64,6 +65,33 @@ '()))) #:guix-name identity)) +(test-equal "recursive-import: skip false packages (toplevel)" + '() + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + +(test-equal "recursive-import: skip false packages (dependency)" + '((package + (name "foo") + (inputs `(("bar" ,bar))))) + (recursive-import "foo" + #:repo 'repo + #:repo->guix-package + (match-lambda* + (("foo" #:version #f #:repo 'repo) + (values '(package + (name "foo") + (inputs `(("bar" ,bar)))) + '("bar"))) + (("bar" #:version #f #:repo 'repo) + (values #f '()))) + #:guix-name identity)) + (test-assert "alist->package with simple source" (let* ((meta '(("name" . "hello") ("version" . "2.10") -- 2.31.1 From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 06 14:05:25 2021 Received: (at 49196) by debbugs.gnu.org; 6 Aug 2021 18:05:25 +0000 Received: from localhost ([127.0.0.1]:51312 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mC4Dl-0000kA-5X for submit@debbugs.gnu.org; Fri, 06 Aug 2021 14:05:25 -0400 Received: from out2.migadu.com ([188.165.223.204]:30607) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mC4Dj-0000ju-Ca for 49196@debbugs.gnu.org; Fri, 06 Aug 2021 14:05:24 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1628273121; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s1inxFO+F2xzKSJHxztFV6a2bxDmmhtaUk7aiendReo=; b=Y4JU98yaNGershbRxo6UROr3H8lDWUgktc3JmdWSpMn575r+suzJWRqRx7ftFmStgFG7C6 qauf4/BDBNj5jtnWy461hlPOdgqDVQ4wlp2nAh8E8ZlgrnyqGWD6x6Mkj6c2Ie8LLvdVq8 wWJKBDKBY93NqT6e9liM8boLuTpUcN0= From: Sarah Morgensen To: 49196@debbugs.gnu.org Subject: [PATCH v6 3/3] import: go: Improve error handling. Date: Fri, 6 Aug 2021 11:05:17 -0700 Message-Id: In-Reply-To: <5d11fc04400d26de06efba709cf2e1b14a1d5949.1628272957.git.iskarian@mgsn.dev> References: <5d11fc04400d26de06efba709cf2e1b14a1d5949.1628272957.git.iskarian@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: zimoun * guix/import/go.scm (go-module->guix-package*): Handle errors, remove memoize. (go-module-recursive-import): Remove 'guard', add memoize. * guix/scripts/import/go.scm (guix-import-go): Adjust. * test [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [188.165.223.204 listed in wl.mailspike.net] 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs [URI: yoctocell.xyz (xyz)] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-Debbugs-Envelope-To: 49196 Cc: zimoun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) From: zimoun * guix/import/go.scm (go-module->guix-package*): Handle errors, remove memoize. (go-module-recursive-import): Remove 'guard', add memoize. * guix/scripts/import/go.scm (guix-import-go): Adjust. * tests/go.scm: Adjust. --- guix/import/go.scm | 50 +++++++++++++++++++++++--------------- guix/scripts/import/go.scm | 6 ++--- tests/go.scm | 2 +- 3 files changed, 35 insertions(+), 23 deletions(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index a4775f973f..4755571209 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -6,6 +6,7 @@ ;;; Copyright © 2021 Ludovic Courtès ;;; Copyright © 2021 Xinglu Chen ;;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2021 Simon Tournier ;;; ;;; This file is part of GNU Guix. ;;; @@ -63,6 +64,7 @@ #:use-module (web uri) #:export (go-module->guix-package + go-module->guix-package* go-module-recursive-import)) ;;; Commentary: @@ -646,7 +648,28 @@ hint: use one of the following available versions ~a\n" dependencies+versions dependencies)))) -(define go-module->guix-package* (memoize go-module->guix-package)) +(define go-module->guix-package* + (lambda args + ;; Disable output buffering so that the following warning gets printed + ;; consistently. + (setvbuf (current-error-port) 'none) + (let ((package-name (match args ((name _ ...) name)))) + (guard (c ((http-get-error? c) + (warning (G_ "Failed to import package ~s. +reason: ~s could not be fetched: HTTP error ~a (~s). +This package and its dependencies won't be imported.~%") + package-name + (uri->string (http-get-error-uri c)) + (http-get-error-code c) + (http-get-error-reason c)) + (values #f '())) + (else + (warning (G_ "Failed to import package ~s. +reason: ~s.~%") + package-name + (exception-args c)) + (values #f '()))) + (apply go-module->guix-package args))))) (define* (go-module-recursive-import package-name #:key (goproxy "https://proxy.golang.org") @@ -656,23 +679,12 @@ hint: use one of the following available versions ~a\n" (recursive-import package-name #:repo->guix-package - (lambda* (name #:key version repo) - ;; Disable output buffering so that the following warning gets printed - ;; consistently. - (setvbuf (current-error-port) 'none) - (guard (c ((http-get-error? c) - (warning (G_ "Failed to import package ~s. -reason: ~s could not be fetched: HTTP error ~a (~s). -This package and its dependencies won't be imported.~%") - name - (uri->string (http-get-error-uri c)) - (http-get-error-code c) - (http-get-error-reason c)) - (values #f '()))) - (receive (package-sexp dependencies) - (go-module->guix-package* name #:goproxy goproxy - #:version version - #:pin-versions? pin-versions?) - (values package-sexp dependencies)))) + (memoize + (lambda* (name #:key version repo) + (receive (package-sexp dependencies) + (go-module->guix-package* name #:goproxy goproxy + #:version version + #:pin-versions? pin-versions?) + (values package-sexp dependencies)))) #:guix-name go-module->guix-package-name #:version version)) diff --git a/guix/scripts/import/go.scm b/guix/scripts/import/go.scm index e08a1e427e..f5cfea8683 100644 --- a/guix/scripts/import/go.scm +++ b/guix/scripts/import/go.scm @@ -112,10 +112,10 @@ that are not yet in Guix")) (map package->definition* (apply go-module-recursive-import arguments)) ;; Single import. - (let ((sexp (apply go-module->guix-package arguments))) + (let ((sexp (apply go-module->guix-package* arguments))) (unless sexp - (leave (G_ "failed to download meta-data for module '~a'~%") - module-name)) + (leave (G_ "failed to download meta-data for module '~a'.~%") + name)) (package->definition* sexp)))))) (() (leave (G_ "too few arguments~%"))) diff --git a/tests/go.scm b/tests/go.scm index 6749f4585f..9e7223ff7c 100644 --- a/tests/go.scm +++ b/tests/go.scm @@ -410,6 +410,6 @@ package.") (nix-base32-string->bytevector "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5") #f))) - (go-module->guix-package "github.com/go-check/check"))))))) + (go-module->guix-package* "github.com/go-check/check"))))))) (test-end "go") -- 2.31.1 From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 01 17:39:58 2021 Received: (at 49196-done) by debbugs.gnu.org; 1 Sep 2021 21:39:58 +0000 Received: from localhost ([127.0.0.1]:39125 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mLXxe-0003jw-Kl for submit@debbugs.gnu.org; Wed, 01 Sep 2021 17:39:58 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52104) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mLXxc-0003ji-F6 for 49196-done@debbugs.gnu.org; Wed, 01 Sep 2021 17:39:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:38960) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLXxW-0006r1-Ey; Wed, 01 Sep 2021 17:39:50 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48832 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLXxW-0001wo-47; Wed, 01 Sep 2021 17:39:50 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Sarah Morgensen Subject: Re: bug#49196: [PATCH] "guix import go" Improve error handling References: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> <5d11fc04400d26de06efba709cf2e1b14a1d5949.1628272957.git.iskarian@mgsn.dev> Date: Wed, 01 Sep 2021 23:39:48 +0200 In-Reply-To: <5d11fc04400d26de06efba709cf2e1b14a1d5949.1628272957.git.iskarian@mgsn.dev> (Sarah Morgensen's message of "Fri, 6 Aug 2021 11:04:43 -0700") Message-ID: <87k0k0561n.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 49196-done Cc: 49196-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Sarah Morgensen skribis: > * guix/import/go.scm (go-module-recursive-import): Explicitly return > false when packages are not found. > --- > Hello Guix, > > I took the liberty of rebasing this patchset since it no longer applies on > master. > > Is there any one else who would like to comment? If not, I believe this h= as > reached a final state. I think so too. :-) Applied; thank you and thanks zimoun, and apologies for the delay! Ludo=E2=80=99. From unknown Wed Jun 18 23:04:34 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 30 Sep 2021 11:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator