From unknown Fri Jun 20 07:21:54 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#37254] [PATCH 0/4] Refactor (guix ci) and (guix import crate) Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 01 Sep 2019 14:47:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 37254 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 37254@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.156734921731431 (code B ref -1); Sun, 01 Sep 2019 14:47:01 +0000 Received: (at submit) by debbugs.gnu.org; 1 Sep 2019 14:46:57 +0000 Received: from localhost ([127.0.0.1]:57040 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4R85-0008As-3q for submit@debbugs.gnu.org; Sun, 01 Sep 2019 10:46:57 -0400 Received: from lists.gnu.org ([209.51.188.17]:38785) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4R83-0008Al-B8 for submit@debbugs.gnu.org; Sun, 01 Sep 2019 10:46:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42430) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i4R81-0003yD-QN for guix-patches@gnu.org; Sun, 01 Sep 2019 10:46:55 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_20, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48505) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i4R81-0000zH-9m; Sun, 01 Sep 2019 10:46:53 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40518 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i4R80-00024B-S1; Sun, 01 Sep 2019 10:46:53 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Sun, 1 Sep 2019 16:46:47 +0200 Message-Id: <20190901144647.15185-1-ludo@gnu.org> X-Mailer: git-send-email 2.23.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Guix! Initially I just wanted to fix “guix import crate blake2-rfc”, which didn’t work as Efraim reported on IRC, but that led me to refactor (guix ci) and (guix import crate) to use ‘define-json-mapping’ to automatically map JSON dictionaries to records. Feedback welcome! Thanks, Ludo’. Ludovic Courtès (4): Add (guix json). ci: Use (guix json) and adjust for Guile-JSON 3.x. import: create: Separate crates.io API from actual conversion. import: crate: Correct interpretation of dual-licensing strings. Makefile.am | 1 + guix/build-system/cargo.scm | 11 ++- guix/ci.scm | 68 ++++++--------- guix/import/crate.scm | 161 ++++++++++++++++++++++++++---------- guix/json.scm | 62 ++++++++++++++ guix/swh.scm | 35 +------- tests/crate.scm | 13 ++- 7 files changed, 229 insertions(+), 122 deletions(-) create mode 100644 guix/json.scm -- 2.23.0 From unknown Fri Jun 20 07:21:54 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#37254] [PATCH 1/4] Add (guix json). References: <20190901144647.15185-1-ludo@gnu.org> In-Reply-To: <20190901144647.15185-1-ludo@gnu.org> Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 01 Sep 2019 14:57:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 37254 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 37254@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 37254-submit@debbugs.gnu.org id=B37254.156734979032269 (code B ref 37254); Sun, 01 Sep 2019 14:57:01 +0000 Received: (at 37254) by debbugs.gnu.org; 1 Sep 2019 14:56:30 +0000 Received: from localhost ([127.0.0.1]:57045 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4RHK-0008OP-3C for submit@debbugs.gnu.org; Sun, 01 Sep 2019 10:56:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34961) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4RHI-0008OA-U5 for 37254@debbugs.gnu.org; Sun, 01 Sep 2019 10:56:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48567) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i4RHB-0004ZA-FA; Sun, 01 Sep 2019 10:56:22 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40532 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i4RH9-0002la-SQ; Sun, 01 Sep 2019 10:56:21 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Sun, 1 Sep 2019 16:56:00 +0200 Message-Id: <20190901145603.15515-1-ludo@gnu.org> X-Mailer: git-send-email 2.23.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/swh.scm (define-json-reader, define-json-mapping): Move to... * guix/json.scm: ... here. New file. * Makefile.am (MODULES): Add it. --- Makefile.am | 1 + guix/json.scm | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++ guix/swh.scm | 35 +---------------------------- 3 files changed, 64 insertions(+), 34 deletions(-) create mode 100644 guix/json.scm diff --git a/Makefile.am b/Makefile.am index fa6bf8fe80..7b96c9473c 100644 --- a/Makefile.am +++ b/Makefile.am @@ -68,6 +68,7 @@ MODULES = \ guix/cpio.scm \ guix/deprecation.scm \ guix/docker.scm \ + guix/json.scm \ guix/records.scm \ guix/pki.scm \ guix/progress.scm \ diff --git a/guix/json.scm b/guix/json.scm new file mode 100644 index 0000000000..20f0bd8f13 --- /dev/null +++ b/guix/json.scm @@ -0,0 +1,62 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2018, 2019 Ludovic Courtès +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix json) + #:use-module (json) + #:use-module (srfi srfi-9) + #:export (define-json-mapping)) + +;;; Commentary: +;;; +;;; Helpers to map JSON objects to SRFI-9 records. Taken from (guix swh). +;;; +;;; Code: + +(define-syntax-rule (define-json-reader json->record ctor spec ...) + "Define JSON->RECORD as a procedure that converts a JSON representation, +read from a port, string, or hash table, into a record created by CTOR and +following SPEC, a series of field specifications." + (define (json->record input) + (let ((table (cond ((port? input) + (json->scm input)) + ((string? input) + (json-string->scm input)) + ((or (null? input) (pair? input)) + input)))) + (let-syntax ((extract-field (syntax-rules () + ((_ table (field key json->value)) + (json->value (assoc-ref table key))) + ((_ table (field key)) + (assoc-ref table key)) + ((_ table (field)) + (assoc-ref table + (symbol->string 'field)))))) + (ctor (extract-field table spec) ...))))) + +(define-syntax-rule (define-json-mapping rtd ctor pred json->record + (field getter spec ...) ...) + "Define RTD as a record type with the given FIELDs and GETTERs, à la SRFI-9, +and define JSON->RECORD as a conversion from JSON to a record of this type." + (begin + (define-record-type rtd + (ctor field ...) + pred + (field getter) ...) + + (define-json-reader json->record ctor + (field spec ...) ...))) diff --git a/guix/swh.scm b/guix/swh.scm index c253e217da..1e8927128c 100644 --- a/guix/swh.scm +++ b/guix/swh.scm @@ -20,6 +20,7 @@ #:use-module (guix base16) #:use-module (guix build utils) #:use-module ((guix build syscalls) #:select (mkdtemp!)) + #:use-module (guix json) #:use-module (web client) #:use-module (web response) #:use-module (json) @@ -129,40 +130,6 @@ url (string-append url "/"))) -(define-syntax-rule (define-json-reader json->record ctor spec ...) - "Define JSON->RECORD as a procedure that converts a JSON representation, -read from a port, string, or hash table, into a record created by CTOR and -following SPEC, a series of field specifications." - (define (json->record input) - (let ((table (cond ((port? input) - (json->scm input)) - ((string? input) - (json-string->scm input)) - ((or (null? input) (pair? input)) - input)))) - (let-syntax ((extract-field (syntax-rules () - ((_ table (field key json->value)) - (json->value (assoc-ref table key))) - ((_ table (field key)) - (assoc-ref table key)) - ((_ table (field)) - (assoc-ref table - (symbol->string 'field)))))) - (ctor (extract-field table spec) ...))))) - -(define-syntax-rule (define-json-mapping rtd ctor pred json->record - (field getter spec ...) ...) - "Define RTD as a record type with the given FIELDs and GETTERs, à la SRFI-9, -and define JSON->RECORD as a conversion from JSON to a record of this type." - (begin - (define-record-type rtd - (ctor field ...) - pred - (field getter) ...) - - (define-json-reader json->record ctor - (field spec ...) ...))) - (define %date-regexp ;; Match strings like "2014-11-17T22:09:38+01:00" or ;; "2018-09-30T23:20:07.815449+00:00"". -- 2.23.0 From unknown Fri Jun 20 07:21:54 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#37254] [PATCH 2/4] ci: Use (guix json) and adjust for Guile-JSON 3.x. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 01 Sep 2019 14:57:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 37254 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 37254@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 37254-submit@debbugs.gnu.org id=B37254.156734979332289 (code B ref 37254); Sun, 01 Sep 2019 14:57:02 +0000 Received: (at 37254) by debbugs.gnu.org; 1 Sep 2019 14:56:33 +0000 Received: from localhost ([127.0.0.1]:57049 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4RHM-0008Od-GA for submit@debbugs.gnu.org; Sun, 01 Sep 2019 10:56:32 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34966) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4RHK-0008OD-Hl for 37254@debbugs.gnu.org; Sun, 01 Sep 2019 10:56:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48568) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i4RHF-0004b2-Bg; Sun, 01 Sep 2019 10:56:25 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40532 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i4RHB-0002la-RN; Sun, 01 Sep 2019 10:56:23 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Sun, 1 Sep 2019 16:56:01 +0200 Message-Id: <20190901145603.15515-2-ludo@gnu.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190901145603.15515-1-ludo@gnu.org> References: <20190901145603.15515-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) This is in part a followup to 81c3dc32244a17241d74eea9fa265edfcb326f6d. * guix/ci.scm (, , ): Define using 'define-json-mapping'. (json->build, json->checkout, json->evaluation): Remove. (queued-builds, latest-builds, latest-evaluations): Pass JSON arrays through 'vector->list' to adjust for Guile-JSON 3.x. (evaluations-for-commit): Fix typo to really export. --- guix/ci.scm | 68 +++++++++++++++++++++-------------------------------- 1 file changed, 27 insertions(+), 41 deletions(-) diff --git a/guix/ci.scm b/guix/ci.scm index 1727297dd7..9e21996023 100644 --- a/guix/ci.scm +++ b/guix/ci.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2018 Ludovic Courtès +;;; Copyright © 2018, 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -18,9 +18,10 @@ (define-module (guix ci) #:use-module (guix http-client) - #:autoload (json parser) (json->scm) + #:use-module (guix json) + #:use-module (json) #:use-module (srfi srfi-1) - #:use-module (srfi srfi-9) + #:use-module (ice-9 match) #:export (build? build-id build-derivation @@ -42,7 +43,7 @@ queued-builds latest-builds latest-evaluations - evaluation-for-commit)) + evaluations-for-commit)) ;;; Commentary: ;;; @@ -51,28 +52,31 @@ ;;; ;;; Code: -(define-record-type - (make-build id derivation system status timestamp) - build? - (id build-id) ;integer +(define-json-mapping make-build build? + json->build + (id build-id "id") ;integer (derivation build-derivation) ;string | #f (system build-system) ;string - (status build-status) ;integer + (status build-status "buildstatus" ) ;integer (timestamp build-timestamp)) ;integer -(define-record-type - (make-checkout commit input) - checkout? +(define-json-mapping make-checkout checkout? + json->checkout (commit checkout-commit) ;string (SHA1) (input checkout-input)) ;string (name) -(define-record-type - (make-evaluation id spec complete? checkouts) - evaluation? +(define-json-mapping make-evaluation evaluation? + json->evaluation (id evaluation-id) ;integer (spec evaluation-spec) ;string - (complete? evaluation-complete?) ;Boolean - (checkouts evaluation-checkouts)) ;* + (complete? evaluation-complete? "in-progress" + (match-lambda + (0 #t) + (_ #f))) ;Boolean + (checkouts evaluation-checkouts "checkouts" ;* + (lambda (checkouts) + (map json->checkout + (vector->list checkouts))))) (define %query-limit ;; Max number of builds requested in queries. @@ -84,18 +88,11 @@ (close-port port) json)) -(define (json->build json) - (make-build (hash-ref json "id") - (hash-ref json "derivation") - (hash-ref json "system") - (hash-ref json "buildstatus") - (hash-ref json "timestamp"))) - (define* (queued-builds url #:optional (limit %query-limit)) "Return the list of queued derivations on URL." (let ((queue (json-fetch (string-append url "/api/queue?nr=" (number->string limit))))) - (map json->build queue))) + (map json->build (vector->list queue)))) (define* (latest-builds url #:optional (limit %query-limit) #:key evaluation system) @@ -114,26 +111,15 @@ string such as \"x86_64-linux\"), restrict to builds for SYSTEM." (option "system" system))))) ;; Note: Hydra does not provide a "derivation" field for entries in ;; 'latestbuilds', but Cuirass does. - (map json->build latest))) - -(define (json->checkout json) - (make-checkout (hash-ref json "commit") - (hash-ref json "input"))) - -(define (json->evaluation json) - (make-evaluation (hash-ref json "id") - (hash-ref json "specification") - (case (hash-ref json "in-progress") - ((0) #t) - (else #f)) - (map json->checkout (hash-ref json "checkouts")))) + (map json->build (vector->list latest)))) (define* (latest-evaluations url #:optional (limit %query-limit)) "Return the latest evaluations performed by the CI server at URL." (map json->evaluation - (json->scm - (http-fetch (string-append url "/api/evaluations?nr=" - (number->string limit)))))) + (vector->list + (json->scm + (http-fetch (string-append url "/api/evaluations?nr=" + (number->string limit))))))) (define* (evaluations-for-commit url commit #:optional (limit %query-limit)) -- 2.23.0 From unknown Fri Jun 20 07:21:54 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#37254] [PATCH 3/4] import: create: Separate crates.io API from actual conversion. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 01 Sep 2019 14:57:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 37254 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 37254@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 37254-submit@debbugs.gnu.org id=B37254.156734979932308 (code B ref 37254); Sun, 01 Sep 2019 14:57:02 +0000 Received: (at 37254) by debbugs.gnu.org; 1 Sep 2019 14:56:39 +0000 Received: from localhost ([127.0.0.1]:57052 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4RHO-0008Ox-Vu for submit@debbugs.gnu.org; Sun, 01 Sep 2019 10:56:38 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34973) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4RHM-0008OF-7O for 37254@debbugs.gnu.org; Sun, 01 Sep 2019 10:56:32 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48569) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i4RHH-0004bc-0O; Sun, 01 Sep 2019 10:56:27 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40532 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i4RHF-0002la-Nw; Sun, 01 Sep 2019 10:56:26 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Sun, 1 Sep 2019 16:56:02 +0200 Message-Id: <20190901145603.15515-3-ludo@gnu.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190901145603.15515-1-ludo@gnu.org> References: <20190901145603.15515-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) This provides a clean separation between bindings to the https://crates.io/api/v1 API and actual conversion to Guix package sexps. As a side-effect, it fixes things like "guix import blake2-rfc", "guix refresh -t crates", etc. * guix/import/crate.scm (, , ): New record types. (lookup-crate, crate-version-dependencies): New procedures. (crate-fetch): Remove. (crate->guix-package): Rewrite to use the new API. (latest-release): Likewise. * guix/build-system/cargo.scm (%crate-base-url): New variable. * tests/crate.scm (test-crate): Update accordingly. --- guix/build-system/cargo.scm | 11 ++- guix/import/crate.scm | 150 ++++++++++++++++++++++++++---------- tests/crate.scm | 13 +++- 3 files changed, 128 insertions(+), 46 deletions(-) diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm index 10a1bac844..1e8b3a578e 100644 --- a/guix/build-system/cargo.scm +++ b/guix/build-system/cargo.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2013, 2014, 2015, 2016 Ludovic Courtès +;;; Copyright © 2013, 2014, 2015, 2016, 2019 Ludovic Courtès ;;; Copyright © 2013 Andreas Enge ;;; Copyright © 2013 Nikita Karetnikov ;;; Copyright © 2016 David Craven @@ -35,12 +35,17 @@ #:export (%cargo-build-system-modules %cargo-utils-modules cargo-build-system + %crate-base-url crate-url crate-url? crate-uri)) -(define crate-url "https://crates.io/api/v1/crates/") -(define crate-url? (cut string-prefix? crate-url <>)) +(define %crate-base-url + (make-parameter "https://crates.io")) +(define crate-url + (string-append (%crate-base-url) "/api/v1/crates/")) +(define crate-url? + (cut string-prefix? crate-url <>)) (define (crate-uri name version) "Return a URI string for the crate package hosted at crates.io corresponding diff --git a/guix/import/crate.scm b/guix/import/crate.scm index 52c5cb1c30..bcd5068e6c 100644 --- a/guix/import/crate.scm +++ b/guix/import/crate.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2016 David Craven +;;; Copyright © 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -22,6 +23,7 @@ #:use-module ((guix download) #:prefix download:) #:use-module (gcrypt hash) #:use-module (guix http-client) + #:use-module (guix json) #:use-module (guix import json) #:use-module (guix import utils) #:use-module ((guix licenses) #:prefix license:) @@ -30,7 +32,6 @@ #:use-module (guix upstream) #:use-module (guix utils) #:use-module (ice-9 match) - #:use-module (ice-9 pretty-print) ; recursive #:use-module (json) #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) @@ -39,46 +40,82 @@ guix-package->crate-name %crate-updater)) -(define (crate-fetch crate-name callback) - "Fetch the metadata for CRATE-NAME from crates.io and call the callback." + +;;; +;;; Interface to https://crates.io/api/v1. +;;; - (define (crates->inputs crates) - (sort (map (cut assoc-ref <> "crate_id") crates) string-ci below. +(define-json-mapping make-crate crate? + json->crate + (name crate-name) ;string + (latest-version crate-latest-version "max_version") ;string + (home-page crate-home-page "homepage") ;string | #nil + (repository crate-repository) ;string + (description crate-description) ;string + (keywords crate-keywords ;list of strings + "keywords" vector->list) + (categories crate-categories ;list of strings + "categories" vector->list) + (versions crate-versions "actual_versions" ;list of + (lambda (vector) + (map json->crate-version + (vector->list vector)))) + (links crate-links)) ;alist - (define (string->license string) - (map spdx-string->license (string-split string #\/))) +;; Crate version. +(define-json-mapping make-crate-version crate-version? + json->crate-version + (id crate-version-id) ;integer + (number crate-version-number "num") ;string + (download-path crate-version-download-path "dl_path") ;string + (readme-path crate-version-readme-path "readme_path") ;string + (license crate-version-license "license") ;string + (links crate-version-links)) ;alist - (define (crate-kind-predicate kind) - (lambda (dep) (string=? (assoc-ref dep "kind") kind))) - - (and-let* ((crate-json (json-fetch (string-append crate-url crate-name))) - (crate (assoc-ref crate-json "crate")) - (name (assoc-ref crate "name")) - (version (assoc-ref crate "max_version")) - (homepage (assoc-ref crate "homepage")) - (repository (assoc-ref crate "repository")) - (synopsis (assoc-ref crate "description")) - (description (assoc-ref crate "description")) - (license (or (and=> (assoc-ref crate "license") - string->license) - '())) ;missing license info - (path (string-append "/" version "/dependencies")) - (deps-json (json-fetch (string-append crate-url name path))) - (deps (vector->list (assoc-ref deps-json "dependencies"))) - (dep-crates (filter (crate-kind-predicate "normal") deps)) - (dev-dep-crates - (filter (lambda (dep) - (not ((crate-kind-predicate "normal") dep))) deps)) - (cargo-inputs (crates->inputs dep-crates)) - (cargo-development-inputs (crates->inputs dev-dep-crates)) - (home-page (match homepage - (() repository) - (_ homepage)))) - (callback #:name name #:version version - #:cargo-inputs cargo-inputs - #:cargo-development-inputs cargo-development-inputs - #:home-page home-page #:synopsis synopsis - #:description description #:license license))) +;; Crate dependency. Each dependency (each edge in the graph) is annotated as +;; being a "normal" dependency or a development dependency. There also +;; information about the minimum required version, such as "^0.0.41". +(define-json-mapping make-crate-dependency + crate-dependency? + json->crate-dependency + (id crate-dependency-id "crate_id") ;string + (kind crate-dependency-kind "kind" ;'normal | 'dev + string->symbol) + (requirement crate-dependency-requirement "req")) ;string + +(define (lookup-crate name) + "Look up NAME on https://crates.io and return the corresopnding +record or #f if it was not found." + (let ((json (json-fetch (string-append (%crate-base-url) "/api/v1/crates/" + name)))) + (and=> (and json (assoc-ref json "crate")) + (lambda (alist) + ;; The "versions" field of ALIST is simply a list of version IDs + ;; (integers). Here, we squeeze in the actual version + ;; dictionaries that are not part of ALIST but are just more + ;; convenient handled this way. + (let ((versions (or (assoc-ref json "versions") '#()))) + (json->crate `(,@alist + ("actual_versions" . ,versions)))))))) + +(define (crate-version-dependencies version) + "Return the list of records of VERSION, a +." + (let* ((path (assoc-ref (crate-version-links version) "dependencies")) + (url (string-append (%crate-base-url) path))) + (match (assoc-ref (or (json-fetch url) '()) "dependencies") + ((? vector? vector) + (map json->crate-dependency (vector->list vector))) + (_ + '())))) + + +;;; +;;; Converting crates to Guix packages. +;;; (define (maybe-cargo-inputs package-names) (match (package-names->package-inputs package-names) @@ -141,7 +178,35 @@ and LICENSE." (define (crate->guix-package crate-name) "Fetch the metadata for CRATE-NAME from crates.io, and return the `package' s-expression corresponding to that package, or #f on failure." - (crate-fetch crate-name make-crate-sexp)) + (define (string->license string) + (map spdx-string->license (string-split string #\/))) + + (define (normal-dependency? dependency) + (eq? (crate-dependency-kind dependency) 'normal)) + + (let* ((crate (lookup-crate crate-name)) + (version (find (lambda (version) + (string=? (crate-version-number version) + (crate-latest-version crate))) + (crate-versions crate))) + (dependencies (crate-version-dependencies version)) + (dep-crates (filter normal-dependency? dependencies)) + (dev-dep-crates (remove normal-dependency? dependencies)) + (cargo-inputs (sort (map crate-dependency-id dep-crates) + string-ci (crate-version-license version) + string->license)))) (define (guix-package->crate-name package) "Return the crate name of PACKAGE." @@ -157,6 +222,7 @@ and LICENSE." (define (crate-name->package-name name) (string-append "rust-" (string-join (string-split name #\_) "-"))) + ;;; ;;; Updater ;;; @@ -175,9 +241,9 @@ and LICENSE." (define (latest-release package) "Return an for the latest release of PACKAGE." (let* ((crate-name (guix-package->crate-name package)) - (callback (lambda* (#:key version #:allow-other-keys) version)) - (version (crate-fetch crate-name callback)) - (url (crate-uri crate-name version))) + (crate (lookup-crate crate-name)) + (version (crate-latest-version crate)) + (url (crate-uri crate-name version))) (upstream-source (package (package-name package)) (version version) diff --git a/tests/crate.scm b/tests/crate.scm index 72c3a13350..8a232ba06c 100644 --- a/tests/crate.scm +++ b/tests/crate.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2014 David Thompson ;;; Copyright © 2016 David Craven +;;; Copyright © 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -32,10 +33,20 @@ \"crate\": { \"max_version\": \"1.0.0\", \"name\": \"foo\", - \"license\": \"MIT/Apache-2.0\", \"description\": \"summary\", \"homepage\": \"http://example.com\", \"repository\": \"http://example.com\", + \"keywords\": [\"dummy\" \"test\"], + \"categories\": [\"test\"] + \"actual_versions\": [ + { \"id\": \"foo\", + \"num\": \"1.0.0\", + \"license\": \"MIT/Apache-2.0\", + \"links\": { + \"dependencies\": \"/api/v1/crates/foo/1.0.0/dependencies\" + } + } + ] } }") -- 2.23.0 From unknown Fri Jun 20 07:21:54 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#37254] [PATCH 4/4] import: crate: Correct interpretation of dual-licensing strings. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 01 Sep 2019 14:57:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 37254 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 37254@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 37254-submit@debbugs.gnu.org id=B37254.156734979932314 (code B ref 37254); Sun, 01 Sep 2019 14:57:03 +0000 Received: (at 37254) by debbugs.gnu.org; 1 Sep 2019 14:56:39 +0000 Received: from localhost ([127.0.0.1]:57054 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4RHS-0008P4-WA for submit@debbugs.gnu.org; Sun, 01 Sep 2019 10:56:39 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34986) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4RHN-0008OH-4i for 37254@debbugs.gnu.org; Sun, 01 Sep 2019 10:56:33 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48571) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i4RHH-0004cU-Uw; Sun, 01 Sep 2019 10:56:27 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40532 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i4RHH-0002la-Cf; Sun, 01 Sep 2019 10:56:27 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Sun, 1 Sep 2019 16:56:03 +0200 Message-Id: <20190901145603.15515-4-ludo@gnu.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190901145603.15515-1-ludo@gnu.org> References: <20190901145603.15515-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * guix/import/crate.scm (%dual-license-rx): New variable. (crate->guix-package)[string->license]: Rewrite to match it. * tests/crate.scm (test-crate): Adjust "license" field to current practice. --- guix/import/crate.scm | 11 ++++++++++- tests/crate.scm | 2 +- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/guix/import/crate.scm b/guix/import/crate.scm index bcd5068e6c..a1cbf33361 100644 --- a/guix/import/crate.scm +++ b/guix/import/crate.scm @@ -32,6 +32,7 @@ #:use-module (guix upstream) #:use-module (guix utils) #:use-module (ice-9 match) + #:use-module (ice-9 regex) #:use-module (json) #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) @@ -175,11 +176,19 @@ and LICENSE." (close-port port) pkg)) +(define %dual-license-rx + ;; Dual licensing is represented by a string such as "MIT OR Apache-2.0". + ;; This regexp matches that. + (make-regexp "^(.*) OR (.*)$")) + (define (crate->guix-package crate-name) "Fetch the metadata for CRATE-NAME from crates.io, and return the `package' s-expression corresponding to that package, or #f on failure." (define (string->license string) - (map spdx-string->license (string-split string #\/))) + (match (regexp-exec %dual-license-rx string) + (#f (spdx-string->license string)) + (m (list (spdx-string->license (match:substring m 1)) + (spdx-string->license (match:substring m 2)))))) (define (normal-dependency? dependency) (eq? (crate-dependency-kind dependency) 'normal)) diff --git a/tests/crate.scm b/tests/crate.scm index 8a232ba06c..c14862ad9f 100644 --- a/tests/crate.scm +++ b/tests/crate.scm @@ -41,7 +41,7 @@ \"actual_versions\": [ { \"id\": \"foo\", \"num\": \"1.0.0\", - \"license\": \"MIT/Apache-2.0\", + \"license\": \"MIT OR Apache-2.0\", \"links\": { \"dependencies\": \"/api/v1/crates/foo/1.0.0/dependencies\" } -- 2.23.0 From unknown Fri Jun 20 07:21:54 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#37254] [PATCH 0/4] Refactor (guix ci) and (guix import crate) Resent-From: Efraim Flashner Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 01 Sep 2019 16:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 37254 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 37254@debbugs.gnu.org Received: via spool by 37254-submit@debbugs.gnu.org id=B37254.15673539246356 (code B ref 37254); Sun, 01 Sep 2019 16:06:02 +0000 Received: (at 37254) by debbugs.gnu.org; 1 Sep 2019 16:05:24 +0000 Received: from localhost ([127.0.0.1]:57091 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4SM0-0001eS-8A for submit@debbugs.gnu.org; Sun, 01 Sep 2019 12:05:24 -0400 Received: from flashner.co.il ([178.62.234.194]:46946) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4SLw-0001eD-Me for 37254@debbugs.gnu.org; Sun, 01 Sep 2019 12:05:22 -0400 Received: from localhost (unknown [188.120.128.191]) by flashner.co.il (Postfix) with ESMTPSA id C9EA1401CD; Sun, 1 Sep 2019 16:05:14 +0000 (UTC) Date: Sun, 1 Sep 2019 19:05:13 +0300 From: Efraim Flashner Message-ID: <20190901160513.GI13917@E5400> References: <20190901144647.15185-1-ludo@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="M9kwpIYUMbI/2cCx" Content-Disposition: inline In-Reply-To: <20190901144647.15185-1-ludo@gnu.org> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --M9kwpIYUMbI/2cCx Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable After my several (panicked :/) messages on IRC it turns out it's working pretty well. I'm able to run 'guix lint -c refresh' and 'guix refresh -t crate'. some errors I've come across, sometimes I get (license (list . #f)), like for rust-ppv-lite86, and sometimes I get (license (list . license:expat))), like with rust-schannel. On Sun, Sep 01, 2019 at 04:46:47PM +0200, Ludovic Court=C3=A8s wrote: > Hello Guix! >=20 > Initially I just wanted to fix =E2=80=9Cguix import crate blake2-rfc=E2= =80=9D, which > didn=E2=80=99t work as Efraim reported on IRC, but that led me to refactor > (guix ci) and (guix import crate) to use =E2=80=98define-json-mapping=E2= =80=99 to > automatically map JSON dictionaries to records. >=20 > Feedback welcome! >=20 > Thanks, > Ludo=E2=80=99. >=20 > Ludovic Court=C3=A8s (4): > Add (guix json). > ci: Use (guix json) and adjust for Guile-JSON 3.x. > import: create: Separate crates.io API from actual conversion. > import: crate: Correct interpretation of dual-licensing strings. >=20 > Makefile.am | 1 + > guix/build-system/cargo.scm | 11 ++- > guix/ci.scm | 68 ++++++--------- > guix/import/crate.scm | 161 ++++++++++++++++++++++++++---------- > guix/json.scm | 62 ++++++++++++++ > guix/swh.scm | 35 +------- > tests/crate.scm | 13 ++- > 7 files changed, 229 insertions(+), 122 deletions(-) > create mode 100644 guix/json.scm >=20 > --=20 > 2.23.0 >=20 >=20 >=20 >=20 --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --M9kwpIYUMbI/2cCx Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl1r7DUACgkQQarn3Mo9 g1FbChAAkFkXSgkTylmxoWlsBl89Qx19SiHoCMQmdt8+DYLbea81tEzGVhbuzIbB 8g9aDPj4A5bbnN7e9DVH/Bg2Nh9OplEgA7AzrswcP9zUTjY1F/BKUNVdFx7qm3hk bzpDlpFGXY5P1KfLYYxrIojHJ6VCjoj3Zko9POrdTRu7GgW2ObClgnDc9kWmRYQW 9m+dheFa8Z0c27Jh0c8QrsxL+ITbzZfgETC2vTAuC/p6a7kVlDcT689b+cba2jFD Wh6bKTI9A3VDUOKD8PjvJG38b5aFh6qJv1xvBjQX/cCHsuHkArOy9SqiKla8l4f+ MhXzh9RhR7Am/tuel40McJcSyaOMtNvZu9M56ch1kaytjDRkrIR7K3qWoJZ9P7TB TBVYTQvLt0helF9Q7I5QnOw+C0IoYRh35dJTelufcAJphbELsI/MVMe04yOnucSS wOEW+5m5SoIdEb82vlFwGTHCUf10AGiqY4P4KtSCqLNFXHmpP1xg+MuhyBOzsiXe +SoLqaKjhlS2yZT8n+06r7AaI/6JyR8//pO3hbJZynns6FHKZg9MkZF+Qqre8ZUX NCJNrxDt16D8BeBl8az3UGqL0BIjARt8mbr3qrpz8TvBOqeDk3Ux1pj4J5j814M1 K5arnbPYlcJRTcox8f3ixO138H8uEmjVAMFILYIm2YU7qiR1TNE= =Lpqm -----END PGP SIGNATURE----- --M9kwpIYUMbI/2cCx-- From unknown Fri Jun 20 07:21:54 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Subject: bug#37254: closed (Re: [bug#37254] [PATCH 0/4] Refactor (guix ci) and (guix import crate)) Message-ID: References: <87ftlc726c.fsf@inria.fr> <20190901144647.15185-1-ludo@gnu.org> X-Gnu-PR-Message: they-closed 37254 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 37254@debbugs.gnu.org Date: Wed, 04 Sep 2019 11:05:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1567595102-2158-1" This is a multi-part message in MIME format... ------------=_1567595102-2158-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #37254: [PATCH 0/4] Refactor (guix ci) and (guix import crate) which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 37254@debbugs.gnu.org. --=20 37254: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D37254 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1567595102-2158-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 37254-done) by debbugs.gnu.org; 4 Sep 2019 11:04:37 +0000 Received: from localhost ([127.0.0.1]:60894 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i5T5Z-0000Xk-1F for submit@debbugs.gnu.org; Wed, 04 Sep 2019 07:04:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53393) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i5T5X-0000XW-Pz for 37254-done@debbugs.gnu.org; Wed, 04 Sep 2019 07:04:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:42906) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i5T5S-00054E-LL; Wed, 04 Sep 2019 07:04:30 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=42418 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i5T5R-0007DT-9j; Wed, 04 Sep 2019 07:04:30 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Efraim Flashner Subject: Re: [bug#37254] [PATCH 0/4] Refactor (guix ci) and (guix import crate) References: <20190901144647.15185-1-ludo@gnu.org> <20190901160513.GI13917@E5400> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 15 Fructidor an 227 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 04 Sep 2019 13:04:27 +0200 Message-ID: <87ftlc726c.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 37254-done Cc: 37254-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, Efraim Flashner skribis: > After my several (panicked :/) messages on IRC it turns out it's working > pretty well. I'm able to run 'guix lint -c refresh' and 'guix refresh -t > crate'. There was room for improvement though: :-) --8<---------------cut here---------------start------------->8--- $ guix refresh -t crates Backtrace: 10 (primitive-load "/home/ludo/.config/guix/current/bin/gu=E2=80= =A6") In guix/ui.scm: 1692:12 9 (run-guix-command _ . _) In ice-9/boot-9.scm: 829:9 8 (catch _ _ # =E2=80=A6) 829:9 7 (catch _ _ # =E2=80=A6) In guix/store.scm: 623:10 6 (call-with-store _) 1803:24 5 (run-with-store # _ # _ =E2= =80=A6) In guix/scripts/refresh.scm: 533:14 4 (_ _) In srfi/srfi-1.scm: 640:9 3 (for-each # =E2=80=A6) In guix/scripts/refresh.scm: 344:2 2 (check-for-package-update # =E2=80=A6) In guix/import/crate.scm: 180:14 1 (latest-release #) In unknown file: 0 (string-append "https://crates.io/api/v1/crates/" "aut=E2=80= =A6" =E2=80=A6) ERROR: In procedure string-append: In procedure string-append: Wrong type (expecting string): #f $ guix import crate blake2-rfc guix import: error: failed to download meta-data for package 'blake2-rfc' $ guix describe=20 Generacio 101 Aug 26 2019 09:31:24 (nuna) guix a707484 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: a707484d64e7e46f8cb8401c660fbb6eb77ab9c6 --8<---------------cut here---------------end--------------->8--- This change fixes that. > some errors I've come across, sometimes I get (license (list . #f)), > like for rust-ppv-lite86, and sometimes I get (license (list . > license:expat))), like with rust-schannel. Oh there was still an issue with this, so I=E2=80=99ve fixed it. But note = that =E2=80=9Cguix import crate schannel=E2=80=9D doesn=E2=80=99t work on master. I=E2=80=99ve pushed the whole series now. Thanks for your feedback! Ludo=E2=80=99. ------------=_1567595102-2158-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 1 Sep 2019 14:46:57 +0000 Received: from localhost ([127.0.0.1]:57040 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4R85-0008As-3q for submit@debbugs.gnu.org; Sun, 01 Sep 2019 10:46:57 -0400 Received: from lists.gnu.org ([209.51.188.17]:38785) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i4R83-0008Al-B8 for submit@debbugs.gnu.org; Sun, 01 Sep 2019 10:46:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42430) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i4R81-0003yD-QN for guix-patches@gnu.org; Sun, 01 Sep 2019 10:46:55 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_20, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48505) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1i4R81-0000zH-9m; Sun, 01 Sep 2019 10:46:53 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40518 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1i4R80-00024B-S1; Sun, 01 Sep 2019 10:46:53 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/4] Refactor (guix ci) and (guix import crate) Date: Sun, 1 Sep 2019 16:46:47 +0200 Message-Id: <20190901144647.15185-1-ludo@gnu.org> X-Mailer: git-send-email 2.23.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Guix! Initially I just wanted to fix “guix import crate blake2-rfc”, which didn’t work as Efraim reported on IRC, but that led me to refactor (guix ci) and (guix import crate) to use ‘define-json-mapping’ to automatically map JSON dictionaries to records. Feedback welcome! Thanks, Ludo’. Ludovic Courtès (4): Add (guix json). ci: Use (guix json) and adjust for Guile-JSON 3.x. import: create: Separate crates.io API from actual conversion. import: crate: Correct interpretation of dual-licensing strings. Makefile.am | 1 + guix/build-system/cargo.scm | 11 ++- guix/ci.scm | 68 ++++++--------- guix/import/crate.scm | 161 ++++++++++++++++++++++++++---------- guix/json.scm | 62 ++++++++++++++ guix/swh.scm | 35 +------- tests/crate.scm | 13 ++- 7 files changed, 229 insertions(+), 122 deletions(-) create mode 100644 guix/json.scm -- 2.23.0 ------------=_1567595102-2158-1--