From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 27 11:58:45 2017 Received: (at submit) by debbugs.gnu.org; 27 Aug 2017 15:58:45 +0000 Received: from localhost ([127.0.0.1]:57998 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlzxU-00016u-TH for submit@debbugs.gnu.org; Sun, 27 Aug 2017 11:58:45 -0400 Received: from eggs.gnu.org ([208.118.235.92]:50770) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlzxT-00016h-AR for submit@debbugs.gnu.org; Sun, 27 Aug 2017 11:58:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dlzxM-0006El-Je for submit@debbugs.gnu.org; Sun, 27 Aug 2017 11:58:37 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:46165) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dlzxM-0006Eh-Go for submit@debbugs.gnu.org; Sun, 27 Aug 2017 11:58:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41741) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dlzxK-0006og-S9 for guix-patches@gnu.org; Sun, 27 Aug 2017 11:58:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dlzxH-0006Dd-HX for guix-patches@gnu.org; Sun, 27 Aug 2017 11:58:34 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21029) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dlzxH-0006DJ-9W for guix-patches@gnu.org; Sun, 27 Aug 2017 11:58:31 -0400 Received: from localhost (46.183.103.8 [46.183.103.8]) by mx.zohomail.com with SMTPS id 1503849505758519.0976110512945; Sun, 27 Aug 2017 08:58:25 -0700 (PDT) From: Ricardo Wurmus To: guix-patches@gnu.org Subject: [PATCH 0/3] Add generic JSON importer Date: Sun, 27 Aug 2017 17:58:20 +0200 Message-Id: <20170827155820.28812-1-rekado@elephly.net> X-Mailer: git-send-email 2.14.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ZohoMailClient: External X-ZohoMail: Z_26063301 SPT_1 Z_26062608 SPT_0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit Cc: Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) Hi Guix, this patch set adds a somewhat unusual importer. Assume we have a file "package.json" with the following contents: --8<---------------cut here---------------start------------->8--- { "name": "hello", "version": "2.10", "source": { "method": "url-fetch", "uri": "mirror://gnu/hello/hello-2.10.tar.gz", "sha256": { "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i" } } "build-system": "gnu", "home-page": "https://www.gnu.org/software/hello/", "synopsis": "Hello, GNU world: An example GNU package", "description": "It really works.", "license": "GPL-3.0+", "inputs": ["r-minimal@3", "ghc-pandoc", "samtools@0"] } --8<---------------cut here---------------end--------------->8--- Let’s run the new “json” importer on this file: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix import json package.json (package (name "hello") (version "2.10") (source (origin (uri (string-append "mirror://gnu/hello/hello-2.10.tar.gz")) (method url-fetch) (sha256 (base32 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")))) (build-system r-build-system) (inputs `(("r-minimal" ,(@ (gnu packages statistics) r-minimal)) ("ghc-pandoc" ,(@ (gnu packages haskell) ghc-pandoc)) ("samtools" ,(@ (gnu packages bioinformatics) samtools-0.1)))) (home-page "https://www.gnu.org/software/hello/") (synopsis "Hello, GNU world: An example GNU package") (description "It really works.") (license gpl3+)) --8<---------------cut here---------------end--------------->8--- What you don’t see here is that the JSON importer internally creates a package object, which could already be built (e.g. from within the REPL) — without having to write it to a file first and setting GUIX_PACKAGE_PATH. What is this good for? Users could create simple Guix packages for their own immature projects using a syntax that they may be more familiar with and generate a proper Scheme package definition from it to allow other people to install it. For more complicated packages they would, of course, be better served by using the usual Scheme syntax for package definitions. To make the importer behave like all other importers, we use the new “package->code” procedure, which takes a package and generates the code, which would create an equivalent package object when evaluated. There are some minor problems with “package->code”, which are marked with FIXME comments. We probably shouldn’t print out “(@ (gnu packages statistics) r-minimal)” and instead let “package->code” return two values: the package code and a list of modules needed to evaluate it. What do you think? Terrible? Exciting? Both? *raises hand* Documentation of this importer is missing because I’m not sure if this is the best way of doing this. Let’s discuss! ~~ Ricardo Ricardo Wurmus (3): packages: Add package->code. import: Add generic data to package converter. import: Add JSON importer. guix/import/utils.scm | 77 ++++++++++++++++++++++++- guix/packages.scm | 131 +++++++++++++++++++++++++++++++++++++++++++ guix/scripts/import.scm | 2 +- guix/scripts/import/json.scm | 101 +++++++++++++++++++++++++++++++++ 4 files changed, 309 insertions(+), 2 deletions(-) create mode 100644 guix/scripts/import/json.scm -- 2.14.1 From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 27 12:00:57 2017 Received: (at 28251) by debbugs.gnu.org; 27 Aug 2017 16:00:57 +0000 Received: from localhost ([127.0.0.1]:58003 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlzzb-0001Bb-Cc for submit@debbugs.gnu.org; Sun, 27 Aug 2017 12:00:57 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21124) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlzzZ-0001BS-5a for 28251@debbugs.gnu.org; Sun, 27 Aug 2017 12:00:53 -0400 Received: from localhost (46.183.103.8 [46.183.103.8]) by mx.zohomail.com with SMTPS id 1503849651202131.1570906600275; Sun, 27 Aug 2017 09:00:51 -0700 (PDT) From: Ricardo Wurmus To: 28251@debbugs.gnu.org Subject: [PATCH 1/3] packages: Add package->code. Date: Sun, 27 Aug 2017 18:00:44 +0200 Message-Id: <20170827160046.29049-1-rekado@elephly.net> X-Mailer: git-send-email 2.14.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ZohoMailClient: External X-ZohoMail: Z_26063301 SPT_1 Z_26062608 SPT_0 X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 28251 Cc: Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) * guix/packages.scm (package->code): New procedure. --- guix/packages.scm | 131 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) diff --git a/guix/packages.scm b/guix/packages.scm index f619d9b37..d25920010 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -4,6 +4,7 @@ ;;; Copyright © 2015 Eric Bavier ;;; Copyright © 2016 Alex Kost ;;; Copyright © 2017 Efraim Flashner +;;; Copyright © 2017 Ricardo Wurmus ;;; ;;; This file is part of GNU Guix. ;;; @@ -31,6 +32,7 @@ #:use-module (guix derivations) #:use-module (guix memoization) #:use-module (guix build-system) + #:use-module (guix licenses) #:use-module (guix search-paths) #:use-module (guix sets) #:use-module (ice-9 match) @@ -84,6 +86,7 @@ package-maintainers package-properties package-location + package->code hidden-package hidden-package? package-superseded @@ -306,6 +309,134 @@ name of its URI." package) 16))))) +;; FIXME: the quasiquoted arguments field may contain embedded package +;; objects, e.g. in #:disallowed-references; they will just be printed with +;; their usual # representation, not as variable names. +(define (package->code package) + "Return an S-expression representing the source code that produces PACKAGE +when evaluated." + ;; The module in which the package PKG is defined + (define (package-module-name pkg) + (map string->symbol + (string-split (string-drop-right + (location-file (package-location pkg)) 4) + #\/))) + + ;; Return the first candidate variable name that is bound to VAL. + ;; TODO: avoid '%pkg-config + (define (variable-name val mod) + (let ((candidates (filter identity + (module-map + (lambda (sym var) + (if (equal? val (variable-ref var)) sym #f)) + (resolve-interface mod))))) + (if (null? candidates) #f (car candidates)))) + + ;; Print either license variable name or the code for a license object + (define (print-license lic) + (let ((var (variable-name lic '(guix licenses)))) + (or var + `(license + (name ,(license-name lic)) + (uri ,(license-uri lic)) + (comment ,(license-comment lic)))))) + + (define (print-search-path-specification spec) + `(search-path-specification + (variable ,(search-path-specification-variable spec)) + (files (list ,@(search-path-specification-files spec))) + (separator ,(search-path-specification-separator spec)) + (file-type (quote ,(search-path-specification-file-type spec))) + (file-pattern ,(search-path-specification-file-pattern spec)))) + + (define (print-source source version) + ;; FIXME: we cannot use factorize-uri because (guix import utils) + ;; cannot be imported in this module. + (let ((factorize-uri (lambda (uri version) + (list uri)))) + (match source + (($ uri method sha256 file-name patches) + `(origin + (uri (string-append ,@(factorize-uri uri version))) + (method ,(procedure-name method)) + (sha256 + (base32 + ,(format #f "~a" (bytevector->nix-base32-string sha256)))) + ;; FIXME: in order to be able to throw away the directory prefix, + ;; we just assume that the patch files can be found with + ;; "search-patches". + ,@(let ((ps (force patches))) + (if (null? ps) '() + `((patches (search-patches ,@(map basename ps))))))))))) + + (define (print-package-lists lsts) + (list 'quasiquote + (map (match-lambda + ((label pkg) + (let ((mod (package-module-name pkg))) + (list label + ;; FIXME: using '@ certainly isn't pretty, but it + ;; avoids having to import the individual package + ;; modules. + (list 'unquote + (list '@ mod (variable-name pkg mod))))))) + lsts))) + + (match package + (($ name version source build-system + arguments inputs propagated-inputs native-inputs + self-native-input? + outputs + native-search-paths + search-paths + replacement + synopsis description license + home-page supported-systems maintainers + properties location) + `(package + (name ,name) + (version ,version) + (source ,(print-source source version)) + ,@(if (null? properties) '() + `((properties ,properties))) + ,@(let ((rep (replacement))) + (if rep + `((replacement ,rep)) + '())) + (build-system ,(symbol-append (build-system-name build-system) + '-build-system)) + ,@(let ((args (arguments))) + (if (null? args) '() + `((arguments ,(list 'quasiquote (arguments)))))) + ,@(if (equal? outputs '("out")) '() + `((outputs (list ,@outputs)))) + ,@(let ((pkgs (native-inputs))) + (if (null? pkgs) '() + `((native-inputs ,(print-package-lists pkgs))))) + ,@(let ((pkgs (inputs))) + (if (null? pkgs) '() + `((inputs ,(print-package-lists pkgs))))) + ,@(let ((pkgs (propagated-inputs))) + (if (null? pkgs) '() + `((propagated-inputs ,(print-package-lists pkgs))))) + ,@(if (lset= string=? supported-systems %supported-systems) + '() + `((supported-systems (list ,@supported-systems)))) + ,@(let ((paths (map print-search-path-specification native-search-paths))) + (if (null? paths) '() + `((native-search-paths + (list ,@paths))))) + ,@(let ((paths (map print-search-path-specification search-paths))) + (if (null? paths) '() + `((search-paths + (list ,@paths))))) + (home-page ,home-page) + (synopsis ,synopsis) + (description ,description) + (license ,(if (list? license) + `(list ,@(map print-license license)) + (print-license license))))))) + (define (package-upstream-name package) "Return the upstream name of PACKAGE, which could be different from the name it has in Guix." -- 2.14.1 From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 27 12:01:01 2017 Received: (at 28251) by debbugs.gnu.org; 27 Aug 2017 16:01:01 +0000 Received: from localhost ([127.0.0.1]:58006 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlzzg-0001Bw-Rx for submit@debbugs.gnu.org; Sun, 27 Aug 2017 12:01:01 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21039) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlzze-0001Bm-Cj for 28251@debbugs.gnu.org; Sun, 27 Aug 2017 12:00:58 -0400 Received: from localhost (46.183.103.8 [46.183.103.8]) by mx.zohomail.com with SMTPS id 150384965472276.2483916561032; Sun, 27 Aug 2017 09:00:54 -0700 (PDT) From: Ricardo Wurmus To: 28251@debbugs.gnu.org Subject: [PATCH 2/3] import: Add generic data to package converter. Date: Sun, 27 Aug 2017 18:00:45 +0200 Message-Id: <20170827160046.29049-2-rekado@elephly.net> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170827160046.29049-1-rekado@elephly.net> References: <20170827160046.29049-1-rekado@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ZohoMailClient: External X-ZohoMail: Z_26063301 SPT_1 Z_26062608 SPT_0 X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 28251 Cc: Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) * guix/import/utils.scm (build-system-modules, guix-modules): New variables. (lookup-build-system-by-name, specs->package-lists, convert-source, data->guix-package): New procedures. --- guix/import/utils.scm | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index be1980d08..edc6fda26 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2012, 2013 Ludovic Courtès ;;; Copyright © 2016 Jelle Licht ;;; Copyright © 2016 David Craven +;;; Copyright © 2017 Ricardo Wurmus ;;; ;;; This file is part of GNU Guix. ;;; @@ -25,6 +26,10 @@ #:use-module (guix http-client) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix utils) + #:use-module (guix packages) + #:use-module (guix discovery) + #:use-module (guix build-system) + #:use-module (gnu packages) #:use-module (ice-9 match) #:use-module (ice-9 regex) #:use-module (srfi srfi-1) @@ -45,7 +50,9 @@ license->symbol snake-case - beautify-description)) + beautify-description + + data->guix-package)) (define (factorize-uri uri version) "Factorize URI, a package tarball URI as a string, such that any occurrences @@ -241,3 +248,71 @@ package definition." (('package ('name (? string? name)) _ ...) `(define-public ,(string->symbol name) ,guix-package)))) + +(define build-system-modules + (all-modules (map (lambda (entry) + `(,entry . "guix/build-system")) + %load-path))) + +(define guix-modules + (all-modules (map (lambda (entry) + `(,entry . "guix")) + %load-path))) + +(define (lookup-build-system-by-name name) + (fold-module-public-variables (lambda (obj result) + (if (and (build-system? obj) + (eq? name (build-system-name obj))) + obj result)) + #f + build-system-modules)) + +(define (specs->package-lists specs) + (map (lambda (spec) + (let ((pkg (specification->package spec))) + (list (package-name pkg) pkg))) + specs)) + +(define (convert-source source) + (match source + ((? string? file) (local-file file)) + (#f #f) + (orig (let ((sha (match (car (assoc-ref orig "sha256")) + (("base32" . value) + (base32 value)) + (_ #f)))) + (origin + (method (match (assoc-ref orig "method") + ("url-fetch" (@ (guix download) url-fetch)) + ("git-fetch" (@ (guix git-download) git-fetch)) + ("svn-fetch" (@ (guix svn-download) svn-fetch)) + ("hg-fetch" (@ (guix hg-download) hg-fetch)) + (_ #f))) + (uri (assoc-ref orig "uri")) + (sha256 sha)))))) + +(define (data->guix-package meta) + (package + (name (assoc-ref meta "name")) + (version (assoc-ref meta "version")) + (source (convert-source (assoc-ref meta "source"))) + (build-system + (lookup-build-system-by-name + (string->symbol (assoc-ref meta "build-system")))) + (native-inputs + (specs->package-lists (or (assoc-ref meta "native-inputs") '()))) + (inputs + (specs->package-lists (or (assoc-ref meta "inputs") '()))) + (propagated-inputs + (specs->package-lists (or (assoc-ref meta "propagated-inputs") '()))) + (home-page + (assoc-ref meta "home-page")) + (synopsis + (assoc-ref meta "synopsis")) + (description + (assoc-ref meta "description")) + (license + (let ((l (assoc-ref meta "license"))) + (or (module-ref (resolve-interface '(guix licenses) #:prefix 'license:) + (spdx-string->license l)) + (fsdg-compatible l)))))) -- 2.14.1 From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 27 12:01:04 2017 Received: (at 28251) by debbugs.gnu.org; 27 Aug 2017 16:01:04 +0000 Received: from localhost ([127.0.0.1]:58011 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlzzk-0001Cn-6G for submit@debbugs.gnu.org; Sun, 27 Aug 2017 12:01:04 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21051) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlzzh-0001Bv-1X for 28251@debbugs.gnu.org; Sun, 27 Aug 2017 12:01:01 -0400 Received: from localhost (46.183.103.8 [46.183.103.8]) by mx.zohomail.com with SMTPS id 1503849659283823.5691822717615; Sun, 27 Aug 2017 09:00:59 -0700 (PDT) From: Ricardo Wurmus To: 28251@debbugs.gnu.org Subject: [PATCH 3/3] import: Add JSON importer. Date: Sun, 27 Aug 2017 18:00:46 +0200 Message-Id: <20170827160046.29049-3-rekado@elephly.net> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170827160046.29049-1-rekado@elephly.net> References: <20170827160046.29049-1-rekado@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ZohoMailClient: External X-ZohoMail: Z_26063301 SPT_1 Z_26062608 SPT_0 X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 28251 Cc: Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) * guix/scripts/import/json.scm: New file. * guix/scripts/import.scm (importers): Add json. --- guix/scripts/import.scm | 2 +- guix/scripts/import/json.scm | 101 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 102 insertions(+), 1 deletion(-) create mode 100644 guix/scripts/import/json.scm diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index 9bba074e8..67bc7a755 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -74,7 +74,7 @@ rather than \\n." ;;; (define importers '("gnu" "nix" "pypi" "cpan" "hackage" "stackage" "elpa" "gem" - "cran" "crate" "texlive")) + "cran" "crate" "texlive" "json")) (define (resolve-importer name) (let ((module (resolve-interface diff --git a/guix/scripts/import/json.scm b/guix/scripts/import/json.scm new file mode 100644 index 000000000..b459ef819 --- /dev/null +++ b/guix/scripts/import/json.scm @@ -0,0 +1,101 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014 Eric Bavier +;;; Copyright © 2015, 2017 Ricardo Wurmus +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts import json) + #:use-module (json) + #:use-module (guix ui) + #:use-module (guix utils) + #:use-module (guix scripts) + #:use-module (guix import utils) + #:use-module (guix scripts import) + #:use-module (guix packages) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-9 gnu) + #:use-module (srfi srfi-11) + #:use-module (srfi srfi-37) + #:use-module (srfi srfi-41) + #:use-module (ice-9 match) + #:use-module (ice-9 rdelim) + #:use-module (ice-9 format) + #:export (guix-import-json)) + + +;;; +;;; Command-line options. +;;; + +(define %default-options + '()) + +(define (show-help) + (display (G_ "Usage: guix import json PACKAGE-FILE +Import and convert the JSON package definition in PACKAGE-FILE.\n")) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + ;; Specification of the command-line options. + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix import json"))) + %standard-import-options)) + + +;;; +;;; Entry point. +;;; + +(define (guix-import-json . args) + (define (parse-options) + ;; Return the alist of option values. + (args-fold* args %options + (lambda (opt name arg result) + (leave (G_ "~A: unrecognized option~%") name)) + (lambda (arg result) + (alist-cons 'argument arg result)) + %default-options)) + + (let* ((opts (parse-options)) + (args (filter-map (match-lambda + (('argument . value) + value) + (_ #f)) + (reverse opts)))) + (match args + ((file-name) + (catch 'json-invalid + (lambda () + (let ((json (json-string->scm + (with-input-from-file file-name read-string)))) + ;; TODO: also print define-module boilerplate + (package->code (data->guix-package (hash-table->alist json))))) + (lambda () + (leave (G_ "invalid JSON in file '~a'~%") file-name)))) + (() + (leave (G_ "too few arguments~%"))) + ((many ...) + (leave (G_ "too many arguments~%")))))) -- 2.14.1 From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 28 08:27:30 2017 Received: (at 28251) by debbugs.gnu.org; 28 Aug 2017 12:27:30 +0000 Received: from localhost ([127.0.0.1]:58925 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dmJ8c-00039z-Hc for submit@debbugs.gnu.org; Mon, 28 Aug 2017 08:27:30 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21075) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dmJ8b-00039q-62 for 28251@debbugs.gnu.org; Mon, 28 Aug 2017 08:27:29 -0400 Received: from localhost (141.80.246.138 [141.80.246.138]) by mx.zohomail.com with SMTPS id 1503923244674341.6584728730435; Mon, 28 Aug 2017 05:27:24 -0700 (PDT) References: <20170827155820.28812-1-rekado@elephly.net> User-agent: mu4e 0.9.18; emacs 25.2.1 From: Ricardo Wurmus To: 28251@debbugs.gnu.org Subject: Re: [PATCH 0/3] Add generic JSON importer In-reply-to: <20170827155820.28812-1-rekado@elephly.net> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Mon, 28 Aug 2017 14:27:22 +0200 Message-ID: <87lgm3ho7p.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ZohoMailClient: External X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 28251 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) I have since made a couple of minor changes like adding a missing (guix gexp) import, adding the new script to the MODULES in Makefile.am, and adding a simple test for “data->guix-package”. I also have a small patch to “guix build -f”; given a file ending on “.json” it will parse the JSON and run “data->guix-package”. I have confirmed that this works for the “hello” package in JSON format. If this looks like a good idea I’ll convert the recursive CRAN importer to produce package objects; as a next step I’d extend “guix build” to accept a “--via=IMPORTER” option, which would cause a package to be imported and then built without requiring the manual work of writing or generating a package definition in Scheme. (Optionally, it could generate the code that could be contributed to Guix.) This is not meant to ever replace Scheme package definitions, but I think it can make importers useful to users, not only developers. -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 01 11:35:58 2017 Received: (at 28251) by debbugs.gnu.org; 1 Sep 2017 15:35:58 +0000 Received: from localhost ([127.0.0.1]:40383 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnnzC-00080Q-Bc for submit@debbugs.gnu.org; Fri, 01 Sep 2017 11:35:58 -0400 Received: from eggs.gnu.org ([208.118.235.92]:59674) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnnzB-00080E-6C for 28251@debbugs.gnu.org; Fri, 01 Sep 2017 11:35:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dnnz2-0000HV-Rx for 28251@debbugs.gnu.org; Fri, 01 Sep 2017 11:35:52 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41065) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dnnz2-0000H3-Kr; Fri, 01 Sep 2017 11:35:48 -0400 Received: from [193.50.110.184] (port=38920 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dnnz2-0007tL-7x; Fri, 01 Sep 2017 11:35:48 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Ricardo Wurmus Subject: Re: [bug#28251] [PATCH 0/3] Add generic JSON importer References: <20170827155820.28812-1-rekado@elephly.net> Date: Fri, 01 Sep 2017 17:35:46 +0200 In-Reply-To: <20170827155820.28812-1-rekado@elephly.net> (Ricardo Wurmus's message of "Sun, 27 Aug 2017 17:58:20 +0200") Message-ID: <87bmmu5t4d.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28251 Cc: 28251@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hi Ricardo! Ricardo Wurmus skribis: > this patch set adds a somewhat unusual importer. Assume we have a file > "package.json" with the following contents: > > { > "name": "hello", > "version": "2.10", > "source": { > "method": "url-fetch", > "uri": "mirror://gnu/hello/hello-2.10.tar.gz", > "sha256": { > "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i" > } > } > "build-system": "gnu", > "home-page": "https://www.gnu.org/software/hello/", > "synopsis": "Hello, GNU world: An example GNU package", > "description": "It really works.", > "license": "GPL-3.0+", > "inputs": ["r-minimal@3", "ghc-pandoc", "samtools@0"] > } Neat! I wonder if we could further simplify the =E2=80=9Csource=E2=80=9D part, li= ke allowing (maybe optionally) for: "source": "mirror://gnu=E2=80=A6" and then letting the importer download the thing and fill in the hash. Likewise for a Git checkout: "git": { "commit": "cabba9e"; "url": "=E2=80=A6" }; Thoughts? > What do you think? Terrible? Exciting? Both? *raises hand* Both! :-) As discussed at the GHM, I don=E2=80=99t fully measure that, but it can pro= bably help us reach out to more people. Thanks! Ludo=E2=80=99, who goes look at the patches. From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 01 11:55:42 2017 Received: (at 28251) by debbugs.gnu.org; 1 Sep 2017 15:55:42 +0000 Received: from localhost ([127.0.0.1]:40409 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnoII-0008RS-8k for submit@debbugs.gnu.org; Fri, 01 Sep 2017 11:55:42 -0400 Received: from eggs.gnu.org ([208.118.235.92]:38854) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dnoIG-0008RF-7T for 28251@debbugs.gnu.org; Fri, 01 Sep 2017 11:55:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dnoI6-0000Lc-0S for 28251@debbugs.gnu.org; Fri, 01 Sep 2017 11:55:34 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41360) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dnoI5-0000LY-TG; Fri, 01 Sep 2017 11:55:29 -0400 Received: from [193.50.110.184] (port=38926 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dnoI5-000782-Fy; Fri, 01 Sep 2017 11:55:29 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Ricardo Wurmus Subject: Re: [bug#28251] [PATCH 1/3] packages: Add package->code. References: <20170827155820.28812-1-rekado@elephly.net> <20170827160046.29049-1-rekado@elephly.net> Date: Fri, 01 Sep 2017 17:55:27 +0200 In-Reply-To: <20170827160046.29049-1-rekado@elephly.net> (Ricardo Wurmus's message of "Sun, 27 Aug 2017 18:00:44 +0200") Message-ID: <87ziae4dn4.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28251 Cc: 28251@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Ricardo Wurmus skribis: > * guix/packages.scm (package->code): New procedure. We=E2=80=99ll need tests for this. :-) I would move it to (guix import utils) or a new (guix import print) module or similar (which will also allow us to use =E2=80=98factorize-uri= =E2=80=99). In my mind, eventually all importers will produce a object directly, and so this will be a core part of the import machinery. Since it=E2=80=99s not a crucial component, I would prefer to have it out of (guix packages) though. > +;; FIXME: the quasiquoted arguments field may contain embedded package > +;; objects, e.g. in #:disallowed-references; they will just be printed w= ith > +;; their usual # representation, not as variable names. Not sure how to solve that; maybe we can ignore for now. > +(define (package->code package) > + "Return an S-expression representing the source code that produces PAC= KAGE > +when evaluated." Like you wrote, it would be nice to also return a spec of modules in scope, like: ((gnu packages r) ((guix licenses) #:prefix license:)) WDYT? That way, we can eventually change =E2=80=98guix import=E2=80=99 to systema= tically print both the =E2=80=98define-module=E2=80=99 clause and the package definition. > + ;; The module in which the package PKG is defined > + (define (package-module-name pkg) > + (map string->symbol > + (string-split (string-drop-right > + (location-file (package-location pkg)) 4) > + #\/))) > + > + ;; Return the first candidate variable name that is bound to VAL. > + ;; TODO: avoid '%pkg-config > + (define (variable-name val mod) > + (let ((candidates (filter identity > + (module-map > + (lambda (sym var) > + (if (equal? val (variable-ref var)) sym= #f)) > + (resolve-interface mod))))) > + (if (null? candidates) #f (car candidates)))) > + I think we should compare values with =E2=80=98eq?=E2=80=99 (usually we=E2= =80=99re concerned with pointer identity of records), and also use =E2=80=98module-for-each=E2= =80=99 + =E2=80=98let/ec=E2=80=99 to avoid building a list for nothing, like: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> (match (let/ec return (module-for-each (lambda (sym var) (if (eq? + (variable-ref var)) (return sym) #f)) the-scm-module)) ((? symbol? sym) sym) (_ #f)) $17 =3D + --8<---------------cut here---------------end--------------->8--- > + ;; Print either license variable name or the code for a license object > + (define (print-license lic) Nitpick: I=E2=80=99d rename all the =E2=80=98print-*=E2=80=99 procedures to= =E2=80=98*->code=E2=80=99. > + (match package > + (($ name version source build-system If we move this to (guix import =E2=80=A6), we can no longer match on the record, but that=E2=80=99s not necessarily a bad thing anyway. :-) > + ,@(let ((args (arguments))) > + (if (null? args) '() > + `((arguments ,(list 'quasiquote (arguments)))))) > + ,@(if (equal? outputs '("out")) '() > + `((outputs (list ,@outputs)))) > + ,@(let ((pkgs (native-inputs))) > + (if (null? pkgs) '() > + `((native-inputs ,(print-package-lists pkgs))))) > + ,@(let ((pkgs (inputs))) > + (if (null? pkgs) '() > + `((inputs ,(print-package-lists pkgs))))) > + ,@(let ((pkgs (propagated-inputs))) > + (if (null? pkgs) '() > + `((propagated-inputs ,(print-package-lists pkgs))))) =E2=80=98match=E2=80=99! :-) This looks pretty cool already! Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 04 09:05:17 2017 Received: (at 28251) by debbugs.gnu.org; 4 Sep 2017 13:05:17 +0000 Received: from localhost ([127.0.0.1]:47974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dor41-0000AE-Jk for submit@debbugs.gnu.org; Mon, 04 Sep 2017 09:05:17 -0400 Received: from eggs.gnu.org ([208.118.235.92]:55417) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dor3z-00009y-QN for 28251@debbugs.gnu.org; Mon, 04 Sep 2017 09:05:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dor2y-00022Q-Lt for 28251@debbugs.gnu.org; Mon, 04 Sep 2017 09:05:10 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56266) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dor2y-00022H-HI; Mon, 04 Sep 2017 09:04:12 -0400 Received: from [193.50.110.184] (port=33138 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dor2y-0008Dl-3D; Mon, 04 Sep 2017 09:04:12 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Ricardo Wurmus Subject: Re: [bug#28251] [PATCH 2/3] import: Add generic data to package converter. References: <20170827160046.29049-1-rekado@elephly.net> <20170827160046.29049-2-rekado@elephly.net> Date: Mon, 04 Sep 2017 15:04:10 +0200 In-Reply-To: <20170827160046.29049-2-rekado@elephly.net> (Ricardo Wurmus's message of "Sun, 27 Aug 2017 18:00:45 +0200") Message-ID: <87vaky399x.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28251 Cc: 28251@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Hello! Ricardo Wurmus skribis: > * guix/import/utils.scm (build-system-modules, guix-modules): New variabl= es. > (lookup-build-system-by-name, specs->package-lists, convert-source, > data->guix-package): New procedures. [...] > +(define build-system-modules > + (all-modules (map (lambda (entry) > + `(,entry . "guix/build-system")) > + %load-path))) > + > +(define guix-modules > + (all-modules (map (lambda (entry) > + `(,entry . "guix")) > + %load-path))) =E2=80=98all-modules=E2=80=99 causes a directory traversal, so it should no= t be called at the top level. The solution is to turn these two things in a promise or a thunk or probably an =E2=80=98mlambda=E2=80=99 thunk (depending on whe= ther they are expected to be called frequently.) > +(define (lookup-build-system-by-name name) > + (fold-module-public-variables (lambda (obj result) Docstring please. :-) > +(define (specs->package-lists specs) > + (map (lambda (spec) > + (let ((pkg (specification->package spec))) > + (list (package-name pkg) pkg))) > + specs)) This should probably use =E2=80=98specification->package+output=E2=80=99 so= that one can use specs like =E2=80=9Chwloc:lib=E2=80=9D. > +(define (convert-source source) Maybe =E2=80=98source-spec->object=E2=80=99? > + (match source > + ((? string? file) (local-file file)) > + (#f #f) > + (orig (let ((sha (match (car (assoc-ref orig "sha256")) > + (("base32" . value) > + (base32 value)) > + (_ #f)))) > + (origin > + (method (match (assoc-ref orig "method") > + ("url-fetch" (@ (guix download) url-fetch)) > + ("git-fetch" (@ (guix git-download) git-fetch)) > + ("svn-fetch" (@ (guix svn-download) svn-fetch)) > + ("hg-fetch" (@ (guix hg-download) hg-fetch)) > + (_ #f))) > + (uri (assoc-ref orig "uri")) > + (sha256 sha)))))) Though as discussed earlier, I=E2=80=99m unsure about exposing =E2=80=9Curl= -fetch=E2=80=9D and co. in the spec that people write. > +(define (data->guix-package meta) Maybe =E2=80=98alist->package=E2=80=99? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 04 09:05:57 2017 Received: (at 28251) by debbugs.gnu.org; 4 Sep 2017 13:05:57 +0000 Received: from localhost ([127.0.0.1]:47977 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dor4e-0000BF-VE for submit@debbugs.gnu.org; Mon, 04 Sep 2017 09:05:57 -0400 Received: from eggs.gnu.org ([208.118.235.92]:56246) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dor4d-0000Ax-9k for 28251@debbugs.gnu.org; Mon, 04 Sep 2017 09:05:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dor3P-0002Y1-CK for 28251@debbugs.gnu.org; Mon, 04 Sep 2017 09:05:50 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56299) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dor3P-0002Xo-8m; Mon, 04 Sep 2017 09:04:39 -0400 Received: from [193.50.110.184] (port=33140 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dor3O-0008FI-LI; Mon, 04 Sep 2017 09:04:39 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Ricardo Wurmus Subject: Re: [bug#28251] [PATCH 3/3] import: Add JSON importer. References: <20170827160046.29049-1-rekado@elephly.net> <20170827160046.29049-3-rekado@elephly.net> Date: Mon, 04 Sep 2017 15:04:37 +0200 In-Reply-To: <20170827160046.29049-3-rekado@elephly.net> (Ricardo Wurmus's message of "Sun, 27 Aug 2017 18:00:46 +0200") Message-ID: <87r2vm3996.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28251 Cc: 28251@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Ricardo Wurmus skribis: > * guix/scripts/import/json.scm: New file. > * guix/scripts/import.scm (importers): Add json. With a bit of doc, this looks all good! Ludo'. From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 28 07:20:12 2017 Received: (at 28251-done) by debbugs.gnu.org; 28 Sep 2017 11:20:12 +0000 Received: from localhost ([127.0.0.1]:35976 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dxWrT-0000c0-HJ for submit@debbugs.gnu.org; Thu, 28 Sep 2017 07:20:11 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21091) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dxWrR-0000bs-Iq for 28251-done@debbugs.gnu.org; Thu, 28 Sep 2017 07:20:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1506597564; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; l=2253; bh=pJ648D0/LTyn1EJdnjqAa2H2V1jzlJrbihYPJ3eohh0=; b=KYyoa1GckBOux7PDa2U8IWVoNDe4vcUFdHBPrxTR4hfugBfhYtvZl/X/Uh8yy1Gy I0tqr6kb+72gDiWXcYz0ucCFeTuFDanUFZy1qzo1o8z90feU96izVCMPm3IFlrlzkOC DUTsp75tGUjLB8tl/uIReCJqy9W/mbr3yjdRszBE= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 150659756474037.178227749173516; Thu, 28 Sep 2017 04:19:24 -0700 (PDT) Received: from localhost (141.80.247.133 [141.80.247.133]) by mx.zohomail.com with SMTPS id 150659756468741.718791634741365; Thu, 28 Sep 2017 04:19:24 -0700 (PDT) References: <20170827155820.28812-1-rekado@elephly.net> <20170827160046.29049-1-rekado@elephly.net> <87ziae4dn4.fsf@gnu.org> User-agent: mu4e 0.9.18; emacs 25.3.1 From: Ricardo Wurmus To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#28251] [PATCH 1/3] packages: Add package->code. In-reply-to: <87ziae4dn4.fsf@gnu.org> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Thu, 28 Sep 2017 13:19:21 +0200 Message-ID: <1425051478.20370.1506597564740.JavaMail.sas@[172.25.246.172]> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 28251-done Cc: 28251-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Ludovic Court=C3=A8s writes: > Ricardo Wurmus skribis: > >> * guix/packages.scm (package->code): New procedure. > > We=E2=80=99ll need tests for this. :-) I=E2=80=99ve added some simple tests to tests/print.scm and import-utils.sc= m. > I would move it to (guix import utils) or a new (guix import print) > module or similar (which will also allow us to use =E2=80=98factorize-uri= =E2=80=99). In > my mind, eventually all importers will produce a object > directly, and so this will be a core part of the import machinery. > Since it=E2=80=99s not a crucial component, I would prefer to have it out= of > (guix packages) though. Okay, done. >> +;; FIXME: the quasiquoted arguments field may contain embedded package >> +;; objects, e.g. in #:disallowed-references; they will just be printed = with >> +;; their usual # representation, not as variable names. > > Not sure how to solve that; maybe we can ignore for now. That=E2=80=99s why I originally experimented with overriding the printer. = For the purposes of a JSON importer, however, this really isn=E2=80=99t importa= nt. >> +(define (package->code package) >> + "Return an S-expression representing the source code that produces PA= CKAGE >> +when evaluated." > > Like you wrote, it would be nice to also return a spec of modules in > scope, like: > > ((gnu packages r) > ((guix licenses) #:prefix license:)) > > WDYT? I=E2=80=99ll leave this as a later improvement, but yes: I=E2=80=99ll add t= his at some point. > I think we should compare values with =E2=80=98eq?=E2=80=99 (usually we= =E2=80=99re concerned > with pointer identity of records), and also use =E2=80=98module-for-each= =E2=80=99 + > =E2=80=98let/ec=E2=80=99 to avoid building a list for nothing That=E2=80=99s good! > Nitpick: I=E2=80=99d rename all the =E2=80=98print-*=E2=80=99 procedures = to =E2=80=98*->code=E2=80=99. Done. I=E2=80=99ve implemented the other suggested changes and added some documentation. I=E2=80=99ll push it to master in a few minutes. Thanks for the comments! -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 28 07:23:48 2017 Received: (at 28251) by debbugs.gnu.org; 28 Sep 2017 11:23:48 +0000 Received: from localhost ([127.0.0.1]:35984 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dxWuy-0000ii-8l for submit@debbugs.gnu.org; Thu, 28 Sep 2017 07:23:48 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21127) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dxWuw-0000ia-PB for 28251@debbugs.gnu.org; Thu, 28 Sep 2017 07:23:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1506597823; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:In-reply-to:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; l=1045; bh=81ZSIhn4Aj41Y2FjrM6OGItiQHNh0FW2kkoCOwjJzYQ=; b=BXAmHoMpWIJ20In4B2TMAhuYGltVByTi9kTCBmXlC7Rn+mkwgMWOEjLC8tzC7TYd XX25j4jyaCFC6QmrHoQidw5idUCZqaJaZmYSU/VF5yyivBQh99KT6wrf/1xS4O06+4g HDC722r6YVu8jdfnUbtboP92Cl/1iIQyN4zuvNGA= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1506597823791885.2028992298087; Thu, 28 Sep 2017 04:23:43 -0700 (PDT) Received: from localhost (141.80.247.133 [141.80.247.133]) by mx.zohomail.com with SMTPS id 1506597823740693.9051637668471; Thu, 28 Sep 2017 04:23:43 -0700 (PDT) References: <20170827155820.28812-1-rekado@elephly.net> <87bmmu5t4d.fsf@gnu.org> User-agent: mu4e 0.9.18; emacs 25.3.1 From: Ricardo Wurmus To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#28251] [PATCH 0/3] Add generic JSON importer In-reply-to: <87bmmu5t4d.fsf@gnu.org> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Thu, 28 Sep 2017 13:23:40 +0200 Message-ID: <1033407664.20475.1506597823791.JavaMail.sas@[172.25.246.172]> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 28251 Cc: 28251@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Ludovic Court=C3=A8s writes: > I wonder if we could further simplify the =E2=80=9Csource=E2=80=9D part, = like allowing > (maybe optionally) for: > > "source": "mirror://gnu=E2=80=A6" > > and then letting the importer download the thing and fill in the hash. > Likewise for a Git checkout: > > "git": { "commit": "cabba9e"; "url": "=E2=80=A6" }; > > Thoughts? I have added support for a simple source string as in the first example. I left the support for a direct translation of origin expressions unchanged, because I=E2=80=99m not sure yet how to express this. So, right now we can do either "source": "mirror://gnu/hello/hello-2.10.tar.gz", or "source": { "method": "url-fetch", "uri": "mirror://gnu/hello/hello-2.10.tar.gz", "sha256": { "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i" } } I=E2=80=99m open to simplifying this further. What do other people think? -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 28 15:58:45 2017 Received: (at 28251) by debbugs.gnu.org; 28 Sep 2017 19:58:45 +0000 Received: from localhost ([127.0.0.1]:37145 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dxexJ-00048U-05 for submit@debbugs.gnu.org; Thu, 28 Sep 2017 15:58:45 -0400 Received: from eggs.gnu.org ([208.118.235.92]:52818) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dxexH-00048I-U5 for 28251@debbugs.gnu.org; Thu, 28 Sep 2017 15:58:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dxexB-0002ed-PK for 28251@debbugs.gnu.org; Thu, 28 Sep 2017 15:58:38 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50346) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dxexB-0002eX-Lo; Thu, 28 Sep 2017 15:58:37 -0400 Received: from [2a01:e0a:1d:7270:6a6c:dc17:fc02:cfda] (port=54720 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dxexB-0000Um-1p; Thu, 28 Sep 2017 15:58:37 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Ricardo Wurmus Subject: Re: [bug#28251] [PATCH 0/3] Add generic JSON importer References: <20170827155820.28812-1-rekado@elephly.net> <87bmmu5t4d.fsf@gnu.org> <1033407664.20475.1506597823791.JavaMail.sas@[172.25.246.172]> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 =?utf-8?Q?Vend=C3=A9miaire?= an 226 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 28 Sep 2017 21:58:34 +0200 In-Reply-To: <1033407664.20475.1506597823791.JavaMail.sas@[172.25.246.172]> (Ricardo Wurmus's message of "Thu\, 28 Sep 2017 13\:23\:40 +0200") Message-ID: <87efqqlhn9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 28251 Cc: 28251@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Ricardo Wurmus skribis: > Ludovic Court=C3=A8s writes: > >> I wonder if we could further simplify the =E2=80=9Csource=E2=80=9D part,= like allowing >> (maybe optionally) for: >> >> "source": "mirror://gnu=E2=80=A6" >> >> and then letting the importer download the thing and fill in the hash. >> Likewise for a Git checkout: >> >> "git": { "commit": "cabba9e"; "url": "=E2=80=A6" }; >> >> Thoughts? > > I have added support for a simple source string as in the first example. > I left the support for a direct translation of origin expressions > unchanged, because I=E2=80=99m not sure yet how to express this. > > So, right now we can do either > > "source": "mirror://gnu/hello/hello-2.10.tar.gz", > > or > > "source": { > "method": "url-fetch", > "uri": "mirror://gnu/hello/hello-2.10.tar.gz", > "sha256": { > "base32": "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i" > } > } > > I=E2=80=99m open to simplifying this further. What do other people think? That sounds good to me. One important consideration is to have a format that is future-proof, which is probably the case here. We could perhaps add a =E2=80=9Cformat-version=E2=80=9D field in the JSON input to be on the safe = side, though it would defeat the whole simplification. (In other news, I=E2=80=99ve been considering the implementation of an =E2=80=9Cupstream=E2=80=9D importer, where =E2=80=9Cguix import upstream UR= L=E2=80=9D would return a package template as complete as possible.) Thanks, Ludo=E2=80=99. From unknown Wed Aug 20 00:02:44 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 27 Oct 2017 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator