From unknown Thu Aug 14 21:52:45 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#62375 <62375@debbugs.gnu.org> To: bug#62375 <62375@debbugs.gnu.org> Subject: Status: [PATCH 0/1] npm binary importer Reply-To: bug#62375 <62375@debbugs.gnu.org> Date: Fri, 15 Aug 2025 04:52:45 +0000 retitle 62375 [PATCH 0/1] npm binary importer reassign 62375 guix-patches submitter 62375 jlicht@fsfe.org severity 62375 normal tag 62375 patch moreinfo thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 22 07:25:20 2023 Received: (at submit) by debbugs.gnu.org; 22 Mar 2023 11:25:20 +0000 Received: from localhost ([127.0.0.1]:33651 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pewal-0004AY-Mk for submit@debbugs.gnu.org; Wed, 22 Mar 2023 07:25:20 -0400 Received: from lists.gnu.org ([209.51.188.17]:39482) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pewai-0004AN-NI for submit@debbugs.gnu.org; Wed, 22 Mar 2023 07:25:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pewag-00022c-Bj for guix-patches@gnu.org; Wed, 22 Mar 2023 07:25:14 -0400 Received: from mail1.fsfe.org ([2001:aa8:ffed:f5f3::151]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pewae-0006UW-1q; Wed, 22 Mar 2023 07:25:14 -0400 From: jlicht@fsfe.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1679484305; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=aHvktX4KrRbyjP0CtnfUSBovzYlZw7d91zc9mKc7tmQ=; b=FLOiOltT2xPz8JO/KAnflzwoMj0XJZ7VoxQh+MBMj7nw/TLrNNQXJ/O27uoGZYQlvibS/f AqO3sBZ9gkc5p2XSGyhb05/5s32iAM6AFG3nYskUnXLghlQA5W31illC+NCAAuRCffKxzr ACnZQn0Clg2u9zFmDpVDjDhPFhHkLsw= To: guix-patches@gnu.org Subject: [PATCH 0/1] npm binary importer Date: Wed, 22 Mar 2023 12:25:04 +0100 Message-Id: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=2001:aa8:ffed:f5f3::151; envelope-from=jlicht@fsfe.org; helo=mail1.fsfe.org X-Spam_score_int: -70 X-Spam_score: -7.1 X-Spam_bar: ------- X-Spam_report: (-7.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: dev@jpoiret.xyz, me@tobias.gr, zimon.toutoune@gmail.com, othacehe@gnu.org, ludo@gnu.org, mail@cbaines.net, rekado@elephly.net, Jelle Licht X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) From: Jelle Licht Folks, Here a revised patch to add the npm binary importer. To give some context, 'binary' here refers to the fact that this downloads archives straight from the npm registry at https://registry.npmjs.org. Some of these downloaded archives may not contain the original sources, so unless properly vetted the output of this importer does not allow one to easily generate package expressions for inclusion in guix's collection of packages. It should work as-is for most simple NPM packages. As noted in an inline comment somewhere, the way both npm and our very own node-build-system treats peer dependencies may require some manual intervention from time to time to hook in the right dependency at the right spot. The upside here is that when this happens, it's either trivial for a human to spot and fix, or wholly incompatible with our current approach so not fixable without writing a custom build system to deal with dependency cycles :-). Please test and review, let us get this merged as it might be useful to many people for building out their personal channels and/or package expressions. Special thanks to Timothy Sample and Lars-Dominik Braun for involvement in realising this, way too long ago. Jelle Licht (1): import: Add binary npm importer. Makefile.am | 2 + guix/import/npm-binary.scm | 269 +++++++++++++++++++++++++++++ guix/scripts/import.scm | 2 +- guix/scripts/import/npm-binary.scm | 113 ++++++++++++ 4 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 guix/import/npm-binary.scm create mode 100644 guix/scripts/import/npm-binary.scm -- 2.39.2 From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 22 07:27:24 2023 Received: (at 62375) by debbugs.gnu.org; 22 Mar 2023 11:27:24 +0000 Received: from localhost ([127.0.0.1]:33662 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pewcl-0004EO-1x for submit@debbugs.gnu.org; Wed, 22 Mar 2023 07:27:24 -0400 Received: from mail1.fsfe.org ([217.69.89.151]:60960) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pewcg-0004EC-M6 for 62375@debbugs.gnu.org; Wed, 22 Mar 2023 07:27:21 -0400 From: jlicht@fsfe.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1679484437; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=h9KQB9dGnO55Ur+khT2vGPB9YbpYuFP3tY1BOgTunuQ=; b=M+MQMrd0RbNvFreCoU3zE+WmC+swantH8PoViOvuSHdhQekn3jv8nnH5XEmBMlzvPH5Nde yxIvGCQNnaxPUj2sVbZKmVFj4SPEzK6xqhUMPNh+BXcGHp3fFRZKkvBkJZho5TlFcdRuDz L+zYQCryg7I80/7N6DncjWhYLyj6i6M= To: 62375@debbugs.gnu.org Subject: [PATCH] import: Add binary npm importer. Date: Wed, 22 Mar 2023 12:27:16 +0100 Message-Id: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 3.0 (+++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Jelle Licht * guix/scripts/import.scm: (importers): Add "npm-binary". * guix/import/npm-binary.scm: New file. * guix/scripts/import/npm-binary.scm: New file. * Makefile.am: Add them. Content analysis details: (3.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.0 MANY_TO_CC Sent to 10+ recipients 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Josselin Poiret , Tobias Geerinckx-Rice , Simon Tournier , Mathieu Othacehe , =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Christopher Baines , Lars-Dominik Braun , Ricardo Wurmus , Jelle Licht X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Jelle Licht * guix/scripts/import.scm: (importers): Add "npm-binary". * guix/import/npm-binary.scm: New file. * guix/scripts/import/npm-binary.scm: New file. * Makefile.am: Add them. Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.0 MANY_TO_CC Sent to 10+ recipients 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager From: Jelle Licht * guix/scripts/import.scm: (importers): Add "npm-binary". * guix/import/npm-binary.scm: New file. * guix/scripts/import/npm-binary.scm: New file. * Makefile.am: Add them. Co-authored-by: Timothy Sample Co-authored-by: Lars-Dominik Braun --- Makefile.am | 2 + guix/import/npm-binary.scm | 269 +++++++++++++++++++++++++++++ guix/scripts/import.scm | 2 +- guix/scripts/import/npm-binary.scm | 113 ++++++++++++ 4 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 guix/import/npm-binary.scm create mode 100644 guix/scripts/import/npm-binary.scm diff --git a/Makefile.am b/Makefile.am index 23b939b674..52def58ae2 100644 --- a/Makefile.am +++ b/Makefile.am @@ -288,6 +288,7 @@ MODULES = \ guix/import/kde.scm \ guix/import/launchpad.scm \ guix/import/minetest.scm \ + guix/import/npm-binary.scm \ guix/import/opam.scm \ guix/import/print.scm \ guix/import/pypi.scm \ @@ -339,6 +340,7 @@ MODULES = \ guix/scripts/import/hexpm.scm \ guix/scripts/import/json.scm \ guix/scripts/import/minetest.scm \ + guix/scripts/import/npm-binary.scm \ guix/scripts/import/opam.scm \ guix/scripts/import/pypi.scm \ guix/scripts/import/stackage.scm \ diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm new file mode 100644 index 0000000000..f9b54263e4 --- /dev/null +++ b/guix/import/npm-binary.scm @@ -0,0 +1,269 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019, 2020 Timothy Sample +;;; Copyright © 2020, 2023 Jelle Licht +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix import npm-binary) + #:use-module (guix import json) + #:use-module (guix import utils) + #:use-module (guix memoization) + #:use-module ((gnu services configuration) #:select (alist?)) + #:use-module (guix utils) + #:use-module (gnu packages) + #:use-module (ice-9 match) + #:use-module (ice-9 regex) + #:use-module (ice-9 receive) + #:use-module (json) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-9) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-41) + #:use-module (web client) + #:use-module (web response) + #:use-module (web uri) + #:export (npm-binary-recursive-import + npm-binary->guix-package + package-json->guix-package + make-versioned-package + name+version->symbol)) + +;; Autoload Guile-Semver so we only have a soft dependency. +(module-autoload! (current-module) + '(semver) '(string->semver semver? semver->string semver=? semver>?)) +(module-autoload! (current-module) + '(semver ranges) '(*semver-range-any* string->semver-range semver-range-contains?)) + +;; Dist-tags +(define-json-mapping make-dist-tags dist-tags? + json->dist-tags + (latest dist-tags-latest "latest" string->semver)) + +(define-record-type + (make-versioned-package name version) + versioned-package? + (name versioned-package-name) ;string + (version versioned-package-version)) ;string + +(define (dependencies->versioned-packages entries) + (match entries + (((names . versions) ...) + (map make-versioned-package names versions)) + (_ '()))) + +(define (extract-license license-string) + (if (unspecified? license-string) + 'unspecified! + (spdx-string->license license-string))) + +(define-json-mapping make-dist dist? + json->dist + (tarball dist-tarball)) + +(define (empty-or-string s) + (if (string? s) s "")) + +(define-json-mapping make-package-revision package-revision? + json->package-revision + (name package-revision-name) + (version package-revision-version "version" string->semver) ;semver + (home-page package-revision-home-page "homepage") ;string + (dependencies package-revision-dependencies "dependencies" ;list of versioned-package + dependencies->versioned-packages) + (dev-dependencies package-revision-dev-dependencies ;list of versioned-package + "devDependencies" dependencies->versioned-packages) + (peer-dependencies package-revision-peer-dependencies ;list of versioned-package + "peerDependencies" dependencies->versioned-packages) + (license package-revision-license "license" ;license | #f + (match-lambda + ((? unspecified?) #f) + ((? string? str) (spdx-string->license str)) + ((? alist? alist) + (match (assoc "type" alist) + ((_ . (? string? type)) + (spdx-string->license type)) + (_ #f))))) + (description package-revision-description ; string + "description" empty-or-string) + (dist package-revision-dist "dist" json->dist)) ;dist + +(define (versions->package-revisions versions) + (match versions + (((version . package-spec) ...) + (map json->package-revision package-spec)) + (_ '()))) + +(define (versions->package-versions versions) + (match versions + (((version . package-spec) ...) + (map string->semver versions)) + (_ '()))) + +(define-json-mapping make-meta-package meta-package? + json->meta-package + (name meta-package-name) ;string + (description meta-package-description) ;string + (dist-tags meta-package-dist-tags "dist-tags" json->dist-tags) ;dist-tags + (revisions meta-package-revisions "versions" versions->package-revisions)) + +;; TODO: Support other registries +(define *registry* "https://registry.npmjs.org") +(define *default-page* "https://www.npmjs.com/package") + +(define (lookup-meta-package name) + (let ((json (json-fetch (string-append *registry* "/" (uri-encode name))))) + (and=> json json->meta-package))) + +(define lookup-meta-package* (memoize lookup-meta-package)) + +(define (http-error-code arglist) + (match arglist + (('http-error _ _ _ (code)) code) + (_ #f))) + +(define (meta-package-versions meta) + (map package-revision-version + (meta-package-revisions meta))) + +(define (meta-package-latest meta) + (and=> (meta-package-dist-tags meta) dist-tags-latest)) + +(define* (meta-package-package meta #:optional + (version (meta-package-latest meta))) + (match version + ((? semver?) (find (lambda (revision) + (semver=? version (package-revision-version revision))) + (meta-package-revisions meta))) + ((? string?) (meta-package-package meta (string->semver version))) + (_ #f))) + +(define* (semver-latest svs #:optional (svr *semver-range-any*)) + (find (cut semver-range-contains? svr <>) + (sort svs semver>?))) + +(define* (resolve-package name #:optional (svr *semver-range-any*)) + (let ((meta (lookup-meta-package* name))) + (and meta + (let* ((version (semver-latest (or (meta-package-versions meta) '()) svr)) + (pkg (meta-package-package meta version))) + pkg)))) + + +;;; +;;; Converting packages +;;; + +(define (hash-url url) + "Downloads the resource at URL and computes the base32 hash for it." + (call-with-temporary-output-file + (lambda (temp port) + (begin ((@ (guix import utils) url-fetch) url temp) + (guix-hash-url temp))))) + +(define (npm-name->name npm-name) + "Return a Guix package name for the npm package with name NPM-NAME." + (define (clean name) + (string-map (lambda (chr) (if (char=? chr #\/) #\- chr)) + (string-filter (negate (cut char=? <> #\@)) name))) + (guix-name "node-" (clean npm-name))) + +(define (name+version->symbol name version) + (string->symbol (string-append name "-" version))) + +(define (package-revision->symbol package) + (let* ((npm-name (package-revision-name package)) + (version (semver->string (package-revision-version package))) + (name (npm-name->name npm-name))) + (name+version->symbol name version))) + +(define (package-revision->input package) + "Return the `inputs' entry for PACKAGE." + (let* ((npm-name (package-revision-name package)) + (name (npm-name->name npm-name))) + `(,name + (,'unquote ,(package-revision->symbol package))))) + +(define (npm-package->package-sexp npm-package) + "Return the `package' s-expression for an NPM-PACKAGE." + (define (new-or-existing-inputs resolved-deps) + (map package-revision->input resolved-deps)) + + (match npm-package + (($ name version home-page dependencies dev-dependencies peer-dependencies license description dist) + (let* ((name (npm-name->name name)) + (url (dist-tarball dist)) + (home-page (if (string? home-page) + home-page + (string-append *default-page* "/" (uri-encode name)))) + (synopsis description) + (resolved-deps (map (match-lambda (($ name version) + (resolve-package name (string->semver-range version)))) (append dependencies peer-dependencies))) + (peer-names (map versioned-package-name peer-dependencies)) + ;; lset-difference for treating peer-dependencies as dependencies, which leads to dependency cycles. + ;; lset-union for treating them as (ignored) dev-dependencies, which leads to broken packages. + (dev-names (lset-union string= (map versioned-package-name dev-dependencies) peer-names)) + (extra-phases (match dev-names + (() '()) + ((dev-names ...) + `((add-after 'patch-dependencies 'delete-dev-dependencies + (lambda _ + (delete-dependencies '(,@(reverse dev-names)))))))))) + (values + `(package + (name ,name) + (version ,(semver->string (package-revision-version npm-package))) + (source (origin + (method url-fetch) + (uri ,url) + (sha256 (base32 ,(hash-url url))))) + (build-system node-build-system) + (arguments + '(#:tests? #f + #:phases (modify-phases %standard-phases + (delete 'build) + ,@extra-phases))) + ,@(match dependencies + (() '()) + ((dependencies ...) + `((inputs + (,'quasiquote ,(map package-revision->input resolved-deps)))))) + (home-page ,home-page) + (synopsis ,synopsis) + (description ,description) + (license ,license)) + (map (match-lambda (($ name version) + (list name (semver->string version)))) + resolved-deps)))) + (_ #f))) + + +;;; +;;; Interface +;;; + +(define npm-binary->guix-package + (lambda* (name #:key (version *semver-range-any*) #:allow-other-keys) + (let* ((svr (match version + ((? string?) (string->semver-range version)) + (_ version))) + (pkg (resolve-package name svr))) + (and=> pkg npm-package->package-sexp)))) + +(define* (npm-binary-recursive-import package-name #:key version) + (recursive-import package-name + #:repo->guix-package (memoize npm-binary->guix-package) + #:version version + #:guix-name npm-name->name)) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index f84a964a53..dccf6488b2 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -47,7 +47,7 @@ (define %standard-import-options '()) (define importers '("gnu" "pypi" "cpan" "hackage" "stackage" "egg" "elpa" "gem" "go" "cran" "crate" "texlive" "json" "opam" - "minetest" "elm" "hexpm")) + "minetest" "elm" "hexpm" "npm-binary")) (define (resolve-importer name) (let ((module (resolve-interface diff --git a/guix/scripts/import/npm-binary.scm b/guix/scripts/import/npm-binary.scm new file mode 100644 index 0000000000..825c43bbc3 --- /dev/null +++ b/guix/scripts/import/npm-binary.scm @@ -0,0 +1,113 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014 David Thompson +;;; Copyright © 2018 Ricardo Wurmus +;;; Copyright © 2019 Timothy Sample +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts import npm-binary) + #:use-module (guix ui) + #:use-module (guix utils) + #:use-module (guix scripts) + #:use-module (guix import npm-binary) + #:use-module (guix scripts import) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-11) + #:use-module (srfi srfi-37) + #:use-module (srfi srfi-41) + #:use-module (ice-9 match) + #:use-module (ice-9 format) + #:export (guix-import-npm-binary)) + + +;;; +;;; Command-line options. +;;; + +(define %default-options + '()) + +(define (show-help) + (display (G_ "Usage: guix import npm-binary PACKAGE-NAME [VERSION] +Import and convert the NPM package PACKAGE-NAME using the +`npm-build-system' (but without building the package from source).")) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -r, --recursive import packages recursively")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + ;; Specification of the command-line options. + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix import npm-binary"))) + (option '(#\r "recursive") #f #f + (lambda (opt name arg result) + (alist-cons 'recursive #t result))) + %standard-import-options)) + + +;;; +;;; Entry point. +;;; + +(define (guix-import-npm-binary . args) + (define (parse-options) + ;; Return the alist of option values. + (args-fold* args %options + (lambda (opt name arg result) + (leave (G_ "~A: unrecognized option~%") name)) + (lambda (arg result) + (alist-cons 'argument arg result)) + %default-options)) + + (let* ((opts (parse-options)) + (args (filter-map (match-lambda + (('argument . value) + value) + (_ #f)) + (reverse opts)))) + (let loop ((args args)) + (match args + ((package-name version) + (if (assoc-ref opts 'recursive) + ;; Recursive import + (map (match-lambda + ((and ('package ('name name) ('version version) . rest) pkg) + `(define-public ,(name+version->symbol name version) + ,pkg)) + (_ #f)) + (npm-binary-recursive-import package-name #:version version)) + ;; Single import + (let ((sexp (npm-binary->guix-package package-name #:version version))) + (unless sexp + (leave (G_ "failed to download meta-data for package '~a@~a'~%") + package-name version)) + sexp))) + ((package-name) + (loop (list package-name "*"))) + (() + (leave (G_ "too few arguments~%"))) + ((many ...) + (leave (G_ "too many arguments~%"))))))) -- 2.39.2 From debbugs-submit-bounces@debbugs.gnu.org Tue Mar 28 11:49:47 2023 Received: (at 62375) by debbugs.gnu.org; 28 Mar 2023 15:49:48 +0000 Received: from localhost ([127.0.0.1]:50545 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1phBZz-0004xb-Fv for submit@debbugs.gnu.org; Tue, 28 Mar 2023 11:49:47 -0400 Received: from eggs.gnu.org ([209.51.188.92]:60690) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1phBZv-0004xJ-DB for 62375@debbugs.gnu.org; Tue, 28 Mar 2023 11:49:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1phBZo-0000YO-5k; Tue, 28 Mar 2023 11:49:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=Y7DrnRzb/9ppGjPYdH6NEeWKbHZWIVoRlJrTBs/fx64=; b=PGZCHVKIfx490nGTK5MJ yJ7tnozcP8374GdjYaqdURLHjxU64jwOx6p/hFzfpejf4yZ8NbdTDJUNUZPtLzNKCl9jOw2hllNTW uHpB4kbjg5PlmQpBEiEQo4Nl32gxeXR4+3zZ99P5SlnUrCwv7IpypXd+SyVVs5WTOd3gPuszy4WHi 4H8t8ez2xcvHPc//SkEXlBpFZed1vjC9+vRcZ2g29GKG5ExUGuuzllbGZ/y849QX3NkhWz4arh50q bOQzHjMGGfsNvxmMgoRq0szFLtroGwRJJGZLqZZci+D/rghwJusDKC2zmPPtxZBur1g6joKujkv2/ +D8UWahn7S6vNA==; Received: from [193.50.110.81] (helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1phBZn-0000zv-Nn; Tue, 28 Mar 2023 11:49:35 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: jlicht@fsfe.org Subject: Re: [PATCH] import: Add binary npm importer. References: X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Octidi 8 Germinal an 231 de la =?utf-8?Q?R=C3=A9volu?= =?utf-8?Q?tion=2C?= jour de la Jonquille X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 28 Mar 2023 17:49:32 +0200 In-Reply-To: (jlicht@fsfe.org's message of "Wed, 22 Mar 2023 12:27:16 +0100") Message-ID: <87tty4g9kj.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Josselin Poiret , Tobias Geerinckx-Rice , Simon Tournier , Mathieu Othacehe , Christopher Baines , Lars-Dominik Braun , 62375@debbugs.gnu.org, Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello Jelle! jlicht@fsfe.org skribis: > From: Jelle Licht > > * guix/scripts/import.scm: (importers): Add "npm-binary". > * guix/import/npm-binary.scm: New file. > * guix/scripts/import/npm-binary.scm: New file. > * Makefile.am: Add them. > > Co-authored-by: Timothy Sample > Co-authored-by: Lars-Dominik Braun Woohoo! I think it=E2=80=99ll be useful to many, even if we know it=E2=80= =99s far from =E2=80=9Cideal=E2=80=9D by our standards. We=E2=80=99ll need doc under =E2=80=9CInvoking guix import=E2=80=9D and tes= ts in =E2=80=98tests/npm-binary.scm=E2=80=99, similar to what=E2=80=99s done for = the other importers. The doc should clarify why it=E2=80=99s called that way and what the limita= tions are. For tests, I=E2=80=99d recommend mocking the npmjs.org HTTP servers using =E2=80=98with-http-server=E2=80=99 as in =E2=80=98tests/cpan.scm=E2=80=99. Also, please add docstrings to top-level procedures. > +;; TODO: Support other registries > +(define *registry* "https://registry.npmjs.org") > +(define *default-page* "https://www.npmjs.com/package") The convention currently is more like =E2=80=98%registry=E2=80=99. > +(define (http-error-code arglist) > + (match arglist > + (('http-error _ _ _ (code)) code) > + (_ #f))) Unused. :-) > +(define (hash-url url) > + "Downloads the resource at URL and computes the base32 hash for it." > + (call-with-temporary-output-file > + (lambda (temp port) > + (begin ((@ (guix import utils) url-fetch) url temp) > + (guix-hash-url temp))))) Maybe something more like: (port-sha256 (http-fetch =E2=80=A6)) ? > +(define (npm-package->package-sexp npm-package) > + "Return the `package' s-expression for an NPM-PACKAGE." > + (define (new-or-existing-inputs resolved-deps) > + (map package-revision->input resolved-deps)) > + > + (match npm-package > + (($ name version home-page dependencies dev-depen= dencies peer-dependencies license description dist) Please use =E2=80=98match-record=E2=80=99 instead and keep lines below 80 c= hars. :-) > + (let* ((name (npm-name->name name)) > + (url (dist-tarball dist)) > + (home-page (if (string? home-page) > + home-page > + (string-append *default-page* "/" (uri-encode= name)))) > + (synopsis description) > + (resolved-deps (map (match-lambda (($ na= me version) > + (resolve-package name (st= ring->semver-range version)))) (append dependencies peer-dependencies))) Likewise. That=E2=80=99s it! Could you send an updated patch? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 06 16:18:35 2023 Received: (at control) by debbugs.gnu.org; 6 Apr 2023 20:18:35 +0000 Received: from localhost ([127.0.0.1]:55271 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pkW43-0003Mw-7w for submit@debbugs.gnu.org; Thu, 06 Apr 2023 16:18:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53058) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pkW42-0003Mf-FH for control@debbugs.gnu.org; Thu, 06 Apr 2023 16:18:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pkW3x-0005cO-80 for control@debbugs.gnu.org; Thu, 06 Apr 2023 16:18:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to: references; bh=1VBEO6oSohAy7O7hiaVi08o5keWi8VdZUaslGRzexgQ=; b=WzzKdh7S+rJOvz OZxXKP3t8WJDmKZye6ZGpZNfmaa2IWPPCd0DIUgALnm4GtwrydIO2skfMYXMI13zMAQIofRe2vAP/ MengpPeGASXyF+45YRrbZXDuUk1iDQGDaeJAT2PGAkDta4EHgPUCSXERrg9xgCkNERwjTJEVU/ztm OV+VWOK8HSM97WxDDHb3POrqSrAze2mZnqhRizMTssc931mrA/3a21V0VfK7zteVZ9h5VMBBGAGL7 5wW4NReHB3GR4UkU6Clp6rFtPv2C3L16GF6s0t/TIZhlbOLfxXH5nDITBpamu8wXWkaYNjel6cI4m berZ8ZS81bZ9OI2BElIA==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pkW3w-0007RT-OJ for control@debbugs.gnu.org; Thu, 06 Apr 2023 16:18:28 -0400 Date: Thu, 06 Apr 2023 22:18:27 +0200 Message-Id: <87lej4hii4.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #62375 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) tags 62375 + moreinfo quit From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 08 14:29:27 2023 Received: (at 62375) by debbugs.gnu.org; 8 Apr 2023 18:29:27 +0000 Received: from localhost ([127.0.0.1]:59503 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1plDJW-0006Cp-N7 for submit@debbugs.gnu.org; Sat, 08 Apr 2023 14:29:27 -0400 Received: from mail1.fsfe.org ([217.69.89.151]:51332) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1plDJV-0006Cg-5Z for 62375@debbugs.gnu.org; Sat, 08 Apr 2023 14:29:26 -0400 From: Jelle Licht DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1680978563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TCVCfpFki70rsR5o/v6/FB9+JLzLyFnbaWiw754CNKc=; b=dAN0LsUiF/bu5xmO6SR/VIInhnckndz5rQJ5dqMF8doFsGxoxqLMKnLd3kHrHaIPNoqslA SXwiq9vNXTkmunhvU7/Dn4Yqofy7ikiUcCd97UkeXzMx063GESOq4uMcgKQwF7hpuGGgJB Q8+ikX67sUelnFC5SLZfk6PYmzclM+k= To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [PATCH] import: Add binary npm importer. In-Reply-To: <87tty4g9kj.fsf@gnu.org> References: <87tty4g9kj.fsf@gnu.org> Date: Sat, 08 Apr 2023 20:29:18 +0200 Message-ID: <871qkurzwh.fsf@fsfe.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Josselin Poiret , Tobias Geerinckx-Rice , Simon Tournier , Mathieu Othacehe , Christopher Baines , Lars-Dominik Braun , 62375@debbugs.gnu.org, Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Ludovic Court=C3=A8s writes: > Hello Jelle! > > jlicht@fsfe.org skribis: > >> From: Jelle Licht >> >> * guix/scripts/import.scm: (importers): Add "npm-binary". >> * guix/import/npm-binary.scm: New file. >> * guix/scripts/import/npm-binary.scm: New file. >> * Makefile.am: Add them. >> >> Co-authored-by: Timothy Sample >> Co-authored-by: Lars-Dominik Braun > > Woohoo! I think it=E2=80=99ll be useful to many, even if we know it=E2= =80=99s far from > =E2=80=9Cideal=E2=80=9D by our standards. > > We=E2=80=99ll need doc under =E2=80=9CInvoking guix import=E2=80=9D and t= ests in > =E2=80=98tests/npm-binary.scm=E2=80=99, similar to what=E2=80=99s done fo= r the other importers. > The doc should clarify why it=E2=80=99s called that way and what the limi= tations > are. > > For tests, I=E2=80=99d recommend mocking the npmjs.org HTTP servers using > =E2=80=98with-http-server=E2=80=99 as in =E2=80=98tests/cpan.scm=E2=80=99. > > Also, please add docstrings to top-level procedures. > >> +;; TODO: Support other registries >> +(define *registry* "https://registry.npmjs.org") >> +(define *default-page* "https://www.npmjs.com/package") > > The convention currently is more like =E2=80=98%registry=E2=80=99. > >> +(define (http-error-code arglist) >> + (match arglist >> + (('http-error _ _ _ (code)) code) >> + (_ #f))) > > Unused. :-) > >> +(define (hash-url url) >> + "Downloads the resource at URL and computes the base32 hash for it." >> + (call-with-temporary-output-file >> + (lambda (temp port) >> + (begin ((@ (guix import utils) url-fetch) url temp) >> + (guix-hash-url temp))))) > > Maybe something more like: (port-sha256 (http-fetch =E2=80=A6)) ? > >> +(define (npm-package->package-sexp npm-package) >> + "Return the `package' s-expression for an NPM-PACKAGE." >> + (define (new-or-existing-inputs resolved-deps) >> + (map package-revision->input resolved-deps)) >> + >> + (match npm-package >> + (($ name version home-page dependencies dev-depe= ndencies peer-dependencies license description dist) > > Please use =E2=80=98match-record=E2=80=99 instead and keep lines below 80= chars. :-) The records generated by `define-json-mapping' through `define-record-type' seem to not work with `match-record'. I could define our very own `define-json-mapping*' that works /w `define-record-type*' instead (and through that, `match-record'), but I thought I'd ask if you had something else in mind first :). >> + (let* ((name (npm-name->name name)) >> + (url (dist-tarball dist)) >> + (home-page (if (string? home-page) >> + home-page >> + (string-append *default-page* "/" (uri-encod= e name)))) >> + (synopsis description) >> + (resolved-deps (map (match-lambda (($ n= ame version) >> + (resolve-package name (s= tring->semver-range version)))) (append dependencies peer-dependencies))) > > Likewise. > > That=E2=80=99s it! Could you send an updated patch? Will do, once I got some proper test cases set up. Thanks again for the review!=20 - Jelle From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 17 17:15:06 2023 Received: (at 62375) by debbugs.gnu.org; 17 Apr 2023 21:15:06 +0000 Received: from localhost ([127.0.0.1]:57068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1poWBm-0003Nj-E6 for submit@debbugs.gnu.org; Mon, 17 Apr 2023 17:15:06 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40818) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1poWBk-00036i-Hy for 62375@debbugs.gnu.org; Mon, 17 Apr 2023 17:15:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1poWBd-0002UI-H1; Mon, 17 Apr 2023 17:14:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=Stye2baZYiufUekKQaWvorfJmKpViG6ppg147FGH8BQ=; b=H+zIcu26YU8mRKviIP3r YHlhHnM7h5ZX1GGRwNBLVcGE9hho65Rki7ubJLxWc9gGituUOOw0khGcpbxMd6lqS9crgGYnOEIg1 zJkNYdsjxAgfu3zWOw+eFegdsgYENF/AX6Svy7/vanazYwAYUOZUsFZQu3w1yYPAL10M2qI7+3C6M dpoZamDp8vm2a38DIo/CyYS42kXui7wgEbdAKGe4ZwFPPJ1B20Cvsn136CzJMzMUEelU67qqDPuDc 4FhjgVEKHDpT+NG5BfkOFTO+H6amjPKxixTdOx3VnHsdFIrVLHgPDG30woOfgcIeDYi1visCWbxSF ho+x5Wko4mfJQA==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1poWBc-0003fD-QY; Mon, 17 Apr 2023 17:14:57 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Jelle Licht Subject: Re: bug#62375: [PATCH 0/1] npm binary importer References: <87tty4g9kj.fsf@gnu.org> <871qkurzwh.fsf@fsfe.org> Date: Mon, 17 Apr 2023 23:14:52 +0200 In-Reply-To: <871qkurzwh.fsf@fsfe.org> (Jelle Licht's message of "Sat, 08 Apr 2023 20:29:18 +0200") Message-ID: <874jpecisz.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Josselin Poiret , Christopher Baines , Simon Tournier , Mathieu Othacehe , Tobias Geerinckx-Rice , Lars-Dominik Braun , 62375@debbugs.gnu.org, Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Jelle Licht skribis: > Ludovic Court=C3=A8s writes: [...] >>> + (match npm-package >>> + (($ name version home-page dependencies dev-dep= endencies peer-dependencies license description dist) >> >> Please use =E2=80=98match-record=E2=80=99 instead and keep lines below 8= 0 chars. :-) > > The records generated by `define-json-mapping' through > `define-record-type' seem to not work with `match-record'. Oh right, my bad. The other option is to call record accessors; it would avoid the risk associated with index-based matches but may be more verbose. Your call! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 18 17:03:51 2023 Received: (at 62375) by debbugs.gnu.org; 18 Jun 2023 21:03:51 +0000 Received: from localhost ([127.0.0.1]:55071 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qAzYs-0007it-Q4 for submit@debbugs.gnu.org; Sun, 18 Jun 2023 17:03:51 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58594) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qAzYq-0007if-FJ for 62375@debbugs.gnu.org; Sun, 18 Jun 2023 17:03:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qAzYi-0004fY-Vf; Sun, 18 Jun 2023 17:03:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=D9rdYrePwFDOeK31dWICzjVk9+R7vgikhvOW/s7P74Y=; b=r9AysnxmBKqpEEt7aWgV RjHSRtzaUQNrwW08xo6yFYpvM+ipVJwUmDI1ME+t9dWlXaVlfRgGwrae+V7ZLSfJy8Gah9KoPjf03 IP6HzdL3Toh/onALGumGo5NQudvJcrTYeinlZxJcfVpEvWpZkpV/g0taRB6TJDVSOfGbFlXSmy2iB fduT6lvpI6EBg/NqkqL91oECQYfaqQiYvegvPIXN4BVCXz7JSG8t98fSsWLe3fsDe+izdYVFpfZs4 Ydq3EWXlwau/ccRuXg7Lplq9eZYYaVCZmdybF8a3vkFhO1X+TJTuw0FS6PillLHtG2qDoTA/cXSln gcMCE1JP55L/2Q==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qAzYh-00065u-7H; Sun, 18 Jun 2023 17:03:40 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Jelle Licht Subject: Re: bug#62375: [PATCH 0/1] npm binary importer References: <87tty4g9kj.fsf@gnu.org> <871qkurzwh.fsf@fsfe.org> <874jpecisz.fsf_-_@gnu.org> Date: Sun, 18 Jun 2023 23:03:35 +0200 In-Reply-To: <874jpecisz.fsf_-_@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s?= =?utf-8?Q?=22's?= message of "Mon, 17 Apr 2023 23:14:52 +0200") Message-ID: <87pm5spjo8.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Josselin Poiret , Christopher Baines , Simon Tournier , Mathieu Othacehe , Tobias Geerinckx-Rice , Lars-Dominik Braun , 62375@debbugs.gnu.org, Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hey Jelle, How far are we from merging? :-) TIA, Ludo=E2=80=99. Ludovic Court=C3=A8s skribis: > Hi, > > Jelle Licht skribis: > >> Ludovic Court=C3=A8s writes: > > [...] > >>>> + (match npm-package >>>> + (($ name version home-page dependencies dev-de= pendencies peer-dependencies license description dist) >>> >>> Please use =E2=80=98match-record=E2=80=99 instead and keep lines below = 80 chars. :-) >> >> The records generated by `define-json-mapping' through >> `define-record-type' seem to not work with `match-record'. > > Oh right, my bad. > > The other option is to call record accessors; it would avoid the risk > associated with index-based matches but may be more verbose. Your call! > > Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 22 05:39:24 2023 Received: (at 62375) by debbugs.gnu.org; 22 Jun 2023 09:39:24 +0000 Received: from localhost ([127.0.0.1]:34848 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qCGmi-00020r-Hz for submit@debbugs.gnu.org; Thu, 22 Jun 2023 05:39:24 -0400 Received: from mail1.fsfe.org ([217.69.89.151]:49624) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qCGmf-00020f-UW for 62375@debbugs.gnu.org; Thu, 22 Jun 2023 05:39:23 -0400 From: Jelle Licht DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1687426759; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gf7QES9+hTr0pNFFaahXo88glC8+xlAfT9AE2BMdAZo=; b=XpCS7DYC38gkhOtrwGeFzJH9ECN/0e9JMpw2FA+S+/1rHSaiMP4MQ2sXLEux/F8CUCNbAs 4gAw8DYToKGQjVGy2oRy7e5eJqUPe2B/J+O2RJaSFhCStTaywZYi35b8dKCi+trOZ1QRU/ GkNeXWiKTC6u50HwY7QY/36Jfnb7npw= To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#62375: [PATCH 0/1] npm binary importer In-Reply-To: <87pm5spjo8.fsf_-_@gnu.org> References: <87tty4g9kj.fsf@gnu.org> <871qkurzwh.fsf@fsfe.org> <874jpecisz.fsf_-_@gnu.org> <87pm5spjo8.fsf_-_@gnu.org> Date: Thu, 22 Jun 2023 11:39:19 +0200 Message-ID: <87352j7s54.fsf@fsfe.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Josselin Poiret , Christopher Baines , Simon Tournier , Mathieu Othacehe , Tobias Geerinckx-Rice , Lars-Dominik Braun , 62375@debbugs.gnu.org, Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Ludovic Court=C3=A8s writes: > Hey Jelle, > > How far are we from merging? :-) Hey Ludo, Thanks for checking in! I haven't had a whole lot of time recently due to a bunch of (positive) personal things, but I'll reach out soon~ish with the next iteration. Cheers, - Jelle From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 07 19:59:37 2024 Received: (at 62375) by debbugs.gnu.org; 8 Feb 2024 00:59:37 +0000 Received: from localhost ([127.0.0.1]:58668 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rXslN-0008Gs-AT for submit@debbugs.gnu.org; Wed, 07 Feb 2024 19:59:37 -0500 Received: from 2.mo583.mail-out.ovh.net ([178.33.109.111]:56291) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rXslK-0008Gi-AN for 62375@debbugs.gnu.org; Wed, 07 Feb 2024 19:59:36 -0500 Received: from director3.ghost.mail-out.ovh.net (unknown [10.108.9.148]) by mo583.mail-out.ovh.net (Postfix) with ESMTP id C74242878E for <62375@debbugs.gnu.org>; Thu, 8 Feb 2024 00:59:18 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-l9gd2 (unknown [10.110.113.89]) by director3.ghost.mail-out.ovh.net (Postfix) with ESMTPS id C73271FE70; Thu, 8 Feb 2024 00:59:16 +0000 (UTC) Received: from ngraves.fr ([37.59.142.102]) by ghost-submission-6684bf9d7b-l9gd2 with ESMTPSA id LIMXLWQnxGUnChIAtmFz7A (envelope-from ); Thu, 08 Feb 2024 00:59:16 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-102R004863b317c-78b4-430f-9c17-78c8418dc472, 1F816C6F1D534EC93874C2C90F666022E64E63F2) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 81.67.146.208 From: Nicolas Graves To: jlicht@fsfe.org, 62375@debbugs.gnu.org Subject: Re: [bug#62375] [PATCH 0/1] npm binary importer In-Reply-To: References: Date: Thu, 08 Feb 2024 01:59:16 +0100 Message-ID: <87eddn4vq3.fsf@ngraves.fr> MIME-Version: 1.0 Content-Type: text/plain X-Ovh-Tracer-Id: 2838956615636017906 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvledrtdefgddvlecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemucehtddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttdenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpeekieegjeefgffhfeeileehveelieevffetkeeiueetueevtdfgfefhvdejleeuveenucffohhmrghinhepnhhpmhhjshdrohhrghenucfkphepuddvjedrtddrtddruddpkedurdeijedrudegiedrvddtkedpfeejrdehledrudegvddruddtvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeduvdejrddtrddtrddupdhmrghilhhfrhhomhepoehnghhrrghvvghssehnghhrrghvvghsrdhfrheqpdhnsggprhgtphhtthhopedupdhrtghpthhtohepiedvfeejheesuggvsggsuhhgshdrghhnuhdrohhrghdpoffvtefjohhsthepmhhoheekfedpmhhouggvpehsmhhtphhouhht X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 62375 Cc: dev@jpoiret.xyz, mail@cbaines.net, zimon.toutoune@gmail.com, othacehe@gnu.org, ludo@gnu.org, me@tobias.gr, rekado@elephly.net, Jelle Licht X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On 2023-03-22 12:25, jlicht@fsfe.org wrote: > From: Jelle Licht > > Folks, > > Here a revised patch to add the npm binary importer. To give some context, > 'binary' here refers to the fact that this downloads archives straight from > the npm registry at https://registry.npmjs.org. Some of these downloaded > archives may not contain the original sources, so unless properly vetted the > output of this importer does not allow one to easily generate package > expressions for inclusion in guix's collection of packages. > > It should work as-is for most simple NPM packages. As noted in an inline > comment somewhere, the way both npm and our very own node-build-system treats > peer dependencies may require some manual intervention from time to time to > hook in the right dependency at the right spot. The upside here is that when > this happens, it's either trivial for a human to spot and fix, or wholly > incompatible with our current approach so not fixable without writing a custom > build system to deal with dependency cycles :-). > > Please test and review, let us get this merged as it might be useful to many > people for building out their personal channels and/or package expressions. > > Special thanks to Timothy Sample and Lars-Dominik Braun for involvement in > realising this, way too long ago. > > > Jelle Licht (1): > import: Add binary npm importer. > > Makefile.am | 2 + > guix/import/npm-binary.scm | 269 +++++++++++++++++++++++++++++ > guix/scripts/import.scm | 2 +- > guix/scripts/import/npm-binary.scm | 113 ++++++++++++ > 4 files changed, 385 insertions(+), 1 deletion(-) > create mode 100644 guix/import/npm-binary.scm > create mode 100644 guix/scripts/import/npm-binary.scm Hey Jelle, Just to understand, is there a reason why you didn't went further with the sucrase approach? -- Best regards, Nicolas Graves From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 24 14:14:28 2024 Received: (at 62375) by debbugs.gnu.org; 24 Mar 2024 18:14:28 +0000 Received: from localhost ([127.0.0.1]:47387 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1roSMU-0007Ao-JO for submit@debbugs.gnu.org; Sun, 24 Mar 2024 14:14:28 -0400 Received: from mail-vi1eur04olkn2022.outbound.protection.outlook.com ([40.92.75.22]:33601 helo=EUR04-VI1-obe.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1roPFI-0006i6-Vj for 62375@debbugs.gnu.org; Sun, 24 Mar 2024 10:54:50 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CW5MRo3Uk1+2eRG5EanNo2/BJEqd/b2rDzQVHTAwZXhP6xAFAx0bOdvqcMGL3ldyTowysOMQ7fkLGe2AN44uLDXKs7LGuTbXz8HdixtDWcZDZrkH9DdbNSrFm+vH8Yk5D9C7tXnfVBUDlEFFGE41C1fliKzDb5yVz7+ADiwCeGQ0rlgDm7Qr+gky0nMBwy/89rqll6gLvadEWmbNaxrapwStuIFSoKINExfSK1rYDxe9T1/XliINkj6sa6mzccNw5zHj4QLsds/rLfbqoG5gW6E2n2TOhixJJpQJazhjkfxCkLv7i7KWlaH9/Jp9HQgbkjbBk+jVxWX3b6rU1hcY+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5TrsKcyRTsCZQQeHzqXBjBYvCjSRwxkN9pJY+fUcarc=; b=Kphi4AIPnLGKByBhXmFruocTLI5EMmnvBuLvreftzAfQ71IMXhP+70cFaXUDHBZGNmZFhkkWdJGs0/i35q1oKKEzDPdKuI+fH2XuU4J9uRF2uqsIXuqI1Ab1M25ZAVG/DHNDS5awkvyJXYo5eyHGM3SuLTH1z7qUVl04VsWI+s5n6xS+PF06RiyxUgzi/rJT7UgMX8tbuE9J8/VOoJHAxIcLk+T/si1khNkOToDwQMLdTCBJdcWqeole+N4GzJet21TCVuvTZoAAtCvMtBJJuRiu4T81inKwXprRzsBLFkmGeZfGw9xlCgGmBVwpOJSMBjYz+DUNl69WtjtmR49qYQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=OUTLOOK.FR; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5TrsKcyRTsCZQQeHzqXBjBYvCjSRwxkN9pJY+fUcarc=; b=PRZR4sfN1pl8kclSB+/39mdz0TSPYNVYOWqbRlywlXh5ZPftpILoGbHedKmWfARIaDbMiPFI9/vaXN1vUd/aUYc5gNyTriyXcaRC3IVtsXec0h9TrJSnfW5DKLxZVdJ2C3TE7ifuM6Fg7bwqsKaZeBo/n2nZ8SGQXO3ucT/jzUzoUHAencgL7Skqey8TqXI3RojSloY5lEFrs5gzIwABZtp2lRq9FTrbjZ9AUfSrJ6Atj8eswzMeFhfdmpNQPitqbBJ9NXsel3v1zamB0fPStMFEOOthBvyeT3JtJS4/AC1gYpHdCsIABbRf9kdKYMuZOmdQt2HJDj94I6C/2iU85g== Received: from AS8P189MB2024.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:52b::6) by VE1P189MB1087.EURP189.PROD.OUTLOOK.COM (2603:10a6:800:16b::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7409.31; Sun, 24 Mar 2024 14:54:00 +0000 Received: from AS8P189MB2024.EURP189.PROD.OUTLOOK.COM ([fe80::bd04:a484:76b3:a5f1]) by AS8P189MB2024.EURP189.PROD.OUTLOOK.COM ([fe80::bd04:a484:76b3:a5f1%7]) with mapi id 15.20.7409.028; Sun, 24 Mar 2024 14:54:00 +0000 From: Pablo Zamora To: "62375@debbugs.gnu.org" <62375@debbugs.gnu.org> Subject: Continue the npm-binary importer Thread-Topic: Continue the npm-binary importer Thread-Index: AQHaffo6iAyc71SpuU+Hym+lQboSTQ== Date: Sun, 24 Mar 2024 14:54:00 +0000 Message-ID: Accept-Language: fr-FR, en-US Content-Language: fr-FR X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: x-tmn: [6UGTQ3QMd9zRU0OsKHfS9y6NSfxql4s7INf3pB+JRjqeMSJOYvRCOTfMZ01BjCWUHoimdhOok9k=] x-ms-publictraffictype: Email x-ms-traffictypediagnostic: AS8P189MB2024:EE_|VE1P189MB1087:EE_ x-ms-office365-filtering-correlation-id: f7d383c4-8e86-4403-618d-08dc4c123d3e x-ms-exchange-slblob-mailprops: EgT5Wr3QDKwnUYJPhqA/guHYz8JO0CG5t72H588CEuMo3YNlTOa82t5d2VnMJvoA62OpGQPs2M8+pvQWEt+t4EUaqHwoErcwyBLkmK7N5VVH6xbQoUq7puGlhAGAuz5soMRSzTsBjrT6V7S+1X+m6mtStH5gIY0E4lpgI7+TeQZU3BAMLJtISTai+VQ2luHDg1Fv9HiwxYk+sxY9XxvdBUEUwbYCDo8N/XQL2XLbXj3xayPSbpQlwKQtxB2fyMzDspo0CQfTZLMZiO9TW50YZwmKEM2zBlYpplj/nMFAMTY/IuY2cNF4VcXTrdfvAnePW94A2oVvunvO7fQtsFvpWQg6enfTEwVm0fMtVps5nf7nGZc9ENbbnMRV0PUSiYO/qHj350UYTqJUTFWa+n45HI9gmakgk/aK4zuMnlLA2dxeYuV2KffNer+Unn9X2vuNElQwEod5a/yq4+/oVfHJiM326sT5c7DivA+TRCuQF6jFFcPcS4tG2QFE1fNYnKpO99JFNkAMRcon7pVGc2gnSDm/CXVpRn1s45GbOYCbtGAHxHR9ISvOQVRC97qzBtRU9n1X2sdIu4TTtSQhcOrUT6zqt8dzH9aEZ+QaWoU2g12fgKplW9uRcPftDpfKd5l2h1CpYT+5JoNnza7407b/z0z4mG2Tl9ThjsovmoL+vxzpWCBFB0/agLZe6kVHiBwC+hV+3ZyJdklKJ4vNE2r1Fm/EO+EVmhNMRIh77xZNZEg= x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: FvyxXpoL5YyiIM5bOauatFRt5ttN1W0Zpb4Vv6Vm+RJ1Dlv2BphRE5YeZbXEH82h2EbA1xOveK8s7UmTOWrpdgwBAslalRNT3qsQLM1DCOhr4IVOZEIsL9DIEpDmCQUuVdQ+l47glxRgvlqvxsdr+GB0jrpeOikl/x7cOnGpXWXGda4v7fjAq9feHDoMOdrYEMC2LXLMtsSLG60p6Rd+fW9aradTCr1qht4btwye5+ad2Zw82dn7Wkc/Vp57xcNxElEo0h58ViB6c7XjeOARGYLgz/wLf9exeMsdWz6SPixjvWGxu0gX4q+djtqmjJGklFLbjcfK4ShzIiBqHGkaB04ZG2r6RnQHobH1BTjbfHmqGH0tbuhsb9fCY26teWURas7OP+9J4b3+gGlspDmD9kSRubK5p+/bHM0dkhZJkpiGulJM7qNzRNXz4cEEaK3JzkmtCbvZN/KgRJapGsEbIGxppUsihJsEY/LM7/QL4H2PKebIGzVS0zmAt4AOPdakyTJZ2kXPPtLdqTDPWroTY0W5Q4vXXsFX7uBY1UFvhf94lIk4e3WT+fvIjjBrGacu37PfgpNTQee057ISDQrsw3344nVmKVNAoalMdKyBxtZz1AESWxBKOfczGlHOLQrOyGKngXLZNSbLzOzs9T8nV2FQ2kmUiIURjv0Tok1tOHP0fnaTzly2nRMk8QATR0Dbot3QzCvg1uvjQiyuUOZuippTsnMpLRRu4wug11AB+gM= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?teTAWf0XAgtHyhwebyG0NbGW9YE2WF/3bSJGCByqiAHQ5d5I1Kw+tlivXv?= =?iso-8859-1?Q?JuE16UCZ/Epb4fH8y6JXYdecbrZR523XzE7x7e9GXi774dcG89u3fqenlB?= =?iso-8859-1?Q?qOWzE0gMGiUWryG+i0PfyI7za5l5bx/qi6BCbufnFKapJYDY0xQj7vtLY8?= =?iso-8859-1?Q?vKJmKSFhuLKNYZjgvdDihoea62ZueRzm6jqB5d1EHfmrN4P6jB82J6MzG7?= =?iso-8859-1?Q?dC1npaMmwVP584Go0p+RQ70UzBph4nDfRdBlOdbSOYAWjAOCxu2Fssld4W?= =?iso-8859-1?Q?n+toEKLc/tXKBYamTR0hB+HUZ53w6E/IkHtyg/+f9jSviqzzUbZFuE30fe?= =?iso-8859-1?Q?e7DGSRDjDGxOFh6lcvTAn2nl3mNA4NpZCixLvJn6It40uVhKjmoH67xbZd?= =?iso-8859-1?Q?3pAxW5YesyZTcvqkLtbrrJtDNiNW6j7MfUGO36lYyJHx6rzWpLA6FDm9gK?= =?iso-8859-1?Q?NcpzUFND8ll3NYEvbvVpitvKN+8mV5HO2pGO8Tn3LLxIaMUbKLp/BnMXBI?= =?iso-8859-1?Q?GVwxtjOXHDMGX9kzAnERGLi6AuikWMMG5osFm/9Z2qY5hXY4R7xYUSB2od?= =?iso-8859-1?Q?rQa1btOOcyUh4eCD9+NMBIg948MVvAk0NlDALIcjn+3S2dSC2z0SH3RTqL?= =?iso-8859-1?Q?sckk/7mtCI2AQowI88884G21iksLidLhAbsr2VeAxTZGr56ztJw5BJE01h?= =?iso-8859-1?Q?8jopuVAPAAg82D2GSkA1BeSeJrgm96oTOtdQASaIk8dpUo9VPY3jiyGjB6?= =?iso-8859-1?Q?xZ+Yw9blj7UjZqEInARaRBHcsx50luEo2Pbq2jVb5KISZACLUzgyL7NXiv?= =?iso-8859-1?Q?izzqp0QDpL+aGJBhw05+ZcKwaRbZMljaPo70ZF0OOnKPuFJ/MaE4Xn8r8r?= =?iso-8859-1?Q?oUvV7/PCHmMA5f7RmxWTVRsScRhz2mtHi1Qkeu4QLy63wqrJgGF6Tw3SRi?= =?iso-8859-1?Q?M47+Qqp2XS7binx9QJnwP3sE38CRUzJOg1kDVafDD67r7ae5QWKZythPAq?= =?iso-8859-1?Q?6vjC8NHSEAjnRkkC/EB8cym3AWsvPvxs+I/ywXcCcPrJNfs1DsuQLYRCvy?= =?iso-8859-1?Q?XDN/4qevOt8muNTeOdmgvqSaSknpwQYzkcRCuXvx+bPIczuTNqgh1f+0Pn?= =?iso-8859-1?Q?QpIPqKZfWLAl1I94hVXctYmQaW7PPmFoZ8ZlcHvvI6x+fzN1CclJZQZLSH?= =?iso-8859-1?Q?SMUo7Gz3IJA4Ua0Qe3S4qTtweMnGzPTeEkxKYUpEXgFznh4S/m7UOJQ4zD?= =?iso-8859-1?Q?flfNRQsI1jQONCDxA2jxHzt5NP+Ym70DD+C2h5UZjuVemcJJcOAeasOJcL?= =?iso-8859-1?Q?6PcIrIHSngSPVJSq+wt9ABrLQPUSLiamyLeyEbQ2Pvka2K4=3D?= Content-Type: multipart/alternative; boundary="_000_AS8P189MB20242416E3975E7D4AC12F3BFD372AS8P189MB2024EURP_" MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4734-24-msonline-outlook-c54b5.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AS8P189MB2024.EURP189.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: f7d383c4-8e86-4403-618d-08dc4c123d3e X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Mar 2024 14:54:00.0376 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1P189MB1087 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 62375 X-Mailman-Approved-At: Sun, 24 Mar 2024 14:14:25 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --_000_AS8P189MB20242416E3975E7D4AC12F3BFD372AS8P189MB2024EURP_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Dear Guix, I am stuck while trying to define the 'bitwarden' package definition on my = personal channel here https://github.com/Pablo12345678901/guix-custom-chann= el/blob/master/dev/my-bitwarden.scm . I am facing issues due to the npm dep= endencies. One of the potential solution would be this importer. I would like to finis= h writing it and perform the tests. On which branch should I send git pull request to continue what has been do= ne so far ? This would be my first commit with guix. Pablo Zamora --_000_AS8P189MB20242416E3975E7D4AC12F3BFD372AS8P189MB2024EURP_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Dear Guix,

I am stuck while trying to define the 'bitwarden' package definition on my = personal channel here https://github.com/Pablo12345678901/guix-custom-channel/blob/master/dev/my-= bitwarden.scm . I am facing issues due to the npm dependencies.

One of the potential solution would be this importer. I would like to finis= h writing it and perform the tests.

On which branch should I send git pull request to continue what has been do= ne so far ? This would be my first commit with guix.

Pablo Zamora
--_000_AS8P189MB20242416E3975E7D4AC12F3BFD372AS8P189MB2024EURP_-- From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 31 15:47:14 2024 Received: (at 62375) by debbugs.gnu.org; 31 Mar 2024 19:47:14 +0000 Received: from localhost ([127.0.0.1]:48489 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rr196-0005hM-By for submit@debbugs.gnu.org; Sun, 31 Mar 2024 15:47:13 -0400 Received: from mail1.fsfe.org ([217.69.89.151]:42988) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rr191-0005gY-Nd for 62375@debbugs.gnu.org; Sun, 31 Mar 2024 15:47:10 -0400 From: jlicht@fsfe.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1711914420; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fiTdm7OjdNZGY3zdjonpbJ0iQVP3onoyT/ygwpE9MWA=; b=cMzPUC/QbwUiwwPsItu/uHGluVH9F4mlxoC8BW4yQwIv+YJq4lTXLpY8KobKzNS5/5NDWp hEKbYalWpuKfBuNT7xgAttTA+xNzTDKGCL8K+OV5HWbRYIyJvNDEPOhnAHRhBP8jMC3qDs D9tBJzymE4wmdKvKMZB/qptWi8gHqOQ= To: 62375@debbugs.gnu.org Subject: [PATCH v2] import: Add binary npm importer. Date: Sun, 31 Mar 2024 21:46:55 +0200 Message-ID: MIME-Version: 1.0 X-Debbugs-Cc: Christopher Baines , Josselin Poiret , Ludovic Courtès , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Jelle Licht , Lars-Dominik Braun , pablo.zamora@outlook.fr X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Jelle Licht * guix/scripts/import.scm: (importers): Add "npm-binary". * guix/import/npm-binary.scm: New file. * guix/scripts/import/npm-binary.scm: New file. * Makefile.am: Add them. Co-authored-by: Timothy Sample Co-authored-by: Lars-Dominik Braun Change-Id: I98a45068cf5b9c42790664cc743feaa7ac76f807 --- Changes in v2: - Change *SOME-VAR* to %SOME-VAR - Removed unused http-error-code - Rebase on master - Refactor hash-url to use port-sha256 helper - use explicit record accessors instead of order-sensitive destructuring - address line-width styling issues - added basic documentation - added some basic tests (using simple mocks instead of with-http-server) - simplify import script entrypoint Makefile.am | 3 + doc/guix.texi | 33 ++++ guix/import/npm-binary.scm | 277 +++++++++++++++++++++++++++++ guix/scripts/import.scm | 2 +- guix/scripts/import/npm-binary.scm | 108 +++++++++++ tests/npm-binary.scm | 146 +++++++++++++++ 6 files changed, 568 insertions(+), 1 deletion(-) create mode 100644 guix/import/npm-binary.scm create mode 100644 guix/scripts/import/npm-binary.scm create mode 100755 tests/npm-binary.scm diff --git a/Makefile.am b/Makefile.am index 1c5688ac13..459c47a954 100644 --- a/Makefile.am +++ b/Makefile.am @@ -305,6 +305,7 @@ MODULES = \ guix/import/kde.scm \ guix/import/launchpad.scm \ guix/import/minetest.scm \ + guix/import/npm-binary.scm \ guix/import/opam.scm \ guix/import/print.scm \ guix/import/pypi.scm \ @@ -359,6 +360,7 @@ MODULES = \ guix/scripts/import/hexpm.scm \ guix/scripts/import/json.scm \ guix/scripts/import/minetest.scm \ + guix/scripts/import/npm-binary.scm \ guix/scripts/import/opam.scm \ guix/scripts/import/pypi.scm \ guix/scripts/import/stackage.scm \ @@ -557,6 +559,7 @@ SCM_TESTS = \ tests/modules.scm \ tests/monads.scm \ tests/nar.scm \ + tests/npm-binary.scm \ tests/networking.scm \ tests/opam.scm \ tests/openpgp.scm \ diff --git a/doc/guix.texi b/doc/guix.texi index 69a904473c..566af6e849 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -14641,6 +14641,39 @@ Invoking guix import in Guix. @end table +@item npm-binary +@cindex npm +@cindex Node.js +Import metadata from the @uref{https://registry.npmjs.org, npm +Registry}, as in this example: + +@example +guix import npm-binary buffer-crc32 +@end example + +The npm-binary importer also allows you to specify a version string: + +@example +guix import npm-binary buffer-crc32 1.0.0 +@end example + +@quotation Note +Generated package expressions skip the build step of the +@code{node-build-system}. As such, generated package expressions often +refer to transpiled or generated files, instead of being built from +source. +@end quotation + +Additional options include: + +@table @code +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table + @item opam @cindex OPAM @cindex OCaml diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm new file mode 100644 index 0000000000..57c985baf2 --- /dev/null +++ b/guix/import/npm-binary.scm @@ -0,0 +1,277 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019, 2020 Timothy Sample +;;; Copyright © 2020, 2023, 2024 Jelle Licht +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix import npm-binary) + #:use-module ((gnu services configuration) #:select (alist?)) + #:use-module (gcrypt hash) + #:use-module (gnu packages) + #:use-module (guix base32) + #:use-module (guix http-client) + #:use-module (guix import json) + #:use-module (guix import utils) + #:use-module (guix memoization) + #:use-module (guix utils) + #:use-module (ice-9 match) + #:use-module (ice-9 receive) + #:use-module (ice-9 regex) + #:use-module (json) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-41) + #:use-module (srfi srfi-9) + #:use-module (web client) + #:use-module (web response) + #:use-module (web uri) + #:export (npm-binary-recursive-import + npm-binary->guix-package + make-versioned-package + name+version->symbol)) + +;; Autoload Guile-Semver so we only have a soft dependency. +(module-autoload! (current-module) + '(semver) + '(string->semver semver? semver->string semver=? semver>?)) +(module-autoload! (current-module) + '(semver ranges) + '(*semver-range-any* string->semver-range semver-range-contains?)) + +;; Dist-tags +(define-json-mapping make-dist-tags dist-tags? + json->dist-tags + (latest dist-tags-latest "latest" string->semver)) + +(define-record-type + (make-versioned-package name version) + versioned-package? + (name versioned-package-name) ;string + (version versioned-package-version)) ;string + +(define (dependencies->versioned-packages entries) + (match entries + (((names . versions) ...) + (map make-versioned-package names versions)) + (_ '()))) + +(define (extract-license license-string) + (if (unspecified? license-string) + 'unspecified! + (spdx-string->license license-string))) + +(define-json-mapping make-dist dist? + json->dist + (tarball dist-tarball)) + +(define (empty-or-string s) + (if (string? s) s "")) + +(define-json-mapping make-package-revision package-revision? + json->package-revision + (name package-revision-name) + (version package-revision-version "version" ;semver + string->semver) + (home-page package-revision-home-page "homepage") ;string + (dependencies package-revision-dependencies ;list of versioned-package + "dependencies" + dependencies->versioned-packages) + (dev-dependencies package-revision-dev-dependencies ;list of versioned-package + "devDependencies" dependencies->versioned-packages) + (peer-dependencies package-revision-peer-dependencies ;list of versioned-package + "peerDependencies" dependencies->versioned-packages) + (license package-revision-license "license" ;license | #f + (match-lambda + ((? unspecified?) #f) + ((? string? str) (spdx-string->license str)) + ((? alist? alist) + (match (assoc "type" alist) + ((_ . (? string? type)) + (spdx-string->license type)) + (_ #f))))) + (description package-revision-description ;string + "description" empty-or-string) + (dist package-revision-dist "dist" json->dist)) ;dist + +(define (versions->package-revisions versions) + (match versions + (((version . package-spec) ...) + (map json->package-revision package-spec)) + (_ '()))) + +(define (versions->package-versions versions) + (match versions + (((version . package-spec) ...) + (map string->semver versions)) + (_ '()))) + +(define-json-mapping make-meta-package meta-package? + json->meta-package + (name meta-package-name) ;string + (description meta-package-description) ;string + (dist-tags meta-package-dist-tags "dist-tags" json->dist-tags) ;dist-tags + (revisions meta-package-revisions "versions" versions->package-revisions)) + +;; TODO: Support other registries +(define %registry "https://registry.npmjs.org") +(define %default-page "https://www.npmjs.com/package") + +(define (lookup-meta-package name) + (let ((json (json-fetch (string-append %registry "/" (uri-encode name))))) + (and=> json json->meta-package))) + +(define lookup-meta-package* (memoize lookup-meta-package)) + +(define (meta-package-versions meta) + (map package-revision-version + (meta-package-revisions meta))) + +(define (meta-package-latest meta) + (and=> (meta-package-dist-tags meta) dist-tags-latest)) + +(define* (meta-package-package meta #:optional + (version (meta-package-latest meta))) + (match version + ((? semver?) (find (lambda (revision) + (semver=? version (package-revision-version revision))) + (meta-package-revisions meta))) + ((? string?) (meta-package-package meta (string->semver version))) + (_ #f))) + +(define* (semver-latest svs #:optional (svr *semver-range-any*)) + (find (cut semver-range-contains? svr <>) + (sort svs semver>?))) + +(define* (resolve-package name #:optional (svr *semver-range-any*)) + (let ((meta (lookup-meta-package* name))) + (and meta + (let* ((version (semver-latest (or (meta-package-versions meta) '()) svr)) + (pkg (meta-package-package meta version))) + pkg)))) + + +;;; +;;; Converting packages +;;; + +(define (hash-url url) + "Downloads the resource at URL and computes the base32 hash for it." + (bytevector->nix-base32-string (port-sha256 (http-fetch url)))) + +(define (npm-name->name npm-name) + "Return a Guix package name for the npm package with name NPM-NAME." + (define (clean name) + (string-map (lambda (chr) (if (char=? chr #\/) #\- chr)) + (string-filter (negate (cut char=? <> #\@)) name))) + (guix-name "node-" (clean npm-name))) + +(define (name+version->symbol name version) + (string->symbol (string-append name "-" version))) + +(define (package-revision->symbol package) + (let* ((npm-name (package-revision-name package)) + (version (semver->string (package-revision-version package))) + (name (npm-name->name npm-name))) + (name+version->symbol name version))) + +(define (npm-package->package-sexp npm-package) + "Return the `package' s-expression for an NPM-PACKAGE." + (define resolve-spec + (match-lambda + (($ name version) + (resolve-package name (string->semver-range version))))) + + (if (package-revision? npm-package) + (let ((name (package-revision-name npm-package)) + (version (package-revision-version npm-package)) + (home-page (package-revision-home-page npm-package)) + (dependencies (package-revision-dependencies npm-package)) + (dev-dependencies (package-revision-dev-dependencies npm-package)) + (peer-dependencies (package-revision-peer-dependencies npm-package)) + (license (package-revision-license npm-package)) + (description (package-revision-description npm-package)) + (dist (package-revision-dist npm-package))) + (let* ((name (npm-name->name name)) + (url (dist-tarball dist)) + (home-page (if (string? home-page) + home-page + (string-append %default-page "/" (uri-encode name)))) + (synopsis description) + (resolved-deps (map resolve-spec + (append dependencies peer-dependencies))) + (peer-names (map versioned-package-name peer-dependencies)) + ;; lset-difference for treating peer-dependencies as dependencies, + ;; which leads to dependency cycles. lset-union for treating them as + ;; (ignored) dev-dependencies, which leads to broken packages. + (dev-names + (lset-union string= + (map versioned-package-name dev-dependencies) + peer-names)) + (extra-phases + (match dev-names + (() '()) + ((dev-names ...) + `((add-after 'patch-dependencies 'delete-dev-dependencies + (lambda _ + (delete-dependencies '(,@(reverse dev-names)))))))))) + (values + `(package + (name ,name) + (version ,(semver->string (package-revision-version npm-package))) + (source (origin + (method url-fetch) + (uri ,url) + (sha256 (base32 ,(hash-url url))))) + (build-system node-build-system) + (arguments + (list + #:tests? #f + #:phases + #~(modify-phases %standard-phases + (delete 'build) + ,@extra-phases))) + ,@(match dependencies + (() '()) + ((dependencies ...) + `((inputs + (list ,@(map package-revision->symbol resolved-deps)))))) + (home-page ,home-page) + (synopsis ,synopsis) + (description ,description) + (license ,license)) + (map (match-lambda (($ name version) + (list name (semver->string version)))) + resolved-deps)))) + (values #f '()))) + + +;;; +;;; Interface +;;; + +(define npm-binary->guix-package + (lambda* (name #:key (version *semver-range-any*) #:allow-other-keys) + (let* ((svr (match version + ((? string?) (string->semver-range version)) + (_ version))) + (pkg (resolve-package name svr))) + (npm-package->package-sexp pkg)))) + +(define* (npm-binary-recursive-import package-name #:key version) + (recursive-import package-name + #:repo->guix-package (memoize npm-binary->guix-package) + #:version version + #:guix-name npm-name->name)) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index 1f34cab088..d724f2bca3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -49,7 +49,7 @@ (define %standard-import-options '()) (define importers '("gnu" "pypi" "cpan" "hackage" "stackage" "egg" "elpa" "gem" "go" "cran" "crate" "texlive" "json" "opam" - "minetest" "elm" "hexpm" "composer")) + "minetest" "elm" "hexpm" "composer" "npm-binary")) (define (resolve-importer name) (let ((module (resolve-interface diff --git a/guix/scripts/import/npm-binary.scm b/guix/scripts/import/npm-binary.scm new file mode 100644 index 0000000000..d16b0f15b0 --- /dev/null +++ b/guix/scripts/import/npm-binary.scm @@ -0,0 +1,108 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014 David Thompson +;;; Copyright © 2018 Ricardo Wurmus +;;; Copyright © 2019 Timothy Sample +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts import npm-binary) + #:use-module (guix ui) + #:use-module (guix utils) + #:use-module (guix scripts) + #:use-module (guix import npm-binary) + #:use-module (guix scripts import) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-11) + #:use-module (srfi srfi-37) + #:use-module (srfi srfi-41) + #:use-module (ice-9 match) + #:use-module (ice-9 format) + #:export (guix-import-npm-binary)) + + +;;; +;;; Command-line options. +;;; + +(define %default-options + '()) + +(define (show-help) + (display (G_ "Usage: guix import npm-binary PACKAGE-NAME [VERSION] +Import and convert the npm package PACKAGE-NAME using the +`node-build-system' (but without building the package from source).")) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -r, --recursive import packages recursively")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + ;; Specification of the command-line options. + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix import npm-binary"))) + (option '(#\r "recursive") #f #f + (lambda (opt name arg result) + (alist-cons 'recursive #t result))) + %standard-import-options)) + + +;;; +;;; Entry point. +;;; + +(define (guix-import-npm-binary . args) + (define (parse-options) + ;; Return the alist of option values. + (args-fold* args %options + (lambda (opt name arg result) + (leave (G_ "~A: unrecognized option~%") name)) + (lambda (arg result) + (alist-cons 'argument arg result)) + %default-options)) + (let* ((opts (parse-options)) + (args (filter-map (match-lambda + (('argument . value) + value) + (_ #f)) + (reverse opts)))) + (let loop ((args args)) + (match args + ((package-name version) + (match (if (assoc-ref opts 'recursive) + ;; Recursive import + (npm-binary-recursive-import package-name #:version version) + ;; Single import + (npm-binary->guix-package package-name #:version version)) + ((or #f '()) + (leave (G_ "failed to download meta-data for package '~a@~a'~%") + package-name version)) + ((? list? sexps) sexps) + (sexp (list sexp)))) + ((package-name) + (loop (list package-name "*"))) + (() + (leave (G_ "too few arguments~%"))) + ((many ...) + (leave (G_ "too many arguments~%"))))))) diff --git a/tests/npm-binary.scm b/tests/npm-binary.scm new file mode 100755 index 0000000000..cf85e572b3 --- /dev/null +++ b/tests/npm-binary.scm @@ -0,0 +1,146 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2024 Jelle Licht +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . +(define-module (test-npm-binary) + #:use-module ((gcrypt hash) + #:select ((sha256 . gcrypt-sha256))) + #:use-module (guix import npm-binary) + #:use-module (guix base32) + #:use-module (guix tests) + #:use-module (srfi srfi-64) + #:use-module (ice-9 iconv) + #:use-module (ice-9 match) + #:export (run-test)) + +(define foo-json + "{ + \"name\": \"foo\", + \"dist-tags\": { + \"latest\": \"1.2.3\", + \"next\": \"2.0.1-beta4\" + }, + \"description\": \"General purpose utilities to foo your bars\", + \"homepage\": \"https://github.com/quartz/foo\", + \"repository\": \"quartz/foo\", + \"versions\": { + \"1.2.3\": { + \"name\": \"foo\", + \"description\": \"General purpose utilities to foo your bars\", + \"version\": \"1.2.3\", + \"author\": \"Jelle Licht \", + \"devDependencies\": { + \"node-megabuilder\": \"^0.0.2\" + }, + \"dependencies\": { + \"bar\": \"^0.1.0\" + }, + \"repository\": { + \"url\": \"quartz/foo\" + }, + \"homepage\": \"https://github.com/quartz/foo\", + \"license\": \"MIT\", + \"dist\": { + \"tarball\": \"https://registry.npmjs.org/foo/-/foo-1.2.3.tgz\" + } + } + } +}") + +(define bar-json + "{ + \"name\": \"bar\", + \"dist-tags\": { + \"latest\": \"0.1.2\" + }, + \"description\": \"Core module in FooBar\", + \"homepage\": \"https://github.com/quartz/bar\", + \"repository\": \"quartz/bar\", + \"versions\": { + \"0.1.2\": { + \"name\": \"bar\", + \"description\": \"Core module in FooBar\", + \"version\": \"0.1.2\", + \"author\": \"Jelle Licht \", + \"repository\": { + \"url\": \"quartz/bar\" + }, + \"homepage\": \"https://github.com/quartz/bar\", + \"license\": \"MIT\", + \"dist\": { + \"tarball\": \"https://registry.npmjs.org/bar/-/bar-0.1.2.tgz\" + } + } + } +}") + +(define test-source-hash + "") + +(define test-source + "Empty file\n") + +(define have-guile-semver? + (false-if-exception (resolve-interface '(semver)))) + +(test-begin "npm") + +(unless have-guile-semver? (test-skip 1)) +(test-assert "npm-binary->guix-package" + (mock ((guix http-client) http-fetch + (lambda* (url #:rest _) + (match url + ("https://registry.npmjs.org/foo" + (values (open-input-string foo-json) + (string-length foo-json))) + ("https://registry.npmjs.org/bar" + (values (open-input-string bar-json) + (string-length bar-json))) + ("https://registry.npmjs.org/foo/-/foo-1.2.3.tgz" + (set! test-source-hash + (bytevector->nix-base32-string + (gcrypt-sha256 (string->bytevector test-source "utf-8")))) + (values (open-input-string test-source) + (string-length test-source)))))) + (match (npm-binary->guix-package "foo") + (`(package + (name "node-foo") + (version "1.2.3") + (source (origin + (method url-fetch) + (uri "https://registry.npmjs.org/foo/-/foo-1.2.3.tgz") + (sha256 + (base32 + ,test-source-hash)))) + (build-system node-build-system) + (arguments + (list #:tests? #f + #:phases + (gexp (modify-phases %standard-phases + (delete 'build) + (add-after 'patch-dependencies 'delete-dev-dependencies + (lambda _ + (delete-dependencies '("node-megabuilder")))))))) + (inputs (list node-bar-0.1.2)) + (home-page "https://github.com/quartz/foo") + (synopsis "General purpose utilities to foo your bars") + (description "General purpose utilities to foo your bars") + (license license:expat)) + #t) + (x + (pk 'fail x #f))))) + +(test-end "npm") base-commit: 4d79a9cd6b5f0d8c5afbab0c6b70ae42740d5470 -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 31 15:57:17 2024 Received: (at 62375) by debbugs.gnu.org; 31 Mar 2024 19:57:17 +0000 Received: from localhost ([127.0.0.1]:48504 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rr1Ir-0006P8-7Z for submit@debbugs.gnu.org; Sun, 31 Mar 2024 15:57:17 -0400 Received: from mail2.fsfe.org ([2001:780:215:1::55]:53906) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rr1Io-0006Ou-MC for 62375@debbugs.gnu.org; Sun, 31 Mar 2024 15:57:16 -0400 From: Jelle Licht DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021081301; t=1711915028; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9o+wBwqqYgWcrdMzPKZ4dLlF0RT66Qxw5WAqLYELs1c=; b=cbS76g6lsc9WMuyk89Yhdv8Ji/GMT98B3H7DNEh8FU7PEH4Ck0mkhIx+VK9G8BWq/eiE0N SRCsFxKDt6BAtSO7ccxUesXX11KXuhOkz9DIVAbMY+IvMrfnJeIYRu1VvMEzpNSQAs0EWY zYdoErIKp586cM78XRSNqGfuJp5shD4= To: Pablo Zamora , "62375@debbugs.gnu.org" <62375@debbugs.gnu.org> Subject: Re: [bug#62375] Continue the npm-binary importer In-Reply-To: References: Date: Sun, 31 Mar 2024 21:57:07 +0200 Message-ID: <8734s6gooc.fsf@fsfe.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 62375 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Pablo, Pablo Zamora writes: > Dear Guix, > > I am stuck while trying to define the 'bitwarden' package definition on my personal channel here https://github.com/Pablo12345678901/guix-custom-channel/blob/master/dev/my-bitwarden.scm . I am facing issues due to the npm dependencies. > > One of the potential solution would be this importer. I would like to finish writing it and perform the tests. That is nice to hear! I just sent v2 of this patch addressing most of the shortcomings that were identified earlier. > On which branch should I send git pull request to continue what has been done so far ? This would be my first commit with guix. I see two ways forward, depending on how much work you foresee and/or are willing to do: 1) This patch gets through another round of review, and afterwards you can send patches based on the 'master' branch. 2) You work on improving this patch while it is still under review, and send a V{3,4,5} patch as a reply to this very thread. Kind regards, Jelle Licht From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 31 16:37:14 2024 Received: (at 62375) by debbugs.gnu.org; 31 Mar 2024 20:37:14 +0000 Received: from localhost ([127.0.0.1]:48547 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rr1vU-0008Tm-Tk for submit@debbugs.gnu.org; Sun, 31 Mar 2024 16:37:14 -0400 Received: from mail1.fsfe.org ([217.69.89.151]:46236) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rr1vQ-0008TY-DU for 62375@debbugs.gnu.org; Sun, 31 Mar 2024 16:37:11 -0400 From: jlicht@fsfe.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1711917422; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=PrAHOkgM9uAyoR6fHFgN/ibCJ5slDIKMLZzzTzHO9i8=; b=MIGE7VqXrmpZMNlNf2d21UnUrz3pxyG0hrEEAZn57trl7f9sIB1LxulT+aSA0dOhHOrgIW c0nK974t0NxO9cm1SN4UdQ4DtG0ssrfIDGvSp83TGTt41tkm6/ystxZV8m4C3sABGcZk2L BaAswJH8YKTlwid3uCOSNhKeZKcq6js= To: 62375@debbugs.gnu.org Subject: [PATCH v3] import: Add binary npm importer. Date: Sun, 31 Mar 2024 22:37:01 +0200 Message-ID: <7148a74f04e1598a18b58fbd0b1b27a2397312b2.1711917305.git.jlicht@fsfe.org> MIME-Version: 1.0 X-Debbugs-Cc: Christopher Baines , Josselin Poiret , Ludovic Courtès , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Jelle Licht , Lars-Dominik Braun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Jelle Licht * guix/scripts/import.scm: (importers): Add "npm-binary". * guix/import/npm-binary.scm: New file. * guix/scripts/import/npm-binary.scm: New file. * Makefile.am: Add them. Co-authored-by: Timothy Sample Co-authored-by: Lars-Dominik Braun Change-Id: I98a45068cf5b9c42790664cc743feaa7ac76f807 --- Changes in v3: - Ensure that package bindings generated during recursive import match the package bindings used in the list of inputs, as this was a regression introduced in the V2 series. Changes in v2: - Change *SOME-VAR* to %SOME-VAR - Removed unused http-error-code - Rebase on master - Refactor hash-url to use port-sha256 helper - use explicit record accessors instead of order-sensitive destructuring - address line-width styling issues - added basic documentation - added some basic tests (using simple mocks instead of with-http-server) - simplify import script entrypoint Makefile.am | 3 + doc/guix.texi | 33 ++++ guix/import/npm-binary.scm | 277 +++++++++++++++++++++++++++++ guix/scripts/import.scm | 2 +- guix/scripts/import/npm-binary.scm | 114 ++++++++++++ tests/npm-binary.scm | 146 +++++++++++++++ 6 files changed, 574 insertions(+), 1 deletion(-) create mode 100644 guix/import/npm-binary.scm create mode 100644 guix/scripts/import/npm-binary.scm create mode 100755 tests/npm-binary.scm diff --git a/Makefile.am b/Makefile.am index 1c5688ac13..459c47a954 100644 --- a/Makefile.am +++ b/Makefile.am @@ -305,6 +305,7 @@ MODULES = \ guix/import/kde.scm \ guix/import/launchpad.scm \ guix/import/minetest.scm \ + guix/import/npm-binary.scm \ guix/import/opam.scm \ guix/import/print.scm \ guix/import/pypi.scm \ @@ -359,6 +360,7 @@ MODULES = \ guix/scripts/import/hexpm.scm \ guix/scripts/import/json.scm \ guix/scripts/import/minetest.scm \ + guix/scripts/import/npm-binary.scm \ guix/scripts/import/opam.scm \ guix/scripts/import/pypi.scm \ guix/scripts/import/stackage.scm \ @@ -557,6 +559,7 @@ SCM_TESTS = \ tests/modules.scm \ tests/monads.scm \ tests/nar.scm \ + tests/npm-binary.scm \ tests/networking.scm \ tests/opam.scm \ tests/openpgp.scm \ diff --git a/doc/guix.texi b/doc/guix.texi index 69a904473c..566af6e849 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -14641,6 +14641,39 @@ Invoking guix import in Guix. @end table +@item npm-binary +@cindex npm +@cindex Node.js +Import metadata from the @uref{https://registry.npmjs.org, npm +Registry}, as in this example: + +@example +guix import npm-binary buffer-crc32 +@end example + +The npm-binary importer also allows you to specify a version string: + +@example +guix import npm-binary buffer-crc32 1.0.0 +@end example + +@quotation Note +Generated package expressions skip the build step of the +@code{node-build-system}. As such, generated package expressions often +refer to transpiled or generated files, instead of being built from +source. +@end quotation + +Additional options include: + +@table @code +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table + @item opam @cindex OPAM @cindex OCaml diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm new file mode 100644 index 0000000000..57c985baf2 --- /dev/null +++ b/guix/import/npm-binary.scm @@ -0,0 +1,277 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019, 2020 Timothy Sample +;;; Copyright © 2020, 2023, 2024 Jelle Licht +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix import npm-binary) + #:use-module ((gnu services configuration) #:select (alist?)) + #:use-module (gcrypt hash) + #:use-module (gnu packages) + #:use-module (guix base32) + #:use-module (guix http-client) + #:use-module (guix import json) + #:use-module (guix import utils) + #:use-module (guix memoization) + #:use-module (guix utils) + #:use-module (ice-9 match) + #:use-module (ice-9 receive) + #:use-module (ice-9 regex) + #:use-module (json) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-41) + #:use-module (srfi srfi-9) + #:use-module (web client) + #:use-module (web response) + #:use-module (web uri) + #:export (npm-binary-recursive-import + npm-binary->guix-package + make-versioned-package + name+version->symbol)) + +;; Autoload Guile-Semver so we only have a soft dependency. +(module-autoload! (current-module) + '(semver) + '(string->semver semver? semver->string semver=? semver>?)) +(module-autoload! (current-module) + '(semver ranges) + '(*semver-range-any* string->semver-range semver-range-contains?)) + +;; Dist-tags +(define-json-mapping make-dist-tags dist-tags? + json->dist-tags + (latest dist-tags-latest "latest" string->semver)) + +(define-record-type + (make-versioned-package name version) + versioned-package? + (name versioned-package-name) ;string + (version versioned-package-version)) ;string + +(define (dependencies->versioned-packages entries) + (match entries + (((names . versions) ...) + (map make-versioned-package names versions)) + (_ '()))) + +(define (extract-license license-string) + (if (unspecified? license-string) + 'unspecified! + (spdx-string->license license-string))) + +(define-json-mapping make-dist dist? + json->dist + (tarball dist-tarball)) + +(define (empty-or-string s) + (if (string? s) s "")) + +(define-json-mapping make-package-revision package-revision? + json->package-revision + (name package-revision-name) + (version package-revision-version "version" ;semver + string->semver) + (home-page package-revision-home-page "homepage") ;string + (dependencies package-revision-dependencies ;list of versioned-package + "dependencies" + dependencies->versioned-packages) + (dev-dependencies package-revision-dev-dependencies ;list of versioned-package + "devDependencies" dependencies->versioned-packages) + (peer-dependencies package-revision-peer-dependencies ;list of versioned-package + "peerDependencies" dependencies->versioned-packages) + (license package-revision-license "license" ;license | #f + (match-lambda + ((? unspecified?) #f) + ((? string? str) (spdx-string->license str)) + ((? alist? alist) + (match (assoc "type" alist) + ((_ . (? string? type)) + (spdx-string->license type)) + (_ #f))))) + (description package-revision-description ;string + "description" empty-or-string) + (dist package-revision-dist "dist" json->dist)) ;dist + +(define (versions->package-revisions versions) + (match versions + (((version . package-spec) ...) + (map json->package-revision package-spec)) + (_ '()))) + +(define (versions->package-versions versions) + (match versions + (((version . package-spec) ...) + (map string->semver versions)) + (_ '()))) + +(define-json-mapping make-meta-package meta-package? + json->meta-package + (name meta-package-name) ;string + (description meta-package-description) ;string + (dist-tags meta-package-dist-tags "dist-tags" json->dist-tags) ;dist-tags + (revisions meta-package-revisions "versions" versions->package-revisions)) + +;; TODO: Support other registries +(define %registry "https://registry.npmjs.org") +(define %default-page "https://www.npmjs.com/package") + +(define (lookup-meta-package name) + (let ((json (json-fetch (string-append %registry "/" (uri-encode name))))) + (and=> json json->meta-package))) + +(define lookup-meta-package* (memoize lookup-meta-package)) + +(define (meta-package-versions meta) + (map package-revision-version + (meta-package-revisions meta))) + +(define (meta-package-latest meta) + (and=> (meta-package-dist-tags meta) dist-tags-latest)) + +(define* (meta-package-package meta #:optional + (version (meta-package-latest meta))) + (match version + ((? semver?) (find (lambda (revision) + (semver=? version (package-revision-version revision))) + (meta-package-revisions meta))) + ((? string?) (meta-package-package meta (string->semver version))) + (_ #f))) + +(define* (semver-latest svs #:optional (svr *semver-range-any*)) + (find (cut semver-range-contains? svr <>) + (sort svs semver>?))) + +(define* (resolve-package name #:optional (svr *semver-range-any*)) + (let ((meta (lookup-meta-package* name))) + (and meta + (let* ((version (semver-latest (or (meta-package-versions meta) '()) svr)) + (pkg (meta-package-package meta version))) + pkg)))) + + +;;; +;;; Converting packages +;;; + +(define (hash-url url) + "Downloads the resource at URL and computes the base32 hash for it." + (bytevector->nix-base32-string (port-sha256 (http-fetch url)))) + +(define (npm-name->name npm-name) + "Return a Guix package name for the npm package with name NPM-NAME." + (define (clean name) + (string-map (lambda (chr) (if (char=? chr #\/) #\- chr)) + (string-filter (negate (cut char=? <> #\@)) name))) + (guix-name "node-" (clean npm-name))) + +(define (name+version->symbol name version) + (string->symbol (string-append name "-" version))) + +(define (package-revision->symbol package) + (let* ((npm-name (package-revision-name package)) + (version (semver->string (package-revision-version package))) + (name (npm-name->name npm-name))) + (name+version->symbol name version))) + +(define (npm-package->package-sexp npm-package) + "Return the `package' s-expression for an NPM-PACKAGE." + (define resolve-spec + (match-lambda + (($ name version) + (resolve-package name (string->semver-range version))))) + + (if (package-revision? npm-package) + (let ((name (package-revision-name npm-package)) + (version (package-revision-version npm-package)) + (home-page (package-revision-home-page npm-package)) + (dependencies (package-revision-dependencies npm-package)) + (dev-dependencies (package-revision-dev-dependencies npm-package)) + (peer-dependencies (package-revision-peer-dependencies npm-package)) + (license (package-revision-license npm-package)) + (description (package-revision-description npm-package)) + (dist (package-revision-dist npm-package))) + (let* ((name (npm-name->name name)) + (url (dist-tarball dist)) + (home-page (if (string? home-page) + home-page + (string-append %default-page "/" (uri-encode name)))) + (synopsis description) + (resolved-deps (map resolve-spec + (append dependencies peer-dependencies))) + (peer-names (map versioned-package-name peer-dependencies)) + ;; lset-difference for treating peer-dependencies as dependencies, + ;; which leads to dependency cycles. lset-union for treating them as + ;; (ignored) dev-dependencies, which leads to broken packages. + (dev-names + (lset-union string= + (map versioned-package-name dev-dependencies) + peer-names)) + (extra-phases + (match dev-names + (() '()) + ((dev-names ...) + `((add-after 'patch-dependencies 'delete-dev-dependencies + (lambda _ + (delete-dependencies '(,@(reverse dev-names)))))))))) + (values + `(package + (name ,name) + (version ,(semver->string (package-revision-version npm-package))) + (source (origin + (method url-fetch) + (uri ,url) + (sha256 (base32 ,(hash-url url))))) + (build-system node-build-system) + (arguments + (list + #:tests? #f + #:phases + #~(modify-phases %standard-phases + (delete 'build) + ,@extra-phases))) + ,@(match dependencies + (() '()) + ((dependencies ...) + `((inputs + (list ,@(map package-revision->symbol resolved-deps)))))) + (home-page ,home-page) + (synopsis ,synopsis) + (description ,description) + (license ,license)) + (map (match-lambda (($ name version) + (list name (semver->string version)))) + resolved-deps)))) + (values #f '()))) + + +;;; +;;; Interface +;;; + +(define npm-binary->guix-package + (lambda* (name #:key (version *semver-range-any*) #:allow-other-keys) + (let* ((svr (match version + ((? string?) (string->semver-range version)) + (_ version))) + (pkg (resolve-package name svr))) + (npm-package->package-sexp pkg)))) + +(define* (npm-binary-recursive-import package-name #:key version) + (recursive-import package-name + #:repo->guix-package (memoize npm-binary->guix-package) + #:version version + #:guix-name npm-name->name)) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index 1f34cab088..d724f2bca3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -49,7 +49,7 @@ (define %standard-import-options '()) (define importers '("gnu" "pypi" "cpan" "hackage" "stackage" "egg" "elpa" "gem" "go" "cran" "crate" "texlive" "json" "opam" - "minetest" "elm" "hexpm" "composer")) + "minetest" "elm" "hexpm" "composer" "npm-binary")) (define (resolve-importer name) (let ((module (resolve-interface diff --git a/guix/scripts/import/npm-binary.scm b/guix/scripts/import/npm-binary.scm new file mode 100644 index 0000000000..4a4daa7945 --- /dev/null +++ b/guix/scripts/import/npm-binary.scm @@ -0,0 +1,114 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014 David Thompson +;;; Copyright © 2018 Ricardo Wurmus +;;; Copyright © 2019 Timothy Sample +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts import npm-binary) + #:use-module (guix ui) + #:use-module (guix utils) + #:use-module (guix scripts) + #:use-module (guix import npm-binary) + #:use-module (guix scripts import) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-11) + #:use-module (srfi srfi-37) + #:use-module (srfi srfi-41) + #:use-module (ice-9 match) + #:use-module (ice-9 format) + #:export (guix-import-npm-binary)) + + +;;; +;;; Command-line options. +;;; + +(define %default-options + '()) + +(define (show-help) + (display (G_ "Usage: guix import npm-binary PACKAGE-NAME [VERSION] +Import and convert the npm package PACKAGE-NAME using the +`node-build-system' (but without building the package from source).")) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -r, --recursive import packages recursively")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + ;; Specification of the command-line options. + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix import npm-binary"))) + (option '(#\r "recursive") #f #f + (lambda (opt name arg result) + (alist-cons 'recursive #t result))) + %standard-import-options)) + + +;;; +;;; Entry point. +;;; + +(define (guix-import-npm-binary . args) + (define (parse-options) + ;; Return the alist of option values. + (args-fold* args %options + (lambda (opt name arg result) + (leave (G_ "~A: unrecognized option~%") name)) + (lambda (arg result) + (alist-cons 'argument arg result)) + %default-options)) + (let* ((opts (parse-options)) + (args (filter-map (match-lambda + (('argument . value) + value) + (_ #f)) + (reverse opts)))) + (let loop ((args args)) + (match args + ((package-name version) + (match (if (assoc-ref opts 'recursive) + ;; Recursive import + (npm-binary-recursive-import package-name #:version version) + ;; Single import + (npm-binary->guix-package package-name #:version version)) + ((or #f '()) + (leave (G_ "failed to download meta-data for package '~a@~a'~%") + package-name version)) + ((? list? sexps) + (map (match-lambda + ((and ('package ('name name) ('version version) . rest) pkg) + `(define-public ,(name+version->symbol name version) + ,pkg)) + (_ #f)) + sexps)) + (sexp (list sexp)))) + ((package-name) + (loop (list package-name "*"))) + (() + (leave (G_ "too few arguments~%"))) + ((many ...) + (leave (G_ "too many arguments~%"))))))) diff --git a/tests/npm-binary.scm b/tests/npm-binary.scm new file mode 100755 index 0000000000..cf85e572b3 --- /dev/null +++ b/tests/npm-binary.scm @@ -0,0 +1,146 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2024 Jelle Licht +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . +(define-module (test-npm-binary) + #:use-module ((gcrypt hash) + #:select ((sha256 . gcrypt-sha256))) + #:use-module (guix import npm-binary) + #:use-module (guix base32) + #:use-module (guix tests) + #:use-module (srfi srfi-64) + #:use-module (ice-9 iconv) + #:use-module (ice-9 match) + #:export (run-test)) + +(define foo-json + "{ + \"name\": \"foo\", + \"dist-tags\": { + \"latest\": \"1.2.3\", + \"next\": \"2.0.1-beta4\" + }, + \"description\": \"General purpose utilities to foo your bars\", + \"homepage\": \"https://github.com/quartz/foo\", + \"repository\": \"quartz/foo\", + \"versions\": { + \"1.2.3\": { + \"name\": \"foo\", + \"description\": \"General purpose utilities to foo your bars\", + \"version\": \"1.2.3\", + \"author\": \"Jelle Licht \", + \"devDependencies\": { + \"node-megabuilder\": \"^0.0.2\" + }, + \"dependencies\": { + \"bar\": \"^0.1.0\" + }, + \"repository\": { + \"url\": \"quartz/foo\" + }, + \"homepage\": \"https://github.com/quartz/foo\", + \"license\": \"MIT\", + \"dist\": { + \"tarball\": \"https://registry.npmjs.org/foo/-/foo-1.2.3.tgz\" + } + } + } +}") + +(define bar-json + "{ + \"name\": \"bar\", + \"dist-tags\": { + \"latest\": \"0.1.2\" + }, + \"description\": \"Core module in FooBar\", + \"homepage\": \"https://github.com/quartz/bar\", + \"repository\": \"quartz/bar\", + \"versions\": { + \"0.1.2\": { + \"name\": \"bar\", + \"description\": \"Core module in FooBar\", + \"version\": \"0.1.2\", + \"author\": \"Jelle Licht \", + \"repository\": { + \"url\": \"quartz/bar\" + }, + \"homepage\": \"https://github.com/quartz/bar\", + \"license\": \"MIT\", + \"dist\": { + \"tarball\": \"https://registry.npmjs.org/bar/-/bar-0.1.2.tgz\" + } + } + } +}") + +(define test-source-hash + "") + +(define test-source + "Empty file\n") + +(define have-guile-semver? + (false-if-exception (resolve-interface '(semver)))) + +(test-begin "npm") + +(unless have-guile-semver? (test-skip 1)) +(test-assert "npm-binary->guix-package" + (mock ((guix http-client) http-fetch + (lambda* (url #:rest _) + (match url + ("https://registry.npmjs.org/foo" + (values (open-input-string foo-json) + (string-length foo-json))) + ("https://registry.npmjs.org/bar" + (values (open-input-string bar-json) + (string-length bar-json))) + ("https://registry.npmjs.org/foo/-/foo-1.2.3.tgz" + (set! test-source-hash + (bytevector->nix-base32-string + (gcrypt-sha256 (string->bytevector test-source "utf-8")))) + (values (open-input-string test-source) + (string-length test-source)))))) + (match (npm-binary->guix-package "foo") + (`(package + (name "node-foo") + (version "1.2.3") + (source (origin + (method url-fetch) + (uri "https://registry.npmjs.org/foo/-/foo-1.2.3.tgz") + (sha256 + (base32 + ,test-source-hash)))) + (build-system node-build-system) + (arguments + (list #:tests? #f + #:phases + (gexp (modify-phases %standard-phases + (delete 'build) + (add-after 'patch-dependencies 'delete-dev-dependencies + (lambda _ + (delete-dependencies '("node-megabuilder")))))))) + (inputs (list node-bar-0.1.2)) + (home-page "https://github.com/quartz/foo") + (synopsis "General purpose utilities to foo your bars") + (description "General purpose utilities to foo your bars") + (license license:expat)) + #t) + (x + (pk 'fail x #f))))) + +(test-end "npm") base-commit: 4d79a9cd6b5f0d8c5afbab0c6b70ae42740d5470 -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 01 01:14:28 2024 Received: (at 62375) by debbugs.gnu.org; 1 Apr 2024 05:14:28 +0000 Received: from localhost ([127.0.0.1]:49289 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrA03-0007cM-MA for submit@debbugs.gnu.org; Mon, 01 Apr 2024 01:14:28 -0400 Received: from mail-vi1eur05olkn2085.outbound.protection.outlook.com ([40.92.90.85]:9057 helo=EUR05-VI1-obe.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rr3H9-0007Jw-P2 for 62375@debbugs.gnu.org; Sun, 31 Mar 2024 18:03:41 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XIK7bcgva+k48NX3rnJWIxIVjJwv+iv173vOhagKiT5fMKWJ49pg4zgY0h+hyAgn6Oz9HYXhp9WlEOuOsbFmnifJFflrTpWhBBSECFfrW1xSYabeAIhGMZjYRoGotpDLq0maGPW1lN+LZ2kWEd9P0QxAGChiYOCmlDjR6fm6ftS+XL2PIqlkykl0G7K7iI3bh6lHCSVuCjjAYDpS/GPCbbiycOJPp+0w3gkrrTDtVO1nhE6OrPy7R0CzyVLDmjWfQFBol9SoBuqShwEOayBPYMGdxhOEt8pL/W6i36vSN7mJx98rngVwZx7V1itmaqcuL5M8qZKmp5kdgVw747Vr4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IN/kPVBqyn16hgx0wpItjgXSWkwOqVJtDqN9YdSnnGs=; b=AK75QO629o/27yiWIdakjEx8ffJStEpfZlqJp1WVgBxLIcHY125VTvUhqBMexJs0wjI6RS+KcCcdvWSFBRtYUf+DhVSZtebab9gLWejw6JEvRHYfQn6xyhHfWxlh6uTrQpwBDkiDZkqbrKo4c4ciZeu0fqUm7yiwEQEHYhj0AHhMgVTepRANqtltQTm9tWNsKfEB+RkXg02+Z0ZOunFGXRlc0ARbNUpRozLfrX+F4sLgM6jSDSuOKgqZtzz70h7INzNXx6qVKYemZ4d5y9kpqmTruEBFOhUUow+eq+X3YLrZlEzBYGuKhN5MhQnZL2eW8AplDu6nro66CcLXRAUN4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=OUTLOOK.FR; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IN/kPVBqyn16hgx0wpItjgXSWkwOqVJtDqN9YdSnnGs=; b=cSmGbYxaCaVFvmMdN/DlOBRBOC/3+5iSs5kznj93UtGgQ6/ytl3uMNgfu8Xez/AT4adkJelF8I9L3AacS8XC/JDEMNkp3gOgPhG4FuHzY+JS8wPI8u9N961zHjoohZWCKPJIeMymM+6rU+deC95PqznRtT6HFR7Z1WqX4VG9dJlnM01IMT5yxbs13VPPpwFM9xuLWWvAsQWSFvPL8TZJcDJw6JBGiX527s5LONGsz4dagAuDjn/NX/OuNN1Wy6xkMxqqTLAcEEAlFsxvB8ISPksQVySfH0H7ViKfEoBTrhEuYMK2cIQ8gUZ8sAdDC83Iu10sQZtb7g4EG4S64sqLyg== Received: from AS8P189MB2024.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:52b::6) by VE1P189MB1151.EURP189.PROD.OUTLOOK.COM (2603:10a6:800:164::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7409.46; Sun, 31 Mar 2024 22:03:30 +0000 Received: from AS8P189MB2024.EURP189.PROD.OUTLOOK.COM ([fe80::bd04:a484:76b3:a5f1]) by AS8P189MB2024.EURP189.PROD.OUTLOOK.COM ([fe80::bd04:a484:76b3:a5f1%7]) with mapi id 15.20.7409.042; Sun, 31 Mar 2024 22:03:29 +0000 From: Pablo Zamora To: "62375@debbugs.gnu.org" <62375@debbugs.gnu.org> Subject: RE: [bug#62375] Continue the npm-binary importer Thread-Topic: [bug#62375] Continue the npm-binary importer Thread-Index: AQHaffo6iAyc71SpuU+Hym+lQboSTbFST0OAgAAh3kI= Date: Sun, 31 Mar 2024 22:03:29 +0000 Message-ID: References: <8734s6gooc.fsf@fsfe.org> In-Reply-To: <8734s6gooc.fsf@fsfe.org> Accept-Language: fr-FR, en-US Content-Language: fr-FR X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: x-tmn: [FDTQrP3qYGKUBrUmymgAHDeiRyy+AMXFdnyX6ZaMmvDoaUs/Ybgf0crHVWGLwBtvq+riD3VuxQE=] x-ms-publictraffictype: Email x-ms-traffictypediagnostic: AS8P189MB2024:EE_|VE1P189MB1151:EE_ x-ms-office365-filtering-correlation-id: dbe19fab-c397-42ff-cf52-08dc51ce661c x-ms-exchange-slblob-mailprops: 02NmSoc12DedRvKN5/4QiK8NHQIQU0k8hI2PoDzU/a36pV4d8eFtfe2Id3c+jii0pskjllp0rpb8AZLDpO0rOmI4PQySPFh5Q6ci9eBzT8AalFp2//ZVaZCMekNb4gWHUGxYWPx7c0av1ftWiBJ2MxUseEWANnHij3GnZKPSgEPmm4zvAgfgeO1jOqFH2Mb1x3HnhJxYepI5uSeYEtqop2YMPKzy51Et7HidG+Ak33O/r+P+wT/GCAvslUEVh2W/g/UQnuDtZMKp55TYptntTz4jgtYMdZjAk3vDivxzClZiyvWdPCuxzPGNuwWpMUg23v2nzgPh4af+XE7wz761il3JFnVJLBkTKEO3P8vNXI4syTWw+jobaufYtLRkcM/t8moyvHKKXp9T9aepH0kIEFRIXK7QSDDSjBUrW9525yjZasDnT1MDyrhuO47dKeItntUobWeVkkXOqxyYta0JeDUX7oJ9EOpY73fZgYnI8QkLdJL9RjcPb/og6FUz5m8ENChcJFTAk2nTJo3fFRQ/XBAXrixIMtMvN0EBgIF4yb8c3X970YLrugf5u4wyWQnGgdKwAJXfe/ZqRUC0TshtLyvM3KEZLb9OUR4rOhPldNbeTzv1YQZLr574wVU8Z0+ChYxshh5nIq2F9dpo5zvsbtoTAj7rx55Li3ZYQ1o6gbvjaP4rQ4tUFSQbRJnrQOd6ZKYlXbaGqXsytBz615gxcyir00/viQbQHQWNxdQ0cC3Su7PE3NtoV9RRj+nvwoiK x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: uJ4Kh2Wm8xP8jA0dfFt2pkug32pd2pYWbt4w8MTXkOqoTcG8/KbMORioLvQUbhnHYXPtnXJSgXaO6ngTbefTXRsJyzbXNwrRsbvwkrJYw3ehFXE7sDIlNJuR+XqIh/bykdCmKMDpHyd0CcLxJNE36vJfWcMfT7ZmaST5wQHIxE8LuMheJ896yqV5WuCKzTSskNM+ilaK3+eDZ2qVrF+Nu8RR7J86eyoevY1a5GBRvDrpIxk6bJQ1EZ9Cqv5Wko3wvx57cWUnPu8e4nW2wBR6v+blI9Hge7ieJ4qnIxylCjw0ypM9g76iXDkzBE/8pajUzP1ql3HFMt1hi9T/PxR5PysHxDncVyUk1Vi0k17z0gU+cI5NwcI9w7XdSl+O3Ie6pubhYaV0/ITUrJzvobKBgPb5HttLZp/GgdWmvQBavjKKOimifa+56D82JD75iuwODn39fE4+qURqVKQ2SvJG3E8dLd1lrP1UOn0b8bN+B1XljHqPynOG9tayLvHUxTsfEiJbsR7QrTuYFabhgx9gKdIjMjE3uxKaIguSUEkphlZeqBSOZkIQ6m4c4bIp6wJ6SeiDkAguKlkkPIrioE/KWqIMb5D6iFnS607DsWdXZMan8VkB3q3HiFKwYym3WuYM+0mXdp+WAr9r92pX2TSjhoPFdUViAnqJOiLOLoHUWV+nTBlIe8TVFdxi2+6FC0jKb2WZVU8MytaB8XtxWr0WEDVODlYdt9En8MjfwoWrwF8= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?D3v60TZrctGidVcszUlHBRAcuyGL6TXO0sbX/RaBxr4pL3LRerWVm39Sf6?= =?iso-8859-1?Q?YcvX2VItNU3ob26psv6d2hYRP/hZRKukTQyomzmDf/Qx/61IVFV6Gmc2U0?= =?iso-8859-1?Q?cZY2W7ZAKJUfluSBHIRKHTOz3oVduwz8E6QKi3fNvrQ1TosnrwK7ZvsdEh?= =?iso-8859-1?Q?c4UGyTCVglRZ69KQG44HHQ6Kh63pahb6+EzrnSLuITY6nS+qe2mqJ8ZYV6?= =?iso-8859-1?Q?OVKX+rEWwgZCIY5vIx/XFiA+KR5RnTYcP4ClriUOLWxMrNL58DIEJX7Vwq?= =?iso-8859-1?Q?TvgDe9IzIJNbGMwqmGF+nCMkr51HjuKJACpi5eLHEt+GvGPEfF/yH1s+G9?= =?iso-8859-1?Q?F6Wgy83GcHX7TuCUZ9E5lO5VUoqTFn8Q9po8jUiwf67BKrFYcbYNlHmUoZ?= =?iso-8859-1?Q?AaKRhDCIm4ZaXDS+CLtdfcyZQq1RBU6R2TaJprMyUiEbuX6Acn0bdnDTSR?= =?iso-8859-1?Q?kRhh+xisRofu740LWPoHQgvBXC6t7OJqsCcLHZ+gcLPZIOmifcHMafvVmC?= =?iso-8859-1?Q?KiigDNrRRNaHkf8bU33aL6nM2lo4cZHiwCeYHkUV7mu3whE9ZwUeHmU0mf?= =?iso-8859-1?Q?lWxla5YcHFPATSCKuZwQ49nM95p8sFZ69GonGZVwvIPrAjIY6hjykEiT+U?= =?iso-8859-1?Q?MkonZwKptkG/+aapbuc37gC1GoltE6FS58E20BSvbLQnR6NJ7ypzzH8tFe?= =?iso-8859-1?Q?HCW2d5z+ho+FwDNpPzCh3vmw/tLqbppEsuCN7LQvoZn9qwPENcoWKAqqw5?= =?iso-8859-1?Q?G/m1brjFtH3+Td1etPjF0pjUKhlgAL7df0MzRgw9t738dohb1rDwBAeoiW?= =?iso-8859-1?Q?etFja+6EfSgwj34OOBctwqHt0dH2oTXHfzHPwGPTkmWyjCdldB9umSixeY?= =?iso-8859-1?Q?b5TTy0fLTRs7SGJcgHRNUoPIIZVY4MAsV9TAZrDde89zKoU4i1oQjJdOL/?= =?iso-8859-1?Q?hKW/eLKT0J8xNCRCpAIdgzA06KS1ffTecn8mVBy49deqvuj2OFsORCdI8a?= =?iso-8859-1?Q?QWeLHP0/imXafS3UWj16xGOcMyffHOI+pz0wSbw5PawABF7qwka/4HnpUU?= =?iso-8859-1?Q?IAX0Yq0nc7EYS5SHrBrC7Lr1NSCVbrRmsCxSA8euFmcBq46JShZiEwcpWq?= =?iso-8859-1?Q?AoEUjPbZFDU88BHR3eDPO9fyLYXSbgbCDMjbtiTb6XWVSbTtIKHpLCc6EE?= =?iso-8859-1?Q?uWVReGODNLLovuYLofVEZBxwSb6Ep+xKaoQpjp1P+Sk11z4itDFPaTYdFa?= =?iso-8859-1?Q?sF0PXBldhwklOkyOg/xXDmi1KmuvqprdpA7P/QeNGWDSV993DNTSC2yp62?= =?iso-8859-1?Q?Qo7tHFIef0WXhE7oeTC152gZPA9bWZEdEEN70khVs7tCPb4=3D?= Content-Type: multipart/alternative; boundary="_000_AS8P189MB20245595664C8E10AA652B7EFD382AS8P189MB2024EURP_" MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-4734-24-msonline-outlook-c54b5.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AS8P189MB2024.EURP189.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: dbe19fab-c397-42ff-cf52-08dc51ce661c X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Mar 2024 22:03:29.7926 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1P189MB1151 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 62375 X-Mailman-Approved-At: Mon, 01 Apr 2024 01:14:24 -0400 Cc: Jelle Licht X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --_000_AS8P189MB20245595664C8E10AA652B7EFD382AS8P189MB2024EURP_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Dear Jelle, Thank you very much for your help. I will shortly (this week) take a thorough look on your patches so I think = of choosing '2)', even temporarily in order to become accustomed on how to = develop a Guix importer. Kind regards, Pablo Zamora ________________________________ De : Jelle Licht Envoy=E9 : dimanche 31 mars 2024 21:57 =C0 : Pablo Zamora ; 62375@debbugs.gnu.org <62375@= debbugs.gnu.org> Objet : Re: [bug#62375] Continue the npm-binary importer Hi Pablo, Pablo Zamora writes: > Dear Guix, > > I am stuck while trying to define the 'bitwarden' package definition on m= y personal channel here https://github.com/Pablo12345678901/guix-custom-cha= nnel/blob/master/dev/my-bitwarden.scm . I am facing issues due to the npm d= ependencies. > > One of the potential solution would be this importer. I would like to fin= ish writing it and perform the tests. That is nice to hear! I just sent v2 of this patch addressing most of the shortcomings that were identified earlier. > On which branch should I send git pull request to continue what has been = done so far ? This would be my first commit with guix. I see two ways forward, depending on how much work you foresee and/or are willing to do: 1) This patch gets through another round of review, and afterwards you can send patches based on the 'master' branch. 2) You work on improving this patch while it is still under review, and send a V{3,4,5} patch as a reply to this very thread. Kind regards, Jelle Licht --_000_AS8P189MB20245595664C8E10AA652B7EFD382AS8P189MB2024EURP_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Dear Jelle,

Thank you very much for your help.

I will shortly (this week) take a thorough look on your patches so I think = of choosing '2)', even temporarily in order to become accustomed on how to = develop a Guix importer.

Kind regards,

Pablo Zamora
De : Jelle Licht <jlicht= @fsfe.org>
Envoy=E9 : dimanche 31 mars 2024 21:57
=C0 : Pablo Zamora <pablo.zamora@outlook.fr>; 62375@debbugs.gn= u.org <62375@debbugs.gnu.org>
Objet : Re: [bug#62375] Continue the npm-binary importer
 

Hi Pablo,

Pablo Zamora <pablo.zamora@outlook.fr> writes:

> Dear Guix,
>
> I am stuck while trying to define the 'bitwarden' package definition o= n my personal channel here https://github.com/Pablo12345678901/guix-custom-channel/blob/master/dev/my-= bitwarden.scm . I am facing issues due to the npm dependencies.
>
> One of the potential solution would be this importer. I would like to = finish writing it and perform the tests.

That is nice to hear! I just sent v2 of this patch addressing most of
the shortcomings that were identified earlier.

> On which branch should I send git pull request to continue what has be= en done so far ? This would be my first commit with guix.

I see two ways forward, depending on how much work you foresee and/or
are willing to do:

1) This patch gets through another round of review, and afterwards you
can send patches based on the 'master' branch.

2) You work on improving this patch while it is still under review, and
send a V{3,4,5} patch as a reply to this very thread.

Kind regards,
Jelle Licht
--_000_AS8P189MB20245595664C8E10AA652B7EFD382AS8P189MB2024EURP_-- From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 01 16:41:43 2024 Received: (at 62375) by debbugs.gnu.org; 1 Apr 2024 20:41:43 +0000 Received: from localhost ([127.0.0.1]:51786 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrOTP-0007kQ-0q for submit@debbugs.gnu.org; Mon, 01 Apr 2024 16:41:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36652) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrOTK-0007kA-VK for 62375@debbugs.gnu.org; Mon, 01 Apr 2024 16:41:42 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rrOTB-0001fG-IU; Mon, 01 Apr 2024 16:41:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=R4Tem82kgGMfIipNSisQle6aBWMQ2g/6vhOpbfTZrNQ=; b=VgbgmZ51HQ6sMtUK1b7l TIT2uZ0VujYeyW01Bt+zIdUGDIbmenDLEYCHhBZX/le+ZQv1SRE900geDL36VlaGXRvs2X1vijcVp uTTrVLBn68rJUcux9P8a0iMOWA65w2jEo4wrmlhxK5n1oDA+XVuv3WkXE+reNtcdJ5HwkSfE+EPPn FY8lD3hOFB1BMs8vG8qYGjQIL2rg43FIoS5gA3JQE2dPjjxqWcMpOixRWK6hcf4TWjkhYuJGtOMrB ecKBoKeQKe3+bZYqjrqv7XfVISTa5Ot5DyPE5xMo9EtJDFZVjDKqxwVNcIDcZ1qv2ozh1UDQpVz/t hODbQ0r/e02bJQ==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: jlicht@fsfe.org Subject: Re: [bug#62375] [PATCH v3] import: Add binary npm importer. In-Reply-To: <7148a74f04e1598a18b58fbd0b1b27a2397312b2.1711917305.git.jlicht@fsfe.org> (jlicht@fsfe.org's message of "Sun, 31 Mar 2024 22:37:01 +0200") References: <7148a74f04e1598a18b58fbd0b1b27a2397312b2.1711917305.git.jlicht@fsfe.org> Date: Mon, 01 Apr 2024 22:41:26 +0200 Message-ID: <871q7odde1.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Josselin Poiret , Simon Tournier , Mathieu Othacehe , Tobias Geerinckx-Rice , Lars-Dominik Braun , 62375@debbugs.gnu.org, Christopher Baines , Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! jlicht@fsfe.org skribis: > From: Jelle Licht > > * guix/scripts/import.scm: (importers): Add "npm-binary". > * guix/import/npm-binary.scm: New file. > * guix/scripts/import/npm-binary.scm: New file. > * Makefile.am: Add them. > > Co-authored-by: Timothy Sample > Co-authored-by: Lars-Dominik Braun > > Change-Id: I98a45068cf5b9c42790664cc743feaa7ac76f807 Yay! > +The npm-binary importer also allows you to specify a version string: > + > +@example > +guix import npm-binary buffer-crc32 1.0.0 > +@end example For consistency with other importers (pypi, gem, cran), could you change the syntax to: guix import npm-binary buffer-crc32@1.0.0 ? That=E2=80=99s the last remaining issue for me. > +;; TODO: Support other registries > +(define %registry "https://registry.npmjs.org") For the purposes of tests, you could make it: (define %npm-registry (make-parameter "https://registry.npmjs.org")) That would allow you to write tests using =E2=80=98with-http-server=E2=80= =99 and =E2=80=98parameterize=E2=80=99 as done in =E2=80=98tests/pypi.scm=E2=80=99 = and others, which I find nicer and more robust than =E2=80=98mock=E2=80=99. Not a blocker though. Thanks! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 01 18:01:53 2024 Received: (at 62375) by debbugs.gnu.org; 1 Apr 2024 22:01:54 +0000 Received: from localhost ([127.0.0.1]:51893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrPiz-0001ge-JZ for submit@debbugs.gnu.org; Mon, 01 Apr 2024 18:01:53 -0400 Received: from mout.web.de ([212.227.15.14]:39085) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrPiw-0001gQ-Jp for 62375@debbugs.gnu.org; Mon, 01 Apr 2024 18:01:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1712008900; x=1712613700; i=jonathan.brielmaier@web.de; bh=DRfY2aWLC5m2p2a/+WWG2JPzEE15uqGAmY0A0aRG+Ds=; h=X-UI-Sender-Class:Date:To:Subject:From; b=EnzPL0q7mPASXwmIzIuLIHk7J9CRljPZx89MXW3ZB8bxzwoTCChldYZfn+paUSBS 0YCbYdsNpIaBDFZRsN+A8o7WlQsIf2JOeTGfFuGHX8xvzREq4n18T2iEu8RIXJlH5 nVYDFFTU6ze+X28BcdXCRyTbnHx0wwP0cwtuL8Vqec4No2SkdGgmcbb/JyvTarwb4 mp5nP+FM2e0RbNEqyxE4KqMJr+5egrBbvgHYOGI7IKJ1csmfziO0JWn1YBTP03ws7 jf0EoITDsTFGB15ssMyaaVW5Qcmnq0yVXVsMvYEBUq61U0JLnEKcQzGHkainW8J/t 9YxrX9oOgiHBe3AQKw== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from [192.168.178.29] ([77.189.203.8]) by smtp.web.de (mrweb005 [213.165.67.108]) with ESMTPSA (Nemesis) id 1MOm0x-1sBYk71ndn-00YCDe for <62375@debbugs.gnu.org>; Tue, 02 Apr 2024 00:01:40 +0200 Message-ID: Date: Tue, 2 Apr 2024 00:01:39 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 To: 62375@debbugs.gnu.org Subject: [PATCH 0/1] npm binary importer Content-Language: de-DE, en-US From: Jonathan Brielmaier Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:K/4mVfydVToQDGHCYNQoKq5N55U5T5zjV3GC/kMHvPLl9/xhMYn zzQh1sCmyf8pSdZ6JtSSC+qt/MZOz/uzt7xOXXvm2xNpkU3Ai/tMyxxauqrDZAJTQiQ8Xyy fg5S9cpB/CXLt8Qj4K+KLAt66zKbd31gTL6oxWcb1TQ5SVE1PDEEXLNPWoXjL7doPMmwQ95 CcoxM4rw0+nIULzmENrng== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:byIA8qN/wig=;pUX6buW8tV8JBo5/bnMRhggjZcG j2h9zbjdRTVeZOY3I9EIzsQPf1D2wafCH6liuV10UmjN8uD6EL5RbsACKP08/0NRJo9QaGuVh a6Vp0haLe8hK1OUuJCVj1UpJUSAvS1aUnGs6jLSw/BcoKDIINBKSSdN/khHbqKIPtFQCRNmt6 MtON87yGCaCxcpFT9eft9Vne+QMVzMy7Px48Z34HoU5FleQpQzpOMf741vyuOqAzEt7vKfZfl fgdZJP0Pn0YKhRsXc6UPZRTnZtmJldOBk63EPgXkKeeoB3y1KTVe/WsBiNBUuZfeu3/yqZkQf iIjHETkxRluk1kUmVLTIRjKAFSerlLnnkTj15gidZ3PaLdKUT8I3A8FoRxnVjn/4aJSl7ZfMV /PmeW0v8N3kKl2LOUmNEcGe/uwKGINmV3yuzoNwRnC5frm9n2aVeQcdib09aCMgsLEgRxNACI yd6zpSCmykohtHSoXdz5Xe6fRBA+S82sohkQGN0PnI8qktVon8vifRKW545LISGM5O01B7Flj 2nVno439n62FLb1oMZWpFASd4tuXCCIbDzBragSxbOoZoWle1daJATiVUp7GYuVe0mAOXZ0r3 3+dkGO/Ain1e1SWHodNLVvsjzNwe3qlzgDVtRY8Ba8sjb/gNOhyWwAECfyb7g0kpTLWX+DTxn 6+AhBgbEdj25wAWCBRIwS7vyiJdxZmAM6nOMoXlpo/0gpcF6+62WjrxttvRmbqB5rD0c/3YnT NICUq0mzvWT3lCOS1UkLPUcN4wrJofKzSoxZDw+T5t4jVoB4QpSDps0enDTHjyddBSQAgZqLb M46o9d5UUaf6pkSffcPzo+UZ+pmzHbYmatp7azaDBnx+g= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 62375 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Jelle, one rather unimportant remark. The commit message mentions Lars-Dominik as co-author: https://issues.guix.gnu.org/62375#12-lineno8 But there is no copyright header for him in any of the added files... ~Jonathan From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 02 10:12:17 2024 Received: (at 62375) by debbugs.gnu.org; 2 Apr 2024 14:12:18 +0000 Received: from localhost ([127.0.0.1]:54840 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rres4-000744-9J for submit@debbugs.gnu.org; Tue, 02 Apr 2024 10:12:17 -0400 Received: from mail1.fsfe.org ([2001:aa8:ffed:f5f3::151]:43980) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rres0-00073h-Q2 for 62375@debbugs.gnu.org; Tue, 02 Apr 2024 10:12:14 -0400 From: Jelle Licht DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1712067125; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KpVz2XsfaT2Unpb54ux8MnLUUnHAkH17qk2c8YDvvC4=; b=rTx2i14SYn/dVXSzx99o0DNjGRR+OXYGe+Jlbu3a4QW8slxlN6X6HgHAjAVc4DHA2lEuxj xYMBausFEBh7QgTid9tY5I86jEm336lSlhUhrl+gQL6622qHO+ZL+W14kmUwqcVxT9nKfe b0yAet8qoQDpe2ThdfuA4nu/qPApaEo= To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: [bug#62375] [PATCH v3] import: Add binary npm importer. In-Reply-To: <871q7odde1.fsf@gnu.org> References: <7148a74f04e1598a18b58fbd0b1b27a2397312b2.1711917305.git.jlicht@fsfe.org> <871q7odde1.fsf@gnu.org> Date: Tue, 02 Apr 2024 16:12:01 +0200 Message-ID: <87r0fng8ge.fsf@fsfe.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Josselin Poiret , Simon Tournier , Mathieu Othacehe , Tobias Geerinckx-Rice , Lars-Dominik Braun , 62375@debbugs.gnu.org, Christopher Baines , Ricardo Wurmus X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hi Ludo', Thanks for the speedy review! Ludovic Court=C3=A8s writes: > Hi! > > jlicht@fsfe.org skribis: > >> From: Jelle Licht >> >> * guix/scripts/import.scm: (importers): Add "npm-binary". >> * guix/import/npm-binary.scm: New file. >> * guix/scripts/import/npm-binary.scm: New file. >> * Makefile.am: Add them. >> >> Co-authored-by: Timothy Sample >> Co-authored-by: Lars-Dominik Braun >> >> Change-Id: I98a45068cf5b9c42790664cc743feaa7ac76f807 > > Yay! > >> +The npm-binary importer also allows you to specify a version string: >> + >> +@example >> +guix import npm-binary buffer-crc32 1.0.0 >> +@end example > > For consistency with other importers (pypi, gem, cran), could you change > the syntax to: > > guix import npm-binary buffer-crc32@1.0.0 > > ? > > That=E2=80=99s the last remaining issue for me. > I needed some custom logic to support the npm scoped packages ("@linthtml/linthtml@1.2.3"), but it should now work. >> +;; TODO: Support other registries >> +(define %registry "https://registry.npmjs.org") > > For the purposes of tests, you could make it: > > (define %npm-registry > (make-parameter "https://registry.npmjs.org")) Easy! > That would allow you to write tests using =E2=80=98with-http-server=E2=80= =99 and > =E2=80=98parameterize=E2=80=99 as done in =E2=80=98tests/pypi.scm=E2=80= =99 and others, which I find > nicer and more robust than =E2=80=98mock=E2=80=99. The `with-http-server' construct does not play well with an interactive REPL workflow due to not cleaning up the used port in all situations, making that port unavailable for running the test again[1]. `mock', brittle as it may be, does not to suffer from this drawback, so I'd vastly prefer using that for now. > Not a blocker though. > > Thanks! > > Ludo=E2=80=99. I'll send a v4, let it simmer for some days and then merge it if no big blockers or requested changes show up. - Jelle From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 02 10:13:58 2024 Received: (at 62375) by debbugs.gnu.org; 2 Apr 2024 14:13:58 +0000 Received: from localhost ([127.0.0.1]:54845 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rretg-0007GX-Ui for submit@debbugs.gnu.org; Tue, 02 Apr 2024 10:13:58 -0400 Received: from mail1.fsfe.org ([217.69.89.151]:42306) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rretc-0007Fk-3z for 62375@debbugs.gnu.org; Tue, 02 Apr 2024 10:13:56 -0400 From: jlicht@fsfe.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1712067227; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jXwdbhuhNoSQBAk6AXEgF2i/nAfzBHcESPbfNJsZZE4=; b=H7dC3hlQMf5fDBz7N2L3rqRC0tBmLR+SRm5yl5tYrrKC9z/QK4/1jVYj84Sho7dRoG5oCK K/ZavK7UqLip+yWNEeqSlbR+smQRWdeVf8Vb0+5bIH9WAh1GJdjUEY7Tx2q7nm9OrYdLZQ LXmSv4YpLt0Tn2LKC+2idcktGKw9jAA= To: 62375@debbugs.gnu.org Subject: [PATCH v4] import: Add binary npm importer. Date: Tue, 2 Apr 2024 16:13:45 +0200 Message-ID: <1892fd7b000320e56cbf10958fd4cf88d3c83ced.1712067148.git.jlicht@fsfe.org> MIME-Version: 1.0 X-Debbugs-Cc: Christopher Baines , Josselin Poiret , Ludovic Courtès , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 62375 Cc: Timothy Sample , Jelle Licht , Lars-Dominik Braun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) From: Jelle Licht * guix/scripts/import.scm: (importers): Add "npm-binary". * guix/import/npm-binary.scm: New file. * guix/scripts/import/npm-binary.scm: New file. * Makefile.am: Add them. Co-authored-by: Timothy Sample Co-authored-by: Lars-Dominik Braun Change-Id: I98a45068cf5b9c42790664cc743feaa7ac76f807 --- Changes in v4: - Add copyright line for LDB - Use package-name@version-spec notation on CLI - Simplify CLI argument handling - Turn %registry into a parameter named %npm-registry Changes in v3: - Ensure that package bindings generated during recursive import match the package bindings used in the list of inputs Changes in v2: - Change *SOME-VAR* to %SOME-VAR - Removed unused http-error-code - Rebase on master - Refactor hash-url to use port-sha256 helper - use explicit record accessors instead of order-sensitive destructuring - address line-width styling issues - added basic documentation - added some basic tests (using simple mocks instead of with-http-server) - simplify import script entrypoint Makefile.am | 3 + doc/guix.texi | 33 ++++ guix/import/npm-binary.scm | 279 +++++++++++++++++++++++++++++ guix/scripts/import.scm | 2 +- guix/scripts/import/npm-binary.scm | 121 +++++++++++++ tests/npm-binary.scm | 146 +++++++++++++++ 6 files changed, 583 insertions(+), 1 deletion(-) create mode 100644 guix/import/npm-binary.scm create mode 100644 guix/scripts/import/npm-binary.scm create mode 100755 tests/npm-binary.scm diff --git a/Makefile.am b/Makefile.am index 1c5688ac13..459c47a954 100644 --- a/Makefile.am +++ b/Makefile.am @@ -305,6 +305,7 @@ MODULES = \ guix/import/kde.scm \ guix/import/launchpad.scm \ guix/import/minetest.scm \ + guix/import/npm-binary.scm \ guix/import/opam.scm \ guix/import/print.scm \ guix/import/pypi.scm \ @@ -359,6 +360,7 @@ MODULES = \ guix/scripts/import/hexpm.scm \ guix/scripts/import/json.scm \ guix/scripts/import/minetest.scm \ + guix/scripts/import/npm-binary.scm \ guix/scripts/import/opam.scm \ guix/scripts/import/pypi.scm \ guix/scripts/import/stackage.scm \ @@ -557,6 +559,7 @@ SCM_TESTS = \ tests/modules.scm \ tests/monads.scm \ tests/nar.scm \ + tests/npm-binary.scm \ tests/networking.scm \ tests/opam.scm \ tests/openpgp.scm \ diff --git a/doc/guix.texi b/doc/guix.texi index 69a904473c..f8f8f7bf3a 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -14641,6 +14641,39 @@ Invoking guix import in Guix. @end table +@item npm-binary +@cindex npm +@cindex Node.js +Import metadata from the @uref{https://registry.npmjs.org, npm +Registry}, as in this example: + +@example +guix import npm-binary buffer-crc32 +@end example + +The npm-binary importer also allows you to specify a version string: + +@example +guix import npm-binary buffer-crc32@@1.0.0 +@end example + +@quotation Note +Generated package expressions skip the build step of the +@code{node-build-system}. As such, generated package expressions often +refer to transpiled or generated files, instead of being built from +source. +@end quotation + +Additional options include: + +@table @code +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table + @item opam @cindex OPAM @cindex OCaml diff --git a/guix/import/npm-binary.scm b/guix/import/npm-binary.scm new file mode 100644 index 0000000000..6dfedc4910 --- /dev/null +++ b/guix/import/npm-binary.scm @@ -0,0 +1,279 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019, 2020 Timothy Sample +;;; Copyright © 2021 Lars-Dominik Braun +;;; Copyright © 2020, 2023, 2024 Jelle Licht +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix import npm-binary) + #:use-module ((gnu services configuration) #:select (alist?)) + #:use-module (gcrypt hash) + #:use-module (gnu packages) + #:use-module (guix base32) + #:use-module (guix http-client) + #:use-module (guix import json) + #:use-module (guix import utils) + #:use-module (guix memoization) + #:use-module (guix utils) + #:use-module (ice-9 match) + #:use-module (ice-9 receive) + #:use-module (ice-9 regex) + #:use-module (json) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-41) + #:use-module (srfi srfi-9) + #:use-module (web client) + #:use-module (web response) + #:use-module (web uri) + #:export (npm-binary-recursive-import + npm-binary->guix-package + %npm-registry + make-versioned-package + name+version->symbol)) + +;; Autoload Guile-Semver so we only have a soft dependency. +(module-autoload! (current-module) + '(semver) + '(string->semver semver? semver->string semver=? semver>?)) +(module-autoload! (current-module) + '(semver ranges) + '(*semver-range-any* string->semver-range semver-range-contains?)) + +;; Dist-tags +(define-json-mapping make-dist-tags dist-tags? + json->dist-tags + (latest dist-tags-latest "latest" string->semver)) + +(define-record-type + (make-versioned-package name version) + versioned-package? + (name versioned-package-name) ;string + (version versioned-package-version)) ;string + +(define (dependencies->versioned-packages entries) + (match entries + (((names . versions) ...) + (map make-versioned-package names versions)) + (_ '()))) + +(define (extract-license license-string) + (if (unspecified? license-string) + 'unspecified! + (spdx-string->license license-string))) + +(define-json-mapping make-dist dist? + json->dist + (tarball dist-tarball)) + +(define (empty-or-string s) + (if (string? s) s "")) + +(define-json-mapping make-package-revision package-revision? + json->package-revision + (name package-revision-name) + (version package-revision-version "version" ;semver + string->semver) + (home-page package-revision-home-page "homepage") ;string + (dependencies package-revision-dependencies ;list of versioned-package + "dependencies" + dependencies->versioned-packages) + (dev-dependencies package-revision-dev-dependencies ;list of versioned-package + "devDependencies" dependencies->versioned-packages) + (peer-dependencies package-revision-peer-dependencies ;list of versioned-package + "peerDependencies" dependencies->versioned-packages) + (license package-revision-license "license" ;license | #f + (match-lambda + ((? unspecified?) #f) + ((? string? str) (spdx-string->license str)) + ((? alist? alist) + (match (assoc "type" alist) + ((_ . (? string? type)) + (spdx-string->license type)) + (_ #f))))) + (description package-revision-description ;string + "description" empty-or-string) + (dist package-revision-dist "dist" json->dist)) ;dist + +(define (versions->package-revisions versions) + (match versions + (((version . package-spec) ...) + (map json->package-revision package-spec)) + (_ '()))) + +(define (versions->package-versions versions) + (match versions + (((version . package-spec) ...) + (map string->semver versions)) + (_ '()))) + +(define-json-mapping make-meta-package meta-package? + json->meta-package + (name meta-package-name) ;string + (description meta-package-description) ;string + (dist-tags meta-package-dist-tags "dist-tags" json->dist-tags) ;dist-tags + (revisions meta-package-revisions "versions" versions->package-revisions)) + +(define %npm-registry + (make-parameter "https://registry.npmjs.org")) +(define %default-page "https://www.npmjs.com/package") + +(define (lookup-meta-package name) + (let ((json (json-fetch (string-append (%npm-registry) "/" (uri-encode name))))) + (and=> json json->meta-package))) + +(define lookup-meta-package* (memoize lookup-meta-package)) + +(define (meta-package-versions meta) + (map package-revision-version + (meta-package-revisions meta))) + +(define (meta-package-latest meta) + (and=> (meta-package-dist-tags meta) dist-tags-latest)) + +(define* (meta-package-package meta #:optional + (version (meta-package-latest meta))) + (match version + ((? semver?) (find (lambda (revision) + (semver=? version (package-revision-version revision))) + (meta-package-revisions meta))) + ((? string?) (meta-package-package meta (string->semver version))) + (_ #f))) + +(define* (semver-latest svs #:optional (svr *semver-range-any*)) + (find (cut semver-range-contains? svr <>) + (sort svs semver>?))) + +(define* (resolve-package name #:optional (svr *semver-range-any*)) + (let ((meta (lookup-meta-package* name))) + (and meta + (let* ((version (semver-latest (or (meta-package-versions meta) '()) svr)) + (pkg (meta-package-package meta version))) + pkg)))) + + +;;; +;;; Converting packages +;;; + +(define (hash-url url) + "Downloads the resource at URL and computes the base32 hash for it." + (bytevector->nix-base32-string (port-sha256 (http-fetch url)))) + +(define (npm-name->name npm-name) + "Return a Guix package name for the npm package with name NPM-NAME." + (define (clean name) + (string-map (lambda (chr) (if (char=? chr #\/) #\- chr)) + (string-filter (negate (cut char=? <> #\@)) name))) + (guix-name "node-" (clean npm-name))) + +(define (name+version->symbol name version) + (string->symbol (string-append name "-" version))) + +(define (package-revision->symbol package) + (let* ((npm-name (package-revision-name package)) + (version (semver->string (package-revision-version package))) + (name (npm-name->name npm-name))) + (name+version->symbol name version))) + +(define (npm-package->package-sexp npm-package) + "Return the `package' s-expression for an NPM-PACKAGE." + (define resolve-spec + (match-lambda + (($ name version) + (resolve-package name (string->semver-range version))))) + + (if (package-revision? npm-package) + (let ((name (package-revision-name npm-package)) + (version (package-revision-version npm-package)) + (home-page (package-revision-home-page npm-package)) + (dependencies (package-revision-dependencies npm-package)) + (dev-dependencies (package-revision-dev-dependencies npm-package)) + (peer-dependencies (package-revision-peer-dependencies npm-package)) + (license (package-revision-license npm-package)) + (description (package-revision-description npm-package)) + (dist (package-revision-dist npm-package))) + (let* ((name (npm-name->name name)) + (url (dist-tarball dist)) + (home-page (if (string? home-page) + home-page + (string-append %default-page "/" (uri-encode name)))) + (synopsis description) + (resolved-deps (map resolve-spec + (append dependencies peer-dependencies))) + (peer-names (map versioned-package-name peer-dependencies)) + ;; lset-difference for treating peer-dependencies as dependencies, + ;; which leads to dependency cycles. lset-union for treating them as + ;; (ignored) dev-dependencies, which leads to broken packages. + (dev-names + (lset-union string= + (map versioned-package-name dev-dependencies) + peer-names)) + (extra-phases + (match dev-names + (() '()) + ((dev-names ...) + `((add-after 'patch-dependencies 'delete-dev-dependencies + (lambda _ + (delete-dependencies '(,@(reverse dev-names)))))))))) + (values + `(package + (name ,name) + (version ,(semver->string (package-revision-version npm-package))) + (source (origin + (method url-fetch) + (uri ,url) + (sha256 (base32 ,(hash-url url))))) + (build-system node-build-system) + (arguments + (list + #:tests? #f + #:phases + #~(modify-phases %standard-phases + (delete 'build) + ,@extra-phases))) + ,@(match dependencies + (() '()) + ((dependencies ...) + `((inputs + (list ,@(map package-revision->symbol resolved-deps)))))) + (home-page ,home-page) + (synopsis ,synopsis) + (description ,description) + (license ,license)) + (map (match-lambda (($ name version) + (list name (semver->string version)))) + resolved-deps)))) + (values #f '()))) + + +;;; +;;; Interface +;;; + +(define npm-binary->guix-package + (lambda* (name #:key (version *semver-range-any*) #:allow-other-keys) + (let* ((svr (match version + ((? string?) (string->semver-range version)) + (_ version))) + (pkg (resolve-package name svr))) + (npm-package->package-sexp pkg)))) + +(define* (npm-binary-recursive-import package-name #:key version) + (recursive-import package-name + #:repo->guix-package (memoize npm-binary->guix-package) + #:version version + #:guix-name npm-name->name)) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index 1f34cab088..d724f2bca3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -49,7 +49,7 @@ (define %standard-import-options '()) (define importers '("gnu" "pypi" "cpan" "hackage" "stackage" "egg" "elpa" "gem" "go" "cran" "crate" "texlive" "json" "opam" - "minetest" "elm" "hexpm" "composer")) + "minetest" "elm" "hexpm" "composer" "npm-binary")) (define (resolve-importer name) (let ((module (resolve-interface diff --git a/guix/scripts/import/npm-binary.scm b/guix/scripts/import/npm-binary.scm new file mode 100644 index 0000000000..3403a69bcc --- /dev/null +++ b/guix/scripts/import/npm-binary.scm @@ -0,0 +1,121 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2014 David Thompson +;;; Copyright © 2018 Ricardo Wurmus +;;; Copyright © 2019 Timothy Sample +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts import npm-binary) + #:use-module (guix ui) + #:use-module (guix utils) + #:use-module (guix scripts) + #:use-module (guix import npm-binary) + #:use-module (guix scripts import) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-11) + #:use-module (srfi srfi-37) + #:use-module (srfi srfi-41) + #:use-module (ice-9 match) + #:use-module (ice-9 format) + #:export (guix-import-npm-binary)) + + +;;; +;;; Command-line options. +;;; + +(define %default-options + '()) + +(define (show-help) + (display (G_ "Usage: guix import npm-binary PACKAGE-NAME [VERSION] +Import and convert the npm package PACKAGE-NAME using the +`node-build-system' (but without building the package from source).")) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -r, --recursive import packages recursively")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + ;; Specification of the command-line options. + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix import npm-binary"))) + (option '(#\r "recursive") #f #f + (lambda (opt name arg result) + (alist-cons 'recursive #t result))) + %standard-import-options)) + +(define* (package-name->name+version* spec) + "Given SPEC, a package name like \"@scope/pac@^0.9.1\", return two values: +\"@scope/pac\" and \"^0.9.1\". When the version part is unavailable, SPEC and \"*\" +are returned. The first part may start with '@', the latter part must not contain +contain '@'." + (match (string-rindex spec delimiter) + (#f (values spec "*")) + (0 (values spec "*")) + (idx (values (substring spec 0 idx) + (substring spec (1+ idx)))))) + + +;;; +;;; Entry point. +;;; + +(define (guix-import-npm-binary . args) + (define (parse-options) + ;; Return the alist of option values. + (parse-command-line args %options (list %default-options) + #:build-options? #f)) + + (let* ((opts (parse-options)) + (args (filter-map (match-lambda + (('argument . value) + value) + (_ #f)) + (reverse opts)))) + (match args + ((spec) + (define-values (package-name version) + (package-name->name+version* spec)) + (match (if (assoc-ref opts 'recursive) + ;; Recursive import + (npm-binary-recursive-import package-name #:version version) + ;; Single import + (npm-binary->guix-package package-name #:version version)) + ((or #f '()) + (leave (G_ "failed to download meta-data for package '~a@~a'~%") + package-name version)) + (('package etc ...) `(package ,@etc)) + ((? list? sexps) + (map (match-lambda + ((and ('package ('name name) ('version version) . rest) pkg) + `(define-public ,(name+version->symbol name version) + ,pkg)) + (_ #f)) + sexps)))) + (() + (leave (G_ "too few arguments~%"))) + ((many ...) + (leave (G_ "too many arguments~%")))))) diff --git a/tests/npm-binary.scm b/tests/npm-binary.scm new file mode 100755 index 0000000000..cf85e572b3 --- /dev/null +++ b/tests/npm-binary.scm @@ -0,0 +1,146 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2024 Jelle Licht +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . +(define-module (test-npm-binary) + #:use-module ((gcrypt hash) + #:select ((sha256 . gcrypt-sha256))) + #:use-module (guix import npm-binary) + #:use-module (guix base32) + #:use-module (guix tests) + #:use-module (srfi srfi-64) + #:use-module (ice-9 iconv) + #:use-module (ice-9 match) + #:export (run-test)) + +(define foo-json + "{ + \"name\": \"foo\", + \"dist-tags\": { + \"latest\": \"1.2.3\", + \"next\": \"2.0.1-beta4\" + }, + \"description\": \"General purpose utilities to foo your bars\", + \"homepage\": \"https://github.com/quartz/foo\", + \"repository\": \"quartz/foo\", + \"versions\": { + \"1.2.3\": { + \"name\": \"foo\", + \"description\": \"General purpose utilities to foo your bars\", + \"version\": \"1.2.3\", + \"author\": \"Jelle Licht \", + \"devDependencies\": { + \"node-megabuilder\": \"^0.0.2\" + }, + \"dependencies\": { + \"bar\": \"^0.1.0\" + }, + \"repository\": { + \"url\": \"quartz/foo\" + }, + \"homepage\": \"https://github.com/quartz/foo\", + \"license\": \"MIT\", + \"dist\": { + \"tarball\": \"https://registry.npmjs.org/foo/-/foo-1.2.3.tgz\" + } + } + } +}") + +(define bar-json + "{ + \"name\": \"bar\", + \"dist-tags\": { + \"latest\": \"0.1.2\" + }, + \"description\": \"Core module in FooBar\", + \"homepage\": \"https://github.com/quartz/bar\", + \"repository\": \"quartz/bar\", + \"versions\": { + \"0.1.2\": { + \"name\": \"bar\", + \"description\": \"Core module in FooBar\", + \"version\": \"0.1.2\", + \"author\": \"Jelle Licht \", + \"repository\": { + \"url\": \"quartz/bar\" + }, + \"homepage\": \"https://github.com/quartz/bar\", + \"license\": \"MIT\", + \"dist\": { + \"tarball\": \"https://registry.npmjs.org/bar/-/bar-0.1.2.tgz\" + } + } + } +}") + +(define test-source-hash + "") + +(define test-source + "Empty file\n") + +(define have-guile-semver? + (false-if-exception (resolve-interface '(semver)))) + +(test-begin "npm") + +(unless have-guile-semver? (test-skip 1)) +(test-assert "npm-binary->guix-package" + (mock ((guix http-client) http-fetch + (lambda* (url #:rest _) + (match url + ("https://registry.npmjs.org/foo" + (values (open-input-string foo-json) + (string-length foo-json))) + ("https://registry.npmjs.org/bar" + (values (open-input-string bar-json) + (string-length bar-json))) + ("https://registry.npmjs.org/foo/-/foo-1.2.3.tgz" + (set! test-source-hash + (bytevector->nix-base32-string + (gcrypt-sha256 (string->bytevector test-source "utf-8")))) + (values (open-input-string test-source) + (string-length test-source)))))) + (match (npm-binary->guix-package "foo") + (`(package + (name "node-foo") + (version "1.2.3") + (source (origin + (method url-fetch) + (uri "https://registry.npmjs.org/foo/-/foo-1.2.3.tgz") + (sha256 + (base32 + ,test-source-hash)))) + (build-system node-build-system) + (arguments + (list #:tests? #f + #:phases + (gexp (modify-phases %standard-phases + (delete 'build) + (add-after 'patch-dependencies 'delete-dev-dependencies + (lambda _ + (delete-dependencies '("node-megabuilder")))))))) + (inputs (list node-bar-0.1.2)) + (home-page "https://github.com/quartz/foo") + (synopsis "General purpose utilities to foo your bars") + (description "General purpose utilities to foo your bars") + (license license:expat)) + #t) + (x + (pk 'fail x #f))))) + +(test-end "npm") base-commit: 4d79a9cd6b5f0d8c5afbab0c6b70ae42740d5470 -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 02 10:17:03 2024 Received: (at 62375) by debbugs.gnu.org; 2 Apr 2024 14:17:03 +0000 Received: from localhost ([127.0.0.1]:54864 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrewg-0007YR-WC for submit@debbugs.gnu.org; Tue, 02 Apr 2024 10:17:03 -0400 Received: from mail1.fsfe.org ([2001:aa8:ffed:f5f3::151]:33850) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrewe-0007Xa-Qd for 62375@debbugs.gnu.org; Tue, 02 Apr 2024 10:17:02 -0400 From: Jelle Licht DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021100501; t=1712067413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hbMWdHlbiQunftnYfcAdga7FC+e1tVS8cym8yUho4y0=; b=NnzU0FBR+94dHkHHJ1+LdjIhYiKfjIoMbGqC3TpS/zwZfAXl9qCZnkgkIPGU15T07M7EbI iRDX1kcVNRFuxTkV+apTgMsM0Wnx2LQJIiICCgdNLj7sqipBRnYsrCJrx6ZtD+sIg3bVs5 qw3/xoAhSekuKVLIjtRT+hHZBMxiJyw= To: Jonathan Brielmaier , 62375@debbugs.gnu.org Subject: Re: [bug#62375] [PATCH 0/1] npm binary importer In-Reply-To: References: Date: Tue, 02 Apr 2024 16:16:53 +0200 Message-ID: <87o7arg88a.fsf@fsfe.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 62375 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Jonathan Brielmaier via Guix-patches via writes: > Hi Jelle, > > one rather unimportant remark. The commit message mentions Lars-Dominik > as co-author: > https://issues.guix.gnu.org/62375#12-lineno8 > > But there is no copyright header for him in any of the added files... Thanks for the reminder; it was a contribution that they shared in 2021, so I've now added the copyright header in V4. - Jelle From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 25 05:14:12 2025 Received: (at 62375-done) by debbugs.gnu.org; 25 Feb 2025 10:14:12 +0000 Received: from localhost ([127.0.0.1]:44886 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tmrx5-00080o-Jn for submit@debbugs.gnu.org; Tue, 25 Feb 2025 05:14:12 -0500 Received: from mail2.fsfe.org ([213.95.165.55]:39924) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tmrx1-00080X-Jd for 62375-done@debbugs.gnu.org; Tue, 25 Feb 2025 05:14:09 -0500 From: Jelle Licht DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fsfe.org; s=2021081301; t=1740478445; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=+ztckiIMxzt/Le+oWNE19GFUdw+r18MigFEpAqLXpHw=; b=U1Nx2gwnVbT24BhF7RZVAZ0XS+6hChkRIPO8zC/E3gcRtLJRhGg/dHvnfZaoQXrEbQOcd/ djRMsvG845PTEtlpDCYRj/ASiEbo/gRm5hbAkIhnhvhXzy5act6S0dXMcwWNvjpuNBlgQF 0BDk3MLHlagrJzy3UEcGDlo2KLObZtM= To: 62375-done@debbugs.gnu.org Subject: Re: [PATCH v4] import: Add binary npm importer. In-Reply-To: <1892fd7b000320e56cbf10958fd4cf88d3c83ced.1712067148.git.jlicht@fsfe.org> References: <1892fd7b000320e56cbf10958fd4cf88d3c83ced.1712067148.git.jlicht@fsfe.org> Date: Tue, 25 Feb 2025 11:14:02 +0100 Message-ID: <87y0xupc0l.fsf@fsfe.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 62375-done Cc: Timothy Sample , Lars-Dominik Braun X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.0 (------) Hey folks, This was merged quite some time ago into master already, so closing. jlicht@fsfe.org writes: > From: Jelle Licht > > * guix/scripts/import.scm: (importers): Add "npm-binary". > * guix/import/npm-binary.scm: New file. > * guix/scripts/import/npm-binary.scm: New file. > * Makefile.am: Add them. > > Co-authored-by: Timothy Sample > Co-authored-by: Lars-Dominik Braun > > Change-Id: I98a45068cf5b9c42790664cc743feaa7ac76f807 > --- > > Changes in v4: > - Add copyright line for LDB > - Use package-name@version-spec notation on CLI > - Simplify CLI argument handling > - Turn %registry into a parameter named %npm-registry > > Changes in v3: > - Ensure that package bindings generated during recursive import match the > package bindings used in the list of inputs > > Changes in v2: > - Change *SOME-VAR* to %SOME-VAR > - Removed unused http-error-code > - Rebase on master > - Refactor hash-url to use port-sha256 helper > - use explicit record accessors instead of order-sensitive destructuring > - address line-width styling issues > - added basic documentation > - added some basic tests (using simple mocks instead of with-http-server) > - simplify import script entrypoint > > Makefile.am | 3 + > doc/guix.texi | 33 ++++ > guix/import/npm-binary.scm | 279 +++++++++++++++++++++++++++++ > guix/scripts/import.scm | 2 +- > guix/scripts/import/npm-binary.scm | 121 +++++++++++++ > tests/npm-binary.scm | 146 +++++++++++++++ > 6 files changed, 583 insertions(+), 1 deletion(-) > create mode 100644 guix/import/npm-binary.scm > create mode 100644 guix/scripts/import/npm-binary.scm > create mode 100755 tests/npm-binary.scm From unknown Thu Aug 14 21:52:45 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 25 Mar 2025 11:24:18 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator