Subject: bug#35813: closed (Re: [bug#35813] [PATCH] Add crate-recursive-import.)

From: [email-masked] (GNU bug Tracking System)
Date: Sun, 13 Oct 2019 07:44:02 +0000

Your bug report

#35813: [PATCH] import: crate: add recursive option

which was filed against the guix-patches package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
35813: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=35813
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: [bug#35813] [PATCH] Add crate-recursive-import.
From: Brian Leung <[email-masked]>
To: Efraim Flashner <[email-masked]>
Cc: [email-masked], Ivan Petkov <[email-masked]>
Date: Sun, 13 Oct 2019 00:42:22 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

Closing this since someone else already merged their own crate recursive
importer.

On Sun, Sep 8, 2019 at 12:57 AM Efraim Flashner <[email-masked]>
wrote:

> As a simple test I ran 'guix import crate encoding -r' and it gave me
> the 6 packages I expected. 'guix import crate winapi -r' only gave me
> rust-winapi, as the dependent crates are already packaged. When I tried
> 'guix import crate rand -r' it found the updated version and started
> importing all the new dependencies also.
>
> It looks good. I'm tempted to leave it running with 'guix import crate
> serde -r' just to see if we're ever going to make it there.
>
> I see that it imports A then B then C then D, and prints out D then C
> then B then A. For the one I tested with is still rust-encoding. I'll
> try my hand at ascii art:
>
>                             encoding
>                                 |
>   -------------------------------------------------------
>   |         |               |           |               |
> japanese  korean      simpchinese    singlebyte     tradchinese
>   |         |               |           |               |
>   -------------------------------------------------------
>                         |
>                  encoding-tests
>
> import went encoding, japanese, tests, korean, simpchinese, singlebyte,
> tradchinese
>
> I think the only thing I would wish for would be to do tests, then the
> languages and then encoding (best for upstreaming one at a time), or to
> do them alphabetically (plop them in alphabetically all at once). This
> I'm happy to live without I think.
>
> The other thing was I ran 'guix import crate security-framework -r' and
> after ~40 crates it crashed on me with:
>     web/http.scm:1186:15: In procedure read-response-line:
>     Bad Read-Header-Line header: #<eof>
> and I would prefer to have the ~40 crates it did grab first to be
> printed out and not lost. Between these two I would like most to not
> lose the imported crates than worrying over the printed order.
>
> Great job! From what I've tested I think it's ready as-is and any
> changes would just be gravy.
>
>
> --
> Efraim Flashner   <[email-masked]>   אפרים פלשנר
> GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted
>

Subject: [PATCH] import: crate: add recursive option
From: Karl Meakin <[email-masked]>
To: [email-masked]
Cc: Karl Meakin <[email-masked]>
Date: Mon, 20 May 2019 19:23:06 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

* guix/script/import/crate.scm: Add recursive option.
* guix/import/crate.scm (crate-recursive-import): New variable.
---
 doc/guix.texi                 |  8 ++++++++
 guix/import/crate.scm         | 24 ++++++++++++++++--------
 guix/scripts/import/crate.scm | 27 ++++++++++++++++++++++-----
 3 files changed, 46 insertions(+), 13 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index ae9ad0739e..636bb7521d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -8787,6 +8787,14 @@ in Guix.
 Import metadata from the crates.io Rust package repository
 @uref{https://crates.io, crates.io}.
 
+@table @code
+@item --recursive
+@itemx -r
+Traverse the dependency graph of the given upstream package recursively
+and generate package expressions for all those packages that are not yet
+in Guix.
+@end table
+
 @item opam
 @cindex OPAM
 @cindex OCaml
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index e0b400d054..d8554b0e7a 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -37,6 +37,7 @@
   #:use-module (srfi srfi-26)
   #:export (crate->guix-package
             guix-package->crate-name
+            crate-recursive-import
             %crate-updater))
 
 (define (crate-fetch crate-name callback)
@@ -86,8 +87,8 @@
 VERSION, INPUTS, NATIVE-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
   (let* ((port (http-fetch (crate-uri name version)))
          (guix-name (crate-name->package-name name))
-         (inputs (map crate-name->package-name inputs))
-         (native-inputs (map crate-name->package-name native-inputs))
+         (input-packages (map crate-name->package-name inputs))
+         (native-input-packages (map crate-name->package-name native-inputs))
          (pkg `(package
                    (name ,guix-name)
                    (version ,version)
@@ -99,8 +100,8 @@ VERSION, INPUTS, NATIVE-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
                               (base32
                                ,(bytevector->nix-base32-string (port-sha256 port))))))
                    (build-system cargo-build-system)
-                   ,@(maybe-native-inputs native-inputs "src")
-                   ,@(maybe-inputs inputs "src")
+                   ,@(maybe-native-inputs native-input-packages "src")
+                   ,@(maybe-inputs input-packages "src")
                    (home-page ,(match home-page
                                  (() "")
                                  (_ home-page)))
@@ -111,12 +112,14 @@ VERSION, INPUTS, NATIVE-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
                                ((license) license)
                                (_ `(list ,@license)))))))
          (close-port port)
-         pkg))
+         (values pkg inputs)))
 
-(define (crate->guix-package crate-name)
-  "Fetch the metadata for CRATE-NAME from crates.io, and return the
+(define crate->guix-package
+  (memoize
+   (lambda* (crate-name _)
+     "Fetch the metadata for CRATE-NAME from crates.io, and return the
 `package' s-expression corresponding to that package, or #f on failure."
-  (crate-fetch crate-name make-crate-sexp))
+     (crate-fetch crate-name make-crate-sexp))))
 
 (define (guix-package->crate-name package)
   "Return the crate name of PACKAGE."
@@ -158,6 +161,11 @@ VERSION, INPUTS, NATIVE-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
      (version version)
      (urls (list url)))))
 
+(define* (crate-recursive-import package-name)
+  (recursive-import package-name #f
+                    #:repo->guix-package crate->guix-package
+                    #:guix-name crate-name->package-name))
+
 (define %crate-updater
   (upstream-updater
    (name 'crates)
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index cab9a4397b..8fadcdd57c 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -27,6 +27,7 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-37)
+  #:use-module (srfi srfi-41)
   #:use-module (ice-9 match)
   #:use-module (ice-9 format)
   #:export (guix-import-crate))
@@ -45,6 +46,8 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
   (display (G_ "
   -h, --help             display this help and exit"))
   (display (G_ "
+  -r, --recursive        import packages recursively"))
+  (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
   (show-bug-report-information))
@@ -58,6 +61,9 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
          (option '(#\V "version") #f #f
                  (lambda args
                    (show-version-and-exit "guix import crate")))
+         (option '(#\r "recursive") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'recursive #t result)))
          %standard-import-options))
 
 
@@ -83,11 +89,22 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
                            (reverse opts))))
     (match args
       ((package-name)
-       (let ((sexp (crate->guix-package package-name)))
-         (unless sexp
-           (leave (G_ "failed to download meta-data for package '~a'~%")
-                  package-name))
-         sexp))
+       (if (assoc-ref opts 'recursive)
+           ;; Recursive import
+           (map (match-lambda
+                  ((and ('package ('name name) . rest) pkg)
+                   `(define-public ,(string->symbol name)
+                      ,pkg))
+                  (_ #f))
+                (reverse
+                 (stream->list
+                  (crate-recursive-import package-name))))
+           ;; Single import
+           (let ((sexp (crate->guix-package package-name #f)))
+             (unless sexp
+               (leave (G_ "failed to download meta-data for package '~a'~%")
+                      package-name))
+             sexp)))
       (()
        (leave (G_ "too few arguments~%")))
       ((many ...)
-- 
2.21.0