GNU bug report logs - #52555
[RFC PATCH 0/3] Decentralized substitute distribution with ERIS

Previous Next

Full log

Message #172 received at 52555 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: pukkamustard <pukkamustard <at> posteo.net>
Cc: 52555 <at> debbugs.gnu.org
Subject: Re: bug#52555: [RFC PATCH 0/3] Decentralized substitute
 distribution with ERIS
Date: Sat, 14 Jan 2023 19:34:06 +0100

pukkamustard <pukkamustard <at> posteo.net> skribis:

> * guix/scripts/publish.scm: (bake-narinfo+nar): Compute ERIS URN of compressed nars.
> (narinfo-string): Add #:eris-urn parameter and honor it.
> * guix/scripts/narinfo.scm: (<narinfo>)[eris-format,eris-urn]: New fields.
> (narinfo-maker): Handle ERIS URN and ERIS format.
> * configure.ac: (HAVE_GUILE_ERIS): New conditional.
> * gnu/packages/package-management.scm: (guix)[native-inputs]: Add guile-eris.

I’d suggest separating the (guix narinfo) part from the rest.

> @@ -135,8 +139,8 @@ (define (narinfo-maker str cache-url)
>    "Return a narinfo constructor for narinfos originating from CACHE-URL.  STR
>  must contain the original contents of a narinfo file."
>    (lambda (path urls compressions file-hashes file-sizes
> -                nar-hash nar-size references deriver system
> -                signature)
> +                nar-hash nar-size eris-format eris-urn references deriver
> +                system signature)

Maybe make ‘eris-format’ and ‘eris-urn’ named parameters.

> +  (define (eris-encode-nar compressions)
> +    (and (member %eris-zstd-compression compressions)
> +         (let* ((nar (nar-cache-file cache item
> +                                     #:compression %eris-zstd-compression))
> +                (stat (stat nar #f)))
> +           (and stat
> +                (call-with-input-file nar
> +                  (lambda (port)
> +                    (let ((eris-urn _
> +                                    (eris-encode port
> +                                                 #:block-size 'large
> +                                                 #:block-reducer rcount
> +                                                 #:convergence-secret
> +                                                 %null-convergence-secret)))
> +                      eris-urn)))))))

I think you remove the ‘let’ and just call ‘eris-encode’ in tail
position (Guile truncates multiple-value returns.)

Where is ‘eris-encode’ storing the blocks?  I don’t see any parameter
here.

I guess it should go to /var/cache/guix/publish/eris or similar.

>    (let ((compression (actual-compressions item compressions)))
>  
>      (for-each (cut compress-nar cache item <>) compressions)
> @@ -640,7 +662,8 @@ (define (compressed-nar-size compression)
>      (match compressions
>        ((main others ...)
>         (let ((narinfo (narinfo-cache-file cache item
> -                                          #:compression main)))
> +                                          #:compression main))
> +             (eris-urn (eris-encode-nar compressions)))
>           (with-atomic-file-output narinfo
>             (lambda (port)
>               ;; Open a new connection to the store.  We cannot reuse the main
> @@ -651,7 +674,8 @@ (define (compressed-nar-size compression)
>                   (display (narinfo-string store item
>                                            #:nar-path nar-path
>                                            #:compressions compressions
> -                                          #:file-sizes sizes)
> +                                          #:file-sizes sizes
> +                                          #:eris-urn eris-urn)

Would be nice to make ERIS encoding optional, with a command-line
switch to turn it on.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.