GNU bug report logs - #39547
[PATCH] website: Provide JSON sources list used by Software Heritage.

Previous Next

Full log

View this message in rfc822 format

From: zimoun <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludovic.courtes <at> inria.fr>
Cc: 39547 <at> debbugs.gnu.org
Subject: [bug#39547] [PATCH] website: Provide JSON sources list used by Software Heritage.
Date: Fri, 14 Feb 2020 10:04:01 +0100

Hi Ludo,

Thank you for the review.


On Fri, 14 Feb 2020 at 09:40, Ludovic Courtès <ludovic.courtes <at> inria.fr> wrote:
> zimoun <zimon.toutoune <at> gmail.com> skribis:

> > Format discussed here <https://forge.softwareheritage.org/D2025#51269>.
> >
> > * website/apps/packages/builder.scm (sources-json-builder): New procedure.
>
> [...]
>
> > +(define (sources-json-builder)
> > +  "Return a JSON page listing all the sources."
>
> Please add a reference to
> <https://forge.softwareheritage.org/D2025#51269> here.

ok


> > +  (define (origin->json origin)
> > +    (define method
> > +      (origin-method origin))
> > +
> > +    (define uri                         ;represented as string
> > +      (origin-uri origin))
> > +
> > +    (define (mirror->url uri)
> > +      (uri->string (car (maybe-expand-mirrors uri %mirrors))))
> > +
> > +    (define (resolve urls)
> > +      (let* ((url (car urls))
> > +             (uri (string->uri url))
> > +             (rest (cdr urls)))
> > +        (case (uri-scheme uri)
> > +          ((mirror) (mirror->url uri))
> > +          ((http) url)
> > +          ((https) url)
> > +          (else
> > +           (if (null? rest)
> > +               url
> > +               (resolve rest))))))
> > +
> > +    `((type . ,(cond ((eq? url-fetch method) 'url)
> > +                     ((eq? git-fetch method) 'git)
> > +                     ((eq? svn-fetch method) 'svn)
> > +                     (else                   #nil)))
> > +      ,@(cond ((eq? url-fetch method)
> > +               `(("url" . ,(match uri
> > +                               ((? string? url) (mirror->url (string->uri url)))
> > +                               ((urls ...) (resolve urls))))))
> > +              ((eq? git-fetch method)
> > +               `(("git_url" . ,(git-reference-url uri))))
> > +              ((eq? svn-fetch method)
> > +               `(("svn_url" . ,(svn-reference-url uri))))
> > +              (else '()))
> > +      ,@(if (eq? method git-fetch)
> > +            `(("git_ref" . ,(git-reference-commit uri)))
> > +            '())
> > +      ,@(if (eq? method svn-fetch)
> > +            `(("svn_revision" . ,(svn-reference-revision
> > +                                  uri)))
> > +            '())))
>
> Could you, in a first patch, move ‘origin->json’ out of
> ‘packages-json-builder’, and in a second patch, add mirror-expansion
> feature?

Yes, I will try but I am a bit lost. There is:
 - packages-json-builder that I did not modified
 - sources-json-builder which the "adaptation" of the former to output
to the sources.json format.

Well, do you want a refactor of 'origin->json' shared by the 2
"{sources,packages}-json-builder"?


> For mirror:// expansion, I think you can just write something like:
>
>   (define uris
>     (append-map (cut maybe-expand-mirrors <> %mirrors)
>                 (match url
>                   ((_ ...) (map string->uri url))
>                   (_       (list (string->uri url))))))
>
> and then pick the first element of the list.

Yes, it is better. :-)


> In parallel, I’d recommend suggesting a format change or addition that
> would allow us to provide all the URLs.  :-)

Ok :-)


> With those changes in place, I think we’ll be ready to go!

Working on updating the package Julia, I have seen that some patches
are 'origin' and live for example upstream
(https://blablab/project.git/patches/fancy-name.patch) and the current
patch will not list them. Therefore, if upstream disappears and/or
change in-place the patches, Guix would not be able to re-build in the
future (time-machine).

I am trying to implement a recursive exporter.
Well, I will try to make that the v2 contains your suggestions and all
the patches.



Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.