Subject: bug#37224: closed (Re: [bug#37224] [PATCH 0/4] Add 'archival' checker for 'guix lint')

From: [email-masked] (GNU bug Tracking System)
Date: Mon, 02 Sep 2019 13:29:02 +0000

Your bug report

#37224: [PATCH 0/4] Add 'archival' checker for 'guix lint'

which was filed against the guix-patches package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
37224: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=37224
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: [bug#37224] [PATCH 0/4] Add 'archival' checker for 'guix lint'
From: Ludovic Courtès <[email-masked]>
To: [email-masked]
Date: Mon, 02 Sep 2019 15:28:01 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hello,

Ludovic Courtès <[email-masked]> skribis:

>   tests: 'with-http-server' accepts multiple responses.
>   swh: Add hooks for rate limiting handling.
>   swh: Make 'commit-id?' public.
>   lint: Add 'archival' checker.

I went ahead and pushed these at commit
55549c7b9b778a79d3e1f3d085861ef36aabdca6.

I asked for feedback on #swh-devel and olasd (Nicolas Dandrimont), one
of the SWH developers, replied:

--8<---------------cut here---------------start------------->8---
<olasd> civodul: this seems like a sensible design to me; Does `guix lint`
	automatically call other network services? maybe the save request
	should be an optional flag  [13:55]
<olasd> (automatically _checking_ is fine; automatically _saving_, I don't
	know)
<civodul> olasd: there's a 'refresh' checker that calls out to services to
	  determine whether a newer version of the package is available, for
	  instance  [14:01]
<civodul> initially i thought about not saving at all, and just writing "you
	  should save this"
<civodul> but then i thought it's more convenient to just do it right away
<civodul> it's unlikely to send garbage anyway, and it'll necessarily send
	  only public code, and very likely only free code  [14:02]
<civodul> or did you have other concerns?
<olasd> I don't think it's going to be an issue for us  [14:08]
<olasd> I would just (personally) be surprised if a lint tool I'm using
	started to have side effects on somewhat unrelated systems :)
								        [14:09]
[...]

<civodul> olasd: ah true, though i guess we just got used to that ;-)  [14:12]
<civodul> anyway, thanks for your feedback!
<olasd> civodul: feel free to quote me by mail if you want to keep it archived
--8<---------------cut here---------------end--------------->8---

Ludo’.

Subject: [PATCH 0/4] Add 'archival' checker for 'guix lint'
From: Ludovic Courtès <[email-masked]>
To: [email-masked]
Cc: Ludovic Courtès <[email-masked]>
Date: Fri, 30 Aug 2019 01:16:53 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hello Guix!

This patch series adds an ‘archival’ checker for ‘guix lint’, documented
like this:

     Checks whether the package’s source code is archived at Software
     Heritage (https://www.softwareheritage.org).

     When the source code that is not archived comes from a
     version-control system (VCS)—e.g., it’s obtained with ‘git-fetch’,
     send Software Heritage a “save” request so that it eventually
     archives it.  This ensures that the source will remain available in
     the long term, and that Guix can fall back to Software Heritage
     should the source code disappear from its original host.  The
     status of recent “save” requests can be viewed on-line
     (https://archive.softwareheritage.org/save/#requests).

     When source code is a tarball obtained with ‘url-fetch’, simply
     print a message when it is not archived.  As of this writing
     Software Heritage does not allow requests to save arbitrary
     tarballs; we are working on ways to ensure that non-VCS source code
     is also archived.

     Software Heritage limits the request rate per IP address
     (https://archive.softwareheritage.org/api/#rate-limiting).  When
     the limit is reached, ‘guix lint’ prints a message and the
     ‘archival’ checker stops doing anything until that limit has been
     reset.

Currently, only 25% of our packages are not fetched with ‘url-fetch’.
For the remaining 75%, this checker can only report whether the tarball
is missing (and apart from ftp.gnu.org and a few other exceptions, it
usually _is_ missing) and cannot actually save it.

Anyway, it’s a first step in that direction.  Feedback welcome!

The second step will be to write a “lister” for Software Heritage that
grabs the list of source code URLs from
<https://guix.gnu.org/packages.json>.  That could would run at SWH
and it could potentially grab the tarballs, not just the VCS checkouts.
Here’s are examples:

  https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/packagist/lister.py
  https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gnu/lister.py

It should be quite easy for a Pythonista to write something similar
for our ‘packages.json’.  Any takers?  :-)

Ludo’.

Ludovic Courtès (4):
  tests: 'with-http-server' accepts multiple responses.
  swh: Add hooks for rate limiting handling.
  swh: Make 'commit-id?' public.
  lint: Add 'archival' checker.

 doc/guix.texi         |  25 ++++++
 guix/lint.scm         |  96 +++++++++++++++++++++-
 guix/swh.scm          |  88 ++++++++++++++++-----
 guix/tests/http.scm   |  39 +++++----
 tests/derivations.scm |  12 +--
 tests/lint.scm        | 179 ++++++++++++++++++++++++++++++++----------
 tests/swh.scm         |  41 +++++++++-
 7 files changed, 395 insertions(+), 85 deletions(-)

-- 
2.23.0