GNU bug report logs -
#37224
[PATCH 0/4] Add 'archival' checker for 'guix lint'
Previous Next
Reported by: Ludovic Courtès <ludo <at> gnu.org>
Date: Thu, 29 Aug 2019 23:18:01 UTC
Severity: normal
Tags: patch
Done: Ludovic Courtès <ludo <at> gnu.org>
Bug is archived. No further changes may be made.
Full log
View this message in rfc822 format
[Message part 1 (text/plain, inline)]
Your message dated Mon, 02 Sep 2019 15:28:01 +0200
with message-id <87h85u96am.fsf <at> gnu.org>
and subject line Re: [bug#37224] [PATCH 0/4] Add 'archival' checker for 'guix lint'
has caused the debbugs.gnu.org bug report #37224,
regarding [PATCH 0/4] Add 'archival' checker for 'guix lint'
to be marked as done.
(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)
--
37224: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=37224
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems
[Message part 2 (message/rfc822, inline)]
Hello Guix!
This patch series adds an ‘archival’ checker for ‘guix lint’, documented
like this:
Checks whether the package’s source code is archived at Software
Heritage (https://www.softwareheritage.org).
When the source code that is not archived comes from a
version-control system (VCS)—e.g., it’s obtained with ‘git-fetch’,
send Software Heritage a “save” request so that it eventually
archives it. This ensures that the source will remain available in
the long term, and that Guix can fall back to Software Heritage
should the source code disappear from its original host. The
status of recent “save” requests can be viewed on-line
(https://archive.softwareheritage.org/save/#requests).
When source code is a tarball obtained with ‘url-fetch’, simply
print a message when it is not archived. As of this writing
Software Heritage does not allow requests to save arbitrary
tarballs; we are working on ways to ensure that non-VCS source code
is also archived.
Software Heritage limits the request rate per IP address
(https://archive.softwareheritage.org/api/#rate-limiting). When
the limit is reached, ‘guix lint’ prints a message and the
‘archival’ checker stops doing anything until that limit has been
reset.
Currently, only 25% of our packages are not fetched with ‘url-fetch’.
For the remaining 75%, this checker can only report whether the tarball
is missing (and apart from ftp.gnu.org and a few other exceptions, it
usually _is_ missing) and cannot actually save it.
Anyway, it’s a first step in that direction. Feedback welcome!
The second step will be to write a “lister” for Software Heritage that
grabs the list of source code URLs from
<https://guix.gnu.org/packages.json>. That could would run at SWH
and it could potentially grab the tarballs, not just the VCS checkouts.
Here’s are examples:
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/packagist/lister.py
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gnu/lister.py
It should be quite easy for a Pythonista to write something similar
for our ‘packages.json’. Any takers? :-)
Ludo’.
Ludovic Courtès (4):
tests: 'with-http-server' accepts multiple responses.
swh: Add hooks for rate limiting handling.
swh: Make 'commit-id?' public.
lint: Add 'archival' checker.
doc/guix.texi | 25 ++++++
guix/lint.scm | 96 +++++++++++++++++++++-
guix/swh.scm | 88 ++++++++++++++++-----
guix/tests/http.scm | 39 +++++----
tests/derivations.scm | 12 +--
tests/lint.scm | 179 ++++++++++++++++++++++++++++++++----------
tests/swh.scm | 41 +++++++++-
7 files changed, 395 insertions(+), 85 deletions(-)
--
2.23.0
[Message part 3 (message/rfc822, inline)]
Hello,
Ludovic Courtès <ludo <at> gnu.org> skribis:
> tests: 'with-http-server' accepts multiple responses.
> swh: Add hooks for rate limiting handling.
> swh: Make 'commit-id?' public.
> lint: Add 'archival' checker.
I went ahead and pushed these at commit
55549c7b9b778a79d3e1f3d085861ef36aabdca6.
I asked for feedback on #swh-devel and olasd (Nicolas Dandrimont), one
of the SWH developers, replied:
--8<---------------cut here---------------start------------->8---
<olasd> civodul: this seems like a sensible design to me; Does `guix lint`
automatically call other network services? maybe the save request
should be an optional flag [13:55]
<olasd> (automatically _checking_ is fine; automatically _saving_, I don't
know)
<civodul> olasd: there's a 'refresh' checker that calls out to services to
determine whether a newer version of the package is available, for
instance [14:01]
<civodul> initially i thought about not saving at all, and just writing "you
should save this"
<civodul> but then i thought it's more convenient to just do it right away
<civodul> it's unlikely to send garbage anyway, and it'll necessarily send
only public code, and very likely only free code [14:02]
<civodul> or did you have other concerns?
<olasd> I don't think it's going to be an issue for us [14:08]
<olasd> I would just (personally) be surprised if a lint tool I'm using
started to have side effects on somewhat unrelated systems :)
[14:09]
[...]
<civodul> olasd: ah true, though i guess we just got used to that ;-) [14:12]
<civodul> anyway, thanks for your feedback!
<olasd> civodul: feel free to quote me by mail if you want to keep it archived
--8<---------------cut here---------------end--------------->8---
Ludo’.
This bug report was last modified 5 years and 247 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.