GNU bug report logs -
#69328
[PATCH 00/12] Better source code recovery from SWH
Previous Next
Reported by: Ludovic Courtès <ludo <at> gnu.org>
Date: Fri, 23 Feb 2024 14:46:01 UTC
Severity: normal
Tags: patch
Done: Ludovic Courtès <ludo <at> gnu.org>
Bug is archived. No further changes may be made.
Full log
View this message in rfc822 format
[Message part 1 (text/plain, inline)]
Your bug report
#69328: [PATCH 00/12] Better source code recovery from SWH
which was filed against the guix-patches package, has been closed.
The explanation is attached below, along with your original report.
If you require more details, please reply to 69328 <at> debbugs.gnu.org.
--
69328: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=69328
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems
[Message part 2 (message/rfc822, inline)]
Hello,
Simon Tournier <zimon.toutoune <at> gmail.com> skribis:
> On mar., 05 mars 2024 at 12:06, Ludovic Courtès <ludo <at> gnu.org> wrote:
>
>> Ludovic Courtès (12):
>> lint: Switch to SRFI-71.
>> lint: archival: Fix crash in non-Git case.
>> lint: archival: Trigger “Save Code Now” for VCSes other than Git.
>> swh: Add ‘type’ field to <visit>.
>> swh: ‘origin-visits’ takes an optional ‘max’ parameter.
>> swh: ‘lookup-origin-revision’ handles branches pointing to
>> directories.
>> hg-download: Use ‘swh-download-directory-by-nar-hash’.
>> svn-download: Use ‘swh-download-directory-by-nar-hash’.
>> bzr-download: Implement nar fallback.
>> download-nar: Distinguish ‘output’ and ‘item’ parameter.
>> perform-download: Allow use of ‘download-nar’ for ‘--check’ builds.
>> download: Honor ‘GUIX_DOWNLOAD_METHODS’ environment variable.
>
> LGTM.
Pushed as 2f441fc738976175d438f7942211b1894e2eb416, thank you & Timothy
for taking a look!
I’ll update the ‘guix’ package in the coming days so we can benefit from
all of this.
> Unrelated things for later. :-)
>
> 1. About CVS, IIRC, there is only one package: gnu-standards. And it
> changes barely. Why not fetch from FTP or else instead of CVS?
Good idea (or maybe someday someone will finally migrate it to some
other VCS?).
> 2. About the lookup, currently it is done item per item when it could be
> done several at once – Timothy does that with PoG. This helps for the
> rate limit. For instance if one uses “guix lint -c archival -m
> manifest.scm”.
>
> 3. The option ’-m’ for “guix lint” seems missing.
>
> These #2 and #3 would help third-party channels, IMHO.
All good ideas.
> Although, I am slowly working on some “guix swh” extension… but I have
> been distraction by another extension “guix try-out”, then distracted by
> another one “guix cite”. Well, I need to finish all my homeworks. ;-)
Heh, sounds exciting!
Ludo’.
[Message part 3 (message/rfc822, inline)]
Hello Guix!
This patch series improves source code recovery from SWH, as a followup
to <https://issues.guix.gnu.org/68741>.
It does several things:
• ‘guix lint -c archival’ now emits save requests for VCSes other
than Git.
• Fix <https://issues.guix.gnu.org/69070>.
• Allow content-addressed recovery of Mercurial and Subversion
checkouts.
• Allow Bazaar recovery using ‘download-nar’ (I didn’t bother with SWH).
• Have all these things honor the ‘GUIX_DOWNLOAD_SEQUENCE’ environment
variable.
You can try the various methods like this:
GUIX_DOWNLOAD_SEQUENCE=nar ./pre-inst-env guix build -S apl --check
GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-wisp --check
GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-gcrypt --check
In the last case, note that you must be running guix-daemon for the checkout
since that uses “builtin:git-download”, which is implemented on the server
side.
There’s a few caveats:
• Mercurial SWH fallback almost works, but not quite, due to this SWH bug:
<https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/5256>.
• Right now, no Subversion checkout has the nar-sha256 ExtID at SWH for
unclear reasons, so retrieving the source of ‘apl’ (say) from SWH
doesn’t work yet.
• Multi-directory Subversion downloads (‘svn-multi-fetch’) is not supported
yet. For that we’ll need to arrange with our SWH friends so they
compute nar-sha256 ExtIDs for combined directories (and we’ll have to
include that info in ‘sources.json’).
Feedback welcome!
Ludo’.
Ludovic Courtès (12):
lint: Switch to SRFI-71.
lint: archival: Fix crash in non-Git case.
lint: archival: Trigger “Save Code Now” for VCSes other than Git.
swh: Add ‘type’ field to <visit>.
swh: ‘origin-visits’ takes an optional ‘max’ parameter.
swh: ‘lookup-origin-revision’ handles branches pointing to
directories.
hg-download: Use ‘swh-download-directory-by-nar-hash’.
svn-download: Use ‘swh-download-directory-by-nar-hash’.
bzr-download: Implement nar fallback.
download-nar: Distinguish ‘output’ and ‘item’ parameter.
perform-download: Allow use of ‘download-nar’ for ‘--check’ builds.
download: Honor ‘GUIX_DOWNLOAD_SEQUENCE’ environment variable.
guix/build/bzr.scm | 3 +-
guix/build/download-nar.scm | 12 +--
guix/build/download.scm | 50 +++++++---
guix/build/git.scm | 27 ++++--
guix/bzr-download.scm | 57 ++++++++---
guix/cvs-download.scm | 24 +++--
guix/download.scm | 53 ++++-------
guix/git-download.scm | 20 ++--
guix/hg-download.scm | 36 ++++---
guix/lint.scm | 151 +++++++++++++++++++-----------
guix/scripts/perform-download.scm | 65 +++++++------
guix/svn-download.scm | 84 +++++++++++------
guix/swh.scm | 71 ++++++++------
tests/lint.scm | 20 ++++
tests/swh.scm | 74 +++++++++++++++
15 files changed, 501 insertions(+), 246 deletions(-)
base-commit: ffcce77ec488e3c89401ad77fafa65fcd9e9f5be
--
2.41.0
This bug report was last modified 1 year and 102 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.