GNU bug report logs - #28745
tarballs generated on github are generated on demand (leading to different hash sums)

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: tracker <at> debbugs.gnu.org
Subject: bug#28745: closed (tarballs generated on github are generated on
 demand (leading to different hash sums))
Date: Fri, 20 Oct 2017 21:05:01 +0000

[Message part 1 (text/plain, inline)]

Your message dated Fri, 20 Oct 2017 23:04:43 +0200
with message-id <871slxcyz8.fsf <at> gnu.org>
and subject line Re: bug#28745: [PATCH] tarballs generated on github are generated on demand (leading to different hash sums)
has caused the debbugs.gnu.org bug report #28745,
regarding tarballs generated on github are generated on demand (leading to different hash sums)
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
28745: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=28745
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: ng0 <ng0 <at> infotropique.org>
To: bug-guix <at> gnu.org
Subject: tarballs generated on github are generated on demand (leading to
 different hash sums)
Date: Sun, 8 Oct 2017 11:40:09 +0000

[Message part 3 (text/plain, inline)]

Past and recent discussion in our IRC channel and on the mailing list
show that we can not rely on tarballs on github keeping the same
hash forever.
According to github they are "generated on demand", leading to
regular hash mismatches.

Since some of our own dependencies are on github (at the very least
guile-git), we need to come up with a solution.

Right now we have around 449 packages with tarball sources from
github in our gnu/packages.
We could:

- Move them all to use git-download and just use
  the commit that has been tagged in the versions that produce
  the tarballs on github.

- Mirror the content somewhere reliable in snapshots for
  some time. Problem here: we start to rely on this "somewhere"
  to be trustworthy and introduce one more point to trust
  (however due to pre-recorded hash sum this is just an annoyance,
  not a grave issue).

- Your idea here.

-- 
ng0
GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588
GnuPG: https://dist.krosos.org/dist/keys/
https://www.infotropique.org https://krosos.org

[signature.asc (application/pgp-signature, inline)]

[Message part 5 (message/rfc822, inline)]

From: ludo <at> gnu.org (Ludovic Courtès)
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: bug#28745 <28745-done <at> debbugs.gnu.org>
Subject: Re: bug#28745: [PATCH] tarballs generated on github are generated on
 demand (leading to different hash sums)
Date: Fri, 20 Oct 2017 23:04:43 +0200

Hi,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> skribis:

> I could finish a script that helped me finding all of our affected
> packages, verify that only the hash but not the content of the archives
> had changed, as well as automate the hash update for those safe to
> update.

Great job!

> Attached is the patch and the scripts I used. I think we might
> want to reuse some of it to extend guix lint to warn packagers that
> archives coming from .*github.*archives URL are not guaranteed to be
> stable and that it would be better, if available, to use manually
> uploaded releases archives.

Unfortunately, it’s become commonplace to publish nothing else than a
Git tag.  Now, in those cases, we could also use ‘git-fetch’, which
wouldn’t be affected by problems with generated tarballs.

Thoughts?

> PS: I've also uploaded the scripts here:
> https://notabug.org/apteryx/fiasco for ease of cloning. Any comments
> about my nascent (ab)use of Scheme are welcome!

The code looks nice!

> From 774a764149ecb0e234ae09c9a0a273af671c3c86 Mon Sep 17 00:00:00 2001
> From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
> Date: Sun, 15 Oct 2017 22:17:12 -0400
> Subject: [PATCH] gnu: packages: Fix the hashes of mutated GitHub archives.
>
> Fixes bug https://bugs.gnu.org/28745.
>
> * gnu/packages/audio.scm (csound): Fix hash.
> * gnu/packages/engineering.scm (fritzing): Likewise.
> * gnu/packages/erlang.scm (erlang): Likewise.
> * gnu/packages/fonts.scm (font-google-material-design-icons): Likewise.
> * gnu/packages/graphics.scm (ogre): Likewise.
> * gnu/packages/java.scm (java-plexus-interpolation, antlr3): Likewise.
> * gnu/packages/serialization.scm (yaml-cpp): Likewise.
> * gnu/packages/version-control.scm (libgit2): Likewise.

I’ve checked the hashes by running:

  ./pre-inst-env guix build -S --no-substitutes csound fritzing erlang \
     font-google-material-design-icons ogre java-plexus-interpolation \
     antlr3 yaml-cpp libgit2  --max-jobs=2

and everything went well.

Pushed as fd75eb6cd4e5c689f9e6ce7dd8d87f423778d308, thanks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.