Subject: Re: bug#43843: git-annex is not Reproducible

From: zimoun <[email-masked]>
Date: Fri, 09 Oct 2020 22:33:43 +0200

Dear,

On Thu, 08 Oct 2020 at 09:17, Efraim Flashner <[email-masked]> wrote:

>>     $ guix time-machine --commit=e13fefbcffa50144d9301d67b61928ac3c964fb3 --
>> build --no-grafts --check git-annex
>>     guix build: error: derivation
>> `/gnu/store/5illriaaijqrhl17aymzgiw6wzaq666s-git-annex-8.20200810.drv' may
>> not be deterministic: output
>> `/gnu/store/32h0xbqdwkm22naqba10fk9124cdgw9q-git-annex-8.20200810' differs
>>
>> However, going back one more to 35e24fab73 (gnu: git-annex: Update to
>> 8.20200810, 2020-08-11), the check was successful:
>>
>>     $ guix time-machine --commit=35e24fab73bc450c6e0d3b9a5bbdfa88d2c425f4 --
>> build --no-grafts --check git-annex
>>     [...]
>>     /gnu/store/m79ri0g2s3d8pk52ysjkdvallk2zqbwb-git-annex-8.20200810

> In between those two commits is also the change to the haskell build
> system update, where we use the dynamic libraries. The best bet is to
> make sure the dependencies are built reproducibly.

Yep, it is what “git bisect” says.

--8<---------------cut here---------------start------------->8---
67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2 is the first bad commit
commit 67cb9fa2357026ee42ec5bb0923ec4dc4a43abe2
Author: Ricardo Wurmus <[email-masked]>
Date:   Tue Jun 16 22:25:48 2020 +0200

    build-system/haskell: Support parallel builds.

    * guix/build-system/haskell.scm (haskell-build): Add keyword
      PARALLEL-BUILD?
    and pass it on to the builder.
    * guix/build/haskell-build-system.scm (build): Accept keyword
      PARALLEL-BUILD?
    and pass the number of parallel jobs to GHC.

 guix/build-system/haskell.scm       | 2 ++
 guix/build/haskell-build-system.scm | 7 +++++--
 2 files changed, 7 insertions(+), 2 deletions(-)
bisect run success
--8<---------------cut here---------------end--------------->8---

I did:

  git bisect start e13fefbc 35e24fab
  git bisest run ./check.sh

where check.sh reads:

--8<---------------cut here---------------start------------->8---
#!/bin/sh

PKG=git-annex

git log -1 --format='%h %s [%an]'
COMMIT=$(git log -1 --format='%H')
echo "Start: $COMMIT"

guix time-machine     \
     --commit=$COMMIT \
     -- build $PKG    \
     --no-grafts

guix time-machine     \
     --commit=$COMMIT \
     -- build $PKG    \
     --no-grafts --check -K
status=$?

echo "End: [$status] $COMMIT"

exit $status
--8<---------------cut here---------------end--------------->8---

And I do not know if it is related, but these packages using the
’haskell-build-system’ are not reproducible neither (304/~640):

 agda
 cabal-install
 cpphs
 darcs
 dhall
 elm-compiler
 ghc-abstract-deque
 ghc-abstract-par
 ghc-adjunctions
 ghc-aeson
 ghc-alex
 ghc-asn1-encoding
 ghc-asn1-types
 ghc-assoc
 ghc-atomic-primops
 ghc-attoparsec
 ghc-auto-update
 ghc-aws
 ghc-base-compat
 ghc-basement
 ghc-base-unicode-symbols
 ghc-bifunctors
 ghc-bitarray
 ghc-blaze-html
 ghc-blaze-markup
 ghc-bloomfilter
 ghc-bytes
 ghc-bytestring-handle
 ghc-c2hs
 ghc-cairo
 ghc-cborg
 ghc-cereal
 ghc-cgi
 ghc-charset
 ghc-chart
 ghc-chasingbottoms
 ghc-cheapskate
 ghc-chunked-data
 ghc-clientsession
 ghc-cmdargs
 ghc-colour
 ghc-comonad
 ghc-concurrent-extra
 ghc-conduit
 ghc-conduit-algorithms
 ghc-conduit-extra
 ghc-configurator
 ghc-constraints
 ghc-convertible
 ghc-crypto-api
 ghc-crypto-api-tests
 ghc-crypto-cipher-tests
 ghc-crypto-cipher-types
 ghc-cryptohash
 ghc-cryptonite
 ghc-crypto-random
 ghc-curl
 ghc-data-accessor
 ghc-dbus
 ghc-descriptive
 ghc-diagrams-core
 ghc-diagrams-lib
 ghc-diagrams-solve
 ghc-diff
 ghc-digest
 ghc-doctest
 ghc-edisonapi
 ghc-edisoncore
 ghc-edit-distance
 ghc-either
 ghc-errors
 ghc-esqueleto
 ghc-exactprint
 ghc-extra
 ghc-fast-logger
 ghc-feed
 ghc-fgl
 ghc-filemanip
 ghc-filepath-bytestring
 ghc-fingertree
 ghc-foldl
 ghc-foundation
 ghc-free
 ghc-fsnotify
 ghc-generic-deriving
 ghc-generics-sop
 ghc-genvalidity-property
 ghc-getopt-generics
 ghc-gluraw
 ghc-glut
 ghc-gnuplot
 ghc-graphviz
 ghc-gtk2hs-buildtools
 ghc-hackage-security
 ghc-haddock-api
 ghc-haddock-library
 ghc-happstack-server
 ghc-happy
 ghc-hashtables
 ghc-haskeline
 ghc-haskell-lexer
 ghc-haskell-src
 ghc-haskell-src-exts
 ghc-haskell-src-exts-util
 ghc-hedgehog
 ghc-highlighting-kate
 ghc-hindent
 ghc-hmatrix
 ghc-hmatrix-gsl
 ghc-hmatrix-gsl-stats
 ghc-hmatrix-special
 ghc-hourglass
 ghc-hpack
 ghc-hslogger
 ghc-hslua
 ghc-hsopenssl
 ghc-hspec
 ghc-hspec-contrib
 ghc-hspec-core
 ghc-hspec-meta
 ghc-hsyaml
 ghc-http
 ghc-http2
 ghc-http-client
 ghc-http-date
 ghc-http-types
 ghc-hunit
 ghc-hxt
 ghc-hxt-regex-xmlschema
 ghc-hxt-unicode
 ghcid
 ghc-inline-c
 ghc-inspection-testing
 ghc-integer-logarithms
 ghc-interpolate
 ghc-intervalmap
 ghc-intervals
 ghc-io-streams
 ghc-jira-wiki-markup
 ghc-json
 ghc-juicypixels
 ghc-kan-extensions
 ghc-language-c
 ghc-language-glsl
 ghc-language-javascript
 ghc-lens
 ghc-lens-family-core
 ghc-libmpd
 ghc-lib-parser
 ghc-libxml
 ghc-libyaml
 ghc-lifted-base
 ghc-linear
 ghc-listlike
 ghc-llvm-hs
 ghc-llvm-hs-pure
 ghc-logging-facade
 ghc-math-functions
 ghc-megaparsec
 ghc-memory
 ghc-missingh
 ghc-monad-par
 ghc-monad-par-extras
 ghc-monads-tf
 ghc-monoid-extras
 ghc-mono-traversable
 ghc-mwc-random
 ghc-ncurses
 ghc-network
 ghc-non-negative
 ghc-opengl
 ghc-openglraw
 ghc-optparse-applicative
 ghc-pandoc
 ghc-pandoc-citeproc
 ghc-pandoc-types
 ghc-parallel
 ghc-parser-combinators
 ghc-parsers
 ghc-patience
 ghc-pem
 ghc-persistent
 ghc-persistent-test
 ghc-pgp-wordlist
 ghc-polyparse
 ghc-pqueue
 ghc-prettyprinter
 ghc-prettyprinter
 ghc-pretty-show
 ghc-pretty-simple
 ghc-primitive
 ghc-profunctors
 ghc-protolude
 ghc-psqueues
 ghc-quickcheck
 ghc-quickcheck-instances
 ghc-rebase
 ghc-reducers
 ghc-regex
 ghc-regex-applicative
 ghc-regex-pcre
 ghc-regex-pcre-builtin
 ghc-regex-posix
 ghc-regex-tdfa
 ghc-regex-tdfa-text
 ghc-rerebase
 ghc-rio
 ghc-safe
 ghc-safesemaphore
 ghc-sandi
 ghc-scientific
 ghc-sdl
 ghc-sdl2
 ghc-sdl2-image
 ghc-sdl-mixer
 ghc-semialign
 ghc-semigroupoids
 ghc-sendfile
 ghc-serialise
 ghc-shakespeare
 ghc-shelly
 ghc-size-based
 ghc-skylighting-core
 ghc-smallcheck
 ghc-snap-core
 ghc-snap-server
 ghc-socks
 ghc-sop-core
 ghc-sourcemap
 ghc-splitmix
 ghc-statistics
 ghc-stm-chans
 ghc-stm-conduit
 ghc-storable-record
 ghc-storablevector
 ghc-streaming-commons
 ghc-strict
 ghc-stringsearch
 ghc-stylish-haskell
 ghc-syb
 ghc-tagged
 ghc-tagsoup
 ghc-tar
 ghc-tasty
 ghc-test-framework
 ghc-testing-feat
 ghc-texmath
 ghc-tf-random
 ghc-these
 ghc-time-compat
 ghc-timezone-olson
 ghc-tls
 ghc-transformers
 ghc-transformers-compat
 ghc-tree-diff
 ghc-trifecta
 ghc-turtle
 ghc-unagi-chan
 ghc-unicode-transforms
 ghc-union-find
 ghc-uniplate
 ghc-unix-time
 ghc-unliftio
 ghc-unliftio-core
 ghc-unordered-containers
 ghc-uri-bytestring
 ghc-utf8-string
 ghc-utility-ht
 ghc-uuid
 ghc-vault
 ghc-vector
 ghc-vector-algorithms
 ghc-wai-extra
 ghc-warp
 ghc-websockets
 ghc-x11
 ghc-x509
 ghc-x509-store
 ghc-x509-system
 ghc-x509-validation
 ghc-xml
 ghc-xml-conduit
 ghc-xmonad-contrib
 ghc-yaml
 ghc-yesod
 ghc-yesod-core
 ghc-yesod-form
 ghc-zstd
 git-annex
 hlint
 hoogle
 hscolour
 hspec-discover
 idris
 kmonad
 ngless
 pandoc
 pandoc-citeproc
 purescript
 raincat
 scroll
 shellcheck
 stylish-haskell
 xmobar

All the best,
simon