Subject: bug#72232: closed (Re: bug#72232: "make dist" is not reproducible)

From: [email-masked] (GNU bug Tracking System)
Date: Sun, 21 Jul 2024 17:46:02 +0000

Your bug report

#72232: "make dist" is not reproducible

which was filed against the coreutils package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
72232: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72232
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#72232: "make dist" is not reproducible
From: Pádraig Brady <[email-masked]>
To: Bruno Haible <[email-masked]>, [email-masked]
Cc: Simon Josefsson <[email-masked]>
Date: Sun, 21 Jul 2024 18:44:04 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

On 21/07/2024 17:18, Bruno Haible wrote:
> Hi,
> 
> Subsequent runs of "make dist" in the same environment produce tarballs with
> different contents.
> 
> How to reproduce:
> 
> In a git checkout of coreutils, do:
> 
> $ echo snapshot > .tarball-version
> $ ./configure; make -k maintainer-clean
> $ ./bootstrap
> $ ./configure; make dist
> $ mkdir 1; (cd 1 && tar xf ../*.tar.gz)
> $ make; make dist
> $ mkdir 2; (cd 2 && tar xf ../*.tar.gz)
> $ diff -r -q 1 2
> 
> The last command produces a difference:
> 
> $ diff -r -q 1 2
> Files 1/coreutils-snapshot/.timestamp and 2/coreutils-snapshot/.timestamp differ
> $ cat 1/coreutils-snapshot/.timestamp
> 1721577765
> $ cat 2/coreutils-snapshot/.timestamp
> 1721577832
> 
> Can this be avoided? Can the contents of '.timestamp' always be the same?
> (Wouldn't it be enough to give it a different modification time, each time?)

Right, we should be able to adjust this.
The .timestamp file was added to support reproducible _builds_:
https://github.com/coreutils/coreutils/commit/c1b3d6587
https://reproducible-builds.org/docs/source-date-epoch/

We can just rely on the timestamp of the .tarball-version
to support reproducible _tarballs_.
That's done in the attached, which I'll apply later.

Marking this as done.

thanks,
Pádraig

From ed1273b00b127cad9bb9867e0fd2be70d60bee1a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <[email-masked]>
Date: Sun, 21 Jul 2024 18:25:05 +0100
Subject: [PATCH] build: support reproducible tarball creation

We already support reproducible builds since commit v8.24-99-gc1b3d6587,
and this adjusts that change to also support reproducible tarballs with
subsequent runs of `make dist`.

* Makefile.am: Don't create a varying .timestamp file, instead ...
* man/local.mk: Rely on the timestamp of the .tarball-version file.
Fixes https://bugs.gnu.org/72232
---
 Makefile.am  | 2 --
 man/local.mk | 3 ++-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index bbbdc7829..3fd546599 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -82,14 +82,12 @@ BUILT_SOURCES = .version
 
 # Have no read-only files in the tarball to allow easy removal.
 # Have .tarball-version based versions only in tarball builds.
-# Have .timestamp based dates only in tarball builds.
 # The perl substitution is to change some key uses of "rm" to "/bin/rm".
 # See the rm_subst comment for details.
 # The touch avoids a subtle, spurious "make distcheck" failure.
 dist-hook: gen-ChangeLog
 	$(AM_V_GEN)chmod -R +rw $(distdir)
 	$(AM_V_GEN)echo $(VERSION) > $(distdir)/.tarball-version
-	$(AM_V_GEN)date +%s > $(distdir)/.timestamp
 	$(AM_V_at)perl -pi -e '$(rm_subst)' $(distdir)/Makefile.in
 	$(AM_V_at)touch $(distdir)/doc/constants.texi \
 	  $(distdir)/doc/coreutils.info
diff --git a/man/local.mk b/man/local.mk
index 24f88745a..1acd73fe7 100644
--- a/man/local.mk
+++ b/man/local.mk
@@ -194,7 +194,8 @@ endif
 	  && $(MKDIR_P) $$t						\
 	  && (cd $$t && $(LN_S) '$(abs_top_builddir)/src/'$$prog$(EXEEXT) \
 				$$argv$(EXEEXT))			\
-	&& : $${SOURCE_DATE_EPOCH=`cat $(srcdir)/.timestamp 2>/dev/null || :`} \
+	&& : $${SOURCE_DATE_EPOCH=`date -r $(srcdir)/.tarball-version +%s \
+				   2>/dev/null || :`} \
 	&& : $${TZ=UTC0} && export TZ					\
 	&& export SOURCE_DATE_EPOCH && $(run_help2man)			\
 		     --source='$(PACKAGE_STRING)'			\
-- 
2.45.2

Subject: "make dist" is not reproducible
From: Bruno Haible <[email-masked]>
To: [email-masked]
Cc: Simon Josefsson <[email-masked]>
Date: Sun, 21 Jul 2024 18:18:06 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

Subsequent runs of "make dist" in the same environment produce tarballs with
different contents.

How to reproduce:

In a git checkout of coreutils, do:

$ echo snapshot > .tarball-version
$ ./configure; make -k maintainer-clean
$ ./bootstrap
$ ./configure; make dist
$ mkdir 1; (cd 1 && tar xf ../*.tar.gz)
$ make; make dist
$ mkdir 2; (cd 2 && tar xf ../*.tar.gz)
$ diff -r -q 1 2

The last command produces a difference:

$ diff -r -q 1 2
Files 1/coreutils-snapshot/.timestamp and 2/coreutils-snapshot/.timestamp differ
$ cat 1/coreutils-snapshot/.timestamp
1721577765
$ cat 2/coreutils-snapshot/.timestamp
1721577832

Can this be avoided? Can the contents of '.timestamp' always be the same?
(Wouldn't it be enough to give it a different modification time, each time?)

Bruno