Subject: bug#18301: closed (Re: texi-vers.am: Problem with parallel builds due to vti.tmp)

From: [email-masked] (GNU bug Tracking System)
Date: Sat, 23 Aug 2014 14:59:02 +0000

Your bug report

#18301: texi-vers.am: Problem with parallel builds due to vti.tmp

which was filed against the automake package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
18301: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18301
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: texi-vers.am: Problem with parallel builds due to vti.tmp
From: Paul Eggert <[email-masked]>
To: [email-masked]
Date: Sat, 23 Aug 2014 07:58:37 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

Thanks, I installed the attached patch, which has the more-usual 
approach of a uniquely-named temporary.
From 28b4fdb0ea4c540f1b7bcb90675e365e3aa11beb Mon Sep 17 00:00:00 2001
From: Paul Eggert <[email-masked]>
Date: Sat, 23 Aug 2014 07:55:28 -0700
Subject: [PATCH] build: fix race in parallel builds

Reported by Friedrich Beckmann in: http://bugs.gnu.org/18301
* lib/am/texi-vers.am (?DIRSTAMP?): Put the process-ID into the
temporary file name.  Use a similar temporary in the source dir.
---
 lib/am/texi-vers.am | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/lib/am/texi-vers.am b/lib/am/texi-vers.am
index bddf382..e98bb82 100644
--- a/lib/am/texi-vers.am
+++ b/lib/am/texi-vers.am
@@ -31,25 +31,25 @@ DIST_COMMON += %VTEXI% %STAMPVTI%
 ## %STAMPVTI% is distributed and %DIRSTAMP% isn't: a distributed file
 ## should never be dependent upon a non-distributed built file.
 ## Therefore we ensure that %DIRSTAMP% exists in the rule.
+## Use cp + mv so that the update of %VTEXI% is atomic even if
+## the source directory is on a different file system.
 ?DIRSTAMP?	@test -f %DIRSTAMP% || $(MAKE) $(AM_MAKEFLAGS) %DIRSTAMP%
 	@(dir=.; test -f ./%TEXI% || dir=$(srcdir); \
 	set `$(SHELL) %MDDIR%mdate-sh $$dir/%TEXI%`; \
 	echo "@set UPDATED $$1 $$2 $$3"; \
 	echo "@set UPDATED-MONTH $$2 $$3"; \
 	echo "@set EDITION $(VERSION)"; \
-	echo "@set VERSION $(VERSION)") > %VTI%.tmp
-## Use cp and rm here because some older "mv"s can't move across
-## filesystems.  Furthermore, GNU "mv" in the AmigaDOS environment
-## can't handle this.
-	@cmp -s %VTI%.tmp %VTEXI% \
-	  || (echo "Updating %VTEXI%"; \
-	      cp %VTI%.tmp %VTEXI%)
-	-@rm -f %VTI%.tmp
+	echo "@set VERSION $(VERSION)") > %VTI%.tmp$$$$ && \
+	(cmp -s %VTI%.tmp$$$$ %VTEXI% \
+	  || (echo "Updating %VTEXI%" && \
+	      cp %VTI%.tmp$$$$ %VTEXI%.tmp$$$$ && \
+	      mv %VTEXI%.tmp$$$$ %VTEXI%)) && \
+	rm -f %VTI%.tmp$$$$ %VTEXI%.$$$$
 	@cp %VTEXI% $@
 
 mostlyclean-am: mostlyclean-%VTI%
 mostlyclean-%VTI%:
-	-rm -f %VTI%.tmp
+	-rm -f %VTI%.tmp* %VTEXI%.tmp*
 
 maintainer-clean-am: maintainer-clean-%VTI%
 maintainer-clean-%VTI%:
-- 
1.9.3

Subject: texi-vers.am: Problem with parallel builds due to vti.tmp
From: Friedrich Beckmann <[email-masked]>
To: [email-masked]
Date: Wed, 20 Aug 2014 11:17:59 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Package: automake
Version: 1.11.6-1

I encounter a problem with automake during parallel builds for the debian pspp package. When I run the build
process non-parallel then it works. The problem only occurs sometimes when I do a parallel build. The build
is a debian package build. Here are the relevant sections from the build.log and Makefile.in: 

======== snippet from pspp_0.8.3-3_amd64.build ============
make[4]: Entering directory `/home/fritz/pspp/new/pspp-0.8.3'
make[4]: »doc/.dirstamp« ist bereits aktualisiert.
make[4]: Leaving directory `/home/fritz/pspp/new/pspp-0.8.3'
Updating ./doc/version.texi
Updating ./doc/version.texi
cp: Aufruf von stat für „vti.tmp“ nicht möglich: Datei oder Verzeichnis nicht gefunden
make[3]: *** [doc/stamp-vti] Fehler 1
make[3]: Leaving directory `/home/fritz/pspp/new/pspp-0.8.3'
make[2]: *** [pdf-recursive] Fehler 1
make[2]: *** Warte auf noch nicht beendete Prozesse…
================================================================

The build log shows „Updating ./doc/version.texi“ two times. This message is generated when
the stamp-vti target is build. So obviously the parallel build results in two parallel processes to build
stamp-vti. 

================ Relevant section from Makefile.in ======================
$(srcdir)/doc/version.texi:  $(srcdir)/doc/stamp-vti
$(srcdir)/doc/stamp-vti: doc/pspp.texinfo $(top_srcdir)/configure
        test -f doc/$(am__dirstamp) || $(MAKE) $(AM_MAKEFLAGS) doc/$(am__dirstamp)
        @(dir=.; test -f ./doc/pspp.texinfo || dir=$(srcdir); \
        set `$(SHELL) $(top_srcdir)/build-aux/mdate-sh $$dir/doc/pspp.texinfo`; \
        echo "@set UPDATED $$1 $$2 $$3"; \
        echo "@set UPDATED-MONTH $$2 $$3"; \
        echo "@set EDITION $(VERSION)"; \
        echo "@set VERSION $(VERSION)") > vti.tmp
        @cmp -s vti.tmp $(srcdir)/doc/version.texi \
          || (echo "Updating $(srcdir)/doc/version.texi"; \
              cp vti.tmp $(srcdir)/doc/version.texi)
        -@rm -f vti.tmp
================================================================

I think the problem is due to the file „vti.tmp“ which is generated and removed during the build. When
one process has already deleted the file and the other process wants to delete it, then the error message
in the build log: "cp: Aufruf von stat für „vti.tmp“ nicht möglich: Datei oder Verzeichnis nicht gefunden“
will show up. This means that the file vti.tmp is not found for the copy action.

A possible solution could be based on unique temporary files, instead of vti.tmp.  The relevant automake file
is http://git.savannah.gnu.org/cgit/automake.git/tree/lib/am/texi-vers.am. I think the relevant part of the file
did not change since version 1.11.6. Attached is the full build.log. 
The bug report in the pspp project is here: https://savannah.gnu.org/bugs/?43029

Friedrich