Subject: bug#64756: closed (Re: bug#64756: some frequent test failures)

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 02 Nov 2023 22:05:02 +0000

Your bug report

#64756: some frequent test failures

which was filed against the automake package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
64756: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=64756
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#64756: some frequent test failures
From: Karl Berry <[email-masked]>
To: [email-masked]
Date: Thu, 2 Nov 2023 16:03:48 -0600
List-Id: <debbugs-submit.debbugs.gnu.org>

Just to close this out: Bogdan worked up a patch to avoid the
fractional-second timestamps unless the new (current autoconf
development) autom4te is in used. Here is the basic change:
https://git.savannah.gnu.org/cgit/automake.git/commit/?id=b6fa73115d094c8d0da1d6759b6e7c7fca1f8a07
(also appended for the archive)

(Comments tweaked a little in subsequent changes.)

Jim is going to start working towards a release over the next days. 
--thanks, karl.

>From b6fa73115d094c8d0da1d6759b6e7c7fca1f8a07 Mon Sep 17 00:00:00 2001
From: Bogdan <[email-masked]>
Date: Wed, 1 Nov 2023 17:40:47 -0700
Subject: m4: fall back to non-fractional timestamps with older autom4te.

* m4/sanity.m4 (_AM_FILESYSTEM_TIMESTAMP_RESOLUTION): if
HiRes is not present in Autom4te/FileUtils.pm, do not consider
fractional sleeps.
---
 m4/sanity.m4 | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/m4/sanity.m4 b/m4/sanity.m4
index db9a1f5..445d1fb 100644
--- a/m4/sanity.m4
+++ b/m4/sanity.m4
@@ -16,10 +16,28 @@ AS_IF([sleep 0.001 2>/dev/null], [am_cv_sleep_fractional_seconds=true], [am_cv_s
 # _AM_FILESYSTEM_TIMESTAMP_RESOLUTION
 # -----------------------------------
 # Determine the filesystem timestamp resolution.  Modern systems are nanosecond
-# capable, but historical systems could be millisecond, second, or even 2-second
-# resolution.
+# capable, but historical systems could have millisecond, second, or even
+# 2-second resolution.
 AC_DEFUN([_AM_FILESYSTEM_TIMESTAMP_RESOLUTION], [dnl
 AC_REQUIRE([_AM_SLEEP_FRACTIONAL_SECONDS])
+#
+# Check if Autom4te uses Time::HiRes. If not, we cannot use fractional sleep,
+# because this sanity test and automated tests will be unreliable due to
+# Autom4te's caching of results and comparing timestamps.
+# More info: long thread around
+#     https://lists.gnu.org/archive/html/automake/2023-04/msg00002.html
+# and https://debbugs.gnu.org/cgi/bugreport.cgi?bug=64756.  
+AC_PATH_PROG([AUTOM4TE], [autom4te])
+if test x"$autom4te_perllibdir" = x; then
+  autom4te_perllibdir=`sed -n \
+   '/autom4te_perllibdir/{s/^.*|| //;s/;$//;s/^.//;s/.$//;p;q}' <$AUTOM4TE`
+fi
+if grep HiRes "$autom4te_perllibdir"/Autom4te/FileUtils.pm >/dev/null; then
+  :
+else
+  am_cv_sleep_fractional_seconds=false
+fi
+
 AC_CACHE_CHECK([the filesystem timestamp resolution], am_cv_filesystem_timestamp_resolution, [dnl
 # Use names that lexically sort older-first when the timestamps are equal.
 rm -f conftest.file.a conftest.file.b
-- 
cgit v1.1

Subject: some frequent test failures
From: Bruno Haible <[email-masked]>
To: [email-masked]
Date: Thu, 20 Jul 2023 23:55:33 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

After checking out autoconf from git (master branch), then
  $ bootstrap
  $ ./configure
  $ make
  $ for n in 2 3 4 5 6 7 8 9; do
      make check ; mv test-suite.log test-suite.log.$n
    done

The various test-suite.log files show different test failures each
time:

$ grep ^FAIL: test-suite.log.2
FAIL: t/aclocal-I-and-install
FAIL: t/backcompat2
FAIL: t/python-prefix
FAIL: t/remake-after-aclocal-m4
FAIL: t/remake-include-configure
FAIL: t/subobj

$ grep ^FAIL: test-suite.log.3
FAIL: t/aclocal-I-and-install
FAIL: t/python-prefix
FAIL: t/remake-after-acinclude-m4
FAIL: t/remake-after-aclocal-m4
FAIL: t/remake-aclocal-version-mismatch

$ grep ^FAIL: test-suite.log.4
FAIL: t/aclocal-I-and-install
FAIL: t/backcompat2
FAIL: t/nodef
FAIL: t/python-prefix
FAIL: t/remake-after-acinclude-m4
FAIL: t/remake-after-aclocal-m4
FAIL: t/remake-include-configure

$ grep ^FAIL: test-suite.log.5
FAIL: t/aclocal-I-and-install
FAIL: t/nodef2
FAIL: t/python-prefix
FAIL: t/remake-after-aclocal-m4
FAIL: t/remake-all-1

$ grep ^FAIL: test-suite.log.6
FAIL: t/aclocal-I-and-install
FAIL: t/backcompat2
FAIL: t/testsuite-summary-reference-log
FAIL: t/python-prefix
FAIL: t/remake-after-acinclude-m4

$ grep ^FAIL: test-suite.log.7
FAIL: t/aclocal-I-and-install
FAIL: t/nodef
FAIL: t/python-prefix

$ grep ^FAIL: test-suite.log.8
FAIL: t/aclocal-I-and-install
FAIL: t/backcompat2
FAIL: t/nodef
FAIL: t/testsuite-summary-reference-log
FAIL: t/python-prefix
FAIL: t/remake-after-configure-ac
FAIL: t/remake-after-aclocal-m4

$ grep ^FAIL: test-suite.log.9
FAIL: t/aclocal-I-and-install
FAIL: t/nodef
FAIL: t/python-prefix
FAIL: t/remake-after-acinclude-m4
FAIL: t/remake-after-aclocal-m4
FAIL: t/remake-all-1

So:
  * t/aclocal-I-and-install and t/python-prefix fail always.
  * The following tests fail randomly:
    t/backcompat2
    t/nodef
    t/nodef2
    t/remake-after-configure-ac
    t/remake-after-acinclude-m4
    t/remake-after-aclocal-m4
    t/remake-aclocal-version-mismatch
    t/remake-include-configure
    t/remake-all-1
    t/subobj
    t/testsuite-summary-reference-log

This is on a system with Linux, glibc 2.35, autoconf 2.71, m4 1.4.19, make 4.3,
perl v5.34.0. No hardware issues (uptime: 20 days).

Any idea?

Bruno