GNU bug report logs - #11413
Issues with output redirection for parallel make

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 11413 in the body.
You can then email your comments to 11413 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: bug-automake <at> gnu.org
Subject: Racy failure in test case 'tap-more.sh'
Date: Sat, 05 May 2012 17:51:50 +0200

Severity: minor

I'm sporadically seeing the failure below in the test 'tap-more.sh'.

Anyone has an insight about what's going on?

Regards,
  Stefano

-*-*-*-

tap-more: exec /bin/sh t/tap-more.sh
Running from installcheck: no
Using TAP: no
PATH = [SNIP]
++ pwd
/devel/stefano/src/am/branches/trunks/t/tap-more.dir
+ fetch_tap_driver
+ case $am_tap_implementation in
+ AM_TAP_AWK=gawk
+ export AM_TAP_AWK
+ sed '1s|#!.*|#! /bin/sh|' /home/stefano/src/am/branches/trunks/lib/tap-driver.sh
+ chmod a+x tap-driver
+ sed 10q tap-driver
#! /bin/sh
# Copyright (C) 2011-2012 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
+ cat
+ cat
+ cat
+ cat
+ cat
+ chmod a+x 1.test 2.test 3.test
+ aclocal-1.12 -Werror
+ autoconf
+ automake-1.12 --foreign -Werror -Wall
+ for try in 0 1
+ test 0 -eq 0
+ mkdir build
+ cd build
+ srcdir=..
+ run_make=make
+ ../configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
+ ls -l
total 64
-rw-r--r-- 1 stefano stefano 27456 May  5 16:57 Makefile
-rw-r--r-- 1 stefano stefano  5004 May  5 16:57 config.log
-rwxr-xr-x 1 stefano stefano 25212 May  5 16:57 config.status
+ make check
+ cat stdout
make  check-TESTS
make[1]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
make[2]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
PASS: 1.test 1 - mu
SKIP: 1.test 2 zardoz # SKIP
PASS: 2.test 1
XFAIL: 2.test 2 # TODO not implemented
PASS: 2.test 3
PASS: 3.test 1 - blah blah blah
# 3.test: Some diagnostic
make[3]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
============================================================================
Testsuite summary for tap-more 1.0
============================================================================
# TOTAL: 6
# PASS:  4
# SKIP:  1
# XFAIL: 1
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[2]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
make[1]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
+ count_test_results total=6 pass=4 fail=0 xpass=0 xfail=1 skip=1 error=0
+ total=ERR
+ pass=ERR
+ fail=ERR
+ xpass=ERR
+ xfail=ERR
+ skip=ERR
+ error=ERR
+ eval total=6 pass=4 fail=0 xpass=0 xfail=1 skip=1 error=0
++ total=6
++ pass=4
++ fail=0
++ xpass=0
++ xfail=1
++ skip=1
++ error=0
+ /bin/grep -E -i '(total|x?pass|x?fail|skip|error)' stdout
PASS: 1.test 1 - mu
SKIP: 1.test 2 zardoz # SKIP
PASS: 2.test 1
XFAIL: 2.test 2 # TODO not implemented
PASS: 2.test 3
PASS: 3.test 1 - blah blah blah
# TOTAL: 6
# PASS:  4
# SKIP:  1
# XFAIL: 1
# FAIL:  0
# XPASS: 0
# ERROR: 0
+ rc=0
+ set +e
++ grep -c '^PASS:' stdout
+ test 4 -eq 4
++ grep -c '^XFAIL:' stdout
+ test 1 -eq 1
++ grep -c '^SKIP:' stdout
+ test 1 -eq 1
++ grep -c '^FAIL:' stdout
+ test 0 -eq 0
++ grep -c '^XPASS:' stdout
+ test 0 -eq 0
++ grep -c '^ERROR:' stdout
+ test 0 -eq 0
+ grep '^# TOTAL:  *6$' stdout
# TOTAL: 6
+ grep '^# PASS:  *4$' stdout
# PASS:  4
+ grep '^# XFAIL:  *1$' stdout
# XFAIL: 1
+ grep '^# SKIP:  *1$' stdout
# SKIP:  1
+ grep '^# FAIL:  *0$' stdout
# FAIL:  0
+ grep '^# XPASS:  *0$' stdout
# XPASS: 0
+ grep '^# ERROR:  *0$' stdout
# ERROR: 0
+ test 0 -eq 0
+ grep '^PASS: 1\.test 1 - mu$' stdout
PASS: 1.test 1 - mu
+ grep '^SKIP: 1\.test 2 zardoz # SKIP$' stdout
SKIP: 1.test 2 zardoz # SKIP
++ /bin/grep -F -c 1.test stdout
+ test 2 -eq 2
+ grep '^PASS: 2\.test 1$' stdout
PASS: 2.test 1
+ grep '^XFAIL: 2\.test 2 # TODO not implemented$' stdout
XFAIL: 2.test 2 # TODO not implemented
+ grep '^PASS: 2\.test 3$' stdout
PASS: 2.test 3
++ /bin/grep -F -c 2.test stdout
+ test 3 -eq 3
+ grep '^PASS: 3\.test 1 - blah blah blah$' stdout
PASS: 3.test 1 - blah blah blah
+ grep '^# 3\.test: Some diagnostic$' stdout
# 3.test: Some diagnostic
++ /bin/grep -F -c 3.test stdout
+ test 2 -eq 2
+ echo dummy
+ echo dummy
+ make check
make[2]: *** [test-suite.log] Error 1
make[1]: *** [check-TESTS] Error 2
make: *** [check-am] Error 2
+ cat stdout
make  check-TESTS
make[1]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
make[2]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
PASS: 1.test 1 - mu
FAIL: 1.test 2 zardoz
PASS: 2.test 1
XFAIL: 2.test 2 # TODO not implemented
PASS: 2.test 3
PASS: 3.test 1 - blah blah blah
# 3.test: Some diagnostic
ERROR: 3.test - Bail out! Kernel Panic
make[3]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
============================================================================
Testsuite summary for tap-more 1.0
============================================================================
# TOTAL: 7
# PASS:  4
# SKIP:  0
# XFAIL: 1
# FAIL:  1
# XPASS: 0
# ERROR: 1
============================================================================
See ./test-suite.log
============================================================================
make[2]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
make[1]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir/build'
+ count_test_results total=7 pass=4 fail=1 xpass=0 xfail=1 skip=0 error=1
+ total=ERR
+ pass=ERR
+ fail=ERR
+ xpass=ERR
+ xfail=ERR
+ skip=ERR
+ error=ERR
+ eval total=7 pass=4 fail=1 xpass=0 xfail=1 skip=0 error=1
++ total=7
++ pass=4
++ fail=1
++ xpass=0
++ xfail=1
++ skip=0
++ error=1
+ /bin/grep -E -i '(total|x?pass|x?fail|skip|error)' stdout
PASS: 1.test 1 - mu
FAIL: 1.test 2 zardoz
PASS: 2.test 1
XFAIL: 2.test 2 # TODO not implemented
PASS: 2.test 3
PASS: 3.test 1 - blah blah blah
ERROR: 3.test - Bail out! Kernel Panic
# TOTAL: 7
# PASS:  4
# SKIP:  0
# XFAIL: 1
# FAIL:  1
# XPASS: 0
# ERROR: 1
+ rc=0
+ set +e
++ grep -c '^PASS:' stdout
+ test 4 -eq 4
++ grep -c '^XFAIL:' stdout
+ test 1 -eq 1
++ grep -c '^SKIP:' stdout
+ test 0 -eq 0
++ grep -c '^FAIL:' stdout
+ test 1 -eq 1
++ grep -c '^XPASS:' stdout
+ test 0 -eq 0
++ grep -c '^ERROR:' stdout
+ test 1 -eq 1
+ grep '^# TOTAL:  *7$' stdout
# TOTAL: 7
+ grep '^# PASS:  *4$' stdout
# PASS:  4
+ grep '^# XFAIL:  *1$' stdout
# XFAIL: 1
+ grep '^# SKIP:  *0$' stdout
# SKIP:  0
+ grep '^# FAIL:  *1$' stdout
# FAIL:  1
+ grep '^# XPASS:  *0$' stdout
# XPASS: 0
+ grep '^# ERROR:  *1$' stdout
# ERROR: 1
+ test 0 -eq 0
+ grep '^PASS: 1\.test 1 - mu$' stdout
PASS: 1.test 1 - mu
+ grep '^FAIL: 1\.test 2 zardoz$' stdout
FAIL: 1.test 2 zardoz
++ /bin/grep -F -c 1.test stdout
+ test 2 -eq 2
+ grep '^PASS: 2\.test 1$' stdout
PASS: 2.test 1
+ grep '^XFAIL: 2\.test 2 # TODO not implemented$' stdout
XFAIL: 2.test 2 # TODO not implemented
+ grep '^PASS: 2\.test 3$' stdout
PASS: 2.test 3
++ /bin/grep -F -c 2.test stdout
+ test 3 -eq 3
+ grep '^PASS: 3\.test 1 - blah blah blah$' stdout
PASS: 3.test 1 - blah blah blah
+ grep '^# 3\.test: Some diagnostic$' stdout
# 3.test: Some diagnostic
+ grep '^ERROR: 3\.test - Bail out! Kernel Panic$' stdout
ERROR: 3.test - Bail out! Kernel Panic
++ /bin/grep -F -c 3.test stdout
+ test 3 -eq 3
+ cd ..
+ for try in 0 1
+ test 1 -eq 0
+ test 1 -eq 1
+ srcdir=.
+ case $MAKE in
+ echo all:
+ for run_make in '"$MAKE -j3"' '"$MAKE -j 3"' '"$MAKE"'
+ make -j3
make: Nothing to be done for `all'.
+ break
+ rm -f Makefile
+ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
+ ls -l
total 320
-rwxr-xr-x 1 stefano stefano    123 May  5 16:57 1.test
-rwxr-xr-x 1 stefano stefano     80 May  5 16:57 2.test
-rwxr-xr-x 1 stefano stefano    141 May  5 16:57 3.test
-rw-r--r-- 1 stefano stefano  27414 May  5 16:57 Makefile
-rw-r--r-- 1 stefano stefano    101 May  5 16:57 Makefile.am
-rw-r--r-- 1 stefano stefano  26796 May  5 16:57 Makefile.in
-rw-r--r-- 1 stefano stefano  21856 May  5 16:57 aclocal.m4
drwxr-xr-x 2 stefano stefano   4096 May  5 16:57 autom4te.cache
drwxr-xr-x 2 stefano stefano   4096 May  5 16:57 build
-rw-r--r-- 1 stefano stefano   4991 May  5 16:57 config.log
-rwxr-xr-x 1 stefano stefano  25363 May  5 16:57 config.status
-rwxr-xr-x 1 stefano stefano 100365 May  5 16:57 configure
-rw-r--r-- 1 stefano stefano    153 May  5 16:57 configure.ac
-rwxr-xr-x 1 stefano stefano  20842 May  5 16:57 depcomp
-rwxr-xr-x 1 stefano stefano  13997 May  5 16:57 install-sh
-rwxr-xr-x 1 stefano stefano  10179 May  5 16:57 missing
-rwxr-xr-x 1 stefano stefano  19514 May  5 16:57 tap-driver
+ make -j3 check
+ cat stdout
make  check-TESTS
make[1]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir'
make[2]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir'
PASS: 1.test 1 - mu
SKIP: 1.test 2 zardoz # SKIP
PASS: 2.test 1
XFAIL: 2.test 2 # TODO not implemented
PASS: 2.test 3
make[3]: Entering directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir'
============================================================================
Testsuite summary for tap-more 1.0
============================================================================
# TOTAL: 6
# PASS:  4
# SKIP:  1
# XFAIL: 1
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[2]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir'
make[1]: Leaving directory `/devel/stefano/src/am/branches/trunks/t/tap-more.dir'
+ count_test_results total=6 pass=4 fail=0 xpass=0 xfail=1 skip=1 error=0
+ total=ERR
+ pass=ERR
+ fail=ERR
+ xpass=ERR
+ xfail=ERR
+ skip=ERR
+ error=ERR
+ eval total=6 pass=4 fail=0 xpass=0 xfail=1 skip=1 error=0
++ total=6
++ pass=4
++ fail=0
++ xpass=0
++ xfail=1
++ skip=1
++ error=0
+ /bin/grep -E -i '(total|x?pass|x?fail|skip|error)' stdout
PASS: 1.test 1 - mu
SKIP: 1.test 2 zardoz # SKIP
PASS: 2.test 1
XFAIL: 2.test 2 # TODO not implemented
PASS: 2.test 3
# TOTAL: 6
# PASS:  4
# SKIP:  1
# XFAIL: 1
# FAIL:  0
# XPASS: 0
# ERROR: 0
+ rc=0
+ set +e
++ grep -c '^PASS:' stdout
+ test 3 -eq 4
+ rc=1
++ grep -c '^XFAIL:' stdout
+ test 1 -eq 1
++ grep -c '^SKIP:' stdout
+ test 1 -eq 1
++ grep -c '^FAIL:' stdout
+ test 0 -eq 0
++ grep -c '^XPASS:' stdout
+ test 0 -eq 0
++ grep -c '^ERROR:' stdout
+ test 0 -eq 0
+ grep '^# TOTAL:  *6$' stdout
# TOTAL: 6
+ grep '^# PASS:  *4$' stdout
# PASS:  4
+ grep '^# XFAIL:  *1$' stdout
# XFAIL: 1
+ grep '^# SKIP:  *1$' stdout
# SKIP:  1
+ grep '^# FAIL:  *0$' stdout
# FAIL:  0
+ grep '^# XPASS:  *0$' stdout
# XPASS: 0
+ grep '^# ERROR:  *0$' stdout
# ERROR: 0
+ test 1 -eq 0
+ exit_status=1
+ set +e
+ cd /home/stefano/src/am/branches/trunks
+ test no = yes
+ case $am_explicit_skips in
+ test 1 -eq 0
+ keep_testdirs=yes
+ am_keeping_testdirs
+ case $keep_testdirs in
+ return 0
+ set +x
tap-more: exit 1

Message #8 received at 11413 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: 11413 <at> debbugs.gnu.org
Cc: "automake-patches <at> gnu.org" <automake-patches <at> gnu.org>
Subject: Re: bug#11413: Racy failure in test case 'tap-more.sh'
Date: Sat, 05 May 2012 19:34:32 +0200

[Message part 1 (text/plain, inline)]

tags 11413 + patch
close 11413
thanks

On 05/05/2012 05:51 PM, Stefano Lattarini wrote:
> Severity: minor
> 
> I'm sporadically seeing the failure below in the test 'tap-more.sh'.
> 
> Anyone has an insight about what's going on?
>
Such an insight is offered by an older commit:

  commit 642699970e75580e0f11f5786302082e25d8f49c
  Author: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
  Date:   Tue Mar 10 21:33:47 2009 +0100

      Use append mode to capture parallel test output.

      * tests/lisp8.test: Use append mode for output from `make -j',
      to avoid dropped lines.
      * tests/parallel-tests3.test: Likewise.

      Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>

I've verified (with the help of the attached script) that such an issue
is indeed present on the affected systems (my Debian box and the Fedora
ppc64 system offered by the GCC compile farm), and I've pushed the
attached patch to fix it.  I'm thus closing this bug report.

Regards,
  Stefano

[foo.sh (application/x-sh, attachment)]

[0002-tests-use-append-mode-to-capture-parallel-make-outpu.patch (text/x-diff, attachment)]

Message #25 received at 11413 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: 11413 <at> debbugs.gnu.org
Subject: update: parallel make redirections
Date: Sun, 26 May 2013 11:20:17 +0200

UPDATE: one year later

The redirection issues described in this report were also making it
impossible to try the Automake testsuite with parallel make, as with:

    make check AM_TESTSUITE_MAKE="make -j8"

Racy, random failures were in fact often present, making it hard to
trust the results of the command above.  Which was especially annoying
because, with today's fast multicore machines, is becoming more and
more important to ensure Automake-generated Makefiles work correctly
when run in parallel mode.

The issue has been fixed for the testsuite as a whole with a recent
set of commits, the most significant of is commit d19eb85c, "tests:
use append mode to capture parallel make output":

  <http://git.savannah.gnu.org/cgit/automake.git/commit/?h=d19eb85c>
  <http://lists.gnu.org/archive/html/automake-patches/2013-05/msg00129.html>

Regards,
  Stefano

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.