Subject: bug#9245: closed (Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness)

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 18 Aug 2011 19:45:03 +0000

Your bug report

#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness

which was filed against the automake package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
9245: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9245
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness
From: Stefano Lattarini <[email-masked]>
To: [email-masked]
Date: Thu, 18 Aug 2011 21:42:32 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

On Tuesday 16 August 2011, Stefano Lattarini wrote:
> On Friday 12 August 2011, Stefano Lattarini wrote:
> > OK, the attached hacky patch seems to fix the bug.
> >
> LOL no, there was an embarassingly stupid error in my patch:
> 
>  diff --git a/lib/am/check.am b/lib/am/check.am
>  index 0b54312..75f8bba 100644
>  --- a/lib/am/check.am
>  +++ b/lib/am/check.am
>  @@ -224,7 +224,9 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
>            col="$$red";                                                  \
>          fi;                                                             \
>          echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std";         \
>  -       $$exit
>  +## The apparently redundant and repeated "exit 1" works around a bug in
>  +## FreeBSD make (present only when running in concurrent mode).
>  +       $$exit || (exit 1); exit 1
>  
> This obviously causes the `check' target to exit unconditionally with a
> non-zero exit status.  Hardly a fix for anything, sigh.
> 
> The "proper fix" is even easier BTW: just use "$$exit || exit 1" instead.
> 
> The updated patch should now work (and I've tested it properly this time).
> This new patch hasn't been derived by "cargo-cult guessing", but from the
> explanation of the relevant FreeBSD make bug provided by Jilles Tjoelker:
>  <http://www.freebsd.org/cgi/query-pr.cgi?pr=159730>
> 
> Now that's something to be documented in the Autoconf manual I'd say ...
>
BTW, done with commit v2.68-84-g96cdf32 "docs: other issues with parallel BSD
make"; see:
 <http://lists.gnu.org/archive/html/autoconf-patches/2011-08/msg00013.html>

> > I have no idea why
> > exactly it is so yet, so we might want to wait to apply the patch until
> > we have fully understood the reasons of the original failure,
> > 
> Glad I've waited this time!  And I will allow another couple of days for
> comments and suggestions before pushing.
> 
> Regards,
>   Stefano
> 
I've pushed the patch now, with the additional squash-in below.  I'm closing
this bug report.

Regards,
  Stefano

-*-*-

diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
index 2f553ed..17a2002 100644
--- a/lib/Automake/tests/Makefile.in
+++ b/lib/Automake/tests/Makefile.in
@@ -405,7 +405,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
          col="$$red";                                                  \
        fi;                                                             \
        echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std";         \
-       $$exit
+       $$exit || exit 1
 
 # Run all the tests.
 check-TESTS:

Subject: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness
From: Stefano Lattarini <[email-masked]>
To: [email-masked]
Date: Fri, 5 Aug 2011 11:08:20 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Here is a brief illustration of the issue:

  $ cat > Makefile.am <<'END'
  AUTOMAKE_OPTIONS = foreign parallel-tests -Wall -Werror
  TEST_LOG_COMPILER = false
  TESTS = foo.test bar.test
  END
  $ touch foo.test bar.test
  $ autoreconf -vi && ./configure
  ...

  $ freebsd-make -j2 check; echo %% STATUS = $? %%
  freebsd-make  check-TESTS
  FAIL: foo.test
  FAIL: bar.test
  ====================
  2 of 2 tests failed
  See ./test-suite.log
  ====================
  %% STATUS = 0 %%

It works as expected with GNU make and NetBSD make though:

 $ gmake -j2 check; echo %% STATUS = $? %%
  gmake  check-TESTS
  gmake[1]: Entering directory `/tmp/...'
  gmake[2]: Entering directory `/tmp/...'
  FAIL: foo.test
  FAIL: bar.test
  ====================
  2 of 2 tests failed
  See ./test-suite.log
  ====================
  gmake[2]: *** [test-suite.log] Error 1
  gmake[2]: Leaving directory `/tmp/...'
  gmake[1]: *** [check-TESTS] Error 2
  gmake[1]: Leaving directory `/tmp/...'
  gmake: *** [check-am] Error 2
  %% STATUS = 2 %%

  $ netbsd-make -j2 check; echo %% STATUS = $? %%
  --- check-am ---
  netbsd-make  check-TESTS
  --- check-TESTS ---
  --- foo.log ---
  --- bar.log ---
  FAIL: bar.test
  --- foo.log ---
  FAIL: foo.test
  --- test-suite.log ---
  ====================
  2 of 2 tests failed
  See ./test-suite.log
  ====================
  *** [test-suite.log] Error code 1
  1 error
  netbsd-make: stopped in /tmp/...
  *** [check-TESTS] Error code 2
  1 error
  netbsd-make: stopped in /tmp/...
  *** [check-am] Error code 2
  1 error
  netbsd-make: stopped in /tmp/...
  %% STATUS = 2 %%

The problem is present both in automake 1.11.1 and in the developement
version from the 'test-protocols' branch.

Regards,
  Stefano