From unknown Thu Aug 14 21:45:18 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#9245 <9245@debbugs.gnu.org> To: bug#9245 <9245@debbugs.gnu.org> Subject: Status: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness Reply-To: bug#9245 <9245@debbugs.gnu.org> Date: Fri, 15 Aug 2025 04:45:18 +0000 retitle 9245 FreeBSD make in concurrent mode report spurious success in aut= omake-generated tests harness reassign 9245 automake submitter 9245 Stefano Lattarini severity 9245 normal tag 9245 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 05 05:09:56 2011 Received: (at submit) by debbugs.gnu.org; 5 Aug 2011 09:09:56 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QpGPk-0007Af-8j for submit@debbugs.gnu.org; Fri, 05 Aug 2011 05:09:56 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QpGPg-0007AW-Lt for submit@debbugs.gnu.org; Fri, 05 Aug 2011 05:09:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QpGOu-0006gN-7j for submit@debbugs.gnu.org; Fri, 05 Aug 2011 05:09:08 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, T_DKIM_INVALID, T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:49907) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QpGOu-0006gH-6B for submit@debbugs.gnu.org; Fri, 05 Aug 2011 05:09:04 -0400 Received: from eggs.gnu.org ([140.186.70.92]:39656) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QpGOq-0000yC-Cj for bug-automake@gnu.org; Fri, 05 Aug 2011 05:09:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QpGOm-0006fV-I2 for bug-automake@gnu.org; Fri, 05 Aug 2011 05:09:00 -0400 Received: from mail-ww0-f49.google.com ([74.125.82.49]:45836) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QpGOm-0006fR-CN for bug-automake@gnu.org; Fri, 05 Aug 2011 05:08:56 -0400 Received: by wwf10 with SMTP id 10so631630wwf.30 for ; Fri, 05 Aug 2011 02:08:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:content-type :content-transfer-encoding:message-id; bh=XfcNQl7UC3dHWhyOYLW+6dnYMeZuB+ihMvI6qnwKMcQ=; b=hE6Iev0oQ9IDhx2qwZLtFay212BAgMvpbDb8x0oCL5BXkVtyJe00L/+oQqnSI3fn58 rVvfBeuL7sa2UbKZCF6ccWy1Fzz2JNsnUE40kfdC69ffRE0HnFKaXptsgOoUGS3+K/6T EIGrpLmVlPsBOOxpko12mcXo22vshTfkutGOU= Received: by 10.216.79.16 with SMTP id h16mr322192wee.80.1312535335583; Fri, 05 Aug 2011 02:08:55 -0700 (PDT) Received: from bigio.localnet (host78-93-dynamic.7-79-r.retail.telecomitalia.it [79.7.93.78]) by mx.google.com with ESMTPS id gb1sm2125168wbb.54.2011.08.05.02.08.49 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 05 Aug 2011 02:08:52 -0700 (PDT) From: Stefano Lattarini To: bug-automake@gnu.org Subject: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness Date: Fri, 5 Aug 2011 11:08:20 +0200 User-Agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201108051108.21060.stefano.lattarini@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.1 (-----) Here is a brief illustration of the issue: $ cat > Makefile.am <<'END' AUTOMAKE_OPTIONS = foreign parallel-tests -Wall -Werror TEST_LOG_COMPILER = false TESTS = foo.test bar.test END $ touch foo.test bar.test $ autoreconf -vi && ./configure ... $ freebsd-make -j2 check; echo %% STATUS = $? %% freebsd-make check-TESTS FAIL: foo.test FAIL: bar.test ==================== 2 of 2 tests failed See ./test-suite.log ==================== %% STATUS = 0 %% It works as expected with GNU make and NetBSD make though: $ gmake -j2 check; echo %% STATUS = $? %% gmake check-TESTS gmake[1]: Entering directory `/tmp/...' gmake[2]: Entering directory `/tmp/...' FAIL: foo.test FAIL: bar.test ==================== 2 of 2 tests failed See ./test-suite.log ==================== gmake[2]: *** [test-suite.log] Error 1 gmake[2]: Leaving directory `/tmp/...' gmake[1]: *** [check-TESTS] Error 2 gmake[1]: Leaving directory `/tmp/...' gmake: *** [check-am] Error 2 %% STATUS = 2 %% $ netbsd-make -j2 check; echo %% STATUS = $? %% --- check-am --- netbsd-make check-TESTS --- check-TESTS --- --- foo.log --- --- bar.log --- FAIL: bar.test --- foo.log --- FAIL: foo.test --- test-suite.log --- ==================== 2 of 2 tests failed See ./test-suite.log ==================== *** [test-suite.log] Error code 1 1 error netbsd-make: stopped in /tmp/... *** [check-TESTS] Error code 2 1 error netbsd-make: stopped in /tmp/... *** [check-am] Error code 2 1 error netbsd-make: stopped in /tmp/... %% STATUS = 2 %% The problem is present both in automake 1.11.1 and in the developement version from the 'test-protocols' branch. Regards, Stefano From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 12 12:34:47 2011 Received: (at submit) by debbugs.gnu.org; 12 Aug 2011 16:34:47 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qruh4-0003Sd-DZ for submit@debbugs.gnu.org; Fri, 12 Aug 2011 12:34:47 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qruh2-0003SW-4r for submit@debbugs.gnu.org; Fri, 12 Aug 2011 12:34:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Qrufc-0007Nj-JP for submit@debbugs.gnu.org; Fri, 12 Aug 2011 12:33:18 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, T_DKIM_INVALID, T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:53950) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qrufc-0007Nf-Hp for submit@debbugs.gnu.org; Fri, 12 Aug 2011 12:33:16 -0400 Received: from eggs.gnu.org ([140.186.70.92]:54947) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qrufa-0003oM-V3 for bug-automake@gnu.org; Fri, 12 Aug 2011 12:33:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QrufZ-0007N5-75 for bug-automake@gnu.org; Fri, 12 Aug 2011 12:33:14 -0400 Received: from mail-fx0-f41.google.com ([209.85.161.41]:36464) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrufY-0007Mz-UK for bug-automake@gnu.org; Fri, 12 Aug 2011 12:33:13 -0400 Received: by fxg9 with SMTP id 9so2904015fxg.0 for ; Fri, 12 Aug 2011 09:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:message-id; bh=PpXR48dpdqUs+RqcEE9iULXEQajWFriDK4BrEonRTu8=; b=K9vdV1TVX+gHCPTpfiUb9EvXv65tLsI1ZbjYmPKSjwIS/0paE8vNrIqAWpTdPpG964 uuu6O4R5gVz1jNxF8ROEH3+51zK4DW7uccCqFxwPi3iiykRNUFq9l3zKcSkjVUewfpir WEGdbtERcjTT8w7GtZpqkz6GrsPUx90GoMAkw= Received: by 10.223.60.65 with SMTP id o1mr1525738fah.58.1313166791903; Fri, 12 Aug 2011 09:33:11 -0700 (PDT) Received: from bigio.localnet (host164-50-dynamic.58-82-r.retail.telecomitalia.it [82.58.50.164]) by mx.google.com with ESMTPS id l22sm2554808fam.13.2011.08.12.09.33.07 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 12 Aug 2011 09:33:08 -0700 (PDT) From: Stefano Lattarini To: bug-automake@gnu.org Subject: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness Date: Fri, 12 Aug 2011 18:32:57 +0200 User-Agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) References: <201108051108.21060.stefano.lattarini@gmail.com> In-Reply-To: <201108051108.21060.stefano.lattarini@gmail.com> MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_6WVROxfIQRFmmoP" Message-Id: <201108121832.58440.stefano.lattarini@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit Cc: 9245@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.1 (-----) --Boundary-00=_6WVROxfIQRFmmoP Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit On Friday 05 August 2011, Stefano Lattarini wrote: > Here is a brief illustration of the issue: > > $ cat > Makefile.am <<'END' > AUTOMAKE_OPTIONS = foreign parallel-tests -Wall -Werror > TEST_LOG_COMPILER = false > TESTS = foo.test bar.test > END > $ touch foo.test bar.test > $ autoreconf -vi && ./configure > ... > > $ freebsd-make -j2 check; echo %% STATUS = $? %% > freebsd-make check-TESTS > FAIL: foo.test > FAIL: bar.test > ==================== > 2 of 2 tests failed > See ./test-suite.log > ==================== > %% STATUS = 0 %% > > It works as expected with GNU make and NetBSD make though: > > $ gmake -j2 check; echo %% STATUS = $? %% > gmake check-TESTS > gmake[1]: Entering directory `/tmp/...' > gmake[2]: Entering directory `/tmp/...' > FAIL: foo.test > FAIL: bar.test > ==================== > 2 of 2 tests failed > See ./test-suite.log > ==================== > gmake[2]: *** [test-suite.log] Error 1 > gmake[2]: Leaving directory `/tmp/...' > gmake[1]: *** [check-TESTS] Error 2 > gmake[1]: Leaving directory `/tmp/...' > gmake: *** [check-am] Error 2 > %% STATUS = 2 %% > > > $ netbsd-make -j2 check; echo %% STATUS = $? %% > --- check-am --- > netbsd-make check-TESTS > --- check-TESTS --- > --- foo.log --- > --- bar.log --- > FAIL: bar.test > --- foo.log --- > FAIL: foo.test > --- test-suite.log --- > ==================== > 2 of 2 tests failed > See ./test-suite.log > ==================== > *** [test-suite.log] Error code 1 > 1 error > netbsd-make: stopped in /tmp/... > *** [check-TESTS] Error code 2 > 1 error > netbsd-make: stopped in /tmp/... > *** [check-am] Error code 2 > 1 error > netbsd-make: stopped in /tmp/... > %% STATUS = 2 %% > > The problem is present both in automake 1.11.1 and in the developement > version from the 'test-protocols' branch. > OK, the attached hacky patch seems to fix the bug. I have no idea why exactly it is so yet, so we might want to wait to apply the patch until we have fully understood the reasons of the original failure, and why the proposed patch seems to fix it. Regards, Stefano --Boundary-00=_6WVROxfIQRFmmoP Content-Type: text/x-patch; charset="utf-8"; name="0001-parallel-tests-no-more-spurious-successes-for-FreeBS.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline; filename="0001-parallel-tests-no-more-spurious-successes-for-FreeBS.patch" =46rom e7568ee0cfaccc5c9998275a5d93643a49ea533d Mon Sep 17 00:00:00 2001 Message-Id: =46rom: Stefano Lattarini Date: Fri, 12 Aug 2011 18:29:28 +0200 Subject: [PATCH] parallel-tests: no more spurious successes for FreeBSD make Work around a bug of FreeBSD make bug that was causing the automake-generated "check" target to complete with success even if some tests failed; this happened only when FreeBSD make was run in concurrent mode (as in, e.g., "make -j2 check"). The bug is not present in NetBSD make. This change fixes automake bug#9245. * lib/am/check.am [%?PARALLEL_TESTS%] $(TEST_SUITE_LOG): Use a more "safe" (and apparently redundant) idiom to exit with error, so that the non-zero exit status is picked up also by FreeBSD make when it's running in concurrent mode. * NEWS: Update. * tests/parallel-tests-concurrency-bug9245.test: New test. * tests/Makefile.am (TESTS): Update. =2D-- ChangeLog | 17 ++++++++ NEWS | 4 +- lib/Automake/tests/Makefile.in | 2 +- lib/am/check.am | 4 +- tests/Makefile.am | 1 + tests/Makefile.in | 3 +- tests/parallel-tests-concurrency-bug9245.test | 55 +++++++++++++++++++++= ++++ 7 files changed, 82 insertions(+), 4 deletions(-) create mode 100755 tests/parallel-tests-concurrency-bug9245.test diff --git a/ChangeLog b/ChangeLog index af2556f..f7a3565 100644 =2D-- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,20 @@ +2011-08-12 Stefano Lattarini + + parallel-tests: no more spurious successes for FreeBSD make + Work around a bug of FreeBSD make bug that was causing the + automake-generated "check" target to complete with success + even if some tests failed; this happened only when FreeBSD + make was run in concurrent mode (as in, e.g., "make -j2 + check"). The bug is not present in NetBSD make. + This change fixes automake bug#9245. + * lib/am/check.am [%?PARALLEL_TESTS%] $(TEST_SUITE_LOG): Use a + more "safe" (and apparently redundant) idiom to exit with error, + so that the non-zero exit status is picked up also by FreeBSD + make when it's running in concurrent mode. + * NEWS: Update. + * tests/parallel-tests-concurrency-bug9245.test: New test. + * tests/Makefile.am (TESTS): Update. + 2011-08-08 Stefano Lattarini =20 test defs: more environment cleanup diff --git a/NEWS b/NEWS index 29c552b..5086f75 100644 =2D-- a/NEWS +++ b/NEWS @@ -23,7 +23,9 @@ Bugs fixed in 1.11.0a: * Bugs introduced by 1.11: =20 - The `parallel-tests' test driver works around a GNU make 3.80 bug with =2D trailing white space in the test list (`TESTS =3D foo $(EMPTY)'). + trailing white space in the test list (`TESTS =3D foo $(EMPTY)'), and + does not report spurious successes when used with concurrent FreeBSD + make (e.g., "make check -j3"). =20 - The `silent-rules' option now also silences all compile rules if depen= dency tracking is disabled. Also, when `silent-rules' is not used, the outp= ut from diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in index 2f553ed..1cb438c 100644 =2D-- a/lib/Automake/tests/Makefile.in +++ b/lib/Automake/tests/Makefile.in @@ -405,7 +405,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) col=3D"$$red"; \ fi; \ echo "$$msg" | $(am__text_box) "col=3D$$col" "std=3D$$std"; \ =2D $$exit + $$exit || (exit 1); exit 1 =20 # Run all the tests. check-TESTS: diff --git a/lib/am/check.am b/lib/am/check.am index 0b54312..75f8bba 100644 =2D-- a/lib/am/check.am +++ b/lib/am/check.am @@ -224,7 +224,9 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) col=3D"$$red"; \ fi; \ echo "$$msg" | $(am__text_box) "col=3D$$col" "std=3D$$std"; \ =2D $$exit +## The apparently redundant and repeated "exit 1" works around a bug in +## FreeBSD make (present only when running in concurrent mode). + $$exit || (exit 1); exit 1 =20 RECHECK_LOGS =3D $(TEST_LOGS) =20 diff --git a/tests/Makefile.am b/tests/Makefile.am index b545dda..a1f7818 100644 =2D-- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -605,6 +605,7 @@ parallel-tests-log-override-1.test \ parallel-tests-log-override-2.test \ parallel-tests-log-override-recheck.test \ parallel-tests-log-compiler-example.test \ +parallel-tests-concurrency-bug9245.test \ parse.test \ percent.test \ percent2.test \ diff --git a/tests/Makefile.in b/tests/Makefile.in index fe44a40..7bb6586 100644 =2D-- a/tests/Makefile.in +++ b/tests/Makefile.in @@ -882,6 +882,7 @@ parallel-tests-log-override-1.test \ parallel-tests-log-override-2.test \ parallel-tests-log-override-recheck.test \ parallel-tests-log-compiler-example.test \ +parallel-tests-concurrency-bug9245.test \ parse.test \ percent.test \ percent2.test \ @@ -1274,7 +1275,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) col=3D"$$red"; \ fi; \ echo "$$msg" | $(am__text_box) "col=3D$$col" "std=3D$$std"; \ =2D $$exit + $$exit || (exit 1); exit 1 =20 # Run all the tests. check-TESTS: diff --git a/tests/parallel-tests-concurrency-bug9245.test b/tests/parallel= =2Dtests-concurrency-bug9245.test new file mode 100755 index 0000000..78bd98d =2D-- /dev/null +++ b/tests/parallel-tests-concurrency-bug9245.test @@ -0,0 +1,55 @@ +#! /bin/sh +# Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2, or (at your option) +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Look for a bug where FreeBSD make in concurrent mode reported success +# even when the Automake-generated parallel testsuite harness failed. +# See automake bug#9245. + +parallel_tests=3Dyes +. ./defs || Exit 1 + +cat >> configure.in << 'END' +AC_OUTPUT +END + +cat > Makefile.am << 'END' +TEST_LOG_COMPILER =3D false +TESTS =3D foo.test bar.test +END + +: > foo.test +: > bar.test + +$ACLOCAL +$AUTOCONF +$AUTOMAKE -a + +./configure + +# Some make implementations don't grok the `-j' option. +$MAKE -j1 || Exit 77 + +for j in '' -j1 -j2; do + $MAKE $j check && Exit 1 + $MAKE $j recheck && Exit 1 + TESTS=3Dfoo.test $MAKE $j -e check && Exit 1 + TEST_LOGS=3Dfoo.log $MAKE $j -e check && Exit 1 + rm -f test-suite.log + $MAKE $j test-suite.log && Exit 1 + test -f test-suite.log +done + +: =2D-=20 1.7.2.3 --Boundary-00=_6WVROxfIQRFmmoP-- From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 12 14:51:07 2011 Received: (at submit) by debbugs.gnu.org; 12 Aug 2011 18:51:07 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qrwp0-0006jm-UK for submit@debbugs.gnu.org; Fri, 12 Aug 2011 14:51:07 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qrwoz-0006jf-0W for submit@debbugs.gnu.org; Fri, 12 Aug 2011 14:51:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QrwnY-0001TQ-JW for submit@debbugs.gnu.org; Fri, 12 Aug 2011 14:49:38 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, HTML_MESSAGE,RCVD_IN_DNSWL_LOW,T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:44057) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrwnY-0001TE-FC for submit@debbugs.gnu.org; Fri, 12 Aug 2011 14:49:36 -0400 Received: from eggs.gnu.org ([140.186.70.92]:36292) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrwnX-0007N6-8p for bug-automake@gnu.org; Fri, 12 Aug 2011 14:49:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QrwnV-0001Rl-Lj for bug-automake@gnu.org; Fri, 12 Aug 2011 14:49:35 -0400 Received: from mail-fx0-f41.google.com ([209.85.161.41]:53236) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrwnV-0001RG-3l; Fri, 12 Aug 2011 14:49:33 -0400 Received: by fxg9 with SMTP id 9so2979626fxg.0 for ; Fri, 12 Aug 2011 11:49:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:x-kmail-markup:content-type:content-transfer-encoding :message-id; bh=BJeImtNu0r0eaWZjvUEmZIxuIZbVSDKxTcjdjGAwTtM=; b=Zs3AqnuvwrskT1m4TR2WCwHGoE/uj1cSyeIgrpeNcbNF4YEJGjlPW9uk38UMxpEygY uQ31YiD/MSX3hL/+K2sHjn7vMDx65edu3ILNWHWwuLwXzLK0ShGpjnP7VvUkKAMBqBmv v44H8XAM8DiDPSW8/W3NbFpPS5CEDQlZUk7qA= Received: by 10.223.159.207 with SMTP id k15mr1700224fax.82.1313174971753; Fri, 12 Aug 2011 11:49:31 -0700 (PDT) Received: from bigio.localdomain (host25-103-dynamic.2-87-r.retail.telecomitalia.it [87.2.103.25]) by mx.google.com with ESMTPS id 6sm2646813fas.2.2011.08.12.11.49.20 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 12 Aug 2011 11:49:28 -0700 (PDT) From: Stefano Lattarini To: bug-automake@gnu.org Subject: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness Date: Fri, 12 Aug 2011 20:49:06 +0200 User-Agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) References: <201108051108.21060.stefano.lattarini@gmail.com> <201108121832.58440.stefano.lattarini@gmail.com> In-Reply-To: <201108121832.58440.stefano.lattarini@gmail.com> MIME-Version: 1.0 X-KMail-Markup: true Content-Type: multipart/alternative; boundary="Boundary-01=_jWXRO+2wZAmUztt" Content-Transfer-Encoding: 7bit Message-Id: <201108122049.07174.stefano.lattarini@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit Cc: bug-autoconf@gnu.org, 9245@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.1 (-----) --Boundary-01=_jWXRO+2wZAmUztt Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit [CC:ing bug-autoconf, about a FreeBSD make bug] On Friday 12 August 2011, Stefano Lattarini wrote: > OK, the attached hacky patch seems to fix the bug. I have no idea why > exactly it is so yet, so we might want to wait to apply the patch until > we have fully understood the reasons of the original failure, and why > the proposed patch seems to fix it. > I have managed to reproduce the FreeBSD make bug with this one liner: $ echo 'all: ; @set +e; false' | make -j2 -f- Notice that all of `-j', `@' and `set +e' are required to trigger the bug: $ echo 'all: ; set +e; false' | make -j2 -f- set +e; false *** Error code 1 1 error $ echo 'all: ; @false' | make -j2 -f- *** Error code 1 1 error $ echo 'all: ; @set +e; false' | make -f- *** Error code 1 Stop in /tmp. A well-placed use of ps might give an hint about the problem: $ echo 'all: ; @set +e; ps -o args $$$$' | make -f- -j2 COMMAND sh -ev That `-v' flag seems truly fishy ... System information: $ uname -rsm FreeBSD 8.2-RELEASE i386 $ echo 'all: ; @echo $(MAKE_VERSION)' | make -f- 5200408120 This issue should IMHO be probably documented in the autoconf manual, and reported to the FreeBSD developers. I'll try to do that soonish (couple of days) if nobody beats me. Regards, Stefano --Boundary-01=_jWXRO+2wZAmUztt Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 7bit

[CC:ing bug-autoconf, about a FreeBSD make bug]

On Friday 12 August 2011, Stefano Lattarini wrote:

> OK, the attached hacky patch seems to fix the bug. I have no idea why

> exactly it is so yet, so we might want to wait to apply the patch until

> we have fully understood the reasons of the original failure, and why

> the proposed patch seems to fix it.

>

I have managed to reproduce the FreeBSD make bug with this one liner:

$ echo 'all: ; @set +e; false' | make -j2 -f-

Notice that all of `-j', `@' and `set +e' are required to trigger the bug:

$ echo 'all: ; set +e; false' | make -j2 -f-

set +e; false

*** Error code 1

1 error

$ echo 'all: ; @false' | make -j2 -f-

*** Error code 1

1 error

$ echo 'all: ; @set +e; false' | make -f-

*** Error code 1

Stop in /tmp.

A well-placed use of ps might give an hint about the problem:

$ echo 'all: ; @set +e; ps -o args $$$$' | make -f- -j2

COMMAND

sh -ev

That `-v' flag seems truly fishy ...

System information:

$ uname -rsm

FreeBSD 8.2-RELEASE i386

$ echo 'all: ; @echo $(MAKE_VERSION)' | make -f-

5200408120

This issue should IMHO be probably documented in the autoconf manual, and

reported to the FreeBSD developers. I'll try to do that soonish (couple

of days) if nobody beats me.

Regards,

Stefano

--Boundary-01=_jWXRO+2wZAmUztt-- From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 12 17:30:12 2011 Received: (at submit) by debbugs.gnu.org; 12 Aug 2011 21:30:12 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QrzIy-00027d-68 for submit@debbugs.gnu.org; Fri, 12 Aug 2011 17:30:12 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QrzIw-00027W-2F for submit@debbugs.gnu.org; Fri, 12 Aug 2011 17:30:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QrzHV-0006xs-QY for submit@debbugs.gnu.org; Fri, 12 Aug 2011 17:28:43 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, HTML_MESSAGE,RCVD_IN_DNSWL_LOW,T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:37178) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrzHV-0006xg-Fq for submit@debbugs.gnu.org; Fri, 12 Aug 2011 17:28:41 -0400 Received: from eggs.gnu.org ([140.186.70.92]:53573) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrzHU-00025B-FU for bug-automake@gnu.org; Fri, 12 Aug 2011 17:28:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QrzHT-0006xU-L2 for bug-automake@gnu.org; Fri, 12 Aug 2011 17:28:40 -0400 Received: from mail-fx0-f41.google.com ([209.85.161.41]:54735) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrzHT-0006xQ-GR; Fri, 12 Aug 2011 17:28:39 -0400 Received: by fxg9 with SMTP id 9so3056442fxg.0 for ; Fri, 12 Aug 2011 14:28:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :x-kmail-markup:mime-version:content-type:content-transfer-encoding :message-id; bh=OyCJhQo1ljs81iLep1/+YdS7tCkCpWJtSsSh23L7Dz8=; b=P6dDRQP75K1BdT46hiOxRpuB9Sc3WfYJrSJ8Rb4t3bG8W0AxbCvsUB2NhHWS345D9z pxuaeCE1UXmq4AGNDcylEGYSl149m1LJ1xklSgg17acJoIW3n8T+LY+rhMdgOL7Gowsv 2l2/TkriboPAj24XGDrXjQ3J3oCNjJrCOxuMo= Received: by 10.223.161.214 with SMTP id s22mr1825628fax.59.1313184518576; Fri, 12 Aug 2011 14:28:38 -0700 (PDT) Received: from bigio.localdomain (host25-103-dynamic.2-87-r.retail.telecomitalia.it [87.2.103.25]) by mx.google.com with ESMTPS id y15sm2745874fah.11.2011.08.12.14.28.36 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 12 Aug 2011 14:28:37 -0700 (PDT) From: Stefano Lattarini To: bug-automake@gnu.org Subject: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness Date: Fri, 12 Aug 2011 23:28:21 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.2-RELEASE; KDE/4.5.5; i386; ; ) References: <201108051108.21060.stefano.lattarini@gmail.com> <201108121832.58440.stefano.lattarini@gmail.com> <201108122049.07174.stefano.lattarini@gmail.com> In-Reply-To: <201108122049.07174.stefano.lattarini@gmail.com> X-KMail-Markup: true MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="Boundary-01=_2rZROp/i2nv53fl" Content-Transfer-Encoding: 7bit Message-Id: <201108122328.22621.stefano.lattarini@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -5.1 (-----) X-Debbugs-Envelope-To: submit Cc: bug-autoconf@gnu.org, 9245@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.1 (-----) --Boundary-01=_2rZROp/i2nv53fl Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On Friday 12 August 2011, Stefano Lattarini wrote: > [CC:ing bug-autoconf, about a FreeBSD make bug] > > [SNIP] > > This issue should IMHO be probably reported to the FreeBSD developers > Done, see freebsd pr `bin/159730': Regards, Stefano --Boundary-01=_2rZROp/i2nv53fl Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 7bit

On Friday 12 August 2011, Stefano Lattarini wrote:

> [CC:ing bug-autoconf, about a FreeBSD make bug]

>

> [SNIP]

>

> This issue should IMHO be probably reported to the FreeBSD developers

>

Done, see freebsd pr `bin/159730':

<http://www.freebsd.org/cgi/query-pr.cgi?pr=159730>

Regards,

Stefano

--Boundary-01=_2rZROp/i2nv53fl-- From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 16 12:06:58 2011 Received: (at submit) by debbugs.gnu.org; 16 Aug 2011 16:06:59 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QtMAM-0006wC-CA for submit@debbugs.gnu.org; Tue, 16 Aug 2011 12:06:58 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QtMAF-0006vw-0t for submit@debbugs.gnu.org; Tue, 16 Aug 2011 12:06:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QtM8S-0007g7-M9 for submit@debbugs.gnu.org; Tue, 16 Aug 2011 12:05:02 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, T_DKIM_INVALID, T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:53228) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QtM8S-0007g1-Hj for submit@debbugs.gnu.org; Tue, 16 Aug 2011 12:05:00 -0400 Received: from eggs.gnu.org ([140.186.70.92]:52731) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QtM8Q-0001UR-6U for bug-automake@gnu.org; Tue, 16 Aug 2011 12:05:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QtM8O-0007fP-82 for bug-automake@gnu.org; Tue, 16 Aug 2011 12:04:58 -0400 Received: from mail-wy0-f169.google.com ([74.125.82.169]:38024) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QtM8N-0007fI-Va for bug-automake@gnu.org; Tue, 16 Aug 2011 12:04:56 -0400 Received: by wyi11 with SMTP id 11so30948wyi.0 for ; Tue, 16 Aug 2011 09:04:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:message-id; bh=lNAmpa//wKvux5RxVv2+UQNOVx8QVeUWZNktkybICiI=; b=UUkZtlQfXjmcI3o7BrZcEDOYv3838mYbbyJxeGj/mci+Fzue+g6PS3N13R6OWYFdsj ZOdFePkwTUhg0sfLx2vrr1Ri1pvdkY/yGb8vRtDPFjKmoo4qBLnObMWt6ukLkqwiWocL aGNtHjC4OoC468b+52TM7JQqYH/v7cAxzH9hg= Received: by 10.227.208.140 with SMTP id gc12mr4645579wbb.32.1313510694459; Tue, 16 Aug 2011 09:04:54 -0700 (PDT) Received: from bigio.localnet (host175-92-dynamic.4-87-r.retail.telecomitalia.it [87.4.92.175]) by mx.google.com with ESMTPS id 8sm141096wbx.51.2011.08.16.09.04.50 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 16 Aug 2011 09:04:51 -0700 (PDT) From: Stefano Lattarini To: bug-automake@gnu.org Subject: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness Date: Tue, 16 Aug 2011 18:04:38 +0200 User-Agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) References: <201108051108.21060.stefano.lattarini@gmail.com> <201108121832.58440.stefano.lattarini@gmail.com> In-Reply-To: <201108121832.58440.stefano.lattarini@gmail.com> MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_XUpSOUzhPmx/DiX" Message-Id: <201108161804.39354.stefano.lattarini@gmail.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -5.1 (-----) X-Debbugs-Envelope-To: submit Cc: 9245@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.1 (-----) --Boundary-00=_XUpSOUzhPmx/DiX Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On Friday 12 August 2011, Stefano Lattarini wrote: > OK, the attached hacky patch seems to fix the bug. > LOL no, there was an embarassingly stupid error in my patch: diff --git a/lib/am/check.am b/lib/am/check.am index 0b54312..75f8bba 100644 --- a/lib/am/check.am +++ b/lib/am/check.am @@ -224,7 +224,9 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) col="$$red"; \ fi; \ echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \ - $$exit +## The apparently redundant and repeated "exit 1" works around a bug in +## FreeBSD make (present only when running in concurrent mode). + $$exit || (exit 1); exit 1 This obviously causes the `check' target to exit unconditionally with a non-zero exit status. Hardly a fix for anything, sigh. The "proper fix" is even easier BTW: just use "$$exit || exit 1" instead. The updated patch should now work (and I've tested it properly this time). This new patch hasn't been derived by "cargo-cult guessing", but from the explanation of the relevant FreeBSD make bug provided by Jilles Tjoelker: Now that's something to be documented in the Autoconf manual I'd say ... > I have no idea why > exactly it is so yet, so we might want to wait to apply the patch until > we have fully understood the reasons of the original failure, > Glad I've waited this time! And I will allow another couple of days for comments and suggestions before pushing. Regards, Stefano --Boundary-00=_XUpSOUzhPmx/DiX Content-Type: text/x-patch; charset="utf-8"; name="0001-parallel-tests-no-more-spurious-successes-for-FreeBS.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline; filename="0001-parallel-tests-no-more-spurious-successes-for-FreeBS.patch" =46rom 3dfdeba20496399036825e321830e6079949e2ce Mon Sep 17 00:00:00 2001 Message-Id: <3dfdeba20496399036825e321830e6079949e2ce.1313510614.git.stefan= o.lattarini@gmail.com> =46rom: Stefano Lattarini Date: Fri, 12 Aug 2011 18:29:28 +0200 Subject: [PATCH] parallel-tests: no more spurious successes for FreeBSD make Work around a bug of FreeBSD make bug that was causing the automake-generated "check" target to complete with success even if some tests failed; this happened only when FreeBSD make was run in concurrent mode (as in, e.g., "make -j2 check"). The bug is not present in NetBSD make. This change fixes automake bug#9245: See also FreeBSD PR bin/159730: * lib/am/check.am [%?PARALLEL_TESTS%] $(TEST_SUITE_LOG): Use a more "safe" (and apparently redundant) idiom to exit with error, so that the non-zero exit status is picked up also by FreeBSD make when it's running in concurrent mode. * NEWS: Update. * tests/check-concurrency-bug9245.test: New test. * tests/Makefile.am (TESTS): Update. =2D-- ChangeLog | 20 +++++++++++ NEWS | 4 ++- lib/am/check.am | 7 +++- tests/Makefile.am | 1 + tests/Makefile.in | 4 ++- tests/check-concurrency-bug9245.test | 62 ++++++++++++++++++++++++++++++= ++++ 6 files changed, 95 insertions(+), 3 deletions(-) create mode 100755 tests/check-concurrency-bug9245.test diff --git a/ChangeLog b/ChangeLog index af2556f..9be4bbf 100644 =2D-- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,23 @@ +2011-08-16 Stefano Lattarini + + parallel-tests: no more spurious successes for FreeBSD make + Work around a bug of FreeBSD make bug that was causing the + automake-generated "check" target to complete with success + even if some tests failed; this happened only when FreeBSD + make was run in concurrent mode (as in, e.g., "make -j2 + check"). The bug is not present in NetBSD make. + This change fixes automake bug#9245: + + See also FreeBSD PR bin/159730: + + * lib/am/check.am [%?PARALLEL_TESTS%] $(TEST_SUITE_LOG): Use a + more "safe" (and apparently redundant) idiom to exit with error, + so that the non-zero exit status is picked up also by FreeBSD + make when it's running in concurrent mode. + * NEWS: Update. + * tests/check-concurrency-bug9245.test: New test. + * tests/Makefile.am (TESTS): Update. + 2011-08-08 Stefano Lattarini =20 test defs: more environment cleanup diff --git a/NEWS b/NEWS index 29c552b..5086f75 100644 =2D-- a/NEWS +++ b/NEWS @@ -23,7 +23,9 @@ Bugs fixed in 1.11.0a: * Bugs introduced by 1.11: =20 - The `parallel-tests' test driver works around a GNU make 3.80 bug with =2D trailing white space in the test list (`TESTS =3D foo $(EMPTY)'). + trailing white space in the test list (`TESTS =3D foo $(EMPTY)'), and + does not report spurious successes when used with concurrent FreeBSD + make (e.g., "make check -j3"). =20 - The `silent-rules' option now also silences all compile rules if depen= dency tracking is disabled. Also, when `silent-rules' is not used, the outp= ut from diff --git a/lib/am/check.am b/lib/am/check.am index 0b54312..3d0188d 100644 =2D-- a/lib/am/check.am +++ b/lib/am/check.am @@ -224,7 +224,12 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) col=3D"$$red"; \ fi; \ echo "$$msg" | $(am__text_box) "col=3D$$col" "std=3D$$std"; \ =2D $$exit +## The use of "exit 1" below is required to work around a FreeBSD make bug +## (present only when running in concurrent mode). See automake bug#9245: +## +## and FreeBSD PR bin/159730: +## . + $$exit || exit 1 =20 RECHECK_LOGS =3D $(TEST_LOGS) =20 diff --git a/tests/Makefile.am b/tests/Makefile.am index b545dda..c2a55fd 100644 =2D-- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -160,6 +160,7 @@ check11.test \ check-exported-srcdir.test \ check-tests-in-builddir.test \ check-tests_environment.test \ +check-concurrency-bug9245.test \ tests-environment-backcompat.test \ checkall.test \ clean.test \ diff --git a/tests/Makefile.in b/tests/Makefile.in index fe44a40..5a3259e 100644 =2D-- a/tests/Makefile.in +++ b/tests/Makefile.in @@ -291,6 +291,7 @@ yacc-dist-nobuild-subdir.test \ txinfo5.test =20 parallel_tests =3D \ +check-concurrency-bug9245-p.test \ check-exported-srcdir-p.test \ check-tests-in-builddir-p.test \ check-tests_environment-p.test \ @@ -437,6 +438,7 @@ check11.test \ check-exported-srcdir.test \ check-tests-in-builddir.test \ check-tests_environment.test \ +check-concurrency-bug9245.test \ tests-environment-backcompat.test \ checkall.test \ clean.test \ @@ -1274,7 +1276,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) col=3D"$$red"; \ fi; \ echo "$$msg" | $(am__text_box) "col=3D$$col" "std=3D$$std"; \ =2D $$exit + $$exit || exit 1 =20 # Run all the tests. check-TESTS: diff --git a/tests/check-concurrency-bug9245.test b/tests/check-concurrency= =2Dbug9245.test new file mode 100755 index 0000000..e5dd153 =2D-- /dev/null +++ b/tests/check-concurrency-bug9245.test @@ -0,0 +1,62 @@ +#! /bin/sh +# Copyright (C) 2011 Free Software Foundation, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2, or (at your option) +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Look for a bug where FreeBSD make in concurrent mode reported success +# even when the Automake-generated parallel testsuite harness failed. +# See automake bug#9245. + +. ./defs || Exit 1 + +cat >> configure.in << 'END' +AC_OUTPUT +END + +cat > Makefile.am << 'END' +TESTS =3D foo.test bar.test +END + +cat > foo.test <<'END' +#!/bin/sh +exit 1 +END +chmod a+x foo.test + +cp foo.test bar.test + +$ACLOCAL +$AUTOCONF +$AUTOMAKE -a + +./configure + +# Some make implementations don't grok the `-j' option. +$MAKE -j1 || Exit 77 + +for j in '' -j1 -j2; do + $MAKE $j check && Exit 1 + TESTS=3Dfoo.test $MAKE $j -e check && Exit 1 + if test x"$parallel_tests" =3D x"yes"; then + $MAKE $j recheck && Exit 1 + TEST_LOGS=3Dfoo.log $MAKE $j -e check && Exit 1 + rm -f test-suite.log + $MAKE $j test-suite.log && Exit 1 + test -f test-suite.log || Exit 1 + else + : # For shells with buggy 'set -e'. + fi +done + +: =2D-=20 1.7.2.3 --Boundary-00=_XUpSOUzhPmx/DiX-- From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 16 12:20:50 2011 Received: (at 9245) by debbugs.gnu.org; 16 Aug 2011 16:20:50 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QtMNl-0007GY-Ts for submit@debbugs.gnu.org; Tue, 16 Aug 2011 12:20:50 -0400 Received: from mx1.redhat.com ([209.132.183.28]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QtMNe-0007GL-2y for 9245@debbugs.gnu.org; Tue, 16 Aug 2011 12:20:48 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p7GGIrOJ010028 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 16 Aug 2011 12:18:53 -0400 Received: from [10.3.113.118] (ovpn-113-118.phx2.redhat.com [10.3.113.118]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p7GGIpWU025180; Tue, 16 Aug 2011 12:18:52 -0400 Message-ID: <4E4A986B.2020308@redhat.com> Date: Tue, 16 Aug 2011 10:18:51 -0600 From: Eric Blake Organization: Red Hat User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110621 Fedora/3.1.11-1.fc14 Lightning/1.0b3pre Mnenhy/0.8.3 Thunderbird/3.1.11 MIME-Version: 1.0 To: Stefano Lattarini Subject: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness References: <201108051108.21060.stefano.lattarini@gmail.com> <201108121832.58440.stefano.lattarini@gmail.com> <201108161804.39354.stefano.lattarini@gmail.com> In-Reply-To: <201108161804.39354.stefano.lattarini@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Spam-Score: -10.3 (----------) X-Debbugs-Envelope-To: 9245 Cc: Autoconf , 9245@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -10.3 (----------) On 08/16/2011 10:04 AM, Stefano Lattarini wrote: > The "proper fix" is even easier BTW: just use "$$exit || exit 1" instead. > > The updated patch should now work (and I've tested it properly this time). > This new patch hasn't been derived by "cargo-cult guessing", but from the > explanation of the relevant FreeBSD make bug provided by Jilles Tjoelker: > > > Now that's something to be documented in the Autoconf manual I'd say ... Indeed. So it all boils down to BSD make rewriting rules (but only for make -j) to be roughly: sh -c 'set -v; your rules; set +v' and losing your exit status (not to mention sharing shell status between commands that are supposed to be in independent shells), when it really should have been rewriting as something more complex: sh -c '(your rule 1); (your rule 2); st=$?; set +v; exit $st' And your workaround of an explicit exit rather than relying on $? at the end of your 'set +e' invocation is indeed the proper way to force the parallel make bug to be avoided, by exiting prior to any post-commands injected by the BSD make rewriting going on that corrupt $?. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 16 12:30:18 2011 Received: (at 9245) by debbugs.gnu.org; 16 Aug 2011 16:30:18 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QtMWu-0007Tw-TZ for submit@debbugs.gnu.org; Tue, 16 Aug 2011 12:30:17 -0400 Received: from mail-wy0-f172.google.com ([74.125.82.172]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QtMWs-0007Tn-8F for 9245@debbugs.gnu.org; Tue, 16 Aug 2011 12:30:15 -0400 Received: by wyg8 with SMTP id 8so42038wyg.3 for <9245@debbugs.gnu.org>; Tue, 16 Aug 2011 09:28:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:content-transfer-encoding:message-id; bh=nuX0BPQY2avUiX7/YkAyZx1PCfM8SP709ihM50vB59k=; b=LQvsaTenzmCUbK/AJhsFbPReoCzNhzHxu3nVuk77N3CWDB0k9kkCoWbY5EZjzmtG9q qX+ALNGE34uPFWOlzIRrle6u9WpSp58cJw30h+fLARpsu4oWMnOAT4dYnjWncM0Ii0nx EV7sCx5fZKu08fG1unXLCyGI7XA3dzCW6eiug= Received: by 10.227.151.196 with SMTP id d4mr1889471wbw.102.1313512105038; Tue, 16 Aug 2011 09:28:25 -0700 (PDT) Received: from bigio.localnet (host175-92-dynamic.4-87-r.retail.telecomitalia.it [87.4.92.175]) by mx.google.com with ESMTPS id fy12sm158508wbb.49.2011.08.16.09.28.21 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 16 Aug 2011 09:28:21 -0700 (PDT) From: Stefano Lattarini To: Eric Blake Subject: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness Date: Tue, 16 Aug 2011 18:28:12 +0200 User-Agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) References: <201108051108.21060.stefano.lattarini@gmail.com> <201108161804.39354.stefano.lattarini@gmail.com> <4E4A986B.2020308@redhat.com> In-Reply-To: <4E4A986B.2020308@redhat.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201108161828.12910.stefano.lattarini@gmail.com> X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: 9245 Cc: Autoconf , 9245@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.0 (----) Hi Eric. On Tuesday 16 August 2011, Eric Blake wrote: > On 08/16/2011 10:04 AM, Stefano Lattarini wrote: > > The "proper fix" is even easier BTW: just use "$$exit || exit 1" instead. > > > > The updated patch should now work (and I've tested it properly this time). > > This new patch hasn't been derived by "cargo-cult guessing", but from the > > explanation of the relevant FreeBSD make bug provided by Jilles Tjoelker: > > > > > > Now that's something to be documented in the Autoconf manual I'd say ... > > Indeed. So it all boils down to BSD make rewriting rules (but only for > make -j) to be roughly: > > sh -c 'set -v; your rules; set +v' > > and losing your exit status (not to mention sharing shell status between > commands that are supposed to be in independent shells), when it really > should have been rewriting as something more complex: > > sh -c '(your rule 1); (your rule 2); st=$?; set +v; exit $st' > > And your workaround of an explicit exit rather than relying on $? at the > end of your 'set +e' invocation is indeed the proper way to force the > parallel make bug to be avoided, by exiting prior to any post-commands > injected by the BSD make rewriting going on that corrupt $?. > I'll have a "draft patch" read soonish. There is ample room for improvements, but I'll post it here anyway since it can benefit from early feedback. Regards, Stefano From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 16 13:03:05 2011 Received: (at control) by debbugs.gnu.org; 16 Aug 2011 17:03:05 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QtN2d-0008G6-W0 for submit@debbugs.gnu.org; Tue, 16 Aug 2011 13:03:05 -0400 Received: from mail-wy0-f172.google.com ([74.125.82.172]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QtN2b-0008Ft-VU for control@debbugs.gnu.org; Tue, 16 Aug 2011 13:03:03 -0400 Received: by wyg8 with SMTP id 8so64948wyg.3 for ; Tue, 16 Aug 2011 10:01:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:content-type :content-transfer-encoding:message-id; bh=l5sz9L5uoWioz60vQZ1CukdygMMY94VYIxZGQaEWQMM=; b=itYH73VRwekCJ2xxQrQ4s6M7D/dZS4fFX+YHGDOqEAhslJHUTY4EKTCT+KbkXj5IGE ce6c54r1RM/724aFGi/nazhwi1+3aAbY0hHXJ8FIGvHUkN6A+3AGjgrANSgAM+6SJClY LQA1BQnn05xfvgzbQHFz4w74XIEG0BpMLwZ8g= Received: by 10.216.180.77 with SMTP id i55mr4421123wem.100.1313514072878; Tue, 16 Aug 2011 10:01:12 -0700 (PDT) Received: from bigio.localnet (host175-92-dynamic.4-87-r.retail.telecomitalia.it [87.4.92.175]) by mx.google.com with ESMTPS id y61sm157164wec.30.2011.08.16.10.01.11 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 16 Aug 2011 10:01:11 -0700 (PDT) From: Stefano Lattarini To: control@debbugs.gnu.org Subject: none Date: Tue, 16 Aug 2011 19:01:02 +0200 User-Agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201108161901.02874.stefano.lattarini@gmail.com> X-Spam-Score: -3.9 (---) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.9 (---) tag 9306 patch tag 9245 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 18 15:44:50 2011 Received: (at 9245-done) by debbugs.gnu.org; 18 Aug 2011 19:44:50 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qu8WH-0008Tp-PH for submit@debbugs.gnu.org; Thu, 18 Aug 2011 15:44:50 -0400 Received: from mail-ww0-f46.google.com ([74.125.82.46]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qu8WC-0008Tb-Vw for 9245-done@debbugs.gnu.org; Thu, 18 Aug 2011 15:44:47 -0400 Received: by wwm17 with SMTP id 17so2419041wwm.15 for <9245-done@debbugs.gnu.org>; Thu, 18 Aug 2011 12:42:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:references:in-reply-to:mime-version :content-type:content-transfer-encoding:message-id; bh=5SMZQ7khHj2Ix439eYS9uopnHKh6buwy2Vd6vzORrmY=; b=x+FZ2cVAF6iFKt6NSXRk12vN1QZuUGIR5Du+9DBqV1QHAa0pmRj4ma8DFqIeqz6WzE AlQftZSCI00CPhC4iAJq+rHfQOfHhp0vgMde8KvBTuFGNam590YErtCG7DQpwkm6mpNl HWyLRC4fBL85bCAtGliUtWVAY/bXCRS/5PmkU= Received: by 10.227.25.194 with SMTP id a2mr1372502wbc.108.1313696563601; Thu, 18 Aug 2011 12:42:43 -0700 (PDT) Received: from bigio.localnet (host131-92-dynamic.4-87-r.retail.telecomitalia.it [87.4.92.131]) by mx.google.com with ESMTPS id fg5sm2014915wbb.23.2011.08.18.12.42.40 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 18 Aug 2011 12:42:41 -0700 (PDT) From: Stefano Lattarini To: 9245-done@debbugs.gnu.org Subject: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness Date: Thu, 18 Aug 2011 21:42:32 +0200 User-Agent: KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) References: <201108051108.21060.stefano.lattarini@gmail.com> <201108121832.58440.stefano.lattarini@gmail.com> <201108161804.39354.stefano.lattarini@gmail.com> In-Reply-To: <201108161804.39354.stefano.lattarini@gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201108182142.32647.stefano.lattarini@gmail.com> X-Spam-Score: -3.9 (---) X-Debbugs-Envelope-To: 9245-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.9 (---) On Tuesday 16 August 2011, Stefano Lattarini wrote: > On Friday 12 August 2011, Stefano Lattarini wrote: > > OK, the attached hacky patch seems to fix the bug. > > > LOL no, there was an embarassingly stupid error in my patch: > > diff --git a/lib/am/check.am b/lib/am/check.am > index 0b54312..75f8bba 100644 > --- a/lib/am/check.am > +++ b/lib/am/check.am > @@ -224,7 +224,9 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) > col="$$red"; \ > fi; \ > echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \ > - $$exit > +## The apparently redundant and repeated "exit 1" works around a bug in > +## FreeBSD make (present only when running in concurrent mode). > + $$exit || (exit 1); exit 1 > > This obviously causes the `check' target to exit unconditionally with a > non-zero exit status. Hardly a fix for anything, sigh. > > The "proper fix" is even easier BTW: just use "$$exit || exit 1" instead. > > The updated patch should now work (and I've tested it properly this time). > This new patch hasn't been derived by "cargo-cult guessing", but from the > explanation of the relevant FreeBSD make bug provided by Jilles Tjoelker: > > > Now that's something to be documented in the Autoconf manual I'd say ... > BTW, done with commit v2.68-84-g96cdf32 "docs: other issues with parallel BSD make"; see: > > I have no idea why > > exactly it is so yet, so we might want to wait to apply the patch until > > we have fully understood the reasons of the original failure, > > > Glad I've waited this time! And I will allow another couple of days for > comments and suggestions before pushing. > > Regards, > Stefano > I've pushed the patch now, with the additional squash-in below. I'm closing this bug report. Regards, Stefano -*-*- diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in index 2f553ed..17a2002 100644 --- a/lib/Automake/tests/Makefile.in +++ b/lib/Automake/tests/Makefile.in @@ -405,7 +405,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) col="$$red"; \ fi; \ echo "$$msg" | $(am__text_box) "col=$$col" "std=$$std"; \ - $$exit + $$exit || exit 1 # Run all the tests. check-TESTS: From unknown Thu Aug 14 21:45:18 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 16 Sep 2011 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator