GNU bug report logs - #16302
1.14.1: check-TESTS is not lazy enough

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 16302 in the body.
You can then email your comments to 16302 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim <at> lrde.epita.fr>
To: Automake Bugs <bug-automake <at> gnu.org>
Subject: 1.14.1: check-TESTS is not lazy enough
Date: Mon, 30 Dec 2013 15:48:48 +0100

Hi all,

I have this piece of software with several APIs, organized in clear
layers.  Building the whole package is costly, especially because of
the top-level layers (dozens of binaries), and the whole test suite
is even costlier (because it requires to build the whole set of binaries,
and then it runs all the tests, including the costly top-level tests).

So, « of course », when developing this piece of software, I seldom run
the full test-suite, and run selected bits.  In particular I avoid
« make all », so, of course, I also avoid « make check » since it depends
on the former.  Rather I use check-TESTS and deal with the dependencies
myself, like a grownup.

Alas, for some reason, while it works well to generate individual test logs,
but when generating (the partial) test-suite.log, « make all-am » is
invoked.

Please try to attached tarball.  It features a lazy test suite with two
test scripts.

> AM_TESTS_ENVIRONMENT = export PATH=$(abs_top_builddir):$PATH;
> RECHECK_LOGS =
> TESTS = fast slow

« slow » runs a binary built by the package, so
of course, its log depends on this binary:

> slow.log: foo

but ‘fast’ depends on nothing, so to run it and only it, I have
this piece of syntactic sugar: 

> check-fast:
> 	$(MAKE) $(AM_MAKEFLAGS) check-TESTS TESTS='fast'

As expected ‘make check’ is lazy.  However, if I touch
fast and foo.c (the dependency of slow), then observe:

> $ touch ~/src/gnu/test-suite/{foo.c,fast}
> $ make check-fast
> /usr/bin/make  check-TESTS TESTS='fast'
> PASS: fast
> depbase=`echo foo.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
> 	ccache gcc-mp-4.8 -DPACKAGE_NAME=\"test-suite\" -DPACKAGE_TARNAME=\"test-suite\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"test-suite\ 1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"test-suite\" -DVERSION=\"1.0\" -I. -I/Users/akim/src/gnu/test-suite   -DNDEBUG -isystem /opt/local/include  -O3 -MT foo.o -MD -MP -MF $depbase.Tpo -c -o foo.o /Users/akim/src/gnu/test-suite/foo.c &&\
> 	mv -f $depbase.Tpo $depbase.Po
> ccache gcc-mp-4.8  -O3  -L/opt/local/lib -o foo foo.o  
> ============================================================================
> Testsuite summary for test-suite 1.0
> ============================================================================
> # TOTAL: 1
> # PASS:  1
> # SKIP:  0
> # XFAIL: 0
> # FAIL:  0
> # XPASS: 0
> # ERROR: 0
> ============================================================================

As you can see, « fast.log » was recreated appropriately, but then,
to build test-suite.log, something fires « all-am » (running make
with -d shows this very well).  It appears to be due to this:

> $(TEST_SUITE_LOG): $(TEST_LOGS)
> 	@$(am__set_TESTS_bases); \
> 	am__f_ok () { test -f "$$1" && test -r "$$1"; }; \
> 	redo_bases=`for i in $$bases; do \
> 	              am__f_ok $$i.trs && am__f_ok $$i.log || echo $$i; \
> 	            done`; \
> 	if test -n "$$redo_bases"; then \
> 	  redo_logs=`for i in $$redo_bases; do echo $$i.log; done`; \
> 	  redo_results=`for i in $$redo_bases; do echo $$i.trs; done`; \
> 	  if $(am__make_dryrun); then :; else \
> 	    rm -f $$redo_logs && rm -f $$redo_results || exit 1; \
> 	  fi; \
> 	fi; \
> 	if test -n "$$am__remaking_logs"; then \
> 	  echo "fatal: making $(TEST_SUITE_LOG): possible infinite" \
> 	       "recursion detected" >&2; \
> 	else \
> 	  am__remaking_logs=yes $(MAKE) $(AM_MAKEFLAGS) $$redo_logs; \
> 	fi; \

in this last bit, « $$redo_logs » is empty, so, blam, a full cost
make all.  What was the point of this piece of code?  At first sight
it seems that it should be guarder by ‘test -n $$redo_log’.

This is *really* costly, I’d be happy to have nice workarounds.

Thanks, and happy new year!

Message #8 received at 16302 <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim <at> lrde.epita.fr>
To: 16302 <at> debbugs.gnu.org
Subject: Re: bug#16302: Acknowledgement (1.14.1: check-TESTS is not lazy
 enough)
Date: Mon, 30 Dec 2013 15:51:19 +0100

[Message part 1 (text/plain, inline)]

Well, the promised tarball…

[test-suite-1.0.tar.gz (application/x-gzip, attachment)]

Message #11 received at 16302 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Akim Demaille <akim <at> lrde.epita.fr>
Cc: 16302 <at> debbugs.gnu.org,
 "automake-patches <at> gnu.org" <automake-patches <at> gnu.org>
Subject: Re: bug#16302: 1.14.1: check-TESTS is not lazy enough
Date: Tue, 31 Dec 2013 00:11:34 +0100

[Message part 1 (text/plain, inline)]

tags 16302 + patch
severity 16302 minor
stop

On 12/30/2013 03:48 PM, Akim Demaille wrote:
> Hi all,
>
Hi Akim.

> I have this piece of software with several APIs, organized in clear
> layers.  Building the whole package is costly, especially because of
> the top-level layers (dozens of binaries), and the whole test suite
> is even costlier (because it requires to build the whole set of binaries,
> and then it runs all the tests, including the costly top-level tests).
> 
> So, « of course », when developing this piece of software, I seldom run
> the full test-suite, and run selected bits.  In particular I avoid
> « make all », so, of course, I also avoid « make check » since it depends
> on the former.  Rather I use check-TESTS and deal with the dependencies
> myself, like a grownup.
> 
> Alas, for some reason, while it works well to generate individual test logs,
> but when generating (the partial) test-suite.log, « make all-am » is
> invoked.
> 
> Please try to attached tarball.  It features a lazy test suite with two
> test scripts.
> 
>> AM_TESTS_ENVIRONMENT = export PATH=$(abs_top_builddir):$PATH;
>> RECHECK_LOGS =
>> TESTS = fast slow
> 
> « slow » runs a binary built by the package, so
> of course, its log depends on this binary:
> 
>> slow.log: foo
> 
> but ‘fast’ depends on nothing, so to run it and only it, I have
> this piece of syntactic sugar: 
> 
>> check-fast:
>> 	$(MAKE) $(AM_MAKEFLAGS) check-TESTS TESTS='fast'
> 
> As expected ‘make check’ is lazy.  However, if I touch
> fast and foo.c (the dependency of slow), then observe:
> 
>> $ touch ~/src/gnu/test-suite/{foo.c,fast}
>> $ make check-fast
>> /usr/bin/make  check-TESTS TESTS='fast'
>> PASS: fast
>> depbase=`echo foo.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
>> 	ccache gcc-mp-4.8 -DPACKAGE_NAME=\"test-suite\" -DPACKAGE_TARNAME=\"test-suite\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"test-suite\ 1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"test-suite\" -DVERSION=\"1.0\" -I. -I/Users/akim/src/gnu/test-suite   -DNDEBUG -isystem /opt/local/include  -O3 -MT foo.o -MD -MP -MF $depbase.Tpo -c -o foo.o /Users/akim/src/gnu/test-suite/foo.c &&\
>> 	mv -f $depbase.Tpo $depbase.Po
>> ccache gcc-mp-4.8  -O3  -L/opt/local/lib -o foo foo.o  
>> ============================================================================
>> Testsuite summary for test-suite 1.0
>> ============================================================================
>> # TOTAL: 1
>> # PASS:  1
>> # SKIP:  0
>> # XFAIL: 0
>> # FAIL:  0
>> # XPASS: 0
>> # ERROR: 0
>> ============================================================================
> 
> As you can see, « fast.log » was recreated appropriately, but then,
> to build test-suite.log, something fires « all-am » (running make
> with -d shows this very well).  It appears to be due to this:
> 
>> $(TEST_SUITE_LOG): $(TEST_LOGS)
>> 	@$(am__set_TESTS_bases); \
>> 	am__f_ok () { test -f "$$1" && test -r "$$1"; }; \
>> 	redo_bases=`for i in $$bases; do \
>> 	              am__f_ok $$i.trs && am__f_ok $$i.log || echo $$i; \
>> 	            done`; \
>> 	if test -n "$$redo_bases"; then \
>> 	  redo_logs=`for i in $$redo_bases; do echo $$i.log; done`; \
>> 	  redo_results=`for i in $$redo_bases; do echo $$i.trs; done`; \
>> 	  if $(am__make_dryrun); then :; else \
>> 	    rm -f $$redo_logs && rm -f $$redo_results || exit 1; \
>> 	  fi; \
>> 	fi; \
>> 	if test -n "$$am__remaking_logs"; then \
>> 	  echo "fatal: making $(TEST_SUITE_LOG): possible infinite" \
>> 	       "recursion detected" >&2; \
>> 	else \
>> 	  am__remaking_logs=yes $(MAKE) $(AM_MAKEFLAGS) $$redo_logs; \
>> 	fi; \
> 
> in this last bit, « $$redo_logs » is empty, so, blam, a full cost
> make all.  What was the point of this piece of code?
>
See the '##'-style comments in lib/am/check.am for detailed explanations
(such comments are stripped from the generated Makefiles, as I'm sure
you know).

> At first sight it seems that it should be guarder by ‘test -n $$redo_log’.
>
Indeed.

> This is *really* costly, I’d be happy to have nice workarounds.
>
Or eve better, to fix the bug :-)  See attached patch.  Does it work
for you?

> Thanks, and happy new year!
> 

Thanks,
  Stefano

[0001-parallel-tests-avoid-possible-implicit-make-all-in-t.patch (text/x-patch, attachment)]

Message #18 received at 16302 <at> debbugs.gnu.org (full text, mbox):

From: Akim Demaille <akim <at> lrde.epita.fr>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 16302 <at> debbugs.gnu.org, Automake Patches <automake-patches <at> gnu.org>
Subject: Re: bug#16302: 1.14.1: check-TESTS is not lazy enough
Date: Tue, 31 Dec 2013 11:17:14 +0100

Le 31 déc. 2013 à 00:11, Stefano Lattarini <stefano.lattarini <at> gmail.com> a écrit :

> Hi Akim.

Hi!

Thanks for the quick answer.

>> At first sight it seems that it should be guarded by ‘test -n $$redo_log’.
>> 
> Indeed.
> 
>> This is *really* costly, I’d be happy to have nice workarounds.
>> 
> Or eve better, to fix the bug :-)  See attached patch.  Does it work
> for you?

I’ve had a hard try to figure out on which branch to apply
it, and failed.  I installed it by hand, and it works fine,
thanks!

But the NEWS entry seems to mean that the patch won’t be
released anytime soon?  We are several to work on this project,
and it would be nice if we could avoid all have to deploy
a git-version of Automake.

> Thanks, and happy new year!

Happy new year to you!

> diff --git a/NEWS b/NEWS
> index dd2812a..a73804a 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -93,6 +93,10 @@ New in 1.15:
>      implementation of the TAP driver (that is documented in the manual)
>      is more portable and has feature parity with the perl implementation.
>  
> +  - The rule generating ‘test-suite.log' no longer risk incurring in an

risks

> +    extra useless "make all" recursive invocation in some corner cases
> +    (automake bug#16302).
> +
>  * Bug fixes:
>  
>    - The user can now extend the special .PRECIOUS target, the same way

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.