GNU bug report logs - #7868
parallel-tests and command-line length limit issue

Previous Next

Full log

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: bug-automake <at> gnu.org
Subject: splitting up test suites
Date: Wed, 19 Jan 2011 23:22:19 +0100

The testsuite is too large for MSYS.

We've finally reached the point where we have more than 1000
tests, $(TESTS) expands to 15k characters, and where 'make check' will
not work at all any more on MSYS, because it cannot spawn sh any more,
presumably in 'make check TESTS="..."'.  (MSYS make doesn't export
macros to the environment of spawned processes even without .NOEXPORT,
presumably otherwise lots of Makefiles would be really unusable here.)
This also clears up the spurious failure of sed a few days ago.


Here's a preliminary plan for multiple testsuites per Makefile.am.
It would be nice if this worked somehow:

# These are all specified by the user:
TEST_SUITE_LOGS = suite1.log suite2.log suite3.log ..
TEST_EXTENSIONS = .test ...
# these undergo $(EXEEXT) autoexpansion internally:
suite1_log_SOURCES = all.test aclocal7.test ...
suite2_log_SOURCES = suffix.test ...

# These are then produced by automake:
TEST_SUITE_HTMLS = $(TEST_SUITE_LOGS:.log=.html)
suite1_log_LOGS = $(am__helper_var:.test=.log)

mostlyclean-generic:
	-test -z "$(suite1_log_LOGS)" || rm -f $(suite1_log_LOGS)
	-test -z "$(suite2_log_LOGS)" || rm -f $(suite2_log_LOGS)


In the <suite>_SOURCES, $(EXEEXT) transformation should take place
(unless no-exeext is given, of course), just as is currently done for
TESTS and *_PROGRAMS.  Hmm, alternatively we could also require all
<suite>_SOURCES to be listed in $(TESTS), that would allow reuse of
this variable as well, at the cost of some specification redundance.

Open questions: how to produce nice results, both on stdout and in suite
log files.  One way is to merge all logs into a final TEST_SUITE_LOG
(that way we could also reuse that variable).  Another is to try to even
hide the summaries of the individual suites, iff a final suite is being
made.

Internally, it would be nice if the register_language could be exploited
(but this is not a requirement for this, as some features with tests are
distinctly different from other languages).

Cheers,
Ralf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.