GNU bug report logs - #11745
the new test-suite summary is confusing

Previous Next

Full log

View this message in rfc822 format

From: Alexandre Duret-Lutz <adl <at> lrde.epita.fr>
To: 11745 <at> debbugs.gnu.org
Subject: bug#11745: the new test-suite summary is confusing
Date: Tue, 19 Jun 2012 18:47:07 +0200

Hi!

I have a project that contains 8 different test-suites, each in its
own directory.  Some test suite are big, other are small, but they
test different parts of the project and I like being able to run "make
check" in the relevant directory.  They are run sequentially (with
SUBDIRS) when "make check" is run from the top-level.

I upgraded to Automake 1.12.1 to discover the each of these test-suite
now displays a huge summary like:

============================================================================
Testsuite summary for spot 0.9.1a
============================================================================
# TOTAL: 2
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

Instead of the older one-line summary I'm used too.

I can live with the extra verbosity, but I find the initial header
saying "Testsuite summary for spot 0.9.1a" in each of the test suites
very confusing.  The above in definitely *not* a testsuite summary for
the project, it's only a test summary for one directory.

I think the easiest fix would be to change the header to just
"Testsuite summary", without mentioning the package.

Another option could be to let the test-suite be named in Makefile.am.
Maybe introduce
    TESTS_DISPLAY_NAME = $(PACKAGE_STRING)
and use
    echo "Testsuite summary for $(TESTS_DISPLAY_NAME)"
so that people with multiple test-suites can give them different names.

-- 
Alexandre Duret-Lutz

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.