Subject: bug#11287: Various issues with the test suite framework

From: Akim Demaille <[email-masked]>
Date: Fri, 20 Apr 2012 13:13:58 +0200

Hi!

I have seen that check-html will be removed, or rather moved
into the contrib part, but there are a few issues:

- the target is not declared recursive, so one has to write
 the bouncing target herself.

- because of that, the "naive" implementation of check-html
 that just bounces the right directories will miss the
 dependencies that check features (in other words, make
 check properly recursively builds before, while check-html
 would just go into the typical tests/ directory, and then
 bounce to "check", so it completely missed the recursion
 in the other directories and their dependencies).

- RECURSIVE_TARGETS cannot be highjacked by the users, that's
 sad.  Could a $(USER_RECURSIVE_TARGETS) be added to
 RECURSIVE_TARGETS so that we can add our own targets that
 traverse the whole package?  Actually maybe this should
 be an Automake option to add recursive targets, and
 then TARGET-am and TARGET-recursive would be generated and
 bound to the proper TARGET.

- the .log.html recipe features $$RST2HTML, but in '', so it is
 useless.  And anyway $(RST2HTML) already does the job.

- "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing.

- the test compiler does not appear as a dependency for
 the test logs that use it.  This is most useful when
 RECHECK_LOG =.

- it would be equally useful to be able to add dependencies
 to a category of test drivers.  Say, with the documented
 example:

    TESTS = foo.pl bar.py baz
    TEST_EXTENSIONS = .pl .py
    PL_LOG_COMPILER = $(PERL)
    AM_PL_LOG_FLAGS = -w
    PY_LOG_COMPILER = $(PYTHON)
    AM_PY_LOG_FLAGS = -v
    LOG_COMPILER = ./wrapper-script
    AM_LOG_FLAGS = -d

 be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and
 LOG_DEPENDENCIES.  I don't think EXTRA dependencies would
 be useful.

Cheers!