GNU bug report logs - #49309
Feature Request: Automake script based tests to print the test name before running it

Previous Next

Full log

View this message in rfc822 format

From: Kasper k <kasperkantz <at> outlook.com>
To: Hans-Bernhard Bröker <HBBroeker <at> t-online.de>, Peter Johansson <trojkan <at> gmail.com>, "49309 <at> debbugs.gnu.org" <49309 <at> debbugs.gnu.org>
Subject: bug#49309: Feature Request: Automake script based tests to print the test name before running it
Date: Fri, 2 Jul 2021 21:05:39 +0000

[Message part 1 (text/plain, inline)]

Hi Hans,

Thanks for taking interest. But I am not sure how would it break the parallelism of tests or anything in the test infra. The ask here is to optionally print an additional extra line before running the test regardless of order or medium in which they will eventually get rendered.

AM script check supports these options:

  *   AM_TESTS_FD_REDIRECT
  *   AM_TESTS_ENVIRONMENT
  *   AM_TESTSUITE_SUMMARY_HEADER

they solve similar problems which are more useful in one case, but not the other one. With the proposed AM_TESTS_SHOW_NAME, you can have parallelism:

    # with AM_TESTS_SHOW_NAME=1
    #
    RUN Foo
    RUN Bar
    PASS Foo
    RUN Baz
    FAILED Baz
    PASS Bar

now as to your point for my specific example where it could be useful, I agree it will not solve that particular problem (of revealing the hanging tests) perfectly in all modes and neither I am looking for the requested option to do so (and neither do the other three supported options are meant to solve everything...)

/K

________________________________
From: Hans-Bernhard Bröker <HBBroeker <at> t-online.de>
Sent: Friday, July 2, 2021 5:02 PM
To: Kasper k <kasperkantz <at> outlook.com>; Peter Johansson <trojkan <at> gmail.com>; 49309 <at> debbugs.gnu.org <49309 <at> debbugs.gnu.org>
Subject: Re: bug#49309: Feature Request: Automake script based tests to print the test name before running it

Am 01.07.2021 um 20:59 schrieb Kasper k:

> Then if some intermittent bad situation happens in
> unattended/disposable CI environment, as an example, then we won't be
> needing to opt into any kind of workaround, it'd be just there right in
> the "regular" make-check logs; without introducing crazy amount of
> verbosity (which set -x, VERBOSE=1 etc. renders).

IMHO that whole idea loses a most, if not all of its appeal when tests
are run in parallel (which CI builds, always pressed for time, will
rather likely want to do).

As soon as there is more than one line of output, you're back to where
you were without this change, or worse.

* -Otarget or -Orecurse will undo the desired effect, because make will
keep the entire output to itself until the test is done.  That's the
back to square one part.

* -Oline or -Onone modes will print outputs from different tests
intermixed, making it virtually impossible to know which result output
belongs to which testcase.  That's considerably worse than single-line
outputs.

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.