GNU bug report logs - #71422
no easy way to generate a test-suite.log without skipped tests

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 71422 in the body.
You can then email your comments to 71422 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-automake <at> gnu.org
Subject: no easy way to generate a test-suite.log without skipped tests
Date: Fri, 07 Jun 2024 18:26:15 +0200

[Message part 1 (text/plain, inline)]

Hi,

In my projects that use Automake and the "parallel" test driver, often
I have 1000 unit tests, of which 100-150 are skipped and 1-5 fail.
When analyzing the "make check" results, the generated test-suite.log
is more handy to use than the 1000 different <test>.log files. However,
before I get an overview of the results, I have to eliminate the logs
of the SKIPped tests (because the 1-5 failures are intermixed with
the 100-150 SKIPped test logs). This is a process that takes 1-2 minutes
each time.

I could write a script that post-processes the test-suite.log to eliminate
them. But it is better if the logs of the SKIPped tests wouldn't appear
in the file in the first place. The SKIPped tests are uninteresting
in my case.

Here is a patch that allows the package maintainer to eliminate the
SKIPped test logs, thus making test-suite.log more useful: either
by running
  make check IGNORE_SKIPPED_LOGS=1
or by defining in the Makefile (or Makefile.am):
  IGNORE_SKIPPED_LOGS = 1

It includes a unit test, documentation updates, and passes "make check".

[0001-automake-Allow-omitting-the-logs-of-skipped-tests-fr.patch (text/x-patch, attachment)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sam James <sam <at> gentoo.org>
To: Bruno Haible <bruno <at> clisp.org>
Cc: bug-automake <at> gnu.org
Subject: Re: no easy way to generate a test-suite.log without skipped tests
Date: Sat, 08 Jun 2024 09:16:27 +0100

[Message part 1 (text/plain, inline)]

Bruno Haible <bruno <at> clisp.org> writes:

> Hi,
>
> In my projects that use Automake and the "parallel" test driver, often
> I have 1000 unit tests, of which 100-150 are skipped and 1-5 fail.
> When analyzing the "make check" results, the generated test-suite.log
> is more handy to use than the 1000 different <test>.log files. However,
> before I get an overview of the results, I have to eliminate the logs
> of the SKIPped tests (because the 1-5 failures are intermixed with
> the 100-150 SKIPped test logs). This is a process that takes 1-2 minutes
> each time.

Thanks for proposing this. I'd find this useful too.

>
> I could write a script that post-processes the test-suite.log to eliminate
> them. But it is better if the logs of the SKIPped tests wouldn't appear
> in the file in the first place. The SKIPped tests are uninteresting
> in my case.
>
> Here is a patch that allows the package maintainer to eliminate the
> SKIPped test logs, thus making test-suite.log more useful: either
> by running
>   make check IGNORE_SKIPPED_LOGS=1
> or by defining in the Makefile (or Makefile.am):
>   IGNORE_SKIPPED_LOGS = 1
>
> It includes a unit test, documentation updates, and passes "make check".
>
> [2. text/x-patch; 0001-automake-Allow-omitting-the-logs-of-skipped-tests-fr.patch]...

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 71422 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: bruno <at> clisp.org
Cc: 71422 <at> debbugs.gnu.org
Subject: Re: bug#71422: no easy way to generate a test-suite.log without
 skipped tests
Date: Mon, 17 Jun 2024 11:07:08 -0600

    Here is a patch that allows the package maintainer to eliminate the
    SKIPped test logs, thus making test-suite.log more useful: either
    by running
      make check IGNORE_SKIPPED_LOGS=1
    or by defining in the Makefile (or Makefile.am):
      IGNORE_SKIPPED_LOGS = 1

Thanks Bruno. I installed this (with minor wording tweaks). -k

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.