GNU bug report logs - #23521
control test "PASS" exit status on a per-test basis

Previous Next

Full log

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: bug-automake <bug-automake <at> gnu.org>
Subject: XFAIL
Date: Thu, 12 May 2016 11:01:11 +0100

[Message part 1 (text/plain, inline)]

The documentation says: "It's not uncommon, especially during early
development stages, that some tests fail for known reasons, and that the
developer doesn't want
to tackle these failures immediately (this is especially true when the
failing tests deal with corner cases)."

Another common use for "expected failure" is to write tests to check that
error conditions arise as expected, for example, by checking that a program
raises an error when given invalid input.

If that's a reasonable use of automake's test harness, perhaps the
documentation could reflect that, e.g. by adding:

"Another use for XFAIL is to mark tests that are supposed to fail, for
example, to check that a program raises an error when given invalid input."

It is often easier to write expected-to-fail tests this way (so that they
can all look the same), rather than have to have, for example, an extra
driver that converts expected errors into success codes for the automake
test harness.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.