Subject: bug#23521: XFAIL

From: Reuben Thomas <[email-masked]>
Date: Thu, 12 May 2016 11:01:11 +0100

The documentation says: "It's not uncommon, especially during early
development stages, that some tests fail for known reasons, and that the
developer doesn't want
to tackle these failures immediately (this is especially true when the
failing tests deal with corner cases)."

Another common use for "expected failure" is to write tests to check that
error conditions arise as expected, for example, by checking that a program
raises an error when given invalid input.

If that's a reasonable use of automake's test harness, perhaps the
documentation could reflect that, e.g. by adding:

"Another use for XFAIL is to mark tests that are supposed to fail, for
example, to check that a program raises an error when given invalid input."

It is often easier to write expected-to-fail tests this way (so that they
can all look the same), rather than have to have, for example, an extra
driver that converts expected errors into success codes for the automake
test harness.

-- 
http://rrt.sc3d.org