On 20 May 2016 at 16:49, Mathieu Lirzin <mthl@gnu.org> wrote:

Reuben Thomas <rrt@sc3d.org> writes:

> On 19 May 2016 at 00:04, Mathieu Lirzin <mthl@gnu.org> wrote:
>
> > It is often easier to write expected-to-fail tests this way (so
> that
> > they can all look the same), rather than have to have, for
> example, an
> > extra driver that converts expected errors into success codes
> for the
> > automake test harness.
>
> What do you mean precisely by “an extra driver”?
>
>
> A custom test driver.

OK, I wasn't sure. Indeed a custom test driver seems a bit heavy just
checking failures. IMO the solution Peter proposed is nice and simple.

What Peter proposed is essentially a custom test driver: I would not expect to duplicate the logic to check the return code &c. in each test expected to fail; rather, I would put it in a custom test driver that would handle expected fails and mark them as passes. (My expected fails are all of the same type, i.e. a non-zero exit code. It might additionally be useful, as Peter suggests, to check that an expected error message is produced.)