GNU bug report logs - #23521
control test "PASS" exit status on a per-test basis

Previous Next

Full log

Message #29 received at 23521 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Mathieu Lirzin <mthl <at> gnu.org>
Cc: 23521 <at> debbugs.gnu.org
Subject: Re: bug#23521: XFAIL
Date: Fri, 20 May 2016 17:00:30 +0100

[Message part 1 (text/plain, inline)]

On 20 May 2016 at 16:49, Mathieu Lirzin <mthl <at> gnu.org> wrote:

> Reuben Thomas <rrt <at> sc3d.org> writes:
>
> > On 19 May 2016 at 00:04, Mathieu Lirzin <mthl <at> gnu.org> wrote:
> >
> >     > It is often easier to write expected-to-fail tests this way (so
> >     that
> >     > they can all look the same), rather than have to have, for
> >     example, an
> >     > extra driver that converts expected errors into success codes
> >     for the
> >     > automake test harness.
> >
> >     What do you mean precisely by “an extra driver”?
> >
> >
> > ​A custom test driver.​
>
> OK, I wasn't sure.  Indeed a custom test driver seems a bit heavy just
> checking failures.  IMO the solution Peter proposed is nice and simple.
>

​What Peter proposed is essentially a custom test driver: I would not
expect to duplicate the logic to check the return code &c. in each test
expected to fail; rather, I would put it in a custom test driver that would
handle expected fails and mark them as passes. (My expected fails are all
of the same type, i.e. a non-zero exit code. It might additionally be
useful, as Peter suggests, to check that an expected error message is
produced.)

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.