GNU bug report logs - #20715
tap-driver.sh 'missing test plan' message contradicts the TAP Standard

Previous Next

Full log

Message #17 received at 20715 <at> debbugs.gnu.org (full text, mbox):

From: Nick Bowler <nbowler <at> elliptictech.com>
To: Arthur Schwarz <aschwarz1309 <at> att.net>
Cc: 20715 <at> debbugs.gnu.org
Subject: Re: bug#20715: tap-driver.sh 'missing test plan' message contradicts
 the TAP Standard
Date: Tue, 2 Jun 2015 13:30:54 -0400

On 2015-06-02 09:40 -0700, Arthur Schwarz wrote:
> [Nick Bowler wrote]
>     The plan line has to be mandatory for this feature to work as
>     intended (i.e., for the TAP consumer to determine whether a
>     producer has run to completion or not).  An optional plan would
>     be useless.
> 
> Sorry to be such a pest.
> 
> Your statement that "... determine whether a producer has run to completion
> or not" is puzzling. The TAP Standard specifically states that if the test
> plan is 1..N and the number of test lines are k < N, then the k+1 .. N
> missing test lines are to be considered as skipped.

Where does it say that?  It is not correct to treat missing test results
as skips.

It is an error for the number of actual test results to differ from the
plan.  This is the whole point of the plan: the consumer always knows
how many test results are expected.  This enables the consumer to tell
the difference between a producer that crashed before completing all the
tests, and a producer that ran normally.

> There is one issue not addressed in the TAP Standard and not addressed in
> the Automake Manual. What happens if k > N?

If k > N, then the number of test cases does not match the plan, so
this is an error.

Regards,
-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.