GNU bug report logs - #20693
SKIP not counted in TAP

Previous Next

Full log

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Arthur Schwarz" <aschwarz1309 <at> att.net>
To: <bug-automake <at> gnu.org>
Subject: SKIP not counted in TAP
Date: Fri, 29 May 2015 08:15:15 -0700

I think this is an error.

skip causes the following issues:
1: It is not reported.
2: It is not summarized.
3: It must be subtracted from the plan (tests run), and
4: The .log file has TAP generated output and the test data intermixed.

I think this is against the TAP definition?

====================== output ======================

PASS: test2.tap 1
FAIL: test2.tap 2
===========================================
Testsuite summary for test 0.5
===========================================
# TOTAL: 2
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
===========================================
See src/test-suite.log
===========================================

====================== test-suite.log ======================

==================================
   test 0.5: src/test-suite.log
==================================

# TOTAL: 2
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: test2
===========

skip
ok
PASS: test2.tap 1
not ok
FAIL: test2.tap 2
1..2

====================== program ======================
 
## Process this file with automake to produce Makefile.in

# select testing format
AUTOMAKE_OPTIONS = parallel-tests 
EXTRA_DIST       = tap-driver.sh


# test options
TEST_EXTENSIONS    = .tap
TAP_LOG_DRIVER     = env AM_TAP_AWK='$(AWK)' $(SHELL) ./tap-driver.sh

# Testing 
check_SCRIPTS      = test2.tap
TESTS              = test2.tap

test2.tap:
	echo '#!/bin/bash'                                  > test2.tap
	echo 'echo test2.tap "$$# [ $$@ ]"   >> test2.log' >> test2.tap
	echo "echo I am a tap script         >> test2.log" >> test2.tap
	echo echo '1..2'                                   >> test2.tap
	echo echo 'ok'                                     >> test2.tap
	echo echo 'not ok'                                 >> test2.tap
	echo echo 'skip'                                   >> test2.tap
	chmod +x test2.tap

CLEANFILES         = $(test_SOURCES)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.