GNU bug report logs -
#41824
Dejagnu's unknown proc aborts testsuite run when triggered in test-case
Previous Next
Full log
Message #73 received at 41824 <at> debbugs.gnu.org (full text, mbox):
On 6/25/20 7:53 PM, Jacob Bachmeyer wrote:
> The original rationale for using UNRESOLVED here was that DejaGnu
> converts a test result to UNRESOLVED if too many errors or warnings were
> produced. The DejaGnu manual indicates (section "A POSIX Compliant Test
> Framework") that UNRESOLVED is correct for a test where execution was
> interrupted or was set up incorrectly. UNTESTED is specifically listed
> as a placeholder for an as-yet-unwritten testcase. A "typical" GDB
> testsuite run has almost a hundred UNTESTED results but (without this
> patch) zero UNRESOLVED results.
>
> To me, this seems that UNRESOLVED is correct here, or that the manual
> has an error.
So having written both, there is always the chance I've interpreted
things differently. If the test is interrupted, or has errors/warnings,
like timing problems for example, then it's UNRESOLVED. But a bug in Tcl
code enough to trigger unknown is UNTESTED, as the test never really
ran. I'd go with which definition the toolchain teams prefer, as this
effects validation testruns. I could go either way.
I don't know if POSIX 1003.3 is still considered a standard, that was
a long time ago. The testsuites were primarily focused on XOpen and
POSIX conformance tests, not toolchains. I was
on the standards committee, so used them as a model cause they seemed a
good standard. But the details of the interpretation can now be whatever
we want it to be. We're probably the only ones left running TET
compliant testsuites anyway. :-)
Anyway, whether a test run aborts on an error or not is not defined,
so it's up to us to decide.
- rob -
This bug report was last modified 4 years and 313 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.