GNU bug report logs - #58065
UNSUPPORTED test cases still can FAIL

Previous Next

Package: dejagnu;

Reported by: Arsen Arsenović <arsen <at> aarsen.me>

Date: Sun, 25 Sep 2022 07:44:01 UTC

Owned by: jcb62281 <at> gmail.com

Severity: normal

Done: Jacob Bachmeyer <jcb62281 <at> gmail.com>

Bug is archived. No further changes may be made.

Full log


Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Arsen Arsenović <arsen <at> aarsen.me>
To: bug-dejagnu <at> gnu.org
Subject: UNSUPPORTED test cases still can FAIL
Date: Sun, 25 Sep 2022 01:39:05 +0200
Hi,

When a testsuite that uses dg.exp returns ::unsupported:: for some test 
from ${tool}-dg-test or ${tool}-dg-prune, dg-{error,warning} markers in 
that test case are still processed. If the ::unsupported:: is returned 
from ${tool}-dg-test, these markers emit a FAIL, since the output does 
not contain their wanted errors. If the ::unsupported:: is returned from 
${tool}-dg-prune, these markers are processed as normal, which can 
sometimes PASS due to compiler resumption, but can also FAIL. I see no 
other way to report ::unsupported:: status from the dg callbacks.
This feels like an oversight, as it makes conditionally skipping tests 
as UNSUPPORTED nigh impossible.

Previous ML discussion:
https://lists.gnu.org/archive/html/dejagnu/2022-09/msg00000.html

Minimized reproduction case:
https://git.sr.ht/~arsen/dejagnu-demo-testcase/
(commit 4e83a1a0820062387985d866286d73299f0691a6)

Suggested solutions:
- Call ${tool}-dg-prune before processing dg-messages.
- Permit ${tool}-dg-test to return a special value to prevent processing 
dg-messages.

Thanks,
-- 
Arsen Arsenović







This bug report was last modified 2 years and 280 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.