GNU bug report logs - #58065
UNSUPPORTED test cases still can FAIL

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Arsen Arsenović <arsen <at> aarsen.me>
Subject: bug#58065: closed (Re: bug#58065: UNSUPPORTED test cases still
 can FAIL)
Date: Thu, 06 Oct 2022 01:25:01 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#58065: UNSUPPORTED test cases still can FAIL

which was filed against the dejagnu package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 58065 <at> debbugs.gnu.org.

-- 
58065: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=58065
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Jacob Bachmeyer <jcb62281 <at> gmail.com>
To: Arsen Arsenović <arsen <at> aarsen.me>
Cc: 58065-done <at> debbugs.gnu.org
Subject: Re: bug#58065: UNSUPPORTED test cases still can FAIL
Date: Wed, 05 Oct 2022 20:23:56 -0500

Arsen Arsenović wrote:
> On Wednesday, 5 October 2022 05:31:17 CEST Jacob Bachmeyer wrote:
>   
>> A preliminary patch has been pushed on the PR58065 branch at Savannah.
>> Please confirm that it resolves this issue.
>>     
> It does.
>   

Thank you.  The PR58065 branch has been merged to master and will be 
included in the future 1.6.4 release.

>> [...]
>>
>> The descriptions given were sufficient that I was able avoid examining
>> the offered test case and thus avoid needing associated copyright
>> paperwork.
>>     
> Huh, would have looking at the provided test even require such papers? 
> That seems a bit excess..
>   

There is a possibility of trouble if code subsequently added to DejaGnu, 
such as a regression test, were to be "significant" and sufficiently 
resemble the offered case, and papers were not in order.  Not looking at 
it short-circuits the whole problem.

In general, the GNU project seems to take a "better safe than sorry" 
angle on the issue, and the prospect of being "low-hanging fruit" for 
some future SCO-alike does not appeal to me.


-- Jacob

[Message part 3 (message/rfc822, inline)]

From: Arsen Arsenović <arsen <at> aarsen.me>
To: bug-dejagnu <at> gnu.org
Subject: UNSUPPORTED test cases still can FAIL
Date: Sun, 25 Sep 2022 01:39:05 +0200

Hi,

When a testsuite that uses dg.exp returns ::unsupported:: for some test 
from ${tool}-dg-test or ${tool}-dg-prune, dg-{error,warning} markers in 
that test case are still processed. If the ::unsupported:: is returned 
from ${tool}-dg-test, these markers emit a FAIL, since the output does 
not contain their wanted errors. If the ::unsupported:: is returned from 
${tool}-dg-prune, these markers are processed as normal, which can 
sometimes PASS due to compiler resumption, but can also FAIL. I see no 
other way to report ::unsupported:: status from the dg callbacks.
This feels like an oversight, as it makes conditionally skipping tests 
as UNSUPPORTED nigh impossible.

Previous ML discussion:
https://lists.gnu.org/archive/html/dejagnu/2022-09/msg00000.html

Minimized reproduction case:
https://git.sr.ht/~arsen/dejagnu-demo-testcase/
(commit 4e83a1a0820062387985d866286d73299f0691a6)

Suggested solutions:
- Call ${tool}-dg-prune before processing dg-messages.
- Permit ${tool}-dg-test to return a special value to prevent processing 
dg-messages.

Thanks,
-- 
Arsen Arsenović

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.