Subject: bug#58065: closed (Re: bug#58065: UNSUPPORTED test cases still can FAIL)

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 06 Oct 2022 01:25:01 +0000

Your bug report

#58065: UNSUPPORTED test cases still can FAIL

which was filed against the dejagnu package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
58065: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=58065
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#58065: UNSUPPORTED test cases still can FAIL
From: Jacob Bachmeyer <[email-masked]>
To: Arsen Arsenović <[email-masked]>
Cc: [email-masked]
Date: Wed, 05 Oct 2022 20:23:56 -0500
List-Id: <debbugs-submit.debbugs.gnu.org>

Arsen Arsenović wrote:
> On Wednesday, 5 October 2022 05:31:17 CEST Jacob Bachmeyer wrote:
>   
>> A preliminary patch has been pushed on the PR58065 branch at Savannah.
>> Please confirm that it resolves this issue.
>>     
> It does.
>   

Thank you.  The PR58065 branch has been merged to master and will be 
included in the future 1.6.4 release.

>> [...]
>>
>> The descriptions given were sufficient that I was able avoid examining
>> the offered test case and thus avoid needing associated copyright
>> paperwork.
>>     
> Huh, would have looking at the provided test even require such papers? 
> That seems a bit excess..
>   

There is a possibility of trouble if code subsequently added to DejaGnu, 
such as a regression test, were to be "significant" and sufficiently 
resemble the offered case, and papers were not in order.  Not looking at 
it short-circuits the whole problem.

In general, the GNU project seems to take a "better safe than sorry" 
angle on the issue, and the prospect of being "low-hanging fruit" for 
some future SCO-alike does not appeal to me.

-- Jacob

Subject: UNSUPPORTED test cases still can FAIL
From: Arsen Arsenović <[email-masked]>
To: [email-masked]
Date: Sun, 25 Sep 2022 01:39:05 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

When a testsuite that uses dg.exp returns ::unsupported:: for some test 
from ${tool}-dg-test or ${tool}-dg-prune, dg-{error,warning} markers in 
that test case are still processed. If the ::unsupported:: is returned 
from ${tool}-dg-test, these markers emit a FAIL, since the output does 
not contain their wanted errors. If the ::unsupported:: is returned from 
${tool}-dg-prune, these markers are processed as normal, which can 
sometimes PASS due to compiler resumption, but can also FAIL. I see no 
other way to report ::unsupported:: status from the dg callbacks.
This feels like an oversight, as it makes conditionally skipping tests 
as UNSUPPORTED nigh impossible.

Previous ML discussion:
https://lists.gnu.org/archive/html/dejagnu/2022-09/msg00000.html

Minimized reproduction case:
https://git.sr.ht/~arsen/dejagnu-demo-testcase/
(commit 4e83a1a0820062387985d866286d73299f0691a6)

Suggested solutions:
- Call ${tool}-dg-prune before processing dg-messages.
- Permit ${tool}-dg-test to return a special value to prevent processing 
dg-messages.

Thanks,
-- 
Arsen Arsenović