GNU bug report logs - #58065
UNSUPPORTED test cases still can FAIL

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: jcb62281 <at> gmail.com
Cc: tracker <at> debbugs.gnu.org
Subject: bug#58065: closed (UNSUPPORTED test cases still can FAIL)
Date: Thu, 06 Oct 2022 01:25:01 +0000

[Message part 1 (text/plain, inline)]

Your message dated Wed, 05 Oct 2022 20:23:56 -0500
with message-id <633E2E2C.6010307 <at> gmail.com>
and subject line Re: bug#58065: UNSUPPORTED test cases still can FAIL
has caused the debbugs.gnu.org bug report #58065,
regarding UNSUPPORTED test cases still can FAIL
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
58065: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=58065
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Arsen Arsenović <arsen <at> aarsen.me>
To: bug-dejagnu <at> gnu.org
Subject: UNSUPPORTED test cases still can FAIL
Date: Sun, 25 Sep 2022 01:39:05 +0200

Hi,

When a testsuite that uses dg.exp returns ::unsupported:: for some test 
from ${tool}-dg-test or ${tool}-dg-prune, dg-{error,warning} markers in 
that test case are still processed. If the ::unsupported:: is returned 
from ${tool}-dg-test, these markers emit a FAIL, since the output does 
not contain their wanted errors. If the ::unsupported:: is returned from 
${tool}-dg-prune, these markers are processed as normal, which can 
sometimes PASS due to compiler resumption, but can also FAIL. I see no 
other way to report ::unsupported:: status from the dg callbacks.
This feels like an oversight, as it makes conditionally skipping tests 
as UNSUPPORTED nigh impossible.

Previous ML discussion:
https://lists.gnu.org/archive/html/dejagnu/2022-09/msg00000.html

Minimized reproduction case:
https://git.sr.ht/~arsen/dejagnu-demo-testcase/
(commit 4e83a1a0820062387985d866286d73299f0691a6)

Suggested solutions:
- Call ${tool}-dg-prune before processing dg-messages.
- Permit ${tool}-dg-test to return a special value to prevent processing 
dg-messages.

Thanks,
-- 
Arsen Arsenović

[Message part 3 (message/rfc822, inline)]

From: Jacob Bachmeyer <jcb62281 <at> gmail.com>
To: Arsen Arsenović <arsen <at> aarsen.me>
Cc: 58065-done <at> debbugs.gnu.org
Subject: Re: bug#58065: UNSUPPORTED test cases still can FAIL
Date: Wed, 05 Oct 2022 20:23:56 -0500

Arsen Arsenović wrote:
> On Wednesday, 5 October 2022 05:31:17 CEST Jacob Bachmeyer wrote:
>   
>> A preliminary patch has been pushed on the PR58065 branch at Savannah.
>> Please confirm that it resolves this issue.
>>     
> It does.
>   

Thank you.  The PR58065 branch has been merged to master and will be 
included in the future 1.6.4 release.

>> [...]
>>
>> The descriptions given were sufficient that I was able avoid examining
>> the offered test case and thus avoid needing associated copyright
>> paperwork.
>>     
> Huh, would have looking at the provided test even require such papers? 
> That seems a bit excess..
>   

There is a possibility of trouble if code subsequently added to DejaGnu, 
such as a regression test, were to be "significant" and sufficiently 
resemble the offered case, and papers were not in order.  Not looking at 
it short-circuits the whole problem.

In general, the GNU project seems to take a "better safe than sorry" 
angle on the issue, and the prospect of being "low-hanging fruit" for 
some future SCO-alike does not appeal to me.


-- Jacob

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.