Subject: bug#76839: [Roi Martin] [PATCH] Fix 'Skip' behavior in erts files

From: Stefan Kangas <[email-masked]>
Date: Sat, 8 Mar 2025 09:57:04 +0000

Roi Martin <[email-masked]> writes:

> Stefan Kangas <[email-masked]> writes:
>
>>> +(ert-deftest ert-test-erts-skip-one ()
>>> +  "Test that Skip does not affect subsequent test cases (Bug#76839)."
>>> +  :expected-result :failed
>>
>> I think this should be avoided, because a test can fail for any reason.
>>
>> Can we turn the condition around, so that this test always passes?
>
> The thing is that I need to determine that the test case following the
> skipped one in the erts file has been executed.  If I know that the test
> will fail, it is easy to identify that, which is why I use
> ':expected-result :failed'.  However, If we expect `ert-test-erts-file'
> to succeed, how can I confirm that the second test case has passed
> instead of being skipped as well?
>
> I guess I could handle the `ert-test-failed' error signaled by
> `ert-test-erts-file' when it calls `ert-fail', but is it much better?

I think it's better, because then you reduce the scope to just that one
error.

The test could also fail for any other reason, including that some form
couldn't be read, an incorrect erts file, and so on.  We would want such
things to be flagged, in case they happen.

Does that make sense?  Could you propose an updated patch along these
lines?