GNU bug report logs -
#79077
host_execute and non-zero exit status
Previous Next
Full log
Message #17 received at 79077 <at> debbugs.gnu.org (full text, mbox):
Hi,
Am Fr., 25. Juli 2025 um 04:14 Uhr schrieb Jacob Bachmeyer <jcb62281 <at> gmail.com>:
[...]
> Is Nieper-Wisskirchen a proper ASCII transliteration of your name for a thanks in the ChangeLog?
Yes, exactly.
> Am Do., 24. Juli 2025 um 05:09 Uhr schrieb Jacob Bachmeyer <jcb62281 <at> gmail.com>:
>
> On 7/23/25 02:31, Marc Nieper-Wißkirchen wrote:
>
> Hi,
>
> The host_execute procedure in dejagnu.exp (see [1]) doesn't seem to
> check the exit status of the executed test programs. A test program
> that simply aborts (e.g. using the C function of the same name) won't
> cause any testsuite failures. This seems brittle and like a
> misfeature.
>
> What is the supposed way to deal with this?
>
> The host_execute procedure is intended for running unit test programs
> that speak a special DejaGnu protocol and ignoring exit codes from unit
> test programs is intended. The DejaGnu unit testing protocol does not
> depend on the exit code of the unit test program at all, because that
> exit code might not be available in all environments.
>
> I am sorry if I wasn't clear enough in my previous email. I didn't
> mean that the exit code should be part of the actual protocol, only
> that running the testcase should be considered unsuccessful if the
> program exits with a failure code (in a POSIX system; in some other
> environment, there may be other indications for failure of execution).
>
> But that would make the exit code part of the protocol. DejaGnu generally supports running tests on "remote" target boards and a target connected over a serial line is unlikely to return an exit code. The only way to be sure that a dependency on the exit code will not creep in is to ignore the exit code.
With the recent addition of the "END" token, this is probably moot;
without, how would have DejaGnu handled a test running remotely, say
over a serial line, where the connection drops early? I saw a non-zero
exit code as an analogue of such a dropped connection, which could be
expressed by an "UNRESOLVED" result.
>
> Admittedly, DejaGnu does not yet properly support running unit tests on remote targets.
>
> Instead, DejaGnu expects an explicit "END" token from the unit test
> program to indicate that the program has reached its intended
> completion. A warning is produced if this token is not observed;
> perhaps a future version of DejaGnu should insert an UNRESOLVED result
> like we currently do when a Tcl test script aborts?
>
> I think inserting an UNRESOLVED result would be more appropriate.
> Otherwise, there would be no formal failure result if a unit test
> breaks due to, say, a segfault.
>
> The explicit "END" token is a relatively recent addition (added after the last release). I was reluctant to outright require it (in case there are any testsuites out there with independent unit test protocol implementations) but I now agree that a warning from the test framework is not enough when a unit test bombs out early.
>
> An initial solution has been pushed to Savannah on the PR79077 branch. DejaGnu can now be run directly from a Git checkout, you should be able to simply pass RUNTEST=/full/name/of/working/tree/runtest on the "make check" command line.
That is great news. Thank you! Moreover, thank you very much for your
ideas about the Valgrind mapper; very helpful, indeed.
Best regards,
Marc
[...]
This bug report was last modified 14 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.