GNU bug report logs - #67312
feature request: avoid creating logs for no tests

Previous Next

To reply to this bug, email your comments to 67312 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tom Tromey <tom <at> tromey.com>
To: bug-dejagnu <at> gnu.org
Subject: dejagnu creates testrun.{sum,log}
Date: Mon, 20 Nov 2023 16:12:35 -0700

I've noticed that if I run 'runtest' in the wrong directory -- for
example in the source directory rather than the appropriate build
directory -- dejagnu will create files testrun.log and testrun.sum.

In this case these files are not actually useful.

And, runtest already emits some warnings about this:

prentzel. runtest 
WARNING: testsuite is not in a testsuite/ directory.
WARNING: Couldn't find the global config file.
WARNING: No tool specified

I think it would be nicer if, when dejagnu notices that the environment
is unsuitable (like, no tool specified), then it should just print
errors and exit -- without creating these files.

thanks
Tom

Message #8 received at 67312 <at> debbugs.gnu.org (full text, mbox):

From: Jacob Bachmeyer <jcb62281 <at> gmail.com>
To: Tom Tromey <tom <at> tromey.com>
Cc: 67312 <at> debbugs.gnu.org
Subject: Re: bug#67312: dejagnu creates testrun.{sum,log}
Date: Mon, 20 Nov 2023 19:54:24 -0600

Tom Tromey wrote:
> I've noticed that if I run 'runtest' in the wrong directory -- for
> example in the source directory rather than the appropriate build
> directory -- dejagnu will create files testrun.log and testrun.sum.
>
> In this case these files are not actually useful.
>
> And, runtest already emits some warnings about this:
>
> prentzel. runtest 
> WARNING: testsuite is not in a testsuite/ directory.
> WARNING: Couldn't find the global config file.
> WARNING: No tool specified
>
> I think it would be nicer if, when dejagnu notices that the environment
> is unsuitable (like, no tool specified), then it should just print
> errors and exit -- without creating these files.

Each of the warnings mentioned is a warning and not an error because, 
while those indicate that something is probably wrong, there may be 
testsuites out there using them.  The lack of a testsuite/ directory 
violates a long-documented requirement, but 1.6.3 is the first release 
that actually checks that in order to resolve an ambiguity between 
recursive and non-recursive Automake-generated Makefiles.  (Several 
DejaGnu tests in the Automake testsuite had to be patched to fix exactly 
the misuse that triggers that warning, so there may be other testsuites 
out there that do not properly use testsuite/.  The ambiguity concerns 
whether srcdir is "testsuite" itself or its parent.)  The global config 
file is part of site configuration, is expected to define remote 
targets, and is actually optional if built-in defaults are sufficient, 
as they usually are for the "unix" target.  Lastly, while failure to 
specify a tool produces a warning, it is a valid invocation of runtest.

The problem with bailing out in this case is that failing to specify a 
tool is a valid (legacy, possibly deprecated) use:  it means to run all 
tests (for all tools) in the testsuite.  Writing log and summary files 
as if a tool named "testrun" was tested is the long-standing behavior in 
this case and there is actually explicit code in lib/framework.exp to 
support this.  (That code may also mean that a tool literally named 
"testrun" cannot actually be tested...)

Of the three warnings, the lack of a testsuite/ directory is the best 
candidate to upgrade to a hard error since it has seemingly always been 
a documented requirement, but all previous releases would silently 
accept a testsuite not using that directory, so moving directly to 
"ERROR:  no testsuite/ directory found.  Stop." is too much to do on a 
point release.

Are the produced files empty?  I will consider if there might be a good 
way to detect a completely bogus run (example:  no test scripts found at 
all) and bail out early without creating logfiles, but this issue seems 
to be a valid use case that is misfiring in your situation.


-- Jacob

Message #11 received at 67312 <at> debbugs.gnu.org (full text, mbox):

From: Tom Tromey <tom <at> tromey.com>
To: Jacob Bachmeyer <jcb62281 <at> gmail.com>
Cc: 67312 <at> debbugs.gnu.org, Tom Tromey <tom <at> tromey.com>
Subject: Re: bug#67312: dejagnu creates testrun.{sum,log}
Date: Tue, 21 Nov 2023 06:59:23 -0700

Jacob> Are the produced files empty?  I will consider if there might be a
Jacob> good way to detect a completely bogus run (example:  no test scripts
Jacob> found at all) and bail out early without creating logfiles, but this
Jacob> issue seems to be a valid use case that is misfiring in your
Jacob> situation.

They aren't zero-length but they don't contain any useful information at
all.  You can easily try it yourself by "cd /tmp; runtest".

Tom

Message #20 received at 67312 <at> debbugs.gnu.org (full text, mbox):

From: Jacob Bachmeyer <jcb62281 <at> gmail.com>
To: Tom Tromey <tom <at> tromey.com>
Cc: 67312 <at> debbugs.gnu.org
Subject: Re: bug#67312: dejagnu creates testrun.{sum,log}
Date: Tue, 21 Nov 2023 22:10:23 -0600

Tom Tromey wrote:
> Jacob> Are the produced files empty?  I will consider if there might be a
> Jacob> good way to detect a completely bogus run (example:  no test scripts
> Jacob> found at all) and bail out early without creating logfiles, but this
> Jacob> issue seems to be a valid use case that is misfiring in your
> Jacob> situation.
>
> They aren't zero-length but they don't contain any useful information at
> all.  You can easily try it yourself by "cd /tmp; runtest".

Indeed, running runtest in an empty directory produces useless 
(boilerplate) log and summary files.  The testsuite has (nominally) been 
located before the logs are opened, but DejaGnu currently has a big ball 
of mud around actually finding and running the tests and does not check 
for test files until after the logs have been opened.

Fixing this has been added to my TODO list, and extracting the "find 
tests" stage from "find and run tests" will be needed for the planned 
future native parallel testing support in any case.  This will require 
some fairly extensive overhauls in runtest.exp to separate that step and 
effectively plan tests before running them instead of just running 
them.  I am also planning to expand tool handling at some point, 
eventually allowing multiple --tool options to be given in a single run; 
the effect will be equivalent to consecutive runtest invocations, but 
may have better performance in parallel mode, since that would allow 
testing of different tools to overlap.

Until then, I will carry this bug report as a feature request.


-- Jacob

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.