GNU bug report logs - #10431
a .trs file left behind for each test

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10431 in the body.
You can then email your comments to 10431 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: bug-automake <at> gnu.org
Subject: a .trs file left behind for each test
Date: Wed, 04 Jan 2012 16:10:05 +0100

Hi Stefano,

[using latest automake from git.master]
For example, in grep and coreutils, after I run "make check", I find
that every test has a corresponding .trs file alongside the usual .log file.

Those are indeed removed via "make clean", but shouldn't they be removed
also upon normal completion of each test?

Message #8 received at 10431 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 10431 <at> debbugs.gnu.org
Subject: Re: bug#10431: a .trs file left behind for each test
Date: Wed, 04 Jan 2012 17:59:49 +0100

tags 10431 notabug
close 10431
thanks

On 01/04/2012 04:10 PM, Jim Meyering wrote:
> Hi Stefano,
>
Hi Jim, thanks for the report.

> [using latest automake from git.master]
> For example, in grep and coreutils, after I run "make check", I find
> that every test has a corresponding .trs file alongside the usual .log file.
>
Yep, that's by design; the new automake-generated testsuite harness needs
to register, for each test script that it runs, more information than the
simple "final result" of the script.  It would have been really awkward and
fragile to do so "in band", i.e., with special directives embedded into the
`.log' files[1]; so we register this new information safely out-of-band,
into these `.trs' files (where "trs" stands for "Test ReSults").  This new
interface should be clearly documented in 'master' version of the automake
manual; if something is missing or unclear, that's a bug, and feel free to
report it!

[1] I had taken the in-bad approach initially:
     <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html>
    but it proved brittle and bug-prone (as predicted by Ralf); hence the
    introduction of the `.trs' files:
     <http://lists.gnu.org/archive/html/automake-patches/2011-07/msg00097.html>
     <http://lists.gnu.org/archive/html/automake-patches/2011-08/msg00001.html>

> Those are indeed removed via "make clean", but shouldn't they be removed
> also upon normal completion of each test?
>
Oh no!  The testsuite harness will need them to perform various tasks; e.g.,
the creation of the final `test-suite.log' file, the display of the testsuite
summary, the decision of which files to re-run upon "make recheck", etc.

HTH,
  Stefano

P.S. Let me reiterate that, while I greatly appreciate your continual use
and testing the bleeding-edge automake versions with real-world packages,
the code from `master' it is IMO *not* stable or mature enough for use in
production code.  So I'd advise against its use in the upcoming coreutils
release (but you could use the version from maint, which has a couple of
bug fixes w.r.t. 1.11.2, and it's IMHO very stable).

Message #11 received at 10431 <at> debbugs.gnu.org (full text, mbox):

From: Peter Rosin <peda <at> lysator.liu.se>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 10431 <at> debbugs.gnu.org, Jim Meyering <jim <at> meyering.net>
Subject: Re: bug#10431: a .trs file left behind for each test
Date: Wed, 04 Jan 2012 18:15:15 +0100

Stefano Lattarini skrev 2012-01-04 17:59:
> P.S. Let me reiterate that, while I greatly appreciate your continual use
> and testing the bleeding-edge automake versions with real-world packages,
> the code from `master' it is IMO *not* stable or mature enough for use in
> production code.  So I'd advise against its use in the upcoming coreutils
> release (but you could use the version from maint, which has a couple of
> bug fixes w.r.t. 1.11.2, and it's IMHO very stable).

But maint has an outdated version of lib/compile and lib/ar-lib isn't
provided in any shape or form.  So, from my point of view, please don't
use maint for production either.

Cheers,
Peter

Message #14 received at 10431 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 10431 <at> debbugs.gnu.org
Subject: Re: bug#10431: a .trs file left behind for each test
Date: Wed, 04 Jan 2012 18:21:43 +0100

Stefano Lattarini wrote:
...
>> [using latest automake from git.master]
>> For example, in grep and coreutils, after I run "make check", I find
>> that every test has a corresponding .trs file alongside the usual .log file.
>>
> Yep, that's by design; the new automake-generated testsuite harness needs
> to register, for each test script that it runs, more information than the
...
>      <http://lists.gnu.org/archive/html/automake-patches/2011-07/msg00097.html>
>      <http://lists.gnu.org/archive/html/automake-patches/2011-08/msg00001.html>

Indeed, I missed those patches and should have looked at the source
before reporting.  Sorry my full schedule led me to take that short-cut.
(you're partly to thank/blame for posting those coreutils snapshot results :-)

> P.S. Let me reiterate that, while I greatly appreciate your continual use
> and testing the bleeding-edge automake versions with real-world packages,
> the code from `master' it is IMO *not* stable or mature enough for use in
> production code.  So I'd advise against its use in the upcoming coreutils
> release (but you could use the version from maint, which has a couple of
> bug fixes w.r.t. 1.11.2, and it's IMHO very stable).

I understand your sentiment and appreciate the concern, but so far all
of the evidence I've seen (continual monitoring of automake commits and
regular testing both of automake and of packages built using the very
latest automake) suggests that you are wrong.

Message #17 received at 10431 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Peter Rosin <peda <at> lysator.liu.se>
Cc: 10431 <at> debbugs.gnu.org, Jim Meyering <jim <at> meyering.net>
Subject: Re: bug#10431: a .trs file left behind for each test
Date: Wed, 04 Jan 2012 19:14:11 +0100

On 01/04/2012 06:15 PM, Peter Rosin wrote:
> Stefano Lattarini skrev 2012-01-04 17:59:
>> P.S. Let me reiterate that, while I greatly appreciate your continual use
>> and testing the bleeding-edge automake versions with real-world packages,
>> the code from `master' it is IMO *not* stable or mature enough for use in
>> production code.  So I'd advise against its use in the upcoming coreutils
>> release (but you could use the version from maint, which has a couple of
>> bug fixes w.r.t. 1.11.2, and it's IMHO very stable).
> 
> But maint has an outdated version of lib/compile and lib/ar-lib [SNIP]
>
My bad!  I should have recommended `branch-1.11', not `maint'.  Consider
my advise updated accordingly :-)

Regards,
  Stefano

Message #20 received at 10431 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 10431 <at> debbugs.gnu.org
Subject: Re: bug#10431: a .trs file left behind for each test
Date: Wed, 04 Jan 2012 19:21:03 +0100

On 01/04/2012 06:21 PM, Jim Meyering wrote:
> Stefano Lattarini wrote:
>>
>> P.S. Let me reiterate that, while I greatly appreciate your continual use
>> and testing the bleeding-edge automake versions with real-world packages,
>> the code from `master' it is IMO *not* stable or mature enough for use in
>> production code.  So I'd advise against its use in the upcoming coreutils
>> release (but you could use the version from maint, which has a couple of
>> bug fixes w.r.t. 1.11.2, and it's IMHO very stable).
> 
> I understand your sentiment and appreciate the concern, but so far all
> of the evidence I've seen (continual monitoring of automake commits and
> regular testing both of automake and of packages built using the very
> latest automake) suggests that you are wrong.
>
When, then for once I'm happy to be wrong :-)

Regards, and thanks,
  Stefano

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.