GNU bug report logs - #8044
Automake should support autotest-based testsuites.

Previous Next

To reply to this bug, email your comments to 8044 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: bug-automake <at> gnu.org
Subject: Automake should support autotest-based testsuites.
Date: Tue, 15 Feb 2011 18:39:17 +0100

Severity: wishlist

Hello automakers.

I think that automake should support autotest-based testsuites
out-of-the-box.

The steps required to set up an autotest-based testsuite are already
excellently described in the autoconf manual:
 <http://www.gnu.org/software/autoconf/manual/html_node/Making-testsuite-Scripts.html>
and seem pretty mechanical, so it shouldn't be too much difficult to
teach automake how to reproduce them.

Do you think that would be worthwhile?  If yes, I might attempt an
implementation (taking as references/inspiration the autoconf manual
and autoconf's own tests/Makefile.am).  I must warn you that that will
probably take me some time, tough.

Regards,
  Stefano

Message #8 received at 8044 <at> debbugs.gnu.org (full text, mbox):

From: Jack Kelly <jack <at> jackkelly.name>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 8044 <at> debbugs.gnu.org
Subject: Re: bug#8044: Automake should support autotest-based testsuites.
Date: Wed, 16 Feb 2011 05:32:19 +1100

On Wed, Feb 16, 2011 at 4:39 AM, Stefano Lattarini
<stefano.lattarini <at> gmail.com> wrote:
> The steps required to set up an autotest-based testsuite are already
> excellently described in the autoconf manual:
>  <http://www.gnu.org/software/autoconf/manual/html_node/Making-testsuite-Scripts.html>
> and seem pretty mechanical, so it shouldn't be too much difficult to
> teach automake how to reproduce them.
>
> Do you think that would be worthwhile?  If yes, I might attempt an
> implementation (taking as references/inspiration the autoconf manual
> and autoconf's own tests/Makefile.am).  I must warn you that that will
> probably take me some time, tough.

It sounds useful for some (probably not me, though), but I'd be wary
of the big warning at the start of the autotest section:

>> N.B.: This section describes a feature which is still
>> stabilizing.  Although we believe that Autotest is useful as-is, this
>> documentation describes an interface which might change in the future:
>> do not depend upon Autotest without subscribing to the Autoconf mailing
>> lists.

-- Jack

Message #11 received at 8044 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Jack Kelly <jack <at> jackkelly.name>
Cc: Autoconf <Autoconf <at> gnu.org>,
	Stefano Lattarini <stefano.lattarini <at> gmail.com>, 8044 <at> debbugs.gnu.org
Subject: Re: bug#8044: Automake should support autotest-based testsuites.
Date: Tue, 15 Feb 2011 11:44:06 -0700

[Message part 1 (text/plain, inline)]

[adding autoconf]

On 02/15/2011 11:32 AM, Jack Kelly wrote:
> On Wed, Feb 16, 2011 at 4:39 AM, Stefano Lattarini
> <stefano.lattarini <at> gmail.com> wrote:
>> The steps required to set up an autotest-based testsuite are already
>> excellently described in the autoconf manual:
>>  <http://www.gnu.org/software/autoconf/manual/html_node/Making-testsuite-Scripts.html>
>> and seem pretty mechanical, so it shouldn't be too much difficult to
>> teach automake how to reproduce them.
>>
>> Do you think that would be worthwhile?

Absolutely!

>>  If yes, I might attempt an
>> implementation (taking as references/inspiration the autoconf manual
>> and autoconf's own tests/Makefile.am).  I must warn you that that will
>> probably take me some time, tough.
> 
> It sounds useful for some (probably not me, though), but I'd be wary
> of the big warning at the start of the autotest section:
> 
>>> N.B.: This section describes a feature which is still
>>> stabilizing.  Although we believe that Autotest is useful as-is, this
>>> documentation describes an interface which might change in the future:
>>> do not depend upon Autotest without subscribing to the Autoconf mailing
>>> lists.

Maybe it's time to nuke that paragraph from the next autoconf release.
I think we've built up enough of autoconf's testsuite to promise a
stable autotest interface, and that we can commit to maintaining
backwards compatibility when adding further macros for easier testing.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #14 received at 8044 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Eric Blake <eblake <at> redhat.com>
Cc: Autoconf <Autoconf <at> gnu.org>,
	Stefano Lattarini <stefano.lattarini <at> gmail.com>,
	Jack Kelly <jack <at> jackkelly.name>, 8044 <at> debbugs.gnu.org
Subject: Re: bug#8044: Automake should support autotest-based testsuites.
Date: Tue, 15 Feb 2011 21:39:25 +0100

Hello,

* Eric Blake wrote on Tue, Feb 15, 2011 at 07:44:06PM CET:
> > It sounds useful for some (probably not me, though), but I'd be wary
> > of the big warning at the start of the autotest section:
> > 
> >>> N.B.: This section describes a feature which is still
> >>> stabilizing.  Although we believe that Autotest is useful as-is, this
> >>> documentation describes an interface which might change in the future:
> >>> do not depend upon Autotest without subscribing to the Autoconf mailing
> >>> lists.
> 
> Maybe it's time to nuke that paragraph from the next autoconf release.
> I think we've built up enough of autoconf's testsuite to promise a
> stable autotest interface, and that we can commit to maintaining
> backwards compatibility when adding further macros for easier testing.

I agree with this.  I should add though, in a sort-of self-advertising
mood, that you may want to consider that there are still possibly-good
changes that would modify the Autotest API somewhat.  For example, this
AT_CHECK_PROGRESS (which I've been using as AT_CHECK for a while now):
http://news.gmane.org/gmane.comp.sysutils.autoconf.patches/cutoff=7562

Anyway.  I too think support for Autotest in Automake would be a good
thing.

Thanks,
Ralf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.