From unknown Mon Aug 18 17:53:45 2025 X-Loop: help-debbugs@gnu.org Subject: bug#11287: Various issues with the test suite framework Resent-From: Akim Demaille Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-automake@gnu.org Resent-Date: Fri, 20 Apr 2012 11:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 11287 X-GNU-PR-Package: automake X-GNU-PR-Keywords: To: 11287@debbugs.gnu.org X-Debbugs-Original-To: Automake Bugs Received: via spool by submit@debbugs.gnu.org id=B.133492115323248 (code B ref -1); Fri, 20 Apr 2012 11:26:01 +0000 Received: (at submit) by debbugs.gnu.org; 20 Apr 2012 11:25:53 +0000 Received: from localhost ([127.0.0.1]:43689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SLByK-00062u-QA for submit@debbugs.gnu.org; Fri, 20 Apr 2012 07:25:53 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51084) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SLByI-00062d-DM for submit@debbugs.gnu.org; Fri, 20 Apr 2012 07:25:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SLBxk-0004yh-Dr for submit@debbugs.gnu.org; Fri, 20 Apr 2012 07:25:20 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:44646) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SLBxk-0004e2-Ac for submit@debbugs.gnu.org; Fri, 20 Apr 2012 07:25:16 -0400 Received: from eggs.gnu.org ([208.118.235.92]:43125) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SLBnW-0008JF-QU for bug-automake@gnu.org; Fri, 20 Apr 2012 07:14:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SLBnO-0000dg-W4 for bug-automake@gnu.org; Fri, 20 Apr 2012 07:14:42 -0400 Received: from sao-paulo.lrde.epita.fr ([163.5.55.1]:42255 helo=kualalumpur.lrde.epita.fr) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SLBnO-0000ZP-Pp for bug-automake@gnu.org; Fri, 20 Apr 2012 07:14:34 -0400 Received: from tenochtitlan-eth.lrde.epita.fr ([192.168.101.154]) by kualalumpur.lrde.epita.fr with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1SLBmq-00013l-Q5 for bug-automake@gnu.org; Fri, 20 Apr 2012 13:14:01 +0200 From: Akim Demaille Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Fri, 20 Apr 2012 13:13:58 +0200 Message-Id: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 208.118.235.17 X-Spam-Score: -6.9 (------) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.9 (------) Hi! I have seen that check-html will be removed, or rather moved into the contrib part, but there are a few issues: - the target is not declared recursive, so one has to write the bouncing target herself. - because of that, the "naive" implementation of check-html that just bounces the right directories will miss the dependencies that check features (in other words, make check properly recursively builds before, while check-html would just go into the typical tests/ directory, and then bounce to "check", so it completely missed the recursion in the other directories and their dependencies). - RECURSIVE_TARGETS cannot be highjacked by the users, that's sad. Could a $(USER_RECURSIVE_TARGETS) be added to RECURSIVE_TARGETS so that we can add our own targets that traverse the whole package? Actually maybe this should be an Automake option to add recursive targets, and then TARGET-am and TARGET-recursive would be generated and bound to the proper TARGET. - the .log.html recipe features $$RST2HTML, but in '', so it is useless. And anyway $(RST2HTML) already does the job. - "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing. - the test compiler does not appear as a dependency for the test logs that use it. This is most useful when RECHECK_LOG =. - it would be equally useful to be able to add dependencies to a category of test drivers. Say, with the documented example: TESTS = foo.pl bar.py baz TEST_EXTENSIONS = .pl .py PL_LOG_COMPILER = $(PERL) AM_PL_LOG_FLAGS = -w PY_LOG_COMPILER = $(PYTHON) AM_PY_LOG_FLAGS = -v LOG_COMPILER = ./wrapper-script AM_LOG_FLAGS = -d be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and LOG_DEPENDENCIES. I don't think EXTRA dependencies would be useful. Cheers! From unknown Mon Aug 18 17:53:45 2025 X-Loop: help-debbugs@gnu.org Subject: bug#11287: Various issues with the test suite framework Resent-From: Stefano Lattarini Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-automake@gnu.org Resent-Date: Tue, 24 Apr 2012 09:44:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11287 X-GNU-PR-Package: automake X-GNU-PR-Keywords: To: Akim Demaille Cc: 11287@debbugs.gnu.org Received: via spool by 11287-submit@debbugs.gnu.org id=B11287.133526058430339 (code B ref 11287); Tue, 24 Apr 2012 09:44:01 +0000 Received: (at 11287) by debbugs.gnu.org; 24 Apr 2012 09:43:04 +0000 Received: from localhost ([127.0.0.1]:50011 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMcH2-0007tI-AR for submit@debbugs.gnu.org; Tue, 24 Apr 2012 05:43:04 -0400 Received: from mail-bk0-f44.google.com ([209.85.214.44]:62839) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMcH0-0007sp-0D for 11287@debbugs.gnu.org; Tue, 24 Apr 2012 05:43:03 -0400 Received: by bkuw5 with SMTP id w5so322796bku.3 for <11287@debbugs.gnu.org>; Tue, 24 Apr 2012 02:42:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=MrGkR5ntJiAzm81eo5mxggSvyVboxfWEH+mahgHzENA=; b=TIeNQx1zntyL+aZs+D6xCZnbS78irXGTHDC6PIIr/7Mv6JhFP/YkOH3KpKMF4VYWZT Bh9hidxQZdvbJeM0OPVz1mJb0YDv1JYnXYceE93V1Smb/jy7ZkEYl7sgo1T1bxWOlp9v dcybEyjyvFmyYN51o2zteFiHEP2qrlQXNdSY5MWmcizCYqjcNIUYRlwtBEHTKRC2Qxiw 4oJ/RV4coGxyi4pzOpRWfY0xrsKxBh9p2HDs/oHFq6cfMYkPHQTf0CIfsUDON0nk26r2 0EYQ9j0HM7b1JIrvcgu5JUgzQwoj11rFZLniSiStho4gHkeWr8/Jo8cwnRUULu7PZlbk tMRQ== Received: by 10.204.150.89 with SMTP id x25mr171357bkv.24.1335260529614; Tue, 24 Apr 2012 02:42:09 -0700 (PDT) Received: from [82.54.101.169] (host169-101-dynamic.54-82-r.retail.telecomitalia.it. [82.54.101.169]) by mx.google.com with ESMTPS id z17sm30492575bkw.12.2012.04.24.02.42.07 (version=SSLv3 cipher=OTHER); Tue, 24 Apr 2012 02:42:08 -0700 (PDT) Message-ID: <4F96756D.9010704@gmail.com> Date: Tue, 24 Apr 2012 11:42:05 +0200 From: Stefano Lattarini MIME-Version: 1.0 References: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> In-Reply-To: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Spam-Score: -2.6 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) On 04/20/2012 01:13 PM, Akim Demaille wrote: > Hi! > Hi Akim, sorry for the delay. > I have seen that check-html will be removed, or rather moved > into the contrib part, but there are a few issues: > > - the target is not declared recursive, so one has to write > the bouncing target herself. > > - because of that, the "naive" implementation of check-html > that just bounces the right directories will miss the > dependencies that check features (in other words, make > check properly recursively builds before, while check-html > would just go into the typical tests/ directory, and then > bounce to "check", so it completely missed the recursion > in the other directories and their dependencies). > Wasn't this issue already present in Automake 1.11.1? (So, not a regression, unless I'm mistaken). In any case, a test case exposing the issue would be appreciated, if you have one at hand. > - the .log.html recipe features $$RST2HTML, but in '', so it is > useless. And anyway $(RST2HTML) already does the job. > > - "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing. > None of these are covered by the Automake test suite sadly, and I see them as really low low priority anyway, so I'm not going to tackle them right now. But I'll gladly accept patches by anyone who has more knowledge or interest than me in this area ;-) > - RECURSIVE_TARGETS cannot be highjacked by the users, that's > sad. Could a $(USER_RECURSIVE_TARGETS) be added to > RECURSIVE_TARGETS so that we can add our own targets that > traverse the whole package? > There had been an attempt to offer an interface for user-defined recursive rules already: But unfortunately the discussion stagnated and died out, and I don't have time to resurrect it nor the relevant patches ATM. Still, it would be a cool feature to have, so any help would be appreciated. > Actually maybe this should > be an Automake option to add recursive targets, and > then TARGET-am and TARGET-recursive would be generated and > bound to the proper TARGET. > > - the test compiler does not appear as a dependency for > the test logs that use it. > This makes sense, since the test compiler is not required to be an in-project script, and not even to be specified as an absolute path -- it can be a command like "sh" or "/usr/bin/env perl". > - it would be equally useful to be able to add dependencies > to a category of test drivers. Say, with the documented > example: > > TESTS = foo.pl bar.py baz > TEST_EXTENSIONS = .pl .py > PL_LOG_COMPILER = $(PERL) > AM_PL_LOG_FLAGS = -w > PY_LOG_COMPILER = $(PYTHON) > AM_PY_LOG_FLAGS = -v > LOG_COMPILER = ./wrapper-script > AM_LOG_FLAGS = -d > > be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and > LOG_DEPENDENCIES. > This seems like a cool feature, but not trivial to implement in the current Automake code base. Still, I have some half-finished patches to rework the parallel-tests harness in Automake-NG (reviews welcome, BTW ;-), and once those are in place, implementing your proposed feature in Automake-NG might be very easy. > I don't think EXTRA dependencies would be useful. > Right. If we'll ever need automatically computed dependencies, we should name them AM_PL_LOG_DEPENDENCIES etc., without impinging on the user namespace. Thanks, Stefano From unknown Mon Aug 18 17:53:45 2025 X-Loop: help-debbugs@gnu.org Subject: bug#11287: Various issues with the test suite framework Resent-From: Akim Demaille Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-automake@gnu.org Resent-Date: Tue, 24 Apr 2012 12:00:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11287 X-GNU-PR-Package: automake X-GNU-PR-Keywords: To: Stefano Lattarini Cc: 11287@debbugs.gnu.org Received: via spool by 11287-submit@debbugs.gnu.org id=B11287.133526877713549 (code B ref 11287); Tue, 24 Apr 2012 12:00:02 +0000 Received: (at 11287) by debbugs.gnu.org; 24 Apr 2012 11:59:37 +0000 Received: from localhost ([127.0.0.1]:50222 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMePA-0003WT-MX for submit@debbugs.gnu.org; Tue, 24 Apr 2012 07:59:37 -0400 Received: from sao-paulo.lrde.epita.fr ([163.5.55.1]:58866 helo=kualalumpur.lrde.epita.fr) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMeP8-0003WG-4E for 11287@debbugs.gnu.org; Tue, 24 Apr 2012 07:59:35 -0400 Received: from tenochtitlan-eth.lrde.epita.fr ([192.168.101.154]) by kualalumpur.lrde.epita.fr with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1SMeOG-0005Nu-RT; Tue, 24 Apr 2012 13:58:40 +0200 Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: Akim Demaille In-Reply-To: <4F96756D.9010704@gmail.com> Date: Tue, 24 Apr 2012 13:58:40 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> <4F96756D.9010704@gmail.com> X-Mailer: Apple Mail (2.1084) X-Spam-Score: -1.9 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) Le 24 avr. 2012 =E0 11:42, Stefano Lattarini a =E9crit : > On 04/20/2012 01:13 PM, Akim Demaille wrote: >> Hi! >>=20 > Hi Akim, sorry for the delay. Hi Stefano, No worries. >> I have seen that check-html will be removed, or rather moved >> into the contrib part, but there are a few issues: >>=20 >> - the target is not declared recursive, so one has to write >> the bouncing target herself. >>=20 >> - because of that, the "naive" implementation of check-html >> that just bounces the right directories will miss the >> dependencies that check features (in other words, make >> check properly recursively builds before, while check-html >> would just go into the typical tests/ directory, and then >> bounce to "check", so it completely missed the recursion >> in the other directories and their dependencies). >>=20 > Wasn't this issue already present in Automake 1.11.1? (So, not a > regression, unless I'm mistaken). In any case, a test case > exposing the issue would be appreciated, if you have one at hand. No, I don't think this is a regression. It is a problem though. I don't know (yet) what contribs/ are in Automake, but this shows that some real assistance from Automake would really help, it's not just a matter of a single Make snippet in a single Makefile, but something more pervasive to the whole package. >> - the .log.html recipe features $$RST2HTML, but in '', so it is >> useless. And anyway $(RST2HTML) already does the job. >>=20 >> - "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing. >>=20 > None of these are covered by the Automake test suite sadly, and I > see them as really low low priority anyway, so I'm not going to > tackle them right now. But I'll gladly accept patches by anyone > who has more knowledge or interest than me in this area ;-) I'll look for someone matching these specs :) >> - RECURSIVE_TARGETS cannot be highjacked by the users, that's >> sad. Could a $(USER_RECURSIVE_TARGETS) be added to >> RECURSIVE_TARGETS so that we can add our own targets that >> traverse the whole package? >>=20 > There had been an attempt to offer an interface for user-defined > recursive rules already: >=20 > >=20 > But unfortunately the discussion stagnated and died out, and I don't > have time to resurrect it nor the relevant patches ATM. Still, it > would be a cool feature to have, so any help would be appreciated. Yes, definitely. >> - the test compiler does not appear as a dependency for >> the test logs that use it. >>=20 > This makes sense, since the test compiler is not required to be an > in-project script, Which is not a problem. I am very happy that my dependencies include headers from /usr and not just those in the package. > and not even to be specified as an absolute path > -- it can be a command like "sh" or "/usr/bin/env perl". which is one, indeed. >> - it would be equally useful to be able to add dependencies >> to a category of test drivers. Say, with the documented >> example: >>=20 >> TESTS =3D foo.pl bar.py baz >> TEST_EXTENSIONS =3D .pl .py >> PL_LOG_COMPILER =3D $(PERL) >> AM_PL_LOG_FLAGS =3D -w >> PY_LOG_COMPILER =3D $(PYTHON) >> AM_PY_LOG_FLAGS =3D -v >> LOG_COMPILER =3D ./wrapper-script >> AM_LOG_FLAGS =3D -d >>=20 >> be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and >> LOG_DEPENDENCIES. >>=20 > This seems like a cool feature, but not trivial to implement in the = current > Automake code base. Still, I have some half-finished patches to = rework > the parallel-tests harness in Automake-NG (reviews welcome, BTW ;-), I'll pull :) > and > once those are in place, implementing your proposed feature in = Automake-NG > might be very easy. Good news.= From unknown Mon Aug 18 17:53:45 2025 X-Loop: help-debbugs@gnu.org Subject: bug#11287: [FYI] {master} coverage: check-html can be used recursively References: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> In-Reply-To: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> Resent-From: Stefano Lattarini Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-automake@gnu.org Resent-Date: Mon, 02 Jul 2012 18:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11287 X-GNU-PR-Package: automake X-GNU-PR-Keywords: To: automake-patches@gnu.org Cc: 11287@debbugs.gnu.org Received: via spool by 11287-submit@debbugs.gnu.org id=B11287.134125433818456 (code B ref 11287); Mon, 02 Jul 2012 18:39:01 +0000 Received: (at 11287) by debbugs.gnu.org; 2 Jul 2012 18:38:58 +0000 Received: from localhost ([127.0.0.1]:43145 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SllWT-0004nc-PG for submit@debbugs.gnu.org; Mon, 02 Jul 2012 14:38:58 -0400 Received: from mail-wg0-f46.google.com ([74.125.82.46]:57240) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SllWR-0004nP-Qn for 11287@debbugs.gnu.org; Mon, 02 Jul 2012 14:38:56 -0400 Received: by wgbdq10 with SMTP id dq10so5206371wgb.15 for <11287@debbugs.gnu.org>; Mon, 02 Jul 2012 11:34:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer; bh=Dz44o083vI29Q2u2+IoRup+SlyweA7WcUnHVwwU6Ep0=; b=tEopcvRLnOr64/Zmlo7t7UKbYeA8oiS6Y1YTz2IcZb8OauYG38chJI87PIK4rV9GvB tgS54JPKHS5SEn/TYhZkwJ6crY76HfmxA6IGlWwohePBqGMlhYgFCfPq1ryl0kZFVwqC fRQzwXJLe5WuYKxdVd2i3XKRUUyPOQJXdQ2OTnLTF1Fej6DO+V04nRBUZsrpFmRXFlzM 6uqb259vusSGUQ84l606ExRlQyu/ZHh5ihS7+r+qAnikSIr4FZ8BiRkbxM4I+F8b9jB/ NmrxlOMs/XqHRrIrEUrG9G4Ztu3gSzLVaG/8kBIgk8DO7k2suKqvKFCcEdvX3gNCdQWy Zm7g== Received: by 10.180.109.195 with SMTP id hu3mr7046639wib.8.1341254057934; Mon, 02 Jul 2012 11:34:17 -0700 (PDT) Received: from localhost.localdomain (host106-37-dynamic.56-82-r.retail.telecomitalia.it. [82.56.37.106]) by mx.google.com with ESMTPS id d10sm47343445wiy.3.2012.07.02.11.34.15 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 02 Jul 2012 11:34:16 -0700 (PDT) From: Stefano Lattarini Date: Mon, 2 Jul 2012 20:34:07 +0200 Message-Id: X-Mailer: git-send-email 1.7.9.5 X-Spam-Score: -0.9 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -0.9 (/) Addresses the main part of Automake bug#11287. * contrib/t/parallel-tests-html-recursive.sh: New test. * contrib/t/local.am: Add it. Signed-off-by: Stefano Lattarini --- contrib/t/local.am | 1 + contrib/t/parallel-tests-html-recursive.sh | 163 ++++++++++++++++++++++++++++ 2 files changed, 164 insertions(+) create mode 100755 contrib/t/parallel-tests-html-recursive.sh diff --git a/contrib/t/local.am b/contrib/t/local.am index 0a5f5c1..3859c70 100644 --- a/contrib/t/local.am +++ b/contrib/t/local.am @@ -20,5 +20,6 @@ contrib_testsuite_dir = contrib/t contrib_TESTS = \ $(contrib_testsuite_dir)/parallel-tests-html.sh \ + $(contrib_testsuite_dir)/parallel-tests-html-recursive.sh \ $(contrib_testsuite_dir)/help-multilib.sh \ $(contrib_testsuite_dir)/multilib.sh diff --git a/contrib/t/parallel-tests-html-recursive.sh b/contrib/t/parallel-tests-html-recursive.sh new file mode 100755 index 0000000..12d967e --- /dev/null +++ b/contrib/t/parallel-tests-html-recursive.sh @@ -0,0 +1,163 @@ +#! /bin/sh +# Copyright (C) 2012 Free Software Foundation, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2, or (at your option) +# any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# Recursive use of 'check-html'. See Automake bug#11287. + +. ./defs || exit 1 + +# Try the variants that are tried in check-html.am. +while :; do + for r2h in $RST2HTML rst2html rst2html.py; do + echo "$me: running $r2h --version" + $r2h --version && break 2 + : For shells with busted 'set -e'. + done + skip_all_ "no proper rst2html program found" +done +unset r2h + +cp "$am_top_srcdir"/contrib/check-html.am . \ + || fatal_ "cannot fetch 'check-html.am' from contrib" + +cat >> configure.ac << 'END' +AM_EXTRA_RECURSIVE_TARGETS([check-html]) +AC_CONFIG_FILES([sub/Makefile sub/more/Makefile]) +AC_OUTPUT +END + +cat > Makefile.am << 'END' +SUBDIRS = sub +EXTRA_DIST = $(TESTS) +TEST_SUITE_LOG = mylog.log +TESTS = foo.test bar.sh mu +XFAIL_TESTS = bar.sh +check_SCRIPTS = bla +bla: + echo '#!/bin/sh' > $@-t + echo 'echo Blah Blah Blah' >> $@-t + chmod a+x,a-w $@-t + mv -f $@-t $@ +CLEANFILES = bla +include $(srcdir)/check-html.am +END + +mkdir sub +echo SUBDIRS = more > sub/Makefile.am + +mkdir sub/more +cat > sub/more/Makefile.am << 'END' +include $(top_srcdir)/check-html.am +TEST_EXTENSIONS = .test .sh +TESTS = sh.sh test.test +LOG_COMPILER = true +test.log: sh.log +nodist_check_DATA = x.txt +$(nodist_check_DATA): + echo $@ > $@ +CLEANFILES = $(nodist_check_DATA) +EXTRA_DIST = $(TESTS) +END + +cat > foo.test <<'END' +#! /bin/sh +./bla +exit 77 +END + +cat > bar.sh <<'END' +#! /bin/sh +echo "this is $0" +exit 1 +END + +cat > mu <<'END' +#! /bin/sh +set -x +test -f sub/more/test.log +test -f sub/more/sh.log +END + +cat > sub/more/test.test << 'END' +#!/bin/sh +echo "this is $0" +set -x +test -f sh.log +test -f x.txt +exit 77 +END + +cat > sub/more/sh.sh << 'END' +#!/bin/sh +set -x +test ! -f test.log +test -f x.txt +END + + +cat > sub/more/mu << 'END' +#!/bin/sh +exit 99 +END + +chmod a+x foo.test bar.sh mu sub/more/test.test sub/more/sh.sh + +$ACLOCAL +$AUTOCONF +$AUTOMAKE -a + +./configure + +$MAKE check-html +grep 'Blah Blah Blah' mylog.html +grep 'this is .*bar\.sh' mylog.html +grep 'this is .*test\.test' sub/more/test-suite.html +# check-html should cause check_SCRIPTS and check_DATA to be created. +test -f bla +test -f sub/more/x.txt + +# "make clean" should remove HTML files. +$MAKE clean +test ! -e mylog.html +test ! -e sub/more/test-suite.html +test ! -e bla +test ! -e sub/more/x.txt + +# Create HTML output for individual tests. + +$MAKE bla +$MAKE foo.html bar.sh.html +grep 'Blah Blah Blah' foo.html +grep 'this is .*bar\.sh' bar.sh.html +test ! -e mu.hml + +ocwd=$(pwd) || fatal_ "getting current workind directory" +( cd sub/more \ + && $MAKE sh.html \ + && test -f sh.html \ + && test ! -e test.html \ + && $MAKE test.html \ + && grep 'this is .*test\.test' test.html) || exit 1 + +# HTML output removed by mostlyclean. +$MAKE check-html +test -f mylog.html +test -f sub/more/test-suite.html +$MAKE mostlyclean +find . -name '*.html' | grep . && exit 1 + +$MAKE distcheck + +: -- 1.7.9.5 From unknown Mon Aug 18 17:53:45 2025 X-Loop: help-debbugs@gnu.org Subject: bug#11287: [FYI] {master} contrib: check-html: pass flags to rst2html invocations References: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> In-Reply-To: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> Resent-From: Stefano Lattarini Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-automake@gnu.org Resent-Date: Mon, 02 Jul 2012 19:19:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11287 X-GNU-PR-Package: automake X-GNU-PR-Keywords: To: automake-patches@gnu.org Cc: 11287@debbugs.gnu.org Received: via spool by 11287-submit@debbugs.gnu.org id=B11287.134125673722053 (code B ref 11287); Mon, 02 Jul 2012 19:19:01 +0000 Received: (at 11287) by debbugs.gnu.org; 2 Jul 2012 19:18:57 +0000 Received: from localhost ([127.0.0.1]:43226 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Slm9B-0005je-Cp for submit@debbugs.gnu.org; Mon, 02 Jul 2012 15:18:57 -0400 Received: from mail-wg0-f46.google.com ([74.125.82.46]:45687) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Slm99-0005jV-8D for 11287@debbugs.gnu.org; Mon, 02 Jul 2012 15:18:56 -0400 Received: by wgbdq10 with SMTP id dq10so5237996wgb.15 for <11287@debbugs.gnu.org>; Mon, 02 Jul 2012 12:14:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer; bh=frjIWVr+S2ipYS7RGAXJJTunE6SGoF276zk9vvbCniY=; b=dXHyOZpQ/myi3KoLQPdFNKAZrjwkwtaQfD8caj39HJvdcikGnkw9O2hDfljmrkqCSg t1q6+jzsCrhKrYGvmZ0VCch6DA7FB4bkrj1vYET1aJIMEza2GXr2ppeJnbu0laKapLpP DcCSQ7l1n7B24HL7JgZ/QqmaRZj+j3691WJHMIQ9VmGmXJy6LIuJMKUdBJoTnkuGE68G M4W8NivFG/JLQ611tCh8nse2A64o54ij3sopXCzOzwCuPK2v3JjbYA9pWDKlgfZfkJOc vEbG4xb2839MfxRr34zxeSdv4xq91ft9XDZY0rXJSQJ4Lo4ilOntIFve/J+BI6reUZ5Z 0JSg== Received: by 10.180.78.99 with SMTP id a3mr18876621wix.15.1341256457247; Mon, 02 Jul 2012 12:14:17 -0700 (PDT) Received: from localhost.localdomain (host106-37-dynamic.56-82-r.retail.telecomitalia.it. [82.56.37.106]) by mx.google.com with ESMTPS id d10sm47679028wiy.3.2012.07.02.12.14.15 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 02 Jul 2012 12:14:16 -0700 (PDT) From: Stefano Lattarini Date: Mon, 2 Jul 2012 21:14:08 +0200 Message-Id: <3a8d589829e0d5d438f2b55fc5199a83e38f1d0e.1341256385.git.stefano.lattarini@gmail.com> X-Mailer: git-send-email 1.7.9.5 X-Spam-Score: -2.6 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) Related to automake bug#11287. * contrib/t/check-html.am (.log.html): Here, passing $(AM_RST2HTMLFLAGS) and $(RST2HTMLFLAGS) (in that order). Signed-off-by: Stefano Lattarini --- contrib/check-html.am | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/check-html.am b/contrib/check-html.am index 47e4f47..36a5dbe 100644 --- a/contrib/check-html.am +++ b/contrib/check-html.am @@ -38,7 +38,8 @@ mostlyclean-check-html: echo "cannot find rst2html, cannot create $@" >&2; \ exit 2; \ done; \ - $$r2h $< >$@-t && mv -f $@-t $@ + $$r2h $(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS) $< >$@-t \ + && mv -f $@-t $@ # Be sure to run check first, and then to convert the result. # Beware of concurrent executions. Run "check" not "check-TESTS", as -- 1.7.9.5 From unknown Mon Aug 18 17:53:45 2025 X-Loop: help-debbugs@gnu.org Subject: bug#11287: [PATCH] contrib: simple improvements to check-html References: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> In-Reply-To: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> Resent-From: Stefano Lattarini Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-automake@gnu.org Resent-Date: Mon, 02 Jul 2012 19:20:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11287 X-GNU-PR-Package: automake X-GNU-PR-Keywords: To: 11287@debbugs.gnu.org Cc: Stefano Lattarini Received: via spool by 11287-submit@debbugs.gnu.org id=B11287.134125677522131 (code B ref 11287); Mon, 02 Jul 2012 19:20:01 +0000 Received: (at 11287) by debbugs.gnu.org; 2 Jul 2012 19:19:35 +0000 Received: from localhost ([127.0.0.1]:43232 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Slm9m-0005kt-Vd for submit@debbugs.gnu.org; Mon, 02 Jul 2012 15:19:35 -0400 Received: from mail-wg0-f46.google.com ([74.125.82.46]:41618) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Slm9k-0005kl-Ty for 11287@debbugs.gnu.org; Mon, 02 Jul 2012 15:19:33 -0400 Received: by wgbdq10 with SMTP id dq10so5238560wgb.15 for <11287@debbugs.gnu.org>; Mon, 02 Jul 2012 12:14:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer; bh=etlc/WL66StaIaTy37RJB3q9VaZcnH7ptB1NRDNzrT4=; b=ILglmSJ7yf6FBxaAGA+oFISltye1FmGib1p8WoxUcVyc2IqeeA8zKO+SX+8T/MqOyl JTftytBptA4br1v8+ieHjmEorOTau1RdgoOxoeW60gLeQzlpe0YJaWDR1B3MO5ST2w3l X8saufFmJM2EHNJhEic3WXtZQL1lnKrxyeJUTv9Wb1W0b2COkqIok4gsHfTm9ZIOHPb9 5ND43mCLe+j70Jq6e3/osR+jh27R3WeugUZAnZWnmgeNGMzAgYCiwiVRGGZ+hcMzSqkm ZBDak0F3QkaDrP39JhlVjEx2LniqQz3z7w6Bm8jWAxHT3nZYr9rI7qu6O+JKaTWQGDfa IMVw== Received: by 10.216.133.130 with SMTP id q2mr2491375wei.131.1341256495365; Mon, 02 Jul 2012 12:14:55 -0700 (PDT) Received: from localhost.localdomain (host106-37-dynamic.56-82-r.retail.telecomitalia.it. [82.56.37.106]) by mx.google.com with ESMTPS id ch9sm47671347wib.8.2012.07.02.12.14.53 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 02 Jul 2012 12:14:54 -0700 (PDT) From: Stefano Lattarini Date: Mon, 2 Jul 2012 21:14:47 +0200 Message-Id: <18dac9077caa49f69aa39e306b146ae700e14e9c.1341256468.git.stefano.lattarini@gmail.com> X-Mailer: git-send-email 1.7.9.5 X-Spam-Score: -2.6 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) Related to automake bug#11287. * contrib/t/check-html.am (.log.html): Do not look for $RST2HTML in the environment, looking for $(RST2HTML) should be enough (also, the pre-existing code was broken, because it single-quoted $RST2HTML). Fix the loop-and-search implementation to be more similar to the one in 'contrib/t/parallel-tests-html.sh'. Prefer 'mv -f' over bare 'mv'. Do not break the recipe in Signed-off-by: Stefano Lattarini --- contrib/check-html.am | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/contrib/check-html.am b/contrib/check-html.am index 4495ac0..47e4f47 100644 --- a/contrib/check-html.am +++ b/contrib/check-html.am @@ -29,18 +29,16 @@ mostlyclean-check-html: rm -f $(TEST_SUITE_HTML) .log.html: - @list='$(RST2HTML) $$RST2HTML rst2html rst2html.py'; \ - for r2h in $$list; do \ - if ($$r2h --version) >/dev/null 2>&1; then \ - R2H=$$r2h; \ - fi; \ - done; \ - if test -z "$$R2H"; then \ - echo >&2 "cannot find rst2html, cannot create $@"; \ + @list='$(RST2HTML) rst2html rst2html.py'; \ + while :; do \ + for r2h in $$list; do \ + if ($$r2h --version) >/dev/null 2>&1; then break 2; \ + else :; fi; \ + done; \ + echo "cannot find rst2html, cannot create $@" >&2; \ exit 2; \ - fi; \ - $$R2H $< >$@.tmp - @mv $@.tmp $@ + done; \ + $$r2h $< >$@-t && mv -f $@-t $@ # Be sure to run check first, and then to convert the result. # Beware of concurrent executions. Run "check" not "check-TESTS", as -- 1.7.9.5 From unknown Mon Aug 18 17:53:45 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.428 (Entity 5.428) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Akim Demaille Subject: bug#11287: closed (Re: bug#11287: Various issues with the test suite framework) Message-ID: References: <4FF1F61A.9020403@gmail.com> <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> X-Gnu-PR-Message: they-closed 11287 X-Gnu-PR-Package: automake Reply-To: 11287@debbugs.gnu.org Date: Mon, 02 Jul 2012 19:33:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1341257582-23388-1" This is a multi-part message in MIME format... ------------=_1341257582-23388-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #11287: Various issues with the test suite framework which was filed against the automake package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 11287@debbugs.gnu.org. --=20 11287: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D11287 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1341257582-23388-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 11287-done) by debbugs.gnu.org; 2 Jul 2012 19:32:13 +0000 Received: from localhost ([127.0.0.1]:43272 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SlmM1-00064F-02 for submit@debbugs.gnu.org; Mon, 02 Jul 2012 15:32:13 -0400 Received: from mail-ee0-f44.google.com ([74.125.83.44]:47936) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SlmLy-000647-Bv for 11287-done@debbugs.gnu.org; Mon, 02 Jul 2012 15:32:11 -0400 Received: by eekd4 with SMTP id d4so2245615eek.3 for <11287-done@debbugs.gnu.org>; Mon, 02 Jul 2012 12:27:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=0t8ROFNHXeaHt8e+pKrao6ot8JioueqADzd0dLweBW8=; b=trjGw0xup5HM0Bee7P1qvNL6KFnwI8XjUXBmNfb11vbu8S2fWlCV3/EBVufGvI9/Xa 1cqn8CIH6BTdJlMD0iT9LYcZAF5Uj9E+2WdqztE8cZykcx+iggIOUU5A2Aq+TFOIrzLE GNm0Wf8DzRfUJZWpZ31F3lx2i80lJoJhfk8N0ABNiEG3qTaNtmm0FDUS1wQUUbR8au/M PrDuJrFkbk7jTXrg8Tq9NZH0m8rwi8fYLiKwmepoo1vKXMRYP1vWPUcHNyOu9njH9VQK s+QnnuvQX/3+YqBi1zXSNPcf7rLSSadat7MrH79Oa67HLbLwOfBU74dWO5PfwVRyIb4x 51Kw== Received: by 10.14.47.139 with SMTP id t11mr3562181eeb.155.1341257252515; Mon, 02 Jul 2012 12:27:32 -0700 (PDT) Received: from [82.56.37.106] (host106-37-dynamic.56-82-r.retail.telecomitalia.it. [82.56.37.106]) by mx.google.com with ESMTPS id e48sm37772195eea.12.2012.07.02.12.27.30 (version=SSLv3 cipher=OTHER); Mon, 02 Jul 2012 12:27:31 -0700 (PDT) Message-ID: <4FF1F61A.9020403@gmail.com> Date: Mon, 02 Jul 2012 21:27:22 +0200 From: Stefano Lattarini MIME-Version: 1.0 To: Akim Demaille Subject: Re: bug#11287: Various issues with the test suite framework References: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> In-Reply-To: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Spam-Score: -2.6 (--) X-Debbugs-Envelope-To: 11287-done Cc: 11287-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) Reference: To sum up the actions related to this bug report: On 04/20/2012 01:13 PM, Akim Demaille wrote: > Hi! > > I have seen that check-html will be removed, or rather moved > into the contrib part, but there are a few issues: > > - the target is not declared recursive, so one has to write > the bouncing target herself. > This isn't a big deal anymore, now that the user can define extra recursive targets through the macro AM_EXTRA_RECURSIVE_TARGETS (features introduced in commit v1.12.1-163-g3da1457); the test 'contrib/t/parallel-tests-html-recursive.sh' (introduced by commit v1.12.1-191-g53d4984) should prove this assertion. > - because of that, the "naive" implementation of check-html > that just bounces the right directories will miss the > dependencies that check features (in other words, make > check properly recursively builds before, while check-html > would just go into the typical tests/ directory, and then > bounce to "check", so it completely missed the recursion > in the other directories and their dependencies). > Adding 'check-html' to AM_EXTRA_RECURSIVE_TARGETS should take care of this issue as well. > - RECURSIVE_TARGETS cannot be highjacked by the users, that's > sad. Could a $(USER_RECURSIVE_TARGETS) be added to > RECURSIVE_TARGETS so that we can add our own targets that > traverse the whole package? Actually maybe this should > be an Automake option to add recursive targets, and > then TARGET-am and TARGET-recursive would be generated and > bound to the proper TARGET. > Addressed by commit v1.12.1-163-g3da1457 (with a different design but comparable functionalities); see this thread for more info: > - the .log.html recipe features $$RST2HTML, but in '', so it is > useless. And anyway $(RST2HTML) already does the job. > Agreed. Fixed by commit v1.12.1-192-g18dac90. > - "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing. > Fixed by commit v1.12.1-193-g67abae4. > - the test compiler does not appear as a dependency for > the test logs that use it. > Reporting what I've stated in an earlier reply: This makes sense, since the test compiler is not required to be an in-project script, and not even to be specified as an absolute path -- it can be a command like "sh" or "/usr/bin/env perl". > (This is most useful when RECHECK_LOG =.) > - it would be equally useful to be able to add dependencies > to a category of test drivers. Say, with the documented > example: > > TESTS = foo.pl bar.py baz > TEST_EXTENSIONS = .pl .py > PL_LOG_COMPILER = $(PERL) > AM_PL_LOG_FLAGS = -w > PY_LOG_COMPILER = $(PYTHON) > AM_PY_LOG_FLAGS = -v > LOG_COMPILER = ./wrapper-script > AM_LOG_FLAGS = -d > > be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and > LOG_DEPENDENCIES. I don't think EXTRA dependencies would > be useful. > This would be tricky to implement in the current Automake code base, and IMHO for a too little benefit; so I'm going to label this part of the report as a "wontfix". However, it was easy to implement in Automake-NG: -*-*-*- All that considered, I'm closing this bug report now. Regards, Stefano ------------=_1341257582-23388-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 20 Apr 2012 11:25:53 +0000 Received: from localhost ([127.0.0.1]:43689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SLByK-00062u-QA for submit@debbugs.gnu.org; Fri, 20 Apr 2012 07:25:53 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51084) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SLByI-00062d-DM for submit@debbugs.gnu.org; Fri, 20 Apr 2012 07:25:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SLBxk-0004yh-Dr for submit@debbugs.gnu.org; Fri, 20 Apr 2012 07:25:20 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:44646) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SLBxk-0004e2-Ac for submit@debbugs.gnu.org; Fri, 20 Apr 2012 07:25:16 -0400 Received: from eggs.gnu.org ([208.118.235.92]:43125) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SLBnW-0008JF-QU for bug-automake@gnu.org; Fri, 20 Apr 2012 07:14:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SLBnO-0000dg-W4 for bug-automake@gnu.org; Fri, 20 Apr 2012 07:14:42 -0400 Received: from sao-paulo.lrde.epita.fr ([163.5.55.1]:42255 helo=kualalumpur.lrde.epita.fr) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SLBnO-0000ZP-Pp for bug-automake@gnu.org; Fri, 20 Apr 2012 07:14:34 -0400 Received: from tenochtitlan-eth.lrde.epita.fr ([192.168.101.154]) by kualalumpur.lrde.epita.fr with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.63) (envelope-from ) id 1SLBmq-00013l-Q5 for bug-automake@gnu.org; Fri, 20 Apr 2012 13:14:01 +0200 From: Akim Demaille Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Various issues with the test suite framework Date: Fri, 20 Apr 2012 13:13:58 +0200 Message-Id: <2ED5838A-9BD1-4ED3-AE4A-E88A68C410B8@lrde.epita.fr> To: Automake Bugs Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 208.118.235.17 X-Spam-Score: -6.9 (------) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.9 (------) Hi! I have seen that check-html will be removed, or rather moved into the contrib part, but there are a few issues: - the target is not declared recursive, so one has to write the bouncing target herself. - because of that, the "naive" implementation of check-html that just bounces the right directories will miss the dependencies that check features (in other words, make check properly recursively builds before, while check-html would just go into the typical tests/ directory, and then bounce to "check", so it completely missed the recursion in the other directories and their dependencies). - RECURSIVE_TARGETS cannot be highjacked by the users, that's sad. Could a $(USER_RECURSIVE_TARGETS) be added to RECURSIVE_TARGETS so that we can add our own targets that traverse the whole package? Actually maybe this should be an Automake option to add recursive targets, and then TARGET-am and TARGET-recursive would be generated and bound to the proper TARGET. - the .log.html recipe features $$RST2HTML, but in '', so it is useless. And anyway $(RST2HTML) already does the job. - "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing. - the test compiler does not appear as a dependency for the test logs that use it. This is most useful when RECHECK_LOG =. - it would be equally useful to be able to add dependencies to a category of test drivers. Say, with the documented example: TESTS = foo.pl bar.py baz TEST_EXTENSIONS = .pl .py PL_LOG_COMPILER = $(PERL) AM_PL_LOG_FLAGS = -w PY_LOG_COMPILER = $(PYTHON) AM_PY_LOG_FLAGS = -v LOG_COMPILER = ./wrapper-script AM_LOG_FLAGS = -d be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and LOG_DEPENDENCIES. I don't think EXTRA dependencies would be useful. Cheers! ------------=_1341257582-23388-1--