Subject: Automake patches for custom test drivers' support break coreutils testsuite (was: Re: [PATCH v4 1/3] parallel-tests: add auxiliary script 'pt-driver', refactor)
From: Stefano Lattarini <[email-masked]>
To: "Ralf Wildenhues" <[email-masked]>
Cc: [email-masked], [email-masked]
Date: Fri, 17 Jun 2011 23:09:22 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

[Adding bug-coreutils]

Reference:
 <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00093.html>

On Friday 17 June 2011, Ralf Wildenhues wrote:
> I generally like the direction this is taking.  The point of best
> separation between which code goes into Makefile.in and which into
> the driver scripts can be fine-tuned when we have more than one such
> script.
> 
> Actually, yes, before deciding on this for real I really do want to see
> a nontrivial other driver script.  There is no point in hardcoding
> too much in several driver scripts if it all needs to be the same
> anyway.
> 
> Please measure the time overhead your changes introduce into the current
> code, for a trivial testsuite (say, 50 tests running 'true'), and a
> nontrivial one like Automake's and one with faster tests.
>
I've tried the coreutils testsuite and ... Ouch!  That gets broken
by my patches :-(

That's due to the overly complex TESTS_ENVIRONMENT employed by
conreutils' tests/Makefile.am:

 TESTS_ENVIRONMENT = \
  . $(srcdir)/lang-default;                     \
  tmp__=$${TMPDIR-/tmp};                        \
  test -d "$$tmp__" && test -w "$$tmp__" || tmp__=.;    \
  . $(srcdir)/envvar-check;                     \
  TMPDIR=$$tmp__; export TMPDIR;                \
  shell_or_perl_() {                            \
    if grep '^\#!/usr/bin/perl' "$$1" > /dev/null; then                 \
      if $(PERL) -e 'use warnings' > /dev/null 2>&1; then               \
        grep '^\#!/usr/bin/perl -T' "$$1" > /dev/null && T_=T || T_=;   \
        $(PERL) -w$$T_ -I$(srcdir) -MCoreutils -MCuSkip                 \
              -M"CuTmpdir qw($$f)" -- "$$1";    \
      else                                      \
        echo 1>&2 "$$tst: configure did not find a usable version of Perl," \
          "so skipping this test";              \
        (exit 77);                              \
      fi;                                       \
    else                                        \
      $(SHELL) "$$1";                           \
    fi;                                         \
  };                                            \
  export                                        \
  VERSION='$(VERSION)'                          \
  LOCALE_FR='$(LOCALE_FR)'                      \
  LOCALE_FR_UTF8='$(LOCALE_FR_UTF8)'            \
  abs_top_builddir='$(abs_top_builddir)'        \
  abs_top_srcdir='$(abs_top_srcdir)'            \
  abs_srcdir='$(abs_srcdir)'                    \
  built_programs="`cat .built-programs`"        \
  host_os=$(host_os)                            \
  host_triplet='$(host_triplet)'                \
  srcdir='$(srcdir)'                            \
  top_srcdir='$(top_srcdir)'                    \
  CONFIG_HEADER='$(abs_top_builddir)/$(CONFIG_INCLUDE)' \
  CU_TEST_NAME=`basename '$(abs_srcdir)'`,`echo $$tst|sed 's,^\./,,;s,/,-,g'` \
  CC='$(CC)'                                    \
  AWK='$(AWK)'                                  \
  EGREP='$(EGREP)'                              \
  EXEEXT='$(EXEEXT)'                            \
  MAKE=$(MAKE)                                  \
  PACKAGE_BUGREPORT='$(PACKAGE_BUGREPORT)'      \
  PACKAGE_VERSION=$(PACKAGE_VERSION)            \
  PERL='$(PERL)'                                \
  PREFERABLY_POSIX_SHELL='$(PREFERABLY_POSIX_SHELL)' \
  REPLACE_GETCWD=$(REPLACE_GETCWD)              \
  ; test -d /usr/xpg4/bin && PATH='/usr/xpg4/bin$(PATH_SEPARATOR)'"$$PATH"; \
  PATH='$(abs_top_builddir)/src$(PATH_SEPARATOR)'"$$PATH" \
  ; shell_or_perl_ 9>&2

In order to work with the upcoming new Automake testsuite harness, coreutils
have two possibilities:
 1. move the `shell_or_perl_' subroutine's functionality into a real acript,
    and define the LOG_COMPILER to point to it; or
 2. add a `.pl' extension to the perl test scripts, and define PL_LOG_COMPILER
    appropriately (might be a little tricky, considering the hops that the
    `shell_or_perl_' subroutine goes through in order to get the flags and
    imports right).

I should have have an FSF copyright assignement in place for coreutils too,
so I can volounteer to write a fix for this situation, if no one wants to
beat me.

Regards,
  Stefano