GNU bug report logs - #13771
GNU make specific stuff cannot be used in RHS of "TESTS ="

Previous Next

To reply to this bug, email your comments to 13771 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Daiki Ueno <ueno <at> gnu.org>
To: bug-automake <at> gnu.org
Subject: make functions cannot be used in RHS of "TESTS ="
Date: Wed, 20 Feb 2013 18:27:13 +0900

Hi,

I noticed that some modules are using make functions to assign values to
TESTS.  For example, I found the following in my GNOME jhbuild checkout:

TESTS = $(filter-out fake-tp-backend individual-zeitgeist,$(noinst_PROGRAMS))
TESTS = $(sort $(tests))

These cause warnings like:

Makefile.am:2: warning: sort $(tests: non-POSIX variable name
Makefile.am:2: (probably a GNU make extension)

and also generate the following rule in Makefile.in, which seems
apparently wrong:

$(sort.log: $(sort
        @p='$(sort'; \
        b='$(sort'; \
        $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \
        --log-file $$b.log --trs-file $$b.trs \
        $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \
        "$$tst" $(AM_TESTS_FD_REDIRECT)

Shall it be fixed in Makefile.am or Automake?

Regards,
-- 
Daiki Ueno

Message #8 received at 13771 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Daiki Ueno <ueno <at> gnu.org>
Cc: 13771 <at> debbugs.gnu.org
Subject: Re: bug#13771: make functions cannot be used in RHS of "TESTS ="
Date: Tue, 05 Mar 2013 16:20:56 +0100

severity 13771 wishlist
stop

Hi Daiki, sorry for the delay.

On 02/20/2013 10:27 AM, Daiki Ueno wrote:
> Hi,
> 
> I noticed that some modules are using make functions to assign values to
> TESTS.  For example, I found the following in my GNOME jhbuild checkout:
> 
> TESTS = $(filter-out fake-tp-backend individual-zeitgeist,$(noinst_PROGRAMS))
> TESTS = $(sort $(tests))
> 
> These cause warnings like:
> 
> Makefile.am:2: warning: sort $(tests: non-POSIX variable name
> Makefile.am:2: (probably a GNU make extension)
> 
> and also generate the following rule in Makefile.in, which seems
> apparently wrong:
> 
> $(sort.log: $(sort
>         @p='$(sort'; \
>         b='$(sort'; \
>         $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \
>         --log-file $$b.log --trs-file $$b.trs \
>         $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \
>         "$$tst" $(AM_TESTS_FD_REDIRECT)
> 
> Shall it be fixed in Makefile.am or Automake?
>
This is actually a long-standing (and probably not easily lifted) Automake
limitation, due to the need to perform $(EXEEXT) appending to entries in
the $(PROGRAMS) primary (and consequently in the $(TESTS) variable too,
since that can contain programs).  I won't object to a patch that tries to
mitigate the issue, but given how old this limitation is, I won't attempt
it myself anytime soon.

In the meantime, I see two solutions:

   * Don't use make functions in the TESTS definition; this might not be
     trivial in general situations, but seems pretty easy in your setup
     above -- instead of having:

         noinst_PROGRAMS = foo bar qux zap
         TESTS = $(filter-out foo bar,$(noinst_PROGRAMS))

     use:

         test_progs = foo bar
         noinst_PROGRAMS = $(test_progs) qux zap
         TESTS = $(test_progs)

    And no need to $(sort) the TESTS -- since you are using the parallel
    driver, they should be able to run in any order anyway.

  * If you only care for the package to build for UNIX (and not for Cygwin
    or MinGW), you can do away with the $(EXEEXT) rewrite machinery by
    using the 'no-exeext' Automake option.

HTH,
  Stefano

Message #13 received at 13771 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Daiki Ueno <ueno <at> gnu.org>
Cc: 13771 <at> debbugs.gnu.org
Subject: Re: bug#13771: make functions cannot be used in RHS of "TESTS ="
Date: Sat, 08 Jun 2013 11:54:00 +0200

Reference:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13771#8>

Resurrecting an old thread to correct some wrong information
(given by myself, sorry about that).

On 03/05/2013 04:20 PM, Stefano Lattarini wrote:
>
> Hi Daiki, sorry for the delay.
> 
> On 02/20/2013 10:27 AM, Daiki Ueno wrote:
>> Hi,
>>
>> I noticed that some modules are using make functions to assign values to
>> TESTS.  For example, I found the following in my GNOME jhbuild checkout:
>>
>> TESTS = $(filter-out fake-tp-backend individual-zeitgeist,$(noinst_PROGRAMS))
>> TESTS = $(sort $(tests))
>>
>> These cause warnings like:
>>
>> Makefile.am:2: warning: sort $(tests: non-POSIX variable name
>> Makefile.am:2: (probably a GNU make extension)
>>
>> and also generate the following rule in Makefile.in, which seems
>> apparently wrong:
>>
>> $(sort.log: $(sort
>>         @p='$(sort'; \
>>         b='$(sort'; \
>>         $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \
>>         --log-file $$b.log --trs-file $$b.trs \
>>         $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \
>>         "$$tst" $(AM_TESTS_FD_REDIRECT)
>>
>> Shall it be fixed in Makefile.am or Automake?
>>
> This is actually a long-standing (and probably not easily lifted) Automake
> limitation, due to the need to perform $(EXEEXT) appending to entries in
> the $(PROGRAMS) primary (and consequently in the $(TESTS) variable too,
> since that can contain programs).
>
Actually, this automake-time processing is required also to be able
to correctly handle extension-less tests; so ...

>  I won't object to a patch that tries to
> mitigate the issue, but given how old this limitation is, I won't attempt
> it myself anytime soon.
> 
> In the meantime, I see two solutions:
> 
>    * Don't use make functions in the TESTS definition; this might not be
>      trivial in general situations, but seems pretty easy in your setup
>      above -- instead of having:
> 
>          noinst_PROGRAMS = foo bar qux zap
>          TESTS = $(filter-out foo bar,$(noinst_PROGRAMS))
> 
>      use:
> 
>          test_progs = foo bar
>          noinst_PROGRAMS = $(test_progs) qux zap
>          TESTS = $(test_progs)
> 
>     And no need to $(sort) the TESTS -- since you are using the parallel
>     driver, they should be able to run in any order anyway.
> 
>   * If you only care for the package to build for UNIX (and not for Cygwin
>     or MinGW), you can do away with the $(EXEEXT) rewrite machinery by
>     using the 'no-exeext' Automake option.
> 
... this proposed work-around won't work :-(

Note that this has come up again recently in automake bug#14561.

There we gave a a possible and quite generic workaround: wrap any
GNU make specific stuff that you want to use in $(TESTS) into a
configure substitution, which will be left alone by automake; e.g.:

  # In configure.ac:
  AC_SUBST([auto_find_tests], ['$(sort $(wildcard *.test))'])

  # In Makefile.am:
  TESTS = @auto_find_tests@

But alas, in order to have this to work correctly, one should only
use tests whose suffix is listed in TESTS_EXTENSIONS (if that
variable is not explicitly defined, it's default value is ".test").
That's because, missing such extension, automake cannot generate
generic suffix rules to create the '.trs' and '.log' files the
testsuite harness relies upon.

So even this workaround wouldn't work in the particular use case
reported by Daiki :-(  Sorry.

Best regards,
  Stefano

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.