Subject: automake 1.12.2 test failure on Mac OS X 10.7
From: Max Horn <[email-masked]>
To: [email-masked]
Date: Tue, 10 Jul 2012 13:44:06 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi there,

here is another test failure I am seeing on Mac OS X 10.7.4 with automake 1.12.2. Specifically, two tap test fail with errors. So when I execute

   make check TESTS="t/suffix8.tap t/suffix10.tap"

then I get this (after removing some fluff):

PASS: t/suffix8.tap 1 - libtoolize
PASS: t/suffix8.tap 2 - aclocal
PASS: t/suffix8.tap 3 - autoconf
PASS: t/suffix8.tap 4 - automake
PASS: t/suffix8.tap 5
ERROR: t/suffix8.tap 5 - configure # OUT-OF-ORDER (expecting 6)
ERROR: t/suffix8.tap 6 - make test0 # OUT-OF-ORDER (expecting 7)
ERROR: t/suffix8.tap 7 - make test1 # OUT-OF-ORDER (expecting 8)
ERROR: t/suffix8.tap 8 - make test2 # OUT-OF-ORDER (expecting 9)
ERROR: t/suffix8.tap 9 - make all # OUT-OF-ORDER (expecting 10)
ERROR: t/suffix8.tap 11 # UNPLANNED
ERROR: t/suffix8.tap 10 - make distcheck # UNPLANNED
ERROR: t/suffix8.tap - too many tests run (expected 10, got 12)
PASS: t/suffix10.tap 1 - libtoolize
PASS: t/suffix10.tap 2 - aclocal
PASS: t/suffix10.tap 3 - autoconf
PASS: t/suffix10.tap 4 - automake
PASS: t/suffix10.tap 5
ERROR: t/suffix10.tap 5 - configure # OUT-OF-ORDER (expecting 6)
ERROR: t/suffix10.tap 6 - make test # OUT-OF-ORDER (expecting 7)
ERROR: t/suffix10.tap 7 - make all # UNPLANNED
ERROR: t/suffix10.tap - too many tests run (expected 7, got 8)

I think the problem is caused by the "configure" script spitting out these messages:

checking for archiver @FILE support... rm: cannot remove `conftest.dSYM': Is a directory
no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm output from cc object... rm: cannot remove `conftest.dSYM': Is a directory
ok

Note the isolated "ok" on a line of its own; apparently this makes the test think that an unplanned test step occurred.

This in turn is caused by a "bug" in libtool -- or rather, there is a peculiar behavior of the Mac OS X compilers (namely to create conftest.dSYM directories under certain circumstances), and libtool hence needs to replace "rm -f conftest*" by "rm -rf conftest*". This was done some years ago, but apparently was forgotten in three cases.

I already reported this to libtool: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11895>
But since it directly affects the test suite of automake, I thought it best to report this here, too. Perhaps you would like to workaround it by making the test a bit more strict about how it detects a completed test step?

Cheers,
Max