GNU bug report logs - #11897
automake 1.12.2 test failure on Mac OS X 10.7

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 11897 in the body.
You can then email your comments to 11897 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Max Horn <max <at> quendi.de>
To: bug-automake <at> gnu.org
Subject: automake 1.12.2 test failure on Mac OS X 10.7
Date: Tue, 10 Jul 2012 13:44:06 +0200

Hi there,

here is another test failure I am seeing on Mac OS X 10.7.4 with automake 1.12.2. Specifically, two tap test fail with errors. So when I execute

   make check TESTS="t/suffix8.tap t/suffix10.tap"

then I get this (after removing some fluff):


PASS: t/suffix8.tap 1 - libtoolize
PASS: t/suffix8.tap 2 - aclocal
PASS: t/suffix8.tap 3 - autoconf
PASS: t/suffix8.tap 4 - automake
PASS: t/suffix8.tap 5
ERROR: t/suffix8.tap 5 - configure # OUT-OF-ORDER (expecting 6)
ERROR: t/suffix8.tap 6 - make test0 # OUT-OF-ORDER (expecting 7)
ERROR: t/suffix8.tap 7 - make test1 # OUT-OF-ORDER (expecting 8)
ERROR: t/suffix8.tap 8 - make test2 # OUT-OF-ORDER (expecting 9)
ERROR: t/suffix8.tap 9 - make all # OUT-OF-ORDER (expecting 10)
ERROR: t/suffix8.tap 11 # UNPLANNED
ERROR: t/suffix8.tap 10 - make distcheck # UNPLANNED
ERROR: t/suffix8.tap - too many tests run (expected 10, got 12)
PASS: t/suffix10.tap 1 - libtoolize
PASS: t/suffix10.tap 2 - aclocal
PASS: t/suffix10.tap 3 - autoconf
PASS: t/suffix10.tap 4 - automake
PASS: t/suffix10.tap 5
ERROR: t/suffix10.tap 5 - configure # OUT-OF-ORDER (expecting 6)
ERROR: t/suffix10.tap 6 - make test # OUT-OF-ORDER (expecting 7)
ERROR: t/suffix10.tap 7 - make all # UNPLANNED
ERROR: t/suffix10.tap - too many tests run (expected 7, got 8)


I think the problem is caused by the "configure" script spitting out these messages:

checking for archiver @FILE support... rm: cannot remove `conftest.dSYM': Is a directory
no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm output from cc object... rm: cannot remove `conftest.dSYM': Is a directory
ok

Note the isolated "ok" on a line of its own; apparently this makes the test think that an unplanned test step occurred.


This in turn is caused by a "bug" in libtool -- or rather, there is a peculiar behavior of the Mac OS X compilers (namely to create conftest.dSYM directories under certain circumstances), and libtool hence needs to replace "rm -f conftest*" by "rm -rf conftest*". This was done some years ago, but apparently was forgotten in three cases.

I already reported this to libtool: <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11895>
But since it directly affects the test suite of automake, I thought it best to report this here, too. Perhaps you would like to workaround it by making the test a bit more strict about how it detects a completed test step?


Cheers,
Max

Message #8 received at 11897 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Max Horn <max <at> quendi.de>
Cc: 11897 <at> debbugs.gnu.org,
	"automake-patches <at> gnu.org" <automake-patches <at> gnu.org>
Subject: Re: bug#11897: automake 1.12.2 test failure on Mac OS X 10.7
Date: Tue, 10 Jul 2012 15:31:47 +0200

[Message part 1 (text/plain, inline)]

tags 11897 + patch
severity 11897 minor
thanks

On 07/10/2012 01:44 PM, Max Horn wrote:
> Hi there,
>
Hi Max, thanks for the report.

> here is another test failure I am seeing on Mac OS X 10.7.4 with
> automake 1.12.2. Specifically, two tap test fail with errors.
> So when I execute
> 
>    make check TESTS="t/suffix8.tap t/suffix10.tap"
> 
> then I get this (after removing some fluff):
> 
> 
> PASS: t/suffix8.tap 1 - libtoolize
> PASS: t/suffix8.tap 2 - aclocal
> PASS: t/suffix8.tap 3 - autoconf
> PASS: t/suffix8.tap 4 - automake
> PASS: t/suffix8.tap 5
> ERROR: t/suffix8.tap 5 - configure # OUT-OF-ORDER (expecting 6)
> ERROR: t/suffix8.tap 6 - make test0 # OUT-OF-ORDER (expecting 7)
> ERROR: t/suffix8.tap 7 - make test1 # OUT-OF-ORDER (expecting 8)
> ERROR: t/suffix8.tap 8 - make test2 # OUT-OF-ORDER (expecting 9)
> ERROR: t/suffix8.tap 9 - make all # OUT-OF-ORDER (expecting 10)
> ERROR: t/suffix8.tap 11 # UNPLANNED
> ERROR: t/suffix8.tap 10 - make distcheck # UNPLANNED
> ERROR: t/suffix8.tap - too many tests run (expected 10, got 12)
> PASS: t/suffix10.tap 1 - libtoolize
> PASS: t/suffix10.tap 2 - aclocal
> PASS: t/suffix10.tap 3 - autoconf
> PASS: t/suffix10.tap 4 - automake
> PASS: t/suffix10.tap 5
> ERROR: t/suffix10.tap 5 - configure # OUT-OF-ORDER (expecting 6)
> ERROR: t/suffix10.tap 6 - make test # OUT-OF-ORDER (expecting 7)
> ERROR: t/suffix10.tap 7 - make all # UNPLANNED
> ERROR: t/suffix10.tap - too many tests run (expected 7, got 8)
> 
> 
> I think the problem is caused by the "configure" script spitting
> out these messages:
> 
> checking for archiver @FILE support... rm: cannot remove `conftest.dSYM': Is a directory
> no
> checking for strip... strip
> checking for ranlib... ranlib
> checking command to parse /usr/bin/nm output from cc object... rm: cannot remove `conftest.dSYM': Is a directory
> ok
> 
> Note the isolated "ok" on a line of its own; apparently this
> makes the test think that an unplanned test step occurred.
>
Your interpretation is absolutely correct.  The attached patch should
take care of the issue.  Could you give it a try?

> This in turn is caused by a "bug" in libtool -- or rather, there is
> a peculiar behavior of the Mac OS X compilers (namely to create
> conftest.dSYM directories under certain circumstances), and libtool
> hence needs to replace "rm -f conftest*" by "rm -rf conftest*". This
> was done some years ago, but apparently was forgotten in three cases.
> 
> I already reported this to libtool:
> <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11895>
>

> But since it directly affects the test suite of automake, I thought
> it best to report this here, too.
>
Well done.  This is a spurious failure that shouldn't interfere with
our test results.

> Perhaps you would like to workaround it by making the test a bit
> more strict about how it detects a completed test step?
>
This is not possible; the '*.tap' tests in the Automake testsuite
use the TAP protocol, and that mandates which output lines must be
interpreted as declaration of test results.  I think the workaround
implemented in the attached patch is the simplest and best fix.

Thanks,
  Stefano

[0001-tests-avoid-spurious-TAP-errors-on-Mac-OS-X-10.7.patch (text/x-diff, attachment)]

Message #15 received at 11897 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Max Horn <postbox <at> quendi.de>
Cc: 11897 <at> debbugs.gnu.org,
	"automake-patches <at> gnu.org" <automake-patches <at> gnu.org>
Subject: Re: bug#11897: automake 1.12.2 test failure on Mac OS X 10.7
Date: Tue, 10 Jul 2012 16:06:37 +0200

[Message part 1 (text/plain, inline)]

On 07/10/2012 03:56 PM, Max Horn wrote:
>
> This fixes most of the errors, except for
> 
> PASS: t/suffix8.tap 10
> ERROR: t/suffix8.tap 10 - make distcheck # UNPLANNED
> ERROR: t/suffix8.tap - too many tests run (expected 10, got 11)
> 
> I looked at the log, this is again the same issue, configure
> is run once again
>
Of course, by "make distcheck"! *faceplam*  I should have been more
careful.

> and once again suffers from the same problem.
> 
Does the attached patch fares better?

Thanks,
  Stefano

[0001-tests-avoid-spurious-TAP-errors-on-Mac-OS-X-10.7.patch (text/x-diff, attachment)]

Message #20 received at 11897-done <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Max Horn <postbox <at> quendi.de>
Cc: 11897-done <at> debbugs.gnu.org,
	"automake-patches <at> gnu.org" <automake-patches <at> gnu.org>
Subject: Re: bug#11897: automake 1.12.2 test failure on Mac OS X 10.7
Date: Tue, 10 Jul 2012 16:32:39 +0200

On 07/10/2012 04:30 PM, Max Horn wrote:
> 
> On 10.07.2012, at 16:06, Stefano Lattarini wrote:
> 
>> On 07/10/2012 03:56 PM, Max Horn wrote:
>>>
>>> This fixes most of the errors, except for
>>>
>>> PASS: t/suffix8.tap 10
>>> ERROR: t/suffix8.tap 10 - make distcheck # UNPLANNED
>>> ERROR: t/suffix8.tap - too many tests run (expected 10, got 11)
>>>
>>> I looked at the log, this is again the same issue, configure
>>> is run once again
>>>
>> Of course, by "make distcheck"! *faceplam*  I should have been more
>> careful.
>>
>>> and once again suffers from the same problem.
>>>
>> Does the attached patch fares better?
> 
> Works perfectly, thanks!
> 
Thanks for confirming.  I've pushed it, and I'm closing this bug
report.

Best regards,
  Stefano

Message #23 received at 11897 <at> debbugs.gnu.org (full text, mbox):

From: Max Horn <postbox <at> quendi.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 11897 <at> debbugs.gnu.org,
	"automake-patches <at> gnu.org" <automake-patches <at> gnu.org>
Subject: Re: bug#11897: automake 1.12.2 test failure on Mac OS X 10.7
Date: Tue, 10 Jul 2012 15:56:44 +0200

Dear Stefano,

On 10.07.2012, at 15:31, Stefano Lattarini wrote:

> tags 11897 + patch
> severity 11897 minor
> thanks
> 
> On 07/10/2012 01:44 PM, Max Horn wrote:
>> Hi there,
>> 
> Hi Max, thanks for the report.

Sure, thank you for your prompt reply and patch!

[...]

>> I think the problem is caused by the "configure" script spitting
>> out these messages:
>> 
>> checking for archiver @FILE support... rm: cannot remove `conftest.dSYM': Is a directory
>> no
>> checking for strip... strip
>> checking for ranlib... ranlib
>> checking command to parse /usr/bin/nm output from cc object... rm: cannot remove `conftest.dSYM': Is a directory
>> ok
>> 
>> Note the isolated "ok" on a line of its own; apparently this
>> makes the test think that an unplanned test step occurred.
>> 
> Your interpretation is absolutely correct.  The attached patch should
> take care of the issue.  Could you give it a try?

This fixes most of the errors, except for

PASS: t/suffix8.tap 10
ERROR: t/suffix8.tap 10 - make distcheck # UNPLANNED
ERROR: t/suffix8.tap - too many tests run (expected 10, got 11)

I looked at the log, this is again the same issue, configure is run once again and once again suffers from the same problem.



Cheers,
Max

Message #26 received at 11897 <at> debbugs.gnu.org (full text, mbox):

From: Max Horn <postbox <at> quendi.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 11897 <at> debbugs.gnu.org,
	"automake-patches <at> gnu.org" <automake-patches <at> gnu.org>
Subject: Re: bug#11897: automake 1.12.2 test failure on Mac OS X 10.7
Date: Tue, 10 Jul 2012 16:30:16 +0200

On 10.07.2012, at 16:06, Stefano Lattarini wrote:

> On 07/10/2012 03:56 PM, Max Horn wrote:
>> 
>> This fixes most of the errors, except for
>> 
>> PASS: t/suffix8.tap 10
>> ERROR: t/suffix8.tap 10 - make distcheck # UNPLANNED
>> ERROR: t/suffix8.tap - too many tests run (expected 10, got 11)
>> 
>> I looked at the log, this is again the same issue, configure
>> is run once again
>> 
> Of course, by "make distcheck"! *faceplam*  I should have been more
> careful.
> 
>> and once again suffers from the same problem.
>> 
> Does the attached patch fares better?

Works perfectly, thanks!

Max

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.