GNU bug report logs - #68119
[GNU automake 1.16.5] test-suite 45 tests failed

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 68119 in the body.
You can then email your comments to 68119 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Erik A Johnson <JohnsonE <at> usc.edu>
To: "bug-automake <at> gnu.org" <bug-automake <at> gnu.org>
Subject: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Fri, 29 Dec 2023 23:12:53 +0000

[Message part 1 (text/plain, inline)]

Clean install of macOS Sonoma 14.2.1 on MacBook Pro M3 max

[test-suite.log (application/octet-stream, attachment)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu
Cc: bug-automake <at> gnu.org
Subject: Re: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Sat, 30 Dec 2023 15:56:25 -0700

    Subject: [GNU automake 1.16.5] test-suite 45 tests failed
    Clean install of macOS Sonoma 14.2.1 on MacBook Pro M3 max

Thanks for the report, but it seems like something in the environment is
causing trouble. For example, one of the failing tests is
dist-vs-built-sources.sh:

    gcc -DPACKAGE_NAME=\"dist-vs-built-sources\" -DPACKAGE_TARNAME=\"dist-vs-built-sources\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"dist-vs-built-sources\ 1.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"dist-vs-built-sources\" -DVERSION=\"1.0\" -I.     -g -O2 -MT foo.o -MD -MP -MF .deps/foo.Tpo -c -o foo.o foo.c
    foo.c:2:19: error: call to undeclared library function 'printf' with type 'int (const char *, ...)'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    int main (void) { printf ("%s\n", F); return 0; }
                      ^
    foo.c:2:19: note: include the header <stdio.h> or explicitly provide a declaration for 'printf'

However, <stdio.h> is already included. The test program is these three
lines (h.h is a trivial #define):
#include <stdio.h>
#include "h.h"
int main (void) { printf ("%s\n", F); return 0; }

So the failure doesn't make sense to me.  I have no access to Sonoma or
other way to debug, unfortunately.

The other failures also look like they relate to the test suite programs
in this particular environment, rather than actual Automake
functionality.

Maybe someone else here has an idea or a way to proceed. --thanks, karl.

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu
Cc: bug-automake <at> gnu.org
Subject: Re: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Sun, 31 Dec 2023 16:18:03 -0700

Hi Erik - thanks for the follow-up. Now I understand. You were testing
automake-1.16.5. But you sent it so soon after we released a pretest, I
didn't notice, and thought you were testing the pretest :).

These test failures (among lots of other things) are supposedly fixed in
the current sources. Could you easily test
https://alpha.gnu.org/gnu/automake/automake-1.16j.tar.xz
?

No need to install it, just ./configure && make && make -j8 check;
for whatever value of "8" you like :). --thanks, karl.

Message #18 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu, 68119 <at> debbugs.gnu.org
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Mon, 1 Jan 2024 16:37:29 -0700

    92 tests fail on 1.16j

Thanks for trying, though a sad result. I have a hunch that a lot of the
failures are timing issues. Although Automake thinks your system
"should" work with a 1-millisecond sleep at various points (I gather
you're already using autoconf-2.72), maybe there are problems in
practice.

If you're up for it, please try setting this variable in the environment
  am_cv_sleep_fractional_seconds=false
  export am_cv_sleep_fractional_seconds
(Bourne-compatible shell syntax, if you're using a C shell, I assume you
know what to do. :)

And then make distclean (to remove the leftovers) and the same configure
&& etc. That's my only immediate idea ... --thanks again, karl.

Message #21 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Erik A Johnson <JohnsonE <at> usc.edu>
To: Karl Berry <karl <at> freefriends.org>
Cc: "68119 <at> debbugs.gnu.org" <68119 <at> debbugs.gnu.org>
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Tue, 2 Jan 2024 23:06:35 +0000

[Message part 1 (text/plain, inline)]

Thanks, Karl.  Yes, using autoconf-2.72.

Using am_cv_sleep_fractional_seconds=false dropped it to 7-11 tests failing.  Strangely (to me), it isn't consistent: I've run this 8 times, each in a new fresh download of 1.16j.  The following 7 tests always fail:

* py-compile-basedir -- it fails because I installed Python 3.12, which has removed the "imp" module in favor of the "importlib" module ("imp" was deprecated starting in Python 3.4).

* yacc-bison-skeleton, yacc-d-basic, yacc-deleted-headers, yacc-depend, yacc-depend2, yacc-mix-c-cxx -- these all fail not finding the yyparse function.  Some of the other yacc-.* tests seem to have yyparse in them so it's not immediately clear to me why these 6 consistently fail.

The following failed 3 of 8 times (tests #1 & #3 & (#6 or #8))

* aclocal-deleted-header
* aclocal-deleted-header-aclocal-amflags

The following failed 2 of 8 times (tests #1 & #3)

* aclocal-deps

The following failed on 1 of 8 times (test #1)

* subdir-add2-pr46

Log files for the 11 failed tests are attached, along with a gzip'd test-suite.log

Erik






> On Jan 1, 2024, at 3:37 PM, Karl Berry <karl <at> freefriends.org> wrote:
>
>    92 tests fail on 1.16j
>
> Thanks for trying, though a sad result. I have a hunch that a lot of the
> failures are timing issues. Although Automake thinks your system
> "should" work with a 1-millisecond sleep at various points (I gather
> you're already using autoconf-2.72), maybe there are problems in
> practice.
>
> If you're up for it, please try setting this variable in the environment
>  am_cv_sleep_fractional_seconds=false
>  export am_cv_sleep_fractional_seconds
> (Bourne-compatible shell syntax, if you're using a C shell, I assume you
> know what to do. :)
>
> And then make distclean (to remove the leftovers) and the same configure
> && etc. That's my only immediate idea ... --thanks again, karl.

[Message part 2 (text/html, inline)]

[aclocal-deleted-header.log (application/octet-stream, attachment)]

[aclocal-deleted-header-aclocal-amflags.log (application/octet-stream, attachment)]

[aclocal-deps.log (application/octet-stream, attachment)]

[py-compile-basedir.log (application/octet-stream, attachment)]

[subdir-add2-pr46.log (application/octet-stream, attachment)]

[yacc-bison-skeleton.log (application/octet-stream, attachment)]

[yacc-d-basic.log (application/octet-stream, attachment)]

[yacc-deleted-headers.log (application/octet-stream, attachment)]

[yacc-depend.log (application/octet-stream, attachment)]

[yacc-depend2.log (application/octet-stream, attachment)]

[yacc-mix-c-cxx.log (application/octet-stream, attachment)]

[test-suite.log.gz (application/x-gzip, attachment)]

Message #24 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu
Cc: 68119 <at> debbugs.gnu.org
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Wed, 3 Jan 2024 16:07:44 -0700

    Using am_cv_sleep_fractional_seconds=false dropped it to 7-11 tests failing.

Erik, thanks for all the trials and detailed analysis.
Nothing is immediately clear to me either :(, but I'll take a look. -k

Message #27 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Erik A Johnson <JohnsonE <at> usc.edu>
To: Karl Berry <karl <at> freefriends.org>
Cc: "68119 <at> debbugs.gnu.org" <68119 <at> debbugs.gnu.org>
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Thu, 4 Jan 2024 06:52:22 +0000

[Message part 1 (text/plain, inline)]

FYI, macOS has an old bison (version 2.3 from 2006).  Installing the current version of bison (3.8.2) eliminated most of the yacc errors.  Here's what I get now with 1.16j ...

FAIL: t/aclocal-deleted-header-aclocal-amflags
FAIL: t/aclocal-deleted-header
FAIL: t/aclocal-deps
FAIL: t/py-compile-basedir
FAIL: t/subdir-add2-pr46
FAIL: t/yacc-mix-c-cxx

In both aclocal-deleted-header.sh and aclocal-deleted-header-aclocal-amflags.sh, isn't an additional "$ACLOCAL -I . && $AUTOMAKE && $AUTOCONF && ./configure" needed before the second $MAKE?






> On Jan 3, 2024, at 3:07 PM, Karl Berry <karl <at> freefriends.org> wrote:
>
>    Using am_cv_sleep_fractional_seconds=false dropped it to 7-11 tests failing.
>
> Erik, thanks for all the trials and detailed analysis.
> Nothing is immediately clear to me either :(, but I'll take a look. -k

[Message part 2 (text/html, inline)]

[Message part 3 (application/applefile, inline)]

[test-suite.log.gz (application/octet-stream, attachment)]

[aclocal-deleted-header-aclocal-amflags.log (application/octet-stream, attachment)]

[aclocal-deleted-header.log (application/octet-stream, attachment)]

[aclocal-deps.log (application/octet-stream, attachment)]

[subdir-add2-pr46.log (application/octet-stream, attachment)]

[yacc-mix-c-cxx.log (application/octet-stream, attachment)]

Message #30 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu
Cc: 68119 <at> debbugs.gnu.org
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Thu, 4 Jan 2024 15:36:59 -0700

    FYI, macOS has an old bison (version 2.3 from 2006).

I'm not sure if automake theoretically support bison going back that
far, but I'm not going to worry about it since many errors disappear
with the new bison. Thanks for thinking of that.

    In both aclocal-deleted-header.sh and
    aclocal-deleted-header-aclocal-amflags.sh, isn't an additional
    "$ACLOCAL -I . && $AUTOMAKE && $AUTOCONF && ./configure" needed
    before the second $MAKE?

Very likely. I saw similar omitted autocalls in other tests.
I will look as soon as I have a chance. --thanks again, karl.

Message #33 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu
Cc: 68119 <at> debbugs.gnu.org
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Sun, 28 Jan 2024 08:50:40 -0700

Hi Erik,

    FAIL: t/aclocal-deleted-header-aclocal-amflags
    FAIL: t/aclocal-deleted-header

I'm not sure, but I think the idea of those tests, and others, is that
after making the various changes to the configure setup, running make
should suffice to rebuild everything. Because that's what is most
convenient and what people (including me :) expect.

So I added some $sleep commands before file modifications instead of
rerunning all the autotools. I hope it will suffice.  I'm mostly just
groping in the dark about these timing issues :(.

    FAIL: t/aclocal-deps
    FAIL: t/subdir-add2-pr46

Similarly added $sleep commands.

    FAIL: t/yacc-mix-c-cxx

Moved the #include <cstdlib> to before the "using" statements,
following the other yacc C++ tests.

    FAIL: t/py-compile-basedir

That one I don't know. py-compile has used importlib for quite a while.
But some changes have been made, so maybe try the current version, and
if it still fails (as it probably will), send the log?

If you have time and energy to try testing out of the git repository
(git clone git://git.savannah.gnu.org/automake.git),
the above changes are committed now. If not, I hope it won't be long
until the next (and last?!) pretest.

Thanks for all,
Karl

Message #36 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu
Cc: 68119 <at> debbugs.gnu.org
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Sun, 28 Jan 2024 15:07:05 -0700

Hi Erik - thanks for the continued testing.

    Hi, Karl.  I get an error on "make" because there is no "makeinfo"
    command on macOS.  

Well, since you've come this far, you could try installing the texinfo
package, if you have Perl.

Or perhaps it would be enough to
  touch doc/automake.info doc/automake-history.info 

    This didn't happen with the git clone from several weeks ago. 

That's surprising to me. It's normal for makeinfo to be required when
building from the dev sources. It's not required when building from a
release (or pretest), since .info are included in release tarballs
precisely to avoid this dependency.

Or maybe I'm forgetting something about building from the repo, in which
case I hope someone here will enlighten us :). --thanks again, karl.

Message #39 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Bogdan <bogdro_rep <at> gmx.us>
To: 68119 <at> debbugs.gnu.org
Subject: [bug#68119][GNU automake 1.16.5] test-suite 45 tests failed
Date: Sun, 28 Jan 2024 23:19:45 +0100

[Message part 1 (text/plain, inline)]

Hello.

 The attached patch fixes the tests for Python 3.12 (and, hopefully
later ones), while still keeping them working for earlier versions.
 A simple fix. Could perhaps be written better (like in a single
Python script), but this is enough.

--
Regards - Bogdan ('bogdro') D.                 (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):    http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft  http://bogdro.evai.pl/soft4asm
www.Xiph.org  www.TorProject.org  www.LibreOffice.org  www.GnuPG.org

[automake-fix-python-tests-3.12-mail.diff (text/x-patch, attachment)]

Message #42 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: bogdro_rep <at> gmx.us, JohnsonE <at> usc.edu
Cc: 68119 <at> debbugs.gnu.org
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Sun, 28 Jan 2024 19:14:09 -0700

      The attached patch fixes the tests for Python 3.12 

Thank you Bogdan! Looks good to me. Installed.

      (and, hopefully later ones), 

Maybe until Python 3.12.2, anyway. --thanks, karl.

Message #45 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Erik A Johnson <JohnsonE <at> usc.edu>
To: Karl Berry <karl <at> freefriends.org>
Cc: "68119 <at> debbugs.gnu.org" <68119 <at> debbugs.gnu.org>
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Mon, 29 Jan 2024 08:34:48 +0000

Success (well nearly), Karl!


1. Both "touch ..." and installing texinfo got compiling to go through.


2. Only one test failed now: line 46 of t/py-compile-basedir.sh is:

	case $(echo "$files" | wc -l) in 4|6) ;; *) false;; esac

but needs to be one of the following

	case $(echo "$files" | wc -l | sed 's/^ *//') in 4|6) ;; *) false;; esac
	case $(echo "$files" | wc -l | awk '{print $1}') in 4|6) ;; *) false;; esac
	linecount=$(echo "$files" | wc -l); case $(echo $linecount) in 4|6) ;; *) false;; esac

because "wc -l" prints some spaces in front of the number (at least on macOS).


Erik


P.S. "python" points to python2 by default on macOS Sonoma; the tests run using python2 but I will use
	export PYTHON=python3
to verify that it works with python 3.xx.

P.P.S. It still needs
	export am_cv_sleep_fractional_seconds=false
to correctly run the tests on macOS.  Without that, 75-90 tests fail.

Message #48 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu
Cc: 68119 <at> debbugs.gnu.org
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Mon, 29 Jan 2024 16:00:38 -0700

Hi Erik - yay for near-success :).

      case $(echo "$files" | wc -l | sed 's/^ *//') in 4|6) ;; *) false;; esac

I went with that sed.

    because "wc -l" prints some spaces in front of the number (at least
    on macOS).

I'm surprised this has not been seen already. But fine.
At least it's easy one to fix.

            export PYTHON=python3
    to verify that it works with python 3.xx.

Fingers crossed.

    P.P.S. It still needs
            export am_cv_sleep_fractional_seconds=false
    to correctly run the tests on macOS.  Without that, 75-90 tests fail.

I'll write separately about that, since this bug is already plenty long :).

Thanks,
Karl

Message #51 received at 68119 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: JohnsonE <at> usc.edu
Cc: 68119 <at> debbugs.gnu.org
Subject: Re: bug#68119: [GNU automake 1.16.5] test-suite 45 tests failed
Date: Wed, 31 Jan 2024 15:47:08 -0700

Erik, I think all the automake test problems you reported here have been
resolved (thanks to your excellent debugging). So closing. Leaving
#68808 open until Zack or I have a chance to look into testing of make
for the subsecond mtimes. Thanks again! -k

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.