GNU bug report logs - #26471
Automake 1.15 generates a recheck target that depends on all, breaking parallel builds

Previous Next

To reply to this bug, email your comments to 26471 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Seth Fowler <seth.fowler <at> me.com>
To: bug-automake <at> gnu.org
Subject: Automake 1.15 generates a recheck target that depends on all,
 breaking parallel builds
Date: Wed, 12 Apr 2017 21:54:00 -0700

Hi folks,

I recently ran into what appears to be a bug in automake and I thought it would be a good idea to let you know.

We had noticed that running “make recheck -j8” if some source files were dirty would cause random build failures. The symptom was that the same file was getting built more than once by different make processes, which led to the resulting objects being corrupted. We’d see messages like this:

    libtool:   error: ‘foo.lo’ is not a valid libtool object

I dug into this a little and the root cause of the problem seems to be that unlike the other top-level targets generated by automake (check, install, etc.), recheck depends on “all” instead of “all-am”. “all” doesn’t declare very many dependencies - it just launches a new copy of make that builds “all-am”. Because recheck also depends on other targets (e.g. everything in check_PROGRAMS) which might depend on some of the same things as “all-am”, in a parallel build the original make process and the make process spawned by “all” can end up attempting to build the same targets.

I fixed this for our project by copying the generated recheck rule into our Makefile.am and replacing “all” with “all-am”. After doing this, I could no longer reproduce the problem with “make recheck -j8”.

It seems like that change should be made in automake itself. Or maybe I just missed something - I’m far from an automake guru. =)

Thanks,
- Seth

Message #8 received at 26471 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Lirzin <mthl <at> gnu.org>
To: Seth Fowler <seth.fowler <at> me.com>
Cc: 26471 <at> debbugs.gnu.org
Subject: Re: bug#26471: Automake 1.15 generates a recheck target that depends
 on all, breaking parallel builds
Date: Thu, 13 Apr 2017 15:52:41 +0200

Hi Seth,

Seth Fowler <seth.fowler <at> me.com> writes:

> I recently ran into what appears to be a bug in automake and I thought it would be a good idea to let you know.
>
> We had noticed that running “make recheck -j8” if some source files
> were dirty would cause random build failures. The symptom was that the
> same file was getting built more than once by different make
> processes, which led to the resulting objects being corrupted. We’d
> see messages like this:
>
>     libtool:   error: ‘foo.lo’ is not a valid libtool object
>
> I dug into this a little and the root cause of the problem seems to be
> that unlike the other top-level targets generated by automake (check,
> install, etc.), recheck depends on “all” instead of “all-am”. “all”
> doesn’t declare very many dependencies - it just launches a new copy
> of make that builds “all-am”. Because recheck also depends on other
> targets (e.g. everything in check_PROGRAMS) which might depend on some
> of the same things as “all-am”, in a parallel build the original make
> process and the make process spawned by “all” can end up attempting to
> build the same targets.
>
> I fixed this for our project by copying the generated recheck rule
> into our Makefile.am and replacing “all” with “all-am”. After doing
> this, I could no longer reproduce the problem with “make recheck -j8”.
>
> It seems like that change should be made in automake itself. Or maybe I just missed something - I’m far from an automake guru. =)

Seems like a bug.

In order to properly fix this in Automake, we need to have a test
that allow us to reproduce the issue.  Ideally you would provide
this test yourself by taking inspiration from:

  https://git.savannah.gnu.org/cgit/automake.git/tree/t/libtool2.sh?h=micro

If you are not comfortable with doing that, a tarball or a link to a
repository containing a minimal automake project that allow us to
reproduce the bug manually is good enough.

Thanks for the report.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Message #13 received at 26471 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Lirzin <mthl <at> gnu.org>
To: Seth Fowler <seth.fowler <at> me.com>
Cc: 26471 <at> debbugs.gnu.org
Subject: Re: bug#26471: Automake 1.15 generates a recheck target that depends
 on all, breaking parallel builds
Date: Fri, 19 Jan 2018 01:37:00 +0100

Hi Seth,

Mathieu Lirzin <mthl <at> gnu.org> writes:

> Seth Fowler <seth.fowler <at> me.com> writes:
>
>> I recently ran into what appears to be a bug in automake and I thought it would be a good idea to let you know.
>>
>> We had noticed that running “make recheck -j8” if some source files
>> were dirty would cause random build failures. The symptom was that the
>> same file was getting built more than once by different make
>> processes, which led to the resulting objects being corrupted. We’d
>> see messages like this:
>>
>>     libtool:   error: ‘foo.lo’ is not a valid libtool object
>>
>> I dug into this a little and the root cause of the problem seems to be
>> that unlike the other top-level targets generated by automake (check,
>> install, etc.), recheck depends on “all” instead of “all-am”. “all”
>> doesn’t declare very many dependencies - it just launches a new copy
>> of make that builds “all-am”. Because recheck also depends on other
>> targets (e.g. everything in check_PROGRAMS) which might depend on some
>> of the same things as “all-am”, in a parallel build the original make
>> process and the make process spawned by “all” can end up attempting to
>> build the same targets.
>>
>> I fixed this for our project by copying the generated recheck rule
>> into our Makefile.am and replacing “all” with “all-am”. After doing
>> this, I could no longer reproduce the problem with “make recheck -j8”.
>>
>> It seems like that change should be made in automake itself. Or maybe I just missed something - I’m far from an automake guru. =)
>
> Seems like a bug.

Any news?  Would be great if you could provide a way to reproduce this
issue.

Thanks.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

Message #16 received at 26471 <at> debbugs.gnu.org (full text, mbox):

From: Bogdan <bogdro_rep <at> gmx.us>
To: automake-patches <at> gnu.org, 68860 <at> debbugs.gnu.org, 26471 <at> debbugs.gnu.org
Subject: [bug#68860] race condition with make recheck
Date: Fri, 23 Aug 2024 23:10:07 +0200

Hi.

I've just noticed that bug #68860 (patched) may be a duplicate of
#26471. Different descriptions and error messages, but looks like the
same cause.

--
Regards - Bogdan ('bogdro') D.                 (GNU/Linux & FreeDOS)
X86 assembly (DOS, GNU/Linux):    http://bogdro.evai.pl/index-en.php
Soft(EN): http://bogdro.evai.pl/soft  http://bogdro.evai.pl/soft4asm
www.Xiph.org  www.TorProject.org  www.LibreOffice.org  www.GnuPG.org

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.