GNU bug report logs - #7374
Automake bug when moving files to different directory

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 7374 in the body.
You can then email your comments to 7374 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Török Edwin <edwin <at> clamav.net>
To: bug-automake <at> gnu.org
Subject: Automake bug when moving files to different directory
Date: Thu, 11 Nov 2010 14:11:44 +0200

[Message part 1 (text/plain, inline)]

Hi,

Whenever I move a C/C++ file from a directory to another (and update
Makefile.am), a subsequent 'make' fails because it is looking for the
file in the old place (the .Plo dependency file actually).

For exampling moving test1/Foo.c to test2/Foo.c results in this error:
make: *** No rule to make target `test1/Foo.c', needed by `Foo.lo'.
Stop.

Tried a 'make clean', but it doesn't fix it, rerunning configure
doesn't fix it either.
Only a 'make distclean' and rerunning configure fixes it.

Versions:
autoconf (GNU Autoconf) 2.67
automake (GNU automake) 1.11.1
ltmain.sh (GNU libtool) 2.2.6b
gcc version 4.4.5 (Debian 4.4.5-6)

Minimal example attached as automake-move-bug.tar.gz, here are the
steps to reproduce the bug:

# Build initially, all good
autoreconf -fvi && ./configure && make
# Clean and rebuild, all good
make clean && make
# Move file around
mkdir test2
mv test1/Foo.c test2/Foo.c
mv test1/foo1.h test2/foo1.h
sed -ie 's/test1/test2/' Makefile.am
# Regen, and rerun configure, good so far
autoreconf -fvi && ./configure
# This one fails, but should work:
make || echo "Make failed, but should have worked!"
# Lets try cleaning and making again, fails too
# However I think 'make clean' should fix this as last resort, not
# distcheck. 
make clean && make || 
echo "Make clean, and remake failed, but should have worked!"
# Lets try distclean, this finally works
make distclean && ./configure && make

This is a bit annoying when it happens, because it breaks our
buildbots, and have to login to each one and run make distclean
manually (or zap the builddir) to fix it. Fortunately we don't move
around files that much (this is only 2nd time it occured in a few
years, and 1st time I wrote a testcase, but forgot to file the
bugreport).

I think that:
 - make clean should remove the dependency files if they're out of
   date/wrong
 - rerunning configure or config.status should regen the dependencies
   properly, and not rely on some old, out-of-date dependencies

Best regards,
--Edwin

[automake-move-bug.tar.gz (application/x-gzip, attachment)]

Message #8 received at 7374 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Török Edwin <edwin <at> clamav.net>
Cc: 7374 <at> debbugs.gnu.org
Subject: Re: bug#7374: Automake bug when moving files to different directory
Date: Thu, 11 Nov 2010 20:42:05 +0100

Hi Török,

thanks for the bug report.

* Török Edwin wrote on Thu, Nov 11, 2010 at 01:11:44PM CET:
> Whenever I move a C/C++ file from a directory to another (and update
> Makefile.am), a subsequent 'make' fails because it is looking for the
> file in the old place (the .Plo dependency file actually).
> 
> For exampling moving test1/Foo.c to test2/Foo.c results in this error:
> make: *** No rule to make target `test1/Foo.c', needed by `Foo.lo'.
> Stop.

Confirmed.  This is the "deleted header file" problem (discussed in the
list archives, and in the manual), except that for you it applies to the
primary source file, not included headers.

For the headers we work around the issue by adding
  header.h:

lines in the .Po files for each included header (non-ancient gcc
versions do this for us, for other compilers depcomp does it).  We don't
do this for the primary source files.  I think this is so that we get a
sensible error from make when some primary source file isn't present
(haven't checked archives on this yet, though).

> This is a bit annoying when it happens, because it breaks our
> buildbots, and have to login to each one and run make distclean
> manually (or zap the builddir) to fix it.

How come this happens at all for your buildbots?  Do they do incremental
builds only, rather than full rebuilds from scratch?  I'd expect bots to
mostly do the latter, unlike developers while hacking.

> Fortunately we don't move
> around files that much (this is only 2nd time it occured in a few
> years, and 1st time I wrote a testcase, but forgot to file the
> bugreport).

Good.  So this isn't a pressing issue.

> I think that:
>  - make clean should remove the dependency files if they're out of
>    date/wrong

That is unfortunately not possible, because make will barf if files
included from the Makefile are missing.  GNU make has '-include' for
this, but there is no equivalent of that for portable make.

This is the reason that config.status produces stub .Po files when they
are not present.

>  - rerunning configure or config.status should regen the dependencies
>    properly, and not rely on some old, out-of-date dependencies

Oh, we really believe in tracking dependencies on the fly for
compilation, i.e., we try to avoid extra invocations of the compiler in
order to compute dependency information (at least with capable
compilers).

Hope that helps.

Cheers,
Ralf

Message #11 received at 7374 <at> debbugs.gnu.org (full text, mbox):

From: Török Edwin <edwin <at> clamav.net>
To: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
Cc: 7374 <at> debbugs.gnu.org
Subject: Re: bug#7374: Automake bug when moving files to different directory
Date: Thu, 11 Nov 2010 22:10:56 +0200

On Thu, 11 Nov 2010 20:42:05 +0100
Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de> wrote:

> Hi Török,
> 
> thanks for the bug report.
> 
> * Török Edwin wrote on Thu, Nov 11, 2010 at 01:11:44PM CET:
> > Whenever I move a C/C++ file from a directory to another (and update
> > Makefile.am), a subsequent 'make' fails because it is looking for
> > the file in the old place (the .Plo dependency file actually).
> > 
> > For exampling moving test1/Foo.c to test2/Foo.c results in this
> > error: make: *** No rule to make target `test1/Foo.c', needed by
> > `Foo.lo'. Stop.
> 
> Confirmed.  This is the "deleted header file" problem (discussed in
> the list archives, and in the manual), except that for you it applies
> to the primary source file, not included headers.
> 
> For the headers we work around the issue by adding
>   header.h:
> 
> lines in the .Po files for each included header (non-ancient gcc
> versions do this for us, for other compilers depcomp does it).  We
> don't do this for the primary source files.  I think this is so that
> we get a sensible error from make when some primary source file isn't
> present (haven't checked archives on this yet, though).
> 
> > This is a bit annoying when it happens, because it breaks our
> > buildbots, and have to login to each one and run make distclean
> > manually (or zap the builddir) to fix it.
> 
> How come this happens at all for your buildbots?  Do they do
> incremental builds only, rather than full rebuilds from scratch?  I'd
> expect bots to mostly do the latter, unlike developers while hacking.

They do incremental builds to save some time, there are parts of the
code that rarely change.
Bots aren't the only ones affected, team members and some git users too.
Have to tell them "automake bug, run 'make distclean'".
Some people know to retry running 'make clean', or rerunning
configure. distclean is not very popular.

> 
> > Fortunately we don't move
> > around files that much (this is only 2nd time it occured in a few
> > years, and 1st time I wrote a testcase, but forgot to file the
> > bugreport).
> 
> Good.  So this isn't a pressing issue.
> 
> > I think that:
> >  - make clean should remove the dependency files if they're out of
> >    date/wrong
> 
> That is unfortunately not possible, because make will barf if files
> included from the Makefile are missing.  GNU make has '-include' for
> this, but there is no equivalent of that for portable make.
> 
> This is the reason that config.status produces stub .Po files when
> they are not present.

OK, then how about:
 - make clean deletes .Po/.Plo files, and invokes config.status with
   proper flags to produce the stub .Po files?

Having the old .Po files around after a make clean isn't going to help
speed up the build anyway since all the obj files are gone.

> 
> >  - rerunning configure or config.status should regen the
> > dependencies properly, and not rely on some old, out-of-date
> > dependencies
> 
> Oh, we really believe in tracking dependencies on the fly for
> compilation, i.e., we try to avoid extra invocations of the compiler
> in order to compute dependency information (at least with capable
> compilers).

See above, it could just generate the dummy .Po files.
But of course that would mean that updating Makefile.am/configure could
result in rebuild of entire project, which is not good.

If there was some way to check that .Po files are still valid, then
config.status could do that, and replace them with dummy ones if
they're not valid. 
Like checking if the source file declared in the .Po file still exists.
Not sure how easy or hard that would be to do in portable shell
programming. Probably not impossible though ;)

Best regards,
--Edwin

Message #14 received at 7374 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Török Edwin <edwin <at> clamav.net>
Cc: 7374 <at> debbugs.gnu.org
Subject: Re: bug#7374: Automake bug when moving files to different directory
Date: Thu, 11 Nov 2010 21:38:01 +0100

* Török Edwin wrote on Thu, Nov 11, 2010 at 09:10:56PM CET:
> On Thu, 11 Nov 2010 20:42:05 +0100 Ralf Wildenhues wrote:
> > * Török Edwin wrote on Thu, Nov 11, 2010 at 01:11:44PM CET:
> > > I think that:
> > >  - make clean should remove the dependency files if they're out of
> > >    date/wrong
> > 
> > That is unfortunately not possible, because make will barf if files
> > included from the Makefile are missing.  GNU make has '-include' for
> > this, but there is no equivalent of that for portable make.
> > 
> > This is the reason that config.status produces stub .Po files when
> > they are not present.
> 
> OK, then how about:
>  - make clean deletes .Po/.Plo files, and invokes config.status with
>    proper flags to produce the stub .Po files?
> 
> Having the old .Po files around after a make clean isn't going to help
> speed up the build anyway since all the obj files are gone.

True, but the above sounds quite ugly as well.

Here's a better suggestion: use the subdir-objects automake option (you
can add it to the options passed to AM_INIT_AUTOMAKE) and configure with
--enable-maintainer-mode.  That way the old .Po file will just not be
read any more, and the maintainer mode will trigger the automake rerun
that then causes the rules in Makefile to be successfully updated.

Hope that is good enough for you.

Cheers,
Ralf

Message #19 received at 7374-done <at> debbugs.gnu.org (full text, mbox):

From: Török Edwin <edwin <at> clamav.net>
To: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
Cc: 7374-done <at> debbugs.gnu.org
Subject: Re: bug#7374: Automake bug when moving files to different directory
Date: Thu, 11 Nov 2010 22:49:39 +0200

On Thu, 11 Nov 2010 21:38:01 +0100
Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de> wrote:

> * Török Edwin wrote on Thu, Nov 11, 2010 at 09:10:56PM CET:
> > On Thu, 11 Nov 2010 20:42:05 +0100 Ralf Wildenhues wrote:
> > > * Török Edwin wrote on Thu, Nov 11, 2010 at 01:11:44PM CET:
> > > > I think that:
> > > >  - make clean should remove the dependency files if they're out
> > > > of date/wrong
> > > 
> > > That is unfortunately not possible, because make will barf if
> > > files included from the Makefile are missing.  GNU make has
> > > '-include' for this, but there is no equivalent of that for
> > > portable make.
> > > 
> > > This is the reason that config.status produces stub .Po files when
> > > they are not present.
> > 
> > OK, then how about:
> >  - make clean deletes .Po/.Plo files, and invokes config.status with
> >    proper flags to produce the stub .Po files?
> > 
> > Having the old .Po files around after a make clean isn't going to
> > help speed up the build anyway since all the obj files are gone.
> 
> True, but the above sounds quite ugly as well.
> 
> Here's a better suggestion: use the subdir-objects automake option
> (you can add it to the options passed to AM_INIT_AUTOMAKE)

Excellent, that does exactly what I wanted! (and my testcase works now)

> and
> configure with --enable-maintainer-mode.
> That way the old .Po file
> will just not be read any more, and the maintainer mode will trigger
> the automake rerun that then causes the rules in Makefile to be
> successfully updated.

Yes, I use maintainer-mode sometimes. For simple edits I tend to just
run 'automake' though, and commit Makefile.in.

> 
> Hope that is good enough for you.

Yes, thanks a lot!
We are in code freeze right now, but I'll definetely make that change
after we're done.

We can close this bug now.

Best regards,
--Edwin

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.