GNU bug report logs - #13107
timestamp bug when files are created just before make is run

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 13107 in the body.
You can then email your comments to 13107 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mikulas Patocka <mikulas <at> artax.karlin.mff.cuni.cz>
To: bug-make <at> gnu.org, bug-automake <at> gnu.org
Subject: timestamp bug when files are created just before make is run
Date: Thu, 6 Dec 2012 23:02:48 +0100 (CET)

Hi

Try this Makefile:
---
a : b
        echo build a
        touch a

b : c
        echo build b
        touch b
---
and run this script:
touch b;sleep 1;touch a c;make

You see
"echo build b
build b
touch b"
but it doesn't remake a.

The apparent problem is that after make rebuilds b, it compares b's time 
with a's time, finds that the times are equal (because a was touched just 
before make was run) and doesn't rebuild a.

I think it is a bug - when b is finished, make should find out whether the 
rule for b modified the file b (if b's time is greater or equal than the 
time when the rule for b was started, then b may have been modified) and 
rebuild a in this case.

---

This bug is causing real-world problems in automake-generated Makefiles. 
This is a simplified piece of Makefile from automake:
all : config.h
        $(MAKE) all-am

config.h : stamp-h1
        echo build config.h

stamp-h1 : config.h.in
        echo build stamp-h1
        rm -f stamp-h1
        touch config.h
        touch stamp-h1

config.h.in : am__configure_deps
        echo build config.h.in
        rm -f stamp-h1
        touch config.h.in

all-am :
        echo ALL-AM

Now, if you run this script, you trigger the bug:

touch config.h.in;sleep 1;touch am__configure_deps;sleep 1;touch config.h stamp-h1;make

- you see "build config.h.in", but the other files are not rebuild and 
all-am is executed
(the essential thing to trigger the bug is that make is run in the same 
second in which config.h and stamp-h1 were created)


The problem that really happens in a real build:

* The user runs autoheader && aclocal && automake && autoconf && ./configure && make -j4
* Configure runs ./config.status, ./config.status writes config.h and 
stamp-h1
* Make sees that am__configure_deps is newer than config.h.in
* Make runs the rule for config.h.in. It sets the new timestamp for 
config.h.in and deletes stamp-h1
* Now make sees that config.h.in has the same time as stamp-h1 (the 
timestamp is read from make's cache even if stamp-h1 no longer exists)
* Because of the same timestamp, make doesn't run the commands for 
stamp-h1 and config.h
* Make executes a subprocess to build the rule all-am

* The subprocess doesn't have the file cache of the parent process, so the 
subprocess knows that stamp-h1 is missing
* The subprocess sees a dependency config.h->stamp-h1, stamp-h1 doesn't 
exist
* The subprocess executes the rule for stamp-h1 which regenerates config.h 
- the problem is that this rule is executed in parallel with other rules 
that build C files that include config.h - changing config.h while it is 
being used results in build failure

Another possible solution for this bug would be to remove rm -f stamp-h1 
from config.h.in rule, but there is some complex explanation in 
/usr/local/share/automake-1.12/am/remake-hdr.am why rm -f stamp-h1 is 
there so it would likely fix one bug and trigger another one.

So it would be better to fix this in make.

Mikulas

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philip Guenther <guenther <at> gmail.com>
To: Mikulas Patocka <mikulas <at> artax.karlin.mff.cuni.cz>
Cc: bug-automake <at> gnu.org, bug-make <at> gnu.org
Subject: Re: timestamp bug when files are created just before make is run
Date: Thu, 6 Dec 2012 15:53:40 -0800

On Thu, Dec 6, 2012 at 2:02 PM, Mikulas Patocka
<mikulas <at> artax.karlin.mff.cuni.cz> wrote:
> The apparent problem is that after make rebuilds b, it compares b's time
> with a's time, finds that the times are equal (because a was touched just
> before make was run) and doesn't rebuild a.
>
> I think it is a bug - when b is finished, make should find out whether the
> rule for b modified the file b (if b's time is greater or equal than the
> time when the rule for b was started, then b may have been modified) and
> rebuild a in this case.

That's not the rule that make has always followed and, indeed, is
required by the standard to follow.  To quote POSIX:

        The make utility attempts to perform the actions required to ensure
        that the specified targets are up-to-date. A target is considered
        out-of-date if it is older than any of its prerequisites or if
it does not
        exist. The make utility shall treat all prerequisites as targets
        themselves and recursively ensure that they are up-to-date, processing
        them in the order in which they appear in the rule. The make utility
        shall use the modification times of files to determine whether the
        corresponding targets are out-of-date.

The fact that the timestamp on a prerequisite has changed do *not*
mean that it's newer than the target; it may have been copied from
some other file which, while newer, is still older than the target
file.  So the change to the make algorithm described in the paragraph
above that starts "I think it is a bug" would not be correct;
something more precise about what timestamps would be considered
"newer" would be necessary.


Note that this problem doesn't arise on systems with high precision
file timestamps.  Many systems have provided those since the mid 90's;
I'm appalled that the modern system that process the involved shell
commands fast enough for this to regularly be a problem don't provide
microsecond or better timestamp precision.


> This bug is causing real-world problems in automake-generated Makefiles.
> This is a simplified piece of Makefile from automake:
> all : config.h
>         $(MAKE) all-am
>
> config.h : stamp-h1
>         echo build config.h
>
> stamp-h1 : config.h.in
>         echo build stamp-h1
>         rm -f stamp-h1
>         touch config.h
>         touch stamp-h1
>
> config.h.in : am__configure_deps
>         echo build config.h.in
>         rm -f stamp-h1
>         touch config.h.in
>
> all-am :
>         echo ALL-AM
>
> Now, if you run this script, you trigger the bug:
>
> touch config.h.in;sleep 1;touch am__configure_deps;sleep 1;touch config.h stamp-h1;make
>
> - you see "build config.h.in", but the other files are not rebuild and
> all-am is executed
> (the essential thing to trigger the bug is that make is run in the same
> second in which config.h and stamp-h1 were created)
...
> Another possible solution for this bug would be to remove rm -f stamp-h1
> from config.h.in rule, but there is some complex explanation in
> /usr/local/share/automake-1.12/am/remake-hdr.am why rm -f stamp-h1 is
> there so it would likely fix one bug and trigger another one.

Hmm, adding a "sleep 1" after that rm -f and before the touch
config.h.in would work without reintroducing the issue described in
that explanation, no?


Philip Guenther

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mikulas Patocka <mikulas <at> artax.karlin.mff.cuni.cz>
To: Philip Guenther <guenther <at> gmail.com>
Cc: bug-automake <at> gnu.org, bug-make <at> gnu.org
Subject: Re: timestamp bug when files are created just before make is run
Date: Fri, 7 Dec 2012 02:55:48 +0100 (CET)

On Thu, 6 Dec 2012, Philip Guenther wrote:

> On Thu, Dec 6, 2012 at 2:02 PM, Mikulas Patocka
> <mikulas <at> artax.karlin.mff.cuni.cz> wrote:
> > The apparent problem is that after make rebuilds b, it compares b's time
> > with a's time, finds that the times are equal (because a was touched just
> > before make was run) and doesn't rebuild a.
> >
> > I think it is a bug - when b is finished, make should find out whether the
> > rule for b modified the file b (if b's time is greater or equal than the
> > time when the rule for b was started, then b may have been modified) and
> > rebuild a in this case.
> 
> That's not the rule that make has always followed and, indeed, is
> required by the standard to follow.  To quote POSIX:
> 
>         The make utility attempts to perform the actions required to ensure
>         that the specified targets are up-to-date. A target is considered
>         out-of-date if it is older than any of its prerequisites or if
> it does not
>         exist. The make utility shall treat all prerequisites as targets
>         themselves and recursively ensure that they are up-to-date, processing
>         them in the order in which they appear in the rule. The make utility
>         shall use the modification times of files to determine whether the
>         corresponding targets are out-of-date.
> 
> The fact that the timestamp on a prerequisite has changed do *not*
> mean that it's newer than the target; it may have been copied from
> some other file which, while newer, is still older than the target
> file.  So the change to the make algorithm described in the paragraph
> above that starts "I think it is a bug" would not be correct;
> something more precise about what timestamps would be considered
> "newer" would be necessary.
> 
> 
> Note that this problem doesn't arise on systems with high precision
> file timestamps.  Many systems have provided those since the mid 90's;
> I'm appalled that the modern system that process the involved shell
> commands fast enough for this to regularly be a problem don't provide
> microsecond or better timestamp precision.

Yes, but it is very counterintuitive to remake file "b" and not remake 
file "a" that depends on "b" when asked to make file "a". It may be ok 
according to the standard, but not what the user wants.

> > This bug is causing real-world problems in automake-generated Makefiles.
> > This is a simplified piece of Makefile from automake:
> > all : config.h
> >         $(MAKE) all-am
> >
> > config.h : stamp-h1
> >         echo build config.h
> >
> > stamp-h1 : config.h.in
> >         echo build stamp-h1
> >         rm -f stamp-h1
> >         touch config.h
> >         touch stamp-h1
> >
> > config.h.in : am__configure_deps
> >         echo build config.h.in
> >         rm -f stamp-h1
> >         touch config.h.in
> >
> > all-am :
> >         echo ALL-AM
> >
> > Now, if you run this script, you trigger the bug:
> >
> > touch config.h.in;sleep 1;touch am__configure_deps;sleep 1;touch config.h stamp-h1;make
> >
> > - you see "build config.h.in", but the other files are not rebuild and
> > all-am is executed
> > (the essential thing to trigger the bug is that make is run in the same
> > second in which config.h and stamp-h1 were created)
> ...
> > Another possible solution for this bug would be to remove rm -f stamp-h1
> > from config.h.in rule, but there is some complex explanation in
> > /usr/local/share/automake-1.12/am/remake-hdr.am why rm -f stamp-h1 is
> > there so it would likely fix one bug and trigger another one.
> 
> Hmm, adding a "sleep 1" after that rm -f and before the touch
> config.h.in would work without reintroducing the issue described in
> that explanation, no?
> 
> 
> Philip Guenther

Yes it fixes it, you can add "sleep 1" to automake there, that fixes the 
bug.

Mikulas

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mikulas Patocka <mikulas <at> artax.karlin.mff.cuni.cz>
To: Philip Guenther <guenther <at> gmail.com>
Cc: bug-automake <at> gnu.org, bug-make <at> gnu.org
Subject: Re: timestamp bug when files are created just before make is run
Date: Fri, 7 Dec 2012 03:16:31 +0100 (CET)

On Thu, 6 Dec 2012, Philip Guenther wrote:

> Note that this problem doesn't arise on systems with high precision
> file timestamps.  Many systems have provided those since the mid 90's;
> I'm appalled that the modern system that process the involved shell
> commands fast enough for this to regularly be a problem don't provide
> microsecond or better timestamp precision.

BTW. on Linux, high precision timestamps have really kernel-tick 
precision, not nanosecond precision.

So the first example that I posted doesn't always work correctly on Linux 
with 100HZ tick - with sufficiently fast computer, touch "a" and the rule 
to make "b" is executed in the same tick - so "a" isn't rebuilt.

Mikulas

Message #17 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> suse.de>
To: Mikulas Patocka <mikulas <at> artax.karlin.mff.cuni.cz>
Cc: Philip Guenther <guenther <at> gmail.com>, bug-automake <at> gnu.org,
	bug-make <at> gnu.org
Subject: Re: timestamp bug when files are created just before make is run
Date: Mon, 10 Dec 2012 10:45:45 +0100

Mikulas Patocka <mikulas <at> artax.karlin.mff.cuni.cz> writes:

> BTW. on Linux, high precision timestamps have really kernel-tick 
> precision, not nanosecond precision.

The precision is as high as what the best hardware timer provides,
independent of the configured value of HZ.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab <at> suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Message #22 received at 13107 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: 13107 <at> debbugs.gnu.org
Subject: Re: bug#13107: timestamp bug when files are created just before make
 is run
Date: Sat, 11 Jul 2020 19:23:45 -0600

Given the zero response, it seems undesirable to add even minor
complexity to support something which may no longer be needed.
Closing. -k

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.