GNU bug report logs - #11377
configure.am - fails to remove configure before attempting replacement

Previous Next

Package: automake;

Reported by: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>

Date: Sat, 28 Apr 2012 22:06:02 UTC

Severity: minor

Tags: moreinfo, wontfix

Done: Stefano Lattarini <stefano.lattarini <at> gmail.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 11377 in the body.
You can then email your comments to 11377 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-automake <at> gnu.org:
bug#11377; Package automake. (Sat, 28 Apr 2012 22:06:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Ronald F. Guilmette" <rfg <at> tristatelogic.com>:
New bug report received and forwarded. Copy sent to bug-automake <at> gnu.org. (Sat, 28 Apr 2012 22:06:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>
To: bug-automake <at> gnu.org
Subject: configure.am - fails to remove configure before attempting replacement
Date: Sat, 28 Apr 2012 13:55:00 -0700

Under certain very reasonable scenarios, the configure file contained within
some source tree may perhaps be marked as read-only.  When and if this occurs,
automake will fail to remove the configure file before it attempts to generate
it anew.  The following trivial patch corrects this problem.

I would really appreciate it if this small patch could be incorporated into
future releases of automake.  It does no harm, and it can provably do some
good in certain circumstances.



diff -rc2 src/automake-1.11.5/lib/am/configure.am build/automake-1.11.5/lib/am/configure.am
*** src/automake-1.11.5/lib/am/configure.am	2012-04-13 07:46:00.000000000 -0700
--- build/automake-1.11.5/lib/am/configure.am	2012-04-25 15:05:50.000000000 -0700
***************
*** 107,111 ****
  
  $(top_srcdir)/configure: %MAINTAINER-MODE% $(am__configure_deps)
! ?TOPDIR_P?	$(am__cd) $(srcdir) && $(AUTOCONF)
  ?!TOPDIR_P?	cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh
  
--- 107,111 ----
  
  $(top_srcdir)/configure: %MAINTAINER-MODE% $(am__configure_deps)
! ?TOPDIR_P?	$(am__cd) $(srcdir) && rm -f $(top_srcdir)/configure && $(AUTOCONF)
  ?!TOPDIR_P?	cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh
  




Information forwarded to bug-automake <at> gnu.org:
bug#11377; Package automake. (Tue, 01 May 2012 13:59:02 GMT) Full text and rfc822 format available.

Message #8 received at 11377 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>
Cc: 11377 <at> debbugs.gnu.org
Subject: Re: bug#11377: configure.am - fails to remove configure before
	attempting replacement
Date: Tue, 01 May 2012 15:57:05 +0200
tags 11377 + moreinfo
severity 11377 minor
thanks

Hi Ronald, sorry for the delay.

On 04/28/2012 10:55 PM, Ronald F. Guilmette wrote:
> Under certain very reasonable scenarios, the configure file contained within
> some source tree may perhaps be marked as read-only.
>
Before continuing, may I ask you which these scenarios are exactly?

> When and if this occurs,
> automake will fail to remove the configure file before it attempts to generate
> it anew.  The following trivial patch corrects this problem.
> 
> I would really appreciate it if this small patch could be incorporated into
> future releases of automake.  It does no harm, and it can provably do some
> good in certain circumstances.
> 
If you convince me that the circumstances you describe are actually reasonable,
we should also enhance your patch to cover the similar cases of aclocal.m4,
Makefile.in, etc. created read-only (and probably add a test case to ensure
we don't regress with future changes).

Thanks,
  Stefano




Information forwarded to bug-automake <at> gnu.org:
bug#11377; Package automake. (Thu, 03 May 2012 22:12:01 GMT) Full text and rfc822 format available.

Message #11 received at 11377 <at> debbugs.gnu.org (full text, mbox):

From: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 11377 <at> debbugs.gnu.org
Subject: Re: bug#11377: configure.am - fails to remove configure before
	attempting replacement
Date: Thu, 03 May 2012 15:09:55 -0700
My apologies for the delay in replying.  I have been tied up due to other
responsibilities.

In message <4F9FEBB1.50304 <at> gmail.com>, you wrote:

>tags 11377 + moreinfo
>severity 11377 minor
>thanks
>
>Hi Ronald, sorry for the delay.
>
>On 04/28/2012 10:55 PM, Ronald F. Guilmette wrote:
>> Under certain very reasonable scenarios, the configure file contained within
>> some source tree may perhaps be marked as read-only.
>>
>Before continuing, may I ask you which these scenarios are exactly?

Well, since you asked...

I have been working on GNU software since 1989... going on 23 years now.  I
was one of the earlier contributors to the GNU software development tools,
in particular the GNU C compiler.

Long long ago I established a personal methodology of working on GNU and
other open source software.  Some people may take issue with my approach,
but it works for me, and I am comfortable with it, so I use it, always.

If I am working towards the goal of making changes in some package, e.g.
fixing bugs or adding enhancements, then I always proceed as follows:

     1)  Unzip & untar the package sources into a "src/" directory.  (For
         example, I am currently working on adding some enhancemente to
         the "gthumb" application, so I have unpacked the sources for
         that into a directory called "src/gthumb-2.14.3".)

     2)  In order to ensure that I have a pristine and original copy of the
         current sources available (relative to which I will later make diffs
         representing my personal patches) I run the command "chmod -R ugo-w"
         on the entire sources tree that I just unpacked.  This step is
	 critical to insuring that any subsequent patches file I generate
	 using `diff -R' will be properly relative to the current official
	 sources for the package in question.  (Making all of the regular
	 files read-only prevents me from inadvertantly making changes to
	 any of the original source files without doing so quite deliberately.)

     3)  I then make a clone of the src/ tree into a separate and parallel
         "work/" tree where I will make all of my local source code changes.
	 On FreeBSD, at least, this can be accomplished via the "cp -R -l"
         command.  So, for example, I would do this:

        mkdir work
	cp -R -l src/gthumb-2.14.3 work/gthumb-2.14.3

        Note that in order to both minimize disk space consumption _and_ to
	dramatically speed up any and all subsequent whole tree recursive
	diffs, I do not actually make physical copies of the regular files
	when constructing my clone "work/" tree.  Rather, the -l option for
	cp causes all regular files to simply be hard-linked into both trees.

    4)  After all of the above, I go to work on my work/ tree, making changes
	as necessary to implement the bug fixes and/or enhancements as I think
	are approproate.  Of course, as I do so I make changes to
	various of the source files that I had previously marked as read-only.
	(Note that all of the regular files of the src/ AND work/ trees have
	been marked as read-only, because they are just hard-linked copies
	of one another.)

	Because the regular files files in the work/ tree are read-only,
	each time I need to make changes to one I have to un-link it from
	its src/ counterpart, making an actual new physical copy as I do,
	and then mark that copy as read/write.  I have developed a trivial
	shell script for doing this in one simple step.  I call it "mw"
	(make writable).  Here is the C-shell source for that:

	=====================================================================
	#!/bin/csh
	
	# mw - make writable
	
	set argv=($*)
	
	foreach pathname ($argv)
	        echo Processing $pathname
	        if (-d $pathname) then
	                echo $0\: cannot mw on a directory\: $pathname
	        else
	                rm -f $pathname-mw$$
	                mv $pathname $pathname-mw$$
	                cp $pathname-mw$$ $pathname
	                touch -a -m -r $pathname-mw$$ $pathname
	                rm -f $pathname-mw$$
	                chmod +w $pathname
	        endif
	end
	=====================================================================

	So, for example, if I an working in my work/ tree and decide that I
	will be neededing to make changes to the foobar.c and foobar.h files,
	then I will just simply type:

	mw foobar.c foobar.h

    5)  Lastly, when I am done making and testing my local changes, I can
	easily extract all of my local source code changes via this simple
	command:

		diff -c 2 -R src work

	Of course, I then contribute these back to the package maintaner, you
	know, for the greater glory of GNU.

One could easily argue that what I am doing with all of the above is, in
effect, to implement a sort of "poor man's revision control system".  I
personally would not argue that point, one way or the other.

There may perhaps be better ways of doing what I am trying to do with the
methodology described above.  I also would not argue that point either.
What I do know is that _everyone_ who works on software had their own
idiosyncratic methods and approaches, and I am no different in that regard.
I am comfortable with the above way of working, and it has served me well
for the past 23 years of contributing to GNU software.

The only problem with the approach outlined above is that sometimes (rarely)
some Makefiles and/or other tools used as part of a build process (e.g.
automake) may occasionally make the assumption that every regular file
within the current (build) tree is writable, and that thus, when one of these
tools is about to perform some operation that will attempt to effectively
over-write some such file, the over-write can be performed _without_ first
deleting the file in question.

It is these exact (naive and, thankfully, quite rare) assumptions that can
cause the development methodology I have described above to malfunction
because, for example, Makefiles may be written in a way that assumes that
a given file will be properly overwritten, even in cases where it won't be,
e.g. because (a) it has been maked as read-only and also (b) the Makefile
in question failed to take the simple and altogether reasonable precaution
of `rm'ing the file in question before attempting the over-write.

I want to reemphasize what I just said.  Removing a given regular file before
attempting some operation that is intended to fully replace the contents of
that file is a simple and altogether reasonable precaution... one that all
Makefiles should take in all such cases.  It provably will never do any harm
to add this `rm' step to the relevant Makefiles, and it is altogether
apparent that it can be helpful, in some cases, to perform the `rm' step.

Makefiles that have been written _without_ this level of care and caution
can and do malfunction, and when they do, they can be expected to do so in
ways that are likely to be more difficult than necessary to debug, because
the Makefile will be incorrectly tricked into thinking that a given file
has been successflly re-made, when that has *not* actually occured.

Take this simple example Makefile:

	send-command:	command
		cat command > NORAD

	command:
		-echo "Stand Down\n" > command

Assuming that the file named "command" exists before this Makefile is invoked,
and that it somehow came to be marked (accidentally or otherwise) "read only",
and that its pre-existing content is the single line "Fire all missles!\n"...
well... the outcome in this case is, shall we say, sub-optimal, and is not
what one would have hoped.  Note however that inserting the command:

		-rm -f command

at the top of the set of commands used to re-make the "command" file would
make all the difference, and would result in a more suitable outcome.

(Personally, I am an old-school adherent to the philosophy known as
"defensive programming" and this extends even to my Makefiles.  The basic
tenets of defensive programming are: (1) if anything can go wrong, it
_will_ god wrong[1] and (2) if you are given a choice between using a
belt, using suspenders, or using a belt _and_ suspenders, you should
always choose both belt and suspenders.  In this case, over-writting
a file that needs to be aover-written is the belt.  Rm'ing the file first
is the suspenders.)

Anyway, exactly this sort of situation does in fact arise within one of the
automake-generated Makefiles of gthumb-2.14.3.  There is a failure to remove
one configure file before attempting to replace it, and if (as in my case)
the configure file in question may have been marked as read-only, then all
hell breaks loose and things go very very wrong.  (It was this exact problem
that prompted my change request.)

The botom line is just this:  It is maximally fastidious to always make it a
point to rm all files that are about to be over-written.  There are -zero-
cases in which doing so causes any harm whatsoever, however there are at
least some cases in which it is helpful to exercize exactly this level of
caution.  Thus, on balance, it is better to always rm files before over-
writting than it is to sometimes fail to do so.

>> When and if this occurs,
>> automake will fail to remove the configure file before it attempts to genera
>te
>> it anew.  The following trivial patch corrects this problem.
>> 
>> I would really appreciate it if this small patch could be incorporated into
>> future releases of automake.  It does no harm, and it can provably do some
>> good in certain circumstances.
>> 
>If you convince me that the circumstances you describe are actually reasonable
>,
>we should also enhance your patch to cover the similar cases of aclocal.m4,
>Makefile.in, etc. created read-only (and probably add a test case to ensure
>we don't regress with future changes).


That all sounds quite reasonable.


Regards,
rfg


=====
[1]  See also:  Murphy's Law




Information forwarded to bug-automake <at> gnu.org:
bug#11377; Package automake. (Thu, 03 May 2012 22:25:02 GMT) Full text and rfc822 format available.

Message #14 received at 11377 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>
Cc: 11377 <at> debbugs.gnu.org, Stefano Lattarini <stefano.lattarini <at> gmail.com>
Subject: Re: bug#11377: configure.am - fails to remove configure before
	attempting replacement
Date: Thu, 03 May 2012 16:22:31 -0600
[Message part 1 (text/plain, inline)]
On 05/03/2012 04:09 PM, Ronald F. Guilmette wrote:
> 
> The only problem with the approach outlined above is that sometimes (rarely)
> some Makefiles and/or other tools used as part of a build process (e.g.
> automake) may occasionally make the assumption that every regular file
> within the current (build) tree is writable, and that thus, when one of these
> tools is about to perform some operation that will attempt to effectively
> over-write some such file, the over-write can be performed _without_ first
> deleting the file in question.

Nothing against your style of coding, but I do want to point out that
recommending anyone delete a file before rebuilding its contents is
racy.  If things crash in the middle, you can be left without the file,
or with a file with incomplete contents.  It is always better to
recommend a style of atomic changes - make your target file in a
temporary location (generally in the same directory as the destination),
then use an atomic rename() to overwrite the original file with the
updated file in one go.

> 
> I want to reemphasize what I just said.  Removing a given regular file before
> attempting some operation that is intended to fully replace the contents of
> that file is a simple and altogether reasonable precaution... 

I want to reemphasize what I just said - removing a given regular file
before rebuilding it is prone to awful surprises when your build process
is interrupted unexpectedly.  Always write to a temporary name and then
use mv to move it into place, for the best safety.

-- 
Eric Blake   eblake <at> redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Information forwarded to bug-automake <at> gnu.org:
bug#11377; Package automake. (Thu, 03 May 2012 23:50:02 GMT) Full text and rfc822 format available.

Message #17 received at 11377 <at> debbugs.gnu.org (full text, mbox):

From: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>
To: Eric Blake <eblake <at> redhat.com>
Cc: 11377 <at> debbugs.gnu.org, Stefano Lattarini <stefano.lattarini <at> gmail.com>
Subject: Re: bug#11377: configure.am - fails to remove configure before
	attempting replacement
Date: Thu, 03 May 2012 16:47:55 -0700
In message <4FA30527.8070709 <at> redhat.com>, you wrote:

>Nothing against your style of coding, but I do want to point out that
>recommending anyone delete a file before rebuilding its contents is
>racy.

I respectfully disagree.

>If things crash in the middle, you can be left without the file,

Yes.  So?  You still have the Makefile, which contains all of the instructions
for re-making the file in question.

>or with a file with incomplete contents.

Could you please explain, for my edification, how inserting an initial `rm'
would affect this possibility in any way whatsoever?  (Some examples might
help to illustrate.)

I'm not even sure how one would generate a file with "incomplete contents",
let alone how that process might or might not be in any way affected by an
initial "rm target".

>It is always better to
>recommend a style of atomic changes - make your target file in a
>temporary location (generally in the same directory as the destination),
>then use an atomic rename() to overwrite the original file with the
>updated file in one go.

Again, I cannot envision any circumstance in which what you are suggesting
would be of any benefit, in practice.  Perhaps you can provide some examples
to illustrate your point.

In any event, the style of Makefile coding you are suggesting, wherein the
final command to remake a given target would be something like:

	mv tempfile target

is not, I would suggest, at all widespread in current practice.  In short,
even if you could make the case that this is the Best Way To Do Things...
a possibility of which I personally am still dubious... in order to get your
wish, I do believe that you would have to sally forth and convince the
developers of 99.9% of all existing Makefile to rewrite the millions of
Makefiles they maintain in order to conform to the new coding standard
you are proposing.  (Not a very practical idea, in practice.)

>I want to reemphasize what I just said - removing a given regular file
>before rebuilding it is prone to awful surprises when your build process
>is interrupted unexpectedly.

I look forward to reviewing examples that would illustrate your point.


Regards,
rfg


P.S.  Your suggestion that removing & replacing the contents of a given target
file only at the very last second (and only in an atomic operation) is in some
sense "best practice" assumes that something would, in fact, go wrong if
the given target file were to be removed sooner, before being replaced.

I cannot entirely discount the possibility that in some exceptionally rare
and obscure cases, target removal prior to replacement might result in some
unwanted effects... although I would still like to see some real examples
before reaching any final conclusion on that point.  However I do believe
that in all of the existing correctly coded GNU-style Makefiles where such
an issue might arise, the Makefiles in question will already contain a
dependency rule for the special .PRECIOUS target.

My only point is that in actual practice, use of GNU Make's special .PRECIOUS
target is in fact exceedingly rare.  I would venture to guess that no more
than one tenth of one percent of all GNU-style Makefiles ever written have
mentioned .PRECIOUS (and even fewer use this target correctly).  And based
upon the rarity of cases where the .PRECIOUS target has been found to be
either useful or necessary, I think that any assertion of the kind you have
made (i.e. that removal of target files in their recipies _before_ they are
actually replaced is either always or frequently inadvisable) is, by and
large, unfounded.  If target removal before replacement always or frequently 
led to trouble, then virtually all GNU-style Makefiles would contain a rule
for the .PRECIOUS target.  The fact that so few do, in practice, tends to
support the view that target removal separate from and prior to replacement
only causes difficulties in exceedingly rare circumstances.  (But regardless
of how rare or how common such difficulties may be for Makefile targets in
general, I think that it is provable that no such difficulties arise in
the specific case of the $(top_srcdir)/configure target, specifically,
which is the only one that my specific change request pertained to.)





Information forwarded to bug-automake <at> gnu.org:
bug#11377; Package automake. (Fri, 04 May 2012 11:59:01 GMT) Full text and rfc822 format available.

Message #20 received at 11377 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>
Cc: 11377 <at> debbugs.gnu.org, Stefano Lattarini <stefano.lattarini <at> gmail.com>
Subject: Re: bug#11377: configure.am - fails to remove configure before
	attempting replacement
Date: Fri, 04 May 2012 05:56:40 -0600
[Message part 1 (text/plain, inline)]
On 05/03/2012 05:47 PM, Ronald F. Guilmette wrote:
> In message <4FA30527.8070709 <at> redhat.com>, you wrote:
> 
>> Nothing against your style of coding, but I do want to point out that
>> recommending anyone delete a file before rebuilding its contents is
>> racy.
> 
> I respectfully disagree.
> 
>> If things crash in the middle, you can be left without the file,
> 
> Yes.  So?  You still have the Makefile, which contains all of the instructions
> for re-making the file in question.
> 
>> or with a file with incomplete contents.
> 
> Could you please explain, for my edification, how inserting an initial `rm'
> would affect this possibility in any way whatsoever?  (Some examples might
> help to illustrate.)

See, for example, this autoconf patch:
http://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=e9cceec

Autoconf was generating code that created a cache file by writing
directly in to the target, and there were actual examples of failure in
the wild where it broke because of a partially written cache file with
incomplete contents and a current timestamp would not be regenerated to
be complete.  In fact, the problem was hit in practice so often that gcc
had resorted to a per-directory config.cache just to work around the
worst aspects of this failure mode.

We switched things over to writing to a temporary file and atomically
mv'ing it into place, and the bug disappeared.

> 
> I'm not even sure how one would generate a file with "incomplete contents",
> let alone how that process might or might not be in any way affected by an
> initial "rm target".

Any time you hit Ctrl-C in the middle of a make run, you might be
interrupting a process that is in the middle of writing contents to an
output file.

> In any event, the style of Makefile coding you are suggesting, wherein the
> final command to remake a given target would be something like:
> 
> 	mv tempfile target
> 
> is not, I would suggest, at all widespread in current practice.

Agreed, but that doesn't mean that we shouldn't advertise it as being
safer than rm.

> 
> P.S.  Your suggestion that removing & replacing the contents of a given target
> file only at the very last second (and only in an atomic operation) is in some
> sense "best practice" assumes that something would, in fact, go wrong if
> the given target file were to be removed sooner, before being replaced.

Yes - in the case of autoconf's config.cache, things DO go wrong in a
subsequent build if config.cache is removed too soon (a partial file can
corrupt the build, and a missing file costs extra time to rebuild the file).

-- 
Eric Blake   eblake <at> redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Information forwarded to bug-automake <at> gnu.org:
bug#11377; Package automake. (Mon, 07 May 2012 21:10:02 GMT) Full text and rfc822 format available.

Message #23 received at 11377 <at> debbugs.gnu.org (full text, mbox):

From: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>
To: Eric Blake <eblake <at> redhat.com>
Cc: 11377 <at> debbugs.gnu.org, Stefano Lattarini <stefano.lattarini <at> gmail.com>
Subject: Re: bug#11377: configure.am - fails to remove configure before
	attempting replacement
Date: Mon, 07 May 2012 14:07:13 -0700
My apologies for the tardy reply.  I have been off working on other bugs
elsewhere.

In message <4FA3C3F8.1010605 <at> redhat.com>, Eric Blake <eblake <at> redhat.com> wrote:

>On 05/03/2012 05:47 PM, Ronald F. Guilmette wrote:
>> In message <4FA30527.8070709 <at> redhat.com>, you wrote:
>>..
>>> If things crash in the middle, you can be left without the file,
>>
>> Yes.  So?  You still have the Makefile, which contains all of the instructions
>> for re-making the file in question.
>>
>>> or with a file with incomplete contents.
>>
>> Could you please explain, for my edification, how inserting an initial `rm'
>> would affect this possibility in any way whatsoever?  (Some examples might
>> help to illustrate.)
>
>See, for example, this autoconf patch:
>http://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=3De9cceec
>
>Autoconf was generating code that created a cache file by writing
>directly in to the target, and there were actual examples of failure in
>the wild where it broke because of a partially written cache file with
>incomplete contents and a current timestamp would not be regenerated to
>be complete.  In fact, the problem was hit in practice so often that gcc
>had resorted to a per-directory config.cache just to work around the
>worst aspects of this failure mode.

OK, so sometimes Makefile target files can get only partially (and
inadequattely) written.  Granted.

However you still haven't demonstrated how an initial `rm' would have
made any difference at all to that specific problem, one way other
the other.  (It doesn't make the problem worse and it doesn't make
the problem better either.  It has no effect at all on the problem
you cite.)  Neither have you provided any example in which an initial
`rm' at the start of the commands recipie to rebuild the target file
would in fact have any deliterious effects whatsoever.

In short, your assertion that doing an initial `rm' would be somehow a
Bad Thing, on account of the potential ``partially written targets''
file problem simply does not stand up to scrutiny.  This potential for
partially-written target files exists and continues to exist entirely
independently of whether or not the rules to rebuild any given target
file include an initial `rm' of that target file.

>We switched things over to writing to a temporary file and atomically
>mv'ing it into place, and the bug disappeared.

That's swell, and that that is certainly one way of eliminating the
your specific ``partially written targets'' issue.  (I can think of a
few other obvious solutions, including creating a .stamp file after
your actual target file has been completely updated... the way Stallman
was doing in the GCC sources, 20+ years ago.)

The important point remains however:  My suggestion for doing an
initial `rm' on certain select files created as a consequence of using
automake has no bearing on, and is entirely irrelevant and orthogonal
to the ``partially written targets'' problem... and vise versa.  Doing
(or not doing) an initial `rm' for a Makefile target file neither creates
that problem, nor solves that problem.  As I say, it is entirely
irrelevant and orthogonal to that entirely different problem.

>> I'm not even sure how one would generate a file with "incomplete contents",

Please excuse me.  That was a downright stupid thing for me to have said.
Of course I know how a partially-written target file might come about,
and I also have seen such things, on occasions, in practice.  Really, I
think that I was just trying to elicit real examples from you in order
to allow me some concrete basis to make my point, i.e. that an initial
`rm' of a Makefile target file is _never_ harmful... well... except in
those few and extremely rare cases associated with use of the .PRECIOUS
pseudo-target.  (But those are the exceptions that prove the rule... an
initial `rm -f' in the recipies for almost all Makefile targets is, in
general and on balance, a Good Thing, leading to higher reliability of
Makefiles in general.)

>> In any event, the style of Makefile coding you are suggesting, wherein
>> the final command to remake a given target would be something like:
>>
>> 	mv tempfile target
>>
>> is not, I would suggest, at all widespread in current practice.
>
>Agreed, but that doesn't mean that we shouldn't advertise it as being
>safer than rm.

See above.  The two possibilities are separate and orthogonal.

Any given target file may or may not be subjected to an initial `rm -f'
at the start of the recipie to rebuild it.  (And as I have tried to point
out, in general it is Better to including such an initial `rm -f' in
essentially _every_ case I can think of.)

Separately and independently, some subset of all Makefile target files
are generated in such a way that they are prone to being only partially
re-written, e.g. if make is interrupted by cntl-C.  (Does this in-
clude even essentially all .o files??  One would think so.)  This is a
separate and unrelated problem.  In some few cases it becomes bad enough
to pay attention to (and to do something about) e.g. as in your autoconf
cache file example.  In those cases, one obvious solution... although
certainly not the only one... is to generate the desired file contents
first into a temporary file and then... ``atomically''... perform a
`mv' to move the completely built file contents to the actual desired
target file name.

These two possibilities are not in any way conflicting.  One can write
the recipie to rebuild a given target such that it contains an initial
`rm' (or not) and also so that it contains a final `mv' (or not).  These
possibilities are not mutually exclusive.

>> P.S.  Your suggestion that removing & replacing the contents of a given target
>> file only at the very last second (and only in an atomic operation) is in some
>> sense "best practice" assumes that something would, in fact, go wrong if
>> the given target file were to be removed sooner, before being replaced.=
>
>Yes - in the case of autoconf's config.cache, things DO go wrong in a
>subsequent build if config.cache is removed too soon (a partial file can
>corrupt the build,

See above.  To the best of my knowledge, the `rm' command is _never_
responsible for creating partially/incompletely written target files.

> and a missing file costs extra time to rebuild the file).

Oh come come now!  You are seriously grasping at straws to make your point
now.

And in fact, your (already weak) argument that doing an `rm' too soon on
a given target file would incur some (tiny?) subsequent processing time
penalty is provably just plain wrong.  Consider a simple example:


	foo: bar
		-rm -f foo
		...

Your claim is that execution of the initial `rm -f' of foo might result 
in foo having to be rebuilt at some later time (e.g. if the make is
interrupted after the `rm -f' and then restrarted later on) and that
this creates some unnecessary processing overhead.

But consider this... If in fact that `rm -f' line had indeed been executed
then isn't it the case that that could ONLY have occured because make,
together with the rules in the Makefile, together with the current state
of the local environment had _already_ determined that foo needed to be
rebuilt anyway?

In other words, you seem to be arguing that the initial `rm -f' might
possibly result in some processing overhead, in certain circumstances
(e.g. ^C) to rebuild `foo', but that argument does not seem to hold water
because it was _already_ the case that foo needed to be rebuilt anyway.
Thus these is no extra or additional overhead created by the presence of
that initial `rm -f'.


Regards,
rfg




Added tag(s) moreinfo. Request was from Stefano Lattarini <stefano.lattarini <at> gmail.com> to control <at> debbugs.gnu.org. (Sun, 22 Jul 2012 16:00:02 GMT) Full text and rfc822 format available.

Severity set to 'minor' from 'normal' Request was from Stefano Lattarini <stefano.lattarini <at> gmail.com> to control <at> debbugs.gnu.org. (Sun, 22 Jul 2012 16:00:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-automake <at> gnu.org:
bug#11377; Package automake. (Thu, 26 Dec 2013 16:16:03 GMT) Full text and rfc822 format available.

Message #30 received at 11377 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: "Ronald F. Guilmette" <rfg <at> tristatelogic.com>
Cc: 11377 <at> debbugs.gnu.org, Eric Blake <eblake <at> redhat.com>
Subject: Re: configure.am - fails to remove configure before attempting
 replacement
Date: Thu, 26 Dec 2013 17:15:12 +0100
tags 11377 + wontfix
close 11377
stop

Reference:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11377>

No more activity on this wishlist and controversial bug for
an year and a half.  I'm thus closing it to reduce the clutter
in our bug tracker.

Regards,
  Stefano




Added tag(s) wontfix. Request was from Stefano Lattarini <stefano.lattarini <at> gmail.com> to control <at> debbugs.gnu.org. (Thu, 26 Dec 2013 16:16:04 GMT) Full text and rfc822 format available.

bug closed, send any further explanations to 11377 <at> debbugs.gnu.org and "Ronald F. Guilmette" <rfg <at> tristatelogic.com> Request was from Stefano Lattarini <stefano.lattarini <at> gmail.com> to control <at> debbugs.gnu.org. (Thu, 26 Dec 2013 16:16:04 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 24 Jan 2014 12:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 11 years and 200 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.