GNU bug report logs - #9106
24.0.50; ./configure causes massive recompilation

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 9106 in the body.
You can then email your comments to 9106 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.0.50; ./configure causes massive recompilation
Date: Sun, 17 Jul 2011 01:30:36 -0400

Invocation of the `configure' script causes recompilation of many
source files, even though nothing has really changed.

It looks like the cause is lib/Makefile which states that several
generated headers in that directory depend on config.status.  So each
`configure' causes those headers to be regenerated, which in turn
triggers many files using those headers to be recompiled.

This is annoying, as building Emacs even on a modern system takes a
significant amount of time (about 3 minutes on this box, whose details
see below).  Can this annoyance be removed, please?


In GNU Emacs 24.0.50.22 (x86_64-unknown-linux-gnu, GTK+ Version 2.20.1)
 of 2011-07-17 on fencepost
configured using `configure  '--with-gif=no' '--with-tiff=no''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: RMAIL

Minor modes in effect:
  display-time-mode: t
  show-paren-mode: t
  savehist-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
B ESC O A ESC O A ESC O A C-x C-f C h a n g TAB RET 
ESC [ 5 ~ ESC [ 5 ~ ESC [ 6 ~ ESC [ 6 ~ ESC [ 6 ~ ESC 
[ 6 ~ ESC O B ESC O B ESC O B ESC O B ESC O B ESC O 
B ESC O B ESC O B ESC O B ESC O B ESC O B ESC O B ESC 
O C ESC O C ESC O C ESC O C ESC O C ESC O C ESC O C 
ESC O C ESC O C ESC O C ESC O C ESC O C ESC O C ESC 
O C ESC O C ESC O C ESC O C ESC O C ESC O C ESC O C 
ESC O C ESC O C ESC O C ESC O C ESC O C ESC O C ESC 
O C ESC O C ESC O C ESC O B ESC O B ESC O B ESC O B 
ESC O B ESC O B ESC O B ESC O B ESC O B ESC O B ESC 
O B ESC O B ESC O B ESC O B ESC O B ESC O A ESC O A 
ESC O A ESC O A C-x b I N B TAB RET C-u g m a i l . 
n e w RET m e m a c s - d e v e l @ g n u . o r g DEL 
DEL DEL DEL DEL DEL DEL DEL SPC DEL @ g n u . o r g 
ESC O B ESC ~ C-x 5 o ESC x r e p o r t - e m TAB 
RET

Recent messages:
scroll-down-command: Beginning of buffer [2 times]
Getting mail from /srv/data/home/e/eliz/mail.new...
Counting new messages...
Counting messages...60
Counting new messages...done (70)
Saving file /home/e/eliz/INBOX...
Wrote /home/e/eliz/INBOX [2 times]
Computing summary lines...done
70 new messages read
Modification-flag cleared

Load-path shadows:
None found.

Features:
(shadow emacsbug sendmail flyspell ispell add-log vc-bzr cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs regexp-opt qp rmailsum rmailmm message format-spec
rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode mailabbrev
gmm-utils mailheader mail-parse rfc2231 rmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils time paren cus-start cus-load
time-date savehist saveplace tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image fringe
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process
dynamic-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

Message #8 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 18 Jul 2011 13:48:45 -0400

Eli Zaretskii wrote:

> Invocation of the `configure' script causes recompilation of many
> source files, even though nothing has really changed.

Why do you want to re-run configure - can you get away with just `make'?
In principle, almost anything could have changed if you have re-run
configure (?). Eg you could in principle be compiling for a different
arch now. I'm not sure it's possible to distinguish those cases from
cases where nothing has really changed.

> It looks like the cause is lib/Makefile which states that several
> generated headers in that directory depend on config.status.  So each
> `configure' causes those headers to be regenerated, which in turn
> triggers many files using those headers to be recompiled.

Message #11 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Jan Djärv <jan.h.d <at> swipnet.se>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 18 Jul 2011 20:20:19 +0200

Glenn Morris skrev 2011-07-18 19.48:
> Eli Zaretskii wrote:
>
>> Invocation of the `configure' script causes recompilation of many
>> source files, even though nothing has really changed.
>
> Why do you want to re-run configure - can you get away with just `make'?
> In principle, almost anything could have changed if you have re-run
> configure (?). Eg you could in principle be compiling for a different
> arch now. I'm not sure it's possible to distinguish those cases from
> cases where nothing has really changed.

'make' after a bzr upd often runs configure nowdays.  More often than not.

	Jan D.

>
>> It looks like the cause is lib/Makefile which states that several
>> generated headers in that directory depend on config.status.  So each
>> `configure' causes those headers to be regenerated, which in turn
>> triggers many files using those headers to be recompiled.
>
>

Message #14 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Jan Djärv <jan.h.d <at> swipnet.se>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 18 Jul 2011 14:37:04 -0400

Jan Djärv wrote:

> 'make' after a bzr upd often runs configure nowdays.  More often than not.

Then I guess the prerequisites of configure (the various Makefile.in's
etc) must be being updated a lot right now. I don't see a way round
this. Re-running configure is The Right Thing.

Message #17 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 18 Jul 2011 21:37:02 +0300

> From: Glenn Morris <rgm <at> gnu.org>
> Cc: 9106 <at> debbugs.gnu.org
> Date: Mon, 18 Jul 2011 13:48:45 -0400
> 
> Eli Zaretskii wrote:
> 
> > Invocation of the `configure' script causes recompilation of many
> > source files, even though nothing has really changed.
> 
> Why do you want to re-run configure - can you get away with just `make'?

How can I know?  I just did a "bzr up", and I don't want to analyze
every file that got updated to see if a mere "make" will do.  E.g.,
what if some Makefile.in got updated?

It used to be the case that if the results of running `configure'
didn't change anything of essence, "make" would do nothing.  This
worked by producing the generated files under temporary names and by
using move-if-change to overwrite the old files if the new ones were
different.  Why cannot we extend this method to the additional files
we generate now?

> In principle, almost anything could have changed if you have re-run
> configure (?). Eg you could in principle be compiling for a different
> arch now. I'm not sure it's possible to distinguish those cases from
> cases where nothing has really changed.

Of course, it's possible: several files, such as src/config.h, will be
different.  We just need to compare them before we overwrite the old
ones with new.

Message #20 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org, jan.h.d <at> swipnet.se
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 18 Jul 2011 21:58:40 +0300

> From: Glenn Morris <rgm <at> gnu.org>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  9106 <at> debbugs.gnu.org
> Date: Mon, 18 Jul 2011 14:37:04 -0400
> 
> Jan Djärv wrote:
> 
> > 'make' after a bzr upd often runs configure nowdays.  More often than not.
> 
> Then I guess the prerequisites of configure (the various Makefile.in's
> etc) must be being updated a lot right now. I don't see a way round
> this. Re-running configure is The Right Thing.

I have nothing against re-running configure.  I just don't want it
unnecessarily compiling things.

Message #23 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 9106 <at> debbugs.gnu.org
Cc: bug-gnulib <bug-gnulib <at> gnu.org>
Subject: Re: 24.0.50; ./configure causes massive recompilation
Date: Tue, 19 Jul 2011 17:39:38 -0700

[cc'ing bug-gnulib as it's related; see <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9106>]

> It used to be the case that if the results of running `configure'
> didn't change anything of essence, "make" would do nothing.  This
> worked by producing the generated files under temporary names and by
> using move-if-change to overwrite the old files if the new ones were
> different.

If memory serves, that process is pretty error-prone.  One can't
simply use move-if-change: one needs a separate time stamp file for
each file that one is doing the move-if-change trick with.  Otherwise,
when you run 'make' again, it will cheerfully regenerate all the .h
files again.  And with the time stamp files, one runs into problems
where the time stamp files are out of sync with reality.

I'm not saying it can't be done, but it would be a pain to have it
done without losing reliability during the build.  Part of the problem
is deciding automatically whether a change is one "of essence".

Message #26 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 01:24:44 -0400

> Date: Tue, 19 Jul 2011 17:39:38 -0700
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> Cc: bug-gnulib <bug-gnulib <at> gnu.org>
> 
> [cc'ing bug-gnulib as it's related; see <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9106>]
> 
> > It used to be the case that if the results of running `configure'
> > didn't change anything of essence, "make" would do nothing.  This
> > worked by producing the generated files under temporary names and by
> > using move-if-change to overwrite the old files if the new ones were
> > different.
> 
> If memory serves, that process is pretty error-prone.  One can't
> simply use move-if-change: one needs a separate time stamp file for
> each file that one is doing the move-if-change trick with.  Otherwise,
> when you run 'make' again, it will cheerfully regenerate all the .h
> files again.

Sorry, I don't see the difficulty.  Perhaps I'm missing something.

The current recipe for producing, e.g., unistd.h from unistd.in.h is
this:

  unistd.h: unistd.in.h $(top_builddir)/config.status $(CXXDEFS_H) $(ARG_NONNULL_H) $(WARN_ON_USE_H)
	  $(AM_V_GEN)rm -f $@-t $@ && \
	  { echo '/* DO NOT EDIT! GENERATED AUTOMATICALLY! */'; \
	    sed -e 's|@''GUARD_PREFIX''@|GL|g' \
	    [...]
		-e '/definition of _GL_WARN_ON_USE/r $(WARN_ON_USE_H)'; \
	  } > $@-t && \
	  mv $@-t $@

What I'm suggesting is to replace the last command ("mv $@-t $@") with
this:

    move-if-change $@-t $@

That's it.  Make will indeed cheerfully regenerate unistd.h-t, but as
long as that file isn't copied over unistd.h, the source files that
include unistd.h won't be recompiled.  Regeneration of unistd.h-t is
very fast; it's the needless recompilation of the plethora of source
files that include unistd.h that is the problem addressed by this bug
report.

It could be the case that some configure.in wizardry would resolve
this even nicer, by doing a similar move-if-change trick with
config.status (whose being a prerequisite of these header files is the
trigger for their regeneration, IIUC).  That will prevent even the
regeneration itself.  But I don't know if this is possible without too
much effort, so the suggested simpler "band-aid" is good enough for
me.

> Part of the problem is deciding automatically whether a change is
> one "of essence".

I think comparing the old file with the new one, like move-if-change
does, is all that's needed.  There's no requirement to detect changes
that are non-essential, like comments etc. -- if any change is
detected, let the files be recompiled.  Am I missing something?

Message #29 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Tue, 19 Jul 2011 22:58:48 -0700

On 07/19/2011 10:24 PM, Eli Zaretskii wrote:
> What I'm suggesting is to replace the last command ("mv $@-t $@") with
> this:
>
>      move-if-change $@-t $@
>
> That's it.  Make will indeed cheerfully regenerate unistd.h-t

... and alloca.h-t.  And getopt.h-t.  And the other ten .h-t files
that are generated on typical platforms.  And this would occur
every time one does a 'make', even when there's no real work
to do.

The unnecessary "make" actions would fill up people's screens,
and would be confusing.  I'm afraid this cure would be worse
than the disease.

Message #32 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 02:29:38 -0400

> Date: Tue, 19 Jul 2011 22:58:48 -0700
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> CC: 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org
> 
> On 07/19/2011 10:24 PM, Eli Zaretskii wrote:
> > What I'm suggesting is to replace the last command ("mv $@-t $@") with
> > this:
> >
> >      move-if-change $@-t $@
> >
> > That's it.  Make will indeed cheerfully regenerate unistd.h-t
> 
> ... and alloca.h-t.  And getopt.h-t.  And the other ten .h-t files
> that are generated on typical platforms.

Yes.

> And this would occur every time one does a 'make', even when there's
> no real work to do.

This occurs already: these headers are regenerated every time I re-run
the `configure' script.  How is my suggestion worse than the current
situation?

> The unnecessary "make" actions would fill up people's screens,
> and would be confusing.

They fill up my screen already, as things are now.

> I'm afraid this cure would be worse than the disease.

I feel there's some kind of misunderstanding here, because with my
proposal, nothing will happen that doesn't already happen.  Perhaps
you could show in more detail which Make actions would happen that
doesn't happen now.

Message #35 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: eggert <at> cs.ucla.edu
Cc: 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 02:38:24 -0400

> Date: Wed, 20 Jul 2011 02:29:38 -0400
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org
> 
> > The unnecessary "make" actions would fill up people's screens,
> > and would be confusing.
> 
> They fill up my screen already, as things are now.
> 
> > I'm afraid this cure would be worse than the disease.
> 
> I feel there's some kind of misunderstanding here, because with my
> proposal, nothing will happen that doesn't already happen.  Perhaps
> you could show in more detail which Make actions would happen that
> doesn't happen now.

Perhaps you thought that a mere "make", even without re-running
`configure', will trigger these rules.  But that is not the case: as
long as config.status is not updated, these rules will not be
triggered, since unistd.h etc. will always be newer than the
corresponding *.in.h templates, due to the fact that move-if-change
_will_ overwrite them with newer versions whenever there's a real
change in the *.in.h templates.

Am I missing something?

Message #38 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>, bug-gnulib <at> gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 08:44:04 +0200

Hello,

* Eli Zaretskii wrote on Wed, Jul 20, 2011 at 08:29:38AM CEST:
> > From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> > And this would occur every time one does a 'make', even when there's
> > no real work to do.
> 
> This occurs already: these headers are regenerated every time I re-run
> the `configure' script.

Yes, but 'make' is run a lot more often than 'configure' in some
workflows.  I think one implicit gnulib expectation is that configure
is not run often.  (FWIW I don't agree, because if your source tree
is several times larger than your gnulib extract, then it really matters
whether you need to rebuild the world even rarely.)

I proposed using stamp files before, but there are (understandable)
reservations against them.

Cheers,
Ralf

Message #41 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu, bug-gnulib <at> gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 08:46:23 +0200

* Eli Zaretskii wrote on Wed, Jul 20, 2011 at 08:38:24AM CEST:
> > > I'm afraid this cure would be worse than the disease.
> > 
> > I feel there's some kind of misunderstanding here, because with my
> > proposal, nothing will happen that doesn't already happen.  Perhaps
> > you could show in more detail which Make actions would happen that
> > doesn't happen now.
> 
> Perhaps you thought that a mere "make", even without re-running
> `configure', will trigger these rules.  But that is not the case: as
> long as config.status is not updated, these rules will not be
> triggered, since unistd.h etc. will always be newer than the
> corresponding *.in.h templates, due to the fact that move-if-change
> _will_ overwrite them with newer versions whenever there's a real
> change in the *.in.h templates.
> 
> Am I missing something?

I think you are.  Once config.status is updated, the .h files' rules
are triggered, but since move-if-change never updates identical outputs
they will be triggered every time from then on.

You need a separate stamp file to avoid this.

Cheers,
Ralf

Message #44 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
Cc: 9106 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu, bug-gnulib <at> gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 04:48:35 -0400

> Cc: eggert <at> cs.ucla.edu, 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org
> Date: Wed, 20 Jul 2011 08:46:23 +0200
> From: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
> 
> I think you are.  Once config.status is updated, the .h files' rules
> are triggered, but since move-if-change never updates identical outputs
> they will be triggered every time from then on.
> 
> You need a separate stamp file to avoid this.

Or use move-if-change with config.status.

Message #47 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, bug-gnulib <at> gnu.org,
	9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 10:23:32 -0700

On 07/20/11 01:48, Eli Zaretskii wrote:
>> You need a separate stamp file to avoid this.
> Or use move-if-change with config.status.

That might be better, but it'd need a separate timestamp file, no?
Otherwise, config.status would appear out-of-date to the
top-level rule that runs 'configure', and that would cause 'make'
to run 'configure' every time.

Message #50 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Ralf.Wildenhues <at> gmx.de, bug-gnulib <at> gnu.org, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 20:51:08 +0300

> Date: Wed, 20 Jul 2011 10:23:32 -0700
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> CC: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, 9106 <at> debbugs.gnu.org, 
>  bug-gnulib <at> gnu.org
> 
> On 07/20/11 01:48, Eli Zaretskii wrote:
> >> You need a separate stamp file to avoid this.
> > Or use move-if-change with config.status.
> 
> That might be better, but it'd need a separate timestamp file, no?

Probably, sigh.  (I hate Makefile's that run configure for me.)

Message #53 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>, bug-gnulib <at> gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Wed, 20 Jul 2011 23:04:17 +0200

* Eli Zaretskii wrote on Wed, Jul 20, 2011 at 07:51:08PM CEST:
> > Date: Wed, 20 Jul 2011 10:23:32 -0700
> > From: Paul Eggert <eggert <at> cs.ucla.edu>
> > CC: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, 9106 <at> debbugs.gnu.org, 
> >  bug-gnulib <at> gnu.org
> > 
> > On 07/20/11 01:48, Eli Zaretskii wrote:
> > >> You need a separate stamp file to avoid this.
> > > Or use move-if-change with config.status.
> > 
> > That might be better, but it'd need a separate timestamp file, no?
> 
> Probably, sigh.  (I hate Makefile's that run configure for me.)

I guess I don't understand why everyone hates stamp files.[1]
The config.h rule (among others) has been using one for years,
and the last time I've heard complaints or bug reports about it
has been years also.

Cheers,
Ralf

[1] Of course they're not clean and pure and all that, but hey,
you wouldn't be using autoconf in the first place if you cared.

Message #56 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gnulib <at> gnu.org
Cc: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, Paul Eggert <eggert <at> cs.ucla.edu>,
	9106 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Thu, 21 Jul 2011 12:57:44 +0200

Hi Ralf,

> I guess I don't understand why everyone hates stamp files.

I don't like them either [1], from past experience.

Maybe we need to look at the operations that are hurt by stamp files:

  - Building distributions. If stamp files are included in a tarball,
    then merely unpacking the tarball (with 'cpio', not 'tar') or
    copying it (with 'cp -r', not 'cp -a') sets the modification times
    of all files, and the modification time of the stamp may end up
    being a little bit earlier than the one of the main file.

  - Building on a NFS mounted file system, with a time shift between
    the server and the client. The problem here was that "echo >> foo"
    and "touch foo" assign different time stamps to the file 'foo'.

  - Removing or touching the main file by hand must cause a rebuild.
    In some variants of the stamp rules, you also had to remove or
    touch the stamp file in order to get "make" do something.

  - "make -n" ends up rebuilding things, while the developer does not
    want "make -n" to do anything.

  - Or, "make -n" displays more or less statements than "make" will
    actually execute. So "make -n" becomes unreliable.

Paul, Jim, Eric, others, do you remember other problems of stamp files?

> The config.h rule (among others) has been using one for years,
> and the last time I've heard complaints or bug reports about it
> has been years also.

For reference, here's the rules automake generates for config.h:

config.h: stamp-h1
        @if test ! -f $@; then \
          rm -f stamp-h1; \
          $(MAKE) $(AM_MAKEFLAGS) stamp-h1; \
        else :; fi

stamp-h1: $(srcdir)/config.h.in $(top_builddir)/config.status
        @rm -f stamp-h1
        cd $(top_builddir) && $(SHELL) ./config.status config.h

$(srcdir)/config.h.in:  $(am__configure_deps)
        ($(am__cd) $(top_srcdir) && $(AUTOHEADER))
        rm -f stamp-h1
        touch $@

distclean-hdr:
        -rm -f config.h stamp-h1

Is that the kind of rule you would recommend?

Bruno

[1] http://lists.gnu.org/archive/html/bug-gnulib/2011-04/msg00045.html
-- 
In memoriam Ludwig Beck <http://en.wikipedia.org/wiki/Ludwig_Beck>

Message #59 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9106 <at> debbugs.gnu.org, Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>,
	bug-gnulib <at> gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Thu, 21 Jul 2011 13:59:21 -0700

On 07/21/11 03:57, Bruno Haible wrote:
> Is that the kind of rule you would recommend?

Yes, something like that might work, for config.status in Emacs.
But I'd rather not debug this sort of thing myself.
Like you, I'm leery of time stamp files; too often
their costs outweigh their benefits.

Message #62 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 9106 <at> debbugs.gnu.org, Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>,
	bug-gnulib <at> gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Thu, 21 Jul 2011 23:27:26 +0200

Paul Eggert wrote:
> Like you, I'm leery of time stamp files; too often
> their costs outweigh their benefits.

What are, concretely, the problems you are fearing, or that you remember
from the past? In other words, which are the tests that we should perform
before committing a change that makes use of stamp files?

Bruno
-- 
In memoriam Ludwig Beck <http://en.wikipedia.org/wiki/Ludwig_Beck>

Message #65 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Thu, 21 Jul 2011 15:00:50 -0700

On 07/21/11 14:27, Bruno Haible wrote:
> What are, concretely, the problems you are fearing, or that you remember
> from the past? In other words, which are the tests that we should perform
> before committing a change that makes use of stamp files?

Your earlier message summarized the problems that I remember.
(Perhaps I've forgotten some.)  I'm afraid that the only reliable
way to test changes in this area is to have many Emacs developers
and installers, who use different styles, build Emacs and report
the problems that they see.  This would be a reasonable thing to
do, if the benefit is large enough relative to the cost (something
that's not clear to me).

Message #68 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 9106 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>, bug-gnulib <at> gnu.org,
	Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Fri, 22 Jul 2011 00:27:00 +0200

Bruno Haible wrote:
> Paul, Jim, Eric, others, do you remember other problems of stamp files?
>
>> The config.h rule (among others) has been using one for years,
>> and the last time I've heard complaints or bug reports about it
>> has been years also.
>
> For reference, here's the rules automake generates for config.h:
>
> config.h: stamp-h1
>         @if test ! -f $@; then \
>           rm -f stamp-h1; \
>           $(MAKE) $(AM_MAKEFLAGS) stamp-h1; \
>         else :; fi
>
> stamp-h1: $(srcdir)/config.h.in $(top_builddir)/config.status
>         @rm -f stamp-h1
>         cd $(top_builddir) && $(SHELL) ./config.status config.h
>
> $(srcdir)/config.h.in:  $(am__configure_deps)
>         ($(am__cd) $(top_srcdir) && $(AUTOHEADER))
>         rm -f stamp-h1
>         touch $@
>
> distclean-hdr:
>         -rm -f config.h stamp-h1
>
> Is that the kind of rule you would recommend?

Hi Bruno,

I know of no problem with that time stamp mechanism.
It's been in use (complaint-free) for a very long time.

Message #71 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: bug-gnulib <at> gnu.org, 9106 <at> debbugs.gnu.org, bruno <at> clisp.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Fri, 22 Jul 2011 09:14:17 +0300

> Date: Thu, 21 Jul 2011 15:00:50 -0700
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> CC: 9106 <at> debbugs.gnu.org, bug-gnulib <at> gnu.org, 
>  Eli Zaretskii <eliz <at> gnu.org>
> 
> [...] if the benefit is large enough relative to the cost (something
> that's not clear to me).

People complain about bzr operations that take 15 seconds where git
takes 3, so I'm sure recovering 3 minutes of needless compilation will
be quite a win, at a cost of a couple of additional recipes in the
top-level Makefile.in.

Message #74 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bug-gnulib <at> gnu.org, 9106 <at> debbugs.gnu.org, bruno <at> clisp.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Fri, 22 Jul 2011 01:06:00 -0700

On 07/21/11 23:14, Eli Zaretskii wrote:
> I'm sure recovering 3 minutes of needless compilation will
> be quite a win, at a cost of a couple of additional recipes

If the only cost were two small make rules, that would indeed be a win.
I worry the cost will be larger than that.  But please feel free
to give it a try, and see.

Message #77 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org
Subject: Re: New build process?
Date: Thu, 28 Jul 2011 03:15:56 -0700

On 07/28/11 03:06, Eli Zaretskii wrote:
> It works, at least for me, thanks.  It also indirectly solves
> bug#9106, so it can be closed now.

You're welcome, but I don't see how the patch solves Bug#9106,
If one runs 'configure', surely a big recompilation is often
needed, even with that change.

I'll CC: this to 9106 <at> debbugs.gnu.org so that the bug-9106 issues
can be discussed there as needed.  For those reading bug 9106,
here's the thread and patch Eli is referring to:

http://lists.gnu.org/archive/html/emacs-devel/2011-07/msg01092.html

Message #80 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 25 Apr 2022 12:40:20 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> Invocation of the `configure' script causes recompilation of many
> source files, even though nothing has really changed.
>
> It looks like the cause is lib/Makefile which states that several
> generated headers in that directory depend on config.status.  So each
> `configure' causes those headers to be regenerated, which in turn
> triggers many files using those headers to be recompiled.
>
> This is annoying, as building Emacs even on a modern system takes a
> significant amount of time (about 3 minutes on this box, whose details
> see below).  Can this annoyance be removed, please?

Reading this bug thread, it seems that various options were suggested
(mostly around different move-if-changed solutions), but they all seemed
to have various problems.

I think that it's unlikely that a user runs configure twice in a row
without changing any options, so I don't personally see any problems
with re-compiling all the .c files after running configure, so I wonder
whether there's really anything to be done here.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #83 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 25 Apr 2022 14:34:12 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: 9106 <at> debbugs.gnu.org
> Date: Mon, 25 Apr 2022 12:40:20 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > Invocation of the `configure' script causes recompilation of many
> > source files, even though nothing has really changed.
> >
> > It looks like the cause is lib/Makefile which states that several
> > generated headers in that directory depend on config.status.  So each
> > `configure' causes those headers to be regenerated, which in turn
> > triggers many files using those headers to be recompiled.
> >
> > This is annoying, as building Emacs even on a modern system takes a
> > significant amount of time (about 3 minutes on this box, whose details
> > see below).  Can this annoyance be removed, please?
> 
> Reading this bug thread, it seems that various options were suggested
> (mostly around different move-if-changed solutions), but they all seemed
> to have various problems.
> 
> I think that it's unlikely that a user runs configure twice in a row
> without changing any options, so I don't personally see any problems
> with re-compiling all the .c files after running configure, so I wonder
> whether there's really anything to be done here.

I don't mind closing this as wontfix.

Message #86 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 25 Apr 2022 14:10:30 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

> I don't mind closing this as wontfix.

OK; closing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #93 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 25 Apr 2022 15:42:00 +0200

>>>>> On Mon, 25 Apr 2022 14:10:30 +0200, Lars Ingebrigtsen <larsi <at> gnus.org> said:

    Lars> Eli Zaretskii <eliz <at> gnu.org> writes:
    >> I don't mind closing this as wontfix.

    Lars> OK; closing.

Hang on: what about

./configure
make
...configure gets run again, WTF?

Admittedly from a tree thatʼs been ./configure'd before (and I donʼt
really think I should need to run 'make clean' or 'make bootstrap'
here).

Robert
-- 

Message #96 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 25 Apr 2022 16:29:09 +0200

Robert Pluim <rpluim <at> gmail.com> writes:

> ./configure
> make
> ...configure gets run again, WTF?

Doesn't happen when I try that.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #99 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 25 Apr 2022 17:49:40 +0200

>>>>> On Mon, 25 Apr 2022 16:29:09 +0200, Lars Ingebrigtsen <larsi <at> gnus.org> said:

    Lars> Robert Pluim <rpluim <at> gmail.com> writes:
    >> ./configure
    >> make
    >> ...configure gets run again, WTF?

    Lars> Doesn't happen when I try that.

And now I canʼt reproduce it. Oh well.

Robert
-- 

Message #102 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Robert Pluim <rpluim <at> gmail.com>
Cc: larsi <at> gnus.org, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Mon, 25 Apr 2022 19:13:35 +0300

> From: Robert Pluim <rpluim <at> gmail.com>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  9106 <at> debbugs.gnu.org
> Date: Mon, 25 Apr 2022 17:49:40 +0200
> 
> >>>>> On Mon, 25 Apr 2022 16:29:09 +0200, Lars Ingebrigtsen <larsi <at> gnus.org> said:
> 
>     Lars> Robert Pluim <rpluim <at> gmail.com> writes:
>     >> ./configure
>     >> make
>     >> ...configure gets run again, WTF?
> 
>     Lars> Doesn't happen when I try that.
> 
> And now I canʼt reproduce it. Oh well.

Maybe you remembered doing "make bootstrap" after "configure", not
just "make".

Message #105 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: larsi <at> gnus.org, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Tue, 26 Apr 2022 17:43:18 +0200

>>>>> On Mon, 25 Apr 2022 19:13:35 +0300, Eli Zaretskii <eliz <at> gnu.org> said:

    Eli> Maybe you remembered doing "make bootstrap" after "configure", not
    Eli> just "make".

Thatʼs always a possibility. Iʼll pay more attention next time 😊

Robert
-- 

Message #108 received at 9106 <at> debbugs.gnu.org (full text, mbox):

From: Robert Pluim <rpluim <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: larsi <at> gnus.org, 9106 <at> debbugs.gnu.org
Subject: Re: bug#9106: 24.0.50; ./configure causes massive recompilation
Date: Thu, 05 May 2022 10:33:30 +0200

>>>>> On Tue, 26 Apr 2022 17:43:18 +0200, Robert Pluim <rpluim <at> gmail.com> said:

>>>>> On Mon, 25 Apr 2022 19:13:35 +0300, Eli Zaretskii <eliz <at> gnu.org> said:
    Eli> Maybe you remembered doing "make bootstrap" after "configure", not
    Eli> just "make".

    Robert> Thatʼs always a possibility. Iʼll pay more attention next time 😊

So this was actually

git pull
./configure
make
=> configure runs again

but thatʼs normal, since the git pull in this case updated
configure.ac, so it all working properly

Robert
-- 

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.