GNU bug report logs - #1705
23.0.60; parallel build failure: `mh-e' was not provided

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 1705 in the body.
You can then email your comments to 1705 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Markus Triska <markus.triska <at> gmx.at>
To: emacs-pretest-bug <at> gnu.org
Subject: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Thu, 25 Dec 2008 23:27:56 +0100 (CET)

A parallel compilation (-j 2) with today's CVS version aborted with:

   In toplevel form:
   mh-e/mh-folder.el:34:1:Error: Required feature `mh-e' was not provided
   Wrote /Users/mt/emacs/lisp/mh-e/mh-e.elc
   Compiling /Users/mt/emacs/lisp/mh-e/mh-funcs.el
   make[1]: *** [/Users/mt/emacs/lisp/mh-e/mh-folder.elc] Error 1
   make[1]: *** Waiting for unfinished jobs....
   make[1]: *** Waiting for unfinished jobs....
   No MH variant found on the system
   Wrote /Users/mt/emacs/lisp/mh-e/mh-funcs.elc
   make: *** [lisp] Error 2

After another "$ make -j 2", the compilation proceeded without errors.
The full compilation log is available from:

   http://www.logic.at/prolog/log20081225.txt

I have seen this problem only once so far in many tries.


In GNU Emacs 23.0.60.1 (i386-apple-darwin8.11.1, GTK+ Version 2.12.9)
 of 2008-12-25 on mt-computer.local
Windowing system distributor `The XFree86 Project, Inc', version 11.0.40400000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default-enable-multibyte-characters: t

Message #10 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Markus Triska <markus.triska <at> gmx.at>, 1705 <at> debbugs.gnu.org
Cc: emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Sat, 27 Dec 2008 14:19:44 +0200

> From: Markus Triska <markus.triska <at> gmx.at>
> Date: Thu, 25 Dec 2008 23:27:56 +0100 (CET)
> Cc: 
> 
> 
> A parallel compilation (-j 2) with today's CVS version aborted with:
> 
>    In toplevel form:
>    mh-e/mh-folder.el:34:1:Error: Required feature `mh-e' was not provided
>    Wrote /Users/mt/emacs/lisp/mh-e/mh-e.elc
>    Compiling /Users/mt/emacs/lisp/mh-e/mh-funcs.el
>    make[1]: *** [/Users/mt/emacs/lisp/mh-e/mh-folder.elc] Error 1
>    make[1]: *** Waiting for unfinished jobs....
>    make[1]: *** Waiting for unfinished jobs....
>    No MH variant found on the system
>    Wrote /Users/mt/emacs/lisp/mh-e/mh-funcs.elc
>    make: *** [lisp] Error 2

This is because we don't tell Make all the truth about the
dependencies between Lisp files, so it by default assumes that
mh-funcs.elc and mh-e.elc are independent, and compiles them in two
different jobs that run in parallel.  But the truth is that
mh-funcs.el says this:

   (require 'mh-e)

which causes Emacs to look for mh-e.elc when it compiles mh-funcs.el.
So there's a race condition here between the 2 jobs that Make runs in
parallel under "-j 2": one job compiles mh-e.el and writes mh-e.elc,
while the other job, which compiles mh-funcs.el, reads from mh-e.elc.
In this case, the reading job evidently accessed mh-e.elc after it was
already created, but before it was written in its fullness (as you see
above, the "Wrote /Users/mt/emacs/lisp/mh-e/mh-e.elc" message is
printed _after_ the error message).  So the "provide" form was not yet
in mh-e.elc when it was read, and Emacs barfed.

In general, we lack such dependencies all over lisp/Makefile.in, so I
submit that "make -j N" works for us by sheer luck: I'm guessing that
this is due to small values of N and the fact that $(ELCFILES) causes
Make to compile files in alphabetic order, combined with very few
dependencies between Lisp files whose names come close to one another
in that order.  But given a large enough value of N (and a
correspondingly large number of processors on the system), we should
see such failures all the time.

I added to lisp/Makefile.in a series of dependencies that should fix
the case of MH-E compilation.  Please see if this fixes the problem
for you.  If you have more than 2 processors, it would make sense to
try "make -j N" with values of N larger than 2, to increase the
probability of such a clash.

Message #25 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 1705 <at> debbugs.gnu.org, Markus Triska <markus.triska <at> gmx.at>,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Tue, 30 Dec 2008 21:30:33 -0500

> I added to lisp/Makefile.in a series of dependencies that should fix
> the case of MH-E compilation.  Please see if this fixes the problem
> for you.  If you have more than 2 processors, it would make sense to
> try "make -j N" with values of N larger than 2, to increase the
> probability of such a clash.

This won't solve it in general, because there are many circular
dependencies among Elisp files.  Another approach is to just close the
race condition: write the byte-compiled file to foo.elc.temp and then
rename it atomically to foo.elc.


        Stefan

Message #40 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Bill Wohler <wohler <at> newt.com>
To: bug-gnu-emacs <at> gnu.org
Cc: emacs-pretest-bug <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Fri, 02 Jan 2009 11:34:40 -0800

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> I added to lisp/Makefile.in a series of dependencies that should fix
>> the case of MH-E compilation.  Please see if this fixes the problem
>> for you.  If you have more than 2 processors, it would make sense to
>> try "make -j N" with values of N larger than 2, to increase the
>> probability of such a clash.
>
> This won't solve it in general, because there are many circular
> dependencies among Elisp files.  Another approach is to just close the
> race condition: write the byte-compiled file to foo.elc.temp and then
> rename it atomically to foo.elc.

Thanks Eli and Stefan. My laptop isn't a multi-processor box (yet) so
I can't be of more assistance. All I can say is that with the last big
MH-E reorganization going into MH-E 8.0, we eliminated all circular
dependencies within MH-E. I can't speak for the packages we require,
however.

-- 
Bill Wohler <wohler <at> newt.com>  http://www.newt.com/wohler/  GnuPG ID:610BD9AD

Message #50 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 1705 <at> debbugs.gnu.org, markus.triska <at> gmx.at,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Sat, 03 Jan 2009 11:22:53 +0200

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: 1705 <at> emacsbugs.donarmstrong.com,  Markus Triska <markus.triska <at> gmx.at>,  emacs-pretest-bug <at> gnu.org,  bug-gnu-emacs <at> gnu.org
> Date: Tue, 30 Dec 2008 21:30:33 -0500
> 
> > I added to lisp/Makefile.in a series of dependencies that should fix
> > the case of MH-E compilation.  Please see if this fixes the problem
> > for you.  If you have more than 2 processors, it would make sense to
> > try "make -j N" with values of N larger than 2, to increase the
> > probability of such a clash.
> 
> This won't solve it in general, because there are many circular
> dependencies among Elisp files.  Another approach is to just close the
> race condition: write the byte-compiled file to foo.elc.temp and then
> rename it atomically to foo.elc.

But adding dependencies is TRT on its own right, even if your
alternative is adopted.  As long as the byte compiler cannot grok
those dependencies by itself, that is.

Message #65 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 1705 <at> debbugs.gnu.org, markus.triska <at> gmx.at,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Sun, 04 Jan 2009 23:12:58 -0500

>> This won't solve it in general, because there are many circular
>> dependencies among Elisp files.  Another approach is to just close the
>> race condition: write the byte-compiled file to foo.elc.temp and then
>> rename it atomically to foo.elc.
> But adding dependencies is TRT on its own right,

Personally, I've hacked my Emacs so that when byte-compiling files,
`require'd files get loaded from the .el in preference to the .elc file
if it's more recent.


        Stefan

Message #80 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 1705 <at> debbugs.gnu.org, markus.triska <at> gmx.at,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Mon, 05 Jan 2009 22:00:26 +0200

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: 1705 <at> emacsbugs.donarmstrong.com,  markus.triska <at> gmx.at,  emacs-pretest-bug <at> gnu.org,  bug-gnu-emacs <at> gnu.org
> Date: Sun, 04 Jan 2009 23:12:58 -0500
> 
> Personally, I've hacked my Emacs so that when byte-compiling files,
> `require'd files get loaded from the .el in preference to the .elc file
> if it's more recent.

I don't think this is TRT in general.

Message #95 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 1705 <at> debbugs.gnu.org, markus.triska <at> gmx.at,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Mon, 05 Jan 2009 17:21:23 -0500

>> Personally, I've hacked my Emacs so that when byte-compiling files,
>> `require'd files get loaded from the .el in preference to the .elc file
>> if it's more recent.

> I don't think this is TRT in general.

Neither is the preference of .elc even if it's older.


        Stefan

Message #110 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 1705 <at> debbugs.gnu.org, markus.triska <at> gmx.at,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Tue, 06 Jan 2009 06:06:50 +0200

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: 1705 <at> emacsbugs.donarmstrong.com,  markus.triska <at> gmx.at,  emacs-pretest-bug <at> gnu.org,  bug-gnu-emacs <at> gnu.org
> Date: Mon, 05 Jan 2009 17:21:23 -0500
> 
> >> Personally, I've hacked my Emacs so that when byte-compiling files,
> >> `require'd files get loaded from the .el in preference to the .elc file
> >> if it's more recent.
> 
> > I don't think this is TRT in general.
> 
> Neither is the preference of .elc even if it's older.

I wasn't arguing for that, I was arguing for adding dependencies to
lisp/Makefile.in.  That is always TRT.

Message #125 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Richard M Stallman <rms <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>, 1705 <at> debbugs.gnu.org
Cc: eliz <at> gnu.org, markus.triska <at> gmx.at, emacs-pretest-bug <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Tue, 06 Jan 2009 07:31:03 -0500

    Neither is the preference of .elc even if it's older.

In general, it is right to load the .elc even if older,
because that way editing the source code does not
immediately force you to use your new code.
You get to choose (by compiling) when to make it active.

Message #135 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 1705 <at> debbugs.gnu.org, markus.triska <at> gmx.at,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Tue, 06 Jan 2009 10:00:59 -0500

>> >> Personally, I've hacked my Emacs so that when byte-compiling files,
>> >> `require'd files get loaded from the .el in preference to the .elc file
>> >> if it's more recent.
>> > I don't think this is TRT in general.
>> Neither is the preference of .elc even if it's older.
> I wasn't arguing for that,

I know.

> I was arguing for adding dependencies to
> lisp/Makefile.in.  That is always TRT.

Except when there are circular dependencies, which (as mentioned) do
exist in fairly large quantities in Emacs.


        Stefan

Message #150 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 1705 <at> debbugs.gnu.org, markus.triska <at> gmx.at,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Tue, 06 Jan 2009 22:36:38 +0200

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: 1705 <at> emacsbugs.donarmstrong.com,  markus.triska <at> gmx.at,  emacs-pretest-bug <at> gnu.org,  bug-gnu-emacs <at> gnu.org
> Date: Tue, 06 Jan 2009 10:00:59 -0500
> 
> > I was arguing for adding dependencies to
> > lisp/Makefile.in.  That is always TRT.
> 
> Except when there are circular dependencies

Of course, that goes without saying.  Make will simply barf on
circular dependencies.

Message #165 received at submit <at> emacsbugs.donarmstrong.com (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 1705 <at> debbugs.gnu.org, markus.triska <at> gmx.at,
        emacs-pretest-bug <at> gnu.org, bug-gnu-emacs <at> gnu.org
Subject: Re: bug#1705: 23.0.60; parallel build failure: `mh-e' was not provided
Date: Tue, 06 Jan 2009 17:15:13 -0500

>> > I was arguing for adding dependencies to
>> > lisp/Makefile.in.  That is always TRT.
>> Except when there are circular dependencies
> Of course, that goes without saying.  Make will simply barf on
> circular dependencies.

Not only that, but the build order will not be correct (at least
insofar as the dependencies are really needed).  Which is why another
solution is also needed.


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.