Subject: bug#19418: AC_CONFIG_FILES / stamp-h? problem

From: "Jeff Squyres (jsquyres)" <[email-masked]>
Date: Sat, 20 Dec 2014 15:48:49 +0000

Greetings.

I have found what appears to be a subtle bug in the Autotools, and I *think* it may be in Automake.  ...but I am not sure; it could also be a bug in our m4 code.

Short version
=============

The nightly "make distcheck" failed last night in the Open MPI project (www.open-mpi.org) due to several stamp-h? files being left in the build tree.

Close examination shows that the order of $config_headers in config.status -- and therefore the initial generation of stamp-h? files in the build tree -- appears to differ from the numbering of stamp-h? files in the top-level Automake-generated Makefile.in. Hence, the "distclean-hdr" rule is attempting to remove different stamp-h? files than were created by config.status, ultimately resulting in "make distcheck" failing.

Is this a known issue?  Or is there a common user/application error that can cause this kind of behavior?

This behavior occurs with AC 2.69, AM 1.14.1, and LT 2.4.2.

More detail
===========

There had been a commit earlier yesterday that added another AC_CONFIG_HEADERS file in the Open MPI configury.  This new file seemed to trigger this new behavior (i.e., "make distcheck" was not failing before last night).

Let me give some specifics.  In a clean Open MPI git checkout:

1. run "autoreconf -ivf"
2. run "./configure --prefix=/blah"
3. run "find . -name stamp-h\?"

Here's the stamp-h? files that I see -- note that the files marked by (*) are generated by sub-configure scripts that are invoked by Open MPI's main configure script; they're not important to this analysis, and are listed here just for completeness:

-----
./opal/include/stamp-h1
./ompi/include/stamp-h2
./oshmem/include/stamp-h3
./opal/mca/hwloc/hwloc191/hwloc/include/private/autogen/stamp-h4
./opal/mca/hwloc/hwloc191/hwloc/include/hwloc/autogen/stamp-h5
./opal/mca/common/libfabric/libfabric/stamp-h6
(*) ./opal/libltdl/stamp-h1
(*) ./opal/mca/event/libevent2021/libevent/stamp-h1
(*) ./ompi/mca/io/romio/romio/adio/include/stamp-h1
(*) ./ompi/contrib/vt/vt/extlib/otf/stamp-h1
(*) ./ompi/contrib/vt/vt/stamp-h1
-----

The first stamp-h1 - stamp-h6 files are generated by Open MPI's main configure script.

Indeed, looking at the main config.status, you can see that $config_headers agrees with this ordering:

-----
$ egrep '^config_headers=' config.status
config_headers=" opal/include/opal_config.h ompi/include/mpi.h oshmem/include/shmem.h opal/mca/hwloc/hwloc191/hwloc/include/private/autogen/config.h opal/mca/hwloc/hwloc191/hwloc/include/hwloc/autogen/config.h opal/mca/common/libfabric/libfabric/config.h"
-----

The problem is in Open MPI's top-level Makefile.in (which was generated by Automake).  The stamp-h? rules and the "rm -f ..." that occurs in distclean-hdr appear to differ in stamp-h<DIGIT> ordering than that of config.status.

Both the stamp-h? rules and the distclean-hdr rule reflect the same ordering, so I'll just show the "rm -f ..." that is in distclean-hdr for brevity (broken into multiple lines for readability):

-----
-rm -f \
    opal/mca/common/libfabric/libfabric/config.h \
    opal/mca/common/libfabric/libfabric/stamp-h4 \
    opal/mca/hwloc/hwloc191/hwloc/include/private/autogen/config.h \
    opal/mca/hwloc/hwloc191/hwloc/include/private/autogen/stamp-h5 \
    opal/mca/hwloc/hwloc191/hwloc/include/hwloc/autogen/config.h \
    opal/mca/hwloc/hwloc191/hwloc/include/hwloc/autogen/stamp-h6
-----

Notice that the "libfabric" directory corresponds to stamp-h4, but the config.status-generated stamp file for this directory is stamp-h6. The two hwloc191 stamp files reflect similar mismatched ordering.

*** Note that the libfabric/config.h is the newly-added
    AC_CONFIG_HEADERS file.

This all leads "make distclean-hdr" to (attempting to) remove the wrong (and non-existent) stamp files, and therefore "make distcheck" ultimately fails.

Unfortunately, the configury for Open MPI is *quite* complex; it spans many .m4 files across many directories.  I've been trying to create a small reproducer outside of the (very large) Open MPI source tree and have been unable to find the magic set of circumstances to make the same behavior occur.  :-(

So before I try to recreate this in a small example even further, let me ask two questions:

1. Is this a known problem?
2. Is there a common user-level mistake (i.e., somewhere in our configure/m4 code) that could cause this behavior to occur?

Thanks for your time.

--
Jeff Squyres
[email-masked]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/