GNU bug report logs - #10816
automake: autoreconf -i fails if macro dir does not exist (it should automatically create new directories)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10816 in the body.
You can then email your comments to 10816 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "David A. Wheeler" <dwheeler <at> dwheeler.com>
To: bug-automake <at> gnu.org
Cc: dwheeler <at> dwheeler.com
Subject: automake: autoreconf -i fails if macro dir does not exist (it
	should automatically create new directories)
Date: Wed, 15 Feb 2012 09:58:20 -0500 (EST)

Problem:
"autoreconf -i" fails if you follow the libtool advice and use put this in configure.ac:
 AC_CONFIG_MACRO_DIR([m4])
and add "-I m4" to ACLOCAL_AMFLAGS in Makefile.am

It will instead report:
aclocal: couldn't open directory `m4': No such file or directory
autoreconf: aclocal failed with exit status: 1

The whole point of "autoreconf -i" is to create "whatever is needed",
and clearly MACRO_DIRs are needed, yet autoreconf fails to do so.  Therefore, this is a bug.

This is *especially* a problem when using the widely-used "git" tool,
because "git" does not store empty directories (only files). The symptom is that
you "git clone" a fresh repository, and then "autoreconf -i" fails.  You can work around this using:
"mkdir m4 ; touch m4/dummy ; git add m4/dummy" or similar, but you shouldn't need
to work around this at all.

This is already a bug report in Debian:
 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=565663
reported against automake 1.11.

Solution:
Please patch aclocal (part of automake) to automatically create these directories.

A patch has already been created in 2010:
  http://permalink.gmane.org/gmane.linux.debian.devel.bugs.general/678260
  http://permalink.gmane.org/gmane.comp.sysutils.automake.patches/3859

--- David A. Wheeler

Message #14 received at 10816 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: automake-patches <at> gnu.org
Cc: 8168 <at> debbugs.gnu.org, dwheeler <at> dwheeler.com, jjardon <at> gnome.org,
	10816 <at> debbugs.gnu.org
Subject: [PATCH] aclocal: create local directory where to install m4 files
Date: Thu, 15 Mar 2012 20:57:22 +0100

Fixes automake bug#8168 and bug#10816.

A call like "aclocal -I m4 --install" used to fail if the 'm4'
directory wasn't pre-existing.  This could be particularly
annoying when running in a checked-out version from a VCS like
git, which doesn't allow empty directories to be tracked.

* aclocal.in (File::Path): New import.
(scan_m4_dirs): Don't die if the first directory of type FT_USER
doesn't exist and the '--install' option was given; that directory
will be created later in ...
(install_file): ... here.  Change signature: now it takes as second
argument the destination directory rather than the destination file.
Crate the destination directory if it doesn't already exist.  In
verbose mode, tell what is being copied where.
(write_aclocal): Update.
* NEWS, THANKS: Update.
* tests/aclocal-install-fail.test: New test.
* tests/aclocal-install-mkdir.test: Likewise.
* tests/aclocal-no-install-no-mkdir.test: Likewise.
* tests/aclocal-verbose-install.test: Likewise.
* tests/list-of-tests.mk: Add them.

Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com>
---

 I will push this in a couple of days if there is no objection.
 Thanks to David and Javier, and sorry for the awful delay.

 Regards,
   Stefano

 NEWS                                   |    4 ++
 THANKS                                 |    2 +
 aclocal.in                             |   40 +++++++++++++++----
 tests/aclocal-install-fail.test        |   65 ++++++++++++++++++++++++++++++++
 tests/aclocal-install-mkdir.test       |   62 ++++++++++++++++++++++++++++++
 tests/aclocal-no-install-no-mkdir.test |   39 +++++++++++++++++++
 tests/aclocal-verbose-install.test     |   54 ++++++++++++++++++++++++++
 tests/list-of-tests.mk                 |    4 ++
 8 files changed, 261 insertions(+), 9 deletions(-)
 create mode 100755 tests/aclocal-install-fail.test
 create mode 100755 tests/aclocal-install-mkdir.test
 create mode 100755 tests/aclocal-no-install-no-mkdir.test
 create mode 100755 tests/aclocal-verbose-install.test

diff --git a/NEWS b/NEWS
index b998ce4..2b5f68f 100644
--- a/NEWS
+++ b/NEWS
@@ -122,6 +122,10 @@ New in 1.11.0a:
     optional space between the -I, -L and -l options and their respective
     arguments, for better POSIX compliance.
 
+  - If "aclocal --install" is used, and the first directory specified with
+    '-I' is non-existent, aclocal will now create it before trying to copy
+    files in it.
+
 Bugs fixed in 1.11.0a:
 
 * Bugs introduced by 1.11.2:
diff --git a/THANKS b/THANKS
index 8363126..d7b3658 100644
--- a/THANKS
+++ b/THANKS
@@ -73,6 +73,7 @@ Dave Brolley		brolley <at> redhat.com
 Dave Korn		dave.korn.cygwin <at> googlemail.com
 Dave Morrison		dave <at> bnl.gov
 David A. Swierczek	swiercze <at> mr.med.ge.com
+David A. Wheeler	dwheeler <at> dwheeler.com
 David Byron		dbyron <at> dbyron.com
 Davyd Madeley		davyd <at> fugro-fsi.com.au
 David Pashley		david <at> davidpashley.com
@@ -149,6 +150,7 @@ Janos Farkas		chexum <at> shadow.banki.hu
 Jared Davis		abiword <at> aiksaurus.com
 Jason Duell		jcduell <at> lbl.gov
 Jason Molenda		crash <at> cygnus.co.jp
+Javier Jardón		jjardon <at> gnome.org
 Jeff Bailey		Jbailey <at> phn.ca
 Jeff Garzik		jgarzik <at> pobox.com
 Jeff Squyres		jsquyres <at> lam-mpi.org
diff --git a/aclocal.in b/aclocal.in
index ed1d8d9..ca15732 100644
--- a/aclocal.in
+++ b/aclocal.in
@@ -7,9 +7,7 @@ eval 'case $# in 0) exec @PERL@ -S "$0";; *) exec @PERL@ -S "$0" "$@";; esac'
 
 # aclocal - create aclocal.m4 by scanning configure.ac
 
-# Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
-# 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software Foundation,
-# Inc.
+# Copyright (C) 1996-2012 Free Software Foundation, Inc.
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -45,6 +43,7 @@ use Automake::FileUtils;
 use File::Basename;
 use File::stat;
 use Cwd;
+use File::Path ();
 
 # Some globals.
 
@@ -181,6 +180,17 @@ sub unlink_tmp
 $SIG{'INT'} = $SIG{'TERM'} = $SIG{'QUIT'} = $SIG{'HUP'} = 'unlink_tmp';
 END { unlink_tmp }
 
+sub xmkdir_p ($)
+{
+  my $dir = shift;
+  local $@ = undef;
+  return
+    if -d $dir or eval { File::Path::mkpath $dir };
+  chomp $@;
+  $@ =~ s/\s+at\s.*\bline\s\d+.*$//;
+  fatal "could not create directory '$dir': $@";
+}
+
 # Check macros in acinclude.m4.  If one is not used, warn.
 sub check_acinclude ()
 {
@@ -210,12 +220,15 @@ sub reset_maps ()
   undef &search;
 }
 
-# install_file ($SRC, $DEST)
+# install_file ($SRC, $DESTDIR)
 sub install_file ($$)
 {
-  my ($src, $dest) = @_;
+  my ($src, $destdir) = @_;
+  my $dest = $destdir . "/" . basename ($src);
   my $diff_dest;
 
+  verb "installing $src to $dest";
+
   if ($force_output
       || !exists $file_contents{$dest}
       || $file_contents{$src} ne $file_contents{$dest})
@@ -268,6 +281,7 @@ sub install_file ($$)
 	}
       elsif (!$dry_run)
 	{
+          xmkdir_p ($destdir);
 	  xsystem ('cp', $src, $dest);
 	}
     }
@@ -307,6 +321,7 @@ sub list_compare (\@\@)
 # --------------------------
 # Scan all M4 files installed in @DIRS for new macro definitions.
 # Register each file as of type $TYPE (one of the FT_* constants).
+my $first_user_m4dir = 1;
 sub scan_m4_dirs ($@)
 {
   my ($type, @dirlist) = @_;
@@ -315,7 +330,16 @@ sub scan_m4_dirs ($@)
     {
       if (! opendir (DIR, $m4dir))
 	{
-	  fatal "couldn't open directory `$m4dir': $!";
+	  if ($install && $type == FT_USER && $first_user_m4dir)
+            {
+              # We will try to create this directory later, so don't
+              # complain if it doesn't exist.
+              # TODO: maybe we should avoid complaining only if errno
+              # is ENONENT?
+              $first_user_m4dir = 0;
+              next;
+            }
+	  fatal "couldn't open directory '$m4dir': $!";
 	}
 
       # We reverse the directory contents so that foo2.m4 gets
@@ -773,9 +797,7 @@ sub write_aclocal ($@)
 	      my $dest;
 	      for my $ifile (@{$file_includes{$file}}, $file)
 		{
-		  $dest = "$user_includes[0]/" . basename $ifile;
-		  verb "installing $ifile to $dest";
-		  install_file ($ifile, $dest);
+		  install_file ($ifile, $user_includes[0]);
 		}
 	      $installed = 1;
 	    }
diff --git a/tests/aclocal-install-fail.test b/tests/aclocal-install-fail.test
new file mode 100755
index 0000000..cf493aa
--- /dev/null
+++ b/tests/aclocal-install-fail.test
@@ -0,0 +1,65 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that "aclocal --install" fails when it should.
+# FIXME: this is a good candidate for a conversion to TAP.
+
+am_create_testdir=empty
+required=ro-dir
+. ./defs || Exit 1
+
+set -e
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+MY_MACRO
+END
+
+mkdir sys-acdir
+cat > sys-acdir/my-defs.m4 <<END
+AC_DEFUN([MY_MACRO], [:])
+END
+
+ACLOCAL="$ACLOCAL -Wnone --system-acdir=sys-acdir"
+
+: > a-regular-file
+mkdir unwritable-dir
+chmod a-w unwritable-dir
+
+$ACLOCAL -I a-regular-file --install 2>stderr \
+  && { cat stderr >&2; Exit 1; }
+cat stderr >&2
+$EGREP '(mkdir:|directory ).*a-regular-file' stderr
+test ! -f aclocal.m4
+
+$ACLOCAL --install -I unwritable-dir/sub 2>stderr \
+  && { cat stderr >&2; Exit 1; }
+cat stderr >&2
+$EGREP '(mkdir:|directory ).*unwritable-dir/sub' stderr
+test ! -f aclocal.m4
+
+$ACLOCAL -I unwritable-dir --install 2>stderr \
+  && { cat stderr >&2; Exit 1; }
+cat stderr >&2
+$EGREP '(cp:|copy ).*unwritable-dir' stderr
+test ! -f aclocal.m4
+
+# Sanity check.
+mkdir m4
+$ACLOCAL -I m4 --install && test -f aclocal.m4 \
+  || fatal_ "aclocal failed also when expected to succeed"
+
+:
diff --git a/tests/aclocal-install-mkdir.test b/tests/aclocal-install-mkdir.test
new file mode 100755
index 0000000..61a3d19
--- /dev/null
+++ b/tests/aclocal-install-mkdir.test
@@ -0,0 +1,62 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that "aclocal --install" creates the local m4 directory if
+# necessary.
+# FIXME: this is a good candidate for a conversion to TAP.
+
+am_create_testdir=empty
+. ./defs || Exit 1
+
+set -e
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+MY_MACRO
+END
+
+mkdir sys-acdir
+cat > sys-acdir/my-defs.m4 <<END
+AC_DEFUN([MY_MACRO], [:])
+END
+
+ACLOCAL="$ACLOCAL --system-acdir=sys-acdir"
+
+$ACLOCAL -I foo --install
+test -f foo/my-defs.m4
+
+$ACLOCAL --install -I "`pwd`/bar"
+test -f bar/my-defs.m4
+
+$ACLOCAL --install -I baz/sub/sub2
+test -f baz/sub/sub2/my-defs.m4
+
+mkdir zardoz
+# What should happen:
+#  * quux should be created, and required m4 files copied into there.
+#  * zardoz shouldn't be preferred to quux, even if the former exists
+#    while the latter does not.
+$ACLOCAL --install -I quux -I zardoz
+test -d quux
+grep MY_MACRO quux/my-defs.m4
+ls zardoz | grep . && Exit 1
+
+# It's better if aclocal doesn't create the first include dir on failure.
+$ACLOCAL --install -I none -I none2 && Exit 1
+test ! -d none
+test ! -d none2
+
+:
diff --git a/tests/aclocal-no-install-no-mkdir.test b/tests/aclocal-no-install-no-mkdir.test
new file mode 100755
index 0000000..73a6116
--- /dev/null
+++ b/tests/aclocal-no-install-no-mkdir.test
@@ -0,0 +1,39 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that aclocal does not create a non-existent local m4 directory
+# if the '--install' option is not given.
+
+am_create_testdir=empty
+. ./defs || Exit 1
+
+set -e
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+MY_MACRO
+END
+
+mkdir sys-acdir
+cat > sys-acdir/my-defs.m4 <<END
+AC_DEFUN([MY_MACRO], [:])
+END
+
+$ACLOCAL -I foo --system-acdir=sys-acdir && Exit 1
+test ! -d foo
+test ! -r foo
+
+:
diff --git a/tests/aclocal-verbose-install.test b/tests/aclocal-verbose-install.test
new file mode 100755
index 0000000..589d540
--- /dev/null
+++ b/tests/aclocal-verbose-install.test
@@ -0,0 +1,54 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check verbose messages by `aclocal --install'.
+
+am_create_testdir=empty
+. ./defs || Exit 1
+
+set -e
+
+cat > configure.in <<END
+AC_INIT([$me], [1.0])
+MY_MACRO_BAR
+MY_MACRO_QUUX
+END
+
+mkdir sys-acdir
+cat > sys-acdir/bar.m4 <<END
+AC_DEFUN([MY_MACRO_BAR], [:])
+END
+cat > sys-acdir/quux.m4 <<END
+AC_DEFUN([MY_MACRO_QUUX], [:])
+END
+
+mkdir foodir
+: > foodir/bar.m4
+
+$ACLOCAL --system-acdir=sys-acdir --install --verbose -I foodir 2>stderr \
+ || { cat stderr >&2; Exit 1; }
+cat stderr >&2
+grep ' installing .*sys-acdir/bar\.m4.* to .*foodir/bar\.m4' stderr
+grep ' installing .*sys-acdir/quux\.m4.* to .*foodir/quux\.m4' stderr
+grep ' overwriting .*foodir/bar\.m4.* with .*sys-acdir/bar\.m4' stderr
+grep ' installing .*foodir/quux\.m4.* from .*sys-acdir/quux\.m4' stderr
+
+# Sanity checks.
+ls -l foodir
+grep MY_MACRO_BAR foodir/bar.m4
+grep MY_MACRO_QUUX foodir/quux.m4
+
+:
diff --git a/tests/list-of-tests.mk b/tests/list-of-tests.mk
index b5a604c..f9109c0 100644
--- a/tests/list-of-tests.mk
+++ b/tests/list-of-tests.mk
@@ -64,6 +64,10 @@ aclocal-path-install.test \
 aclocal-path-install-serial.test \
 aclocal-path-nonexistent.test \
 aclocal-path-precedence.test \
+aclocal-install-fail.test \
+aclocal-install-mkdir.test \
+aclocal-no-install-no-mkdir.test \
+aclocal-verbose-install.test \
 acoutnoq.test \
 acoutpt.test \
 acoutpt2.test \
-- 
1.7.9

Message #19 received at 10816-done <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: automake-patches <at> gnu.org
Cc: 10816-done <at> debbugs.gnu.org, 8168-done <at> debbugs.gnu.org,
	dwheeler <at> dwheeler.com, jjardon <at> gnome.org
Subject: Re: bug#10816: [PATCH] aclocal: create local directory where to
	install m4 files
Date: Sat, 17 Mar 2012 09:46:30 +0100

On 03/15/2012 08:57 PM, Stefano Lattarini wrote:
> Fixes automake bug#8168 and bug#10816.
> 
> A call like "aclocal -I m4 --install" used to fail if the 'm4'
> directory wasn't pre-existing.  This could be particularly
> annoying when running in a checked-out version from a VCS like
> git, which doesn't allow empty directories to be tracked.
> 
> * aclocal.in (File::Path): New import.
> (scan_m4_dirs): Don't die if the first directory of type FT_USER
> doesn't exist and the '--install' option was given; that directory
> will be created later in ...
> (install_file): ... here.  Change signature: now it takes as second
> argument the destination directory rather than the destination file.
> Crate the destination directory if it doesn't already exist.  In
> verbose mode, tell what is being copied where.
> (write_aclocal): Update.
> * NEWS, THANKS: Update.
> * tests/aclocal-install-fail.test: New test.
> * tests/aclocal-install-mkdir.test: Likewise.
> * tests/aclocal-no-install-no-mkdir.test: Likewise.
> * tests/aclocal-verbose-install.test: Likewise.
> * tests/list-of-tests.mk: Add them.
> 
> Signed-off-by: Stefano Lattarini <stefano.lattarini <at> gmail.com>
> ---
> 
>  I will push this in a couple of days if there is no objection.
>  Thanks to David and Javier, and sorry for the awful delay.
> 
No objection seen so far, so I've pushed this patch (with minor
adjustments, see commit a75a1a52).  I'm also closing the relevant
bug reports.

Regards,
  Stefano

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.