Subject: Re: Improving user experience for non-recursive builds

From: Peter Rosin <[email-masked]>
Date: Tue, 22 Jan 2013 11:22:09 +0100

On 2013-01-22 10:18, Stefano Lattarini wrote:
> [+cc bug-automake, so that we won't forget about the issue]
> [future replies should drop the automake list]
> 
> On 01/22/2013 02:22 AM, Miles Bader wrote:
>> Stefano Lattarini <[email-masked]> writes:
>>> The best solution is on the user-side IMHO: fix the build system to
>>> use less (ideally none) make recursion.  Both the parallel and serial
>>> testsuite harness should support that setup OOTB.
>>
>> It would be nice if automake had some more features for that...
>>
> Indeed (albeit the present situation is already mostly good enough
> for most medium-sized packages).
> 
>> E.g., if I have a directory "foo" that has sources etc, and builds
>> some specific targets, then I can isolate the automake stuff for foo
>> by using an include file "foo/Makefile.am.inc" or something, and then
>> putting an appropriate include in the top-level Makefile.am.
>>
>> But it's a bit annoying, in that AFAICT, all filenames, etc, in foo's
>> Makefile fragment must explicitly include the directory name.
>>
> Yes, and this issue has come up several times already.  Nobody has
> been bothered enough to attempt a patch, though, at least so far.
> 
>> E.g., if it builds a library, "foo/Makefile.am.inc" might look like:
>>
>>    libfoo_a_SOURCES = foo/oink.c foo/barf.c foo/barf.h ...
>>
>> For longish directory names, this can really bloat things up...
>>
> Someone (probably Eric Blake, but I'm not 100% sure) once noted that this
> issue could be mitigated with simple indirections with usual make macros:
> 
>   d1 = wow/a/very/very/insanely/long/directory/name
> 
>   wow_a_very_very_insanely_long_directory_name_prog_SOURCES = \
>    $(d1)/a.c $(d1)/b.c ... $(d1)/z.c
> 
>> It would be really cool if there was some way of telling automake
>> "hey, for every filename mentioned in this file, try to use a prefix
>> of ..."
>>
> This is probably too automatic; but Bob Friesenhahn suggested Automake
> could recognize special substitutions, like %CURDIR% and %XCURDIR%, so
> that you could simply use in
> 
>     %XCURDIR%_prog_SOURCES = %CURDIR%/a.c %CURDIR%/b.c ... %CURDIR%/z.c
> 
> in 'wow/a/very/very/insanely/long/directory/name/local,.mk', include this
> '.mk' fragment from the top-level Makefile.am, and have DTRT.  I think
> that could be implemented in a simple pre-processing step, before
> Automake even parses the Makefile contents (this too was Bob's suggestion,
> IIRC); in which case, the change would probably be unobtrusive and mostly
> safe.  Patches (even WIP) are welcome.

This is proof of concept, and I'm not a perl hacker etc, but it seems to
work ok. Feel free to improve or toss or add documentation or whatever :-)

Patch is based on the maint branch, but being a new feature it perhaps
belongs on master instead. But as I said, do what you want with it...

Cheers,
Peter

>From 5cc9c775dbe46343b651a7e6ac378f71e6a3b6c1 Mon Sep 17 00:00:00 2001
From: Peter Rosin <[email-masked]>
Date: Tue, 22 Jan 2013 11:17:11 +0100
Subject: [PATCH] reldir: Add support for relative names in included fragments

automake.in (read_am_file): Add third argument specifying the
relative directory of this makefile fragment compared to the
main level makefile. Replace @am_reldir@ in the fragment with
this relative directory.
(read_main_am_file): Adjust.
t/reldir.sh: New test.
t/list-of-tests.mk: Augment.
---
 automake.in        |   28 +++++++++++++++++----
 t/list-of-tests.mk |    1 +
 t/reldir.sh        |   68 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 92 insertions(+), 5 deletions(-)
 create mode 100755 t/reldir.sh

diff --git a/automake.in b/automake.in
index 0e3b882..4e52aca 100644
--- a/automake.in
+++ b/automake.in
@@ -6330,15 +6330,15 @@ sub check_trailing_slash ($\$)
 }

-# &read_am_file ($AMFILE, $WHERE)
-# -------------------------------
+# &read_am_file ($AMFILE, $WHERE, $RELDIR)
+# ----------------------------------------
 # Read Makefile.am and set up %contents.  Simultaneously copy lines
 # from Makefile.am into $output_trailer, or define variables as
 # appropriate.  NOTE we put rules in the trailer section.  We want
 # user rules to come after our generated stuff.
 sub read_am_file ($$)
 {
-    my ($amfile, $where) = @_;
+    my ($amfile, $where, $reldir) = @_;
 
     my $am_file = new Automake::XFile ("< $amfile");
     verb "reading $amfile";
@@ -6423,6 +6423,18 @@ sub read_am_file ($$)
 
 	my $new_saw_bk = check_trailing_slash ($where, $_);
 
+	if ($reldir ne '.')
+	  {
+	    my $rel_dir = &canonicalize ($reldir);
+	    $_ =~ s/\@am_reldir\@\//${reldir}\//g;
+	    $_ =~ s/\@am_reldir\@_/${rel_dir}_/g;
+	  }
+	else
+	  {
+	    $_ =~ s/\@am_reldir\@\///g;
+	    $_ =~ s/\@am_reldir\@_//g;
+	  }
+
 	if (/$IGNORE_PATTERN/o)
 	{
 	    # Merely delete comments beginning with two hashes.
@@ -6584,8 +6596,14 @@ sub read_am_file ($$)
 		push_dist_common ("\$\(srcdir\)/$path");
 		$path = $relative_dir . "/" . $path if $relative_dir ne '.';
 	      }
+	    my $new_reldir = $path;
+	    # $new_reldir =~ s/\$\($reldir\)\///;
+	    if ($new_reldir =~ s/\/[^\/]*$// == 0)
+	      {
+	        $new_reldir = '.';
+	      }
 	    $where->push_context ("'$path' included from here");
-	    &read_am_file ($path, $where);
+	    &read_am_file ($path, $where, $new_reldir);
 	    $where->pop_context;
 	}
 	else
@@ -6658,7 +6676,7 @@ sub read_main_am_file ($$)
     &define_standard_variables;
 
     # Read user file, which might override some of our values.
-    &read_am_file ($amfile, new Automake::Location);
+    &read_am_file ($amfile, new Automake::Location, '.');
 }

diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk
index 44f598e..9501ed4 100644
--- a/t/list-of-tests.mk
+++ b/t/list-of-tests.mk
@@ -783,6 +783,7 @@ t/parallel-tests-no-spurious-summary.sh \
 t/parallel-tests-exit-statuses.sh \
 t/parallel-tests-console-output.sh \
 t/parallel-tests-once.sh \
+t/reldir.sh \
 t/tests-environment.sh \
 t/am-tests-environment.sh \
 t/tests-environment-backcompat.sh \
diff --git a/t/reldir.sh b/t/reldir.sh
new file mode 100755
index 0000000..97fd0c2
--- /dev/null
+++ b/t/reldir.sh
@@ -0,0 +1,68 @@
+#! /bin/sh
+# Copyright (C) 2003-2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test @am_reldir@
+
+. test-init.sh
+
+cat >> configure.ac << 'END'
+AC_PROG_CC
+AM_PROG_CC_C_O
+AC_OUTPUT
+END
+
+mkdir foo
+mkdir foo/bar
+mkdir foo/foobar
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = subdir-objects
+bin_PROGRAMS =
+include foo/Makefile.am
+include foo/foobar/Makefile.am
+END
+
+cat > foo/Makefile.am << 'END'
+@am_reldir@_echo:
+	@echo "I am @am_reldir@/Makefile.am"
+
+bin_PROGRAMS += @am_reldir@/mumble
+@am_reldir@_mumble_SOURCES = @am_reldir@/one.c
+END
+
+cat > foo/one.c << 'END'
+int main(void) { return 0; }
+END
+
+cp foo/Makefile.am foo/bar
+cp foo/Makefile.am foo/foobar
+echo "include @am_reldir@/bar/Makefile.am" >> foo/Makefile.am
+
+cp foo/one.c foo/bar
+cp foo/one.c foo/foobar
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+./configure
+
+$MAKE foo_echo
+$MAKE foo_bar_echo
+$MAKE foo_foobar_echo
+$MAKE
+foo/mumble
+foo/bar/mumble
+foo/foobar/mumble
-- 
1.7.9