Subject: Re: bug#7873: Automake should support adding to DejaGnu site.exp

From: Stefano Lattarini <[email-masked]>
Date: Tue, 1 Feb 2011 09:58:35 +0100

On Thursday 20 January 2011, Rainer Orth wrote:
> Currently, Automake doesn't support adding package-specific content to
> the site.exp file generated for DejaGnu testsuites.  In GCC alone, three
> testsuites need that facility and have to override the
> Automake-generated target wholesale to achieve what they need:
> 
> boehm-gc/testsuite/Makefile.am
> libjava/testsuite/Makefile.am
> libstdc++-v3/testsuite/Makefile.am
> 
> These Makefile.am's carry the following comment:
> 
> # We need more things in site.exp, but automake completely controls the
> # creation of that file; there's no way to append to it without messing up
> # the dependancy chains.  So we overrule automake.  This rule is exactly
> # what it would have generated, plus our own additions.
> 
> Since in the future it is easily possible that the target/commands
> generated by Automake deviate from what is currently emitted there,
> there should be a Automake facility to directly support adding to the
> generated site.exp.
> 
> Thanks.
> 	Rainer
> 

What about the attached patch?  It should be applied to a temporary branch
based off of the commit v1.11-249-g253c2e7 "Extend and improve tests on
DejaGnu support", and could be merged to either maint or master.

Regards,
   Stefano
From 41c9eaaae6792b5fde2e8fcbb6591ee27a7a1a5a Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <[email-masked]>
Date: Tue, 1 Feb 2011 09:52:43 +0100
Subject: [PATCH 2/2] dejagnu: allow the package developer to extend site.exp

* lib/am/dejagnu.am (site.exp): Depend from the files listed in
$(EXTRA_DEJAGNU_SITE_CONFIG), if any.  Append their contents to
the generated site.exp (still preserving user edits).
* doc/automake.texi (Dejagnu Tests): Update.
* tests/dejagnu-siteexp-append.test: New test.
* tests/dejagnu-siteexp-extend.test: Likewise.
* tests/dejagnu-siteexp-useredit.test: Likewise.
* tests/Makefile.am (TESTS): Update.

Suggestion by Rainer Orth.
---
 ChangeLog                           |   13 +++
 doc/automake.texi                   |    8 ++
 lib/am/dejagnu.am                   |   16 +++-
 tests/Makefile.am                   |    3 +
 tests/Makefile.in                   |    3 +
 tests/dejagnu-siteexp-append.test   |   72 ++++++++++++++++
 tests/dejagnu-siteexp-extend.test   |  161 +++++++++++++++++++++++++++++++++++
 tests/dejagnu-siteexp-useredit.test |   69 +++++++++++++++
 8 files changed, 341 insertions(+), 4 deletions(-)
 create mode 100755 tests/dejagnu-siteexp-append.test
 create mode 100755 tests/dejagnu-siteexp-extend.test
 create mode 100755 tests/dejagnu-siteexp-useredit.test

diff --git a/ChangeLog b/ChangeLog
index f17c01f..0b66757 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2011-02-01  Stefano Lattarini  <[email-masked]>
+
+	dejagnu: allow the package developer to extend site.exp
+	* lib/am/dejagnu.am (site.exp): Depend from the files listed in
+	$(EXTRA_DEJAGNU_SITE_CONFIG), if any.  Append their contents to
+	the generated site.exp (still preserving user edits).
+	* doc/automake.texi (Dejagnu Tests): Update.
+	* tests/dejagnu-siteexp-append.test: New test.
+	* tests/dejagnu-siteexp-extend.test: Likewise.
+	* tests/dejagnu-siteexp-useredit.test: Likewise.
+	* tests/Makefile.am (TESTS): Update.
+	Suggestion by Rainer Orth.
+
 2010-12-10  Stefano Lattarini  <[email-masked]>
 
 	Extend and improve tests on DejaGnu support.
diff --git a/doc/automake.texi b/doc/automake.texi
index 5a805b3..63ddaf0 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8828,6 +8828,14 @@ not the place where the test suite author should define new variables:
 this should be done elsewhere in the real test suite code.
 Especially, @file{site.exp} should not be distributed.
 
+Still, if the package author has legitimate reasons to extend
+@file{site.exp} at make time, he can do so by defining the variable
+@code{EXTRA_DEJAGNU_SITE_CONFIG}; the files listed there will be
+considered @file{site.exp} prerequisites, and their content will
+be appended to it (in the same order in which they appear in
+@code{EXTRA_DEJAGNU_SITE_CONFIG}).  Note that files are @emph{not}
+distributed by default.
+
 For more information regarding DejaGnu test suites, see @ref{Top, , ,
 dejagnu, The DejaGnu Manual}.
 
diff --git a/lib/am/dejagnu.am b/lib/am/dejagnu.am
index 08de45c..8cfb4f0 100644
--- a/lib/am/dejagnu.am
+++ b/lib/am/dejagnu.am
@@ -72,7 +72,7 @@ check-DEJAGNU: site.exp
 ## Note that in the rule we don't directly generate site.exp to avoid
 ## the possibility of a corrupted site.exp if make is interrupted.
 ## Jim Meyering has some useful text on this topic.
-site.exp: Makefile
+site.exp: Makefile $(EXTRA_DEJAGNU_SITE_CONFIG)
 	@echo 'Making a new site.exp file...'
 	@echo '## these variables are automatically generated by make ##' >site.tmp
 	@echo '# Do not edit here.  If you wish to override these values' >>site.tmp
@@ -86,9 +86,17 @@ site.exp: Makefile
 ?HOST?	@echo 'set host_triplet $(host_triplet)' >>site.tmp
 ?TARGET?	@echo 'set target_alias "$(target_alias)"' >>site.tmp
 ?TARGET?	@echo 'set target_triplet $(target_triplet)' >>site.tmp
-	@echo '## All variables above are generated by configure. Do Not Edit ##' >>site.tmp
-	@test ! -f site.exp || \
-	  sed '1,/^## All variables above are.*##/ d' site.exp >> site.tmp
+## Allow the package author to extend site.exp.
+	@list='$(EXTRA_DEJAGNU_SITE_CONFIG)'; for f in $$list; do \
+	  echo "## Begin content included from file $$f.  Do not modify. ##" \
+	   && cat `test -f "$$f" || echo '$(srcdir)/'`$$f \
+	   && echo "## End content included from file $$f. ##" \
+	   || exit 1; \
+	 done >> site.tmp
+	@echo "## End of auto-generated content; you can edit from here. ##" >> site.tmp
+	@if test -f site.exp; then \
+	   sed -e '1,/^## End of auto-generated content.*##/d' site.exp >> site.tmp; \
+	 fi
 	@-rm -f site.bak
 	@test ! -f site.exp || mv site.exp site.bak
 	@mv site.tmp site.exp
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 3f40361..d947cb4 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -255,6 +255,9 @@ dejagnu4.test \
 dejagnu5.test \
 dejagnu6.test \
 dejagnu7.test \
+dejagnu-siteexp-extend.test \
+dejagnu-siteexp-append.test \
+dejagnu-siteexp-useredit.test \
 depacl2.test \
 depcomp.test \
 depcomp2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 40b4697..323d4db 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -523,6 +523,9 @@ dejagnu4.test \
 dejagnu5.test \
 dejagnu6.test \
 dejagnu7.test \
+dejagnu-siteexp-extend.test \
+dejagnu-siteexp-append.test \
+dejagnu-siteexp-useredit.test \
 depacl2.test \
 depcomp.test \
 depcomp2.test \
diff --git a/tests/dejagnu-siteexp-append.test b/tests/dejagnu-siteexp-append.test
new file mode 100755
index 0000000..7bd34ed
--- /dev/null
+++ b/tests/dejagnu-siteexp-append.test
@@ -0,0 +1,72 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that the files in $(EXTRA_DEJAGNU_SITE_CONFIG) get appended to
+# site.exp in the same order in which they're listed in that variable.
+
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = dejagnu
+DEJATOOL = tool
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+
+./configure
+
+$MAKE site.exp
+sed '/^##.*##$/d' site.exp > expected
+
+cat > foo.exp << 'END'
+foo1
+foo2
+foo3
+END
+
+cat foo.exp - >> expected << 'END'
+BAR 1
+# foo
+BAR 2
+END
+
+cat >> Makefile.am << 'END'
+EXTRA_DEJAGNU_SITE_CONFIG = foo.exp bar.exp
+bar.exp:
+	@(echo 'BAR 1' && echo '# foo' && echo 'BAR 2') > $@
+END
+
+$AUTOMAKE Makefile
+./config.status Makefile
+
+rm -f site.exp
+$MAKE site.exp
+sed '/^##.*##$/d' site.exp > obtained
+
+cat expected
+cat site.exp
+
+diff expected obtained
+
+:
diff --git a/tests/dejagnu-siteexp-extend.test b/tests/dejagnu-siteexp-extend.test
new file mode 100755
index 0000000..591267e
--- /dev/null
+++ b/tests/dejagnu-siteexp-extend.test
@@ -0,0 +1,161 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that the developer can extend the site.exp generated by the
+# automake-generated Makefile.
+
+required=runtest
+. ./defs || Exit 1
+
+set -e
+
+write_check_for ()
+{
+  echo "send_user \"$1: \$$1\\n\""
+  cat << END
+if { \$$1 == "/$1/" } {
+    pass "test_$1"
+} else {
+    fail "test_$1"
+}
+END
+}
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = dejagnu
+DEJATOOL = tool
+
+EXTRA_DIST = tool.test/tool.exp
+
+EXTRA_DEJAGNU_SITE_CONFIG = foo.exp
+EXTRA_DIST += foo.exp
+END
+
+echo 'set foo "/foo/"' > foo.exp
+
+mkdir tool.test
+write_check_for foo > tool.test/tool.exp
+cat tool.test/tool.exp
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+
+./configure
+
+$MAKE check
+cat foo.exp
+cat site.exp
+grep 'PASS: test_foo' tool.sum
+
+write_check_for bar >> tool.test/tool.exp
+write_check_for baz >> tool.test/tool.exp
+cat tool.test/tool.exp
+
+# Ensure that foo.exp will be newer than site.exp, which will
+# thus have to be remade.
+$sleep
+# With this, below we'll also check that settings in files coming later in
+# $(EXTRA_DEJAGNU_SITE_CONFIG) override those in files coming earlier.
+cat >> foo.exp <<'END'
+set bar "/foo/"
+set baz "/foo/"
+set qux "/foo/"
+END
+
+$MAKE check && { cat site.exp; Exit 1; }
+grep 'PASS: test_foo' tool.sum
+grep 'FAIL: test_bar' tool.sum
+grep 'FAIL: test_baz' tool.sum
+
+cat >> Makefile.am << 'END'
+EXTRA_DEJAGNU_SITE_CONFIG += bar bar.dir/bar
+EXTRA_DIST += bar
+DISTCLEANFILES = bar.dir/bar
+bar.dir/bar:
+	test -d bar.dir || mkdir bar.dir
+	echo 'set baz "/baz/"' > $@
+END
+
+echo 'set bar "/bar/"' > bar
+# This will allow us to check one more time that settings in files
+# coming later in $(EXTRA_DEJAGNU_SITE_CONFIG) override those in
+# files coming earlier.
+echo 'set baz "/xyz/"' >> bar
+
+# Ensure that the Makefile will be newer than site.exp, which will
+# thus have to be remade.
+$sleep
+$AUTOMAKE Makefile
+./config.status Makefile
+
+$MAKE check || { cat site.exp; Exit 1; }
+cat site.exp
+cat bar.dir/bar
+$FGREP '/bar/' site.exp
+$FGREP '/baz/' site.exp
+grep 'PASS: test_foo' tool.sum
+grep 'PASS: test_bar' tool.sum
+grep 'PASS: test_baz' tool.sum
+
+# Check that the features we're testing behave well in VPATH builds.
+$MAKE distcheck
+
+# Check that the user can edit the site.exp file, and that his edits
+# are retained.
+write_check_for zardoz >> tool.test/tool.exp
+cat tool.test/tool.exp
+echo 'set zardoz "/zardoz/"' >> site.exp
+
+$MAKE check
+cat site.exp
+grep 'PASS: test_zardoz' tool.sum
+
+cat >> Makefile.am << 'END'
+EXTRA_DEJAGNU_SITE_CONFIG += quux.exp
+quux.exp:
+	echo 'set zardoz "/quux/"' > $@
+END
+
+# Ensure that the Makefile will be newer than on site.exp, which will
+# thus have to be remade.
+$sleep
+$AUTOMAKE Makefile
+./config.status Makefile
+grep 'zardoz.*/quux/' Makefile
+
+$MAKE site.exp
+cat site.exp
+cat quux.exp
+grep 'zardoz.*/quux/' site.exp
+
+$MAKE check
+grep 'PASS: test_zardoz' tool.sum
+grep 'zardoz: /zardoz/' tool.log
+grep 'zardoz.*quux' tool.log && Exit 1
+
+# Check that files in $(EXTRA_DEJAGNU_SITE_CONFIG) are not distributed
+# by default.
+$MAKE distdir
+ls -l $me-1.0
+test ! -r $me-1.0/bar.dir/bar
+test ! -r $me-1.0/quux.exp
+
+:
diff --git a/tests/dejagnu-siteexp-useredit.test b/tests/dejagnu-siteexp-useredit.test
new file mode 100755
index 0000000..a51bc40
--- /dev/null
+++ b/tests/dejagnu-siteexp-useredit.test
@@ -0,0 +1,69 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that the user can edit the Makefile-generated site.exp, and
+# have its edits survive to the remaking of that file.
+
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = dejagnu
+DEJATOOL = foo
+END
+
+# Deliberately select a variable defined automatically by
+# the Makefile-generated site.exp.
+mkdir foo.test
+cat > foo.test/foo.exp << 'END'
+send_user "objdir: $objdir\n"
+set pipe "|"
+if { $objdir == "${pipe}objdir${pipe}" } {
+    pass "test_obj"
+} else {
+    fail "test_obj"
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+
+./configure
+
+$MAKE site.exp
+echo 'set objdir "|objdir|"' >> site.exp
+cat site.exp
+$sleep
+touch Makefile
+$MAKE site.exp
+cat site.exp
+is_newest site.exp Makefile  # Sanity check.
+grep '|objdir|' site.exp
+test `grep -c '|objdir|' site.exp` -eq 1
+
+# We can done a "more semantic" check if DejaGnu is available.
+if runtest SOMEPROGRAM=someprogram --version; then
+  $MAKE check
+  grep 'PASS: test_obj' foo.sum
+fi
+
+:
-- 
1.7.2.3