GNU bug report logs - #8969
improve synchronization between examples in the manual and test cases

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8969 in the body.
You can then email your comments to 8969 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: bug-automake <at> gnu.org
Cc: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, automake-patches <at> gnu.org
Subject: improve synchronization between examples in the manual and test cases
Date: Fri, 1 Jul 2011 00:39:12 +0200

Severity: wishlist
thanks

Reference:
 <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00220.html>

On Tuesday 28 June 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Tue, Jun 28, 2011 at 08:24:13AM CEST:
> > On Monday 27 June 2011, Ralf Wildenhues wrote:
> > > Sure.  Well, do they use some format already?
> > >
> > Well, there are only two of them, and both follows this format:
> > 
> > @c The test case for the setup described here is
> > @c     test/subdircond2.test
> > @c Try to keep it in sync.
> > 
> > (which is wrongish BTW, as the testsuite subdirectory is `tests/',
> > not `test/')
> > 
> > I'd go with one of these simple formats:
> >   @c Keep in sync with subdircond2.test
> 
> I like this one (with a trailing dot ;-)
> 
> ;-)
> 
I'll soon post a patch that improves the synchronization between
some examples from the manual and some test cases.  In the process
of writing it, I've noticed that various non-trivial examples from
the manual are not tested; and while I've remedied to part of this
problem with my patch, a lot remains to be done; so I'm opening
this bug to avoid forgetting about the issue.

Regards,
  Stefano

Message #8 received at 8969 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: 8969 <at> debbugs.gnu.org
Cc: automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Fri, 1 Jul 2011 00:59:53 +0200

[Message part 1 (text/plain, inline)]

On Friday 01 July 2011, Stefano Lattarini wrote:
> Severity: wishlist
> thanks
> 
> Reference:
>  <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00220.html>
> 
> On Tuesday 28 June 2011, Ralf Wildenhues wrote:
> > * Stefano Lattarini wrote on Tue, Jun 28, 2011 at 08:24:13AM CEST:
> > > On Monday 27 June 2011, Ralf Wildenhues wrote:
> > > > Sure.  Well, do they use some format already?
> > > >
> > > Well, there are only two of them, and both follows this format:
> > > 
> > > @c The test case for the setup described here is
> > > @c     test/subdircond2.test
> > > @c Try to keep it in sync.
> > > 
> > > (which is wrongish BTW, as the testsuite subdirectory is `tests/',
> > > not `test/')
> > > 
> > > I'd go with one of these simple formats:
> > >   @c Keep in sync with subdircond2.test
> > 
> > I like this one (with a trailing dot ;-)
> > 
> > ;-)
> > 
> I'll soon post a patch that improves the synchronization between
> some examples from the manual and some test cases.  In the process
> of writing it, I've noticed that various non-trivial examples from
> the manual are not tested; and while I've remedied to part of this
> problem with my patch, a lot remains to be done; so I'm opening
> this bug to avoid forgetting about the issue.
> 
> Regards,
>   Stefano
>
And here it is the promised patch.  I will push in 72 hours if there
is no objection by then.

Regards,
  Stefano

[0001-docs-tests-synchronize-examples-from-docs-to-tests.patch (text/x-patch, inline)]

From 2dd820b423bea12eac7b94fd8ed2563ee21cf9e1 Mon Sep 17 00:00:00 2001
Message-Id: <2dd820b423bea12eac7b94fd8ed2563ee21cf9e1.1309473910.git.stefano.lattarini <at> gmail.com>
From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Date: Thu, 30 Jun 2011 20:05:25 +0200
Subject: [PATCH] docs, tests: synchronize examples from docs to tests

* tests/README (Writing test cases): Give suggestions on how to
keep test cases and examples in the documentation synchronized.
* doc/automake.texi: Improve or fix existing testcase-referencing
comments, and add many new ones.
* HACKING (Administrivia): Suggest to test complex examples and
idioms from the manual.
* tests/specflg8.test: Improve synchronization with the example
in the manual.
* tests/output11.test:Likewise.
* tests/txinfo21.test:Likewise.
* tests/interp.test: Likewise.  Since we are at it, and enable
the `errexit' shell flag, do related changes, and add trailing
`:'command.
* tests/amhello-cflags.test: New test.
* tests/amhello-cross-compile.test: Likewise.
* tests/amhello-binpkg.test: Likewise.
* tests/tests-environment-backcompat: Likewise.
* tests/parallel-tests-log-compiler-example.test: Likewise.
* tests/Makefile.am (TESTS): Update.
---
 ChangeLog                                      |   23 ++++++++
 HACKING                                        |    7 ++-
 doc/automake.texi                              |   54 ++++++++++++++----
 tests/Makefile.am                              |    5 ++
 tests/Makefile.in                              |    5 ++
 tests/README                                   |    7 ++
 tests/amhello-binpkg.test                      |   44 +++++++++++++++
 tests/amhello-cflags.test                      |   49 ++++++++++++++++
 tests/amhello-cross-compile.test               |   54 ++++++++++++++++++
 tests/interp.test                              |   18 ++++--
 tests/parallel-tests-log-compiler-example.test |   71 ++++++++++++++++++++++++
 tests/specflg8.test                            |   35 ++++++------
 tests/tests-environment-backcompat.test        |   65 ++++++++++++++++++++++
 tests/txinfo21.test                            |    4 +-
 14 files changed, 402 insertions(+), 39 deletions(-)
 create mode 100755 tests/amhello-binpkg.test
 create mode 100755 tests/amhello-cflags.test
 create mode 100755 tests/amhello-cross-compile.test
 create mode 100755 tests/parallel-tests-log-compiler-example.test
 create mode 100755 tests/tests-environment-backcompat.test

diff --git a/ChangeLog b/ChangeLog
index f1e10c2..c77326d 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,26 @@
+2011-07-01  Stefano Lattarini  <stefano.lattarini <at> gmail.com>
+
+	docs, tests: synchronize examples from docs to tests
+	* tests/README (Writing test cases): Give suggestions on how to
+	keep test cases and examples in the documentation synchronized.
+	* doc/automake.texi: Improve or fix existing testcase-referencing
+	comments, and add many new ones.
+	* HACKING (Administrivia): Suggest to test complex examples and
+	idioms from the manual.
+	* tests/specflg8.test: Improve synchronization with the example
+	in the manual.
+	* tests/output11.test:Likewise.
+	* tests/txinfo21.test:Likewise.
+	* tests/interp.test: Likewise.  Since we are at it, and enable
+	the `errexit' shell flag, do related changes, and add trailing
+	`:'command.
+	* tests/amhello-cflags.test: New test.
+	* tests/amhello-cross-compile.test: Likewise.
+	* tests/amhello-binpkg.test: Likewise.
+	* tests/tests-environment-backcompat: Likewise.
+	* tests/parallel-tests-log-compiler-example.test: Likewise.
+	* tests/Makefile.am (TESTS): Update.
+
 2011-06-23  Stefano Lattarini  <stefano.lattarini <at> gmail.com>
 
 	docs: avoid a footnote, some related rewordings and improvements
diff --git a/HACKING b/HACKING
index a3e37f0..334d74c 100644
--- a/HACKING
+++ b/HACKING
@@ -20,6 +20,10 @@
 * If somebody reports a new bug, mention his name in the ChangeLog entry
   and in the test case you write.  Put him into THANKS.
 
+* When documenting a non-trivial idiom or example in the manual, be
+  sure to add a test case for it, and to reference such test case from
+  a proper Texinfo comment.
+
 * The correct response to most actual bugs is to write a new test case
   which demonstrates the bug.  Then fix the bug, re-run the test suite,
   and check everything in.
@@ -240,7 +244,8 @@
 
 -----
 
-Copyright (C) 2003, 2007, 2008, 2010 Free Software Foundation, Inc.
+Copyright (C) 2003, 2007, 2008, 2010, 2011 Free Software Foundation,
+Inc.
 
 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
diff --git a/doc/automake.texi b/doc/automake.texi
index e4c96ae..f0a3c43 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -904,6 +904,7 @@ GNU Make Manual}).
 VPATH builds have other interesting uses.  One is to build the same
 sources with multiple configurations.  For instance:
 
+@c Keep in sync with amhello-cflags.test.
 @example
 ~ % @kbd{tar zxf ~/amhello-1.0.tar.gz}
 ~ % @kbd{cd amhello-1.0}
@@ -1055,6 +1056,7 @@ instance my cross-compiler for MinGW32 has its binaries called
 Here is how we could build @code{amhello-1.0} for
 @code{i586-mingw32msvc} on a GNU/Linux PC.
 
+@c Keep in sync with amhello-cross-compile.test.
 @smallexample
 ~/amhello-1.0 % @kbd{./configure --build i686-pc-linux-gnu --host i586-mingw32msvc}
 checking for a BSD-compatible install... /usr/bin/install -c
@@ -1169,6 +1171,7 @@ by some means.
 For instance here is how we could create a binary package containing a
 snapshot of all the files to be installed.
 
+@c Keep in sync with amhello-binpkg.test.
 @example
 ~/amhello-1.0 % @kbd{./configure --prefix /usr}
 @dots{}
@@ -1848,6 +1851,7 @@ When examining a variable definition, Automake will recursively examine
 variables referenced in the definition.  For example, if Automake is
 looking at the content of @code{foo_SOURCES} in this snippet
 
+@c Keep in sync with interp.test.
 @example
 xs = a.c b.c
 foo_SOURCES = c.c $(xs)
@@ -2026,6 +2030,7 @@ defined (e.g., @samp{zardir}).
 For instance, the following snippet will install @file{file.xml} into
 @samp{$(datadir)/xml}.
 
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @example
 xmldir = $(datadir)/xml
 xml_DATA = file.xml
@@ -2036,6 +2041,7 @@ performs to diagnose suspicious directory/primary couples (in the
 unlikely case these checks are undesirable, and you really know what
 you're doing).  For example, Automake would error out on this input:
 
+@c Should be tested in primary-prefix-invalid-couples.test.
 @example
 # Forbidden directory combinations, automake will error out on this.
 pkglib_PROGRAMS = foo
@@ -2045,6 +2051,7 @@ doc_LIBRARIES = libquux.a
 @noindent
 but it will succeed with this:
 
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @example
 # Work around forbidden directory combinations.  Do not use this
 # without a very good reason!
@@ -2125,6 +2132,7 @@ data_DATA = file1 @dots{} file <at> var{N} file <at> var{N+1} @dots{} file <at> var{2N}
 @noindent
 may also be written as
 
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @example
 data_DATA = file1 @dots{} file <at> var{N}
 data2dir = $(datadir)
@@ -2455,6 +2463,7 @@ example in the manual.  However if you were to build @code{true} and
 @code{false} in real life, you would probably use per-program
 compilation flags, like so:
 
+@c Keep in sync with specflg7.test and specflg8.test.
 @example
 bin_PROGRAMS = false true
 
@@ -2778,6 +2787,7 @@ literals.  If part of the specification uses shell variables,
 @command{automake} will not be able to fulfill this setup, and you will
 have to complete the missing bits by hand.  For instance, on
 
+@c Keep in sync with output11.test.
 @example
 file=input
 @dots{}
@@ -2793,6 +2803,7 @@ source file.)
 
 Similarly
 
+@c Keep in sync with output11.test.
 @example
 file=output
 file2=out:in
@@ -3420,6 +3431,7 @@ installs this macro so that @command{aclocal} will find it.
 A macro file's name should end in @file{.m4}.  Such files should be
 installed in @file{$(datadir)/aclocal}.  This is as simple as writing:
 
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @example
 aclocaldir = $(datadir)/aclocal
 aclocal_DATA = mymacro.m4 myothermacro.m4
@@ -4258,9 +4270,7 @@ does not know the possible values of these variables.  In this case
 @cindex @code{SUBDIRS} and @code{AM_CONDITIONAL}
 @cindex @code{AM_CONDITIONAL} and @code{SUBDIRS}
 
-@c The test case for the setup described here is
-@c     test/subdircond2.test
-@c Try to keep it in sync.
+@c Keep in sync with subcond2.test.
 
 @file{configure} should output the @file{Makefile} for each directory
 and define a condition into which @file{opt/} should be built.
@@ -4300,9 +4310,7 @@ automatically because it knows that @code{MAYBE_OPT} can contain
 @cindex @code{SUBDIRS} and @code{AC_SUBST}
 @cindex @code{AC_SUBST} and @code{SUBDIRS}
 
-@c The test case for the setup described here is
-@c     test/subdircond3.test
-@c Try to keep it in sync.
+@c Keep in sync with subcond3.test.
 
 Another possibility is to define @code{MAYBE_OPT} from
 @file{./configure} using @code{AC_SUBST}:
@@ -4460,6 +4468,7 @@ often be replaced by several variables, one for each destination
 directory (@pxref{Uniform}).  For instance, the last example could be
 rewritten as follows:
 
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @example
 imagesdir = $(pkgdatadir)/images
 soundsdir = $(pkgdatadir)/sounds
@@ -4910,6 +4919,7 @@ You can also use Automake conditionals (@pxref{Conditionals}) to
 select programs to be built.  In this case you don't have to worry
 about @samp{$(EXEEXT)} or @code{EXTRA_PROGRAMS}.
 
+@c Keep in sync with exeext.test.
 @example
 bin_PROGRAMS = cpio pax
 if WANT_MT
@@ -4960,6 +4970,7 @@ Extra objects can be added to a library using the
 @code{@var{library}_LIBADD} variable.  This should be used for objects
 determined by @command{configure}.  Again from @code{cpio}:
 
+@c Keep in sync with pr401c.test.
 @example
 libcpio_a_LIBADD = $(LIBOBJS) $(ALLOCA)
 @end example
@@ -5163,6 +5174,7 @@ relates to @file{libfoo.la} or @file{libbar.la} at the time it creates
 the link rule for these two libraries.  Therefore the @option{-rpath}
 argument must be explicitly supplied.
 
+@c Keep in sync with ltcond.test.
 @example
 EXTRA_LTLIBRARIES = libfoo.la libbar.la
 lib_LTLIBRARIES = $(WANTEDLIBS)
@@ -5178,6 +5190,7 @@ Automake is able to compute the @option{-rpath} setting itself, because
 it's clear that both libraries will end up in @samp{$(libdir)} if they
 are installed.
 
+@c Keep in sync with ltcond.test.
 @example
 lib_LTLIBRARIES =
 if WANT_LIBFOO
@@ -5204,6 +5217,7 @@ we could build a @file{libhello.la} library using either
 @file{hello-linux.c} or @file{hello-generic.c} with the following
 @file{Makefile.am}.
 
+@c Keep in sync with ltcond2.test.
 @example
 lib_LTLIBRARIES = libhello.la
 libhello_la_SOURCES = hello-common.c
@@ -5218,6 +5232,7 @@ either @file{hello-linux.lo} or @file{hello-@-generic.lo}.
 
 Or we could simply use an Automake conditional as follows.
 
+@c Keep in sync with ltcond2.test.
 @example
 lib_LTLIBRARIES = libhello.la
 libhello_la_SOURCES = hello-common.c
@@ -5257,6 +5272,7 @@ dependency anywhere it won't be built (this is why
 Here is a sample setup merging libtool convenience libraries from
 subdirectories into one main @file{libtop.la} library.
 
+@c Keep in sync with ltconv.test.
 @example
 # -- Top-level Makefile.am --
 SUBDIRS = sub1 sub2 @dots{}
@@ -7475,6 +7491,7 @@ variable explicitly prevents byte-compilation.
 
 Since Automake 1.8, we now recommend using @code{lisp_DATA} instead:
 
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @example
 lisp_DATA = file1.el file2.el
 @end example
@@ -7528,6 +7545,7 @@ files are not included in the distribution, you should use the
 Here is a typical setup for distributing @file{.java} files and
 installing the @file{.class} files resulting from their compilation.
 
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @example
 javadir = $(datadir)/java
 dist_java_JAVA = a.java b.java @dots{}
@@ -7601,6 +7619,7 @@ Automake ships with an Autoconf macro called @code{AM_PATH_PYTHON}
 that will determine some Python-related directory variables (see
 below).  If you have called @code{AM_PATH_PYTHON} from
 @file{configure.ac}, then you may use the variables
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @code{python_PYTHON} or @code{pkgpython_PYTHON} to list Python source
 files in your @file{Makefile.am}, depending on where you want your files
 installed (see the definitions of @code{pythondir} and
@@ -7689,6 +7708,7 @@ This is the directory where Python extension modules (shared libraries)
 should be installed.  An extension module written in C could be declared
 as follows to Automake:
 
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
 @example
 pyexec_LTLIBRARIES = quaternion.la
 quaternion_la_SOURCES = quaternion.c support.c support.h
@@ -7885,6 +7905,7 @@ passed to @code{makeinfo} when building @file{.info} files; and
 @samp{$(AM_MAKEINFOHTMLFLAGS)} is used when building @file{.html}
 files.
 
+@c Keep in sync with txinfo21.test.
 For instance, the following setting can be used to obtain one single
 @file{.html} file per manual, without node separators.
 @example
@@ -8110,10 +8131,11 @@ Variables using the standard directory prefixes @samp{bin},
 For instance, @code{data_DATA} files are installed by @code{install-data},
 while @code{bin_PROGRAMS} files are installed by @code{install-exec}.
 
-Any variable using a user-defined directory prefix with @samp{exec} in
-the name (e.g., @code{myexecbin_PROGRAMS}) is installed by
-@code{install-exec}.  All other user-defined prefixes are installed by
-@code{install-data}.
+Any variable using a user-defined directory prefix with
+@samp{exec} in the name (e.g.,
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
+@code{myexecbin_PROGRAMS}) is installed by @code{install-exec}.  All
+other user-defined prefixes are installed by @code{install-data}.
 
 @node Extending Installation
 @section Extending Installation
@@ -8298,7 +8320,7 @@ included if they are found in the current directory (either physically,
 or as the target of a @file{Makefile.am} rule); this list is printed by
 @samp{automake --help}.  Note that some files in this list are actually
 distributed only if other certain conditions hold (for example,
-@c The following example is covered by autodist-config-headers.test.
+@c Keep in sync with autodist-config-headers.test.
 the @file{config.h.top} and @file{config.h.bot} files are automatically
 distributed only if, e.g., @samp{AC_CONFIG_HEADERS([config.h])} is used
 in @file{configure.ac}).  Also, files that are read by @command{configure}
@@ -8488,6 +8510,7 @@ If you want @code{distcleancheck} to ignore built files that have not
 been cleaned because they are also part of the distribution, add the
 following definition instead:
 
+@c Keep in sync with distcleancheck.test.
 @example
 distcleancheck_listfiles = \
   find . -type f -exec sh -c 'test -f $(srcdir)/$$1 || echo $$1' \
@@ -8636,6 +8659,7 @@ set @code{TESTS_ENVIRONMENT} to an invocation of the shell (e.g.
 interpreter.  For instance, the following setup may be used to run tests
 with Perl:
 
+@c Keep in sync with tests-environment-backcompat.test.
 @example
 TESTS_ENVIRONMENT = $(PERL) -Mstrict -w
 TESTS = foo.pl bar.pl baz.pl
@@ -8731,6 +8755,7 @@ this extension to be called with this driver.  For all tests without a
 registered extension, the variables @code{LOG_COMPILER},
 @code{AM_LOG_FLAGS}, and @code{LOG_FLAGS} may be used.  For example,
 
+@c Keep in sync with parallel-tests-log-compiler-example.test.
 @example
 TESTS = foo.pl bar.py baz
 TEST_EXTENSIONS = .pl .py
@@ -8804,6 +8829,7 @@ whose testsuite takes long time to execute.  Luckily, this problem can
 easily be avoided by overriding also @code{TEST_SUITE_LOG} at runtime;
 for example,
 
+@c Keep in sync with parallel-tests-log-override-2.test.
 @example
 env TEST_SUITE_LOG=partial.log TESTS="..." make -e check
 @end example
@@ -9507,6 +9533,7 @@ For instance, the following definition prevents Automake from misinterpreting
 the @samp{.idlC.cpp:} rule as an attempt to transform @file{.idlC} files into
 @file{.cpp} files.
 
+@c Keep in sync with suffix7.test.
 @example
 SUFFIXES = .idl C.cpp
 .idlC.cpp:
@@ -10335,6 +10362,7 @@ Checks, autoconf, The Autoconf Manual}) and use @samp{$(LN_S)} in
 For instance, here is how you could install a versioned copy of a
 program using @samp{$(LN_S)}:
 
+@c Keep in sync with insthook.test
 @example
 install-exec-hook:
         cd $(DESTDIR)$(bindir) && \
@@ -10349,7 +10377,9 @@ destination directory in order to create relative links.
 When writing @code{install-exec-hook} or @code{install-data-hook},
 please bear in mind that the exec/data distinction is based on the
 installation directory, not on the primary used (@pxref{The Two Parts of
-Install}).  So a @code{foo_SCRIPTS} will be installed by
+Install}).
+@c Keep in sync with primary-prefix-couples-documented-valid.test.
+So a @code{foo_SCRIPTS} will be installed by
 @code{install-data}, and a @code{barexec_SCRIPTS} will be installed by
 @code{install-exec}.  You should define your hooks consequently.
 
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 4f78e9a..8da5a21 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -85,6 +85,9 @@ alloca.test \
 alloca2.test \
 alpha.test \
 alpha2.test \
+amhello-cflags.test \
+amhello-cross-compile.test \
+amhello-binpkg.test \
 amassign.test \
 ammissing.test \
 amopt.test \
@@ -157,6 +160,7 @@ check11.test \
 check-exported-srcdir.test \
 check-tests-in-builddir.test \
 check-tests_environment.test \
+tests-environment-backcompat.test \
 checkall.test \
 clean.test \
 clean2.test \
@@ -600,6 +604,7 @@ parallel-tests-subdir.test \
 parallel-tests-log-override-1.test \
 parallel-tests-log-override-2.test \
 parallel-tests-log-override-recheck.test \
+parallel-tests-log-compiler-example.test \
 parse.test \
 percent.test \
 percent2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index a6ff6f5..1504233 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -362,6 +362,9 @@ alloca.test \
 alloca2.test \
 alpha.test \
 alpha2.test \
+amhello-cflags.test \
+amhello-cross-compile.test \
+amhello-binpkg.test \
 amassign.test \
 ammissing.test \
 amopt.test \
@@ -434,6 +437,7 @@ check11.test \
 check-exported-srcdir.test \
 check-tests-in-builddir.test \
 check-tests_environment.test \
+tests-environment-backcompat.test \
 checkall.test \
 clean.test \
 clean2.test \
@@ -877,6 +881,7 @@ parallel-tests-subdir.test \
 parallel-tests-log-override-1.test \
 parallel-tests-log-override-2.test \
 parallel-tests-log-override-recheck.test \
+parallel-tests-log-compiler-example.test \
 parse.test \
 percent.test \
 percent2.test \
diff --git a/tests/README b/tests/README
index 57b2ddb..ca17c4f 100644
--- a/tests/README
+++ b/tests/README
@@ -93,6 +93,13 @@ Do
   Cite the PR number (if any), and the original reporter (if any), so
   we can find or ask for information if needed.
 
+  If a test checks examples or idioms given in the documentation, make
+  sure the documentation reference them appropriately in comments, as in:
+    @c Keep in sync with autodist-config-headers.test.
+    @example
+    ...
+    @end example
+
   Use `required=...' for required tools.  Do not explicitly require
   tools which can be taken for granted because they're listed in the
   GNU Coding Standards (for example, `gzip').
diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
new file mode 100755
index 0000000..8da3495
--- /dev/null
+++ b/tests/amhello-binpkg.test
@@ -0,0 +1,44 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Document an example from the manual about the `amhello' package:
+# using DESDIR to build simple, no-frills binary packages.
+
+required=i586-mingw32msvc-gcc
+. ./defs || Exit 1
+
+set -e
+
+cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
+  || fatal_ "cannot get amhello tarball"
+
+tar zxf amhello-1.0.tar.gz
+cd amhello-1.0
+
+./configure --prefix /usr
+make
+make DESTDIR="`pwd`/inst" install
+cd inst
+find . -type f -print > ../files.lst
+tar cvf amhello-1.0-i686.tar.gz `cat ../files.lst` > t
+LC_ALL=C sort t > tar.got
+
+diff - tar.got <<'END'
+./usr/bin/hello
+./usr/share/doc/amhello/README
+END
+
+:
diff --git a/tests/amhello-cflags.test b/tests/amhello-cflags.test
new file mode 100755
index 0000000..5b36fb2
--- /dev/null
+++ b/tests/amhello-cflags.test
@@ -0,0 +1,49 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Document an example from the manual about the `amhello' package:
+# using VPATH builds to configure the same package with different options.
+
+required=gcc
+. ./defs || Exit 1
+
+set -e
+
+cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
+  || fatal_ "cannot get amhello tarball"
+
+tar zxf amhello-1.0.tar.gz
+cd amhello-1.0
+mkdir debug optim
+cd debug
+../configure CFLAGS='-g -O0'
+$MAKE
+ls -l . src # For debugging.
+cd ../optim
+../configure CFLAGS='-O3 -fomit-frame-pointer'
+$MAKE
+ls -l . src # For debugging.
+cd ..
+
+# Check that we have really compiled the objects with two different
+# configurations.
+for exeext in '' .exe :; do
+  test -f optim/src/hello$exeext && break
+  test "$exeext" = : && fatal_ "cannot determine extension of executables"
+done
+cmp optim/src/hello$exeext debug/src/hello$exeext && Exit 1
+
+:
diff --git a/tests/amhello-cross-compile.test b/tests/amhello-cross-compile.test
new file mode 100755
index 0000000..8859ce4
--- /dev/null
+++ b/tests/amhello-cross-compile.test
@@ -0,0 +1,54 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Document an example from the manual about the `amhello' package:
+# cross-compiling a package from Linux/Unix to MinGW.
+
+required=i586-mingw32msvc-gcc
+. ./defs || Exit 1
+
+set -e
+
+cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
+  || fatal_ "cannot get amhello tarball"
+
+host=i586-mingw32msvc
+build=`"$testsrcdir"/../lib/config.guess` && test -n "$build" \
+  || fatal_ "cannot guess build platform"
+case $build in *mingw*) skip_ "build system is MinGW too";; esac
+
+tar zxf amhello-1.0.tar.gz
+cd amhello-1.0
+
+./configure --build "$build" --host "$host" > stdout \
+  || { cat stdout ; Exit 1; }
+cat stdout
+grep '^checking for i586-mingw32msvc-strip\.\.\.' stdout
+grep '^checking for i586-mingw32msvc-gcc\.\.\.' stdout
+grep '^checking for suffix of executables\.\.\. \.exe *$' stdout
+grep '^checking for suffix of object files\.\.\. o *$' stdout
+grep '^checking whether i586-mingw32msvc-gcc accepts -g\.\.\. yes' stdout
+
+$MAKE
+
+cd src
+file hello.exe > whatis
+cat whatis
+$EGREP 'DOS|Win' whatis
+grep 'executable' whatis
+grep 'ELF' whatis && Exit 1
+
+:
diff --git a/tests/interp.test b/tests/interp.test
index af82e33..05464d2 100755
--- a/tests/interp.test
+++ b/tests/interp.test
@@ -18,17 +18,23 @@
 
 . ./defs || Exit 1
 
+set -e
+
 cat >> configure.in << 'END'
 AC_PROG_CC
 END
 
 cat > Makefile.am << 'END'
-bin_PROGRAMS = qqq
-s1 = z.c
-qqq_SOURCES = $(s1)
+bin_PROGRAMS = foo
+xs = a.c b.c
+foo_SOURCES = c.c $(xs)
 END
 
-$ACLOCAL || Exit 1
-$AUTOMAKE || Exit 1
+$ACLOCAL
+$AUTOMAKE
+
+$FGREP ' a.$(OBJEXT)' Makefile.in
+$FGREP ' b.$(OBJEXT)' Makefile.in
+$FGREP ' c.$(OBJEXT)' Makefile.in
 
-$FGREP 'z.$(OBJEXT)' Makefile.in
+:
diff --git a/tests/parallel-tests-log-compiler-example.test b/tests/parallel-tests-log-compiler-example.test
new file mode 100755
index 0000000..746c128
--- /dev/null
+++ b/tests/parallel-tests-log-compiler-example.test
@@ -0,0 +1,71 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test the example of usage of generic and extension-specific
+# LOG_COMPILER and LOG_FLAGS given in the manual.
+
+parallel_tests=yes
+required=python
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in <<END
+AC_SUBST([PERL], ['$PERL'])
+AM_PATH_PYTHON
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS = foo.pl bar.py baz
+TEST_EXTENSIONS = .pl .py
+PL_LOG_COMPILER = $(PERL)
+AM_PL_LOG_FLAGS = -w
+PY_LOG_COMPILER = $(PYTHON)
+AM_PY_LOG_FLAGS = -v
+LOG_COMPILER = ./wrapper-script
+AM_LOG_FLAGS = -d
+END
+
+echo 'my $a =+ 2; exit (0);' > foo.pl
+echo 'import sys; sys.exit(0);' > bar.py
+: > baz
+
+cat > wrapper-script <<'END'
+#!/bin/sh
+echo "wrapper args: $*"
+END
+chmod a+x wrapper-script
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+st=0
+$MAKE check || st=$?
+cat foo.log
+cat bar.log
+cat baz.log
+test $st -eq 0 || Exit $st
+
+# Check that the wrappers have been run with the expected flags.
+grep '[rR]eversed.*+=.*operator.*foo\.pl' foo.log
+grep '^# *[cC]lear.*sys\.argv' bar.log
+grep '^wrapper args:.* -d .*baz' baz.log
+
+:
diff --git a/tests/specflg8.test b/tests/specflg8.test
index 1abcaa2..79daa15 100755
--- a/tests/specflg8.test
+++ b/tests/specflg8.test
@@ -1,5 +1,5 @@
 #! /bin/sh
-# Copyright (C) 2002, 2004  Free Software Foundation, Inc.
+# Copyright (C) 2002, 2004, 2011 Free Software Foundation, Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -34,25 +34,18 @@ END
 # different flags.
 
 cat > Makefile.am << 'END'
-FALSESOURCE = false.c
+TRUESOURCE = true.c
 bin_PROGRAMS = false true
-true_SOURCES = $(FALSESOURCE)
-true_CPPFLAGS = -DAM_TRUE
-false_SOURCES = $(FALSESOURCE)
-false_CPPFLAGS = -DAM_FALSE
+true_SOURCES = $(TRUESOURCE)
+true_CPPFLAGS = -DEXIT_CODE=0
+false_SOURCES = $(TRUESOURCE)
+false_CPPFLAGS = -DEXIT_CODE=1
 END
 
-cat > false.c << 'END'
-#include <stdio.h>
-int
-main (int argc, char *argv[])
+cat > true.c << 'END'
+int main (void)
 {
-#ifdef AM_TRUE
-   puts ("true");
-#else
-   puts ("false");
-#endif
-   return 0;
+   return EXIT_CODE;
 }
 END
 
@@ -62,5 +55,11 @@ $AUTOMAKE -a
 
 ./configure
 $MAKE
-./true | grep true
-./false | grep false
+
+./true
+./false && Exit 1
+
+test -f ./true-true.o
+test -f ./true-true.o
+
+:
diff --git a/tests/tests-environment-backcompat.test b/tests/tests-environment-backcompat.test
new file mode 100755
index 0000000..6749bc2
--- /dev/null
+++ b/tests/tests-environment-backcompat.test
@@ -0,0 +1,65 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# With old serial testsuite driver, TESTS_ENVIRONMENT can be used to
+# define the "test runner", i.e. the program that the test scripts must
+# be run by (with the parallel-tests driver one should use LOG_COMPILER
+# for this).  The behaviour tested here is also documented in the manual.
+
+parallel_tests=no
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in <<END
+AC_SUBST([PERL], ['$PERL'])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TESTS_ENVIRONMENT = $(PERL) -Mstrict -w
+TESTS = foo.pl bar.pl baz.pl
+XFAIL_TESTS = baz.pl
+EXTRA_DIST = $(TESTS)
+END
+
+echo 'exit (0);' > foo.pl
+echo 'exit (0);' > bar.pl
+
+cat > baz.pl << 'END'
+# With "use strict" enacted, this will cause an error, since the
+# variable `$x' is not declared with `my' nor specified with an
+# explicit package name.
+$x = 0;
+exit ($x);
+END
+
+cat > baz.pl << 'END'
+exit (1);
+END
+
+chmod a+x *.pl
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+$MAKE check
+$MAKE distcheck
+
+:
diff --git a/tests/txinfo21.test b/tests/txinfo21.test
index 5644c0b..ae1d985 100755
--- a/tests/txinfo21.test
+++ b/tests/txinfo21.test
@@ -118,8 +118,8 @@ test ! -f rec/main3.html
 
 # Make sure AM_MAKEINFOHTMLFLAGS is supported, and override AM_MAKEINFO.
 cat >>Makefile.am <<\EOF
-AM_MAKEINFOHTMLFLAGS=--no-split
-AM_MAKEINFOFLAGS=--unsupported-option
+AM_MAKEINFOHTMLFLAGS = --no-headers --no-split
+AM_MAKEINFOFLAGS = --unsupported-option
 EOF
 $AUTOMAKE
 ./configure --prefix "`pwd`"
-- 
1.7.2.3

Message #11 received at 8969 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: 8969 <at> debbugs.gnu.org
Cc: automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Fri, 1 Jul 2011 10:10:43 +0200

On Friday 01 July 2011, Stefano Lattarini wrote:
> On Friday 01 July 2011, Stefano Lattarini wrote:
> > I'll soon post a patch that improves the synchronization between
> > some examples from the manual and some test cases.  In the process
> > of writing it, I've noticed that various non-trivial examples from
> > the manual are not tested; and while I've remedied to part of this
> > problem with my patch, a lot remains to be done; so I'm opening
> > this bug to avoid forgetting about the issue.
> > 
> And here it is the promised patch.  I will push in 72 hours if there
> is no objection by then.
> 
Oops, one of the "amhello" tests has an unecessary requirement (copy &
paste blunder).  Consider the following squashed in:

  diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
  index 8da3495..47add73 100755
  --- a/tests/amhello-binpkg.test
  +++ b/tests/amhello-binpkg.test
  @@ -17,7 +17,6 @@
   # Document an example from the manual about the `amhello' package:
   # using DESDIR to build simple, no-frills binary packages.
   
  -required=i586-mingw32msvc-gcc
   . ./defs || Exit 1
   
   set -e

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Peter Breitenlohner <peb <at> mppmu.mpg.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 8969 <at> debbugs.gnu.org, owner <at> debbugs.gnu.org, bug-automake <at> gnu.org,
	automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Fri, 1 Jul 2011 11:13:49 +0200 (CEST)

On Fri, 1 Jul 2011, Stefano Lattarini wrote:

> Oops, one of the "amhello" tests has an unecessary requirement (copy &
> paste blunder).  Consider the following squashed in:
>
>  diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
>  index 8da3495..47add73 100755
>  --- a/tests/amhello-binpkg.test
>  +++ b/tests/amhello-binpkg.test
>  @@ -17,7 +17,6 @@
>   # Document an example from the manual about the `amhello' package:
>   # using DESDIR to build simple, no-frills binary packages.

Hi Stefano,

could this be a typo (DESDIR vs DESTDIR)?

Regards
Peter Breitenlohner <peb <at> mppmu.mpg.de>

Message #20 received at 8969 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Peter Breitenlohner <peb <at> mppmu.mpg.de>
Cc: 8969 <at> debbugs.gnu.org, automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Fri, 1 Jul 2011 11:34:26 +0200

Hello Peter.

On Friday 01 July 2011, Peter Breitenlohner wrote:
> On Fri, 1 Jul 2011, Stefano Lattarini wrote:
> 
> > Oops, one of the "amhello" tests has an unecessary requirement (copy &
> > paste blunder).  Consider the following squashed in:
> >
> >  diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
> >  index 8da3495..47add73 100755
> >  --- a/tests/amhello-binpkg.test
> >  +++ b/tests/amhello-binpkg.test
> >  @@ -17,7 +17,6 @@
> >   # Document an example from the manual about the `amhello' package:
> >   # using DESDIR to build simple, no-frills binary packages.
> 
> Hi Stefano,
> 
> could this be a typo (DESDIR vs DESTDIR)?
>
Yes it is; fixed now.  Thanks for spotting it!

And since we are out to hunt typos, all the new amhello tests have the
same error in the heading descriptions, using: 
  "Document an example from the manual about the `amhello' package:"
where this should be used instead:
  "Test an example from the manual about the `amhello' package:"
Consider these fixed too.

Thanks,
  Stefano

Message #23 received at 8969 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: Peter Breitenlohner <peb <at> mppmu.mpg.de>
Cc: 8969 <at> debbugs.gnu.org, automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Sun, 3 Jul 2011 15:49:21 +0200

On Friday 01 July 2011, Stefano Lattarini wrote:
> Hello Peter.
> 
> On Friday 01 July 2011, Peter Breitenlohner wrote:
> > On Fri, 1 Jul 2011, Stefano Lattarini wrote:
> > 
> > > Oops, one of the "amhello" tests has an unecessary requirement (copy &
> > > paste blunder).  Consider the following squashed in:
> > >
> > >  diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
> > >  index 8da3495..47add73 100755
> > >  --- a/tests/amhello-binpkg.test
> > >  +++ b/tests/amhello-binpkg.test
> > >  @@ -17,7 +17,6 @@
> > >   # Document an example from the manual about the `amhello' package:
> > >   # using DESDIR to build simple, no-frills binary packages.
> > 
> > Hi Stefano,
> > 
> > could this be a typo (DESDIR vs DESTDIR)?
> >
> Yes it is; fixed now.  Thanks for spotting it!
> 
> And since we are out to hunt typos, all the new amhello tests have the
> same error in the heading descriptions, using: 
>   "Document an example from the manual about the `amhello' package:"
> where this should be used instead:
>   "Test an example from the manual about the `amhello' package:"
> Consider these fixed too.
> 
> Thanks,
>   Stefano
>
I've applied this patch to maint, merged maint into master, merged master
into testsuite-work, and pushed.

Regards,
  Stefano

Message #26 received at 8969 <at> debbugs.gnu.org (full text, mbox):

From: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 8969 <at> debbugs.gnu.org, automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Wed, 06 Jul 2011 08:40:10 +0200

* Stefano Lattarini wrote on Fri, Jul 01, 2011 at 12:59:53AM CEST:
> Subject: [PATCH] docs, tests: synchronize examples from docs to tests
> 
> * tests/README (Writing test cases): Give suggestions on how to
> keep test cases and examples in the documentation synchronized.
> * doc/automake.texi: Improve or fix existing testcase-referencing
> comments, and add many new ones.
> * HACKING (Administrivia): Suggest to test complex examples and
> idioms from the manual.
> * tests/specflg8.test: Improve synchronization with the example
> in the manual.
> * tests/output11.test:Likewise.
> * tests/txinfo21.test:Likewise.
> * tests/interp.test: Likewise.  Since we are at it, and enable
> the `errexit' shell flag, do related changes, and add trailing
> `:'command.
> * tests/amhello-cflags.test: New test.
> * tests/amhello-cross-compile.test: Likewise.
> * tests/amhello-binpkg.test: Likewise.
> * tests/tests-environment-backcompat: Likewise.
> * tests/parallel-tests-log-compiler-example.test: Likewise.
> * tests/Makefile.am (TESTS): Update.


> --- /dev/null
> +++ b/tests/amhello-binpkg.test
> @@ -0,0 +1,44 @@

> +# Document an example from the manual about the `amhello' package:
> +# using DESDIR to build simple, no-frills binary packages.

(DESTDIR was already fixed IIUC)

> +required=i586-mingw32msvc-gcc
> +. ./defs || Exit 1
> +
> +set -e
> +
> +cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
> +  || fatal_ "cannot get amhello tarball"
> +
> +tar zxf amhello-1.0.tar.gz

The z flag is not portable to all tars, the portable spelling is
  gzip -dc amhello-1.0.tar.gz | tar xf -

(several instances).  I'm ok with fixing the manual also, although it's
usually clear for people still having to use those vendor tars (and
inconvenient for the rest).

> +cd amhello-1.0
> +
> +./configure --prefix /usr
> +make
> +make DESTDIR="`pwd`/inst" install
> +cd inst
> +find . -type f -print > ../files.lst
> +tar cvf amhello-1.0-i686.tar.gz `cat ../files.lst` > t
> +LC_ALL=C sort t > tar.got
> +
> +diff - tar.got <<'END'
> +./usr/bin/hello
> +./usr/share/doc/amhello/README
> +END
[...]

Nice patch btw!

Thanks,
Ralf

Message #29 received at 8969 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
Cc: 8969 <at> debbugs.gnu.org, automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Wed, 6 Jul 2011 10:32:56 +0200

[Message part 1 (text/plain, inline)]

On Wednesday 06 July 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Fri, Jul 01, 2011 at 12:59:53AM CEST:
> > Subject: [PATCH] docs, tests: synchronize examples from docs to tests
> > 
> > * tests/README (Writing test cases): Give suggestions on how to
> > keep test cases and examples in the documentation synchronized.
> > * doc/automake.texi: Improve or fix existing testcase-referencing
> > comments, and add many new ones.
> > * HACKING (Administrivia): Suggest to test complex examples and
> > idioms from the manual.
> > * tests/specflg8.test: Improve synchronization with the example
> > in the manual.
> > * tests/output11.test:Likewise.
> > * tests/txinfo21.test:Likewise.
> > * tests/interp.test: Likewise.  Since we are at it, and enable
> > the `errexit' shell flag, do related changes, and add trailing
> > `:'command.
> > * tests/amhello-cflags.test: New test.
> > * tests/amhello-cross-compile.test: Likewise.
> > * tests/amhello-binpkg.test: Likewise.
> > * tests/tests-environment-backcompat: Likewise.
> > * tests/parallel-tests-log-compiler-example.test: Likewise.
> > * tests/Makefile.am (TESTS): Update.
> 
> 
> > --- /dev/null
> > +++ b/tests/amhello-binpkg.test
> > @@ -0,0 +1,44 @@
> 
> > +# Document an example from the manual about the `amhello' package:
> > +# using DESDIR to build simple, no-frills binary packages.
> 
> (DESTDIR was already fixed IIUC)
> 
> > +required=i586-mingw32msvc-gcc
> > +. ./defs || Exit 1
> > +
> > +set -e
> > +
> > +cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
> > +  || fatal_ "cannot get amhello tarball"
> > +
> > +tar zxf amhello-1.0.tar.gz
> 
> The z flag is not portable to all tars, the portable spelling is
>   gzip -dc amhello-1.0.tar.gz | tar xf -
> 
> (several instances).
>
Oops, sorry.  Fixed by the attached patch.  OK for maint?  I'll wait
the customary 72 hours before pushing.

> I'm ok with fixing the manual also, although it's
> usually clear for people still having to use those vendor tars (and
> inconvenient for the rest).
>
Yes, I'd say we leave the examples in manual untouched.  Agreed?

> > +cd amhello-1.0
> > +
> > +./configure --prefix /usr
> > +make
> > +make DESTDIR="`pwd`/inst" install
> > +cd inst
> > +find . -type f -print > ../files.lst
> > +tar cvf amhello-1.0-i686.tar.gz `cat ../files.lst` > t
> > +LC_ALL=C sort t > tar.got
> > +
> > +diff - tar.got <<'END'
> > +./usr/bin/hello
> > +./usr/share/doc/amhello/README
> > +END
> [...]
> 
> Nice patch btw!
>
Thanks :-)

Regards,
  Stefano

[0001-tests-portability-fixes-in-tests-on-amhello-examples.patch (text/x-patch, inline)]

From d7e03b029b3ae322ed719ef501810d7334eaf288 Mon Sep 17 00:00:00 2001
Message-Id: <d7e03b029b3ae322ed719ef501810d7334eaf288.1309941072.git.stefano.lattarini <at> gmail.com>
From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Date: Wed, 6 Jul 2011 10:31:03 +0200
Subject: [PATCH] tests: portability fixes in tests on amhello examples

* tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
extract a gzip-compressed tarball, that's unportable to some
tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
idiom instead.
* tests/amhello-cflags.test: Likewise.
* tests/amhello-cross-compile.test: Likewise.

Suggestion from Ralf Wildenhues.
---
 ChangeLog                        |   11 +++++++++++
 tests/amhello-binpkg.test        |    3 ++-
 tests/amhello-cflags.test        |    4 +++-
 tests/amhello-cross-compile.test |    3 ++-
 4 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index c77326d..c6f8d05 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2011-07-06  Stefano Lattarini  <stefano.lattarini <at> gmail.com>
+
+	tests: portability fixes in tests on amhello examples
+	* tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
+	extract a gzip-compressed tarball, that's unportable to some
+	tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
+	idiom instead.
+	* tests/amhello-cflags.test: Likewise.
+	* tests/amhello-cross-compile.test: Likewise.
+	Suggestion from Ralf Wildenhues.
+
 2011-07-01  Stefano Lattarini  <stefano.lattarini <at> gmail.com>
 
 	docs, tests: synchronize examples from docs to tests
diff --git a/tests/amhello-binpkg.test b/tests/amhello-binpkg.test
index 8085a23..951f713 100755
--- a/tests/amhello-binpkg.test
+++ b/tests/amhello-binpkg.test
@@ -24,7 +24,8 @@ set -e
 cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
   || fatal_ "cannot get amhello tarball"
 
-tar zxf amhello-1.0.tar.gz
+# "tar zxf" in unportable to some tar implementations.
+gzip -dc amhello-1.0.tar.gz | tar xf -
 cd amhello-1.0
 
 ./configure --prefix /usr
diff --git a/tests/amhello-cflags.test b/tests/amhello-cflags.test
index 1c3e516..25b0a52 100755
--- a/tests/amhello-cflags.test
+++ b/tests/amhello-cflags.test
@@ -26,8 +26,10 @@ set -e
 cp "$testsrcdir"/../doc/amhello-1.0.tar.gz . \
   || fatal_ "cannot get amhello tarball"
 
-tar zxf amhello-1.0.tar.gz
+# "tar zxf" in unportable to some tar implementations.
+gzip -dc amhello-1.0.tar.gz | tar xf -
 cd amhello-1.0
+
 mkdir debug optim
 cd debug
 ../configure CFLAGS='-g -O0'
diff --git a/tests/amhello-cross-compile.test b/tests/amhello-cross-compile.test
index 862a079..7c05fc0 100755
--- a/tests/amhello-cross-compile.test
+++ b/tests/amhello-cross-compile.test
@@ -30,7 +30,8 @@ build=`"$testsrcdir"/../lib/config.guess` && test -n "$build" \
   || fatal_ "cannot guess build platform"
 case $build in *mingw*) skip_ "build system is MinGW too";; esac
 
-tar zxf amhello-1.0.tar.gz
+# "tar zxf" in unportable to some tar implementations.
+gzip -dc amhello-1.0.tar.gz | tar xf -
 cd amhello-1.0
 
 ./configure --build "$build" --host "$host" > stdout \
-- 
1.7.2.3

Message #32 received at 8969 <at> debbugs.gnu.org (full text, mbox):

From: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
To: Stefano Lattarini <stefano.lattarini <at> gmail.com>
Cc: 8969 <at> debbugs.gnu.org, automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Wed, 06 Jul 2011 21:53:49 +0200

* Stefano Lattarini wrote on Wed, Jul 06, 2011 at 10:32:56AM CEST:
> Oops, sorry.  Fixed by the attached patch.  OK for maint?  I'll wait
> the customary 72 hours before pushing.

Thanks.  OK, but please remove all the comments about tar unportability
in the tests.  It's so obvious.  ;-)
(and there are probably a dozen other places in the Automake source tree
that you'd have to put the comment at as well, for any amount of
consistency.  I think having one comment, the one that already exists in
lib/am/distdir.am, is fully sufficient.)

> > I'm ok with fixing the manual also, although it's
> > usually clear for people still having to use those vendor tars (and
> > inconvenient for the rest).
> >
> Yes, I'd say we leave the examples in manual untouched.  Agreed?

OK.

> Subject: [PATCH] tests: portability fixes in tests on amhello examples
> 
> * tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
> extract a gzip-compressed tarball, that's unportable to some
> tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
> idiom instead.
> * tests/amhello-cflags.test: Likewise.
> * tests/amhello-cross-compile.test: Likewise.

Thanks,
Ralf

Message #35 received at 8969 <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: "Ralf Wildenhues" <Ralf.Wildenhues <at> gmx.de>
Cc: 8969 <at> debbugs.gnu.org, automake-patches <at> gnu.org
Subject: Re: bug#8969: improve synchronization between examples in the manual
	and test cases
Date: Fri, 8 Jul 2011 09:58:58 +0200

On Wednesday 06 July 2011, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Wed, Jul 06, 2011 at 10:32:56AM CEST:
> > Oops, sorry.  Fixed by the attached patch.  OK for maint?  I'll wait
> > the customary 72 hours before pushing.
> 
> Thanks.  OK, but please remove all the comments about tar unportability
> in the tests.  It's so obvious.  ;-)
>
OK done.

> (and there are probably a dozen other places in the Automake source tree
> that you'd have to put the comment at as well, for any amount of
> consistency.  I think having one comment, the one that already exists in
> lib/am/distdir.am, is fully sufficient.)
> 
> > > I'm ok with fixing the manual also, although it's
> > > usually clear for people still having to use those vendor tars (and
> > > inconvenient for the rest).
> > >
> > Yes, I'd say we leave the examples in manual untouched.  Agreed?
> 
> OK.
> 
> > Subject: [PATCH] tests: portability fixes in tests on amhello examples
> > 
> > * tests/amhello-binpkg.test: Don't use "tar xzf too.tag.gz" to
> > extract a gzip-compressed tarball, that's unportable to some
> > tar implementations; use the "gzip -dc fo.tar.gz | tar xf -"
> > idiom instead.
> > * tests/amhello-cflags.test: Likewise.
> > * tests/amhello-cross-compile.test: Likewise.
> 
> Thanks,
> Ralf
> 
Pushed now.

Thanks,
  Stefano

Message #40 received at 8969-done <at> debbugs.gnu.org (full text, mbox):

From: Stefano Lattarini <stefano.lattarini <at> gmail.com>
To: 8969-done <at> debbugs.gnu.org
Subject: close bug 8969
Date: Wed, 09 Jan 2013 14:10:41 +0100

Reference:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8969>

One and a half year ago, I wrote :

> I'll soon post a patch that improves the synchronization between
> some examples from the manual and some test cases.  In the process
> of writing it, I've noticed that various non-trivial examples from
> the manual are not tested; and while I've remedied to part of this
> problem with my patch, a lot remains to be done; so I'm opening
> this bug to avoid forgetting about the issue.

I now think that keeping a bug report open about this is overkill,
and introduce more noise rather than helping out.  So, since the
referenced patch has been since long written and applied, and the
issues originally spotted have been fixed, I'm closing the report.

Regards,
  Stefano

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.