Subject: [PATCH v3] tap: new `--stderr-prefix' option to prefix test script's stderr

From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 19:00:57 -0500

It took a couple of months, but FSF accepted the copyright assignment. 
And then I promptly forgot to follow up on this because I was in the 
middle of something else.  Sorry about that!

Supposedly I have not appeared in copyright.list for Automake yet.  (FSF 
signed on 2024-09-24, so I would expect to be in by now.)  Please let me 
know if I need to do anything, such as forward the mutually signed 
agreement PDF to someone.

Attached is v3 of the patch series.  Changes from v2:
   * Fixed a race condition
   * Fixed accidental discard of test script's final line if the output 
didn't end with a terminating newline
   * Moved the line prefixing logic to a separate function for 
readability and to make it easier to add comments for subtle points
   * The driver's own stderr is no longer mixed with the test script's 
stderr
   * Rewrote the new tests as *.tap
   * Docs now suggest `[stderr]' prefix instead of `<stderr>'
   * Minor improvements to Automake's own test infrastructure

Thanks,
Richard
From 788cab38c23cd013a93a66b42d1bbc66d3fdb81b Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:50 -0500
Subject: [PATCH 01/10] tests: turn off `set -x' in `diag_'

* t/ax/tap-functions.sh (diag_): Turn off `set -x' to reduce log spam.
---
 t/ax/tap-functions.sh | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/t/ax/tap-functions.sh b/t/ax/tap-functions.sh
index 41dd3f407..619cad183 100644
--- a/t/ax/tap-functions.sh
+++ b/t/ax/tap-functions.sh
@@ -74,9 +74,10 @@ planned_=none
 # done to allow better interplay with TAP drivers that allow such a string
 # to be configured.
 diag_ ()
-{
+(
+  set +x
   test $# -eq 0 || echo "$diag_string_ $*"
-}
+)
 
 # Used by the 'diag_' function above.  User-overridable.
 diag_string_="#"
-- 
2.48.1

From 7a47605461313937b8804e688f0908df9a0aeef7 Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:52 -0500
Subject: [PATCH 02/10] tests: prefix every line of diag_ text, not just the
 first

* t/ax/tap-function.sh (diag_): Prefix every line of the diagnostic
text, not just the first.
---
 t/ax/tap-functions.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/t/ax/tap-functions.sh b/t/ax/tap-functions.sh
index 619cad183..29b9a028c 100644
--- a/t/ax/tap-functions.sh
+++ b/t/ax/tap-functions.sh
@@ -76,7 +76,11 @@ planned_=none
 diag_ ()
 (
   set +x
-  test $# -eq 0 || echo "$diag_string_ $*"
+  test $# -eq 0 || while IFS= read -r line; do
+    echo "$diag_string_ $line"
+  done <<EOF
+$*
+EOF
 )
 
 # Used by the 'diag_' function above.  User-overridable.
-- 
2.48.1

From e448ac38694893aa71980757c5703702b6954dff Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:53 -0500
Subject: [PATCH 03/10] tests: use `printf' instead of `echo'

* t/ax/tap-functions (diag_): Use `printf' instead of `echo' in case
`echo' processes backslash escapes or the argument looks like an
option.
---
 t/ax/tap-functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/ax/tap-functions.sh b/t/ax/tap-functions.sh
index 29b9a028c..a458b773e 100644
--- a/t/ax/tap-functions.sh
+++ b/t/ax/tap-functions.sh
@@ -77,7 +77,7 @@ diag_ ()
 (
   set +x
   test $# -eq 0 || while IFS= read -r line; do
-    echo "$diag_string_ $line"
+    printf %s\\n "$diag_string_ $line"
   done <<EOF
 $*
 EOF
-- 
2.48.1

From 61b82d6c3f9e494e851c0bb53acc6e9b516a0b35 Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:54 -0500
Subject: [PATCH 04/10] tests: support getting `diag_' message from stdin

* t/ax/tap-functions.sh (diag_): Read the diagnostic message from
stdin if no arguments are given.
---
 t/ax/tap-functions.sh | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/t/ax/tap-functions.sh b/t/ax/tap-functions.sh
index a458b773e..a6e0cf4a4 100644
--- a/t/ax/tap-functions.sh
+++ b/t/ax/tap-functions.sh
@@ -69,18 +69,17 @@ planned_=none
 
 # diag_ [EXPLANATION]
 # ------------------
-# Report the given text as TAP diagnostic.  Assumes the string denoting
-# TAP diagnostic lines is stored in the '$diag_string_' variable; this is
-# done to allow better interplay with TAP drivers that allow such a string
-# to be configured.
+# Report the given text, or stdin if no arguments, as TAP diagnostic.
+# Assumes the string denoting TAP diagnostic lines is stored in the
+# '$diag_string_' variable; this is done to allow better interplay
+# with TAP drivers that allow such a string to be configured.
 diag_ ()
 (
   set +x
-  test $# -eq 0 || while IFS= read -r line; do
+  test $# -eq 0 || { printf %s\\n "$*" | diag_; return; }
+  while IFS= read -r line || [ -n "$line" ]; do
     printf %s\\n "$diag_string_ $line"
-  done <<EOF
-$*
-EOF
+  done
 )
 
 # Used by the 'diag_' function above.  User-overridable.
-- 
2.48.1

From 73d85af20829e6f4c5efbd5bdaf3b325d5f068c0 Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:54 -0500
Subject: [PATCH 05/10] tests: move space from `diag_()' to `$diag_string_'

* t/ax/tap-functions.sh (diag_): Moved the space separating
`$diag_string_' from the diagnostic message to `$diag_string_' so that
callers can eliminate the space.
---
 t/ax/tap-functions.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/ax/tap-functions.sh b/t/ax/tap-functions.sh
index a6e0cf4a4..0536db98c 100644
--- a/t/ax/tap-functions.sh
+++ b/t/ax/tap-functions.sh
@@ -78,12 +78,12 @@ diag_ ()
   set +x
   test $# -eq 0 || { printf %s\\n "$*" | diag_; return; }
   while IFS= read -r line || [ -n "$line" ]; do
-    printf %s\\n "$diag_string_ $line"
+    printf %s\\n "$diag_string_$line"
   done
 )
 
 # Used by the 'diag_' function above.  User-overridable.
-diag_string_="#"
+diag_string_="# "
 
 # warn_ [EXPLANATION]
 # ------------------
-- 
2.48.1

From 6b478dd4e361ca091689b548f5014f63479e8634 Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:55 -0500
Subject: [PATCH 06/10] tap: close fd 3 before invoking the test script

* lib/tap-driver.sh: Close file descriptor 3 before invoking the test
script to avoid potential conflicts with the test script.
---
 lib/tap-driver.sh | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/lib/tap-driver.sh b/lib/tap-driver.sh
index 75e767605..0bd79a6d4 100755
--- a/lib/tap-driver.sh
+++ b/lib/tap-driver.sh
@@ -23,7 +23,7 @@
 # bugs to <[email-masked]> or send patches to
 # <[email-masked]>.
 
-scriptversion=2024-12-03.03; # UTC
+scriptversion=2025-02-03.01; # UTC
 
 # Make unconditional expansion of undefined variables an error.  This
 # helps a lot in preventing typo-related bugs.
@@ -144,7 +144,7 @@ fi
     else
       exec 2>&3
     fi
-    "$@"
+    "$@" 3>&-
     echo $?
   ) | LC_ALL=C ${AM_TAP_AWK-awk} \
         -v me="$me" \
@@ -638,8 +638,6 @@ exit 0
 
 } # End of "BEGIN" block.
 '
-
-# TODO: document that we consume the file descriptor 3 :-(
 } 3>"$log_file"
 
 test $? -eq 0 || fatal "I/O or internal error"
-- 
2.48.1

From 00f6913f96ec1053c2f0e29a2987f37b2070efec Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:56 -0500
Subject: [PATCH 07/10] tap: minimize stderr redirection to just the test
 script

* lib/tap-driver.sh: Keep driver stderr separate from test script
stderr to aid with debugging (e.g., `set -x').
---
 lib/tap-driver.sh | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/lib/tap-driver.sh b/lib/tap-driver.sh
index 0bd79a6d4..93ee81823 100755
--- a/lib/tap-driver.sh
+++ b/lib/tap-driver.sh
@@ -23,7 +23,7 @@
 # bugs to <[email-masked]> or send patches to
 # <[email-masked]>.
 
-scriptversion=2025-02-03.01; # UTC
+scriptversion=2025-02-03.02; # UTC
 
 # Make unconditional expansion of undefined variables an error.  This
 # helps a lot in preventing typo-related bugs.
@@ -139,12 +139,18 @@ fi
     # <https://lists.gnu.org/archive/html/bug-autoconf/2011-09/msg00004.html>
     # <http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2009-February/004121.html>
     trap : 1 3 2 13 15
+    # Determine where to send the test script's stderr.  Only the
+    # test's stderr should go here; if `exec 2>&$stderr_fd' were run,
+    # this script's stderr (e.g., `set -x' output, if turned on to
+    # help with debugging) would mix with the test script's stderr and
+    # go to the log (via `awk', if `--merge' is enabled), not the
+    # terminal.
     if test $merge -gt 0; then
-      exec 2>&1
+      stderr_fd=1  # send stderr to awk, which will copy it to the log
     else
-      exec 2>&3
+      stderr_fd=3  # send stderr directly to the log file
     fi
-    "$@" 3>&-
+    "$@" 2>&$stderr_fd 3>&-
     echo $?
   ) | LC_ALL=C ${AM_TAP_AWK-awk} \
         -v me="$me" \
-- 
2.48.1

From 423e3ca59d3db4772fcc51ccd007cd670ea00737 Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:57 -0500
Subject: [PATCH 08/10] tap: new `--stderr-prefix' option to prefix test
 script's stderr

* lib/tap-driver.sh: Add a new `--stderr-prefix' option to instruct
the TAP driver to prefix each line of the test script's standard error
with the given string.  This is useful when `--merge' is enabled and
standard error lines might be confused with test results.
* doc/automake.texi: Document the new `--stderr-prefix' option.
* t/tap-stderr-prefix.tap: New test.
* t/list-of-tests.mk (handwritten_TESTS): Add it.
---
 doc/automake.texi       | 80 ++++++++++++++++++++++++++++++++++-
 lib/tap-driver.sh       | 75 ++++++++++++++++++++++++++++++---
 t/list-of-tests.mk      |  1 +
 t/tap-stderr-prefix.tap | 92 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 242 insertions(+), 6 deletions(-)
 create mode 100644 t/tap-stderr-prefix.tap

diff --git a/doc/automake.texi b/doc/automake.texi
index 03d99856c..6685ff6ae 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -10524,9 +10524,35 @@ relative to test results; this can be of great help in debugging
 (especially if your test scripts are shell scripts run with shell
 tracing active).  As a downside, this option might cause the test
 harness to get confused if anything that appears on standard error
-looks like a test result.
+looks like a test result.  Use @option{--stderr-prefix} to prefix each
+line of standard error to avoid this problem.
 @item --no-merge
 Revert the effects of @option{--merge}.
+@item --stderr-prefix @var{STRING}
+Prefix each line of the test script's standard error with
+@code{@var{STRING}}.  Defaults to the empty string.  This option makes
+it possible to avoid parsing ambiguity when @option{--merge} is in use,
+but it can also be used without @option{--merge} to make it easier to
+read test log files.  When used with @option{--merge}, users will
+typically want to start @code{@var{STRING}} with @samp{#}, which starts
+a TAP comment.  The @samp{#} character is difficult to portably include
+in a @command{make} variable, so the following quadrigraph
+substitutions, copied from Autoconf (@pxref{Quadrigraphs, , , autoconf,
+The Autoconf Manual}), are performed on @code{@var{STRING}}:
+
+@table @samp
+@item @@%:@@
+Replaced with @samp{#}.
+@item @@&t@@
+Replaced with the empty string.  This makes it possible to include the
+literal characters @samp{@@%:@@} in the prefix (@samp{@@%@@&t@@:@@}, for
+example).
+@end table
+
+For example, in shell, @samp{--stderr-prefix "@@%:@@[stderr] "} will
+prefix each line of the test script's standard error with
+@samp{#[stderr] }.
+
 @item --diagnostic-string @var{STRING}
 Change the string that introduces TAP diagnostics from the default value
 of ``@code{#}'' to @code{@var{STRING}}.  This can be useful if your
@@ -10614,6 +10640,58 @@ PASS: baz.test 1
 ...
 % @kbd{echo exit status: $?}
 exit status: 0
+
+% @kbd{cat stderr-demo.test}
+#!/bin/sh
+echo 'ok 1'
+# These sleeps help highlight the reordering of standard output and
+# standard error lines that can happen if --merge is not used.
+sleep 1
+# Note that this is a diagnostic message written to standard error, so
+# it should not affect the tests.  However, it will break the tests if
+# --merge is used without --stderr-prefix because "Bail out!" will be
+# interpreted as a TAP directive.
+echo 'Bail out! (this should not actually bail out)' >&2
+sleep 1
+echo 'ok 2'
+echo '1..2'
+
+% @kbd{make check TESTS=stderr-demo.test}
+...
+PASS: stderr-demo.test 1
+PASS: stderr-demo.test 2
+...
+
+% @kbd{cat stderr-demo.log}
+Bail out! (this should not actually bail out)
+ok 1
+PASS: stderr-demo.test 1
+ok 2
+PASS: stderr-demo.test 2
+1..2
+
+% @kbd{make check TESTS=stderr-demo.test \
+      TEST_LOG_DRIVER_FLAGS='--merge'}
+...
+PASS: stderr-demo.test 1
+ERROR: stderr-demo.test - Bail out! (this should not actually bail out)
+...
+
+% @kbd{make check TESTS=stderr-demo.test \
+      TEST_LOG_DRIVER_FLAGS='--merge --stderr-prefix "@@%:@@[stderr] "'}
+...
+PASS: stderr-demo.test 1
+PASS: stderr-demo.test 2
+...
+
+% @kbd{cat stderr-demo.log}
+ok 1
+PASS: stderr-demo.test 1
+#[stderr] Bail out! (this should not actually bail out)
+ok 2
+PASS: stderr-demo.test 2
+1..2
+
 @end example
 
 @node Incompatibility with other TAP parsers and drivers
diff --git a/lib/tap-driver.sh b/lib/tap-driver.sh
index 93ee81823..0c7653b69 100755
--- a/lib/tap-driver.sh
+++ b/lib/tap-driver.sh
@@ -23,7 +23,7 @@
 # bugs to <[email-masked]> or send patches to
 # <[email-masked]>.
 
-scriptversion=2025-02-03.02; # UTC
+scriptversion=2025-02-03.03; # UTC
 
 # Make unconditional expansion of undefined variables an error.  This
 # helps a lot in preventing typo-related bugs.
@@ -52,7 +52,8 @@ Usage:
                 [--expect-failure {yes|no}] [--color-tests {yes|no}]
                 [--enable-hard-errors {yes|no}] [--ignore-exit]
                 [--diagnostic-string STRING] [--merge|--no-merge]
-                [--comments|--no-comments] [--] TEST-COMMAND
+                [--stderr-prefix STRING] [--comments|--no-comments]
+                [--] TEST-COMMAND
 The '--test-name', '-log-file' and '--trs-file' options are mandatory.
 
 Report bugs to <[email-masked]>.
@@ -69,6 +70,7 @@ trs_file=  # Where to save the metadata of the test run.
 expect_failure=0
 color_tests=0
 merge=0
+stderr_prefix=
 ignore_exit=0
 comments=0
 diag_string='#'
@@ -84,6 +86,7 @@ while test $# -gt 0; do
   --enable-hard-errors) shift;; # No-op.
   --merge) merge=1;;
   --no-merge) merge=0;;
+  --stderr-prefix) stderr_prefix=$2; shift;;
   --ignore-exit) ignore_exit=1;;
   --comments) comments=1;;
   --no-comments) comments=0;;
@@ -94,6 +97,46 @@ while test $# -gt 0; do
   shift
 done
 
+# Quadrigraph substitutions for `--stderr-prefix'.  Note that the
+# empty substitution MUST be done last, otherwise `@%@&t@:@' will
+# become `#', not `@%:@'.
+for q_r in '@%:@ #' '@&t@ '; do
+  q=${q_r%% *} # quadrigraph
+  r=${q_r#* } # replacement
+  while true; do
+    case $stderr_prefix in
+    *"$q"*) stderr_prefix=${stderr_prefix%%"$q"*}$r${stderr_prefix#*"$q"};;
+    *) break;;
+    esac
+  done
+done
+
+# Prefixes each line of its stdin with the first argument and writes
+# the result to stdout.  If the final line of stdin is non-empty and
+# does not end with a terminating newline, a newline is added.
+prefix_lines() {
+  # Implementation note: This function is used to prefix the test
+  # script's stderr lines.  Preserving the order of the test script's
+  # stdout and stderr lines is important for debugging, so this
+  # function is sensitive to input and output buffering.  A shell loop
+  # is used to prefix the lines instead of `$AM_TAP_AWK' (which would
+  # probably be more efficient) because `mawk' aggressively buffers
+  # its input (except with the `-Winteractive' command-line option),
+  # which would defeat the purpose of the `--merge' option.  `sed' or
+  # `perl' could be used instead of a shell loop, but those would add
+  # a dependency to this script.
+
+  # <https://stackoverflow.com/a/6399568> explains `IFS='.  The `||'
+  # check ensures that stdin's final line is written to stdout even if
+  # it is missing a terminating newline.
+  while IFS= read -r line || [ -n "$line" ]; do
+    # `printf' is preferred over `echo' because `echo' might process
+    # backslash escapes or behave unexpectedly if its argument looks
+    # like an option.
+    printf %s\\n "$1$line"
+  done
+}
+
 test $# -gt 0 || usage_error "missing test command"
 
 case $expect_failure in
@@ -139,6 +182,10 @@ fi
     # <https://lists.gnu.org/archive/html/bug-autoconf/2011-09/msg00004.html>
     # <http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2009-February/004121.html>
     trap : 1 3 2 13 15
+    # Duplicate the stdout fd (which connects to awk's stdin) to fd 4
+    # so that we can reuse fd 1 for pipelines and command
+    # substitutions below.
+    exec 4>&1
     # Determine where to send the test script's stderr.  Only the
     # test's stderr should go here; if `exec 2>&$stderr_fd' were run,
     # this script's stderr (e.g., `set -x' output, if turned on to
@@ -146,12 +193,30 @@ fi
     # go to the log (via `awk', if `--merge' is enabled), not the
     # terminal.
     if test $merge -gt 0; then
-      stderr_fd=1  # send stderr to awk, which will copy it to the log
+      stderr_fd=4  # send stderr to awk, which will copy it to the log
     else
       stderr_fd=3  # send stderr directly to the log file
     fi
-    "$@" 2>&$stderr_fd 3>&-
-    echo $?
+    # Set to the test script's numeric exit status.
+    status=$(
+        exec 5>&1
+        {
+            "$@" 2>&1 1>&4 3>&- 4>&- 5>&-
+            # Capturing the status in a variable then writing the
+            # variable value to awk below may seem like unnecessary
+            # steps: Why not just write the status directly to awk
+            # here?  The reason is because it avoids a race condition.
+            # The awk script below *requires* the final line of its
+            # input to be the test script's exit status, but writing
+            # to fd 4 here would not provide that guarantee.  This
+            # `echo' is running concurrently with `prefix_lines',
+            # which is writing to fd 4 if `--merge' is enabled, so a
+            # prefixed and merged stderr line could be written to fd 4
+            # /after/ this status is written.
+            echo $? 1>&5
+        } | prefix_lines "$stderr_prefix" 1>&$stderr_fd
+    )
+    printf %s\\n "$status"
   ) | LC_ALL=C ${AM_TAP_AWK-awk} \
         -v me="$me" \
         -v test_script_name="$test_name" \
diff --git a/t/list-of-tests.mk b/t/list-of-tests.mk
index 2c6c271ea..7b6bbcfe8 100644
--- a/t/list-of-tests.mk
+++ b/t/list-of-tests.mk
@@ -1186,6 +1186,7 @@ t/tap-planskip-whitespace.sh \
 t/tap-planskip-badexit.sh \
 t/tap-planskip-bailout.sh \
 t/tap-planskip-later-errors.sh \
+t/tap-stderr-prefix.tap \
 t/tap-test-number-0.sh \
 t/tap-recheck-logs.sh \
 t/tap-result-comment.sh \
diff --git a/t/tap-stderr-prefix.tap b/t/tap-stderr-prefix.tap
new file mode 100644
index 000000000..30ef743a2
--- /dev/null
+++ b/t/tap-stderr-prefix.tap
@@ -0,0 +1,92 @@
+#! /bin/sh
+# Copyright (C) 2025 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+# TAP support:
+#  - The Automake TAP driver has an option that instructs it to prefix the
+#    test script's stderr with a given string.
+
+. test-init.sh
+plan_ later
+
+dumpf() (
+    set +x
+    diag_string_='#' diag_ "####### Contents of $1:"
+    diag_ <$1
+    diag_string_='#' diag_ "#######"
+)
+
+cat >Makefile.am <<END || bailout_ "failed to create Makefile.am"
+TESTS = all.test
+END
+dumpf Makefile.am
+
+. tap-setup.sh
+
+cat >all.test <<END || bailout_ "failed to create all.test"
+#!/bin/sh
+echo 1..2
+echo ok 1
+echo 'Bail out!' >&2
+echo ok 2
+END
+chmod a+x all.test || bailout_ "failed to make all.test executable"
+
+command_ok_ "no merge, no prefix: make check passes" -- \
+    run_make -O AM_TEST_LOG_DRIVER_FLAGS= check
+dumpf all.log
+command_ok_ "no merge, no prefix: result counts are correct" -- \
+    count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
+command_ok_ "no merge, no prefix: stderr in log" -- \
+    grep -q '^Bail out!$' all.log
+
+command_ok_ "--merge: make check fails" -- \
+    run_make -e FAIL AM_TEST_LOG_DRIVER_FLAGS=--merge check
+dumpf all.log
+# Don't bother checking the counts -- they're meaningless due to the merged
+# stderr "corrupting" the TAP output.  It is sufficient to just test that
+# `make check' returned non-zero.
+command_ok_ "--merge: stderr in log" -- \
+    grep -q '^Bail out!$' all.log
+
+# Included in the prefix:
+#   1. quadrigraph for # (@%:@)
+#   2. literal # (difficult to include in a Makefile variable, but easy to
+#      pass in a command-line make variable assignment)
+#   3. a string that expands to @%:@ (quadrigraph for #) after quadrigraph
+#      replacement, accomplished by embedding a quadrigraph for the empty
+#      string inside the quadrigraph for #
+# The end result should be: ##@%:@
+PFXFLAG='--stderr-prefix "@%:@#@%@&t@:@ "'
+
+command_ok_ "$PFXFLAG: make check passes" -- \
+    run_make -O AM_TEST_LOG_DRIVER_FLAGS="$PFXFLAG" check
+dumpf all.log
+command_ok_ "$PFXFLAG: result counts are correct" -- \
+    count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
+command_ok_ "$PFXFLAG: prefixed stderr in log" -- \
+    grep -q '^##@%:@ Bail out!$' all.log
+
+command_ok_ "$PFXFLAG --merge: make check passes" -- \
+    run_make -O AM_TEST_LOG_DRIVER_FLAGS="$PFXFLAG --merge" check
+dumpf all.log
+command_ok_ "$PFXFLAG --merge: result counts are correct" -- \
+    count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
+command_ok_ "$PFXFLAG --merge: prefixed stderr in log" -- \
+    grep -q '^##@%:@ Bail out!$' all.log
+
+plan_ now
+
+:
-- 
2.48.1

From 8b83b152f3a74b82d10ea171f52b32294194f15f Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:58 -0500
Subject: [PATCH 09/10] tap: include malformed output in error message to aid
 with debugging

* lib/tap-driver.sh: If the line that should contain the test script's exit
status is not an integer (which would indicate a bug with the driver, not the
code under test), include the line in the error message to help with
debugging.
---
 lib/tap-driver.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/tap-driver.sh b/lib/tap-driver.sh
index 0c7653b69..ed997edd0 100755
--- a/lib/tap-driver.sh
+++ b/lib/tap-driver.sh
@@ -23,7 +23,7 @@
 # bugs to <[email-masked]> or send patches to
 # <[email-masked]>.
 
-scriptversion=2025-02-03.03; # UTC
+scriptversion=2025-02-03.04; # UTC
 
 # Make unconditional expansion of undefined variables an error.  This
 # helps a lot in preventing typo-related bugs.
@@ -526,7 +526,7 @@ function get_test_exit_message(status)
   if (status == 0)
     return ""
   if (status !~ /^[1-9][0-9]*$/)
-    abort("getting exit status")
+    abort("getting exit status: not an integer: " status)
   if (status < 127)
     exit_details = ""
   else if (status == 127)
-- 
2.48.1

From 8e571302fe4293621f83aa3279c154dff89bd75f Mon Sep 17 00:00:00 2001
From: Richard Hansen <[email-masked]>
Date: Mon, 3 Feb 2025 18:26:59 -0500
Subject: [PATCH 10/10] tests: use `--stderr-prefix' for TAP tests

* t/local.mk: Pass `--stderr-prefix' to `lib/tap-driver.sh' when
running TAP tests to prevent stderr lines from being confused with
test results and to improve the readability of the test log files.
---
 t/local.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/local.mk b/t/local.mk
index 890d50ad6..ad11cc4d3 100644
--- a/t/local.mk
+++ b/t/local.mk
@@ -29,7 +29,7 @@ AM_PL_LOG_FLAGS = -Mstrict -w
 
 TAP_LOG_DRIVER = AM_TAP_AWK='$(AWK)' $(SHELL) $(srcdir)/lib/tap-driver.sh
 
-AM_TAP_LOG_DRIVER_FLAGS = --merge
+AM_TAP_LOG_DRIVER_FLAGS = --merge --stderr-prefix '@%:@[stderr]'
 
 EXTRA_DIST += %D%/README %D%/ax/is %D%/ax/is_newest %D%/ax/deltree.pl
 
-- 
2.48.1