GNU bug report logs - #6427
Autotest: enable colored test results.

Previous Next

To reply to this bug, email your comments to 6427 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: autoconf-patches <at> gnu.org, bug-coreutils <bug-coreutils <at> gnu.org>
Subject: Re: Autotest: enable colored test results.
Date: Mon, 14 Jun 2010 14:23:51 -0600

[Message part 1 (text/plain, inline)]

[the coreutils portion of this email starts half-way down]

On 06/14/2010 01:45 PM, Ralf Wildenhues wrote:
> Hi Eric,
> 
> quoting your reply out of order:
> 
> * Eric Blake wrote on Mon, Jun 14, 2010 at 07:22:31PM CEST:
>> On 06/13/2010 12:50 AM, Ralf Wildenhues wrote:
>>> The logical next step for Autotest to be on par with Automake's
>>> parallel-test.
>>>
>>> Unlike Automake, the testsuite author does not have to do anything for
>>> the user to be able to use color; AT_COLOR_TESTS only changes the
>>> default to yes.  We could easily let it accept an optional argument, if
>>> you think it is useful.
>>
>> I'm debating whether:
>>
>> AT_INIT([testsuite])
>> AT_COLOR_TESTS
> 
> This currently does not work:
> 
>> Did you make sure that AT_COLOR_TESTS can be specified either side of
>> AT_INIT, or is there a fixed invocation order that authors must be aware of?
> 
> Right now it has to be specified before AT_INIT.  If there's a diversion
> thinhy we can easily fix that with, great, otherwise I guess it would
> need documentation and order warning in the code.  Help?

It seems like it would merely be a matter of unconditionally emitting
at_color=no in an early diversion, and having AT_COLOR_TESTS emit
at_color=auto in an intermediate diversion, all before the option
parsing that appears in a later diversion.  That way, if the user didn't
specify --color, they pick up the appropriate default.

That also implies that anything else that checks at m4-time whether
AT_color is defined (such as your m4_ifdef for changing the --help
output text) must be m4_wrap'd to delay any output based on that
decision of AT_color until after the user's source file has been
completely read in.  Then again, we already m4_wrap some of our --help
text in order to accurately tell the user how many AT_SETUPs we
encountered, so this may already be taken care of given the current
architecture of autotest.

I'll see if I can get some time to help you figure that out, if that
wasn't enough of a hint.

>> Personally, I like tri-state color options: --color=no or --color=never
>> to disable, --color=yes or --color=always to enable, and --color=auto or
>> the simpler --color to depend on tty status.  Are you making 'yes' a
>> synonym for 'always' or for 'auto'?  Should we support other common
>> synonyms?
> 
> I meant yes=auto.  I realize this is different from how ls does it, and
> I'm fine with doing it another way.

My quick testing shows that when directed to a tty, both ls and grep
treat all four of '--color', '--color=auto', '--color=yes', and
'--color=always' as a command to turn on color.  More interesting is
collecting the output:

ls --color => color
ls --color=yes => color
ls --color=auto => plain
ls --color=always => color
grep --color => plain
grep --color=yes => color
grep --color=auto => plain
grep --color=always => color

That is, ls --color implies ls --color=always, while grep --color
implies grep --color=always.

It's probably worth a bug report to coreutils (cc'd), since I prefer the
grep behavior.  Should we also raise a bug against the GNU Coding
Standards to codify a preferred variant?

>  I'm also fine with 'never' and
> 'auto' as additional synonyms.  However, I very much think that if some
> option --foo[=ARG] accepts no and yes and other arguments, then yes
> --foo should be a synonym for --foo=yes, not anything else.

So you are arguing yet a third behavior from either coreutils or grep,
which is that --color implies --color=yes, but that yes is an alias for
'auto' rather than 'always'.  I could see your behavior making sense as
well.  But again, it argues for standardization among GNU projects.

Any other examples we can look at, to see if there is already a majority
in behavior?

>> I don't know how to easily detect ascii vs. ebcdic ESC (which is a
>> different encoding); \e and \E escapes are common, but non-portable.
> 
> Is there any chance that an EBCDIC system accepts ANSI terminal color
> escape sequences?  If not, then what would be the equivalent there?
> Am I confusing different entities here?

I have no idea of anyone that uses an EBCDIC system that also uses
terminal escape sequences.  Let's just go with hard-coding ASCII and
wait for the bug reports (after all, we DID get some bug reports, along
with a patch, about one use of m4_translit in 2.65, which have already
been applied to make autoconf.git more EBCDIC-friendly).

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #8 received at 6427 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Eric Blake <eblake <at> redhat.com>
Cc: 6427 <at> debbugs.gnu.org, autoconf-patches <at> gnu.org
Subject: Re: bug#6427: Autotest: enable colored test results.
Date: Tue, 15 Jun 2010 10:50:01 +0100

On 14/06/10 21:23, Eric Blake wrote:
>>> Personally, I like tri-state color options: --color=no or --color=never
>>> to disable, --color=yes or --color=always to enable, and --color=auto or
>>> the simpler --color to depend on tty status.  Are you making 'yes' a
>>> synonym for 'always' or for 'auto'?  Should we support other common
>>> synonyms?
>>
>> I meant yes=auto.  I realize this is different from how ls does it, and
>> I'm fine with doing it another way.
> 
> My quick testing shows that when directed to a tty, both ls and grep
> treat all four of '--color', '--color=auto', '--color=yes', and
> '--color=always' as a command to turn on color.  More interesting is
> collecting the output:
> 
> ls --color => color
> ls --color=yes => color
> ls --color=auto => plain
> ls --color=always => color
> grep --color => plain
> grep --color=yes => color
> grep --color=auto => plain
> grep --color=always => color
> 
> That is, ls --color implies ls --color=always, while grep --color
> implies grep --color=always.

grep --color=auto you mean.

> It's probably worth a bug report to coreutils (cc'd), since I prefer the
> grep behavior.  Should we also raise a bug against the GNU Coding
> Standards to codify a preferred variant?

Yes I prefer the grep behavior, but I'm not sure we can
change ls at this stage?

>>  I'm also fine with 'never' and
>> 'auto' as additional synonyms.  However, I very much think that if some
>> option --foo[=ARG] accepts no and yes and other arguments, then yes
>> --foo should be a synonym for --foo=yes, not anything else.
> 
> So you are arguing yet a third behavior from either coreutils or grep,
> which is that --color implies --color=yes, but that yes is an alias for
> 'auto' rather than 'always'.  I could see your behavior making sense as
> well.  But again, it argues for standardization among GNU projects.

ls has these synonyms:

  "always", "yes", "force",
  "never", "no", "none",
  "auto", "tty", "if-tty", NULL

We only document the first column which is a good thing.

> Any other examples we can look at, to see if there is already a majority
> in behavior?
> 
>>> I don't know how to easily detect ascii vs. ebcdic ESC (which is a
>>> different encoding); \e and \E escapes are common, but non-portable.
>>
>> Is there any chance that an EBCDIC system accepts ANSI terminal color
>> escape sequences?  If not, then what would be the equivalent there?
>> Am I confusing different entities here?
> 
> I have no idea of anyone that uses an EBCDIC system that also uses
> terminal escape sequences.  Let's just go with hard-coding ASCII and
> wait for the bug reports (after all, we DID get some bug reports, along
> with a patch, about one use of m4_translit in 2.65, which have already
> been applied to make autoconf.git more EBCDIC-friendly).

I've wondered about how general the color codes we use are.
Using termcap/terminfo would be most general but since
we've had no reports about it yet I think we should stick
with the simple hardcoded scheme.

I.E. the following currently outputs colors:

  TERM=dumb ls --color=auto

The l¹ script I use to wrap `ls` works around
this "limitation" by essentially doing:

  if ! tput setaf 1 >/dev/null 2>&1 && ! tput AF 1 >/dev/null 2>&1; then
    color_when=never
  else
    color_when=auto
  fi
  ls --color=$color_when

cheers,
Pádraig.

¹ http://www.pixelbeat.org/scripts/l

Message #11 received at 6427 <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Pádraig Brady <P <at> draigBrady.com>, 6427 <at> debbugs.gnu.org, 
	autoconf-patches <at> gnu.org
Subject: Re: bug#6427: Autotest: enable colored test results.
Date: Thu, 17 Jun 2010 13:54:24 -0600

[Message part 1 (text/plain, inline)]

On 06/17/2010 01:06 PM, Ralf Wildenhues wrote:
> * Pádraig Brady wrote on Tue, Jun 15, 2010 at 11:50:01AM CEST:
>> On 14/06/10 21:23, Eric Blake wrote:
>>> ls --color => color
>>> ls --color=yes => color
>>> ls --color=auto => plain
>>> ls --color=always => color
>>> grep --color => plain
>>> grep --color=yes => color
>>> grep --color=auto => plain
>>> grep --color=always => color
>>>
>>> That is, ls --color implies ls --color=always, while grep --color
>>> implies grep --color=always.
>>
>> grep --color=auto you mean.

Yep, sorry for my typo.

> I don't know either.  However, I also think Autotest need not wait for
> standardization in order to gain color support.  We can still fix the
> details later.

True enough.

> 
>> ls has these synonyms:
>>
>>   "always", "yes", "force",
>>   "never", "no", "none",
>>   "auto", "tty", "if-tty", NULL
>>
>> We only document the first column which is a good thing.
> 
> I agree, and implemented that in the updated patch.
> 
> The patch below has --color equal --color=always.  Since I'm less
> concerned about this detail than about the fact that the patch may not
> go in at all, I'm adding a second patch, to be squashed in with the
> first, if you prefer --color to be equal to --color=auto.
> 
> OK to commit 1 or 1+2?

For now, let's match ls (--color equals --color=always, patch 1 only),
and you are free to push after fixing one nit:

> +    --color )
> +	at_color=always
> +	;;
> +    --color=* )
> +	case $at_optarg in
> +	no | never | none) at_color=never ;;
> +	auto | tty | if-tty) at_color=auto ;;
> +	always | yes | force) at_color=always ;;
> +	esac

Let's add a *) option and cause a fatal error if we don't recognize the
argument given to the --color option.

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #14 received at 6427 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Pádraig Brady <P <at> draigBrady.com>
Cc: 6427 <at> debbugs.gnu.org, autoconf-patches <at> gnu.org,
	Eric Blake <eblake <at> redhat.com>
Subject: Re: bug#6427: Autotest: enable colored test results.
Date: Thu, 17 Jun 2010 21:06:55 +0200

* Pádraig Brady wrote on Tue, Jun 15, 2010 at 11:50:01AM CEST:
> On 14/06/10 21:23, Eric Blake wrote:
> > ls --color => color
> > ls --color=yes => color
> > ls --color=auto => plain
> > ls --color=always => color
> > grep --color => plain
> > grep --color=yes => color
> > grep --color=auto => plain
> > grep --color=always => color
> > 
> > That is, ls --color implies ls --color=always, while grep --color
> > implies grep --color=always.
> 
> grep --color=auto you mean.
> 
> > It's probably worth a bug report to coreutils (cc'd), since I prefer the
> > grep behavior.  Should we also raise a bug against the GNU Coding
> > Standards to codify a preferred variant?
> 
> Yes I prefer the grep behavior, but I'm not sure we can
> change ls at this stage?

I don't know either.  However, I also think Autotest need not wait for
standardization in order to gain color support.  We can still fix the
details later.

> ls has these synonyms:
> 
>   "always", "yes", "force",
>   "never", "no", "none",
>   "auto", "tty", "if-tty", NULL
> 
> We only document the first column which is a good thing.

I agree, and implemented that in the updated patch.

> I've wondered about how general the color codes we use are.
> Using termcap/terminfo would be most general but since
> we've had no reports about it yet I think we should stick
> with the simple hardcoded scheme.
> 
> I.E. the following currently outputs colors:
> 
>   TERM=dumb ls --color=auto

I regard it as fairly unproblematic if --color does the wrong thing on a
dumb terminal for now (Automake hasn't seen a bug report to this end
yet either).  Thanks for the suggestions.

In the attched patch, I also moved the AT_COLOR_TESTS entry up a bit,
because the paragraph after AT_TESTED really belongs to the macro
definition.  And added another diversion HELP_TUNING_BEGIN, to cope with
the ordering issues.

The patch below has --color equal --color=always.  Since I'm less
concerned about this detail than about the fact that the patch may not
go in at all, I'm adding a second patch, to be squashed in with the
first, if you prefer --color to be equal to --color=auto.

OK to commit 1 or 1+2?

Thanks,
Ralf

    Autotest: enable colored test results.
    
    * lib/autotest/general.m4 (HELP_TUNING_BEGIN): New diversion.
    (HELP_TUNING, HELP_OTHER, HELP_END): Bump diversion numbers.
    (AT_INIT): Accept
    --color and --color=never|auto|always.  If desired, colorize
    test results and testsuite summary on standard output.
    [HELP_TUNING]: Divert content instead to ...
    [HELP_TUNING_BEGIN]: ... this diversion, m4_wrapped until the
    end, when we know whether AT_COLOR_TESTS has been specified.
    (AT_COLOR_TESTS): New macro, set the default for color to auto.
    * doc/autoconf.texi (Writing Testsuites): Document it.
    (testsuite Invocation): Document --color* options.
    * tests/local.at: Call AT_COLOR_TESTS for Autoconf's testsuite.
    * tests/autotest.at (color test results): New test, mirroring
    color.test from Automake.
    * NEWS: Update.

diff --git a/NEWS b/NEWS
index f2290a1..0ff482f 100644
--- a/NEWS
+++ b/NEWS
@@ -36,6 +36,8 @@ GNU Autoconf NEWS - User visible changes.
    AT_ARG_OPTION has been changed in that the negative of a long option
    --OPTION is now --no-OPTION rather than --noOPTION.
 
+** Autotest testsuites may optionally provide colored test results.
+
 * Major changes in Autoconf 2.65 (2009-11-21) [stable]
   Released by Eric Blake, based on git versions 2.64.*.
 
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 48f8589..3bb6b6d 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -23478,6 +23478,12 @@ It it recommended that you use a package-specific prefix to @var{options}
 names in order to avoid clashes with future Autotest built-in options.
 @end defmac
 
+@defmac AT_COLOR_TESTS
+@atindex{COLOR_TESTS}
+Enable colored test results by default when the output is connected to
+a terminal.
+@end defmac
+
 @defmac AT_TESTED (@var{executables})
 @atindex{TESTED}
 Log the file name and answer to @option{--version} of each program in
@@ -23868,6 +23874,14 @@ then concurrently running tests will finish before exiting.
 Force more verbosity in the detailed output of what is being done.  This
 is the default for debugging scripts.
 
+@item --color
+@itemx --color <at> r{[}=never <at> r{|}auto <at> r{|}always <at> r{]}
+Enable colored test results.  Without an argument, or with @samp{always},
+test results will be colored.  With @samp{never}, color mode is turned
+off.  Otherwise, if either the macro @code{AT_COLOR_TESTS} is used by
+the testsuite author, or the argument @samp{auto} is given, then test
+results are colored if standard output is connected to a terminal.
+
 @item --debug
 @itemx -d
 Do not remove the files after a test group was performed ---but they are
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 0bc529f..68b80b4 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -62,8 +62,10 @@ Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008,
 #  - HELP_MODES
 #    Modes help text.  Additional modes can be appended as self-contained
 #    cat'd here-docs as generated by AS_HELP_STRING.
+#  - HELP_TUNING_BEGIN
+#    Tuning help text.  This is for Autotest-provided text.
 #  - HELP_TUNING
-#    Tuning help text.  Additional tuning options can be appended as
+#    Additional tuning options' help text can be appended here as
 #    self-contained cat'd here-docs as generated by AS_HELP_STRING.
 #  - HELP_OTHER
 #    User help can be appended to this as self-contained cat'd here-docs.
@@ -104,9 +106,10 @@ m4_define([_m4_divert(PARSE_ARGS)],         201)
 m4_define([_m4_divert(PARSE_ARGS_END)],     202)
 m4_define([_m4_divert(HELP)],               300)
 m4_define([_m4_divert(HELP_MODES)],         301)
-m4_define([_m4_divert(HELP_TUNING)],        302)
-m4_define([_m4_divert(HELP_OTHER)],         303)
-m4_define([_m4_divert(HELP_END)],           304)
+m4_define([_m4_divert(HELP_TUNING_BEGIN)],  302)
+m4_define([_m4_divert(HELP_TUNING)],        303)
+m4_define([_m4_divert(HELP_OTHER)],         304)
+m4_define([_m4_divert(HELP_END)],           305)
 m4_define([_m4_divert(VERSION)],            350)
 m4_define([_m4_divert(VERSION_NOTICES)],    351)
 m4_define([_m4_divert(VERSION_END)],        352)
@@ -423,6 +426,8 @@ m4_define([_AT_FINISH],
 [m4_ifdef([AT_ingroup], [m4_fatal([missing AT_CLEANUP detected])])dnl
 m4_divert_text([DEFAULTS],
 [
+# Whether to enable colored test results.
+at_color=m4_ifdef([AT_color], [AT_color], [no])
 # List of the tested programs.
 at_tested='m4_ifdef([AT_tested],
   [m4_translit(m4_dquote(m4_defn([AT_tested])), [ ], m4_newline)])'
@@ -492,6 +497,17 @@ do
 	at_clean=:
 	;;
 
+    --color )
+	at_color=always
+	;;
+    --color=* )
+	case $at_optarg in
+	no | never | none) at_color=never ;;
+	auto | tty | if-tty) at_color=auto ;;
+	always | yes | force) at_color=always ;;
+	esac
+	;;
+
     --debug | -d )
 	at_debug_p=:
 	;;
@@ -663,6 +679,17 @@ else
   # Sort the tests, removing duplicates.
   at_groups=`AS_ECHO(["$at_groups"]) | tr ' ' "$as_nl" | sort -nu`
 fi
+
+if test x"$at_color" = xalways \
+   || { test x"$at_color" = xauto && test -t 1; }; then
+  at_red=`printf '\033@<:@0;31m'`
+  at_grn=`printf '\033@<:@0;32m'`
+  at_lgn=`printf '\033@<:@1;32m'`
+  at_blu=`printf '\033@<:@1;34m'`
+  at_std=`printf '\033@<:@m'`
+else
+  at_red= at_grn= at_lgn= at_blu= at_std=
+fi
 m4_divert_pop([PARSE_ARGS_END])dnl
 m4_divert_push([HELP])dnl
 
@@ -697,13 +724,17 @@ Operation modes:
   -l, --list     describes all the tests, or the selected TESTS
 _ATEOF
 m4_divert_pop([HELP_MODES])dnl
-m4_divert_push([HELP_TUNING])dnl
+m4_wrap([m4_divert_push([HELP_TUNING_BEGIN])dnl
 cat <<_ATEOF || at_write_fail=1
 
 dnl extra quoting prevents emacs whitespace mode from putting tabs in output
 Execution tuning:
   -C, --directory=DIR
 [                 change to directory DIR before starting]
+      --color[[=never|auto|always]]
+[                 ]m4_ifdef([AT_color],
+		      [disable colored test results, or enable even without terminal],
+		      [enable colored test results on terminal, or always])
   -j, --jobs[[=N]]
 [                 Allow N jobs at once; infinite jobs with no arg (default 1)]
   -k, --keywords=KEYWORDS
@@ -717,7 +748,7 @@ Execution tuning:
 [                 default for debugging scripts]
   -x, --trace    enable tests shell tracing
 _ATEOF
-m4_divert_pop([HELP_TUNING])dnl
+m4_divert_pop([HELP_TUNING_BEGIN])])dnl
 m4_divert_push([HELP_END])dnl
 cat <<_ATEOF || at_write_fail=1
 
@@ -1156,35 +1187,40 @@ _ATEOF
 	at_msg="UNEXPECTED PASS"
 	at_res=xpass
 	at_errexit=$at_errexit_p
+	at_color=$at_red
 	;;
     no:0)
 	at_msg="ok"
 	at_res=pass
 	at_errexit=false
+	at_color=$at_grn
 	;;
     *:77)
 	at_msg='skipped ('`cat "$at_check_line_file"`')'
 	at_res=skip
 	at_errexit=false
+	at_color=$at_blu
 	;;
     no:* | *:99)
 	at_msg='FAILED ('`cat "$at_check_line_file"`')'
 	at_res=fail
 	at_errexit=$at_errexit_p
+	at_color=$at_red
 	;;
     yes:*)
 	at_msg='expected failure ('`cat "$at_check_line_file"`')'
 	at_res=xfail
 	at_errexit=false
+	at_color=$at_lgn
 	;;
   esac
   echo "$at_res" > "$at_job_dir/$at_res"
   # In parallel mode, output the summary line only afterwards.
   if test $at_jobs -ne 1 && test -n "$at_verbose"; then
-    AS_ECHO(["$at_desc_line $at_msg"])
+    AS_ECHO(["$at_desc_line $at_color$at_msg$at_std"])
   else
     # Make sure there is a separator even with long titles.
-    AS_ECHO([" $at_msg"])
+    AS_ECHO([" $at_color$at_msg$at_std"])
   fi
   at_log_msg="$at_group. $at_desc ($at_setup_line): $at_msg"
   case $at_status in
@@ -1485,12 +1521,14 @@ if $at_errexit_p && test $at_unexpected_count != 0; then
     at_result="$at_result $at_were run, one failed"
   fi
   at_result="$at_result unexpectedly and inhibited subsequent tests."
+  at_color=$at_red
 else
   # Don't you just love exponential explosion of the number of cases?
+  at_color=$at_red
   case $at_xpass_count:$at_fail_count:$at_xfail_count in
     # So far, so good.
-    0:0:0) at_result="$at_result $at_were successful." ;;
-    0:0:*) at_result="$at_result behaved as expected." ;;
+    0:0:0) at_result="$at_result $at_were successful." at_color=$at_grn ;;
+    0:0:*) at_result="$at_result behaved as expected." at_color=$at_lgn ;;
 
     # Some unexpected failures
     0:*:0) at_result="$at_result $at_were run,
@@ -1538,10 +1576,10 @@ $at_skip_count tests were skipped." ;;
 esac
 
 if test $at_unexpected_count = 0; then
-  echo "$at_result"
+  echo "$at_color$at_result$at_std"
   echo "$at_result" >&AS_MESSAGE_LOG_FD
 else
-  echo "ERROR: $at_result" >&2
+  echo "${at_color}ERROR: $at_result$at_std" >&2
   echo "ERROR: $at_result" >&AS_MESSAGE_LOG_FD
   {
     echo
@@ -1757,6 +1795,12 @@ m4_define([AT_COPYRIGHT],
 [m4_default([$2], [m4_newline])([$1])])])# AT_COPYRIGHT
 
 
+# AT_COLOR_TESTS
+# --------------
+# Enable colored test results if standard error is connected to a terminal.
+m4_define([AT_COLOR_TESTS],
+[m4_define([AT_color], [auto])])
+
 # AT_SETUP(DESCRIPTION)
 # ---------------------
 # Start a group of related tests, all to be executed in the same subshell.
diff --git a/tests/autotest.at b/tests/autotest.at
index e09e189..fd3787a 100644
--- a/tests/autotest.at
+++ b/tests/autotest.at
@@ -1505,6 +1505,93 @@ AT_CHECK([test -s sigpipe-stamp || test ! -f micro-suite.dir/7/micro-suite.log],
 
 AT_CLEANUP
 
+
+# --color
+AT_CHECK_AT_TEST([colored test results],
+  [AT_CHECK([:])
+   AT_CLEANUP
+   AT_SETUP([fail])
+   AT_CHECK([exit 1])
+   AT_CLEANUP
+   AT_SETUP([xpass])
+   AT_XFAIL_IF([:])
+   AT_CHECK([:])
+   AT_CLEANUP
+   AT_SETUP([xfail])
+   AT_XFAIL_IF([:])
+   AT_CHECK([exit 1])
+   AT_CLEANUP
+   AT_SETUP([skip])
+   AT_CHECK([exit 77])
+   AT_CLEANUP
+   AT_SETUP([hardfail])
+   AT_XFAIL_IF([:])
+   AT_CHECK([exit 99])
+], [], [], [], [], [], [
+
+TERM=ansi
+export TERM
+
+red=`printf '\033@<:@0;31m'`
+grn=`printf '\033@<:@0;32m'`
+lgn=`printf '\033@<:@1;32m'`
+blu=`printf '\033@<:@1;34m'`
+std=`printf '\033@<:@m'`
+
+# Check that grep can parse nonprinting characters.
+# BSD 'grep' works from a pipe, but not a seekable file.
+# GNU or BSD 'grep -a' works on files, but is not portable.
+AT_CHECK([case `echo "$std" | grep .` in #'' restore font-lock
+	    $std) :;;
+	    *) Exit 77;;
+	  esac], [], [ignore], [],
+	 [echo "grep can't parse nonprinting characters" >&2])
+
+if echo 'ab*c' | grep -F 'ab*c' >/dev/null 2>&1; then
+  FGREP="grep -F"
+else
+  FGREP=fgrep
+fi
+
+# No color.
+AT_CHECK([$CONFIG_SHELL ./micro-suite], [1], [stdout], [stderr])
+for color in "$red" "$grn" "$lgn" "$blu"; do
+  AT_CHECK([cat stdout stderr | $FGREP "$color"], [1])
+done
+
+# Color of test group results.
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always], [1], [stdout], [stderr])
+AT_CHECK([cat stdout | grep " only " | $FGREP "$grn"], [], [ignore])
+AT_CHECK([cat stdout | grep " fail " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stdout | grep " xfail " | $FGREP "$lgn"], [], [ignore])
+AT_CHECK([cat stdout | grep " xpass " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stdout | grep " skip " | $FGREP "$blu"], [], [ignore])
+AT_CHECK([cat stdout | grep " hardfail " | $FGREP "$red"], [], [ignore])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+
+# The summary is green if all tests were successful, light green if all
+# behaved as expected, and red otherwise.
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always 1 -k skip],
+	 [0], [stdout])
+AT_CHECK([cat stdout | grep 'test.*successful' | $FGREP "$grn"],
+	 [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always 1 -k xfail -k skip],
+	 [0], [stdout])
+AT_CHECK([cat stdout | grep 'as expected' | $FGREP "$lgn"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k fail],
+	 [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k xpass],
+	 [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+AT_CHECK([$CONFIG_SHELL ./micro-suite --color=always -k hardfail],
+	 [1], [ignore], [stderr])
+AT_CHECK([cat stderr | grep ERROR | $FGREP "$red"], [], [ignore])
+# Reset color on verbose output.
+printf %s\\n "$std"
+], [1])
+
+
 ## ------------------- ##
 ## srcdir propagation. ##
 ## ------------------- ##
diff --git a/tests/local.at b/tests/local.at
index a812c43..39360ef 100644
--- a/tests/local.at
+++ b/tests/local.at
@@ -25,6 +25,8 @@ m4_pattern_allow([^m4_(define|shift)$])
 # Programs this package provides
 AT_TESTED([autom4te autoconf autoheader autoupdate autoreconf ifnames])
 
+# Enable colored test output.
+AT_COLOR_TESTS
 
 ## ---------------- ##
 ## Utility macros.  ##




    --color implies --color=auto, not --color=always

diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 3bb6b6d..3c67a0d 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -23876,11 +23876,12 @@ is the default for debugging scripts.
 
 @item --color
 @itemx --color <at> r{[}=never <at> r{|}auto <at> r{|}always <at> r{]}
-Enable colored test results.  Without an argument, or with @samp{always},
-test results will be colored.  With @samp{never}, color mode is turned
-off.  Otherwise, if either the macro @code{AT_COLOR_TESTS} is used by
-the testsuite author, or the argument @samp{auto} is given, then test
-results are colored if standard output is connected to a terminal.
+Enable colored test results.  Without an argument of @samp{always},
+test results will always be colored, with @samp{never}, color mode is
+turned off.  Otherwise, if either the macro @code{AT_COLOR_TESTS} is
+used by the testsuite author, or no argument or @samp{auto} is given,
+then test results are colored if standard output is connected to a
+terminal.
 
 @item --debug
 @itemx -d
diff --git a/lib/autotest/general.m4 b/lib/autotest/general.m4
index 68b80b4..9389d16 100644
--- a/lib/autotest/general.m4
+++ b/lib/autotest/general.m4
@@ -498,7 +498,7 @@ do
 	;;
 
     --color )
-	at_color=always
+	at_color=auto
 	;;
     --color=* )
 	case $at_optarg in

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.