From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 31 13:59:39 2024 Received: (at submit) by debbugs.gnu.org; 31 Jan 2024 18:59:39 +0000 Received: from localhost ([127.0.0.1]:39040 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rVFoA-0000JQ-T3 for submit@debbugs.gnu.org; Wed, 31 Jan 2024 13:59:39 -0500 Received: from lists.gnu.org ([2001:470:142::17]:42918) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rVFo8-0000J8-9k for submit@debbugs.gnu.org; Wed, 31 Jan 2024 13:59:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVFnt-00060f-AN for automake-patches@gnu.org; Wed, 31 Jan 2024 13:59:21 -0500 Received: from frenzy.freefriends.org ([198.99.81.75] helo=freefriends.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVFnq-0000Kj-3b for automake-patches@gnu.org; Wed, 31 Jan 2024 13:59:21 -0500 X-Envelope-From: karl@freefriends.org X-Envelope-To: Received: from freefriends.org (localhost [127.0.0.1]) by freefriends.org (8.14.7/8.14.7) with ESMTP id 40VIxEY2002140 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 31 Jan 2024 11:59:14 -0700 Received: (from apache@localhost) by freefriends.org (8.14.7/8.14.7/Submit) id 40VIxEKi002139; Wed, 31 Jan 2024 11:59:14 -0700 Date: Wed, 31 Jan 2024 11:59:14 -0700 Message-Id: <202401311859.40VIxEKi002139@freefriends.org> From: Karl Berry To: automake-patches@gnu.org Subject: wording for user on test suite failure? Received-SPF: pass client-ip=198.99.81.75; envelope-from=karl@freefriends.org; helo=freefriends.org X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) I was thinking about making the message when the test suite failure more explicit about what to do, as in below. If any comments or criticisms, let me know. I was actually inspired to do this not so much to tweak these words, but because I was thinking of adding yet more instructions about how to help debug the subsecond mtime stuff (for automake only, of course), i.e., setting am_cv_sleep_fractional_seconds=false. But with the new possibility that checking make's behavior will be enough to obviate those random failures, that doesn't seem necessary/desirable now. But having gone to all the trouble of figuring out how to change the text and which tests need to be changed to keep them in sync, I guess I still hope people think it's worth it, or at least ok, to make this change. The output would look like this: ============================================================================ See tests/test-suite.log for debugging. Some test(s) failed. Please report this to
, together with the test-suite.log file (gzipped) and your system information. Thanks. ============================================================================ Hopefully I did not break the colorization, but I don't have a good way to test that. --thanks, karl. ----------------------------------------------------------------------------- test: make bug-reporting message on test failure more explicit. * NEWS: mention this. * lib/am/check.am ($(TEST_SUITE_LOG)) : new fn, new wording. * t/ax/test-defs.in (failure_footer_text): new wording. (failure_footer_text_colorized): add red. * t/ax/tap-summary-aux.sh (failure_footer): use failure_footer_text*. * t/tap-doc2.sh: change grep. * t/testsuite-summary-color.sh: likewise. * t/testsuite-summary-count-many.sh: likewise. * t/testsuite-summary-count.sh: likewise. * t/testsuite-summary-reference-log.sh: likewise. diff --git a/NEWS b/NEWS index 153cb7733..964d82e8c 100644 --- a/NEWS +++ b/NEWS @@ -44,6 +44,10 @@ New in 1.17: - The missing script also supports autoreconf, autogen, and perl. (https://lists.gnu.org/archive/html/automake-patches/2015-08/msg00000.html) + - test-suite.log now contains basic system information, and the + console message about bug reporting on failure has a bit more detail. + (bug#68746) + * Bugs fixed - Generated file timestamp checks handle filesystems with sub-second diff --git a/lib/am/check.am b/lib/am/check.am index d210172a6..b6fa3aaad 100644 --- a/lib/am/check.am +++ b/lib/am/check.am @@ -246,6 +246,7 @@ am--force-recheck: ## Exists only to be overridden. See bug#11745. AM_TESTSUITE_SUMMARY_HEADER = ' for $(PACKAGE_STRING)' +## emacs-page $(TEST_SUITE_LOG): $(TEST_LOGS) @$(am__set_TESTS_bases); \ ## Helper shell function, tells whether a path refers to an existing, @@ -360,7 +361,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) result_count $$1 "ERROR:" $$error "$$mgn"; \ }; \ ## -## Write some basic system info in the log. +## Function to write some basic system info in the log. output_system_information () \ { \ ## Omit the hostname for privacy. In practice it's a single word? @@ -372,6 +373,16 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) cat /etc/issue; \ fi; \ }; \ +## +## Function to output the "please report bugs" message. + please_report () \ + { \ +echo "Some test(s) failed. Please report this to $(PACKAGE_BUGREPORT),"; \ +echo "together with the test-suite.log file (gzipped) and your system"; \ +echo "information. Thanks."; \ + }; \ +## +## emacs-page ## Write "global" testsuite log. { \ echo "$(PACKAGE_STRING): $(subdir)/$(TEST_SUITE_LOG)" | \ @@ -379,8 +390,10 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) create_testsuite_report --no-color; \ output_system_information; \ echo; \ -## What is this line for? It seems to be intended for machine parsing, -## but I can't find any other reference to it. --karl, 27jan24. +## What is this line for? It's a Sphinx reStructuredText directive that +## would insert a table of contents that goes two levels deep on headings. +## But why is it in test-suite.log? No other references to it are evident. +## Please inform automake@gnu.org. echo ".. contents:: :depth: 2"; \ echo; \ for b in $$bases; do echo $$b; done \ @@ -403,9 +416,9 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) create_testsuite_report --maybe-color; \ echo "$$col$$br$$std"; \ if $$success; then :; else \ - echo "$${col}See $(subdir)/$(TEST_SUITE_LOG)$${std}"; \ + echo "$${col}See $(subdir)/$(TEST_SUITE_LOG) for debugging.$${std}";\ if test -n "$(PACKAGE_BUGREPORT)"; then \ - echo "$${col}Please report to $(PACKAGE_BUGREPORT)$${std}"; \ + please_report | sed -e "s/^/$${col}/" -e s/'$$'/"$${std}"/; \ fi; \ echo "$$col$$br$$std"; \ fi; \ diff --git a/t/ax/tap-summary-aux.sh b/t/ax/tap-summary-aux.sh index d1e2888c3..a709f1ef3 100644 --- a/t/ax/tap-summary-aux.sh +++ b/t/ax/tap-summary-aux.sh @@ -87,8 +87,17 @@ if test $use_colors = yes; then mgn="$esc[0;35m" brg="$esc[1m" std="$esc[m" + failure_footer="\ +${red}${br}${std} +$failure_footer_text_colorized +${red}${br}${std}" + else red= grn= lgn= blu= mgn= brg= std= + failure_footer="\ +${red}${br}${std} +$failure_footer_text +${red}${br}${std}" fi success_header="\ @@ -101,12 +110,6 @@ ${grn}${br}${std}" failure_header="\ ${red}${br}${std} ${red}Testsuite summary for GNU AutoTAP 5.12${std} -${red}${br}${std}" - - failure_footer="\ -${red}${br}${std} -${red}See ./test-suite.log${std} -${red}Please report to bug-automake@gnu.org${std} ${red}${br}${std}" $ACLOCAL diff --git a/t/ax/test-defs.in b/t/ax/test-defs.in index 8ea842355..c1ead418e 100644 --- a/t/ax/test-defs.in +++ b/t/ax/test-defs.in @@ -215,6 +215,20 @@ sleep="sleep $MTIME_RESOLUTION" # in UTC; see . old_timestamp=198309271735.59 +# We need this in multiple tests and the wording is subject to change. +# Don't include the ${br} lines since br is not defined at this point. +# +failure_footer_text="\ +See ./test-suite.log for debugging. +Some test(s) failed. Please report this to bug-automake@gnu.org, +together with the test-suite.log file (gzipped) and your system +information. Thanks." +# +# And with color. +red='' +std=''; +failure_footer_text_colorized="`echo \"$failure_footer_text\" | sed -e "s/^/${red}/" -e s/'$'/"${std}"/`" + # Make our wrapper script (or installed scripts, if running under # "installcheck") accessible by default. And avoid uselessly # extending $PATH multiple times if this file is sourced multiple times. diff --git a/t/tap-doc2.sh b/t/tap-doc2.sh index fdca66ad8..54c69b631 100644 --- a/t/tap-doc2.sh +++ b/t/tap-doc2.sh @@ -102,7 +102,7 @@ cat exp cat got diff exp got -grep '^Please report to bug-automake@gnu\.org$' stdout +grep 'Please report this to bug-automake@gnu\.org' stdout run_make -O check \ TESTS='foo.test baz.test' \ diff --git a/t/testsuite-summary-color.sh b/t/testsuite-summary-color.sh index 7b6254f61..ffbe5c158 100644 --- a/t/testsuite-summary-color.sh +++ b/t/testsuite-summary-color.sh @@ -46,8 +46,7 @@ ${red}${br}${std}" failure_footer="\ ${red}${br}${std} -${red}See ./test-suite.log${std} -${red}Please report to bug-automake@gnu.org${std} +$failure_footer_text_colorized ${red}${br}${std}" do_check '' <> configure.ac diff --git a/t/testsuite-summary-count.sh b/t/testsuite-summary-count.sh index c8f15389d..9cd2fa58e 100644 --- a/t/testsuite-summary-count.sh +++ b/t/testsuite-summary-count.sh @@ -32,8 +32,7 @@ success_footer=${br} failure_footer="\ ${br} -See ./test-suite.log -Please report to bug-automake@gnu.org +$failure_footer_text ${br}" # Corner cases. diff --git a/t/testsuite-summary-reference-log.sh b/t/testsuite-summary-reference-log.sh index 64cc4eb6c..f27e6fc39 100644 --- a/t/testsuite-summary-reference-log.sh +++ b/t/testsuite-summary-reference-log.sh @@ -46,11 +46,11 @@ cd build ../configure run_make -O -e FAIL check -grep '^See \./my_test_suite\.log$' stdout +grep '^See \./my_test_suite\.log for debugging\.$' stdout mkdir bar run_make -O -e FAIL TEST_SUITE_LOG=bar/bar.log check -grep '^See \./bar/bar\.log$' stdout +grep '^See \./bar/bar\.log for debugging\.$' stdout cd .. @@ -70,13 +70,13 @@ $AUTOMAKE ./configure run_make -O -e FAIL check -grep '^See sub/test-suite\.log$' stdout +grep '^See sub/test-suite\.log for debugging\.$' stdout cd sub run_make -O -e FAIL check -grep '^See sub/test-suite\.log$' stdout +grep '^See sub/test-suite\.log for debugging\.$' stdout cd .. run_make -O -e FAIL TEST_SUITE_LOG=foo.log check -grep '^See sub/foo\.log$' stdout +grep '^See sub/foo\.log for debugging\.$' stdout : compile finished at Wed Jan 31 10:56:25 2024 From debbugs-submit-bounces@debbugs.gnu.org Wed May 15 16:27:34 2024 Received: (at 68855) by debbugs.gnu.org; 15 May 2024 20:27:34 +0000 Received: from localhost ([127.0.0.1]:44294 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s7LDp-0007Fa-W4 for submit@debbugs.gnu.org; Wed, 15 May 2024 16:27:34 -0400 Received: from frenzy.freefriends.org ([198.99.81.75]:33248 helo=freefriends.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s7LDm-0007FN-Pe; Wed, 15 May 2024 16:27:33 -0400 X-Envelope-From: karl@freefriends.org Received: from freefriends.org (localhost [127.0.0.1]) by freefriends.org (8.14.7/8.14.7) with ESMTP id 44FKRRiA020099 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 May 2024 14:27:27 -0600 Received: (from apache@localhost) by freefriends.org (8.14.7/8.14.7/Submit) id 44FKRQaf020098; Wed, 15 May 2024 14:27:26 -0600 Date: Wed, 15 May 2024 14:27:26 -0600 Message-Id: <202405152027.44FKRQaf020098@freefriends.org> From: Karl Berry To: 68855@debbugs.gnu.org Subject: Re: [bug#68855] wording for user on test suite failure? In-Reply-To: <202401311859.40VIxEKi002139@freefriends.org> X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 68855 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) I was thinking about making the message when the test suite failure more explicit ... (lib/am/check.am) +echo "Some test(s) failed. Please report this to $(PACKAGE_BUGREPORT),"; \ +echo "together with the test-suite.log file (gzipped) and your system"; \ +echo "information. Thanks."; \ No one has replied or noted anything since I committed the change at the end of January, so closing ... -k From unknown Sat Aug 09 05:04:11 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 13 Jun 2024 11:24:09 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator