Subject: Re: bug#16941: [PATCH] tests/mb-non-UTF8-performance: increase timeout

From: Jim Meyering <[email-masked]>
Date: Sun, 9 Mar 2014 22:12:01 -0700

On Thu, Mar 6, 2014 at 12:15 AM, Andreas Schwab <[email-masked]> wrote:
> Jim Meyering <[email-masked]> writes:
>
>> On Wed, Mar 5, 2014 at 8:09 AM, Andreas Schwab <[email-masked]> wrote:
>>> ---
>>>  tests/mb-non-UTF8-performance | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/tests/mb-non-UTF8-performance b/tests/mb-non-UTF8-performance
>>> index 282f0c4..9b581a3 100755
>>> --- a/tests/mb-non-UTF8-performance
>>> +++ b/tests/mb-non-UTF8-performance
>>> @@ -26,7 +26,7 @@ require_JP_EUC_locale_
>>>  yes $(printf '%078d' 0) | head -50000 > in || framework_failure_
>>>
>>>  # Expect no match, i.e., exit status of 1.  Anything else is an error.
>>> -timeout 4 grep -i foobar in; st=$?
>>> +timeout 50 grep -i foobar in; st=$?
>>>  test $st = 1 || fail=1
>>
>> Thanks.  For what environment is that useful?
>
> For m68k-suse-linux.

Thanks again for the patch, but if I were to use it, the timeout
would be so long that the test would mistakenly pass
(the timeout would not trigger) even if the bug were reintroduced.
Instead, I've rewritten the test to make it less sensitive to the
actual hardware used to run it.  Please let me know if this works
for you:
From a6a92037b56f4fad9c75409f2b9ee89555c8c54c Mon Sep 17 00:00:00 2001
From: Jim Meyering <[email-masked]>
Date: Sun, 9 Mar 2014 21:25:51 -0700
Subject: [PATCH] tests: make a perf.-related test less system-sensitive

Andreas Schwab reported in http://debbugs.gnu.org/16941
that this test would timeout on m68k-suse-linux.
Rather than testing absolute duration with a limit tuned
to today's hardware, compare performance of grep with LC_ALL=C
against that same command using LC_ALL=ja_JP.eucJP.
* tests/init.cfg (require_hi_res_time_): New function.
* tests/mb-non-UTF8-performance: Rewrite to use it:
record absolute duration D of the first (normally much faster)
command, and set a timeout of 8*D for the command running in
an affected locale.
---
 tests/init.cfg                | 15 +++++++++++++++
 tests/mb-non-UTF8-performance | 17 ++++++++++++++---
 2 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/tests/init.cfg b/tests/init.cfg
index 5555e2d..3e9ab03 100644
--- a/tests/init.cfg
+++ b/tests/init.cfg
@@ -103,6 +103,21 @@ require_unibyte_locale()
   skip_ 'no unibyte locale found'
 }

+# Define hi_res_time_ to a function that prints the current time
+# as a floating point number with greater than 1-second resolution.
+# Otherwise, skip the requiring test.
+require_hi_res_time_()
+{
+  local cmd
+  for cmd in 'date +%s.%N' \
+          'perl -le "use Time::HiRes qw(time); print scalar time()"'; do
+      case $($cmd) in
+          *.[0-9]*) eval 'hi_res_time_() { '"$cmd"'; }'; break;;
+      esac
+  done
+  type hi_res_time_ || skip_ no high-resolution timer support
+}
+
 require_JP_EUC_locale_()
 {
   local locale=ja_JP.eucJP
diff --git a/tests/mb-non-UTF8-performance b/tests/mb-non-UTF8-performance
index 282f0c4..640d5db 100755
--- a/tests/mb-non-UTF8-performance
+++ b/tests/mb-non-UTF8-performance
@@ -18,15 +18,26 @@

 . "${srcdir=.}/init.sh"; path_prepend_ ../src
 require_timeout_
+require_hi_res_time_

 fail=0

-require_JP_EUC_locale_
-
 yes $(printf '%078d' 0) | head -50000 > in || framework_failure_

+start=$(hi_res_time_)
+LC_ALL=C grep -i foobar in; st=$?
+stop=$(hi_res_time_)
+
+# Use a multiple of the LC_ALL=C duration as the timeout for the JP/EUC test.
+# Typically, a multiple of 3 seems to be enough.
+timeout=$($AWK 'BEGIN { print 8 * ('$stop' - '$start')}' < /dev/null)
+
+test $st = 1 || fail=1
+
+require_JP_EUC_locale_
+
 # Expect no match, i.e., exit status of 1.  Anything else is an error.
-timeout 4 grep -i foobar in; st=$?
+timeout $timeout grep -i foobar in; st=$?
 test $st = 1 || fail=1

 Exit $fail
-- 
1.9.0.152.g6ab4ae2