Subject: Re: bug#24530: tests: revamp multibyte-white-space test to be more permissive
From: Jim Meyering <[email-masked]>
To: [email-masked], "Nelson H. F. Beebe" <[email-masked]>
Date: Sat, 24 Sep 2016 17:25:23 -0700
List-Id: <debbugs-submit.debbugs.gnu.org>

On Sat, Sep 24, 2016 at 4:02 PM, Jim Meyering <[email-masked]> wrote:
> grep's multibyte-white-space would too often fail.
> Its failure was mainly a reflection on the system's poor locale
> support, so this test did not give good signal on whether one would be
> well-advised to install the resulting grep binary.
>
> I've done this:
>
>         tests: revamp multibyte-white-space test to be more permissive
>         This test elicits too many failures. Whether a system has accurate
>         unicode "whitespace" attributes should not influence whether grep's
>         test suite passes.  In many cases, now you will see a warning that
>         some multibyte characters do not pass whitespace-related tests, but
>         this test no longer fails.  However, if you run this test on a modern
>         enough system, it does require that \s and \S do work properly with
>         most of the listed characters.
>         * tests/multibyte-white-space: Confirm that Fedora 24's locale
>         tables still declare those four Unicode code points *not* whitespace.
>         Honor a new column telling how to handle failure.  Provide more
>         information in each diagnostic.
>
> With the attached patch, even on Fedora 24, we see new warnings like
> this (before those characters were not even checked), and the test
> passes as it did before:
>
>  warning: \s failed to match \xe2\x80\x87 in the en_US.UTF-8 locale
>  warning: \S mistakenly matched \xe2\x80\x87 in the en_US.UTF-8 locale
>  warning: \s failed to match \xe2\x80\x8b in the en_US.UTF-8 locale
>  warning: \S mistakenly matched \xe2\x80\x8b in the en_US.UTF-8 locale
>  warning: \s failed to match \xe2\x80\xaf in the en_US.UTF-8 locale
>  warning: \S mistakenly matched \xe2\x80\xaf in the en_US.UTF-8 locale
>
> More importantly, on less modern systems, while this test would fail
> before, now it will merely emit warnings like the above.

Pushed: http://git.sv.gnu.org/cgit/grep.git/commit/?id=7c4c69400c6ab