Subject: bug#60621: closed (Re: bug#60618: unicode characters are not identified as such for \w and \b with -P)

From: [email-masked] (GNU bug Tracking System)
Date: Sun, 08 Jan 2023 02:30:03 +0000

Your bug report

#60618: grep -P does not set PCRE2_UCP

which was filed against the grep package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
60618: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=60618
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#60618: unicode characters are not identified as such for \w and \b with -P
From: Jim Meyering <[email-masked]>
To: Carlo Arenas <[email-masked]>
Cc: [email-masked]
Date: Sat, 7 Jan 2023 18:28:49 -0800
List-Id: <debbugs-submit.debbugs.gnu.org>

On Fri, Jan 6, 2023 at 11:37 PM Jim Meyering <[email-masked]> wrote:
> On Fri, Jan 6, 2023 at 11:28 PM Jim Meyering <[email-masked]> wrote:
> > On Fri, Jan 6, 2023 at 7:49 PM Carlo Arenas <[email-masked]> wrote:
> > > Reported to PCRE[1] with mention of GNU grep being also affected.
> > >
> > > [1] https://github.com/PCRE2Project/pcre2/issues/185
> >
> > Yikes. This is a big deal.
> > Thank you for the patch and added test.

I've also added the new names to THANKS.in and pushed this:
https://git.savannah.gnu.org/cgit/grep.git/commit/?id=5e3b760f65f13856e5717e5b9d935f5b4a615be3

Subject: grep -P does not set PCRE2_UCP
From: Karl Pettersson <[email-masked]>
To: bug-grep <[email-masked]>
Date: Fri, 6 Jan 2023 21:41:53 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi

Using grep -P for boundary matches yields incorrect results with
non-ASCII letters:

$ echo 'Öst' | grep -P '\bs'
Öst

The output should be nothing in this case, and the culprit seems to be
this line in pcresearch.c:

      flags |= PCRE2_UTF;

If the PCRE2_UCP flag is added according to this, the program behaves
correctly:

      flags |= PCRE2_UTF|PCRE2_UCP;

The pcre2grep test program in the pcre2 has the same problem, and I
filed an issue there too:

https://github.com/PCRE2Project/pcre2/issues/185

A Twitter discussion with more examples:

https://twitter.com/gro_tsen/status/1610972356972875777

Kind regards
-- 
Karl Pettersson
Uppsala, Sverige/Sweden

https://static-dust.klpn.se/