GNU bug report logs - #60618
unicode characters are not identified as such for \w and \b with -P

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Karl Pettersson <karl.pettersson <at> klpn.se>
Subject: bug#60621: closed (Re: bug#60618: unicode characters are not
 identified as such for \w and \b with -P)
Date: Sun, 08 Jan 2023 02:30:03 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#60618: grep -P does not set PCRE2_UCP

which was filed against the grep package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 60621 <at> debbugs.gnu.org.

-- 
60618: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=60618
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Jim Meyering <jim <at> meyering.net>
To: Carlo Arenas <carenas <at> gmail.com>
Cc: 60618-done <at> debbugs.gnu.org
Subject: Re: bug#60618: unicode characters are not identified as such for \w
 and \b with -P
Date: Sat, 7 Jan 2023 18:28:49 -0800

On Fri, Jan 6, 2023 at 11:37 PM Jim Meyering <jim <at> meyering.net> wrote:
> On Fri, Jan 6, 2023 at 11:28 PM Jim Meyering <jim <at> meyering.net> wrote:
> > On Fri, Jan 6, 2023 at 7:49 PM Carlo Arenas <carenas <at> gmail.com> wrote:
> > > Reported to PCRE[1] with mention of GNU grep being also affected.
> > >
> > > [1] https://github.com/PCRE2Project/pcre2/issues/185
> >
> > Yikes. This is a big deal.
> > Thank you for the patch and added test.

I've also added the new names to THANKS.in and pushed this:
https://git.savannah.gnu.org/cgit/grep.git/commit/?id=5e3b760f65f13856e5717e5b9d935f5b4a615be3

[Message part 3 (message/rfc822, inline)]

From: Karl Pettersson <karl.pettersson <at> klpn.se>
To: bug-grep <bug-grep <at> gnu.org>
Subject: grep -P does not set PCRE2_UCP
Date: Fri, 6 Jan 2023 21:41:53 +0100

Hi

Using grep -P for boundary matches yields incorrect results with
non-ASCII letters:

$ echo 'Öst' | grep -P '\bs'
Öst

The output should be nothing in this case, and the culprit seems to be
this line in pcresearch.c:

      flags |= PCRE2_UTF;

If the PCRE2_UCP flag is added according to this, the program behaves
correctly:

      flags |= PCRE2_UTF|PCRE2_UCP;

The pcre2grep test program in the pcre2 has the same problem, and I
filed an issue there too:

https://github.com/PCRE2Project/pcre2/issues/185

A Twitter discussion with more examples:

https://twitter.com/gro_tsen/status/1610972356972875777

Kind regards
-- 
Karl Pettersson
Uppsala, Sverige/Sweden

https://static-dust.klpn.se/

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.