GNU bug report logs - #60690
[PATCH v2] grep: correctly identify utf-8 characters with \{b,w} in -P

Previous Next

Full log

View this message in rfc822 format

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Jim Meyering <jim <at> meyering.net>
Cc: demerphq <at> gmail.com, 60690 <at> debbugs.gnu.org, mega lith01 <megalith01 <at> gmail.com>, Carlo Marcelo Arenas Belón <carenas <at> gmail.com>, Ævar Arnfjörð Bjarmason <avarab <at> gmail.com>, gitster <at> pobox.com, pcre-dev <at> exim.org, Tukusej’s Sirs <tukusejssirs <at> protonmail.com>, git <at> vger.kernel.org
Subject: bug#60690: -P '\d' in GNU and git grep
Date: Mon, 3 Apr 2023 23:46:57 -0700

On 2023-04-03 20:30, Jim Meyering wrote:
> have you seen justification
> (other than for compatibility with some other tool or language) for
> allowing \d to match non-ASCII by default, in spite of the risks?

In the example Ævar supplied in <https://bugs.gnu.org/60690>, my 
impression was that it was better when \d matched non-ASCII digits. That 
is, in a UTF-8 locale it's better when \d finds matches in these lines:

>> 	> git-gui/po/ja.po:"- 第１行: 何をしたか、を１行で要約。\n"
>> 	> git-gui/po/ja.po:"- 第２行: 空白\n"

because they contain the Japanese digits "１" and "２". This was the only 
example I recall being given.

Also, I find it odd that grep -P '^[\w\d]*$' matches lines containing 
any sort of Arabic word characters, but it rejects lines containing 
Arabic digits like "٣" that are perfectly reasonable in Arabic-language 
text. I also find it odd that [\d] and [[:digit:]] mean different things.

There are arguments on the other side, otherwise we wouldn't be having 
this discussion. And it's true that grep -P '\d' formerly rejected 
Arabic digits (though it's also true that grep -P '\w' formerly rejected 
Arabic letters...). Still, the cure's oddness and incompatibility with 
Git, Perl, etc. appears to me to be worse than the disease of dealing 
with grep -P invocations that need to use [0-9] or LC_ALL="C" anyway if 
they want to be portable to any program other than GNU grep.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.