Subject: bug#18454: closed (Re: bug#18454: Improve performance when -P (PCRE) is used in UTF-8 locales)

From: [email-masked] (GNU bug Tracking System)
Date: Wed, 24 Nov 2021 03:37:04 +0000

Your bug report

#18454: Improve performance when -P (PCRE) is used in UTF-8 locales

which was filed against the grep package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
18454: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18454
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#18454: Improve performance when -P (PCRE) is used in UTF-8 locales
From: Paul Eggert <[email-masked]>
To: [email-masked]
Cc: Santiago Ruano Rincón <[email-masked]>, Norihiro Tanaka <[email-masked]>, Jim Meyering <[email-masked]>, Vincent Lefevre <[email-masked]>,Zoltán Herczeg <[email-masked]>, Eric Blake <[email-masked]>
Date: Tue, 23 Nov 2021 19:36:11 -0800
List-Id: <debbugs-submit.debbugs.gnu.org>

On 9/30/14 12:39, Paul Eggert wrote:

> GNU grep is smart 
> enough to start matching at character boundaries without checking the 
> validity of the input data.  This helps it run faster.  However, because 
> libpcre requires a validity prepass, grep -P must slow down and do the 
> validity check one way or another.  Grep does this only when libpcre is 
> used, and that's one reason grep -P is slower than plain grep.

Now that Grep master on Savannah has been changed to use PCRE2 instead 
of PCRE, the 'grep -P' performance problem seems to have been fixed, in 
that the following commands now take about the same amount of time:

grep -P zzzyyyxxx 10840.pdf
pcre2grep -U zzzyyyxxx 10840.pdf

where the file is from <http://research.nhm.org/pdfs/10840/10840.pdf>. 
Formerly, 'grep -P' was about 10x slower on this test.

My guess is that the grep -P performance boost comes from bleeding-edge 
grep using PCRE2's PCRE2_MATCH_INVALID_UTF option.

I'm closing this old bug report <https://bugs.gnu.org/18454>. We can 
always reopen it if there are still performance issues that I've missed.

Subject: Improve performance when -P (PCRE) is used in UTF-8 locales
From: Vincent Lefevre <[email-masked]>
To: [email-masked]
Date: Fri, 12 Sep 2014 03:24:49 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

With the patch that fixes bug 18266, grep -P works again on binary
files (with invalid UTF-8 sequences), but it is now significantly
slower than old versions (which could yield undefined behavior).

Timings with the Debian packages on my personal svn working copy
(binary + text files):

2.18-2   0.9s with -P, 0.4s without -P
2.20-3  11.6s with -P, 0.4s without -P

On this example, that's a 13x slowdown! Though the performance issue
would better be fixed in libpcre3, I suppose that it is not so simple
and won't occur any time soon. Things could be done in grep:

1. Ignore -P when the pattern would have the same meaning without -P
   (patterns could also be transformed, e.g. "a\d+b" -> "a[0-9]\+b",
   at least for the simplest cases).

2. Call PCRE in the C locale when this is equivalent.

3. Transform invalid bytes to null bytes in-place before the PCRE
   call. This changes the current semantic, but:
   * the semantic on invalid bytes has never been specified, AFAIK;
   * the best *practical* behavior may not be the current one
     (I personally prefer to be able to match invalid bytes, just
     like one can match top-bit-set characters in the C locale, and
     seeing such invalid bytes as equivalent to null bytes would
     not be a problem for most users, IMHO -- things can also be
     configurable).

-- 
Vincent Lefèvre <[email-masked]> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)