GNU bug report logs - #40634
Massive pattern list handling with -E format seems very slow since 2.28.

Previous Next

Full log

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: fryasu <at> yahoo.co.jp
To: "bug-grep <at> gnu.org" <bug-grep <at> gnu.org>
Subject: Massive pattern list handling with -E format seems very slow since
 2.28.
Date: Wed, 15 Apr 2020 09:26:55 +0900 (JST)

Hi,

Massive pattern list handling with -E format seems very
slow, since grep 2.28.

Conversion from -E format to -F format may have problem
about performance.

When the processing time is measured by the script below,
the result isobviously different between 2.28 and 2.27.

----
#!/bin/bash
export LC_ALL=C

rm -f grep-patterns.txt
for i in {1..2000}; do
     pat=$(echo -n "$i" | sha1sum | cut -f1 -d ' ')
     echo -e "$pat$pat(\$|$pat)" >> grep-patterns.txt
done

echo executing grep...
time grep -E -v -m1 -f grep-patterns.txt /dev/null
----

The following is the results in my PC with fedora's RPM.
https://koji.fedoraproject.org/koji/packageinfo?packageID=1023

- result with grep 2.28

  real 0m11.087s / user 0m11.027s / sys 0m0.037s

- result with grep 2.27

  real 0m0.144s / user 0m0.116s / sys 0m0.027s

With also recent 3.4, result is same.


I hope you find it useful.


regards,

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.