GNU bug report logs - #73721
grep perf docs barely mention mem usage

Previous Next

Full log

Message #10 received at 73721-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: mark.yagnatinsky <at> barclays.com
Cc: 73721-done <at> debbugs.gnu.org
Subject: Re: bug#73721: grep perf docs barely mention mem usage
Date: Wed, 9 Oct 2024 12:03:16 -0700

On 2024-10-09 10:01, mark.yagnatinsky--- via Bug reports for GNU grep wrote:
> After a bit of research, it seems that once upon a time, grep used mmap where possible, but it no longer does this.
> Thus, peak memory usage will be proportional to the length of the longest line in the file.
> Thus, if use the "-z multiline hack" to search across lines, grep will read the whole file into memory.
> Thus, if I try this on a huge file, I will likely have a bad time.
> (e.g., a 5 gig file would fail in a 32-bit grep, and would increase memory pressure on the system on a 64-bit grep.)
> 
> Is the above about right?

Sounds right.

mmap likely wouldn't help much. As I recall, it typically made 'grep' 
slower.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.