GNU bug report logs - #21527
avoiding "memory exhausted" in 2.5.4?

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21527 in the body.
You can then email your comments to 21527 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Aaron Davies <aaron.davies <at> gmail.com>
To: bug-grep <at> gnu.org
Subject: avoiding "memory exhausted" in 2.5.4?
Date: Sun, 20 Sep 2015 17:47:24 -0400

i'm trying to grep an entire volume for a short, fixed string, but no matter what i do, i keep getting "grep: memory exhausted" errors

the only versions i have immediately available to me are 2.5.4 and and earlier; i can probably get an upgrade to the current one installed eventually, but it will take some time

since the string is fixed, i'm using -F

i've tried both with and without --mmap

i've tried both a -r recursive grep and separate runs using find -print0|xargs -r0 grep

since i'm really only interested in finding the file containing the string and if might be a binary file, i've also tried with and without -z on the theory that the problem was binaries without newlines, and NUL was more likely to occur than newline in such files (a suggestion i found online)

none of it's helped -- they all fail with "memory exhausted" at some point

does anyone have any other suggestions as to how to solve this? is it something fixed in a more recent grep? is there some other set of tools i could be using instead?

i'm currently working on getting some statistics on the line lengths (whether delimited by \n or \0) of the files involved, but that may take a bit of time
-- 
Aaron Davies
aaron.davies <at> gmail.com

Message #8 received at 21527 <at> debbugs.gnu.org (full text, mbox):

From: Aaron Davies <aaron.davies <at> gmail.com>
To: "21527 <at> debbugs.gnu.org" <21527 <at> debbugs.gnu.org>
Subject: Re: bug#21527: avoiding "memory exhausted" in 2.5.4?
Date: Mon, 21 Sep 2015 02:16:21 -0400

On Sep 20, 2015, at 5:47 PM, Aaron Davies <aaron.davies <at> gmail.com> wrote:

> i'm trying to grep an entire volume for a short, fixed string, but no matter what i do, i keep getting "grep: memory exhausted" errors
> 
> the only versions i have immediately available to me are 2.5.4 and and earlier; i can probably get an upgrade to the current one installed eventually, but it will take some time
> 
> since the string is fixed, i'm using -F
> 
> i've tried both with and without --mmap
> 
> i've tried both a -r recursive grep and separate runs using find -print0|xargs -r0 grep
> 
> since i'm really only interested in finding the file containing the string and if might be a binary file, i've also tried with and without -z on the theory that the problem was binaries without newlines, and NUL was more likely to occur than newline in such files (a suggestion i found online)
> 
> none of it's helped -- they all fail with "memory exhausted" at some point
> 
> does anyone have any other suggestions as to how to solve this? is it something fixed in a more recent grep? is there some other set of tools i could be using instead?
> 
> i'm currently working on getting some statistics on the line lengths (whether delimited by \n or \0) of the files involved, but that may take a bit of time

i forgot to mention, i'm also using -l on all these--i don't care about the line the string is found in (which may not even be a meaningful concept with binary files), only the file
-- 
Aaron Davies
aaron.davies <at> gmail.com

Message #11 received at 21527 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Aaron Davies <aaron.davies <at> gmail.com>
Cc: "21527 <at> debbugs.gnu.org" <21527 <at> debbugs.gnu.org>
Subject: Re: bug#21527: avoiding "memory exhausted" in 2.5.4?
Date: Mon, 21 Sep 2015 09:02:45 +0100

On Mon, Sep 21, 2015 at 7:16 AM, Aaron Davies <aaron.davies <at> gmail.com> wrote:
> On Sep 20, 2015, at 5:47 PM, Aaron Davies <aaron.davies <at> gmail.com> wrote:
>
>> i'm trying to grep an entire volume for a short, fixed string, but no matter what i do, i keep getting "grep: memory exhausted" errors
>>
>> the only versions i have immediately available to me are 2.5.4 and and earlier; i can probably get an upgrade to the current one installed eventually, but it will take some time
>>
>> since the string is fixed, i'm using -F
>>
>> i've tried both with and without --mmap
>>
>> i've tried both a -r recursive grep and separate runs using find -print0|xargs -r0 grep
>>
>> since i'm really only interested in finding the file containing the string and if might be a binary file, i've also tried with and without -z on the theory that the problem was binaries without newlines, and NUL was more likely to occur than newline in such files (a suggestion i found online)
>>
>> none of it's helped -- they all fail with "memory exhausted" at some point
>>
>> does anyone have any other suggestions as to how to solve this? is it something fixed in a more recent grep? is there some other set of tools i could be using instead?

2.5.4 is more than 6 years old.
Please try the latest release, grep-2.21.
There have been many improvements (at least 16 stable releases) since 2.5.4.

Message #18 received at 21527-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Aaron Davies <aaron.davies <at> gmail.com>
Cc: 21527-done <at> debbugs.gnu.org
Subject: Re: avoiding "memory exhausted" in 2.5.4?
Date: Sat, 24 Sep 2016 14:40:23 -0700

No further comment and it has been about a year. Closing the bug report.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.