GNU bug report logs - #43999
sed: couldn't re-allocate memory

Previous Next

To reply to this bug, email your comments to 43999 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mark Filipak <markfilipak <at> neo.rr.com>
To: bug-sed <at> gnu.org
Subject: sed: couldn't re-allocate memory
Date: Wed, 14 Oct 2020 16:40:42 -0400

Kindly expedite this and contact me for any reason.

The file "1,073,709,056 bytes" provokes an error (& zero output), but only if piped from 'tr' and 
only for a particular pattern: /00000100/.
the file "1,073,739,776 bytes" succeeds with identical parameters.
The pipe through 'tr' appears not to be the problem.

$ sed --version
sed (GNU sed) 4.2.2
...
$ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt
sed: couldn't re-allocate memory
$ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 's/000001/\x0D\x0A&/g' > foo.txt
$ xxd -p -u "1,073,709,056 bytes" | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt
$ xxd -p -u "1,073,739,776 bytes" | tr -d '\n' | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt
$

You probably want the two source files, or perhaps only the source file that provokes the error. 
Kindly let me know how I can send it to you.

Regards,
Mark Filipak.

PS: For your sake, I posted a request for vetting this case
https://forums.linuxmint.com/viewtopic.php?f=47&t=332836
I got no responses, so it has not been replicated by someone else.

Message #8 received at 43999 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: markfilipak.linux+gnuorg <at> gmail.com, 43999 <at> debbugs.gnu.org
Subject: Re: bug#43999: sed: couldn't re-allocate memory
Date: Thu, 15 Oct 2020 00:35:42 -0600

tag 43999 moreinfo
stop

Hello,

On 2020-10-14 2:40 p.m., Mark Filipak wrote:
> The file "1,073,709,056 bytes" provokes an error (& zero output), but 
> only if piped from 'tr' and only for a particular pattern: /00000100/.
> the file "1,073,739,776 bytes" succeeds with identical parameters.
> The pipe through 'tr' appears not to be the problem.
> 
> $ sed --version
> sed (GNU sed) 4.2.2

First,
sed version 4.2.2 is 8 years old.
It is not supported any more.
Please try with the latest version of sed and report back if you still 
get the error.

Second,
please send some information about your environment, namely "uname -a",
are you using a 32bit or 64bit system,
and how much RAM and SWAP does your machine have.

> ...
> $ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 
> 's/00000100/\x0D\x0A&/g' > foo.txt
> sed: couldn't re-allocate memory

Third,
You did not describe what your input file contains.
I implicitly assume from the file name that the file is large (~1GB?).
I did not try reproduce this error, but my hunch is that by deleting
input newlines you are asking sed to load a very large input string and
store it in memory - so a memory-related error is not unexpected.

regards,
 - assaf

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.