GNU bug report logs - #9307
sort crashes on big files

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 9307 in the body.
You can then email your comments to 9307 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Michael Stahn <michael.stahn.42 <at> googlemail.com>
To: bug-coreutils <at> gnu.org
Subject: sort crashes on big files
Date: Mon, 15 Aug 2011 21:56:11 +0200

Hi all,

Calling sort -u -S 1G file.txt > output.txt from coreutils-8.7 crashes
sort. Increasing memory size doesn't help. The size of file.txt is
approximately 1G. The program was called with other text-files with similar
size and it crashes only in some cases (same file every time). I cant upload it
here because of the size, but it's just a plain textfile.
This bug only seems to happen if sort is called with "-u" option.

Steps to Reproduce:
1. call sort -u -S 1G file.txt > output.txt with a min 1G plaintext-file

Actual Results:
Sort crashes with "Killed" or "Segmenation fault", depending on the file. Also
some sortXXXX-files in /tmp ARE NOT deleted if sort crashes! Your disk will
become clobbered with trash in a very short time when sorting big files.

Backtrace (without debug symbols):

Starting program: /bin/sort -u -S 1G bigfile.txt
[Thread debugging using libthread_db enabled]
[New Thread 0x77af2b70 (LWP 7473)]
[Thread 0x77af2b70 (LWP 7473) exited]
[New Thread 0x77af2b70 (LWP 7828)]
[Thread 0x77af2b70 (LWP 7828) exited]
[New Thread 0x77af2b70 (LWP 8248)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x77af2b70 (LWP 8248)]
0x0804c3a3 in ?? ()
#0  0x0804c3a3 in ?? ()
No symbol table info available.
#1  0x0804c65d in ?? ()
No symbol table info available.
#2  0x0804d4c4 in ?? ()
No symbol table info available.
#3  0x0804da61 in ?? ()
No symbol table info available.
#4  0xb7fb4d23 in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#5  0xb7f22bfe in clone () from /lib/libc.so.6
No symbol table info available.

Message #10 received at 9307-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Michael Stahn <michael.stahn.42 <at> googlemail.com>
Cc: 9307-done <at> debbugs.gnu.org
Subject: Re: bug#9307: sort crashes on big files (coreutils-8.7)
Date: Mon, 15 Aug 2011 22:10:20 +0200

Michael Stahn wrote:
> Calling sort -u -S 1G file.txt > output.txt from coreutils-8.7 crashes
> sort. Increasing memory size doesn't help. The size of file.txt is
> approximately 1G. The program was called with other text-files with similar
> size and it crashes only in some cases (same file every time). I cant upload it
> here because of the size, but it's just a plain textfile.
> This bug only seems to happen if sort is called with "-u" option.
>
> Steps to Reproduce:
> 1. call sort -u -S 1G file.txt > output.txt with a min 1G plaintext-file
>
> Actual Results:
> Sort crashes with "Killed" or "Segmenation fault", depending on the file. Also
> some sortXXXX-files in /tmp ARE NOT deleted if sort crashes! Your disk will
> become clobbered with trash in a very short time when sorting big files.

Thanks for the report.
That sounds like a bug that was fixed in coreutils-8.8:

    * Noteworthy changes in release 8.8 (2010-12-22) [stable]

    ** Bug fixes
      ...

      sort with at least two threads no longer segfaults due to use of pointers
      into the stack of an expired thread. [bug introduced in coreutils-8.6]

The latest is coreutils-8.12.
If you can make the latest fail, please let us know.
In the mean time, I'm closing this issue.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.