GNU bug report logs - #9307
sort crashes on big files

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Jim Meyering <jim <at> meyering.net>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#9307: closed (sort crashes on big files)
Date: Mon, 15 Aug 2011 20:13:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Mon, 15 Aug 2011 22:10:20 +0200
with message-id <87bovqfnbn.fsf <at> rho.meyering.net>
and subject line Re: bug#9307: sort crashes on big files (coreutils-8.7)
has caused the GNU bug report #9307,
regarding sort crashes on big files
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
9307: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9307
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Michael Stahn <michael.stahn.42 <at> googlemail.com>
To: bug-coreutils <at> gnu.org
Subject: sort crashes on big files
Date: Mon, 15 Aug 2011 21:56:11 +0200

Hi all,

Calling sort -u -S 1G file.txt > output.txt from coreutils-8.7 crashes
sort. Increasing memory size doesn't help. The size of file.txt is
approximately 1G. The program was called with other text-files with similar
size and it crashes only in some cases (same file every time). I cant upload it
here because of the size, but it's just a plain textfile.
This bug only seems to happen if sort is called with "-u" option.

Steps to Reproduce:
1. call sort -u -S 1G file.txt > output.txt with a min 1G plaintext-file

Actual Results:
Sort crashes with "Killed" or "Segmenation fault", depending on the file. Also
some sortXXXX-files in /tmp ARE NOT deleted if sort crashes! Your disk will
become clobbered with trash in a very short time when sorting big files.

Backtrace (without debug symbols):

Starting program: /bin/sort -u -S 1G bigfile.txt
[Thread debugging using libthread_db enabled]
[New Thread 0x77af2b70 (LWP 7473)]
[Thread 0x77af2b70 (LWP 7473) exited]
[New Thread 0x77af2b70 (LWP 7828)]
[Thread 0x77af2b70 (LWP 7828) exited]
[New Thread 0x77af2b70 (LWP 8248)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x77af2b70 (LWP 8248)]
0x0804c3a3 in ?? ()
#0  0x0804c3a3 in ?? ()
No symbol table info available.
#1  0x0804c65d in ?? ()
No symbol table info available.
#2  0x0804d4c4 in ?? ()
No symbol table info available.
#3  0x0804da61 in ?? ()
No symbol table info available.
#4  0xb7fb4d23 in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#5  0xb7f22bfe in clone () from /lib/libc.so.6
No symbol table info available.

[Message part 3 (message/rfc822, inline)]

From: Jim Meyering <jim <at> meyering.net>
To: Michael Stahn <michael.stahn.42 <at> googlemail.com>
Cc: 9307-done <at> debbugs.gnu.org
Subject: Re: bug#9307: sort crashes on big files (coreutils-8.7)
Date: Mon, 15 Aug 2011 22:10:20 +0200

Michael Stahn wrote:
> Calling sort -u -S 1G file.txt > output.txt from coreutils-8.7 crashes
> sort. Increasing memory size doesn't help. The size of file.txt is
> approximately 1G. The program was called with other text-files with similar
> size and it crashes only in some cases (same file every time). I cant upload it
> here because of the size, but it's just a plain textfile.
> This bug only seems to happen if sort is called with "-u" option.
>
> Steps to Reproduce:
> 1. call sort -u -S 1G file.txt > output.txt with a min 1G plaintext-file
>
> Actual Results:
> Sort crashes with "Killed" or "Segmenation fault", depending on the file. Also
> some sortXXXX-files in /tmp ARE NOT deleted if sort crashes! Your disk will
> become clobbered with trash in a very short time when sorting big files.

Thanks for the report.
That sounds like a bug that was fixed in coreutils-8.8:

    * Noteworthy changes in release 8.8 (2010-12-22) [stable]

    ** Bug fixes
      ...

      sort with at least two threads no longer segfaults due to use of pointers
      into the stack of an expired thread. [bug introduced in coreutils-8.6]

The latest is coreutils-8.12.
If you can make the latest fail, please let us know.
In the mean time, I'm closing this issue.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.