From unknown Mon Jun 23 07:50:13 2025 X-Loop: help-debbugs@gnu.org Subject: bug#9307: sort crashes on big files Resent-From: Michael Stahn Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Mon, 15 Aug 2011 20:01:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 9307 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: To: 9307@debbugs.gnu.org X-Debbugs-Original-To: bug-coreutils@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.131343844610288 (code B ref -1); Mon, 15 Aug 2011 20:01:02 +0000 Received: (at submit) by debbugs.gnu.org; 15 Aug 2011 20:00:46 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qt3L4-0002ft-3G for submit@debbugs.gnu.org; Mon, 15 Aug 2011 16:00:46 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qt3IO-0002b9-OE for submit@debbugs.gnu.org; Mon, 15 Aug 2011 15:58:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Qt3Gi-0004O9-BM for submit@debbugs.gnu.org; Mon, 15 Aug 2011 15:56:17 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, T_DKIM_INVALID, T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:48076) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qt3Gi-0004O5-9r for submit@debbugs.gnu.org; Mon, 15 Aug 2011 15:56:16 -0400 Received: from eggs.gnu.org ([140.186.70.92]:43060) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qt3Gh-00043A-1P for bug-coreutils@gnu.org; Mon, 15 Aug 2011 15:56:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Qt3Gf-0004Nb-Dn for bug-coreutils@gnu.org; Mon, 15 Aug 2011 15:56:15 -0400 Received: from mail-ww0-f49.google.com ([74.125.82.49]:38624) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qt3Gf-0004NL-1G for bug-coreutils@gnu.org; Mon, 15 Aug 2011 15:56:13 -0400 Received: by wwf10 with SMTP id 10so4117256wwf.30 for ; Mon, 15 Aug 2011 12:56:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=UflAAP8bdPNONqNpCxw+6noN/jql52DqA1zRFc3/8fk=; b=PqsF3uPJRGvsVfg5VJFLZqxOCg4qPtTgTG9ACIeWPQFlX1NEDxZH8dLKEMaxtgumTD mEVQyH/DQ/Br7csJgeDtvmQO8sm38iJut3jwLUBxlj9bBq1YxVRHqyiuSsg35HVvhAr8 l5KLHiyPoJK19+i3zT4H3fj+wq+NsTgLKU6j8= MIME-Version: 1.0 Received: by 10.227.11.70 with SMTP id s6mr3831586wbs.40.1313438171409; Mon, 15 Aug 2011 12:56:11 -0700 (PDT) Received: by 10.216.11.212 with HTTP; Mon, 15 Aug 2011 12:56:11 -0700 (PDT) Date: Mon, 15 Aug 2011 21:56:11 +0200 Message-ID: From: Michael Stahn Content-Type: text/plain; charset=ISO-8859-1 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -5.9 (-----) X-Mailman-Approved-At: Mon, 15 Aug 2011 16:00:44 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) Hi all, Calling sort -u -S 1G file.txt > output.txt from coreutils-8.7 crashes sort. Increasing memory size doesn't help. The size of file.txt is approximately 1G. The program was called with other text-files with similar size and it crashes only in some cases (same file every time). I cant upload it here because of the size, but it's just a plain textfile. This bug only seems to happen if sort is called with "-u" option. Steps to Reproduce: 1. call sort -u -S 1G file.txt > output.txt with a min 1G plaintext-file Actual Results: Sort crashes with "Killed" or "Segmenation fault", depending on the file. Also some sortXXXX-files in /tmp ARE NOT deleted if sort crashes! Your disk will become clobbered with trash in a very short time when sorting big files. Backtrace (without debug symbols): Starting program: /bin/sort -u -S 1G bigfile.txt [Thread debugging using libthread_db enabled] [New Thread 0x77af2b70 (LWP 7473)] [Thread 0x77af2b70 (LWP 7473) exited] [New Thread 0x77af2b70 (LWP 7828)] [Thread 0x77af2b70 (LWP 7828) exited] [New Thread 0x77af2b70 (LWP 8248)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x77af2b70 (LWP 8248)] 0x0804c3a3 in ?? () #0 0x0804c3a3 in ?? () No symbol table info available. #1 0x0804c65d in ?? () No symbol table info available. #2 0x0804d4c4 in ?? () No symbol table info available. #3 0x0804da61 in ?? () No symbol table info available. #4 0xb7fb4d23 in start_thread () from /lib/libpthread.so.0 No symbol table info available. #5 0xb7f22bfe in clone () from /lib/libc.so.6 No symbol table info available. From unknown Mon Jun 23 07:50:13 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.427 (Entity 5.427) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Michael Stahn Subject: bug#9307: closed (Re: bug#9307: sort crashes on big files (coreutils-8.7)) Message-ID: References: <87bovqfnbn.fsf@rho.meyering.net> X-Gnu-PR-Message: they-closed 9307 X-Gnu-PR-Package: coreutils Reply-To: 9307@debbugs.gnu.org Date: Mon, 15 Aug 2011 20:13:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1313439182-11394-1" This is a multi-part message in MIME format... ------------=_1313439182-11394-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #9307: sort crashes on big files which was filed against the coreutils package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 9307@debbugs.gnu.org. --=20 9307: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D9307 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1313439182-11394-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 9307-done) by debbugs.gnu.org; 15 Aug 2011 20:12:11 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qt3W3-0002wb-JR for submit@debbugs.gnu.org; Mon, 15 Aug 2011 16:12:11 -0400 Received: from mx.meyering.net ([82.230.74.64]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qt3W0-0002wT-KK for 9307-done@debbugs.gnu.org; Mon, 15 Aug 2011 16:12:06 -0400 Received: from rho.meyering.net (localhost.localdomain [127.0.0.1]) by rho.meyering.net (Acme Bit-Twister) with ESMTP id 579B56002D; Mon, 15 Aug 2011 22:10:20 +0200 (CEST) From: Jim Meyering To: Michael Stahn Subject: Re: bug#9307: sort crashes on big files (coreutils-8.7) In-Reply-To: (Michael Stahn's message of "Mon, 15 Aug 2011 21:56:11 +0200") References: Date: Mon, 15 Aug 2011 22:10:20 +0200 Message-ID: <87bovqfnbn.fsf@rho.meyering.net> Lines: 30 MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -6.1 (------) X-Debbugs-Envelope-To: 9307-done Cc: 9307-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.1 (------) Michael Stahn wrote: > Calling sort -u -S 1G file.txt > output.txt from coreutils-8.7 crashes > sort. Increasing memory size doesn't help. The size of file.txt is > approximately 1G. The program was called with other text-files with similar > size and it crashes only in some cases (same file every time). I cant upload it > here because of the size, but it's just a plain textfile. > This bug only seems to happen if sort is called with "-u" option. > > Steps to Reproduce: > 1. call sort -u -S 1G file.txt > output.txt with a min 1G plaintext-file > > Actual Results: > Sort crashes with "Killed" or "Segmenation fault", depending on the file. Also > some sortXXXX-files in /tmp ARE NOT deleted if sort crashes! Your disk will > become clobbered with trash in a very short time when sorting big files. Thanks for the report. That sounds like a bug that was fixed in coreutils-8.8: * Noteworthy changes in release 8.8 (2010-12-22) [stable] ** Bug fixes ... sort with at least two threads no longer segfaults due to use of pointers into the stack of an expired thread. [bug introduced in coreutils-8.6] The latest is coreutils-8.12. If you can make the latest fail, please let us know. In the mean time, I'm closing this issue. ------------=_1313439182-11394-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 15 Aug 2011 20:00:46 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qt3L4-0002ft-3G for submit@debbugs.gnu.org; Mon, 15 Aug 2011 16:00:46 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Qt3IO-0002b9-OE for submit@debbugs.gnu.org; Mon, 15 Aug 2011 15:58:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Qt3Gi-0004O9-BM for submit@debbugs.gnu.org; Mon, 15 Aug 2011 15:56:17 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, T_DKIM_INVALID, T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:48076) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qt3Gi-0004O5-9r for submit@debbugs.gnu.org; Mon, 15 Aug 2011 15:56:16 -0400 Received: from eggs.gnu.org ([140.186.70.92]:43060) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qt3Gh-00043A-1P for bug-coreutils@gnu.org; Mon, 15 Aug 2011 15:56:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Qt3Gf-0004Nb-Dn for bug-coreutils@gnu.org; Mon, 15 Aug 2011 15:56:15 -0400 Received: from mail-ww0-f49.google.com ([74.125.82.49]:38624) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qt3Gf-0004NL-1G for bug-coreutils@gnu.org; Mon, 15 Aug 2011 15:56:13 -0400 Received: by wwf10 with SMTP id 10so4117256wwf.30 for ; Mon, 15 Aug 2011 12:56:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=UflAAP8bdPNONqNpCxw+6noN/jql52DqA1zRFc3/8fk=; b=PqsF3uPJRGvsVfg5VJFLZqxOCg4qPtTgTG9ACIeWPQFlX1NEDxZH8dLKEMaxtgumTD mEVQyH/DQ/Br7csJgeDtvmQO8sm38iJut3jwLUBxlj9bBq1YxVRHqyiuSsg35HVvhAr8 l5KLHiyPoJK19+i3zT4H3fj+wq+NsTgLKU6j8= MIME-Version: 1.0 Received: by 10.227.11.70 with SMTP id s6mr3831586wbs.40.1313438171409; Mon, 15 Aug 2011 12:56:11 -0700 (PDT) Received: by 10.216.11.212 with HTTP; Mon, 15 Aug 2011 12:56:11 -0700 (PDT) Date: Mon, 15 Aug 2011 21:56:11 +0200 Message-ID: Subject: sort crashes on big files From: Michael Stahn To: bug-coreutils@gnu.org Content-Type: text/plain; charset=ISO-8859-1 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -5.9 (-----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Mon, 15 Aug 2011 16:00:44 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) Hi all, Calling sort -u -S 1G file.txt > output.txt from coreutils-8.7 crashes sort. Increasing memory size doesn't help. The size of file.txt is approximately 1G. The program was called with other text-files with similar size and it crashes only in some cases (same file every time). I cant upload it here because of the size, but it's just a plain textfile. This bug only seems to happen if sort is called with "-u" option. Steps to Reproduce: 1. call sort -u -S 1G file.txt > output.txt with a min 1G plaintext-file Actual Results: Sort crashes with "Killed" or "Segmenation fault", depending on the file. Also some sortXXXX-files in /tmp ARE NOT deleted if sort crashes! Your disk will become clobbered with trash in a very short time when sorting big files. Backtrace (without debug symbols): Starting program: /bin/sort -u -S 1G bigfile.txt [Thread debugging using libthread_db enabled] [New Thread 0x77af2b70 (LWP 7473)] [Thread 0x77af2b70 (LWP 7473) exited] [New Thread 0x77af2b70 (LWP 7828)] [Thread 0x77af2b70 (LWP 7828) exited] [New Thread 0x77af2b70 (LWP 8248)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x77af2b70 (LWP 8248)] 0x0804c3a3 in ?? () #0 0x0804c3a3 in ?? () No symbol table info available. #1 0x0804c65d in ?? () No symbol table info available. #2 0x0804d4c4 in ?? () No symbol table info available. #3 0x0804da61 in ?? () No symbol table info available. #4 0xb7fb4d23 in start_thread () from /lib/libpthread.so.0 No symbol table info available. #5 0xb7f22bfe in clone () from /lib/libc.so.6 No symbol table info available. ------------=_1313439182-11394-1--