GNU bug report logs - #12738
'sort' wastes cpu-cycles of all available cpus

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 12738 in the body.
You can then email your comments to 12738 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Schwidom, Frank" <Frank.Schwidom <at> HS-Lausitz.de>
To: "bug-coreutils <at> gnu.org" <bug-coreutils <at> gnu.org>
Subject: 'sort' wastes cpu-cycles of all available cpus
Date: Fri, 26 Oct 2012 09:12:40 +0200

[Message part 1 (text/plain, inline)]

In some situations sort wastes cpu-cycles of all available cpus.

Hi,

I refer to 'sort' of the following Version:

$ sort --version
sort (GNU coreutils) 8.7
Packaged by Gentoo (8.7 (p1))
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and Paul Eggert.

on the system:

$ uname -a
Linux CUDA 2.6.38-gentoo-r6 #12 SMP Fri Sep 14 12:59:07 CEST 2012 x86_64 Intel(R) Xeon(R) CPU E5645 @ 2.40GHz GenuineIntel GNU/Linux

The problem causes, if the input of sort is very long but the output is not taken.

if i call:

# seq 1 10000000 | sort -R | wc -l
10000000

then sort itself uses only circa 10 cpus:

top:
 8316 root      20   0  359m 185m  676 S 1009  1.5   4:08.18 sort

or ca. 4 cpus in this case:

# seq 1 10000000 | sort -n | wc -l
10000000

top:
 8782 root      20   0  295m 185m  672 S  382  1.5   0:25.18 sort

while sort is running, this behaviour is ok. it would be ok too, if
it would use all cpus.

but, if i use:

# seq 1 1000000 | sort -n | less

and while im not scrolling to the bottom in 'less'
sort uses all cpus

top:
 9421 root      20   0  447m 109m  660 R 2295  0.9   3:12.72 sort

in this case, sort is already done sorting (it took and sorted
the whole input and 'less' displays values).
but it takes a lot of cpu-cycles on all cpus.

if i scroll to the bottom, 'less' takes all values of the sort output
and sort will end; this is ok.

Regards


Mit freundlichen Grüßen

Frank Schwidom
Dipl.-Inf. (FH)

Hochschule Lausitz (FH)
Fertigungstechnik/ Tribologie
Prof. Dr.-Ing. R. Winkelmann
Großenhainer Str. 57
01968 Senftenberg

Fone :       +49 (0) 3573 85 435
Fax:          +49 (0) 3573 85 426
Email:       Frank.Schwidom <at> HS-Lausitz.de<mailto:Frank.Schwidom <at> HS-Lausitz.de>
Homepage:  http://www2.fh-lausitz.de/fhl/mb/labor/tribologie/Index.html

[Message part 2 (text/html, inline)]

Message #10 received at 12738-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: "Schwidom\, Frank" <Frank.Schwidom <at> HS-Lausitz.de>
Cc: 12738-done <at> debbugs.gnu.org
Subject: Re: bug#12738: 'sort' wastes cpu-cycles of all available cpus
Date: Fri, 26 Oct 2012 11:42:31 +0200

Schwidom, Frank wrote:
> In some situations sort wastes cpu-cycles of all available cpus.
>
> $ sort --version
> sort (GNU coreutils) 8.7
...
> but, if i use:
>
> # seq 1 1000000 | sort -n | less
>
> and while im not scrolling to the bottom in 'less'
> sort uses all cpus
>
> top:
>  9421 root      20   0  447m 109m  660 R 2295  0.9   3:12.72 sort
>
> in this case, sort is already done sorting (it took and sorted
> the whole input and 'less' displays values).
> but it takes a lot of cpu-cycles on all cpus.
>
> if i scroll to the bottom, 'less' takes all values of the sort output
> and sort will end; this is ok.

Thank you for the report.
However, your version of sort is quite old and
that bug was fixed almost two years ago:
(from "NEWS")

    * Noteworthy changes in release 8.8 (2010-12-22) [stable]

    ** Bug fixes

      sort with at least two threads and with blocked output would busy-loop
      (spinlock) all threads, often using 100% of available CPU cycles to
      do no work.  I.e., "sort < big-file | less" could waste a lot of power.
      [bug introduced in coreutils-8.6]

The latest release is coreutils-8.20.

Message #13 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mike Frysinger <vapier <at> gentoo.org>
To: bug-coreutils <at> gnu.org
Cc: 12738-done <at> debbugs.gnu.org, "Schwidom,
	Frank" <Frank.Schwidom <at> hs-lausitz.de>, Jim Meyering <jim <at> meyering.net>
Subject: Re: bug#12738: 'sort' wastes cpu-cycles of all available cpus
Date: Fri, 26 Oct 2012 17:28:04 -0400

[Message part 1 (text/plain, inline)]

On Friday 26 October 2012 05:42:31 Jim Meyering wrote:
> Schwidom, Frank wrote:
> > In some situations sort wastes cpu-cycles of all available cpus.
> > 
> > $ sort --version
> > sort (GNU coreutils) 8.7
> > Packaged by Gentoo (8.7 (p1))
> 
> However, your version of sort is quite old and
> that bug was fixed almost two years ago:
> (from "NEWS")
> 
>     * Noteworthy changes in release 8.8 (2010-12-22) [stable]
> 
>     ** Bug fixes
> 
>       sort with at least two threads and with blocked output would
> busy-loop (spinlock) all threads, often using 100% of available CPU cycles
> to do no work.  I.e., "sort < big-file | less" could waste a lot of power.
> [bug introduced in coreutils-8.6]
> 
> The latest release is coreutils-8.20.

and Gentoo has stabilized newer versions in the interim with 8.16 going stable 
most recently in Aug 2012.
-mike

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.