GNU bug report logs - #7489
[coreutils] over aggressive threads in sort

Previous Next

Full log

View this message in rfc822 format

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Chen Guo <chen.guo.0625 <at> gmail.com>
Cc: coreutils <at> gnu.org, Jim Meyering <jim <at> meyering.net>, 7489 <at> debbugs.gnu.org, DJ Lucas <dj <at> linuxfromscratch.org>
Subject: bug#7489: [coreutils] over aggressive threads in sort
Date: Fri, 03 Dec 2010 13:10:58 -0800

On 12/03/10 12:18, Chen Guo wrote:
> I'll try out Professor Eggert's suggestion, of switching to mutexes
> only at the top level merge.

I'm having second thoughts about that.  Yes, that'll prevent the
top-level merge (which is generating the actual output) from chewing
up CPU time.  But it already has that property, since it's outputting
to stdout.  And if second-level merges use mutexes, then the third-level
merges will spin.  We'll have to use mutexes at all levels, unless
I'm missing something.

How about this idea instead.  Keep using spin locks everywhere, but
have the top-level merge output to memory, as the lower merges already
do.  The main thread can wait for the top level merge to finish and
then generate output.  That way, none of the merges will have to wait
on an output pipe (or a slow output file).

Either option (either switch to mutexes everywhere, or have the top-level
merge go to memory) should work.  Perhaps we should try both and benchmark
them.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.