GNU bug report logs - #7489
[coreutils] over aggressive threads in sort

Previous Next

Full log

View this message in rfc822 format

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Jim Meyering <jim <at> meyering.net>
Cc: Chen Guo <chen.guo.0625 <at> gmail.com>, Pádraig Brady <P <at> draigBrady.com>, DJ Lucas <dj <at> linuxfromscratch.org>, 7489 <at> debbugs.gnu.org, coreutils <at> gnu.org
Subject: bug#7489: [coreutils] over aggressive threads in sort
Date: Mon, 06 Dec 2010 18:23:38 -0800

On 12/05/10 03:21, Jim Meyering wrote:
>   seq -w 200000 > exp && tac exp > in
>   PATH=.:$PATH ./sort --compress-program=dzip -S 1k in > out
> 
> That gets stuck in waitpid (from sort.c's reap), waiting for a
> dzip invocation that appears will never terminate.  This is also
> on that same 4-core system, and is relatively easy to reproduce,
> so it should be easy to identify the offending change, but I'm
> out of time, now.
> 

I've verified this bug.  It occurs regardless of whether the
December 3 changes are applied, so it looks like it was an
earlier change.  I've also verified that it occurs with
--parallel=1 so it seems unlikely it would be thread-related.

What happens is that 'sort' creates a pipe to a compressor
but then never closes the pipe.  When it waits for the compressor
to finish there is an obvious deadlock.

I'll see if I can track it down, but my guess it's somewhere
in the compressor/decompressor code.  It's doing some lazy
evaluation and I wouldn't be surprised if that goes haywire
in some cases.

I'd guess there's also a bug in the thread code too,
having to do with your core dump (and Chen's; could be
two bugs).

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.