GNU bug report logs - #8961
stdbuf has no effect on some programs

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Bruno Haible <bruno <at> clisp.org>
Subject: bug#8961: closed (Re: bug#8961: stdbuf has no effect on some
 programs)
Date: Wed, 29 Jun 2011 23:31:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#8961: stdbuf has no effect on some programs

which was filed against the coreutils package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 8961 <at> debbugs.gnu.org.

-- 
8961: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8961
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Pádraig Brady <P <at> draigBrady.com>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 8961-done <at> debbugs.gnu.org
Subject: Re: bug#8961: stdbuf has no effect on some programs
Date: Thu, 30 Jun 2011 00:29:22 +0100

On 30/06/11 00:05, Pádraig Brady wrote:
> On 29/06/11 21:59, Bruno Haible wrote:
>> Hi,
>>
>> The glibc 'iconv' program buffers its input, and some people don't like this.
>> I thought that the 'stdbuf' program could remove the buffering, but it does not
>> work.
> 
> The following shows I think that iconv is bypassing stdio and buffering internally?
> 
> (echo; sleep 3; echo) | ltrace iconv -f ASCII
> 
> The stdbuf man page notes that:
> 
> NOTE: If COMMAND adjusts the buffering of its standard  streams  (`tee'
> does  for  e.g.) then that will override corresponding settings changed
> by `stdbuf'.  Also some filters (like `dd' and `cat'  etc.)  don't  use
> streams for I/O, and are thus unaffected by `stdbuf' settings.

In fact iconv seems to buffer for ever and so it not scalable,
as demonstrated with this consuming all of memory:

  yes | iconv

It would be OK to treat '\n' simply in all
unibyte encodings and utf8 for example,
but that would introduce an inconsistency I suppose.
Though maybe iconv could employ more scalable
buffering at least, for all encodings?

cheers,
Pádraig.

[Message part 3 (message/rfc822, inline)]

From: Bruno Haible <bruno <at> clisp.org>
To: bug-coreutils <at> gnu.org
Subject: stdbuf has no effect on some programs
Date: Wed, 29 Jun 2011 22:59:13 +0200

Hi,

The glibc 'iconv' program buffers its input, and some people don't like this.
I thought that the 'stdbuf' program could remove the buffering, but it does not
work.

How to reproduce:

Create this script and make it executable:
================================== producer ==================================
#!/bin/sh
echo Hello
/bin/sleep 3
echo World
==============================================================================

$ ./producer | /usr/bin/iconv -f ASCII
Hello
World

All the output comes at the end.

$ stdbuf -o 0 ./producer | /usr/bin/iconv -f ASCII
Hello
World

All the output comes at the end.

$ ./producer | stdbuf -i 0 /usr/bin/iconv -f ASCII
Hello
World

All the output comes at the end.

$ stdbuf -o 0 ./producer | stdbuf -i 0 /usr/bin/iconv -f ASCII
Hello
World

All the output comes at the end.

What do I need to do to get the output of the first line immediately?

/usr/bin/iconv is from glibc, but I get the same behaviour from libiconv's
'iconv' program too.

In $ ./producer | /bin/cat
I get the first line immediately, but the coreutils documentation
<http://www.gnu.org/software/coreutils/manual/html_node/stdbuf-invocation.html>
makes me think that 'stdbuf' is meant for those programs that do not work like
'cat'.

Bruno
-- 
In memoriam José Olaya <http://es.wikipedia.org/wiki/José_Olaya>

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.