Subject: bug#8961: closed (Re: bug#8961: stdbuf has no effect on some programs)

From: [email-masked] (GNU bug Tracking System)
Date: Wed, 29 Jun 2011 23:31:02 +0000

Your bug report

#8961: stdbuf has no effect on some programs

which was filed against the coreutils package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
8961: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8961
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#8961: stdbuf has no effect on some programs
From: Pádraig Brady <[email-masked]>
To: Bruno Haible <[email-masked]>
Cc: [email-masked]
Date: Thu, 30 Jun 2011 00:29:22 +0100
List-Id: <debbugs-submit.debbugs.gnu.org>

On 30/06/11 00:05, Pádraig Brady wrote:
> On 29/06/11 21:59, Bruno Haible wrote:
>> Hi,
>>
>> The glibc 'iconv' program buffers its input, and some people don't like this.
>> I thought that the 'stdbuf' program could remove the buffering, but it does not
>> work.
> 
> The following shows I think that iconv is bypassing stdio and buffering internally?
> 
> (echo; sleep 3; echo) | ltrace iconv -f ASCII
> 
> The stdbuf man page notes that:
> 
> NOTE: If COMMAND adjusts the buffering of its standard  streams  (`tee'
> does  for  e.g.) then that will override corresponding settings changed
> by `stdbuf'.  Also some filters (like `dd' and `cat'  etc.)  don't  use
> streams for I/O, and are thus unaffected by `stdbuf' settings.

In fact iconv seems to buffer for ever and so it not scalable,
as demonstrated with this consuming all of memory:

  yes | iconv

It would be OK to treat '\n' simply in all
unibyte encodings and utf8 for example,
but that would introduce an inconsistency I suppose.
Though maybe iconv could employ more scalable
buffering at least, for all encodings?

cheers,
Pádraig.

Subject: stdbuf has no effect on some programs
From: Bruno Haible <[email-masked]>
To: [email-masked]
Date: Wed, 29 Jun 2011 22:59:13 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

The glibc 'iconv' program buffers its input, and some people don't like this.
I thought that the 'stdbuf' program could remove the buffering, but it does not
work.

How to reproduce:

Create this script and make it executable:
================================== producer ==================================
#!/bin/sh
echo Hello
/bin/sleep 3
echo World
==============================================================================

$ ./producer | /usr/bin/iconv -f ASCII
Hello
World

All the output comes at the end.

$ stdbuf -o 0 ./producer | /usr/bin/iconv -f ASCII
Hello
World

All the output comes at the end.

$ ./producer | stdbuf -i 0 /usr/bin/iconv -f ASCII
Hello
World

All the output comes at the end.

$ stdbuf -o 0 ./producer | stdbuf -i 0 /usr/bin/iconv -f ASCII
Hello
World

All the output comes at the end.

What do I need to do to get the output of the first line immediately?

/usr/bin/iconv is from glibc, but I get the same behaviour from libiconv's
'iconv' program too.

In $ ./producer | /bin/cat
I get the first line immediately, but the coreutils documentation
<http://www.gnu.org/software/coreutils/manual/html_node/stdbuf-invocation.html>
makes me think that 'stdbuf' is meant for those programs that do not work like
'cat'.

Bruno
-- 
In memoriam José Olaya <http://es.wikipedia.org/wiki/José_Olaya>