Subject: Re: bug#8961: stdbuf has no effect on some programs

From: Pádraig Brady <[email-masked]>
Date: Thu, 30 Jun 2011 00:29:22 +0100

On 30/06/11 00:05, Pádraig Brady wrote:
> On 29/06/11 21:59, Bruno Haible wrote:
>> Hi,
>>
>> The glibc 'iconv' program buffers its input, and some people don't like this.
>> I thought that the 'stdbuf' program could remove the buffering, but it does not
>> work.
> 
> The following shows I think that iconv is bypassing stdio and buffering internally?
> 
> (echo; sleep 3; echo) | ltrace iconv -f ASCII
> 
> The stdbuf man page notes that:
> 
> NOTE: If COMMAND adjusts the buffering of its standard  streams  (`tee'
> does  for  e.g.) then that will override corresponding settings changed
> by `stdbuf'.  Also some filters (like `dd' and `cat'  etc.)  don't  use
> streams for I/O, and are thus unaffected by `stdbuf' settings.

In fact iconv seems to buffer for ever and so it not scalable,
as demonstrated with this consuming all of memory:

  yes | iconv

It would be OK to treat '\n' simply in all
unibyte encodings and utf8 for example,
but that would introduce an inconsistency I suppose.
Though maybe iconv could employ more scalable
buffering at least, for all encodings?

cheers,
Pádraig.