Subject: Re: bug#7362: dd strangeness

From: Lucia Rotger <[email-masked]>
Date: Fri, 12 Nov 2010 15:53:19 +0100

On 10/11/2010 16:29, Pádraig Brady wrote:
> On 10/11/10 10:22, Lucia Rotger wrote:
>> I see this behavior in Solaris, Linux and BSD dd: if I send a big enough
>> file they all read it short at the end of the stream.
>>
>> This works as expected:
>>
>> # cat /dev/zero | dd bs=512 count=293601280 | wc
>>
>> I get the expected results, dd reads exactly 293601280 blocks and wc
>> sees 150323855360 characters, 140 GB
>>
>> Whereas substituting cat for zfs send doesn't:
>>
>> # zfs send<backup>  | dd bs=512 count=293601280 | wc
>
> different write sizes to the pipe mean
> in the later case, dd will get short reads.
> IMHO dd is doing the wrong/most surprising thing here,
> but it can't be changed for compatibility reasons.
> You can get coreutils dd to do what you want with:
>
> dd iflag=fullblock

Thanks for the explanation. Now, if I may abuse your time a bit more, 
how exactly is a block size determined when reading from a pipe? is it 
the size returned by each read()? And why does dd stop short at the last 
block?

Sorry about those problems in the man page being already fixed. I 
checked and I have dd version 6.1, these Ubuntu guys don't update very well.

I have somewhat solved it in a more portable way doing

# zfs send<backup> | dd obs=512 | dd bs=512 count=293601280 | wc

I guess as long as no writes from zfs send are smaller than 512 bytes. 
It works nicely now.

I suppose adding ibs=1 to the first dd would work with everything but it 
would probably take ages to finish.

Thanks