GNU bug report logs - #13135
Loss of data while copying

Previous Next

Full log

Message #15 received at 13135 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: 13135 <at> debbugs.gnu.org, xojoc <at> gmx.com
Subject: Re: bug#13135: Loss of data while copying
Date: Mon, 10 Dec 2012 15:21:29 +0000

On 12/10/2012 03:09 PM, Eric Blake wrote:
> tag 13135 notabug
> thanks
>
> On 12/10/2012 07:58 AM, Cojocaru Alexandru wrote:
>> bash$ yes $(for i in $(seq 1 100000); do echo -n a; done) | dd of=big-lines ibs=100001 count=10000
>> 9924+76 records in
>
> Thanks for the report.  Based on this output, short reads occurred.  dd
> transferred exactly 10000 reads as requested, but since some of those
> were short, it transferred less than 10000*100001 bytes.  This is
> expected (and the behavior is described in that way by POSIX); the
> solution you are looking for is to _also_ use the iconv=fullblock
> option, to force dd to re-read until it has a full input block rather
> than immediately transferring short input reads to output.
>
> As such, I'm closing this as not a bug.  Do feel free to add further
> comments to this thread, though, if you have more questions about why dd
> does this.
>
> Oh, and by the way, 'echo -n' is not portable.  You want to use
> printf(1) instead.
>

Yes, because a count was specified,
dd will operate in its default awkward but POSIX specified mode
of counting each read() call, even if it returned less than specified.
This is especially noticeable with pipes:

  yes blah | dd of=/dev/null ibs=100001 count=10000

To avoid that you can use iflag=fullblock (not iconv as Eric mentioned above):

  yes blah | dd of=/dev/null ibs=100001 count=10000 iflag=fullblock

cheers,
Pádraig.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.