From unknown Fri Aug 15 03:38:07 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#9734 <9734@debbugs.gnu.org> To: bug#9734 <9734@debbugs.gnu.org> Subject: Status: [solaris] `dd if=/dev/urandom of=file bs=1024k count=1' gets a file of 133120 bytes Reply-To: bug#9734 <9734@debbugs.gnu.org> Date: Fri, 15 Aug 2025 10:38:07 +0000 retitle 9734 [solaris] `dd if=3D/dev/urandom of=3Dfile bs=3D1024k count=3D1= ' gets a file of 133120 bytes reassign 9734 coreutils submitter 9734 "Clark J. Wang" severity 9734 normal tag 9734 notabug thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 12 08:33:11 2011 Received: (at submit) by debbugs.gnu.org; 12 Oct 2011 12:33:11 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RDxzi-0007xO-Ld for submit@debbugs.gnu.org; Wed, 12 Oct 2011 08:33:11 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RDu5T-0001ES-Nx for submit@debbugs.gnu.org; Wed, 12 Oct 2011 04:22:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RDu51-0005iA-SP for submit@debbugs.gnu.org; Wed, 12 Oct 2011 04:22:24 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, HTML_MESSAGE,RCVD_IN_DNSWL_LOW,T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:47660) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RDu51-0005i6-Qr for submit@debbugs.gnu.org; Wed, 12 Oct 2011 04:22:23 -0400 Received: from eggs.gnu.org ([140.186.70.92]:42467) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RDu4v-0004rT-Hc for bug-coreutils@gnu.org; Wed, 12 Oct 2011 04:22:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RDu4t-0005gX-V6 for bug-coreutils@gnu.org; Wed, 12 Oct 2011 04:22:17 -0400 Received: from mail-qw0-f41.google.com ([209.85.216.41]:46559) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RDu4t-0005gA-ST for bug-coreutils@gnu.org; Wed, 12 Oct 2011 04:22:15 -0400 Received: by qadb17 with SMTP id b17so408645qad.0 for ; Wed, 12 Oct 2011 01:22:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=endqVINfPV7p2TUu57aQf55N7cQ96C0aDqSpHGIybH4=; b=CMvwzxpo++7IUJpbXfMIICEaVNf3b/7a7sjDwa/PspoT9gDP4JoAuobMSANejEh41m M4x86MIfUi4IOi3OU9L44Z7rt3CmGHwpokqONA86lyy9KtFrENJ9FZO0PARQW26pGZe9 R2twktnM7QZxowQQ6sGigV5DaSx0ybKoM0YZ0= MIME-Version: 1.0 Received: by 10.224.174.68 with SMTP id s4mr13937698qaz.4.1318407732349; Wed, 12 Oct 2011 01:22:12 -0700 (PDT) Received: by 10.224.54.143 with HTTP; Wed, 12 Oct 2011 01:22:12 -0700 (PDT) Date: Wed, 12 Oct 2011 16:22:12 +0800 Message-ID: Subject: [solaris] `dd if=/dev/urandom of=file bs=1024k count=1' gets a file of 133120 bytes From: "Clark J. Wang" To: bug-coreutils@gnu.org Content-Type: multipart/alternative; boundary=485b397dd5b98e49fa04af15b8a5 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -5.9 (-----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Wed, 12 Oct 2011 08:33:09 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) --485b397dd5b98e49fa04af15b8a5 Content-Type: text/plain; charset=UTF-8 I'm not sure if it's a bug but it's not reasonable to me. On Solaris 11 (SunOS 5.11 snv_174, i86pc): $ uname -a SunOS sollab-242.cn.oracle.com 5.11 snv_174 i86pc i386 i86pc $ pkg list gnu-coreutils NAME (PUBLISHER) VERSION IFO file/gnu-coreutils 8.5-0.174.0.0.0.0.504 i-- $ /usr/gnu/bin/dd if=/dev/urandom of=file bs=1024k count=1 0+1 records in 0+1 records out 133120 bytes (133 kB) copied, 0.00290536 s, 45.8 MB/s $ ls -l file -rw-r--r-- 1 root root 133120 2011-10-12 16:12 file $ I'm new to Solaris but I've never seen this problem whe I use Linux so it really suprises me. I found this in the man page of /dev/urandom on Solaris: "The limitation per read for /dev/random is 1040 bytes. The limit for /dev/urandom is (128 * 1040 = 133120)." That seems to be the reason but I think dd should handle that and check the return value of the read() system call and make sure 1024k bytes have really been read from /dev/urandom. Any idea? Thanks. -Clark --485b397dd5b98e49fa04af15b8a5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I'm not sure if it's a bug but it's not reasonable to me. On So= laris 11 (SunOS 5.11 snv_174, i86pc):

$ uname -a
SunOS sollab-242.cn.oracle.com 5.11 snv_174= i86pc i386 i86pc
$ pkg list gnu-coreutils
NAME (PUBLISHER)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 VERSION=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 IFO
f= ile/gnu-coreutils=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 8.5-0.174.0.0.0.0.50= 4=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 i--
$ /usr/gnu/bin/dd if=3D/dev/urandom = of=3Dfile bs=3D1024k count=3D1
0+1 records in
0+1 records out
133120 bytes (133 kB) copied, 0.002905= 36 s, 45.8 MB/s
$ ls -l file
-rw-r--r-- 1 root root 133120 2011-10-12= 16:12 file
$

I'm new to Solaris but I've never seen this= problem whe I use Linux so it really suprises me.

I found this in the man page of /dev/urandom on Solaris: "The limi= tation per read for /dev/random is 1040 bytes. The limit for /dev/urandom i= s (128 * 1040 =3D 133120)." That seems to be the reason but I think dd= should handle that and check the return value of the read() system call an= d make sure 1024k bytes have really been read from /dev/urandom.

Any idea?

Thanks.

-Clark
--485b397dd5b98e49fa04af15b8a5-- From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 12 10:14:47 2011 Received: (at control) by debbugs.gnu.org; 12 Oct 2011 14:14:47 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RDza3-0002BF-5v for submit@debbugs.gnu.org; Wed, 12 Oct 2011 10:14:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RDzZz-0002Au-Cb; Wed, 12 Oct 2011 10:14:44 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p9CEEG7B002883 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 12 Oct 2011 10:14:16 -0400 Received: from [10.3.113.147] (ovpn-113-147.phx2.redhat.com [10.3.113.147]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p9CEEFUw000560; Wed, 12 Oct 2011 10:14:16 -0400 Message-ID: <4E95A0B7.3020905@redhat.com> Date: Wed, 12 Oct 2011 08:14:15 -0600 From: Eric Blake Organization: Red Hat User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110928 Fedora/3.1.15-1.fc14 Lightning/1.0b3pre Mnenhy/0.8.4 Thunderbird/3.1.15 MIME-Version: 1.0 To: "Clark J. Wang" Subject: Re: bug#9734: [solaris] `dd if=/dev/urandom of=file bs=1024k count=1' gets a file of 133120 bytes References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Spam-Score: -10.3 (----------) X-Debbugs-Envelope-To: control Cc: 9734-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -10.3 (----------) tag 9734 notabug thanks On 10/12/2011 02:22 AM, Clark J. Wang wrote: > I'm not sure if it's a bug but it's not reasonable to me. On Solaris 11 > (SunOS 5.11 snv_174, i86pc): > > $ uname -a > SunOS sollab-242.cn.oracle.com 5.11 snv_174 i86pc i386 i86pc > $ pkg list gnu-coreutils > NAME (PUBLISHER) VERSION > IFO > file/gnu-coreutils 8.5-0.174.0.0.0.0.504 > i-- > $ /usr/gnu/bin/dd if=/dev/urandom of=file bs=1024k count=1 > 0+1 records in Notice that this means you read a partial record - read() tried to read 1024k bytes, but the read ended short at only 133120 bytes. > 0+1 records out And because you didn't request dd to group multiple short reads before doing a full write, you got a single (short) record written. > I'm new to Solaris but I've never seen this problem whe I use Linux so it > really suprises me. Solaris and Linux kernels differ on when you will get short reads, and magic files like /dev/urandom are more likely to display the issue than regular files. That said, Linux also has the "problem" of short reads; it's especially noticeable when passing the output of dd to a pipe. You probably wanted to use this GNU extension: dd if=/dev/urandom of=file bs=1024k count=1 iconv=fullblock where the iconv flag requests that dd pile together multiple read()s until it has a full block, so that you no longer have a partial block output. > > I found this in the man page of /dev/urandom on Solaris: "The limitation per > read for /dev/random is 1040 bytes. The limit for /dev/urandom is (128 * > 1040 = 133120)." That seems to be the reason but I think dd should handle > that and check the return value of the read() system call and make sure > 1024k bytes have really been read from /dev/urandom. Only if the iconv=fullblock flag is specified, since it is a violation of POSIX to do more than one read() without an explicit flag requesting multiple reads per block. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 12 11:08:58 2011 Received: (at submit) by debbugs.gnu.org; 12 Oct 2011 15:08:58 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RE0QT-000504-Gd for submit@debbugs.gnu.org; Wed, 12 Oct 2011 11:08:58 -0400 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RE0QR-0004zl-Oc for submit@debbugs.gnu.org; Wed, 12 Oct 2011 11:08:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RE0Py-0000mw-IQ for submit@debbugs.gnu.org; Wed, 12 Oct 2011 11:08:27 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([140.186.70.17]:38894) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RE0Py-0000ms-Gz for submit@debbugs.gnu.org; Wed, 12 Oct 2011 11:08:26 -0400 Received: from eggs.gnu.org ([140.186.70.92]:42613) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RE0Px-0005IO-5u for bug-coreutils@gnu.org; Wed, 12 Oct 2011 11:08:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RE0Pw-0000lb-1r for bug-coreutils@gnu.org; Wed, 12 Oct 2011 11:08:25 -0400 Received: from bitwagon.com ([74.82.39.175]:57126) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1RE0Pv-0000hZ-QF for bug-coreutils@gnu.org; Wed, 12 Oct 2011 11:08:24 -0400 Received: from f14-64.local ([76.105.149.67]) by bitwagon.com for ; Wed, 12 Oct 2011 08:08:21 -0700 Message-ID: <4E95ADDA.2020003@bitwagon.com> Date: Wed, 12 Oct 2011 08:10:18 -0700 From: John Reiser Organization: - User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110928 Fedora/3.1.15-1.fc14 Thunderbird/3.1.15 MIME-Version: 1.0 To: bug-coreutils@gnu.org Subject: Re: bug#9734: [solaris] `dd if=/dev/urandom of=file bs=1024k count=1' gets a file of 133120 bytes References: In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 140.186.70.17 X-Spam-Score: -6.6 (------) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.6 (------) > I found this in the man page of /dev/urandom on Solaris: "The limitation per > read for /dev/random is 1040 bytes. The limit for /dev/urandom is (128 * > 1040 = 133120)." That seems to be the reason but I think dd should handle > that and check the return value of the read() system call and make sure > 1024k bytes have really been read from /dev/urandom. "bs=N" means "write(,,read(,,N))" [error checking has been elided for presentation here] so if the read is short then the write will be also. Check the manual page. Consider ibs=, obs=, iflag=fullblock. -- From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 12 11:43:18 2011 Received: (at 9734) by debbugs.gnu.org; 12 Oct 2011 15:43:18 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RE0xi-0005ot-68 for submit@debbugs.gnu.org; Wed, 12 Oct 2011 11:43:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RE0xf-0005ol-Jy for 9734@debbugs.gnu.org; Wed, 12 Oct 2011 11:43:16 -0400 Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p9CFgpMx017693 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for <9734@debbugs.gnu.org>; Wed, 12 Oct 2011 11:42:51 -0400 Received: from [10.3.113.77] (ovpn-113-77.phx2.redhat.com [10.3.113.77]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id p9CFgnYl014196 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 12 Oct 2011 11:42:50 -0400 Message-ID: <4E95B579.6050106@draigBrady.com> Date: Wed, 12 Oct 2011 16:42:49 +0100 From: =?UTF-8?B?UMOhZHJhaWcgQnJhZHk=?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20110816 Thunderbird/6.0 MIME-Version: 1.0 To: 9734@debbugs.gnu.org, eblake@redhat.com Subject: Re: bug#9734: [solaris] `dd if=/dev/urandom of=file bs=1024k count=1' gets a file of 133120 bytes References: <4E95A0B7.3020905@redhat.com> In-Reply-To: <4E95A0B7.3020905@redhat.com> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=UTF-8 X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by mx1.redhat.com id p9CFgpMx017693 X-Spam-Score: -10.6 (----------) X-Debbugs-Envelope-To: 9734 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -10.6 (----------) On 10/12/2011 03:14 PM, Eric Blake wrote: >=20 > On 10/12/2011 02:22 AM, Clark J. Wang wrote: >> I'm not sure if it's a bug but it's not reasonable to me. On Solaris 1= 1 >> (SunOS 5.11 snv_174, i86pc): >> >> $ uname -a >> SunOS sollab-242.cn.oracle.com 5.11 snv_174 i86pc i386 i86pc >> $ pkg list gnu-coreutils >> NAME (PUBLISHER) VERSION >> IFO >> file/gnu-coreutils 8.5-0.174.0.0.0.0.50= 4 >> i-- >> $ /usr/gnu/bin/dd if=3D/dev/urandom of=3Dfile bs=3D1024k count=3D1 >> 0+1 records in >=20 > Notice that this means you read a partial record - read() tried to read= 1024k bytes, but the read ended short at only 133120 bytes. >=20 >> 0+1 records out >=20 > And because you didn't request dd to group multiple short reads before = doing a full write, you got a single (short) record written. >=20 >> I'm new to Solaris but I've never seen this problem whe I use Linux so= it >> really suprises me. >=20 > Solaris and Linux kernels differ on when you will get short reads, and = magic files like /dev/urandom are more likely to display the issue than r= egular files. That said, Linux also has the "problem" of short reads; it= 's especially noticeable when passing the output of dd to a pipe. >=20 > You probably wanted to use this GNU extension: >=20 > dd if=3D/dev/urandom of=3Dfile bs=3D1024k count=3D1 iconv=3Dfullblock >=20 > where the iconv flag requests that dd pile together multiple read()s un= til it has a full block, so that you no longer have a partial block outpu= t. Right, but for the record it's iflag=3Dfullblock (available since coreuti= ls 7.0) This common issue is warned about by coreutils >=3D 8.11, where it will suggest using iflag=3Dfullblock Note the particular case where count=3D1 is not warned about, as with a single read, one doesn't know if we're just at EOF. Also it's probably a quite common idiom to, consume available data up to $bs bytes. cheers, P=C3=A1draig. From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 12 23:45:31 2011 Received: (at 9734-done) by debbugs.gnu.org; 13 Oct 2011 03:45:31 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RECEb-00017h-SR for submit@debbugs.gnu.org; Wed, 12 Oct 2011 23:45:30 -0400 Received: from mail-qw0-f44.google.com ([209.85.216.44]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1REBDi-00088f-DF for 9734-done@debbugs.gnu.org; Wed, 12 Oct 2011 22:40:31 -0400 Received: by qadb12 with SMTP id b12so1156181qad.3 for <9734-done@debbugs.gnu.org>; Wed, 12 Oct 2011 19:39:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hV309itF2Se9chOv3Roc64KFIrYlCAIKfmu38sAkRlg=; b=Cwl2f88zw4LlERKKznjOp7lPx1ftjdLOD4D/DBG7dxsUpEg3SHx3kkZ9II3TT8tGY1 Wb2yFUoJnN8i8MIwnelPIKGEFQNoblLxMHVPIgiFc5byJ8bTyBtqnyug77LEE4AayWx5 x2G0uT279IX4Gt9/Qi53OpMNVksvMbKF9VvU0= MIME-Version: 1.0 Received: by 10.224.174.68 with SMTP id s4mr1504403qaz.4.1318473598486; Wed, 12 Oct 2011 19:39:58 -0700 (PDT) Received: by 10.224.54.143 with HTTP; Wed, 12 Oct 2011 19:39:58 -0700 (PDT) In-Reply-To: <4E95A0B7.3020905@redhat.com> References: <4E95A0B7.3020905@redhat.com> Date: Thu, 13 Oct 2011 10:39:58 +0800 Message-ID: Subject: Re: bug#9734: [solaris] `dd if=/dev/urandom of=file bs=1024k count=1' gets a file of 133120 bytes From: "Clark J. Wang" To: Eric Blake Content-Type: multipart/alternative; boundary=485b397dd5b97bc87004af250eb1 X-Spam-Score: -4.8 (----) X-Debbugs-Envelope-To: 9734-done X-Mailman-Approved-At: Wed, 12 Oct 2011 23:45:28 -0400 Cc: 9734-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.4 (----) --485b397dd5b97bc87004af250eb1 Content-Type: text/plain; charset=UTF-8 On Wed, Oct 12, 2011 at 10:14 PM, Eric Blake wrote: > tag 9734 notabug > thanks > > > On 10/12/2011 02:22 AM, Clark J. Wang wrote: > >> I'm not sure if it's a bug but it's not reasonable to me. On Solaris 11 >> (SunOS 5.11 snv_174, i86pc): >> >> $ uname -a >> SunOS sollab-242.cn.oracle.com 5.11 snv_174 i86pc i386 i86pc >> $ pkg list gnu-coreutils >> NAME (PUBLISHER) VERSION >> IFO >> file/gnu-coreutils 8.5-0.174.0.0.0.0.504 >> i-- >> $ /usr/gnu/bin/dd if=/dev/urandom of=file bs=1024k count=1 >> 0+1 records in >> > > Notice that this means you read a partial record - read() tried to read > 1024k bytes, but the read ended short at only 133120 bytes. > > 0+1 records out >> > > And because you didn't request dd to group multiple short reads before > doing a full write, you got a single (short) record written. I've never noticed what "0+1 records in/out" really means since dd always does what I mean. Thanks a lot for all the clarification. > > I'm new to Solaris but I've never seen this problem whe I use Linux so it >> really suprises me. >> > > Solaris and Linux kernels differ on when you will get short reads, and > magic files like /dev/urandom are more likely to display the issue than > regular files. That said, Linux also has the "problem" of short reads; it's > especially noticeable when passing the output of dd to a pipe. > > You probably wanted to use this GNU extension: > > dd if=/dev/urandom of=file bs=1024k count=1 iconv=fullblock > > where the iconv flag requests that dd pile together multiple read()s until > it has a full block, so that you no longer have a partial block output. > > > >> I found this in the man page of /dev/urandom on Solaris: "The limitation >> per >> read for /dev/random is 1040 bytes. The limit for /dev/urandom is (128 * >> 1040 = 133120)." That seems to be the reason but I think dd should handle >> that and check the return value of the read() system call and make sure >> 1024k bytes have really been read from /dev/urandom. >> > > Only if the iconv=fullblock flag is specified, since it is a violation of > POSIX to do more than one read() without an explicit flag requesting > multiple reads per block. > > -- > Eric Blake eblake@redhat.com +1-801-349-2682 > Libvirt virtualization library http://libvirt.org > --485b397dd5b97bc87004af250eb1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, Oct 12, 2011 at 10:14 PM, Eric Blake <eblake@redhat.com> wrote:
tag 9734 notabug
thanks


On 10/12/2011 02:22 AM, Clark J. Wang wrote:
I'm not sure if it's a bug but it's not reasonable to me. On So= laris 11
(SunOS 5.11 snv_174, i86pc):

$ uname -a
SunOS sollab-= 242.cn.oracle.com 5.11 snv_174 i86pc i386 i86pc
$ pkg list gnu-coreutils
NAME (PUBLISHER) =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0VERSION
IFO
file/gnu-coreutils =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A08.5-0.174.0.0.0.0.50= 4
i--
$ /usr/gnu/bin/dd if=3D/dev/urandom of=3Dfile bs=3D1024k count=3D1
0+1 records in

Notice that this means you read a partial record - read() tried to read 102= 4k bytes, but the read ended short at only 133120 bytes.

0+1 records out

And because you didn't request dd to group multiple short reads before = doing a full write, you got a single (short) record written.

I've never noticed what "0+1 records in/out" really me= ans since dd always does what I mean. Thanks a lot for all the clarificatio= n.



I'm new to Solaris but I've never seen this problem whe I use Linux= so it
really suprises me.

Solaris and Linux kernels differ on when you will get short reads, and magi= c files like /dev/urandom are more likely to display the issue than regular= files. =C2=A0That said, Linux also has the "problem" of short re= ads; it's especially noticeable when passing the output of dd to a pipe= .

You probably wanted to use this GNU extension:

dd if=3D/dev/urandom of=3Dfile bs=3D1024k count=3D1 iconv=3Dfullblock

where the iconv flag requests that dd pile together multiple read()s until = it has a full block, so that you no longer have a partial block output.


I found this in the man page of /dev/urandom on Solaris: "The limitati= on per
read for /dev/random is 1040 bytes. The limit for /dev/urandom is (128 * 1040 =3D 133120)." That seems to be the reason but I think dd should h= andle
that and check the return value of the read() system call and make sure
1024k bytes have really been read from /dev/urandom.

Only if the iconv=3Dfullblock flag is specified, since it is a violation of= POSIX to do more than one read() without an explicit flag requesting multi= ple reads per block.

--
Eric Blake =C2=A0 eb= lake@redhat.com =C2=A0 =C2=A0+1-801-349-2682
Libvirt virtualization library http://libvirt.org

--485b397dd5b97bc87004af250eb1-- From unknown Fri Aug 15 03:38:07 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 10 Nov 2011 12:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator