From unknown Mon Jun 23 04:09:15 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#22277 <22277@debbugs.gnu.org> To: bug#22277 <22277@debbugs.gnu.org> Subject: Status: 'dd' - stats are not what expected Reply-To: bug#22277 <22277@debbugs.gnu.org> Date: Mon, 23 Jun 2025 11:09:15 +0000 retitle 22277 'dd' - stats are not what expected reassign 22277 coreutils submitter 22277 Mike Fiedler severity 22277 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 30 21:21:02 2015 Received: (at submit) by debbugs.gnu.org; 31 Dec 2015 02:21:02 +0000 Received: from localhost ([127.0.0.1]:50784 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aESrN-0001f7-Fm for submit@debbugs.gnu.org; Wed, 30 Dec 2015 21:21:02 -0500 Received: from eggs.gnu.org ([208.118.235.92]:60081) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aERmC-0008G8-Sf for submit@debbugs.gnu.org; Wed, 30 Dec 2015 20:11:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aERm6-0004aS-Pm for submit@debbugs.gnu.org; Wed, 30 Dec 2015 20:11:31 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: * X-Spam-Status: No, score=1.9 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE,HTML_MIME_NO_HTML_TAG,MIME_HTML_ONLY,T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:41011) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aERm6-0004aN-Mj for submit@debbugs.gnu.org; Wed, 30 Dec 2015 20:11:30 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51026) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aERm5-0005Qd-Q7 for bug-coreutils@gnu.org; Wed, 30 Dec 2015 20:11:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aERm0-0004WW-Pu for bug-coreutils@gnu.org; Wed, 30 Dec 2015 20:11:29 -0500 Received: from forward19h.cmail.yandex.net ([87.250.230.161]:54193) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aERm0-0004Vh-FO for bug-coreutils@gnu.org; Wed, 30 Dec 2015 20:11:24 -0500 Received: from web20h.yandex.ru (web20h.yandex.ru [IPv6:2a02:6b8:0:f05::30]) by forward19h.cmail.yandex.net (Yandex) with ESMTP id 864972167B for ; Thu, 31 Dec 2015 04:11:20 +0300 (MSK) Received: from 127.0.0.1 (localhost [127.0.0.1]) by web20h.yandex.ru (Yandex) with ESMTP id 24A931322CAB; Thu, 31 Dec 2015 04:11:20 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.com; s=mail; t=1451524280; bh=ljZmrEG3lhurqbY5ZdIQ22mEIV5pY43/6qmv8Bmx9Bg=; h=From:To:Subject:Date; b=jNwkw+sUzFiFOy6DskuoaDyp+JzlHZsP9Jo0i88/HknrB6HpO+DQ4o/dlBlQfyuPg vwc5j0cnxPKov3cPKJeyazmT+f5n5uD2zRpjGEzLb6SH6YDNsQ9Ndw3xZ/ZpqZiOZZ uVVZeN+yCo57Jvc/3pNLe0PS8T4avR+fuZh2Btbo= Received: by web20h.yandex.ru with HTTP; Thu, 31 Dec 2015 04:11:19 +0300 From: Mike Fiedler To: bug-coreutils@gnu.org Subject: 'dd' - stats are not what expected MIME-Version: 1.0 Message-Id: <1626821451524279@web20h.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Wed, 30 Dec 2015 20:11:19 -0500 Content-Transfer-Encoding: base64 Content-Type: text/html; charset=koi8-r X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Wed, 30 Dec 2015 21:20:59 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) PGRpdj6aPC9kaXY+PGRpdj5IaSw8L2Rpdj48ZGl2Ppo8L2Rpdj48ZGl2PkkgcmFuIG9uZSBvZiBt eSBmYXZvcml0ZSB1dGlsaXRpZXMgJ2RkJyBhZ2FpbiB0aGlzIGV2ZW5pbmcsIHRoaXMgdGltZSB3 aXRoIGJzPTFHICggSUVDICkgLSBJIHVzdWFsbHkgZG8gMU0gYnV0IHRoaXMgdGltZSBJIGRlYWx0 IHdpdGggbW9yZSBkYXRhIHRvIGJlIGNvcGllZC4uLjwvZGl2PjxkaXY+SSBoYWQgdG8gY29weSBh Ym91dCAyMTUgR2lCIG9mIGRhdGEgZnJvbSBvbmUgdG8gYW5vdGhlciBkcml2ZSAoIG9mZnNldCAy MTUgR2lCIHdhcyBhYm91dCB0aGUgZW5kIG9mIHRoZSBsYXN0IHBhcnRpdGlvbiApLjwvZGl2Pjxk aXY+U28gSSBkaWQ6PC9kaXY+PGRpdj6aPC9kaXY+PGRpdj4kmiBkZCBpZj0vZGV2L3NkYiBvZj0v ZGV2L3NkYSBicz08c3Ryb25nPjFHPC9zdHJvbmc+IGNvdW50PTxzdHJvbmc+MjIyPC9zdHJvbmc+ PGJyIC8+MjIyKzAgcmVjb3JkcyBpbjxiciAvPjIyMiswIHJlY29yZHMgb3V0PGJyIC8+MjM4Mzcw Njg0OTI4IGJ5dGVzICg8c3Ryb25nPjIzOCBHQjwvc3Ryb25nPikgY29waWVkLCAxMjc1LjAzIHMs IDE4NyBNQi9zPGJyIC8+PGJyIC8+PC9kaXY+PGRpdj5XaGVuIGl0IGZpbmlzaGVkLCBJIGdvdCBh IGJpdCBjb25mdXNlZCwgYW5kIEkgYXNrZWQgbXlzZWxmIGEgcXVlc3Rpb24gaWYgdGhlIGRhdGEg SSByZXF1ZXN0ZWQgZGlkIHJlYWxseSBnZXQgY29waWVkLi6aIG9mIGNvdXJzZSBpdCBkaWQsIGJ1 dCBJIHdhcyBub3QgZXhwZWN0aW5nIDIzOCBHQiB0byBiZSBzaG93bi48L2Rpdj48ZGl2PlRvIG1h a2Ugc3VyZSBJIGNhbGN1bGF0ZWQgdGhlIDUxMiBieXRlIHNlY3RvciBlbmQgbnVtYmVyIG91dCBv ZiB0aGUgMjM4MzcwNjg0OTI4IGJ5dGVzICdkZCcgcmVzdWx0IGFuZCBjb21wYXJlZCBpdCB3aXRo IHRoZSBvdXRwdXQgb2YgZmRpc2sgc2hvd2luZyB0aGUgbGFzdCBzZWN0b3Igb2YgdGhlIGxhc3Qg cGFydGl0aW9uLi4uIEkgd2FzIGZpbmUuPC9kaXY+PGRpdj6aPC9kaXY+PGRpdj5JIHRoaW5rLCBh bmQgbWFueSBvdGhlcnMgaGF2ZSBhIHNhbWUgb3BpbmlvbiwgMWtCID0gMTAwMEIsIGV0Yywgc2hv dWxkIGJlIGJhbm5lZCBmcm9tIHVzZSBpbiB0aGUgSVQgd29ybGQsIGFuZCBiYW5uZWQgZnJvbSB1 c2UgYnkgdGhlIHNhbGVzIHBlb3BsZS48L2Rpdj48ZGl2Ppo8L2Rpdj48ZGl2PlRoZSBwb2ludCBp cywgYXMgeW91IHByb2JhYmx5IG5vdGljZWQsIGlmIGRkIGlzIHRvbGQgdG8gdXNlIElFQywgbGV0 J3Mgc3RpY2sgdG8gSUVDIGFuZCBub3QgZ2V0IHRoZSByZXN1bHRzIGluIHdoYXRldmVyIGFydGlm aWNpYWwgZGVjaW1hbCBjcmFwLi4uLjwvZGl2PjxkaXY+SXQgY2FuIG5vdCBvbmx5IGNvbmZ1c2Us IGJ1dCB1dGlsaXR5IGxpa2UgJ2RkJyBzaG91bGQgYmUgMTAwJSBzcGVjaWZpYyBhYm91dCBoYW5k bGluZyB0aGUgdW5pdHMsIGFuZCB0aGVyZSBzaG91bGQgYmUgbm90IGEgYml0IG9mIGRvdWJ0IHdo ZW4gaXQgc3BpdHMgb3V0IHRoZSByZXN1bHRzLjwvZGl2PjxkaXY+SWYgSSB3b3VsZCB1c2UgMUsg aW4gdGhpcyBjYXNlLCBJIHdvdWxkIG5vdCBub3RpY2UgdGhlIGRpZmZlcmVuY2UgLSBteSBicmFp biBpcyBzaW1wbHkgdG9vIHNpbXBsZSwgYW5kIHNtYWxsLCBidXQgMUcgc2hvdWxkIGF0IGxlYXN0 IHJlc3VsdCBpbiBkaXNwbGF5aW5nIDIyMiBHaUIgYW5kIGZvciBzdXJlIG5vdCBHQi48L2Rpdj48 ZGl2Ppo8L2Rpdj48ZGl2PlRoYW5rIHlvdSBHTlUgZm9yIGNvcmV1dGlscywgYW5kIGZvciBvcGVu aW5nIHRoZSB3aW5kb3dzIHRvIHRoZSB3b3JsZC48L2Rpdj48ZGl2Ppo8L2Rpdj48ZGl2PldoZW4g SSBkaWUsIEkgd2lsbCByZXN0IHdpdGggYSAiR05VL0xpbnV4IGZvciBkdW1taWVzIiBib29rIG9u IG15IGhlYXJ0LjwvZGl2PjxkaXY+mjwvZGl2PjxkaXY+mjwvZGl2PjxkaXY+U2luY2VyZWx5Ljwv ZGl2PjxkaXY+mjwvZGl2PjxkaXY+TWlrZSBGaWVkbGVyPC9kaXY+ From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 31 05:18:19 2015 Received: (at 22277) by debbugs.gnu.org; 31 Dec 2015 10:18:19 +0000 Received: from localhost ([127.0.0.1]:50958 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aEaJH-0000mX-1v for submit@debbugs.gnu.org; Thu, 31 Dec 2015 05:18:19 -0500 Received: from mail1.vodafone.ie ([213.233.128.43]:49541) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aEaJF-0000mI-SH for 22277@debbugs.gnu.org; Thu, 31 Dec 2015 05:18:18 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtEMALb/hFZtT1zR/2dsb2JhbABeKAECgj5RHjRtAYhYtjIihWoBAgKBFkwBAQEBAQGBC4Q1AQEEEhEPAVYLDQEKAgIFFgsCAgkDAgECAUUGAQwIAQEeiBEKpAuKK4VtixUBCyGBAYRZhXuHc4FJBZcGhUCFVoQtNIcYhT2OOmSCDgMcgV0+NAGFDwEBAQ Received: from unknown (HELO localhost.localdomain) ([109.79.92.209]) by mail1.vodafone.ie with ESMTP; 31 Dec 2015 10:18:11 +0000 Subject: Re: bug#22277: 'dd' - stats are not what expected To: Mike Fiedler , 22277@debbugs.gnu.org References: <1626821451524279@web20h.yandex.ru> From: =?UTF-8?Q?P=c3=a1draig_Brady?= X-Enigmail-Draft-Status: N1110 Message-ID: <568500E2.6060105@draigBrady.com> Date: Thu, 31 Dec 2015 10:18:10 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <1626821451524279@web20h.yandex.ru> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 22277 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) unarchive 17505 forcemerge 17505 22277 stop On 31/12/15 01:11, Mike Fiedler wrote: > > Hi, > > I ran one of my favorite utilities 'dd' again this evening, this time with bs=1G ( IEC ) - I usually do 1M but this time I dealt with more data to be copied... > I had to copy about 215 GiB of data from one to another drive ( offset 215 GiB was about the end of the last partition ). > So I did: > > $ dd if=/dev/sdb of=/dev/sda bs=*1G* count=*222* > 222+0 records in > 222+0 records out > 238370684928 bytes (*238 GB*) copied, 1275.03 s, 187 MB/s > > When it finished, I got a bit confused, and I asked myself a question if the data I requested did really get copied.. of course it did, but I was not expecting 238 GB to be shown. > To make sure I calculated the 512 byte sector end number out of the 238370684928 bytes 'dd' result and compared it with the output of fdisk showing the last sector of the last partition... I was fine. > > I think, and many others have a same opinion, 1kB = 1000B, etc, should be banned from use in the IT world, and banned from use by the sales people. > > The point is, as you probably noticed, if dd is told to use IEC, let's stick to IEC and not get the results in whatever artificial decimal crap.... > It can not only confuse, but utility like 'dd' should be 100% specific about handling the units, and there should be not a bit of doubt when it spits out the results. > If I would use 1K in this case, I would not notice the difference - my brain is simply too simple, and small, but 1G should at least result in displaying 222 GiB and for sure not GB. I have to agree, and this has come up a few times now. The number in brackets is not exact and informational for human consumption, so we should make an effort to be less confusing. There was a proposed patch at: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17505#37 which auto determines the appropriate base from the amount output, to output the number with the least amount of info loss. There were some issues noted with that, but IMHO they were lesser than the current issue. We will have to be careful to not corrupt output when switching with status=progress (due to possibly shorter status line). I'll have another look. thanks, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 31 12:35:38 2015 Received: (at control) by debbugs.gnu.org; 31 Dec 2015 17:35:38 +0000 Received: from localhost ([127.0.0.1]:51912 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aEh8U-00056K-2X for submit@debbugs.gnu.org; Thu, 31 Dec 2015 12:35:38 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:55591) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aEh8R-000566-Nd for control@debbugs.gnu.org; Thu, 31 Dec 2015 12:35:36 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id B8260160F50 for ; Thu, 31 Dec 2015 09:35:29 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 44T598xRnVkM for ; Thu, 31 Dec 2015 09:35:28 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 7F26F160F59 for ; Thu, 31 Dec 2015 09:35:28 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id M8d0CbwSiFzM for ; Thu, 31 Dec 2015 09:35:28 -0800 (PST) Received: from [192.168.1.9] (pool-100-32-155-148.lsanca.fios.verizon.net [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 64CCA160F50 for ; Thu, 31 Dec 2015 09:35:28 -0800 (PST) To: control@debbugs.gnu.org From: Paul Eggert Subject: merge 17505 and 22277 Organization: UCLA Computer Science Department Message-ID: <5685675D.8070309@cs.ucla.edu> Date: Thu, 31 Dec 2015 09:35:25 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) unarchive 17505 forcemerge 17505 22277 stop From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 01 19:00:45 2016 Received: (at 22277) by debbugs.gnu.org; 2 Jan 2016 00:00:45 +0000 Received: from localhost ([127.0.0.1]:33933 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aF9ci-0000RL-K6 for submit@debbugs.gnu.org; Fri, 01 Jan 2016 19:00:44 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:36777) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aF9ch-0000R6-4M for 22277@debbugs.gnu.org; Fri, 01 Jan 2016 19:00:43 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id CA728160F6B; Thu, 31 Dec 2015 14:14:50 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 9JurDEUQ2obV; Thu, 31 Dec 2015 14:14:49 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 3C97C160F72; Thu, 31 Dec 2015 14:14:49 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id SrsrC8t-5R7W; Thu, 31 Dec 2015 14:14:49 -0800 (PST) Received: from [192.168.1.9] (pool-100-32-155-148.lsanca.fios.verizon.net [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 104EE160F6B; Thu, 31 Dec 2015 14:14:49 -0800 (PST) Subject: Re: bug#22277: 'dd' - stats are not what expected To: =?UTF-8?Q?P=c3=a1draig_Brady?= , Mike Fiedler , 22277@debbugs.gnu.org References: <1626821451524279@web20h.yandex.ru> <568500E2.6060105@draigBrady.com> <56858878.7070106@cs.ucla.edu> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <5685A8D8.2010902@cs.ucla.edu> Date: Thu, 31 Dec 2015 14:14:48 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <56858878.7070106@cs.ucla.edu> Content-Type: multipart/mixed; boundary="------------000403090506090007060501" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 22277 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) This is a multi-part message in MIME format. --------------000403090506090007060501 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Paul Eggert wrote: > Ah. Is this due to glitches when the size grows from (say) "1023 KiB" to "1.0 > MiB", or is it something else? Assuming that's the problem, I installed the attached further patches to try to fix it. The gnulib update fixes an off-by-one bug I encountered in gnulib while testing this. --------------000403090506090007060501 Content-Type: text/x-diff; name="0001-dd-append-spaces-to-shorter-status-progress-line.patch" Content-Disposition: attachment; filename*0="0001-dd-append-spaces-to-shorter-status-progress-line.patch" Content-Transfer-Encoding: quoted-printable >From 2dab6cd3c2e18eb574b24e54fba86a33c80b6a27 Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Thu, 31 Dec 2015 14:09:05 -0800 Subject: [PATCH 1/2] dd: append spaces to shorter status=3Dprogress line MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit Problem noted by P=C3=A1draig Brady in: http://bugs.gnu.org/22277#8 Also, make the output a bit more precise while we're at it. * NEWS: Document this. * src/dd.c (previous_time): Remove, replacing with ... (next_time): New var. All uses changed. This avoids some rounding errors, and should be a bit faster. (newline_pending): Remove, replacing with ... (progress_len): New var. All uses changed. This lets us keep track of how many trailing spaces to append. (print_xfer_stats): Get the time first thing, so that it's closer to being correct. Count the bytes output, and append trailing spaces if needed. Add remarks to translators about translation lengths. --- NEWS | 2 ++ src/dd.c | 91 +++++++++++++++++++++++++++++++++-------------------------= ------ 2 files changed, 49 insertions(+), 44 deletions(-) diff --git a/NEWS b/NEWS index ae8d9e0..e7b73ca 100644 --- a/NEWS +++ b/NEWS @@ -32,6 +32,8 @@ GNU coreutils NEWS -= *- outline -*- dd now summarizes sizes in --human-readable format too, not just --si. E.g., "3441325000 bytes (3.4 GB, 3.2 GiB) copied". It omits the summa= ries if they would not provide useful information, e.g., "3 bytes copied". + Its status=3Dprogress output now uses the same format as ordinary stat= us, + perhaps with trailing spaces to erase previous progress output. =20 md5sum now supports the --ignore-missing option to allow verifying a subset of files given a larger list of checksums. diff --git a/src/dd.c b/src/dd.c index 4a24775..dc9f3d9 100644 --- a/src/dd.c +++ b/src/dd.c @@ -212,11 +212,11 @@ static uintmax_t w_bytes =3D 0; /* Time that dd started. */ static xtime_t start_time; =20 -/* Previous time for periodic progress. */ -static xtime_t previous_time; +/* Next time to report periodic progress. */ +static xtime_t next_time; =20 -/* Whether a '\n' is pending after writing progress. */ -static bool newline_pending; +/* If positive, the number of bytes output in the current progress line.= */ +static int progress_len; =20 /* True if input is seekable. */ static bool input_seekable; @@ -530,10 +530,10 @@ maybe_close_stdout (void) static void _GL_ATTRIBUTE_FORMAT ((__printf__, 3, 4)) nl_error (int status, int errnum, const char *fmt, ...) { - if (newline_pending) + if (0 < progress_len) { fputc ('\n', stderr); - newline_pending =3D false; + progress_len =3D 0; } =20 va_list ap; @@ -764,37 +764,23 @@ abbreviation_lacks_prefix (char const *message) static void print_xfer_stats (xtime_t progress_time) { - char hbuf[2][LONGEST_HUMAN_READABLE + 1]; + xtime_t now =3D progress_time ? progress_time : gethrxtime (); + char hbuf[3][LONGEST_HUMAN_READABLE + 1]; double delta_s; char const *bytes_per_second; char const *si =3D human_readable (w_bytes, hbuf[0], human_opts, 1, 1)= ; char const *iec =3D human_readable (w_bytes, hbuf[1], human_opts | human_base_1024, 1, 1); - if (progress_time) - fputc ('\r', stderr); =20 /* Use integer arithmetic to compute the transfer rate, since that makes it easy to use SI abbreviations. */ - - fprintf (stderr, - ngettext ("%"PRIuMAX" byte copied", - (abbreviation_lacks_prefix (si) - ? "%"PRIuMAX" bytes copied" - : abbreviation_lacks_prefix (iec) - ? "%"PRIuMAX" bytes (%s) copied" - : "%"PRIuMAX" bytes (%s, %s) copied"), - select_plural (w_bytes)), - w_bytes, si, iec); - - xtime_t now =3D progress_time ? progress_time : gethrxtime (); - if (start_time < now) { double XTIME_PRECISIONe0 =3D XTIME_PRECISION; uintmax_t delta_xtime =3D now; delta_xtime -=3D start_time; delta_s =3D delta_xtime / XTIME_PRECISIONe0; - bytes_per_second =3D human_readable (w_bytes, hbuf[0], human_opts, + bytes_per_second =3D human_readable (w_bytes, hbuf[2], human_opts, XTIME_PRECISION, delta_xtime); } else @@ -803,22 +789,41 @@ print_xfer_stats (xtime_t progress_time) bytes_per_second =3D _("Infinity B"); } =20 - /* TRANSLATORS: The two instances of "s" in this string are the SI - symbol "s" (meaning second), and should not be translated. - - This format used to be: + if (progress_time) + fputc ('\r', stderr); =20 - ngettext (", %g second, %s/s\n", ", %g seconds, %s/s\n", delta_s =3D= =3D 1) + /* TRANSLATORS: The instances of "s" in the following formats are + the SI symbol "s" (meaning second), and should not be translated. + The strings use SI symbols for better internationalization even + though they may be a bit more confusing in English. If one of + these formats A looks shorter on the screen than another format + B, then A's string length should be less than B's, and appending + strlen (B) - strlen (A) spaces to A should make it appear to be + at least as long as B. */ + + int stats_len + =3D (abbreviation_lacks_prefix (si) + ? fprintf (stderr, + ngettext ("%"PRIuMAX" byte copied, %g s, %s/s", + "%"PRIuMAX" bytes copied, %g s, %s/s", + select_plural (w_bytes)), + w_bytes, delta_s, bytes_per_second) + : abbreviation_lacks_prefix (iec) + ? fprintf (stderr, + _("%"PRIuMAX" bytes (%s) copied, %g s, %s/s"), + w_bytes, si, delta_s, bytes_per_second) + : fprintf (stderr, + _("%"PRIuMAX" bytes (%s, %s) copied, %g s, %s/s"), + w_bytes, si, iec, delta_s, bytes_per_second)); =20 - but that was incorrect for languages like Polish. To fix this - bug we now use SI symbols even though they're a bit more - confusing in English. */ - char const *time_fmt =3D _(", %g s, %s/s\n"); if (progress_time) - time_fmt =3D _(", %.6f s, %s/s"); /* OK with '\r' as increasing wid= th. */ - fprintf (stderr, time_fmt, delta_s, bytes_per_second); - - newline_pending =3D !!progress_time; + { + if (0 <=3D stats_len && stats_len < progress_len) + fprintf (stderr, "%*s", progress_len - stats_len, ""); + progress_len =3D stats_len; + } + else + fputc ('\n', stderr); } =20 static void @@ -827,10 +832,10 @@ print_stats (void) if (status_level =3D=3D STATUS_NONE) return; =20 - if (newline_pending) + if (0 < progress_len) { fputc ('\n', stderr); - newline_pending =3D false; + progress_len =3D 0; } =20 fprintf (stderr, @@ -2112,13 +2117,10 @@ dd_copy (void) if (status_level =3D=3D STATUS_PROGRESS) { xtime_t progress_time =3D gethrxtime (); - uintmax_t delta_xtime =3D progress_time; - delta_xtime -=3D previous_time; - double XTIME_PRECISIONe0 =3D XTIME_PRECISION; - if (delta_xtime / XTIME_PRECISIONe0 > 1) + if (next_time <=3D progress_time) { print_xfer_stats (progress_time); - previous_time =3D progress_time; + next_time +=3D XTIME_PRECISION; } } =20 @@ -2439,7 +2441,8 @@ main (int argc, char **argv) } } =20 - start_time =3D previous_time =3D gethrxtime (); + start_time =3D gethrxtime (); + next_time =3D start_time + XTIME_PRECISION; =20 exit_status =3D dd_copy (); =20 --=20 2.5.0 --------------000403090506090007060501 Content-Type: text/x-diff; name="0002-build-update-gnulib-submodule-to-latest.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0002-build-update-gnulib-submodule-to-latest.patch" >From 8213c959c222e54cbbae3d8d00f86b8fa879d95a Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Thu, 31 Dec 2015 14:13:23 -0800 Subject: [PATCH 2/2] build: update gnulib submodule to latest --- gnulib | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gnulib b/gnulib index 81d0a26..92bbc1b 160000 --- a/gnulib +++ b/gnulib @@ -1 +1 @@ -Subproject commit 81d0a26465c44b2041fcf9a6ec08a0cd43776825 +Subproject commit 92bbc1b583743b7e463cdfbcd048d9d52063b8c4 -- 2.5.0 --------------000403090506090007060501-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 01 19:05:46 2016 Received: (at 22277) by debbugs.gnu.org; 2 Jan 2016 00:05:46 +0000 Received: from localhost ([127.0.0.1]:33945 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aF9hZ-0000ZQ-Rr for submit@debbugs.gnu.org; Fri, 01 Jan 2016 19:05:46 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:36846) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aF9hW-0000ZB-EI for 22277@debbugs.gnu.org; Fri, 01 Jan 2016 19:05:44 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id EAFAB160D4E; Thu, 31 Dec 2015 11:56:42 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 3K_AJZtpGfX3; Thu, 31 Dec 2015 11:56:41 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 6187F160F59; Thu, 31 Dec 2015 11:56:41 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id pSlsFMHZSYIl; Thu, 31 Dec 2015 11:56:41 -0800 (PST) Received: from [192.168.1.9] (pool-100-32-155-148.lsanca.fios.verizon.net [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 3943A160D4E; Thu, 31 Dec 2015 11:56:41 -0800 (PST) Subject: Re: bug#22277: 'dd' - stats are not what expected To: =?UTF-8?Q?P=c3=a1draig_Brady?= , Mike Fiedler , 22277@debbugs.gnu.org References: <1626821451524279@web20h.yandex.ru> <568500E2.6060105@draigBrady.com> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <56858878.7070106@cs.ucla.edu> Date: Thu, 31 Dec 2015 11:56:40 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <568500E2.6060105@draigBrady.com> Content-Type: multipart/mixed; boundary="------------030809040408080008090400" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 22277 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) This is a multi-part message in MIME format. --------------030809040408080008090400 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable P=C3=A1draig Brady wrote: > There were some issues noted with that, > but IMHO they were lesser than the current issue. No matter which format we pick, fans of the other format will complain. I= nstead,=20 let's output the information in both formats. I did that with the attache= d patch. > We will have to be careful to not corrupt output > when switching with status=3Dprogress (due to possibly shorter status l= ine). I didn't know about this problem. Could you explain it? Ah. Is this due to glitches when the size grows from (say) "1023 KiB" to = "1.0=20 MiB", or is it something else? --------------030809040408080008090400 Content-Type: text/x-diff; name="0001-dd-summarize-in-human-readable-format-too.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-dd-summarize-in-human-readable-format-too.patch" >From 9d4e79f5c2d23544f989e73a8b37a3d8908f3b4a Mon Sep 17 00:00:00 2001 From: Paul Eggert Date: Thu, 31 Dec 2015 11:43:09 -0800 Subject: [PATCH] dd: summarize in --human-readable format too Problem reported by Linda Walsh in: http://bugs.gnu.org/17505 * NEWS: Document this. * doc/coreutils.texi (dd invocation): Use a simpler script. Adjust output example to match new behavior. * src/dd.c (human_size): Remove. All uses changed to use human_readable and ... (human_opts): ... this new constant. (abbreviation_lacks_prefix): New function. (print_xfer_stats): Use it. Output both --si and --human-readable summaries, but only if they have prefixes. * tests/dd/reblock.sh, tests/dd/stats.sh: Test new behavior. --- NEWS | 4 +++ doc/coreutils.texi | 22 ++++++++--------- src/dd.c | 70 ++++++++++++++++++++++++++++++++--------------------- tests/dd/reblock.sh | 4 +-- tests/dd/stats.sh | 6 ++--- 5 files changed, 63 insertions(+), 43 deletions(-) diff --git a/NEWS b/NEWS index 5080f0f..ae8d9e0 100644 --- a/NEWS +++ b/NEWS @@ -29,6 +29,10 @@ GNU coreutils NEWS -*- outline -*- ** New features + dd now summarizes sizes in --human-readable format too, not just --si. + E.g., "3441325000 bytes (3.4 GB, 3.2 GiB) copied". It omits the summaries + if they would not provide useful information, e.g., "3 bytes copied". + md5sum now supports the --ignore-missing option to allow verifying a subset of files given a larger list of checksums. This also affects sha1sum, sha224sum, sha256sum, sha384sum and sha512sum. diff --git a/doc/coreutils.texi b/doc/coreutils.texi index 64419c7..8eb2a3e 100644 --- a/doc/coreutils.texi +++ b/doc/coreutils.texi @@ -8835,23 +8835,23 @@ trap '' USR1 # which can be triggered by reception of signals. dd iflag=fullblock if=/dev/zero of=/dev/null count=5000000 bs=1000 & pid=$! -# Output stats every half second -until ! kill -s USR1 $pid 2>/dev/null; do sleep .5; done +# Output stats every second. +while kill -s USR1 $pid 2>/dev/null; do sleep 1; done @end example -The above script will output in the following format +The above script will output in the following format: @example -859+0 records in -859+0 records out -4295000000 bytes (4.3 GB) copied, 0.539934 s, 8.0 GB/s -1000+0 records in -1000+0 records out -5000000000 bytes (5.0 GB) copied, 0.630785 s, 7.9 GB/s +3441325+0 records in +3441325+0 records out +3441325000 bytes (3.4 GB, 3.2 GiB) copied, 1.00036 s, 3.4 GB/s +5000000+0 records in +5000000+0 records out +5000000000 bytes (5.0 GB, 4.7 GiB) copied, 1.44433 s, 3.5 GB/s @end example -Note also the @samp{status=progress} option which periodically updates -the last line of the transfer statistics above. +The @samp{status=progress} option periodically updates the last line +of the transfer statistics above. @vindex POSIXLY_CORRECT On systems lacking the @samp{INFO} signal @command{dd} responds to the diff --git a/src/dd.c b/src/dd.c index a5557a8..4a24775 100644 --- a/src/dd.c +++ b/src/dd.c @@ -673,15 +673,10 @@ Options are:\n\ exit (status); } -static char * -human_size (size_t n) -{ - static char hbuf[LONGEST_HUMAN_READABLE + 1]; - int human_opts = - (human_autoscale | human_round_to_nearest | human_base_1024 - | human_space_before_unit | human_SI | human_B); - return human_readable (n, hbuf, human_opts, 1, 1); -} +/* Common options to use when displaying sizes and rates. */ + +enum { human_opts = (human_autoscale | human_round_to_nearest + | human_space_before_unit | human_SI | human_B) }; /* Ensure input buffer IBUF is allocated. */ @@ -693,9 +688,15 @@ alloc_ibuf (void) char *real_buf = malloc (input_blocksize + INPUT_BLOCK_SLOP); if (!real_buf) - error (EXIT_FAILURE, 0, - _("memory exhausted by input buffer of size %"PRIuMAX" bytes (%s)"), - (uintmax_t) input_blocksize, human_size (input_blocksize)); + { + uintmax_t ibs = input_blocksize; + char hbuf[LONGEST_HUMAN_READABLE + 1]; + error (EXIT_FAILURE, 0, + _("memory exhausted by input buffer of size %"PRIuMAX" bytes (%s)"), + ibs, + human_readable (input_blocksize, hbuf, + human_opts | human_base_1024, 1, 1)); + } real_buf += SWAB_ALIGN_OFFSET; /* allow space for swab */ @@ -715,10 +716,16 @@ alloc_obuf (void) /* Page-align the output buffer, too. */ char *real_obuf = malloc (output_blocksize + OUTPUT_BLOCK_SLOP); if (!real_obuf) - error (EXIT_FAILURE, 0, - _("memory exhausted by output buffer of size %"PRIuMAX - " bytes (%s)"), - (uintmax_t) output_blocksize, human_size (output_blocksize)); + { + uintmax_t obs = output_blocksize; + char hbuf[LONGEST_HUMAN_READABLE + 1]; + error (EXIT_FAILURE, 0, + _("memory exhausted by output buffer of size %"PRIuMAX + " bytes (%s)"), + obs, + human_readable (output_blocksize, hbuf, + human_opts | human_base_1024, 1, 1)); + } obuf = ptr_align (real_obuf, page_size); } else @@ -746,17 +753,23 @@ multiple_bits_set (int i) return MULTIPLE_BITS_SET (i); } +static bool +abbreviation_lacks_prefix (char const *message) +{ + return message[strlen (message) - 2] == ' '; +} + /* Print transfer statistics. */ static void -print_xfer_stats (xtime_t progress_time) { - char hbuf[LONGEST_HUMAN_READABLE + 1]; - int human_opts = - (human_autoscale | human_round_to_nearest - | human_space_before_unit | human_SI | human_B); +print_xfer_stats (xtime_t progress_time) +{ + char hbuf[2][LONGEST_HUMAN_READABLE + 1]; double delta_s; char const *bytes_per_second; - + char const *si = human_readable (w_bytes, hbuf[0], human_opts, 1, 1); + char const *iec = human_readable (w_bytes, hbuf[1], + human_opts | human_base_1024, 1, 1); if (progress_time) fputc ('\r', stderr); @@ -764,11 +777,14 @@ print_xfer_stats (xtime_t progress_time) { since that makes it easy to use SI abbreviations. */ fprintf (stderr, - ngettext ("%"PRIuMAX" byte (%s) copied", - "%"PRIuMAX" bytes (%s) copied", + ngettext ("%"PRIuMAX" byte copied", + (abbreviation_lacks_prefix (si) + ? "%"PRIuMAX" bytes copied" + : abbreviation_lacks_prefix (iec) + ? "%"PRIuMAX" bytes (%s) copied" + : "%"PRIuMAX" bytes (%s, %s) copied"), select_plural (w_bytes)), - w_bytes, - human_readable (w_bytes, hbuf, human_opts, 1, 1)); + w_bytes, si, iec); xtime_t now = progress_time ? progress_time : gethrxtime (); @@ -778,7 +794,7 @@ print_xfer_stats (xtime_t progress_time) { uintmax_t delta_xtime = now; delta_xtime -= start_time; delta_s = delta_xtime / XTIME_PRECISIONe0; - bytes_per_second = human_readable (w_bytes, hbuf, human_opts, + bytes_per_second = human_readable (w_bytes, hbuf[0], human_opts, XTIME_PRECISION, delta_xtime); } else diff --git a/tests/dd/reblock.sh b/tests/dd/reblock.sh index cd9d942..d950b39 100755 --- a/tests/dd/reblock.sh +++ b/tests/dd/reblock.sh @@ -23,14 +23,14 @@ print_ver_ dd cat <<\EOF > exp-reblock || framework_failure_ 0+2 records in 1+1 records out -4 bytes (4 B) copied +4 bytes copied EOF # 2 short reads -> 2 partial writes cat <<\EOF > exp-no-reblock || framework_failure_ 0+2 records in 0+2 records out -4 bytes (4 B) copied +4 bytes copied EOF diff --git a/tests/dd/stats.sh b/tests/dd/stats.sh index da2c2d2..f2cbef9 100755 --- a/tests/dd/stats.sh +++ b/tests/dd/stats.sh @@ -60,14 +60,14 @@ for open in '' '1'; do wait # Ensure all data processed and at least last status written - grep '250000000 bytes .* copied' err || { cat err; fail=1; } + grep '250000000 bytes (250 MB, 238 MiB) copied' err || { cat err; fail=1; } done progress_output() { { sleep "$1"; echo 1; } | dd bs=1 status=progress of=/dev/null 2>err - # Progress output should be for "byte ... copied", while final is "bytes ..." - grep 'byte .* copied' err + # Progress output should be for "byte copied", while final is "bytes ..." + grep 'byte copied' err } retry_delay_ progress_output 1 4 || { cat err; fail=1; } -- 2.5.0 --------------030809040408080008090400-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 01 19:11:07 2016 Received: (at 22277) by debbugs.gnu.org; 2 Jan 2016 00:11:07 +0000 Received: from localhost ([127.0.0.1]:33956 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aF9mk-0000i5-Ut for submit@debbugs.gnu.org; Fri, 01 Jan 2016 19:11:07 -0500 Received: from mail1.vodafone.ie ([213.233.128.43]:40619) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aF9mi-0000hP-30 for 22277@debbugs.gnu.org; Fri, 01 Jan 2016 19:11:05 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AoQFACeihVZtT1zR/2dsb2JhbABeKAECgj5RUmcGiFmsZ4kcIoVqAQICgRVMAQEBAQEBgQuENAEBAQQSEWYLDQEDAwECAQkWCwICCQMCAQIBPQgGAQkDBgIBARYIiBEFBaRHiiuFbYsNAQEBAQEFAQEBAQEBARMJhVqFe4QmEQECPYJ8gUkFh16PKIJygWRqhVaEFxY0hxiFPYVYgRmHSWSCDgMcgV0+NAGDTYFCAQEB Received: from unknown (HELO localhost.localdomain) ([109.79.92.209]) by mail1.vodafone.ie with ESMTP; 31 Dec 2015 21:51:19 +0000 Subject: Re: bug#22277: 'dd' - stats are not what expected To: Mike Fiedler , 22277@debbugs.gnu.org References: <1626821451524279@web20h.yandex.ru> <568500E2.6060105@draigBrady.com> From: =?UTF-8?Q?P=c3=a1draig_Brady?= Message-ID: <5685A357.7000004@draigBrady.com> Date: Thu, 31 Dec 2015 21:51:19 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <568500E2.6060105@draigBrady.com> Content-Type: multipart/mixed; boundary="------------080601030600030605090507" X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 22277 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.5 (/) This is a multi-part message in MIME format. --------------080601030600030605090507 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit On 31/12/15 10:18, Pádraig Brady wrote: > unarchive 17505 > forcemerge 17505 22277 > stop > > On 31/12/15 01:11, Mike Fiedler wrote: >> >> Hi, >> >> I ran one of my favorite utilities 'dd' again this evening, this time with bs=1G ( IEC ) - I usually do 1M but this time I dealt with more data to be copied... >> I had to copy about 215 GiB of data from one to another drive ( offset 215 GiB was about the end of the last partition ). >> So I did: >> >> $ dd if=/dev/sdb of=/dev/sda bs=*1G* count=*222* >> 222+0 records in >> 222+0 records out >> 238370684928 bytes (*238 GB*) copied, 1275.03 s, 187 MB/s >> >> When it finished, I got a bit confused, and I asked myself a question if the data I requested did really get copied.. of course it did, but I was not expecting 238 GB to be shown. >> To make sure I calculated the 512 byte sector end number out of the 238370684928 bytes 'dd' result and compared it with the output of fdisk showing the last sector of the last partition... I was fine. >> >> I think, and many others have a same opinion, 1kB = 1000B, etc, should be banned from use in the IT world, and banned from use by the sales people. >> >> The point is, as you probably noticed, if dd is told to use IEC, let's stick to IEC and not get the results in whatever artificial decimal crap.... >> It can not only confuse, but utility like 'dd' should be 100% specific about handling the units, and there should be not a bit of doubt when it spits out the results. >> If I would use 1K in this case, I would not notice the difference - my brain is simply too simple, and small, but 1G should at least result in displaying 222 GiB and for sure not GB. > > I have to agree, and this has come up a few times now. > > The number in brackets is not exact and informational for human consumption, > so we should make an effort to be less confusing. > There was a proposed patch at: > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17505#37 > which auto determines the appropriate base from the amount output, > to output the number with the least amount of info loss. > > There were some issues noted with that, > but IMHO they were lesser than the current issue. > > We will have to be careful to not corrupt output > when switching with status=progress (due to possibly shorter status line). Resending as previous email seems dropped somewhere: The attached auto sets the units. For status=progress this is done based on output block size, for the final transfer stats, it's done based on the transferred byte count. cheers, Pádraig --------------080601030600030605090507 Content-Type: text/x-patch; name="dd-stats-units.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dd-stats-units.patch" >From 5885398ed2252af1ce433232e00933f15e0a318f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= Date: Fri, 16 May 2014 10:32:43 +0100 Subject: [PATCH] dd: output base 1024 transfer counts in IEC units * src/dd.c (human_size): Add "base" and "from/to block size" params so we can reuse this function for all output size conversions. (print_xfer_stats): Use IEC units for status=progress output when the obs is a multiple of 2 and not a multiple of 10. Also use IEC units for the final status summary when the number of output bytes is a power of 1024, and not a power of 1000. * tests/dd/stats.sh: Update to test status units. * NEWS: Mention the change in behavior. Fixes http://bugs.gnu.org/17505 --- NEWS | 2 ++ src/dd.c | 44 +++++++++++++++++++++++++++++--------------- tests/dd/stats.sh | 10 +++++++--- 3 files changed, 38 insertions(+), 18 deletions(-) diff --git a/NEWS b/NEWS index 5080f0f..f7d219e 100644 --- a/NEWS +++ b/NEWS @@ -45,6 +45,8 @@ GNU coreutils NEWS -*- outline -*- date --iso-8601 now uses +00:00 timezone format rather than +0000. The standard states to use this "extended" format throughout a timestamp. + dd now uses IEC units if appropriate for the human readable byte count status. + df now prefers sources towards the root of a device when eliding duplicate bind mounted entries. diff --git a/src/dd.c b/src/dd.c index a5557a8..870e938 100644 --- a/src/dd.c +++ b/src/dd.c @@ -674,13 +674,23 @@ Options are:\n\ } static char * -human_size (size_t n) +human_size (size_t n, size_t base, uintmax_t from_bs, uintmax_t to_bs) { static char hbuf[LONGEST_HUMAN_READABLE + 1]; + int human_opts = - (human_autoscale | human_round_to_nearest | human_base_1024 + (human_autoscale | human_round_to_nearest | human_space_before_unit | human_SI | human_B); - return human_readable (n, hbuf, human_opts, 1, 1); + + if (! base) + { + if ((n % 1000) && ! (n % 1024)) + human_opts |= human_base_1024; + } + else if (base == 1024) + human_opts |= human_base_1024; + + return human_readable (n, hbuf, human_opts, from_bs, to_bs); } /* Ensure input buffer IBUF is allocated. */ @@ -695,7 +705,8 @@ alloc_ibuf (void) if (!real_buf) error (EXIT_FAILURE, 0, _("memory exhausted by input buffer of size %"PRIuMAX" bytes (%s)"), - (uintmax_t) input_blocksize, human_size (input_blocksize)); + (uintmax_t) input_blocksize, + human_size (input_blocksize, 1024, 1, 1)); real_buf += SWAB_ALIGN_OFFSET; /* allow space for swab */ @@ -718,7 +729,8 @@ alloc_obuf (void) error (EXIT_FAILURE, 0, _("memory exhausted by output buffer of size %"PRIuMAX " bytes (%s)"), - (uintmax_t) output_blocksize, human_size (output_blocksize)); + (uintmax_t) output_blocksize, + human_size (output_blocksize, 1024, 1, 1)); obuf = ptr_align (real_obuf, page_size); } else @@ -749,16 +761,19 @@ multiple_bits_set (int i) /* Print transfer statistics. */ static void -print_xfer_stats (xtime_t progress_time) { - char hbuf[LONGEST_HUMAN_READABLE + 1]; - int human_opts = - (human_autoscale | human_round_to_nearest - | human_space_before_unit | human_SI | human_B); +print_xfer_stats (xtime_t progress_time) +{ double delta_s; char const *bytes_per_second; + size_t w_bytes_base = 0; /* auto for final stats. */ if (progress_time) - fputc ('\r', stderr); + { + fputc ('\r', stderr); + w_bytes_base = 1000; + if ((output_blocksize % 10) && ! (output_blocksize % 2)) + w_bytes_base = 1024; + } /* Use integer arithmetic to compute the transfer rate, since that makes it easy to use SI abbreviations. */ @@ -767,8 +782,7 @@ print_xfer_stats (xtime_t progress_time) { ngettext ("%"PRIuMAX" byte (%s) copied", "%"PRIuMAX" bytes (%s) copied", select_plural (w_bytes)), - w_bytes, - human_readable (w_bytes, hbuf, human_opts, 1, 1)); + w_bytes, human_size (w_bytes, w_bytes_base, 1, 1)); xtime_t now = progress_time ? progress_time : gethrxtime (); @@ -778,8 +792,8 @@ print_xfer_stats (xtime_t progress_time) { uintmax_t delta_xtime = now; delta_xtime -= start_time; delta_s = delta_xtime / XTIME_PRECISIONe0; - bytes_per_second = human_readable (w_bytes, hbuf, human_opts, - XTIME_PRECISION, delta_xtime); + bytes_per_second = human_size (w_bytes, 1000, XTIME_PRECISION, + delta_xtime); } else { diff --git a/tests/dd/stats.sh b/tests/dd/stats.sh index da2c2d2..7b7c856 100755 --- a/tests/dd/stats.sh +++ b/tests/dd/stats.sh @@ -63,11 +63,15 @@ for open in '' '1'; do grep '250000000 bytes .* copied' err || { cat err; fail=1; } done +# Check progress is output, and check that the correct +# IEC/SI units are used both for the progress and final status output. progress_output() { - { sleep "$1"; echo 1; } | dd bs=1 status=progress of=/dev/null 2>err - # Progress output should be for "byte ... copied", while final is "bytes ..." - grep 'byte .* copied' err + { sleep "$1"; dd status=none bs=1024 count=1 if=/dev/zero; } | + dd bs=1000 of=/dev/null status=progress 2>err + + grep -F '1000 bytes (1.0 kB) copied' err && # Progress line + grep -F '1024 bytes (1.0 KiB) copied' err # Final status line } retry_delay_ progress_output 1 4 || { cat err; fail=1; } -- 2.5.0 --------------080601030600030605090507-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 01 19:11:07 2016 Received: (at control) by debbugs.gnu.org; 2 Jan 2016 00:11:07 +0000 Received: from localhost ([127.0.0.1]:33958 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aF9ml-0000iD-CI for submit@debbugs.gnu.org; Fri, 01 Jan 2016 19:11:07 -0500 Received: from mail1.vodafone.ie ([213.233.128.43]:40619) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aF9mj-0000hP-RK for control@debbugs.gnu.org; Fri, 01 Jan 2016 19:11:06 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlEHADVthVZtT1zR/2dsb2JhbABeKAECgj5RIn6IeLYDhgc1akwBAQEBAQGBC0ESg3oHCipUDQIFFgsCCwMCAQIBOQYCAggNCAEBHogRnx2FRIorhW2LOYEBhFmNNAwuE4E2BZcGlw+Td2SBSgEBAQcCAYIzPoNqgVoCAQI Received: from unknown (HELO localhost.localdomain) ([109.79.92.209]) by mail1.vodafone.ie with ESMTP; 31 Dec 2015 18:02:32 +0000 To: GNU bug tracker automated control server From: =?UTF-8?Q?P=c3=a1draig_Brady?= Message-ID: <56856DB8.4030100@draigBrady.com> Date: Thu, 31 Dec 2015 18:02:32 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: unarchive 17505 forcemerge 17505 22277 stop [...] Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [213.233.128.43 listed in list.dnswl.org] 0.5 DATE_IN_PAST_24_48 Date: is 24 to 48 hours before Received: date 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: unarchive 17505 forcemerge 17505 22277 stop [...] Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [213.233.128.43 listed in list.dnswl.org] 0.5 DATE_IN_PAST_24_48 Date: is 24 to 48 hours before Received: date 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject unarchive 17505 forcemerge 17505 22277 stop From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 01 19:55:44 2016 Received: (at 22277) by debbugs.gnu.org; 2 Jan 2016 00:55:44 +0000 Received: from localhost ([127.0.0.1]:34022 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aFATw-0003S8-Ad for submit@debbugs.gnu.org; Fri, 01 Jan 2016 19:55:44 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:37678) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aFATv-0003Rw-0r for 22277@debbugs.gnu.org; Fri, 01 Jan 2016 19:55:43 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 24F69160F79; Thu, 31 Dec 2015 17:27:47 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id tXnC8S-m91VJ; Thu, 31 Dec 2015 17:27:46 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 52A2B160F76; Thu, 31 Dec 2015 17:27:46 -0800 (PST) X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 8X66hjZF5H-P; Thu, 31 Dec 2015 17:27:46 -0800 (PST) Received: from [192.168.1.9] (pool-100-32-155-148.lsanca.fios.verizon.net [100.32.155.148]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 33E8F160F73; Thu, 31 Dec 2015 17:27:46 -0800 (PST) Subject: Re: bug#22277: 'dd' - stats are not what expected To: =?UTF-8?Q?P=c3=a1draig_Brady?= References: <1626821451524279@web20h.yandex.ru> <568500E2.6060105@draigBrady.com> <56858878.7070106@cs.ucla.edu> <5685A8D8.2010902@cs.ucla.edu> <5685BA17.5020009@draigBrady.com> From: Paul Eggert Organization: UCLA Computer Science Department Message-ID: <5685D612.3000000@cs.ucla.edu> Date: Thu, 31 Dec 2015 17:27:46 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <5685BA17.5020009@draigBrady.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 22277 Cc: 22277@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) P=C3=A1draig Brady wrote: > I had proposed a patch to erase that extra > output with \e[K. Sorry, I didn't notice that proposal (maybe mailing list problems?). Is \e[K portable? If so, the code in my recently-installed patch could be= =20 simplified, since it wouldn't need to count columns. From unknown Mon Jun 23 04:09:15 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 30 Jan 2016 12:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator