From unknown Sat Jun 21 10:08:03 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#7715 <7715@debbugs.gnu.org> To: bug#7715 <7715@debbugs.gnu.org> Subject: Status: cp command on Linux Reply-To: bug#7715 <7715@debbugs.gnu.org> Date: Sat, 21 Jun 2025 17:08:03 +0000 retitle 7715 cp command on Linux=20 reassign 7715 coreutils submitter 7715 "Hemant Rumde" severity 7715 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 22 18:48:07 2010 Received: (at submit) by debbugs.gnu.org; 22 Dec 2010 23:48:07 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PVYPd-0007Rc-Q4 for submit@debbugs.gnu.org; Wed, 22 Dec 2010 18:48:06 -0500 Received: from eggs.gnu.org ([140.186.70.92]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PVVnn-0002vp-Ti for submit@debbugs.gnu.org; Wed, 22 Dec 2010 16:00:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PVVuA-0000eI-D2 for submit@debbugs.gnu.org; Wed, 22 Dec 2010 16:07:27 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,HTML_MESSAGE, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 Received: from lists.gnu.org ([199.232.76.165]:41217) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PVVuA-0000eD-AF for submit@debbugs.gnu.org; Wed, 22 Dec 2010 16:07:26 -0500 Received: from [140.186.70.92] (port=51212 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PVVu9-0004XH-E7 for bug-coreutils@gnu.org; Wed, 22 Dec 2010 16:07:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PVVu7-0000dh-Hw for bug-coreutils@gnu.org; Wed, 22 Dec 2010 16:07:25 -0500 Received: from bld9527051.us.ing.com ([204.86.34.37]:45214) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PVVu7-0000dQ-Bd for bug-coreutils@gnu.org; Wed, 22 Dec 2010 16:07:23 -0500 Received: from unknown (HELO PJAXAIPI1000.us.americas.intranet) ([10.172.17.171]) by BLD9527051.us.ing.com with ESMTP; 22 Dec 2010 14:05:45 -0700 X-IronPort-AV: E=Sophos;i="4.60,215,1291611600"; d="scan'208,217";a="476165" Received: from mspa336000.us.americas.intranet (HELO mspa336000.common.ecamericas) ([10.163.221.101]) by PJAXAIPI1000.us.americas.intranet with ESMTP; 22 Dec 2010 16:06:01 -0500 Received: from ironportqcy.corp.citistreet.org ([172.29.24.163]) by mspa336000.common.ecamericas with Microsoft SMTPSVC(6.0.3790.3959); Wed, 22 Dec 2010 15:07:18 -0600 Received: from hwpms008.corp.citistreet.org ([172.29.24.128]) by IronPortQcy.corp.citistreet.org with ESMTP; 22 Dec 2010 16:07:17 -0500 Received: from hwpms002.corp.citistreet.org ([172.29.144.219]) by hwpms008.corp.citistreet.org with Microsoft SMTPSVC(6.0.3790.4675); Wed, 22 Dec 2010 16:07:17 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CBA21C.36DEC7B2" Subject: cp command on Linux Date: Wed, 22 Dec 2010 16:07:20 -0500 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: cp command on Linux Thread-Index: AcuiHDi0J7cLf/2fS7Goad5DosBBZw== From: "Hemant Rumde" To: X-OriginalArrivalTime: 22 Dec 2010 21:07:17.0444 (UTC) FILETIME=[36EC7040:01CBA21C] X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Spam-Score: -6.0 (------) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Wed, 22 Dec 2010 18:48:04 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -6.0 (------) This is a multi-part message in MIME format. ------_=_NextPart_001_01CBA21C.36DEC7B2 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi=20 I do not log any bug for cp command. In our company, we copy huge Cobol files before processing data. This is to rollback our data files.=20 Suppose A1 is my huge file of 60GB and A1.bk is its backup file, before we process ( change ) data into A1. Then which of our method would be faster? 1. Method-1 ( A1.bk exists )=20 $ cp A1 A1.bk=20 2. Method-2 =20 $ rm -f A1.bk=20 $ cp A1 A1.bk=20 3. Method-3=20 $ cp --remove-destination A1 A1,bk=20 This operation is very simple. But our operators tell, in some cases cp takes longer time. How can we reduce copying time? Thanks=20 Hemant Rumde=20 ING US=20 Boston US --------------------------------------------------------- NOTICE: The information contained in this electronic mail message is confid= ential and intended only for certain recipients. If you are not an intende= d recipient, you are hereby notified that any disclosure, reproduction, dis= tribution or other use of this communication and any attachments is strictl= y prohibited. If you have received this communication in error, please not= ify the sender by reply transmission and delete the message without copying= or disclosing it. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ------_=_NextPart_001_01CBA21C.36DEC7B2 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable cp command on Linux

Hi

I do not log any bug for cp command. In ou= r company, we copy huge Cobol files before processing data. This is to roll= back our data files.

Suppose A1 is my huge file of 60GB and A1.= bk is its backup file, before we process ( change ) data into A1. Then whic= h of our method

would be faster?

1. Method-1 ( A1.bk exists )
    $ cp  A1 A1.bk

2. Method-2 
    $ rm -f A1.bk
    $ cp A1 A1.bk

3. Method-3
    $ cp --remove-destinat= ion A1 A1,bk

This operation is very simple. But our ope= rators tell, in some cases cp takes longer time. How can we reduce copying = time?

Thanks
Hemant Rumde
ING US
Boston US 

---------------------------------------------------------

NOTICE: The information contained in this electronic mail message is confid=
ential and intended only for certain recipients.  If you are not an intende=
d recipient, you are hereby notified that any disclosure, reproduction, dis=
tribution or other use of this communication and any attachments is strictl=
y prohibited.  If you have received this communication in error, please not=
ify the sender by reply transmission and delete the message without copying=
 or disclosing it.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
------_=_NextPart_001_01CBA21C.36DEC7B2-- From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 22 20:44:27 2010 Received: (at 7715-done) by debbugs.gnu.org; 23 Dec 2010 01:44:28 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PVaEE-0002yK-N0 for submit@debbugs.gnu.org; Wed, 22 Dec 2010 20:44:27 -0500 Received: from smtp.cs.ucla.edu ([131.179.128.62]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PVaEC-0002y6-GW for 7715-done@debbugs.gnu.org; Wed, 22 Dec 2010 20:44:25 -0500 Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp.cs.ucla.edu (Postfix) with ESMTP id BEFA539E80F8; Wed, 22 Dec 2010 17:50:59 -0800 (PST) X-Virus-Scanned: amavisd-new at smtp.cs.ucla.edu Received: from smtp.cs.ucla.edu ([127.0.0.1]) by localhost (smtp.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QlUHKl6lrCfZ; Wed, 22 Dec 2010 17:50:59 -0800 (PST) Received: from [192.168.1.10] (pool-71-189-109-235.lsanca.fios.verizon.net [71.189.109.235]) by smtp.cs.ucla.edu (Postfix) with ESMTPSA id 3569439E80F2; Wed, 22 Dec 2010 17:50:59 -0800 (PST) Message-ID: <4D12AAFC.2000203@cs.ucla.edu> Date: Wed, 22 Dec 2010 17:50:52 -0800 From: Paul Eggert Organization: UCLA Computer Science Department User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7 MIME-Version: 1.0 To: Hemant Rumde Subject: Re: bug#7715: cp command on Linux References: In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Score: -2.9 (--) X-Debbugs-Envelope-To: 7715-done Cc: 7715-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.9 (--) On 12/22/2010 01:07 PM, Hemant Rumde wrote: > Then which of our method would be faster? Method 3 is probably a tad faster than method 2, but whether 1 or 3 is faster depends on the underlying implementation. If you're seeing wildly different performance numbers I'd look there, not at 'cp' itself. From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 22 21:10:02 2010 Received: (at 7715-done) by debbugs.gnu.org; 23 Dec 2010 02:10:03 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PVad0-0003Wl-67 for submit@debbugs.gnu.org; Wed, 22 Dec 2010 21:10:02 -0500 Received: from joseki.proulx.com ([216.17.153.58]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PVacy-0003WP-E9 for 7715-done@debbugs.gnu.org; Wed, 22 Dec 2010 21:10:01 -0500 Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 5D130213C1; Wed, 22 Dec 2010 19:16:36 -0700 (MST) Received: by hysteria.proulx.com (Postfix, from userid 1000) id 4B6A12DD7D; Wed, 22 Dec 2010 19:16:36 -0700 (MST) Date: Wed, 22 Dec 2010 19:16:36 -0700 From: Bob Proulx To: Hemant Rumde Subject: Re: cp command performance Message-ID: <20101223021636.GA26341@hysteria.proulx.com> Mail-Followup-To: coreutils@gnu.org, Hemant Rumde References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-Spam-Score: -2.4 (--) X-Debbugs-Envelope-To: 7715-done Cc: 7715-done@debbugs.gnu.org, coreutils@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list Reply-To: coreutils@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.4 (--) Hemant Rumde wrote: > I do not log any bug for cp command. In that case I will close the bug report that you have opened. Let's have the discussion on the discussion mailing list coreutils@gnu.org. That is the more appropriate place. I have set the mail headers to direct discussion there but if your mailer doesn't comply please manually redirect it. > In our company, we copy huge Cobol files before processing > data. This is to rollback our data files. Suppose A1 is my huge > file of 60GB and A1.bk is its backup file, before we process ( > change ) data into A1. Then which of our method would be faster? > > 1. Method-1 ( A1.bk exists ) > $ cp A1 A1.bk > > 2. Method-2 > $ rm -f A1.bk > $ cp A1 A1.bk > > 3. Method-3 > $ cp --remove-destination A1 A1,bk All three of those should be virtually the same, especially the last two. But benchmarking it is always good. I created a 10G test file using dd and copied it once to set up the test and then performed the following operations on a ext3 filesystem. $ time cp testdata testdata.bak real 3m34.435s user 0m0.108s sys 0m30.950s $ time ( rm -f testdata.bak ; cp testdata testdata.bak ) real 3m27.941s user 0m0.092s sys 0m30.914s $ time cp --remove-destination testdata testdata.bak real 3m36.931s user 0m0.068s sys 0m30.862s As you can see the times for all three operations are with limits of being exactly the same. > This operation is very simple. But our operators tell, in some cases cp > takes longer time. How can we reduce copying time? I do not doubt that there will be differences in times consumed for just the raw command. With such a large file I think this will be dependent upon outside influences. Such as what filesystem you are using for the copy and how much ram you have available for buffer cache and whether extraneous sync and fsync calls are happening at the same time and so forth. I could send for-examples but I don't want to send you off on in the wrong direction and so will resist. Bob From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 23 09:28:59 2010 Received: (at 7715-done) by debbugs.gnu.org; 23 Dec 2010 14:29:00 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PVmA7-0003yS-A1 for submit@debbugs.gnu.org; Thu, 23 Dec 2010 09:28:59 -0500 Received: from bld9527050.us.ing.com ([204.86.34.36]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1PVmA3-0003yD-KR for 7715-done@debbugs.gnu.org; Thu, 23 Dec 2010 09:28:57 -0500 Received: from unknown (HELO PJAXAIPI1000.us.americas.intranet) ([10.172.17.171]) by bld9527050.us.ing.com with ESMTP; 23 Dec 2010 07:23:39 -0700 X-IronPort-AV: E=Sophos;i="4.60,219,1291611600"; d="scan'208";a="499846" Received: from mspa336000.us.americas.intranet (HELO mspa336000.common.ecamericas) ([10.163.221.101]) by PJAXAIPI1000.us.americas.intranet with ESMTP; 23 Dec 2010 09:34:07 -0500 Received: from ironportqcy.corp.citistreet.org ([172.29.24.163]) by mspa336000.common.ecamericas with Microsoft SMTPSVC(6.0.3790.3959); Thu, 23 Dec 2010 08:34:54 -0600 Received: from hwpms008.corp.citistreet.org ([172.29.24.128]) by IronPortQcy.corp.citistreet.org with ESMTP; 23 Dec 2010 09:34:53 -0500 Received: from hwpms002.corp.citistreet.org ([172.29.144.219]) by hwpms008.corp.citistreet.org with Microsoft SMTPSVC(6.0.3790.4675); Thu, 23 Dec 2010 09:34:53 -0500 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Subject: RE: cp command performance Date: Thu, 23 Dec 2010 09:34:52 -0500 Message-ID: In-Reply-To: <20101223021636.GA26341@hysteria.proulx.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: cp command performance Thread-Index: AcuiR29SjYHXeFGUSZaasgTgE4CpVwAZYtIw References: <20101223021636.GA26341@hysteria.proulx.com> From: "Hemant Rumde" To: X-OriginalArrivalTime: 23 Dec 2010 14:34:53.0626 (UTC) FILETIME=[9020A9A0:01CBA2AE] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Spam-Score: -8.3 (--------) X-Debbugs-Envelope-To: 7715-done Cc: 7715-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -9.1 (---------) Hi Bob Thanks for your quick response. I really appreciate your reply! We are using HP Storage. I guess, our infrastructure is ok.=20 Lets discuss on "cp A1 A1.bk". Correct me if I am wrong.=20 In this cp, OS needs to cache all A1.bk data blocks from storage to overwrite with A1 block. I guess, some time would be=20 utilized for this. =20 However, if A1.bk is new, then it would take free data=20 Blocks from super block. I guess, this should be faster. Apart from this, read/write hits can make some difference in performance. When you use dd, I guess most of your data would be in buffer-cache and read-hit rate would be more And very few calls would go to backend storage.=20 Does this make any sense? Thanks=20 Hemant =20 -----Original Message----- From: Bob Proulx [mailto:bob@proulx.com]=20 Sent: Wednesday, December 22, 2010 9:17 PM To: Hemant Rumde Cc: 7715-done@debbugs.gnu.org; coreutils@gnu.org Subject: Re: cp command performance Hemant Rumde wrote: > I do not log any bug for cp command. In that case I will close the bug report that you have opened. Let's have the discussion on the discussion mailing list coreutils@gnu.org. That is the more appropriate place. I have set the mail headers to direct discussion there but if your mailer doesn't comply please manually redirect it. > In our company, we copy huge Cobol files before processing data. This=20 > is to rollback our data files. Suppose A1 is my huge file of 60GB and > A1.bk is its backup file, before we process ( change ) data into A1.=20 > Then which of our method would be faster? >=20 > 1. Method-1 ( A1.bk exists )=20 > $ cp A1 A1.bk >=20 > 2. Method-2 =20 > $ rm -f A1.bk=20 > $ cp A1 A1.bk >=20 > 3. Method-3=20 > $ cp --remove-destination A1 A1,bk All three of those should be virtually the same, especially the last two. But benchmarking it is always good. I created a 10G test file using dd and copied it once to set up the test and then performed the following operations on a ext3 filesystem. $ time cp testdata testdata.bak real 3m34.435s user 0m0.108s sys 0m30.950s $ time ( rm -f testdata.bak ; cp testdata testdata.bak ) real 3m27.941s user 0m0.092s sys 0m30.914s $ time cp --remove-destination testdata testdata.bak real 3m36.931s user 0m0.068s sys 0m30.862s As you can see the times for all three operations are with limits of being exactly the same. > This operation is very simple. But our operators tell, in some cases=20 > cp takes longer time. How can we reduce copying time? I do not doubt that there will be differences in times consumed for just the raw command. With such a large file I think this will be dependent upon outside influences. Such as what filesystem you are using for the copy and how much ram you have available for buffer cache and whether extraneous sync and fsync calls are happening at the same time and so forth. I could send for-examples but I don't want to send you off on in the wrong direction and so will resist. Bob --------------------------------------------------------- NOTICE: The information contained in this electronic mail message is confid= ential and intended only for certain recipients. If you are not an intende= d recipient, you are hereby notified that any disclosure, reproduction, dis= tribution or other use of this communication and any attachments is strictl= y prohibited. If you have received this communication in error, please not= ify the sender by reply transmission and delete the message without copying= or disclosing it. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D From unknown Sat Jun 21 10:08:03 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 21 Jan 2011 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator