GNU bug report logs - #16309
cp is a cpu hog

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 16309 in the body.
You can then email your comments to 16309 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "U.Mutlu" <um <at> mutluit.com>
To: bug-coreutils <at> gnu.org
Subject: cp is a cpu hog
Date: Tue, 31 Dec 2013 08:37:35 +0100

Hello,
there is a big bug with the cp command: it eats all cpu resources,
the system is not responsive while copying a huge file,
say a file of size 10 GB.

There are other _desasterous_ consequences of this behaviour,
if interessted I can tell you.

OS: Debian, 64bit, latest, happens also on other distros like Ubuntu,
see for example these links:
 http://ubuntuforums.org/showthread.php?t=1206936
 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/365775

http://stackoverflow.com/questions/4290679/why-high-io-rate-operations-slow-everything-on-linux
 ...

I suggest you just place a nanosleep(a_small_value)
in the loop that does the copy operation.
From my experience in simular situations this helps much.

Best regards & Happy New Year
U.Mutlu

Message #8 received at 16309 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: "U.Mutlu" <um <at> mutluit.com>
Cc: 16309 <at> debbugs.gnu.org
Subject: Re: bug#16309: cp is a cpu hog
Date: Tue, 31 Dec 2013 21:24:13 +0000

On 12/31/2013 07:37 AM, U.Mutlu wrote:
> Hello,
> there is a big bug with the cp command: it eats all cpu resources,
> the system is not responsive while copying a huge file,
> say a file of size 10 GB.
> 
> There are other _desasterous_ consequences of this behaviour,
> if interessted I can tell you.
> 
> OS: Debian, 64bit, latest, happens also on other distros like Ubuntu,
> see for example these links:
>  http://ubuntuforums.org/showthread.php?t=1206936
>  https://bugs.launchpad.net/ubuntu/+source/linux/+bug/365775
> 
> http://stackoverflow.com/questions/4290679/why-high-io-rate-operations-slow-everything-on-linux
>  ...
> 
> I suggest you just place a nanosleep(a_small_value)
> in the loop that does the copy operation.
> From my experience in simular situations this helps much.

In my experience if a process consumes all available CPU,
it has little negative impact on system responsiveness.
If however I/O resources are saturated, then the system
can become unresponsive.  So I'd first like to see confirmation
of cp consuming all CPU (which isn't necessarily a bad thing anyway).

Also I've noticed a couple of related issues in the Linux VM.
1. It doesn't handle large files very well at present, caching very large
portions (given today's common large RAM sizes) before starting to write out.
2. The resultant writes block other system (I/O) operations for long periods of time.

You could avoid the Linux VM by using direct I/O.
For example you could see if you had the same issue with dd:

  dd bs=1M if=big.file of=/the/copy

Then see if direct I/O improves things:

  dd bs=1M if=big.file of=/the/copy oflag=direct

You could also check if streaming without polluting the cache helps
(while taking advantage of the readahead cache) using:

  dd bs=1M if=big.file of=/the/copy iflag=nocache oflag=nocache

thanks,
Pádraig.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.