GNU bug report logs - #16309
cp is a cpu hog

Previous Next

Full log

View this message in rfc822 format

From: Pádraig Brady <P <at> draigBrady.com>
To: "U.Mutlu" <um <at> mutluit.com>
Cc: 16309 <at> debbugs.gnu.org
Subject: bug#16309: cp is a cpu hog
Date: Tue, 31 Dec 2013 21:24:13 +0000

On 12/31/2013 07:37 AM, U.Mutlu wrote:
> Hello,
> there is a big bug with the cp command: it eats all cpu resources,
> the system is not responsive while copying a huge file,
> say a file of size 10 GB.
> 
> There are other _desasterous_ consequences of this behaviour,
> if interessted I can tell you.
> 
> OS: Debian, 64bit, latest, happens also on other distros like Ubuntu,
> see for example these links:
>  http://ubuntuforums.org/showthread.php?t=1206936
>  https://bugs.launchpad.net/ubuntu/+source/linux/+bug/365775
> 
> http://stackoverflow.com/questions/4290679/why-high-io-rate-operations-slow-everything-on-linux
>  ...
> 
> I suggest you just place a nanosleep(a_small_value)
> in the loop that does the copy operation.
> From my experience in simular situations this helps much.

In my experience if a process consumes all available CPU,
it has little negative impact on system responsiveness.
If however I/O resources are saturated, then the system
can become unresponsive.  So I'd first like to see confirmation
of cp consuming all CPU (which isn't necessarily a bad thing anyway).

Also I've noticed a couple of related issues in the Linux VM.
1. It doesn't handle large files very well at present, caching very large
portions (given today's common large RAM sizes) before starting to write out.
2. The resultant writes block other system (I/O) operations for long periods of time.

You could avoid the Linux VM by using direct I/O.
For example you could see if you had the same issue with dd:

  dd bs=1M if=big.file of=/the/copy

Then see if direct I/O improves things:

  dd bs=1M if=big.file of=/the/copy oflag=direct

You could also check if streaming without polluting the cache helps
(while taking advantage of the readahead cache) using:

  dd bs=1M if=big.file of=/the/copy iflag=nocache oflag=nocache

thanks,
Pádraig.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.