From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 31 14:22:16 2013 Received: (at submit) by debbugs.gnu.org; 31 Dec 2013 19:22:16 +0000 Received: from localhost ([127.0.0.1]:55013 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vy4tK-0004Qy-JH for submit@debbugs.gnu.org; Tue, 31 Dec 2013 14:22:16 -0500 Received: from eggs.gnu.org ([208.118.235.92]:50022) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vxtto-000406-Ry for submit@debbugs.gnu.org; Tue, 31 Dec 2013 02:38:02 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Vxttj-0007u8-4b for submit@debbugs.gnu.org; Tue, 31 Dec 2013 02:38:00 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:53957) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vxttj-0007tw-1D for submit@debbugs.gnu.org; Tue, 31 Dec 2013 02:37:55 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51712) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vxttd-0001kW-Qj for bug-coreutils@gnu.org; Tue, 31 Dec 2013 02:37:54 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VxttX-0007qS-B2 for bug-coreutils@gnu.org; Tue, 31 Dec 2013 02:37:49 -0500 Received: from mutluit.com ([82.211.8.197]:35361) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VxttX-0007q6-2U for bug-coreutils@gnu.org; Tue, 31 Dec 2013 02:37:43 -0500 X-AuthUser: admin-mb@mutluit.com Received: from [192.168.64.22] ([77.21.29.203]:61429) by mutluit.com with [XMail 1.27 ESMTP Server] id for from ; Tue, 31 Dec 2013 08:37:38 +0100 Message-ID: <52C2743F.1050405@mutluit.com> Date: Tue, 31 Dec 2013 08:37:35 +0100 From: "U.Mutlu" Organization: mutluit.com User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:20.0) Gecko/20100101 Firefox/20.0 SeaMonkey/2.17.1 MIME-Version: 1.0 To: bug-coreutils@gnu.org Subject: cp is a cpu hog Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.1 (----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Tue, 31 Dec 2013 14:22:13 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.1 (----) Hello, there is a big bug with the cp command: it eats all cpu resources, the system is not responsive while copying a huge file, say a file of size 10 GB. There are other _desasterous_ consequences of this behaviour, if interessted I can tell you. OS: Debian, 64bit, latest, happens also on other distros like Ubuntu, see for example these links: http://ubuntuforums.org/showthread.php?t=1206936 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/365775 http://stackoverflow.com/questions/4290679/why-high-io-rate-operations-slow-everything-on-linux ... I suggest you just place a nanosleep(a_small_value) in the loop that does the copy operation. From my experience in simular situations this helps much. Best regards & Happy New Year U.Mutlu From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 31 16:24:21 2013 Received: (at 16309) by debbugs.gnu.org; 31 Dec 2013 21:24:21 +0000 Received: from localhost ([127.0.0.1]:55114 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vy6nU-0000qZ-Tp for submit@debbugs.gnu.org; Tue, 31 Dec 2013 16:24:21 -0500 Received: from mail4.vodafone.ie ([213.233.128.170]:50398) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vy6nQ-0000qK-57 for 16309@debbugs.gnu.org; Tue, 31 Dec 2013 16:24:18 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApMBAIc1w1JtTg2L/2dsb2JhbAANMRqDQ7lwgTCDGQEBAQQyAUYQCw0LCRYPCQMCAQIBRQYNAQcBAYgFCDapR5pTF48dB4Q2AQOZR4U8jlU Received: from unknown (HELO [192.168.1.79]) ([109.78.13.139]) by mail3.vodafone.ie with ESMTP; 31 Dec 2013 21:24:15 +0000 Message-ID: <52C335FD.6000402@draigBrady.com> Date: Tue, 31 Dec 2013 21:24:13 +0000 From: =?ISO-8859-15?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130110 Thunderbird/17.0.2 MIME-Version: 1.0 To: "U.Mutlu" Subject: Re: bug#16309: cp is a cpu hog References: <52C2743F.1050405@mutluit.com> In-Reply-To: <52C2743F.1050405@mutluit.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 16309 Cc: 16309@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) On 12/31/2013 07:37 AM, U.Mutlu wrote: > Hello, > there is a big bug with the cp command: it eats all cpu resources, > the system is not responsive while copying a huge file, > say a file of size 10 GB. > > There are other _desasterous_ consequences of this behaviour, > if interessted I can tell you. > > OS: Debian, 64bit, latest, happens also on other distros like Ubuntu, > see for example these links: > http://ubuntuforums.org/showthread.php?t=1206936 > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/365775 > > http://stackoverflow.com/questions/4290679/why-high-io-rate-operations-slow-everything-on-linux > ... > > I suggest you just place a nanosleep(a_small_value) > in the loop that does the copy operation. > From my experience in simular situations this helps much. In my experience if a process consumes all available CPU, it has little negative impact on system responsiveness. If however I/O resources are saturated, then the system can become unresponsive. So I'd first like to see confirmation of cp consuming all CPU (which isn't necessarily a bad thing anyway). Also I've noticed a couple of related issues in the Linux VM. 1. It doesn't handle large files very well at present, caching very large portions (given today's common large RAM sizes) before starting to write out. 2. The resultant writes block other system (I/O) operations for long periods of time. You could avoid the Linux VM by using direct I/O. For example you could see if you had the same issue with dd: dd bs=1M if=big.file of=/the/copy Then see if direct I/O improves things: dd bs=1M if=big.file of=/the/copy oflag=direct You could also check if streaming without polluting the cache helps (while taking advantage of the readahead cache) using: dd bs=1M if=big.file of=/the/copy iflag=nocache oflag=nocache thanks, Pádraig. From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 11 18:15:48 2018 Received: (at control) by debbugs.gnu.org; 11 Oct 2018 22:15:48 +0000 Received: from localhost ([127.0.0.1]:45654 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gAjFE-0002PD-5i for submit@debbugs.gnu.org; Thu, 11 Oct 2018 18:15:48 -0400 Received: from mail-pf1-f181.google.com ([209.85.210.181]:39391) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gAjFC-0002Ow-D5 for control@debbugs.gnu.org; Thu, 11 Oct 2018 18:15:46 -0400 Received: by mail-pf1-f181.google.com with SMTP id c25-v6so5102440pfe.6 for ; Thu, 11 Oct 2018 15:15:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:message-id:date:user-agent:mime-version:content-language :content-transfer-encoding; bh=aCMPf+7kyoxN+rBTekFZz3oWrRjDvMuc6vTuYT21Ayk=; b=jQSTj1oIOh/kU+0QgvVa9jAf+WwO4i158LaHlPg5KcDpTdIIMVsexf46xGQWHnYLM8 +zReHcVThWS5DKY2PtiaYQGoq0/qHoDTNNNQ7BTSHUqQwf5HTyh5a6RobQNQugQFtydJ Rt2cbxuXfVQHEozp/uVR62CVqhNPR6aaaYGnlGU20ZoIjt4uN5rnGh2BcOMDV5upRjvo 6dwGWMSXjJC8ddSe+KJfFeLw6KnDTzBrUeAiZj247qIw5PooKZAmz9jkgQfx2LmRWPJC 98xDJquUSuqjjWj758gJwUlPLaWqlEn+H43oIvxpqV7epCT+uuams8HD7MkjZy1uo3/8 nwXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:message-id:date:user-agent:mime-version :content-language:content-transfer-encoding; bh=aCMPf+7kyoxN+rBTekFZz3oWrRjDvMuc6vTuYT21Ayk=; b=XAcJmAEOey9bb2WzHT6+srgmRjRzR4cDJbuTkbhLNQxG/XSET2F5+c/jjtle2ISd0V YsQW4UwM1Z1NbdhHaf5HYWoZ0bZ6A7oO7bzRAWXn4000cNozruOCjhAqqpwg2RAkePfP CboI+2L8zn9621bO6EApDM3/7ZZVOJt7Ca3HO7PZ4EXgZXAorXCKS93G/O64ggjklOud 3XKuBNBnYabnsHdE00C3fOPEUO9WVB+MFDzrf7X37xHyNiFxeTz24yLJJxTZ5gEYlMWQ s0j/PGkwzaDWVWEWjs5iKC2GQvuq9P92h+5Y5lCllSnlj8/pChugcBAQtQhPz3RZnIbG h/8A== X-Gm-Message-State: ABuFfogJB04BZVDOwb0VfdxCQKygSQpusXbFwqFvlObU9/zrOBl+BIHK lIYX2UAUTb546doifRv5Qh/++IfM X-Google-Smtp-Source: ACcGV63/p1ZxnixeZlKyAhWdRojntCOIqhMBML/uGDJBsBc9xOlhDy1myTnczOQ7SPvTJ1b7zRMyWA== X-Received: by 2002:a62:3384:: with SMTP id z126-v6mr3277051pfz.85.1539296139769; Thu, 11 Oct 2018 15:15:39 -0700 (PDT) Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38]) by smtp.googlemail.com with ESMTPSA id t15-v6sm66768319pfj.7.2018.10.11.15.15.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Oct 2018 15:15:38 -0700 (PDT) To: control@debbugs.gnu.org From: Assaf Gordon Message-ID: Date: Thu, 11 Oct 2018 16:15:36 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 2.0 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: tags 15308 notabug close 15308 tags 15634 notabug close 15634 tags 16004 notabug severity 16004 wishlist close 16004 [...] Content analysis details: (2.0 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (assafgordon[at]gmail.com) -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [209.85.210.181 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.210.181 listed in wl.mailspike.net] 1.8 MISSING_SUBJECT Missing Subject: header 0.2 NO_SUBJECT Extra score for no subject 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) tags 15308 notabug close 15308 tags 15634 notabug close 15634 tags 16004 notabug severity 16004 wishlist close 16004 tags 16245 notabug close 16245 tags 16249 notabug close 16249 tags 16249 notabug close 16249 close 16309 tags 16468 notabug close 16468 tag 16530 notabug close 16530 tags 16718 notabug close 16718 tags 16742 +moreinfo close 16742 tags 16831 wontfix close 16831 tags 16838 wontfix close 16838 tags 16872 fixed close 16872 close 16945 close 17224 tags 17503 + notabug close 17503 close 17546 tags 17904 notabug close 17904 From unknown Sat Jun 21 03:28:00 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 09 Nov 2018 12:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator