From unknown Tue Jun 24 05:10:17 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#16875 <16875@debbugs.gnu.org> To: bug#16875 <16875@debbugs.gnu.org> Subject: Status: python, comint-mode: Large output makes Emacs freeze Reply-To: bug#16875 <16875@debbugs.gnu.org> Date: Tue, 24 Jun 2025 12:10:17 +0000 retitle 16875 python, comint-mode: Large output makes Emacs freeze reassign 16875 emacs submitter 16875 Andreas R=C3=B6hler severity 16875 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 25 04:17:54 2014 Received: (at submit) by debbugs.gnu.org; 25 Feb 2014 09:17:54 +0000 Received: from localhost ([127.0.0.1]:38587 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1WIE9C-0004KE-5B for submit@debbugs.gnu.org; Tue, 25 Feb 2014 04:17:54 -0500 Received: from eggs.gnu.org ([208.118.235.92]:41254) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1WIE99-0004Jx-PQ for submit@debbugs.gnu.org; Tue, 25 Feb 2014 04:17:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WIE8x-0005Ew-By for submit@debbugs.gnu.org; Tue, 25 Feb 2014 04:17:46 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:32938) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WIE8x-0005Es-8r for submit@debbugs.gnu.org; Tue, 25 Feb 2014 04:17:39 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42895) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WIE8r-0003ky-8r for bug-gnu-emacs@gnu.org; Tue, 25 Feb 2014 04:17:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WIE8l-00059Q-AE for bug-gnu-emacs@gnu.org; Tue, 25 Feb 2014 04:17:33 -0500 Received: from moutng.kundenserver.de ([212.227.17.13]:59016) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WIE8l-000597-1b for bug-gnu-emacs@gnu.org; Tue, 25 Feb 2014 04:17:27 -0500 Received: from purzel.sitgens (brln-4db91a31.pool.mediaWays.net [77.185.26.49]) by mrelayeu.kundenserver.de (node=mreue105) with ESMTP (Nemesis) id 0M9GsI-1WOIPN47LS-00Cgys; Tue, 25 Feb 2014 10:17:24 +0100 Message-ID: <530C60A7.1030009@easy-emacs.de> Date: Tue, 25 Feb 2014 10:21:43 +0100 From: =?ISO-8859-15?Q?Andreas_R=F6hler?= User-Agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: bug-gnu-emacs@gnu.org Subject: python, comint-mode: Large output makes Emacs freeze Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:3+kDleE1jbTaqG5JkebSRdxWwv6HozWNds7pSvgomlZ +CVAnUK17WSOaxSBe5RWTBeYIZ+1BtpDAyawZ81rdby4AE/dPS CN5JhsFnJzO7ye4p56lPjXBBprqxuTsMLonqp1X7WY85z/vNdy ilv3SAcFncfmYZrNOguUCj9fB+omRnQey4XuOWPrKUpvogaEOU wI1Ml2B4k6GsSy1Blgo874JrKDrPGV73j8h+Ph0FxotgOa19WW Mo3JjoqJOeTfHnqNbfPojDp1W2GDNSrxbDU+pauu7fF6eBE3Md 3pet6lRqnYMvqqlJUXztjYN/pI2qKECzEExDcRjbFN5TVOHAIa iA+cCRSRRhR11zu9ZCT/wh8aSi0sI0+1MPoFj+BI0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) May confirm what is reported here: http://stackoverflow.com/questions/20128425/can-i-stop-the-repl-locking-up-when-using-emacs-and-python Form sent is [[[False] * 200 for i in range(3)] for j in range(200)] Does only occur with plain Python-shell, not if IPython is running. IIUC that's because IPython return is passed through a kind of pretty-print, inserting newlines. Comints dealing with long lines seems the cause. With process-send-string it works. From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 24 21:19:15 2014 Received: (at 16875) by debbugs.gnu.org; 25 Jun 2014 01:19:15 +0000 Received: from localhost ([127.0.0.1]:59806 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Wzbrn-0004WZ-Fg for submit@debbugs.gnu.org; Tue, 24 Jun 2014 21:19:15 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:42215 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Wzbrl-0004WQ-8g for 16875@debbugs.gnu.org; Tue, 24 Jun 2014 21:19:13 -0400 Received: from 222-99-245-190.fibertel.com.ar ([190.245.99.222]:39070 helo=localhost) by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1Wzbrk-00061U-Dy for 16875@debbugs.gnu.org; Tue, 24 Jun 2014 21:19:12 -0400 From: fgallina@gnu.org (=?utf-8?Q?Fabi=C3=A1n?= Ezequiel Gallina) To: 16875@debbugs.gnu.org Subject: python, comint-mode: Large output makes Emacs freeze Date: Tue, 24 Jun 2014 22:19:08 -0300 Message-ID: <87bnth4whf.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 16875 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) This looks related to: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D13675 Some ways to attenuate irresponsiveness: 1. Disable python shell font-locking: (setq python-shell-enable-font-lock nil) 2. Use Python's pprint to print such expressions, this makes it behave pretty much like iPython: >>> import pprint >>> pprint.pprint([[[False] * 200 for i in range(3)] for j in range= (200)]) >From the python.el side there's not much we can do. Fabi=C3=A1n From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 25 01:28:03 2014 Received: (at submit) by debbugs.gnu.org; 25 Jun 2014 05:28:03 +0000 Received: from localhost ([127.0.0.1]:59860 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1WzfkY-0003Lh-IF for submit@debbugs.gnu.org; Wed, 25 Jun 2014 01:28:02 -0400 Received: from eggs.gnu.org ([208.118.235.92]:38247) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1WzfkV-0003LD-Hs for submit@debbugs.gnu.org; Wed, 25 Jun 2014 01:28:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WzfkJ-0002rA-L2 for submit@debbugs.gnu.org; Wed, 25 Jun 2014 01:27:54 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:55887) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzfkJ-0002r5-Ik for submit@debbugs.gnu.org; Wed, 25 Jun 2014 01:27:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39962) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzfkD-0002I0-JA for bug-gnu-emacs@gnu.org; Wed, 25 Jun 2014 01:27:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Wzfk7-0002q5-KO for bug-gnu-emacs@gnu.org; Wed, 25 Jun 2014 01:27:41 -0400 Received: from mout.kundenserver.de ([212.227.126.131]:59670) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wzfk7-0002px-B0 for bug-gnu-emacs@gnu.org; Wed, 25 Jun 2014 01:27:35 -0400 Received: from [192.168.178.32] (brln-4dba04fb.pool.mediaWays.net [77.186.4.251]) by mrelayeu.kundenserver.de (node=mreue005) with ESMTP (Nemesis) id 0LzFY5-1We26M2VQv-014XpV; Wed, 25 Jun 2014 07:27:31 +0200 Message-ID: <53AA5DA5.8060700@easy-emacs.de> Date: Wed, 25 Jun 2014 07:27:01 +0200 From: =?UTF-8?B?QW5kcmVhcyBSw7ZobGVy?= User-Agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Icedove/24.6.0 MIME-Version: 1.0 To: bug-gnu-emacs@gnu.org Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze References: <530C60A7.1030009@easy-emacs.de> <87bnth4whf.fsf@gnu.org> In-Reply-To: <87bnth4whf.fsf@gnu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Provags-ID: V02:K0:INcoeN/3cv0R33BgglAb/Iypuj0m12hsApjl4bDZ+7x VXiVZa2IojFEuff0dxNuuLah7E9FPb0WTa04Z5v70EMZgLOu9g iD9idclbM11NaLI4N1h3ehy7294WxWaDqUH5L+KfcMzwkcC7iv UXDZTTVPZQ8bZZ0H4mlvYUaKWhSucBfcHXLLiaGAg8xkFdRXjM MYdn9yDwhp+tUevLcLKGfhgDKd2rTyBSlxDCf+Ykmp+7jnWtwl FTkdPNJv3Kya3QbVIZ3SFk4/JpjIb/v9TPkXtB+fKFwkH1Z0kO eezEBziqjGnrPePdYaJVUoNK11cxl6Nrdo8jDBb08x8Z4k4438 3Wz2kWRTT3nCpJPdJxLLz/480CBqsCqYd85f4zxoE X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) On 25.06.2014 03:19, Fabián Ezequiel Gallina wrote: > > This looks related to: > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13675 > To a certain extend. There some more operations done by comint, which aren't needed when executing Python code from source but may slow down it. Thus solution in python-mode.el is to surpass comint-mode completely and use "start-process" and "process-send-string" See py-fast-process. > Some ways to attenuate irresponsiveness: > > 1. Disable python shell font-locking: > (setq python-shell-enable-font-lock nil) > > 2. Use Python's pprint to print such expressions, this makes it behave > pretty much like iPython: > > >>> import pprint > >>> pprint.pprint([[[False] * 200 for i in range(3)] for j in range(200)]) > Indeed, "pprint" should solve it already. Maybe make that the default? IMO looks better anyway. Andreas From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 25 10:26:36 2014 Received: (at 16875) by debbugs.gnu.org; 25 Jun 2014 14:26:37 +0000 Received: from localhost ([127.0.0.1]:60577 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Wzo9k-0005EJ-2j for submit@debbugs.gnu.org; Wed, 25 Jun 2014 10:26:36 -0400 Received: from relais.videotron.ca ([24.201.245.36]:30106) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Wzo9g-0005E0-Aq for 16875@debbugs.gnu.org; Wed, 25 Jun 2014 10:26:33 -0400 MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ceviche.home ([24.201.170.218]) by VL-VM-MR003.ip.videotron.ca (Oracle Communications Messaging Exchange Server 7u4-22.01 64bit (built Apr 21 2011)) with ESMTP id <0N7Q008JAAS6E680@VL-VM-MR003.ip.videotron.ca> for 16875@debbugs.gnu.org; Wed, 25 Jun 2014 10:26:30 -0400 (EDT) Received: by ceviche.home (Postfix, from userid 20848) id 0D5E066167; Wed, 25 Jun 2014 10:26:30 -0400 (EDT) From: Stefan Monnier To: Andreas =?windows-1252?Q?R=F6hler?= Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze Message-id: References: <530C60A7.1030009@easy-emacs.de> <87bnth4whf.fsf@gnu.org> <53AA5DA5.8060700@easy-emacs.de> Date: Wed, 25 Jun 2014 10:26:30 -0400 In-reply-to: <53AA5DA5.8060700@easy-emacs.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4.50 (gnu/linux) X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 16875 Cc: 16875@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) >> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13675 > To a certain extend. > There some more operations done by comint, which aren't needed when > executing Python code from source but may slow down it. I expect the main slowdown comes from font-lock. But indeed, it''d be good to try and profile it to see where the time is spent. Last time I tried to speed up M-x grep, I changed compile.el so that it only processes (font-lock and friends) the output up to the last \n. This way, when a very long line is received in 100 chunks, it doesn't get re-processed 100 times. Maybe comint.el could do the same (especially since it already assumes that "text between the last \n and EOB is a prompt"). > Thus solution in python-mode.el is to surpass comint-mode completely and use > "start-process" and "process-send-string" If the problem is really in "unneeded comint functionality" (or call it "bloat"), that's an option, but of course if we don't know where the performance problem comes, we may end up with the same problem anyway. > Indeed, "pprint" should solve it already. Maybe make that the default? > IMO looks better anyway. [ I don't use Python, so I don't have an opinion on that, but if it works, fine by me. ] Maybe comint.el could also be changed so that it "wraps" lines if they get past some arbitrary maximum length (like 10K chars, for example). It wouldn't be a great solution, but if the performance sucks really bad past 10K chars, wrapping the line might be a lesser evil. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 25 11:29:39 2014 Received: (at 16875) by debbugs.gnu.org; 25 Jun 2014 15:29:39 +0000 Received: from localhost ([127.0.0.1]:60610 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Wzp8g-0007SN-IX for submit@debbugs.gnu.org; Wed, 25 Jun 2014 11:29:39 -0400 Received: from mout.kundenserver.de ([212.227.126.187]:65196) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Wzp8Z-0007Rp-O3 for 16875@debbugs.gnu.org; Wed, 25 Jun 2014 11:29:32 -0400 Received: from [192.168.178.32] (brln-4dba04fb.pool.mediaWays.net [77.186.4.251]) by mrelayeu.kundenserver.de (node=mreue003) with ESMTP (Nemesis) id 0LjwWD-1WOYVl2Sk5-00boKd; Wed, 25 Jun 2014 17:29:21 +0200 Message-ID: <53AAEAB9.9070408@easy-emacs.de> Date: Wed, 25 Jun 2014 17:28:57 +0200 From: =?ISO-8859-1?Q?Andreas_R=F6hler?= User-Agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Icedove/24.6.0 MIME-Version: 1.0 To: Stefan Monnier Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze References: <530C60A7.1030009@easy-emacs.de> <87bnth4whf.fsf@gnu.org> <53AA5DA5.8060700@easy-emacs.de> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:0qY51wMiT3GEUr3yX1wQb38e/3HCF+MlLYJkslJX1zW FhOjxNfWycIjjS8JAKbkKtQ7BZscv8Ninmtp8RxUWuxglDPoOX 1WbAvBfHez05xpus7V/5ZTxa5SW35prmzasQxPB2Ej5wksOUjP nw7jycV9YQoc3WxrjyfR9zY1kLIhpMu5sqLctVqtuPpOrJI2W+ HrNU8Zb53y/WtiIj7kgPx68E95SyUwIosEc4wFU85S5LW52EZX 2rWnC7GRujW9zowwK1IZOEUCKgiTUZdrQe88XPVO7G6D6LejMF mh3xl3otLZg3E8nrm1Zb6fpeaHrd2ETiSjHAyuxOWGnV5pVWD3 3CxGWyUUY2Psm9EZ2L8k81/xUtQ9EmO3tfqbKImDl X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 16875 Cc: 16875@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) On 25.06.2014 16:26, Stefan Monnier wrote: [ ... ] > Maybe comint.el could also be changed so that it "wraps" lines if they > get past some arbitrary maximum length (like 10K chars, for example). > It wouldn't be a great solution, but if the performance sucks really bad > past 10K chars, wrapping the line might be a lesser evil. > > > Stefan > Sounds promising, thanks! As comint is built for human interaction, readability of output matters. Andreas From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 20 18:27:53 2014 Received: (at 16875) by debbugs.gnu.org; 20 Jul 2014 22:27:53 +0000 Received: from localhost ([127.0.0.1]:60679 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1X8zaC-00029e-Nq for submit@debbugs.gnu.org; Sun, 20 Jul 2014 18:27:53 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:56801 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1X8za9-00029U-Mx for 16875@debbugs.gnu.org; Sun, 20 Jul 2014 18:27:50 -0400 Received: from 222-99-245-190.fibertel.com.ar ([190.245.99.222]:54175 helo=localhost) by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1X8za8-00013V-Ci; Sun, 20 Jul 2014 18:27:48 -0400 From: fgallina@gnu.org (=?utf-8?Q?Fabi=C3=A1n?= Ezequiel Gallina) To: Stefan Monnier Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze References: <530C60A7.1030009@easy-emacs.de> <87bnth4whf.fsf@gnu.org> <53AA5DA5.8060700@easy-emacs.de> Date: Sun, 20 Jul 2014 19:27:46 -0300 In-Reply-To: (Stefan Monnier's message of "Wed, 25 Jun 2014 10:26:30 -0400") Message-ID: <87tx6bhdi5.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 16875 Cc: Andreas =?utf-8?Q?R=C3=B6hler?= , 16875@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Andreas R=C3=B6hler writes: >> Indeed, "pprint" should solve it already. Maybe make that the default? >> IMO looks better anyway. That's changing the Python shell default behavior and I that's out of the scope of python.el. Stefan Monnier writes: >>> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D13675 >> To a certain extend. >> There some more operations done by comint, which aren't needed when >> executing Python code from source but may slow down it. > > I expect the main slowdown comes from font-lock. But indeed, it''d be > good to try and profile it to see where the time is spent. > >From profiling, a lot of time spent on font-locking while another bunch is spent at `comint-postoutput-scroll-to-bottom' (default member of `comint-output-filter-functions'). After disabling both, the output is sent pretty quickly and Emacs, while it's a bit slower, it's still responsive. WRT the font-lock slowness I must say I'm not happy the way it's implemented, there should be a better way to just fontify the text after the current prompt. That should avoid any extreme fontification cases. > > If the problem is really in "unneeded comint functionality" (or call it > "bloat"), that's an option, but of course if we don't know where the > performance problem comes, we may end up with the same problem anyway. > Now about the `comint-postoutput-scroll-to-bottom', this alone can also make Emacs be extremely irresponsive (pretty much like with the font-locking enabled). The real culprit of it's slowness is `recenter' function which seems to get really slow with long lines. Here are the elp-results with font-lock disabled (with a smaller sample of output): comint-output-filter 103 29.795948749 = 0.2892810558 comint-postoutput-scroll-to-bottom 104 28.636891969 = 0.2753547304 recenter 104 28.635311547 = 0.2753395341 python-pdbtrack-comint-output-filter-function 104 0.990129131 = 0.0095204724 comint-watch-for-password-prompt 104 0.0924025480 = 0.0008884860 > Maybe comint.el could also be changed so that it "wraps" lines if they > get past some arbitrary maximum length (like 10K chars, for example). > It wouldn't be a great solution, but if the performance sucks really bad > past 10K chars, wrapping the line might be a lesser evil. > Disabling font-lock and avoiding the `recenter' calls works pretty well, so I rather focus on improving those than modifying comint to workaround this. As a final related note, to avoid unnecessary filters, I think the `inferior-python-mode' should set the `comint-output-filter-functions' to a minimum and point users to append custom desired filters using the `inferior-python-mode-hook'. Regards, Fabi=C3=A1n From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 23 02:21:35 2014 Received: (at 16875) by debbugs.gnu.org; 23 Jul 2014 06:21:35 +0000 Received: from localhost ([127.0.0.1]:35203 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1X9pvj-0000X8-3o for submit@debbugs.gnu.org; Wed, 23 Jul 2014 02:21:35 -0400 Received: from mout.kundenserver.de ([212.227.126.131]:53240) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1X9pvd-0000Wm-LQ for 16875@debbugs.gnu.org; Wed, 23 Jul 2014 02:21:32 -0400 Received: from [192.168.178.32] (brln-4db9e851.pool.mediaWays.net [77.185.232.81]) by mrelayeu.kundenserver.de (node=mreue006) with ESMTP (Nemesis) id 0LkEtr-1WXlPr1Y4J-00cTWr; Wed, 23 Jul 2014 08:21:16 +0200 Message-ID: <53CF5440.7080503@easy-emacs.de> Date: Wed, 23 Jul 2014 08:20:48 +0200 From: =?UTF-8?B?QW5kcmVhcyBSw7ZobGVy?= User-Agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Icedove/24.6.0 MIME-Version: 1.0 To: =?UTF-8?B?RmFiacOhbiBFemVxdWllbCBHYWxsaW5h?= , Stefan Monnier Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze References: <530C60A7.1030009@easy-emacs.de> <87bnth4whf.fsf@gnu.org> <53AA5DA5.8060700@easy-emacs.de> <87tx6bhdi5.fsf@gnu.org> In-Reply-To: <87tx6bhdi5.fsf@gnu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Provags-ID: V02:K0:A6WsoADYhElO6t5zBk+wne5UWsoZZfN6TspG2KWZw+o xXPgrrDBm/Sk3UFJ1MDYJ5D3ZqbpN5Xw9Td3XbX4ml4q54dCLF SpLLyQPJ1Sroi7s1jHbdxyojp4qi8nuXbXVvlBJ1BLoUTUkWJu yoKFzswhEA2ER0o9khVV5LdRa9kExXggkGUfTeFYanknYLDzd9 aAH18Jp08aFME+taqw1NNkft8uhBEr0t6sQZeo77PScBkC/Ohh Cj8z4E4fPKUgYeLA3lbzU17bssvAfX/lO3BXLWg8eZCW6uhbLY jRsgXrmop7GmLDyIIHoWM0vqVlAuIGEBjJfGY+PRJIEHmKyIqf 8ClsM3EZ+sM/WSMATczWAbjuh0ioX6Lr8O2D1Q8Sl X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 16875 Cc: 16875@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.0 (/) On 21.07.2014 00:27, Fabián Ezequiel Gallina wrote: > [ ... ] > Disabling font-lock and avoiding the `recenter' calls works pretty well, > so I rather focus on improving those than modifying comint to workaround > this. > Given font-locking is an issue only when triggered while output arrives, maybe there is a way to disable it alongside with RET, but enabling it as soon no output is pending? Andreas From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 26 22:22:47 2014 Received: (at 16875-done) by debbugs.gnu.org; 27 Jul 2014 02:22:47 +0000 Received: from localhost ([127.0.0.1]:39142 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XBE6p-0004uw-4l for submit@debbugs.gnu.org; Sat, 26 Jul 2014 22:22:47 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:48312 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XBE6m-0004uo-K7 for 16875-done@debbugs.gnu.org; Sat, 26 Jul 2014 22:22:45 -0400 Received: from 222-99-245-190.fibertel.com.ar ([190.245.99.222]:60618 helo=localhost) by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1XBE6l-0007ME-Vo for 16875-done@debbugs.gnu.org; Sat, 26 Jul 2014 22:22:44 -0400 From: fgallina@gnu.org (=?utf-8?Q?Fabi=C3=A1n?= Ezequiel Gallina) To: 16875-done@debbugs.gnu.org Subject: python, comint-mode: Large output makes Emacs freeze Date: Sat, 26 Jul 2014 23:22:41 -0300 Message-ID: <871tt7levi.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 16875-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Fixed in revno 117582 in trunk Emacs is still slow, but at least responds to commands. I feel that everything that could be optimized on the comint side is pretty much there and that the slowness being experienced is in fact related to the long-lines slowness bug mentioned before[0]. Here's the current output for elp-results: comint-output-filter 206 3.542= 0630980 0.0171944810 python-pdbtrack-comint-output-filter-function 207 2.528= 3979559 0.0122144828 ansi-color-filter-apply 618 1.566= 2161150 0.0025343302 python-comint-postoutput-scroll-to-bottom 207 0.400= 739811 0.0019359411 comint-postoutput-scroll-to-bottom 1 0.365= 398304 0.365398304 recenter 1 0.365= 378877 0.365378877 python-shell-comint-end-of-output-p 412 0.055= 4427629 0.0001345698 python-shell-font-lock-comint-output-filter-function 207 0.032= 279105 0.0001559377 ansi-color-process-output 207 0.010= 7249640 5.181...e-05 ansi-color-apply-on-region 207 0.009= 8468030 4.756...e-05 comint-carriage-motion 206 0.001= 4299259 6.941...e-06 python-shell-font-lock-post-command-hook 3 0.000= 34127 0.0001137566 process-mark 828 0.000= 202256 2.442...e-07 process-kill-buffer-query-function 206 0.000= 1694799 8.227...e-07 comint-send-input 1 0.000= 149412 0.000149412 ansi-color--find-face 207 8.684= 8e-05 4.195...e-07 process-status 208 7.752= ...e-05 3.727...e-07 process-buffer 207 7.367= ...e-05 3.558...e-07 ansi-color-apply-overlay-face 207 6.695= ...e-05 3.234...e-07 python-util-text-properties-replace-name 1 4.839= 4e-05 4.8394e-05 comint-simple-send 1 3.887= 9e-05 3.8879e-05 python-shell-font-lock-cleanup-buffer 1 2.344= e-05 2.344e-05 comint-send-string 1 2.220= 1e-05 2.2201e-05 process-send-string 1 1.557= 1e-05 1.5571e-05 comint-add-to-input-history 1 1.282= 3e-05 1.2823e-05 comint-adjust-window-point 1 4.201= e-06 4.201e-06 comint-preinput-scroll-to-bottom 3 2.708= e-06 9.026...e-07 python-util-comint-last-prompt 6 2.490= ...e-06 4.150...e-07 comint-snapshot-last-prompt 2 2.462= e-06 1.231e-06 [0] http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D13675 Cheers, Fabi=C3=A1n From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 27 00:03:11 2014 Received: (at 16875) by debbugs.gnu.org; 27 Jul 2014 04:03:11 +0000 Received: from localhost ([127.0.0.1]:39154 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XBFfy-0007Nj-Q2 for submit@debbugs.gnu.org; Sun, 27 Jul 2014 00:03:11 -0400 Received: from mtaout24.012.net.il ([80.179.55.180]:34549) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XBFfv-0007N6-6U for 16875@debbugs.gnu.org; Sun, 27 Jul 2014 00:03:08 -0400 Received: from conversion-daemon.mtaout24.012.net.il by mtaout24.012.net.il (HyperSendmail v2007.08) id <0N9C00C00R14WL00@mtaout24.012.net.il> for 16875@debbugs.gnu.org; Sun, 27 Jul 2014 06:58:54 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout24.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0N9C00CHRR26FI00@mtaout24.012.net.il>; Sun, 27 Jul 2014 06:58:54 +0300 (IDT) Date: Sun, 27 Jul 2014 07:03:12 +0300 From: Eli Zaretskii Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze In-reply-to: <871tt7levi.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: fgallina@gnu.org (=?utf-8?Q?Fabi=C3=A1n?= Ezequiel Gallina) Message-id: <837g2zih33.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 8BIT References: <530C60A7.1030009@easy-emacs.de> <871tt7levi.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 16875 Cc: andreas.roehler@easy-emacs.de, 16875@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: fgallina@gnu.org (Fabián Ezequiel Gallina) > Date: Sat, 26 Jul 2014 23:22:41 -0300 > > > Emacs is still slow, but at least responds to commands. I feel that > everything that could be optimized on the comint side is pretty much > there and that the slowness being experienced is in fact related to the > long-lines slowness bug mentioned before[0]. That bug is about displaying long lines. Is the code that is slow involved in displaying such lines? If there's no display involved, then the discussion you point to is not relevant. Also, please tell how long are your lines, on the average. From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 27 06:20:44 2014 Received: (at 16875) by debbugs.gnu.org; 27 Jul 2014 10:20:44 +0000 Received: from localhost ([127.0.0.1]:39189 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XBLZM-0008MQ-0Y for submit@debbugs.gnu.org; Sun, 27 Jul 2014 06:20:44 -0400 Received: from smtp-pri-02-2.vtxnet.net ([212.147.62.156]:53398 helo=smtp-pri-02.vtxnet.net) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XBLZI-0008MH-LX for 16875@debbugs.gnu.org; Sun, 27 Jul 2014 06:20:41 -0400 Received: from fmsmemgm.homelinux.net (dyn.83-228-141-040.dsl.vtx.ch [83.228.141.40]) by smtp-pri-02.vtxnet.net (VTX Services SA) with ESMTP id A7F3037DCA9; Sun, 27 Jul 2014 12:20:39 +0200 (CEST) Received: by fmsmemgm.homelinux.net (Postfix, from userid 20848) id 7428EAE1BC; Sun, 27 Jul 2014 06:20:39 -0400 (EDT) From: Stefan Monnier To: 16875@debbugs.gnu.org Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze Message-ID: References: <530C60A7.1030009@easy-emacs.de> <871tt7levi.fsf@gnu.org> Date: Sun, 27 Jul 2014 06:20:39 -0400 In-Reply-To: <871tt7levi.fsf@gnu.org> (=?windows-1252?Q?=22Fabi=E1n?= Ezequiel Gallina"'s message of "Sat, 26 Jul 2014 23:22:41 -0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 16875 Cc: fgallina@gnu.org, andreas.roehler@easy-emacs.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Emacs is still slow, but at least responds to commands. I feel that > everything that could be optimized on the comint side is pretty much > there and that the slowness being experienced is in fact related to the > long-lines slowness bug mentioned before[0]. Have you tried to ensure that font-lock is only applied once a line is complete (i.e. after seeing the corresponding \n)? Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 28 17:35:17 2014 Received: (at 16875) by debbugs.gnu.org; 28 Jul 2014 21:35:17 +0000 Received: from localhost ([127.0.0.1]:41308 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XBsZg-0006ex-43 for submit@debbugs.gnu.org; Mon, 28 Jul 2014 17:35:16 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:57484 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XBsZd-0006eo-7L for 16875@debbugs.gnu.org; Mon, 28 Jul 2014 17:35:14 -0400 Received: from 222-99-245-190.fibertel.com.ar ([190.245.99.222]:47115 helo=localhost) by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1XBsZb-0000DI-Pd; Mon, 28 Jul 2014 17:35:12 -0400 From: fgallina@gnu.org (=?utf-8?Q?Fabi=C3=A1n?= Ezequiel Gallina) To: Eli Zaretskii Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze References: <530C60A7.1030009@easy-emacs.de> <871tt7levi.fsf@gnu.org> <837g2zih33.fsf@gnu.org> Date: Mon, 28 Jul 2014 18:35:09 -0300 In-Reply-To: <837g2zih33.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 27 Jul 2014 07:03:12 +0300") Message-ID: <878und2mlu.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -5.7 (-----) X-Debbugs-Envelope-To: 16875 Cc: andreas.roehler@easy-emacs.de, 16875@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.7 (-----) Eli Zaretskii writes: >> From: fgallina@gnu.org (Fabi=C3=A1n Ezequiel Gallina) >> Date: Sat, 26 Jul 2014 23:22:41 -0300 >>=20 >>=20 >> Emacs is still slow, but at least responds to commands. I feel that >> everything that could be optimized on the comint side is pretty much >> there and that the slowness being experienced is in fact related to the >> long-lines slowness bug mentioned before[0]. > > That bug is about displaying long lines. Is the code that is slow > involved in displaying such lines? If there's no display involved, > then the discussion you point to is not relevant. > > Also, please tell how long are your lines, on the average. > Yes, such code outputs a long line consisting of 841601 chars. One way to get that output in a file for testing is executing the following shell command: python -c 'print ([[[False] * 200 for i in range(3)] for j in range(200= )])' > /tmp/out.log Even opening that file in fundamental-mode makes Emacs slow. For instance calling `previous-line' from the end of buffer several times takes 1.1 seconds on average to complete. previous-line 30 33.263101639 1.1087700546 Next line is a bit more responsive though: next-line 30 14.028573178 0.4676191059 Operations seem to take longer and longer as point moves away from the beginning of buffer. Cheers, Fabi=C3=A1n From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 29 03:32:09 2014 Received: (at 16875) by debbugs.gnu.org; 29 Jul 2014 07:32:09 +0000 Received: from localhost ([127.0.0.1]:41580 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XC1tF-0006hA-6J for submit@debbugs.gnu.org; Tue, 29 Jul 2014 03:32:08 -0400 Received: from mtaout27.012.net.il ([80.179.55.183]:34217) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1XC1t9-0006gc-HI for 16875@debbugs.gnu.org; Tue, 29 Jul 2014 03:32:03 -0400 Received: from conversion-daemon.mtaout27.012.net.il by mtaout27.012.net.il (HyperSendmail v2007.08) id <0N9G00800PYIJP00@mtaout27.012.net.il> for 16875@debbugs.gnu.org; Tue, 29 Jul 2014 10:27:16 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout27.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0N9G00NAZQ1F9MA0@mtaout27.012.net.il>; Tue, 29 Jul 2014 10:27:16 +0300 (IDT) Date: Tue, 29 Jul 2014 10:32:10 +0300 From: Eli Zaretskii Subject: Re: bug#16875: python, comint-mode: Large output makes Emacs freeze In-reply-to: <878und2mlu.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: fgallina@gnu.org (=?utf-8?Q?Fabi=C3=A1n?= Ezequiel Gallina) Message-id: <834my0hb7p.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 8BIT References: <530C60A7.1030009@easy-emacs.de> <871tt7levi.fsf@gnu.org> <837g2zih33.fsf@gnu.org> <878und2mlu.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 16875 Cc: andreas.roehler@easy-emacs.de, 16875@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: fgallina@gnu.org (Fabián Ezequiel Gallina) > Cc: 16875@debbugs.gnu.org, andreas.roehler@easy-emacs.de > Date: Mon, 28 Jul 2014 18:35:09 -0300 > > Eli Zaretskii writes: > > >> From: fgallina@gnu.org (Fabián Ezequiel Gallina) > >> Date: Sat, 26 Jul 2014 23:22:41 -0300 > >> > >> > >> Emacs is still slow, but at least responds to commands. I feel that > >> everything that could be optimized on the comint side is pretty much > >> there and that the slowness being experienced is in fact related to the > >> long-lines slowness bug mentioned before[0]. > > > > That bug is about displaying long lines. Is the code that is slow > > involved in displaying such lines? If there's no display involved, > > then the discussion you point to is not relevant. > > > > Also, please tell how long are your lines, on the average. > > > > Yes, such code outputs a long line consisting of 841601 chars. In that case, yes, that's the "long line display slowness" problem. If you profile Emacs with 'profiler-start' and 'profiler-report', you should see that most of the time is spent in redisplay. > Operations seem to take longer and longer as point moves away from the > beginning of buffer. That's expected, although the time should level out after some number of lines. From unknown Tue Jun 24 05:10:17 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 26 Aug 2014 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator