From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 14:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 20154@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.142686159528031 (code B ref -1); Fri, 20 Mar 2015 14:27:01 +0000 Received: (at submit) by debbugs.gnu.org; 20 Mar 2015 14:26:35 +0000 Received: from localhost ([127.0.0.1]:59210 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYxsg-0007I1-NK for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:26:35 -0400 Received: from eggs.gnu.org ([208.118.235.92]:37766) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYxsf-0007Hp-2t for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:26:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YYxsY-0001nd-L4 for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:26:27 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_40,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:40474) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YYxsY-0001nZ-IR for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:26:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39475) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YYxsX-0006s0-3w for bug-gnu-emacs@gnu.org; Fri, 20 Mar 2015 10:26:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YYxsP-0001mR-Qi for bug-gnu-emacs@gnu.org; Fri, 20 Mar 2015 10:26:25 -0400 Received: from mail-wg0-x22c.google.com ([2a00:1450:400c:c00::22c]:33796) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YYxsP-0001m7-JT for bug-gnu-emacs@gnu.org; Fri, 20 Mar 2015 10:26:17 -0400 Received: by wggv3 with SMTP id v3so90849697wgg.1 for ; Fri, 20 Mar 2015 07:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:subject:date:message-id:mime-version:content-type; bh=rbArsYm15+hrChPK5LGAz/BhBJuSPCOyJ6hmmF2uhJI=; b=b1ZGtuG7zHI4PbKiXadhlFtGPIj+4GqdPFLoTP8iD/bkrLUAJmux0aUNcHwi0g01bm /z1ENHiDU8HW4M2D/KnpHVhKLj8Jcq3zwvTjK7L3dNZX0bCJ6nm+V+hm/KFRxD1hb5iN doUpFfFoevYfDNeAf3P7l0CzKi+3VA7tsG6x3XL10WZa+RkpL9FoeZa7k/QJGZtczVlc L968DC9nLa20OMdSDx/m4EplVEPRZbOQnCewQHtxL+bt/vOgWib+A9ZmzPntUqLqD3bd z8SAbHKoLFoemv+Bf0JztNqQnq9E8LlDi9lTAq/kFcbjconhBNrikZ6Bm3c6IGk9AZ/q XF+g== X-Received: by 10.180.202.228 with SMTP id kl4mr5681314wic.66.1426861576108; Fri, 20 Mar 2015 07:26:16 -0700 (PDT) Received: from axl ([82.102.93.54]) by mx.google.com with ESMTPSA id z13sm6570117wjr.44.2015.03.20.07.26.12 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 07:26:15 -0700 (PDT) From: Dmitry Gutov Date: Fri, 20 Mar 2015 16:26:07 +0200 Message-ID: <86twxf68zk.fsf@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 208.118.235.17 X-Spam-Score: -4.0 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) A 300Kb string takes 0.5s to encode on my machine. Example: (defvar s (apply #'concat (cl-loop for i from 1 to 30000 collect "0123456789\n"))) (length (json-encode-string s)) For a comparison, the built-in json module in my local Python installation takes only 2ms to do that for the same string. This is important for advanced code completion in general, because JSON is a common transport format, and sending the contents of the current buffer to the server is a common approach to avoid needlessly saving it (and running associated hooks, etc). And in this specific case, our JSON encoding speed is a bottleneck when working with ycmd, the editor-agnostic code completion daemon extracted from a popular Vim package: https://github.com/company-mode/company-mode/issues/325#issuecomment-83120928 I've tried to reimplement this function using `replace-regexp-in-string' or `re-search-forward' with a temp buffer, to minimize the number of concatenations and `json-encode-char' calls in the fast case (all characters are ASCII), but as long as characters that need to be encoded (such as newlines) still occur throughout the contents of the string, the speed improvement is nowhere near the acceptable level. Should it be written in C? In GNU Emacs 25.0.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2) of 2015-03-20 on axl Repository revision: 8142fc97af742e083fb83e4d0470da59b123a467 Windowing system distributor `The X.Org Foundation', version 11.0.11601901 System Description: Ubuntu 14.10 From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 14:35:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142686208728918 (code B ref 20154); Fri, 20 Mar 2015 14:35:01 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 14:34:47 +0000 Received: from localhost ([127.0.0.1]:59247 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYy0c-0007WM-Td for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:34:47 -0400 Received: from mtaout23.012.net.il ([80.179.55.175]:52946) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYy0Z-0007W7-JT for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 10:34:44 -0400 Received: from conversion-daemon.a-mtaout23.012.net.il by a-mtaout23.012.net.il (HyperSendmail v2007.08) id <0NLI00600LQGZ600@a-mtaout23.012.net.il> for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 16:34:36 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout23.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLI00676LTOY710@a-mtaout23.012.net.il>; Fri, 20 Mar 2015 16:34:36 +0200 (IST) Date: Fri, 20 Mar 2015 16:34:34 +0200 From: Eli Zaretskii In-reply-to: <86twxf68zk.fsf@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83384zwxdx.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Dmitry Gutov > Date: Fri, 20 Mar 2015 16:26:07 +0200 > > A 300Kb string takes 0.5s to encode on my machine. Example: > > (defvar s (apply #'concat (cl-loop for i from 1 to 30000 > collect "0123456789\n"))) > > (length (json-encode-string s)) > > For a comparison, the built-in json module in my local Python > installation takes only 2ms to do that for the same string. > > This is important for advanced code completion in general, because JSON > is a common transport format, and sending the contents of the current > buffer to the server is a common approach to avoid needlessly saving it > (and running associated hooks, etc). > > And in this specific case, our JSON encoding speed is a bottleneck when > working with ycmd, the editor-agnostic code completion daemon extracted > from a popular Vim package: > https://github.com/company-mode/company-mode/issues/325#issuecomment-83120928 > > I've tried to reimplement this function using `replace-regexp-in-string' > or `re-search-forward' with a temp buffer, to minimize the number of > concatenations and `json-encode-char' calls in the fast case (all > characters are ASCII), but as long as characters that need to be encoded > (such as newlines) still occur throughout the contents of the string, > the speed improvement is nowhere near the acceptable level. Should it be > written in C? I suggest to start with a detailed profile of the current implementation, because only then we will be able to talk intelligently about what part(s) need(s) to be sped up. Thanks. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 14:44:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142686262729697 (code B ref 20154); Fri, 20 Mar 2015 14:44:01 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 14:43:47 +0000 Received: from localhost ([127.0.0.1]:59255 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYy9L-0007iu-Cv for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:43:47 -0400 Received: from mail-wi0-f170.google.com ([209.85.212.170]:35609) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYy9J-0007ih-95 for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 10:43:45 -0400 Received: by wibdy8 with SMTP id dy8so18925591wib.0 for <20154@debbugs.gnu.org>; Fri, 20 Mar 2015 07:43:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=BgulrS5fQkbkaMtMI8doP0rGyIVpV3Gp9Kzx1dsb+Hw=; b=yCnet5HAZZTHB8GTfE9OQzzCJh2sWgcHUNmofW7lESHN5mIGyl6ugKo7SCwsmq+QyI Q0ApkbX9UGFl9bN1pScR6585xEgFZ+WQomnVKnlZA937aIR60Dej9vx+3ny9GPdLaXXg IFCYKr7ADz7pkDmx7KrwBZYDg3pnMd0qJYTssezixQfBcDN3xUAxE9Xivfhg/8wMYkr+ UsDVfNtztzrdWLGMO86j/MYoF1EE7AleRNOJ7oGRqMuVWjlVFL2RYxI5VfFNvLSIrkjA CbpU9K2FJsk46VmdbjZzfhvfb3DFjFnnkTSzMxNwiig9eZ7/hpn4+VfdxKAxZ/Oi7D6Q l2AA== X-Received: by 10.194.121.68 with SMTP id li4mr162638588wjb.138.1426862619595; Fri, 20 Mar 2015 07:43:39 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id hl8sm6638594wjb.38.2015.03.20.07.43.38 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 07:43:39 -0700 (PDT) Message-ID: <550C3218.4000903@yandex.ru> Date: Fri, 20 Mar 2015 16:43:36 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> In-Reply-To: <83384zwxdx.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/20/2015 04:34 PM, Eli Zaretskii wrote: > I suggest to start with a detailed profile of the current > implementation, Any suggestions there? By the way, I've included the example, so you can also profile it yourself. Here's the output of the built-in profiler, after using the current implementation: CPU: - command-execute 1147 79% - call-interactively 1147 79% - funcall-interactively 1082 74% - eval-last-sexp 1035 71% - elisp--eval-last-sexp 1035 71% - eval 1035 71% - length 1035 71% - json-encode-string 1035 71% - mapconcat 874 60% json-encode-char 571 39% + execute-extended-command 40 2% + previous-line 7 0% + byte-code 65 4% - ... 298 20% Automatic GC 298 20% Memory: - command-execute 255,362,537 99% - call-interactively 255,362,537 99% - funcall-interactively 255,349,159 99% - eval-last-sexp 248,605,484 97% - elisp--eval-last-sexp 248,605,484 97% - eval 217,011,432 84% - length 217,011,432 84% - json-encode-string 217,011,432 84% - mapconcat 93,689,099 36% json-encode-char 81,784,197 32% format 814 0% elisp--eval-last-sexp-print-value 11,954 0% + elisp--preceding-sexp 2,048 0% + execute-extended-command 6,743,643 2% + previous-line 32 0% + byte-code 13,378 0% + xselect-convert-to-string 176 0% ... 0 0% From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 15:05:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142686385031589 (code B ref 20154); Fri, 20 Mar 2015 15:05:01 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 15:04:10 +0000 Received: from localhost ([127.0.0.1]:59263 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYyT3-0008DR-Sm for submit@debbugs.gnu.org; Fri, 20 Mar 2015 11:04:10 -0400 Received: from mtaout23.012.net.il ([80.179.55.175]:54562) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYyT0-0008Cw-QF for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 11:04:08 -0400 Received: from conversion-daemon.a-mtaout23.012.net.il by a-mtaout23.012.net.il (HyperSendmail v2007.08) id <0NLI00700MVH1200@a-mtaout23.012.net.il> for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 17:03:59 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout23.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLI006VQN6NY740@a-mtaout23.012.net.il>; Fri, 20 Mar 2015 17:03:59 +0200 (IST) Date: Fri, 20 Mar 2015 17:03:57 +0200 From: Eli Zaretskii In-reply-to: <550C3218.4000903@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <831tkjww0y.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Fri, 20 Mar 2015 16:43:36 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > On 03/20/2015 04:34 PM, Eli Zaretskii wrote: > > > I suggest to start with a detailed profile of the current > > implementation, > > Any suggestions there? json-encode-char and mapconcat take most of the time, so it seems. > By the way, I've included the example, so you can also profile it > yourself. Yes, I could. What's your point, though? From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 15:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.1426864837536 (code B ref 20154); Fri, 20 Mar 2015 15:21:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 15:20:37 +0000 Received: from localhost ([127.0.0.1]:59267 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYyiy-00008a-Tv for submit@debbugs.gnu.org; Fri, 20 Mar 2015 11:20:37 -0400 Received: from mail-we0-f181.google.com ([74.125.82.181]:34911) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYyiw-00008M-If for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 11:20:35 -0400 Received: by webcq43 with SMTP id cq43so84757348web.2 for <20154@debbugs.gnu.org>; Fri, 20 Mar 2015 08:20:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=E3ln7TD5kQRyJHAIDGHJwlJrKsdyh0zWWtIJocg7IHs=; b=WyJ6Z/TRTnAdF6taVQLEqdYjOJt2nOviv7FWWzBqj6/90fBUAnn0XDd3LMLJEjWTSb AN87fuNnuklOEhq574NVBJrK5py9cvJQiEAAz70dwrApu7lBjk+g8SLryYQP7qQChh/n ZcgsbOBjeavmP3cg1aE+ewpz8t51UDnUHnGpB/tktif9jIVE2KSWcCv8wScjx3oToh42 8e13zi7kIbeuLpXHBU+ADG6b1LzeXPMUWbNlebbr6e8F3kh/epz5a34hSK4Gn+JK3bpt FW1EnOeGuRCkH9F+KPBXn8QM7Y6pJ103Fq5uhzAQipAQjOSjZtyQVnuCmwnXTlsNYCEB HVDw== X-Received: by 10.180.80.7 with SMTP id n7mr19558890wix.60.1426864829006; Fri, 20 Mar 2015 08:20:29 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id n1sm3440156wib.11.2015.03.20.08.20.27 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 08:20:28 -0700 (PDT) Message-ID: <550C3AB9.7020403@yandex.ru> Date: Fri, 20 Mar 2015 17:20:25 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> In-Reply-To: <831tkjww0y.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/20/2015 05:03 PM, Eli Zaretskii wrote: > Yes, I could. What's your point, though? That if asking the question takes the same time as doing the profiling yourself, the latter would be more efficient. I don't really mind, just puzzled. > json-encode-char and mapconcat take most of the time, so it seems. So it does. But here's an alternative implementation I tried: (defun json-encode-big-string (str) (with-temp-buffer (insert str) (goto-char (point-min)) (while (re-search-forward "[\"\\/\b\f\b\r\t]\\|[^ -~]" nil t) (replace-match (json-encode-char (char-after (match-beginning 0))) t t)) (format "\"%s\"" (buffer-string)))) It takes 0.15s here, which is still too long. Here's its profile. CPU: - command-execute 1245 95% - call-interactively 1245 95% - funcall-interactively 1184 90% - eval-last-sexp 1140 87% - elisp--eval-last-sexp 1140 87% - eval 1140 87% - length 1140 87% - json-encode-big-string 1140 87% - let 1140 87% - save-current-buffer 1140 87% - unwind-protect 1136 87% - progn 1136 87% - while 980 75% - replace-match 332 25% - json-encode-char 212 16% char-after 4 0% format 4 0% + execute-extended-command 37 2% + previous-line 7 0% + byte-code 61 4% + ... 57 4% Memory: - command-execute 76,018,070 100% - call-interactively 76,018,070 100% - funcall-interactively 76,005,728 99% - eval-last-sexp 69,257,352 91% - elisp--eval-last-sexp 69,257,352 91% - eval 69,242,772 91% - length 69,242,772 91% - json-encode-big-string 69,242,772 91% - let 69,242,772 91% - save-current-buffer 69,234,412 91% - unwind-protect 69,040,810 90% - progn 69,033,546 90% - while 55,201,778 72% - replace-match 17,829,052 23% json-encode-char 10,471,476 13% format 2,640,256 3% generate-new-buffer 8,360 0% elisp--eval-last-sexp-print-value 12,532 0% + elisp--preceding-sexp 2,048 0% - execute-extended-command 6,748,360 8% - command-execute 6,685,480 8% - call-interactively 6,685,480 8% - funcall-interactively 6,685,464 8% + profiler-report 6,681,851 8% + profiler-start 3,613 0% + sit-for 3,320 0% + previous-line 16 0% + byte-code 12,342 0% ... 0 0% From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 16:04:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14268673894637 (code B ref 20154); Fri, 20 Mar 2015 16:04:01 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 16:03:09 +0000 Received: from localhost ([127.0.0.1]:59295 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYzO8-0001Ci-TQ for submit@debbugs.gnu.org; Fri, 20 Mar 2015 12:03:09 -0400 Received: from mtaout28.012.net.il ([80.179.55.184]:51390) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYzO6-0001CC-9C for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 12:03:07 -0400 Received: from conversion-daemon.mtaout28.012.net.il by mtaout28.012.net.il (HyperSendmail v2007.08) id <0NLI00100PSXB600@mtaout28.012.net.il> for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 18:01:35 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout28.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLI00DV6PUNPVA0@mtaout28.012.net.il>; Fri, 20 Mar 2015 18:01:35 +0200 (IST) Date: Fri, 20 Mar 2015 18:02:57 +0200 From: Eli Zaretskii In-reply-to: <550C3AB9.7020403@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83wq2bveq6.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Fri, 20 Mar 2015 17:20:25 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > On 03/20/2015 05:03 PM, Eli Zaretskii wrote: > > > Yes, I could. What's your point, though? > > That if asking the question takes the same time as doing the profiling > yourself, the latter would be more efficient. I don't really mind, just > puzzled. I have other things on my plate while I read email. If my advice bothers you, I can shut up in the future. > > json-encode-char and mapconcat take most of the time, so it seems. > > So it does. But here's an alternative implementation I tried: > > (defun json-encode-big-string (str) > (with-temp-buffer > (insert str) > (goto-char (point-min)) > (while (re-search-forward "[\"\\/\b\f\b\r\t]\\|[^ -~]" nil t) > (replace-match (json-encode-char (char-after (match-beginning 0))) > t t)) > (format "\"%s\"" (buffer-string)))) > > It takes 0.15s here, which is still too long. I suggest to rewrite json-encode-char, it does a lot of unnecessary stuff, starting with the call to encode-char (which was needed in Emacs 22 and before, but no more). The call to rassoc is also redundant, since you already have that covered in your regexp. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 16:22:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14268685166314 (code B ref 20154); Fri, 20 Mar 2015 16:22:01 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 16:21:56 +0000 Received: from localhost ([127.0.0.1]:59303 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYzgK-0001dm-8T for submit@debbugs.gnu.org; Fri, 20 Mar 2015 12:21:56 -0400 Received: from mail-we0-f180.google.com ([74.125.82.180]:33830) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYzgI-0001dX-3R for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 12:21:54 -0400 Received: by wegp1 with SMTP id p1so86018841weg.1 for <20154@debbugs.gnu.org>; Fri, 20 Mar 2015 09:21:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=sdenkB+voBeJYJ6L5ovkt/dDBSWx55fWvr0GTMrxRXg=; b=TbhuVuympXy54pCuZIleM0fn6qPaLQnwR/fZ9Ja+zc9s8O4xtpchGCZMbxZ9L8mh4r cag6tG8IiagvJUNUxMoBv8zFLnPf43zp64gPBP8arB4sOh7UJVvJniLBnk4ptEc9qCWp loHDDaM7PrkkteU3hANSsu2fAZdY7yDGtd4bEIgIf7AK8e3tozVUmlPnYpawjXSE2pLU mu+zCDrli1atmp8cI2bhyrFqQpwa3aHBMBrTc2OU4MgO56x+6NdzGwRHzwUWr5EM4r+K Hfvgw+2oF9ovjnK3+9XTS8rFSLc+bj53VXceqeCy1meMpDaD9X0kbrDcq1eGNIz5k2Gl 5S1g== X-Received: by 10.180.104.66 with SMTP id gc2mr26019636wib.34.1426868508235; Fri, 20 Mar 2015 09:21:48 -0700 (PDT) Received: from [192.168.0.185] (static-nbl2-118.cytanet.com.cy. [212.31.107.118]) by mx.google.com with ESMTPSA id fo8sm3651204wib.14.2015.03.20.09.21.47 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 09:21:47 -0700 (PDT) Message-ID: <550C491A.6000909@yandex.ru> Date: Fri, 20 Mar 2015 18:21:46 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> In-Reply-To: <83wq2bveq6.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/20/2015 06:02 PM, Eli Zaretskii wrote: > I have other things on my plate while I read email. If my advice > bothers you, I can shut up in the future. Very well. Please continue with the advice. > I suggest to rewrite json-encode-char, it does a lot of unnecessary > stuff, starting with the call to encode-char (which was needed in > Emacs 22 and before, but no more). The call to rassoc is also > redundant, since you already have that covered in your regexp. Yes, I thought about that, but as the number of calls to `json-encode-char' must have decreased by 10 in the new version (only each 10th character needs to be encoded), and the runtime only decreased by 3 (or by 2, in a different example I have), the total improvement can't be dramatic enough even if `json-encode-char' is lightning-fast. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 16:45:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14268698878469 (code B ref 20154); Fri, 20 Mar 2015 16:45:01 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 16:44:47 +0000 Received: from localhost ([127.0.0.1]:59319 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ02Q-0002CW-SE for submit@debbugs.gnu.org; Fri, 20 Mar 2015 12:44:47 -0400 Received: from mtaout21.012.net.il ([80.179.55.169]:64523) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ02O-0002CJ-In for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 12:44:45 -0400 Received: from conversion-daemon.a-mtaout21.012.net.il by a-mtaout21.012.net.il (HyperSendmail v2007.08) id <0NLI00M00RTDT100@a-mtaout21.012.net.il> for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 18:44:38 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout21.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLI00MSFRUDRD30@a-mtaout21.012.net.il>; Fri, 20 Mar 2015 18:44:38 +0200 (IST) Date: Fri, 20 Mar 2015 18:44:35 +0200 From: Eli Zaretskii In-reply-to: <550C491A.6000909@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83siczvcss.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Fri, 20 Mar 2015 18:21:46 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > > I suggest to rewrite json-encode-char, it does a lot of unnecessary > > stuff, starting with the call to encode-char (which was needed in > > Emacs 22 and before, but no more). The call to rassoc is also > > redundant, since you already have that covered in your regexp. > > Yes, I thought about that, but as the number of calls to > `json-encode-char' must have decreased by 10 in the new version (only > each 10th character needs to be encoded), and the runtime only decreased > by 3 (or by 2, in a different example I have), the total improvement > can't be dramatic enough even if `json-encode-char' is lightning-fast. To see how much of the time is taken by json-encode-char, replace it with something trivial, like 1+, and see what speedup you get. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 16:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14268703569189 (code B ref 20154); Fri, 20 Mar 2015 16:53:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 16:52:36 +0000 Received: from localhost ([127.0.0.1]:59334 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ0A0-0002O9-Ao for submit@debbugs.gnu.org; Fri, 20 Mar 2015 12:52:36 -0400 Received: from mail-wi0-f169.google.com ([209.85.212.169]:38787) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ09x-0002Nw-RG for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 12:52:34 -0400 Received: by wibgn9 with SMTP id gn9so20017505wib.1 for <20154@debbugs.gnu.org>; Fri, 20 Mar 2015 09:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=59oqKGOUm+S/MhdLyzIA/UQKpPV3rQtkYpTgfWNxvnQ=; b=Mv+4ChQnsXSM2ZGQ8hrxo37WKSTia7vybye+9gpGJnswtlYIPTpPABaQZBZpXH7/fq xo0V8C5q6RENUmbYBY7JfIw4C9z7AdyBJltiUvezsyBeqmMusvNKSRvnSY15QHxr/FPt uGmeckSbsuuXHcy6dUkdqkVaV+mAqeD02Nf8R5lIVlfXUDavzkTlOho/eEQE68Gm/W2S nwVQwMzWvKBEjKnByQha0nvyvYkTJLRIaiBl0vWWLBkhHFM3vz3pROtLboelGEapccza QEysVT4vbtsiwdrO5ZJ1MomP+TNlvc/Cze1SZEiBFFU6mVVthTaxoadjUl04DEOIphNh TD6Q== X-Received: by 10.180.35.72 with SMTP id f8mr19434434wij.31.1426870348109; Fri, 20 Mar 2015 09:52:28 -0700 (PDT) Received: from [192.168.0.185] (static-nbl2-118.cytanet.com.cy. [212.31.107.118]) by mx.google.com with ESMTPSA id e18sm7092711wjz.27.2015.03.20.09.52.27 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 09:52:27 -0700 (PDT) Message-ID: <550C504A.10708@yandex.ru> Date: Fri, 20 Mar 2015 18:52:26 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> In-Reply-To: <83siczvcss.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/20/2015 06:44 PM, Eli Zaretskii wrote: > To see how much of the time is taken by json-encode-char, replace it > with something trivial, like 1+, and see what speedup you get. Yep. Replacing the second definition with (defun json-encode-big-string (str) (with-temp-buffer (insert str) (goto-char (point-min)) (while (re-search-forward "[\"\\/\b\f\b\r\t]\\|[^ -~]" nil t) (replace-match "z" t t)) (format "\"%s\"" (buffer-string)))) still makes it take ~100ms on the example string (as opposed to 2ms in the optimized Python implementation). From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 17:45:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142687348918964 (code B ref 20154); Fri, 20 Mar 2015 17:45:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 17:44:49 +0000 Received: from localhost ([127.0.0.1]:59344 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ0yW-0004vm-VU for submit@debbugs.gnu.org; Fri, 20 Mar 2015 13:44:49 -0400 Received: from mtaout27.012.net.il ([80.179.55.183]:55731) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ0yU-0004v3-BJ for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 13:44:47 -0400 Received: from conversion-daemon.mtaout27.012.net.il by mtaout27.012.net.il (HyperSendmail v2007.08) id <0NLI00300U8RXT00@mtaout27.012.net.il> for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 19:39:21 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout27.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLI001WAUDL3B20@mtaout27.012.net.il>; Fri, 20 Mar 2015 19:39:21 +0200 (IST) Date: Fri, 20 Mar 2015 19:44:37 +0200 From: Eli Zaretskii In-reply-to: <550C504A.10708@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83r3sjva0q.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Fri, 20 Mar 2015 18:52:26 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > On 03/20/2015 06:44 PM, Eli Zaretskii wrote: > > > To see how much of the time is taken by json-encode-char, replace it > > with something trivial, like 1+, and see what speedup you get. > > Yep. Replacing the second definition with > > (defun json-encode-big-string (str) > (with-temp-buffer > (insert str) > (goto-char (point-min)) > (while (re-search-forward "[\"\\/\b\f\b\r\t]\\|[^ -~]" nil t) > (replace-match "z" t t)) > (format "\"%s\"" (buffer-string)))) > > still makes it take ~100ms on the example string (as opposed to 2ms in > the optimized Python implementation). That's not what I see here. I cannot get the time above 1 sec even with a 1000 time longer input string, if I replace json-encode-char with 1+. So I think your 100ms is the constant overhead of some kind. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 18:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142687694532441 (code B ref 20154); Fri, 20 Mar 2015 18:43:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 18:42:25 +0000 Received: from localhost ([127.0.0.1]:59362 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ1sG-0008R8-R8 for submit@debbugs.gnu.org; Fri, 20 Mar 2015 14:42:25 -0400 Received: from mail-we0-f175.google.com ([74.125.82.175]:34006) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ1sE-0008QX-QQ for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 14:42:23 -0400 Received: by wegp1 with SMTP id p1so88821803weg.1 for <20154@debbugs.gnu.org>; Fri, 20 Mar 2015 11:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=D1y3rkXeKtX/rRqwHt3Lt6ZA26R/7a0jOmm1ThL73po=; b=hiKQ/MWIjMToXe2VNGEHW5xjl8ZbcCA2n2TVBOqWZFbX32XQCaKAw9Ojwb7ctR/+J2 uosac/isAIxsY2YpObwy3pwold5dX1RJFaMc8eiAPvgFbesNrtYiTw1So+RafpPuj0K5 hhLuGi3lhT2AXkL/FTr4XbnSGVJLatqf6ve3qs+wzigomPWiv+W1LWD2qVGJnU64Zqub ZgDyTLESsVhGXJW4DaGjai+ahRO3m5bevnBH4TB6C8I7Ox3jezBMRfOI0yx2yy7JQr8O oaiewBBPBo3AEgeDfrjMw1riHquYL4VjeqqbRcVp1vcghI/fhSE28BJvw5d6kfHY2qR3 DH9A== X-Received: by 10.194.63.206 with SMTP id i14mr166418089wjs.107.1426876936936; Fri, 20 Mar 2015 11:42:16 -0700 (PDT) Received: from [192.168.0.185] (static-nbl2-118.cytanet.com.cy. [212.31.107.118]) by mx.google.com with ESMTPSA id fs8sm72252wib.8.2015.03.20.11.42.15 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 11:42:16 -0700 (PDT) Message-ID: <550C6A06.6040203@yandex.ru> Date: Fri, 20 Mar 2015 20:42:14 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> In-Reply-To: <83r3sjva0q.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/20/2015 07:44 PM, Eli Zaretskii wrote: > That's not what I see here. I cannot get the time above 1 sec even > with a 1000 time longer input string, if I replace json-encode-char > with 1+. What code exactly have you tried? You can't just replace json-encode-char with 1+. The former returns a string, the latter returns a number (or a char, I guess). > So I think your 100ms is the constant overhead of some kind. If you just changed the upper bound in the defvar init form (from 30000 to something), I suspect you forgot to use C-M-x instead of C-x C-e, to update the actual value of the variable. Making the string 10 times longer increases the runtime by ~5 here (0.1 -> 0.5). Another 10x increase in length makes it run 4.3 seconds. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 21:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142688608021889 (code B ref 20154); Fri, 20 Mar 2015 21:15:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 21:14:40 +0000 Received: from localhost ([127.0.0.1]:59394 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ4Fb-0005gz-O0 for submit@debbugs.gnu.org; Fri, 20 Mar 2015 17:14:39 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:65462) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ4FW-0005gi-Ij for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 17:14:38 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0NLJ00100416P400@a-mtaout20.012.net.il> for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 23:14:28 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLJ001OF4C3OT10@a-mtaout20.012.net.il>; Fri, 20 Mar 2015 23:14:28 +0200 (IST) Date: Fri, 20 Mar 2015 23:14:26 +0200 From: Eli Zaretskii In-reply-to: <550C6A06.6040203@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83fv8zv0b1.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Fri, 20 Mar 2015 20:42:14 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > Making the string 10 times longer increases the runtime by ~5 here (0.1 > -> 0.5). Another 10x increase in length makes it run 4.3 seconds. So maybe writing this in C is the way to go. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 22:03:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142688898026221 (code B ref 20154); Fri, 20 Mar 2015 22:03:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 22:03:00 +0000 Received: from localhost ([127.0.0.1]:59415 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ50O-0006or-DN for submit@debbugs.gnu.org; Fri, 20 Mar 2015 18:03:00 -0400 Received: from mail-wi0-f175.google.com ([209.85.212.175]:37215) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ50M-0006of-Qz for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 18:02:59 -0400 Received: by wixw10 with SMTP id w10so237619wix.0 for <20154@debbugs.gnu.org>; Fri, 20 Mar 2015 15:02:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=O2z+Z+A2DmhEkaCjxIJU7ydIb9U/eGvrqeKGPNA943Y=; b=VfRZUzniFeCZcMa4Ga7WfhKDUn3D+hFca/K1+BZcxMi3zS3x0cN5HFFHT3lPpm2ui+ 52p8wbqHIDewT4s+A1t4nUSyIsQvJxNZKwYT+dzONgAdOZvwwP7NF1n4mc821WPBZkXQ mq+oNQMGZzGAl9RFMnygGhFcHLFtsy/Y5GBCyi5B9bl9MPt0pTsztaDstob8SzvxovVW JJGp0jWg13utBi5/P1CIuWoxNOoHa16kDAtJzpJTIbjUyxQRmDkWzuOSASmmgTx99KdF LliQM7G8dg9PfXseAJ6Sxlp2cNRKQL1o2yS5Mu/ZMYJOCB2O7CBg/LWF3RmFspYXRqv5 uUkg== X-Received: by 10.194.177.195 with SMTP id cs3mr161093617wjc.141.1426888973181; Fri, 20 Mar 2015 15:02:53 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id yr10sm8093017wjc.0.2015.03.20.15.02.52 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 15:02:52 -0700 (PDT) Message-ID: <550C990B.8080505@yandex.ru> Date: Sat, 21 Mar 2015 00:02:51 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> In-Reply-To: <83fv8zv0b1.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/20/2015 11:14 PM, Eli Zaretskii wrote: >> Making the string 10 times longer increases the runtime by ~5 here (0.1 >> -> 0.5). Another 10x increase in length makes it run 4.3 seconds. > > So maybe writing this in C is the way to go. Maybe implementing `json-encode-string` itself in C isn't strictly necessary, or even particularly advantageous. How about trying to optimize `replace-match' or `replace-regexp-in-string' (which are the main two approaches we can use to implement `json-encode-string') for the case of large input? Take this example: (setq s1 (apply #'concat (cl-loop for i from 1 to 30000 collect "123456789\n")) s2 (apply #'concat (cl-loop for i from 1 to 15000 collect "1234567890123456789\n"))) On my machine, (replace-regexp-in-string "\n" "z" s1 t t) takes ~0.13s, while (replace-regexp-in-string "\n" "z" s2 t t) clocks at ~0.08-0.10. Which is, again, pretty slow by modern standards. (And I've only now realized that the above function is implemented in Lisp; all the more reason to move it to C). Replacing "z" with #'identity (so now we include a function call overhead) increases the averages to 0.15s and 0.10s respectively. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Mar 2015 22:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142689037328239 (code B ref 20154); Fri, 20 Mar 2015 22:27:01 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Mar 2015 22:26:13 +0000 Received: from localhost ([127.0.0.1]:59424 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ5Mq-0007LP-Rp for submit@debbugs.gnu.org; Fri, 20 Mar 2015 18:26:13 -0400 Received: from mail-wi0-f177.google.com ([209.85.212.177]:34856) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZ5Mo-0007LC-Tx for 20154@debbugs.gnu.org; Fri, 20 Mar 2015 18:26:11 -0400 Received: by wibdy8 with SMTP id dy8so3115097wib.0 for <20154@debbugs.gnu.org>; Fri, 20 Mar 2015 15:26:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=qOz0MBJty54AEl2aK0Mq8XmW+WP5/N+v303YzmcoM8s=; b=hEYgtkz4inQMVVyCv82pX7GCSqq6XL9zzDq3nliMc4RlgqKWrS+swY1bHGKtz5p4w8 8VIwNd9PTU17UTaPxQQMdc/FVi+ZVXt8TXniB2+PLMBS4WqsoziBv97A9FWBVs7psfl3 l07RY3xyRktFMN71k5KzgwPx9KzRA3cF8T9bCWIf4NEMQAg04rzCYvJtmazRNy5lUjMA BqIGRDhYrzQV3sX1C8nAL+qOtTETAzqr8p/ZsZD9PhqQrhKITbkL59EyFGlmSO4AHzxQ wXoz0e/FvPxBPVXGDAOYLOnFlmwKPtVp88nzRYew9Zrks+GOqWTByp9C4dtV6OhADZRw 4Vkw== X-Received: by 10.180.106.68 with SMTP id gs4mr28207325wib.39.1426890365408; Fri, 20 Mar 2015 15:26:05 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id lu13sm65896wic.10.2015.03.20.15.26.04 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 15:26:04 -0700 (PDT) Message-ID: <550C9E7B.4060907@yandex.ru> Date: Sat, 21 Mar 2015 00:26:03 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> In-Reply-To: <83fv8zv0b1.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) As per your comment, this seems to be the best we can do for long strings, without diving into C: (defun json-encode-string-1 (string) "Return a JSON representation of STRING." (with-temp-buffer (insert string) (goto-char (point-min)) ;; Skip over ASCIIish printable characters. (while (re-search-forward "\\([\"\\/\b\f\n\r\t]\\)\\|[^ -~]" nil t) (replace-match (if (match-beginning 1) ;; Special JSON character (\n, \r, etc.). (format "\\%c" (char-after (match-beginning 0))) ;; Fallback: UCS code point in \uNNNN form. (format "\\u%04x" (char-after (match-beginning 0)))) t t)) (format "\"%s\"" (buffer-string)))) It brings the execution time down to ~0.14s here, on the same example. And there'll need to be a fallback for short strings, because `with-temp-buffer' overhead is non-trivial. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 07:59:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142692473418602 (code B ref 20154); Sat, 21 Mar 2015 07:59:01 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 07:58:54 +0000 Received: from localhost ([127.0.0.1]:59544 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZEJ3-0004px-Ti for submit@debbugs.gnu.org; Sat, 21 Mar 2015 03:58:54 -0400 Received: from mtaout21.012.net.il ([80.179.55.169]:62093) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZEJ0-0004pg-Aq for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 03:58:51 -0400 Received: from conversion-daemon.a-mtaout21.012.net.il by a-mtaout21.012.net.il (HyperSendmail v2007.08) id <0NLJ00100XYHNL00@a-mtaout21.012.net.il> for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 09:58:43 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout21.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLJ001UKY5VJC90@a-mtaout21.012.net.il>; Sat, 21 Mar 2015 09:58:43 +0200 (IST) Date: Sat, 21 Mar 2015 09:58:43 +0200 From: Eli Zaretskii In-reply-to: <550C990B.8080505@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <838ueqvl1o.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sat, 21 Mar 2015 00:02:51 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > On 03/20/2015 11:14 PM, Eli Zaretskii wrote: > > >> Making the string 10 times longer increases the runtime by ~5 here (0.1 > >> -> 0.5). Another 10x increase in length makes it run 4.3 seconds. > > > > So maybe writing this in C is the way to go. > > Maybe implementing `json-encode-string` itself in C isn't strictly > necessary, or even particularly advantageous. It depends on your requirements. How fast would it need to run to satisfy your needs? > How about trying to optimize `replace-match' or > `replace-regexp-in-string' (which are the main two approaches we can use > to implement `json-encode-string') for the case of large input? > > Take this example: > > (setq s1 (apply #'concat (cl-loop for i from 1 to 30000 > collect "123456789\n")) > s2 (apply #'concat (cl-loop for i from 1 to 15000 > collect "1234567890123456789\n"))) > > On my machine, > > (replace-regexp-in-string "\n" "z" s1 t t) > > takes ~0.13s, while > > (replace-regexp-in-string "\n" "z" s2 t t) > > clocks at ~0.08-0.10. > > Which is, again, pretty slow by modern standards. You don't really need regexp replacement functions with all its features here, do you? What you need is a way to skip characters that are "okay", then replace the character that is "not okay" with its encoded form, then repeat. So an alternative strategy would be to use 'skip-chars-forward' to skip to the next locus where encoding is necessary, and 'append' to construct the output string from the "okay" parts and the encoded "not okay" part. This bypasses most of the unneeded complications in 'replace-match' and 'replace-regexp-in-string'. I don't know if that will be faster, but I think it's worth trying. For starters, how fast can you iterate through the string with 'skip-chars-forward', stopping at characters that need encoding, without actually encoding them, but just consing the output string by appending the parts delimited by places where 'skip-chars-forward' stopped? That's the lower bound on performance using this method. > (And I've only now realized that the above function is implemented in > Lisp; all the more reason to move it to C). I think the latest tendency is the opposite: move to Lisp everything that doesn't need to be in C. If some specific application needs more speed than we can provide, the first thing I'd try is think of a new primitive by abstracting your use case enough to be more useful than just for JSON. Of course, implementing the precise use case in C first is probably a prerequisite, since it could turn out that the problem is somewhere else, or that even in C you won't get the speed you want. > Replacing "z" with #'identity (so now we include a function call > overhead) increases the averages to 0.15s and 0.10s respectively. Sounds like the overhead of the Lisp interpreter is a significant factor here, no? From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 08:08:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142692525519521 (code B ref 20154); Sat, 21 Mar 2015 08:08:01 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 08:07:35 +0000 Received: from localhost ([127.0.0.1]:59548 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZERT-00054n-97 for submit@debbugs.gnu.org; Sat, 21 Mar 2015 04:07:35 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:61004) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZERN-00054T-GD for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 04:07:33 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0NLJ00J00Y7XAM00@a-mtaout22.012.net.il> for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 10:07:23 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLJ00J1DYKA0OA0@a-mtaout22.012.net.il>; Sat, 21 Mar 2015 10:07:23 +0200 (IST) Date: Sat, 21 Mar 2015 10:07:22 +0200 From: Eli Zaretskii In-reply-to: <550C9E7B.4060907@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <837fuavkn9.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sat, 21 Mar 2015 00:26:03 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > As per your comment, this seems to be the best we can do for long > strings, without diving into C: > > (defun json-encode-string-1 (string) > "Return a JSON representation of STRING." > (with-temp-buffer > (insert string) > (goto-char (point-min)) > ;; Skip over ASCIIish printable characters. > (while (re-search-forward "\\([\"\\/\b\f\n\r\t]\\)\\|[^ -~]" nil t) > (replace-match > (if (match-beginning 1) > ;; Special JSON character (\n, \r, etc.). > (format "\\%c" (char-after (match-beginning 0))) Do you really need 'format' here? Why not (concat "\\" (char-to-string (char-after (match-beginning 0)))) instead? Or even simply insert these two parts at the match point, after deleting the match. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 08:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: dgutov@yandex.ru Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142692558420069 (code B ref 20154); Sat, 21 Mar 2015 08:14:02 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 08:13:04 +0000 Received: from localhost ([127.0.0.1]:59552 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZEWm-0005Dd-44 for submit@debbugs.gnu.org; Sat, 21 Mar 2015 04:13:04 -0400 Received: from mtaout23.012.net.il ([80.179.55.175]:55833) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZEWj-0005D7-SU for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 04:13:02 -0400 Received: from conversion-daemon.a-mtaout23.012.net.il by a-mtaout23.012.net.il (HyperSendmail v2007.08) id <0NLJ00900Y9UFC00@a-mtaout23.012.net.il> for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 10:12:55 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout23.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLJ009A4YTJAB80@a-mtaout23.012.net.il>; Sat, 21 Mar 2015 10:12:55 +0200 (IST) Date: Sat, 21 Mar 2015 10:12:55 +0200 From: Eli Zaretskii In-reply-to: <838ueqvl1o.fsf@gnu.org> X-012-Sender: halo1@inter.net.il Message-id: <83619uvke0.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sat, 21 Mar 2015 09:58:43 +0200 > From: Eli Zaretskii > Cc: 20154@debbugs.gnu.org > > So an alternative strategy would be to use 'skip-chars-forward' to > skip to the next locus where encoding is necessary, and 'append' to ^^^^^^^^ Sorry, I meant 'concat', of course. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 20:01:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142696805927662 (code B ref 20154); Sat, 21 Mar 2015 20:01:02 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 20:00:59 +0000 Received: from localhost ([127.0.0.1]:60056 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZPZq-0007C5-FZ for submit@debbugs.gnu.org; Sat, 21 Mar 2015 16:00:58 -0400 Received: from mail-wi0-f176.google.com ([209.85.212.176]:36078) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZPZo-0007Bs-M5 for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 16:00:57 -0400 Received: by wibg7 with SMTP id g7so14991389wib.1 for <20154@debbugs.gnu.org>; Sat, 21 Mar 2015 13:00:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=h2JNZLOtoYbtiu90xRB6XYKmCmac3MUq6wjyIFmpX9A=; b=FjNJHV0mTCZ/v96Lsr/ZVIIqo6y5r590Uc4jLgBzTDUq/TE9hW2mDrB4SAz4aHlONo EkmiDkZCwzfcyJmZxY8iSKEQBz7HDhNcFvViUcVTawpbBz5VO9vZxNpSWV/yfmvx3uSB jPwbmYvOfRBegNF9I65oqASc2If7lm0VYE9fyJp8loyAL+8GsY0yW+4jSkZIZq1rpl/J H2SoAN9Z+vZ/ERKmSMjhZBwIgY7nk7wRrWznf9QpTr3UtCyD1wLkQoDKdPuXbIirnEOb i30w47pL0V6ZnW+O486hRJtxj57A6nzA170lAqh+Og6ZyxBa27fCZEDVF5VWsnO2umcF yM2g== X-Received: by 10.180.206.13 with SMTP id lk13mr6391232wic.95.1426968051041; Sat, 21 Mar 2015 13:00:51 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id dj4sm11948294wjc.13.2015.03.21.13.00.49 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Mar 2015 13:00:50 -0700 (PDT) Message-ID: <550DCDEE.4090900@yandex.ru> Date: Sat, 21 Mar 2015 22:00:46 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> In-Reply-To: <838ueqvl1o.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/21/2015 09:58 AM, Eli Zaretskii wrote: > It depends on your requirements. How fast would it need to run to > satisfy your needs? In this case, the buffer contents are encoded to JSON at most once per keypress. So 50ms or below should be fast enough, especially since most files are smaller than that. Of course, I'm sure there are use cases for fast JSON encoding/decoding of even bigger volumes of data, but they can probably wait until we have FFI. > You don't really need regexp replacement functions with all its > features here, do you? What you need is a way to skip characters that > are "okay", then replace the character that is "not okay" with its > encoded form, then repeat. It doesn't seem like regexp searching is the slow part: save for the GC pauses, looking for the non-matching regexp in the same string - (replace-regexp-in-string "x" "z" s1 t t) - only takes ~3ms. And likewise, after changing them to use `concat' instead of `format', both alternative json-encode-string implementations that I have "encode" a numbers-only (without newlines) string of the same length in a few milliseconds. Again, save for the GC pauses, which can add 30-40ms. > For starters, how fast > can you iterate through the string with 'skip-chars-forward', stopping > at characters that need encoding, without actually encoding them, but > just consing the output string by appending the parts delimited by > places where 'skip-chars-forward' stopped? That's the lower bound on > performance using this method. 70-90ms if we simply skip 0-9, even without nreverse-ing and concatenating. But the change in runtime after adding an (apply #'concat (nreverse res)) step doesn't look statistically insignificant. Here's the implementation I tried: (defun foofoo (string) (with-temp-buffer (insert string) (goto-char (point-min)) (let (res) (while (not (eobp)) (let ((skipped (skip-chars-forward "0-9"))) (push (buffer-substring (- (point) skipped) (point)) res)) (forward-char 1)) res))) But that actually goes down to 30ms if we don't accumulate the result. > I think the latest tendency is the opposite: move to Lisp everything > that doesn't need to be in C. Yes, and often that's great, if we're dealing with some piece of UI infrastructure that only gets called at most a few times per command, with inputs of size we can anticipate in advance. > If some specific application needs more > speed than we can provide, the first thing I'd try is think of a new > primitive by abstracting your use case enough to be more useful than > just for JSON. That's why I suggested to do that with `replace-regexp-in-string' first. That's a very common feature, and in Python and Ruby it's written in C. Ruby's calling convention is even pretty close (the replacement can be a string, or it can take a block, which is a kind of a function). > Of course, implementing the precise use case in C first is probably a > prerequisite, since it could turn out that the problem is somewhere > else, or that even in C you won't get the speed you want. A fast `replace-regexp-in-string' may not get us where I want, but it should get us close. It will still be generally useful, and it'll save us from having two `json-encode-string' implementations - for long and short strings. >> Replacing "z" with #'identity (so now we include a function call >> overhead) increases the averages to 0.15s and 0.10s respectively. > > Sounds like the overhead of the Lisp interpreter is a significant > factor here, no? Yes and no. Given the 50ms budget, I think we can live with it for now, when it's the only problem. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 20:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142696957229960 (code B ref 20154); Sat, 21 Mar 2015 20:27:02 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 20:26:12 +0000 Received: from localhost ([127.0.0.1]:60076 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZPyF-0007n9-J6 for submit@debbugs.gnu.org; Sat, 21 Mar 2015 16:26:11 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:62051) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZPyC-0007mv-M6 for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 16:26:09 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0NLK00100WFSB400@a-mtaout22.012.net.il> for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 22:26:02 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLK001R8WRD2N90@a-mtaout22.012.net.il>; Sat, 21 Mar 2015 22:26:02 +0200 (IST) Date: Sat, 21 Mar 2015 22:25:48 +0200 From: Eli Zaretskii In-reply-to: <550DCDEE.4090900@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83zj76rtbn.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sat, 21 Mar 2015 22:00:46 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > On 03/21/2015 09:58 AM, Eli Zaretskii wrote: > > > It depends on your requirements. How fast would it need to run to > > satisfy your needs? > > In this case, the buffer contents are encoded to JSON at most once per > keypress. So 50ms or below should be fast enough, especially since most > files are smaller than that. So each keypress you need to encode the whole buffer, including the last keypress and all those before it? I guess I don't really understand why each keypress should trigger encoding of the whole buffer. > > You don't really need regexp replacement functions with all its > > features here, do you? What you need is a way to skip characters that > > are "okay", then replace the character that is "not okay" with its > > encoded form, then repeat. > > It doesn't seem like regexp searching is the slow part: save for the GC > pauses, looking for the non-matching regexp in the same string - > > (replace-regexp-in-string "x" "z" s1 t t) > > - only takes ~3ms. Then a series of calls to replace-regexp-in-string, one each for every one of the "special" characters, should get you close to your goal, right? > And likewise, after changing them to use `concat' instead of `format', > both alternative json-encode-string implementations that I have "encode" > a numbers-only (without newlines) string of the same length in a few > milliseconds. Again, save for the GC pauses, which can add 30-40ms. So does this mean you have your solution? From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Drew Adams Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 21:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov , Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14269719641237 (code B ref 20154); Sat, 21 Mar 2015 21:07:01 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 21:06:04 +0000 Received: from localhost ([127.0.0.1]:60099 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZQap-0000Js-TO for submit@debbugs.gnu.org; Sat, 21 Mar 2015 17:06:04 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:18105) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZQan-0000JM-I9 for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 17:06:02 -0400 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id t2LL5rHr025684 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sat, 21 Mar 2015 21:05:54 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.13.8/8.13.8) with ESMTP id t2LL5rnW015265 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Sat, 21 Mar 2015 21:05:53 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by aserv0122.oracle.com (8.13.8/8.13.8) with ESMTP id t2LL5rxK018038; Sat, 21 Mar 2015 21:05:53 GMT MIME-Version: 1.0 Message-ID: Date: Sat, 21 Mar 2015 14:05:55 -0700 (PDT) From: Drew Adams References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> In-Reply-To: <550DCDEE.4090900@yandex.ru> X-Priority: 3 X-Mailer: Oracle Beehive Extensions for Outlook 2.0.1.8.2 (807160) [OL 12.0.6691.5000 (x86)] Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Source-IP: aserv0022.oracle.com [141.146.126.234] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) > > I think the latest tendency is the opposite: move to Lisp everything > > that doesn't need to be in C. >=20 > Yes, and often that's great, if we're dealing with some piece of UI > infrastructure that only gets called at most a few times per command, > with inputs of size we can anticipate in advance. (FYI, I'm not following this thread.) I will just say that if you want or need to have something like `json-encode-string' be coded in C for speed, an alternative might be for the actual code to invoke a Lisp function when bound to a variable, e.g., `json-encode-string-function'. That is, it can be OK to define something like the default encoding of JSON in C, but perhaps you can give users the possibility of optionally providing their own encoding Lisp function as well. What would, I think, be too bad would be to make it impossible or difficult for users to provide their own encoding function (without messing with C and rebuilding Emacs). From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 21:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14269722091670 (code B ref 20154); Sat, 21 Mar 2015 21:11:01 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 21:10:09 +0000 Received: from localhost ([127.0.0.1]:60103 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZQem-0000Qs-LI for submit@debbugs.gnu.org; Sat, 21 Mar 2015 17:10:08 -0400 Received: from mail-wg0-f46.google.com ([74.125.82.46]:35107) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZQek-0000PU-JT for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 17:10:07 -0400 Received: by wgdm6 with SMTP id m6so116111661wgd.2 for <20154@debbugs.gnu.org>; Sat, 21 Mar 2015 14:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=/QtLjDS3MLezyOkzusuoq7MqQ0pDEb5rDr5OEUJZj5Q=; b=cH3YxLqn0O7GMwlqlxFZNjaXdyCgsAamhpLN4JsHdHJ+Vud1hGwTaP+GHAstwMvP/Y g2Hkb3zWTs3R/V/2vHXr6bhzFanNeEWIihEgngrzZg2VOVYsGkCGtANZuFY0YRXLoQz0 6UfIkZFtZoqxN8Cy7ETxKF7fGp2wnsnt+kFfSXuamoXH2vqwbHOIR+tcw0v/8hj9cjnV FXENHPxCPNddUPKJCCQ4KmOQwybP8h5aTcUw5j9AQ8bQ1brZmrkdTAxLAR0RSo5o586B XStIRiSwZEcHhStikUSIFT+5naGFfpDPwqMNEw27aE9YAn2/3+oal/91xzYJS83rlZht ig6g== X-Received: by 10.180.189.35 with SMTP id gf3mr6909909wic.5.1426972201122; Sat, 21 Mar 2015 14:10:01 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id nb4sm12151830wjc.20.2015.03.21.14.09.59 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Mar 2015 14:10:00 -0700 (PDT) Message-ID: <550DDE25.1090000@yandex.ru> Date: Sat, 21 Mar 2015 23:09:57 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> In-Reply-To: <837fuavkn9.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/21/2015 10:07 AM, Eli Zaretskii wrote: > Do you really need 'format' here? Why not > > (concat "\\" (char-to-string (char-after (match-beginning 0)))) > > instead? > > Or even simply insert these two parts at the match point, after > deleting the match. Yes, thanks. It gave a small improvement, by 5ms or so. Here's the updated definition (by the way, `json-special-chars' is still needed, to convert ?\n to ?n, and so on, and the performance hit is negligible). (defun json-encode-string-1 (string) "Return a JSON representation of STRING." (with-temp-buffer (insert string) (goto-char (point-min)) ;; Skip over ASCIIish printable characters. (while (re-search-forward "\\([\"\\/\b\f\n\r\t]\\)\\|[^ -~]" nil t) (let ((c (char-before))) (delete-region (1- (point)) (point)) (if (match-beginning 1) ;; Special JSON character (\n, \r, etc.). (insert "\\" (car (rassoc c json-special-chars))) ;; Fallback: UCS code point in \uNNNN form. (insert (format "\\u%04x" c))))) (concat "\"" (buffer-string) "\""))) Futher, it seems I didn't measure its performance well enough to begin with. The average time out of 10 runs comes down to 85ms. Compare it to 150ms, from this implementation: (defun json-encode-string-2 (string) "Return a JSON representation of STRING." (concat "\"" (replace-regexp-in-string "\\([\"\\/\b\f\n\r\t]\\)\\|[^ -~]" (lambda (s) (if (match-beginning 1) (format "\\%c" (car (rassoc (string-to-char s) json-special-chars))) (format "\\u%04x" (string-to-char s)))) string t t) "\"")) From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 21:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14269731913159 (code B ref 20154); Sat, 21 Mar 2015 21:27:01 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 21:26:31 +0000 Received: from localhost ([127.0.0.1]:60109 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZQuc-0000os-Bl for submit@debbugs.gnu.org; Sat, 21 Mar 2015 17:26:30 -0400 Received: from mail-wi0-f176.google.com ([209.85.212.176]:36539) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZQua-0000of-3t for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 17:26:28 -0400 Received: by wibg7 with SMTP id g7so15932887wib.1 for <20154@debbugs.gnu.org>; Sat, 21 Mar 2015 14:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=WL++uxWQ4ZmvkDuj/GwXARDGUkllgp4vpRMrOOXSJg4=; b=W6MlUfFDOvi0IHiAZ047SvSX8rF3o/4gTaVNIRIRrMmbNLd1boMDgavdNcY9RnVj/l x57gGuySxV/5N0yn8n8yc3AzXBg5Um7/iP2/e+EtbryxbRNDNHiX5YAH0tadxIETB0u+ YXSyb2bWRIxirWDBsqlUFawJyW3aCQjdfNVEERu1/MLK+TEh7tgTd9/7+3PJxpnd9iAg wwaVqlLOqu+JZoa2YC4R+Szz4Pgu8RcIL4g4/HeD3EcY+e/DmHJGymymzX0ATuMb+/U2 +3XlfqDNKQb+LY6MuzlFBmdEhFvDll4djnPI1I9/a+qUA3L5aIuQlZMAjHlu7/UWZy2g wjJA== X-Received: by 10.194.62.167 with SMTP id z7mr174065781wjr.106.1426973182428; Sat, 21 Mar 2015 14:26:22 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id gd6sm3987639wib.17.2015.03.21.14.26.21 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Mar 2015 14:26:21 -0700 (PDT) Message-ID: <550DE1FB.2060409@yandex.ru> Date: Sat, 21 Mar 2015 23:26:19 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> <83zj76rtbn.fsf@gnu.org> In-Reply-To: <83zj76rtbn.fsf@gnu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/21/2015 10:25 PM, Eli Zaretskii wrote: > So each keypress you need to encode the whole buffer, including the > last keypress and all those before it? Pretty much. Ycmd server uses caching heavily, so it's not bothered by the frequent requests. And when extracting it from the YCM Vim package, the author measured the transport overhead, saw it's negligible, and went with the "send everything" approach. Here's the blog post about it: https://plus.google.com/+StrahinjaMarković/posts/Zmr5uf2jCHm (He's saying there the json module used is pure Python; this part he's most likely mistaken about). > I guess I don't really understand why each keypress should trigger > encoding of the whole buffer. It's not necessary, just the recommended workflow. The server can take it: https://github.com/company-mode/company-mode/issues/325#issuecomment-83154084, and this way the suggestions reach the user the soonest. Or course, we can wait until Emacs is idle for a bit, but even so if encoding takes 100ms (never mind 500ms it takes now), that can create visible stutters where they don't have to be, if the user starts typing again in the middle of it. >> (replace-regexp-in-string "x" "z" s1 t t) >> >> - only takes ~3ms. > > Then a series of calls to replace-regexp-in-string, one each for every > one of the "special" characters, should get you close to your goal, > right? No no no. There are no "x" characters in s1. I just wanted to demonstrate that the regexp searching by itself is not a bottleneck, so `skip-chars-forward' isn't really warranted. As long as we're replacing an actual character present in the string, it takes well above 3ms. >> And likewise, after changing them to use `concat' instead of `format', >> both alternative json-encode-string implementations that I have "encode" >> a numbers-only (without newlines) string of the same length in a few >> milliseconds. Again, save for the GC pauses, which can add 30-40ms. > > So does this mean you have your solution? No. An actual buffer has lots of newlines, which need to be encoded. Again, the above is about the speed of the regexp engine. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 21:33:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Drew Adams , Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14269735813779 (code B ref 20154); Sat, 21 Mar 2015 21:33:01 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 21:33:01 +0000 Received: from localhost ([127.0.0.1]:60113 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZR0u-0000ys-Md for submit@debbugs.gnu.org; Sat, 21 Mar 2015 17:33:00 -0400 Received: from mail-we0-f171.google.com ([74.125.82.171]:34659) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZR0s-0000yg-9H for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 17:32:58 -0400 Received: by wegp1 with SMTP id p1so107916310weg.1 for <20154@debbugs.gnu.org>; Sat, 21 Mar 2015 14:32:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=wKpJ6CGfN71DBiO0JfeTboE+UYaHG7kgKpVsGep7Jns=; b=vmVQe1XWzNMM8cuyT+tPqn/BSdhcjsi2SmOip53/3+6/wF7AWNC/jVdIQLMCkXLZkN v6ycfekXEfUXqYZFJ4FuvxcCgnm0mBWv4lYyvSPSAHY3M8sfhhNKCKseRI5g/gOujezh 12dSrAhOIdG8P/yhAzHr+9G3AEPFAFwBNDk7UOi1ygPgGiGkGjhLc7JxqmSSTuuizeTZ eygeUGq1fl7Krq11wqPHjhf0qmRtSINPTHYst+A/vs5PsMdGANo0XgXybirMG9Kfl6L/ lP8j61r9i5bMnVRT4xikcP5/DWgKuvmY6tlK91GyOoRrK8KrTEm0D28IoDFhX/9gUkAj DoTw== X-Received: by 10.180.98.137 with SMTP id ei9mr6879915wib.92.1426973572792; Sat, 21 Mar 2015 14:32:52 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id e2sm12198045wjy.46.2015.03.21.14.32.51 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Mar 2015 14:32:52 -0700 (PDT) Message-ID: <550DE381.1030709@yandex.ru> Date: Sat, 21 Mar 2015 23:32:49 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/21/2015 11:05 PM, Drew Adams wrote: > (FYI, I'm not following this thread.) I will just say that if you want > or need to have something like `json-encode-string' be coded in C for > speed, an alternative might be for the actual code to invoke a Lisp > function when bound to a variable, e.g., `json-encode-string-function'. Rather, `json-encode-function'. But that's solving a different problem, one we'd be lucky to have. > What > would, I think, be too bad would be to make it impossible or difficult > for users to provide their own encoding function (without messing with > C and rebuilding Emacs). Yes, without FFI in Emacs that's pretty useless. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Ivan Shmakov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 22:22:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14269764658441 (code B ref 20154); Sat, 21 Mar 2015 22:22:02 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 22:21:05 +0000 Received: from localhost ([127.0.0.1]:60127 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZRlQ-0002C5-KT for submit@debbugs.gnu.org; Sat, 21 Mar 2015 18:21:04 -0400 Received: from fely.am-1.org ([78.47.74.50]:60886) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZRlN-0002BQ-C1 for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 18:21:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=siamics.net; s=a2013295; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:Sender:References:Subject:To:From; bh=SgBT2RjGPOKyV3dG1gJhZbelKoHJgCZA3H7c78vYzP8=; b=cptk0t0Rij2rsIFTLC0ok619j9VeimTxMiFodbpxSf88iqfp+0IyvTmBkwpMMGHtgmL8xPSenP9JCwoHtYmR6wG8XCRtsqm6JTMIu+ivBHFE7GF8fKQuf8R+XZJ4fNef/sKH/WisbKVOHIlhcYajasDiFy5ZTUwIArCY4/CGAOg=; Received: from [2a02:2560:6d4:26ca::1:1d] (helo=violet.siamics.net) by fely.am-1.org with esmtps (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1YZRlM-0006DN-89 for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 22:21:00 +0000 Received: from localhost ([::1] helo=violet.siamics.net) by violet.siamics.net with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1YZRlF-0002kv-93 for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 05:20:53 +0700 From: Ivan Shmakov References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> Mail-Followup-To: 20154@debbugs.gnu.org Date: Sat, 21 Mar 2015 22:20:51 +0000 In-Reply-To: <550DDE25.1090000@yandex.ru> (Dmitry Gutov's message of "Sat, 21 Mar 2015 23:09:57 +0200") Message-ID: <87a8z6vvp8.fsf@violet.siamics.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.7 (/) >>>>> Dmitry Gutov writes: [=E2=80=A6] > Here's the updated definition (by the way, `json-special-chars' is > still needed, to convert ?\n to ?n, and so on, and the performance > hit is negligible). Perhaps a plain vector may fit there? > (defun json-encode-string-1 (string) > "Return a JSON representation of STRING." > (with-temp-buffer > (insert string) > (goto-char (point-min)) > ;; Skip over ASCIIish printable characters. > (while (re-search-forward "\\([\"\\/\b\f\n\r\t]\\)\\|[^ -~]" nil t) > (let ((c (char-before))) > (delete-region (1- (point)) (point)) > (if (match-beginning 1) > ;; Special JSON character (\n, \r, etc.). > (insert "\\" (car (rassoc c json-special-chars))) > ;; Fallback: UCS code point in \uNNNN form. > (insert (format "\\u%04x" c))))) > (concat "\"" (buffer-string) "\""))) FWIW, using replace-match in the loop seem to speed up the routine by another few percents. (while (re-search-forward "\\([\"\\/\b\f\n\r\t]\\)\\|[^ -~]" nil t) (let ((c (char-before))) (replace-match (if (match-beginning 1) ;; Special JSON character (\n, \r, etc.). (string ?\\ (car (rassq c json-special-chars))) ;; Fallback: UCS code point in \uNNNN form. (format "\\u%04x" c)) t t))) [=E2=80=A6] --=20 FSF associate member #7257 http://boycottsystemd.org/ =E2=80=A6 3013 B6A0= 230E 334A From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 21 Mar 2015 23:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 20154@debbugs.gnu.org Cc: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142698102515073 (code B ref 20154); Sat, 21 Mar 2015 23:38:02 +0000 Received: (at 20154) by debbugs.gnu.org; 21 Mar 2015 23:37:05 +0000 Received: from localhost ([127.0.0.1]:60141 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZSwz-0003v3-83 for submit@debbugs.gnu.org; Sat, 21 Mar 2015 19:37:05 -0400 Received: from mail-wg0-f45.google.com ([74.125.82.45]:35482) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZSwx-0003uZ-Fg for 20154@debbugs.gnu.org; Sat, 21 Mar 2015 19:37:03 -0400 Received: by wgdm6 with SMTP id m6so117561371wgd.2 for <20154@debbugs.gnu.org>; Sat, 21 Mar 2015 16:36:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=vk68D52HyfT0cElfPL3LtZ7Dgq5wPkoJmLQ1k0s/9ro=; b=YpGZqpVxCG9gbIIbgYpBYp0BgaU5ikHbaoaU7cPrwUwU+FhsegFMHY764QhG2e7LFI RKsX21aOR7bVacwSVv14+OyiswoWsTfmhTuG50ouLcl1CMVJELMzJrbp2VmDG4d79iTi X/fZTsybQph1YXJCu3g9oYv4suZxEFZdHOrRJ95TCbd5XFit6urF/GPzvgbRPUAIG/ZH r4UebOHQGM/2DHJQo2MRtwuVN3KMrbVAMAgDa49Lm7i/q2rBrVK90uFf4BR97YEnlYq/ tr0qt6g9VQRbnfhdzCtmAKsrp5ViH7jq/rEEOFOEUN2LgPYEwfOBkjgdNso8jf+ZtvYw y3cQ== X-Received: by 10.194.57.206 with SMTP id k14mr175653690wjq.1.1426981017721; Sat, 21 Mar 2015 16:36:57 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id kr5sm12584415wjc.1.2015.03.21.16.36.56 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Mar 2015 16:36:57 -0700 (PDT) Message-ID: <550E0097.9000500@yandex.ru> Date: Sun, 22 Mar 2015 01:36:55 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <87a8z6vvp8.fsf@violet.siamics.net> In-Reply-To: <87a8z6vvp8.fsf@violet.siamics.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/22/2015 12:20 AM, Ivan Shmakov wrote: > Perhaps a plain vector may fit there? Where? > FWIW, using replace-match in the loop seem to speed up the > routine by another few percents. Indeed, it seems so, counter to Eli's advice earlier. Just by a bit. Anyway, the small boost is nice to have, but the buffer-based implementation is actually worse than the current one on small strings (because of `with-temp-buffer'). So I don't think we can simply replace it. A fast `replace-regexp-in-string' implementation would make that possible. P.S. Please keep the discussion participants in Cc, even if you prefer not to receive the copy email. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 14:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14270359344766 (code B ref 20154); Sun, 22 Mar 2015 14:53:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 14:52:14 +0000 Received: from localhost ([127.0.0.1]:60756 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZhEc-0001Eo-8J for submit@debbugs.gnu.org; Sun, 22 Mar 2015 10:52:14 -0400 Received: from mail-wg0-f54.google.com ([74.125.82.54]:35884) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZhEa-0001EZ-5N for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 10:52:12 -0400 Received: by wgra20 with SMTP id a20so126806149wgr.3 for <20154@debbugs.gnu.org>; Sun, 22 Mar 2015 07:52:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=Hhq8a9JCba7HhKpR7XAneiyScVWWyKSUXJ6JiUL32Pw=; b=duRNrsDrGlzjPlGRzQlo9t/tYGqSyvRJ/iiy+AfdLB3eusTcgutpdhzsCXp3+iueE/ 3cg7l3qNw6rs6hwdVigIY9/umgSqTiPCe8dJxBpmpl6yQdP/eeX9vYXOLQfbQAhLb+Vc 9YbQ/5QDumff0KIc3LgqAMFr0EsK9b95HVU5lBi5JXVBYXNSQSYui35TO/P7wzwCH14j 8pXuVfe1FRYiOYvPBj8iv68mnBRuU8SLGWCZVYGsb2zSGvwRFpFNkBBDBtMwbtlAkaM/ /i4gxog3jb7C9La3OOhgxEzzfBTaj9CyA8Ag7H5Oq9ypRbFIX0L0ZpaYrc8Bv/dONNAU PyYA== X-Received: by 10.195.12.97 with SMTP id ep1mr180679163wjd.134.1427035926506; Sun, 22 Mar 2015 07:52:06 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id vh8sm15161755wjc.12.2015.03.22.07.52.05 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Mar 2015 07:52:06 -0700 (PDT) Message-ID: <550ED713.2010201@yandex.ru> Date: Sun, 22 Mar 2015 16:52:03 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> In-Reply-To: <550DDE25.1090000@yandex.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Here's the version I've arrived at: (defun json-encode-string-3 (string) "Return a JSON representation of STRING." ;; Reimplement the meat of `replace-regexp-in-string', for ;; performance (bug#20154). (let ((l (length string)) (start 0) (res (list "\""))) ;; Skip over ASCIIish printable characters. (while (string-match "[\"\\/\b\f\n\r\t]\\|[^ -~]" string start) (let* ((mb (match-beginning 0)) (c (aref string mb)) (special (rassoc c json-special-chars))) (push (substring string start mb) res) (push (if special ;; Special JSON character (\n, \r, etc.). (string ?\\ (car special)) ;; Fallback: UCS code point in \uNNNN form. (format "\\u%04x" c)) res) (setq start (1+ mb)))) (push (substring string start l) res) (push "\"" res) (apply #'concat (nreverse res)))) A bit slower than the temp-buffer version (90ms vs 80ms), but consistently faster than the current version, even on small strings. Probably the best we can do in Lisp. If no one has any better ideas, I'm going to install it. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Ivan Shmakov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 16:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 20154@debbugs.gnu.org, Dmitry Gutov Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704096112505 (code B ref 20154); Sun, 22 Mar 2015 16:16:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 16:16:01 +0000 Received: from localhost ([127.0.0.1]:60808 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZiXg-0003Fc-KU for submit@debbugs.gnu.org; Sun, 22 Mar 2015 12:16:00 -0400 Received: from fely.am-1.org ([78.47.74.50]:60970) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZiXe-0003FR-Aq for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 12:15:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=siamics.net; s=a2013295; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:Sender:References:Subject:To:From; bh=9dYFOJ4BZ/9VcRBeEPSfwyUgqKep/LuIURX09JLu3+4=; b=LHwmeNLPiWRfKCdGNm+tBPyangtNAVCnfO4xJsEjkHqyWgCPkgY04nAuEJ7fv6iYYpc3bAGquKas9LxJpxkg/1GyTnnr59sF/Z1gqgmFZgbrK6J5usFsCYmX4gXcWK4x/m0R1OickrZ5vfuCdg4ezuPdbeRLxWSOo5rX7Xu7ZYc=; Received: from [2a02:2560:6d4:26ca::1:1d] (helo=violet.siamics.net) by fely.am-1.org with esmtps (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1YZiXc-000747-Ns; Sun, 22 Mar 2015 16:15:56 +0000 Received: from localhost ([::1] helo=violet.siamics.net) by violet.siamics.net with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1YZiXV-0005Ej-Gp; Sun, 22 Mar 2015 23:15:49 +0700 From: Ivan Shmakov References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <550E0097.9000500@yandex.ru> <550ED713.2010201@yandex.ru> Mail-Followup-To: 20154@debbugs.gnu.org, Dmitry Gutov Date: Sun, 22 Mar 2015 16:15:48 +0000 In-Reply-To: <550ED713.2010201@yandex.ru> (Dmitry Gutov's message of "Sun, 22 Mar 2015 16:52:03 +0200") Message-ID: <87384xvwi3.fsf@violet.siamics.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.7 (/) >>>>> Dmitry Gutov writes: > (defun json-encode-string-3 (string) > "Return a JSON representation of STRING." > (let ((l (length string)) > (start 0) > (res (list "\""))) > ;; Skip over ASCIIish printable characters. > (while (string-match "[\"\\/\b\f\n\r\t]\\|[^ -~]" string start) > (let* ((mb (match-beginning 0)) Why not =E2=80=98let=E2=80=99 mb above and use (while (setq mb (string-mat= ch =E2=80=A6)) =E2=80=A6) here (instead of going through match-beginning)? > (c (aref string mb)) > (special (rassoc c json-special-chars))) Is there a specific reason to prefer rassoc over rassq here? > (push (substring string start mb) res) > (push (if special > ;; Special JSON character (\n, \r, etc.). > (string ?\\ (car special)) > ;; Fallback: UCS code point in \uNNNN form. > (format "\\u%04x" c)) > res) > (setq start (1+ mb)))) > (push (substring string start l) res) > (push "\"" res) > (apply #'concat (nreverse res)))) I guess you can (apply #'concat "\"" (substring =E2=80=A6) (nreverse =E2= =80=A6)) just as well, instead of pushing to the list just before getting rid of it. [=E2=80=A6] > Please keep the discussion participants in Cc, even if you prefer not > to receive the copy email. Curiously, per my experience, the practice of Cc:-ing the subscribers tends to be frowned upon when it comes to the lists dedicated to free software. For the reasons I do not know or understand, Emacs lists are apparently an exception, though. I don=E2=80=99t seem to recall any issues with me trying to stick to the custom (of /not/ Cc:-ing) I=E2=80=99ve learned earlier; but if necessary, I surely can remember to Cc: those who so request. --=20 FSF associate member #7257 http://boycottsystemd.org/ =E2=80=A6 3013 B6A0= 230E 334A From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 16:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704285615515 (code B ref 20154); Sun, 22 Mar 2015 16:48:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 16:47:36 +0000 Received: from localhost ([127.0.0.1]:60838 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZj2F-00042B-T4 for submit@debbugs.gnu.org; Sun, 22 Mar 2015 12:47:36 -0400 Received: from mail-wi0-f173.google.com ([209.85.212.173]:36832) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZj2D-00041y-Kt for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 12:47:34 -0400 Received: by wibg7 with SMTP id g7so28302737wib.1 for <20154@debbugs.gnu.org>; Sun, 22 Mar 2015 09:47:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=BUfJRX8NnRIa0qHr4/3Ya28iu7jMFiEk62LLIta5U94=; b=yuMGYaftjEgalfJLWsgfxHXeVoxO/3rLetR4XO+5Ep4I4dqq8G7pJgoT8LaDstKuSM b/GrqFr+0wHXoxzVhDFK3xPJxDq5PiO8VzfXDVTGSTA0XE9b9yf7cWBG2xn2z0inpQ2R gPgjyueQnuyxRr1aeBL0vdB2AuunDb8vZQjfc5TlIW/F6tEbrRo3ZhZr6+CmP6Khr8sd IahG4NH+nhdJJRDr2xVnjoYiaQ9SzTuEwWWZU1kCtQ7nzodczvDOKn7hGAoEENjPjQWl yZVOAZ5T27EtbW1rTuUY+f0l/9xXiy9RXS9n9vv4RydmV9jesjjR4ywRzm/QmjnikXVx hrBg== X-Received: by 10.194.85.46 with SMTP id e14mr172462658wjz.110.1427042847822; Sun, 22 Mar 2015 09:47:27 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id r3sm15553380wjw.7.2015.03.22.09.47.26 for <20154@debbugs.gnu.org> (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Mar 2015 09:47:27 -0700 (PDT) Message-ID: <550EF21C.6060403@yandex.ru> Date: Sun, 22 Mar 2015 18:47:24 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <550E0097.9000500@yandex.ru> <550ED713.2010201@yandex.ru> <87384xvwi3.fsf@violet.siamics.net> In-Reply-To: <87384xvwi3.fsf@violet.siamics.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/22/2015 06:15 PM, Ivan Shmakov wrote: > Why not ‘let’ mb above and use (while (setq mb (string-match …)) > …) here (instead of going through match-beginning)? Good point, thanks. It wins a few milliseconds. > Is there a specific reason to prefer rassoc over rassq here? Not at all. Good call, though no performance improvement. > > (push (substring string start l) res) > > (push "\"" res) > > (apply #'concat (nreverse res)))) > > I guess you can (apply #'concat "\"" (substring …) (nreverse …)) > just as well, instead of pushing to the list just before getting > rid of it. Also good idea, but partially. That gets rid of the initial binding for `res', but those (substring ...) value and quote have to go to the end of the string. We can't put them as the last arguments to `apply'. > > Please keep the discussion participants in Cc, even if you prefer not > > to receive the copy email. > > Curiously, per my experience, the practice of Cc:-ing the > subscribers tends to be frowned upon when it comes to the lists > dedicated to free software. For the reasons I do not know or > understand, Emacs lists are apparently an exception, though. To the best of my knowledge, debbugs only sends a copy to the bug's author, and there's no way to subscribe. So that excludes Eli (although he probably subscribes to all bugs anyway). From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 16:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704303215808 (code B ref 20154); Sun, 22 Mar 2015 16:51:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 16:50:32 +0000 Received: from localhost ([127.0.0.1]:60846 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZj55-00046t-4D for submit@debbugs.gnu.org; Sun, 22 Mar 2015 12:50:31 -0400 Received: from mtaout24.012.net.il ([80.179.55.180]:48627) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZj52-00046c-Ny for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 12:50:29 -0400 Received: from conversion-daemon.mtaout24.012.net.il by mtaout24.012.net.il (HyperSendmail v2007.08) id <0NLM00400GPP6G00@mtaout24.012.net.il> for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 18:42:40 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout24.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLM00OR4H34SI50@mtaout24.012.net.il>; Sun, 22 Mar 2015 18:42:40 +0200 (IST) Date: Sun, 22 Mar 2015 18:50:05 +0200 From: Eli Zaretskii In-reply-to: <550ED713.2010201@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83vbhtrn7m.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <550ED713.2010201@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sun, 22 Mar 2015 16:52:03 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > (defun json-encode-string-3 (string) > "Return a JSON representation of STRING." > ;; Reimplement the meat of `replace-regexp-in-string', for > ;; performance (bug#20154). > (let ((l (length string)) > (start 0) > (res (list "\""))) > ;; Skip over ASCIIish printable characters. > (while (string-match "[\"\\/\b\f\n\r\t]\\|[^ -~]" string start) > (let* ((mb (match-beginning 0)) > (c (aref string mb)) > (special (rassoc c json-special-chars))) Did you try a 'cond' with specific characters instead of calling rassoc every time? The list in json-special-chars is not long, is it? From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 16:52:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ivan Shmakov Cc: dgutov@yandex.ru, 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704308415909 (code B ref 20154); Sun, 22 Mar 2015 16:52:01 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 16:51:24 +0000 Received: from localhost ([127.0.0.1]:60850 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZj5w-00048X-2F for submit@debbugs.gnu.org; Sun, 22 Mar 2015 12:51:24 -0400 Received: from mtaout24.012.net.il ([80.179.55.180]:57078) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZj5u-00048K-MG for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 12:51:23 -0400 Received: from conversion-daemon.mtaout24.012.net.il by mtaout24.012.net.il (HyperSendmail v2007.08) id <0NLM00600H4GB600@mtaout24.012.net.il> for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 18:43:36 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout24.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLM00OXEH4OSI50@mtaout24.012.net.il>; Sun, 22 Mar 2015 18:43:36 +0200 (IST) Date: Sun, 22 Mar 2015 18:51:00 +0200 From: Eli Zaretskii In-reply-to: <87384xvwi3.fsf@violet.siamics.net> X-012-Sender: halo1@inter.net.il Message-id: <83twxdrn63.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <550E0097.9000500@yandex.ru> <550ED713.2010201@yandex.ru> <87384xvwi3.fsf@violet.siamics.net> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Ivan Shmakov > Date: Sun, 22 Mar 2015 16:15:48 +0000 > > > Please keep the discussion participants in Cc, even if you prefer not > > to receive the copy email. > > Curiously, per my experience, the practice of Cc:-ing the > subscribers tends to be frowned upon when it comes to the lists > dedicated to free software. For the reasons I do not know or > understand, Emacs lists are apparently an exception, though. The lists that require you NOT to CC are a nuisance, IMO. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 17:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704421417684 (code B ref 20154); Sun, 22 Mar 2015 17:11:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 17:10:14 +0000 Received: from localhost ([127.0.0.1]:60855 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZjO9-0004b9-VM for submit@debbugs.gnu.org; Sun, 22 Mar 2015 13:10:14 -0400 Received: from mail-we0-f182.google.com ([74.125.82.182]:33084) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZjO8-0004ax-7F for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 13:10:12 -0400 Received: by weop45 with SMTP id p45so120218003weo.0 for <20154@debbugs.gnu.org>; Sun, 22 Mar 2015 10:10:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=r7U2Y7nZA8728O1BMXoNcORfbpI88MHvNDPoHUP9UIM=; b=HJhQx++PchAPnvvwegTFWTkAFsj7NPp7yrH6Lq6ed3ejceqaxavnnb1BOJi6xPlgIi 0Jsnq/1vqxEpPe2lYeTdBY1crIyHCwclrOE0ZSUEAXIwsAQxZj5JHUpvxXaDSIzpsQfp vy1kIFMixNZBxZb8SE4tMyVT8ujHo/nduoSd8/PdXUZUMKUQukKNYU+KjLMwRV8Ekibl W5neQBkWr6ihbxvtcLK91VMnW8OO0Bhx7XYPjD+bBnmuKye653vYC1FdRM/GwO7AOP3p MqqlaS6gSZDom1EEtY8hzB+GSe5cohhwWG/y4No348zipDf+hxUv8H9qjoOyb6T0NOyW Z1+A== X-Received: by 10.180.89.163 with SMTP id bp3mr12654576wib.88.1427044206497; Sun, 22 Mar 2015 10:10:06 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id hd10sm7390773wib.7.2015.03.22.10.10.05 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Mar 2015 10:10:05 -0700 (PDT) Message-ID: <550EF76A.3030506@yandex.ru> Date: Sun, 22 Mar 2015 19:10:02 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <550ED713.2010201@yandex.ru> <83vbhtrn7m.fsf@gnu.org> In-Reply-To: <83vbhtrn7m.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/22/2015 06:50 PM, Eli Zaretskii wrote: > Did you try a 'cond' with specific characters instead of calling > rassoc every time? The list in json-special-chars is not long, is it? Tried that now, but it only made performance worse, by a few ms. And anyway, I wouldn't want to duplicate json-special-chars, that list is also used in another function. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 17:32:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704550919698 (code B ref 20154); Sun, 22 Mar 2015 17:32:01 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 17:31:49 +0000 Received: from localhost ([127.0.0.1]:60859 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZjj3-00057d-8M for submit@debbugs.gnu.org; Sun, 22 Mar 2015 13:31:49 -0400 Received: from mtaout21.012.net.il ([80.179.55.169]:56711) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZjj0-00057N-Tz for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 13:31:47 -0400 Received: from conversion-daemon.a-mtaout21.012.net.il by a-mtaout21.012.net.il (HyperSendmail v2007.08) id <0NLM00700JBB2C00@a-mtaout21.012.net.il> for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 19:31:40 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout21.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLM007YUJCS1A10@a-mtaout21.012.net.il>; Sun, 22 Mar 2015 19:31:40 +0200 (IST) Date: Sun, 22 Mar 2015 19:31:24 +0200 From: Eli Zaretskii In-reply-to: <550DE1FB.2060409@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83r3shrlar.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 8BIT References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> <83zj76rtbn.fsf@gnu.org> <550DE1FB.2060409@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sat, 21 Mar 2015 23:26:19 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > On 03/21/2015 10:25 PM, Eli Zaretskii wrote: > > > So each keypress you need to encode the whole buffer, including the > > last keypress and all those before it? > > Pretty much. Ycmd server uses caching heavily, so it's not bothered by > the frequent requests. And when extracting it from the YCM Vim package, > the author measured the transport overhead, saw it's negligible, and > went with the "send everything" approach. Here's the blog post about it: > > https://plus.google.com/+StrahinjaMarković/posts/Zmr5uf2jCHm > > (He's saying there the json module used is pure Python; this part he's > most likely mistaken about). > > > I guess I don't really understand why each keypress should trigger > > encoding of the whole buffer. > > It's not necessary, just the recommended workflow. The server can take > it: > https://github.com/company-mode/company-mode/issues/325#issuecomment-83154084, > and this way the suggestions reach the user the soonest. I understand why you _send_ everything, but not why you need to _encode_ everything. Why not encode only the new stuff? > >> (replace-regexp-in-string "x" "z" s1 t t) > >> > >> - only takes ~3ms. > > > > Then a series of calls to replace-regexp-in-string, one each for every > > one of the "special" characters, should get you close to your goal, > > right? > > No no no. There are no "x" characters in s1. I know. I meant something like (replace-regexp-in-string "\n" "\\n" s1 t t) (replace-regexp-in-string "\f" "\\f" s1 t t) etc. After all, the list of characters to be encoded is not very long, is it? > > So does this mean you have your solution? > > No. An actual buffer has lots of newlines, which need to be encoded. > Again, the above is about the speed of the regexp engine. But when you've encoded them once, you only need to encode the additions, no? If you can do this incrementally, the amount of work for each keystroke will be much smaller, I think. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 17:44:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704620920787 (code B ref 20154); Sun, 22 Mar 2015 17:44:01 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 17:43:29 +0000 Received: from localhost ([127.0.0.1]:60869 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZjuL-0005PD-DY for submit@debbugs.gnu.org; Sun, 22 Mar 2015 13:43:29 -0400 Received: from mtaout26.012.net.il ([80.179.55.182]:43625) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZjuI-0005Oy-MB for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 13:43:27 -0400 Received: from conversion-daemon.mtaout26.012.net.il by mtaout26.012.net.il (HyperSendmail v2007.08) id <0NLM00900JMJBO00@mtaout26.012.net.il> for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 19:44:14 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout26.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLM00235JXP4A80@mtaout26.012.net.il>; Sun, 22 Mar 2015 19:44:14 +0200 (IST) Date: Sun, 22 Mar 2015 19:43:04 +0200 From: Eli Zaretskii In-reply-to: <550EF21C.6060403@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83k2y9rkrb.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <550E0097.9000500@yandex.ru> <550ED713.2010201@yandex.ru> <87384xvwi3.fsf@violet.siamics.net> <550EF21C.6060403@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sun, 22 Mar 2015 18:47:24 +0200 > From: Dmitry Gutov > > To the best of my knowledge, debbugs only sends a copy to the bug's > author, and there's no way to subscribe. So that excludes Eli (although > he probably subscribes to all bugs anyway). I subscribe to the list, actually. But people who send messages shouldn't rely on that, and not everyone subscribes anyway. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 18:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704802223931 (code B ref 20154); Sun, 22 Mar 2015 18:14:01 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 18:13:42 +0000 Received: from localhost ([127.0.0.1]:60899 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZkNa-0006Dt-6j for submit@debbugs.gnu.org; Sun, 22 Mar 2015 14:13:42 -0400 Received: from mail-we0-f181.google.com ([74.125.82.181]:36822) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZkNX-0006DQ-RN for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 14:13:40 -0400 Received: by wetk59 with SMTP id k59so120974283wet.3 for <20154@debbugs.gnu.org>; Sun, 22 Mar 2015 11:13:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=3/lAJSeN/EJbG5Ejl8dit5vMwv2VqcJr4tmBczTdVP0=; b=Er4PdWayjE6nWy+K+jnnIxS/IOJLDqQ9b79jVci638HdT1epQ0klFIxnDrTmV5uC5R 8XQhjT/5NhKkZ7QeGEgrEs531lIgHfAmuNvw+If+Qe+5AMKs8JyXvKcJ2xYHetvnvPRK GFO2rieGwvyKGJcv/14rWFMMDVk8KQOqneCxajgxJ1WHZMWVRsaLbWTlEh0prDT+0z03 6LDvT2tkBcTqkqrXxU6Tmc9zmycCsQ5MF+I+93uw4j8+BWiT+SlMkWMTg6mTdkXN3+Q2 iWG+o8czW2oY7GIWBqQl0jKfoRGy3WhJrJTuZb/2KJk8Vl9BNz2oYaz4BcyN92sSdENt J0Mg== X-Received: by 10.194.48.12 with SMTP id h12mr183104177wjn.74.1427048014302; Sun, 22 Mar 2015 11:13:34 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id k4sm4734418wiz.0.2015.03.22.11.13.32 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Mar 2015 11:13:33 -0700 (PDT) Message-ID: <550F064A.5030909@yandex.ru> Date: Sun, 22 Mar 2015 20:13:30 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> <83zj76rtbn.fsf@gnu.org> <550DE1FB.2060409@yandex.ru> <83r3shrlar.fsf@gnu.org> In-Reply-To: <83r3shrlar.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/22/2015 07:31 PM, Eli Zaretskii wrote: > I understand why you _send_ everything, but not why you need to > _encode_ everything. Why not encode only the new stuff? That's the protocol. You're welcome to bring the question up with the author, but for now, as already described, there has been no need to complicate it, because Vim compiled with Python support can encode even a large buffer quickly enough. >>> Then a series of calls to replace-regexp-in-string, one each for every >>> one of the "special" characters, should get you close to your goal, >>> right? Actually, that wouldn't work anyway: aside from the special characters, JSON \\u1234 needs to encode any non-ASCII characters. Look at the "Fallback: UCS code point" comment. > I meant something like > > (replace-regexp-in-string "\n" "\\n" s1 t t) > (replace-regexp-in-string "\f" "\\f" s1 t t) > > etc. After all, the list of characters to be encoded is not very > long, is it? One (replace-regexp-in-string "\n" "\\n" s1 t t) call already takes ~100ms, which is more than the latest proposed json-encode-string implementation takes. > But when you've encoded them once, you only need to encode the > additions, no? If you can do this incrementally, the amount of work > for each keystroke will be much smaller, I think. Sure, that's optimizable, with a sufficiently smart server (which ycmd currently isn't), and at the cost of some buffer state tracking and diffing logic. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Glenn Morris Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 18:23:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 20154@debbugs.gnu.org Cc: Dmitry Gutov Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704853025014 (code B ref 20154); Sun, 22 Mar 2015 18:23:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 18:22:10 +0000 Received: from localhost ([127.0.0.1]:60916 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZkVl-0006VN-OM for submit@debbugs.gnu.org; Sun, 22 Mar 2015 14:22:10 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:40010 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZkVj-0006VF-8Q for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 14:22:07 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1YZkVh-0003lH-27; Sun, 22 Mar 2015 14:22:05 -0400 From: Glenn Morris References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <550E0097.9000500@yandex.ru> <550ED713.2010201@yandex.ru> <87384xvwi3.fsf@violet.siamics.net> X-Spook: Marxist monarchist quiche Operation Iraqi Freedom X-Ran: gDusM7||@ub9Z&P~X)&wQ~oT!RDFv%<`HDe"Le}Mdbk(U,1eU(GR*2S'^Br/V:'A+vQFab X-Hue: cyan X-Attribution: GM Date: Sun, 22 Mar 2015 14:22:04 -0400 In-Reply-To: <87384xvwi3.fsf@violet.siamics.net> (Ivan Shmakov's message of "Sun, 22 Mar 2015 16:15:48 +0000") Message-ID: <2zj7451v7.fsf@fencepost.gnu.org> User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Ivan Shmakov wrote: > Curiously, per my experience, the practice of Cc:-ing the > subscribers tends to be frowned upon when it comes to the lists > dedicated to free software. For the reasons I do not know or > understand, Emacs lists are apparently an exception, though. Both statements are contrary to my own experience. > I don't seem to recall any issues with me trying to stick to the > custom (of /not/ Cc:-ing) I've learned earlier; Obviously _you_ won't see any issues. > but if necessary, I surely can remember to Cc: those who so request. People should not have to "request" this. AFAIK the policy on GNU mailing lists has always been "reply-to-all, and do not assume people are subscribed." Works fine for me. Mailman has optional duplicate suppression, and so does any decent mail client (eg Gnus). This is especially true for a bug list, where there should be zero expectation for any correspondent to be subscribed. I'm actually slightly surprised that https://www.gnu.org/prep/maintain/html_node/Mail.html does not mention this. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 18:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704881025443 (code B ref 20154); Sun, 22 Mar 2015 18:27:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 18:26:50 +0000 Received: from localhost ([127.0.0.1]:60920 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZkaH-0006cI-RG for submit@debbugs.gnu.org; Sun, 22 Mar 2015 14:26:50 -0400 Received: from mail-wi0-f181.google.com ([209.85.212.181]:34067) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZkaG-0006c6-6v for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 14:26:48 -0400 Received: by wibg7 with SMTP id g7so22323614wib.1 for <20154@debbugs.gnu.org>; Sun, 22 Mar 2015 11:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=rttDyztl/zmc/RqUg39hqP4xrKLwL4iOVq3S3S0I8f8=; b=ZOf8Cnz4r5qiqaw4MpScMvDtO8tHDpAqSzpW7TrI4mvVBhP9J9z8j3qcM1OPHmJUBx 6z+5W+77CcETFXFv8080EcprKaZTzWvA+cCcvc4b/MgmU0RUgYmE/Mtj1tHEXbhIQ/DJ /wiQDEWz1YuQEVFI611ZPScNEMu+gAj1mI7fUjbGyNkQ1FyIBCXaR20sRSf0tnenD/df zMj3dDS651NayjPSVpUgDje/HPF4xcBUIkNnqRF2GrvY0+MV/ysGGa4yae8d3ZyqqaEO D+xF5ODq1BeVuPcTb/ktbz3H5elM7gzO80E8J40DyIWFGtUEAxxBBcxUOdRMy8o1WH97 H1wA== X-Received: by 10.194.234.40 with SMTP id ub8mr181505209wjc.100.1427048802655; Sun, 22 Mar 2015 11:26:42 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id ax10sm15879684wjc.26.2015.03.22.11.26.41 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Mar 2015 11:26:42 -0700 (PDT) Message-ID: <550F095D.8050007@yandex.ru> Date: Sun, 22 Mar 2015 20:26:37 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> <83zj76rtbn.fsf@gnu.org> <550DE1FB.2060409@yandex.ru> <83r3shrlar.fsf@gnu.org> In-Reply-To: <83r3shrlar.fsf@gnu.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/22/2015 07:31 PM, Eli Zaretskii wrote: > But when you've encoded them once, you only need to encode the > additions, no? If you can do this incrementally, the amount of work > for each keystroke will be much smaller, I think. It seems I've misunderstood you here, sorry. The question of "why encode everything again" comes to down programmer's convenience, and not re-implementing parts of the JSON encoder. At least until `json-encode' has a way to pass an already-encoded string verbatim, how else would you encode an alist like `(("file_data" . ((,full-path . (("contents" . ,file-contents) ("filetypes" . ,file-types))))) ("filepath" . ,full-path) ("line_num" . ,line-num) ("column_num" . ,column-num)) to JSON, except by encoding everything again? From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 18:34:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142704918426075 (code B ref 20154); Sun, 22 Mar 2015 18:34:01 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 18:33:04 +0000 Received: from localhost ([127.0.0.1]:60924 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZkgJ-0006mV-NR for submit@debbugs.gnu.org; Sun, 22 Mar 2015 14:33:03 -0400 Received: from mtaout24.012.net.il ([80.179.55.180]:60270) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZkgH-0006ly-8o for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 14:33:02 -0400 Received: from conversion-daemon.mtaout24.012.net.il by mtaout24.012.net.il (HyperSendmail v2007.08) id <0NLM00300LIRSS00@mtaout24.012.net.il> for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 20:25:13 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout24.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLM00NH6LU18Z60@mtaout24.012.net.il>; Sun, 22 Mar 2015 20:25:13 +0200 (IST) Date: Sun, 22 Mar 2015 20:32:38 +0200 From: Eli Zaretskii In-reply-to: <550F095D.8050007@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83bnjksx15.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> <83zj76rtbn.fsf@gnu.org> <550DE1FB.2060409@yandex.ru> <83r3shrlar.fsf@gnu.org> <550F095D.8050007@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sun, 22 Mar 2015 20:26:37 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > The question of "why encode everything again" comes to down programmer's > convenience, and not re-implementing parts of the JSON encoder. > > At least until `json-encode' has a way to pass an already-encoded string > verbatim, how else would you encode an alist like > > `(("file_data" . > ((,full-path . (("contents" . ,file-contents) > ("filetypes" . ,file-types))))) > ("filepath" . ,full-path) > ("line_num" . ,line-num) > ("column_num" . ,column-num)) > > to JSON, except by encoding everything again? Caveat: I'm probably missing something simple here, so excuse in advance for asking stupid questions. You said you need to encode everything on every keystroke, so I was wondering why you couldn't encode just the new keystroke, and append the result to what you already encoded earlier. Then send everything to the server, as it expects. The problem is in encoding, not in sending. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 19:04:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142705101128945 (code B ref 20154); Sun, 22 Mar 2015 19:04:01 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 19:03:31 +0000 Received: from localhost ([127.0.0.1]:60952 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZl9m-0007Wl-Lg for submit@debbugs.gnu.org; Sun, 22 Mar 2015 15:03:31 -0400 Received: from mail-wi0-f169.google.com ([209.85.212.169]:38865) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZl9k-0007WY-Bl for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 15:03:28 -0400 Received: by wibgn9 with SMTP id gn9so40908429wib.1 for <20154@debbugs.gnu.org>; Sun, 22 Mar 2015 12:03:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=1I2UfkNXC4ALbJXb/OJPq9J2dq0I1VnZiXgugkRqPzo=; b=Ag4H3BDpUmYvbRwn6FzK3sGN41xHxPKRVyW9nVEfbjQyyTvl5owlBaKZGjpRfa5H6j SjaIBHYCc8FiUBbCdrxBoqJ0fp5GYT51VFRwlPm03G4u8RjW8rm4D51ci0HXnZL45/bc uJP3NThH8t+w2PS4CXteZuOoNnJosmCFNhYYR72VweqWcM/kSMFTVPMCSgn1ZteXYbLP r147UtYjcTZB5dLJJFjcH6VQHogAybJw3NsBekGMJFMgZOwtCFUIzUGyfJbuWHssBpQS 8vX4Hn1j2DQkdmvjObM/9+utqe2cBkNvShmjNly/ShPUqAzgH0JozfGfr+RPD0jyJdHK XEKA== X-Received: by 10.194.174.106 with SMTP id br10mr128070633wjc.21.1427051002782; Sun, 22 Mar 2015 12:03:22 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id dj5sm16003572wjb.28.2015.03.22.12.03.21 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Mar 2015 12:03:22 -0700 (PDT) Message-ID: <550F11F7.3010701@yandex.ru> Date: Sun, 22 Mar 2015 21:03:19 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <838ueqvl1o.fsf@gnu.org> <550DCDEE.4090900@yandex.ru> <83zj76rtbn.fsf@gnu.org> <550DE1FB.2060409@yandex.ru> <83r3shrlar.fsf@gnu.org> <550F095D.8050007@yandex.ru> <83bnjksx15.fsf@gnu.org> In-Reply-To: <83bnjksx15.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/22/2015 08:32 PM, Eli Zaretskii wrote: > You said you need to encode everything on every keystroke, so I was That was a simplification: "every keystroke" is an approximate description of frequency. But anyway, not every keystroke inserts a character. Some delete them, some invoke various commands, which may do complex things to buffer contents. > wondering why you couldn't encode just the new keystroke, and append > the result to what you already encoded earlier. > Then send everything > to the server, as it expects. The problem is in encoding, not in > sending. The server expects a certain JSON-encoded structure. You can't just append stuff at the end, that wouldn't be valid JSON. I suppose you could first encode a "skeleton" alist, with some tagged values replacing the values that you'd expect to be big strings, and then substitute the values in before sending. I'm sure you can imagine the (admittedly rare) edge cases when this will break, and it imposes new constraints on the organization of the code: where you could before just pass Emacs structures between functions to be encoded at the end just before sending, now you have to worry about the stuff above. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Ivan Shmakov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 19:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 20154@debbugs.gnu.org, Dmitry Gutov Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142705173230030 (code B ref 20154); Sun, 22 Mar 2015 19:16:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 19:15:32 +0000 Received: from localhost ([127.0.0.1]:60956 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZlLQ-0007oH-2w for submit@debbugs.gnu.org; Sun, 22 Mar 2015 15:15:32 -0400 Received: from fely.am-1.org ([78.47.74.50]:60988) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZlLM-0007o7-Br for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 15:15:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=siamics.net; s=a2013295; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:Sender:References:Subject:To:From; bh=yTlFap2doUxwQV08molA9TMrSneYyXicwqRFw9oj3lU=; b=S4grmek8H4m/T79v2X3/RQsdW5W+0SLKnE0FuopDIC9fKp8b6LwSslf0hDm9QmYqpF872BQ/LiX5/+zKcpyyXmQPRfXN4P0AY+CMSCYSAog8Wjhpz9k3qaZr+OG4eF0NIcAaZdcWDtYL/bwh92AyYOLmn0lMwASV9p+GgWdBtLc=; Received: from [2a02:2560:6d4:26ca::1:1d] (helo=violet.siamics.net) by fely.am-1.org with esmtps (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1YZlLK-0000kF-EP; Sun, 22 Mar 2015 19:15:26 +0000 Received: from localhost ([::1] helo=violet.siamics.net) by violet.siamics.net with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1YZlLD-0005kG-NO; Mon, 23 Mar 2015 02:15:19 +0700 From: Ivan Shmakov References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C9E7B.4060907@yandex.ru> <837fuavkn9.fsf@gnu.org> <550DDE25.1090000@yandex.ru> <550E0097.9000500@yandex.ru> <550ED713.2010201@yandex.ru> <87384xvwi3.fsf@violet.siamics.net> <550EF21C.6060403@yandex.ru> Mail-Followup-To: 20154@debbugs.gnu.org, Dmitry Gutov Date: Sun, 22 Mar 2015 19:15:18 +0000 In-Reply-To: <550EF21C.6060403@yandex.ru> (Dmitry Gutov's message of "Sun, 22 Mar 2015 18:47:24 +0200") Message-ID: <87vbhsvo6x.fsf@violet.siamics.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.7 (/) >>>>> Dmitry Gutov writes: >>>>> On 03/22/2015 06:15 PM, Ivan Shmakov wrote: [=E2=80=A6] >>> (push (substring string start l) res) >>> (push "\"" res) >>> (apply #'concat (nreverse res)))) >> I guess you can (apply #'concat "\"" (substring =E2=80=A6) (nreverse = =E2=80=A6)) >> just as well, instead of pushing to the list just before getting rid >> of it. > Also good idea, but partially. That gets rid of the initial binding > for res', but those (substring ...) value and quote have to go to the > end of the string. We can't put them as the last arguments to > `apply'. Indeed, I=E2=80=99ve misunderstood the code a bit. [=E2=80=A6] >> Curiously, per my experience, the practice of Cc:-ing the >> subscribers tends to be frowned upon when it comes to the lists >> dedicated to free software. For the reasons I do not know or >> understand, Emacs lists are apparently an exception, though. > To the best of my knowledge, debbugs only sends a copy to the bug's > author, Not at all. > and there's no way to subscribe. It=E2=80=99s possible to subscribe to bug-gnu-emacs@. It=E2=80=99s also possible to subscribe to nntp://news.gmane.org/gmane.emacs.bugs/ instead; that way only the messages requested by the user agent (assuming one which does support both mail /and/ news) will actually be transferred. No clutter to the mailbox, too. Moreover, unless I be mistaken, that=E2=80=99s the limitation of the debbugs.gnu.org instance =E2=80=93 the software itself /does/ allow for such subscription. > So that excludes Eli (although he probably subscribes to all bugs > anyway). Yes. --=20 FSF associate member #7257 http://boycottsystemd.org/ =E2=80=A6 3013 B6A0= 230E 334A From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 22 Mar 2015 22:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142706505023758 (code B ref 20154); Sun, 22 Mar 2015 22:58:02 +0000 Received: (at 20154) by debbugs.gnu.org; 22 Mar 2015 22:57:30 +0000 Received: from localhost ([127.0.0.1]:32815 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZooE-0006B8-Gb for submit@debbugs.gnu.org; Sun, 22 Mar 2015 18:57:30 -0400 Received: from mail-wi0-f170.google.com ([209.85.212.170]:33116) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YZooD-0006Aw-3k for 20154@debbugs.gnu.org; Sun, 22 Mar 2015 18:57:29 -0400 Received: by wixw10 with SMTP id w10so24861335wix.0 for <20154@debbugs.gnu.org>; Sun, 22 Mar 2015 15:57:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=A9KMLGzDmibQJAfCInSMYncdAEsN/05BzWMY+cFaCYU=; b=RRNYsMXHYvn4Bu0pP+k88jb/2pPDEElHP54K56eb1Jy0eu/dvxcxWxl3sp4TO5bgL8 5gjfvyjKLWtTE/C5hfeTRBnoWFdL0W7w2x9sygfvdw+rYJxFahcH9UOxfV13PL5pQOIp eWcUCCj1HrTtdR9aHD88IAUA46yhjzUZyPcEdywZ6miHX98NLfw9DiLMePkMCCFlX74h jo0YvT0MLMXtgct07tU6kEYSykNmTzZy1mKBKDmmDDF+NZnPbSJ9CLjsiBq1fo/bzLZY p5bDKbdNZEX3mZDVtMbYE5VPSbKr/0Zh/UumVMTWQfLfowBjVTL9fOyTz+uQlywGrIU/ Vorg== X-Received: by 10.180.103.136 with SMTP id fw8mr14359674wib.46.1427065043443; Sun, 22 Mar 2015 15:57:23 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id n3sm16742149wja.36.2015.03.22.15.57.22 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Mar 2015 15:57:22 -0700 (PDT) Message-ID: <550F48CF.6010002@yandex.ru> Date: Mon, 23 Mar 2015 00:57:19 +0200 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> In-Reply-To: <83wq2bveq6.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/20/2015 06:02 PM, Eli Zaretskii wrote: > I suggest to rewrite json-encode-char, it does a lot of unnecessary > stuff, starting with the call to encode-char (which was needed in > Emacs 22 and before, but no more). Continuing this line of thought, should we also remove the alias for `decode-char', as well as its one use in `json-read-escaped-char'? From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 23 Mar 2015 15:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Reply-To: Eli Zaretskii Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.14271250448985 (code B ref 20154); Mon, 23 Mar 2015 15:38:02 +0000 Received: (at 20154) by debbugs.gnu.org; 23 Mar 2015 15:37:24 +0000 Received: from localhost ([127.0.0.1]:33978 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Ya4Pr-0002Kp-QC for submit@debbugs.gnu.org; Mon, 23 Mar 2015 11:37:24 -0400 Received: from mtaout21.012.net.il ([80.179.55.169]:57852) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Ya4Po-0002KS-Hz for 20154@debbugs.gnu.org; Mon, 23 Mar 2015 11:37:21 -0400 Received: from conversion-daemon.a-mtaout21.012.net.il by a-mtaout21.012.net.il (HyperSendmail v2007.08) id <0NLO00A008EJPF00@a-mtaout21.012.net.il> for 20154@debbugs.gnu.org; Mon, 23 Mar 2015 17:37:14 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout21.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLO00AR88Q1OI30@a-mtaout21.012.net.il>; Mon, 23 Mar 2015 17:37:14 +0200 (IST) Date: Mon, 23 Mar 2015 17:37:00 +0200 From: Eli Zaretskii In-reply-to: <550F48CF.6010002@yandex.ru> X-012-Sender: halo1@inter.net.il Message-id: <83y4mnrahv.fsf@gnu.org> References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550F48CF.6010002@yandex.ru> X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Mon, 23 Mar 2015 00:57:19 +0200 > From: Dmitry Gutov > CC: 20154@debbugs.gnu.org > > On 03/20/2015 06:02 PM, Eli Zaretskii wrote: > > > I suggest to rewrite json-encode-char, it does a lot of unnecessary > > stuff, starting with the call to encode-char (which was needed in > > Emacs 22 and before, but no more). > > Continuing this line of thought, should we also remove the alias for > `decode-char', as well as its one use in `json-read-escaped-char'? Yes, I recommend that. From unknown Mon Jun 23 14:57:46 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.503 (Entity 5.503) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Dmitry Gutov Subject: bug#20154: closed (Re: bug#20154: 25.0.50; json-encode-string is too slow for large strings) Message-ID: References: <5523DC43.5000409@yandex.ru> <86twxf68zk.fsf@yandex.ru> X-Gnu-PR-Message: they-closed 20154 X-Gnu-PR-Package: emacs Reply-To: 20154@debbugs.gnu.org Date: Tue, 07 Apr 2015 13:32:06 +0000 Content-Type: multipart/mixed; boundary="----------=_1428413526-6502-1" This is a multi-part message in MIME format... ------------=_1428413526-6502-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #20154: 25.0.50; json-encode-string is too slow for large strings which was filed against the emacs package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 20154@debbugs.gnu.org. --=20 20154: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D20154 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1428413526-6502-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 20154-done) by debbugs.gnu.org; 7 Apr 2015 13:32:02 +0000 Received: from localhost ([127.0.0.1]:47236 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YfTbl-0001gd-3p for submit@debbugs.gnu.org; Tue, 07 Apr 2015 09:32:01 -0400 Received: from mail-wi0-f179.google.com ([209.85.212.179]:38585) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YfTbg-0001gE-CW for 20154-done@debbugs.gnu.org; Tue, 07 Apr 2015 09:31:57 -0400 Received: by wiun10 with SMTP id n10so18823958wiu.1 for <20154-done@debbugs.gnu.org>; Tue, 07 Apr 2015 06:31:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=DVOolyjUt7/qGK9tSsOyEtCUdfYAeTJccWT3kkiWBlE=; b=j2lHBnHDkkNaemCu7zgOEKIo3k3BymqR06JY9E8qDhy675u6vfiwuE5gQVMBk+F9f0 sxpzuNHvrYCCFI4FOhVPp6ydmuUZWvlAZubReD0KgVc+eRzexe0fTjsbJEnsbu2a4CU4 ael7pO0a7zXx8riEKAmK9QzebOty6tiy4eHeh7BQ/UaA6NHq+TvW9kWn6uPd9R/KV+yV g6pGmJTxF8v96/a+Q2hrhgrNqcBa/G5VOViF7KSNogDapwqBjlyqeQFSAjQ6jI1dOv3T KbTvIABcXe+6+4sRvXQe9Vs9I2xB1J7wzMauAXjDPbGrguVdZb0N0NPdwXjOuDQ8/rxO vFpw== X-Received: by 10.180.74.37 with SMTP id q5mr4694879wiv.59.1428413510849; Tue, 07 Apr 2015 06:31:50 -0700 (PDT) Received: from [192.168.1.3] ([82.102.93.54]) by mx.google.com with ESMTPSA id l3sm11041824wik.16.2015.04.07.06.31.49 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Apr 2015 06:31:50 -0700 (PDT) Message-ID: <5523DC43.5000409@yandex.ru> Date: Tue, 07 Apr 2015 16:31:47 +0300 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 To: Eli Zaretskii Subject: Re: bug#20154: 25.0.50; json-encode-string is too slow for large strings References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550F48CF.6010002@yandex.ru> <83y4mnrahv.fsf@gnu.org> In-Reply-To: <83y4mnrahv.fsf@gnu.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 20154-done Cc: 20154-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 03/23/2015 05:37 PM, Eli Zaretskii wrote: > Yes, I recommend that. Thanks. It appears there's not much that can be done in Lisp. The current speed is much better, although it could still use improvement. ------------=_1428413526-6502-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 20 Mar 2015 14:26:35 +0000 Received: from localhost ([127.0.0.1]:59210 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYxsg-0007I1-NK for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:26:35 -0400 Received: from eggs.gnu.org ([208.118.235.92]:37766) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YYxsf-0007Hp-2t for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:26:33 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YYxsY-0001nd-L4 for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:26:27 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_40,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:40474) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YYxsY-0001nZ-IR for submit@debbugs.gnu.org; Fri, 20 Mar 2015 10:26:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39475) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YYxsX-0006s0-3w for bug-gnu-emacs@gnu.org; Fri, 20 Mar 2015 10:26:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YYxsP-0001mR-Qi for bug-gnu-emacs@gnu.org; Fri, 20 Mar 2015 10:26:25 -0400 Received: from mail-wg0-x22c.google.com ([2a00:1450:400c:c00::22c]:33796) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YYxsP-0001m7-JT for bug-gnu-emacs@gnu.org; Fri, 20 Mar 2015 10:26:17 -0400 Received: by wggv3 with SMTP id v3so90849697wgg.1 for ; Fri, 20 Mar 2015 07:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:subject:date:message-id:mime-version:content-type; bh=rbArsYm15+hrChPK5LGAz/BhBJuSPCOyJ6hmmF2uhJI=; b=b1ZGtuG7zHI4PbKiXadhlFtGPIj+4GqdPFLoTP8iD/bkrLUAJmux0aUNcHwi0g01bm /z1ENHiDU8HW4M2D/KnpHVhKLj8Jcq3zwvTjK7L3dNZX0bCJ6nm+V+hm/KFRxD1hb5iN doUpFfFoevYfDNeAf3P7l0CzKi+3VA7tsG6x3XL10WZa+RkpL9FoeZa7k/QJGZtczVlc L968DC9nLa20OMdSDx/m4EplVEPRZbOQnCewQHtxL+bt/vOgWib+A9ZmzPntUqLqD3bd z8SAbHKoLFoemv+Bf0JztNqQnq9E8LlDi9lTAq/kFcbjconhBNrikZ6Bm3c6IGk9AZ/q XF+g== X-Received: by 10.180.202.228 with SMTP id kl4mr5681314wic.66.1426861576108; Fri, 20 Mar 2015 07:26:16 -0700 (PDT) Received: from axl ([82.102.93.54]) by mx.google.com with ESMTPSA id z13sm6570117wjr.44.2015.03.20.07.26.12 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Mar 2015 07:26:15 -0700 (PDT) From: Dmitry Gutov To: bug-gnu-emacs@gnu.org Subject: 25.0.50; json-encode-string is too slow for large strings Date: Fri, 20 Mar 2015 16:26:07 +0200 Message-ID: <86twxf68zk.fsf@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 208.118.235.17 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) A 300Kb string takes 0.5s to encode on my machine. Example: (defvar s (apply #'concat (cl-loop for i from 1 to 30000 collect "0123456789\n"))) (length (json-encode-string s)) For a comparison, the built-in json module in my local Python installation takes only 2ms to do that for the same string. This is important for advanced code completion in general, because JSON is a common transport format, and sending the contents of the current buffer to the server is a common approach to avoid needlessly saving it (and running associated hooks, etc). And in this specific case, our JSON encoding speed is a bottleneck when working with ycmd, the editor-agnostic code completion daemon extracted from a popular Vim package: https://github.com/company-mode/company-mode/issues/325#issuecomment-83120928 I've tried to reimplement this function using `replace-regexp-in-string' or `re-search-forward' with a temp buffer, to minimize the number of concatenations and `json-encode-char' calls in the fast case (all characters are ASCII), but as long as characters that need to be encoded (such as newlines) still occur throughout the contents of the string, the speed improvement is nowhere near the acceptable level. Should it be written in C? In GNU Emacs 25.0.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2) of 2015-03-20 on axl Repository revision: 8142fc97af742e083fb83e4d0470da59b123a467 Windowing system distributor `The X.Org Foundation', version 11.0.11601901 System Description: Ubuntu 14.10 ------------=_1428413526-6502-1-- From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Ted Zlatanov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 20 Apr 2015 22:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: Eli Zaretskii , 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142956844013135 (code B ref 20154); Mon, 20 Apr 2015 22:21:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Apr 2015 22:20:40 +0000 Received: from localhost ([127.0.0.1]:33954 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YkK3S-0003Pj-Ur for submit@debbugs.gnu.org; Mon, 20 Apr 2015 18:20:39 -0400 Received: from mail-qc0-f170.google.com ([209.85.216.170]:34590) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YkK3P-0003PT-Tf for 20154@debbugs.gnu.org; Mon, 20 Apr 2015 18:20:36 -0400 Received: by qcyk17 with SMTP id k17so66906396qcy.1 for <20154@debbugs.gnu.org>; Mon, 20 Apr 2015 15:20:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lifelogs.com; s=google; h=from:to:cc:subject:organization:references:mail-copies-to :gmane-reply-to-list:date:in-reply-to:message-id:user-agent :mime-version:content-type; bh=l0sF2vHDr3P9gD3U8PEO/hosOJbAnoKfyw6eJp5aSV0=; b=VQW2oUQOirFxfZLSASKt+AHz8pCoTXF11EIr88qomyNVf808tK85CZqGMfATvNUuzK uI7DVuNAx74pd27KrlhGZintJT8VgikH4KyjERhDqO5LK4OxK+Oig72cbS8vxpIVXwS1 nz7BTQzHFa7/g4ESxw4qRP6BxRNe8TAd7U9/M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:organization:references :mail-copies-to:gmane-reply-to-list:date:in-reply-to:message-id :user-agent:mime-version:content-type; bh=l0sF2vHDr3P9gD3U8PEO/hosOJbAnoKfyw6eJp5aSV0=; b=f9gClzSJ+bYpt+nB0ZpnJBx+XPKKfv03Mf8pamr7qdKqDhnIkTZDsPajZemy9bpj+N MGeKzGX9Dbkp2oQr4KquRhViG+i7Xi17/0bZx0LF0gqudH9Tn9casO6oWzmyXWEnLlFM vSNEE9xmG/WvwTf8wuJwYHoVSPTPuqbp6znzKbxKJ34cTTEf7Pwk8Ja/BPE08qf+NeEy QnF2vn4jQ5lOQPtm7qVs1+kK2FNK4grX2OghznauHAI1JltD3YND6e7sb4CyS3txvX6k /jFtCJJolFk1vN4BFQh6bgIGO9uLsrghQdpgIbImXg0Ercu9iHRJ4uHbRezbR2UYKVsf z12A== X-Gm-Message-State: ALoCoQkkKemN7w5924b2PpQUsP60f9UbEWZIfvoTYUWqUXbJmApv8r7Of/45+YTc5zQRf+YkqpIf X-Received: by 10.55.22.194 with SMTP id 63mr33051079qkw.3.1429568430353; Mon, 20 Apr 2015 15:20:30 -0700 (PDT) Received: from flea (c-98-229-61-72.hsd1.ma.comcast.net. [98.229.61.72]) by mx.google.com with ESMTPSA id 21sm15499720qhy.1.2015.04.20.15.20.29 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 20 Apr 2015 15:20:29 -0700 (PDT) From: Ted Zlatanov Organization: =?UTF-8?Q?=D0=A2=D0=B5=D0=BE=D0=B4=D0=BE=D1=80_?= =?UTF-8?Q?=D0=97=D0=BB=D0=B0=D1=82=D0=B0=D0=BD=D0=BE=D0=B2?= @ Cienfuegos References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never Gmane-Reply-To-List: yes Date: Mon, 20 Apr 2015 18:20:32 -0400 In-Reply-To: <550C990B.8080505@yandex.ru> (Dmitry Gutov's message of "Sat, 21 Mar 2015 00:02:51 +0200") Message-ID: <871tjemogf.fsf@lifelogs.com> User-Agent: Gnus/5.130012 (Ma Gnus v0.12) Emacs/25.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Sat, 21 Mar 2015 00:02:51 +0200 Dmitry Gutov wrote: DG> Maybe implementing `json-encode-string` itself in C isn't strictly DG> necessary, or even particularly advantageous. Absolutely, implementing it ourselves is dumb. Because it's been *done already* many times, in libjson in particular. DG> How about trying to optimize `replace-match' or DG> `replace-regexp-in-string' (which are the main two approaches we can DG> use to implement `json-encode-string') for the case of large input? I think all the time and effort spent on that would be better spent on the FFI work, which would also enable libyaml integration and many other improvements. Ted From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 20 Apr 2015 22:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Ted Zlatanov Cc: Eli Zaretskii , 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142956972615190 (code B ref 20154); Mon, 20 Apr 2015 22:43:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Apr 2015 22:42:06 +0000 Received: from localhost ([127.0.0.1]:33974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YkKOD-0003wv-OV for submit@debbugs.gnu.org; Mon, 20 Apr 2015 18:42:06 -0400 Received: from mail-wi0-f172.google.com ([209.85.212.172]:36645) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YkKOB-0003wR-Ka for 20154@debbugs.gnu.org; Mon, 20 Apr 2015 18:42:04 -0400 Received: by wizk4 with SMTP id k4so117618250wiz.1 for <20154@debbugs.gnu.org>; Mon, 20 Apr 2015 15:41:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=ldbCZaTtl1wQTKb9xk2w0tXgeFuSkbL0BMYDt1TodGU=; b=Wmpdvb2mnRQxpFy2h9nOCpn604NVuQwDNjUcuPVCMMtEDs3bg4sMM3gWPAlZtrIKhY FwIPsJMeuiCHcayNZS0/nTnXfBpUXvDX41nJLwBycJK0y7ODMtwF7Ew4GrnqrCNXvmRg KcWWULitJRGHQrvktAp8IrH4e3uFOJIvDX7h9N5/LwiMXlUS/03gpV/31tUZc2o/VrHV 7jzV1+in8xJS/V32LaaNOHnGohBd9ZtX44HoSmfppBGDdiLtKB6QV2aOOHXttKjyh4DU itLGTog7L/P+0pDs5hRh+J8795SH7Y26jKrg29msyp7r/zHaR7PKZaWzGM8CQ2WAaSo7 7Wbg== X-Received: by 10.194.201.164 with SMTP id kb4mr34794595wjc.32.1429569717966; Mon, 20 Apr 2015 15:41:57 -0700 (PDT) Received: from [192.168.1.2] ([82.102.93.54]) by mx.google.com with ESMTPSA id wo10sm10020wjb.35.2015.04.20.15.41.56 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Apr 2015 15:41:57 -0700 (PDT) Message-ID: <553580B2.2090001@yandex.ru> Date: Tue, 21 Apr 2015 01:41:54 +0300 From: Dmitry Gutov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:36.0) Gecko/20100101 Thunderbird/36.0 MIME-Version: 1.0 References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <871tjemogf.fsf@lifelogs.com> In-Reply-To: <871tjemogf.fsf@lifelogs.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On 04/21/2015 01:20 AM, Ted Zlatanov wrote: > Absolutely, implementing it ourselves is dumb. Because it's been *done > already* many times, in libjson in particular. Is libjson available on all our target platforms? > I think all the time and effort spent on that would be better spent on > the FFI work, which would also enable libyaml integration and many other > improvements. A faster `replace-regexp-in-string' could also mean a slight speed improvement in many different places. And that project would be like two orders of magnitude simpler than FFI. From unknown Mon Jun 23 14:57:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#20154: 25.0.50; json-encode-string is too slow for large strings Resent-From: Ted Zlatanov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 20 Apr 2015 23:12:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20154 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov Cc: 20154@debbugs.gnu.org Received: via spool by 20154-submit@debbugs.gnu.org id=B20154.142957147523359 (code B ref 20154); Mon, 20 Apr 2015 23:12:02 +0000 Received: (at 20154) by debbugs.gnu.org; 20 Apr 2015 23:11:15 +0000 Received: from localhost ([127.0.0.1]:34005 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YkKqQ-00064f-VH for submit@debbugs.gnu.org; Mon, 20 Apr 2015 19:11:15 -0400 Received: from mail-qk0-f175.google.com ([209.85.220.175]:35904) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YkKqK-00064F-ND for 20154@debbugs.gnu.org; Mon, 20 Apr 2015 19:11:09 -0400 Received: by qku63 with SMTP id 63so200943113qku.3 for <20154@debbugs.gnu.org>; Mon, 20 Apr 2015 16:11:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lifelogs.com; s=google; h=from:to:cc:subject:organization:references:mail-copies-to :gmane-reply-to-list:date:in-reply-to:message-id:user-agent :mime-version:content-type; bh=V8dH7JT1B569A0yWahlHQibUqbit7N9P5VB+Ql1pMtQ=; b=rKdWxKkaM43ZSRJQWTDYEOtO/IXULyHgpYO7Y+4w3QpXTdmkmCcuqogM5Jo8Wgl9yy KFqEnaDwDu6rmbqKgkqGvCWxyJgjIoWTRSCw+jcEQTrrMm7bLVmXnSjEl6rp9OwuWFMc IC6FlHp4WqOz1xy6HpziL1cF9+znTZMqdrNYY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:organization:references :mail-copies-to:gmane-reply-to-list:date:in-reply-to:message-id :user-agent:mime-version:content-type; bh=V8dH7JT1B569A0yWahlHQibUqbit7N9P5VB+Ql1pMtQ=; b=ENbFUc0e9PoSn5JpTvRwZP/6K1iYsVZJvSg22AxQVhGDQvN73l+Hw6CfbNDFL9AY1n uJIlZQ0+AaftyxzoVstN7P3PIRigdLD647CJn0yJ4wcxrEQ9o4EihiK9Mbi+It7rP/rp zNlm4qJWxI6xgChd75OSzzp+w98KzDdUEDWMtpV1BUbz7SF6e9nkUr7yXc7B9c6KiZBx UvtuyLJlHvsRKPP9EMaSrd1uwWiLDyaqAX7VSK7WiTnBY6ktXbPziKPJg1//HO+OpDmW pEhMjpC7Bcx07qfJB5I8NUeU0sdDAnDHm7j6NsjcZSu9cRrqFnbs0km5Wz9W21QR0J6X X+ww== X-Gm-Message-State: ALoCoQlZnkR7YP3ZqYpnozHcxS3VpJYRQylDJ9G5KyZHd4J+a32RlssdX2H/kiQ2VuM843xXDsrL X-Received: by 10.140.98.3 with SMTP id n3mr19916363qge.62.1429571463404; Mon, 20 Apr 2015 16:11:03 -0700 (PDT) Received: from flea (c-98-229-61-72.hsd1.ma.comcast.net. [98.229.61.72]) by mx.google.com with ESMTPSA id n13sm81737qkh.8.2015.04.20.16.11.02 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 20 Apr 2015 16:11:02 -0700 (PDT) From: Ted Zlatanov Organization: =?UTF-8?Q?=D0=A2=D0=B5=D0=BE=D0=B4=D0=BE=D1=80_?= =?UTF-8?Q?=D0=97=D0=BB=D0=B0=D1=82=D0=B0=D0=BD=D0=BE=D0=B2?= @ Cienfuegos References: <86twxf68zk.fsf@yandex.ru> <83384zwxdx.fsf@gnu.org> <550C3218.4000903@yandex.ru> <831tkjww0y.fsf@gnu.org> <550C3AB9.7020403@yandex.ru> <83wq2bveq6.fsf@gnu.org> <550C491A.6000909@yandex.ru> <83siczvcss.fsf@gnu.org> <550C504A.10708@yandex.ru> <83r3sjva0q.fsf@gnu.org> <550C6A06.6040203@yandex.ru> <83fv8zv0b1.fsf@gnu.org> <550C990B.8080505@yandex.ru> <871tjemogf.fsf@lifelogs.com> <553580B2.2090001@yandex.ru> X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6; d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never Gmane-Reply-To-List: yes Date: Mon, 20 Apr 2015 19:11:05 -0400 In-Reply-To: <553580B2.2090001@yandex.ru> (Dmitry Gutov's message of "Tue, 21 Apr 2015 01:41:54 +0300") Message-ID: <87sibul7jq.fsf@lifelogs.com> User-Agent: Gnus/5.130012 (Ma Gnus v0.12) Emacs/25.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) On Tue, 21 Apr 2015 01:41:54 +0300 Dmitry Gutov wrote: DG> On 04/21/2015 01:20 AM, Ted Zlatanov wrote: >> Absolutely, implementing it ourselves is dumb. Because it's been *done >> already* many times, in libjson in particular. DG> Is libjson available on all our target platforms? I think it's available on the free ones, but I'm not positive about *all* of them. It's under the GPL, in any case, and I think in 2015 we can safely assume any platform of strategic interest to the GNU project will have it. Do you know of any counterexamples? Besides the relatively large performance gains from libjson, please don't disregard the significant work already sunk into making sure it is bug-free. >> I think all the time and effort spent on that would be better spent on >> the FFI work, which would also enable libyaml integration and many other >> improvements. DG> A faster `replace-regexp-in-string' could also mean a slight speed DG> improvement in many different places. And that project would be like DG> two orders of magnitude simpler than FFI. I didn't say it wouldn't be simpler to do things as this thread proposes. I am also not saying improvements in `replace-regexp-in-string' are a waste of time. I am specifically talking about making them for the sake of speeding up the encoding of JSON data. Ted