GNU bug report logs - #46761
28.0.50; Speed up json.el encoding

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 46761 in the body.
You can then email your comments to 46761 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; Speed up json.el encoding
Date: Thu, 25 Feb 2021 01:33:47 +0000

[Message part 1 (text/plain, inline)]

Severity: wishlist
Tags: patch

The attached patch speeds up json-encode by inserting into a buffer
rather than concatenating strings.  It does so backward compatibly by
creating a new json--print-* namespace that mirrors the existing
json-encode-* namespace, cleaning it up a bit and reducing code
duplication in the process.

Using my usual benchmark from bug#40693#89:

  canada.json
  old (1.412693239 96 0.736882091)
  new (1.154423962 32 0.248241551)

  citm_catalog.json
  old (0.676292855 68 0.5285956769999993)
  new (0.306573098 12 0.0965493740000003)

  twitter.json
  old (0.353447016 40 0.28536439900000055)
  new (0.142140227  8 0.05943713899999992)

Note that one of the unit tests depends on the patch to map.el in
bug#46754 in order to pass.

WDYT?

Thanks,

-- 
Basil

[0001-Speed-up-json.el-encoding.patch (text/x-diff, attachment)]

Message #8 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Thu, 25 Feb 2021 18:21:20 +0000

[Message part 1 (text/plain, inline)]

"Basil L. Contovounesios" <contovob <at> tcd.ie> writes:

> The attached patch speeds up json-encode by inserting into a buffer
> rather than concatenating strings.  It does so backward compatibly by
> creating a new json--print-* namespace that mirrors the existing
> json-encode-* namespace, cleaning it up a bit and reducing code
> duplication in the process.
>
> Using my usual benchmark from bug#40693#89:
>
>   canada.json
>   old (1.412693239 96 0.736882091)
>   new (1.154423962 32 0.248241551)
>
>   citm_catalog.json
>   old (0.676292855 68 0.5285956769999993)
>   new (0.306573098 12 0.0965493740000003)
>
>   twitter.json
>   old (0.353447016 40 0.28536439900000055)
>   new (0.142140227  8 0.05943713899999992)

This additional change:

[json.diff (text/x-diff, inline)]

diff --git a/lisp/json.el b/lisp/json.el
index eb655162d3..461f688f9c 100644
--- a/lisp/json.el
+++ b/lisp/json.el
@@ -199,6 +199,8 @@ json--with-output-to-string
      (with-current-buffer standard-output
        ;; This affords decent performance gains.
        (setq-local inhibit-modification-hooks t)
+       ;; Ignore `read-only' property once and for all (bug#43549).
+       (setq-local inhibit-read-only t)
        ,@body)))
 
 (defmacro json--with-indentation (&rest body)
@@ -458,9 +460,9 @@ json-read-string
 (defun json--print-string (string &optional from)
   "Insert a JSON representation of STRING at point.
 FROM is the index of STRING to start from and defaults to 0."
-  (goto-char (prog1 (1+ (point))
-               ;; Strip `read-only' property (bug#43549).
-               (insert ?\" (substring-no-properties string from))))
+  (goto-char (prog1 (1+ (point)) (insert ?\" string)))
+  (set-text-properties (point) (point-max) ())
+  (and from (delete-char from))
   ;; Escape only quotation mark, backslash, and the control
   ;; characters U+0000 to U+001F (RFC 4627, ECMA-404).
   (while (re-search-forward (rx (in ?\" ?\\ cntrl)) nil 'move)

[Message part 3 (text/plain, inline)]

improves things slightly further (slight inconsistencies with the above
are because my laptop's currently on battery power):

  canada.json
  old (1.450930341 96 0.7616264250000002)
  new (1.161926076 32 0.24752529000000045)

  citm_catalog.json
  old (0.686048204 68 0.5394565070000006)
  new (0.267222201  6 0.048179708000000154)

  twitter.json
  old (0.362725099 40 0.2935560630000005)
  new (0.099399607  2 0.01469844000000009)

And yes, I have added a test case for this locally.

-- 
Basil

Message #11 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Thu, 25 Feb 2021 19:36:14 +0000

"Basil L. Contovounesios" <contovob <at> tcd.ie> writes:

> @@ -458,9 +460,9 @@ json-read-string
>  (defun json--print-string (string &optional from)
>    "Insert a JSON representation of STRING at point.
>  FROM is the index of STRING to start from and defaults to 0."
> -  (goto-char (prog1 (1+ (point))
> -               ;; Strip `read-only' property (bug#43549).
> -               (insert ?\" (substring-no-properties string from))))
> +  (goto-char (prog1 (1+ (point)) (insert ?\" string)))
> +  (set-text-properties (point) (point-max) ())
> +  (and from (delete-char from))

AKA:

  (insert ?\")
  (goto-char (prog1 (point) (princ string)))
  (and from (delete-char from))

-- 
Basil

Message #14 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>, 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Sat, 27 Feb 2021 20:51:05 +0200

On 25.02.2021 03:33, Basil L. Contovounesios wrote:
> Severity: wishlist
> Tags: patch
> 
> The attached patch speeds up json-encode by inserting into a buffer
> rather than concatenating strings.  It does so backward compatibly by
> creating a new json--print-* namespace that mirrors the existing
> json-encode-* namespace, cleaning it up a bit and reducing code
> duplication in the process.
> 
> Using my usual benchmark from bug#40693#89:
> 
>    canada.json
>    old (1.412693239 96 0.736882091)
>    new (1.154423962 32 0.248241551)
> 
>    citm_catalog.json
>    old (0.676292855 68 0.5285956769999993)
>    new (0.306573098 12 0.0965493740000003)
> 
>    twitter.json
>    old (0.353447016 40 0.28536439900000055)
>    new (0.142140227  8 0.05943713899999992)
> 
> Note that one of the unit tests depends on the patch to map.el in
> bug#46754 in order to pass.

Looking good.

I'm guessing there is an approximate size where structures smaller than 
that size will get slower to encode because of this change (creating a 
temp buffer and switching to it are not entirely free), but I can't 
think of a use case where this would matter.

Message #17 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Sat, 27 Feb 2021 19:30:59 +0000

Dmitry Gutov <dgutov <at> yandex.ru> writes:

> Looking good.

Thanks.

> I'm guessing there is an approximate size where structures smaller than that
> size will get slower to encode because of this change (creating a temp buffer
> and switching to it are not entirely free),

Of course, e.g. {"a": true}.

> but I can't think of a use case where this would matter.

Agreed, because by the time a single with-temp-buffer becomes the
limiting factor, the encoding time for such small objects is trivially
small.

-- 
Basil

Message #20 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Sat, 27 Feb 2021 22:01:29 +0200

On 27.02.2021 21:30, Basil L. Contovounesios wrote:
>> but I can't think of a use case where this would matter.
> Agreed, because by the time a single with-temp-buffer becomes the
> limiting factor, the encoding time for such small objects is trivially
> small.

I suppose someone somewhere could be encoding lots of small objects in a 
loop. That will likely become slower.

But we can install the patch and then wait for someone to report such 
problem.

Message #23 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Sat, 27 Feb 2021 21:09:02 +0000

Dmitry Gutov <dgutov <at> yandex.ru> writes:

> On 27.02.2021 21:30, Basil L. Contovounesios wrote:
>> Agreed, because by the time a single with-temp-buffer becomes the
>> limiting factor, the encoding time for such small objects is trivially
>> small.
>
> I suppose someone somewhere could be encoding lots of small objects in a
> loop. That will likely become slower.
>
> But we can install the patch and then wait for someone to report such problem.

If only Emacs had some sort of JSON support for such performance
critical applications ;).

-- 
Basil

Message #26 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Sat, 27 Feb 2021 23:10:17 +0200

On 27.02.2021 23:09, Basil L. Contovounesios wrote:
> If only Emacs had some sort of JSON support for such performance
> critical applications;).

Fair enough :-)

Message #29 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Sat, 27 Feb 2021 21:16:32 +0000

Dmitry Gutov <dgutov <at> yandex.ru> writes:

> I'm guessing there is an approximate size where structures smaller than that
> size will get slower to encode because of this change (creating a temp buffer
> and switching to it are not entirely free)

BTW, we could avoid creating a new buffer each time, e.g. as
json-encode-string did previously, but that comes with its own set of
problems that I think outweigh the benefits.

-- 
Basil

Message #32 received at 46761 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: "Basil L. Contovounesios" <contovob <at> tcd.ie>
Cc: 46761 <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Sat, 27 Feb 2021 23:21:15 +0200

On 27.02.2021 23:16, Basil L. Contovounesios wrote:
> BTW, we could avoid creating a new buffer each time, e.g. as
> json-encode-string did previously, but that comes with its own set of
> problems that I think outweigh the benefits.

Yes, let's not try to solve this now.

I just wanted to bring it up in case someone has any directly relevant 
use case in mind.

Message #39 received at 46761-done <at> debbugs.gnu.org (full text, mbox):

From: "Basil L. Contovounesios" <contovob <at> tcd.ie>
To: 46761-done <at> debbugs.gnu.org
Subject: Re: bug#46761: 28.0.50; Speed up json.el encoding
Date: Sat, 06 Mar 2021 18:37:08 +0000

tags 46761 fixed
close 46761 28.1
quit

Now pushed and closing.

Speed up json.el encoding
428339e231 2021-03-06 18:25:44 +0000
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=428339e2316a552713b265193d6648125042cc98

-- 
Basil

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.