From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 04 15:15:35 2015 Received: (at submit) by debbugs.gnu.org; 4 Oct 2015 19:15:35 +0000 Received: from localhost ([127.0.0.1]:54664 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Ziokv-0004O7-Vc for submit@debbugs.gnu.org; Sun, 04 Oct 2015 15:15:34 -0400 Received: from eggs.gnu.org ([208.118.235.92]:53338) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Ziokt-0004Nx-Ca for submit@debbugs.gnu.org; Sun, 04 Oct 2015 15:15:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Ziokr-0004fn-Fo for submit@debbugs.gnu.org; Sun, 04 Oct 2015 15:15:31 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:52522) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ziokr-0004fK-By for submit@debbugs.gnu.org; Sun, 04 Oct 2015 15:15:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35094) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ziokp-0005Sf-LL for bug-gnu-emacs@gnu.org; Sun, 04 Oct 2015 15:15:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Ziokm-0004RA-BW for bug-gnu-emacs@gnu.org; Sun, 04 Oct 2015 15:15:27 -0400 Received: from mail-la0-x22e.google.com ([2a00:1450:4010:c03::22e]:33381) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ziokl-0004Of-VA for bug-gnu-emacs@gnu.org; Sun, 04 Oct 2015 15:15:24 -0400 Received: by lafb9 with SMTP id b9so26302546laf.0 for ; Sun, 04 Oct 2015 12:15:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:mime-version:content-type; bh=gWiFKmcBlXTXBZAg/YBmcJb6iywixtpN3hTL7jHTyWk=; b=qFB6olXvYpe0fTxhN9ytpATfYVPZUkHtXcKYGAKcpt/IAuSAcwQgfCVLY4lMGzVSpp j7rgifOHgVI/8RZQH15XUXdsNcKfHdWspSbg7F2nMnFk4SrckxZa84TzYnCF+uAHQaeA MPIJWQ+0bjWloAH1By/EV1ljm/FuLBpUKIa336hrVtCtT9Vwht7gnFFpX8NGD6pvN5UA /fhbdsU+yc6+BE2ImFwpzbV3oeG5PRSUr2Au1QpDbaS/Bw4RhkfBtgc1R4tFtcAVfoH7 b0q8prbpoJHJLgFs+/JnpxJNdQ5MPaHWxykEzhLmArSX1ZDLqNQx+qPiYdkuhmS/LkUU 9drw== X-Received: by 10.112.184.137 with SMTP id eu9mr9780564lbc.21.1443986122789; Sun, 04 Oct 2015 12:15:22 -0700 (PDT) Received: from x240 (cm-84.210.143.4.getinternet.no. [84.210.143.4]) by smtp.gmail.com with ESMTPSA id ym7sm3464823lbb.13.2015.10.04.12.15.21 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 04 Oct 2015 12:15:22 -0700 (PDT) From: =?utf-8?Q?Simen_Heggest=C3=B8yl?= To: bug-gnu-emacs@gnu.org Subject: [PATCH] Enable sorting of JSON object keys when encoding Date: Sun, 04 Oct 2015 21:15:21 +0200 Message-ID: <87twq6e986.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) When working with JSON data, it is often convenient to be able to prettify the data, and having object keys sorted in a fixed order. This patch adds the variable `json-encoding-object-sort-key' which enables defining a sort key to be used when encoding JSON objects. Additionally, the commands `json-pretty-print-ordered' and `json-pretty-print-buffer-ordered' are added for convenience, providing prettification with alphabetical ordering of object keys. This gets rid of a lot of redundant code from `json-encode-hash-table' and `json-encode-plist' by going via alists, and using the logic of `json-encode-alist' commonly for all the structures. I was in doubt whether to make `json-pretty-print-ordered' and `json-pretty-print-buffer-ordered' their own commands, or if it would be better to provide this functionality by having `json-pretty-print' and `json-pretty-print-buffer' accept prefix arguments. I decided on the former, because I think it makes the commands easier to discover. A proposed patch follows! -- Simen >From bf32702520b17e9bb975c68adb79c0d84e2cbf41 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Simen=20Heggest=C3=B8yl?= Date: Sun, 4 Oct 2015 12:21:57 +0200 Subject: [PATCH] Enable sorting of JSON object keys when encoding * lisp/json.el (json-encoding-object-sort-key): New variable for specifying a sort key for JSON objects during encoding. (json--plist-to-alist): New utility function. (json-encode-hash-table): Re-use `json-encode-alist'. (json-encode-alist): Sort output by `json-encoding-object-sort-key, when set. (json-encode-plist): Re-use `json-encode-alist'. (json-pretty-print-buffer-ordered): New command to pretty print the buffer with object keys sorted alphabetically. (json-pretty-print-ordered): New command to pretty print the region with object keys sorted alphabetically. * test/automated/json-tests.el (test-json-plist-to-alist) (test-json-encode-plist, test-json-encode-hash-table) (test-json-encode-with-sort-key): New tests. * etc/NEWS: Add an entry for the new commands. --- etc/NEWS | 4 +++ lisp/json.el | 77 +++++++++++++++++++++----------------------- test/automated/json-tests.el | 22 +++++++++++++ 3 files changed, 63 insertions(+), 40 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index dbe0de3..062f17a 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -318,6 +318,10 @@ standards. --- *** `json-pretty-print' and `json-pretty-print-buffer' now maintain the ordering of object keys by default. +--- +*** New commands `json-pretty-print-ordered' and +`json-pretty-print-buffer-ordered' pretty prints JSON objects with +object keys sorted alphabetically. ** You can recompute the VC state of a file buffer with `M-x vc-refresh-state' ** Prog mode has some support for multi-mode indentation. diff --git a/lisp/json.el b/lisp/json.el index e2c7cc7..33247bf 100644 --- a/lisp/json.el +++ b/lisp/json.el @@ -52,6 +52,8 @@ ;;; Code: +(require 'map) + ;; Parameters (defvar json-object-type 'alist @@ -100,6 +102,13 @@ this around your call to `json-read' instead of `setq'ing it.") "The default indentation level for encoding. Used only when `json-encoding-pretty-print' is non-nil.") +(defvar json-encoding-object-sort-key nil + "Sort key for JSON object keys during encoding. +If nil, no sorting is performed. Else, JSON object keys are +ordered by the specified sort key during encoding. For instance, +setting this to `string<' will have JSON object keys ordered +alphabetically.") + (defvar json--encoding-current-indentation "\n" "Internally used to keep track of the current indentation level of encoding. Used only when `json-encoding-pretty-print' is non-nil.") @@ -148,6 +157,15 @@ Unlike `reverse', this keeps the property-value pairs intact." (push prop res))) res)) +(defun json--plist-to-alist (plist) + "Return an alist of the property-value pairs in PLIST." + (let (res) + (while plist + (let ((prop (pop plist)) + (val (pop plist))) + (push (cons prop val) res))) + (nreverse res))) + (defmacro json--with-indentation (body) `(let ((json--encoding-current-indentation (if json-encoding-pretty-print @@ -421,32 +439,17 @@ Please see the documentation of `json-object-type' and `json-key-type'." (defun json-encode-hash-table (hash-table) "Return a JSON representation of HASH-TABLE." - (format "{%s%s}" - (json-join - (let (r) - (json--with-indentation - (maphash - (lambda (k v) - (push (format - (if json-encoding-pretty-print - "%s%s: %s" - "%s%s:%s") - json--encoding-current-indentation - (json-encode-key k) - (json-encode v)) - r)) - hash-table)) - r) - json-encoding-separator) - (if (or (not json-encoding-pretty-print) - json-encoding-lisp-style-closings) - "" - json--encoding-current-indentation))) + (json-encode-alist (map-into hash-table 'list))) ;; List encoding (including alists and plists) (defun json-encode-alist (alist) "Return a JSON representation of ALIST." + (when json-encoding-object-sort-key + (setq alist + (sort alist (lambda (a b) + (funcall json-encoding-object-sort-key + (car a) (car b)))))) (format "{%s%s}" (json-join (json--with-indentation @@ -466,25 +469,7 @@ Please see the documentation of `json-object-type' and `json-key-type'." (defun json-encode-plist (plist) "Return a JSON representation of PLIST." - (let (result) - (json--with-indentation - (while plist - (push (concat - json--encoding-current-indentation - (json-encode-key (car plist)) - (if json-encoding-pretty-print - ": " - ":") - (json-encode (cadr plist))) - result) - (setq plist (cddr plist)))) - (concat "{" - (json-join (nreverse result) json-encoding-separator) - (if (and json-encoding-pretty-print - (not json-encoding-lisp-style-closings)) - json--encoding-current-indentation - "") - "}"))) + (json-encode-alist (json--plist-to-alist plist))) (defun json-encode-list (list) "Return a JSON representation of LIST. @@ -622,6 +607,18 @@ Advances point just past JSON object." (txt (delete-and-extract-region begin end))) (insert (json-encode (json-read-from-string txt)))))) +(defun json-pretty-print-buffer-ordered () + "Pretty-print current buffer with object keys ordered." + (interactive) + (let ((json-encoding-object-sort-key 'string<)) + (json-pretty-print-buffer))) + +(defun json-pretty-print-ordered (begin end) + "Pretty-print the region with object keys ordered." + (interactive "r") + (let ((json-encoding-object-sort-key 'string<)) + (json-pretty-print begin end))) + (provide 'json) ;;; json.el ends here diff --git a/test/automated/json-tests.el b/test/automated/json-tests.el index d1b7a2f..cdf85cc 100644 --- a/test/automated/json-tests.el +++ b/test/automated/json-tests.el @@ -28,11 +28,33 @@ (should (equal (json--plist-reverse '(:a 1 :b 2 :c 3)) '(:c 3 :b 2 :a 1)))) +(ert-deftest test-json-plist-to-alist () + (should (equal (json--plist-to-alist '()) '())) + (should (equal (json--plist-to-alist '(:a 1)) '((:a . 1)))) + (should (equal (json--plist-to-alist '(:a 1 :b 2 :c 3)) + '((:a . 1) (:b . 2) (:c . 3))))) + +(ert-deftest test-json-encode-plist () + (let ((plist '(:a 1 :b 2))) + (should (equal (json-encode plist) "{\"a\":1,\"b\":2}")))) + (ert-deftest json-encode-simple-alist () (should (equal (json-encode '((a . 1) (b . 2))) "{\"a\":1,\"b\":2}"))) +(ert-deftest test-json-encode-hash-table () + (let ((hash-table (make-hash-table)) + (json-encoding-object-sort-key 'string<)) + (puthash :a 1 hash-table) + (puthash :b 2 hash-table) + (should (equal (json-encode hash-table) "{\"a\":1,\"b\":2}")))) + +(ert-deftest test-json-encode-with-sort-key () + (let ((alist '((:a . 1) (:b . 2) (:c . 3))) + (json-encoding-object-sort-key 'string>)) + (should (equal (json-encode alist) "{\"c\":3,\"b\":2,\"a\":1}")))) + (ert-deftest json-read-simple-alist () (let ((json-object-type 'alist)) (should (equal (json-read-from-string "{\"a\": 1, \"b\": 2}") -- 2.5.3 From debbugs-submit-bounces@debbugs.gnu.org Sun Nov 08 19:01:30 2015 Received: (at 21616) by debbugs.gnu.org; 9 Nov 2015 00:01:30 +0000 Received: from localhost ([127.0.0.1]:57957 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZvZtq-0001Yl-8T for submit@debbugs.gnu.org; Sun, 08 Nov 2015 19:01:30 -0500 Received: from mail-wi0-f169.google.com ([209.85.212.169]:33420) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZvZto-0001Yd-Sg for 21616@debbugs.gnu.org; Sun, 08 Nov 2015 19:01:29 -0500 Received: by wiby19 with SMTP id y19so8591711wib.0 for <21616@debbugs.gnu.org>; Sun, 08 Nov 2015 16:01:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=E5eE14R0s3ZafOkMnS979YZQXyb/WtnaUtZiiT9/VFA=; b=lOkoUHfu4mBL5ykohrUznxU/64mqaITKi9VSP66rqnDx1T1foTWZcgWT1l80nB+Dwe KADlMfvjE/9+XPX/7gyfexpioRhq7zJBajGYDxLCEKdKGvPLKMtDzqUHziPBHOBxbJpN fLgv2/t2DcmGGgiGK19ir632J3afipUJlutkVHK0WiVncdOz3PEceHkPkqfVVaZjDAmB NI7jh0hkCIS5pNC9kbeouWzVHF0WwdaN8xTv/n7J+Bk8UrMdgicQSSz3xV7hpfx8yEw4 YCrHYmxR27ftFWSoAM5Kgj45TUEzOpaKtk0xA6wy6uVTYSHtfMbHK96XbX0UPySXf9cL Gmzg== X-Received: by 10.194.52.72 with SMTP id r8mr26991974wjo.8.1447027288269; Sun, 08 Nov 2015 16:01:28 -0800 (PST) Received: from [192.168.1.2] ([185.105.175.24]) by smtp.googlemail.com with ESMTPSA id lb2sm12319011wjc.15.2015.11.08.16.01.26 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 08 Nov 2015 16:01:27 -0800 (PST) Subject: Re: bug#21616: [PATCH] Enable sorting of JSON object keys when encoding To: =?UTF-8?Q?Simen_Heggest=c3=b8yl?= , 21616@debbugs.gnu.org References: <87twq6e986.fsf@gmail.com> From: Dmitry Gutov Message-ID: <563FE255.5080506@yandex.ru> Date: Mon, 9 Nov 2015 02:01:25 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:42.0) Gecko/20100101 Thunderbird/42.0 MIME-Version: 1.0 In-Reply-To: <87twq6e986.fsf@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21616 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Hi Simen, On 10/04/2015 10:15 PM, Simen Heggestøyl wrote: > When working with JSON data, it is often convenient to be able to > prettify the data, and having object keys sorted in a fixed order. > This patch adds the variable `json-encoding-object-sort-key' which I think it's a "predicate", not a "key". See the argument names in `sort' and `cl-sort': the latter has a :key keyword argument, but it has different purpose. Call it json-[encode-]key-sort-predicate, maybe? > This gets rid of a lot of redundant code from `json-encode-hash-table' > and `json-encode-plist' by going via alists, and using the logic of > `json-encode-alist' commonly for all the structures. The unification part makes me concerned, again, from the performance standpoint. Did you do any measuring here? > I was in doubt whether to make `json-pretty-print-ordered' and > `json-pretty-print-buffer-ordered' their own commands, or if it would > be better to provide this functionality by having `json-pretty-print' > and `json-pretty-print-buffer' accept prefix arguments. I decided on > the former, because I think it makes the commands easier to discover. Yes, making them separate seems to be more in line with other Emacs commands. From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 11 14:00:00 2015 Received: (at 21616) by debbugs.gnu.org; 11 Nov 2015 19:00:00 +0000 Received: from localhost ([127.0.0.1]:33547 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zwach-00030t-Kk for submit@debbugs.gnu.org; Wed, 11 Nov 2015 14:00:00 -0500 Received: from mail-lb0-f179.google.com ([209.85.217.179]:36792) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZwacM-00030R-EU for 21616@debbugs.gnu.org; Wed, 11 Nov 2015 13:59:58 -0500 Received: by lbblt2 with SMTP id lt2so22181262lbb.3 for <21616@debbugs.gnu.org>; Wed, 11 Nov 2015 10:59:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:subject:to:cc:message-id:in-reply-to:references :mime-version:content-type; bh=/jEzkzok/W0dkb10TRFV2x/yarnluKvyMBEKwi8X5zo=; b=lSb0aYeR+cxsenBV8IXFGUoAY9xeVF1UsdMBOG4mSwff1c/2X3ktYfSCnKCnzeOvoj Vb3VvIe+qFBFe2P0EFn4GLxH2j8zf9PrhIp32JF0uPxlFs2jxkJd9rv/w5ghPp/BYt9k 2y5du9VPXoi1xsbAQJykIxHmwOxBV8OT6AyUDdi77hqABvGJCF4qpzBVAFRgeZx7cwZB pjPdOhs63VOUL006cvcrrZ98oUTBUUBNaGq3r4fXE1lHKBYRpvPnz2eIZB+1DRtwEY6b 3HmbwCp3iqlr0DEjzte3TW9MzuVSTD6aJ8b0DGZ1u5HKRWlWkVS1gCZvgfugEsjaMtZd Bk/g== X-Received: by 10.112.198.234 with SMTP id jf10mr4968385lbc.49.1447268377072; Wed, 11 Nov 2015 10:59:37 -0800 (PST) Received: from [192.168.100.7] (cm-84.210.143.4.getinternet.no. [84.210.143.4]) by smtp.gmail.com with ESMTPSA id m80sm1682794lfm.15.2015.11.11.10.59.35 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Nov 2015 10:59:36 -0800 (PST) Date: Wed, 11 Nov 2015 19:59:33 +0100 From: Simen =?iso-8859-1?q?Heggest=F8yl?= Subject: Re: bug#21616: [PATCH] Enable sorting of JSON object keys when encoding To: Dmitry Gutov Message-Id: <1447268373.11049.0@smtp.gmail.com> In-Reply-To: <563FE255.5080506@yandex.ru> References: <87twq6e986.fsf@gmail.com> <563FE255.5080506@yandex.ru> X-Mailer: geary/0.10.0 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-X111p62ssqJCQOySR+V0" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21616 Cc: 21616@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --=-X111p62ssqJCQOySR+V0 Content-Type: multipart/alternative; boundary="=-0g6Te+qRbGj0MEFpYXmf" --=-0g6Te+qRbGj0MEFpYXmf Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Hello, Dmitry. On Mon, Nov 9, 2015 at 1:01 AM, Dmitry Gutov wrote: > Call it json-[encode-]key-sort-predicate, maybe? Indeed, "predicate" seems more appropriate. I'll change the name. > The unification part makes me concerned, again, from the performance=20 > standpoint. Did you do any measuring here? I didn't before now, but as you feared, the performance became measurably worse when encoding plists and hash-tables. I performed some benchmarks on the same 560K JSON file as before (http://folk.uio.no/simenheg/huge.json), with the following results: With json-object-type set to hash-table: Before the patch: (benchmark-run 100 (json-encode huge)) =E2=87=92 (17.396825399 800 7.954568797999999) After the patch: (benchmark-run 100 (json-encode huge)) =E2=87=92 (19.338006244000002 700 9.664285849000013) With json-object-type set to plist: Before the patch: (benchmark-run 100 (json-encode huge)) =E2=87=92 (15.152158676 1101 7.109026906) After the patch: (benchmark-run 100 (json-encode huge)) =E2=87=92 (18.122579887 777 8.46547749400001) How about keeping the old encoding code as the default, and only do the {hash-table, plist} =E2=86=92 alist transform when the output is to be sort= ed? That keeps new code to a minimum, and the they would need to be transformed to an intermediate structure to be sorted anyway. A patch implementing this suggestion is attached. Here are the same benchmarks with the new patch applied: With json-object-type set to hash-table: (benchmark-run 100 (json-encode huge)) =E2=87=92 (17.229601604 750 8.015517397999995) With json-object-type set to plist: (benchmark-run 100 (json-encode huge)) =E2=87=92 (14.363185863 1101 6.992225689000007) -- Simen = --=-0g6Te+qRbGj0MEFpYXmf Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hello, Dmitry.

On Mon, Nov 9, 2015 at 1:01 AM, Dmi= try Gutov <dgutov@yandex.ru> wrote:
Call it json-[encode-]key-sort-predicate, maybe?

Indeed, "predicate" seems more appro= priate. I'll change the name.

The unification part makes = me concerned, again, from the performance standpoint. Did you do any measur= ing here?

I didn't before now, but as you f= eared, the performance became
measurably worse when encoding plis= ts and hash-tables.

I performed some benchmarks on= the same 560K JSON file as before
(http://folk.uio.no/simenheg/huge.json), with the f= ollowing results:

With json-object-type set to has= h-table:

  Before the patch:
 = (benchmark-run 100 (json-encode huge))
      &nbs= p;=E2=87=92 (17.396825399 800 7.954568797999999)

&= nbsp; After the patch:
  (benchmark-run 100 (json-encode hug= e))
       =E2=87=92 (19.338006244000002 700 = 9.664285849000013)

With json-object-type set to pl= ist:

  Before the patch:
  (be= nchmark-run 100 (json-encode huge))
       = =E2=87=92 (15.152158676 1101 7.109026906)

  A= fter the patch:
  (benchmark-run 100 (json-encode huge))
       =E2=87=92 (18.122579887 777 8.46547749400= 001)

How about keeping the old encoding code as th= e default, and only do the
{hash-table, plist} =E2=86=92 alist tr= ansform when the output is to be sorted?
That keeps new code to a= minimum, and the they would need to be
transformed to an interme= diate structure to be sorted anyway.

A patch imple= menting this suggestion is attached. Here are the same
benchmarks= with the new patch applied:

With json-object-type= set to hash-table:

  (benchmark-run 100 (jso= n-encode huge))
       =E2=87=92 (17.22960160= 4 750 8.015517397999995)

With json-object-type set= to plist:

  (benchmark-run 100 (json-encode = huge))
       =E2=87=92 (14.363185863 1101 6.= 992225689000007)

-- Simen
= --=-0g6Te+qRbGj0MEFpYXmf-- --=-X111p62ssqJCQOySR+V0 Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Enable-sorting-of-JSON-object-keys-when-encoding.patch >From 07ae7e0e3aa2db77a6bd2a9c97ebd34367c76972 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Simen=20Heggest=C3=B8yl?= Date: Sun, 4 Oct 2015 12:21:57 +0200 Subject: [PATCH] Enable sorting of JSON object keys when encoding * lisp/json.el (json-encoding-object-sort-predicate): New variable for specifying a sorting predicate for JSON objects during encoding. (json--plist-to-alist): New utility function. (json-encode-hash-table): Re-use `json-encode-alist' when object keys are to be sorted. (json-encode-alist): Sort output by `json-encoding-object-sort-predicate, when set. (json-encode-plist): Re-use `json-encode-alist' when object keys are to be sorted. (json-pretty-print-buffer-ordered): New command to pretty print the buffer with object keys sorted alphabetically. (json-pretty-print-ordered): New command to pretty print the region with object keys sorted alphabetically. * test/automated/json-tests.el (test-json-plist-to-alist) (test-json-encode-plist, test-json-encode-hash-table) (test-json-encode-alist-with-sort-predicate) (test-json-encode-plist-with-sort-predicate): New tests. * etc/NEWS: Add an entry for the new commands. --- etc/NEWS | 4 ++ lisp/json.el | 117 ++++++++++++++++++++++++++++--------------- test/automated/json-tests.el | 29 +++++++++++ 3 files changed, 111 insertions(+), 39 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index f3df92e..46910b0 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -332,6 +332,10 @@ unlike `bookmark-set' which silently updates an existing bookmark. --- *** `json-pretty-print' and `json-pretty-print-buffer' now maintain the ordering of object keys by default. +--- +*** New commands `json-pretty-print-ordered' and +`json-pretty-print-buffer-ordered' pretty prints JSON objects with +object keys sorted alphabetically. ** You can recompute the VC state of a file buffer with `M-x vc-refresh-state' ** Prog mode has some support for multi-mode indentation. diff --git a/lisp/json.el b/lisp/json.el index 97cf993..0214a3e 100644 --- a/lisp/json.el +++ b/lisp/json.el @@ -52,6 +52,8 @@ ;;; Code: +(require 'map) + ;; Parameters (defvar json-object-type 'alist @@ -111,6 +113,13 @@ json-encoding-lisp-style-closings "If non-nil, ] and } closings will be formatted lisp-style, without indentation.") +(defvar json-encoding-object-sort-predicate nil + "Sorting predicate for JSON object keys during encoding. +If nil, no sorting is performed. Else, JSON object keys are +ordered by the specified sort predicate during encoding. For +instance, setting this to `string<' will have JSON object keys +ordered alphabetically.") + (defvar json-pre-element-read-function nil "Function called (if non-nil) by `json-read-array' and `json-read-object' right before reading a JSON array or object, @@ -159,6 +168,15 @@ json--plist-reverse (push prop res))) res)) +(defun json--plist-to-alist (plist) + "Return an alist of the property-value pairs in PLIST." + (let (res) + (while plist + (let ((prop (pop plist)) + (val (pop plist))) + (push (cons prop val) res))) + (nreverse res))) + (defmacro json--with-indentation (body) `(let ((json--encoding-current-indentation (if json-encoding-pretty-print @@ -492,32 +510,39 @@ json-read-object (defun json-encode-hash-table (hash-table) "Return a JSON representation of HASH-TABLE." - (format "{%s%s}" - (json-join - (let (r) - (json--with-indentation - (maphash - (lambda (k v) - (push (format - (if json-encoding-pretty-print - "%s%s: %s" - "%s%s:%s") - json--encoding-current-indentation - (json-encode-key k) - (json-encode v)) - r)) - hash-table)) - r) - json-encoding-separator) - (if (or (not json-encoding-pretty-print) - json-encoding-lisp-style-closings) - "" - json--encoding-current-indentation))) + (if json-encoding-object-sort-predicate + (json-encode-alist (map-into hash-table 'list)) + (format "{%s%s}" + (json-join + (let (r) + (json--with-indentation + (maphash + (lambda (k v) + (push (format + (if json-encoding-pretty-print + "%s%s: %s" + "%s%s:%s") + json--encoding-current-indentation + (json-encode-key k) + (json-encode v)) + r)) + hash-table)) + r) + json-encoding-separator) + (if (or (not json-encoding-pretty-print) + json-encoding-lisp-style-closings) + "" + json--encoding-current-indentation)))) ;; List encoding (including alists and plists) (defun json-encode-alist (alist) "Return a JSON representation of ALIST." + (when json-encoding-object-sort-predicate + (setq alist + (sort alist (lambda (a b) + (funcall json-encoding-object-sort-predicate + (car a) (car b)))))) (format "{%s%s}" (json-join (json--with-indentation @@ -537,25 +562,27 @@ json-encode-alist (defun json-encode-plist (plist) "Return a JSON representation of PLIST." - (let (result) - (json--with-indentation - (while plist - (push (concat - json--encoding-current-indentation - (json-encode-key (car plist)) - (if json-encoding-pretty-print - ": " - ":") - (json-encode (cadr plist))) - result) - (setq plist (cddr plist)))) - (concat "{" - (json-join (nreverse result) json-encoding-separator) - (if (and json-encoding-pretty-print - (not json-encoding-lisp-style-closings)) + (if json-encoding-object-sort-predicate + (json-encode-alist (json--plist-to-alist plist)) + (let (result) + (json--with-indentation + (while plist + (push (concat json--encoding-current-indentation - "") - "}"))) + (json-encode-key (car plist)) + (if json-encoding-pretty-print + ": " + ":") + (json-encode (cadr plist))) + result) + (setq plist (cddr plist)))) + (concat "{" + (json-join (nreverse result) json-encoding-separator) + (if (and json-encoding-pretty-print + (not json-encoding-lisp-style-closings)) + json--encoding-current-indentation + "") + "}")))) (defun json-encode-list (list) "Return a JSON representation of LIST. @@ -698,6 +725,18 @@ json-pretty-print (txt (delete-and-extract-region begin end))) (insert (json-encode (json-read-from-string txt)))))) +(defun json-pretty-print-buffer-ordered () + "Pretty-print current buffer with object keys ordered." + (interactive) + (let ((json-encoding-object-sort-predicate 'string<)) + (json-pretty-print-buffer))) + +(defun json-pretty-print-ordered (begin end) + "Pretty-print the region with object keys ordered." + (interactive "r") + (let ((json-encoding-object-sort-predicate 'string<)) + (json-pretty-print begin end))) + (provide 'json) ;;; json.el ends here diff --git a/test/automated/json-tests.el b/test/automated/json-tests.el index fa1f548..8f0cd6f 100644 --- a/test/automated/json-tests.el +++ b/test/automated/json-tests.el @@ -28,11 +28,40 @@ (should (equal (json--plist-reverse '(:a 1 :b 2 :c 3)) '(:c 3 :b 2 :a 1)))) +(ert-deftest test-json-plist-to-alist () + (should (equal (json--plist-to-alist '()) '())) + (should (equal (json--plist-to-alist '(:a 1)) '((:a . 1)))) + (should (equal (json--plist-to-alist '(:a 1 :b 2 :c 3)) + '((:a . 1) (:b . 2) (:c . 3))))) + +(ert-deftest test-json-encode-plist () + (let ((plist '(:a 1 :b 2))) + (should (equal (json-encode plist) "{\"a\":1,\"b\":2}")))) + (ert-deftest json-encode-simple-alist () (should (equal (json-encode '((a . 1) (b . 2))) "{\"a\":1,\"b\":2}"))) +(ert-deftest test-json-encode-hash-table () + (let ((hash-table (make-hash-table)) + (json-encoding-object-sort-predicate 'string<)) + (puthash :a 1 hash-table) + (puthash :b 2 hash-table) + (puthash :c 3 hash-table) + (should (equal (json-encode hash-table) + "{\"a\":1,\"b\":2,\"c\":3}")))) + +(ert-deftest test-json-encode-alist-with-sort-predicate () + (let ((alist '((:c . 3) (:a . 1) (:b . 2))) + (json-encoding-object-sort-predicate 'string<)) + (should (equal (json-encode alist) "{\"a\":1,\"b\":2,\"c\":3}")))) + +(ert-deftest test-json-encode-plist-with-sort-predicate () + (let ((plist '(:c 3 :a 1 :b 2)) + (json-encoding-object-sort-predicate 'string<)) + (should (equal (json-encode plist) "{\"a\":1,\"b\":2,\"c\":3}")))) + (ert-deftest json-read-simple-alist () (let ((json-object-type 'alist)) (should (equal (json-read-from-string "{\"a\": 1, \"b\": 2}") -- 2.6.2 --=-X111p62ssqJCQOySR+V0-- From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 11 21:39:27 2015 Received: (at 21616) by debbugs.gnu.org; 12 Nov 2015 02:39:27 +0000 Received: from localhost ([127.0.0.1]:33808 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZwhnK-0008Ck-TK for submit@debbugs.gnu.org; Wed, 11 Nov 2015 21:39:27 -0500 Received: from mail-wm0-f49.google.com ([74.125.82.49]:34753) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zwhmz-0008CC-Ic for 21616@debbugs.gnu.org; Wed, 11 Nov 2015 21:39:24 -0500 Received: by wmvv187 with SMTP id v187so12219963wmv.1 for <21616@debbugs.gnu.org>; Wed, 11 Nov 2015 18:39:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=caCoQp4ME712Yh+xxxcq4NVqK75ZARcqyFW5bOKYbk4=; b=ulk1hWOX95t27uTaibXf/m5U0RNolFpRHf1yboPtCq/g+/EJJNR0sgpTVCQt7/mO6b DUv3/nsS1lMWQbSUNj9Lxs0+Zs9iCdVUAaGOKIJLNdO3+UAoBtx2JQzvPVSxnmT/EKKs G3EO7KXdzMCD8lYcMuoDmYjB2Kk1L7ud+94ysaVkeVgdfFogj2uOARhKCvqDzMa+PFVI 91DBn0VJvsLng7z9Epy0Yaa3IobHRzhvVl5PfqeVhZl9nsrbujZkqMiHNdP7nJi1xzQ0 Cmj8AtbBGHXkqAAtrSEAb2JJKt6ObMGt6yh63AQs5gAw4JHC25fkzDTVaym3ys4LBk9v 8ePg== X-Received: by 10.28.215.205 with SMTP id o196mr15197956wmg.95.1447295944736; Wed, 11 Nov 2015 18:39:04 -0800 (PST) Received: from [10.9.0.103] (nat.webazilla.com. [78.140.128.228]) by smtp.googlemail.com with ESMTPSA id v192sm11388088wmv.5.2015.11.11.18.39.03 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Nov 2015 18:39:04 -0800 (PST) Subject: Re: bug#21616: [PATCH] Enable sorting of JSON object keys when encoding To: =?UTF-8?Q?Simen_Heggest=c3=b8yl?= References: <87twq6e986.fsf@gmail.com> <563FE255.5080506@yandex.ru> <1447268373.11049.0@smtp.gmail.com> From: Dmitry Gutov Message-ID: <5643FBC6.2000401@yandex.ru> Date: Thu, 12 Nov 2015 04:39:02 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:42.0) Gecko/20100101 Thunderbird/42.0 MIME-Version: 1.0 In-Reply-To: <1447268373.11049.0@smtp.gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21616 Cc: 21616@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) Hi Simen, On 11/11/2015 08:59 PM, Simen Heggestøyl wrote: > How about keeping the old encoding code as the default, and only do the > {hash-table, plist} → alist transform when the output is to be sorted? > That keeps new code to a minimum, and the they would need to be > transformed to an intermediate structure to be sorted anyway. > > A patch implementing this suggestion is attached. Here are the same > benchmarks with the new patch applied: LGTM, please install. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Thu Nov 12 12:37:57 2015 Received: (at 21616-done) by debbugs.gnu.org; 12 Nov 2015 17:37:57 +0000 Received: from localhost ([127.0.0.1]:34943 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zwvoq-0002KM-DW for submit@debbugs.gnu.org; Thu, 12 Nov 2015 12:37:56 -0500 Received: from mail-lf0-f50.google.com ([209.85.215.50]:33609) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZwvoW-0002Jr-9l for 21616-done@debbugs.gnu.org; Thu, 12 Nov 2015 12:37:54 -0500 Received: by lffz63 with SMTP id z63so38512988lff.0 for <21616-done@debbugs.gnu.org>; Thu, 12 Nov 2015 09:37:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:subject:to:cc:message-id:in-reply-to:references :mime-version:content-type; bh=PGSHDEPBa6WdWOHWh4LC31+B8HaJI4I7/YSjjX/UEKo=; b=fzGMEwgol27AeDDBAFqcJtdmGRYOiCzgk6vwd9rOriACOG+DYaC24MH/jT8wH92k0F TDT4RTstAfpN9TJ3XNHl5/I8nrP4XIuFEKMZCJiAnlE5hSFOxevDJ0/QtTNo0yvYpSBs layXdBIsDzZ4ILDJ9X6rC8UHyhZ5qsB58qQce58VY+Yy6Kf9wfqbSpq3Xo8FwDbgdqEA yHX7k01TUuhYu7okJviiKRRyrVpRG1jQP7AlM336TJH874dur+kMmNtUQZHljusQ/Lkw n9lRxqaGpGMcLm1HZMqk4m4cno0Tn+LYFoTbx6VZLV2jvzKQWc/FgeJ+v+J/z37JpI/A S7ng== X-Received: by 10.25.132.147 with SMTP id g141mr8109178lfd.106.1447349855404; Thu, 12 Nov 2015 09:37:35 -0800 (PST) Received: from [192.168.100.7] (cm-84.210.143.4.getinternet.no. [84.210.143.4]) by smtp.gmail.com with ESMTPSA id q1sm2432639lbb.5.2015.11.12.09.37.34 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Nov 2015 09:37:34 -0800 (PST) Date: Thu, 12 Nov 2015 18:37:33 +0100 From: Simen =?iso-8859-1?q?Heggest=F8yl?= Subject: Re: bug#21616: [PATCH] Enable sorting of JSON object keys when encoding To: Dmitry Gutov Message-Id: <1447349853.2400.0@smtp.gmail.com> In-Reply-To: <5643FBC6.2000401@yandex.ru> References: <87twq6e986.fsf@gmail.com> <563FE255.5080506@yandex.ru> <1447268373.11049.0@smtp.gmail.com> <5643FBC6.2000401@yandex.ru> X-Mailer: geary/0.10.0 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=-3kDNtzq+6ZttuzjdkXBi" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 21616-done Cc: 21616-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --=-3kDNtzq+6ZttuzjdkXBi Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Thanks for your time, Dmitry. Installed! On Thu, Nov 12, 2015 at 3:39 AM, Dmitry Gutov wrote: > Hi Simen, >=20 > On 11/11/2015 08:59 PM, Simen Heggest=C3=B8yl wrote: >=20 >> How about keeping the old encoding code as the default, and only do=20 >> the >> {hash-table, plist} =E2=86=92 alist transform when the output is to be=20 >> sorted? >> That keeps new code to a minimum, and the they would need to be >> transformed to an intermediate structure to be sorted anyway. >>=20 >> A patch implementing this suggestion is attached. Here are the same >> benchmarks with the new patch applied: >=20 > LGTM, please install. Thanks. = --=-3kDNtzq+6ZttuzjdkXBi Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Thanks for your time, Dmitry. Installed!

On Thu, Nov 12, 2015 at 3:= 39 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:
Hi Simen, On 11/11/2015 08:59 PM, Simen Heggest=C3=B8yl wrote:
How about keeping the old encoding code as the default, and onl= y do the {hash-table, plist} =E2=86=92 alist transform when the output is to be sort= ed? That keeps new code to a minimum, and the they would need to be transformed to an intermediate structure to be sorted anyway. A patch implementing this suggestion is attached. Here are the same benchmarks with the new patch applied:
LGTM, please install. Thanks.
= --=-3kDNtzq+6ZttuzjdkXBi-- From unknown Sun Jun 15 13:01:22 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 11 Dec 2015 12:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator