Subject: Re: bug#20154: 25.0.50; json-encode-string is too slow for large strings

From: Eli Zaretskii <[email-masked]>
Date: Fri, 20 Mar 2015 16:34:34 +0200

> From: Dmitry Gutov <[email-masked]>
> Date: Fri, 20 Mar 2015 16:26:07 +0200
> 
> A 300Kb string takes 0.5s to encode on my machine. Example:
> 
> (defvar s (apply #'concat (cl-loop for i from 1 to 30000
>                                    collect "0123456789\n")))
> 
> (length (json-encode-string s))
> 
> For a comparison, the built-in json module in my local Python
> installation takes only 2ms to do that for the same string.
> 
> This is important for advanced code completion in general, because JSON
> is a common transport format, and sending the contents of the current
> buffer to the server is a common approach to avoid needlessly saving it
> (and running associated hooks, etc).
> 
> And in this specific case, our JSON encoding speed is a bottleneck when
> working with ycmd, the editor-agnostic code completion daemon extracted
> from a popular Vim package:
> https://github.com/company-mode/company-mode/issues/325#issuecomment-83120928
> 
> I've tried to reimplement this function using `replace-regexp-in-string'
> or `re-search-forward' with a temp buffer, to minimize the number of
> concatenations and `json-encode-char' calls in the fast case (all
> characters are ASCII), but as long as characters that need to be encoded
> (such as newlines) still occur throughout the contents of the string,
> the speed improvement is nowhere near the acceptable level. Should it be
> written in C?

I suggest to start with a detailed profile of the current
implementation, because only then we will be able to talk
intelligently about what part(s) need(s) to be sped up.

Thanks.