GNU bug report logs - #20154
25.0.50; json-encode-string is too slow for large strings

Previous Next

Full log

Message #98 received at 20154 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 20154 <at> debbugs.gnu.org
Subject: Re: bug#20154: 25.0.50;
 json-encode-string is too slow for large strings
Date: Sun, 22 Mar 2015 19:31:24 +0200

> Date: Sat, 21 Mar 2015 23:26:19 +0200
> From: Dmitry Gutov <dgutov <at> yandex.ru>
> CC: 20154 <at> debbugs.gnu.org
> 
> On 03/21/2015 10:25 PM, Eli Zaretskii wrote:
> 
> > So each keypress you need to encode the whole buffer, including the
> > last keypress and all those before it?
> 
> Pretty much. Ycmd server uses caching heavily, so it's not bothered by 
> the frequent requests. And when extracting it from the YCM Vim package, 
> the author measured the transport overhead, saw it's negligible, and 
> went with the "send everything" approach. Here's the blog post about it:
> 
> https://plus.google.com/+StrahinjaMarković/posts/Zmr5uf2jCHm
> 
> (He's saying there the json module used is pure Python; this part he's 
> most likely mistaken about).
> 
> > I guess I don't really understand why each keypress should trigger
> > encoding of the whole buffer.
> 
> It's not necessary, just the recommended workflow. The server can take 
> it: 
> https://github.com/company-mode/company-mode/issues/325#issuecomment-83154084, 
> and this way the suggestions reach the user the soonest.

I understand why you _send_ everything, but not why you need to
_encode_ everything.  Why not encode only the new stuff?

> >> (replace-regexp-in-string "x" "z" s1 t t)
> >>
> >> - only takes ~3ms.
> >
> > Then a series of calls to replace-regexp-in-string, one each for every
> > one of the "special" characters, should get you close to your goal,
> > right?
> 
> No no no. There are no "x" characters in s1.

I know.  I meant something like

  (replace-regexp-in-string "\n" "\\n" s1 t t)
  (replace-regexp-in-string "\f" "\\f" s1 t t)

etc.  After all, the list of characters to be encoded is not very
long, is it?

> > So does this mean you have your solution?
> 
> No. An actual buffer has lots of newlines, which need to be encoded. 
> Again, the above is about the speed of the regexp engine.

But when you've encoded them once, you only need to encode the
additions, no?  If you can do this incrementally, the amount of work
for each keystroke will be much smaller, I think.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.