GNU bug report logs - #21093
Web server: response bodies systematically loaded in memory

Previous Next

Full log

View this message in rfc822 format

From: ludo <at> gnu.org (Ludovic Courtès)
To: Andy Wingo <wingo <at> pobox.com>
Cc: guile-devel <at> gnu.org, 21093 <at> debbugs.gnu.org
Subject: bug#21093: Web server: response bodies systematically loaded in memory
Date: Fri, 24 Jun 2016 10:58:21 +0200

Andy Wingo <wingo <at> pobox.com> skribis:

> On Mon 20 Jul 2015 00:10, ludo <at> gnu.org (Ludovic Courtès) writes:
>
>> The ‘sanitize-response’ procedure systematically loads the whole
>> response body in memory, which causes obvious scalability issues (this
>> is in 2.0.11.)
>>
>> In particular, when a request handler returns a port-taking procedure as
>> its second return value, ‘sanitize-request’ will just call that
>> procedure passing it a string output port.
>>
>> This procedure should instead be called from the server implementation’s
>> ‘write’ hook, but that would necessitate an API change.
>>
>> Thoughts?
>
> How would you set the Content-Length?  Just leave it off and set
> Connection: close ?  Set it in the headers perhaps?  Then you have to
> verify later, which I dunno how nice that is.  Maybe it is OK.

I think it could work this way:

  1. By default, provide no ‘Content-Length’ and force chunked encoding
     (so that the recipient can make sure it received everything.)

     This is useful for data generated on the fly.

  2. Provide an optional mechanism allowing the user to specify the
     content length upfront.

     Useful for statically-generated data that cannot fit in memory.

I haven’t thought yet about the actual API (I’ll be happy to do so as
time permits; to be clear, I don’t consider it a 2.0.12 blocker.)

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.