GNU bug report logs - #40993
cuirass: Add build products download support.

Previous Next

Full log

View this message in rfc822 format

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <m.othacehe <at> gmail.com>
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>, 40993 <at> debbugs.gnu.org
Subject: [bug#40993] cuirass: Add build products download support.
Date: Fri, 01 May 2020 23:17:09 +0200

Hello!

Mathieu Othacehe <m.othacehe <at> gmail.com> skribis:

> With the (very) hacky patch attached, I fool sanitize-response, by
> sending the file name as a bytevector. This allows me to save gigabytes
> of RAM when downloading disk images.

Yay!  This is similar to what ‘guix publish’ does.  :-)

> From 0c5e91c170639d50d1cc339fa0b0e68ea4fba68c Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <m.othacehe <at> gmail.com>
> Date: Fri, 1 May 2020 15:03:12 +0200
> Subject: [PATCH] cuirass: Use sendfiles instead of raw copies.
>
> * src/cuirass/http.scm (respond-file): Send the file name as an UTF8
> bytevector, instead of the raw file content,
> (respond-gzipped-file): ditto. Also set 'content-disposition header.
> * src/web/server/fiberized.scm (client-loop): Check if 'content-disposition is
> set. If it's the case, assume that the bytevector is the file name, and use
> sendfiles to send it. Otherwise, keep the existing behaviour and send directly
> the received bytevector.


> +(define extend-response (@@ (web server) extend-response))

@@ is evil and it’s not guaranteed to work with Guile 3: the procedure
might be inlined.

But you can use these ‘guix publish’ helper procedures, which rely on
(srfi srfi-9 gnu):

  (define (strip-headers response)
    "Return RESPONSE's headers minus 'Content-Length' and our internal headers."
    (fold alist-delete
          (response-headers response)
          '(content-length x-raw-file x-nar-compression)))

  (define (with-content-length response length)
    "Return RESPONSE with a 'content-length' header set to LENGTH."
    (set-field response (response-headers)
               (alist-cons 'content-length length
                           (strip-headers response))))

> +                      (call-with-input-file file
> +                        (lambda (port)
> +                          (write-response
> +                           (extend-response response 'content-length
> +                                            file-size)
> +                           client)
> +                          (sendfile client port file-size))))

I didn’t look at the other patches, but note that ‘sendfile’ blocks.
Since Cuirass is fiberized, you shouldn’t block a fiber.

‘guix publish’ doesn’t use Fibers but it shouldn’t block either while
sending a nar, so what it does is spawn a new thread for the ‘sendfile’
call.

HTH!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.