GNU bug report logs - #54723
'guix publish' without '--cache' can send garbled narinfo responses

Previous Next

Full log

View this message in rfc822 format

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Simon Streit <simon <at> netpanic.org>, 54723 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: [bug#54723] [PATCH] Check URI when verifying narinfo validity.
Date: Thu, 14 Apr 2022 12:18:51 +0000

[Message part 1 (text/plain, inline)]

Guillaume Le Vaillant <glv <at> posteo.net> skribis:

> Ludovic Courtès <ludo <at> gnu.org> skribis:
>
>> Woow.  How do you build and run ‘guix publish’?  Is it a distro package
>> or is it coming straight from Guix?  What command-line options are you
>> passing?
>>
>> I’ve never seen this, although we have it running on several servers,
>> notably ci.guix.  I wonder what could cause this.
>>
>> Thanks,
>> Ludo’.
>
> I'm using guix-publish-service-type in the operating-system definition
> to manage the "guix publish" server, using the on-the-fly mode and
> fast Zstandard compression:
>
> (service guix-publish-service-type
>          (guix-publish-configuration
>           (host "0.0.0.0")
>           (port 8080)
>           (compression '(("zstd" 3)))
>           (advertise? #t)))
>
>
> When booting the machine, shepherd starts the server with the following
> command-line options:
>
> /gnu/store/059svbd32i4s0l9s5i7z0krcnl666bjy-guix-1.3.0-24.2fb4304/libexec/guix/guile \ /gnu/store/059svbd32i4s0l9s5i7z0krcnl666bjy-guix-1.3.0-24.2fb4304/bin/guix publish -u guix-publish -p 8080 -C zstd:3 --nar-path=nar --listen=0.0.0.0 --advertise
>
> There's another report about this at <https://issues.guix.gnu.org/53668>
> I had forgotten about, where Simon Streit and Maxim Cournoyer indicate
> that they have seen this issue too.

I tested a few things, and it looks like there is a concurrency issue in
the 'http-write' function of the publish server.
After removing the 'call-with-new-thread' in the part of the function
sending narinfos, I can't reproduce the "broken pipe" error anymore.

--8<---------------cut here---------------start------------->8---
diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm
index 870dfc11e9..e9fad5845a 100644
--- a/guix/scripts/publish.scm
+++ b/guix/scripts/publish.scm
@@ -1002,28 +1002,25 @@ (define compression
            (begin
              (when (keep-alive? response)
                (keep-alive client))
-             (call-with-new-thread
-              (lambda ()
-                (set-thread-name "publish narinfo")
-                (let* ((narinfo
-                        (with-store store
-                          (narinfo-string store (utf8->string body)
-                                          #:nar-path nar-path
-                                          #:compressions compressions)))
-                       (narinfo-bv (string->bytevector narinfo "UTF-8"))
-                       (narinfo-length
-                        (bytevector-length narinfo-bv))
-                       (response (write-response
-                                  (with-content-length response
-                                                       narinfo-length)
-                                  client))
-                       (output (response-port response)))
-                  (configure-socket client)
-                  (put-bytevector output narinfo-bv)
-                  (force-output output)
-                  (unless (keep-alive? response)
-                    (close-port output))
-                  (values)))))
+             (let* ((narinfo
+                     (with-store store
+                       (narinfo-string store (utf8->string body)
+                                       #:nar-path nar-path
+                                       #:compressions compressions)))
+                    (narinfo-bv (string->bytevector narinfo "UTF-8"))
+                    (narinfo-length
+                     (bytevector-length narinfo-bv))
+                    (response (write-response
+                               (with-content-length response
+                                                    narinfo-length)
+                               client))
+                    (output (response-port response)))
+               (configure-socket client)
+               (put-bytevector output narinfo-bv)
+               (force-output output)
+               (unless (keep-alive? response)
+                 (close-port output))
+               (values)))
            (%http-write server client response body))))
     (_
      (match (assoc-ref (response-headers response) 'x-raw-file)
--8<---------------cut here---------------end--------------->8---

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.