From unknown Sun Aug 10 07:29:04 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#42548 <42548@debbugs.gnu.org> To: bug#42548 <42548@debbugs.gnu.org> Subject: Status: Cuirass 504 errors Reply-To: bug#42548 <42548@debbugs.gnu.org> Date: Sun, 10 Aug 2025 14:29:04 +0000 retitle 42548 Cuirass 504 errors reassign 42548 guix submitter 42548 Mathieu Othacehe severity 42548 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 26 12:11:00 2020 Received: (at submit) by debbugs.gnu.org; 26 Jul 2020 16:11:00 +0000 Received: from localhost ([127.0.0.1]:53342 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jzjEq-0002FW-CE for submit@debbugs.gnu.org; Sun, 26 Jul 2020 12:11:00 -0400 Received: from lists.gnu.org ([209.51.188.17]:44652) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jzjEo-0002FP-VD for submit@debbugs.gnu.org; Sun, 26 Jul 2020 12:10:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48816) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jzjEo-0004Tr-R1 for bug-guix@gnu.org; Sun, 26 Jul 2020 12:10:58 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:36957) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jzjEo-0008HV-I1 for bug-guix@gnu.org; Sun, 26 Jul 2020 12:10:58 -0400 Received: from [2a01:cb18:832e:5f00:387f:d83e:9705:d871] (port=38150 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jzjEo-00058j-1I for bug-guix@gnu.org; Sun, 26 Jul 2020 12:10:58 -0400 From: Mathieu Othacehe To: bug-guix@gnu.org Subject: Cuirass 504 errors Date: Sun, 26 Jul 2020 18:10:55 +0200 Message-ID: <87eeoy9rzk.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, Back from holidays, perfect time to fix some Cuirass issues :) The Cuirass web interface frequently serves 504 errors for all requests, requiring a service restart on berlin. Having a look to /var/log/cuirass-web.log it seems that we have indeed multiple things going wrong. A first problem is caused by checkout entries pointing to remove inputs. This should be fix with f71f026a41d8e68e4a7f11ef6e708964594a599c in Cuirass. A second issue is caused when a build product download is started, then aborted. In that case, sendfile throws an exception or enters an endless loop. There's a third issue, but the cause is not clear to me: --8<---------------cut here---------------start------------->8--- Uncaught exception in fiber ##f: In ice-9/boot-9.scm: 1736:10 5 (with-exception-handler _ _ #:unwind? _ # _) In web/server/fiberized.scm: 160:26 4 (_) In ice-9/suspendable-ports.scm: 83:4 3 (write-bytes # #vu8(60 33 ?) ?) In unknown file: 2 (port-write # #vu8(60 33 # ?) ?) In ice-9/boot-9.scm: 1669:16 1 (raise-exception _ #:continuable? _) 1669:16 0 (raise-exception _ #:continuable? _) ice-9/boot-9.scm:1669:16: In procedure raise-exception: --8<---------------cut here---------------end--------------->8--- Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 27 18:12:08 2020 Received: (at 42548) by debbugs.gnu.org; 27 Jul 2020 22:12:08 +0000 Received: from localhost ([127.0.0.1]:55949 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k0BLs-0000Nt-2h for submit@debbugs.gnu.org; Mon, 27 Jul 2020 18:12:08 -0400 Received: from mail-wm1-f45.google.com ([209.85.128.45]:50980) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k0BLn-0000NM-6y for 42548@debbugs.gnu.org; Mon, 27 Jul 2020 18:12:07 -0400 Received: by mail-wm1-f45.google.com with SMTP id c80so15478575wme.0 for <42548@debbugs.gnu.org>; Mon, 27 Jul 2020 15:12:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:in-reply-to:references:date:message-id:mime-version :content-transfer-encoding; bh=6MDsxf1ofj/cx14Zi7DEgt0JlV2/19wYLrScMLZ1LPs=; b=eb6ApsbD8To0hF40NTtKlFs6ZReEg3nO05sKD27IidTdmdFVitD6UoQkph+qLeyc95 AMUHyLTMUvC5SEC3nx7ViyBUiIWpHXHP+c8TEv7T1cTheuBEXBolYJYt2qIlSX7Pw+xt y6o9fb/4EWbmfiReCRpKorbaLUjEQTdJ1EzA9phtwqCUAP/Y3hCOV3L2JIZDVWgeBTEp 4oL2K7oeBgcCMxTsLRxGzBQrQ2kHm1GA2seBMVadDFd02CLSpAGnMfGHczZfFAXbn8zb 8YGr+0wcXgUzbvCL7Hmfs6SfmyoCeC6xG37akbYyujnT1OJQkDU6d/4UdLsrs3QTmXqp KOBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=6MDsxf1ofj/cx14Zi7DEgt0JlV2/19wYLrScMLZ1LPs=; b=ryDrV+NR8oEIqwfVD43k18NhND+3eM6POp1O+EuGBLYzA4yr6n/8Uy/ecVv91YCG2V LcCl9k2ya2I2Pkj75Bn91wGRDkC/s14SaTNag2QUfKWLjiQlIailHDfzDNKUvBq2wrKi aUYFaET6ERcScxgQWAmb9dOpZr4GM5eRg/u9ksxRYwZ/4popxlXDbfj4lazpG3xqK+pb Yh0vM+1htaCxFa6DQvt7ps4DxXIu8wgKML/BxBViQb4WqIMsjwAqmhLVlUydW91I8uJ+ cWif1wWG4YUPhnd1rsgOxaOcKdRpfjNCaw7JbV1BLq9V5UPE7VwFri/kJvhbnmZcl0op egbg== X-Gm-Message-State: AOAM531owZ/R7iF9rmRmhu/gqAMrOA5eslr3jYOD92y9fr8BahNt/XPZ HjcyRMyIJ1C+Kr4XewrvsBVTQOh/ X-Google-Smtp-Source: ABdhPJwjnbcqhB9wz/HgaF+LIKJFYOuPUy44IGUI3nPcoMB+9QTpurYG2bd8YJBHSd1RhOg82tBbjw== X-Received: by 2002:a1c:5986:: with SMTP id n128mr1065784wmb.112.1595887917131; Mon, 27 Jul 2020 15:11:57 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id q4sm1266862wme.31.2020.07.27.15.11.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jul 2020 15:11:56 -0700 (PDT) From: zimoun To: Mathieu Othacehe , 42548@debbugs.gnu.org Subject: Re: bug#42548: Cuirass 504 errors In-Reply-To: <87eeoy9rzk.fsf@gnu.org> References: <87eeoy9rzk.fsf@gnu.org> Date: Tue, 28 Jul 2020 00:11:55 +0200 Message-ID: <86zh7kzjys.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 42548 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Mathieu, On Sun, 26 Jul 2020 at 18:10, Mathieu Othacehe wrote: > A second issue is caused when a build product download is started, then > aborted. In that case, sendfile throws an exception or enters an endless > loop. What do you mean by =E2=80=9Cbuild product download is started, then aborte= d=E2=80=9D? Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 28 03:32:22 2020 Received: (at 42548) by debbugs.gnu.org; 28 Jul 2020 07:32:22 +0000 Received: from localhost ([127.0.0.1]:57098 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k0K62-0001tz-DW for submit@debbugs.gnu.org; Tue, 28 Jul 2020 03:32:22 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43850) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k0K5y-0001tj-D6 for 42548@debbugs.gnu.org; Tue, 28 Jul 2020 03:32:20 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:46188) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0K5t-0002Lg-4t; Tue, 28 Jul 2020 03:32:13 -0400 Received: from [2a01:cb18:832e:5f00:cd85:33f5:211:292b] (port=42188 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k0K5r-0005Bt-VM; Tue, 28 Jul 2020 03:32:12 -0400 From: Mathieu Othacehe To: zimoun Subject: Re: bug#42548: Cuirass 504 errors References: <87eeoy9rzk.fsf@gnu.org> <86zh7kzjys.fsf@gmail.com> Date: Tue, 28 Jul 2020 09:32:09 +0200 In-Reply-To: <86zh7kzjys.fsf@gmail.com> (zimoun's message of "Tue, 28 Jul 2020 00:11:55 +0200") Message-ID: <87sgdckscm.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 42548 Cc: 42548@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hey zimoun, > What do you mean by =E2=80=9Cbuild product download is started, then abor= ted=E2=80=9D? Here I mean clicking on the downloadable image here[1] and then hit "cancel" when the download popup appears, or the abort button later on, when the download is started. Thanks, Mathieu [1]: https://ci.guix.gnu.org/build/3031091/details From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 28 04:49:55 2020 Received: (at 42548) by debbugs.gnu.org; 28 Jul 2020 08:49:55 +0000 Received: from localhost ([127.0.0.1]:57214 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k0LJ5-0003qx-5S for submit@debbugs.gnu.org; Tue, 28 Jul 2020 04:49:55 -0400 Received: from mail-wm1-f54.google.com ([209.85.128.54]:53186) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k0LJ1-0003qe-8z for 42548@debbugs.gnu.org; Tue, 28 Jul 2020 04:49:54 -0400 Received: by mail-wm1-f54.google.com with SMTP id x5so15965180wmi.2 for <42548@debbugs.gnu.org>; Tue, 28 Jul 2020 01:49:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=E0EJUEkrt5s1nPTxGIqDx+t+b70C2k5tjOTuKyR747Y=; b=UKYXCciKZbmAT9FFK91Jv6uCHKTM8zISfAP24nTP8x+zRCf8/+ocJdmbV0E8n7HrLs mnWBVLUftFoS+h/XxY9r7fn1kUAf7zOOnstsVqrJefEJq9Ygn1oG2mBHPY5TOH7GZRzH Q3mbyRo6R2t3HFhbyt+XbLyJKGd17zLaCA/nhOM8xzBVBj6yzjR06wck79wJQSa9XhCg 95pK/xk/nnE7nBivWSBho8FRU4uCfzAvsxp8/I9nhgRTkhGz3k+awKHioMI4w44G+kzI g2ilDAEVrNrXRmtBZeY01rioMsaJpGSNqnDyIBNrlqzEUUAqzUITuH9D7CHZxdRcO2Na GxWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=E0EJUEkrt5s1nPTxGIqDx+t+b70C2k5tjOTuKyR747Y=; b=BsrabKeim5vsCtdpmex9mkCkcfkTfnqeg6vy682mUMve2DpYJLecko0Xo0NNmpMtJk lSQemuQbDnYYQcVWByWVxWIvVIx2DxOAkC1LnmBJ0Bp3/W84DdiR7P0IZtv0pBgEkTjM PtKLXc2dRnGQ+YqaTar9aM/P0Qljy/5JpIUPYwEdw6uqYu7K61ZPzdi6AGMXIamLp6+b j9xEYXPydvZLZiqiphhMZVAK/v9gu/llfMlQLoXdjhTKwztqTVffn4fno5P68ToECbEH KDELGGhzO5kTUiN+YPD3TUmbmfimGdC95oBdUCA21YBmhFQ9rqZANCRd6CId1azlpul3 EoLg== X-Gm-Message-State: AOAM5333yRb6ErEUVOwWB0JOC8jq7y6MIpecriSDD3WaKJLO+6a7oLKi uDW+NpmgafLCjAMzeNO/ScQfngsb X-Google-Smtp-Source: ABdhPJwGg03YcmSv4ON2rQQb2/gkt/U5o19nl3sVBYYnP3V2l2EAZZn7H25NP1WKM/mAEgGuS27UQQ== X-Received: by 2002:a1c:5f41:: with SMTP id t62mr2878094wmb.53.1595926184981; Tue, 28 Jul 2020 01:49:44 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id c136sm3193405wmd.10.2020.07.28.01.49.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jul 2020 01:49:44 -0700 (PDT) From: zimoun To: Mathieu Othacehe Subject: Re: bug#42548: Cuirass 504 errors In-Reply-To: <87sgdckscm.fsf@gnu.org> References: <87eeoy9rzk.fsf@gnu.org> <86zh7kzjys.fsf@gmail.com> <87sgdckscm.fsf@gnu.org> Date: Tue, 28 Jul 2020 10:49:43 +0200 Message-ID: <86d04gyqfs.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 42548 Cc: 42548@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Mathieu, On Tue, 28 Jul 2020 at 09:32, Mathieu Othacehe wrote: > Here I mean clicking on the downloadable image here[1] and then hit > "cancel" when the download popup appears, or the abort button later on, > when the download is started. Ah that=E2=80=99 annoying indeed. :-) And does it mess Cuirass if the connection is lost e.g. down the network? Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 28 10:57:11 2020 Received: (at 42548) by debbugs.gnu.org; 28 Jul 2020 14:57:11 +0000 Received: from localhost ([127.0.0.1]:58797 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k0R2V-0002hR-G9 for submit@debbugs.gnu.org; Tue, 28 Jul 2020 10:57:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55928) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k0R2T-0002hC-TM for 42548@debbugs.gnu.org; Tue, 28 Jul 2020 10:57:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51313) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0R2O-0005G6-DB; Tue, 28 Jul 2020 10:57:04 -0400 Received: from [2a01:cb18:832e:5f00:cd85:33f5:211:292b] (port=45634 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k0R2K-0000Qg-VZ; Tue, 28 Jul 2020 10:57:01 -0400 From: Mathieu Othacehe To: zimoun Subject: Re: bug#42548: Cuirass 504 errors References: <87eeoy9rzk.fsf@gnu.org> <86zh7kzjys.fsf@gmail.com> <87sgdckscm.fsf@gnu.org> <86d04gyqfs.fsf@gmail.com> Date: Tue, 28 Jul 2020 16:56:57 +0200 In-Reply-To: <86d04gyqfs.fsf@gmail.com> (zimoun's message of "Tue, 28 Jul 2020 10:49:43 +0200") Message-ID: <87mu3jheme.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 42548 Cc: 42548@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > And does it mess Cuirass if the connection is lost e.g. down the > network? Not sure yet, I also found this message: --8<---------------cut here---------------start------------->8--- Uncaught exception in fiber ##f: In ice-9/boot-9.scm: 1736:10 5 (with-exception-handler _ _ #:unwind? _ # _) In web/server/fiberized.scm: 160:26 4 (_) In ice-9/suspendable-ports.scm: 83:4 3 (write-bytes # #vu8(60 33 ?) ?) In unknown file: 2 (port-write # #vu8(60 33 # ?) ?) In ice-9/boot-9.scm: 1669:16 1 (raise-exception _ #:continuable? _) 1669:16 0 (raise-exception _ #:continuable? _) ice-9/boot-9.scm:1669:16: In procedure raise-exception: In procedure fport_write: Broken pipe --8<---------------cut here---------------end--------------->8--- that suggests that we try to write something to a closed file. To be investigated :) Mathieu From debbugs-submit-bounces@debbugs.gnu.org Thu Jul 30 10:47:23 2020 Received: (at 42548) by debbugs.gnu.org; 30 Jul 2020 14:47:23 +0000 Received: from localhost ([127.0.0.1]:36485 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k19q7-00054i-3t for submit@debbugs.gnu.org; Thu, 30 Jul 2020 10:47:23 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52456) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k19q5-00054W-Ip for 42548@debbugs.gnu.org; Thu, 30 Jul 2020 10:47:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:39094) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k19q0-0000sv-9g for 42548@debbugs.gnu.org; Thu, 30 Jul 2020 10:47:16 -0400 Received: from [2a01:cb18:832e:5f00:e0a7:8073:f3be:5cd4] (port=50264 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k19pz-0002jK-Cj for 42548@debbugs.gnu.org; Thu, 30 Jul 2020 10:47:15 -0400 From: Mathieu Othacehe To: 42548@debbugs.gnu.org Subject: Re: bug#42548: Cuirass 504 errors References: <87eeoy9rzk.fsf@gnu.org> Date: Thu, 30 Jul 2020 16:47:12 +0200 In-Reply-To: <87eeoy9rzk.fsf@gnu.org> (Mathieu Othacehe's message of "Sun, 26 Jul 2020 18:10:55 +0200") Message-ID: <87r1st83gv.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 42548 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hey, > A second issue is caused when a build product download is started, then > aborted. In that case, sendfile throws an exception or enters an endless > loop. Ok, so I found a couple of errors here. First, I noticed that it was not possible to download simultaneously two build products, because the first download was blocking the whole process. This is solved by: 6ad9c602697ffe33c8fbb09ccd796b74bf600223. In short, current-fiber was set to #f, both in the context of the caller and the spawned thread. So I think the get-message operating was blocking the whole thread instead of suspending the current fiber. But if someone else could take a look it would be nice :). Second issue, sendfile may throw EPIPE or ECONNRESET if the client disconnects before the end of the transfer. I think, besides the dirty backtrace, it was not harmful. But anyway, its better to catch this as we are doing in "guix publish", see: 0955a11abd9e27c96a1375cca6a1c97869b5780a. I fear it won't be enough to fix the 504 errors, but at least it's a start. Thanks, Mathieu From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 04 12:48:38 2020 Received: (at 42548) by debbugs.gnu.org; 4 Aug 2020 16:48:38 +0000 Received: from localhost ([127.0.0.1]:49028 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k307B-0003nf-SM for submit@debbugs.gnu.org; Tue, 04 Aug 2020 12:48:38 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45484) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k3078-0003nQ-1m for 42548@debbugs.gnu.org; Tue, 04 Aug 2020 12:48:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:39679) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k3072-0003Q4-Oq for 42548@debbugs.gnu.org; Tue, 04 Aug 2020 12:48:28 -0400 Received: from [109.190.253.14] (port=52204 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k3072-0000QH-3V for 42548@debbugs.gnu.org; Tue, 04 Aug 2020 12:48:28 -0400 From: Mathieu Othacehe To: 42548@debbugs.gnu.org Subject: Re: bug#42548: Cuirass 504 errors References: <87eeoy9rzk.fsf@gnu.org> <86zh7kzjys.fsf@gmail.com> <87sgdckscm.fsf@gnu.org> <86d04gyqfs.fsf@gmail.com> <87mu3jheme.fsf@gnu.org> Date: Tue, 04 Aug 2020 18:48:24 +0200 In-Reply-To: <87mu3jheme.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 28 Jul 2020 16:56:57 +0200") Message-ID: <87bljq8ihz.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hello, > that suggests that we try to write something to a closed file. > > To be investigated :) Ok, so I have a better grasp on what's going on. Cuirass web server is receiving some requests such as "/builds/1234)" which were not rejected, but worst, caused SQL queries such as "select * from Bui [...] Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.92 listed in list.dnswl.org] 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [109.190.253.14 listed in zen.spamhaus.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-Debbugs-Envelope-To: 42548 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) Hello, > that suggests that we try to write something to a closed file. > > To be investigated :) Ok, so I have a better grasp on what's going on. Cuirass web server is receiving some requests such as "/builds/1234)" which were not rejected, but worst, caused SQL queries such as "select * from Builds". As the table is quite large, it caused some of the DB workers to hang. Once all DB workers were hanging, the queries started to accumulate until the open fd limit (1024) was reached. I did consolidate the HTTP queries validation, and Cuirass web server is now running since 48 hours, which has not happened in months I think. I also added some warnings to detect DB workers hanging for more than 5 seconds. The next step is to log all SQL queries using[1]. This should allow us to spot this kind of issues more easily. Logging the duration of each query should also help us to optimize the queries. I'm still waiting a few days before closing this issue. Thanks, Mathieu [1]: https://notabug.org/guile-sqlite3/guile-sqlite3/pulls/16 From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 06 04:17:04 2020 Received: (at 42548-done) by debbugs.gnu.org; 6 Aug 2020 08:17:04 +0000 Received: from localhost ([127.0.0.1]:53086 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k3b5E-00009G-6a for submit@debbugs.gnu.org; Thu, 06 Aug 2020 04:17:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53944) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1k3b5B-0008Uw-Ea for 42548-done@debbugs.gnu.org; Thu, 06 Aug 2020 04:17:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47644) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k3b56-0006rI-8N for 42548-done@debbugs.gnu.org; Thu, 06 Aug 2020 04:16:56 -0400 Received: from lfbn-ann-1-356-169.w86-200.abo.wanadoo.fr ([86.200.73.169]:45220 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k3b55-0005DH-CO for 42548-done@debbugs.gnu.org; Thu, 06 Aug 2020 04:16:55 -0400 From: Mathieu Othacehe To: 42548-done@debbugs.gnu.org Subject: Re: bug#42548: Cuirass 504 errors References: <87eeoy9rzk.fsf@gnu.org> <86zh7kzjys.fsf@gmail.com> <87sgdckscm.fsf@gnu.org> <86d04gyqfs.fsf@gmail.com> <87mu3jheme.fsf@gnu.org> <87bljq8ihz.fsf@gnu.org> Date: Thu, 06 Aug 2020 10:16:52 +0200 In-Reply-To: <87bljq8ihz.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 04 Aug 2020 18:48:24 +0200") Message-ID: <87a6z8ur2j.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 42548-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello, > I'm still waiting a few days before closing this issue. No issues so far, closing this one. Mathieu From unknown Sun Aug 10 07:29:04 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 03 Sep 2020 11:24:08 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator