From unknown Sun Jun 22 04:12:39 2025 X-Loop: help-debbugs@gnu.org Subject: bug#61646: Bandwidth-induced offload timeout abort whole operating Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 20 Feb 2023 03:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 61646 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 61646@debbugs.gnu.org X-Debbugs-Original-To: bug-guix Received: via spool by submit@debbugs.gnu.org id=B.16768637132114 (code B ref -1); Mon, 20 Feb 2023 03:29:02 +0000 Received: (at submit) by debbugs.gnu.org; 20 Feb 2023 03:28:33 +0000 Received: from localhost ([127.0.0.1]:50110 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTwqu-0000Y1-RE for submit@debbugs.gnu.org; Sun, 19 Feb 2023 22:28:33 -0500 Received: from lists.gnu.org ([209.51.188.17]:56166) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTwqt-0000Xu-Az for submit@debbugs.gnu.org; Sun, 19 Feb 2023 22:28:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pTwqt-0006V7-5c for bug-guix@gnu.org; Sun, 19 Feb 2023 22:28:31 -0500 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pTwqr-0007kA-Kh for bug-guix@gnu.org; Sun, 19 Feb 2023 22:28:30 -0500 Received: by mail-qt1-x830.google.com with SMTP id f2so1135078qtj.2 for ; Sun, 19 Feb 2023 19:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:message-id:date:subject:to:from:from:to:cc:subject :date:message-id:reply-to; bh=LI0fE7Bf+doI3tBCHTg+/C1kBe/HdJSeHgGdQpfYv54=; b=WdSwj09n1+bc2ntuwicMc8mtRpFEqQss5BfUVnHOsrwA+u5k+9hcux76iaT5F+xzjq +dtj2E272d3hVeYkKlPU54Dz3xc9EjomwFF2KFQX+YFFyiInZIic3TS8EO/XgSx758Vd Tp59NvEzQAPtwRRO+GxDiT9KOkNS7oXfZVXI5vBmWFwNnIMOeaem86bXEI2u48OBub7r xyhzV6EICUdASQKjJwxbe0d/LPByKb0N8G2QlG+6OD2qtJ3N6gx8knFL3UvNIVSw1JT9 fpVmRC0IXU0v5s/B2LRJ03F+sV836uv1kwwWkCXMclaZb1APxB0d6yVGkcFDUV0VAu3c olzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:message-id:date:subject:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=LI0fE7Bf+doI3tBCHTg+/C1kBe/HdJSeHgGdQpfYv54=; b=A6NOIbVU7Op8MGeMpe5d39Quk+cVKDrJlYJp8az7/G7cmuofGRFSOySg0AoMoyomuq mzLcrDl7Zv/x/exHa+Wr9KPzIKIR95Db+4D28VW9iXchGCw1av9jPWUWWdFhIU0BeTxi ZuPu1PhGRkdy3nhdUgtvm6LYYMr7zlQBNgnlCXrUijtGqxWktGgQz9FHHaKTlOsCC9/K Ju/8fH+yxAW5KClcY4Nmv2eT/PkOcO0Xy9+QWfvksEiNQ0sNnwtwXZWw31Z2Kk/+Y/La Ps4790x9hKQl4kZ/L7UrveMlGdmNJpb4CEHRC2bZvzWPrB2oYOj6Cr/ZqOTmlLAQ4q50 VvOg== X-Gm-Message-State: AO0yUKUzxfv2GIUdT80SLk8JF1NmZt4nlBQlVJDsUDbV7QrSlDUemaoR mdBJPG/sho/aC8UxXH2ymsf3nSBsigvd6A== X-Google-Smtp-Source: AK7set/O6K02xfgHeXJbsxbfpQzlI1apzyquafhEcdQGH3/T6iMnRsa9hFFkvvJnJYxLvg+e95QhSw== X-Received: by 2002:a05:622a:1787:b0:3b9:a4be:fc4a with SMTP id s7-20020a05622a178700b003b9a4befc4amr180260qtk.14.1676863708354; Sun, 19 Feb 2023 19:28:28 -0800 (PST) Received: from hurd (dsl-158-213.b2b2c.ca. [66.158.158.213]) by smtp.gmail.com with ESMTPSA id p4-20020a05622a00c400b003b68d445654sm8185265qtw.91.2023.02.19.19.28.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Feb 2023 19:28:27 -0800 (PST) From: Maxim Cournoyer Date: Sun, 19 Feb 2023 22:28:16 -0500 Message-ID: <87ilfxm2wf.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::830; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qt1-x830.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Hi Guix, I can reproduce this rather easily on my system: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix build icedove The following derivations will be built: /gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv /gnu/store/8zi808086b3vlfjrhdm87fgljziwdqx2-icedove-l10n-102.7.2.drv /gnu/store/v0sq7rb8fk36kjasb27a71z1a27wxb1s-icedove-minimal-102.7.2.drv process 19542 acquired build slot '/var/guix/offload/localhost:6666/0' normalized load on machine 'localhost' is 0.08 building /gnu/store/8zi808086b3vlfjrhdm87fgljziwdqx2-icedove-l10n-102.7.2.drv... process 19548 acquired build slot '/var/guix/offload/localhost:6666/1' normalized load on machine 'localhost' is 0.08 building /gnu/store/v0sq7rb8fk36kjasb27a71z1a27wxb1s-icedove-minimal-102.7.2.drv... guix offload: sending 1 store item (558 MiB) to 'localhost'... exporting path `/gnu/store/bwb5hcdyzgq16kmbsva7ax0zq6lzg78z-icedove-102.7.2.tar.xz' guix offload: error: failed to connect to 'localhost': Timeout connecting to localhost cannot build derivation `/gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv': 1 dependencies couldn't be built guix build: error: build of `/gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv' failed --8<---------------cut here---------------end--------------->8--- The third derivation tries to get a build slot and times out, because the first two have already saturated the bandwidth of the link and it takes more time than expected to get a reply. The workaround is to use '-k', for "--keep-continuing", and retry the 3rd failing derivation after the first two completed. I don't have a clear idea on how to improve the situation other than use longer timeouts... but perhaps these timeouts could be dynamic based on the load of the network/CPU/ ? -- Thanks, Maxim From unknown Sun Jun 22 04:12:39 2025 X-Loop: help-debbugs@gnu.org Subject: bug#61646: Bandwidth-induced offload timeout abort whole operating Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 23 Feb 2023 22:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61646 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Maxim Cournoyer Cc: 61646@debbugs.gnu.org Received: via spool by 61646-submit@debbugs.gnu.org id=B61646.167719119321072 (code B ref 61646); Thu, 23 Feb 2023 22:27:01 +0000 Received: (at 61646) by debbugs.gnu.org; 23 Feb 2023 22:26:33 +0000 Received: from localhost ([127.0.0.1]:35517 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pVK2r-0005To-0Z for submit@debbugs.gnu.org; Thu, 23 Feb 2023 17:26:33 -0500 Received: from eggs.gnu.org ([209.51.188.92]:47892) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pVK2p-0005Tb-PJ for 61646@debbugs.gnu.org; Thu, 23 Feb 2023 17:26:32 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVK2j-0000qt-Pc; Thu, 23 Feb 2023 17:26:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=vyoVMlqf5dpWFo1EBF1z8a6DyCtgeheqy1RKTLEssAY=; b=kmAkq912WPJ8ToH2tt8B qQlNYkOvkT0C269gZ1jV0oePNnoKYgr1uOIx2uUGCW3FtCRPAfCMOqYjjc9o6IOywhqDBMtM3xF1j o5flnNwA/XuXvWuZ4MsFiG2dscq1YXjeeHPGkBvNhdlgbFOZihyJYGla0kSqnivdgscA3lEViLpUe h9enwMnQwmGVP+7WrmMUzXmu7fJydxwkhUAs4dWKhLOsMP3YxgbfVQlmqk7rO2mZcM9JeFbeU+weh LHGr6232fsy7RDvsP9XOEPPzW5c/9+u5tDxfxzW+4WrE20S7xylSePjjiR0XzUpGa52GKAiNwPruF 5lTlfppx4UKOGQ==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVK2j-00006y-Cy; Thu, 23 Feb 2023 17:26:25 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87ilfxm2wf.fsf@gmail.com> Date: Thu, 23 Feb 2023 23:26:22 +0100 In-Reply-To: <87ilfxm2wf.fsf@gmail.com> (Maxim Cournoyer's message of "Sun, 19 Feb 2023 22:28:16 -0500") Message-ID: <87wn483to1.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Maxim, Maxim Cournoyer skribis: > I can reproduce this rather easily on my system: > > $ ./pre-inst-env guix build icedove > The following derivations will be built: > /gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv > /gnu/store/8zi808086b3vlfjrhdm87fgljziwdqx2-icedove-l10n-102.7.2.drv > /gnu/store/v0sq7rb8fk36kjasb27a71z1a27wxb1s-icedove-minimal-102.7.2.drv > process 19542 acquired build slot '/var/guix/offload/localhost:6666/0' > normalized load on machine 'localhost' is 0.08 > building /gnu/store/8zi808086b3vlfjrhdm87fgljziwdqx2-icedove-l10n-102.7.2= .drv... > process 19548 acquired build slot '/var/guix/offload/localhost:6666/1' > normalized load on machine 'localhost' is 0.08 > building /gnu/store/v0sq7rb8fk36kjasb27a71z1a27wxb1s-icedove-minimal-102.= 7.2.drv... > guix offload: sending 1 store item (558 MiB) to 'localhost'... > exporting path `/gnu/store/bwb5hcdyzgq16kmbsva7ax0zq6lzg78z-icedove-102.7= .2.tar.xz' > guix offload: error: failed to connect to 'localhost': Timeout connecting= to localhost > cannot build derivation `/gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-iced= ove-102.7.2.drv': 1 dependencies couldn't be built > guix build: error: build of > `/gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv' failed > > The third derivation tries to get a build slot and times out, because > the first two have already saturated the bandwidth of the link and it > takes more time than expected to get a reply. Weird. Since the it=E2=80=99s a timeout while connecting, I suppose the pa= tch below would improve the situation: --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/guix/scripts/offload.scm b/guix/scripts/offload.scm index 578b3b9888..90cf97401c 100644 --- a/guix/scripts/offload.scm +++ b/guix/scripts/offload.scm @@ -220,7 +220,7 @@ (define* (open-ssh-session machine #:optional max-silent-time) (session (make-session #:user (build-machine-user machine) #:host (build-machine-name machine) #:port (build-machine-port machine) - #:timeout 10 ;initial timeout (seconds) + #:timeout 30 ;initial timeout (seconds) ;; #:log-verbosity 'protocol #:identity (build-machine-private-key machine) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 DQpXRFlUPw0KDQpMdWRv4oCZLg0K --=-=-=-- From unknown Sun Jun 22 04:12:39 2025 X-Loop: help-debbugs@gnu.org Subject: bug#61646: Bandwidth-induced offload timeout abort whole operating Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 25 Feb 2023 02:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61646 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 61646@debbugs.gnu.org Received: via spool by 61646-submit@debbugs.gnu.org id=B61646.167729319828608 (code B ref 61646); Sat, 25 Feb 2023 02:47:02 +0000 Received: (at 61646) by debbugs.gnu.org; 25 Feb 2023 02:46:38 +0000 Received: from localhost ([127.0.0.1]:38701 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pVka5-0007RK-SA for submit@debbugs.gnu.org; Fri, 24 Feb 2023 21:46:38 -0500 Received: from mail-qv1-f46.google.com ([209.85.219.46]:45818) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pVka4-0007R6-JF for 61646@debbugs.gnu.org; Fri, 24 Feb 2023 21:46:37 -0500 Received: by mail-qv1-f46.google.com with SMTP id bo10so773742qvb.12 for <61646@debbugs.gnu.org>; Fri, 24 Feb 2023 18:46:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=JFXcMVu5tyhw9PPCJYODl0Ww+WgeqiekixYly1mB2Kk=; b=F/AFcPLPen6pP+v1e6YQLFShyl8wMy7IBCv0ho8g2VyO5kKB5EPNmYxDWWHLrbZegD lCOtFFWyRdBhdF2qtLVhxgP37YnuGyVKIdNzPx1UbDAnYxmwjw1nNdDphMS7Zs3m4gJQ bCc5o0VV2pZfMLV8Jt4rIMTjx9k8epz4aABxVKGodcpsETBha/nURtSMZ5yHK4BrZcHo sjcb0vNceaGHwWUXX79AGT+aAWETpUEl2CcMF85xW6uRqQ/8j9QjhfZ/gdXCTPVQ+HlM U4vngeP0lQ2EMGh6Pd9CsYS9g1fRnr9xlsB158CyFzQpNbF83RPz4fY8VhP1Cx5muArX qn4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=JFXcMVu5tyhw9PPCJYODl0Ww+WgeqiekixYly1mB2Kk=; b=Dnp8MpSVnGiJVDZfNw5CHA+J3H7QlwO5qnESN2aWA7Q5bLR7/PYt33JFeQfYsAKiyA gzBccaKzYP7/bPJ+WlwObz4f7P2CfdTfuxM4+k8yKCBPDA3SvW/+EqvaCXKQN40FgkrY Rv+muAiuBzeHyka6YH6+5LDlFVLA4kN32dT94Aaj8n7CMglhoxHYYZAsUDjB10G0551A AX4aafBbzwgQasHt6l9toOACY5wR3PjfiV5WaLQ4m0s8mFMnTqZl+wjfxVyzVQPssXGX ciBHqMDKFWj5vH5anYe0DRJbmTq5y+wGgbNbZGBa+a4+GFe99LFm6JYS96WxT4Dg2rUR bCrg== X-Gm-Message-State: AO0yUKWmRPbt9xt+biNYEdBXMGNYu26oWPl0+U1bLiJaXctYeswXZp1H P8hXY6JfPGzCW98Kpmh8x+V11ehOZ/NoBQ== X-Google-Smtp-Source: AK7set/MOWnOIMKxPbfbrqmrOgabCUriNZXP+ihsOSkn+QEOHFh5+1w7Uvh9ejUecEUYrvhB9OT5kw== X-Received: by 2002:a05:6214:400a:b0:56e:f3b3:989e with SMTP id kd10-20020a056214400a00b0056ef3b3989emr31493054qvb.26.1677293190779; Fri, 24 Feb 2023 18:46:30 -0800 (PST) Received: from hurd (dsl-141-27.b2b2c.ca. [66.158.141.27]) by smtp.gmail.com with ESMTPSA id r135-20020a37448d000000b00706bc44fda8sm474472qka.79.2023.02.24.18.46.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Feb 2023 18:46:30 -0800 (PST) From: Maxim Cournoyer References: <87ilfxm2wf.fsf@gmail.com> <87wn483to1.fsf@gnu.org> Date: Fri, 24 Feb 2023 21:46:29 -0500 In-Reply-To: <87wn483to1.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Thu, 23 Feb 2023 23:26:22 +0100") Message-ID: <87356uh37e.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Ludovic, Ludovic Court=C3=A8s writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> I can reproduce this rather easily on my system: >> >> $ ./pre-inst-env guix build icedove >> The following derivations will be built: >> /gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv >> /gnu/store/8zi808086b3vlfjrhdm87fgljziwdqx2-icedove-l10n-102.7.2.drv >> /gnu/store/v0sq7rb8fk36kjasb27a71z1a27wxb1s-icedove-minimal-102.7.2.drv >> process 19542 acquired build slot '/var/guix/offload/localhost:6666/0' >> normalized load on machine 'localhost' is 0.08 >> building /gnu/store/8zi808086b3vlfjrhdm87fgljziwdqx2-icedove-l10n-102.7.= 2.drv... >> process 19548 acquired build slot '/var/guix/offload/localhost:6666/1' >> normalized load on machine 'localhost' is 0.08 >> building /gnu/store/v0sq7rb8fk36kjasb27a71z1a27wxb1s-icedove-minimal-102= .7.2.drv... >> guix offload: sending 1 store item (558 MiB) to 'localhost'... >> exporting path `/gnu/store/bwb5hcdyzgq16kmbsva7ax0zq6lzg78z-icedove-102.= 7.2.tar.xz' >> guix offload: error: failed to connect to 'localhost': Timeout connectin= g to localhost >> cannot build derivation >> `/gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv': 1 >> dependencies couldn't be built >> guix build: error: build of >> `/gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv' fail= ed >> >> The third derivation tries to get a build slot and times out, because >> the first two have already saturated the bandwidth of the link and it >> takes more time than expected to get a reply. > > Weird. Since the it=E2=80=99s a timeout while connecting, I suppose the = patch > below would improve the situation: > > diff --git a/guix/scripts/offload.scm b/guix/scripts/offload.scm > index 578b3b9888..90cf97401c 100644 > --- a/guix/scripts/offload.scm > +++ b/guix/scripts/offload.scm > @@ -220,7 +220,7 @@ (define* (open-ssh-session machine #:optional max-sil= ent-time) > (session (make-session #:user (build-machine-user machine) > #:host (build-machine-name machine) > #:port (build-machine-port machine) > - #:timeout 10 ;initial timeout (seco= nds) > + #:timeout 30 ;initial timeout (seco= nds) > ;; #:log-verbosity 'protocol > #:identity (build-machine-private-key mac= hine) Hm, how can I test this again? I tried launching a daemon both on the remote and locally, with something like: sudo -E ./pre-inst-env ./guix-daemon --build-users-group guixbuild --max-silent-time 0 --timeout 0 --log-compression none --discover=3Dyes --substitute-urls "https://ci.guix.gnu.org https://bordeaux.guix.gnu.org" --max-jobs=3D20 and the code edited doesn't seem to run (I put an (error 'hello) in there and nothing happened). --=20 Thanks, Maxim From unknown Sun Jun 22 04:12:39 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Maxim Cournoyer Subject: bug#61646: closed (Re: bug#61646: Bandwidth-induced offload timeout abort whole operating) Message-ID: References: <87y1omfnnk.fsf@gmail.com> <87ilfxm2wf.fsf@gmail.com> X-Gnu-PR-Message: they-closed 61646 X-Gnu-PR-Package: guix Reply-To: 61646@debbugs.gnu.org Date: Sat, 25 Feb 2023 03:08:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1677294482-7545-1" This is a multi-part message in MIME format... ------------=_1677294482-7545-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #61646: Bandwidth-induced offload timeout abort whole operating which was filed against the guix package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 61646@debbugs.gnu.org. --=20 61646: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D61646 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1677294482-7545-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 61646-done) by debbugs.gnu.org; 25 Feb 2023 03:07:52 +0000 Received: from localhost ([127.0.0.1]:38716 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pVkue-0001xK-Ad for submit@debbugs.gnu.org; Fri, 24 Feb 2023 22:07:52 -0500 Received: from mail-qt1-f181.google.com ([209.85.160.181]:46857) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pVkuc-0001x7-Uh for 61646-done@debbugs.gnu.org; Fri, 24 Feb 2023 22:07:51 -0500 Received: by mail-qt1-f181.google.com with SMTP id c19so1435701qtn.13 for <61646-done@debbugs.gnu.org>; Fri, 24 Feb 2023 19:07:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=FUF2/3acOf1ce0jCw4dN9CSLc4WAnnI3dd8hBR4qVAY=; b=h2zwmkiyGmoY7cfYHsRIhRVWpPWqzbig5uiCdYD8euQ8fhd6Tq4PewbeMEFgPYc3ol gcFbFTEEg+nWy6TBzn23Av1DSNGHS/LFHgS7/QAnDFbMBLEvOhuhTUEQ7J0LMoNNNh5B kpXddGIDJR3UB7Y1gmFmvE/e/HNLmjgudHb+oFmreezEtCQvLbfOPEtLSIYGWfuezAbN aG2QQ+tombmzNURC4b2AuDxTyFH9+9MXvocyzNYPOe6CHjNNf+MLFxwqrAFUj2JyUSce hH7oPlek6f8RT37wxUqS/t4x1ZIa+qnx0xNFw4D0vKnB2CxwVdQ+o9a9kWDOcU9fEHQr Rvkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=FUF2/3acOf1ce0jCw4dN9CSLc4WAnnI3dd8hBR4qVAY=; b=TKlqJk78tVt0mPgGsObCPu0RYPvhNLDq3QZoKJzIVwjCCmhZGPyj6mzIzuw9GY3BZe eCfVwQNURY4HsMmqxkZKy/VuL24+01XPWPJdur//oi4cNL40Hq8jaHpxWzVECsg/kT70 LQhBjEUatlJqWVg3L+MRvlhcmM/3JJroKQACtnc3UUvtBxIW+ETNR2DKYVerNB2kJ5yT mRv1tl/yV9m25R240V2Nbm0/Bh7scNBewfNt1tIAqM5IumcQ5o09yziaBJD5EuiJfkiN o8p+E4FpK0Xs+w62PjGWtyGfZTWq+iUdtnZKjh7qpsK2PrLiZ+OnGZYFdCQj2C/H6D4K 84lg== X-Gm-Message-State: AO0yUKUlvI7dPkEbQ2FcDAzA+iSDXfizWcIUfHszNW3I5Ke0BV4lHGXV v4kL/UceGp8ZmyZxmuZuo1ursMHK7uZR2w== X-Google-Smtp-Source: AK7set8exXcK864sIiccrY4fMKTTV37RlB2I38M7X0TF0V1CmY5x45netFWi59Dnf+W4Q9xxfK3Euw== X-Received: by 2002:ac8:5a42:0:b0:3b8:6a9f:9144 with SMTP id o2-20020ac85a42000000b003b86a9f9144mr34134120qta.46.1677294465056; Fri, 24 Feb 2023 19:07:45 -0800 (PST) Received: from hurd (dsl-141-27.b2b2c.ca. [66.158.141.27]) by smtp.gmail.com with ESMTPSA id g3-20020ac84803000000b003be56bdd3b1sm621726qtq.92.2023.02.24.19.07.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Feb 2023 19:07:44 -0800 (PST) From: Maxim Cournoyer To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#61646: Bandwidth-induced offload timeout abort whole operating References: <87ilfxm2wf.fsf@gmail.com> <87wn483to1.fsf@gnu.org> Date: Fri, 24 Feb 2023 22:07:43 -0500 In-Reply-To: <87wn483to1.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Thu, 23 Feb 2023 23:26:22 +0100") Message-ID: <87y1omfnnk.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 61646-done Cc: 61646-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Ludovic Court=C3=A8s writes: [...] > Weird. Since the it=E2=80=99s a timeout while connecting, I suppose the = patch > below would improve the situation: > > diff --git a/guix/scripts/offload.scm b/guix/scripts/offload.scm > index 578b3b9888..90cf97401c 100644 > --- a/guix/scripts/offload.scm > +++ b/guix/scripts/offload.scm > @@ -220,7 +220,7 @@ (define* (open-ssh-session machine #:optional max-sil= ent-time) > (session (make-session #:user (build-machine-user machine) > #:host (build-machine-name machine) > #:port (build-machine-port machine) > - #:timeout 10 ;initial timeout (seco= nds) > + #:timeout 30 ;initial timeout (seco= nds) > ;; #:log-verbosity 'protocol > #:identity (build-machine-private-key mac= hine) Nevermind my previous message, it was --sysconfdir that had not been set, thus ignoring my offload setup (/etc/guix/machines.scm). The command worked to test the change from the local machine: --8<---------------cut here---------------start------------->8--- sudo -E ./pre-inst-env ./guix-daemon --build-users-group guixbuild \ --max-silent-time 0 --timeout 0 --log-compression none --discover=3Dyes \ --substitute-urls "https://ci.guix.gnu.org https://bordeaux.guix.gnu.org" \ --max-jobs=3D4 --8<---------------cut here---------------end--------------->8--- I pushed the fix in commit 53d718f61b. Closing, thank you! --=20 Thanks, Maxim ------------=_1677294482-7545-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 20 Feb 2023 03:28:33 +0000 Received: from localhost ([127.0.0.1]:50110 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTwqu-0000Y1-RE for submit@debbugs.gnu.org; Sun, 19 Feb 2023 22:28:33 -0500 Received: from lists.gnu.org ([209.51.188.17]:56166) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pTwqt-0000Xu-Az for submit@debbugs.gnu.org; Sun, 19 Feb 2023 22:28:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pTwqt-0006V7-5c for bug-guix@gnu.org; Sun, 19 Feb 2023 22:28:31 -0500 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pTwqr-0007kA-Kh for bug-guix@gnu.org; Sun, 19 Feb 2023 22:28:30 -0500 Received: by mail-qt1-x830.google.com with SMTP id f2so1135078qtj.2 for ; Sun, 19 Feb 2023 19:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:message-id:date:subject:to:from:from:to:cc:subject :date:message-id:reply-to; bh=LI0fE7Bf+doI3tBCHTg+/C1kBe/HdJSeHgGdQpfYv54=; b=WdSwj09n1+bc2ntuwicMc8mtRpFEqQss5BfUVnHOsrwA+u5k+9hcux76iaT5F+xzjq +dtj2E272d3hVeYkKlPU54Dz3xc9EjomwFF2KFQX+YFFyiInZIic3TS8EO/XgSx758Vd Tp59NvEzQAPtwRRO+GxDiT9KOkNS7oXfZVXI5vBmWFwNnIMOeaem86bXEI2u48OBub7r xyhzV6EICUdASQKjJwxbe0d/LPByKb0N8G2QlG+6OD2qtJ3N6gx8knFL3UvNIVSw1JT9 fpVmRC0IXU0v5s/B2LRJ03F+sV836uv1kwwWkCXMclaZb1APxB0d6yVGkcFDUV0VAu3c olzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:message-id:date:subject:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=LI0fE7Bf+doI3tBCHTg+/C1kBe/HdJSeHgGdQpfYv54=; b=A6NOIbVU7Op8MGeMpe5d39Quk+cVKDrJlYJp8az7/G7cmuofGRFSOySg0AoMoyomuq mzLcrDl7Zv/x/exHa+Wr9KPzIKIR95Db+4D28VW9iXchGCw1av9jPWUWWdFhIU0BeTxi ZuPu1PhGRkdy3nhdUgtvm6LYYMr7zlQBNgnlCXrUijtGqxWktGgQz9FHHaKTlOsCC9/K Ju/8fH+yxAW5KClcY4Nmv2eT/PkOcO0Xy9+QWfvksEiNQ0sNnwtwXZWw31Z2Kk/+Y/La Ps4790x9hKQl4kZ/L7UrveMlGdmNJpb4CEHRC2bZvzWPrB2oYOj6Cr/ZqOTmlLAQ4q50 VvOg== X-Gm-Message-State: AO0yUKUzxfv2GIUdT80SLk8JF1NmZt4nlBQlVJDsUDbV7QrSlDUemaoR mdBJPG/sho/aC8UxXH2ymsf3nSBsigvd6A== X-Google-Smtp-Source: AK7set/O6K02xfgHeXJbsxbfpQzlI1apzyquafhEcdQGH3/T6iMnRsa9hFFkvvJnJYxLvg+e95QhSw== X-Received: by 2002:a05:622a:1787:b0:3b9:a4be:fc4a with SMTP id s7-20020a05622a178700b003b9a4befc4amr180260qtk.14.1676863708354; Sun, 19 Feb 2023 19:28:28 -0800 (PST) Received: from hurd (dsl-158-213.b2b2c.ca. [66.158.158.213]) by smtp.gmail.com with ESMTPSA id p4-20020a05622a00c400b003b68d445654sm8185265qtw.91.2023.02.19.19.28.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Feb 2023 19:28:27 -0800 (PST) From: Maxim Cournoyer To: bug-guix Subject: Bandwidth-induced offload timeout abort whole operating Date: Sun, 19 Feb 2023 22:28:16 -0500 Message-ID: <87ilfxm2wf.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::830; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qt1-x830.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Hi Guix, I can reproduce this rather easily on my system: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix build icedove The following derivations will be built: /gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv /gnu/store/8zi808086b3vlfjrhdm87fgljziwdqx2-icedove-l10n-102.7.2.drv /gnu/store/v0sq7rb8fk36kjasb27a71z1a27wxb1s-icedove-minimal-102.7.2.drv process 19542 acquired build slot '/var/guix/offload/localhost:6666/0' normalized load on machine 'localhost' is 0.08 building /gnu/store/8zi808086b3vlfjrhdm87fgljziwdqx2-icedove-l10n-102.7.2.drv... process 19548 acquired build slot '/var/guix/offload/localhost:6666/1' normalized load on machine 'localhost' is 0.08 building /gnu/store/v0sq7rb8fk36kjasb27a71z1a27wxb1s-icedove-minimal-102.7.2.drv... guix offload: sending 1 store item (558 MiB) to 'localhost'... exporting path `/gnu/store/bwb5hcdyzgq16kmbsva7ax0zq6lzg78z-icedove-102.7.2.tar.xz' guix offload: error: failed to connect to 'localhost': Timeout connecting to localhost cannot build derivation `/gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv': 1 dependencies couldn't be built guix build: error: build of `/gnu/store/l6r93asndd0kwv7024iyrl71zd0lbpbq-icedove-102.7.2.drv' failed --8<---------------cut here---------------end--------------->8--- The third derivation tries to get a build slot and times out, because the first two have already saturated the bandwidth of the link and it takes more time than expected to get a reply. The workaround is to use '-k', for "--keep-continuing", and retry the 3rd failing derivation after the first two completed. I don't have a clear idea on how to improve the situation other than use longer timeouts... but perhaps these timeouts could be dynamic based on the load of the network/CPU/ ? -- Thanks, Maxim ------------=_1677294482-7545-1--