From unknown Wed Jun 18 23:05:27 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#30290 <30290@debbugs.gnu.org> To: bug#30290 <30290@debbugs.gnu.org> Subject: Status: guix-daemon slows to a crawl when a substitute server is offline Reply-To: bug#30290 <30290@debbugs.gnu.org> Date: Thu, 19 Jun 2025 06:05:27 +0000 retitle 30290 guix-daemon slows to a crawl when a substitute server is offl= ine reassign 30290 guix submitter 30290 Maxim Cournoyer severity 30290 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 29 22:08:04 2018 Received: (at submit) by debbugs.gnu.org; 30 Jan 2018 03:08:04 +0000 Received: from localhost ([127.0.0.1]:48403 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1egMHE-0000QD-0R for submit@debbugs.gnu.org; Mon, 29 Jan 2018 22:08:04 -0500 Received: from eggs.gnu.org ([208.118.235.92]:55301) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1egMHC-0000Ph-H4 for submit@debbugs.gnu.org; Mon, 29 Jan 2018 22:08:02 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1egMH6-0002Lw-Gv for submit@debbugs.gnu.org; Mon, 29 Jan 2018 22:07:57 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:54211) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1egMH6-0002Lf-Ci for submit@debbugs.gnu.org; Mon, 29 Jan 2018 22:07:56 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46261) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1egMH5-000397-Ac for bug-guix@gnu.org; Mon, 29 Jan 2018 22:07:56 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1egMH2-0002Iw-4V for bug-guix@gnu.org; Mon, 29 Jan 2018 22:07:55 -0500 Received: from mail-it0-x231.google.com ([2607:f8b0:4001:c0b::231]:51959) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1egMH1-0002IX-VF for bug-guix@gnu.org; Mon, 29 Jan 2018 22:07:52 -0500 Received: by mail-it0-x231.google.com with SMTP id p139so10790673itb.1 for ; Mon, 29 Jan 2018 19:07:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:user-agent:mime-version; bh=HZrV870pvnu3h0KHzLjpMCG0EUepQxbKrKh1dcs3yyM=; b=hN3SwdL9oahkklRwyAOXIw93UeQZMURISVrNudk5uCllH0jdDgGC5vfL+HZNY+SREo uKV5xNMcKDP+HCAZYj6J2uHu7qIMDZVkPzuKjhVEmzjigDYb7pCwez81Kuh/FcbS5LHq HpslFsghA30QKLWXjZ58YgmOvOL6hS0FaE8E++YhFygkYEy97fYCs7UkLHtk1h2+3/6h 1xGpFkFcghjyVD5cWg1Q/w1t2VZneVeGP3AOlwot1qyMJjIlBTOJ4nBry/GaA7otZniv yx3zn2Qyw1iuRpEQUXl/8y8LyLUuYfA/Wy1tunSlJ++/rblv1P7eWIj/tx5tbnGcyWgG aklw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version; bh=HZrV870pvnu3h0KHzLjpMCG0EUepQxbKrKh1dcs3yyM=; b=UANhPP6NPUVsEHfEC9v0EYv9a8FDabzl3N6p3BsX7yzk0722E/HtFzYa83tqESHHeD INaB+wRT6fc7gYjMZbL3MH7rPTLNdngKU8GFeTbgYg68MhrMx6dPZ9Gt0v4bfjLoVU+b +mar2CbReUUTMDEdClZDt4f395cEIeBxqptpi1XVwfdrwVy3CBc6hnpcXmU1M8y4WZiO 1OpuXN69BDDczyl+swd/D+oDh1I0jS7zPY7Vr/WY53D+y/AXVxE3CU5xmeziyKB8ic48 +r1cbb7vkmdmIrTygVIPXAAorFH+pvH2Y9K3lEI6RuCrY1qDPiwe9w/kY8zykt188Z7i liqw== X-Gm-Message-State: AKwxytfK4bgvxm6Kl4yrxglu4zLBZ1S7CttnCq/HiClR6RioDv9Y2Owb UPs547uXSfZgGk7AoekVhuV5uA== X-Google-Smtp-Source: AH8x225nPfsM3wbGOBu3NdG+vefB/cKJk08tQ6e0y6GN3emWZxG6zZyt/lL6bs/rD7OZOcQE+p9FUQ== X-Received: by 10.36.148.139 with SMTP id j133mr30508142ite.148.1517281670836; Mon, 29 Jan 2018 19:07:50 -0800 (PST) Received: from apteryx ([45.72.232.234]) by smtp.gmail.com with ESMTPSA id z84sm5079347itc.21.2018.01.29.19.07.49 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 29 Jan 2018 19:07:50 -0800 (PST) From: Maxim Cournoyer To: bug-guix Subject: guix-daemon slows to a crawl when a substitute server is offline Date: Mon, 29 Jan 2018 22:07:49 -0500 Message-ID: <87fu6o2ge2.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.0 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.0 (----) When a substitute server used by guix-daemon is offline, the daemon will keep attempting to connect to it, even when it shouldn't need any data (ran 'sudo guix system reconfigure my-config.scm' multiple times in a row. With the disconnected server (bayfront in my case), that command would take close to 8 minutes, with many system calls like: --8<---------------cut here---------------start------------->8--- connect(14, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("141.255.128.56")}, 16) = -1 EINPROGRESS --8<---------------cut here---------------end--------------->8--- which wasted 5 seconds each time. After removing this server from my substitute servers list, the same operation (system reconfigure) is 8 times faster (1 minute). Suggestion: the daemon should stop trying to use the offline substitute server after trying for X times, and print a warning about it. Maxim From debbugs-submit-bounces@debbugs.gnu.org Wed Dec 02 19:25:46 2020 Received: (at 30290) by debbugs.gnu.org; 3 Dec 2020 00:25:46 +0000 Received: from localhost ([127.0.0.1]:37742 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kkcRN-0002Td-QP for submit@debbugs.gnu.org; Wed, 02 Dec 2020 19:25:45 -0500 Received: from mail-wr1-f52.google.com ([209.85.221.52]:43942) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kkcRN-0002TN-5f for 30290@debbugs.gnu.org; Wed, 02 Dec 2020 19:25:45 -0500 Received: by mail-wr1-f52.google.com with SMTP id s8so80734wrw.10 for <30290@debbugs.gnu.org>; Wed, 02 Dec 2020 16:25:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=Lk8z8s7PCaULIiFoulALFpITb8aPYRTgtDUhAj2cgKw=; b=SZ9FS1jN3Xob7FpECAbPQOYH0seMwuxX8NpVuwHLcZyyHTKmO2k86A5dgyvsK9Romz 1ql7M+S+quHurGZxee88X5RVGs7W5K0A83cd8bvR3Q7VsNv+3xPSTDyBmYaJALzgBreb 8lwPhEfUGiN7VJAaCxDsVmbjmEuZQ2Hl8Yen1V2cZ6tKI0GCH48UqdacQZiNAAL6ImUm gVU95eK0a4YVygvs5NTH98T9LL1irviIL4edRo3SVQRQbYi1LyfbwaZZdXfhO/k8p/P6 ycTbAIIH1zAyUlP9q72yYOzOYrx2mV59BGPOrzlTfW9iaVqm+3CZUdWaVkbCp4vMc/aY 5CWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=Lk8z8s7PCaULIiFoulALFpITb8aPYRTgtDUhAj2cgKw=; b=uewiQKSnfFpYzUs6/VvWvX5b0cigrC3t2VSgOoWVDNO1HKfQbTiGR7sI63vICY7Mqz oMzOeKIowEsp3CPLwZ8Vzg879ciWAMX1/NKE0h1aFlvkTJ9Yn6967sTI7UuUyzMARo9m GVdKp8dquJsU4X4AiOglOCfuvlc5SuZzT3SNUVu71hxgnQke8As5vaPTOD4QH4fanTP7 UgVX7P4ATR2OL3dxvuzhU7oHZNwQ4KcKIPZGXDIDa5xDNE6Sb3+VbPVrWEw/GzPvNKOn OcY5QJX7NqDdM0/rfA1VMiF24VsxhTHttphboGuVOXN7Z4qVifPNl1OSsi7lqp1mskmy z91w== X-Gm-Message-State: AOAM531DF3VzSPSmDB1iRiGTj4RZZBF6EsdyC2A4rbq7BOv0vUVgnZX7 RFnLd1AizGsTGjuIqpD7Km/XhHruz47Cbw== X-Google-Smtp-Source: ABdhPJztoTpiwKemPBfBuaCOy6Oc1zHl5xX1o3pWqm+5yrpfWtxkp9FcUjCqDuPWJSYD+YyQVUu5UA== X-Received: by 2002:adf:fa05:: with SMTP id m5mr688564wrr.26.1606955139112; Wed, 02 Dec 2020 16:25:39 -0800 (PST) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id l24sm373003wrb.28.2020.12.02.16.25.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Dec 2020 16:25:38 -0800 (PST) From: zimoun To: Maxim Cournoyer Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> Date: Thu, 03 Dec 2020 01:20:07 +0100 In-Reply-To: <87fu6o2ge2.fsf@gmail.com> (Maxim Cournoyer's message of "Mon, 29 Jan 2018 22:07:49 -0500") Message-ID: <86mtyvzqo8.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Maxim, On Mon, 29 Jan 2018 at 22:07, Maxim Cournoyer wrote: > When a substitute server used by guix-daemon is offline, the daemon will > keep attempting to connect to it, even when it shouldn't need any data > (ran 'sudo guix system reconfigure my-config.scm' multiple times in a > row. > > With the disconnected server (bayfront in my case), that command would > take close to 8 minutes, with many system calls like: > > connect(14, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("141.255.128.56")}, 16) = -1 EINPROGRESS > > which wasted 5 seconds each time. > > After removing this server from my substitute servers list, the same > operation (system reconfigure) is 8 times faster (1 minute). > > Suggestion: the daemon should stop trying to use the offline substitute > server after trying for X times, and print a warning about it. This looks like as a wishlist, right? Do it make sense to include such feature to the recent discussions about the revamp of offloading, Cuirass, publish, etc. All the best, simon From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 18 22:04:15 2020 Received: (at 30290) by debbugs.gnu.org; 19 Dec 2020 03:04:15 +0000 Received: from localhost ([127.0.0.1]:40622 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kqSXX-0005PZ-89 for submit@debbugs.gnu.org; Fri, 18 Dec 2020 22:04:15 -0500 Received: from mail-qk1-f172.google.com ([209.85.222.172]:45970) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kqSXV-0005PN-Iz for 30290@debbugs.gnu.org; Fri, 18 Dec 2020 22:04:13 -0500 Received: by mail-qk1-f172.google.com with SMTP id b64so4071706qkc.12 for <30290@debbugs.gnu.org>; Fri, 18 Dec 2020 19:04:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=SfZypc43K7mTAIIrdWyT6DElSlzUe31ihGZjRuTTcxI=; b=jxNmILHa+G9gZRGX4LeHD363xfqGua+HZj30VmRFElRexy+r6/YmswORqxGqXjDEds dYbuOorvBTbQ9wa4nucoFlyrTl+kSLObigXwo5+x0HlkH/KHltsezRQ+ZTkJcgNnMOiK xGVkGlG8h+8/r4PSJ7Q/0Osr4XV2Q5Eun7hJPxtHH9V6nDthnFOEV8SByfNjeYrIYfdR u1JfmcKeunYizFlcaPvnKEHZaprl8scruAMfvM6JLdIOBy1E4Jwb9khcpVEjN7yEFwVv K2BQCrQ/gwIFOeAT+6Mmg41rKWF/OL/Ryl4dvcVUzoafLdVPOQ137kDWUSZcmqc47Kgi 8POg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=SfZypc43K7mTAIIrdWyT6DElSlzUe31ihGZjRuTTcxI=; b=JcukoQ6/te10m8IHjehP3yL6WrWdJ6cwGgcf6QMI7nSWLHNOqb4dUYkKNc0KQEVtnl ToFdQmlfUGqA9AoFTTkBbPkH5fdBK0wM5bnVWKIH79vOXIFejYXs7ZHCTt21hdILk83K WjmVV3G+cCF3VwCqFp/EaQruoilBNakUC1fb5rAyCYmsW3ocepUZZAFZ9+famPTlyBOQ 09iFFhbN9el7iJTu179hMy6v8wRpnyM/TJ4qcoO2gqFJbQzsqhUZiAvyorpcYiVupXEM phCOtLca0toFNKqt/ypiSRMaRxdUZcGtP1Ye+He4fMrOxIpEzY8wVjLVHbHhoTBiNb3c urEg== X-Gm-Message-State: AOAM532b1ZplwjJ4i+sn4lpwGwz4BIqO4k9X6B/HdSK21Vs+26vFV/gf SZEUJNYWncoxnzmk7N/szJEI50IZJwHAgsR2 X-Google-Smtp-Source: ABdhPJzQeAR+CswnDBj1KZvJNhQ8SO26JNFA8kIVqx98NoLm45IKXQbRVYO11kUQeIwxjuqEvjv59A== X-Received: by 2002:a37:b083:: with SMTP id z125mr8171132qke.246.1608347048051; Fri, 18 Dec 2020 19:04:08 -0800 (PST) Received: from hurd (dsl-10-146-235.b2b2c.ca. [72.10.146.235]) by smtp.gmail.com with ESMTPSA id b12sm6500486qtj.12.2020.12.18.19.04.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Dec 2020 19:04:07 -0800 (PST) From: Maxim Cournoyer To: zimoun Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <86mtyvzqo8.fsf@gmail.com> Date: Fri, 18 Dec 2020 22:04:04 -0500 In-Reply-To: <86mtyvzqo8.fsf@gmail.com> (zimoun's message of "Thu, 03 Dec 2020 01:20:07 +0100") Message-ID: <87h7oiebtn.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi! zimoun writes: > Hi Maxim, > > On Mon, 29 Jan 2018 at 22:07, Maxim Cournoyer wrote: >> When a substitute server used by guix-daemon is offline, the daemon will >> keep attempting to connect to it, even when it shouldn't need any data >> (ran 'sudo guix system reconfigure my-config.scm' multiple times in a >> row. >> >> With the disconnected server (bayfront in my case), that command would >> take close to 8 minutes, with many system calls like: >> >> connect(14, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("141.255.128.56")}, 16) = -1 EINPROGRESS >> >> which wasted 5 seconds each time. >> >> After removing this server from my substitute servers list, the same >> operation (system reconfigure) is 8 times faster (1 minute). >> >> Suggestion: the daemon should stop trying to use the offline substitute >> server after trying for X times, and print a warning about it. > > This looks like as a wishlist, right? Do it make sense to include such > feature to the recent discussions about the revamp of offloading, > Cuirass, publish, etc. To me it's an issue more than a feature request, especially in a build farm setting; having a substitute machine down shouldn't cause a slow down for as long as it's down! I'm not sure if the recent offloading work that Mathieu did touched that topic. I'd need to test the scenario. Perhaps a system test would be useful. Maxim From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 22 10:16:16 2020 Received: (at 30290) by debbugs.gnu.org; 22 Dec 2020 15:16:16 +0000 Received: from localhost ([127.0.0.1]:50693 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1krjOa-0001Pb-CZ for submit@debbugs.gnu.org; Tue, 22 Dec 2020 10:16:16 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38894) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1krjOZ-0001PQ-OP for 30290@debbugs.gnu.org; Tue, 22 Dec 2020 10:16:16 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59604) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1krjOU-0007C6-JQ; Tue, 22 Dec 2020 10:16:10 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44072 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1krjOU-0002MU-2q; Tue, 22 Dec 2020 10:16:10 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> Date: Tue, 22 Dec 2020 16:16:08 +0100 In-Reply-To: <87fu6o2ge2.fsf@gmail.com> (Maxim Cournoyer's message of "Mon, 29 Jan 2018 22:07:49 -0500") Message-ID: <87r1nhzxaf.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 30290 Cc: 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Maxim Cournoyer skribis: > When a substitute server used by guix-daemon is offline, the daemon will > keep attempting to connect to it, even when it shouldn't need any data > (ran 'sudo guix system reconfigure my-config.scm' multiple times in a > row. > > With the disconnected server (bayfront in my case), that command would > take close to 8 minutes, with many system calls like: > > connect(14, {sa_family=3DAF_INET, sin_port=3Dhtons(443), sin_addr=3Dinet_= addr("141.255.128.56")}, 16) =3D -1 EINPROGRESS > > which wasted 5 seconds each time. Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98guix substitute --query=E2=80=99 process won=E2=80=99t retry connections to an u= nreachable host. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Dec 28 07:19:49 2020 Received: (at 30290) by debbugs.gnu.org; 28 Dec 2020 12:19:49 +0000 Received: from localhost ([127.0.0.1]:34482 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktrV7-00015Q-E0 for submit@debbugs.gnu.org; Mon, 28 Dec 2020 07:19:49 -0500 Received: from flashner.co.il ([178.62.234.194]:55266) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktrV5-00015D-4e for 30290@debbugs.gnu.org; Mon, 28 Dec 2020 07:19:48 -0500 Received: from localhost (unknown [5.102.238.249]) by flashner.co.il (Postfix) with ESMTPSA id 4E88840148; Mon, 28 Dec 2020 12:19:41 +0000 (UTC) Date: Mon, 28 Dec 2020 14:19:02 +0200 From: Efraim Flashner To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline Message-ID: References: <87fu6o2ge2.fsf@gmail.com> <87r1nhzxaf.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="BDZvg5AoZ6C4XZS9" Content-Disposition: inline In-Reply-To: <87r1nhzxaf.fsf@gnu.org> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: 30290@debbugs.gnu.org, Maxim Cournoyer X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --BDZvg5AoZ6C4XZS9 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Dec 22, 2020 at 04:16:08PM +0100, Ludovic Court=C3=A8s wrote: > Hi, >=20 > Maxim Cournoyer skribis: >=20 > > When a substitute server used by guix-daemon is offline, the daemon will > > keep attempting to connect to it, even when it shouldn't need any data > > (ran 'sudo guix system reconfigure my-config.scm' multiple times in a > > row. > > > > With the disconnected server (bayfront in my case), that command would > > take close to 8 minutes, with many system calls like: > > > > connect(14, {sa_family=3DAF_INET, sin_port=3Dhtons(443), sin_addr=3Dine= t_addr("141.255.128.56")}, 16) =3D -1 EINPROGRESS > > > > which wasted 5 seconds each time. >=20 > Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 > (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98guix > substitute --query=E2=80=99 process won=E2=80=99t retry connections to an= unreachable > host. >=20 > Ludo=E2=80=99. >=20 Occasionally my internet drops itself, and I find I'm left forever waiting for a timeout to see what sources I have cached locally. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --BDZvg5AoZ6C4XZS9 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl/pzTMACgkQQarn3Mo9 g1F/jhAAj2cB9IKVLCIsvozRZXNMd9WCT25zaNNN6oRPuNoRozDD4555fic0FPdw tv/S3aFnKrHjGuHsMs/qhQd1RzJXsFPO8szbPnwckyC8Cn70+1SvQIMCDfY7osXa r94XBxPl0P9gAYNCmDSIgRGT8WhAlKAeuXAA6DnqrWyU0DvLADGsTLPab+olbe9F 28uIqkQDf0689Z6lYgPd0FvjzRiR869/B3DYY+Kk1hSYWd3xTWan/NCdzHrvNAzg M5oYXUUur1BqLPT+deV2JBvPrJ5p2vsWZIjrx4AvI4cDB/xvYIAhJJuWD9+uP6as 0oZt9wxSsvr8zYwpO2YO1Q6E7a/fuEzhCYGoVtw4pDLx5BKWBYBVWM0K+grrZq5n amJ2fKcDjYHb0wWmk57wzaSqcuPGM22QA+WSehbYrkXsZCuCtGhkIawQAmUHmXDl u05hShPVy0CjhNA35dMlupNdXzOledt87K+H31YVdGAOaHVEd0NJ/RYLkm0aFkhU 5cHFg8gJidOeAfiy1qxgO1msU0EParCBRb2D1c6JaXl1ClWz/8NszWcw6cbIkodV NeglZHLn1f4btEWkdigEjctxM4NL/DhJxleXXBp30VNGMmJ07dMhmgBTDqESK1SO xVZ4LudByowYJU9Pg57y083YuJqSB0XVPJC+laR2Mw8A1nXZT50= =IRoi -----END PGP SIGNATURE----- --BDZvg5AoZ6C4XZS9-- From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 24 17:58:34 2021 Received: (at 30290) by debbugs.gnu.org; 24 Mar 2021 21:58:34 +0000 Received: from localhost ([127.0.0.1]:36276 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lPBWM-0006J1-6C for submit@debbugs.gnu.org; Wed, 24 Mar 2021 17:58:34 -0400 Received: from mail-wm1-f52.google.com ([209.85.128.52]:40699) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lPBWH-0006IT-MR for 30290@debbugs.gnu.org; Wed, 24 Mar 2021 17:58:30 -0400 Received: by mail-wm1-f52.google.com with SMTP id y124-20020a1c32820000b029010c93864955so2017502wmy.5 for <30290@debbugs.gnu.org>; Wed, 24 Mar 2021 14:58:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=phY8xc7EbJm6MstPaMvYf9sMFaEIhTgigQU3rIYOIHw=; b=c/CmF/R4oj6xt1ff0NgUo6V6UiOQsG2xu5Bztps+y7Ii6hAwH/eWhQNE16QqyogISB LvZfv0D002AIhCpTXSsKO9mO/h4eukqjSQJIVUQMlsMyt2qs7Mti/Tm8PqO9mAat6yPZ ul+wRqeATcX4RNCMZ4ViwmtFU+2OiblZN/7NjxbOhITvqxvgKJmwG8k3g/KeaCu3tzIb M2DrFKdVVbQ630qqbpp8JY+rS5yGTvdlFcru44Bd3/Uej0U3m3a8Pfv3O35asdfXkS8s Ye3msXD/yj7H9rFiyaN/VrhhowWzrzp6uCfZlz3KttlxAhHTTAPZEWf0JKJRD4QdyWSq fcHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=phY8xc7EbJm6MstPaMvYf9sMFaEIhTgigQU3rIYOIHw=; b=Iikml5Bu5tTTgVIf9Y2jJbiHMkVPcZvlo0BA/tRDimuZaUD+PISXYIbU75iwVDL1I0 buWKs+L8lTieAwSYZ3P7S8XZTu3rEehCOW82/e9CZA3sDDsc8KpwVHDGoHOa+g6RS4Cp TiE3ojO4zLYb44z47kYhxqXdOFKQQO0LmaGYzaVzNzmCXVXLBA3QunpXrQqTZj61FFxM Xn+3mAzPHV0AUXAIwS18zn2sIMHxKU5gtE6Ew9hKEtaFcaQuAVoRb/z88zdeZy3egNkU cuZU61Ogj10TgR1/jDbybt60HIYGYc/VW3JNRrRN0+aLA2nD1AfVGMkWD2AFudHFvbdo Kzpg== X-Gm-Message-State: AOAM532zeQllQp65mhF8esRDHOeGhsbfPz8cIiCyRUU/035TksILDnuF anE5Hq5Qg3d+xslA8oZlCrX1DOQDq8M= X-Google-Smtp-Source: ABdhPJySPgjnp2T1h/ke7WGeocpn9zdh7uZMeaBiYxO/1x9QnQVAs7wPcwxmEHQSrrZZvH5tglAsXQ== X-Received: by 2002:a05:600c:1548:: with SMTP id f8mr4864462wmg.81.1616623104040; Wed, 24 Mar 2021 14:58:24 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id s3sm3830474wmd.21.2021.03.24.14.58.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Mar 2021 14:58:23 -0700 (PDT) From: zimoun To: Maxim Cournoyer , Mathieu Othacehe Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> Date: Wed, 24 Mar 2021 22:55:13 +0100 In-Reply-To: <87fu6o2ge2.fsf@gmail.com> (Maxim Cournoyer's message of "Mon, 29 Jan 2018 22:07:49 -0500") Message-ID: <868s6cb4ji.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Mon, 29 Jan 2018 at 22:07, Maxim Cournoyer w= rote: > When a substitute server used by guix-daemon is offline, the daemon will > keep attempting to connect to it, even when it shouldn't need any data > (ran 'sudo guix system reconfigure my-config.scm' multiple times in a > row. > > With the disconnected server (bayfront in my case), that command would > take close to 8 minutes, with many system calls like: > > connect(14, {sa_family=3DAF_INET, sin_port=3Dhtons(443), sin_addr=3Dinet_= addr("141.255.128.56")}, 16) =3D -1 EINPROGRESS > > which wasted 5 seconds each time. > > After removing this server from my substitute servers list, the same > operation (system reconfigure) is 8 times faster (1 minute). > > Suggestion: the daemon should stop trying to use the offline substitute > server after trying for X times, and print a warning about it. What is the status of this bug? Especially with the recent additions in Cuirass? Is it still an issue? Is some timeout still happening? Well, in summary, the 3 relevant messages are: -------------------- Start of forwarded message -------------------- From: Maxim Cournoyer Date: Fri, 18 Dec 2020 22:04:04 -0500 I'm not sure if the recent offloading work that Mathieu did touched that topic. I'd need to test the scenario. Perhaps a system test would be useful. -------------------- End of forwarded message -------------------- -------------------- Start of forwarded message -------------------- From: Ludovic Court=C3=A8s Date: Tue, 22 Dec 2020 16:16:08 +0100 Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98guix substitute --query=E2=80=99 process won=E2=80=99t retry connections to an u= nreachable host. -------------------- End of forwarded message -------------------- -------------------- Start of forwarded message -------------------- Date: Mon, 28 Dec 2020 14:19:02 +0200 From: Efraim Flashner Occasionally my internet drops itself, and I find I'm left forever waiting for a timeout to see what sources I have cached locally. -------------------- End of forwarded message -------------------- Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 09 17:43:53 2021 Received: (at 30290) by debbugs.gnu.org; 9 Jun 2021 21:43:54 +0000 Received: from localhost ([127.0.0.1]:34800 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lr5zN-0003Z6-Gk for submit@debbugs.gnu.org; Wed, 09 Jun 2021 17:43:53 -0400 Received: from mail-wm1-f54.google.com ([209.85.128.54]:45898) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lr5zJ-0003Yk-KD for 30290@debbugs.gnu.org; Wed, 09 Jun 2021 17:43:51 -0400 Received: by mail-wm1-f54.google.com with SMTP id v206-20020a1cded70000b02901a586d3fa23so5122124wmg.4 for <30290@debbugs.gnu.org>; Wed, 09 Jun 2021 14:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=UxHsHEt+SAL86OWzSp4gM+5EWgdjjEC/fsuWSFKmHz8=; b=LGrWRKNsKIvXs4rIz6W1GFjnN9iR0IP7/8++czCRCnFdbVFZMJf3myMf/iY2XolZue gd7HuQ2p6c6ku7F2CO9eQF+xUo9zW3dKHInvRwKrp7L1f0uCv+j7G+v/BbYj8SK9uZZL I/9wxJ5C4v1reKuxh4IR7u+kIB/4Em/J7w+pkMREQiy/O70LRyRuIa6PR4QiDnKTyGJm iDH3LUfAd6Rgvaw5lrPpnZNlXJofSxW8yl8+QmCsiVODbJjmn4Ht8ngyWR2pvvuD9FFS yaxE4AyM4qe95OkmfHXGRVnt+q9RSQh2RrDBBi7Eq8MfNi2T6m4emkkVTmxsXw/RKOeN fltA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=UxHsHEt+SAL86OWzSp4gM+5EWgdjjEC/fsuWSFKmHz8=; b=JnPzFNC3L7Rw/I9ZVvfbAKcI2Jg1meEsgjsq200H/u6gcUPYO939kt6AoFlZ9k33/i zb51mXWfBKo6jhOrS1rju2FudbxzxubH25ODPi1kL+pxv/yZk0wqtiSyOzwT24KsAZaH N2EWkm+93D+DCTY81K5Qr85I2foYgAcrgrphhyBj7sv45K7Wuz69Ec8trtBH2h2k1sLI T9ZLWySvldO2U1i3EBCvS4JBrp219PXJIBJRJ464xxLO2es73Oc0HhlfLBQEsedFsxLh n5FZ1pfhUDcFwLwjsRSDCCi8j5fY1RbgKmnEQvKndYc6rstZRbdnTSMWJTUeFygTdWBE ULHQ== X-Gm-Message-State: AOAM530jM9nSEhkDLyUogAoBbmQundw+wEtvWqRsfj3TImfQAVGTD9UT e8zP5T1bQuRKIY/HYdyHcuqX3Be0RSo= X-Google-Smtp-Source: ABdhPJxQou9nZXU0vy5mfszeDc9VvMbOXZCiS79d4BF72z2+SBUdjIdAcc0SfhzdaiD4OKwmK6+6YQ== X-Received: by 2002:a1c:98d0:: with SMTP id a199mr11914498wme.22.1623275023800; Wed, 09 Jun 2021 14:43:43 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id j7sm1136495wrm.93.2021.06.09.14.43.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 14:43:43 -0700 (PDT) From: zimoun To: Maxim Cournoyer Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> Date: Wed, 09 Jun 2021 23:34:41 +0200 In-Reply-To: <868s6cb4ji.fsf@gmail.com> (zimoun's message of "Wed, 24 Mar 2021 22:55:13 +0100") Message-ID: <86y2bilo26.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: Mathieu Othacehe , 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, On Wed, 24 Mar 2021 at 22:55, zimoun wrote: > On Mon, 29 Jan 2018 at 22:07, Maxim Cournoyer = wrote: >> When a substitute server used by guix-daemon is offline, the daemon will >> keep attempting to connect to it, even when it shouldn't need any data >> (ran 'sudo guix system reconfigure my-config.scm' multiple times in a >> row. >> >> With the disconnected server (bayfront in my case), that command would >> take close to 8 minutes, with many system calls like: >> >> connect(14, {sa_family=3DAF_INET, sin_port=3Dhtons(443), >> sin_addr=3Dinet_addr("141.255.128.56")}, 16) =3D -1 EINPROGRESS >> >> which wasted 5 seconds each time. >> >> After removing this server from my substitute servers list, the same >> operation (system reconfigure) is 8 times faster (1 minute). >> >> Suggestion: the daemon should stop trying to use the offline substitute >> server after trying for X times, and print a warning about it. [...] > From: Maxim Cournoyer > Date: Fri, 18 Dec 2020 22:04:04 -0500 (24 weeks, 4 days, 18 hours ago) > > I'm not sure if the recent offloading work that Mathieu did touched that > topic. I'd need to test the scenario. Perhaps a system test would be > useful. > ---------- > > From: Ludovic Court=C3=A8s > Date: Tue, 22 Dec 2020 16:16:08 +0100 > Date: Tue, 22 Dec 2020 16:16:08 +0100 (24 weeks, 1 day, 6 hours ago) > > Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 > (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98guix > substitute --query=E2=80=99 process won=E2=80=99t retry connections to an= unreachable > host. > ---------- > > From: Efraim Flashner > Date: Mon, 28 Dec 2020 14:19:02 +0200 > Date: Mon, 28 Dec 2020 14:19:02 +0200 (23 weeks, 2 days, 9 hours ago) > > Occasionally my internet drops itself, and I find I'm left forever > waiting for a timeout to see what sources I have cached locally. > ---------- What is the current stats of this bug? Is it still happening with the recent improvements of Cuirass? Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 13 05:02:16 2021 Received: (at 30290) by debbugs.gnu.org; 13 Jul 2021 09:02:16 +0000 Received: from localhost ([127.0.0.1]:39426 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m3EIy-0004OH-3m for submit@debbugs.gnu.org; Tue, 13 Jul 2021 05:02:16 -0400 Received: from mail-wr1-f48.google.com ([209.85.221.48]:34339) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m3EIw-0004Nl-Jj for 30290@debbugs.gnu.org; Tue, 13 Jul 2021 05:02:15 -0400 Received: by mail-wr1-f48.google.com with SMTP id p8so29460551wrr.1 for <30290@debbugs.gnu.org>; Tue, 13 Jul 2021 02:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=cuSjQ7lN0u4sbvY9mm/0cT0mOIwf1d1iF5YwdgprEJw=; b=dVA2rNha8Xgl8r/+N32nnws6zfIMUXJR7Q+jInHtJjEBTfOIixuOuCTrn8g1uMD0BZ CnMeBrfMnX3AO/+/BDrojVNLLFJkoEi6tSWDensWgXRb8YlP9nNXAMcUrZlc3AHmo4H5 L7Llopip9Y1RU+KOKFdBgbmmzE1oQSHEPlxudAotZ4sU711PEGwhFtBdB1g9iGx3/qwj se9f9jSLRzYmjQOqlVMlRwc+3+qh24FsWuSg+FT/yvPGI3OGqW88ndvNfF9tig750l6w Gp+N84tZdK8LnMVZMVHIqK5l+gUogctZnvus8p6tRS00SxwvehfR2NV+/S1BFWeFZnNl os1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=cuSjQ7lN0u4sbvY9mm/0cT0mOIwf1d1iF5YwdgprEJw=; b=X+3RT2+dcwHjK00Hi7+SSGQXKw6fLtMRtJ+k5dg1kirOMMv1T+xUzEepYkBWKYtJ1q gp0mhPEnO2mtVtsLQSWyYHFuhTtkpscYgzO/oHw+DQsK8tW32ou7YNqF4jklt3E4uO6E GojXRNqk30mwYmBDkHaHl4V7jac0H26SRs96vgXtG8vRRUdKbLJNUuLA94Uot3X7B5cj AOw8JOyZRoXB54dr7Ar57OAHEovvlouRqfPvy4hFfUYhd13oz0yTB5hskEUZzMpowzIx SuNlwGUdotDAWq6WPGIEfAk6flMMAY8H7RpvU05TaWSf/mKFphzNV0kFU1pshN7w8J84 0ntw== X-Gm-Message-State: AOAM531NVeBSYgNeMDgWb+6NbLkXPN3yKUXpVUlFwHpXJWgORFaMagJW P6tuu1506McHMdd9deSkDS77AFjAkao= X-Google-Smtp-Source: ABdhPJxHmE+Tbo4eTg5lX/5uhDJQairYgi8um03heuBil3+iLW5jdO8SeS8hA/VSJ7whzNmvuMoSxw== X-Received: by 2002:adf:f9c9:: with SMTP id w9mr4211032wrr.107.1626166929050; Tue, 13 Jul 2021 02:02:09 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id l39sm829488wms.1.2021.07.13.02.02.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jul 2021 02:02:08 -0700 (PDT) From: zimoun To: Maxim Cournoyer Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> Date: Tue, 13 Jul 2021 10:49:51 +0200 In-Reply-To: <86y2bilo26.fsf@gmail.com> (zimoun's message of "Wed, 09 Jun 2021 23:34:41 +0200") Message-ID: <86wnpuoaxc.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: Mathieu Othacehe , 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, What is the status of this old bug#30290 [1]? 1: On Wed, 09 Jun 2021 at 23:34, zimoun wrote: > On Wed, 24 Mar 2021 at 22:55, zimoun wrote: >> On Mon, 29 Jan 2018 at 22:07, Maxim Cournoyer >> wrote: >>> When a substitute server used by guix-daemon is offline, the daemon will >>> keep attempting to connect to it, even when it shouldn't need any data >>> (ran 'sudo guix system reconfigure my-config.scm' multiple times in a >>> row. >>> >>> With the disconnected server (bayfront in my case), that command would >>> take close to 8 minutes, with many system calls like: >>> >>> connect(14, {sa_family=3DAF_INET, sin_port=3Dhtons(443), >>> sin_addr=3Dinet_addr("141.255.128.56")}, 16) =3D -1 EINPROGRESS >>> >>> which wasted 5 seconds each time. >>> >>> After removing this server from my substitute servers list, the same >>> operation (system reconfigure) is 8 times faster (1 minute). >>> >>> Suggestion: the daemon should stop trying to use the offline substitute >>> server after trying for X times, and print a warning about it. > > [...] > >> From: Maxim Cournoyer >> Date: Fri, 18 Dec 2020 22:04:04 -0500 (24 weeks, 4 days, 18 hours ago) >> >> I'm not sure if the recent offloading work that Mathieu did touched that >> topic. I'd need to test the scenario. Perhaps a system test would be >> useful. >> ---------- >> >> From: Ludovic Court=C3=A8s >> Date: Tue, 22 Dec 2020 16:16:08 +0100 >> Date: Tue, 22 Dec 2020 16:16:08 +0100 (24 weeks, 1 day, 6 hours ago) >> >> Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 >> (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98gu= ix >> substitute --query=E2=80=99 process won=E2=80=99t retry connections to a= n unreachable >> host. >> ---------- >> >> From: Efraim Flashner >> Date: Mon, 28 Dec 2020 14:19:02 +0200 >> Date: Mon, 28 Dec 2020 14:19:02 +0200 (23 weeks, 2 days, 9 hours ago) >> >> Occasionally my internet drops itself, and I find I'm left forever >> waiting for a timeout to see what sources I have cached locally. >> ---------- > > What is the current stats of this bug? Is it still happening with the > recent improvements of Cuirass? After reading all this, I think this bug can be closed. WDYT? Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 18 07:36:39 2021 Received: (at 30290) by debbugs.gnu.org; 18 Aug 2021 11:36:39 +0000 Received: from localhost ([127.0.0.1]:55016 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGJs7-0003g2-Hk for submit@debbugs.gnu.org; Wed, 18 Aug 2021 07:36:39 -0400 Received: from mail-wr1-f51.google.com ([209.85.221.51]:42754) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGJs5-0003fp-Ur for 30290@debbugs.gnu.org; Wed, 18 Aug 2021 07:36:38 -0400 Received: by mail-wr1-f51.google.com with SMTP id q11so2989931wrr.9 for <30290@debbugs.gnu.org>; Wed, 18 Aug 2021 04:36:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=5uJILfTqLoP7NYakR0DRs8KVr9RjnhbTgUs08UJYZaY=; b=upYcEDkJdgatW+GBa2mk19uIPMPfH512nq3BinULkRUgLWj0uO92LRj87k1J10jQm/ 8YbGB6Dh+Fnw8h80lDq1poVqx05g6fhWZ5M0kSmz+B/xXp5w1SNN1tUXqsfQf10JMmHO ahe+qEL1R68kFWQCPI++hCyO1KQXbEFTRUc3O9QYKuE3HznrE3+CCmCQNLuMAvtwDXaw W/xxohFKFQa274cKeI32rO4YGHBqDBRtpS6NX7744MjEyGTETSqnfyDmly8JziVcYlyB uRHhxUiL97iGk+JeHm0BT2m4ssC8x7IaCgHxzCyrb74ZUVM6fJqQ3L8z8AJoGYh7fAvL w0Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=5uJILfTqLoP7NYakR0DRs8KVr9RjnhbTgUs08UJYZaY=; b=SE1UsYq9L8mN9UmOjFzekrD9B9SXEXQ/febY78F/7MoTGuX8+ASNF/9Z66AbBqU7m9 VNplSkjUoLEv74PNeUzlrUblXsa9pfsQOCyEKH4JPogevIGhkUT9q0YPWRAh7stMNv21 xc+SR759zlXmykoM5Aaf0jNVxCWq5KHmGz+pUylHSYvOWZcpm7X8dd3hzt+CF1ILUvlh G07Kyud+GkTv17D+aLdbFuZN9IY7YOY7mubl4nVZ14Ny2s/9dAkOascWI5n5E9uQ3RP7 Nxz2kLrHOIV9AIDJELiFj5nkZvTKMuCWGlWPIBliN0VHP9OGTXXIMrzC5H1dwOLxE4ap TNvw== X-Gm-Message-State: AOAM531k2dn2oawpnNiayc8XY0m6FfVG19Noz7okSQ7Q+1tVYyuTdmyu EIyww+f0HCGM5YHrSjt7eKA+XeSvgFGnkA== X-Google-Smtp-Source: ABdhPJy8AWCN7HpItw/OC8hjGHQd3gKEXVvCIfRBLyVxbJdXgF/nxzB9+NEMTH9n9AOcaryft8uW1g== X-Received: by 2002:adf:ee90:: with SMTP id b16mr9855175wro.185.1629286591897; Wed, 18 Aug 2021 04:36:31 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id u6sm5565219wrp.83.2021.08.18.04.36.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Aug 2021 04:36:31 -0700 (PDT) From: zimoun To: Maxim Cournoyer Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> <86wnpuoaxc.fsf@gmail.com> Date: Wed, 18 Aug 2021 13:19:14 +0200 In-Reply-To: <86wnpuoaxc.fsf@gmail.com> (zimoun's message of "Tue, 13 Jul 2021 10:49:51 +0200") Message-ID: <86k0kjknkd.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: Mathieu Othacehe , 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Maxim, Reading the discussion=E2=80=A6 On Tue, 13 Jul 2021 at 10:49, zimoun wrote: > What is the status of this old bug#30290 [1]? > > 1: > > On Wed, 09 Jun 2021 at 23:34, zimoun wrote: >> On Wed, 24 Mar 2021 at 22:55, zimoun wrote: >>> On Mon, 29 Jan 2018 at 22:07, Maxim Cournoyer >>> wrote: >>>> When a substitute server used by guix-daemon is offline, the daemon wi= ll >>>> keep attempting to connect to it, even when it shouldn't need any data >>>> (ran 'sudo guix system reconfigure my-config.scm' multiple times in a >>>> row. >>>> >>>> With the disconnected server (bayfront in my case), that command would >>>> take close to 8 minutes, with many system calls like: >>>> >>>> connect(14, {sa_family=3DAF_INET, sin_port=3Dhtons(443), >>>> sin_addr=3Dinet_addr("141.255.128.56")}, 16) =3D -1 EINPROGRESS >>>> >>>> which wasted 5 seconds each time. >>>> >>>> After removing this server from my substitute servers list, the same >>>> operation (system reconfigure) is 8 times faster (1 minute). >>>> >>>> Suggestion: the daemon should stop trying to use the offline substitute >>>> server after trying for X times, and print a warning about it. >> >> [...] >> >>> From: Maxim Cournoyer >>> Date: Fri, 18 Dec 2020 22:04:04 -0500 (24 weeks, 4 days, 18 hours ago) >>> >>> I'm not sure if the recent offloading work that Mathieu did touched that >>> topic. I'd need to test the scenario. Perhaps a system test would be >>> useful. >>> ---------- >>> >>> From: Ludovic Court=C3=A8s >>> Date: Tue, 22 Dec 2020 16:16:08 +0100 >>> Date: Tue, 22 Dec 2020 16:16:08 +0100 (24 weeks, 1 day, 6 hours ago) >>> >>> Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 >>> (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98g= uix >>> substitute --query=E2=80=99 process won=E2=80=99t retry connections to = an unreachable >>> host. >>> ---------- >>> >>> From: Efraim Flashner >>> Date: Mon, 28 Dec 2020 14:19:02 +0200 >>> Date: Mon, 28 Dec 2020 14:19:02 +0200 (23 weeks, 2 days, 9 hours ago) >>> >>> Occasionally my internet drops itself, and I find I'm left forever >>> waiting for a timeout to see what sources I have cached locally. >>> ---------- >> >> What is the current stats of this bug? Is it still happening with the >> recent improvements of Cuirass? > > After reading all this, I think this bug can be closed. WDYT? =E2=80=A6I appears to me that this bug could be close. WDYT? Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 18 09:18:15 2021 Received: (at 30290) by debbugs.gnu.org; 18 Aug 2021 13:18:15 +0000 Received: from localhost ([127.0.0.1]:55246 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGLSR-0006Fz-J4 for submit@debbugs.gnu.org; Wed, 18 Aug 2021 09:18:15 -0400 Received: from mail-qk1-f179.google.com ([209.85.222.179]:35749) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGLSQ-0006Fd-2D for 30290@debbugs.gnu.org; Wed, 18 Aug 2021 09:18:14 -0400 Received: by mail-qk1-f179.google.com with SMTP id 22so2923745qkg.2 for <30290@debbugs.gnu.org>; Wed, 18 Aug 2021 06:18:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=he7RucPYnu3QN3qXZobF6EfgjYXFFPE+f6TYDH5PePI=; b=rmye+XPpLwI7QzNbcLbMSdRLfIEQMpW4Jyyz3MY0Vd1bXcJhbWg7nV9e/tlkExpVUy yQmfCYDQ+ol//30UkzkaTRJ7/Gb8bWLLbxTrWLsKPVRftYz15K12dAowY/yoOPO4qXEm Jb9Fwe7NRuQnXf9NiSBIUD4aRyBR+SrODQpdLNXHs77e5qkMc7xI9HWy/on9Cw5gXbzI cjsQewp7zBFDjhrrYhBpBJhzO29q5AcfvdbKsGTTDFOFrDLzatKcvpBfgJyWiKfjkISB cOv+ymO5qUiWcTIriUpHoFrRBLLT4DqGnFu/Hsx3X5JMEI/Msf0occ7UzXqaSB2OdNU+ 6Uww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=he7RucPYnu3QN3qXZobF6EfgjYXFFPE+f6TYDH5PePI=; b=lh2hlInQcgvuI2ozRRluZiuNRQA5Lj14J1jApZSwMLZVmhWBYXrp1I4eIQppBACCFu 31g5EBO8GHHCHCuycQue0tyndgD0/MtheuXZM1WXNmkpTBxcHR8MBxgdAjthjN2d/SVL GsuMHgOExasAmx1voOsqqtOhTrYyiycqw/sVvDgSTv2JNICpSU3x9Htlog47XxrOx9gv E9JvR1cJKGjk82XwwoQPxUoGGHgNTVtuDkmFZAVOohEh0D1PSnv8vlX93rTMJIb00KSu Ji/OQP57y8L/ODHAE+hEc0XgAFrzOe27LBzrxIDpa5KQThGWR7a8hQPAg/RFizquo33/ 78mg== X-Gm-Message-State: AOAM531nEFfAkEHVuMpOUA2jlRmSLkRXTM6ljlifScIvYYpJ7WEKKWph 3vV1ivjjY/WOBLrXbnufyWvafoeMJwc= X-Google-Smtp-Source: ABdhPJxMY3x9dCBoRNISSP8FPn3giHay1vjUkQkr+kFGZy+P64X9sKaPdu8DuMBj2uFjCUSKQsFjgQ== X-Received: by 2002:a05:620a:b0f:: with SMTP id t15mr9470226qkg.352.1629292688461; Wed, 18 Aug 2021 06:18:08 -0700 (PDT) Received: from hurd (dsl-236-123-187.b2b2c.ca. [207.236.123.187]) by smtp.gmail.com with ESMTPSA id i14sm3398262qka.66.2021.08.18.06.18.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Aug 2021 06:18:07 -0700 (PDT) From: Maxim Cournoyer To: zimoun Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> <86wnpuoaxc.fsf@gmail.com> Date: Wed, 18 Aug 2021 09:18:06 -0400 In-Reply-To: <86wnpuoaxc.fsf@gmail.com> (zimoun's message of "Tue, 13 Jul 2021 10:49:51 +0200") Message-ID: <87r1eqx569.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: Mathieu Othacehe , 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, And sorry for failing to produce a reply earlier :-). zimoun writes: [...] >>> From: Maxim Cournoyer >>> Date: Fri, 18 Dec 2020 22:04:04 -0500 (24 weeks, 4 days, 18 hours ago) >>> >>> I'm not sure if the recent offloading work that Mathieu did touched that >>> topic. I'd need to test the scenario. Perhaps a system test would be >>> useful. >>> ---------- >>> >>> From: Ludovic Court=C3=A8s >>> Date: Tue, 22 Dec 2020 16:16:08 +0100 >>> Date: Tue, 22 Dec 2020 16:16:08 +0100 (24 weeks, 1 day, 6 hours ago) >>> >>> Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 >>> (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98g= uix >>> substitute --query=E2=80=99 process won=E2=80=99t retry connections to = an unreachable >>> host. >>> ---------- >>> >>> From: Efraim Flashner >>> Date: Mon, 28 Dec 2020 14:19:02 +0200 >>> Date: Mon, 28 Dec 2020 14:19:02 +0200 (23 weeks, 2 days, 9 hours ago) >>> >>> Occasionally my internet drops itself, and I find I'm left forever >>> waiting for a timeout to see what sources I have cached locally. >>> ---------- >> >> What is the current stats of this bug? Is it still happening with the >> recent improvements of Cuirass? > > After reading all this, I think this bug can be closed. WDYT? Were you able to replay a scenario in which a substitute server is made unreachable? That's the information that I'd like to have/see before closing. I don't come across unreachable substitute servers often, and can't think of a way to easily test this. I could make it hang by dropping the input/output connections with iptables to a remote guix publish server, but then SSH also hangs, so perhaps that's expected. I'll try to configure a couple local machines to act as publish servers, and disconnect them from the network to see what happens. Thanks, Maxim From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 18 21:55:04 2021 Received: (at 30290) by debbugs.gnu.org; 19 Aug 2021 01:55:04 +0000 Received: from localhost ([127.0.0.1]:57828 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGXGp-0007ZT-5J for submit@debbugs.gnu.org; Wed, 18 Aug 2021 21:55:04 -0400 Received: from mail-qt1-f179.google.com ([209.85.160.179]:44645) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGXGl-0007Yv-Bq for 30290@debbugs.gnu.org; Wed, 18 Aug 2021 21:55:01 -0400 Received: by mail-qt1-f179.google.com with SMTP id r21so3280473qtw.11 for <30290@debbugs.gnu.org>; Wed, 18 Aug 2021 18:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=yOU3dDgTsSk/jobRNeah8mqHIVuNQWSE7u9W/YoMiFI=; b=VDS/nnMlDk8M+zzcbFt8XTCbj3Pde5txNU11fgH+d1+cweCEjqpUA6RIv43fLUcYTe nrB8MEUdNa5yG42jXqoqn77SxFIo1pNYIDbrYphFZSi54DGl+65XTnoc6rDVfM1uiXvh Vh0pJtQAsKtL3q2svUqP6OvtrYkiysf0ZnN0SlC2rfkjSLF41oT6FFUcXXsDs30tGZyQ BiCgZIYc/PsvV4wpgsJmqLzCiYLz0eQ4HhQP86AWxKHKRD0FANRLV6HY2EYSXtA735Ye XdkE9MjNBIetRqB6orbEojOA5KB6lMnIbJtVXOTVDM8sWAP1Oi5uoAJvP49kWjBgcG+I WobQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=yOU3dDgTsSk/jobRNeah8mqHIVuNQWSE7u9W/YoMiFI=; b=ZX4AXgQYEArUgjQnmKMlxp11yLvy33uomdkc7HKp5PWtjvAQ3PIzRfqWj5UFFPGdDr W2lF1qWD7oPRhpHGrR4iYHqy47lt+gmtsJaWgpP/uJdiBOq3bXaSAiGjz5lHRPRkDsz8 UZc+l6mTVINVVjRuna/5DsyEOoUwm3Haea+u0qxIxS1qW88mzhesfodVuChBtBm8FhBj InCs/SnOq6+DzRxbxqUG2ohr1gK5usosnCB1y0jhHmXMH6Ce5Zrj4NEWwWPap7KXx7ql f9i/ELERAHIUAiCaKrtfuajd/ORrgIQKJpXHf3s4G5V3QjlpHuhsmo7NtC92TCpCETQH JK3A== X-Gm-Message-State: AOAM530rN9JDklt5tvTxIy/CAc8uMRfzNPlAGiBwa9Zsx47ItfilbAS8 VRnBStHO7XzbBWst7YTTx0ZlsXfeAp0= X-Google-Smtp-Source: ABdhPJzA60juAupW2ML45L76daZ1pdeK7WaUEzdWQTvI5LxOpK0//zWJhpvoe16ke3JNdyVvHQFTrA== X-Received: by 2002:a05:622a:c1:: with SMTP id p1mr10695535qtw.365.1629338093527; Wed, 18 Aug 2021 18:54:53 -0700 (PDT) Received: from hurd (dsl-236-123-187.b2b2c.ca. [207.236.123.187]) by smtp.gmail.com with ESMTPSA id c4sm697697qtd.62.2021.08.18.18.54.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Aug 2021 18:54:52 -0700 (PDT) From: Maxim Cournoyer To: zimoun Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> <86wnpuoaxc.fsf@gmail.com> <87r1eqx569.fsf@gmail.com> Date: Wed, 18 Aug 2021 21:54:46 -0400 In-Reply-To: <87r1eqx569.fsf@gmail.com> (Maxim Cournoyer's message of "Wed, 18 Aug 2021 09:18:06 -0400") Message-ID: <87mtpew655.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: Mathieu Othacehe , 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Maxim Cournoyer writes: > Hi, > > And sorry for failing to produce a reply earlier :-). > > zimoun writes: > > [...] > >>>> From: Maxim Cournoyer >>>> Date: Fri, 18 Dec 2020 22:04:04 -0500 (24 weeks, 4 days, 18 hours ago) >>>> >>>> I'm not sure if the recent offloading work that Mathieu did touched th= at >>>> topic. I'd need to test the scenario. Perhaps a system test would be >>>> useful. >>>> ---------- >>>> >>>> From: Ludovic Court=C3=A8s >>>> Date: Tue, 22 Dec 2020 16:16:08 +0100 >>>> Date: Tue, 22 Dec 2020 16:16:08 +0100 (24 weeks, 1 day, 6 hours ago) >>>> >>>> Is it still a problem? Commit 4f5234be0378368e6af25925db46612838d25e58 >>>> (Nov. 2019) added a table of unreachable hosts. That way, a =E2=80=98= guix >>>> substitute --query=E2=80=99 process won=E2=80=99t retry connections to= an unreachable >>>> host. >>>> ---------- >>>> >>>> From: Efraim Flashner >>>> Date: Mon, 28 Dec 2020 14:19:02 +0200 >>>> Date: Mon, 28 Dec 2020 14:19:02 +0200 (23 weeks, 2 days, 9 hours ago) >>>> >>>> Occasionally my internet drops itself, and I find I'm left forever >>>> waiting for a timeout to see what sources I have cached locally. >>>> ---------- >>> >>> What is the current stats of this bug? Is it still happening with the >>> recent improvements of Cuirass? >> >> After reading all this, I think this bug can be closed. WDYT? > > Were you able to replay a scenario in which a substitute server is made > unreachable? That's the information that I'd like to have/see before > closing. I don't come across unreachable substitute servers often, and > can't think of a way to easily test this. > > I could make it hang by dropping the input/output connections with > iptables to a remote guix publish server, but then SSH also hangs, so > perhaps that's expected. > > I'll try to configure a couple local machines to act as publish servers, > and disconnect them from the network to see what happens. > > Thanks, > > Maxim I managed to get some problematic behavior from guix substitute: My test protocole was roughly like this: 1. Setup a 2nd machine (machine B) to act as a substitute server, and guix pull to the same commit as that of my main machine (machine A). 2. Run guix build -m manifest.scm on machine B (IP: 192.168.10.172). 3. On the machine A, run the command below, explicitly listing machine A as a substitute URL, along ci.guix.gnu.org. During a download from A, break the connection (I pulled the wifi USB dongle out): $ guix build -m ~/stow/guix/manifest.scm --substitute-urls=3D'http://192.16= 8.10.172 https://ci.guix.gnu.org' --no-offload substitute: updating substitutes from 'http://192.168.10.172:80'... 100.0% substitute: updating substitutes from 'http://192.168.10.172'... 100.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% The following derivations will be built: /gnu/store/lxm7brkbrkkv58c4kzlw1lh3wc0bm8wz-gimp-2.10.24.drv /gnu/store/ddv8jyzwk92nsg1dkv9n3scf6f7w83g5-keepassxc-2.6.6.drv /gnu/store/xky1y32mccplxsb448ziq68by2mvkdaz-ruby-asciidoctor-2.0.10.drv /gnu/store/0ph0sjib0d13n2fsl8w9prnky8g5fkvf-ruby-haml-5.0.4.drv /gnu/store/4dfwfj9qinw4vs6290gdy5qbnqbczm2v-ruby-temple-0.8.2.drv /gnu/store/b12krypa196yg6gzk2bvrh35i1fg5c7x-ruby-tilt-2.0.10.drv /gnu/store/131d8193hi1485ylnb9w8jm3jnlv3iyx-ruby-slim-4.1.0.drv /gnu/store/489nq0jqjby92kv3c6nwrdfqg45l40nw-ruby-sinatra-2.0.8.1.drv /gnu/store/yay3sa8nnq4j0ixwhp3bxfg5vfisfmf1-sflvault-client-0.9.2-1.8de3= 902.drv /gnu/store/2n1xyy0y3nnkrp3mpdifn8r7wf6pzpb0-sflvault-0.9.2-1.8de3902-che= ckout.drv /gnu/store/jsyhy4vxzr9yyg66kzk7w28xffyx050c-python-keyring-1.6.1.drv /gnu/store/kiwn3x2la23f1pa3a5ypsihhc6ja19y5-python-keyring-1.6.1-checkou= t.drv The following files will be downloaded: /gnu/store/2qphwngpvawl6f06d33b2jr18vk1hyc9-module-import-compiled /gnu/store/r7vsb0vl4y66jbq7b56zmrm60q2507zl-wireshark-3.4.7 /gnu/store/wnzx9anjdkmbnkcg5qdd3j77q1w2j1bd-yelp-3.32.2 /gnu/store/vcxwcwlwhvhxj15ma8ik8lghmz8sb2vq-vinagre-3.22.0 /gnu/store/yg8r6kz95p8v03gz0rglpwzrj21npzzw-spacefm-1.0.6 /gnu/store/bn35x60w72ad59a5pd7gmvxgjwgkqvag-youtube-dl-2021.06.06 /gnu/store/xkn540dzpz75hr9cx19xgd3b1r7vgswi-mpv-0.33.1 /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-desktop-4.2.5-debug /gnu/store/4h8ixlh5by2l09vv3rvknmlxv2gm9d6s-linphone-desktop-4.2.5 /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1.4.2 /gnu/store/zczjaxs118155n3mx8w91c24izhx0h0f-ruby-asciimath-2.0.1 /gnu/store/zll4p79a29hw95d2gsh4vjdvd856ry4s-ruby-cucumber-html-formatter= -7.0.0 [...] /gnu/store/sa6hvh9bnw73mpplasbjb3idlv71rvcb-gnome-boxes-3.36.6 /gnu/store/6gy957mhm07zaa001avzkv2d8zhjdl5h-poppler-data-0.4.10 /gnu/store/7kwgmhlsy6qal56h3z19anxmw4c7pf35-diffoscope-177 /gnu/store/hxvlcb4wgw0fpyi9ssc4x6f8w3hlng55-gst-plugins-good-1.18.2 /gnu/store/7bqpzvzanmvb4g1g6gqb1jmrw2j8gv3d-gst-plugins-bad-1.18.2 /gnu/store/f8hzmmnp8cm4yqq0y9cf7rgxl05hf423-cheese-3.38.0 substituting /gnu/store/7kwgmhlsy6qal56h3z19anxmw4c7pf35-diffoscope-177... substituting /gnu/store/ns4n01xgbk6ccvd2z127v71d806rnr6f-inkscape-1.1... substituting /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1.4.= 2... substituting /gnu/store/4h8ixlh5by2l09vv3rvknmlxv2gm9d6s-linphone-desktop-4= .2.5... downloading from http://192.168.10.172/nar/zstd/7kwgmhlsy6qal56h3z19anxmw4c= 7pf35-diffoscope-177 ... diffoscope-177 10.5M= iB/s 00:00 | 128KiB transferreddownloading from http://192.168.10.172/nar/z= std/ns4n01xgbk6ccvd2z127v71d806rnr6f-inkscape-1.1 ... downloading from http://192.168.10.172/nar/zstd/f10an83xvya46ndh61y59qaw5vv= s5f7n-libreoffice-7.1.4.2 ... downloading from http://192.168.10.172/nar/zstd/4h8ixlh5by2l09vv3rvknmlxv2g= m9d6s-linphone-desktop-4.2.5 ... substitution of /gnu/store/4h8ixlh5by2l09vv3rvknmlxv2gm9d6s-linphone-deskto= p-4.2.5 complete substituting /gnu/store/7bqpzvzanmvb4g1g6gqb1jmrw2j8gv3d-gst-plugins-bad-1.= 18.2... downloading from http://192.168.10.172/nar/zstd/7bqpzvzanmvb4g1g6gqb1jmrw2j= 8gv3d-gst-plugins-bad-1.18.2 ... substitution of /gnu/store/7kwgmhlsy6qal56h3z19anxmw4c7pf35-diffoscope-177 = complete substituting /gnu/store/hxvlcb4wgw0fpyi9ssc4x6f8w3hlng55-gst-plugins-good-1= .18.2... downloading from http://192.168.10.172/nar/zstd/hxvlcb4wgw0fpyi9ssc4x6f8w3h= lng55-gst-plugins-good-1.18.2 ... =20 ^ It hung up there, waiting indefinitely. What I would have expected instead, would have been to find out about the network failure, and retry from the other available substitute URL, else build locally. At that time, all the 'substitute' processes are blocked on a read(2) call, while one of the guix-daemon is also, and 2 others are blocked on select. That's not the same as the original report though; let's try to reproduce that one by running the same command again, while the substitute server B is still disconnected: --8<---------------cut here---------------start------------->8--- $ time guix build -m ~/stow/guix/manifest.scm --substitute-urls=3D'http://1= 92.168.10.172 https://ci.guix.gnu.org' --no-offload substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0%g= uix substitute: warning: 192.168.10.172: connection failed: No route to host substitute:=20 substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% The following derivations will be built: /gnu/store/lxm7brkbrkkv58c4kzlw1lh3wc0bm8wz-gimp-2.10.24.drv /gnu/store/ddv8jyzwk92nsg1dkv9n3scf6f7w83g5-keepassxc-2.6.6.drv /gnu/store/xky1y32mccplxsb448ziq68by2mvkdaz-ruby-asciidoctor-2.0.10.drv /gnu/store/0ph0sjib0d13n2fsl8w9prnky8g5fkvf-ruby-haml-5.0.4.drv /gnu/store/4dfwfj9qinw4vs6290gdy5qbnqbczm2v-ruby-temple-0.8.2.drv /gnu/store/b12krypa196yg6gzk2bvrh35i1fg5c7x-ruby-tilt-2.0.10.drv /gnu/store/131d8193hi1485ylnb9w8jm3jnlv3iyx-ruby-slim-4.1.0.drv /gnu/store/489nq0jqjby92kv3c6nwrdfqg45l40nw-ruby-sinatra-2.0.8.1.drv /gnu/store/yay3sa8nnq4j0ixwhp3bxfg5vfisfmf1-sflvault-client-0.9.2-1.8de3= 902.drv /gnu/store/2n1xyy0y3nnkrp3mpdifn8r7wf6pzpb0-sflvault-0.9.2-1.8de3902-che= ckout.drv /gnu/store/jsyhy4vxzr9yyg66kzk7w28xffyx050c-python-keyring-1.6.1.drv /gnu/store/kiwn3x2la23f1pa3a5ypsihhc6ja19y5-python-keyring-1.6.1-checkou= t.drv The following files will be downloaded: /gnu/store/2qphwngpvawl6f06d33b2jr18vk1hyc9-module-import-compiled /gnu/store/r7vsb0vl4y66jbq7b56zmrm60q2507zl-wireshark-3.4.7 /gnu/store/wnzx9anjdkmbnkcg5qdd3j77q1w2j1bd-yelp-3.32.2 /gnu/store/vcxwcwlwhvhxj15ma8ik8lghmz8sb2vq-vinagre-3.22.0 /gnu/store/yg8r6kz95p8v03gz0rglpwzrj21npzzw-spacefm-1.0.6 [...] /gnu/store/zvnnafb7hmiklj8wpvn9qdc85w8rdprl-gnucash-4.2-doc /gnu/store/rp2ai59zvx5m0k6db0cnkx6nn9n41qjd-gnucash-4.2 /gnu/store/hmy026sjdl489sy3i25r2kz9f70h3awm-gnucash-4.2-python /gnu/store/1bspzx0103mr17mxhgw0d9zdlgca2psq-spice-gtk-0.37 /gnu/store/bribnmf6djvh1d3rjr2vs5y97141ad97-osinfo-db-20201218 /gnu/store/r1a25sizf07nmh388ri4qybshzlcxbqd-libosinfo-1.7.1 /gnu/store/2z7p7ynamiarxkx4hnk8dk377xqgm3zl-tracker-2.3.5 /gnu/store/458bw9h0f0ybjdqwg4zm5gjjsmfxbalx-webkitgtk-2.32.3 /gnu/store/sa6hvh9bnw73mpplasbjb3idlv71rvcb-gnome-boxes-3.36.6 /gnu/store/6gy957mhm07zaa001avzkv2d8zhjdl5h-poppler-data-0.4.10 /gnu/store/hxvlcb4wgw0fpyi9ssc4x6f8w3hlng55-gst-plugins-good-1.18.2 /gnu/store/7bqpzvzanmvb4g1g6gqb1jmrw2j8gv3d-gst-plugins-bad-1.18.2 /gnu/store/f8hzmmnp8cm4yqq0y9cf7rgxl05hf423-cheese-3.38.0 substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substituting /gnu/store/ns4n01xgbk6ccvd2z127v71d806rnr6f-inkscape-1.1... substituting /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1.4.= 2... substituting /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-desktop-4= .2.5-debug... substituting /gnu/store/bribnmf6djvh1d3rjr2vs5y97141ad97-osinfo-db-20201218= ... substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% guix substitute: error: connect*: No route to host guix substitute: error: connect*: No route to host guix substitute: error: connect*: No route to host guix substitute: warning: 192.168.10.172: connection failed: No route to ho= st downloading from https://ci.guix.gnu.org/nar/lzip/bribnmf6djvh1d3rjr2vs5y97= 141ad97-osinfo-db-20201218 ... osinfo-db-20201218 88KiB 5.9MiB/s 00= :00 [############# ] 73.1%substitution of /gnu/store/ns4n01xgbk6ccvd2z= 127v71d806rnr6f-inkscape-1.1 failed substitution of /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1= .4.2 failed substitution of /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-deskto= p-4.2.5-debug failed guix build: error: corrupt input while restoring archive from # real 1m13.549s user 0m25.348s sys 0m0.721s --8<---------------cut here---------------end--------------->8--- Hmm. Let's try again, --8<---------------cut here---------------start------------->8--- $ time guix build -m ~/stow/guix/manifest.scm --substitute-urls=3D'http://1= 92.168.10.172 https://ci.guix.gnu.org' --no-offload substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0%g= uix substitute: warning: 192.168.10.172: connection failed: No route to host substitute:=20 substitute: updating substitutes from 'http://192.168.10.172'... 0.0% The following derivations will be built: /gnu/store/lxm7brkbrkkv58c4kzlw1lh3wc0bm8wz-gimp-2.10.24.drv /gnu/store/ddv8jyzwk92nsg1dkv9n3scf6f7w83g5-keepassxc-2.6.6.drv /gnu/store/xky1y32mccplxsb448ziq68by2mvkdaz-ruby-asciidoctor-2.0.10.drv /gnu/store/0ph0sjib0d13n2fsl8w9prnky8g5fkvf-ruby-haml-5.0.4.drv /gnu/store/4dfwfj9qinw4vs6290gdy5qbnqbczm2v-ruby-temple-0.8.2.drv /gnu/store/b12krypa196yg6gzk2bvrh35i1fg5c7x-ruby-tilt-2.0.10.drv /gnu/store/131d8193hi1485ylnb9w8jm3jnlv3iyx-ruby-slim-4.1.0.drv /gnu/store/489nq0jqjby92kv3c6nwrdfqg45l40nw-ruby-sinatra-2.0.8.1.drv /gnu/store/yay3sa8nnq4j0ixwhp3bxfg5vfisfmf1-sflvault-client-0.9.2-1.8de3= 902.drv /gnu/store/2n1xyy0y3nnkrp3mpdifn8r7wf6pzpb0-sflvault-0.9.2-1.8de3902-che= ckout.drv /gnu/store/jsyhy4vxzr9yyg66kzk7w28xffyx050c-python-keyring-1.6.1.drv /gnu/store/kiwn3x2la23f1pa3a5ypsihhc6ja19y5-python-keyring-1.6.1-checkou= t.drv The following files will be downloaded: /gnu/store/2qphwngpvawl6f06d33b2jr18vk1hyc9-module-import-compiled /gnu/store/r7vsb0vl4y66jbq7b56zmrm60q2507zl-wireshark-3.4.7 /gnu/store/wnzx9anjdkmbnkcg5qdd3j77q1w2j1bd-yelp-3.32.2 [...] /gnu/store/sa6hvh9bnw73mpplasbjb3idlv71rvcb-gnome-boxes-3.36.6 /gnu/store/6gy957mhm07zaa001avzkv2d8zhjdl5h-poppler-data-0.4.10 /gnu/store/hxvlcb4wgw0fpyi9ssc4x6f8w3hlng55-gst-plugins-good-1.18.2 /gnu/store/7bqpzvzanmvb4g1g6gqb1jmrw2j8gv3d-gst-plugins-bad-1.18.2 /gnu/store/f8hzmmnp8cm4yqq0y9cf7rgxl05hf423-cheese-3.38.0 substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% =20 substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% substituting /gnu/store/ns4n01xgbk6ccvd2z127v71d806rnr6f-inkscape-1.1... substituting /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1.4.= 2... substituting /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-desktop-4= .2.5-debug... substituting /gnu/store/bribnmf6djvh1d3rjr2vs5y97141ad97-osinfo-db-20201218= ... substitute: updating substitutes from 'http://192.168.10.172:80'... 0.0% substitute: updating substitutes from 'http://192.168.10.172'... 0.0% guix substitute: error: connect*: No route to host guix substitute: error: connect*: No route to host guix substitute: error: connect*: No route to host guix substitute: warning: 192.168.10.172: connection failed: No route to ho= st downloading from https://ci.guix.gnu.org/nar/lzip/bribnmf6djvh1d3rjr2vs5y97= 141ad97-osinfo-db-20201218 ... osinfo-db-20201218 88KiB 6.0MiB/s 00= :00 [############# ] 73.1%substitution of /gnu/store/ns4n01xgbk6ccvd2z= 127v71d806rnr6f-inkscape-1.1 failed substitution of /gnu/store/f10an83xvya46ndh61y59qaw5vvs5f7n-libreoffice-7.1= .4.2 failed substitution of /gnu/store/6abwn23grk710qvzvvg1384bs3kc2f8i-linphone-deskto= p-4.2.5-debug failed guix build: error: corrupt input while restoring archive from # real 1m15.216s user 0m24.963s sys 0m0.702s --8<---------------cut here---------------end--------------->8--- Same thing, the daemon is still trying really hard to get something from that dead substitute server, slowing things down. That corrupted archive failure is curious, I wonder if it may be related. We'll have to keep this bug open I'm afraid :-/. Thanks, Maxim From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 18 22:25:20 2021 Received: (at 30290) by debbugs.gnu.org; 19 Aug 2021 02:25:20 +0000 Received: from localhost ([127.0.0.1]:57851 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGXk8-0008Ip-5G for submit@debbugs.gnu.org; Wed, 18 Aug 2021 22:25:20 -0400 Received: from mail-qv1-f54.google.com ([209.85.219.54]:40680) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mGXk7-0008Ic-5h for 30290@debbugs.gnu.org; Wed, 18 Aug 2021 22:25:19 -0400 Received: by mail-qv1-f54.google.com with SMTP id v1so2850924qva.7 for <30290@debbugs.gnu.org>; Wed, 18 Aug 2021 19:25:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=ooli3lBc4FPT3Urz2KdBClAL5WYuQo4lJlEuja6zfyc=; b=JxTEdQT3hXtHkEH46cVJMr9vRMl8GTwSJOdzAbduRTKIfTGJnruOVv9JAKOfLZSC+E MZy/GKBzDMAOgBn2fxjLf8mpqgOdsFpjhBZ4tsjDjKyYJlHfdBUPrA3nSRNPcSAINbPD GGY1K03G2wNtLe+XNqWKLbVQ7blx4ipbXFZR8O3fcAyP+zM9f5nTEbUdMj+0aOtftrug d7ZaiRJ2B+NkX3sM8CyH4agkDGyXUAgUDsDI14Phbzk5QBpHdsCB5WzyRh3RLy9VmVH1 C9dj3Fjy2/DoHaJWm074JeyLSZ7J+EyJb/Ld12TMPfbCqoBGZdx0pbZG1YocmJCgBdPD qAeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=ooli3lBc4FPT3Urz2KdBClAL5WYuQo4lJlEuja6zfyc=; b=dcFf0OvkcBN+bfDJfh7N51TBB9OHijiusrMItPHwx+BVlnoZKJkY9vMbTofII504Il MGZf5TJE5fsVgIcPSh8pmPH9xuvmoLdzLjjsR5wTe5w8TpeurE6sMY0+ga///oHxtlfG dhU3f28yvaaWtOA4HrYCfWPlLmmOuNc37xdnGwTdsPo3g1GHWa1J1ilUPWICt+Tm2K2u 1pmiAqAAJuiUXVw1U3e0GSWy32zjLAhFNXF3GOGzon6gq6QzG/9u52UmVreeoKu1ePTj 3s6TG/ZU5OG8Kbh8izKA3pCCNgYi027OR3gTVbaPDfrFwF7od6iM39nlTAf5PY/C6ZsR yKwQ== X-Gm-Message-State: AOAM533V7rEvo8dJZWxEsCfOcixSgSkSj5GiDEmSnc2mj4XydEC0Ccie IcZneU4GfZV/7FWdVNDWKKXP9UHGXyA= X-Google-Smtp-Source: ABdhPJy5fW3SNNQyoR2ZCkbGDtcjjH3W/2PF6NQRazpb1mlM7z00yoGtXxELzFXKCCzqpC3kry9jdw== X-Received: by 2002:a0c:f248:: with SMTP id z8mr12155875qvl.5.1629339913334; Wed, 18 Aug 2021 19:25:13 -0700 (PDT) Received: from hurd (dsl-236-123-187.b2b2c.ca. [207.236.123.187]) by smtp.gmail.com with ESMTPSA id d20sm755487qtw.53.2021.08.18.19.25.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Aug 2021 19:25:12 -0700 (PDT) From: Maxim Cournoyer To: zimoun Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> <86wnpuoaxc.fsf@gmail.com> <87r1eqx569.fsf@gmail.com> <87mtpew655.fsf@gmail.com> Date: Wed, 18 Aug 2021 22:25:12 -0400 In-Reply-To: <87mtpew655.fsf@gmail.com> (Maxim Cournoyer's message of "Wed, 18 Aug 2021 21:54:46 -0400") Message-ID: <87fsv6w4qf.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: Mathieu Othacehe , 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Extra note: the problems reported earlier (hang or backtrace instead of graceful fallback to other substitute servers) also affect the scenario where substitutes are fetched from uDNS discovered substitute servers (I just tried). Thanks, Maxim From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 23 06:51:53 2022 Received: (at 30290) by debbugs.gnu.org; 23 Mar 2022 10:51:53 +0000 Received: from localhost ([127.0.0.1]:42876 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nWyan-0007hE-B9 for submit@debbugs.gnu.org; Wed, 23 Mar 2022 06:51:53 -0400 Received: from mail-wr1-f41.google.com ([209.85.221.41]:46739) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nWyai-0007gO-Ph for 30290@debbugs.gnu.org; Wed, 23 Mar 2022 06:51:49 -0400 Received: by mail-wr1-f41.google.com with SMTP id h4so1481134wrc.13 for <30290@debbugs.gnu.org>; Wed, 23 Mar 2022 03:51:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=/eFt3y4B7Wxk/l2Ml+j4KzR5YW7I7A/gx9+NskaffLI=; b=hpVnqNlQ4mMP7q9D/JEy1bh33hIClTjrOfyr5hprNikfa7aCg2frPV72KXWA9yoWML FeigIDHKlFwOBbsfqFfFnVoHs1ua/FBuVsLrV5x144ZdCJP2ytMc0jT1y7Gc+7FlVSgK 9dovd7UEV4jMZcK0ZlpVjJZtZn1AwB+HxDI5IfkXqet7TU+GkXMhYsj259PRDzaSTI3U aLLnGJebR+1NDte6s+i63ekTNOM8UOcU3AgPowqqGjQJ/4SfDJXa95s+grvmKPqcRgCH MKJ0heI3/kWcJxUIdHPVICaAOxozj27rB7YiYWGtSSOzIWNNnrBmYyeQERfKyImYXVRK DiZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=/eFt3y4B7Wxk/l2Ml+j4KzR5YW7I7A/gx9+NskaffLI=; b=GQWjXKl0twsfF6urmzwivmaniEUOMylr2fWhw5gefBC2v/TicUurj72yPiBsX3xggC wIX9rPzPIIx9YiV5WBpt4zTpzf1jqhwgYeA8D+FqUPTwLH5vo0vjtjvShlRhKkzNCjiJ CUuP3JJCdIQYP6DAI8Wr2EV1A0TzVovUoELKtzaP9fne9AAKtTNsl43RYhOmyv6r4CWi GPhMcWyiROtM9220cY6QqGNgbpVr6FFJ0ZZHwj3Uc+qwJg9gKQRrRqoxEhI77zGDfM5o 2Oo4m9uUPfDtKxAUOUc727+h0FR6BlrBNsVWO0ghxkl2SevDeKmQW1EPv/QzSIMoTKyx QDnQ== X-Gm-Message-State: AOAM531y3xZvGVZT1po4821Ou+rmQsIw/IKX2yFBXSz5VnFqkbl+when Oteheh1RmZ9tEzS3We9ZVhx9mmK+yi6rsA== X-Google-Smtp-Source: ABdhPJzCNOyUS5hMuWm99NhXsyB4Pv0SfECpWoYtJT88OeXp1I6zvOlF5b3NjnLIFXgxn920bRWpXg== X-Received: by 2002:adf:dbc6:0:b0:205:8e9a:29ab with SMTP id e6-20020adfdbc6000000b002058e9a29abmr351506wrj.7.1648032703118; Wed, 23 Mar 2022 03:51:43 -0700 (PDT) Received: from lili (client-eduroam632.canalip.upmc.fr. [134.157.122.122]) by smtp.gmail.com with ESMTPSA id p12-20020a5d48cc000000b001e6114938a8sm17688002wrs.56.2022.03.23.03.51.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Mar 2022 03:51:42 -0700 (PDT) From: zimoun To: Maxim Cournoyer , ludo@gnu.org Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> <86wnpuoaxc.fsf@gmail.com> <87r1eqx569.fsf@gmail.com> <87mtpew655.fsf@gmail.com> <87fsv6w4qf.fsf@gmail.com> Date: Wed, 23 Mar 2022 11:50:55 +0100 In-Reply-To: <87fsv6w4qf.fsf@gmail.com> (Maxim Cournoyer's message of "Wed, 18 Aug 2021 22:25:12 -0400") Message-ID: <86fsn90wog.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: Mathieu Othacehe , 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Maxim, On Wed, 18 Aug 2021 at 22:25, Maxim Cournoyer wrote: > Extra note: the problems reported earlier (hang or backtrace instead of > graceful fallback to other substitute servers) also affect the scenario > where substitutes are fetched from uDNS discovered substitute servers (I > just tried). I guess this old bug [1] about the daemon hanging is still there, right? How such issue could be tackled? 1: Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 08 11:17:11 2022 Received: (at 30290) by debbugs.gnu.org; 8 Oct 2022 15:17:11 +0000 Received: from localhost ([127.0.0.1]:41235 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ohBZe-00045X-HV for submit@debbugs.gnu.org; Sat, 08 Oct 2022 11:17:10 -0400 Received: from mail-wm1-f44.google.com ([209.85.128.44]:51800) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ohBZX-00042A-RM for 30290@debbugs.gnu.org; Sat, 08 Oct 2022 11:17:07 -0400 Received: by mail-wm1-f44.google.com with SMTP id o5so4447406wms.1 for <30290@debbugs.gnu.org>; Sat, 08 Oct 2022 08:17:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=6zulQCgDtK1kWZ/klxZDIYz8C3yAJrkncLhYaqR2aB8=; b=JRuVoYtwZ+N9SaFzIntyGSGdJVVcM3Wy5DLSk4NuBeAgzttiDPzKcchtRmXumzsYMw l4/M2ivpoU9uPKW+YPlNgkF1cpkqtwB8UVmpmK4mWFbeAyfuxCoZFxE+kYS50UOGj95V 4zTWh65E9IN/8HaRBNztHbS1FnjDE/b7iGZB1Qsk6ysZ858bP66KJjFEgRCpl+rcqorJ TSxEVmjlLmJ6peb3wwWn+VvbFCwQWR+Mc/wi1X7rmuXRInoDoslUKH7fp/0+zMIx255F xtWohofPUF4gT8dHB7HI4k/qSCI8eilVfoyEuFISH+oP/4EaZFP47Xq/93LieMleT0Pr +ngg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=6zulQCgDtK1kWZ/klxZDIYz8C3yAJrkncLhYaqR2aB8=; b=U0C2ThH9vWznYCAP/nUc+cdmcoCBkeHbWo+yofZTr4oeqJKFa09qtn9j7bUwObRX6c e0GbpSQRgpClEoHkJiZpIzWq8nC+nAGizit9qZr0mhtcZaQd6gY3aDv75DM9b3Di8B6g SGw/rwF+GDc7Ls/7lRoqDm5CQjfCXqaHLi26I28YiaLWw5n8rHxsxGVBo/6j2yQoI/rZ sfVqDhkw95QKwG2iScy6XrOkAXY8hSGW47IsYEhX6ecUWAPR51BcH7lPlRYQfEt8do57 lrHq1GVfDR9oXwLKcM7vMROrcq+OyxIhNsVgwihoy4VenPYfD0pj2TH3u0qJ4Zb9ML52 HQ4w== X-Gm-Message-State: ACrzQf0h08xcOE13pK+6saXtFFHc9hH0qi4mCTGnBXoKKnIibkNEAmSI N7bxXFHzIKBkMGhYPf7Wy+dwZF16ttw= X-Google-Smtp-Source: AMsMyM7Zqm/ESUTxVfqVdWWIzu4qRt02nh0y5tpCAWe4LcWT00entWF2jv+U+KA9kZzYx8yyX4J0VA== X-Received: by 2002:a05:600c:a4c:b0:3b4:fc1b:81 with SMTP id c12-20020a05600c0a4c00b003b4fc1b0081mr6642627wmq.125.1665242218330; Sat, 08 Oct 2022 08:16:58 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id t10-20020a05600001ca00b0022ccae2fa62sm4708660wrx.22.2022.10.08.08.16.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 08:16:57 -0700 (PDT) From: zimoun To: Maxim Cournoyer Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> <86wnpuoaxc.fsf@gmail.com> <87r1eqx569.fsf@gmail.com> <87mtpew655.fsf@gmail.com> <87fsv6w4qf.fsf@gmail.com> <86fsn90wog.fsf@gmail.com> Date: Sat, 08 Oct 2022 16:57:48 +0200 In-Reply-To: <86fsn90wog.fsf@gmail.com> (zimoun's message of "Wed, 23 Mar 2022 11:50:55 +0100") Message-ID: <86h70es6qb.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290 Cc: Mathieu Othacehe , ludo@gnu.org, 30290@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Maxim, On Wed, 23 Mar 2022 at 11:50, zimoun wrote: > On Wed, 18 Aug 2021 at 22:25, Maxim Cournoyer wrote: >> Extra note: the problems reported earlier (hang or backtrace instead of >> graceful fallback to other substitute servers) also affect the scenario >> where substitutes are fetched from uDNS discovered substitute servers (I >> just tried). > > I guess this old bug [1] about the daemon hanging is still there, right? > > How such issue could be tackled? > > 1: What could be actionable for tackling this old bug? Cheers, simon From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 09 23:03:26 2022 Received: (at 30290-done) by debbugs.gnu.org; 10 Oct 2022 03:03:26 +0000 Received: from localhost ([127.0.0.1]:45190 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ohj4g-0006TM-Ge for submit@debbugs.gnu.org; Sun, 09 Oct 2022 23:03:26 -0400 Received: from mail-qk1-f175.google.com ([209.85.222.175]:43915) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ohj4f-0006T9-62 for 30290-done@debbugs.gnu.org; Sun, 09 Oct 2022 23:03:25 -0400 Received: by mail-qk1-f175.google.com with SMTP id i12so201900qkm.10 for <30290-done@debbugs.gnu.org>; Sun, 09 Oct 2022 20:03:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=c5FNuC6vsfkDhYlQCB4qfgQsL+/pbxhJMYQIUlADZwI=; b=h3DJ3mFyGPzXCOjxH9sDsQdXLCHGgk/H24nseB4v2zbunl1Tf3OO83D2OCyh9J4CwE gv6vGZ5cVyLzITZvKWzUj8CZA+0HtbEEs5HhbhnFm8Zpa/678hKDnVjIcDEy+sNVjmcj hhEaItz8xA5zEIFN+i0MBjJRcHNp7YQA8DCCx8NZBFAVtNQ07Qkw+780dc4cTo0rZgze faSMjEI4rM6Zky+IR+QV58hiZUyjK1SyhgHTYMR/xQ+r+AFBpoyPeFajICH9NvtYdpOf uJz6kmUqwwiwjgDz6XQqLmzXhdGWKCeNROmjHtFyFDFK3rmnbryjfB4RuzeMD4hFSyZ+ zs7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=c5FNuC6vsfkDhYlQCB4qfgQsL+/pbxhJMYQIUlADZwI=; b=zCwikNQ5itA2yAMfm9kZBnT5ka0YX1yvvOmpr3hSqRO5knnuEwcNXzrbXKS4N28Jn/ M8FEci/T4kH2r2BzeGGK9KuyIRjo7anKSlm5kWO2ezKyhdSAA2pkP1eU+QmUbNmVGNIg p7bWt8Jn7F0w6YrBK6fo+fOXIBJ7e3aLNA9S4iKT5N8oedcs/Ln9G+rhgoZmI+qZgzeD TwuZ8qtsQFcnkX6v7LS+B6/auReJe0Wgb0eZ7aUHEGeNaf/PEepat0MVd3Dfc74wwLm8 41xisNXwydMcvZlrOU1IxTltk3zgxAEdzWGpWB/UhjcDBZ+TvyDsDz4RA+X5muBdAyxq qUFw== X-Gm-Message-State: ACrzQf2H7epX1ZMZnTdlgej6px7offYeDK3QrcsEEc7hES4YRthl1bp/ FZhPqOCs1R0JSy7BRx0eIWrr3WJXAws= X-Google-Smtp-Source: AMsMyM6wXjjBV2BIeo1J5VwrAyPkOyTN5mqptjYWsA+yOz6DQZXP+d+r97XZb7AGjM8tWB1p/KjBOQ== X-Received: by 2002:a37:82c1:0:b0:6cf:8445:6710 with SMTP id e184-20020a3782c1000000b006cf84456710mr11334419qkd.717.1665370999426; Sun, 09 Oct 2022 20:03:19 -0700 (PDT) Received: from hurd (dsl-150-38.b2b2c.ca. [66.158.150.38]) by smtp.gmail.com with ESMTPSA id o8-20020ac841c8000000b00398313f286dsm4415595qtm.40.2022.10.09.20.03.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 09 Oct 2022 20:03:18 -0700 (PDT) From: Maxim Cournoyer To: zimoun Subject: Re: bug#30290: guix-daemon slows to a crawl when a substitute server is offline References: <87fu6o2ge2.fsf@gmail.com> <868s6cb4ji.fsf@gmail.com> <86y2bilo26.fsf@gmail.com> <86wnpuoaxc.fsf@gmail.com> <87r1eqx569.fsf@gmail.com> <87mtpew655.fsf@gmail.com> <87fsv6w4qf.fsf@gmail.com> <86fsn90wog.fsf@gmail.com> <86h70es6qb.fsf@gmail.com> Date: Sun, 09 Oct 2022 23:03:16 -0400 In-Reply-To: <86h70es6qb.fsf@gmail.com> (zimoun's message of "Sat, 08 Oct 2022 16:57:48 +0200") Message-ID: <87zge42xe3.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 30290-done Cc: Mathieu Othacehe , ludo@gnu.org, 30290-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Simon, zimoun writes: > Hi Maxim, > > On Wed, 23 Mar 2022 at 11:50, zimoun wrote: >> On Wed, 18 Aug 2021 at 22:25, Maxim Cournoyer wrote: > >>> Extra note: the problems reported earlier (hang or backtrace instead of >>> graceful fallback to other substitute servers) also affect the scenario >>> where substitutes are fetched from uDNS discovered substitute servers (I >>> just tried). >> >> I guess this old bug [1] about the daemon hanging is still there, right? >> >> How such issue could be tackled? >> >> 1: > > What could be actionable for tackling this old bug? I can't seem to reproduce the original issue, which was that an offline server would slow down fetching of substitutes by repeatedly trying to reach it, at least when setting e.g.: --8<---------------cut here---------------start------------->8--- 127.0.0.2 ci.guix.gnu.org --8<---------------cut here---------------end--------------->8--- In my /etc/hosts file to simulate it's unreachable, with at least another substitute server available in my --substitute-urls list. It goes right to that second server without loosing any time. Closing for now, we can always reopen the issue if it manifests itself in the future. -- Thanks, Maxim From unknown Wed Jun 18 23:05:27 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 07 Nov 2022 12:24:11 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator