From unknown Wed Sep 10 17:01:35 2025 X-Loop: help-debbugs@gnu.org Subject: bug#77283: Shepherd can't parse logs. Resent-From: MSavoritias Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 26 Mar 2025 13:43:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 77283 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 77283@debbugs.gnu.org X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.174299656118358 (code B ref -1); Wed, 26 Mar 2025 13:43:01 +0000 Received: (at submit) by debbugs.gnu.org; 26 Mar 2025 13:42:41 +0000 Received: from localhost ([127.0.0.1]:42129 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1txR1k-0004m2-PR for submit@debbugs.gnu.org; Wed, 26 Mar 2025 09:42:41 -0400 Received: from lists.gnu.org ([2001:470:142::17]:59538) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1txR1i-0004lh-Dp for submit@debbugs.gnu.org; Wed, 26 Mar 2025 09:42:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1txR1V-0001AK-AA for bug-guix@gnu.org; Wed, 26 Mar 2025 09:42:25 -0400 Received: from mail.webarch.email ([81.95.52.48]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1txR1S-0001BA-RL for bug-guix@gnu.org; Wed, 26 Mar 2025 09:42:24 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id E7B3F1A851BA for ; Wed, 26 Mar 2025 13:42:08 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=msavoritias.me; s=20210930; t=1742996530; h=from:subject:date:message-id:to:mime-version:content-type: content-transfer-encoding:content-language; bh=F6J1A/+revhMwMgDfVdcfsWBSELTgZNVMx/ruuCivuo=; b=LbEcl/6Ikr32+fJQEM4bkqaBcQWg82TAbi5gb27e1/HeoftVeneUeV5bzmkjpyIQvaJBpG WGQisuTmGZHN3LYTB3lkS2MwDiFFXlyUvPWXD1celzwGgojcVKU8Ose4lMBw2e2fwhkQN3 xNIfwOmGflGTQQPyFOlvpYoizl0G9FXyMO+DpfDfYcUhIZr4PgJpM7lIxuoJVHRiTAUGfp EZu37O2b1IsDHLCH/KWNkSZSaSbIOFL/oe/kh4oLT/3dubdRuQ0IPCofB3hIGKgvK/Lm4j vOyBSqubKhWl4XUGEj0NpqK5oF3Y2PkefA2GEZQIQd6lMdUaJODDXfAeY4gfwQ== Message-ID: Date: Wed, 26 Mar 2025 15:42:06 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US From: MSavoritias Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Last-TLS-Session-Version: TLSv1.3 Received-SPF: pass client-ip=81.95.52.48; envelope-from=email@msavoritias.me; helo=mail.webarch.email X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Shepherd has been crashing our system with the message: https://upload.magicbroccoli.de/fb6e7843fef693538a5c7c156c0a98fc0aeb6bac/KmA7ovtC2Fcw0Nm0PSCtphwSyDLI1VUF3QrT9PtD/Kuvakaappaus_-_2025-03-26_13-27-11.png This ends up with prosody and openssh crashing, nginx still working. the picture shows shepherd system-log crashing with: uncaught exception, value out of range in fibers. It references an "unknown file" and a string with prosody. I looked into git commits in the last month (bug didn't exist before end of January/beginning of February, it seems that the bug is either shepherd itself, it was updated recently and that is what i was told in the irc. Or it is the certbot timer. Certbot timer is the only service we have installed from the ones that changed. Being certbot makes less sense since prosody is mentioned specifically tho so it seems a bug in how shepherd parses logs. Our whole config is here -> https://codeberg.org/joinjabber/Infra MSavoritias From unknown Wed Sep 10 17:01:35 2025 X-Loop: help-debbugs@gnu.org Subject: bug#77283: Shepherd can't parse logs. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 30 Mar 2025 21:03:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77283 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: MSavoritias Cc: 77283@debbugs.gnu.org Received: via spool by 77283-submit@debbugs.gnu.org id=B77283.174336854612863 (code B ref 77283); Sun, 30 Mar 2025 21:03:01 +0000 Received: (at 77283) by debbugs.gnu.org; 30 Mar 2025 21:02:26 +0000 Received: from localhost ([127.0.0.1]:38249 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tyznW-0003LO-5V for submit@debbugs.gnu.org; Sun, 30 Mar 2025 17:02:26 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:60238) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tyznT-0003Kt-Gq for 77283@debbugs.gnu.org; Sun, 30 Mar 2025 17:02:24 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 27EC12B6; Sun, 30 Mar 2025 23:02:10 +0200 (CEST) Authentication-Results: hera.aquilenet.fr; none X-Virus-Scanned: Debian amavis at hera.aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavis, port 10024) with ESMTP id qNjXOr_OtY31; Sun, 30 Mar 2025 23:02:09 +0200 (CEST) Received: from ribbon (91-160-117-201.subs.proxad.net [91.160.117.201]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 7EA301F3; Sun, 30 Mar 2025 23:02:07 +0200 (CEST) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: (MSavoritias's message of "Wed, 26 Mar 2025 15:42:06 +0200") References: Date: Sun, 30 Mar 2025 23:02:06 +0200 Message-ID: <87bjtidygh.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 27EC12B6 X-Spamd-Result: default: False [5.46 / 15.00]; SPAM_FLAG(5.00)[]; BAYES_HAM(-3.00)[99.99%]; NEURAL_SPAM(3.00)[1.000]; R_MIXED_CHARSET(0.56)[subject]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Level: ***** X-Rspamd-Action: greylist X-Spamd-Bar: +++++ X-Rspamd-Server: hera X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi MSavoritias, MSavoritias skribis: > Shepherd has been crashing our system with the message: > > https://upload.magicbroccoli.de/fb6e7843fef693538a5c7c156c0a98fc0aeb6bac/= KmA7ovtC2Fcw0Nm0PSCtphwSyDLI1VUF3QrT9PtD/Kuvakaappaus_-_2025-03-26_13-27-11= .png > > > This ends up with prosody and openssh crashing, nginx still > working. the picture shows shepherd system-log crashing with: > > uncaught exception, value out of range in fibers. It references an > "unknown file" and a string with prosody. The code that throws that out-of-range exception on =E2=80=98substring=E2= =80=99 is (match:substring m 3) in =E2=80=98parse-system-log-message=E2=80=99. Normally, =E2=80=98regexp-exec=E2=80=99 returns match structures with corre= ct boundaries, but in this case it didn=E2=80=99t. I reported it here: https://issues.guix.gnu.org/77392 Several options come to mind: 1. Run shepherd in a UTF-8 locale; a quick workaround is to run: sudo herd eval root '(setlocale LC_ALL "C.UTF-8")' but we could also do that in Guix System. 2. Avoid (ice-9 regex) altogether in =E2=80=98system-log.scm=E2=80=99 and= use a pure Scheme implementation, SRFI-115-style. Thanks for your bug report! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 31 16:44:28 2025 Received: (at control) by debbugs.gnu.org; 31 Mar 2025 20:44:28 +0000 Received: from localhost ([127.0.0.1]:43125 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tzLzg-00027h-8z for submit@debbugs.gnu.org; Mon, 31 Mar 2025 16:44:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45666) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tzLzd-000272-U9 for control@debbugs.gnu.org; Mon, 31 Mar 2025 16:44:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tzLzU-0004dS-GM for control@debbugs.gnu.org; Mon, 31 Mar 2025 16:44:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to: references; bh=0AUilOlgCxVbSr4fSlsZAIGNWoQZnx2vvqCsIEXUiFA=; b=PTpthguslv+TnC vQ1COHOcr8MbJQ0FNrHeyBxd7cwg+00/DLKqq4VF21ywp+5X5OKdBA4sQdI8oTBuAM+G5Q6Ua79w6 dMCMtaBWbhQJMLHob8NMP/yD8M9nNxzjeD44v8+Z+uWwqzoF0+Gq0q6YCIRx7VldIszN8WlVyhza7 g3RdMjD3TnE7UwzgZUvr/vHvhLBUfCs3uvlMRcux3eraq9rHQxgib4Y0tmcFOyfH67JOEadW05VwA hFYfHNvAl3jGKCL35D4NS+JRdDpNvBKsQ0lSGg2Zsurm7U2SYl9Uri4yXjrEQRvs0caz+6IcpXaFK xYozGzIoMaE+CW5ibt7Q==; Date: Mon, 31 Mar 2025 22:44:13 +0200 Message-Id: <874iz9j5gi.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #77283 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) retitle 77283 [Shepherd] System log crashes when parsing message quit From unknown Wed Sep 10 17:01:35 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: MSavoritias Subject: bug#77283: closed (Re: bug#77283: Shepherd can't parse logs.) Message-ID: References: <87zfh0j3xr.fsf@gnu.org> X-Gnu-PR-Message: they-closed 77283 X-Gnu-PR-Package: guix Reply-To: 77283@debbugs.gnu.org Date: Mon, 31 Mar 2025 21:18:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1743455882-12094-1" This is a multi-part message in MIME format... ------------=_1743455882-12094-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #77283: [Shepherd] System log crashes when parsing message which was filed against the guix package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 77283@debbugs.gnu.org. --=20 77283: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D77283 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1743455882-12094-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 77283-done) by debbugs.gnu.org; 31 Mar 2025 21:17:21 +0000 Received: from localhost ([127.0.0.1]:43166 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tzMVU-00035H-OY for submit@debbugs.gnu.org; Mon, 31 Mar 2025 17:17:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60634) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tzMVR-00034e-RY for 77283-done@debbugs.gnu.org; Mon, 31 Mar 2025 17:17:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tzMVJ-0008OR-QG; Mon, 31 Mar 2025 17:17:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=4T8RVmNxH8G1loyCuXhU2dmjc42abXhN8QEaZisfUEk=; b=is2sVnSuqdwqOkYhVk3R LstmY2eFSeESB46GpctPRC5x+dfImejaLgQ9kTnV/5H0KgTj4dYAzqxR/RWN8h9TNyadP1q++JRYE Lxz1XeqJZ2RbZfFpOmZ8RtyWEzg5x2zR2PeJ7oyZGbgWgcrmxP003O5+iCzhaUhXW2Ss6ejtXveKy PW+Vr8rVMudhrnGZROZJ53LLlgojkZho7+QlJ622fmEg3PQVLEoheW79GFCcrIloQpqC4+cxT2k/l kEg3GHsi0SPrntz8aHzZDGNw5bMyusB2/C1G/ifchoCsNpeRCkF5M4pimardbwGwQFQliwGZ4x1G4 nd/RRyZIoWSEBg==; From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: MSavoritias Subject: Re: bug#77283: Shepherd can't parse logs. In-Reply-To: <87bjtidygh.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Sun, 30 Mar 2025 23:02:06 +0200") References: <87bjtidygh.fsf@gnu.org> Date: Mon, 31 Mar 2025 23:17:04 +0200 Message-ID: <87zfh0j3xr.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 77283-done Cc: 77283-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Ludovic Court=C3=A8s skribis: > The code that throws that out-of-range exception on =E2=80=98substring=E2= =80=99 is > (match:substring m 3) in =E2=80=98parse-system-log-message=E2=80=99. > > Normally, =E2=80=98regexp-exec=E2=80=99 returns match structures with cor= rect > boundaries, but in this case it didn=E2=80=99t. I reported it here: > > https://issues.guix.gnu.org/77392 Fixed in d98d61a8a3f20de46d18ce4a8af05c93fab20b89 by sidestepping the Guile bug. > 1. Run shepherd in a UTF-8 locale; a quick workaround is to run: > > sudo herd eval root '(setlocale LC_ALL "C.UTF-8")' I recommend doing this while waiting for 1.0.4 to be out, normally in less than a month. Ludo=E2=80=99. ------------=_1743455882-12094-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 26 Mar 2025 13:42:41 +0000 Received: from localhost ([127.0.0.1]:42129 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1txR1k-0004m2-PR for submit@debbugs.gnu.org; Wed, 26 Mar 2025 09:42:41 -0400 Received: from lists.gnu.org ([2001:470:142::17]:59538) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1txR1i-0004lh-Dp for submit@debbugs.gnu.org; Wed, 26 Mar 2025 09:42:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1txR1V-0001AK-AA for bug-guix@gnu.org; Wed, 26 Mar 2025 09:42:25 -0400 Received: from mail.webarch.email ([81.95.52.48]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1txR1S-0001BA-RL for bug-guix@gnu.org; Wed, 26 Mar 2025 09:42:24 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id E7B3F1A851BA for ; Wed, 26 Mar 2025 13:42:08 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=msavoritias.me; s=20210930; t=1742996530; h=from:subject:date:message-id:to:mime-version:content-type: content-transfer-encoding:content-language; bh=F6J1A/+revhMwMgDfVdcfsWBSELTgZNVMx/ruuCivuo=; b=LbEcl/6Ikr32+fJQEM4bkqaBcQWg82TAbi5gb27e1/HeoftVeneUeV5bzmkjpyIQvaJBpG WGQisuTmGZHN3LYTB3lkS2MwDiFFXlyUvPWXD1celzwGgojcVKU8Ose4lMBw2e2fwhkQN3 xNIfwOmGflGTQQPyFOlvpYoizl0G9FXyMO+DpfDfYcUhIZr4PgJpM7lIxuoJVHRiTAUGfp EZu37O2b1IsDHLCH/KWNkSZSaSbIOFL/oe/kh4oLT/3dubdRuQ0IPCofB3hIGKgvK/Lm4j vOyBSqubKhWl4XUGEj0NpqK5oF3Y2PkefA2GEZQIQd6lMdUaJODDXfAeY4gfwQ== Message-ID: Date: Wed, 26 Mar 2025 15:42:06 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: bug-guix@gnu.org From: MSavoritias Subject: Shepherd can't parse logs. Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Last-TLS-Session-Version: TLSv1.3 Received-SPF: pass client-ip=81.95.52.48; envelope-from=email@msavoritias.me; helo=mail.webarch.email X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Shepherd has been crashing our system with the message: https://upload.magicbroccoli.de/fb6e7843fef693538a5c7c156c0a98fc0aeb6bac/KmA7ovtC2Fcw0Nm0PSCtphwSyDLI1VUF3QrT9PtD/Kuvakaappaus_-_2025-03-26_13-27-11.png This ends up with prosody and openssh crashing, nginx still working. the picture shows shepherd system-log crashing with: uncaught exception, value out of range in fibers. It references an "unknown file" and a string with prosody. I looked into git commits in the last month (bug didn't exist before end of January/beginning of February, it seems that the bug is either shepherd itself, it was updated recently and that is what i was told in the irc. Or it is the certbot timer. Certbot timer is the only service we have installed from the ones that changed. Being certbot makes less sense since prosody is mentioned specifically tho so it seems a bug in how shepherd parses logs. Our whole config is here -> https://codeberg.org/joinjabber/Infra MSavoritias ------------=_1743455882-12094-1-- From unknown Wed Sep 10 17:01:35 2025 X-Loop: help-debbugs@gnu.org Subject: bug#77283: [Shepherd] System log crashes when parsing message References: Resent-From: Arun Isaac Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 10 Apr 2025 00:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 77283 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 77283@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , MSavoritias Received: via spool by 77283-submit@debbugs.gnu.org id=B77283.174424633925918 (code B ref 77283); Thu, 10 Apr 2025 00:53:02 +0000 Received: (at 77283) by debbugs.gnu.org; 10 Apr 2025 00:52:19 +0000 Received: from localhost ([127.0.0.1]:42823 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u2g9T-0006jy-8c for submit@debbugs.gnu.org; Wed, 09 Apr 2025 20:52:19 -0400 Received: from mugam.systemreboot.net ([139.59.75.54]:51572) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u2g9P-0006je-OW for 77283@debbugs.gnu.org; Wed, 09 Apr 2025 20:52:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=systemreboot.net; s=default; t=1744246317; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=x/B70E22lYL4Q3MiBmW+jZ3UzRwfgW4My+eOeoHryD0=; b=nRKRKNG8VzOY1ebENIci6xtrAKRnkkMl5jUcqEqc3CL06yCqpMq5kmIIFVLZN6QwZzh3O9 ZvAOYZ7sXFkEmcfEeqm1frw8trH7x7g4BZFYOyXWh4f5PZuIwGG69a7cn7ZUsUsrkgqsi5 jjlcwIeVOjeUutL+rhNaFHdtaBCxSAP8WbRy9CCqR7ObgYeHaRyra28ggLLe8UCFIaWHMv G/XmQ/NJ26YbBLjecH2OgQwN+WhMsfpTcUZWteC1/+mnsN6DsahtZgb2BKCOAp0tonCXm8 gBGcGTuMEKIVTlaZSvBFWyP2q4JmR9bg7tWKn8HaV9LZkkk524ZxAJKqJl0Y/A== Received: from localhost ( [192.168.2.1]) by mugam.systemreboot.net (OpenSMTPD) with ESMTPSA id c5f71adf (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 10 Apr 2025 00:51:57 +0000 (UTC) From: Arun Isaac In-Reply-To: <87zfh0j3xr.fsf@gnu.org> Date: Thu, 10 Apr 2025 01:51:51 +0100 Message-ID: <87semgg7o8.fsf@systemreboot.net> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) I ran into this issue today after a system upgrade. Thank you both for identifying and figuring out the issue! I have applied the suggested workaround now. Cheers!