From unknown Tue Jun 17 01:35:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#69596: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?publish=E2=80=99?= memory leak Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 06 Mar 2024 21:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 69596 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 69596@debbugs.gnu.org X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.170976119121679 (code B ref -1); Wed, 06 Mar 2024 21:40:01 +0000 Received: (at submit) by debbugs.gnu.org; 6 Mar 2024 21:39:51 +0000 Received: from localhost ([127.0.0.1]:51797 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rhyzP-0005dZ-2V for submit@debbugs.gnu.org; Wed, 06 Mar 2024 16:39:51 -0500 Received: from lists.gnu.org ([209.51.188.17]:40462) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rhyzN-0005dS-9p for submit@debbugs.gnu.org; Wed, 06 Mar 2024 16:39:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhyyq-00075o-U6 for bug-guix@gnu.org; Wed, 06 Mar 2024 16:39:17 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhyyq-0001LX-I0 for bug-guix@gnu.org; Wed, 06 Mar 2024 16:39:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=cR2wWvr0foz1YvKiA76O2Vr+w000yQC5TGnQ9hAQ0Dw=; b=MlswYMaH/4xxjH tYy8LMeHGdYk+akwD1qkm5Enn8d1msQJxVukJnFvYXHJXPLH6CJBBBmZObbpKw9b6Zmd8LQ7vk5Z/ TuZgIsVUGHcSJtePmZMMrc4M5dHcCa6ze2x6P9vpZR0h623BbZAmXtbhwYLBwC+fKMZbzPo36HcSz m138EYakx2uW1WRisGzUCceUVuvNNJLY2oCwFWs/nk9SG2D4JSCgFBibma20vgDkGc+d95HVTAKI3 25e0PQBj+ijN4zvyJzbwwCNL5aQalWbabGR7y3+lwsgzdOkIowyCnY+BDOIkVxSDeg1GaEwftg1FF 4x5AHiMjvvH0T38bEoaQ==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Septidi 17 =?UTF-8?Q?Vent=C3=B4se?= an 232 de la =?UTF-8?Q?R=C3=A9volution,?= jour du Doronic X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 06 Mar 2024 22:39:09 +0100 Message-ID: <87zfvb6nwi.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) It seems that =E2=80=98guix publish=E2=80=99 has been leaking memory notice= ably I=E2=80=99d say since the beginning of the year on berlin. After roughly 10 days, it has several GiB resident and needs to be restarted or it becomes too unresponsive. I wonder what could be causing this because we used to let it run for months without problems I believe. This is what=E2=80=99s currently running on berlin: /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libexec= /guix/guile \ /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/gui= x \ publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=3Dnar \ --listen=3Dlocalhost --workers=3D8 --ttl=3D15552000s \ --cache=3D/var/cache/guix/publish --cache-bypass-threshold=3D157286400 =E2=80=A6 coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8. I don=E2=80=99t see anything that changed recently, be it in maintenance.git (for the parameters), =E2=80=98guix publish=E2=80=99, guile-{lzlib,zstd}, o= r Guile. Maybe we had just been lucky? Anyone else seeing this? Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 04 17:20:27 2024 Received: (at control) by debbugs.gnu.org; 4 Apr 2024 21:20:27 +0000 Received: from localhost ([127.0.0.1]:34779 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rsUVX-0005ce-Ek for submit@debbugs.gnu.org; Thu, 04 Apr 2024 17:20:27 -0400 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:36082) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rsUVT-0005bK-4k for control@debbugs.gnu.org; Thu, 04 Apr 2024 17:20:25 -0400 Authentication-Results: mail3-relais-sop.national.inria.fr; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=ludo@gnu.org; dmarc=fail (p=none dis=none) d=gnu.org X-IronPort-AV: E=Sophos;i="6.07,180,1708383600"; d="scan'208";a="83931838" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2024 23:20:11 +0200 Date: Thu, 04 Apr 2024 23:20:10 +0200 Message-Id: <87il0wu8ol.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #69596 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) severity 69596 important quit From unknown Tue Jun 17 01:35:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#69596: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?publish=E2=80=99?= memory leak Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 28 Apr 2024 22:05:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 69596 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 69596@debbugs.gnu.org Received: via spool by 69596-submit@debbugs.gnu.org id=B69596.171434186810383 (code B ref 69596); Sun, 28 Apr 2024 22:05:01 +0000 Received: (at 69596) by debbugs.gnu.org; 28 Apr 2024 22:04:28 +0000 Received: from localhost ([127.0.0.1]:53129 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s1CdH-0002hO-L2 for submit@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42008) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s1CdF-0002hG-Ng for 69596@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s1Ccq-0003O6-Qj for 69596@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=AB0aSjCjRLKl/piNrs9wyI4nJqdpC3fBD02dxcPerBY=; b=AQHkT6xLgNar6O+8HlxH 655exyDUmvhqyDuXL+VgQgF1lpKAIsy18GSl0x25iUysOjW4rkuGZQaH/eRImsBk0WMze/M+osNOd vNYXPEyPFAlEysPqvDlC4NfcpEocq8m6bQ/GmZ+UJifPY0Nl/oNafis9cejIghTvgumv7lxS5HypT q9DtEGHTYOFxgFIkcemso34ThWC6Hv4LHSNKHXniEZG5yW2s3dseidpvDCE1UPhdspVxs1gN5Ajf5 2vx8hj9Yf2ZD0hwVWZ9nTajkZGNP9qmwn9J1QYg+NfcBURthLj1+/2W1XlbuptshK0/WQMKcngXaZ /g6TapgFPbwhoA==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: <87zfvb6nwi.fsf@inria.fr> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Wed, 06 Mar 2024 22:39:09 +0100") References: <87zfvb6nwi.fsf@inria.fr> Date: Mon, 29 Apr 2024 00:03:48 +0200 Message-ID: <87le4xgn5n.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Ludovic Court=C3=A8s skribis: > It seems that =E2=80=98guix publish=E2=80=99 has been leaking memory noti= ceably I=E2=80=99d say > since the beginning of the year on berlin. After roughly 10 days, it > has several GiB resident and needs to be restarted or it becomes too > unresponsive. > > I wonder what could be causing this because we used to let it run for > months without problems I believe. > > This is what=E2=80=99s currently running on berlin: > > /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/libex= ec/guix/guile \ > /gnu/store/cpnshv80n3mar5lwp4qqa2dxxxv4zb03-guix-1.4.0-16.aeb4943/bin/g= uix \ > publish -u guix-publish -p 3000 -C lzip:9 -C zstd:19 --nar-path=3Dnar \ > --listen=3Dlocalhost --workers=3D8 --ttl=3D15552000s \ > --cache=3D/var/cache/guix/publish --cache-bypass-threshold=3D157286400 > > =E2=80=A6 coming from commit 21e4d6cd6913eca131f2c0fd0cd509fc843c7eb8. It turned out to be a guile-lzlib leak that had always been present: https://notabug.org/guile-lzlib/guile-lzlib/commit/74bd35b690801a10ed775d= 486fffc7372b1b341c The reason we were seeing it more on berlin is probably because we increased the cache-bypass-threshold, which goes through the =E2=80=98make-lzip-output-port=E2=80=99 code path (as opposed to =E2=80=98call-with-lzip-output-port=E2=80=99). The bug could be reproduced with: guix publish -p 8124 =E2=80=A6 & while true ; do wget -q -O/dev/full http://localhost:8124/nar/lzip/=E2=80= =A6-coreutils-9.1 ; done (Replace the ellipses with the actual store file name of coreutils.) Fixed by commit 7cef6b7ba555a9dfaf6d09cb7e112b0df77d5141, which updates guile-lzlib. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 28 18:04:38 2024 Received: (at control) by debbugs.gnu.org; 28 Apr 2024 22:04:38 +0000 Received: from localhost ([127.0.0.1]:53136 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s1CdS-0002hh-0k for submit@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42872) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s1CdP-0002hW-W7 for control@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s1Cd1-0003Oy-3R for control@debbugs.gnu.org; Sun, 28 Apr 2024 18:04:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to: references; bh=WX3eg4nXR5stbYbQquqk6b7auanIWHWr1ZNh8dKbG3c=; b=seCDcrZ1gwJXdh AcJzY7b7C+qreja8RuDf+5wtO2YB2A0d4wiFyiwqXIsdYMsywbYGveFY9rToHfSSJW3gMzeNQOvZY OPgEOJZWVTnuec0ThVoX3eR/UWqTOviTNV5Zm6dOSb/cCWJmaJJKlDd+3KrbIr3VnWafRLth1wyyA 3CKkYm38hJMDE6z8EH4ZvMoLMUhq/Q94Ygd9IjBq8yX4WeQT53Btdj9JpgM0dPL4MqAS5mlBMuUN3 Hkpo8aVwjQQmFnbESthhYdkrCdHPq4/LP0exqV7UeMBZawqMRIFZ8LzAcCUtUZomeUmpz+28YxH8w Yapaj75yKKQAjjMzx3Gw==; Date: Mon, 29 Apr 2024 00:04:00 +0200 Message-Id: <87jzkhgn5b.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #69596 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) close 69596 quit From unknown Tue Jun 17 01:35:46 2025 X-Loop: help-debbugs@gnu.org Subject: bug#69596: Possibly breaks Guix pull on foreign distros References: <87zfvb6nwi.fsf@inria.fr> In-Reply-To: <87zfvb6nwi.fsf@inria.fr> Resent-From: Jonathan Roberts Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 30 Apr 2024 10:41:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 69596 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 69596@debbugs.gnu.org Received: via spool by 69596-submit@debbugs.gnu.org id=B69596.171447362915906 (code B ref 69596); Tue, 30 Apr 2024 10:41:03 +0000 Received: (at 69596) by debbugs.gnu.org; 30 Apr 2024 10:40:29 +0000 Received: from localhost ([127.0.0.1]:59352 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s1kuS-00048P-5O for submit@debbugs.gnu.org; Tue, 30 Apr 2024 06:40:29 -0400 Received: from mail-lf1-x12b.google.com ([2a00:1450:4864:20::12b]:51453) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s1j16-0002vM-7J for 69596@debbugs.gnu.org; Tue, 30 Apr 2024 04:39:13 -0400 Received: by mail-lf1-x12b.google.com with SMTP id 2adb3069b0e04-51ae2e37a87so6599694e87.2 for <69596@debbugs.gnu.org>; Tue, 30 Apr 2024 01:38:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20230601; t=1714466325; x=1715071125; darn=debbugs.gnu.org; h=to:message-id:subject:date:mime-version:from :content-transfer-encoding:from:to:cc:subject:date:message-id :reply-to; bh=XsO5C6eDSsuUwLBvr9aU+fVs6inMoZIWSV5qGcl4kNw=; b=N17ItfgkqlXqSz85qDkryYtfvvSh9nSYOnJMZ/y+kF/78P6cusMfrlUFZt8UjFmehm di1jCLkN1byZhpIBoEs7cXgterbdm7nv3ibS9YFJp4TXVmNzU88BhzSMADBeSymuKnK0 YVuOMXdSQ/G5kT5FprZH5oMp0yeyoMZRTtxO70iuiZBkWULNvHIhPltv/r35onDSmvhj 0VJVC1wy7gepMeHraeV8/5jB2pQdo3DRfcvBOE3N9aQhli9E6sDDbLWRugDEZE2d9E9v zD/uryGeOUUtciYKUiery6LPGtGqbvOHcQqlq6sANg3t0l2lL4OL/o50h+YM1Q92HAey Yiwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714466325; x=1715071125; h=to:message-id:subject:date:mime-version:from :content-transfer-encoding:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=XsO5C6eDSsuUwLBvr9aU+fVs6inMoZIWSV5qGcl4kNw=; b=eJgr3TzNkD4KyRIq4+/5gAk7zYrYrfCeXwP/rx0W7VUh1tQwnGGSSDQW0tEs8Xar+i B4LWzDjnV+2h7qVR1etiEIwIWowRDerg33aHIDQpKH0uOWQaE5yXs1wLjZXdutcYtNjI IirBTGsiCaMEAjv7nnqejIYX8tfgqtgf0uBw+w65r5sB8i5coZJ3QoUN8QUZxfZqp0Dz k5RvvlfdiFNX0/Ma6aiav/qG4jjlqCw9HEdtwblh6ZrWBsky0nfbJCTTvglUJNvV7t9A /u5VorjNV6XPrAI32UW3CveNZrlFYB2OjKin2Hrr+i2ZC3txZg/5G8+h4WcfE685bPAv YCZA== X-Gm-Message-State: AOJu0Yw/M7efwk461Vy4IQEoQTYDJKXfBKFegcB095XS6TjKOqJpT5wO mu12knS/8hRSyX3Vb2SWZCMpk3+tLNBISERgSjojYyqsJUPwIFI7ZsUK1A== X-Google-Smtp-Source: AGHT+IF4wBMLWHi+D+z0QdmAv4XRXLi/EUG6aw+S1SkejrUeBXWz1KEtd1tORb3EwJgxcP+eRFmnaA== X-Received: by 2002:a19:640c:0:b0:51e:150e:2c45 with SMTP id y12-20020a19640c000000b0051e150e2c45mr1954225lfb.63.1714466324950; Tue, 30 Apr 2024 01:38:44 -0700 (PDT) Received: from smtpclient.apple ([2a0e:3700:200a:5c00:7d1e:fc37:c111:da11]) by smtp.gmail.com with ESMTPSA id bi24-20020a05600c3d9800b0041be4065adasm11496095wmb.22.2024.04.30.01.38.44 for <69596@debbugs.gnu.org> (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 30 Apr 2024 01:38:44 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable From: Jonathan Roberts Mime-Version: 1.0 (1.0) Date: Tue, 30 Apr 2024 09:38:33 +0100 Message-Id: <48BA9CD1-7ED9-4FB9-9BB6-3A4E91E0C5CA@googlemail.com> X-Mailer: iPhone Mail (20B82) X-Spam-Score: 0.0 (/) X-Mailman-Approved-At: Tue, 30 Apr 2024 06:40:24 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Ran in to the error that a lot of people are reporting - too many heap secti= ons on Guix pull.=20 Worked through commits to see where Guix pull stopped working for me, and th= e answer seems to be that it broke with 7cef6b7ba - gnu: guile-lzlib: Update= to 0.3.0. Could this be confirmed and reverted if it is breaking pull?=20 Sent from my iPhone=