From unknown Sat Jun 21 17:35:18 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#49439 <49439@debbugs.gnu.org> To: bug#49439 <49439@debbugs.gnu.org> Subject: Status: grafts cause =?UTF-8?Q?=E2=80=9Cguix_?= =?UTF-8?Q?environment=E2=80=9D?= to get killed with OOM Reply-To: bug#49439 <49439@debbugs.gnu.org> Date: Sun, 22 Jun 2025 00:35:18 +0000 retitle 49439 grafts cause =E2=80=9Cguix environment=E2=80=9D to get killed= with OOM reassign 49439 guix submitter 49439 Ricardo Wurmus severity 49439 important thanks From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 06 10:38:42 2021 Received: (at submit) by debbugs.gnu.org; 6 Jul 2021 14:38:42 +0000 Received: from localhost ([127.0.0.1]:49619 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m0mDi-0004Gy-Gz for submit@debbugs.gnu.org; Tue, 06 Jul 2021 10:38:42 -0400 Received: from lists.gnu.org ([209.51.188.17]:44896) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m0mDg-0004Gr-TT for submit@debbugs.gnu.org; Tue, 06 Jul 2021 10:38:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m0mDg-0007fq-By for bug-guix@gnu.org; Tue, 06 Jul 2021 10:38:40 -0400 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21136) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m0mDe-0006LY-3n for bug-guix@gnu.org; Tue, 06 Jul 2021 10:38:40 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1625582312; cv=none; d=zohomail.com; s=zohoarc; b=OeoBW/yCYuTXVauEeK5hmsnFBu/CDhKwTf9BlwTb0AZMDxEcz4z7Vuf9i5dV6IJ4K2bQ0RDkvW5A8pmUi2xfaXdPSkvPPpdnj4o/E0YBScveWxH/FY+BTL2iiuwPB+uN7z9ajjjlENpsc9bLWWov6syXJFDlio1CBNOVeClhC3E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1625582312; h=Content-Type:Content-Transfer-Encoding:Date:From:MIME-Version:Message-ID:Subject:To; bh=RWkcaAO2A/t3ODtW8jyTvEYx+mr7fT6VTDKsiptxKDM=; b=LC6SyQWv7+rCVcJ5A/JMoO3seTT7z5xBsqMe3qS5KxYqWJyr1tbrCZw7xTBQ1YzPN57Er+aXn9MKV8KsJCYwAjmP8UVUmhuQzqq5A7Ko7TVnqJPFk7F45HE6FRRMCnpozTUqlZ3PJhHI4VXjAN74HMvo3gImIAS80OIKPZY+gnQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1625582312; s=zoho; d=elephly.net; i=rekado@elephly.net; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=RWkcaAO2A/t3ODtW8jyTvEYx+mr7fT6VTDKsiptxKDM=; b=DNjLXRgRj/puvvIUs2/OpW2CFCuBdQyVEkY1uZmV0qkxqEekuAXpa4dsJ8s9bIMX CAaphRzaSbsbWtchKKMBrqxh0CsQopMtkrmYmzNXLc7JOmQ7E8pJWNPiBJ1K2EyyWqO h1Pc1kT7FBmlelsMzw+EbeG9emFmC7sdjryXLIs0= Received: from localhost (p54ad4c3a.dip0.t-ipconnect.de [84.173.76.58]) by mx.zohomail.com with SMTPS id 1625582311274952.9963472206382; Tue, 6 Jul 2021 07:38:31 -0700 (PDT) User-agent: mu4e 1.4.15; emacs 27.2 From: Ricardo Wurmus To: bug-guix@gnu.org Subject: grafts cause =?utf-8?Q?=E2=80=9Cguix_environment=E2=80=9D?= to get killed with OOM X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Tue, 06 Jul 2021 16:38:28 +0200 Message-ID: <87sg0rse1n.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) With a recent version of Guix, =E2=80=9Cguix environment=E2=80=9D will not= =20 terminate on its own, keeps the CPU busy, and gets killed when the=20 system eventually runs out of memory. $ guix describe -f channels --8<---------------cut here---------------start------------->8--- (list (channel (name 'guix) (url "/home/rekado/dev/gx/branches/master") (commit "685cfdec94e5e48c4ad28de53466a28dfc258edb"))) --8<---------------cut here---------------end--------------->8--- $ guix environment pigx-scrnaseq [wait until it gets killed] The problem disappears when grafts are disabled: $ guix environment --no-grafts pigx-scrnaseq $ [env] yay! --=20 Ricardo From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 09 05:31:32 2021 Received: (at control) by debbugs.gnu.org; 9 Jul 2021 09:31:32 +0000 Received: from localhost ([127.0.0.1]:58367 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1mr6-0004IM-1j for submit@debbugs.gnu.org; Fri, 09 Jul 2021 05:31:32 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51484) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m1mr3-0004I8-TY for control@debbugs.gnu.org; Fri, 09 Jul 2021 05:31:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:45196) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m1mqx-0006g3-6j for control@debbugs.gnu.org; Fri, 09 Jul 2021 05:31:24 -0400 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=36682 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m1mqw-0002G1-V5 for control@debbugs.gnu.org; Fri, 09 Jul 2021 05:31:23 -0400 Date: Fri, 09 Jul 2021 11:31:21 +0200 Message-Id: <87eec7j0k6.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #49439 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) severity 49439 important quit From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 23 00:59:32 2021 Received: (at 49439) by debbugs.gnu.org; 23 Jul 2021 04:59:32 +0000 Received: from localhost ([127.0.0.1]:42082 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m6nHY-0000N0-HW for submit@debbugs.gnu.org; Fri, 23 Jul 2021 00:59:32 -0400 Received: from out2.migadu.com ([188.165.223.204]:33794) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m6nHT-0000Mn-BO for 49439@debbugs.gnu.org; Fri, 23 Jul 2021 00:59:30 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1627016365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=24SdvVd059qkU2e4OdaujT405WmyvmPX/eQEIq1IJpQ=; b=Bt5Sh3DUnql78ZMy5VhIIdRHvQmfLJcBo5aGv1QJgt0g2vYOJ9uXGUhei5EMFeo5da84Um 56xCopehKnAyJZhGE1Jdn8FRZt1YDiDwUYWY7s2CX8MbNVpamvSBLoczFkutS72fCnxPhB hSDvhig3mAisSA4PmVuYYFihFznTWek= From: Sarah Morgensen To: Ricardo Wurmus Subject: Re: bug#49439: grafts cause =?utf-8?Q?=E2=80=9Cguix_environment?= =?utf-8?Q?=E2=80=9D?= to get killed with OOM References: <87sg0rse1n.fsf@elephly.net> Date: Thu, 22 Jul 2021 21:59:21 -0700 In-Reply-To: <87sg0rse1n.fsf@elephly.net> (Ricardo Wurmus's message of "Tue, 06 Jul 2021 16:38:28 +0200") Message-ID: <86tuklr5g6.fsf@mgsn.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: iskarian@mgsn.dev X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 49439 Cc: 49439@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Ricardo Wurmus writes: > With a recent version of Guix, =E2=80=9Cguix environment=E2=80=9D will no= t=20 > terminate on its own, keeps the CPU busy, and gets killed when the=20 > system eventually runs out of memory. > > $ guix describe -f channels > > (list (channel > (name 'guix) > (url "/home/rekado/dev/gx/branches/master") > (commit > "685cfdec94e5e48c4ad28de53466a28dfc258edb"))) > > > $ guix environment pigx-scrnaseq > [wait until it gets killed] I can reproduce this with pigx-scrnaseq as well a number of other packages (listed below). $ ./pre-inst-env guix describe -f channels (list (channel (name 'guix) (url "/home/sarah/guix") (commit "3217a04b0352c2dd13323257b369604eeabfccc3"))) Does not complete within 5 minutes: package # inputs # transitive inputs (from package-transitive-inputs) pigx-chipseq 48 338 pigx-scrnaseq 41 321 r-cellchat 34 110 pigx-rnaseq 34 343 pigx-bsseq 32 358 pigx-sars-cov2-ww 25 261 r-circus 16 134 Does complete: r-chipseq 6 37 completes in >2m r-shortread 17 36 completes in >1m python-scanpy 25 113 completes in <15s I suspect it has something to do with the number of transitive inputs, because it is so prevalent with these R packages, which all use propagated inputs. However... python-scanpy succeeds in under 15 seconds, and it has more transitive inputs than r-chipseq. Can we reproduce this with a large number of low-transitivity packages directly on the command line? > > The problem disappears when grafts are disabled: > > $ guix environment --no-grafts pigx-scrnaseq > $ [env] yay! -- Sarah From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 27 10:53:12 2021 Received: (at submit) by debbugs.gnu.org; 27 Jul 2021 14:53:12 +0000 Received: from localhost ([127.0.0.1]:53534 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m8OSF-0007TU-OQ for submit@debbugs.gnu.org; Tue, 27 Jul 2021 10:53:12 -0400 Received: from lists.gnu.org ([209.51.188.17]:35998) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m8OSE-0007TN-Bw for submit@debbugs.gnu.org; Tue, 27 Jul 2021 10:53:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8OSE-0000vA-3X for bug-guix@gnu.org; Tue, 27 Jul 2021 10:53:10 -0400 Received: from lepiller.eu ([2a00:5884:8208::1]:42562) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8OS9-00054B-KU for bug-guix@gnu.org; Tue, 27 Jul 2021 10:53:09 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id d20c2537; Tue, 27 Jul 2021 14:52:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date :in-reply-to:references:mime-version:content-type :content-transfer-encoding:subject:to:cc:from:message-id; s= dkim; bh=AqQtugjQphSi3WixQ/xCqEtGTXBRTyG1qSnUwcWps0o=; b=I6u3cvT 5Knx2v+0o84lDdNPmuku5/8QZVqtJ1pdTuO+ENBB1dMrWieiWxeo2apcbyulBM/R m0E/hVBwi4EN1Attjf+LJy9yln0vsB9WBSmDu1EHHMzMNZJ328ebH4Fn47/+2QDE 8fgGKOb5ixIHNy5tNnu7JX9ZjNIDhX4icZF+uyc0YoP1BMCk1YqIC3c2IKNLzhFD Vpnnk+Qdj35K+lrwzv6FYx7y4qgod2T0glPU3qBTpXOvk9gViIhZrBClyCIDTM3a aBWU2qL5IHghMoLV7p6qLVjVgaaciwQF7lFKeJGATmNks0svxcb4Gp3xpThpUwoN QHFgOxeQBJIN/Pw== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 0325c994 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Tue, 27 Jul 2021 14:52:56 +0000 (UTC) Date: Tue, 27 Jul 2021 10:52:44 -0400 User-Agent: K-9 Mail for Android In-Reply-To: <86tuklr5g6.fsf@mgsn.dev> References: <87sg0rse1n.fsf@elephly.net> <86tuklr5g6.fsf@mgsn.dev> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----WOJJJVY4L3END7ODIMFCSXM2YM5MTH" Content-Transfer-Encoding: 7bit Subject: =?UTF-8?Q?Re=3A_bug=2349439=3A_grafts_cause_=E2=80=9Cguix_?= =?UTF-8?Q?environment=E2=80=9D_to_get_killed_with_OOM?= To: bug-guix@gnu.org, Sarah Morgensen , Ricardo Wurmus From: Julien Lepiller Message-ID: <464151E3-FBF5-4B80-947E-0F1291FD879D@lepiller.eu> Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: 49439@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) ------WOJJJVY4L3END7ODIMFCSXM2YM5MTH Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I have a similar issue with an ocaml package I use at work=2E It's not free= software, but all its dependencies are=2E The dependencies are not all yee= t in guix, so to reproduce you might have to import them first with "guix i= mport opam -r foo" for ocaml-foo=2E The package depends on ocaml-ounit, ocaml-lp, ocaml-apronext, menhir, ocam= l-async, ocaml-core, ocaml-graph, ocaml-libsvm, ocaml-minisat, ocaml-ppx-de= riving-yojson, ocaml-yojson, ocaml-z3, ocaml-zarith and z3=2E In total, that's 118 transitive inputs=2E Building the profile takes 30 mi= nutes for me, on an SSD=2E The builder takes 1=2E5g resident=2E Other than that, I measured time and memory for creating the environment w= hen the profile was already built (no more derivation to build): `which time` -v ~/guix/pre-inst-env guix environment ocaml-dummy-package -= - echo done User time: 121=2E43s System time: 2=2E28s Maximum resident: 1803028kB (1=2E8 GB) With a warning from GC: Repeated allocation of very large block (approx=2E size 35606528) Note that I get the same numbers with --no-grafts, so it might be a differ= ent issue=2E "guix build" terminates quickly=2E Le 23 juillet 2021 00:59:21 GMT-04:00, Sarah Morgensen a =C3=A9crit : >Hello, > >Ricardo Wurmus writes: > >> With a recent version of Guix, =E2=80=9Cguix environment=E2=80=9D will = not=20 >> terminate on its own, keeps the CPU busy, and gets killed when the=20 >> system eventually runs out of memory=2E >> >> $ guix describe -f channels >> >> (list (channel >> (name 'guix) >> (url "/home/rekado/dev/gx/branches/master") >> (commit >> "685cfdec94e5e48c4ad28de53466a28dfc258edb"))) >> >> >> $ guix environment pigx-scrnaseq >> [wait until it gets killed] > >I can reproduce this with pigx-scrnaseq as well a number of other >packages (listed below)=2E > >$ =2E/pre-inst-env guix describe -f channels >(list (channel > (name 'guix) > (url "/home/sarah/guix") > (commit > "3217a04b0352c2dd13323257b369604eeabfccc3"))) > >Does not complete within 5 minutes: >package # inputs # transitive inputs > (from package-transitive-inputs) >pigx-chipseq 48 338 >pigx-scrnaseq 41 321 >r-cellchat 34 110 >pigx-rnaseq 34 343 >pigx-bsseq 32 358 >pigx-sars-cov2-ww 25 261 >r-circus 16 134 > >Does complete: >r-chipseq 6 37 completes in >2m >r-shortread 17 36 completes in >1m >python-scanpy 25 113 completes in <15s > >I suspect it has something to do with the number of transitive inputs, >because it is so prevalent with these R packages, which all use >propagated inputs=2E However=2E=2E=2E python-scanpy succeeds in under 15 >seconds, and it has more transitive inputs than r-chipseq=2E > >Can we reproduce this with a large number of low-transitivity packages >directly on the command line? > >> >> The problem disappears when grafts are disabled: >> >> $ guix environment --no-grafts pigx-scrnaseq >> $ [env] yay! > >-- >Sarah ------WOJJJVY4L3END7ODIMFCSXM2YM5MTH Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable I have a similar issue with an ocaml package I use= at work=2E It's not free software, but all its dependencies are=2E The dep= endencies are not all yeet in guix, so to reproduce you might have to impor= t them first with "guix import opam -r foo" for ocaml-foo=2E

The pac= kage depends on ocaml-ounit, ocaml-lp, ocaml-apronext, menhir, ocaml-async,= ocaml-core, ocaml-graph, ocaml-libsvm, ocaml-minisat, ocaml-ppx-deriving-y= ojson, ocaml-yojson, ocaml-z3, ocaml-zarith and z3=2E

In total, that= 's 118 transitive inputs=2E Building the profile takes 30 minutes for me, o= n an SSD=2E The builder takes 1=2E5g resident=2E

Other than that, I = measured time and memory for creating the environment when the profile was = already built (no more derivation to build):

`which time` -v ~/guix/= pre-inst-env guix environment ocaml-dummy-package -- echo done

User = time: 121=2E43s
System time: 2=2E28s
Maximum resident: 1803028kB (1= =2E8 GB)

With a warning from GC:

Repeated allocation of very = large block (approx=2E size 35606528)

Note that I get the same numbe= rs with --no-grafts, so it might be a different issue=2E

"guix build= " terminates quickly=2E

Le 23 juillet 202= 1 00:59:21 GMT-04:00, Sarah Morgensen <iskarian@mgsn=2Edev> a =C3=A9c= rit :
Hello,

Ricardo Wurmus <rekado@elephly=2Ene=
t> writes:

 With=
 a recent version of Guix, =E2=80=9Cguix environment=E2=80=9D will not 
=
 terminate on its own, keeps the CPU busy, and gets killed when the 
 sy=
stem eventually runs out of memory=2E

 $ guix describe -f channels
 (list (channel
         (name 'guix)
         (url "/home/reka=
do/dev/gx/branches/master")
         (commit
           "685cfdec94e5=
e48c4ad28de53466a28dfc258edb")))


 $ guix environment pigx-scrnas=
eq
 [wait until it gets killed]

I can reproduce this=
 with pigx-scrnaseq as well a number of other
packages (listed below)=2E=


$ =2E/pre-inst-env guix describe -f channels
(list (channel
 =
       (name 'guix)
        (url "/home/sarah/guix")
        (commit<=
br>          "3217a04b0352c2dd13323257b369604eeabfccc3")))

Does not =
complete within 5 minutes:
package            # inputs  # transitive inp=
uts
                                 (from package-transitive-inputs)pigx-chipseq       48        338
pigx-scrnaseq      41        321
r-=
cellchat         34        110
pigx-rnaseq        34        343
pigx-=
bsseq         32        358
pigx-sars-cov2-ww  25        261
r-circus=
           16        134

Does complete:
r-chipseq          6     =
    37     completes in >2m
r-shortread        17        36     compl=
etes in >1m
python-scanpy      25        113    completes in <15s<=
br>
I suspect it has something to do with the number of transitive input=
s,
because it is so prevalent with these R packages, which all use
pr=
opagated inputs=2E However=2E=2E=2E python-scanpy succeeds in under 15
s=
econds, and it has more transitive inputs than r-chipseq=2E

Can we r=
eproduce this with a large number of low-transitivity packages
directly =
on the command line?


User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -1.1 (-)
X-Debbugs-Envelope-To: 49439
Cc: Ricardo Wurmus , 49439@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: 
List-Unsubscribe: , 
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
 
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" 
X-Spam-Score: -2.1 (--)

Hi,

Sarah Morgensen  skribis:

> Ricardo Wurmus  writes:
>
>> With a recent version of Guix, =E2=80=9Cguix environment=E2=80=9D will n=
ot=20
>> terminate on its own, keeps the CPU busy, and gets killed when the=20
>> system eventually runs out of memory.
>>
>> $ guix describe -f channels
>>
>> (list (channel
>>         (name 'guix)
>>         (url "/home/rekado/dev/gx/branches/master")
>>         (commit
>>           "685cfdec94e5e48c4ad28de53466a28dfc258edb")))
>>
>>
>> $ guix environment pigx-scrnaseq
>> [wait until it gets killed]
>
> I can reproduce this with pigx-scrnaseq as well a number of other
> packages (listed below).
>
> $ ./pre-inst-env guix describe -f channels
> (list (channel
>         (name 'guix)
>         (url "/home/sarah/guix")
>         (commit
>           "3217a04b0352c2dd13323257b369604eeabfccc3")))
>
> Does not complete within 5 minutes:

What hardware are you using?

Here=E2=80=99s what I observe (with everything already in store and on a hot
cache, with an i7 CPU):

--8<---------------cut here---------------start------------->8---
$ guix describe
Generacio 188	Jul 25 2021 12:47:29	(nuna)
  guix a92dfbc
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: a92dfbce30777de6ca05031e275410cf9f56c84c
$ time GUIX_PROFILING=3Dgc guix environment pigx-scrnaseq --search-paths --=
no-grafts >/dev/null
Garbage collection statistics:
  heap size:        160.31 MiB
  allocated:        1440.70 MiB
  GC times:         39
  time spent in GC: 4.51 seconds (46% of user time)

real	0m7.534s
user	0m9.747s
sys	0m0.167s
$ time GUIX_PROFILING=3Dgc guix environment pigx-scrnaseq --search-paths  >=
/dev/null
Garbage collection statistics:
  heap size:        168.31 MiB
  allocated:        2111.71 MiB
  GC times:         53
  time spent in GC: 6.92 seconds (45% of user time)

real	0m12.100s
user	0m15.517s
sys	0m0.198s
--8<---------------cut here---------------end--------------->8---

Commit 78daf9e02e5bc51f91488d8237cab2050cc060cf optimizes
=E2=80=98coalesce-duplicate-inputs=E2=80=99, which was quadratic in the num=
ber of inputs (!).
After that change, I get:

--8<---------------cut here---------------start------------->8---
$ time GUIX_PROFILING=3Dgc ./pre-inst-env guix environment pigx-scrnaseq --=
search-paths --no-grafts  >/dev/null
Garbage collection statistics:
  heap size:        168.31 MiB
  allocated:        716.58 MiB
  GC times:         24
  time spent in GC: 2.65 seconds (40% of user time)

real	0m5.720s
user	0m6.708s
sys	0m0.149s
$ time GUIX_PROFILING=3Dgc ./pre-inst-env guix environment pigx-scrnaseq --=
search-paths  >/dev/null
Garbage collection statistics:
  heap size:        160.31 MiB
  allocated:        1387.96 MiB
  GC times:         42
  time spent in GC: 5.87 seconds (43% of user time)

real	0m10.821s
user	0m13.513s
sys	0m0.198s
--8<---------------cut here---------------end--------------->8---

Could you tell me if it improves the situation for you?

It=E2=80=99s not the end of the road, but it should be an improvement.

Thanks,
Ludo=E2=80=99.




From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 27 19:35:18 2021
Received: (at 49439) by debbugs.gnu.org; 27 Jul 2021 23:35:18 +0000
Received: from localhost ([127.0.0.1]:54252 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from )
	id 1m8WbV-0001hg-Gk
	for submit@debbugs.gnu.org; Tue, 27 Jul 2021 19:35:17 -0400
Received: from out1.migadu.com ([91.121.223.63]:10843)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from ) id 1m8WbQ-0001hF-83
 for 49439@debbugs.gnu.org; Tue, 27 Jul 2021 19:35:16 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1627428910;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=kGd7hX7wigiuvw94qOTEBN/yRYCSqxEBsgMYay/ebV8=;
 b=a8Ssmggr3M3qtB79hrXtVM8rBVvWo/qe5a1hrNPxnUDvR+JU4wavWiBkAoUSh38CjNwQll
 ht3iXZfsLHtZ6xxaqUIn0wnrEO+Qz3+30QBMQNnJQmaiHeXSxm9GdAYTrC9DKPb4GWJhfr
 I5wDCETERyNSLU2u4KvSlowJwlUkDck=
From: Sarah Morgensen 
To: Ludovic =?utf-8?Q?Court=C3=A8s?= 
Subject: Re: bug#49439: grafts cause =?utf-8?Q?=E2=80=9Cguix_environment?=
 =?utf-8?Q?=E2=80=9D?= to get killed with OOM
References: <87sg0rse1n.fsf@elephly.net> <86tuklr5g6.fsf@mgsn.dev>
 <87y29r3enb.fsf@gnu.org>
Date: Tue, 27 Jul 2021 16:35:06 -0700
In-Reply-To: <87y29r3enb.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
 =?utf-8?Q?s?= message of "Tue, 27 Jul 2021 18:28:08 +0200")
Message-ID: <867dhbpbyt.fsf@mgsn.dev>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@mgsn.dev
X-Spam-Score: 0.5 (/)
X-Debbugs-Envelope-To: 49439
Cc: Ricardo Wurmus , 49439@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: 
List-Unsubscribe: , 
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
 
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" 
X-Spam-Score: -0.5 (/)

Hi,

Ludovic Court=C3=A8s  writes:

[...]

>> I can reproduce this with pigx-scrnaseq as well a number of other
>> packages (listed below).
>>
>> $ ./pre-inst-env guix describe -f channels
>> (list (channel
>>         (name 'guix)
>>         (url "/home/sarah/guix")
>>         (commit
>>           "3217a04b0352c2dd13323257b369604eeabfccc3")))
>>
>> Does not complete within 5 minutes:

Okay, so all of a sudden I can't reproduce this; even with the same
commit as above, it completes in ~20s.

  guix time-machine --commit=3D3217a04 -- environment pigx-scrnaseq --searc=
h-paths >/dev/null

> What hardware are you using?

Virtualbox VM with VT-x etc. on a host i7-6700. The VM has 6GB of memory.

>
> Here=E2=80=99s what I observe (with everything already in store and on a =
hot
> cache, with an i7 CPU):
>
> $ guix describe
> Generacio 188	Jul 25 2021 12:47:29	(nuna)
>   guix a92dfbc
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: master
>     commit: a92dfbce30777de6ca05031e275410cf9f56c84c
> $ time GUIX_PROFILING=3Dgc guix environment pigx-scrnaseq --search-paths =
--no-grafts >/dev/null
> Garbage collection statistics:
>   heap size:        160.31 MiB
>   allocated:        1440.70 MiB
>   GC times:         39
>   time spent in GC: 4.51 seconds (46% of user time)
>
> real	0m7.534s
> user	0m9.747s
> sys	0m0.167s
> $ time GUIX_PROFILING=3Dgc guix environment pigx-scrnaseq --search-paths =
 >/dev/null
> Garbage collection statistics:
>   heap size:        168.31 MiB
>   allocated:        2111.71 MiB
>   GC times:         53
>   time spent in GC: 6.92 seconds (45% of user time)
>
> real	0m12.100s
> user	0m15.517s
> sys	0m0.198s
>
>
> Commit 78daf9e02e5bc51f91488d8237cab2050cc060cf optimizes
> =E2=80=98coalesce-duplicate-inputs=E2=80=99, which was quadratic in the n=
umber of inputs (!).
> After that change, I get:
>
> $ time GUIX_PROFILING=3Dgc ./pre-inst-env guix environment pigx-scrnaseq =
--search-paths --no-grafts  >/dev/null
> Garbage collection statistics:
>   heap size:        168.31 MiB
>   allocated:        716.58 MiB
>   GC times:         24
>   time spent in GC: 2.65 seconds (40% of user time)
>
> real	0m5.720s
> user	0m6.708s
> sys	0m0.149s
> $ time GUIX_PROFILING=3Dgc ./pre-inst-env guix environment pigx-scrnaseq =
--search-paths  >/dev/null
> Garbage collection statistics:
>   heap size:        160.31 MiB
>   allocated:        1387.96 MiB
>   GC times:         42
>   time spent in GC: 5.87 seconds (43% of user time)
>
> real	0m10.821s
> user	0m13.513s
> sys	0m0.198s
>
> Could you tell me if it improves the situation for you?

*Now* my experience is like yours:

--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 9	Jul 27 2021 12:35:05	(current)
  guix d0ec739
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: d0ec73907c2995034a34339f4a7c2c72c2e21fea

time GUIX_PROFILING=3Dgc guix time-machine --commit=3D3217a04 -- environmen=
t pigx-scrnaseq --search-paths >/dev/null
Garbage collection statistics:
  heap size:        176.31 MiB
  allocated:        2107.82 MiB
  GC times:         52
  time spent in GC: 5.26 seconds (23% of user time)

real	0m20.471s
user	0m22.605s
sys	0m0.372s

$ time GUIX_PROFILING=3Dgc guix environment pigx-scrnaseq --search-paths >/=
dev/null
Garbage collection statistics:
  heap size:        152.31 MiB
  allocated:        1367.16 MiB
  GC times:         40
  time spent in GC: 3.25 seconds (21% of user time)

real	0m14.701s
user	0m15.698s
sys	0m0.361s
--8<---------------cut here---------------end--------------->8---

But why was it occurring before? The only I thing I can think of is that
I didn't have everything in the store first. Is there a way I can prune
just the relevant items from the store to test this?

>
> It=E2=80=99s not the end of the road, but it should be an improvement.
>
> Thanks,
> Ludo=E2=80=99.

--
Sarah




From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 28 06:01:09 2021
Received: (at 49439) by debbugs.gnu.org; 28 Jul 2021 10:01:09 +0000
Received: from localhost ([127.0.0.1]:54703 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from )
	id 1m8gN8-0008TY-UM
	for submit@debbugs.gnu.org; Wed, 28 Jul 2021 06:01:09 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39844)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from ) id 1m8gMs-0008Si-GW
 for 49439@debbugs.gnu.org; Wed, 28 Jul 2021 06:01:05 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:49768)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from )
 id 1m8gMl-00015D-9Y; Wed, 28 Jul 2021 06:00:43 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45318 helo=ribbon)
 by fencepost.gnu.org with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1)
 (envelope-from )
 id 1m8gMl-0003Ey-1T; Wed, 28 Jul 2021 06:00:43 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= 
To: Sarah Morgensen 
Subject: Re: bug#49439: grafts cause =?utf-8?Q?=E2=80=9Cguix_environment?=
 =?utf-8?Q?=E2=80=9D?= to get killed with OOM
References: <87sg0rse1n.fsf@elephly.net> <86tuklr5g6.fsf@mgsn.dev>
 <87y29r3enb.fsf@gnu.org> <867dhbpbyt.fsf@mgsn.dev>
Date: Wed, 28 Jul 2021 12:00:39 +0200
In-Reply-To: <867dhbpbyt.fsf@mgsn.dev> (Sarah Morgensen's message of "Tue, 27
 Jul 2021 16:35:06 -0700")
Message-ID: <87k0la3ghk.fsf@gnu.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 49439
Cc: Ricardo Wurmus , 49439@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: 
List-Unsubscribe: , 
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
 
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" 
X-Spam-Score: -3.3 (---)

Hi,

Sarah Morgensen  skribis:

> But why was it occurring before? The only I thing I can think of is that
> I didn't have everything in the store first. Is there a way I can prune
> just the relevant items from the store to test this?

You could try something like:

  guix gc -D $(guix gc --references $(guix build pigx-scrnaseq)) \
    $(guix gc --references $(guix build pigx-scrnaseq --no-grafts))

Thinking about it, the grafts code depends on what=E2=80=99s in the store: =
when
nothing is in the store, it bounces to the =E2=80=9Cbuild handler=E2=80=9D,=
 which
accumulates the list of missing store items, until it starts building
them.

Let=E2=80=99s see if I can reproduce that behavior, too=E2=80=A6

Thanks,
Ludo=E2=80=99.




From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 28 18:03:57 2021
Received: (at 49439) by debbugs.gnu.org; 28 Jul 2021 22:03:57 +0000
Received: from localhost ([127.0.0.1]:55997 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from )
	id 1m8ref-00040L-24
	for submit@debbugs.gnu.org; Wed, 28 Jul 2021 18:03:57 -0400
Received: from eggs.gnu.org ([209.51.188.92]:37074)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from ) id 1m8red-000400-V2
 for 49439@debbugs.gnu.org; Wed, 28 Jul 2021 18:03:56 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:42200)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from )
 id 1m8reX-0007y4-B1; Wed, 28 Jul 2021 18:03:49 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45320 helo=ribbon)
 by fencepost.gnu.org with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1)
 (envelope-from )
 id 1m8reX-0001g7-3L; Wed, 28 Jul 2021 18:03:49 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= 
To: Sarah Morgensen 
Subject: Re: bug#49439: grafts cause =?utf-8?Q?=E2=80=9Cguix_environment?=
 =?utf-8?Q?=E2=80=9D?= to get killed with OOM
References: <87sg0rse1n.fsf@elephly.net> <86tuklr5g6.fsf@mgsn.dev>
 <87y29r3enb.fsf@gnu.org> <867dhbpbyt.fsf@mgsn.dev>
 <87k0la3ghk.fsf@gnu.org>
Date: Thu, 29 Jul 2021 00:03:46 +0200
In-Reply-To: <87k0la3ghk.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
 =?utf-8?Q?s?= message of "Wed, 28 Jul 2021 12:00:39 +0200")
Message-ID: <87wnpa14fx.fsf@gnu.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 49439
Cc: Ricardo Wurmus , 49439@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: 
List-Unsubscribe: , 
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
 
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" 
X-Spam-Score: -3.3 (---)

Hi,

Ludovic Court=C3=A8s  skribis:

> Thinking about it, the grafts code depends on what=E2=80=99s in the store=
: when
> nothing is in the store, it bounces to the =E2=80=9Cbuild handler=E2=80=
=9D, which
> accumulates the list of missing store items, until it starts building
> them.

So I can reproduce the problem Ricardo and you initially reported by
running:

  ./pre-inst-env guix environment pigx-scrnaseq --search-paths

after removing some of the ungrafted store items with:

  guix gc -D $(guix build r-rlang --no-grafts) \
    $(guix gc --references $(guix build pigx-scrnaseq --no-grafts))

The seemingly endless CPU usage and unbound memory use comes from the
=E2=80=98build-accumulator=E2=80=99 build handler.  Here, the graph of =E2=
=80=98pigx-scrnaseq=E2=80=99
has many nodes, and many of them depend on, say, =E2=80=98r-rlang=E2=80=99.=
  Thus, when
=E2=80=98r-rlang=E2=80=99 is not in the store, the grafting code keeps aski=
ng for it by
calling =E2=80=98build-derivations=E2=80=99, which aborts to the build hand=
ler; the
build handler saves the .drv file name and the continuation and keeps
going.  But since the next package also depends on =E2=80=98r-langr=E2=80=
=99, we abort
again to the build handler, and so on.

The end result is a very long list of  nodes, probably of
this order in this case:

  $ guix graph -t reverse-package r-rlang |grep 'label =3D "'|wc -l
  594

Presumably, the captured continuations occupy quite a bit of memory,
hence the quick growth.

I suppose one solution is to fire suspended builds when the build
handler sees a build request for a given derivation for the second time.
It needs more thought and testing=E2=80=A6

Ludo=E2=80=99.

PS: Did you know =E2=80=98pigx-scrnaseq=E2=80=99 has twice as many nodes as
    =E2=80=98libreoffice=E2=80=99?

      $ guix graph -t bag pigx-scrnaseq |grep 'label =3D "'|wc -l
      1359
      $ guix graph -t bag libreoffice |grep 'label =3D "'|wc -l
      699

    That makes it a great example to study and fix scalability issues!




From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 28 23:20:43 2021
Received: (at 49439) by debbugs.gnu.org; 29 Jul 2021 03:20:43 +0000
Received: from localhost ([127.0.0.1]:56166 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from )
	id 1m8wbD-0002qe-6r
	for submit@debbugs.gnu.org; Wed, 28 Jul 2021 23:20:43 -0400
Received: from out1.migadu.com ([91.121.223.63]:37277)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from ) id 1m8wbA-0002qS-Pq
 for 49439@debbugs.gnu.org; Wed, 28 Jul 2021 23:20:42 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1627528839;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=+qtAJKm0St2Sdm/YsDnnmt2nfD5A5LMO2wQLOM1JIxc=;
 b=YVoUgApOhuIV6CLXZvMUtp8NycZ96BC8cqttoWrs4h4dQiBztHiK1SGjJXntj+yt42KjaM
 5tj/sT+CNA66IBo2kxzpykZQlwoSkCKmNLLVUSUUh9QuNqrrSKv+Hgw+Geknc88fo8t2vI
 7uJ0tG0Rx2HB8cElt4XkflV2vuQ6vis=
From: Sarah Morgensen 
To: Ludovic =?utf-8?Q?Court=C3=A8s?= 
Subject: Re: bug#49439: grafts cause =?utf-8?Q?=E2=80=9Cguix_environment?=
 =?utf-8?Q?=E2=80=9D?= to get killed with OOM
References: <87sg0rse1n.fsf@elephly.net> <86tuklr5g6.fsf@mgsn.dev>
 <87y29r3enb.fsf@gnu.org> <867dhbpbyt.fsf@mgsn.dev>
 <87k0la3ghk.fsf@gnu.org> <87wnpa14fx.fsf@gnu.org>
Date: Wed, 28 Jul 2021 20:20:36 -0700
In-Reply-To: <87wnpa14fx.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
 =?utf-8?Q?s?= message of "Thu, 29 Jul 2021 00:03:46 +0200")
Message-ID: <861r7hpzzv.fsf@mgsn.dev>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@mgsn.dev
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 49439
Cc: Ricardo Wurmus , 49439@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: 
List-Unsubscribe: , 
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
 
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" 
X-Spam-Score: -1.7 (-)

Hi,

Ludovic Court=C3=A8s  writes:

> Hi,
>
> Ludovic Court=C3=A8s  skribis:
>
>> Thinking about it, the grafts code depends on what=E2=80=99s in the stor=
e: when
>> nothing is in the store, it bounces to the =E2=80=9Cbuild handler=E2=80=
=9D, which
>> accumulates the list of missing store items, until it starts building
>> them.
>
> So I can reproduce the problem Ricardo and you initially reported by
> running:
>
>   ./pre-inst-env guix environment pigx-scrnaseq --search-paths
>
> after removing some of the ungrafted store items with:
>
>   guix gc -D $(guix build r-rlang --no-grafts) \
>     $(guix gc --references $(guix build pigx-scrnaseq --no-grafts))

Same here. I'm glad we were able to pinpoint this!

>
> The seemingly endless CPU usage and unbound memory use comes from the
> =E2=80=98build-accumulator=E2=80=99 build handler.  Here, the graph of =
=E2=80=98pigx-scrnaseq=E2=80=99
> has many nodes, and many of them depend on, say, =E2=80=98r-rlang=E2=80=
=99.  Thus, when
> =E2=80=98r-rlang=E2=80=99 is not in the store, the grafting code keeps as=
king for it by
> calling =E2=80=98build-derivations=E2=80=99, which aborts to the build ha=
ndler; the
> build handler saves the .drv file name and the continuation and keeps
> going.  But since the next package also depends on =E2=80=98r-langr=E2=80=
=99, we abort
> again to the build handler, and so on.
>
> The end result is a very long list of  nodes, probably of
> this order in this case:
>
>   $ guix graph -t reverse-package r-rlang |grep 'label =3D "'|wc -l
>   594
>
> Presumably, the captured continuations occupy quite a bit of memory,
> hence the quick growth.
>
> I suppose one solution is to fire suspended builds when the build
> handler sees a build request for a given derivation for the second time.
> It needs more thought and testing=E2=80=A6

I wonder if instead it's possible to reduce the memory taken by the
continuations? As someone who has absolutely no experience with the
build/derivation system, it seems like all we *should* need to save is
the derivation file name.

>
> Ludo=E2=80=99.
>
> PS: Did you know =E2=80=98pigx-scrnaseq=E2=80=99 has twice as many nodes =
as
>     =E2=80=98libreoffice=E2=80=99?
>
>       $ guix graph -t bag pigx-scrnaseq |grep 'label =3D "'|wc -l
>       1359
>       $ guix graph -t bag libreoffice |grep 'label =3D "'|wc -l
>       699
>
>     That makes it a great example to study and fix scalability issues!

For those interested, I've compiled a list of the top 60
highest-dependency packages, as reported by package-closure:

pigx             	1630
nextcloud-client 	1539
akregator        	1486
kmail            	1484
korganizer       	1481
kdepim-runtime   	1480
kincidenceeditor 	1478
keventviews      	1477
kmailcommon      	1476
kcalendarsupport 	1475
kmessagelib      	1474
knotes           	1472
kaddressbook     	1469
libksieve        	1467
kdepim-apps-libs 	1465
libgravatar      	1463
kpimcommon       	1462
akonadi-calendar 	1453
pigx-bsseq       	1448
elisa            	1446
kaffeine         	1432
kdenlive         	1431
kmailtransport   	1431
dolphin-plugins  	1426
k3b              	1424
libkgapi         	1422
dolphin          	1421
kopete           	1403
pigx-sars-cov2-ww	1401
krdc             	1398
baloo-widgets    	1397
baloo            	1396
pigx-chipseq     	1396
krfb             	1389
ffmpegthumbs     	1388
kget             	1382
kmplayer         	1381
kdelibs4support  	1375
pigx-scrnaseq    	1342
kdevelop         	1340
kmailimporter    	1326
libkdepim        	1325
pigx-rnaseq      	1324
labplot          	1316
smb4k            	1315
kleopatra        	1311
kalarmcal        	1311
choqok           	1311
okular           	1310
ktnef            	1310
ktorrent         	1310
kate             	1308
akonadi-search   	1308
kcalutils        	1307
yakuake          	1306
khelpcenter      	1305
libksysguard     	1305
kdeconnect       	1304
konsole          	1304
libkleo          	1304

There seem to be a lot of kde packages in there, so perhaps this issue
isn't as rare as we might otherwise expect?

--
Sarah




From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 09 17:26:15 2021
Received: (at 49439) by debbugs.gnu.org; 9 Aug 2021 21:26:15 +0000
Received: from localhost ([127.0.0.1]:57941 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from )
	id 1mDCmk-0007sh-Ss
	for submit@debbugs.gnu.org; Mon, 09 Aug 2021 17:26:15 -0400
Received: from eggs.gnu.org ([209.51.188.92]:45206)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from ) id 1mDCmi-0007sU-Qh
 for 49439@debbugs.gnu.org; Mon, 09 Aug 2021 17:26:13 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:56448)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from )
 id 1mDCmc-0003hp-6K; Mon, 09 Aug 2021 17:26:06 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48708 helo=ribbon)
 by fencepost.gnu.org with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1)
 (envelope-from )
 id 1mDCmb-0003QM-U2; Mon, 09 Aug 2021 17:26:06 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= 
To: Sarah Morgensen 
Subject: Re: bug#49439: grafts cause =?utf-8?Q?=E2=80=9Cguix_environment?=
 =?utf-8?Q?=E2=80=9D?= to get killed with OOM
References: <87sg0rse1n.fsf@elephly.net> <86tuklr5g6.fsf@mgsn.dev>
 <87y29r3enb.fsf@gnu.org> <867dhbpbyt.fsf@mgsn.dev>
 <87k0la3ghk.fsf@gnu.org> <87wnpa14fx.fsf@gnu.org>
 <861r7hpzzv.fsf@mgsn.dev>
Date: Mon, 09 Aug 2021 23:26:03 +0200
Message-ID: <87czqmtims.fsf@inria.fr>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 49439
Cc: Ricardo Wurmus , 49439@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: 
List-Unsubscribe: , 
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
 
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" 
X-Spam-Score: -3.3 (---)

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi!

Sarah Morgensen  skribis:

> Ludovic Court=C3=A8s  writes:

[...]

>> The seemingly endless CPU usage and unbound memory use comes from the
>> =E2=80=98build-accumulator=E2=80=99 build handler.  Here, the graph of =
=E2=80=98pigx-scrnaseq=E2=80=99
>> has many nodes, and many of them depend on, say, =E2=80=98r-rlang=E2=80=
=99.  Thus, when
>> =E2=80=98r-rlang=E2=80=99 is not in the store, the grafting code keeps a=
sking for it by
>> calling =E2=80=98build-derivations=E2=80=99, which aborts to the build h=
andler; the
>> build handler saves the .drv file name and the continuation and keeps
>> going.  But since the next package also depends on =E2=80=98r-langr=E2=
=80=99, we abort
>> again to the build handler, and so on.
>>
>> The end result is a very long list of  nodes, probably of
>> this order in this case:
>>
>>   $ guix graph -t reverse-package r-rlang |grep 'label =3D "'|wc -l
>>   594
>>
>> Presumably, the captured continuations occupy quite a bit of memory,
>> hence the quick growth.
>>
>> I suppose one solution is to fire suspended builds when the build
>> handler sees a build request for a given derivation for the second time.
>> It needs more thought and testing=E2=80=A6
>
> I wonder if instead it's possible to reduce the memory taken by the
> continuations? As someone who has absolutely no experience with the
> build/derivation system, it seems like all we *should* need to save is
> the derivation file name.

Continuations may take a bit of room but the main problem is that we=E2=80=
=99re
storing too many of them.  :-)

Roughly, I think we keep one  node per incoming edge into a
package not in the store (=E2=80=98r-rlang=E2=80=99 in the example above). =
 In a case
like =E2=80=98pigx-scrnaseq=E2=80=99, that=E2=80=99s a lot of items.

I added counters and =E2=80=98pk=E2=80=99 calls in =E2=80=98map/accumulate-=
builds=E2=80=99 & co. to see
what happens.  AFAICS, a cutoff as in the attached patch does the job.
It=E2=80=99s a basic heuristic to avoid unbounded growth, at the expense of
possibly reduced accumulation.  For example, here=E2=80=99s the effect on my
user profile with 300+ packages:

--8<---------------cut here---------------start------------->8---
$ # with cutoff
$ time GUIX_PROFILING=3Dgc ./pre-inst-env guix upgrade -n

[...]


2,926.7 MB would be downloaded
Garbage collection statistics:
  heap size:        119.37 MiB
  allocated:        849.56 MiB
  GC times:         30
  time spent in GC: 2.88 seconds (30% of user time)

real	0m8.987s
user	0m9.482s
sys	0m0.186s
$=20
$ # without cutoff
$ time GUIX_PROFILING=3Dgc ./pre-inst-env guix upgrade -n

[...]

3,402.5 MB would be downloaded
Garbage collection statistics:
  heap size:        127.37 MiB
  allocated:        1504.59 MiB
  GC times:         46
  time spent in GC: 5.31 seconds (29% of user time)

real	0m21.616s
user	0m18.082s
sys	0m0.255s
--8<---------------cut here---------------end--------------->8---

You can tell that, without the cutoff (2nd run), the command more
accurately finds out what it=E2=80=99s going to have to download since the
=E2=80=9Cwould be downloaded=E2=80=9D figure is higher; with the cutoff (1s=
t run), it
=E2=80=9Csees less=E2=80=9D of what it=E2=80=99s going to end up downloadin=
g.  This could be
annoying from a UI viewpoint in =E2=80=9Cextreme=E2=80=9D cases like this o=
ne, but most
likely the difference would be invisible in many cases.

More importantly, having this cutoff halves the execution time, which is
nice.

The command initially given in this bug report now behaves like this:

--8<---------------cut here---------------start------------->8---
$ time GUIX_PROFILING=3Dgc  ./pre-inst-env guix environment pigx-scrnaseq -=
-search-paths -n -v2
41.8 MB would be downloaded:
   /gnu/store/difgj9gf8l7y5bsilcl42x2vzgh493c6-r-utf8-1.2.2
   /gnu/store/z4rpk1sn3jhy8brsr30zccln8mira3z5-r-purrr-0.3.4
   /gnu/store/nkiga9rfd8a9gdfsp2rjcqz39s8p2hg3-r-magrittr-2.0.1
   /gnu/store/c5y0xlw1fbcwa5p5qnk4xji286hd7cqk-r-vctrs-0.3.8
   /gnu/store/86vz257dqy4vm6s7axq7zl2jqxacgngf-r-ellipsis-0.3.2
   /gnu/store/m2m7h497g5n6vdrp5pxsnr2v8hsriq5f-r-glue-1.4.2
   /gnu/store/0zv1sl91fz3ddq8namdfvf6yr0j1p2vx-texlive-bin-20190410
   /gnu/store/7ks38sv5cpg7x76g2fdp9cra3x7dpbyq-texlive-union-51265
   /gnu/store/vkdjhkl5s025jsjy7jrr9q7kxlsi2sc4-r-minimal-4.1.0
   /gnu/store/pysc6ixb5fj1m2n50dac6aq5lgd5bnv4-r-rlang-0.4.11
Garbage collection statistics:
  heap size:        127.31 MiB
  allocated:        621.97 MiB
  GC times:         24
  time spent in GC: 2.82 seconds (37% of user time)

real	0m6.493s
user	0m7.584s
sys	0m0.117s
--8<---------------cut here---------------end--------------->8---

This time it completes, which is already an improvement ;-).  The
41.8=C2=A0MB download reported is underestimated, but again that=E2=80=99s =
the
tradeoff we=E2=80=99re making.

Attached is the patch.  I=E2=80=99ll go ahead with that if there are no
objections.

> For those interested, I've compiled a list of the top 60
> highest-dependency packages, as reported by package-closure:
>
> pigx             	1630
> nextcloud-client 	1539
> akregator        	1486
> kmail            	1484
> korganizer       	1481
> kdepim-runtime   	1480
> kincidenceeditor 	1478
> keventviews      	1477
> kmailcommon      	1476
> kcalendarsupport 	1475

Nice!  I really need to stop taking =E2=80=98libreoffice=E2=80=99 as an exa=
mple.

Thanks,
Ludo=E2=80=99.


--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline

diff --git a/guix/store.scm b/guix/store.scm
index 1ab2b08b47..340ad8bc10 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -1358,11 +1358,27 @@ on the build output of a previous derivation."
 (define (map/accumulate-builds store proc lst)
   "Apply PROC over each element of LST, accumulating 'build-things' calls and
 coalescing them into a single call."
-  (define result
-    (map (lambda (obj)
-           (with-build-handler build-accumulator
-             (proc obj)))
-         lst))
+  (define accumulation-cutoff
+    ;; Threshold above which we stop accumulating unresolved nodes to avoid
+    ;; pessimal behavior.  See .
+    30)
+
+  (define-values (result rest)
+    (let loop ((lst lst)
+               (result '())
+               (unresolved 0))
+      (match lst
+        ((head . tail)
+         (match (with-build-handler build-accumulator
+                  (proc head))
+           ((? unresolved? obj)
+            (if (> unresolved accumulation-cutoff)
+                (values (reverse (cons obj result)) tail)
+                (loop tail (cons obj result) (+ 1 unresolved))))
+           (obj
+            (loop tail (cons obj result) unresolved))))
+        (()
+         (values (reverse result) lst)))))
 
   (match (append-map (lambda (obj)
                        (if (unresolved? obj)
@@ -1370,6 +1386,7 @@ coalescing them into a single call."
                            '()))
                      result)
     (()
+     ;; REST is necessarily empty.
      result)
     (to-build
      ;; We've accumulated things TO-BUILD.  Actually build them and resume the
@@ -1382,7 +1399,7 @@ coalescing them into a single call."
                                   ;; unnecessary.
                                   ((unresolved-continuation obj) #f)
                                   obj))
-                            result))))
+                            (append result rest)))))
 
 (define build-things
   (let ((build (operation (build-things (string-list things)

--=-=-=--




From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 10 11:36:36 2021
Received: (at 49439-done) by debbugs.gnu.org; 10 Aug 2021 15:36:36 +0000
Received: from localhost ([127.0.0.1]:60486 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from )
	id 1mDTnw-0008Pn-2E
	for submit@debbugs.gnu.org; Tue, 10 Aug 2021 11:36:36 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35686)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from ) id 1mDTnu-0008PK-HW
 for 49439-done@debbugs.gnu.org; Tue, 10 Aug 2021 11:36:34 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:54100)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from )
 id 1mDTnp-0005zk-2P; Tue, 10 Aug 2021 11:36:29 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=45072 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from )
 id 1mDTno-0004wq-Q6; Tue, 10 Aug 2021 11:36:28 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= 
To: Sarah Morgensen 
Subject: Re: bug#49439: grafts cause =?utf-8?Q?=E2=80=9Cguix_environment?=
 =?utf-8?Q?=E2=80=9D?= to get killed with OOM
References: <87sg0rse1n.fsf@elephly.net> <86tuklr5g6.fsf@mgsn.dev>
 <87y29r3enb.fsf@gnu.org> <867dhbpbyt.fsf@mgsn.dev>
 <87k0la3ghk.fsf@gnu.org> <87wnpa14fx.fsf@gnu.org>
 <861r7hpzzv.fsf@mgsn.dev> <87czqmtims.fsf@inria.fr>
Date: Tue, 10 Aug 2021 17:36:27 +0200
In-Reply-To: <87czqmtims.fsf@inria.fr> ("Ludovic =?utf-8?Q?Court=C3=A8s=22?=
 =?utf-8?Q?'s?= message of "Mon, 09 Aug 2021 23:26:03 +0200")
Message-ID: <87v94dpb0k.fsf@gnu.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 49439-done
Cc: Ricardo Wurmus , 49439-done@debbugs.gnu.org
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: 
List-Unsubscribe: , 
 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
 
Errors-To: debbugs-submit-bounces@debbugs.gnu.org
Sender: "Debbugs-submit" 
X-Spam-Score: -3.3 (---)

Hello!

Ludovic Court=C3=A8s  skribis:

> I added counters and =E2=80=98pk=E2=80=99 calls in =E2=80=98map/accumulat=
e-builds=E2=80=99 & co. to see
> what happens.  AFAICS, a cutoff as in the attached patch does the job.
> It=E2=80=99s a basic heuristic to avoid unbounded growth, at the expense =
of
> possibly reduced accumulation.  For example, here=E2=80=99s the effect on=
 my
> user profile with 300+ packages:

Pushed as fa81971cbae85b39183ccf8f51e8d96ac88fb4ac.

I saw your message on IRC, Sarah; note that because grafts are =E2=80=9Cdyn=
amic
dependencies=E2=80=9D (they depend on previous build results), we cannot kn=
ow in
advance which grafts we=E2=80=99re going to build, so there=E2=80=99s neces=
sarily at
least a second phase during which grafts get built.  See
 for background.

Thanks!

Ludo=E2=80=99.




From unknown Sat Jun 21 17:35:18 2025
Received: (at fakecontrol) by fakecontrolmessage;
To: internal_control@debbugs.gnu.org
From: Debbugs Internal Request 
Subject: Internal Control
Message-Id: bug archived.
Date: Wed, 08 Sep 2021 11:24:04 +0000
User-Agent: Fakemail v42.6.9

# This is a fake control message.
#
# The action:
# bug archived.
thanks
# This fakemail brought to you by your local debbugs
# administrator