From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 11 18:23:58 2021 Received: (at submit) by debbugs.gnu.org; 11 Jun 2021 22:23:58 +0000 Received: from localhost ([127.0.0.1]:40036 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lrpZF-00036E-QA for submit@debbugs.gnu.org; Fri, 11 Jun 2021 18:23:57 -0400 Received: from lists.gnu.org ([209.51.188.17]:59224) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lrpZD-000365-L4 for submit@debbugs.gnu.org; Fri, 11 Jun 2021 18:23:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36744) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lrpZD-0008Rz-2O for bug-guix@gnu.org; Fri, 11 Jun 2021 18:23:55 -0400 Received: from lepiller.eu ([2a00:5884:8208::1]:33574) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lrpZ4-0003oi-P1 for bug-guix@gnu.org; Fri, 11 Jun 2021 18:23:54 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id e1d1ae67 for ; Fri, 11 Jun 2021 22:23:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:message-id:mime-version:content-type :content-transfer-encoding; s=dkim; bh=xELXAE+25RC0RLz5f2tatODDa IGFsvBz0mXKD7hPylY=; b=nP3RY/eY+Qa9mbRAwtKLMZ7eHlMm72DsclMJEXiXK og/l/jckqWlDF67mCRYagbOv4UaMkeVsV8pixJzFlCQxluIU5UrEaJKCD84rEosf X9l11vvvKdmbjAXWNe114rRfX9EFp1MmgwnroqQutqNEZx097dnhFsxIkOPVc2Z8 ka0qHcpIEp6DfHkeBhmJYP3M+LbAEcJbCG9eHJN/WfFnj5t33DfMx0pXbsOvzyjW 25VSp3ZjKcmbbEIXk7VWvCEiKWTKoQ/o1d1ZiB3RuRT78r0rle6JnronU+R7SDjs c9zUqOtJfy9Scklm/+fZB59toFPPW6rqVR2i2/PV5Fs4g== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 870dd4da (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO) for ; Fri, 11 Jun 2021 22:23:40 +0000 (UTC) Date: Sat, 12 Jun 2021 00:23:30 +0200 From: Julien Lepiller To: bug-guix@gnu.org Subject: "guix build guix" halts after exhausting memory Message-ID: <20210612002330.64cd06c1@tachikoma.lepiller.eu> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Hi Guix! I tried updating my system on my armhf board (2GB of RAM), but during "guix system reconfigure", guix tries to build itself (the guix package from (gnu packages package-management)). This package uses too much memory to build, and I start getting GC warnings like so: GC Warning: Out of memory - trying to allocate requested amount (552 bytes)... GC Warning: Header allocation failed: dropping block GC Warning: Out of Memory! Heap size: 2571 MiB. Returning NULL! Warning: Unwind-only out of memory exception; skipping pre-unwind handler. after some more progress and a lot more warnings, the build stops (CPU is at 2%), but the memory is not freed. The build stayed stuck for a long time until I decided to cancel it. As a work-around, I tried using the guix-daemon package instead (by changing the guix field in guix-configuration), but "guix system" still wants to build the guix package anyway. I tried finding usages of the guix package (grepping for "package-management" yields a relatively short list of files, and none of them seems to be used by my config, except (gnu services base) which always uses the guix from the configuration). Why? Can we instead break the build (at the Makefile level) into multiple smaller chunks, that require less memory, in the same way (guix self) works? From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 11 18:30:27 2021 Received: (at 48963) by debbugs.gnu.org; 11 Jun 2021 22:30:27 +0000 Received: from localhost ([127.0.0.1]:40046 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lrpfX-0003If-3w for submit@debbugs.gnu.org; Fri, 11 Jun 2021 18:30:27 -0400 Received: from lepiller.eu ([89.234.186.109]:44388) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lrpfR-0003IR-In for 48963@debbugs.gnu.org; Fri, 11 Jun 2021 18:30:26 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 27271df4 for <48963@debbugs.gnu.org>; Fri, 11 Jun 2021 22:30:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=dkim; bh=EbcBDlaU+/pX rhvWWaaF9MJhxif+Rc97IUm4SJxEx1I=; b=EgP+MhWpSEvqr8kPuG+m+kXP4b9x 7fAHbBOxoaNocSClP6fha8uuIJUiuPFfJyoNHWULv5VBCE5U6eMNCTROG7eP6sjI BH8adiVkse9zDWzYfNEt8A/XSAmTKH+E7ygKIUC5y6lyLcUAnwjVggt+HThRuNi7 6/KFp3xDneleMeKr3RYL/kHogrUirpfOUuVJzqlun1NyVtou3Wf0t3XurUDi/gAV 9FdUKLfwMVTNTZULHVk9znsfXJvU107qjXWeT/wS69KWNRKWhiLeml6ZA48AmpZm hlygnCsGx9Dm7W1SkAzdKTdcgZMok+qJs6H7P5B9iP3VW9ubL97TvLqXRQ== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 59c29b4f (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO) for <48963@debbugs.gnu.org>; Fri, 11 Jun 2021 22:30:19 +0000 (UTC) Date: Sat, 12 Jun 2021 00:30:14 +0200 From: Julien Lepiller To: 48963@debbugs.gnu.org Subject: Re: bug#48963: "guix build guix" halts after exhausting memory Message-ID: <20210612003014.6f9481de@tachikoma.lepiller.eu> In-Reply-To: <20210612002330.64cd06c1@tachikoma.lepiller.eu> References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 48963 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Le Sat, 12 Jun 2021 00:23:30 +0200, Julien Lepiller a =C3=A9crit : > Hi Guix! >=20 > I tried updating my system on my armhf board (2GB of RAM), but during > "guix system reconfigure", guix tries to build itself (the guix > package from (gnu packages package-management)). This package uses > too much memory to build, and I start getting GC warnings like so: >=20 > GC Warning: Out of memory - trying to allocate requested amount (552 > bytes)... > GC Warning: Header allocation failed: dropping block > GC Warning: Out of Memory! Heap size: 2571 MiB. Returning NULL! > Warning: Unwind-only out of memory exception; skipping pre-unwind > handler. >=20 > after some more progress and a lot more warnings, the build stops (CPU > is at 2%), but the memory is not freed. The build stayed stuck for a > long time until I decided to cancel it. >=20 > As a work-around, I tried using the guix-daemon package instead (by > changing the guix field in guix-configuration), but "guix system" > still wants to build the guix package anyway. I tried finding usages > of the guix package (grepping for "package-management" yields a > relatively short list of files, and none of them seems to be used by > my config, except (gnu services base) which always uses the guix from > the configuration). Why? >=20 > Can we instead break the build (at the Makefile level) into multiple > smaller chunks, that require less memory, in the same way (guix self) > works? >=20 >=20 >=20 also note it's not an OOM issue at the system level, because guile doesn't seem to care I have 8GB of free swap it could use. Top reports 1.8GB of resident memory being used by the process. I tried to stop all services at the time in the hope it would save some memory and help, but guile just kept swallowing all that free memory I gave it. From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 15 08:49:28 2021 Received: (at control) by debbugs.gnu.org; 15 Jun 2021 12:49:28 +0000 Received: from localhost ([127.0.0.1]:48666 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt8VU-0004WE-Jq for submit@debbugs.gnu.org; Tue, 15 Jun 2021 08:49:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50682) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lt8VS-0004W1-IL for control@debbugs.gnu.org; Tue, 15 Jun 2021 08:49:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:52272) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lt8VN-0005fT-9r for control@debbugs.gnu.org; Tue, 15 Jun 2021 08:49:21 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=60092 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lt8VN-0001Sc-0r for control@debbugs.gnu.org; Tue, 15 Jun 2021 08:49:21 -0400 Date: Tue, 15 Jun 2021 14:49:19 +0200 Message-Id: <87mtrrs374.fsf@gnu.org> To: control@debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: control message for bug #48963 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) severity 48963 important quit From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 16 16:01:31 2021 Received: (at 48963) by debbugs.gnu.org; 16 Jun 2021 20:01:31 +0000 Received: from localhost ([127.0.0.1]:53391 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltbj9-00033s-7h for submit@debbugs.gnu.org; Wed, 16 Jun 2021 16:01:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:41074) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltbj8-00033c-2J for 48963@debbugs.gnu.org; Wed, 16 Jun 2021 16:01:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60172) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ltbj1-0006Bd-Gk; Wed, 16 Jun 2021 16:01:23 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=35826 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ltbj0-0004Lw-5v; Wed, 16 Jun 2021 16:01:23 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Julien Lepiller Subject: Re: bug#48963: "guix build guix" halts after exhausting memory References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> Date: Wed, 16 Jun 2021 22:01:20 +0200 In-Reply-To: <20210612002330.64cd06c1@tachikoma.lepiller.eu> (Julien Lepiller's message of "Sat, 12 Jun 2021 00:23:30 +0200") Message-ID: <87im2dpoj3.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 48963 Cc: 48963@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Julien Lepiller skribis: > I tried updating my system on my armhf board (2GB of RAM), but during > "guix system reconfigure", guix tries to build itself (the guix package > from (gnu packages package-management)). This package uses too much > memory to build, and I start getting GC warnings like so: > > GC Warning: Out of memory - trying to allocate requested amount (552 > bytes)... > GC Warning: Header allocation failed: dropping block > GC Warning: Out of Memory! Heap size: 2571 MiB. Returning NULL! > Warning: Unwind-only out of memory exception; skipping pre-unwind > handler. > > after some more progress and a lot more warnings, the build stops (CPU > is at 2%), but the memory is not freed. The build stayed stuck for a > long time until I decided to cancel it. This is ridiculous. :-/ > Can we instead break the build (at the Makefile level) into multiple > smaller chunks, that require less memory, in the same way (guix self) > works? Yes, that=E2=80=99s a good idea. Could you check the extent to which the attached patch helps on this machine? It doesn=E2=80=99t split as much as (guix self) does, only in three pieces,= but hopefully that helps a bit. A side effect is that progress report is now off, but we can fix it later. Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/Makefile.am b/Makefile.am index aa21b5383b..758d8b9b8a 100644 --- a/Makefile.am +++ b/Makefile.am @@ -663,7 +663,11 @@ CLEANFILES = \ # the whole thing. Likewise, set 'XDG_CACHE_HOME' to avoid loading possibly # stale files from ~/.cache/guile/ccache. %.go: make-go ; @: -make-go: $(MODULES) guix/config.scm $(dist_noinst_DATA) +make-go: make-core-go make-packages-go make-system-go + +define guile-compilation-rule + +$(1): $(2) $(AM_V_at)echo "Compiling Scheme modules..." ; \ unset GUILE_LOAD_COMPILED_PATH ; \ XDG_CACHE_HOME=/nowhere \ @@ -671,7 +675,19 @@ make-go: $(MODULES) guix/config.scm $(dist_noinst_DATA) $(top_builddir)/pre-inst-env \ $(GUILE) -L "$(top_builddir)" -L "$(top_srcdir)" \ --no-auto-compile \ - -s "$(top_srcdir)"/build-aux/compile-all.scm $^ + -s "$(top_srcdir)"/build-aux/compile-all.scm $$(filter %.scm,$$^) + +.PHONY: $(1) + +endef + +MODULES_CORE = $(filter guix/%,$(MODULES)) +MODULES_PACKAGES = $(filter gnu/packages/%,$(MODULES)) +MODULES_SYSTEM = $(filter-out gnu/packages/%,$(filter gnu/%,$(MODULES))) + +$(eval $(call guile-compilation-rule,make-core-go,$(MODULES_CORE) guix/config.scm $(dist_noinst_DATA))) +$(eval $(call guile-compilation-rule,make-packages-go,$(MODULES_PACKAGES) make-core-go)) +$(eval $(call guile-compilation-rule,make-system-go,$(MODULES_SYSTEM) make-packages-go make-core-go)) SUFFIXES = .go --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 17 09:58:25 2021 Received: (at 48963) by debbugs.gnu.org; 17 Jun 2021 13:58:25 +0000 Received: from localhost ([127.0.0.1]:54823 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltsXJ-0001dW-5F for submit@debbugs.gnu.org; Thu, 17 Jun 2021 09:58:25 -0400 Received: from lepiller.eu ([89.234.186.109]:44964) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ltsXF-0001dM-SZ for 48963@debbugs.gnu.org; Thu, 17 Jun 2021 09:58:23 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id c3552866; Thu, 17 Jun 2021 13:58:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date :in-reply-to:references:mime-version:content-type :content-transfer-encoding:subject:to:cc:from:message-id; s= dkim; bh=RmrCZOHUeQ7lA/N7v5wuXp2gfQo3JuR3vrsiPl6s6j8=; b=a2+YtqM jCeNz/D2pNva64bu5PUtetMmc7mO+ElHrhWt+Pt33bdj6LTWUXDzfEF5J4RgB1cN KYAc1skohASGzCThZ3haI/7KOucAbhVy30KKhlMMQZKvljpY2tX2R5J15RGbiXgN 58pHSyxj0g6t9s5PKLgNhVc/yjdH1fVsSBmE0MhgR2AcpLLzr1/qp5I6PmJurd9+ uHXaYXMXyK96Xljw2BwcrWlAHdlye0xalFYvVgqNZfxMzzj4EAjYx4CfZp/fZ3uI WVfikv0P+tBolSFk4vkLSbOfIJ6Obcc6kSj4Q7sAPwIOq0zNW3mLYWbjed3h7e9W gqp4vrcHPvwvcdQ== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 32610678 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Thu, 17 Jun 2021 13:58:18 +0000 (UTC) Date: Thu, 17 Jun 2021 09:58:09 -0400 User-Agent: K-9 Mail for Android In-Reply-To: <87im2dpoj3.fsf@gnu.org> References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> <87im2dpoj3.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----4E586CT4QFKF8A3P5UW6OUFENW2YG3" Content-Transfer-Encoding: 7bit Subject: Re: bug#48963: "guix build guix" halts after exhausting memory To: =?ISO-8859-1?Q?Ludovic_Court=E8s?= From: Julien Lepiller Message-ID: <78EA8A84-D7DC-4506-91EB-1C304E5DB363@lepiller.eu> X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 48963 Cc: 48963@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) ------4E586CT4QFKF8A3P5UW6OUFENW2YG3 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, I tried your patch by creating a variant of the guix package=2E My first a= ttempt was a failure because it's still using almost all my memory and I fo= rgot to enable my swap=2E In the second attempt, the build phase succeeded,= but the build failed during the test phase (test/inferior)=2E I've started= a new build hoping it's a non-deterministic failure=2E Le 16 juin 2021 16:01:20 GMT-04:00, "Ludovic Court=C3=A8s" a =C3=A9crit : >Hi, > >Julien Lepiller skribis: > >> I tried updating my system on my armhf board (2GB of RAM), but during >> "guix system reconfigure", guix tries to build itself (the guix >package >> from (gnu packages package-management))=2E This package uses too much >> memory to build, and I start getting GC warnings like so: >> >> GC Warning: Out of memory - trying to allocate requested amount (552 >> bytes)=2E=2E=2E >> GC Warning: Header allocation failed: dropping block >> GC Warning: Out of Memory! Heap size: 2571 MiB=2E Returning NULL! >> Warning: Unwind-only out of memory exception; skipping pre-unwind >> handler=2E >> >> after some more progress and a lot more warnings, the build stops >(CPU >> is at 2%), but the memory is not freed=2E The build stayed stuck for a >> long time until I decided to cancel it=2E > >This is ridiculous=2E :-/ > >> Can we instead break the build (at the Makefile level) into multiple >> smaller chunks, that require less memory, in the same way (guix self) >> works? > >Yes, that=E2=80=99s a good idea=2E > >Could you check the extent to which the attached patch helps on this >machine? > >It doesn=E2=80=99t split as much as (guix self) does, only in three piece= s, but >hopefully that helps a bit=2E A side effect is that progress report is >now off, but we can fix it later=2E > >Thanks, >Ludo=E2=80=99=2E ------4E586CT4QFKF8A3P5UW6OUFENW2YG3 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo,

I tried your patch by creating a v= ariant of the guix package=2E My first attempt was a failure because it's s= till using almost all my memory and I forgot to enable my swap=2E In the se= cond attempt, the build phase succeeded, but the build failed during the te= st phase (test/inferior)=2E I've started a new build hoping it's a non-dete= rministic failure=2E

Le 16 juin 2021 16:0= 1:20 GMT-04:00, "Ludovic Court=C3=A8s" <ludo@gnu=2Eorg> a =C3=A9crit = :
Hi,

Julien Lepiller <julien@lepiller=2Eeu&=
gt; skribis:

 I tri=
ed updating my system on my armhf board (2GB of RAM), but during
 "guix =
system reconfigure", guix tries to build itself (the guix package
 from =
(gnu packages package-management))=2E This package uses too much
 memory=
 to build, and I start getting GC warnings like so:

 GC Warning: Out=
 of memory - trying to allocate requested amount (552
 bytes)=2E=2E=2E GC Warning: Header allocation failed: dropping block
 GC Warning: Out=
 of Memory! Heap size: 2571 MiB=2E Returning NULL!
 Warning: Unwind-only=
 out of memory exception; skipping pre-unwind
 handler=2E

 after =
some more progress and a lot more warnings, the build stops (CPU
 is at =
2%), but the memory is not freed=2E The build stayed stuck for a
 long t=
ime until I decided to cancel it=2E

This is ridiculous=
=2E  :-/

Can we ins=
tead break the build (at the Makefile level) into multiple
smaller chunk=
s, that require less memory, in the same way (guix self)
works?

Yes, that=E2=80=99s a good idea=2E

Could you check the e=
xtent to which the attached patch helps on this
machine?

It doesn=
=E2=80=99t split as much as (guix self) does, only in three pieces, but
=
hopefully that helps a bit=2E  A side effect is that progress report is
=
now off, but we can fix it later=2E

Thanks,
Ludo=E2=80=99=2E
<=
br>
------4E586CT4QFKF8A3P5UW6OUFENW2YG3-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jun 17 18:17:51 2021 Received: (at submit) by debbugs.gnu.org; 17 Jun 2021 22:17:52 +0000 Received: from localhost ([127.0.0.1]:55327 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lu0Kd-0001pd-Ng for submit@debbugs.gnu.org; Thu, 17 Jun 2021 18:17:51 -0400 Received: from lists.gnu.org ([209.51.188.17]:40402) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lu0Kb-0001pW-Sz for submit@debbugs.gnu.org; Thu, 17 Jun 2021 18:17:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lu0Kb-0006e3-BM for bug-guix@gnu.org; Thu, 17 Jun 2021 18:17:49 -0400 Received: from lepiller.eu ([2a00:5884:8208::1]:34196) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lu0KZ-0005Lf-AL for bug-guix@gnu.org; Thu, 17 Jun 2021 18:17:49 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 0d418abf for ; Thu, 17 Jun 2021 22:17:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=dkim; bh=tidsHGjffNkf sv7SGJ6vX2sOIS+q/jZmGDnvzZgOPtQ=; b=kVeCSInepT8oixKDqhTXFGEjNlqo DysTRMXuAdTHEua+PoaoGBjwOxRv9ZLVBEaY/v0iqgYV5qdFxgqZ3m/u5JK4P4Mk e74id9xM5SbDg9fw9rM6J9vZVsMoobRKT8okwpGG4PxDSJlTggdE3LOEaJ2PY0G0 wNa14LGJ+mpyRQ/Ca6ewcA2v6Ip0+p1OC+uhnzhD3bc+fQ/AMFTIZNTWC1nl4gk9 zOZKxQZPfkb5s+BYRyhjdeXqYsUpmBXjwEwjWQkD8Ussi+Wq/9Re5YxhQ3+Wotru SYqQf95S8urupYF8kNU8Um2vinqDJirqVHIWyJTTanN0/byscoslTi5hHA== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 786ab63e (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO) for ; Thu, 17 Jun 2021 22:17:40 +0000 (UTC) Date: Fri, 18 Jun 2021 00:17:37 +0200 From: Julien Lepiller To: bug-guix@gnu.org Subject: Re: bug#48963: "guix build guix" halts after exhausting memory Message-ID: <20210618001737.105e2e5a@tachikoma.lepiller.eu> In-Reply-To: <78EA8A84-D7DC-4506-91EB-1C304E5DB363@lepiller.eu> References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> <87im2dpoj3.fsf@gnu.org> <78EA8A84-D7DC-4506-91EB-1C304E5DB363@lepiller.eu> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Le Thu, 17 Jun 2021 09:58:09 -0400, Julien Lepiller a =C3=A9crit : > Hi Ludo, >=20 > I tried your patch by creating a variant of the guix package. My > first attempt was a failure because it's still using almost all my > memory and I forgot to enable my swap. In the second attempt, the > build phase succeeded, but the build failed during the test phase > (test/inferior). I've started a new build hoping it's a > non-deterministic failure. I'm afraid after testing 2 more times that the test failure is happening consistently, at least of armhf. From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 18 05:51:23 2021 Received: (at 48963) by debbugs.gnu.org; 18 Jun 2021 09:51:23 +0000 Received: from localhost ([127.0.0.1]:55811 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1luB9n-0004np-3v for submit@debbugs.gnu.org; Fri, 18 Jun 2021 05:51:23 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56236) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1luB9m-0004nd-At for 48963@debbugs.gnu.org; Fri, 18 Jun 2021 05:51:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51542) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1luB9g-0006t0-4J; Fri, 18 Jun 2021 05:51:16 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=53036 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1luB9f-0003Ok-TG; Fri, 18 Jun 2021 05:51:16 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Julien Lepiller Subject: Re: bug#48963: "guix build guix" halts after exhausting memory References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> <87im2dpoj3.fsf@gnu.org> <78EA8A84-D7DC-4506-91EB-1C304E5DB363@lepiller.eu> <20210618001737.105e2e5a@tachikoma.lepiller.eu> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 30 Prairial an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 18 Jun 2021 11:51:14 +0200 In-Reply-To: <20210618001737.105e2e5a@tachikoma.lepiller.eu> (Julien Lepiller's message of "Fri, 18 Jun 2021 00:17:37 +0200") Message-ID: <87lf77h565.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 48963 Cc: 48963@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Julien Lepiller skribis: > Le Thu, 17 Jun 2021 09:58:09 -0400, > Julien Lepiller a =C3=A9crit : > >> Hi Ludo, >>=20 >> I tried your patch by creating a variant of the guix package. My >> first attempt was a failure because it's still using almost all my >> memory and I forgot to enable my swap. In the second attempt, the >> build phase succeeded, but the build failed during the test phase >> (test/inferior). I've started a new build hoping it's a >> non-deterministic failure. > > I'm afraid after testing 2 more times that the test failure is > happening consistently, at least of armhf. (The _build_ failure, right?) Have you been able to see how this affects max RSS? Does it go any further than without the patch? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 18 06:52:26 2021 Received: (at 48963) by debbugs.gnu.org; 18 Jun 2021 10:52:26 +0000 Received: from localhost ([127.0.0.1]:55897 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1luC6s-0006YV-EU for submit@debbugs.gnu.org; Fri, 18 Jun 2021 06:52:26 -0400 Received: from lepiller.eu ([89.234.186.109]:45056) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1luC6q-0006YL-8W for 48963@debbugs.gnu.org; Fri, 18 Jun 2021 06:52:25 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 30a8b5e4; Fri, 18 Jun 2021 10:52:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date :in-reply-to:references:mime-version:content-type :content-transfer-encoding:subject:to:cc:from:message-id; s= dkim; bh=TacGd9rR4FjT2g3n1tDQbGVUBGlANEr/QPt+JI4y7pM=; b=eKuF7w3 1movNZBW36lzKj7/zxxaVQaELfEOCt4C5jSMHYHL/2aW/fT37t+nx/Tj3obLpR8v IMPgPdK1qXxK11bqolZsaFOR4gfwd0ZBjzrqNoZcQOpIHjC6n7JvPzH3aV/fr9wE hRUdUogAfcgwJ2tj2dKi6OC+b1B8NrKhKVn1wyxNy11dmC/JuxpKVcJ921doDVzp dee3qqykRLwZujz+xCwmpGs1l7CGxJxEt0NdaXwT+NDcKEfel1a+J/ZRK/cZUntb PXq+iZhE3zNItrJCbjGKJaOqyazlLD8No5iYIoq+n3B5o2Hu13SI/A4g3XgyFfw7 szz3vr73nJlNNMw== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id b8fa5b4b (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Fri, 18 Jun 2021 10:52:20 +0000 (UTC) Date: Fri, 18 Jun 2021 06:52:10 -0400 User-Agent: K-9 Mail for Android In-Reply-To: <87lf77h565.fsf@gnu.org> References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> <87im2dpoj3.fsf@gnu.org> <78EA8A84-D7DC-4506-91EB-1C304E5DB363@lepiller.eu> <20210618001737.105e2e5a@tachikoma.lepiller.eu> <87lf77h565.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----KC6XKW7O2H5R9MJ3QW9NP54CTSPIQ1" Content-Transfer-Encoding: 7bit Subject: Re: bug#48963: "guix build guix" halts after exhausting memory To: =?ISO-8859-1?Q?Ludovic_Court=E8s?= From: Julien Lepiller Message-ID: X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 48963 Cc: 48963@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) ------KC6XKW7O2H5R9MJ3QW9NP54CTSPIQ1 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable No, the _test_ failure is consistent=2E The build itself now always complet= es, which is much better that before=2E Le 18 juin 2021 05:51:14 GMT-04:00, "Ludovic Court=C3=A8s" a =C3=A9crit : >Hi, > >Julien Lepiller skribis: > >> Le Thu, 17 Jun 2021 09:58:09 -0400, >> Julien Lepiller a =C3=A9crit : >> >>> Hi Ludo, >>>=20 >>> I tried your patch by creating a variant of the guix package=2E My >>> first attempt was a failure because it's still using almost all my >>> memory and I forgot to enable my swap=2E In the second attempt, the >>> build phase succeeded, but the build failed during the test phase >>> (test/inferior)=2E I've started a new build hoping it's a >>> non-deterministic failure=2E >> >> I'm afraid after testing 2 more times that the test failure is >> happening consistently, at least of armhf=2E > >(The _build_ failure, right?) > >Have you been able to see how this affects max RSS? > >Does it go any further than without the patch? > >Thanks, >Ludo=E2=80=99=2E ------KC6XKW7O2H5R9MJ3QW9NP54CTSPIQ1 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable No, the _test_ failure is consistent=2E The build = itself now always completes, which is much better that before=2E

Le 18 juin 2021 05:51:14 GMT-04:00, "Ludovic Court= =C3=A8s" <ludo@gnu=2Eorg> a =C3=A9crit :
Hi,

Julien Lepiller <julien@lepiller=2Eeu&=
gt; skribis:

 Le Th=
u, 17 Jun 2021 09:58:09 -0400,
 Julien Lepiller <julien@lepiller=2Eeu=
> a =C3=A9crit :

Hi Ludo,

I tried your patch by creating a variant of the guix packa=
ge=2E My
first attempt was a failure because it's still using almost all=
 my
memory and I forgot to enable my swap=2E In the second attempt, the<=
br>build phase succeeded, but the build failed during the test phase
(te=
st/inferior)=2E I've started a new build hoping it's a
non-deterministic=
 failure=2E

 I'm afraid after testing 2 more times that=
 the test failure is
 happening consistently, at least of armhf=2E

(The _build_ failure, right?)

Have you been able to s=
ee how this affects max RSS?

Does it go any further than without the=
 patch?

Thanks,
Ludo=E2=80=99=2E
------KC6XKW7O2H5R9MJ3QW9NP54CTSPIQ1-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 20 17:01:02 2021 Received: (at 48963) by debbugs.gnu.org; 20 Jun 2021 21:01:02 +0000 Received: from localhost ([127.0.0.1]:34917 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lv4Yw-00072b-1w for submit@debbugs.gnu.org; Sun, 20 Jun 2021 17:01:02 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37726) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lv4Yu-000720-UK for 48963@debbugs.gnu.org; Sun, 20 Jun 2021 17:01:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49936) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lv4Yp-0004Em-Gc; Sun, 20 Jun 2021 17:00:55 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41570 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lv4Yp-0006oL-29; Sun, 20 Jun 2021 17:00:55 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Julien Lepiller Subject: Re: bug#48963: "guix build guix" halts after exhausting memory References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> <87im2dpoj3.fsf@gnu.org> <78EA8A84-D7DC-4506-91EB-1C304E5DB363@lepiller.eu> <20210618001737.105e2e5a@tachikoma.lepiller.eu> <87lf77h565.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 2 Messidor an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 20 Jun 2021 23:00:53 +0200 In-Reply-To: (Julien Lepiller's message of "Fri, 18 Jun 2021 06:52:10 -0400") Message-ID: <87v9688d4q.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 48963 Cc: 48963@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Julien Lepiller skribis: > No, the _test_ failure is consistent. The build itself now always complet= es, which is much better that before. Ah, that=E2=80=99s great! I can work on a variant of this patch that shows correct completion numbers. Could you share =E2=80=98tests/inferiors.log=E2=80=99 if that=E2=80=99s the= one that=E2=80=99s failing? Is it also failing for you on x86_64? Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 21 07:05:20 2021 Received: (at 48963) by debbugs.gnu.org; 21 Jun 2021 11:05:20 +0000 Received: from localhost ([127.0.0.1]:35574 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvHjz-0005ws-TX for submit@debbugs.gnu.org; Mon, 21 Jun 2021 07:05:20 -0400 Received: from lepiller.eu ([89.234.186.109]:45368) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvHjx-0005wh-Cc for 48963@debbugs.gnu.org; Mon, 21 Jun 2021 07:05:18 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 1cfb7bc6; Mon, 21 Jun 2021 11:05:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=dkim; bh=S4eptgMIlvOI n2zWsejNVMi9eYFFhuAvjzdGoblzRRw=; b=KmWZCh9iO7mMUDrmhPbJKtZwRIWX sP8HUPaCQWZoq7TEBHXTKpbo+J9TiGF/YF+3Ia+Ys6w6Kn9eqBoUhektLbzCCbHF BHm4ime9c0Wztbp6OTl8zVnnajh4KfXpK1nfHRKZPJA6mMTt0snQw2z6mAVzwxTe paz1zRuJNKBbYy/fHKbE1XfkeIM1JiL3pWTLkQdyaiTZxCm5H0sAWgc2SOduh65d N79i5eM82bxkPtmD3GX9ntYEPCbS2V9TMTmjJfiBlLRl2LkgA7HlcPYOiRZLGarD rG92RzN2OAlXNg9pACClyy7Kh/X9JEqhhCjT4S0MVtQUqAyweoG49isLwg== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 4a9a1a58 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Mon, 21 Jun 2021 11:05:14 +0000 (UTC) Date: Mon, 21 Jun 2021 13:05:10 +0200 From: Julien Lepiller To: Ludovic =?UTF-8?B?Q291cnTDqHM=?= Subject: Re: bug#48963: "guix build guix" halts after exhausting memory Message-ID: <20210621130510.0140bf8c@tachikoma.lepiller.eu> In-Reply-To: <87v9688d4q.fsf@gnu.org> References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> <87im2dpoj3.fsf@gnu.org> <78EA8A84-D7DC-4506-91EB-1C304E5DB363@lepiller.eu> <20210618001737.105e2e5a@tachikoma.lepiller.eu> <87lf77h565.fsf@gnu.org> <87v9688d4q.fsf@gnu.org> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 48963 Cc: 48963@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Le Sun, 20 Jun 2021 23:00:53 +0200, Ludovic Court=C3=A8s a =C3=A9crit : > Hi! >=20 > Julien Lepiller skribis: >=20 > > No, the _test_ failure is consistent. The build itself now always > > completes, which is much better that before. =20 >=20 > Ah, that=E2=80=99s great! I can work on a variant of this patch that sho= ws > correct completion numbers. >=20 > Could you share =E2=80=98tests/inferiors.log=E2=80=99 if that=E2=80=99s t= he one that=E2=80=99s > failing? Is it also failing for you on x86_64? >=20 > Thanks, > Ludo=E2=80=99. Here is the content of `tests/inferiors.log`. The failure is the same on armhf and x86_64: Backtrace: 2 (primitive-load-path "tests/inferior.scm") In ice-9/eval.scm: 626:19 1 (_ #) In unknown file: 0 (dirname #f) ERROR: In procedure dirname: In procedure scm_to_utf8_stringn: Wrong type argument in position 1 (expecting string): #f From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 23 17:44:07 2021 Received: (at 48963-done) by debbugs.gnu.org; 23 Jun 2021 21:44:07 +0000 Received: from localhost ([127.0.0.1]:41993 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwAfH-0006ge-5y for submit@debbugs.gnu.org; Wed, 23 Jun 2021 17:44:07 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33996) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lwAfF-0006g3-F3 for 48963-done@debbugs.gnu.org; Wed, 23 Jun 2021 17:44:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:51960) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lwAf9-0004iL-Fi; Wed, 23 Jun 2021 17:43:59 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=35424 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lwAf9-0006Fz-89; Wed, 23 Jun 2021 17:43:59 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Julien Lepiller Subject: Re: bug#48963: "guix build guix" halts after exhausting memory References: <20210612002330.64cd06c1@tachikoma.lepiller.eu> <87im2dpoj3.fsf@gnu.org> Date: Wed, 23 Jun 2021 23:43:55 +0200 In-Reply-To: <87im2dpoj3.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 16 Jun 2021 22:01:20 +0200") Message-ID: <87zgvgtfxg.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 48963-done Cc: 48963-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Ludovic Court=C3=A8s skribis: > Julien Lepiller skribis: >> Can we instead break the build (at the Makefile level) into multiple >> smaller chunks, that require less memory, in the same way (guix self) >> works? > > Yes, that=E2=80=99s a good idea. > > Could you check the extent to which the attached patch helps on this > machine? > > It doesn=E2=80=99t split as much as (guix self) does, only in three piece= s, but > hopefully that helps a bit. A side effect is that progress report is > now off, but we can fix it later. I pushed a variant of this patch, resorting to an ugly hack so that compile-all.scm can still estimate progress. The estimate is correct if you=E2=80=99re building all the .go files, not so much if you just do, say: make rm guix/cpio.go && make because guix/cpio.scm is in the first module set, so it=E2=80=99ll think it= =E2=80=99s starting from scratch and display =E2=80=9C0%=E2=80=9D. Fixing it is left as an exercise to the reader. :-) It seemed more important to me to fix the memory exhaustion issue first. Thanks, Ludo=E2=80=99. From unknown Mon Jun 23 23:51:15 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 22 Jul 2021 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator