GNU bug report logs - #48963
"guix build guix" halts after exhausting memory

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#48963: closed ("guix build guix" halts after exhausting memory)
Date: Wed, 23 Jun 2021 21:45:02 +0000

[Message part 1 (text/plain, inline)]

Your message dated Wed, 23 Jun 2021 23:43:55 +0200
with message-id <87zgvgtfxg.fsf <at> gnu.org>
and subject line Re: bug#48963: "guix build guix" halts after exhausting memory
has caused the debbugs.gnu.org bug report #48963,
regarding "guix build guix" halts after exhausting memory
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
48963: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=48963
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Julien Lepiller <julien <at> lepiller.eu>
To: bug-guix <at> gnu.org
Subject: "guix build guix" halts after exhausting memory
Date: Sat, 12 Jun 2021 00:23:30 +0200

Hi Guix!

I tried updating my system on my armhf board (2GB of RAM), but during
"guix system reconfigure", guix tries to build itself (the guix package
from (gnu packages package-management)). This package uses too much
memory to build, and I start getting GC warnings like so:

GC Warning: Out of memory - trying to allocate requested amount (552
bytes)...
GC Warning: Header allocation failed: dropping block
GC Warning: Out of Memory! Heap size: 2571 MiB. Returning NULL!
Warning: Unwind-only out of memory exception; skipping pre-unwind
handler.

after some more progress and a lot more warnings, the build stops (CPU
is at 2%), but the memory is not freed. The build stayed stuck for a
long time until I decided to cancel it.

As a work-around, I tried using the guix-daemon package instead (by
changing the guix field in guix-configuration), but "guix system" still
wants to build the guix package anyway. I tried finding usages of the
guix package (grepping for "package-management" yields a relatively
short list of files, and none of them seems to be used by my config,
except (gnu services base) which always uses the guix from the
configuration). Why?

Can we instead break the build (at the Makefile level) into multiple
smaller chunks, that require less memory, in the same way (guix self)
works?

[Message part 3 (message/rfc822, inline)]

From: Ludovic Courtès <ludo <at> gnu.org>
To: Julien Lepiller <julien <at> lepiller.eu>
Cc: 48963-done <at> debbugs.gnu.org
Subject: Re: bug#48963: "guix build guix" halts after exhausting memory
Date: Wed, 23 Jun 2021 23:43:55 +0200

Hi,

Ludovic Courtès <ludo <at> gnu.org> skribis:

> Julien Lepiller <julien <at> lepiller.eu> skribis:

>> Can we instead break the build (at the Makefile level) into multiple
>> smaller chunks, that require less memory, in the same way (guix self)
>> works?
>
> Yes, that’s a good idea.
>
> Could you check the extent to which the attached patch helps on this
> machine?
>
> It doesn’t split as much as (guix self) does, only in three pieces, but
> hopefully that helps a bit.  A side effect is that progress report is
> now off, but we can fix it later.

I pushed a variant of this patch, resorting to an ugly hack so that
compile-all.scm can still estimate progress.  The estimate is correct if
you’re building all the .go files, not so much if you just do, say:

  make
  rm guix/cpio.go && make

because guix/cpio.scm is in the first module set, so it’ll think it’s
starting from scratch and display “0%”.

Fixing it is left as an exercise to the reader.  :-)

It seemed more important to me to fix the memory exhaustion issue first.

Thanks,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.