GNU bug report logs - #49439
grafts cause “guix environment” to get killed with OOM

Previous Next

Full log

Message #28 received at 49439 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Sarah Morgensen <iskarian <at> mgsn.dev>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 49439 <at> debbugs.gnu.org
Subject: Re: bug#49439: grafts cause “guix environment” to get killed with OOM
Date: Thu, 29 Jul 2021 00:03:46 +0200

Hi,

Ludovic Courtès <ludo <at> gnu.org> skribis:

> Thinking about it, the grafts code depends on what’s in the store: when
> nothing is in the store, it bounces to the “build handler”, which
> accumulates the list of missing store items, until it starts building
> them.

So I can reproduce the problem Ricardo and you initially reported by
running:

  ./pre-inst-env guix environment pigx-scrnaseq --search-paths

after removing some of the ungrafted store items with:

  guix gc -D $(guix build r-rlang --no-grafts) \
    $(guix gc --references $(guix build pigx-scrnaseq --no-grafts))

The seemingly endless CPU usage and unbound memory use comes from the
‘build-accumulator’ build handler.  Here, the graph of ‘pigx-scrnaseq’
has many nodes, and many of them depend on, say, ‘r-rlang’.  Thus, when
‘r-rlang’ is not in the store, the grafting code keeps asking for it by
calling ‘build-derivations’, which aborts to the build handler; the
build handler saves the .drv file name and the continuation and keeps
going.  But since the next package also depends on ‘r-langr’, we abort
again to the build handler, and so on.

The end result is a very long list of <unresolved> nodes, probably of
this order in this case:

  $ guix graph -t reverse-package r-rlang |grep 'label = "'|wc -l
  594

Presumably, the captured continuations occupy quite a bit of memory,
hence the quick growth.

I suppose one solution is to fire suspended builds when the build
handler sees a build request for a given derivation for the second time.
It needs more thought and testing…

Ludo’.

PS: Did you know ‘pigx-scrnaseq’ has twice as many nodes as
    ‘libreoffice’?

      $ guix graph -t bag pigx-scrnaseq |grep 'label = "'|wc -l
      1359
      $ guix graph -t bag libreoffice |grep 'label = "'|wc -l
      699

    That makes it a great example to study and fix scalability issues!

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.