GNU bug report logs - #17827
“Too many open files” error while substituter is running

Previous Next

Full log

View this message in rfc822 format

From: ludo <at> gnu.org (Ludovic Courtès)
To: 17827 <at> debbugs.gnu.org
Subject: bug#17827: “Too many open files” error while substituter is running
Date: Sat, 21 Jun 2014 16:24:05 +0200

While running the substituter in a disk/VM image built with Guix (on the
metal or on QEMU), I quickly stumble upon EMFILE.  It goes like this:

  1. Run ‘guix build emacs -n’ to warm up /var/guix/substitute-binary;
     this (currently) uses many threads and everything, but doesn’t
     cause any problems.

  2. Run ‘guix build emacs’.  This starts downloading substitutes, via
     ‘guix substitute-binary --substitute’.  It works fine, except that
     running something in parallel like ‘guix package --help’ quickly
     fails with EMFILE (“Too many open files”.)

Looking at /proc/sys/fs/file-nr suggests we’re still way below the
threshold at that point.  Increasing file-max doesn’t make a difference.

Looking at the strace output of the failing ‘guix package --help’ shows
that it does lots of open/close calls, but remains at 10 open files.
Eventually, one of its close(2) calls (!) returns EMFILE, and all the
subsequent open(2) calls fail similarly.

The problem seems to manifest more easily when there’s a lot of RAM,
which made me thing of a Scheme file port leak.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.