GNU bug report logs - #17827
“Too many open files” error while substituter is running

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: ludo <at> gnu.org (Ludovic Courtès)
Subject: bug#17827: closed (Re: bug#17827: “Too many
 open files” error while substituter is running)
Date: Sat, 21 Jun 2014 15:38:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#17827: “Too many open files” error while substituter is running

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 17827 <at> debbugs.gnu.org.

-- 
17827: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17827
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: ludo <at> gnu.org (Ludovic Courtès)
To: 17827-done <at> debbugs.gnu.org
Subject: Re: bug#17827: “Too many open files” error while substituter is running
Date: Sat, 21 Jun 2014 17:37:26 +0200

The problem was that the unionfs process for the root file system would
run out of file descriptors (the default limit is 1024.)

Fixed in commit 9b9148d.

Ludo’.

[Message part 3 (message/rfc822, inline)]

From: ludo <at> gnu.org (Ludovic Courtès)
To: bug-guix <at> gnu.org
Subject: “Too many open files” error
 while substituter is running
Date: Sat, 21 Jun 2014 16:24:05 +0200

While running the substituter in a disk/VM image built with Guix (on the
metal or on QEMU), I quickly stumble upon EMFILE.  It goes like this:

  1. Run ‘guix build emacs -n’ to warm up /var/guix/substitute-binary;
     this (currently) uses many threads and everything, but doesn’t
     cause any problems.

  2. Run ‘guix build emacs’.  This starts downloading substitutes, via
     ‘guix substitute-binary --substitute’.  It works fine, except that
     running something in parallel like ‘guix package --help’ quickly
     fails with EMFILE (“Too many open files”.)

Looking at /proc/sys/fs/file-nr suggests we’re still way below the
threshold at that point.  Increasing file-max doesn’t make a difference.

Looking at the strace output of the failing ‘guix package --help’ shows
that it does lots of open/close calls, but remains at 10 open files.
Eventually, one of its close(2) calls (!) returns EMFILE, and all the
subsequent open(2) calls fail similarly.

The problem seems to manifest more easily when there’s a lot of RAM,
which made me thing of a Scheme file port leak.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.