GNU bug report logs - #22814
25.0.91; Emacs runs out of file descriptors on OS X

Previous Next

Full log

Message #66 received at 22814 <at> debbugs.gnu.org (full text, mbox):

From: Anders Lindgren <andlind <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 22814 <at> debbugs.gnu.org
Subject: Re: bug#22814: 25.0.91; Emacs runs out of file descriptors on OS X
Date: Sat, 27 Feb 2016 20:00:30 +0100

[Message part 1 (text/plain, inline)]

Hi!

I just gave this some thought (without knowing anything about the kqueue
notifications).

I don't think the problem is what kqueue does when it run out of file
descriptors, but how the rest of the Emacs process acts when this happens.
For example, can it even read and write files? Can subprocesses be started?

The question, then, is how should the kqueue system work so that it doesn't
run Emacs out of file descriptors? For example, if it would be possible to
check how many are available, it could stop when there are, say, 50
remaining. Effectively, this would give a user 200 files that are
auto-reverted using the notification system, and the rest would be handled
by the old timer system.

    -- Anders

On Sat, Feb 27, 2016 at 2:26 PM, Michael Albinus <michael.albinus <at> gmx.de>
wrote:

> Eli Zaretskii <eliz <at> gnu.org> writes:
>
> >> > One simple way to handle this is to define a variable with "max"
> >> > number of files the notification system can handle. We can set this
> >> > to, say, 200 on OS X and unlimited on other systems.
> >>
> >> Would be possible, yes. I would prefer to set the limit to a system
> >> related value. Does there exist a portable way to detect, how many file
> >> descriptors / processes Emacs is able to consume?
> >
> > Not portably, AFAIK.
>
> On POSIX systems, we could propagate the result of
>
> if !(getrlimit (RLIMIT_NOFILE, &rlim))
>    return make_number (rlim.rlim_cur);
> else
>    return Qnil;
>
> Maybe we add a function like `<library>-max-descriptors' to the
> libraries. Or maybe not, and the backends check this themselves, and
> cease to work when reaching an internal limit. Such an internal limit
> could be half the number of soft RLIMIT_NOFILE on POSIX systems, for
> example.
>
> > Also, different implementations use different
> > resources for receiving file notifications.  For example, w32notify
> > uses one handle and one thread per watched file/directory.  The number
> > of handles a process can have on Windows is very large, and the
> > theoretical max number of threads is 32K, but both are limited by the
> > resources already consumed by the Emacs process.  So determining the
> > practical maximum at any given point will require a non-trivial
> > function.
>
> There must be different scenarios for different file notification
> libraries. But using RLIMIT_NOFILE as basis when possible gives us
> better results on systems like OS X and FreeBSD, which both use kqueue,
> but provide different ressource limits.
>
> Best regards, Michael.
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.