GNU bug report logs - #59321
ice-9's open-input-pipe is unexpectedly slow on some systems

Previous Next

Full log

Message #26 received at 59321 <at> debbugs.gnu.org (full text, mbox):

From: Andrew Whatson <whatson <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: hylophile <at> posteo.de, 59321 <at> debbugs.gnu.org
Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some
 systems
Date: Thu, 8 Dec 2022 22:02:04 +1000

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> wrote:
>
> Andrew Whatson <whatson <at> gmail.com> skribis:
>
> > We also need equivalent functionality around SOCK_CLOEXEC.  It seems
> > this is implemented for ‘accept’, but not ‘socket’ or ‘socketpair’.
>
> It’s possible to use SOCK_CLOEXEC with ‘socket’ and ‘socketpair’
> already, as in:
>
>   (socket AF_INET (logior SOCK_CLOEXEC SOCK_STREAM) 0)
>
> With commit 1d313bf5f0d296d766bd3a0e6d030df37c71711b, ‘pipe’ is also
> covered.
>
> So I think we have pretty much everything we need, at least starting
> with 3.0.9.

Ah, nice!  In that case it might be possible to deprecate this
auto-closing behaviour in a future version.

> > Python's PEP 433 contains a good explanation of the issues which can
> > arise from leaked file descriptors:
> > https://peps.python.org/pep-0433/#inherited-file-descriptors-issues
> >
> > Given the risks, I'm convinced that Guile's conservative approach is
> > actually quite sensible.  It seems like the best path forward would be
> > to implement platform-specific optimizations where possible.
> >
> > I've attached a draft patch which implements a fast-path on systems
> > where "/proc/self/fd" is available.
>
> The patch LGTM; it’s certainly an improvement on systems configured with
> a high per-process FD limit.
>
> Now, I believe use of ‘posix_spawn’ as proposed in
> <https://issues.guix.gnu.org/52835> makes that unnecessary.  Let’s take
> a closer look at that other patch and so we can see…

Playing with the wip-posix-spawn branch, it has the same slowdown
(actually a bit worse).  I've updated the "/proc/self/fd" fast-path
patch for posix_spawn, please find attached.

> Thanks,
> Ludo’.

Thank you!

[close-inherited-with-proc-self-fd-psawn.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.