From debbugs-submit-bounces@debbugs.gnu.org Wed Nov 16 15:56:27 2022 Received: (at submit) by debbugs.gnu.org; 16 Nov 2022 20:56:27 +0000 Received: from localhost ([127.0.0.1]:58296 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ovPSM-0001fc-1h for submit@debbugs.gnu.org; Wed, 16 Nov 2022 15:56:27 -0500 Received: from lists.gnu.org ([209.51.188.17]:46690) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ovMnf-0001KU-3e for submit@debbugs.gnu.org; Wed, 16 Nov 2022 13:06:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovMne-000064-VM for bug-guile@gnu.org; Wed, 16 Nov 2022 13:06:14 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovMnb-0006OC-NI for bug-guile@gnu.org; Wed, 16 Nov 2022 13:06:14 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 72DB4240106 for ; Wed, 16 Nov 2022 19:06:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.de; s=2017; t=1668621968; bh=IIoD+jhfwHM6ZuuCarXFMvsSagbavs9K9tTw5EPtQL8=; h=Date:From:To:Subject:From; b=L4aEJ4Y9WclUgj2Jdf5bJHBj7xLn/DbGpnq9MFWGTL1r47gqMSkAscmNyTRxTNny5 liNm/sjOnTd1ti1URzsW/EeVZAosMwjwemxyUDbL/WrjlnFyzKA2mw/cY5N45nkCNy Ez2opvoKjCzFUQtmslmIoxQMbMsBy4XnYjHE9LmK7ce7ENKn2KmbgAkkXRCveJEj+0 mCtU5tnenFzwnPJhYYmzJQiYTwPncCG0ZzoLiClyrpaVJT2XD0tOTF+TWpQvCbRnt3 oJyOVeQzQwi0x9HgWdbm7sM25VBJ2fdZwCBKxRV+j2DvQDmsQxq+2wYrjo7UOnLDtM WzD5Lk+czlWeQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4NC9vz1S70z6tpm for ; Wed, 16 Nov 2022 19:06:07 +0100 (CET) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 16 Nov 2022 18:06:06 +0000 From: hylophile@posteo.de To: bug-guile@gnu.org Subject: ice-9's open-input-pipe is unexpectedly slow on some systems Message-ID: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> Received-SPF: pass client-ip=185.67.36.66; envelope-from=hylophile@posteo.de; helo=mout02.posteo.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Wed, 16 Nov 2022 15:56:23 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Hi, while doing my first steps in Guile and porting a small shell script, I noticed that capturing stdout from a subshell is very slow: $ time guile -c '(begin (use-modules (ice-9 popen) (ice-9 rdelim)) (display (read-delimited "" (open-input-pipe "ls"))))' # real 0m0,313s user 0m0,008s sys 0m0,007s It takes around 300ms where I would expect around 40ms. The slowness happens on one of my machines, but not the other (at least not noticeably). However, I am fairly sure that my machine is not at fault, because this doesn't happen with other languages. Additionally, another Guile user tried this on their machines and also noticed slowness (around 200ms) on one of their machines. We did notice that the machines where the operation runs fast also run on better hardware, but the stark difference is still unexpected. Unfortunately I am not sure what exactly causes the issue, but I am happy to provide any further information needed. The mentioned Guile user was able to pinpoint the delay to be between two read system calls by going through an strace. To reproduce: 1. Create an empty folder with one file named "testfile" in it 2. Run: $ strace -o ice-9.trace -s 256 --timestamps=format:time,precision:ms guile -c '(begin (use-modules (ice-9 popen) (ice-9 rdelim)) (display (read-delimited "" (open-input-pipe "ls"))))' 3. In the resulting ice-9.trace, search for "testfile" and go up to the first read call which reads the output of ls. Between the first and the second read call, there is a noticeable delay. For one of my systems it's around 300ms, for the other it's around 90ms: 18:37:08.880 close(6) = 0 18:37:08.880 read(5, "i", 1) = 1 18:37:09.189 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=188843, si_uid=1000, si_status=0, si_utime=15 /* 0.15 s */, si_stime=14 /* 0.14 s */} --- 18:37:09.189 read(5, "c", 1) = 1 18:37:09.189 read(5, "e", 1) = 1 Regards, Nate System information: guile (GNU Guile) 3.0.8 I am using the Arch Linux guile package, I believe the build configuration can be found here: https://github.com/archlinux/svntogit-packages/blob/packages/guile/trunk/PKGBUILD#L23 Output of config.guess: x86_64-pc-linux-gnu From debbugs-submit-bounces@debbugs.gnu.org Thu Nov 17 23:49:24 2022 Received: (at 59321) by debbugs.gnu.org; 18 Nov 2022 04:49:24 +0000 Received: from localhost ([127.0.0.1]:34896 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ovtJb-0007s7-Qo for submit@debbugs.gnu.org; Thu, 17 Nov 2022 23:49:24 -0500 Received: from mail-yw1-f181.google.com ([209.85.128.181]:36355) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ovtJZ-0007rv-Pm for 59321@debbugs.gnu.org; Thu, 17 Nov 2022 23:49:22 -0500 Received: by mail-yw1-f181.google.com with SMTP id 00721157ae682-37063f855e5so39386727b3.3 for <59321@debbugs.gnu.org>; Thu, 17 Nov 2022 20:49:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=4e+HqZh93T+Omb3LJMI6aXQuk+8ehnB9Ch9VuVY/o4Q=; b=Avs/rxumSHTGa3FdNyAqbFqtQrKaIUXL+A5XaaqQWUNNDy4I1m/MdFRIDjAYb8di+9 m50H8r8CQmRFPKLnXs1FU+zLD6ThU66JmyELBOtMuOwcOuHOwOxHDCFJ/f7FYrwW1Yim LIAhLQ/4zzv5TtgUW8EbttStH+tamXqCDpXI8fziD3K72a/ojn6YIODkKxHzBlcNI6w+ qxj9UKAQh8AvO+QSyeLAmY0C3pyBhsGL6GFi2WArsFqOPkwU6QL32OHJLPs+EPd7biVN zMPXMRfG69Sz/N/l06C2B7DjOG9U6e8bYTtZZFm1jJJHfcYYKUYoh+KjQ5PU7MSikr+5 unIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=4e+HqZh93T+Omb3LJMI6aXQuk+8ehnB9Ch9VuVY/o4Q=; b=L00YfqYlVEf1U/PTEe7JDqYdHb8aLe96a7THtrExBJQ39Zv1UZ/Wq8KS35fhgXjv5q y3wQlE7vfRRljqtjMZLmOFwzVI52GXpjQDqymgIYDiYj/hZ4ZAA91yz+it939CJdnk1i IV/GqUlgY/62lGiW+A159CSXB3jvDbrhGE5xi+D6aylHFJQTRRkL2kij90MoHEcqYNWR Gbuz57R+n28u7FSBIUxC+hgpxPVNyfq1vZGGi0mHccxXGG4/adwEth1SFZPWRru2XJR/ uPTI/QJDIwKpgNbLSbVjGjY6YaMYJdFG1EmLWjv/0qRgOFNRKa4PuYb6AApA5DTJQOMM 4oxQ== X-Gm-Message-State: ANoB5plUGpW8NPXIgh8Ncko//y0hbPwXOl+TDm+RqmdDqcdHlpHZVjLk IlzjuAMQsAghkacvCdDDsKrk/LQKrEOM4egbESg= X-Google-Smtp-Source: AA0mqf5EeohGz6EY+7nU0I/KcBPSvR5iPFt3OZtGd/0uBj7/ApV6W9ggT8R+QxQXIfEM4Ktg3N1gIZP9GbjlKcP3yg0= X-Received: by 2002:a81:49c4:0:b0:368:f9d0:1f28 with SMTP id w187-20020a8149c4000000b00368f9d01f28mr5218260ywa.374.1668746956005; Thu, 17 Nov 2022 20:49:16 -0800 (PST) MIME-Version: 1.0 From: Andrew Whatson Date: Fri, 18 Nov 2022 14:49:05 +1000 Message-ID: Subject: Re: ice-9's open-input-pipe is unexpectedly slow on some systems To: hylophile@posteo.de Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 59321 Cc: 59321@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Nate, Thanks for reporting this! I'm able to reproduce on my systems, and have identified the culprit: > strace -c -f guile -c '(begin (use-modules (ice-9 popen) (ice-9 rdelim)) (display (read-delimited "" (open-input-pipe "ls"))))' % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ------------------ 93.38 0.309261 1 250107 249994 close 4.61 0.015268 135 113 21 futex 0.52 0.001730 4 430 270 newfstatat 0.38 0.001263 6 183 mmap 0.29 0.000953 4 197 92 openat 0.16 0.000542 180 3 execve <...snip...> These close calls are from the start_child routine in libguile/posix.c: static pid_t start_child (const char *exec_file, char **exec_argv, int reading, int c2p[2], int writing, int p2c[2], int in, int out, int err) { int pid; int max_fd = 1024; #if defined (HAVE_GETRLIMIT) && defined (RLIMIT_NOFILE) { struct rlimit lim = { 0, 0 }; if (getrlimit (RLIMIT_NOFILE, &lim) == 0) max_fd = lim.rlim_cur; } #endif <...snip...> /* Close all file descriptors in ports inherited from the parent except for in, out, and err. Heavy-handed, but robust. */ while (max_fd--) if (max_fd != in && max_fd != out && max_fd != err) close (max_fd); <...snip...> On my system, `ulimit -n` shows the maximum number of open files as 250000, explaining the 250000 calls to close shown by strace. Testing a build of guile with the max_fd loop commented out shows the expected performance: # unmodified guile > time guile -c '(begin (use-modules (ice-9 popen) (ice-9 rdelim)) (display (read-delimited "" (open-input-pipe "ls"))))' real 0m0.202s # guile patched to remove max_fd loop > time guile -c '(begin (use-modules (ice-9 popen) (ice-9 rdelim)) (display (read-delimited "" (open-input-pipe "ls"))))' real 0m0.056s Forcibly closing file descriptors like this shouldn't be necessary if the application has properly opened descriptors with the FD_CLOEXEC flag. It would be good to get input from some more experienced Guile hackers on the potential consequences of this change. Cheers, Andrew From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 18 00:52:36 2022 Received: (at 59321) by debbugs.gnu.org; 18 Nov 2022 05:52:36 +0000 Received: from localhost ([127.0.0.1]:35002 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ovuIm-00019G-Cx for submit@debbugs.gnu.org; Fri, 18 Nov 2022 00:52:36 -0500 Received: from mail.tuxteam.de ([5.199.139.25]:51462) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ovuIh-000194-5e for 59321@debbugs.gnu.org; Fri, 18 Nov 2022 00:52:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=tuxteam.de; s=mail; h=From:In-Reply-To:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:Date:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=RjTO6+JUen6jDFDM0/QvQ+gj7Gj2zYRFeow8MPQmzec=; b=Q5PC6T4UfBa4IYtF9SwJoq6eTd j23tlR9ajRzeC6hMewbl8EnmScYagbEqyxRsNabowhfSJ9Kb209rk0dyTmSurH87c85o1p9HfFew+ u0kHZXb7ltPabr93f5nItCnRbNU48ssRvEVDlwSFgKrScuAfcyuLq3RkWytO1zZOKG4pjJFe6ykP6 M05jrt4EHlC/bNBuzqe1sgOLgwYnsWwVrlab04CJNXyGZjgNwTCAx/uWiVIcqExY0cIWTiEFA10kA mrzu5hkRIUSl9vWE35VkbWpClQl2iFA/TWXEzKhT7DssGc//abdGT6tswg+gMOeB+JOM6jyiMmH9V fsLql7uQ==; Received: from tomas by mail.tuxteam.de with local (Exim 4.94.2) (envelope-from ) id 1ovuId-0004yZ-7A; Fri, 18 Nov 2022 06:52:27 +0100 Date: Fri, 18 Nov 2022 06:52:27 +0100 To: Andrew Whatson Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems Message-ID: References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HphKHzm1chwGZDfA" Content-Disposition: inline In-Reply-To: From: X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 59321 Cc: hylophile@posteo.de, 59321@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --HphKHzm1chwGZDfA Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 18, 2022 at 02:49:05PM +1000, Andrew Whatson wrote: > Hi Nate, >=20 > Thanks for reporting this! I'm able to reproduce on my systems, and > have identified the culprit: [250k calls to close()] Well spotted. Here [1] is a good discussion on the problem. Spoiler alert: there is no nice, simple and portable way around it. There may be complex, platform-dependent solutions (like, for example, under Linux go look in /proc//fd; BSD has other tricks). Of course, that doesn't mean that one should give up trying :) Cheers [1] https://stackoverflow.com/questions/56650579/why-should-i-close-all-fil= e-descriptors-after-calling-fork-and-prior-to-callin --=20 t --HphKHzm1chwGZDfA Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCY3cdkgAKCRAFyCz1etHa RsbVAJ97Yg5D78wAQwgvTWWklh9x7C7hEACeNOEVu093R3BYgmPtvPVtNjYdHHc= =rpL5 -----END PGP SIGNATURE----- --HphKHzm1chwGZDfA-- From debbugs-submit-bounces@debbugs.gnu.org Sun Nov 20 12:25:07 2022 Received: (at 59321) by debbugs.gnu.org; 20 Nov 2022 17:25:07 +0000 Received: from localhost ([127.0.0.1]:44461 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owo42-0002WP-Rk for submit@debbugs.gnu.org; Sun, 20 Nov 2022 12:25:07 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38526) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owo41-0002VL-IL for 59321@debbugs.gnu.org; Sun, 20 Nov 2022 12:25:06 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1owo3w-0007Ua-8s; Sun, 20 Nov 2022 12:25:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=VECu9CtbGGfcWDVFEtiqSfN+IqwGvLvXiONCC45XmHw=; b=FP2dRBru/n65hdg1xcKR C39LCbUlFp7gChgHXfEGWpUZmUhfa0+e6bo5PVGewRx4BLdgZLmjWnZ98CYiWaE+gS+MOiCn+UX9b FEGW7FXCvn3GQDtaf1FGzxn09RBwQydZQRQYDN4/ZKQhcDo1yQVda7EQFUTN470WyLZy4HuoGJ20Q jMz9guP1Ov4QjgW8eNts7Ryx/tXOkzDBjuABv6HZLKiHJzUSxYnk3msMot6U9BBdrJcKYTXQ3e9YL 7yvDQCjBzo4VyYlZaZYIuyh5ZXHbzjefv2gHdsjHGdnr34RlSEfqAE5ifUTwF2MOcul0CGHheqxxL 8FfGdEYn0aS38A==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1owo3v-0002PR-TF; Sun, 20 Nov 2022 12:25:00 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Andrew Whatson Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> Date: Sun, 20 Nov 2022 18:24:57 +0100 In-Reply-To: (Andrew Whatson's message of "Fri, 18 Nov 2022 14:49:05 +1000") Message-ID: <87r0xx5yja.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 59321 Cc: hylophile@posteo.de, 59321@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Andrew Whatson skribis: > Forcibly closing file descriptors like this shouldn't be necessary if > the application has properly opened descriptors with the FD_CLOEXEC > flag. It would be good to get input from some more experienced Guile > hackers on the potential consequences of this change. Libguile opens all its own file descriptors at O_CLOEXEC (one omission was recently fixed in 0aa1a9976fc3c6af4d1087e59d728cb8fe7d369a) so it may be possible to remove that FD-closing loop. There=E2=80=99s still the possibility that application bug unwillingly leaks FDs, but we could consider it=E2=80=99s none of our business. Thoughts? Similarly, with commit a356ceebee000efe91a2a16dbcaa64d6c6a3a922, it=E2=80= =99s possible to pass =E2=80=98open-file=E2=80=99 a flag that corresponds to O_C= LOEXEC, which wasn=E2=80=99t possible before. I=E2=80=99ve also been thinking that files= opened with =E2=80=98call-with-*=E2=80=99 should be O_CLOEXEC. That=E2=80=99d be an in= compatible change though, so maybe not something for 3.0.x. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Sun Nov 20 14:00:42 2022 Received: (at submit) by debbugs.gnu.org; 20 Nov 2022 19:00:42 +0000 Received: from localhost ([127.0.0.1]:44652 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owpYY-0007GM-00 for submit@debbugs.gnu.org; Sun, 20 Nov 2022 14:00:42 -0500 Received: from lists.gnu.org ([209.51.188.17]:54168) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owpYW-0007GD-ON for submit@debbugs.gnu.org; Sun, 20 Nov 2022 14:00:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1owpYW-0000Fo-A0 for bug-guile@gnu.org; Sun, 20 Nov 2022 14:00:40 -0500 Received: from mail.tuxteam.de ([5.199.139.25]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1owpYU-0004iN-HI for bug-guile@gnu.org; Sun, 20 Nov 2022 14:00:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=tuxteam.de; s=mail; h=From:In-Reply-To:Content-Type:MIME-Version:References:Message-ID: Subject:To:Date:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=6MF26W4pz0E0vIZb8jzS2WPEV8sMBkMdAJHbF4VwaRI=; b=NGtEHE20gSASMObfHpH0IOZKpv 69oCb1vfkwbvmHcAhHZFVm6GRfGJK3gN40kgD0PL5Rzfy7IuvUSdOsQMo3NIF/spunaId81XbVvbt qN87DVn2Mf8P/rYBOkye0jcSUx6uEAD3zX/JRTmf7fccbZVu3R83ow/YNOd/J2BObwS7GDuMsC4F0 6J793M6rFOgTQoNc6YnF/KFIO4/VGYak7hBzhJ6RiQI3m7vLHeIYlsR6Gvhts0fGUgotZ5fjLekJj yJs4vDY+oClHmp+Q89PqMKyaTsw8L2qd5CwjSJAGTlj3fBXYv+dN01ea4tHDer2P3r2s28hgA+EDL Mym+/+jQ==; Received: from tomas by mail.tuxteam.de with local (Exim 4.94.2) (envelope-from ) id 1owpYQ-0007js-Mf for bug-guile@gnu.org; Sun, 20 Nov 2022 20:00:34 +0100 Date: Sun, 20 Nov 2022 20:00:34 +0100 To: bug-guile@gnu.org Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems Message-ID: References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> <87r0xx5yja.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="QD+SUpFxjK9Bt2f4" Content-Disposition: inline In-Reply-To: <87r0xx5yja.fsf@gnu.org> From: Received-SPF: pass client-ip=5.199.139.25; envelope-from=tomas@tuxteam.de; helo=mail.tuxteam.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --QD+SUpFxjK9Bt2f4 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Nov 20, 2022 at 06:24:57PM +0100, Ludovic Court=C3=A8s wrote: > Hi, >=20 > Andrew Whatson skribis: >=20 > > Forcibly closing file descriptors like this shouldn't be necessary if > > the application has properly opened descriptors with the FD_CLOEXEC > > flag. It would be good to get input from some more experienced Guile > > hackers on the potential consequences of this change. >=20 > Libguile opens all its own file descriptors at O_CLOEXEC (one omission > was recently fixed in 0aa1a9976fc3c6af4d1087e59d728cb8fe7d369a) so it > may be possible to remove that FD-closing loop. There=E2=80=99s still the > possibility that application bug unwillingly leaks FDs, but we could > consider it=E2=80=99s none of our business. >=20 > Thoughts? Hm. Socket FDs don't "have" O_CLOEXEC. Arguably, they are at least as "interesting" as file FDs (meaning: source of obscure bugs). Moreover, misbehaving C libraries can be an additional source of bugs we have no control of. The reference I posted upthread makes a compelling case for at least needing an option for this (admittedly ugly) close orgy (perhaps with some additional platform-dependent mitigations, but that's an ugliness in its own, sigh). Cheers --=20 t --QD+SUpFxjK9Bt2f4 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCY3p5TAAKCRAFyCz1etHa RkdhAJ9OwLKxr//x1fHiEDF6JXcaNPJ7PwCePE7SZITdpX5l9tufDzMsnLFPboQ= =8Oqy -----END PGP SIGNATURE----- --QD+SUpFxjK9Bt2f4-- From debbugs-submit-bounces@debbugs.gnu.org Sun Nov 20 23:23:14 2022 Received: (at 59321) by debbugs.gnu.org; 21 Nov 2022 04:23:14 +0000 Received: from localhost ([127.0.0.1]:45013 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owyKv-0006aO-Jv for submit@debbugs.gnu.org; Sun, 20 Nov 2022 23:23:14 -0500 Received: from mail-yw1-f178.google.com ([209.85.128.178]:43785) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1owyKr-0006a9-8k for 59321@debbugs.gnu.org; Sun, 20 Nov 2022 23:23:11 -0500 Received: by mail-yw1-f178.google.com with SMTP id 00721157ae682-39451671bdfso77289697b3.10 for <59321@debbugs.gnu.org>; Sun, 20 Nov 2022 20:23:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=zQ+NYjrXwu4DhvYqqjBfUY0BCM9MgnIc428sFv5gVPg=; b=Ie75V/5J6LHL8uTh2pzj413MY532n4LLtrohHIDhD2n0aL3sxTLnY8S2LkxQheNw3m 8JqhO1ReJmseIYbrkO5bV18ZHSWHKGbnUMRP0gw9/qI06xlI7S90LtkPutl38NGrAKOF 5B3yu9HItjVpwKxbEKWROIj3LyxtmMRbdKmSj6WgqNSykvW62qPgxMzCOMxDWPePagbq j/NLNq4y9FAAiPBIIeWs9gh+VWtBGEi1FsajTvaAZs6La2f7YcyHhElsP76IzU/BmGca AC6YmNwbJ5+v14/c63VQTfsiEHevl1s9x+hA6CJJtjUfFRWtsHeb/k/SvHnM0szlvNin qKJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zQ+NYjrXwu4DhvYqqjBfUY0BCM9MgnIc428sFv5gVPg=; b=msgmayxMEsdtXTp98FGLIz88KxX3ktHhNDSqWS6N++xZKrcX/sy3ztGH8Ro62eAxyE 9wsrbCR08QsN5zRYTykdxn1Cq8SJ6tss72hvM8a8RNHHVVr+N9PcRnhcdg8S14WYQDmz tDJDAPY4FyU1tpkQqRLMEt49h4uqWod+NBzjzgfrD9EUXbBhdKvJ5do66fvVPTtwKyjY sDwkwsvaubX+Wr7+VfCE0QvWrzKDXvpxfsH5qTxpHcA5gkkpi1NIviRU0xkweHfQE4Hn 4b1XVkq9goAmMjEnqjkMXr8870e2wYRSIdSU11pw51jbnI2BNgB+8j6M24PE5glUKhXA bVFw== X-Gm-Message-State: ANoB5pk4uf65/kVHnfnTf60CwisK6WWbEqJ/Wh7bXcYLwup5I1Sv12Fo bTyAe7fC5D3WljkQQWImVB8+atT4u/9l5ig0GkU= X-Google-Smtp-Source: AA0mqf7fXLeWWTvTc2SjohFJsZ4YkP4tAaP1lRM6yQOvTQYU/pouiz08qsVqhQAQRKcNHKxdEjqkhxUXP64iX6o1SHw= X-Received: by 2002:a81:b04f:0:b0:357:94cb:7a9 with SMTP id x15-20020a81b04f000000b0035794cb07a9mr15114990ywk.328.1669004583522; Sun, 20 Nov 2022 20:23:03 -0800 (PST) MIME-Version: 1.0 References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> <87r0xx5yja.fsf@gnu.org> In-Reply-To: <87r0xx5yja.fsf@gnu.org> From: Andrew Whatson Date: Mon, 21 Nov 2022 14:22:52 +1000 Message-ID: Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Content-Type: multipart/mixed; boundary="000000000000558d8305edf369cc" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 59321 Cc: hylophile@posteo.de, 59321@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --000000000000558d8305edf369cc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s wrote: > > Andrew Whatson skribis: > > > Forcibly closing file descriptors like this shouldn't be necessary if > > the application has properly opened descriptors with the FD_CLOEXEC > > flag. It would be good to get input from some more experienced Guile > > hackers on the potential consequences of this change. > > Libguile opens all its own file descriptors at O_CLOEXEC (one omission > was recently fixed in 0aa1a9976fc3c6af4d1087e59d728cb8fe7d369a) so it > may be possible to remove that FD-closing loop. There=E2=80=99s still th= e > possibility that application bug unwillingly leaks FDs, but we could > consider it=E2=80=99s none of our business. > > Thoughts? I agree with this approach in principle, but from what Tomas is saying it seems like it's not currently possible for applications to do the right thing in all cases. > Similarly, with commit a356ceebee000efe91a2a16dbcaa64d6c6a3a922, it=E2=80= =99s > possible to pass =E2=80=98open-file=E2=80=99 a flag that corresponds to O= _CLOEXEC, > which wasn=E2=80=99t possible before. Nice! > I=E2=80=99ve also been thinking that files opened with =E2=80=98call-with= -*=E2=80=99 should be > O_CLOEXEC. That=E2=80=99d be an incompatible change though, so maybe not > something for 3.0.x. This sounds reasonable to me. We also need equivalent functionality around SOCK_CLOEXEC. It seems this is implemented for =E2=80=98accept=E2=80=99, but not =E2=80=98socket= =E2=80=99 or =E2=80=98socketpair=E2=80=99. Python's PEP 433 contains a good explanation of the issues which can arise from leaked file descriptors: https://peps.python.org/pep-0433/#inherited-file-descriptors-issues Given the risks, I'm convinced that Guile's conservative approach is actually quite sensible. It seems like the best path forward would be to implement platform-specific optimizations where possible. I've attached a draft patch which implements a fast-path on systems where "/proc/self/fd" is available. --000000000000558d8305edf369cc Content-Type: text/x-patch; charset="US-ASCII"; name="close-inherited-with-proc-self-fd.patch" Content-Disposition: attachment; filename="close-inherited-with-proc-self-fd.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_laqa7xar0 Y29tbWl0IDA4OTQzY2FlOTA1NDVkZGRlYTQ0Y2E1NWVhYjY4MDQ3ZTVhZTJmOWQKQXV0aG9yOiBB bmRyZXcgV2hhdHNvbiA8d2hhdHNvbkBnbWFpbC5jb20+CkRhdGU6ICAgTW9uIE5vdiAyMSAxMzo0 MDozMyAyMDIyICsxMDAwCgogICAgUmVkdWNlIHJlZHVuZGFudCBjbG9zZSgpIGNhbGxzIHdoZW4g Zm9ya2luZyBvbiBzb21lIHN5c3RlbXMuCiAgICAKICAgIFNvbWUgc3lzdGVtcyBwcm92aWRlICIv cHJvYy9zZWxmL2ZkIiB3aGljaCBpcyBhIGRpcmVjdG9yeSBjb250YWluaW5nIGFuCiAgICBlbnRy eSBmb3IgZWFjaCBvcGVuIGZpbGUgZGVzY3JpcHRvciBpbiB0aGUgY3VycmVudCBwcm9jZXNzLiAg V2UgdXNlIHRoaXMKICAgIHRvIGxpbWl0IHRoZSBudW1iZXIgb2YgY2xvc2UoKSBjYWxscyBuZWVk ZWQgdG8gZW5zdXJlIGZpbGUgZGVzY3JpcHRvcnMKICAgIGFyZW4ndCBsZWFrZWQgdG8gdGhlIGNo aWxkIHByb2Nlc3Mgd2hlbiBmb3JraW5nLgogICAgCiAgICAqIGxpYmd1aWxlL3Bvc2l4LmMgKGNs b3NlX2luaGVyaXRlZF9mZHNfc2xvdyk6CiAgICAoY2xvc2VfaW5oZXJpdGVkX2Zkcyk6IE5ldyBz dGF0aWMgaGVscGVyIGZ1bmN0aW9ucy4KICAgIChzdGFydF9jaGlsZCk6IEF0dGVtcHQgdG8gY2xv c2UgaW5oZXJpdGVkIGZpbGUgZGVzY3JpcHRvcnMgZWZmaWNpZW50bHkKICAgIHVzaW5nICdjbG9z ZV9pbmhlcml0ZWRfZmRzJywgZmFsbGluZyBiYWNrIHRvIHRoZSBicnV0ZS1mb3JjZSBhcHByb2Fj aCBpbgogICAgJ2Nsb3NlX2luaGVyaXRlZF9mZHNfc2xvdycuCgpkaWZmIC0tZ2l0IGEvbGliZ3Vp bGUvcG9zaXguYyBiL2xpYmd1aWxlL3Bvc2l4LmMKaW5kZXggYjUzNTJjMmM0Li5mYzM1MTIwNTQg MTAwNjQ0Ci0tLSBhL2xpYmd1aWxlL3Bvc2l4LmMKKysrIGIvbGliZ3VpbGUvcG9zaXguYwpAQCAt MjQsNiArMjQsNyBAQAogIyAgaW5jbHVkZSA8Y29uZmlnLmg+CiAjZW5kaWYKIAorI2luY2x1ZGUg PGRpcmVudC5oPgogI2luY2x1ZGUgPGVycm5vLmg+CiAjaW5jbHVkZSA8ZmNudGwuaD4KICNpbmNs dWRlIDxzdGRpby5oPgpAQCAtMTMzNyw2ICsxMzM4LDQ2IEBAIHJlbnVtYmVyX2ZpbGVfZGVzY3Jp cHRvciAoaW50IGZkLCBpbnQgZXJyKQogfQogI2VuZGlmIC8qIEhBVkVfRk9SSyAqLwogCitzdGF0 aWMgdm9pZAorY2xvc2VfaW5oZXJpdGVkX2Zkc19zbG93KGludCBtYXhfZmQsIGludCBpbiwgaW50 IG91dCwgaW50IGVycikKK3sKKyAgd2hpbGUgKG1heF9mZC0tKQorICAgIGlmIChtYXhfZmQgIT0g aW4gJiYgbWF4X2ZkICE9IG91dCAmJiBtYXhfZmQgIT0gZXJyKQorICAgICAgY2xvc2UgKG1heF9m ZCk7Cit9CisKK3N0YXRpYyB2b2lkCitjbG9zZV9pbmhlcml0ZWRfZmRzKGludCBtYXhfZmQsIGlu dCBpbiwgaW50IG91dCwgaW50IGVycikKK3sKKyAgRElSICpkaXJwOworICBzdHJ1Y3QgZGlyZW50 ICpkOworICBpbnQgZmQ7CisKKyAgLyogVHJ5IHRvIHVzZSB0aGUgcGxhdGZvcm0tc3BlY2lmaWMg bGlzdCBvZiBvcGVuIGZpbGUgZGVzY3JpcHRvcnMsIHNvCisgICAgIHdlIGRvbid0IG5lZWQgdG8g dXNlIHRoZSBicnV0ZSBmb3JjZSBhcHByb2FjaC4gKi8KKyAgZGlycCA9IG9wZW5kaXIgKCIvcHJv Yy9zZWxmL2ZkIik7CisKKyAgaWYgKGRpcnAgPT0gTlVMTCkKKyAgICByZXR1cm4gY2xvc2VfaW5o ZXJpdGVkX2Zkc19zbG93IChtYXhfZmQsIGluLCBvdXQsIGVycik7CisKKyAgd2hpbGUgKChkID0g cmVhZGRpciAoZGlycCkpICE9IE5VTEwpCisgICAgeworICAgICAgZmQgPSBhdG9pIChkLT5kX25h bWUpOworCisgICAgICAvKiBTa2lwICIuIiBhbmQgIi4uIiwgYW5kIGFueSBnYXJiYWdlIGVudHJp ZXMuICovCisgICAgICBpZiAoZmQgPD0gMCkKKyAgICAgICAgY29udGludWU7CisKKyAgICAgIC8q IEtlZXAgaW4vb3V0L2VyciBvcGVuLiAqLworICAgICAgaWYgKGZkID09IGluIHx8IGZkID09IG91 dCB8fCBmZCA9PSBlcnIpCisgICAgICAgIGNvbnRpbnVlOworCisgICAgICBjbG9zZSAoZmQpOwor ICAgIH0KKworICBjbG9zZWRpciAoZGlycCk7Cit9CisKICNpZmRlZiBIQVZFX0ZPUksKICNkZWZp bmUgSEFWRV9TVEFSVF9DSElMRCAxCiAvKiBTaW5jZSBHdWlsZSB1c2VzIHRocmVhZHMsIHdlIGhh dmUgdG8gYmUgdmVyeSBjYXJlZnVsIHRvIGF2b2lkIGNhbGxpbmcKQEAgLTEzNzMsOSArMTQxNCw3 IEBAIHN0YXJ0X2NoaWxkIChjb25zdCBjaGFyICpleGVjX2ZpbGUsIGNoYXIgKipleGVjX2FyZ3Ys CiAKICAgLyogQ2xvc2UgYWxsIGZpbGUgZGVzY3JpcHRvcnMgaW4gcG9ydHMgaW5oZXJpdGVkIGZy b20gdGhlIHBhcmVudAogICAgICBleGNlcHQgZm9yIGluLCBvdXQsIGFuZCBlcnIuICBIZWF2eS1o YW5kZWQsIGJ1dCByb2J1c3QuICAqLwotICB3aGlsZSAobWF4X2ZkLS0pCi0gICAgaWYgKG1heF9m ZCAhPSBpbiAmJiBtYXhfZmQgIT0gb3V0ICYmIG1heF9mZCAhPSBlcnIpCi0gICAgICBjbG9zZSAo bWF4X2ZkKTsKKyAgY2xvc2VfaW5oZXJpdGVkX2ZkcyAobWF4X2ZkLCBpbiwgb3V0LCBlcnIpOwog CiAgIC8qIElnbm9yZSBlcnJvcnMgb24gdGhlc2Ugb3BlbigpIGNhbGxzLiAgKi8KICAgaWYgKGlu ID09IC0xKQo= --000000000000558d8305edf369cc-- From debbugs-submit-bounces@debbugs.gnu.org Sat Nov 26 09:39:21 2022 Received: (at 59321) by debbugs.gnu.org; 26 Nov 2022 14:39:21 +0000 Received: from localhost ([127.0.0.1]:38334 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oywKu-0005Tt-Mx for submit@debbugs.gnu.org; Sat, 26 Nov 2022 09:39:21 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oywKp-0005Tc-R3 for 59321@debbugs.gnu.org; Sat, 26 Nov 2022 09:39:19 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oywKj-0005Lb-W5; Sat, 26 Nov 2022 09:39:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=2peMiC7JxeskFb9ZIWapZ0SaZu2cvaYfa79Jq78FVRU=; b=mmjfWoo1hecu3t5HEdmY FzBifjdN5k93DwV7zVPxlMcFHtJ+gioNZiarSz9JedHdkl/8DtjKm03BmYIHEeIEVMfVN4pV26K8L xuDTaBJi4eAUf10DcA/BeeHuPURhEBZxCdh6goW/O4/iR5EGzwMPp1ISrkgQQq0ucqoef46Fm7uyv mnF8+Knx0NYFPhMeiGZMK5Jwr8EACxQfX3D52jutAvAych6OStrsT8xYlTW97xcdIjoiGtGC2+N64 qTZhU/CmS0O7EzYQkf9ZmvQbpoICZGyR2QKDy31cTsRjc8xVfyasJSTpzRMbFt7ai8JO8iSlFogwo 1cNnNvSyeunm6w==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201] helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oywKj-0002c5-Go; Sat, 26 Nov 2022 09:39:09 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Andrew Whatson Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> <87r0xx5yja.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Sextidi 6 Frimaire an 231 de la =?utf-8?Q?R=C3=A9vol?= =?utf-8?Q?ution=2C?= jour de la =?utf-8?Q?M=C3=A2che?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 26 Nov 2022 15:39:06 +0100 In-Reply-To: (Andrew Whatson's message of "Mon, 21 Nov 2022 14:22:52 +1000") Message-ID: <871qppaigl.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 59321 Cc: hylophile@posteo.de, 59321@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi Andrew, Andrew Whatson skribis: > Ludovic Court=C3=A8s wrote: [...] >> Libguile opens all its own file descriptors at O_CLOEXEC (one omission >> was recently fixed in 0aa1a9976fc3c6af4d1087e59d728cb8fe7d369a) so it >> may be possible to remove that FD-closing loop. There=E2=80=99s still t= he >> possibility that application bug unwillingly leaks FDs, but we could >> consider it=E2=80=99s none of our business. >> >> Thoughts? > > I agree with this approach in principle, but from what Tomas is saying > it seems like it's not currently possible for applications to do the > right thing in all cases. OK. [...] > We also need equivalent functionality around SOCK_CLOEXEC. It seems > this is implemented for =E2=80=98accept=E2=80=99, but not =E2=80=98socket= =E2=80=99 or =E2=80=98socketpair=E2=80=99. It=E2=80=99s possible to use SOCK_CLOEXEC with =E2=80=98socket=E2=80=99 and= =E2=80=98socketpair=E2=80=99 already, as in: (socket AF_INET (logior SOCK_CLOEXEC SOCK_STREAM) 0) With commit 1d313bf5f0d296d766bd3a0e6d030df37c71711b, =E2=80=98pipe=E2=80= =99 is also covered. So I think we have pretty much everything we need, at least starting with 3.0.9. > Python's PEP 433 contains a good explanation of the issues which can > arise from leaked file descriptors: > https://peps.python.org/pep-0433/#inherited-file-descriptors-issues > > Given the risks, I'm convinced that Guile's conservative approach is > actually quite sensible. It seems like the best path forward would be > to implement platform-specific optimizations where possible. > > I've attached a draft patch which implements a fast-path on systems > where "/proc/self/fd" is available. The patch LGTM; it=E2=80=99s certainly an improvement on systems configured= with a high per-process FD limit. Now, I believe use of =E2=80=98posix_spawn=E2=80=99 as proposed in makes that unnecessary. Let=E2=80=99s = take a closer look at that other patch and so we can see=E2=80=A6 Thanks, Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 08 07:02:24 2022 Received: (at 59321) by debbugs.gnu.org; 8 Dec 2022 12:02:24 +0000 Received: from localhost ([127.0.0.1]:56375 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1p3Fbc-0008Le-0j for submit@debbugs.gnu.org; Thu, 08 Dec 2022 07:02:24 -0500 Received: from mail-yb1-f170.google.com ([209.85.219.170]:43563) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1p3FbZ-0008LW-BL for 59321@debbugs.gnu.org; Thu, 08 Dec 2022 07:02:22 -0500 Received: by mail-yb1-f170.google.com with SMTP id d128so1343083ybf.10 for <59321@debbugs.gnu.org>; Thu, 08 Dec 2022 04:02:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=QTBvVFb9XJDAEYE/LNp4ZbMEgNV0h1utU7vA9UQViTU=; b=VEQDF7302cRKMU+DrpcKXEAbck+xYtb8tkDxQFErv/qX+8QegRDCduIVo418F2323P h/BGkLk6ytmay8QTXCLtK6GQRO4DbQSBecQ0gDykw0hXrtwFaMunC6Ubp7Qq9R7VVARH rNDkCUcAeoh29ccU3gFAFiKZtHAujYznf75CmSX7F7gScah6YZikd863wF7WovDQU/ru MwM0rdIyA9AD1gmnJ8xvzTxpdiKWBhGgpTUKIZUGO3nr8fPI+F7CD63j5CUMPXDNWih4 tOboo9ZOdcmARhXEekzoag1cshvwtGbexQWDGGTEVMBaPgOhIoQEYDlW86nQae/ENuj2 mcLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QTBvVFb9XJDAEYE/LNp4ZbMEgNV0h1utU7vA9UQViTU=; b=SpA9cN2OJ2fBxnH89dVR+ZWPa2z3CL4orfo9ZRw9xP0liHiPGgF8OJlFYk0OZAWrG8 8J6jFKtGFQjsdaNxoJeU6yANCHUPa2qoT46RI1f7Cvs2Tn6cA8p7rvr3tMrNm8yTs2h+ TjJStCwO5D3x3sIBk6aRg+Vm8No/QB5pM8PmJ5K/O9GoERAKbOCQAxQRm/kTS2+cW5/W HizUUM9Iz73oseRRNmwkJrd9b914Q+Nu5ICdvtk0mmQuVf6IvkFYBaBSxMYYtniwVkYm GJI3iBzK5fH4bCmn0JWJRh4qUoaaHIS8HMVaaGB83ABwi+WnS1YKhIYa9adchaz94hRb YQ7w== X-Gm-Message-State: ANoB5pk/OHPZLPF3g+zdDIQFPYiYszusKwTIXRaXXJiI3JwnER0WCedZ oVhRTUt4dA0xeR+AAYrDlG5vtGw5PRQU4rOmplQ= X-Google-Smtp-Source: AA0mqf7ej2mCLK/YJ8o5VHE18xK1f+loofrn7cmpX8jGDl/7mC5RZA3+l4c91Ws+/iPuBR3NZGblXrRwbrm1ALL29dk= X-Received: by 2002:a25:880e:0:b0:6e6:e31e:3dc5 with SMTP id c14-20020a25880e000000b006e6e31e3dc5mr82802095ybl.534.1670500935546; Thu, 08 Dec 2022 04:02:15 -0800 (PST) MIME-Version: 1.0 References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> <87r0xx5yja.fsf@gnu.org> <871qppaigl.fsf@gnu.org> In-Reply-To: <871qppaigl.fsf@gnu.org> From: Andrew Whatson Date: Thu, 8 Dec 2022 22:02:04 +1000 Message-ID: Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Content-Type: multipart/mixed; boundary="000000000000dcd55e05ef4fce9c" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 59321 Cc: hylophile@posteo.de, 59321@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --000000000000dcd55e05ef4fce9c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s wrote: > > Andrew Whatson skribis: > > > We also need equivalent functionality around SOCK_CLOEXEC. It seems > > this is implemented for =E2=80=98accept=E2=80=99, but not =E2=80=98sock= et=E2=80=99 or =E2=80=98socketpair=E2=80=99. > > It=E2=80=99s possible to use SOCK_CLOEXEC with =E2=80=98socket=E2=80=99 a= nd =E2=80=98socketpair=E2=80=99 > already, as in: > > (socket AF_INET (logior SOCK_CLOEXEC SOCK_STREAM) 0) > > With commit 1d313bf5f0d296d766bd3a0e6d030df37c71711b, =E2=80=98pipe=E2=80= =99 is also > covered. > > So I think we have pretty much everything we need, at least starting > with 3.0.9. Ah, nice! In that case it might be possible to deprecate this auto-closing behaviour in a future version. > > Python's PEP 433 contains a good explanation of the issues which can > > arise from leaked file descriptors: > > https://peps.python.org/pep-0433/#inherited-file-descriptors-issues > > > > Given the risks, I'm convinced that Guile's conservative approach is > > actually quite sensible. It seems like the best path forward would be > > to implement platform-specific optimizations where possible. > > > > I've attached a draft patch which implements a fast-path on systems > > where "/proc/self/fd" is available. > > The patch LGTM; it=E2=80=99s certainly an improvement on systems configur= ed with > a high per-process FD limit. > > Now, I believe use of =E2=80=98posix_spawn=E2=80=99 as proposed in > makes that unnecessary. Let=E2=80=99= s take > a closer look at that other patch and so we can see=E2=80=A6 Playing with the wip-posix-spawn branch, it has the same slowdown (actually a bit worse). I've updated the "/proc/self/fd" fast-path patch for posix_spawn, please find attached. > Thanks, > Ludo=E2=80=99. Thank you! --000000000000dcd55e05ef4fce9c Content-Type: text/x-patch; charset="US-ASCII"; name="close-inherited-with-proc-self-fd-psawn.patch" Content-Disposition: attachment; filename="close-inherited-with-proc-self-fd-psawn.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_lbf12l8l0 Y29tbWl0IGMwMTJkN2IwZDUyNDhhOTlhM2E5Mjc4MDY4N2E2NzZjNWQ0MjBmNWYKQXV0aG9yOiBB bmRyZXcgV2hhdHNvbiA8d2hhdHNvbkBnbWFpbC5jb20+CkRhdGU6ICAgVGh1IERlYyA4IDIxOjQz OjI4IDIwMjIgKzEwMDAKCiAgICBSZWR1Y2UgcmVkdW5kYW50IGNsb3NlKCkgY2FsbHMgd2hlbiBm b3JraW5nIG9uIHNvbWUgc3lzdGVtcy4KICAgIAogICAgU29tZSBzeXN0ZW1zIHByb3ZpZGUgIi9w cm9jL3NlbGYvZmQiIHdoaWNoIGlzIGEgZGlyZWN0b3J5IGNvbnRhaW5pbmcgYW4KICAgIGVudHJ5 IGZvciBlYWNoIG9wZW4gZmlsZSBkZXNjcmlwdG9yIGluIHRoZSBjdXJyZW50IHByb2Nlc3MuICBX ZSB1c2UgdGhpcwogICAgdG8gbGltaXQgdGhlIG51bWJlciBvZiBjbG9zZSgpIGNhbGxzIG5lZWRl ZCB0byBlbnN1cmUgZmlsZSBkZXNjcmlwdG9ycwogICAgYXJlbid0IGxlYWtlZCB0byB0aGUgY2hp bGQgcHJvY2VzcyB3aGVuIGZvcmtpbmcuCiAgICAKICAgICogbGliZ3VpbGUvcG9zaXguYyAoY2xv c2VfaW5oZXJpdGVkX2Zkc19zbG93KToKICAgIChjbG9zZV9pbmhlcml0ZWRfZmRzKTogTmV3IHN0 YXRpYyBoZWxwZXIgZnVuY3Rpb25zLgogICAgKHNjbV9zcGF3bl9wcm9jZXNzKTogQXR0ZW1wdCB0 byBjbG9zZSBpbmhlcml0ZWQgZmlsZSBkZXNjcmlwdG9ycwogICAgZWZmaWNpZW50bHkgdXNpbmcg J2Nsb3NlX2luaGVyaXRlZF9mZHMnLCBmYWxsaW5nIGJhY2sgdG8gdGhlIGJydXRlLWZvcmNlCiAg ICBhcHByb2FjaCBpbiAnY2xvc2VfaW5oZXJpdGVkX2Zkc19zbG93Jy4KCmRpZmYgLS1naXQgYS9s aWJndWlsZS9wb3NpeC5jIGIvbGliZ3VpbGUvcG9zaXguYwppbmRleCA4N2IzMjlkYTkuLjhjOTAy MjExNiAxMDA2NDQKLS0tIGEvbGliZ3VpbGUvcG9zaXguYworKysgYi9saWJndWlsZS9wb3NpeC5j CkBAIC0yNCw2ICsyNCw3IEBACiAjICBpbmNsdWRlIDxjb25maWcuaD4KICNlbmRpZgogCisjaW5j bHVkZSA8ZGlyZW50Lmg+CiAjaW5jbHVkZSA8ZXJybm8uaD4KICNpbmNsdWRlIDxmY250bC5oPgog I2luY2x1ZGUgPHN0ZGlvLmg+CkBAIC0xMzA5LDYgKzEzMTAsNDEgQEAgU0NNX0RFRklORSAoc2Nt X2ZvcmssICJwcmltaXRpdmUtZm9yayIsIDAsIDAsIDAsCiAjdW5kZWYgRlVOQ19OQU1FCiAjZW5k aWYgLyogSEFWRV9GT1JLICovCiAKK3N0YXRpYyB2b2lkCitjbG9zZV9pbmhlcml0ZWRfZmRzX3Ns b3cgKHBvc2l4X3NwYXduX2ZpbGVfYWN0aW9uc190ICphY3Rpb25zLCBpbnQgbWF4X2ZkKQorewor ICB3aGlsZSAoLS1tYXhfZmQgPiAyKQorICAgIHBvc2l4X3NwYXduX2ZpbGVfYWN0aW9uc19hZGRj bG9zZSAoYWN0aW9ucywgbWF4X2ZkKTsKK30KKworc3RhdGljIHZvaWQKK2Nsb3NlX2luaGVyaXRl ZF9mZHMgKHBvc2l4X3NwYXduX2ZpbGVfYWN0aW9uc190ICphY3Rpb25zLCBpbnQgbWF4X2ZkKQor eworICBESVIgKmRpcnA7CisgIHN0cnVjdCBkaXJlbnQgKmQ7CisgIGludCBmZDsKKworICAvKiBU cnkgdG8gdXNlIHRoZSBwbGF0Zm9ybS1zcGVjaWZpYyBsaXN0IG9mIG9wZW4gZmlsZSBkZXNjcmlw dG9ycywgc28KKyAgICAgd2UgZG9uJ3QgbmVlZCB0byB1c2UgdGhlIGJydXRlIGZvcmNlIGFwcHJv YWNoLiAqLworICBkaXJwID0gb3BlbmRpciAoIi9wcm9jL3NlbGYvZmQiKTsKKworICBpZiAoZGly cCA9PSBOVUxMKQorICAgIHJldHVybiBjbG9zZV9pbmhlcml0ZWRfZmRzX3Nsb3cgKGFjdGlvbnMs IG1heF9mZCk7CisKKyAgd2hpbGUgKChkID0gcmVhZGRpciAoZGlycCkpICE9IE5VTEwpCisgICAg eworICAgICAgZmQgPSBhdG9pIChkLT5kX25hbWUpOworCisgICAgICAvKiBTa2lwICIuIiBhbmQg Ii4uIiwgZ2FyYmFnZSBlbnRyaWVzLCBzdGRpbi9zdGRvdXQvc3RkZXJyLiAqLworICAgICAgaWYg KGZkIDw9IDIpCisgICAgICAgIGNvbnRpbnVlOworCisgICAgICBwb3NpeF9zcGF3bl9maWxlX2Fj dGlvbnNfYWRkY2xvc2UgKGFjdGlvbnMsIGZkKTsKKyAgICB9CisKKyAgY2xvc2VkaXIgKGRpcnAp OworfQorCiBzdGF0aWMgU0NNCiBzY21fc3Bhd25fcHJvY2VzcyAoU0NNIHByb2csIFNDTSBhcmdz LCBTQ00gc2NtX2luLCBTQ00gc2NtX291dCwgU0NNIHNjbV9lcnIpCiAjZGVmaW5lIEZVTkNfTkFN RSAic3Bhd24qIgpAQCAtMTM2Myw4ICsxMzk5LDcgQEAgc2NtX3NwYXduX3Byb2Nlc3MgKFNDTSBw cm9nLCBTQ00gYXJncywgU0NNIHNjbV9pbiwgU0NNIHNjbV9vdXQsIFNDTSBzY21fZXJyKQogICBw b3NpeF9zcGF3bl9maWxlX2FjdGlvbnNfYWRkZHVwMiAoJmFjdGlvbnMsIGZkX3Nsb3RbMV0sIDEp OwogICBwb3NpeF9zcGF3bl9maWxlX2FjdGlvbnNfYWRkZHVwMiAoJmFjdGlvbnMsIGZkX3Nsb3Rb Ml0sIDIpOwogCi0gIHdoaWxlICgtLW1heF9mZCA+IDIpCi0gICAgcG9zaXhfc3Bhd25fZmlsZV9h Y3Rpb25zX2FkZGNsb3NlICgmYWN0aW9ucywgbWF4X2ZkKTsKKyAgY2xvc2VfaW5oZXJpdGVkX2Zk cyAoJmFjdGlvbnMsIG1heF9mZCk7CiAKICAgaWYgKHBvc2l4X3NwYXducCAoJnBpZCwgZXhlY19m aWxlLCAmYWN0aW9ucywgYXR0cnAsIGV4ZWNfYXJndiwgZW52aXJvbikgIT0gMCkKICAgICB7Cg== --000000000000dcd55e05ef4fce9c-- From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 08 09:35:10 2022 Received: (at 59321) by debbugs.gnu.org; 8 Dec 2022 14:35:10 +0000 Received: from localhost ([127.0.0.1]:57144 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1p3HzS-0003dX-9P for submit@debbugs.gnu.org; Thu, 08 Dec 2022 09:35:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48878) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1p3HzQ-0003dO-N8 for 59321@debbugs.gnu.org; Thu, 08 Dec 2022 09:35:09 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p3HzD-0007mH-H4; Thu, 08 Dec 2022 09:35:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=fvkzKBUlKD4H/7gmixS92x0NAkq6EEYvfX5L7asCl4w=; b=PYfbO/s3JVAfLk+zAJen uRNyqPWJ1Q2G+Q4z6QRv479l/wTWTdrQzwq9KV7paZJE3wCIHwOowDBkbuWbxbMfeBqz9P5HgC8d5 Sb30ezHvfljWCnujA5Z+wRHvzyjLDK5GayuEUOWn4zv4Brc+G6+QybV86PwnzK5/bqPtAbuZDQ0mf 0Du9KNhA3VsRyfW5m1gZML0mTcJiPI5zSdLkrd7ywoGvP8qRhdo04WcXX3nSpFylmJxOL+yNCrM0G csrWuKQ50tEeZWZE005GXiiR5wTFnh9LL7sXwUhkORYc189yzgegPppDL5yZ4GQ1RIYLQCzFyxWjR Aqov5CWqa8lq8w==; Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p3HzC-0003oV-Kx; Thu, 08 Dec 2022 09:34:55 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Andrew Whatson Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> <87r0xx5yja.fsf@gnu.org> <871qppaigl.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Octidi 18 Frimaire an 231 de la =?utf-8?Q?R=C3=A9vol?= =?utf-8?Q?ution=2C?= jour du Lierre X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 08 Dec 2022 15:34:51 +0100 In-Reply-To: (Andrew Whatson's message of "Thu, 8 Dec 2022 22:02:04 +1000") Message-ID: <87359qdkvo.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 59321 Cc: hylophile@posteo.de, 59321@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi, Andrew Whatson skribis: > Playing with the wip-posix-spawn branch, it has the same slowdown > (actually a bit worse). I've updated the "/proc/self/fd" fast-path > patch for posix_spawn, please find attached. Ooh that=E2=80=99s perfect, thank you. I=E2=80=99ll apply it once we=E2=80=99ve sorted out remaining issues on the= branch. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 13 10:37:35 2023 Received: (at 59321-done) by debbugs.gnu.org; 13 Jan 2023 15:37:35 +0000 Received: from localhost ([127.0.0.1]:52161 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pGM7a-0005iv-NR for submit@debbugs.gnu.org; Fri, 13 Jan 2023 10:37:34 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33112) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pGM7Z-0005ii-AG for 59321-done@debbugs.gnu.org; Fri, 13 Jan 2023 10:37:33 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pGM7T-00040b-Ss; Fri, 13 Jan 2023 10:37:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=WsR4aSD8kctQRNvR1AywZenlKqADo8VJNOtFqqSxIfI=; b=YKhaAjcscudI/pZue86w hZjfSVs57jb3YQXHds1HVOTkuMxspL/FKqHZLebFIgPjSmXE5dvjFOFoHfuxIeWJKLLGaeQEEGLG+ 1PESExZQlDmjLSJSZt2AMXX9B84yIE407fBhECN0UXGUET96G7u2Aa30FxX6asf1uNddPSVQSc4Jh J8y5azQy9n4YCeAAFiqVnQtijDT8A7rfw2dFZOu0Gx7PemB4c9+CwdYIklIxC62bziTrFpeM576X4 qM6pSNr9sQxB/HLichkv4sKRl+0gVTAjSwM9QgKjOtc+ZyUsYY7CZCsyHt/RN75bDwUtLM/0yFn/y NafLGDlUrP6Flg==; Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pGM7T-00030J-Fw; Fri, 13 Jan 2023 10:37:27 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Andrew Whatson Subject: Re: bug#59321: ice-9's open-input-pipe is unexpectedly slow on some systems References: <8d55cf7d1e5382c874cfcaee1f4cddd3@posteo.de> <87r0xx5yja.fsf@gnu.org> <871qppaigl.fsf@gnu.org> Date: Fri, 13 Jan 2023 16:37:24 +0100 In-Reply-To: (Andrew Whatson's message of "Thu, 8 Dec 2022 22:02:04 +1000") Message-ID: <87fscebg3v.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 59321-done Cc: hylophile@posteo.de, 59321-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello! Andrew Whatson skribis: > commit c012d7b0d5248a99a3a92780687a676c5d420f5f > Author: Andrew Whatson > Date: Thu Dec 8 21:43:28 2022 +1000 > > Reduce redundant close() calls when forking on some systems. >=20=20=20=20=20 > Some systems provide "/proc/self/fd" which is a directory containing = an > entry for each open file descriptor in the current process. We use t= his > to limit the number of close() calls needed to ensure file descriptors > aren't leaked to the child process when forking. >=20=20=20=20=20 > * libguile/posix.c (close_inherited_fds_slow): > (close_inherited_fds): New static helper functions. > (scm_spawn_process): Attempt to close inherited file descriptors > efficiently using 'close_inherited_fds', falling back to the brute-fo= rce > approach in 'close_inherited_fds_slow'. Finally applied on top of the =E2=80=98posix_spawn=E2=80=99 series as commit 9332b632407894c2e1951cce1bc678f19e1fa8e4. Thanks! Ludo=E2=80=99. From unknown Sat Aug 16 16:22:44 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 11 Feb 2023 12:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator