From unknown Tue Jun 24 20:52:04 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#13188 <13188@debbugs.gnu.org> To: bug#13188 <13188@debbugs.gnu.org> Subject: Status: par-map causes VM stack overflow Reply-To: bug#13188 <13188@debbugs.gnu.org> Date: Wed, 25 Jun 2025 03:52:04 +0000 retitle 13188 par-map causes VM stack overflow reassign 13188 guile submitter 13188 Nala Ginrut severity 13188 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 15 03:13:52 2012 Received: (at submit) by debbugs.gnu.org; 15 Dec 2012 08:13:52 +0000 Received: from localhost ([127.0.0.1]:43299 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TjmsZ-000505-GI for submit@debbugs.gnu.org; Sat, 15 Dec 2012 03:13:52 -0500 Received: from eggs.gnu.org ([208.118.235.92]:60873) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1TjmsY-0004zy-1U for submit@debbugs.gnu.org; Sat, 15 Dec 2012 03:13:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TjmrW-0005iO-MZ for submit@debbugs.gnu.org; Sat, 15 Dec 2012 03:12:47 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-102.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, T_DKIM_INVALID, USER_IN_WHITELIST autolearn=unavailable version=3.3.2 Received: from lists.gnu.org ([208.118.235.17]:45549) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TjmrW-0005iK-IJ for submit@debbugs.gnu.org; Sat, 15 Dec 2012 03:12:46 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44001) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TjmrV-0003LJ-Qj for bug-guile@gnu.org; Sat, 15 Dec 2012 03:12:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TjmrU-0005hH-Sx for bug-guile@gnu.org; Sat, 15 Dec 2012 03:12:45 -0500 Received: from mail-pa0-f41.google.com ([209.85.220.41]:36495) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TjmrU-0005h8-Jg for bug-guile@gnu.org; Sat, 15 Dec 2012 03:12:44 -0500 Received: by mail-pa0-f41.google.com with SMTP id bj3so2703191pad.0 for ; Sat, 15 Dec 2012 00:12:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:date:organization:content-type:x-mailer :mime-version:content-transfer-encoding; bh=CRlahw4OaYzA/hIt11x0rqxKROREy1vpzaRBnLvDvXU=; b=kcehIGELtVs9iBGSiVskSr8xx7wBiMOqwOYbGp/MxooAWgDfnWtS936xesMLwX8dL6 IVjhMEuUolIGMsMddVpe3uHeAH+FLWA2CpipvMgGLypSJBTCJ2rJMZT6bcOziMeyTtl6 IEJj8e/vahW2dLMtLAH/O1CpFmpFePsbXjNYtvcbHr+1VpNOzR/84/JhG+hd+x3nFXr/ n2JDw8EYpaxEwLuDogXIg15Ll63Y7QJNP5YKiJLVVIe7q3JxwggUb7TvfwuHtsNLWiIv C+cxXILNND9GAchiotK+ve72GztP5FHo6A7RYyWS5rRfl03D50Nl9l+cpHvy+7KORb42 szGw== Received: by 10.66.74.98 with SMTP id s2mr22686843pav.64.1355559162055; Sat, 15 Dec 2012 00:12:42 -0800 (PST) Received: from [192.168.100.100] ([59.40.201.219]) by mx.google.com with ESMTPS id o5sm4673557pay.5.2012.12.15.00.12.38 (version=SSLv3 cipher=OTHER); Sat, 15 Dec 2012 00:12:40 -0800 (PST) Message-ID: <1355559152.27310.5.camel@Renee-desktop.suse> Subject: par-map causes VM stack overflow From: Nala Ginrut To: bug-guile@gnu.org Date: Sat, 15 Dec 2012 16:12:32 +0800 Organization: HFG Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.4.4 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 208.118.235.17 X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.2 (----) Here is the error message. Anyway, par-map shouldn't cause stack overflow. -----------------------err msg------------------------------ scheme@(guile-user)> (par-map 1+ (iota 10000)) While executing meta-command: ERROR: Throw to key `vm-error' with args `(vm-run "VM: Stack overflow" ())'. -------------------------end-------------------------------- From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 27 13:14:47 2013 Received: (at 13188-done) by debbugs.gnu.org; 27 Mar 2013 17:14:47 +0000 Received: from localhost ([127.0.0.1]:47927 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UKtvy-00044Q-Ov for submit@debbugs.gnu.org; Wed, 27 Mar 2013 13:14:47 -0400 Received: from xanadu.aquilenet.fr ([88.191.123.111]:54244) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UKtvw-00044D-9w for 13188-done@debbugs.gnu.org; Wed, 27 Mar 2013 13:14:45 -0400 Received: from localhost (localhost [127.0.0.1]) by xanadu.aquilenet.fr (Postfix) with ESMTP id 5DD30BBBF; Wed, 27 Mar 2013 18:12:17 +0100 (CET) Received: from xanadu.aquilenet.fr ([127.0.0.1]) by localhost (xanadu.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IW0hoeXJ5ij7; Wed, 27 Mar 2013 18:12:17 +0100 (CET) Received: from pluto (reverse-83.fdn.fr [80.67.176.83]) by xanadu.aquilenet.fr (Postfix) with ESMTPSA id ECBD7EB9; Wed, 27 Mar 2013 18:12:16 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Nala Ginrut Subject: Re: bug#13188: par-map causes VM stack overflow References: <1355559152.27310.5.camel@Renee-desktop.suse> Date: Wed, 27 Mar 2013 18:12:16 +0100 In-Reply-To: <1355559152.27310.5.camel@Renee-desktop.suse> (Nala Ginrut's message of "Sat, 15 Dec 2012 16:12:32 +0800") Message-ID: <87y5d8rclr.fsf@gnu.org> User-Agent: Gnus/5.130005 (Ma Gnus v0.5) Emacs/24.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, Nala Ginrut skribis: > scheme@(guile-user)> (par-map 1+ (iota 10000)) > While executing meta-command: > ERROR: Throw to key `vm-error' with args `(vm-run "VM: Stack > overflow" ())'. [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.7 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.5000] X-Debbugs-Envelope-To: 13188-done Cc: 13188-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, Nala Ginrut skribis: > scheme@(guile-user)> (par-map 1+ (iota 10000)) > While executing meta-command: > ERROR: Throw to key `vm-error' with args `(vm-run "VM: Stack > overflow" ())'. [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.7 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.4625] Hi, Nala Ginrut skribis: > scheme@(guile-user)> (par-map 1+ (iota 10000)) > While executing meta-command: > ERROR: Throw to key `vm-error' with args `(vm-run "VM: Stack > overflow" ())'. Commit 8a177d3 fixes this. I added a paragraph in the documentation that explains what happens: delimited continuations to the rescue once again! ;-) Comments welcome. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Wed Mar 27 22:58:10 2013 Received: (at 13188-done) by debbugs.gnu.org; 28 Mar 2013 02:58:10 +0000 Received: from localhost ([127.0.0.1]:48618 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UL32X-0005gd-9K for submit@debbugs.gnu.org; Wed, 27 Mar 2013 22:58:09 -0400 Received: from mail-da0-f49.google.com ([209.85.210.49]:43344) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UL32V-0005gT-8c for 13188-done@debbugs.gnu.org; Wed, 27 Mar 2013 22:58:08 -0400 Received: by mail-da0-f49.google.com with SMTP id t11so4271997daj.8 for <13188-done@debbugs.gnu.org>; Wed, 27 Mar 2013 19:55:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:subject:from:to:cc:date:in-reply-to :references:organization:content-type:x-mailer:mime-version :content-transfer-encoding; bh=/lysIWY6rOENsivLwZP/28gKKwNvOxpWt9visQk++y0=; b=Q7bfowKPsRvn4+grzcCx1iLcyT5yqba/IHH/rUmJdKNShto2OPbty+wWk0Btqxb0VI W7cc8c74UjpjcQa0GFAtmNni3mKYTNdQjUQgvKNRqaYIZoUo9mqOTmNArfCMF9fG2k1d M3ezeV930yDnT0VXbE7d0u1JKJna2tcATE1vI+TDUu8dUD0xunthNfJtJbsLKrmsUyFX 2fNxiP25qIaYFYxWTdITAof9yljTQgpG7NsgdN0fGTalnXH7ERk1nDz/LARqyLlymCU4 zbgI1LVLypKAzj/OORBNIs0XdsfM4SNh/TANRgmPrzJspdphLo+dT8RBJh/EO5LQiqA6 xnOA== X-Received: by 10.66.72.37 with SMTP id a5mr33169205pav.193.1364439338367; Wed, 27 Mar 2013 19:55:38 -0700 (PDT) Received: from [147.2.147.112] ([61.14.130.226]) by mx.google.com with ESMTPS id hs8sm23615696pbc.27.2013.03.27.19.55.36 (version=SSLv3 cipher=RC4-SHA bits=128/128); Wed, 27 Mar 2013 19:55:37 -0700 (PDT) Message-ID: <1364439334.2730.41.camel@Renee-desktop.suse> Subject: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow) From: Nala Ginrut To: Ludovic =?ISO-8859-1?Q?Court=E8s?= , guile-devel@gnu.org Date: Thu, 28 Mar 2013 10:55:34 +0800 In-Reply-To: <87y5d8rclr.fsf@gnu.org> References: <1355559152.27310.5.camel@Renee-desktop.suse> <87y5d8rclr.fsf@gnu.org> Organization: HFG Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.4.4 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.1 (/) X-Debbugs-Envelope-To: 13188-done Cc: 13188-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) On Wed, 2013-03-27 at 18:12 +0100, Ludovic Courtès wrote: > Hi, > > Nala Ginrut skribis: > > > scheme@(guile-user)> (par-map 1+ (iota 10000)) > > While executing meta-command: > > ERROR: Throw to key `vm-error' with args `(vm-run "VM: Stack > > overflow" ())'. > > Commit 8a177d3 fixes this. I added a paragraph in the documentation > that explains what happens: delimited continuations to the rescue once > again! ;-) > > Comments welcome. > oh~I love delimited continuations! But I'm still puzzled with the performance of par-map: --------------------cut------------------- scheme@(guile-user)> ,time (define a (map (lambda (x) (expt x 5)) (iota 10000))) ;; 0.008019s real time, 0.007979s run time. 0.000000s spent in GC. scheme@(guile-user)> ,time (define a (par-map (lambda (x) (expt x 5)) (iota 10000))) ;; 6.596471s real time, 6.579375s run time. 1.513880s spent in GC. --------------------end------------------- So my question is, what's the proper scenario to use par-map? > Ludo’. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 28 01:08:18 2013 Received: (at 13188-done) by debbugs.gnu.org; 28 Mar 2013 05:08:18 +0000 Received: from localhost ([127.0.0.1]:48726 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UL54T-0000X2-UA for submit@debbugs.gnu.org; Thu, 28 Mar 2013 01:08:18 -0400 Received: from world.peace.net ([96.39.62.75]:50219) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UL54R-0000Wt-Aj for 13188-done@debbugs.gnu.org; Thu, 28 Mar 2013 01:08:16 -0400 Received: from 209-6-91-212.c3-0.smr-ubr1.sbo-smr.ma.cable.rcn.com ([209.6.91.212] helo=tines.lan) by world.peace.net with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1UL51w-0005Nt-NE; Thu, 28 Mar 2013 01:05:40 -0400 From: Mark H Weaver To: Nala Ginrut Subject: Re: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow) References: <1355559152.27310.5.camel@Renee-desktop.suse> <87y5d8rclr.fsf@gnu.org> <1364439334.2730.41.camel@Renee-desktop.suse> Date: Thu, 28 Mar 2013 01:05:32 -0400 In-Reply-To: <1364439334.2730.41.camel@Renee-desktop.suse> (Nala Ginrut's message of "Thu, 28 Mar 2013 10:55:34 +0800") Message-ID: <874nfwazc3.fsf@tines.lan> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 13188-done Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , 13188-done@debbugs.gnu.org, guile-devel@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) Nala Ginrut writes: > But I'm still puzzled with the performance of par-map: > --------------------cut------------------- > scheme@(guile-user)> ,time (define a (map (lambda (x) (expt x 5)) (iota > 10000))) > ;; 0.008019s real time, 0.007979s run time. 0.000000s spent in GC. > scheme@(guile-user)> ,time (define a (par-map (lambda (x) (expt x 5)) > (iota 10000))) > ;; 6.596471s real time, 6.579375s run time. 1.513880s spent in GC. > --------------------end------------------- > > So my question is, what's the proper scenario to use par-map? It only makes sense to use 'par-map' when the procedure is fairly expensive to compute. There is inevitably a lot of overhead in creating and joining the threads. Granted, we should be able to do much better than we're doing now, but it would *never* make sense to use 'par-map' when each computation is as simple as (expt x 5). Regards, Mark From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 28 09:46:40 2013 Received: (at 13188-done) by debbugs.gnu.org; 28 Mar 2013 13:46:40 +0000 Received: from localhost ([127.0.0.1]:49260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULDA7-00066y-4h for submit@debbugs.gnu.org; Thu, 28 Mar 2013 09:46:39 -0400 Received: from xanadu.aquilenet.fr ([88.191.123.111]:45171) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULDA3-00066o-9q for 13188-done@debbugs.gnu.org; Thu, 28 Mar 2013 09:46:36 -0400 Received: from localhost (localhost [127.0.0.1]) by xanadu.aquilenet.fr (Postfix) with ESMTP id BDE298D01; Thu, 28 Mar 2013 14:44:03 +0100 (CET) Received: from xanadu.aquilenet.fr ([127.0.0.1]) by localhost (xanadu.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6NaxBGGPvb1p; Thu, 28 Mar 2013 14:44:03 +0100 (CET) Received: from pluto (unknown [193.50.110.140]) by xanadu.aquilenet.fr (Postfix) with ESMTPSA id 7140B6BBE; Thu, 28 Mar 2013 14:44:03 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Mark H Weaver Subject: Re: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow) References: <1355559152.27310.5.camel@Renee-desktop.suse> <87y5d8rclr.fsf@gnu.org> <1364439334.2730.41.camel@Renee-desktop.suse> <874nfwazc3.fsf@tines.lan> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 8 Germinal an 221 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0xEA52ECF4 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 83C4 F8E5 10A3 3B4C 5BEA D15D 77DD 95E2 EA52 ECF4 X-OS: x86_64-unknown-linux-gnu Date: Thu, 28 Mar 2013 14:44:03 +0100 In-Reply-To: <874nfwazc3.fsf@tines.lan> (Mark H. Weaver's message of "Thu, 28 Mar 2013 01:05:32 -0400") Message-ID: <87r4izprks.fsf@gnu.org> User-Agent: Gnus/5.130005 (Ma Gnus v0.5) Emacs/24.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.7 (/) X-Debbugs-Envelope-To: 13188-done Cc: 13188-done@debbugs.gnu.org, guile-devel@gnu.org, Nala Ginrut X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.2 (-) Mark H Weaver skribis: > Nala Ginrut writes: > >> But I'm still puzzled with the performance of par-map: >> --------------------cut------------------- >> scheme@(guile-user)> ,time (define a (map (lambda (x) (expt x 5)) (iota >> 10000))) >> ;; 0.008019s real time, 0.007979s run time. 0.000000s spent in GC. >> scheme@(guile-user)> ,time (define a (par-map (lambda (x) (expt x 5)) >> (iota 10000))) >> ;; 6.596471s real time, 6.579375s run time. 1.513880s spent in GC. >> --------------------end------------------- >> >> So my question is, what's the proper scenario to use par-map? > > It only makes sense to use 'par-map' when the procedure is fairly > expensive to compute. Indeed. > There is inevitably a lot of overhead in creating and joining the > threads. We use a thread pool, so there=E2=80=99s no such cost. But there are other costs. When delimited continuations are used, we=E2=80= =99re on the slow path. Also, Guile=E2=80=99s fat mutexes & co. are terribly inefficient. And finally, there may be contention on the futexes mutex (esp. when the computations is too small.) So yes, there=E2=80=99s room for improvement. Yet, it should be fruitful, provided you use it for reasonably long computations, as Mark outlines. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 28 14:03:42 2013 Received: (at 13188-done) by debbugs.gnu.org; 28 Mar 2013 18:03:42 +0000 Received: from localhost ([127.0.0.1]:49881 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULHAr-0004f4-P3 for submit@debbugs.gnu.org; Thu, 28 Mar 2013 14:03:42 -0400 Received: from world.peace.net ([96.39.62.75]:50853) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULHAo-0004ev-Sa for 13188-done@debbugs.gnu.org; Thu, 28 Mar 2013 14:03:40 -0400 Received: from 209-6-91-212.c3-0.smr-ubr1.sbo-smr.ma.cable.rcn.com ([209.6.91.212] helo=tines.lan) by world.peace.net with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1ULH8G-0007Nc-4C; Thu, 28 Mar 2013 14:01:00 -0400 From: Mark H Weaver To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow) References: <1355559152.27310.5.camel@Renee-desktop.suse> <87y5d8rclr.fsf@gnu.org> <1364439334.2730.41.camel@Renee-desktop.suse> <874nfwazc3.fsf@tines.lan> <87r4izprks.fsf@gnu.org> Date: Thu, 28 Mar 2013 14:00:52 -0400 In-Reply-To: <87r4izprks.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 28 Mar 2013 14:44:03 +0100") Message-ID: <87ip4b9zfv.fsf@tines.lan> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.8 (/) X-Debbugs-Envelope-To: 13188-done Cc: 13188-done@debbugs.gnu.org, guile-devel@gnu.org, Nala Ginrut X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) Hi Ludovic, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Mark H Weaver skribis: > >> It only makes sense to use 'par-map' when the procedure is fairly >> expensive to compute. > > Indeed. > >> There is inevitably a lot of overhead in creating and joining the >> threads. > > We use a thread pool, so there=E2=80=99s no such cost. Sorry, I was using the term 'threads' not in the sense of OS-level threads, but in a more general sense. I should have been more clear. What I meant is that from the user's perspective, threads are being created and joined, and even if you build those using a pool of OS-level threads, this inevitably involves thread synchronization, which is very expensive on modern architectures. So I maintain that there _is_ such a cost, and it can't be avoided. The point I was really trying to make here, in the simplest possible terms, is that it will *never* make sense to replace all uses of 'map' with 'par-map' wherever it is safe to do so. > But there are other costs. When delimited continuations are used, we=E2= =80=99re > on the slow path. Also, Guile=E2=80=99s fat mutexes & co. are terribly > inefficient. And finally, there may be contention on the futexes mutex > (esp. when the computations is too small.) Indeed, I wouldn't be surprised if we could improve this by an order of magnitude. More items for my TODO list :) > So yes, there=E2=80=99s room for improvement. Yet, it should be fruitful, > provided you use it for reasonably long computations, as Mark outlines. Regards, Mark From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 28 16:09:50 2013 Received: (at 13188-done) by debbugs.gnu.org; 28 Mar 2013 20:09:50 +0000 Received: from localhost ([127.0.0.1]:49962 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULJ8v-000857-Uf for submit@debbugs.gnu.org; Thu, 28 Mar 2013 16:09:50 -0400 Received: from xanadu.aquilenet.fr ([88.191.123.111]:57601) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULJ8t-00084z-Db for 13188-done@debbugs.gnu.org; Thu, 28 Mar 2013 16:09:48 -0400 Received: from localhost (localhost [127.0.0.1]) by xanadu.aquilenet.fr (Postfix) with ESMTP id 6807FCDBE; Thu, 28 Mar 2013 21:07:14 +0100 (CET) Received: from xanadu.aquilenet.fr ([127.0.0.1]) by localhost (xanadu.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dzgo47yLBJg6; Thu, 28 Mar 2013 21:07:14 +0100 (CET) Received: from pluto (reverse-83.fdn.fr [80.67.176.83]) by xanadu.aquilenet.fr (Postfix) with ESMTPSA id B11C4CDB6; Thu, 28 Mar 2013 21:07:13 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Mark H Weaver Subject: Re: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow) References: <1355559152.27310.5.camel@Renee-desktop.suse> <87y5d8rclr.fsf@gnu.org> <1364439334.2730.41.camel@Renee-desktop.suse> <874nfwazc3.fsf@tines.lan> <87r4izprks.fsf@gnu.org> <87ip4b9zfv.fsf@tines.lan> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 8 Germinal an 221 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0xEA52ECF4 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 83C4 F8E5 10A3 3B4C 5BEA D15D 77DD 95E2 EA52 ECF4 X-OS: x86_64-unknown-linux-gnu Date: Thu, 28 Mar 2013 21:07:12 +0100 In-Reply-To: <87ip4b9zfv.fsf@tines.lan> (Mark H. Weaver's message of "Thu, 28 Mar 2013 14:00:52 -0400") Message-ID: <87sj3fxp8v.fsf@gnu.org> User-Agent: Gnus/5.130005 (Ma Gnus v0.5) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Mark H Weaver skribis: > ludo@gnu.org (Ludovic Courtès) writes: > >> Mark H Weaver skribis: >> >>> It only makes sense to use 'par-map' when the procedure is fairly >>> expensive to compute. >> >> Indeed. >> >>> There is inevitably a lot of overhead in creating and joining the >>> threads. >> >> We use a thread pool, so there’s no such cost. > > Sorry, I was using the term 'threads' not in the sense of OS-level > threads, but in a more general sense. I should have been more clear. > > What I meant is that from the user's perspective, threads are being > created and joined, and even if you build those using a pool of OS-level > threads, this inevitably involves thread synchronization, which is very > expensive on modern architectures. So I maintain that there _is_ such a > cost, and it can't be avoided. [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.7 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.4024] X-Debbugs-Envelope-To: 13188-done Cc: 13188-done@debbugs.gnu.org, guile-devel@gnu.org, Nala Ginrut X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.2 (-) Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> Mark H Weaver skribis: >> >>> It only makes sense to use 'par-map' when the procedure is fairly >>> expensive to compute. >> >> Indeed. >> >>> There is inevitably a lot of overhead in creating and joining the >>> threads. >> >> We use a thread pool, so there=E2=80=99s no such cost. > > Sorry, I was using the term 'threads' not in the sense of OS-level > threads, but in a more general sense. I should have been more clear. > > What I meant is that from the user's perspective, threads are being > created and joined, and even if you build those using a pool of OS-level > threads, this inevitably involves thread synchronization, which is very > expensive on modern architectures. So I maintain that there _is_ such a > cost, and it can't be avoided. Ah yes, OK. > The point I was really trying to make here, in the simplest possible > terms, is that it will *never* make sense to replace all uses of 'map' > with 'par-map' wherever it is safe to do so. Indeed! Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Thu Mar 28 22:39:35 2013 Received: (at 13188-done) by debbugs.gnu.org; 29 Mar 2013 02:39:35 +0000 Received: from localhost ([127.0.0.1]:50313 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULPE6-0002Yq-J8 for submit@debbugs.gnu.org; Thu, 28 Mar 2013 22:39:35 -0400 Received: from mail-pd0-f180.google.com ([209.85.192.180]:58313) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULPE4-0002Yg-B9 for 13188-done@debbugs.gnu.org; Thu, 28 Mar 2013 22:39:33 -0400 Received: by mail-pd0-f180.google.com with SMTP id g10so72493pdj.39 for <13188-done@debbugs.gnu.org>; Thu, 28 Mar 2013 19:36:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:subject:from:to:cc:date:in-reply-to :references:organization:content-type:x-mailer:mime-version :content-transfer-encoding; bh=2sJmLC0vT9xmHgWjWsT0WhRhF0U0neqWcQgk2aaW7ew=; b=s2K3/WF1dmhxqsFq1Mp8oNFALZYK72lnuW5fZUmozFzs3+KxUkE9jaaOUZRfBSQHym FbN9ad1dXbEZg64X7kz7Kx80kyhX6UUYXWlnsUs+YTYv0Ax38vrNN7IL+M/DzbcOn0Ny 2zhAZjgsf7py8fzq4jWIpJlSLkQ3BcxJZCCI7PTYCud7fKGrw7S2x63JUSQCTibC7Qaj qKojyVQfZEV2H9pR2KFgGEeA9qPq1NO4MVBiVBRdnG2CVle20jugF8wFEotAamuJ/013 rxfZPvRGtXymo42mgg8oqSJmXkhbK399bTBAt3k6+Gj/XrNefy8gwnM5ERjEKd7hntWT kucg== X-Received: by 10.66.43.244 with SMTP id z20mr1989786pal.200.1364524617849; Thu, 28 Mar 2013 19:36:57 -0700 (PDT) Received: from [147.2.147.112] ([61.14.130.226]) by mx.google.com with ESMTPS id z1sm996446pbw.19.2013.03.28.19.36.55 (version=SSLv3 cipher=RC4-SHA bits=128/128); Thu, 28 Mar 2013 19:36:56 -0700 (PDT) Message-ID: <1364524610.2730.48.camel@Renee-desktop.suse> Subject: Re: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow) From: Nala Ginrut To: Mark H Weaver Date: Fri, 29 Mar 2013 10:36:50 +0800 In-Reply-To: <874nfwazc3.fsf@tines.lan> References: <1355559152.27310.5.camel@Renee-desktop.suse> <87y5d8rclr.fsf@gnu.org> <1364439334.2730.41.camel@Renee-desktop.suse> <874nfwazc3.fsf@tines.lan> Organization: HFG Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.4.4 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 13188-done Cc: Ludovic =?ISO-8859-1?Q?Court=E8s?= , 13188-done@debbugs.gnu.org, guile-devel@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) On Thu, 2013-03-28 at 01:05 -0400, Mark H Weaver wrote: > Nala Ginrut writes: > > > But I'm still puzzled with the performance of par-map: > > --------------------cut------------------- > > scheme@(guile-user)> ,time (define a (map (lambda (x) (expt x 5)) (iota > > 10000))) > > ;; 0.008019s real time, 0.007979s run time. 0.000000s spent in GC. > > scheme@(guile-user)> ,time (define a (par-map (lambda (x) (expt x 5)) > > (iota 10000))) > > ;; 6.596471s real time, 6.579375s run time. 1.513880s spent in GC. > > --------------------end------------------- > > > > So my question is, what's the proper scenario to use par-map? > > It only makes sense to use 'par-map' when the procedure is fairly > expensive to compute. There is inevitably a lot of overhead in creating > and joining the threads. Granted, we should be able to do much better > than we're doing now, but it would *never* make sense to use 'par-map' > when each computation is as simple as (expt x 5). > Well, is there any example? And there're two possible applications: 1. handle the requests in a server 2. read files from disk (but how big file is proper for par-map) Are these ways heavy enough for par-map? Potentially, I inclined to use the lovely delimited-continuation to handle the requests, but seems ludo think this way is slower? Or there's improvement room? > Regards, > Mark From debbugs-submit-bounces@debbugs.gnu.org Fri Mar 29 05:55:28 2013 Received: (at 13188-done) by debbugs.gnu.org; 29 Mar 2013 09:55:29 +0000 Received: from localhost ([127.0.0.1]:50642 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULW1w-0005pe-N8 for submit@debbugs.gnu.org; Fri, 29 Mar 2013 05:55:28 -0400 Received: from xanadu.aquilenet.fr ([88.191.123.111]:52484) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1ULW1v-0005pY-4N for 13188-done@debbugs.gnu.org; Fri, 29 Mar 2013 05:55:27 -0400 Received: from localhost (localhost [127.0.0.1]) by xanadu.aquilenet.fr (Postfix) with ESMTP id C507BC9B8; Fri, 29 Mar 2013 10:52:51 +0100 (CET) Received: from xanadu.aquilenet.fr ([127.0.0.1]) by localhost (xanadu.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JD7awu2lEdFF; Fri, 29 Mar 2013 10:52:51 +0100 (CET) Received: from pluto (unknown [193.50.110.192]) by xanadu.aquilenet.fr (Postfix) with ESMTPSA id 74AB2C79E; Fri, 29 Mar 2013 10:52:51 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Nala Ginrut Subject: Re: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow) References: <1355559152.27310.5.camel@Renee-desktop.suse> <87y5d8rclr.fsf@gnu.org> <1364439334.2730.41.camel@Renee-desktop.suse> <874nfwazc3.fsf@tines.lan> <1364524610.2730.48.camel@Renee-desktop.suse> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 9 Germinal an 221 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0xEA52ECF4 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 83C4 F8E5 10A3 3B4C 5BEA D15D 77DD 95E2 EA52 ECF4 X-OS: x86_64-unknown-linux-gnu Date: Fri, 29 Mar 2013 10:52:51 +0100 In-Reply-To: <1364524610.2730.48.camel@Renee-desktop.suse> (Nala Ginrut's message of "Fri, 29 Mar 2013 10:36:50 +0800") Message-ID: <87ppyimt1o.fsf@gnu.org> User-Agent: Gnus/5.130005 (Ma Gnus v0.5) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.5 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Nala Ginrut skribis: > And there're two possible applications: > 1. handle the requests in a server > 2. read files from disk (but how big file is proper for par-map) Quoting the fine manual: [...] Content analysis details: (1.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.7 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% [score: 0.4982] X-Debbugs-Envelope-To: 13188-done Cc: Mark H Weaver , 13188-done@debbugs.gnu.org, guile-devel@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: 0.7 (/) Nala Ginrut skribis: > And there're two possible applications: > 1. handle the requests in a server > 2. read files from disk (but how big file is proper for par-map) Quoting the fine manual: Note that futures are intended for the evaluation of purely functional expressions. Expressions that have side-effects or rely on I/O may require additional care, such as explicit synchronization (*note Mutexes and Condition Variables::). IOW, think twice before you use it for something not purely computational. :-) Ludo=E2=80=99. From unknown Tue Jun 24 20:52:04 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 26 Apr 2013 11:24:04 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator