From unknown Sat Jun 14 00:10:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#39361: continuation and gc performance Resent-From: Stefan Israelsson Tampe Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 30 Jan 2020 21:12:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 39361 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: 39361@debbugs.gnu.org X-Debbugs-Original-To: bug-guile@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.15804186682529 (code B ref -1); Thu, 30 Jan 2020 21:12:01 +0000 Received: (at submit) by debbugs.gnu.org; 30 Jan 2020 21:11:08 +0000 Received: from localhost ([127.0.0.1]:36328 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ixH5g-0000ei-E5 for submit@debbugs.gnu.org; Thu, 30 Jan 2020 16:11:08 -0500 Received: from lists.gnu.org ([209.51.188.17]:46161) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ixH5f-0000eb-5d for submit@debbugs.gnu.org; Thu, 30 Jan 2020 16:11:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49752) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ixH5b-0003PE-Qb for bug-guile@gnu.org; Thu, 30 Jan 2020 16:11:07 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ixH5a-0000He-Qh for bug-guile@gnu.org; Thu, 30 Jan 2020 16:11:03 -0500 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]:39550) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ixH5a-0000Fi-KO for bug-guile@gnu.org; Thu, 30 Jan 2020 16:11:02 -0500 Received: by mail-wr1-x42b.google.com with SMTP id y11so5914177wrt.6 for ; Thu, 30 Jan 2020 13:11:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=ptd5pGFZbxUMz0SkETfp0/UqXLYRhxderXv7JKXCrv0=; b=XnwXc58VFQhO9dPPlyzV7UM//JUHJVBwvTSdNs6bHXayqL/ZG6uf7hOVqPyUAxvu5i 30R0kNkSmau6oqR1YQS2wSGHZxmcVjiAfwmp6fvttAdke0AueihstJ25L4GLCYY2qZwy QlfQQUn6+vJ2bdxqusTJkr6wCv0BBtwQCASiQHrMeBacCeqGNjuu05uVlcNzUIUREaVn WfPSisGXqyAHywwTIi1STBMD2+n7qcIFnS3bsp8FP2leJHqiIBY0tt0gbEc3ECcIDpHn 9gMyhghm9gd1iV+KcfZim5cHUN9DEa9m8SAm83XS2HmhwLzny6PfOYF6Cx29zMS8xJ40 rhbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ptd5pGFZbxUMz0SkETfp0/UqXLYRhxderXv7JKXCrv0=; b=VGmILck6KVo+zFllU7dcTaEMzDYcPHQfuCgqG7Cm8jpslIh1b/MG+peJtOe13CYxcM BPYUsmEE+U8vCSuBB8L73bNJHYxWkL8LPuztuZxDRqmtb9unX9SsGDpw3GSQTN3ZPuFj 8GyEn/5dkjo0A+jOEh5BTEWatBG/lGGSKNUalljohJdLZUYTW66OOkt20WDKl/x/B573 ewlXGE/MJmdyAU4wT8ORLNMAlMGFi/ziAjbxHMe9rb596X042ROW+I2RowH5sZzZCe+l IuGjAHbXzhKFEDu698c2y08+k3WE0OGozBiGNIikai1oOWkWoqEJDo3gsP10G0Juz1Gb 6IUg== X-Gm-Message-State: APjAAAUr50qBrQjqRohAOa1JLoJAyzcmolyZWT/ON/RK1YdB8t+hr8KV pupkmy+9E6vU+iDk0TP6/g2fnWKi9CRIn2r+uOnVmg== X-Google-Smtp-Source: APXvYqzf8T3LaeNIEdIcBme9kn24Bc6DVSfUW/gVe9u/Kl8Umhj+UN9eP1D7lSlQ2X81DYyknAUAw0Itr4Lla8gaJ10= X-Received: by 2002:a5d:4d8d:: with SMTP id b13mr7685488wru.6.1580418660812; Thu, 30 Jan 2020 13:11:00 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Stefan Israelsson Tampe Date: Thu, 30 Jan 2020 22:10:50 +0100 Message-ID: Content-Type: multipart/alternative; boundary="000000000000dfd1ff059d61e4f8" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::42b X-Spam-Score: 0.3 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) --000000000000dfd1ff059d61e4f8 Content-Type: text/plain; charset="UTF-8" I think I found a gc leak in guile 3.0 Isn't it so that so the continuation keep a copy of the stack. The issue is that in the stack a raw integer or float may be present and so the gc properties is less then ideal as those may be interpreted as pointers by the GC and lead to parts of the heap being kept from garnage collecting. The information about a slot being a raw value or a scm value is available as we do the correct gc updating of the stack inside guile 3.0. May I propose that we add a bitvector to the continuation that indicate that if a lslot is raw or not. Then add a pass that collect the rawness information in the creation of the closure. Finally a custom made mark procedure for closures can be made that uses all this information to make sure to mark only scm slots in the stored continuation therby improving gc perfromance. With this information it would also be possible to serialize continuations even if they have slots that are raw values. Happy Hacking --000000000000dfd1ff059d61e4f8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I think I found a gc leak in guile 3= .0

Isn't it so that so the continuation keep a= copy of the stack. The issue is that in the stack a raw integer or float m= ay be present and so the gc properties is less then ideal as those may be i= nterpreted as pointers by the GC and lead to parts of the heap being kept f= rom garnage collecting.

The information about a sl= ot being a raw value or a scm value is available=C2=A0as we do the correct = gc updating of the=C2=A0stack inside guile 3.0. May I propose that we add a= bitvector to the continuation that indicate that if a lslot is raw or not.= Then add a pass that collect the rawness information in the creation of th= e closure. Finally a custom made mark procedure for closures can be made th= at uses all this information to make sure to mark only scm slots in the sto= red continuation therby improving gc perfromance.

= With this information it would also be possible to serialize continuations = even if they have slots that are raw values.

Happy= Hacking
=C2=A0
--000000000000dfd1ff059d61e4f8-- From unknown Sat Jun 14 00:10:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#39361: continuation and gc performance Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Sat, 21 Mar 2020 17:34:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39361 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: Stefan Israelsson Tampe Cc: 39361@debbugs.gnu.org Received: via spool by 39361-submit@debbugs.gnu.org id=B39361.1584812029404 (code B ref 39361); Sat, 21 Mar 2020 17:34:02 +0000 Received: (at 39361) by debbugs.gnu.org; 21 Mar 2020 17:33:49 +0000 Received: from localhost ([127.0.0.1]:47938 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jFi0K-00006S-TF for submit@debbugs.gnu.org; Sat, 21 Mar 2020 13:33:49 -0400 Received: from eggs.gnu.org ([209.51.188.92]:41009) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jFi0J-00006F-Rb for 39361@debbugs.gnu.org; Sat, 21 Mar 2020 13:33:48 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48402) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1jFi0E-0004MO-NS; Sat, 21 Mar 2020 13:33:42 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=56122 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jFi0E-0003cK-9K; Sat, 21 Mar 2020 13:33:42 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: Date: Sat, 21 Mar 2020 18:33:41 +0100 In-Reply-To: (Stefan Israelsson Tampe's message of "Thu, 30 Jan 2020 22:10:50 +0100") Message-ID: <878sjtzk7e.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi Stefan, Stefan Israelsson Tampe skribis: > I think I found a gc leak in guile 3.0 > > Isn't it so that so the continuation keep a copy of the stack. The issue = is > that in the stack a raw integer or float may be present and so the gc > properties is less then ideal as those may be interpreted as pointers by > the GC and lead to parts of the heap being kept from garnage collecting. > > The information about a slot being a raw value or a scm value is > available as we do the correct gc updating of the stack inside guile 3.0. > May I propose that we add a bitvector to the continuation that indicate > that if a lslot is raw or not. Then add a pass that collect the rawness > information in the creation of the closure. Finally a custom made mark > procedure for closures can be made that uses all this information to make > sure to mark only scm slots in the stored continuation therby improving gc > perfromance. I believe what you describe is already what happens in =E2=80=98scm_i_vm_mark_stack=E2=80=99. Or am I missing something? Thanks, Ludo=E2=80=99. From unknown Sat Jun 14 00:10:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#39361: continuation and gc performance Resent-From: Stefan Israelsson Tampe Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Sat, 21 Mar 2020 20:45:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39361 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 39361@debbugs.gnu.org Received: via spool by 39361-submit@debbugs.gnu.org id=B39361.158482344821953 (code B ref 39361); Sat, 21 Mar 2020 20:45:02 +0000 Received: (at 39361) by debbugs.gnu.org; 21 Mar 2020 20:44:08 +0000 Received: from localhost ([127.0.0.1]:48256 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jFkyW-0005i1-GY for submit@debbugs.gnu.org; Sat, 21 Mar 2020 16:44:08 -0400 Received: from mail-wm1-f54.google.com ([209.85.128.54]:33867) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jFkyU-0005hd-Vc for 39361@debbugs.gnu.org; Sat, 21 Mar 2020 16:44:07 -0400 Received: by mail-wm1-f54.google.com with SMTP id 26so5805201wmk.1 for <39361@debbugs.gnu.org>; Sat, 21 Mar 2020 13:44:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=JYP+20OXUcsTl6EkDJxgSaQ8iHxoS+9Q4lEnh5AfZQE=; b=bM9ltLCO1SW5D27Y1S9QOJkmEEDIpHh0a+OAixz697mLP89jmmsBemlN/eYpJ1DR0w BNuOtTBviBJ+DNs114AtT4IaiRyxn1w1E/NTomAyWsCg0uaCPATj4j5N4SPuh+jw3L2g 8KQrZFHtEV4bNzngtmC/BkgwJu1iaBmOvr9TIbw8s2GlmpOk0h0ar/Xni44F6Ea0akPF KdEOTVnu1yAJmO6UUiz6jqX0y095GlyYwqdBFcmpDK+ET37s0IlXl7cxem6asL8Env7+ yBsCAk3CNl4ssEH3AFueb9KIfCtFUKZxpgtq2sir+QWeAdMz21t+Eojfr7qSvR1A4eGQ yQng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JYP+20OXUcsTl6EkDJxgSaQ8iHxoS+9Q4lEnh5AfZQE=; b=nxBMOlsaJTClMpKSPO5UwhE0t7ukTSWu0N1hI3v0XOaAtqwvWsgWhEdPQVvKDn0L2+ iLUt+f5V6tcrP0fxR2bjFcSDr/eJnEyuClBgmWQC5UK1Qw3oQzOeQ/NvNomcIyBjFYN8 eIEj7UBH0iY3ItMvZ6RPy9pFLiHheebKl8QW5a418CKVPOkKeQAqMLHiDO+PRKW+Ku6q on1EouYccm4M9o7zkuZNUfdntB+b+04sK7zLtHNYZtvOXR4dex7ocpyk3k69jWhJ9P5a 1O1Mmw0YQhc0fA6Ko5x+DZ+n7PiyUYe+0Zm6NQuC4u1L4RlkEEe3b528GnfeclowrPE7 uW2A== X-Gm-Message-State: ANhLgQ1cJaGzICjpVhh2IiaSVBI1FDVj3lYKqHvMEmOjBWjdnjpeS3JO GphNunveKc8wL+rruz+OqZRslGmU0DcMWrAakSY= X-Google-Smtp-Source: ADFU+vs59Ty2dqOXS1nEjd3HL3jGxxAq2ZL+x7KJawHGYxjpJ46ySAquByqsCfQIsv5q+cF0n3uQ7dGLyDA44bC3rvQ= X-Received: by 2002:a7b:c74d:: with SMTP id w13mr7242426wmk.189.1584823440980; Sat, 21 Mar 2020 13:44:00 -0700 (PDT) MIME-Version: 1.0 References: <878sjtzk7e.fsf@gnu.org> In-Reply-To: <878sjtzk7e.fsf@gnu.org> From: Stefan Israelsson Tampe Date: Sat, 21 Mar 2020 21:43:49 +0100 Message-ID: Content-Type: multipart/alternative; boundary="0000000000003b3fbb05a1637673" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --0000000000003b3fbb05a1637673 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable That function marks the working stack no, what about stack segments in continuations will they be marked correctly as well? On Sat, Mar 21, 2020 at 6:33 PM Ludovic Court=C3=A8s wrote: > Hi Stefan, > > Stefan Israelsson Tampe skribis: > > > I think I found a gc leak in guile 3.0 > > > > Isn't it so that so the continuation keep a copy of the stack. The issu= e > is > > that in the stack a raw integer or float may be present and so the gc > > properties is less then ideal as those may be interpreted as pointers b= y > > the GC and lead to parts of the heap being kept from garnage collecting= . > > > > The information about a slot being a raw value or a scm value is > > available as we do the correct gc updating of the stack inside guile 3.= 0. > > May I propose that we add a bitvector to the continuation that indicate > > that if a lslot is raw or not. Then add a pass that collect the rawness > > information in the creation of the closure. Finally a custom made mark > > procedure for closures can be made that uses all this information to ma= ke > > sure to mark only scm slots in the stored continuation therby improving > gc > > perfromance. > > I believe what you describe is already what happens in > =E2=80=98scm_i_vm_mark_stack=E2=80=99. Or am I missing something? > > Thanks, > Ludo=E2=80=99. > --0000000000003b3fbb05a1637673 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
That function marks the working stack no, what about stack= segments in continuations will they be marked correctly as well?

=
On Sat, Ma= r 21, 2020 at 6:33 PM Ludovic Court=C3=A8s <ludo@gnu.org> wrote:
Hi Stefan,

Stefan Israelsson Tampe <stefan.itampe@gmail.com> skribis:

> I think I found a gc leak in guile 3.0
>
> Isn't it so that so the continuation keep a copy of the stack. The= issue is
> that in the stack a raw integer or float may be present and so the gc<= br> > properties is less then ideal as those may be interpreted as pointers = by
> the GC and lead to parts of the heap being kept from garnage collectin= g.
>
> The information about a slot being a raw value or a scm value is
> available as we do the correct gc updating of the stack inside guile 3= .0.
> May I propose that we add a bitvector to the continuation that indicat= e
> that if a lslot is raw or not. Then add a pass that collect the rawnes= s
> information in the creation of the closure. Finally a custom made mark=
> procedure for closures can be made that uses all this information to m= ake
> sure to mark only scm slots in the stored continuation therby improvin= g gc
> perfromance.

I believe what you describe is already what happens in
=E2=80=98scm_i_vm_mark_stack=E2=80=99.=C2=A0 Or am I missing something?

Thanks,
Ludo=E2=80=99.
--0000000000003b3fbb05a1637673-- From unknown Sat Jun 14 00:10:24 2025 X-Loop: help-debbugs@gnu.org Subject: bug#39361: continuation and gc performance Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Sun, 22 Mar 2020 20:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39361 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: Stefan Israelsson Tampe Cc: 39361@debbugs.gnu.org Received: via spool by 39361-submit@debbugs.gnu.org id=B39361.158491030021422 (code B ref 39361); Sun, 22 Mar 2020 20:52:02 +0000 Received: (at 39361) by debbugs.gnu.org; 22 Mar 2020 20:51:40 +0000 Received: from localhost ([127.0.0.1]:51012 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jG7ZM-0005ZR-2B for submit@debbugs.gnu.org; Sun, 22 Mar 2020 16:51:40 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53697) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jG7ZK-0005ZD-S5 for 39361@debbugs.gnu.org; Sun, 22 Mar 2020 16:51:39 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40174) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1jG7ZF-00009j-8I; Sun, 22 Mar 2020 16:51:33 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=38634 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jG7ZE-0003rv-NP; Sun, 22 Mar 2020 16:51:33 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <878sjtzk7e.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 3 Germinal an 228 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sun, 22 Mar 2020 21:51:31 +0100 In-Reply-To: (Stefan Israelsson Tampe's message of "Sat, 21 Mar 2020 21:43:49 +0100") Message-ID: <87wo7ct8oc.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Hi, Stefan Israelsson Tampe skribis: > That function marks the working stack no, what about stack segments in > continuations will they be marked correctly as well? Oh you=E2=80=99re right, from a quick look at continuations.c, a continuati= on=E2=80=99s stack appears to be conservatively scanned (allocated with =E2=80=98scm_gc_malloc=E2=80=99). My intuition is that it=E2=80=99s =E2=80=9Cgood enough=E2=80=9D in most cas= es, but could affect GC performance a tiny bit in continuation-heavy code, such as Fibers. Ludo=E2=80=99.