From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: Weak tables in 2.2.2 grow indefinitely Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Mon, 25 Sep 2017 08:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: 28590@debbugs.gnu.org X-Debbugs-Original-To: bug-guile@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.150632936931470 (code B ref -1); Mon, 25 Sep 2017 08:50:01 +0000 Received: (at submit) by debbugs.gnu.org; 25 Sep 2017 08:49:29 +0000 Received: from localhost ([127.0.0.1]:57246 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwP4y-0008BW-Vo for submit@debbugs.gnu.org; Mon, 25 Sep 2017 04:49:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:60520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwP4x-0008BJ-BN for submit@debbugs.gnu.org; Mon, 25 Sep 2017 04:49:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dwP4r-0006wL-F0 for submit@debbugs.gnu.org; Mon, 25 Sep 2017 04:49:22 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:33424) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dwP4r-0006wA-CY for submit@debbugs.gnu.org; Mon, 25 Sep 2017 04:49:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51456) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwP4q-0003Wu-AY for bug-guile@gnu.org; Mon, 25 Sep 2017 04:49:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dwP4n-0006u5-21 for bug-guile@gnu.org; Mon, 25 Sep 2017 04:49:20 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34829) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwP4m-0006tt-Vh for bug-guile@gnu.org; Mon, 25 Sep 2017 04:49:17 -0400 Received: from [193.50.110.164] (port=33444 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dwP4m-0003vz-G1 for bug-guile@gnu.org; Mon, 25 Sep 2017 04:49:16 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 =?UTF-8?Q?Vend=C3=A9miaire?= an 226 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 25 Sep 2017 10:49:13 +0200 Message-ID: <87poafkvsm.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Consider this program: --8<---------------cut here---------------start------------->8--- (use-modules (ice-9 format)) (define loops 3000000) (define table (make-weak-key-hash-table)) (let loop ((i loops)) (unless #f ;(zero? i) (when (zero? (modulo i 100000)) (format #t "heap-size: ~,2h MiB table: ~s~%" (/ (assoc-ref (gc-stats) 'heap-size) (expt 2. 20)) table)) (hashq-set! table (cons 1 2) #t) (loop (1- i)))) --8<---------------cut here---------------end--------------->8--- On 2.0.14, the heap size stays at around 24=C2=A0MiB, and the table size is stable at 224,717 buckets (?). On 2.2.2, the heap grows indefinitely (though logarithmically). It=E2=80= =99s not deterministic though: sometimes the heap size stabilizes in the 140=E2=80=93300=C2=A0MiB range, and sometimes it keeps growing endlessly ev= en though the table size reaches a maxium at 7,190,537 entries. Ludo=E2=80=99. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 25 05:14:11 2017 Received: (at control) by debbugs.gnu.org; 25 Sep 2017 09:14:11 +0000 Received: from localhost ([127.0.0.1]:57265 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwPSs-0000KG-VU for submit@debbugs.gnu.org; Mon, 25 Sep 2017 05:14:11 -0400 Received: from eggs.gnu.org ([208.118.235.92]:40081) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwPSr-0000Ju-FV for control@debbugs.gnu.org; Mon, 25 Sep 2017 05:14:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dwPSi-0002sY-5p for control@debbugs.gnu.org; Mon, 25 Sep 2017 05:14:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35138) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwPSi-0002sM-2O for control@debbugs.gnu.org; Mon, 25 Sep 2017 05:14:00 -0400 Received: from [193.50.110.164] (port=35038 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dwPSh-0003GY-GZ for control@debbugs.gnu.org; Mon, 25 Sep 2017 05:13:59 -0400 Date: Mon, 25 Sep 2017 11:13:57 +0200 Message-Id: <87bmlzkune.fsf@gnu.org> To: control@debbugs.gnu.org From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: control message for bug #28590 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) severity 28590 serious From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: Weak tables in 2.2.2 grow indefinitely Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 26 Sep 2017 09:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150641988827293 (code B ref 28590); Tue, 26 Sep 2017 09:59:02 +0000 Received: (at 28590) by debbugs.gnu.org; 26 Sep 2017 09:58:08 +0000 Received: from localhost ([127.0.0.1]:59722 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwmcy-000769-JL for submit@debbugs.gnu.org; Tue, 26 Sep 2017 05:58:08 -0400 Received: from eggs.gnu.org ([208.118.235.92]:47401) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwmcw-00075f-H9 for 28590@debbugs.gnu.org; Tue, 26 Sep 2017 05:58:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dwmco-0008O4-Kv for 28590@debbugs.gnu.org; Tue, 26 Sep 2017 05:58:01 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56310) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwmco-0008Nx-HL for 28590@debbugs.gnu.org; Tue, 26 Sep 2017 05:57:58 -0400 Received: from [193.50.110.164] (port=35450 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dwmcn-0004rq-JF for 28590@debbugs.gnu.org; Tue, 26 Sep 2017 05:57:57 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <87poafkvsm.fsf@gnu.org> Date: Tue, 26 Sep 2017 11:57:54 +0200 In-Reply-To: <87poafkvsm.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 25 Sep 2017 10:49:13 +0200") Message-ID: <87shf9zsrh.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > Consider this program: > > (use-modules (ice-9 format)) > > (define loops 3000000) > > (define table > (make-weak-key-hash-table)) > > (let loop ((i loops)) > (unless #f ;(zero? i) > (when (zero? (modulo i 100000)) > (format #t "heap-size: ~,2h MiB table: ~s~%" > (/ (assoc-ref (gc-stats) 'heap-size) (expt 2. 20)) > table)) > > (hashq-set! table (cons 1 2) #t) > (loop (1- i)))) I=E2=80=99ve changed it to do: (hashq-set! table (cons 1 2) (make-vector 10000)) Then if we build Guile with -DGC_DEBUG=3D1, link it against libgc built with -DKEEP_BACK_PTRS=3D1, and run with =E2=80=9CGC_BACKTRACES=3D1=E2=80=9D= , we see that the heap is full of those 10,000-element vectors: 0x54ae9030 (../libguile/gc.h:232, sz=3D80008, NORMAL) Reference could not be found 0x1bee5030 (../libguile/gc.h:232, sz=3D80008, NORMAL) Reference could not be found 0x60491030 (../libguile/gc.h:232, sz=3D80008, NORMAL) Reference could not be found Problem is that =E2=80=9CReference could not be found=E2=80=9D corresponds = to GC_UNREFERENCED in , meaning that there=E2=80=99s no reference holding this object. Interesting no? (Though it could also be a bug in the back-pointer code, or pretty much anything!) Ludo=E2=80=99. From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: Weak tables in 2.2.2 grow indefinitely Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Mon, 02 Oct 2017 09:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: 28590@debbugs.gnu.org Cc: Andy Wingo Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150693804926617 (code B ref 28590); Mon, 02 Oct 2017 09:55:02 +0000 Received: (at 28590) by debbugs.gnu.org; 2 Oct 2017 09:54:09 +0000 Received: from localhost ([127.0.0.1]:43450 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dyxQP-0006vF-5T for submit@debbugs.gnu.org; Mon, 02 Oct 2017 05:54:09 -0400 Received: from eggs.gnu.org ([208.118.235.92]:40027) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dyxQN-0006ul-2F for 28590@debbugs.gnu.org; Mon, 02 Oct 2017 05:54:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dyxQD-0005f5-E2 for 28590@debbugs.gnu.org; Mon, 02 Oct 2017 05:54:01 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:35508) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dyxPU-0005P9-My; Mon, 02 Oct 2017 05:53:12 -0400 Received: from [193.50.110.164] (port=41908 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dyxPU-0005EE-4T; Mon, 02 Oct 2017 05:53:12 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <87poafkvsm.fsf@gnu.org> Date: Mon, 02 Oct 2017 11:53:09 +0200 In-Reply-To: <87poafkvsm.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 25 Sep 2017 10:49:13 +0200") Message-ID: <8760bxri4a.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > On 2.2.2, the heap grows indefinitely (though logarithmically). It=E2=80= =99s > not deterministic though: sometimes the heap size stabilizes in the > 140=E2=80=93300=C2=A0MiB range, and sometimes it keeps growing endlessly = even though > the table size reaches a maxium at 7,190,537 entries. Below is a simple reproducer in C that I=E2=80=99ve also reported at : --8<---------------cut here---------------start------------->8--- #define GC_DEBUG 1 #include #include #include int main () { GC_INIT (); while (1) { unsigned int count =3D 777 * (random () % 1000); void **p =3D GC_MALLOC_ATOMIC (count * sizeof *p); for (unsigned int i =3D 0; i < count; i++) { void *key =3D GC_MALLOC_ATOMIC (10); p[i] =3D key; GC_GENERAL_REGISTER_DISAPPEARING_LINK (&p[i], key); /* <- !!! */ } static unsigned int loops =3D 0; if (loops++ % 10 =3D=3D 0) printf ("iteration %4u, heap size =3D %li MiB\n", loops, GC_get_heap_size () / (1024 * 1024)); } } --8<---------------cut here---------------end--------------->8--- Ludo=E2=80=99. From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 2/7] weak-table: Stress the GC a little less when resizing. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 03 Oct 2017 11:46:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150703111325839 (code B ref 28590); Tue, 03 Oct 2017 11:46:02 +0000 Received: (at 28590) by debbugs.gnu.org; 3 Oct 2017 11:45:13 +0000 Received: from localhost ([127.0.0.1]:45306 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdQ-0006iW-Q6 for submit@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:13 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42629) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdN-0006hj-Ml for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzLdF-0005ro-D7 for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59152) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzLcV-0005Aq-QA; Tue, 03 Oct 2017 07:44:15 -0400 Received: from [193.50.110.164] (port=33316 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzLcV-0000d1-AW; Tue, 03 Oct 2017 07:44:15 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Tue, 3 Oct 2017 13:43:47 +0200 Message-Id: <20171003114352.13984-3-ludo@gnu.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171003114352.13984-1-ludo@gnu.org> References: <20171003114352.13984-1-ludo@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) * libguile/weak-table.c (resize_table): Move 'allocate_entries' call outside of the loop. --- libguile/weak-table.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/libguile/weak-table.c b/libguile/weak-table.c index a0bebca5e..1aa2a0fcc 100644 --- a/libguile/weak-table.c +++ b/libguile/weak-table.c @@ -476,10 +476,11 @@ resize_table (scm_t_weak_table *table) if (new_size_index == table->size_index) return; new_size = hashtable_size[new_size_index]; - new_entries = allocate_entries (new_size, table->kind); } while (!is_acceptable_size_index (table, new_size_index)); + new_entries = allocate_entries (new_size, table->kind); + old_entries = table->entries; old_size = table->size; -- 2.14.2 From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 1/7] weak-table: Fix unbounded growth of the disappearing link table. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 03 Oct 2017 11:46:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150703111325847 (code B ref 28590); Tue, 03 Oct 2017 11:46:02 +0000 Received: (at 28590) by debbugs.gnu.org; 3 Oct 2017 11:45:13 +0000 Received: from localhost ([127.0.0.1]:45308 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdR-0006ii-3H for submit@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:13 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42628) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdN-0006hi-Mq for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzLdF-0005rt-Cz for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59151) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzLcU-0005AP-ST; Tue, 03 Oct 2017 07:44:14 -0400 Received: from [193.50.110.164] (port=33316 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzLcU-0000d1-Df; Tue, 03 Oct 2017 07:44:14 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Tue, 3 Oct 2017 13:43:46 +0200 Message-Id: <20171003114352.13984-2-ludo@gnu.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171003114352.13984-1-ludo@gnu.org> References: <20171003114352.13984-1-ludo@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Partly fixes . * libguile/weak-table.c (resize_table): Remove all disappearing links on OLD_ENTRIES, and reset all of OLD_ENTRIES. --- libguile/weak-table.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/libguile/weak-table.c b/libguile/weak-table.c index 599c4cf0e..a0bebca5e 100644 --- a/libguile/weak-table.c +++ b/libguile/weak-table.c @@ -498,11 +498,19 @@ resize_table (scm_t_weak_table *table) scm_t_weak_entry copy; unsigned long new_k, distance; + /* Make sure we don't leave a disappearing link behind us. + See . */ + unregister_disappearing_links (&old_entries[old_k], table->kind); + if (!old_entries[old_k].hash) - continue; - + { + old_entries[old_k].key = old_entries[old_k].value = 0; + continue; + } + copy_weak_entry (&old_entries[old_k], ©); - + old_entries[old_k].key = old_entries[old_k].value = 0; + if (!copy.key || !copy.value) continue; -- 2.14.2 From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 0/7] Attempt to reduce memory growth References: <87poafkvsm.fsf@gnu.org> In-Reply-To: <87poafkvsm.fsf@gnu.org> Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 03 Oct 2017 11:46:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150703111325854 (code B ref 28590); Tue, 03 Oct 2017 11:46:03 +0000 Received: (at 28590) by debbugs.gnu.org; 3 Oct 2017 11:45:13 +0000 Received: from localhost ([127.0.0.1]:45310 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdR-0006iq-F8 for submit@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:13 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42625) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdN-0006hh-Mn for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzLdF-0005rd-Ci for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD, URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59150) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzLcT-0005AC-Vr; Tue, 03 Oct 2017 07:44:14 -0400 Received: from [193.50.110.164] (port=33316 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzLcT-0000d1-Fl; Tue, 03 Oct 2017 07:44:13 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Tue, 3 Oct 2017 13:43:45 +0200 Message-Id: <20171003114352.13984-1-ludo@gnu.org> X-Mailer: git-send-email 2.14.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) So! This is an attempt to mitigate memory growth in the use case shown at . Unfortunately, it doesn’t that much: on the python.scm compilation “benchmark”, there’s a bit less than 10% gain both in memory consumption and CPU time. I’ll try to combine that with incremental marking of the weak table, but I’m not very hopeful. Andy: I need your help! :-) Ludo’. Ludovic Courtès (7): weak-table: Fix unbounded growth of the disappearing link table. weak-table: Stress the GC a little less when resizing. weak-table: Make sure 'move_disappearing_links' actually moves links. weak-table: Always unregister previous links when inserting an entry. weak-table: 'move_weak_entry' reports disappeared links. weak-table: 'rob_from_rich' accounts for disappeared entries. weak-table: Resize less frequently. libguile/weak-table.c | 144 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 114 insertions(+), 30 deletions(-) -- 2.14.2 From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 7/7] weak-table: Resize less frequently. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 03 Oct 2017 11:46:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150703111425863 (code B ref 28590); Tue, 03 Oct 2017 11:46:03 +0000 Received: (at 28590) by debbugs.gnu.org; 3 Oct 2017 11:45:14 +0000 Received: from localhost ([127.0.0.1]:45312 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdR-0006iy-Mn for submit@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:13 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42624) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdN-0006hf-Mp for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzLdF-0005rc-C5 for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59157) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzLca-0005E5-Ae; Tue, 03 Oct 2017 07:44:20 -0400 Received: from [193.50.110.164] (port=33316 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzLcZ-0000d1-S5; Tue, 03 Oct 2017 07:44:20 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Tue, 3 Oct 2017 13:43:52 +0200 Message-Id: <20171003114352.13984-8-ludo@gnu.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171003114352.13984-1-ludo@gnu.org> References: <20171003114352.13984-1-ludo@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) * libguile/weak-table.c (do_count_entries, update_entry_count): New functions. (resize_table): Use it; check 'n_items' and return if there's nothing to do. --- libguile/weak-table.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/libguile/weak-table.c b/libguile/weak-table.c index 24fff4e73..1a99a130f 100644 --- a/libguile/weak-table.c +++ b/libguile/weak-table.c @@ -504,6 +504,32 @@ is_acceptable_size_index (scm_t_weak_table *table, int size_index) return 0; } +static void * +do_count_entries (void *data) +{ + size_t i, count = 0; + scm_t_weak_table *table = data; + + for (i = 0; i < table->size; i++) + { + if (table->entries[i].hash != 0 + && table->entries[i].key != 0 + && table->entries[i].value != 0) + count++; + } + + table->n_items = count; + + return table; +} + +/* Traverse all of TABLE and updates its 'n_items' field. */ +static void +update_entry_count (scm_t_weak_table *table) +{ + GC_call_with_alloc_lock (do_count_entries, table); +} + static void resize_table (scm_t_weak_table *table) { @@ -511,6 +537,14 @@ resize_table (scm_t_weak_table *table) int new_size_index; unsigned long old_size, new_size, old_k; + /* Make sure we have an accurate view of TABLE's load factor before + attempting to resize it. */ + update_entry_count (table); + + if (table->n_items > table->lower + && table->n_items < table->upper) + return; + do { new_size_index = compute_size_index (table); -- 2.14.2 From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 5/7] weak-table: 'move_weak_entry' reports disappeared links. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 03 Oct 2017 11:46:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150703111425869 (code B ref 28590); Tue, 03 Oct 2017 11:46:04 +0000 Received: (at 28590) by debbugs.gnu.org; 3 Oct 2017 11:45:14 +0000 Received: from localhost ([127.0.0.1]:45314 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdR-0006j6-W0 for submit@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:14 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42626) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdN-0006hg-Mr for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzLdF-0005rz-DU for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59155) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzLcY-0005CT-HU; Tue, 03 Oct 2017 07:44:18 -0400 Received: from [193.50.110.164] (port=33316 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzLcY-0000d1-2S; Tue, 03 Oct 2017 07:44:18 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Tue, 3 Oct 2017 13:43:50 +0200 Message-Id: <20171003114352.13984-6-ludo@gnu.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171003114352.13984-1-ludo@gnu.org> References: <20171003114352.13984-1-ludo@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) * libguile/weak-table.c (move_weak_entry): Change to return a Boolean. (give_to_poor): Remove 'copy_weak_entry' call and adjust accordingly. --- libguile/weak-table.c | 46 +++++++++++++++++++++++++++++----------------- 1 file changed, 29 insertions(+), 17 deletions(-) diff --git a/libguile/weak-table.c b/libguile/weak-table.c index b5db3ef48..5c4b3d30a 100644 --- a/libguile/weak-table.c +++ b/libguile/weak-table.c @@ -199,28 +199,45 @@ move_disappearing_links (scm_t_weak_entry *from, scm_t_weak_entry *to, } } -static void +/* Move weak entry FROM to TO. Return non-zero on success, and zero if + the entry at FROM disappeared in the meantime. */ +static int move_weak_entry (scm_t_weak_entry *from, scm_t_weak_entry *to, scm_t_weak_table_kind kind) { if (from->hash) { scm_t_weak_entry copy; - + copy_weak_entry (from, ©); - to->hash = copy.hash; - to->key = copy.key; - to->value = copy.value; - move_disappearing_links (from, to, - SCM_PACK (copy.key), SCM_PACK (copy.value), - kind); + if (copy.key != 0 && copy.value != 0) + { + to->hash = copy.hash; + to->key = copy.key; + to->value = copy.value; + + move_disappearing_links (from, to, + SCM_PACK (copy.key), SCM_PACK (copy.value), + kind); + return 1; + } + else + { + /* Lost weak reference: make sure all the disappearing links + are registered (in the case of a doubly-weak table, one of + the disappearing links could still be there.) */ + memset (to, '\0', sizeof *to); + unregister_disappearing_links (from, kind); + return 0; + } } else { + unregister_disappearing_links (from, kind); to->hash = 0; - to->key = 0; - to->value = 0; + to->key = to->value = 0; + return 0; } } @@ -301,16 +318,14 @@ give_to_poor (scm_t_weak_table *table, unsigned long k) { unsigned long next = (k + 1) % size; unsigned long hash; - scm_t_weak_entry copy; hash = table->entries[next].hash; if (!hash || hash_to_index (hash, size) == next) break; - copy_weak_entry (&table->entries[next], ©); - - if (!copy.key || !copy.value) + if (!move_weak_entry (&table->entries[next], &table->entries[k], + table->kind)) /* Lost weak reference. */ { give_to_poor (table, next); @@ -318,9 +333,6 @@ give_to_poor (scm_t_weak_table *table, unsigned long k) continue; } - move_weak_entry (&table->entries[next], &table->entries[k], - table->kind); - k = next; } -- 2.14.2 From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 6/7] weak-table: 'rob_from_rich' accounts for disappeared entries. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 03 Oct 2017 11:46:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150703111525879 (code B ref 28590); Tue, 03 Oct 2017 11:46:04 +0000 Received: (at 28590) by debbugs.gnu.org; 3 Oct 2017 11:45:15 +0000 Received: from localhost ([127.0.0.1]:45316 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdT-0006jJ-Ba for submit@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:15 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42658) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdQ-0006hn-8e for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzLdF-0005rZ-Bp for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:07 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59156) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzLcZ-0005DO-DN; Tue, 03 Oct 2017 07:44:19 -0400 Received: from [193.50.110.164] (port=33316 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzLcY-0000d1-Vr; Tue, 03 Oct 2017 07:44:19 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Tue, 3 Oct 2017 13:43:51 +0200 Message-Id: <20171003114352.13984-7-ludo@gnu.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171003114352.13984-1-ludo@gnu.org> References: <20171003114352.13984-1-ludo@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) * libguile/weak-table.c (rob_from_rich): Leave the loop also if 'key' or 'value' is zero. Reset 'hash'. --- libguile/weak-table.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/libguile/weak-table.c b/libguile/weak-table.c index 5c4b3d30a..24fff4e73 100644 --- a/libguile/weak-table.c +++ b/libguile/weak-table.c @@ -292,7 +292,11 @@ rob_from_rich (scm_t_weak_table *table, unsigned long k) empty = k; do empty = (empty + 1) % size; - while (table->entries[empty].hash); + while (table->entries[empty].hash + && table->entries[empty].key + && table->entries[empty].value); + + table->entries[empty].hash = 0; do { -- 2.14.2 From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 4/7] weak-table: Always unregister previous links when inserting an entry. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 03 Oct 2017 11:46:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150703111525886 (code B ref 28590); Tue, 03 Oct 2017 11:46:05 +0000 Received: (at 28590) by debbugs.gnu.org; 3 Oct 2017 11:45:15 +0000 Received: from localhost ([127.0.0.1]:45318 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdT-0006jM-JY for submit@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:15 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42623) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdQ-0006ho-Jl for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzLdF-0005rT-9d for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59154) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzLcX-0005Br-KI; Tue, 03 Oct 2017 07:44:17 -0400 Received: from [193.50.110.164] (port=33316 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzLcX-0000d1-4j; Tue, 03 Oct 2017 07:44:17 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Tue, 3 Oct 2017 13:43:49 +0200 Message-Id: <20171003114352.13984-5-ludo@gnu.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171003114352.13984-1-ludo@gnu.org> References: <20171003114352.13984-1-ludo@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) * libguile/weak-table.c (weak_table_put_x): Always call 'unregister_disappearing_links' before returning. --- libguile/weak-table.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/libguile/weak-table.c b/libguile/weak-table.c index 7d8633165..b5db3ef48 100644 --- a/libguile/weak-table.c +++ b/libguile/weak-table.c @@ -731,9 +731,9 @@ weak_table_put_x (scm_t_weak_table *table, unsigned long hash, return; } - if (entries[k].hash) - unregister_disappearing_links (&entries[k], table->kind); - else + unregister_disappearing_links (&entries[k], table->kind); + + if (!entries[k].hash) table->n_items++; entries[k].hash = hash; -- 2.14.2 From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 3/7] weak-table: Make sure 'move_disappearing_links' actually moves links. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 03 Oct 2017 11:46:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150703111625892 (code B ref 28590); Tue, 03 Oct 2017 11:46:05 +0000 Received: (at 28590) by debbugs.gnu.org; 3 Oct 2017 11:45:16 +0000 Received: from localhost ([127.0.0.1]:45320 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdT-0006jT-RJ for submit@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:16 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzLdQ-0006hp-Jo for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzLdF-0005s3-Dj for 28590@debbugs.gnu.org; Tue, 03 Oct 2017 07:45:04 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:59153) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzLcW-0005BN-Mg; Tue, 03 Oct 2017 07:44:16 -0400 Received: from [193.50.110.164] (port=33316 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzLcW-0000d1-7s; Tue, 03 Oct 2017 07:44:16 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Tue, 3 Oct 2017 13:43:48 +0200 Message-Id: <20171003114352.13984-4-ludo@gnu.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171003114352.13984-1-ludo@gnu.org> References: <20171003114352.13984-1-ludo@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) * libguile/weak-table.c (move_disappearing_links): Check the return value of 'GC_move_disappearing_link' and call 'register_disappearing_links' upon GC_NOT_FOUND. (GC_move_disappearing_link) [!HAVE_GC_MOVE_DISAPPEARING_LINK]: Adjust to return the error code. --- libguile/weak-table.c | 35 ++++++++++++++++++++++++++++++----- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/libguile/weak-table.c b/libguile/weak-table.c index 1aa2a0fcc..7d8633165 100644 --- a/libguile/weak-table.c +++ b/libguile/weak-table.c @@ -153,11 +153,16 @@ unregister_disappearing_links (scm_t_weak_entry *entry, } #ifndef HAVE_GC_MOVE_DISAPPEARING_LINK -static void +static int GC_move_disappearing_link (void **from, void **to) { - GC_unregister_disappearing_link (from); - SCM_I_REGISTER_DISAPPEARING_LINK (to, *to); + int err; + + err = GC_unregister_disappearing_link (from); + if (err == GC_SUCCESS) + err = SCM_I_REGISTER_DISAPPEARING_LINK (to, *to); + + return err; } #endif @@ -165,13 +170,33 @@ static void move_disappearing_links (scm_t_weak_entry *from, scm_t_weak_entry *to, SCM key, SCM value, scm_t_weak_table_kind kind) { + int err; + if ((kind == SCM_WEAK_TABLE_KIND_KEY || kind == SCM_WEAK_TABLE_KIND_BOTH) && SCM_HEAP_OBJECT_P (key)) - GC_move_disappearing_link ((void **) &from->key, (void **) &to->key); + { + err = GC_move_disappearing_link ((void **) &from->key, + (void **) &to->key); + if (err == GC_NOT_FOUND) + /* Link disappeared. */ + register_disappearing_links (to, key, value, + SCM_WEAK_TABLE_KIND_KEY); + else + assert (err == GC_SUCCESS); + } if ((kind == SCM_WEAK_TABLE_KIND_VALUE || kind == SCM_WEAK_TABLE_KIND_BOTH) && SCM_HEAP_OBJECT_P (value)) - GC_move_disappearing_link ((void **) &from->value, (void **) &to->value); + { + err = GC_move_disappearing_link ((void **) &from->value, + (void **) &to->value); + if (err == GC_NOT_FOUND) + /* Link disappeared. */ + register_disappearing_links (to, key, value, + SCM_WEAK_TABLE_KIND_VALUE); + else + assert (err == GC_SUCCESS); + } } static void -- 2.14.2 From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 0/7] Attempt to reduce memory growth Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Wed, 04 Oct 2017 13:10:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.150712256923121 (code B ref 28590); Wed, 04 Oct 2017 13:10:01 +0000 Received: (at 28590) by debbugs.gnu.org; 4 Oct 2017 13:09:29 +0000 Received: from localhost ([127.0.0.1]:47733 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzjQX-00060q-7i for submit@debbugs.gnu.org; Wed, 04 Oct 2017 09:09:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51965) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dzjQV-00060d-Cr for 28590@debbugs.gnu.org; Wed, 04 Oct 2017 09:09:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzjQO-0007hZ-KH for 28590@debbugs.gnu.org; Wed, 04 Oct 2017 09:09:22 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:56019) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzjQO-0007hF-Fn; Wed, 04 Oct 2017 09:09:20 -0400 Received: from [2a01:e0a:1d:7270:6a6c:dc17:fc02:cfda] (port=49612 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dzjQO-0006OY-0E; Wed, 04 Oct 2017 09:09:20 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <87poafkvsm.fsf@gnu.org> <20171003114352.13984-1-ludo@gnu.org> Date: Wed, 04 Oct 2017 15:09:17 +0200 In-Reply-To: <20171003114352.13984-1-ludo@gnu.org> ("Ludovic \=\?utf-8\?Q\?Cou\?\= \=\?utf-8\?Q\?rt\=C3\=A8s\=22's\?\= message of "Tue, 3 Oct 2017 13:43:45 +0200") Message-ID: <8760bv9i0y.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s skribis: > I=E2=80=99ll try to combine that with incremental marking of the weak tab= le, but > I=E2=80=99m not very hopeful. Here=E2=80=99s an attempt to mark tables incrementally. Unfortunately it misses references sometimes (e.g., values of a weak-key table do not get marked even though they should.) I=E2=80=99m unsure whether incremental marking is actually doable: the mark procedure only knows its own part of the mark state, not the global mark state. For instance, it cannot distinguish between different mark phases. I tried to fix that by memorizing the =E2=80=98GC_mark_no=E2=80=99= value we were in when we left the mark procedure, but that didn=E2=80=99t help. Ideas? Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/libguile/weak-table.c b/libguile/weak-table.c index 1a99a130f..cec9ee8ae 100644 --- a/libguile/weak-table.c +++ b/libguile/weak-table.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2011, 2012, 2013, 2014 Free Software Foundation, Inc. +/* Copyright (C) 2011, 2012, 2013, 2014, 2017 Free Software Foundation, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public License @@ -94,6 +94,15 @@ typedef struct { scm_t_bits value; } scm_t_weak_entry; +#define WEAK_MAGIC 0x9876123450UL + +struct weak_mark_state +{ + unsigned long magic; /* distinguished magic number */ + unsigned long index; /* index where to resume marking */ + unsigned long count; /* number of entries in the array */ + scm_t_weak_entry *entries; /* pointer to the array of entries */ +}; struct weak_entry_data { scm_t_weak_entry *in; @@ -252,6 +261,7 @@ typedef struct { unsigned long upper; /* when to grow */ int size_index; /* index into hashtable_size */ int min_size_index; /* minimum size_index */ + struct weak_mark_state *mark_state; } scm_t_weak_table; @@ -288,7 +298,11 @@ rob_from_rich (scm_t_weak_table *table, unsigned long k) /* If we are to free up slot K in the table, we need room to do so. */ assert (table->n_items < size); - + + /* To be on the safe side, have the next mark phase start from the + beginning. */ + table->mark_state->index = 0; + empty = k; do empty = (empty + 1) % size; @@ -318,6 +332,10 @@ give_to_poor (scm_t_weak_table *table, unsigned long k) /* Slot K was just freed up; possibly shuffle others down. */ unsigned long size = table->size; + /* To be on the safe side, have the next mark phase start from the + beginning. */ + table->mark_state->index = 0; + while (1) { unsigned long next = (k + 1) % size; @@ -354,49 +372,96 @@ give_to_poor (scm_t_weak_table *table, unsigned long k) static int weak_key_gc_kind; static int weak_value_gc_kind; +extern GC_word GC_mark_no; + static struct GC_ms_entry * mark_weak_key_table (GC_word *addr, struct GC_ms_entry *mark_stack_ptr, struct GC_ms_entry *mark_stack_limit, GC_word env) { - scm_t_weak_entry *entries = (scm_t_weak_entry*) addr; - unsigned long k, size = GC_size (addr) / sizeof (scm_t_weak_entry); - - for (k = 0; k < size; k++) + struct weak_mark_state *state = (struct weak_mark_state *) addr; + scm_t_weak_entry *entries = state->entries; + unsigned long k, size = state->count; + unsigned long credit = GC_PROC_BYTES / sizeof (GC_word); + + if (state->magic != WEAK_MAGIC) + /* STATE might point to a freed element. */ + return mark_stack_ptr; + + for (k = state->index; + credit > 0 && k < size; + k++) if (entries[k].hash && entries[k].key) { SCM value = SCM_PACK (entries[k].value); - mark_stack_ptr = GC_MARK_AND_PUSH ((GC_word*) SCM2PTR (value), + mark_stack_ptr = GC_MARK_AND_PUSH ((GC_word *) SCM2PTR (value), mark_stack_ptr, mark_stack_limit, NULL); + credit--; } + if (k == size) + /* We marked ENTRIES till the end. */ + state->index = 0; + else + { + /* Save the current index and push ADDR back on the mark stack so + we can resume later. */ + state->index = k; + mark_stack_ptr = GC_MARK_AND_PUSH (addr, + mark_stack_ptr, mark_stack_limit, + NULL); + } + return mark_stack_ptr; } static struct GC_ms_entry * mark_weak_value_table (GC_word *addr, struct GC_ms_entry *mark_stack_ptr, - struct GC_ms_entry *mark_stack_limit, GC_word env) + struct GC_ms_entry *mark_stack_limit, GC_word env) { - scm_t_weak_entry *entries = (scm_t_weak_entry*) addr; - unsigned long k, size = GC_size (addr) / sizeof (scm_t_weak_entry); - - for (k = 0; k < size; k++) + struct weak_mark_state *state = (struct weak_mark_state *) addr; + scm_t_weak_entry *entries = state->entries; + unsigned long k, size = state->count; + unsigned long credit = GC_PROC_BYTES / sizeof (GC_word); + + if (state->magic != WEAK_MAGIC) + /* STATE might point to a freed element. */ + return mark_stack_ptr; + + for (k = state->index; + credit > 0 && k < size; + k++) if (entries[k].hash && entries[k].value) { SCM key = SCM_PACK (entries[k].key); - mark_stack_ptr = GC_MARK_AND_PUSH ((GC_word*) SCM2PTR (key), + mark_stack_ptr = GC_MARK_AND_PUSH ((GC_word *) SCM2PTR (key), mark_stack_ptr, mark_stack_limit, NULL); + credit--; } + if (k == size) + /* We marked ENTRIES till the end. */ + state->index = 0; + else + { + /* Save the current index and push ADDR back on the mark stack so + we can resume later. */ + state->index = k; + mark_stack_ptr = GC_MARK_AND_PUSH (addr, + mark_stack_ptr, mark_stack_limit, + NULL); + } + return mark_stack_ptr; } -static scm_t_weak_entry * + +static struct weak_mark_state * allocate_entries (unsigned long size, scm_t_weak_table_kind kind) { - scm_t_weak_entry *ret; - size_t bytes = size * sizeof (*ret); + struct weak_mark_state *ret; + size_t bytes = size * sizeof (scm_t_weak_entry) + sizeof *ret; switch (kind) { @@ -414,6 +479,9 @@ allocate_entries (unsigned long size, scm_t_weak_table_kind kind) } memset (ret, 0, bytes); + ret->magic = WEAK_MAGIC; + ret->count = size; + ret->entries = (scm_t_weak_entry *) ((char *) ret + sizeof (*ret)); return ret; } @@ -533,6 +601,7 @@ update_entry_count (scm_t_weak_table *table) static void resize_table (scm_t_weak_table *table) { + struct weak_mark_state *mark_state, *old_state; scm_t_weak_entry *old_entries, *new_entries; int new_size_index; unsigned long old_size, new_size, old_k; @@ -554,11 +623,13 @@ resize_table (scm_t_weak_table *table) } while (!is_acceptable_size_index (table, new_size_index)); - new_entries = allocate_entries (new_size, table->kind); + mark_state = allocate_entries (new_size, table->kind); + new_entries = mark_state->entries; old_entries = table->entries; old_size = table->size; - + + table->mark_state = mark_state; table->size_index = new_size_index; table->size = new_size; if (new_size_index <= table->min_size_index) @@ -618,6 +689,8 @@ resize_table (scm_t_weak_table *table) SCM_PACK (copy.key), SCM_PACK (copy.value), table->kind); } + + old_state->magic = 0; } /* Run after GC via do_vacuum_weak_table, this function runs over the @@ -807,6 +880,9 @@ weak_table_remove_x (scm_t_weak_table *table, unsigned long hash, hash = (hash << 1) | 0x1; k = hash_to_index (hash, size); + /* To be on the safe side, mark the whole array of entries again. */ + table->mark_state->index = 0; + for (distance = 0; distance < size; distance++, k = (k + 1) % size) { unsigned long other_hash; @@ -869,7 +945,8 @@ make_weak_table (unsigned long k, scm_t_weak_table_kind kind) n = hashtable_size[i]; table = scm_gc_malloc (sizeof (*table), "weak-table"); - table->entries = allocate_entries (n, kind); + table->mark_state = allocate_entries (n, kind); + table->entries = table->mark_state->entries; table->kind = kind; table->n_items = 0; table->size = n; --=-=-=-- From unknown Mon Jun 23 13:15:27 2025 X-Loop: help-debbugs@gnu.org Subject: bug#28590: [PATCH 0/7] Attempt to reduce memory growth Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 05 Oct 2017 14:10:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 28590 X-GNU-PR-Package: guile X-GNU-PR-Keywords: To: wingo@igalia.com Cc: 28590@debbugs.gnu.org Received: via spool by 28590-submit@debbugs.gnu.org id=B28590.15072125594183 (code B ref 28590); Thu, 05 Oct 2017 14:10:01 +0000 Received: (at 28590) by debbugs.gnu.org; 5 Oct 2017 14:09:19 +0000 Received: from localhost ([127.0.0.1]:51653 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e06py-00015P-OV for submit@debbugs.gnu.org; Thu, 05 Oct 2017 10:09:18 -0400 Received: from eggs.gnu.org ([208.118.235.92]:40112) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e06pw-000158-Ky for 28590@debbugs.gnu.org; Thu, 05 Oct 2017 10:09:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e06pn-0007Tv-18 for 28590@debbugs.gnu.org; Thu, 05 Oct 2017 10:09:11 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50181) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e06pm-0007Tq-Tl; Thu, 05 Oct 2017 10:09:06 -0400 Received: from [193.50.110.164] (port=57560 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1e06pm-00048C-Ad; Thu, 05 Oct 2017 10:09:06 -0400 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) References: <87poafkvsm.fsf@gnu.org> <20171003114352.13984-1-ludo@gnu.org> <8760bv9i0y.fsf@gnu.org> Date: Thu, 05 Oct 2017 16:09:04 +0200 In-Reply-To: <8760bv9i0y.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Wed, 04 Oct 2017 15:09:17 +0200") Message-ID: <877ew9660v.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > Here=E2=80=99s an attempt to mark tables incrementally. > > Unfortunately it misses references sometimes (e.g., values of a weak-key > table do not get marked even though they should.) I=E2=80=99ve thrown a message in a bottle, with a reproducer in C: https://github.com/ivmai/bdwgc/issues/183 Ludo=E2=80=99. From unknown Mon Jun 23 13:15:27 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Subject: bug#28590: closed (Re: bug#28590: Weak tables in 2.2.2 grow indefinitely) Message-ID: References: <874lq69wy5.fsf@gnu.org> <87poafkvsm.fsf@gnu.org> X-Gnu-PR-Message: they-closed 28590 X-Gnu-PR-Package: guile Reply-To: 28590@debbugs.gnu.org Date: Tue, 07 Nov 2017 11:00:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1510052402-14467-1" This is a multi-part message in MIME format... ------------=_1510052402-14467-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #28590: Weak tables in 2.2.2 grow indefinitely which was filed against the guile package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 28590@debbugs.gnu.org. --=20 28590: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D28590 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1510052402-14467-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 28590-done) by debbugs.gnu.org; 7 Nov 2017 10:59:01 +0000 Received: from localhost ([127.0.0.1]:55233 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eC1av-0003jG-Bo for submit@debbugs.gnu.org; Tue, 07 Nov 2017 05:59:01 -0500 Received: from hera.aquilenet.fr ([141.255.128.1]:37520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eC1at-0003j5-R7 for 28590-done@debbugs.gnu.org; Tue, 07 Nov 2017 05:59:00 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id D4F54F6C1 for <28590-done@debbugs.gnu.org>; Tue, 7 Nov 2017 11:59:00 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5B2ssMmNCBTv for <28590-done@debbugs.gnu.org>; Tue, 7 Nov 2017 11:59:00 +0100 (CET) Received: from ribbon (unknown [193.50.110.150]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 15FB7F6B6 for <28590-done@debbugs.gnu.org>; Tue, 7 Nov 2017 11:59:00 +0100 (CET) From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: 28590-done@debbugs.gnu.org Subject: Re: bug#28590: Weak tables in 2.2.2 grow indefinitely References: <87poafkvsm.fsf@gnu.org> Date: Tue, 07 Nov 2017 11:58:58 +0100 In-Reply-To: <87poafkvsm.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 25 Sep 2017 10:49:13 +0200") Message-ID: <874lq69wy5.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 28590-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > Consider this program: > > (use-modules (ice-9 format)) > > (define loops 3000000) > > (define table > (make-weak-key-hash-table)) > > (let loop ((i loops)) > (unless #f ;(zero? i) > (when (zero? (modulo i 100000)) > (format #t "heap-size: ~,2h MiB table: ~s~%" > (/ (assoc-ref (gc-stats) 'heap-size) (expt 2. 20)) > table)) > > (hashq-set! table (cons 1 2) #t) > (loop (1- i)))) > > On 2.0.14, the heap size stays at around 24=C2=A0MiB, and the table size = is > stable at 224,717 buckets (?). > > On 2.2.2, the heap grows indefinitely (though logarithmically). It=E2=80= =99s > not deterministic though: sometimes the heap size stabilizes in the > 140=E2=80=93300=C2=A0MiB range, and sometimes it keeps growing endlessly = even though > the table size reaches a maxium at 7,190,537 entries. This is fixed with Andy=E2=80=99s rewrite of weak tables to bucket-and-chai= ns as in 2.0: https://git.savannah.gnu.org/cgit/guile.git/commit/?h=3Dstable-2.2&id=3Da= 053c0510c4a644f9453166b7b385cf30f6d3a21 https://git.savannah.gnu.org/cgit/guile.git/commit/?h=3Dstable-2.2&id=3Dd= 01addeb1feba830ddd703e27f89576864a063ff https://git.savannah.gnu.org/cgit/guile.git/commit/?h=3Dstable-2.2&id=3Dd= c8dda77e0c937abae42a76ea88c6e7995adbd9a https://lists.gnu.org/archive/html/guile-devel/2017-10/msg00051.html \o/ Ludo=E2=80=99. ------------=_1510052402-14467-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 25 Sep 2017 08:49:29 +0000 Received: from localhost ([127.0.0.1]:57246 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwP4y-0008BW-Vo for submit@debbugs.gnu.org; Mon, 25 Sep 2017 04:49:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:60520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwP4x-0008BJ-BN for submit@debbugs.gnu.org; Mon, 25 Sep 2017 04:49:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dwP4r-0006wL-F0 for submit@debbugs.gnu.org; Mon, 25 Sep 2017 04:49:22 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:33424) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dwP4r-0006wA-CY for submit@debbugs.gnu.org; Mon, 25 Sep 2017 04:49:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51456) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwP4q-0003Wu-AY for bug-guile@gnu.org; Mon, 25 Sep 2017 04:49:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dwP4n-0006u5-21 for bug-guile@gnu.org; Mon, 25 Sep 2017 04:49:20 -0400 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34829) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwP4m-0006tt-Vh for bug-guile@gnu.org; Mon, 25 Sep 2017 04:49:17 -0400 Received: from [193.50.110.164] (port=33444 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1dwP4m-0003vz-G1 for bug-guile@gnu.org; Mon, 25 Sep 2017 04:49:16 -0400 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: bug-guile@gnu.org Subject: Weak tables in 2.2.2 grow indefinitely X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 =?utf-8?Q?Vend=C3=A9miaire?= an 226 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 25 Sep 2017 10:49:13 +0200 Message-ID: <87poafkvsm.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.0 (-----) Consider this program: --8<---------------cut here---------------start------------->8--- (use-modules (ice-9 format)) (define loops 3000000) (define table (make-weak-key-hash-table)) (let loop ((i loops)) (unless #f ;(zero? i) (when (zero? (modulo i 100000)) (format #t "heap-size: ~,2h MiB table: ~s~%" (/ (assoc-ref (gc-stats) 'heap-size) (expt 2. 20)) table)) (hashq-set! table (cons 1 2) #t) (loop (1- i)))) --8<---------------cut here---------------end--------------->8--- On 2.0.14, the heap size stays at around 24=C2=A0MiB, and the table size is stable at 224,717 buckets (?). On 2.2.2, the heap grows indefinitely (though logarithmically). It=E2=80= =99s not deterministic though: sometimes the heap size stabilizes in the 140=E2=80=93300=C2=A0MiB range, and sometimes it keeps growing endlessly ev= en though the table size reaches a maxium at 7,190,537 entries. Ludo=E2=80=99. ------------=_1510052402-14467-1--