From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 24 05:59:27 2023 Received: (at submit) by debbugs.gnu.org; 24 Aug 2023 09:59:27 +0000 Received: from localhost ([127.0.0.1]:36159 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZ77e-0005sH-Df for submit@debbugs.gnu.org; Thu, 24 Aug 2023 05:59:27 -0400 Received: from lists.gnu.org ([2001:470:142::17]:45210) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZ77Z-0005s0-FR for submit@debbugs.gnu.org; Thu, 24 Aug 2023 05:59:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qZ77P-0005Fk-GL for bug-gnu-emacs@gnu.org; Thu, 24 Aug 2023 05:59:12 -0400 Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qZ77L-0002RH-CG for bug-gnu-emacs@gnu.org; Thu, 24 Aug 2023 05:59:11 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 0613E240028 for ; Thu, 24 Aug 2023 11:59:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1692871145; bh=CP5Sp1pfQxVr0rDgniFPDcYdXbp1yRqpqEhKtF4Duts=; h=From:To:Subject:Date:Message-ID:MIME-Version:From; b=sPLggslNPrkF1x0pYCc5x6zHcNzuTOVtpGXCisMHOENMVmfoaHS4jBsoMRdfYOy9S B44ymXJddBPJ+yOHsW5CwC8l430B1gUyXQP0tZzVtSUxKpkmbeeDfSxUt2niM/rJE+ 8qWHYhEkm05BLApsGWYudsvZWdl3DkimzV0Of4zV+aa5WwGidLkgZQD+XPSXDAJ1IE QFVkuvsEdrXFMVsotdm4RB2dfeH29glpw5WUOwiQYDWlM3ciIjGv+ZsIDuP8/Y5BLR 3mZF0iM2HvuBjIirx5IMIBsl9eN3pY9+pV4oZqKe1CchtQKPEW+cdhJbK6ZvUQZG5F 6JuCKqKDRypdQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4RWdpJ1XqLz6tyc for ; Thu, 24 Aug 2023 11:59:04 +0200 (CEST) From: Ihor Radchenko To: bug-gnu-emacs@gnu.org Subject: [PATCH] Improve performance allocating vectors Date: Thu, 24 Aug 2023 09:59:33 +0000 Message-ID: <87y1i0iwvu.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.65; envelope-from=yantar92@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Tags: patch Hi, Following up the bignum performance discussion in https://yhetil.org/emacs-devel/87bkfdsmde.fsf@localhost This patch adds a heuristic that reduces the time spent searching `vector_free_lists' when trying to allocate a new vector. `vector_free_lists' is a rather long array with few hundreds of elements. And it does not make sense to check the whole array in `allocate_vector_from_block' if we can get information about free vector that was recently made available of if we know for sure that no free vectors are available (after GC). In the patch, we start searching `vector_free_lists' no earlier than the last known index of the available free vector, or skip the search entirely when there is known index (after sweep). The described approach may sometimes miss free vectors in `vector_free_lists', especially if the allocation happens from larger vector to smaller. The cost will be slightly higher memory consumption - no larger than VECTOR_MAX_FREE_LIST_INDEX extra free vectors. With the patch, CPU time spent allocating new vectors in the fib.eln test from https://yhetil.org/emacs-devel/87bkfdsmde.fsf@localhost drops 10x. Also, the patch gives a noticeable improvement when running elisp-benchmarks (ELPA package). The overall speedup is around 10% (including unaffected tests). No single test gets worse within error margins and the following tests get a significant speedup: - eieio 1.32=C2=B10.04 -> 1.04=C2=B10.03 - pack-unpack 0.40=C2=B10.00 -> 0.35=C2=B10.01 - pidigits 6.00=C2=B10.06 -> 4.08=C2=B10.12 pidigits is no surprise as it likely uses bignums. * Results without the patch | test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s= ) | tot avg err (s) | |--------------------+----------------+------------+---------+-----------= --+-----------------| | bubble | 0.68 | 0.06 | 1 | 0.7= 3 | 0.05 | | bubble-no-cons | 1.17 | 0.00 | 0 | 1.1= 7 | 0.07 | | bytecomp | 1.64 | 0.32 | 13 | 1.9= 5 | 0.03 | | dhrystone | 2.13 | 0.00 | 0 | 2.1= 3 | 0.02 | | eieio | 1.19 | 0.13 | 7 | 1.3= 2 | 0.04 | | fibn | 0.00 | 0.00 | 0 | 0.0= 0 | 0.00 | | fibn-named-let | 1.47 | 0.00 | 0 | 1.4= 7 | 0.04 | | fibn-rec | 0.00 | 0.00 | 0 | 0.0= 0 | 0.00 | | fibn-tc | 0.00 | 0.00 | 0 | 0.0= 0 | 0.00 | | flet | 1.41 | 0.00 | 0 | 1.4= 1 | 0.03 | | inclist | 0.84 | 0.00 | 0 | 0.8= 4 | 0.03 | | inclist-type-hints | 0.76 | 0.00 | 0 | 0.7= 6 | 0.00 | | listlen-tc | 0.12 | 0.00 | 0 | 0.1= 2 | 0.01 | | map-closure | 5.25 | 0.00 | 0 | 5.2= 5 | 0.02 | | nbody | 1.47 | 0.15 | 1 | 1.6= 2 | 0.07 | | pack-unpack | 0.38 | 0.02 | 1 | 0.4= 0 | 0.00 | | pack-unpack-old | 1.13 | 0.05 | 3 | 1.1= 9 | 0.03 | | pcase | 1.77 | 0.00 | 0 | 1.7= 7 | 0.01 | | pidigits | 5.04 | 0.97 | 17 | 6.0= 0 | 0.06 | | scroll | 0.58 | 0.00 | 0 | 0.5= 8 | 0.02 | | smie | 1.47 | 0.05 | 2 | 1.5= 2 | 0.02 | |--------------------+----------------+------------+---------+-----------= --+-----------------| | total | 28.49 | 1.74 | 45 | 30.2= 3 | 0.16 | * Results with the patch | test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s= ) | tot avg err (s) | |--------------------+----------------+------------+---------+-----------= --+-----------------| | bubble | 0.69 | 0.05 | 1 | 0.7= 4 | 0.07 | | bubble-no-cons | 1.03 | 0.00 | 0 | 1.0= 3 | 0.02 | | bytecomp | 1.45 | 0.25 | 13 | 1.7= 0 | 0.10 | | dhrystone | 1.98 | 0.00 | 0 | 1.9= 8 | 0.05 | | eieio | 0.92 | 0.12 | 7 | 1.0= 4 | 0.03 | | fibn | 0.00 | 0.00 | 0 | 0.0= 0 | 0.00 | | fibn-named-let | 1.45 | 0.00 | 0 | 1.4= 5 | 0.12 | | fibn-rec | 0.00 | 0.00 | 0 | 0.0= 0 | 0.00 | | fibn-tc | 0.00 | 0.00 | 0 | 0.0= 0 | 0.00 | | flet | 1.39 | 0.00 | 0 | 1.3= 9 | 0.07 | | inclist | 0.83 | 0.00 | 0 | 0.8= 3 | 0.03 | | inclist-type-hints | 0.77 | 0.00 | 0 | 0.7= 7 | 0.03 | | listlen-tc | 0.10 | 0.00 | 0 | 0.1= 0 | 0.00 | | map-closure | 5.25 | 0.00 | 0 | 5.2= 5 | 0.37 | | nbody | 1.45 | 0.16 | 1 | 1.6= 0 | 0.05 | | pack-unpack | 0.33 | 0.02 | 1 | 0.3= 5 | 0.01 | | pack-unpack-old | 1.07 | 0.06 | 3 | 1.1= 3 | 0.08 | | pcase | 1.78 | 0.00 | 0 | 1.7= 8 | 0.05 | | pidigits | 3.15 | 0.92 | 17 | 4.0= 8 | 0.12 | | scroll | 0.55 | 0.00 | 0 | 0.5= 5 | 0.01 | | smie | 1.45 | 0.04 | 2 | 1.4= 9 | 0.06 | |--------------------+----------------+------------+---------+-----------= --+-----------------| | total | 25.63 | 1.62 | 45 | 27.2= 5 | 0.45 | In GNU Emacs 30.0.50 (build 54, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, cairo version 1.17.8) of 2023-08-22 built on localhost Repository revision: c09d78f3c0818d7391760e84f94a442e8beb22dd Repository branch: master Windowing system distributor 'The X.Org Foundation', version 11.0.12101008 System Description: Gentoo Linux Configured using: 'configure --with-native-compilation JAVAC=3D/etc/java-config-2/current-system-vm/bin/javac' --=-=-= Content-Type: text/patch Content-Disposition: attachment; filename=0001-Improve-performance-allocating-vectors.patch >From 476db65dc780d8c501d73dfaaf92c53eb9c73b2f Mon Sep 17 00:00:00 2001 Message-ID: <476db65dc780d8c501d73dfaaf92c53eb9c73b2f.1692869114.git.yantar92@posteo.net> From: Ihor Radchenko Date: Thu, 24 Aug 2023 11:05:18 +0300 Subject: [PATCH] Improve performance allocating vectors * src/alloc.c (last_known_vector_free_idx): New global variable holding the last known free allocated vector index in `vector_free_lists'. (setup_on_free_list): Record the vector index when adding a new allocated vector to `vector_free_lists'. (allocate_vector_from_block): Add heuristics searching for an existing allocated vector no earlier than `last_known_vector_free_idx'. This heuristics may sometimes skip available allocated vectors, but reduces the number of the `for' loop iterations significantly. (sweep_vectors): Set `last_known_vector_free_idx' to large value - no free allocated vectors remain available after sweep operation. Link: https://yhetil.org/emacs-devel/87bkfdsmde.fsf@localhost --- src/alloc.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/alloc.c b/src/alloc.c index c77bdc6372d..4293de2bdf3 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -3145,6 +3145,7 @@ large_vector_vec (struct large_vector *p) static struct Lisp_Vector *vector_free_lists[VECTOR_MAX_FREE_LIST_INDEX]; +static ptrdiff_t last_known_vector_free_idx = VECTOR_MAX_FREE_LIST_INDEX; /* Singly-linked list of large vectors. */ static struct large_vector *large_vectors; @@ -3180,6 +3181,7 @@ setup_on_free_list (struct Lisp_Vector *v, ptrdiff_t nbytes) set_next_vector (v, vector_free_lists[vindex]); ASAN_POISON_VECTOR_CONTENTS (v, nbytes - header_size); vector_free_lists[vindex] = v; + last_known_vector_free_idx = vindex; } /* Get a new vector block. */ @@ -3234,7 +3236,7 @@ allocate_vector_from_block (ptrdiff_t nbytes) /* Next, check free lists containing larger vectors. Since we will split the result, we should have remaining space large enough to use for one-slot vector at least. */ - for (index = VINDEX (nbytes + VBLOCK_BYTES_MIN); + for (index = max (VINDEX (nbytes + VBLOCK_BYTES_MIN), last_known_vector_free_idx); index < VECTOR_MAX_FREE_LIST_INDEX; index++) if (vector_free_lists[index]) { @@ -3426,6 +3428,7 @@ sweep_vectors (void) gcstat.total_vectors = 0; gcstat.total_vector_slots = gcstat.total_free_vector_slots = 0; memset (vector_free_lists, 0, sizeof (vector_free_lists)); + last_known_vector_free_idx = VECTOR_MAX_FREE_LIST_INDEX; /* Looking through vector blocks. */ -- 2.41.0 --=-=-= Content-Type: text/plain -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 03:14:15 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 07:14:15 +0000 Received: from localhost ([127.0.0.1]:41622 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZnUt-0002kV-4c for submit@debbugs.gnu.org; Sat, 26 Aug 2023 03:14:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40374) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZnUp-0002kG-Az for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 03:14:13 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qZnUe-000564-0y; Sat, 26 Aug 2023 03:14:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=aSPkoQSdPSeLxS2o9w5A3ewQn7Er8B2/f6z/fqtFf9w=; b=U/JfocdYoPtbX+op+T0I r6j1ntueQFV2MGT9cWWKgJmfdQCEPCyrz1L/WNjDPYK5QSWGsyfD9aj63+f4iYUg9UMjhEvbmmR89 wbCF3LdKuXIrXn0qwrbnfp3XabZZdIsqiDDkVGnGFMhsF2wShD6dAjS2cTGW9tHHxlpm3iOz8FOBg 2H1Wt78KdLTSX7KJ+QAHHzZs4DO82UJy4jeU+wlPQv6BUCeumfzRm+8ATmLjecbpT+R5xMKsdD+C9 486zWYxH0posXjZazO8hbMH3mENCnX0szj8thzoyHEb6bHq/5A7Hy/7lLDTQQX3vQWHSwXcWdXQFR QNw90HP1jKA4vw==; Date: Sat, 26 Aug 2023 10:14:28 +0300 Message-Id: <83bkeumg17.fsf@gnu.org> From: Eli Zaretskii To: Ihor Radchenko , Stefan Monnier In-Reply-To: <87y1i0iwvu.fsf@localhost> (message from Ihor Radchenko on Thu, 24 Aug 2023 09:59:33 +0000) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <87y1i0iwvu.fsf@localhost> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Ihor Radchenko > Date: Thu, 24 Aug 2023 09:59:33 +0000 > > Following up the bignum performance discussion in > https://yhetil.org/emacs-devel/87bkfdsmde.fsf@localhost > > This patch adds a heuristic that reduces the time spent searching > `vector_free_lists' when trying to allocate a new vector. > > `vector_free_lists' is a rather long array with few hundreds of > elements. And it does not make sense to check the whole array in > `allocate_vector_from_block' if we can get information about free vector > that was recently made available of if we know for sure that no free > vectors are available (after GC). > > In the patch, we start searching `vector_free_lists' no earlier than the > last known index of the available free vector, or skip the search > entirely when there is known index (after sweep). > > The described approach may sometimes miss free vectors in > `vector_free_lists', especially if the allocation happens from larger > vector to smaller. The cost will be slightly higher memory consumption - > no larger than VECTOR_MAX_FREE_LIST_INDEX extra free vectors. > > With the patch, CPU time spent allocating new vectors in the fib.eln > test from https://yhetil.org/emacs-devel/87bkfdsmde.fsf@localhost drops 10x. > > Also, the patch gives a noticeable improvement when running > elisp-benchmarks (ELPA package). The overall speedup is around 10% > (including unaffected tests). > > No single test gets worse within error margins and the following tests > get a significant speedup: > > - eieio 1.32±0.04 -> 1.04±0.03 > - pack-unpack 0.40±0.00 -> 0.35±0.01 > - pidigits 6.00±0.06 -> 4.08±0.12 > > pidigits is no surprise as it likely uses bignums. > > * Results without the patch > > | test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) | > |--------------------+----------------+------------+---------+-------------+-----------------| > | bubble | 0.68 | 0.06 | 1 | 0.73 | 0.05 | > | bubble-no-cons | 1.17 | 0.00 | 0 | 1.17 | 0.07 | > | bytecomp | 1.64 | 0.32 | 13 | 1.95 | 0.03 | > | dhrystone | 2.13 | 0.00 | 0 | 2.13 | 0.02 | > | eieio | 1.19 | 0.13 | 7 | 1.32 | 0.04 | > | fibn | 0.00 | 0.00 | 0 | 0.00 | 0.00 | > | fibn-named-let | 1.47 | 0.00 | 0 | 1.47 | 0.04 | > | fibn-rec | 0.00 | 0.00 | 0 | 0.00 | 0.00 | > | fibn-tc | 0.00 | 0.00 | 0 | 0.00 | 0.00 | > | flet | 1.41 | 0.00 | 0 | 1.41 | 0.03 | > | inclist | 0.84 | 0.00 | 0 | 0.84 | 0.03 | > | inclist-type-hints | 0.76 | 0.00 | 0 | 0.76 | 0.00 | > | listlen-tc | 0.12 | 0.00 | 0 | 0.12 | 0.01 | > | map-closure | 5.25 | 0.00 | 0 | 5.25 | 0.02 | > | nbody | 1.47 | 0.15 | 1 | 1.62 | 0.07 | > | pack-unpack | 0.38 | 0.02 | 1 | 0.40 | 0.00 | > | pack-unpack-old | 1.13 | 0.05 | 3 | 1.19 | 0.03 | > | pcase | 1.77 | 0.00 | 0 | 1.77 | 0.01 | > | pidigits | 5.04 | 0.97 | 17 | 6.00 | 0.06 | > | scroll | 0.58 | 0.00 | 0 | 0.58 | 0.02 | > | smie | 1.47 | 0.05 | 2 | 1.52 | 0.02 | > |--------------------+----------------+------------+---------+-------------+-----------------| > | total | 28.49 | 1.74 | 45 | 30.23 | 0.16 | > > * Results with the patch > > | test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) | > |--------------------+----------------+------------+---------+-------------+-----------------| > | bubble | 0.69 | 0.05 | 1 | 0.74 | 0.07 | > | bubble-no-cons | 1.03 | 0.00 | 0 | 1.03 | 0.02 | > | bytecomp | 1.45 | 0.25 | 13 | 1.70 | 0.10 | > | dhrystone | 1.98 | 0.00 | 0 | 1.98 | 0.05 | > | eieio | 0.92 | 0.12 | 7 | 1.04 | 0.03 | > | fibn | 0.00 | 0.00 | 0 | 0.00 | 0.00 | > | fibn-named-let | 1.45 | 0.00 | 0 | 1.45 | 0.12 | > | fibn-rec | 0.00 | 0.00 | 0 | 0.00 | 0.00 | > | fibn-tc | 0.00 | 0.00 | 0 | 0.00 | 0.00 | > | flet | 1.39 | 0.00 | 0 | 1.39 | 0.07 | > | inclist | 0.83 | 0.00 | 0 | 0.83 | 0.03 | > | inclist-type-hints | 0.77 | 0.00 | 0 | 0.77 | 0.03 | > | listlen-tc | 0.10 | 0.00 | 0 | 0.10 | 0.00 | > | map-closure | 5.25 | 0.00 | 0 | 5.25 | 0.37 | > | nbody | 1.45 | 0.16 | 1 | 1.60 | 0.05 | > | pack-unpack | 0.33 | 0.02 | 1 | 0.35 | 0.01 | > | pack-unpack-old | 1.07 | 0.06 | 3 | 1.13 | 0.08 | > | pcase | 1.78 | 0.00 | 0 | 1.78 | 0.05 | > | pidigits | 3.15 | 0.92 | 17 | 4.08 | 0.12 | > | scroll | 0.55 | 0.00 | 0 | 0.55 | 0.01 | > | smie | 1.45 | 0.04 | 2 | 1.49 | 0.06 | > |--------------------+----------------+------------+---------+-------------+-----------------| > | total | 25.63 | 1.62 | 45 | 27.25 | 0.45 | > Stefan, any comments? my comment is that the savings are quite small, so it seems, so I'm not sure we should install this. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 03:27:13 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 07:27:13 +0000 Received: from localhost ([127.0.0.1]:41637 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZnhQ-00034k-Nf for submit@debbugs.gnu.org; Sat, 26 Aug 2023 03:27:13 -0400 Received: from mout02.posteo.de ([185.67.36.66]:35533) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZnhM-000346-Vy for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 03:27:12 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 1F3D3240101 for <65491@debbugs.gnu.org>; Sat, 26 Aug 2023 09:26:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1693034818; bh=j+gTQ9dTbV2E72R7hD76QGr/SVtxtSPVNCKL3TU9tTU=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=VQVJdI4ZBRIoOPivj4jnmVPzGC1HF5+QXWbT1yK9ax39nwM8xSl712DR6gUfrGqqj Ygtg9/YK/cHRH5gzdZJFMywbtXJ2yvRXPh8wqJaZ4ctutAW1GMwqoTmO/6Qwn/HPtj EgnJWi8JLxYTkm1xCol2bZJxxrTN5vR2k2ZV855fPt5Ggi0r+yER8I2GJOqlA5W03P y1hQJPDRRLvs6JM+6W1FK0dkxGuTsGntn4ieEgRwX3CabvricLqI8bC2Rl7mROoaNv x5QlJTeOjexVf3Y1YAWakVGppeO21J6xcutn8mDj8OTqTlMOCQ0x6gtCyhcqwazrlg LRhqhGGzhJLaw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4RXpKr6QVxz6twF; Sat, 26 Aug 2023 09:26:56 +0200 (CEST) From: Ihor Radchenko To: Eli Zaretskii Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <83bkeumg17.fsf@gnu.org> References: <87y1i0iwvu.fsf@localhost> <83bkeumg17.fsf@gnu.org> Date: Sat, 26 Aug 2023 07:27:27 +0000 Message-ID: <87ttsm9sbk.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: > my comment is that the savings are quite small, so it seems, so I'm > not sure we should install this. Overall saving are small because the effect is blurred by the code that does not allocate vectors - my patch has 0 effect on such code. Calculations involving vectors are pidigits that is 30% faster and bignum version of fib.el - over 50% faster. If you know any useful code that makes heavy use of vector allocation, I can also benchmark it. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 03:31:35 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 07:31:35 +0000 Received: from localhost ([127.0.0.1]:41655 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZnlf-0003De-DZ for submit@debbugs.gnu.org; Sat, 26 Aug 2023 03:31:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41794) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZnld-0003DR-OZ for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 03:31:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qZnlT-0008Cx-C7; Sat, 26 Aug 2023 03:31:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=RxjfNs9jJZ41ehbpL2sxBMxl2LGNXQY+k4aLjwFntWY=; b=ejjYfUWJy0Wy Fyi33nHAT7TyZOnyGOSOAhP0z9qayXIcdxCvvpws+GPOL7GG6iCBFjY7CqDoDBv163AXip73DouJ7 qYbVnzyWdyJBcciOxEye+YJN3jOfBKPketSTzXk+Ho8LPP3boqOiShf5F1MMi0RtIM/oI19r1n1Xc PGqPS1BsLjSQoFIDq/xyYZ4guGX9J3yQhBZ+IBq3pFb+UDDESSNp+QK/yXsglzy5easE9UKBznTz5 6jNFzhbR1pwDF0kbjV/XX3O72k+jtAHioNVgOwO0fYKYh8JJyh25ygomus1gw0sKu25sqwBVdPu46 pU1pQFebAhsDAgrj0sVPOQ==; Date: Sat, 26 Aug 2023 10:31:52 +0300 Message-Id: <837cpimf87.fsf@gnu.org> From: Eli Zaretskii To: Ihor Radchenko In-Reply-To: <87ttsm9sbk.fsf@localhost> (message from Ihor Radchenko on Sat, 26 Aug 2023 07:27:27 +0000) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <87y1i0iwvu.fsf@localhost> <83bkeumg17.fsf@gnu.org> <87ttsm9sbk.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Ihor Radchenko > Cc: Stefan Monnier , 65491@debbugs.gnu.org > Date: Sat, 26 Aug 2023 07:27:27 +0000 > > If you know any useful code that makes heavy use of vector allocation, I > can also benchmark it. Look in the area of encoding/decoding and automatic compositions -- these tend to use vectors quite a lot. For example, rendering text that uses a script where most of characters are composed, such as Arabic or Hangul (Korean) should allocate vectors. Note that this is the case where we call Lisp from C display code. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 03:47:22 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 07:47:22 +0000 Received: from localhost ([127.0.0.1]:41659 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZo0v-0003Zt-0D for submit@debbugs.gnu.org; Sat, 26 Aug 2023 03:47:22 -0400 Received: from mout01.posteo.de ([185.67.36.65]:47651) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZo0q-0003Zd-12 for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 03:47:19 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id B2885240028 for <65491@debbugs.gnu.org>; Sat, 26 Aug 2023 09:47:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1693036024; bh=ldBFpx03w3NCpZMFhP9hdFXOss02ObsT0JjzIf/Lfgk=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=OLg6efkjUoiTM1YVf7lW6igOjVABAOauWl8dNPZuSeSymA8Gm1fzpWRB80rR0eomE P60ZwfvlcHrM0P4vWRlE/RxzxkXNLueHyQzMky0PbP6yCwTFX+xYjqt7YnLV2CH50J 08yJwarv+YWxR+4DpP8brHSdhX1g9H/6p1w8OcNR5MSqcYjRKLSF15sJDshvDO4Cc+ nT/IUDExiwXaRlIf5Oq1DPaeA3gfID0y3G+aH++TB6NzWhd60WzLun5EeiOgsT7wTG RynZ2yNWlu2fCOtFiG5Rdw+mR1wLJwaCeI2TYH0U/tQsiO8FyAB9BB8qQI7kjOlRpN rXoFRgUoWeIIQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4RXpn36zH5z9s4T; Sat, 26 Aug 2023 09:47:03 +0200 (CEST) From: Ihor Radchenko To: Eli Zaretskii Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <87ttsm9sbk.fsf@localhost> References: <87y1i0iwvu.fsf@localhost> <83bkeumg17.fsf@gnu.org> <87ttsm9sbk.fsf@localhost> Date: Sat, 26 Aug 2023 07:47:34 +0000 Message-ID: <87r0nq9re1.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ihor Radchenko writes: > If you know any useful code that makes heavy use of vector allocation, I > can also benchmark it. One example is the newer versions of Org parser, where I know for sure that vector allocation is used. Although the code is mostly doing regexp search for parsing, so a better real world example would be still useful. I ran a quick parser run on a large Org file using (let ((large-file-warning-threshold nil) (gc-cons-threshold most-positive-fixnum) (org-inhibit-startup t) (org-element-cache-persistent nil)) (with-current-buffer (find-file-noselect "~/Org/notes.org") (message "%s" (benchmark-call (lambda () (org-element-parse-buffer nil nil 'defer)))))) Without the patch: $ perf record ./src/emacs -Q -batch -L ~/Git/org-mode/lisp -L ~/.emacs.d/eln-cache/30.0.50-17ebec90/ -l org-element -l /tmp/test.el (15.724130802000001 1 0.429624333) without the patch: 26.60% emacs emacs [.] exec_byte_code 12.40% emacs emacs [.] re_match_2_internal 8.70% emacs emacs [.] re_search_2 8.19% emacs emacs [.] re_compile_pattern 3.73% emacs emacs [.] re_iswctype 3.60% emacs emacs [.] funcall_subr 3.08% emacs emacs [.] allocate_vectorlike 2.70% emacs emacs [.] plist_get 1.80% emacs emacs [.] buf_charpos_to_bytepos Vector allocation takes about 3% of the time. ----------- with the patch, vector allocation contributions drops to 0.42% - 6x decrease. ----------- $ perf record ./src/emacs -Q -batch -L ~/Git/org-mode/lisp -L ~/.emacs.d/eln-cache/30.0.50-17ebec90/ -l org-element -l /tmp/test.el (15.100695088 1 0.43508134400000004) 27.06% emacs emacs [.] exec_byte_code 12.96% emacs emacs [.] re_match_2_internal 8.90% emacs emacs [.] re_search_2 8.55% emacs emacs [.] re_compile_pattern 3.68% emacs emacs [.] re_iswctype 3.57% emacs emacs [.] funcall_subr 2.80% emacs emacs [.] plist_get ... 0.42% emacs emacs [.] allocate_vectorlike -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 03:51:34 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 07:51:34 +0000 Received: from localhost ([127.0.0.1]:41664 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZo50-0003kx-5v for submit@debbugs.gnu.org; Sat, 26 Aug 2023 03:51:34 -0400 Received: from mout02.posteo.de ([185.67.36.66]:46093) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZo4x-0003kk-Qs for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 03:51:32 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id E8EA9240101 for <65491@debbugs.gnu.org>; Sat, 26 Aug 2023 09:51:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1693036280; bh=RI27QKoby/yr/wU2uXsfPf0ywmLeWBrUMEWisRcjZRQ=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=OD+iGo7jRLcqhGOOKxyLqfmTgFz/CxLaGFOkN8sANNzRGWFWqGyscL8NBPTY1HlYd i0oJvMfpH8N+eRIiJJ3i7SRbCWjq89UqS0bO2rISLo2TZm3qeJQFtcr90Zx+k0gZou cZh/dSXNgo3g/LG7RYzlg2kdkZ1DDr2SO8VGASFEY14f7oELHY/OMMFvBkUeTE1VtP nXFwYf6uuXSCwUIvB/PwQC0FKV6VEaP561eEs+qeU2pA0Wau/d6vG+b9qoMeydkFgU Cn6gUAgB0Ea8OOOA08TsIbbDQOujrVKBhl1UZH95WSF8F0YJZ4bvOB5PGoUl0HnrPH QSleYzn4e+Vkg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4RXpt01w3bz9s4W; Sat, 26 Aug 2023 09:51:20 +0200 (CEST) From: Ihor Radchenko To: Eli Zaretskii Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <837cpimf87.fsf@gnu.org> References: <87y1i0iwvu.fsf@localhost> <83bkeumg17.fsf@gnu.org> <87ttsm9sbk.fsf@localhost> <837cpimf87.fsf@gnu.org> Date: Sat, 26 Aug 2023 07:51:50 +0000 Message-ID: <87o7iu9r6x.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: >> If you know any useful code that makes heavy use of vector allocation, I >> can also benchmark it. > > Look in the area of encoding/decoding and automatic compositions -- > these tend to use vectors quite a lot. For example, rendering text > that uses a script where most of characters are composed, such as > Arabic or Hangul (Korean) should allocate vectors. Note that this is > the case where we call Lisp from C display code. Do you mean something like scrolling performance when scrolling a large Arabic/Korean text file? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 04:07:13 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 08:07:13 +0000 Received: from localhost ([127.0.0.1]:41701 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZoK9-0004Bn-31 for submit@debbugs.gnu.org; Sat, 26 Aug 2023 04:07:13 -0400 Received: from mout01.posteo.de ([185.67.36.65]:58271) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZoK6-0004BX-Uk for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 04:07:12 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 3F02524002B for <65491@debbugs.gnu.org>; Sat, 26 Aug 2023 10:07:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1693037220; bh=kWpydbsLfe1lf0vlHuCZfzhZIHiHGhDfqakuhHZuyqM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=T5TAYwQ9BZg19CoJsmPW6TRBlchi5cQKo06yXPvX981jlFBnLNSmOxdlPoEqBfzwx BDt8mgntql9s8JnssT5IsKzVV3jPmj0U9wFyuFUStUhM0rd5z+DRc8piC7pXiYK5SC jvHC5vpTB2GcA32wK4YPGiKn9eLul3zzys+HfueGdN8CtQTB8J8pxQoQ6z9ZfzpIhz 78KFisx+wAiU81B0dC5rlb0HtZvx6aS/6sBg+s3bLyGHtEOjGS3m9Ku8h7iBGXYfGC kVCuC1MF6ofAZ3bHRlEl+K3Bx6cfvt/BVsTXTlEeeXL5i+ZW4j7a5inlyxG+OARJHk C/SH4BbXfNYcA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4RXqD31YZ9z9rxF; Sat, 26 Aug 2023 10:06:59 +0200 (CEST) From: Ihor Radchenko To: Eli Zaretskii Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <87o7iu9r6x.fsf@localhost> References: <87y1i0iwvu.fsf@localhost> <83bkeumg17.fsf@gnu.org> <87ttsm9sbk.fsf@localhost> <837cpimf87.fsf@gnu.org> <87o7iu9r6x.fsf@localhost> Date: Sat, 26 Aug 2023 08:07:29 +0000 Message-ID: <871qfquszi.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.1 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.1 (---) Ihor Radchenko writes: > Do you mean something like scrolling performance when scrolling a large > Arabic/Korean text file? I downloaded Quran from https://tanzil.net/download/, opened it Emacs like $ perf record ./src/emacs -Q ~/Downloads/quran-simple-plain.txt and scrolled the buffer with PgDown all the way to the bottom. It does not look like allocation vectors is all that heavy here (compared to the edge case with fib.el with bignums). without the patch: 10.14% emacs emacs [.] process_mark_stack 3.79% emacs emacs [.] sub_char_table_ref 3.68% emacs emacs [.] x_draw_glyph_string 2.75% emacs libc.so.6 [.] pthread_mutex_lock 1.75% emacs libc.so.6 [.] 0x0000000000088b0a -> 1.41% emacs emacs [.] allocate_vectorlike 1.30% emacs emacs [.] sweep_vectors 1.27% emacs emacs [.] char_table_ref with the patch 9.97% emacs emacs [.] process_mark_stack 4.30% emacs emacs [.] x_draw_glyph_string 3.58% emacs emacs [.] sub_char_table_ref 2.66% emacs libc.so.6 [.] pthread_mutex_lock 1.87% emacs libc.so.6 [.] 0x0000000000088b0a 1.29% emacs emacs [.] sweep_vectors 1.22% emacs emacs [.] face_for_font 1.17% emacs emacs [.] char_table_ref ... -> 0.26% emacs emacs [.] allocate_vectorlike 1.41% down to 0.26% ~5x speedup. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 05:01:37 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 09:01:37 +0000 Received: from localhost ([127.0.0.1]:41737 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZpAn-0005mf-D0 for submit@debbugs.gnu.org; Sat, 26 Aug 2023 05:01:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58690) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZpAk-0005mK-5I for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 05:01:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qZpAY-0005iV-TZ; Sat, 26 Aug 2023 05:01:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=JHgVi5lUXbVO2oZ19xl+kxStiU5VqD5xZnMVv+1xkNM=; b=dGmqlgox3MUw RZw7QKQWGdW/AJrbd5cfuTe4aJSY2yNywJ9sOL30Cn56i+yb8EKIvvvEjs01nHk5d3jWtaPrQWTy0 3DPis0YGTgZFFek825Zh+XEOEp6rsYoPUaE6WP2XLWWdJPYmYp+938gtLZb8BYazrmL6pVRuJfWp9 9Y+NLHm+Me3GkbEmJDCEGvFO746X2cL8tnrjDXhSmoVrNURFGKvi9o+kP3KpIpMg8kB8y6AxTdFp9 B6dPcO9ouZVwoX4AC0MRtfsqjZDuQJ8YzSDB6I5EgeUeIaS23j11NNPXVIWbQ4JYe3sS/VVitlvvn 5w6EveZUr7XfKl/F9zHE4w==; Date: Sat, 26 Aug 2023 12:01:50 +0300 Message-Id: <83ttsmkwht.fsf@gnu.org> From: Eli Zaretskii To: Ihor Radchenko In-Reply-To: <87o7iu9r6x.fsf@localhost> (message from Ihor Radchenko on Sat, 26 Aug 2023 07:51:50 +0000) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <87y1i0iwvu.fsf@localhost> <83bkeumg17.fsf@gnu.org> <87ttsm9sbk.fsf@localhost> <837cpimf87.fsf@gnu.org> <87o7iu9r6x.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Ihor Radchenko > Cc: monnier@iro.umontreal.ca, 65491@debbugs.gnu.org > Date: Sat, 26 Aug 2023 07:51:50 +0000 > > Eli Zaretskii writes: > > >> If you know any useful code that makes heavy use of vector allocation, I > >> can also benchmark it. > > > > Look in the area of encoding/decoding and automatic compositions -- > > these tend to use vectors quite a lot. For example, rendering text > > that uses a script where most of characters are composed, such as > > Arabic or Hangul (Korean) should allocate vectors. Note that this is > > the case where we call Lisp from C display code. > > Do you mean something like scrolling performance when scrolling a large > Arabic/Korean text file? Yes. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 08:02:05 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 12:02:05 +0000 Received: from localhost ([127.0.0.1]:41822 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZrzQ-0004xC-PZ for submit@debbugs.gnu.org; Sat, 26 Aug 2023 08:02:05 -0400 Received: from mail-lf1-x135.google.com ([2a00:1450:4864:20::135]:44299) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZrzM-0004wc-DD for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 08:02:02 -0400 Received: by mail-lf1-x135.google.com with SMTP id 2adb3069b0e04-4fe27849e6aso2660991e87.1 for <65491@debbugs.gnu.org>; Sat, 26 Aug 2023 05:01:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693051309; x=1693656109; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=Om4K9CZlcyhtR9Rq4wlSQatF1DHKs7DPdgEbHn2VuYE=; b=CNBQ1PqM+YFspaf6S+ASpP2vPSPqkgzZxFf/QyB2DSSYTJOvh2cha6qz4/z1CCVhQw ZGZd5btBd9Bc0eXiVmzAyE2H/3OIqkDXCQT2EbQcWpAxxvYNDe8TU66kYhlBwIVDAD7O q1XMrIllwQXECTeHJDgcxsCisOFGihZgwu7NeQCNmBQW+MP2oN/8CzTQ1ITzfB65ZTAw YNC7LK5ctiIX1xC/FobX0AlI38dR9O/p+1jM33/+0V9hLUzd+/UpnZW4yyJiiFSyFPhh yS+Z0D089Fg8V/NI0Fge4S7vnSSuzoO1dP15HEn+XSY4/Lq+hJnf5sc0c2obXMjgBo2f i2kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693051309; x=1693656109; h=to:cc:date:message-id:subject:mime-version :content-transfer-encoding:from:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Om4K9CZlcyhtR9Rq4wlSQatF1DHKs7DPdgEbHn2VuYE=; b=lahycCu61j1/kR3b2zs1HDpm6b1u0a2VnhcWDpmbBNhvn9dMJWmdhFcJRp1sEnFFVd AlB53UVmyyHS+T/P8n+GlFvKnku8Rl2KY90uLzNEvrLJlFRnFz0yV5XKyGJfewW4L7wS WjttpWThbjaNACJejQdyhzl/5NlvU9fbL4MccPvgGJHNEuMWmvOHmLEQK+zURp5Hr01R oIZrrBhDdStW2bjd8gri4uDd09QD7DkifnUlOO8kKYlSqi1eBK7nyTGrGl7QFgZv7P3o 0kQGSfGS6+sWSD3TjRlMk+onQRtxd2qfMgbBIq9x5B95bGzDtWKY5Gm/d1tt8bbBuiXw ifNQ== X-Gm-Message-State: AOJu0Yx3ugfcb8uH9rYC2yAaPouEiUaeSuUwOVNKpHG7Vg8l6IW15FWs 6lBR+hlOOMdBa0DdPbDxkHKF86rdovc= X-Google-Smtp-Source: AGHT+IFCyeuwtvCtxgN74apJ8n1evDPvcj9vVR9SPlyxacrsCYdSy/Qrf4TN3U1wY2P3y29gc0x/CQ== X-Received: by 2002:a05:6512:b1f:b0:4fc:3755:37d9 with SMTP id w31-20020a0565120b1f00b004fc375537d9mr16927028lfu.68.1693051309069; Sat, 26 Aug 2023 05:01:49 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id z8-20020ac24188000000b004ff8e245260sm688761lfh.15.2023.08.26.05.01.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 26 Aug 2023 05:01:48 -0700 (PDT) From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: bug#65491: [PATCH] Improve performance allocating vectors Message-Id: Date: Sat, 26 Aug 2023 14:01:48 +0200 To: 65491@debbugs.gnu.org X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: Eli Zaretskii , Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) First of all, let's leave bignums out of this entirely. They are not = relevant here. Second, please do not motivate any perceived performance problem from = benchmarks found on the internet, especially anything derived from the = Gabriel benchmarks. This includes the benchmarks in ELPA. That said, vectorlike object allocation in general is definitely = relevant and can certainly be improved but I'm not persuaded by the = proposed patch. Please do not apply it right away. However, the important part is not the patch but the problem it = highlights, and here there is evidently plenty to do.=20 For example: - isn't vector_free_list twice as big as it needs to be? - 4096 bytes for vector blocks seems a tad small - to what extent are we duplicating the work done by modern libc = allocators (very generously including glibc here)? - not happy with the way struct vector_block co-allocates metadata with = a big power-of-two data chunk - next_vector is a dangerously unstable concoction of C undefined = behaviour and so on. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 10:54:49 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 14:54:49 +0000 Received: from localhost ([127.0.0.1]:43294 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZuga-0001FS-KI for submit@debbugs.gnu.org; Sat, 26 Aug 2023 10:54:48 -0400 Received: from mout02.posteo.de ([185.67.36.66]:38315) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZugW-0001FB-Mc for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 10:54:47 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 954C8240104 for <65491@debbugs.gnu.org>; Sat, 26 Aug 2023 16:54:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1693061673; bh=bcERCLW0G9xl9+b84Ba/w7MaXonLew1ubKmRg7Vi1I8=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version: Content-Transfer-Encoding:From; b=MZ/fRmpG53OQNvVY1OKl7AWoaVp1j9vL4Fg1hswRTylXOfAZ13RiKozBYEozV898Y nKG/lxHYJQmFPb+Uy0gdTjEAIaxJlj6tmdqnKgYo9ErSzG9YA+DAK3853W31B109SH UDki8vAyUNprl5ZFKF4oYNx3+vyC/48zIZK70Juhx4JEFuc63g/9yi9g8cA17845fy WYOrno+cAr42nMl7vmtubitPcayffICHZ3s1s1MxuNDcDqvgSOmmLY8XQiLZ5EtBPm bZTMa1nzYy4XyximKyiHHwsDMTRRfi96nbiniSCj3UcGaGZJyZIvNgDhhReLNAcz7n a6o4GENuFVIng== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4RY0GC2VY4z6txf; Sat, 26 Aug 2023 16:54:27 +0200 (CEST) From: Ihor Radchenko To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: References: <87y1i0iwvu.fsf@localhost> Date: Sat, 26 Aug 2023 14:54:55 +0000 Message-ID: <87y1hxj1kw.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Eli Zaretskii , Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Mattias Engdeg=C3=A5rd writes: > First of all, let's leave bignums out of this entirely. They are not rele= vant here. > > Second, please do not motivate any perceived performance problem from > benchmarks found on the internet, especially anything derived from the > Gabriel benchmarks. This includes the benchmarks in ELPA. > That said, vectorlike object allocation in general is definitely > relevant and can certainly be improved but I'm not persuaded by the > proposed patch. Please do not apply it right away. I have no problem with this and I have supplied more relevant benchmarks with Org and with composition (as suggested by Eli). Of course, the problem is not with bignums - it just revealed the inefficiency with array iteration I tried to address. If more can be done with vectorlike allocation, it will be even better. > However, the important part is not the patch but the problem it highlight= s, and here there is evidently plenty to do.=20 > > For example: > > - isn't vector_free_list twice as big as it needs to be? AFAIR, trying to reduce this array size was the first thing I tried. When I touched VECTOR_BLOCK_SIZE, I got segfaults and compilation failures. (Do note that I am missing understanding about the motivation behind this constant). > - to what extent are we duplicating the work done by modern libc allocato= rs (very generously including glibc here)? > - next_vector is a dangerously unstable concoction of C undefined behavio= ur Isn't vector_free_list following the pattern used across alloc.c? For example, Fcons uses a similar idea with holding pre-allocated memory as a chain of pointers. --=20 Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 26 10:56:14 2023 Received: (at 65491) by debbugs.gnu.org; 26 Aug 2023 14:56:14 +0000 Received: from localhost ([127.0.0.1]:43299 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZuhy-0001Hy-4H for submit@debbugs.gnu.org; Sat, 26 Aug 2023 10:56:14 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:63451) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qZuhs-0001Hg-Em for 65491@debbugs.gnu.org; Sat, 26 Aug 2023 10:56:12 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id CAA2C440B91; Sat, 26 Aug 2023 10:55:57 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1693061756; bh=+YSqNtRlHrYVt2ifV2ARwLRSyNgCV05DM+OewdkrmXY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=GbFqDO8l+wOpMwfcKDsZ40oZePvW0r6UmD3aeCse/h8kIKvP006Mg2RNE2A0nXJE9 O1Kw967nD0OQMnkbEJxwoJEjOKNjS5sdsUTvj7Cv2smUAVn+jXpvQ/SP1xha718ccZ NPeeH4FaWnm/7BPXK/s3htANUCmZdR1haIMVsZTzDfOzNscYk/SJdO3m/BUlPN8ZqF lelCYgu1JgV76tz2QeiJXRTytj9+LN0tviiL22UHvtlF9NNjrdq+VgLVml30XBa71i l/CFQAUObxsqQOZQGKzxldbzDaLvoCW0Qp29/tq2gJQKNSSuu5jhZfNI118o9lOu0S U0xBvL0sq28wA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 77D6B440B8A; Sat, 26 Aug 2023 10:55:56 -0400 (EDT) Received: from pastel (unknown [108.175.234.188]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 51E8E1200D7; Sat, 26 Aug 2023 10:55:56 -0400 (EDT) From: Stefan Monnier To: Mattias =?windows-1252?Q?Engdeg=E5rd?= Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: ("Mattias =?windows-1252?Q?Engdeg=E5rd=22's?= message of "Sat, 26 Aug 2023 14:01:48 +0200") Message-ID: References: Date: Sat, 26 Aug 2023 10:55:55 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.064 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > - isn't vector_free_list twice as big as it needs to be? > - 4096 bytes for vector blocks seems a tad small AFAIK this number was chosen arbitrarily and we never tried to tune it, so there's probably room for improvement, indeed. Note that many vectors will be small (they're really Lisp structs), tho. > - to what extent are we duplicating the work done by modern libc allocators > (very generously including glibc here)? We are exactly duplicating that code (and poorly so, as is the rule), but there's a reason for it: we need to allocate those vectors within specifically marked/known address ranges so that our conservative stack scanning can distinguish "potential valid pointer to a vector" from "definitely not a pointer to a vector". It would be nice if we could more or less literally copy the code of a good malloc library. [ The previous vector allocation code was even more costly because every vector allocation came together with allocation and insertion of a new node in the red-black tree where we keep track of which ranges of memory hold which kind of data. ] It's important to improve efficiency of vector allocation because it allows us to get rid of specialized allocations we're using for other Lisp types. E.g. we got rid of the specialized `marker_block`s we used to use for markers and overlays (and a handful of more esoteric data types), and next in line are the `symbol_block`s and `interval_block`s. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 27 05:54:58 2023 Received: (at 65491) by debbugs.gnu.org; 27 Aug 2023 09:54:58 +0000 Received: from localhost ([127.0.0.1]:44194 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qaCTy-0005xu-78 for submit@debbugs.gnu.org; Sun, 27 Aug 2023 05:54:58 -0400 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]:59413) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qaCTs-0005xc-5r for 65491@debbugs.gnu.org; Sun, 27 Aug 2023 05:54:56 -0400 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-500913779f5so3577043e87.2 for <65491@debbugs.gnu.org>; Sun, 27 Aug 2023 02:54:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693130080; x=1693734880; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=LfKs1F/+u11tAlFmIjUGcdagJ5EfJ15HA44vvPBQ8tM=; b=WiwnTJSGOp2kbWrKYixQmAYhtWSZ1Y/Qa1PEh907YOHbuLa2/soYRJz60bfvnE9MOz ZaSoDKxC3c1OlIEj1YZTylo85LBJil8PtCo+m31v1T67tHRUwb73XdAMkgrImkNtmD6C wLhZTK4Yh2Kpa5blZV88ewJinwZ15ybZYigNLI7dDGztHHBl2VVi37+oo4sjOsDblN6J T6gIH4LsPF2Ug9/Sf8qUG66t7qybQOVScH8THlRdJR4CoexYRm4bTbTWNcqFsYwqLL5x uW9+lfNDoJ8oO0aK4sUMh2YuDgj6mLWzsn95ZeSNrcEWA36NgqoQFWNI9haM47qIyCPI pQ3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693130080; x=1693734880; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=LfKs1F/+u11tAlFmIjUGcdagJ5EfJ15HA44vvPBQ8tM=; b=I2GK2J/MUfNknrLY7sJ9tYuZuy/O/FQFYDz2z1bmbJ24NQNGwU9o3BPHpLRfSsKHGR 5Ei5cBK7jbXhE/4j3tF05QDfURK3nAxZ5DSl+WCfjJiTXAY7hHRgkl1L1NcA9WkOrnuy V96fZ9lSEpKy/87ZCkt6QL1qA9k7KDZmHjCTDoZflNar9mJmWcftXZtJmWYT0YA+sU76 E571n0QJmCzZ4xMqDCQStIXoDoosC3n7tCHecNmWyi3tJxh1eB0BEIJsL4c8OX+WiIOy OCEfUViftjMktnqsOzSbk7je0ThA0Jv2j7USPeZzpt6f64yI6a9E4O3WwjnoXCb1VhJG ISqg== X-Gm-Message-State: AOJu0YxVHfyLzJZKPjeYUKt8mVFvs3ZxtH+CEgAe/PrB70rdGte9KexJ 1xM5WoFTVnvqE+WEM6FCRhw= X-Google-Smtp-Source: AGHT+IEuGqjtCGethniEYNGudQks5UnBSIwwt2kWzroGL+XzTPH1/gwJvsk0sPKs2exzhmXTPYgXJw== X-Received: by 2002:a19:5f19:0:b0:4ff:840b:a37b with SMTP id t25-20020a195f19000000b004ff840ba37bmr14427132lfb.14.1693130080104; Sun, 27 Aug 2023 02:54:40 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id y24-20020ac24478000000b0050096712dc0sm1084630lfl.222.2023.08.27.02.54.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Aug 2023 02:54:39 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Sun, 27 Aug 2023 11:54:38 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <311389CC-D412-42B0-92BA-CA52972C3E95@gmail.com> References: To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 26 aug. 2023 kl. 16.55 skrev Stefan Monnier : >> - 4096 bytes for vector blocks seems a tad small >=20 > AFAIK this number was chosen arbitrarily and we never tried to tune = it, > so there's probably room for improvement, indeed. > Note that many vectors will be small (they're really Lisp structs), = tho. That's definitely the case, and the same allocation mechanism is used = for many other fairly small objects. Another observation is that most vectorlike allocations are fixed-size. = This is mostly true even for nominally variable-sized objects (vectors = and records) because as you say, they are often used as (fixed-sized) = structs. We might want to segregate allocations by size entirely and cease = splitting longer vectors. That, with larger and N-bit aligned blocks, = would provide rapid access to common metadata from any object pointer. = This may reduce GC costs; will have to look at that. Naturally, allocation and GC costs must be considered together (and = run-time, in case we change representation which is occasionally = warranted). > It's important to improve efficiency of vector allocation because it > allows us to get rid of specialized allocations we're using for other > Lisp types. E.g. we got rid of the specialized `marker_block`s we = used > to use for markers and overlays (and a handful of more esoteric data > types), and next in line are the `symbol_block`s and = `interval_block`s. Yes, many important object types are fixed-sized vectorlike allocations. = Allocations tend to be spikey in size; that's why vector_free_lists is = so sparse and should probably be reformed. From debbugs-submit-bounces@debbugs.gnu.org Sun Aug 27 12:21:50 2023 Received: (at 65491) by debbugs.gnu.org; 27 Aug 2023 16:21:50 +0000 Received: from localhost ([127.0.0.1]:46031 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qaIWM-0007xR-0i for submit@debbugs.gnu.org; Sun, 27 Aug 2023 12:21:50 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:61525) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qaIWG-0007wx-2j for 65491@debbugs.gnu.org; Sun, 27 Aug 2023 12:21:45 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 7B5C380793; Sun, 27 Aug 2023 12:21:32 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1693153291; bh=pdtBJS3jKdo2xIh2v8soKcVJQQ/3lKXROYKDwSHgDJE=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=Kwdn/StgbZHSeu+K/qX2pA+b3Rzfdga7utMn/PyfR7c60vMVWFh2mQ6A+N0dGLXnV pfRJC3YvMV7aWDBaTy+zRuTpmqsafFGBOyO3d/pUsNh2PAkhIZJFPWr2LOCR+98BAX j+gYceFcdvs06Sa9F2arW2011t9dqO2Wqu0h3A6casmBvvrb2it8qMuUX6DXfac6Yn vQbnUgezw4/MDZNM756G3jhjWNN3DmHcH8ovdie1e3IvFwRdg7I0MAX3RXbsa9LLHK LNVO6INPB+cdUBawqiA9sEA0UP2uPopfZtJ8Kt3DLpX5Wi0/OI+lxLoNFYBWmv0gI9 QXrfU6Wm9nK9A== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 17EE9805DE; Sun, 27 Aug 2023 12:21:31 -0400 (EDT) Received: from pastel (unknown [108.175.234.188]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E294A12024D; Sun, 27 Aug 2023 12:21:30 -0400 (EDT) From: Stefan Monnier To: Ihor Radchenko Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <87y1i0iwvu.fsf@localhost> (Ihor Radchenko's message of "Thu, 24 Aug 2023 09:59:33 +0000") Message-ID: References: <87y1i0iwvu.fsf@localhost> Date: Sun, 27 Aug 2023 12:21:30 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.020 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > This patch adds a heuristic that reduces the time spent searching > `vector_free_lists' when trying to allocate a new vector. The microbenchmarks give surprisingly good performance improvements. > `vector_free_lists' is a rather long array with few hundreds of > elements. And it does not make sense to check the whole array in > `allocate_vector_from_block' if we can get information about free vector > that was recently made available of if we know for sure that no free > vectors are available (after GC). Hmm... after GC we should usually have a non-zero number of free vectors available (the unused parts of vector blocks which could not be `free`d because they still contain a live vector), no? [ See comment below. ] > The described approach may sometimes miss free vectors in > `vector_free_lists', especially if the allocation happens from larger > vector to smaller. Right, it could lead to an increase in fragmentation, tho it might tend to allocate temporally related objects together, which might be beneficial. > - pack-unpack 0.40=B10.00 -> 0.35=B10.01 Interesting. I didn't expect such a large effect there. > @@ -3145,6 +3145,7 @@ large_vector_vec (struct large_vector *p) >=20=20 > static struct Lisp_Vector *vector_free_lists[VECTOR_MAX_FREE_LIST_INDEX]; >=20=20 > +static ptrdiff_t last_known_vector_free_idx =3D VECTOR_MAX_FREE_LIST_IND= EX; > /* Singly-linked list of large vectors. */ >=20=20 > static struct large_vector *large_vectors; There's clearly some spacing issue with the following comment but more importantly the new var would need a good comment explaining what the variable should hold and why it's useful, so we know when it's safe and desirable to set or use the var. > @@ -3180,6 +3181,7 @@ setup_on_free_list (struct Lisp_Vector *v, ptrdiff_= t nbytes) > set_next_vector (v, vector_free_lists[vindex]); > ASAN_POISON_VECTOR_CONTENTS (v, nbytes - header_size); > vector_free_lists[vindex] =3D v; > + last_known_vector_free_idx =3D vindex; > } >=20=20 > /* Get a new vector block. */ > @@ -3234,7 +3236,7 @@ allocate_vector_from_block (ptrdiff_t nbytes) > /* Next, check free lists containing larger vectors. Since > we will split the result, we should have remaining space > large enough to use for one-slot vector at least. */ > - for (index =3D VINDEX (nbytes + VBLOCK_BYTES_MIN); > + for (index =3D max (VINDEX (nbytes + VBLOCK_BYTES_MIN), last_known_vec= tor_free_idx); > index < VECTOR_MAX_FREE_LIST_INDEX; index++) > if (vector_free_lists[index]) > { IIUC that's the core of your patch. Nice. > @@ -3426,6 +3428,7 @@ sweep_vectors (void) > gcstat.total_vectors =3D 0; > gcstat.total_vector_slots =3D gcstat.total_free_vector_slots =3D 0; > memset (vector_free_lists, 0, sizeof (vector_free_lists)); > + last_known_vector_free_idx =3D VECTOR_MAX_FREE_LIST_INDEX; >=20=20 > /* Looking through vector blocks. */ Hmm... so I was wrong and after GC there are aren't any free vectors? I need to go re-read that code, then, because it doesn't match my mental model of how it work(s|ed). Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 28 06:14:35 2023 Received: (at 65491) by debbugs.gnu.org; 28 Aug 2023 10:14:35 +0000 Received: from localhost ([127.0.0.1]:47035 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qaZGV-0005Z2-Cb for submit@debbugs.gnu.org; Mon, 28 Aug 2023 06:14:35 -0400 Received: from mout01.posteo.de ([185.67.36.65]:49485) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qaZGS-0005Ym-K0 for 65491@debbugs.gnu.org; Mon, 28 Aug 2023 06:14:34 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 0CF30240027 for <65491@debbugs.gnu.org>; Mon, 28 Aug 2023 12:14:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1693217660; bh=dxRiCP9mDMWCb/5rmWmsSMFxzFgsKBHUOTzH0tH7AEQ=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=Q3t1HR0ugVBVl+V3FOFMyFU51jK+CPs5kl+7xQ4EGDAbxcqJZ3jaK5oHcfhvn0NgA IjyQ6+nDGod5jeKujvjgCTbZJzmnmL3bx4YhsBW+2Vg3NWFfYfZCicq+7s6dyjSwtk aMzxJ23Vj6XCNbad0B93bVy5sBmZxDsx9QUiBFCsJiBHO888OYA8NH/M4shW+UWflU SeA+Wu4BMFW7Yc/k3MV/jLf/18r6t/fK3v8HRCFOV6gi9ajAiK400TYTjk1mwfhE9k 8xrcZbiO2FLp6mB6naak2TlyyC+wukXkcpXkQ30GPlXMetGV5TPo5U6YoN8WtdGZOT zfu5CLtK1pVdw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4RZ5y32PbBz9rxD; Mon, 28 Aug 2023 12:14:18 +0200 (CEST) From: Ihor Radchenko To: Stefan Monnier Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: References: <87y1i0iwvu.fsf@localhost> Date: Mon, 28 Aug 2023 10:14:49 +0000 Message-ID: <87il8z323q.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Stefan Monnier writes: >> +static ptrdiff_t last_known_vector_free_idx = VECTOR_MAX_FREE_LIST_INDEX; >> /* Singly-linked list of large vectors. */ >> >> static struct large_vector *large_vectors; > > There's clearly some spacing issue with the following comment but more > importantly the new var would need a good comment explaining what the > variable should hold and why it's useful, so we know when it's safe and > desirable to set or use the var. Sure. I can add a comment. However, from previous answers, we may better rewrite the vector allocation code more significantly. >> - for (index = VINDEX (nbytes + VBLOCK_BYTES_MIN); >> + for (index = max (VINDEX (nbytes + VBLOCK_BYTES_MIN), last_known_vector_free_idx); >> index < VECTOR_MAX_FREE_LIST_INDEX; index++) >> if (vector_free_lists[index]) >> { > > IIUC that's the core of your patch. Nice. >> memset (vector_free_lists, 0, sizeof (vector_free_lists)); >> + last_known_vector_free_idx = VECTOR_MAX_FREE_LIST_INDEX; >> >> /* Looking through vector blocks. */ > > Hmm... so I was wrong and after GC there are aren't any free vectors? > I need to go re-read that code, then, because it doesn't match my mental > model of how it work(s|ed). Interestingly, when I just tried to skip searching vector_free_lists when there are no vectors available there, it yielded to no significant improvement. Only that max(VINDEX (...), last_known_vector_free_idx) yielded a good performance improvement. The reason might be that vector sizes are distributed non-uniformly and some segments of vector_free_lists are filled more than others. It could be a good exercise to look into statistics of how vector_free_lists is filled. Is there some standard way to print debug information from the allocation code? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 28 08:48:01 2023 Received: (at 65491) by debbugs.gnu.org; 28 Aug 2023 12:48:01 +0000 Received: from localhost ([127.0.0.1]:47352 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qabey-00047Q-PH for submit@debbugs.gnu.org; Mon, 28 Aug 2023 08:48:01 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:21781) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qabex-00046k-E8 for 65491@debbugs.gnu.org; Mon, 28 Aug 2023 08:47:59 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id A470E8072D; Mon, 28 Aug 2023 08:47:47 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1693226866; bh=7lZXsAQpuS2suHoS5Mykn2KlWpGzjbfo8WHyzefBo1Q=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=Kr9WU9q5dof6wMLXuX2zQq9BGTzl2QvNpX0/W7I5NWNZkuIZZBuGLCv5oEdMX83ak vOkZPlhV5xOODYtW/qzQDCrqtn2wRvNC4jMHZELeWXNnoIwZhQjtC8ryUlb2fkpTHA l4LzIpYvAqNPtlcd1pUgN22avfggU4ibcuh40FciEqhEHty70dA01/rl26bNAWflSG wF87iGC85mrEJw7EImYeuETaofl4dHijJY8rU8jdUQ0JkelwZyHjzFEdMmVlV3jKh1 H3xcYLmHwvi6/WzA1LCR6bs6fFLJMSpOabP9SgiCII39eRlHacpKTWC9OnCBoufzQp /E18pfVY0T+xw== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 8074080348; Mon, 28 Aug 2023 08:47:46 -0400 (EDT) Received: from pastel (unknown [108.175.234.188]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 603C1120298; Mon, 28 Aug 2023 08:47:46 -0400 (EDT) From: Stefan Monnier To: Ihor Radchenko Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: (Stefan Monnier's message of "Sun, 27 Aug 2023 12:21:30 -0400") Message-ID: References: <87y1i0iwvu.fsf@localhost> Date: Mon, 28 Aug 2023 08:47:45 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.027 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> @@ -3426,6 +3428,7 @@ sweep_vectors (void) >> gcstat.total_vectors = 0; >> gcstat.total_vector_slots = gcstat.total_free_vector_slots = 0; >> memset (vector_free_lists, 0, sizeof (vector_free_lists)); >> + last_known_vector_free_idx = VECTOR_MAX_FREE_LIST_INDEX; >> >> /* Looking through vector blocks. */ > > Hmm... so I was wrong and after GC there are aren't any free vectors? > I need to go re-read that code, then, because it doesn't match my mental > model of how it work(s|ed). Ah, no, I see what's going on: the above code is run at the *beginning* of `sweep_vectors` before we discover which (free) vectors remain. Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 28 12:32:23 2023 Received: (at 65491) by debbugs.gnu.org; 28 Aug 2023 16:32:23 +0000 Received: from localhost ([127.0.0.1]:48894 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qafA7-0002Np-Cc for submit@debbugs.gnu.org; Mon, 28 Aug 2023 12:32:23 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:1228) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qafA4-0002Na-Aq for 65491@debbugs.gnu.org; Mon, 28 Aug 2023 12:32:21 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id E566E4414C7; Mon, 28 Aug 2023 12:32:07 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1693240321; bh=r8YpZBXLcYz7h8DgLkIqBmFpzLpR4ou6ZXawvvFH2lo=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=bYdh+YryUjv7jesNdmmeN2VIdu8CkWgBspZx3qyFzFgow4nYwUhgOJ6dkljwPH2C/ 2qo1a3BuVxH/TmxfpxSMpdox2wPeGbdF0G3KS/EWRwskhr0g5ajOEvukihTjKdtKXS DjxVm7jJ9CHqaTCsEgOOikKDQyvtwhIczSEwk5Zx2fE+cGQNHVv1xUb4l9q5JTjSuv 5rBYy5rBlckRyyTk08ZZihUI/gIBMTSTG/FgSHyuQbvQrilSQ9imqHE6ZkATknVjWO mFPXIatiVw4UayoLUevZgzoUPqQw379+oJsQkgjbKhmNr7bMRJyiUmUNmmvmFryWm5 Dl+y3ksgDb8mA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id CB3DC440BB5; Mon, 28 Aug 2023 12:32:01 -0400 (EDT) Received: from pastel (unknown [108.175.234.188]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id AAAE21203CF; Mon, 28 Aug 2023 12:32:01 -0400 (EDT) From: Stefan Monnier To: Ihor Radchenko Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <87il8z323q.fsf@localhost> (Ihor Radchenko's message of "Mon, 28 Aug 2023 10:14:49 +0000") Message-ID: References: <87y1i0iwvu.fsf@localhost> <87il8z323q.fsf@localhost> Date: Mon, 28 Aug 2023 12:32:01 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.015 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > The reason might be that vector sizes are distributed non-uniformly and > some segments of vector_free_lists are filled more than others. Indeed, I'd expect that at any given time, many buckets of `vector_free_lists` are empty. Also, when ELisp code allocates many structs of the same size, the first few allocations may be satisfied by free elements in the appropriate bucket, but very quickly that bucket will become empty, so we'll look for the next non-empty bucket, and the remaining slightly-smaller free vector will then be put back into a lower bucket but the next allocation will go through the same loop to find that same "slightly-smaller free vector", to make it yet a bit smaller, ... until it's all consumed after which we'll look for the next bigger free vector etc... So your code makes a lot of sense from that point of view. Many mallocs approach the problem by creating whole pages dedicated to a given object size, so when the bucket is empty, N new free vectors of that exact size are created (in a new vector-block) and put into the appropriate bucket so the next N allocations of that same size will quickly find a free vector. Since all those vector-blocks are made of vectors of the same size, the size info can be shared between them instead of each one of them carrying its own size. [ In our case, vectors need to carry their size for other purposes than memory management, so it's not clear we'd gain much there. But it could help make `live_small_vector_holding` faster. ] Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Sep 13 08:38:25 2023 Received: (at control) by debbugs.gnu.org; 13 Sep 2023 12:38:25 +0000 Received: from localhost ([127.0.0.1]:32828 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgP8T-0006tr-DG for submit@debbugs.gnu.org; Wed, 13 Sep 2023 08:38:25 -0400 Received: from mail-lj1-x234.google.com ([2a00:1450:4864:20::234]:45554) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qgP8R-0006tb-Fg for control@debbugs.gnu.org; Wed, 13 Sep 2023 08:38:24 -0400 Received: by mail-lj1-x234.google.com with SMTP id 38308e7fff4ca-2ba1e9b1fa9so115502591fa.3 for ; Wed, 13 Sep 2023 05:38:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694608692; x=1695213492; darn=debbugs.gnu.org; h=to:subject:message-id:date:mime-version:from:from:to:cc:subject :date:message-id:reply-to; bh=KA5/s2wyx9AIyFTdJm6ZuKxxEWdnf3AN6gEeK3FGgt8=; b=erwuwh9imEoP+N0Gdu/mWoiWQUXjc+Bb2mfGdxJ9BYSyou47LTvtyX6IvcUMoRMjLp U7rzMarAT+NOATUiZrp95XrvpWsZ/KnmQWzk4G+J52vuq3C+qm6dbVKSflXsFu/GYcV0 zbchi//2EiQKSdMnvsMGFG5rMog1Q1qgDCN/JKxu/KJ8ljBjjy4F+HS62FXShlutKu8L PP4hdPuwI3dKE2mnoV3MLDhp5a+UqVr7+fADuqPDIloYNpnTYCm51tul9mA2KPWKBDdk xtglXHHHFVhAiIYj2FM2Sg2LUGBNNgG3p2FBQ+HK1w0MLc7WBrsAGeWum3+b4NypwXBS 6lEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694608692; x=1695213492; h=to:subject:message-id:date:mime-version:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=KA5/s2wyx9AIyFTdJm6ZuKxxEWdnf3AN6gEeK3FGgt8=; b=R78wgpTDJf1/zQ2aJVAWiDaY6L2uKnl0b5TNXAFoBom6GZR4+wXazvXJL/tduOygND UANim+r0CfJGemA8p1sHOV+/8hqd3aF6lMgdU5bBDgZ93n59ex9iOmR27vucaIS2xcjO H/Vdo192ulIq7xQEiStuaPcbfsMYOmk3OtHkJjubxSN/IHFkx/VLnw52lbAj3OJZWVCg 386JA3ysH8s0ojzOqR9tFZIpG6YNlY0tmNLLRhAUVZy5tjH4aO39D8ycq5ITAHRVfQXa kJ4vuM6sHLLmpwoJP76TkxTmx9O3kF4DeNVcs9pbvE9l83Sft5aqz/rQ37UA/jFikT1e 0j9Q== X-Gm-Message-State: AOJu0YxWDQbbD5EiIrxzW8Qa+00vavaRVMaK7RU1zn+l+n3anHyU/kzr UXyFoWNc5Rx/RRLPA2Uay1jX6+1HZtQmkqH9sr2147+D X-Google-Smtp-Source: AGHT+IEGw/BUUtPgxIswxu1vHQfvlINOLdwYIu7ooGJDvk0sWE7Lpu/hGRDVj1TZqbjpCRMMOAikbM99tA/TyFm4MGk= X-Received: by 2002:a2e:8744:0:b0:2b6:decf:5cbd with SMTP id q4-20020a2e8744000000b002b6decf5cbdmr2311801ljj.36.1694608692167; Wed, 13 Sep 2023 05:38:12 -0700 (PDT) Received: from 753933720722 named unknown by gmailapi.google.com with HTTPREST; Wed, 13 Sep 2023 05:38:11 -0700 From: Stefan Kangas MIME-Version: 1.0 Date: Wed, 13 Sep 2023 05:38:11 -0700 Message-ID: Subject: control message for bug #65491 To: control@debbugs.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) severity 65491 wishlist quit From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 10:58:20 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 14:58:20 +0000 Received: from localhost ([127.0.0.1]:48257 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhWkW-0000Mj-Aj for submit@debbugs.gnu.org; Sat, 16 Sep 2023 10:58:20 -0400 Received: from mail-lf1-x12b.google.com ([2a00:1450:4864:20::12b]:58770) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhWkS-0000MU-4o for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 10:58:19 -0400 Received: by mail-lf1-x12b.google.com with SMTP id 2adb3069b0e04-503056c8195so647714e87.1 for <65491@debbugs.gnu.org>; Sat, 16 Sep 2023 07:58:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1694876283; x=1695481083; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=KEorY5+NHQYg5+umqCFM0wAFmxUxY64eeNlLQjJSh/0=; b=M2EfZaewYBxSmMiFEiokkJ4/DVh7Tb7Fuj0HBS58rRkN7o83VgTzy5+Y8ion3liWTP 0uMYY0rQdWlx69dioUJiEHp4t+d+wZEO80JriCIPRMBbcgrz5uTo71LAl5YeUHLd1ihC fz67TqhbvGXGb3b2J3Jew5lw4lEjZM9ssSVmpM3melsfTGHEaeE+w7DdtpNZi17mQZvW 4M8g8wpmv2TSlQQ1JQS0AZ4Dnlqg5S8AgiwTm+1HFrY30jn4AryE8/9EgOL9VAxDRWx1 tUml/EfTu76ZY/Ea85+eoZlCS/n3d3OD13fraqyeibvp5cVoXfrHbi9bPRnRHE5lf+kc GJyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694876283; x=1695481083; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=KEorY5+NHQYg5+umqCFM0wAFmxUxY64eeNlLQjJSh/0=; b=qTCe5sQKnsuwh198FZwOLgAcKRrr0MJAPkdHb+jWIN+1yVRxjRU3oEFl8GSP3arwJO w/Km/nQzF3eb43egpRiS8yuMo2O7WjoHewM5on31NNVN5HZCtBiLgzYkpUSGZn3RRooX UP1Tx0syYmo8CVRcfyYcouw4iTdzelkKYQxVH2PHxKdgm0KN7P+6ks1j2WRnDm5OP494 v0leanYMNxFbY0dlSlH4wKywKeC4RmRQW77CUpuCF1g1aNRHXqrkDEpKVte1A2JIf5mO 4QhilOVn1PlY8VyCW+KIdwScJyNJ8dTaWkaQjyTXuMLfM6nFrIFKVx+tNGjYnDlz9ZTS +AJA== X-Gm-Message-State: AOJu0YzlK+XZr6v7nnmH0UoN+QgN5y1MK7zFWsM6Fi1hmImjh0FniFjM R330vD1n7oZqxU40xb3I8Km2b6aUUFg= X-Google-Smtp-Source: AGHT+IFtQ0ulQOuCg2yF4ftVRoGGV2fRKhQRdQz0nSRAl36I3Vs8Pdyv/F+FijHyQqF3Ssnr/EtVrQ== X-Received: by 2002:a19:500b:0:b0:501:ba56:65f8 with SMTP id e11-20020a19500b000000b00501ba5665f8mr3676588lfb.45.1694876282535; Sat, 16 Sep 2023 07:58:02 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id l10-20020ac24a8a000000b004fdd9d43e05sm1065025lfp.171.2023.09.16.07.58.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 16 Sep 2023 07:58:02 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Sat, 16 Sep 2023 16:58:01 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> References: To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Results of some further experiments: - Reducing the `vector_free_lists` array to its actually used first half = does improve performance a bit, even with subsequent scanning speed-ups. - Enlarging `struct vector_block` from 4 KiB does not necessarily speed = things up but more measurement is needed. - Using a bitmap for faster `vector_free_lists` scanning is definitely = beneficial. - The previously mentioned hack where scanning begins in the most = recently added bucket is surprisingly effective, often even more so than = a bitmap, but I'm wary of it being brittle. Need more measurements. - Our special-casing of the bool-vector representation is annoying. It = would be good if we didn't have to pay the price on 64-bit platforms. I also pushed two small changes that were essential to working with the = code so we might just as well have them in now: b1881d7dab (fix a static = assertion) and 056c99a34c (better Lisp_Object untagging). I'd like to try size-homogeneous blocks as well, but this requires some = GC updates. Just a matter of taking the time, of course. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 12:13:02 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 16:13:02 +0000 Received: from localhost ([127.0.0.1]:48384 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhXuo-0002i7-CD for submit@debbugs.gnu.org; Sat, 16 Sep 2023 12:13:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53938) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhXul-0002hh-32 for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 12:13:00 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhXuX-0003bq-V6; Sat, 16 Sep 2023 12:12:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=ewGVqglYgN/G1frVeGH0HAvce08VwGzKijs7MKGtXHE=; b=bLzd7UGujP4kdqRPFniJ pO/sEbh/W9upwou3+kMcjuwCf5XqZCvEoW3rBs/k3RPBIRpdFXCql3/FUFc/Zyl0tPTfYy9C7Azed svXlo/H3HQXFtI5ik7H2fBANM1UB4nWKI7fCYQJYHsbTg/6WAEjsssvubT4rBgwtHgCh3FD9idAkD w12MEW66130SwTDNbPYJKEci4MgTBUyXzqwQFs6kJLMMNJ7+jZft0Ml4kVmA+hC7EY5r32BZ/4jry 5XmiOXyApEhTu9XHzjj2csl7eWi81Gnm2DKEq4flV/cAbFnNU6NzXMY4omPElS83Y8ylsexzWAVFW KyLXXIsb/dKGcQ==; Date: Sat, 16 Sep 2023 19:12:40 +0300 Message-Id: <83v8cagkqv.fsf@gnu.org> From: Eli Zaretskii To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= In-Reply-To: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> (message from Mattias =?utf-8?Q?Engdeg=C3=A5rd?= on Sat, 16 Sep 2023 16:58:01 +0200) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Mattias Engdegård > Date: Sat, 16 Sep 2023 16:58:01 +0200 > Cc: 65491@debbugs.gnu.org, > Eli Zaretskii > > I also pushed two small changes that were essential to working with the code so we might just as well have them in now: b1881d7dab (fix a static assertion) and 056c99a34c (better Lisp_Object untagging). The latter one completely broke the 32-bit build --with-wide-int, most probably because the last argument to XUNTAG is frequently a pointer to a 64-bit type, where uintptr_t is only 32-bit wide. As result, I cannot even compile Emacs: each C file emits the same warnings, which fill up my screen in a matter of milliseconds. Please try to fix this ASAP, as it makes me unable to work on master. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 12:17:40 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 16:17:41 +0000 Received: from localhost ([127.0.0.1]:48401 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhXzI-0002qR-M8 for submit@debbugs.gnu.org; Sat, 16 Sep 2023 12:17:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48740) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhXzG-0002qE-Pb for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 12:17:39 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhXz4-0004aq-8P; Sat, 16 Sep 2023 12:17:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Ey7fiVN7dgfWSBZeR+4SGMTi12Q7YFgmb9kW37QSkqw=; b=kwjJdzN031ia R3Q56COHra6Ee367wZhzDX8TCUuScbqnctcVfbCfL/vR44k7sZXxCoSsuROnhxC1Y16aw/gon+qSa ejYn4gS21eQMH2RFWLGNCmYWGbWAw2mho/2WlHahEFS883SDUKkkMhKDdLlULIAxdMKopvOTt3l9Q BexzUKzeoJDwOuAR+7E+8xfNbTXKODaAoTKQTBbt6T4YeUjhmGK2trJEEzL95IkttsQJWDjylOjNg b3qzo0WITyomi6qo4h58QV2P9lIV1sEAKpSOObILpmTGvAoVnjJy7PuuodHxdZDzqqoFzV6HCVQbN Fm6+4HcKfDrdkuXUWiundw==; Date: Sat, 16 Sep 2023 19:17:21 +0300 Message-Id: <83ttrugkj2.fsf@gnu.org> From: Eli Zaretskii To: mattias.engdegard@gmail.com In-Reply-To: <83v8cagkqv.fsf@gnu.org> (message from Eli Zaretskii on Sat, 16 Sep 2023 19:12:40 +0300) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca > Date: Sat, 16 Sep 2023 19:12:40 +0300 > From: Eli Zaretskii > > The latter one completely broke the 32-bit build --with-wide-int, most > probably because the last argument to XUNTAG is frequently a pointer > to a 64-bit type, where uintptr_t is only 32-bit wide. No, that's not it: the reason is that the _first_ argument is a 64-bit data type, and then casting XLP(a) to uintptr_t causes the warning, because uintptr_t is a 32-bit type. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 12:32:52 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 16:32:52 +0000 Received: from localhost ([127.0.0.1]:48406 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYE0-0003GQ-7H for submit@debbugs.gnu.org; Sat, 16 Sep 2023 12:32:52 -0400 Received: from mail-lf1-x130.google.com ([2a00:1450:4864:20::130]:49597) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYDv-0003G9-C2 for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 12:32:50 -0400 Received: by mail-lf1-x130.google.com with SMTP id 2adb3069b0e04-502984f5018so5187631e87.3 for <65491@debbugs.gnu.org>; Sat, 16 Sep 2023 09:32:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1694881953; x=1695486753; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=H7bmbRajsVEk756mKnkxsIMozbsc5g2cXAeevBVrcPY=; b=XcHPT+KV5KW7BiFp2LD+tEju9yhwMO1VxNtRfqLc4fZISVSUkJrFtyQ3LCkmIfHW5X FOQ6bWpB99Gle+JfooTJ27TFy/rQ2/qs3R865JCEIP+YvPR8HheZ+qnPKoPhVMOm3pc9 RZPKN8NtJW0/bFCGHL6cSW+5R3LZU3T5aBRmHUNhBNBBapJr7LEW9n/KPVFDn6/OiEmx iVW0408e3EafHxIBqzA06mF1AngZRR7B6Lk+w6IIaPKjPpbcWJcCTKFMM31XWSKPrscr gdVJgxg0SyJ3OITuoluiIksXfyYPoJFyF4+p9r4JjVD9wdnLPpFPee7HacYKapG3caza vEyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694881953; x=1695486753; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=H7bmbRajsVEk756mKnkxsIMozbsc5g2cXAeevBVrcPY=; b=IbeoLJDqCsTEppB8hU5WqPD403inPSZ99rGlUw6uhjGVNBBVQRzrnp6NsD7xJpidY/ dtF3xH42y1pojOEHoMfgv6BhJ/4FlIRber+Tx8cUmr/y++Uh3WEoPJCtFyBbghN4TVRK 5tES/giuR/vpZNwlmME2zFCtSMIwLbdaaVHkBTTO0jP7rdG9U2MIRL2sCR7hd/bAx8a9 exOc9Vyz/yY0V+cficdnEU2vjkN4kRHsYgSNq/VvsMf97Q+XpY9H+X5aKCglpPhHtzUa 20XntqbIiyrokf7iWg77SZ6VdTVpra7U5mCHhtxEAuvbdBc0kCqcmT1X9sUK4rVXbcxk nd8w== X-Gm-Message-State: AOJu0Yxru0nPl5CVhhMwktj8eMQFqB52JKwwOpgbEYn9iN6WjXDfiyT/ 1zxjUWm92oz/kTkFC3u5HdU= X-Google-Smtp-Source: AGHT+IFqbIYs8W/NTsJOfE3mgAK/A03L3vr0MUeJF6W0XIYJT0rfKMJvSUqTtNL3NdF/hE7iGPaxEQ== X-Received: by 2002:a05:6512:539:b0:500:ba68:2344 with SMTP id o25-20020a056512053900b00500ba682344mr3642848lfc.20.1694881953284; Sat, 16 Sep 2023 09:32:33 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id n9-20020ac242c9000000b00501c022163dsm1077446lfl.222.2023.09.16.09.32.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 16 Sep 2023 09:32:32 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: <83ttrugkj2.fsf@gnu.org> Date: Sat, 16 Sep 2023 18:32:31 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> To: Eli Zaretskii X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 16 sep. 2023 kl. 18.17 skrev Eli Zaretskii : >> The latter one completely broke the 32-bit build --with-wide-int, = most >> probably because the last argument to XUNTAG is frequently a pointer >> to a 64-bit type, where uintptr_t is only 32-bit wide. >=20 > No, that's not it: the reason is that the _first_ argument is a 64-bit > data type, and then casting XLP(a) to uintptr_t causes the warning, > because uintptr_t is a 32-bit type. Let's see if I understand this correctly, as I can't try that = configuration myself. In your configuration, Lisp_Object is a 64-bit integer (or a struct = containing one). LISP_WORD_TAG(type) is also a 64-bit integer with tag bits at the top = (USE_LSB_TAG=3D0). The old XUNTAG was: #define XUNTAG(a, type, ctype) ((ctype *) \ ((char *) XLP (a) - LISP_WORD_TAG = (type))) so that we get a subtraction of a pointer and a very large 64-bit = number, which results in just the pointer. The new XUNTAG is: #define XUNTAG(a, type, ctype) ((ctype *) \ ((uintptr_t) XLP (a) - LISP_WORD_TAG = (type))) so you get a warning from what, conversion of a 64-bit number to (ctype = *)? Does changing the definition to #define XUNTAG(a, type, ctype) \ ((ctype *) ((uintptr_t) XLP (a) - (uintptr_t)LISP_WORD_TAG (type))) help? (That is, cast the LISP_WORD_TAG return value to uintptr_t.) From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 12:54:23 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 16:54:23 +0000 Received: from localhost ([127.0.0.1]:48458 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYYo-0003uW-Mu for submit@debbugs.gnu.org; Sat, 16 Sep 2023 12:54:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47174) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYYm-0003uC-MA for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 12:54:21 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhYYZ-000433-HO; Sat, 16 Sep 2023 12:54:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=5ZUA7Kwp61RMpxBdjbKt20AuVd4of1L8Y5JmsWxec38=; b=BZ9dw00Pg3VFYzcDPmkY avJuffhW9lZtUx6hltPbEhAmIh7Umn8CzhHU//vCOxLJk/RIiyxouc6U/Tl+b8FW74Y/7MbQr2z9y Cm07x3CHqCQginbBD3YnDNKFfA9GwrXO7ZQxRGaYZDUCjemlqfgKpRGDFsD1L4vhuB6jb6QTHazHU uQ03Rc6Y6CyqQEqwMKUZN7mfQgck0hnyJ96pVfV8JvsCmP7gAo4sIkdq2Zs+ltxewJ0W80jEtLCDn OR67Mxd/fGW2wAALDvXYWxWyTx/i0FCfr7YupiRawv4snZwtjStt/7r0rQjM1lq0FVYMSfTPjZibB ylD84DdxX937BQ==; Date: Sat, 16 Sep 2023 19:54:02 +0300 Message-Id: <83sf7egitx.fsf@gnu.org> From: Eli Zaretskii To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= In-Reply-To: (message from Mattias =?utf-8?Q?Engdeg=C3=A5rd?= on Sat, 16 Sep 2023 18:32:31 +0200) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Mattias Engdegård > Date: Sat, 16 Sep 2023 18:32:31 +0200 > Cc: 65491@debbugs.gnu.org, > monnier@iro.umontreal.ca > > The new XUNTAG is: > > #define XUNTAG(a, type, ctype) ((ctype *) \ > ((uintptr_t) XLP (a) - LISP_WORD_TAG (type))) > > so you get a warning from what, conversion of a 64-bit number to (ctype *)? Yes, I think so: In file included from dispnew.c:27: lisp.h: In function 'PSEUDOVECTORP': lisp.h:813:33: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] 813 | #define XUNTAG(a, type, ctype) ((ctype *) \ | ^ lisp.h:374:6: note: in expansion of macro 'XUNTAG' 374 | ((XUNTAG ((a), Lisp_Vectorlike, union vectorlike_header)->size \ | ^~~~~~ lisp.h:1127:10: note: in expansion of macro 'lisp_h_PSEUDOVECTORP' 1127 | return lisp_h_PSEUDOVECTORP (a, code); | ^~~~~~~~~~~~~~~~~~~~ lisp.h: In function 'XSYMBOL_WITH_POS': lisp.h:813:33: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] 813 | #define XUNTAG(a, type, ctype) ((ctype *) \ | ^ lisp.h:1152:12: note: in expansion of macro 'XUNTAG' 1152 | return XUNTAG (a, Lisp_Vectorlike, struct Lisp_Symbol_With_Pos); | ^~~~~~ lisp.h: In function 'XBARE_SYMBOL': lisp.h:813:33: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] 813 | #define XUNTAG(a, type, ctype) ((ctype *) \ | ^ lisp.h:1159:27: note: in expansion of macro 'XUNTAG' 1159 | intptr_t i = (intptr_t) XUNTAG (a, Lisp_Symbol, struct Lisp_Symbol); | ^~~~~~ > Does changing the definition to > > #define XUNTAG(a, type, ctype) \ > ((ctype *) ((uintptr_t) XLP (a) - (uintptr_t)LISP_WORD_TAG (type))) > > help? (That is, cast the LISP_WORD_TAG return value to uintptr_t.) It does, but LISP_WORD_TAG(type) is a 64=bit type with the only bits set above 32 bit, so how casting it to uintptr_t is TRT? Why did you need to change the original cast in the first place? From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 12:54:44 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 16:54:44 +0000 Received: from localhost ([127.0.0.1]:48463 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYZA-0003vF-48 for submit@debbugs.gnu.org; Sat, 16 Sep 2023 12:54:44 -0400 Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]:47540) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYZ8-0003ux-Jt for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 12:54:43 -0400 Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-2b962c226ceso50548511fa.3 for <65491@debbugs.gnu.org>; Sat, 16 Sep 2023 09:54:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1694883269; x=1695488069; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=6xzHKryRnaJoqbrxZvVm8ehcgFXqPdF1ZZucTLCMGAc=; b=Me7Bd5gUOdwuO7p9AtrbWV91Hi7y+vbc4YgLVqZe2+igpYw1lIFoRsaM9DSBe23/ib dqZ5dxTzmgSJIiTXoDkoOtpGPi9x3pAEYflDXj1Ckfm6WrwTHW3RlU/yf7q/XtRZ3JAc XutwUz30imUTP3RydY/q347rzRG5MSRVgbEpKCih/iGHThbPb4LN7YBOl8ignQhfmwkc 7uQeO4HlOiNKwDEPpJ4tvDm0dMfFyVx4Tac1QkoYvWDLPIzVk4D+0Tjs2probMhalbs2 DW76E3BFS7ika3x27BZDd6vZKYxnQTBMdbsdmsmY88PKr7uKW82PiVD/BUojzSEd+UPN KqoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694883269; x=1695488069; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=6xzHKryRnaJoqbrxZvVm8ehcgFXqPdF1ZZucTLCMGAc=; b=GJrVPZmv53bdXmAUu1uvcUM3Ohb1/jc/8JjmI5dj9v65eWjKT3Jqpsa4NKg9ZNBJ2t 9zrAPrhwQTPlYJjIDMCM/ktMFfQNWLO5G6VagEghr599ZZsNtL9hARzAeovpLu7tUZ9M I+E0KAemolsHZxo2F6xVcloScuzEidpRNj7/O1Kx2Tk1Ol+0Z2GpmLUcXu906Y0J8f3y 64BhZNKMJ0T/HT8Wh9rr2w6duKm6C1sNReSDb2nPI6/NJFP7JSPMv2N4qhOWRZYcZLkY +OkWVSyxsGYGANq/b6YzmJXoxOSnkaakDR/8IVylXuTRaMSkUtyT6mZTNLnPgxT9f6OE osQg== X-Gm-Message-State: AOJu0Yx491CrXlmbLF9JehmI/i2wBGbusI30OD3xuSSf8+HZhEeVm7fV mTR/JcWjwmD3i4Pq5oJyxCw= X-Google-Smtp-Source: AGHT+IHRM4m1dT5IFOATrxHmut8x0VrDUxakqP4EiKpG2ilZ/oGR9iCZbY92D4+O7TXEg2qme+ROrA== X-Received: by 2002:a19:ae07:0:b0:500:9524:f733 with SMTP id f7-20020a19ae07000000b005009524f733mr3780750lfc.20.1694883269118; Sat, 16 Sep 2023 09:54:29 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id l25-20020ac24a99000000b004fc863a0bf6sm1086887lfp.223.2023.09.16.09.54.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 16 Sep 2023 09:54:28 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: Date: Sat, 16 Sep 2023 18:54:27 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> To: Eli Zaretskii X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 16 sep. 2023 kl. 18.32 skrev Mattias Engdeg=C3=A5rd = : > #define XUNTAG(a, type, ctype) \ > ((ctype *) ((uintptr_t) XLP (a) - (uintptr_t)LISP_WORD_TAG (type))) I pushed this to master after some experiments. Interestingly, Clang didn't warn about the new code at all. On the other = hand, it did probably warn about the old code (with your configuration), = because char *g(char *p) {return p+(1ULL<<62);} results in :8:26: warning: the pointer incremented by 4611686018427387904 = refers past the last possible element for an array in 32-bit address = space containing 8-bit (1-byte) elements (max possible 4294967296 = elements) [-Warray-bounds] char *g(char *p) {return p+(1ULL<<62);} ^ ~~~~~~~~ :8:9: note: array 'p' declared here char *g(char *p) {return p+(1ULL<<62);} ^ but GCC thinks it's fine. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 13:03:52 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 17:03:52 +0000 Received: from localhost ([127.0.0.1]:48470 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYi0-0004CU-5a for submit@debbugs.gnu.org; Sat, 16 Sep 2023 13:03:52 -0400 Received: from mail-lj1-x229.google.com ([2a00:1450:4864:20::229]:51358) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYhw-0004CE-Ts for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 13:03:51 -0400 Received: by mail-lj1-x229.google.com with SMTP id 38308e7fff4ca-2bf66a32f25so49804111fa.2 for <65491@debbugs.gnu.org>; Sat, 16 Sep 2023 10:03:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1694883815; x=1695488615; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=6LZRVKtJxbt27vAzemoK0vZbfgDDZ4MnpuSuoPptUfU=; b=F7undlMuKmv0Vyso8Pn8zlnp2fs/u3VCGfb3BQtINwu5gM0CAki6Qh1umylmlYTKzA kpRF3s0USjG2D+Fw6PJzdN9WrcQtAzBMcujvOLUVMoT0eq+lfQILHE5r2e9OVBEqN6rq qP3+XWfpuzOvCr90PkpUNtsW0eZGVR+IvluizYwIdg7heJuy1kkHlPcODPc08d+X58sm VKu5OOupUGul2nOE/4V7Vv2LW1Du3nE6pZo9B/g99FQpebpIJEgCnh8djabmgKlaCkvu c6Z0I5jSqCqL4uao64VzwbP1L8soFqAWFj3F+mXjsvbEGsfTx+3gPX6wDhkWc9/bfyL7 fZww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694883815; x=1695488615; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=6LZRVKtJxbt27vAzemoK0vZbfgDDZ4MnpuSuoPptUfU=; b=WpT70dSFc4Z/brKgQTqWKTwEvivyVrjguIoKMvOsAGEPa5+d5gRUXsMqZMXOYmBYhG /QPYP4ly/ee1uFCFAO6Vpyt6R0QCeq4umbw0nYJnRlMl9JNpgZikHXMhQp277zhyf4dR cv3I9tfyormajHGrjdT5W06hGLnQjIP47fRwS6/fKsOuXpy3DDyGzOB0P5KWGbPgL6wy WH95cRfZ9nnczJpj2C+3PAVArqsc+S2tDSVpAvNo7GeyiU6Xl5vcQOesTSAhaeJDG1yV WYRcUZN1xSRf/nPHn7T2X/bBbiV4p/wy3/LhJQpUlfl+DbbB7Bc4dKWwcWktyLhsuTRG KBAg== X-Gm-Message-State: AOJu0YzzxDt7X1vXN27q3e1ItdqKeclJIDsz/faVd1lBSa0e8xulwjVA OLR3Gv7UCjfaRNufK8AuKls= X-Google-Smtp-Source: AGHT+IFxT7JwgcmTHnR41f9WhsiEnmy7uxYyIcdf+MY4aa4gyVKIthGj+kDg2S64CXPi7KD0ChIAYg== X-Received: by 2002:a05:651c:149:b0:2b9:c4f6:fdd with SMTP id c9-20020a05651c014900b002b9c4f60fddmr4137189ljd.14.1694883815230; Sat, 16 Sep 2023 10:03:35 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id r7-20020a2eb607000000b002bf7fd1d336sm1243792ljn.14.2023.09.16.10.03.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 16 Sep 2023 10:03:34 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: <83sf7egitx.fsf@gnu.org> Date: Sat, 16 Sep 2023 19:03:33 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83sf7egitx.fsf@gnu.org> To: Eli Zaretskii X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 16 sep. 2023 kl. 18.54 skrev Eli Zaretskii : > It does, but LISP_WORD_TAG(type) is a 64=3Dbit type with the only bits > set above 32 bit, so how casting it to uintptr_t is TRT? Because XUNTAG is used to get the pointer part; we don't want the tag = bits.=20 > Why did you need to change the original cast in the first place? The commit message tried to explain that, but in essence, the old code = untagged a Lisp_Object value by casting it to char *, then do pointer = arithmetic on that, and then cast the result to whatever pointer we = want. The C standard severely restricts pointer arithmetic: in particular, for = P+X where X is an integer and P is a pointer, P cannot be null (nor will = P+X, since both P and P+X must be pointers to objects in the same = array). This means that XUNTAG could never reliably untag a null pointer and = this did cause mayhem in some places. We have just been lucky not to = trigger it so far but I noticed when attempting to make some = innocent-looking changes. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 13:09:24 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 17:09:24 +0000 Received: from localhost ([127.0.0.1]:48479 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYnM-0004Lp-7r for submit@debbugs.gnu.org; Sat, 16 Sep 2023 13:09:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56714) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYnI-0004LW-B5 for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 13:09:23 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhYn5-0007Ut-6S; Sat, 16 Sep 2023 13:09:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=vgIjwl749QWDJH8iZMAqUsUHbQFzbDFAZ9d1tpMIJd4=; b=K20u8oxyKA+ufXT5oJZl DhLNxb4gEnekp4s7PvmGRWGF1tkWyZwcDhF+iUVCq0Bg5wQAoeWl7DGk168i30KpZNWNsf9ygqGQg j64Cei9kefaARmaKnRJyIy7lwDbFMcDpZ+qI7RTQmIh6qIk1WbVfM00YAngvTb2UQyBOpODotPg4Y pkddvfvgQP3bJW8vWYYHnXtaDG94zd6yHU/XPQE6ujeQjQjI51n8ChovYB0S2/k4rMMPNGppidtgu 5EK624C/CX9qBqNyZZN4euO0lI8P7YDLsZdignuaa53QL0gb6s9yCMxtQ+SE8QYRKvqRCsMbJ58SZ TaaBoA4awSjMfA==; Date: Sat, 16 Sep 2023 20:09:01 +0300 Message-Id: <83r0mygi4y.fsf@gnu.org> From: Eli Zaretskii To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= , Paul Eggert In-Reply-To: (message from Mattias =?utf-8?Q?Engdeg=C3=A5rd?= on Sat, 16 Sep 2023 18:54:27 +0200) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Mattias Engdegård > Date: Sat, 16 Sep 2023 18:54:27 +0200 > Cc: 65491@debbugs.gnu.org, > monnier@iro.umontreal.ca > > 16 sep. 2023 kl. 18.32 skrev Mattias Engdegård : > > > #define XUNTAG(a, type, ctype) \ > > ((ctype *) ((uintptr_t) XLP (a) - (uintptr_t)LISP_WORD_TAG (type))) > > I pushed this to master after some experiments. Sorry, I cannot accept this kind of "discussions" when such tricky issues come up. What's the rush of installing changes when you still didn't answer my questions, and we still are not sure these changes are correct? We are touching the deepest bowels of the implementation, so some caution, even trepidation, is in order. So I've reverted the last two changes. Let's finish discussing the need for this change and why you think the last variant is TRT, and install then whatever we decide is needed. I also added Paul to the discussion, since he wrote most of the involved macros. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 13:12:16 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 17:12:16 +0000 Received: from localhost ([127.0.0.1]:48484 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYq7-0004Qc-Pn for submit@debbugs.gnu.org; Sat, 16 Sep 2023 13:12:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37812) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhYq5-0004QP-BL for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 13:12:15 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhYps-0000Be-PZ; Sat, 16 Sep 2023 13:12:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=bX9lTN+Nl0dd8hItPeUHcRcl6IuTdQGbfdDVZsBkY4E=; b=B8Ot3u+ON4IAdUHzQCb2 qhEWP2DOku3UjfN+cB35hsMEhQ2sx8vap+OLwI8dh6GSx4cuoF3EX1fHzBVt/ujW69DH/3dlL+6B3 ww7pIh8yc2+rK2h6VBeci1fGRssa3vcd6CCcm2OTCoK++kQZJpC8B5Fk15T5EySXaz0vx75aeysel QKZ66YJZMI33R5nFKhDdzYHJRabllUVWcB6iu45kiloe1cAzjY0loax1WCyZZVGTGAqE5kf344K31 sGyTADrsGeSSY6uoeOsulJVzb4LML1tTSx7iShTwoEjFQXDOihT+fLr72GzC/Dx2Dt/+OZeYy4P7e nSScBUQPFXoIDQ==; Date: Sat, 16 Sep 2023 20:11:56 +0300 Message-Id: <83pm2igi03.fsf@gnu.org> From: Eli Zaretskii To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= In-Reply-To: (message from Mattias =?utf-8?Q?Engdeg=C3=A5rd?= on Sat, 16 Sep 2023 19:03:33 +0200) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83sf7egitx.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Mattias Engdegård > Date: Sat, 16 Sep 2023 19:03:33 +0200 > Cc: 65491@debbugs.gnu.org, > monnier@iro.umontreal.ca > > 16 sep. 2023 kl. 18.54 skrev Eli Zaretskii : > > > It does, but LISP_WORD_TAG(type) is a 64=bit type with the only bits > > set above 32 bit, so how casting it to uintptr_t is TRT? > > Because XUNTAG is used to get the pointer part; we don't want the tag bits. Then just casting should do, no? Why the subtraction? > > Why did you need to change the original cast in the first place? > > The commit message tried to explain that, but in essence, the old code untagged a Lisp_Object value by casting it to char *, then do pointer arithmetic on that, and then cast the result to whatever pointer we want. > > The C standard severely restricts pointer arithmetic: in particular, for P+X where X is an integer and P is a pointer, P cannot be null (nor will P+X, since both P and P+X must be pointers to objects in the same array). > > This means that XUNTAG could never reliably untag a null pointer and this did cause mayhem in some places. We have just been lucky not to trigger it so far but I noticed when attempting to make some innocent-looking changes. Sorry, this is not a reason good enough to make changes in these parts. However "unreliable" the old code worked for many moons, luck or no luck, so if there's no other reason, I prefer to leave it alone. And I'd like to hear from Paul before we make any further changes in that area, please. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 13:23:15 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 17:23:15 +0000 Received: from localhost ([127.0.0.1]:48494 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhZ0l-0004j7-Dm for submit@debbugs.gnu.org; Sat, 16 Sep 2023 13:23:15 -0400 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]:49196) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhZ0g-0004iZ-P5 for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 13:23:13 -0400 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-501cba1ec0aso5239633e87.2 for <65491@debbugs.gnu.org>; Sat, 16 Sep 2023 10:23:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1694884977; x=1695489777; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=7D8G7uCOTYGgEgMURpVzf7uu4OEanmvRIucjpaFQw8Y=; b=gjC83R99FMLXqjqCQoMikS5DelncOM/vKn9tp31j5YoI/v6qUzuynAmHabQVfzvgqe SjjxrBmgwAyNuclTeVPxkDxLYBFJtrRNcsdxFNOFjMXoG85PBQCaSET5QUIr1r8/nqIr VpyVwx5tCD3GVEQv7Zo6OwcFkr0M1Mu22AknIKd/hXxWHnJbxMXx6WAs2UpdxLZFiMk1 2tDR1EN5L1ph6xwNp7hEFaG3nbeolasghTBpe6B4uUeREzXhpjjWEJs3zymJy8PZUx38 Bt8NVkCZWFkyDrPEYF+w3CiR96WcsmBGfSLd7SXUx2J54sMgZQVyN022FM33X6hRwUM5 hC2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694884977; x=1695489777; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=7D8G7uCOTYGgEgMURpVzf7uu4OEanmvRIucjpaFQw8Y=; b=d5CDRX9grbrI69/lgPmwtCcn6Gbk2JEld3sVbWwqndJBLAhgV4ug/4uawqAjTq+bRY rZHmdQElmkDMpYdK8AZJMs6yT7zTCr4vsxuzzLaEAtws5sXs28xBOZWLG5vXzSRbZ/or Mxg3f5N6PkCPd6g2Ra/2rOfz+8i0OynGwLVxuza6klzJef4jUC7xWQxxh5CU6aKQqqz8 070v3qycjh+oj3T89+0SX+T377sbEmcKcoSwM+erihT/fqPYEXiGgBibvAMIdsFudDJ6 ZemV60O2PBI91NqvI/Wzlpdyr/NkHTYS4fHW9bUubW+dz9wcp4lR4QuMbqokzGuEn9dh dRfQ== X-Gm-Message-State: AOJu0YwvCzcPLKF6jAIg840t5p+11MyhjYmoqPyCStkOFXWI6F/MJYHm 4xbAeRX0T7uk4urTUmrRVyYq5B/NpdY= X-Google-Smtp-Source: AGHT+IFTg7hVHlwZ6IFMM0l616bcchCcEcwY+Fn0liSYmLba+Z2TPgSQ7HMyyXeyZD3Vq8CIDI8ivw== X-Received: by 2002:ac2:5633:0:b0:500:c348:7efd with SMTP id b19-20020ac25633000000b00500c3487efdmr4054149lff.59.1694884977291; Sat, 16 Sep 2023 10:22:57 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id q17-20020ac25151000000b004fe3c53bb69sm1100909lfd.307.2023.09.16.10.22.55 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 16 Sep 2023 10:22:55 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: <83r0mygi4y.fsf@gnu.org> Date: Sat, 16 Sep 2023 19:22:54 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <8758CDE8-40F1-48DB-9B94-38F771DC8C6C@gmail.com> References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> To: Eli Zaretskii X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Paul Eggert , monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 16 sep. 2023 kl. 19.09 skrev Eli Zaretskii : > Sorry, I cannot accept this kind of "discussions" when such tricky > issues come up. What's the rush of installing changes when you still > didn't answer my questions, and we still are not sure these changes > are correct? I'm confident that they are correct. Moreover, I'm also confident that = the old code was incorrect, which is why the change was carried out. = Both the C standard and modern C compilers agree. There's nothing strange or unusual that the 32-bit --with-wide-int = configuration sees unexpected warnings when code is changed. You must = have seen that many times before. It doesn't mean that there is anything = wrong with the change; in this case it was just a somewhat pedantic GCC = warning, quickly silenced. >>> It does, but LISP_WORD_TAG(type) is a 64=3Dbit type with the only = bits >>> set above 32 bit, so how casting it to uintptr_t is TRT? >>=20 >> Because XUNTAG is used to get the pointer part; we don't want the tag = bits.=20 >=20 > Then just casting should do, no? Why the subtraction? Because when Lisp_Object is pointer-sized we need to remove the tag bits = from the word. Only in the special configuration with a Lisp_Object that = is larger than pointers can we simply cast away the tag bits. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 14:19:54 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 18:19:54 +0000 Received: from localhost ([127.0.0.1]:48508 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhZtZ-0006Ih-Nc for submit@debbugs.gnu.org; Sat, 16 Sep 2023 14:19:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhZtU-0006IN-MQ for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 14:19:51 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhZtG-0005Qa-LM; Sat, 16 Sep 2023 14:19:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=ahpGsW+Eb0HHxgbjYzd1V17g7eo8gpdTcbqJJap/nPc=; b=NSv9LFmsTXMsBhIIsnoE Yxiqhi9VS4lAxoWBbG+fFTatrh4/QMrEXG0X+gekb9aj38VX9q70UU5VyUMZUuMrp8PdpxFs+MgXA EIdK7jPypszd5v9S1HmxNdQd4GgmhVzNdZnRETf1NWkz8HijL0cFoOTqG4gNmJQklGUMIJGIphAzM JrGU6Y76hYMS0vaHEi6b6xoPF1bHU168lPUFC+JXBmvxVrQOYtsY4HJAnlfa2drSEQTBxqBJXWAaa 3xsePPGmocDDGc5HDWqohUAKwU3v2kYbyS5p+1WIdrh3qJh7RqD4JI9wgqHZwwGWfqKjZdndz7dgC Am9lemWWxv12tQ==; Date: Sat, 16 Sep 2023 21:19:29 +0300 Message-Id: <83o7i2gevi.fsf@gnu.org> From: Eli Zaretskii To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= In-Reply-To: <8758CDE8-40F1-48DB-9B94-38F771DC8C6C@gmail.com> (message from Mattias =?utf-8?Q?Engdeg=C3=A5rd?= on Sat, 16 Sep 2023 19:22:54 +0200) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8758CDE8-40F1-48DB-9B94-38F771DC8C6C@gmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, eggert@cs.ucla.edu, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Mattias Engdegård > Date: Sat, 16 Sep 2023 19:22:54 +0200 > Cc: Paul Eggert , > 65491@debbugs.gnu.org, > monnier@iro.umontreal.ca > > 16 sep. 2023 kl. 19.09 skrev Eli Zaretskii : > > > Sorry, I cannot accept this kind of "discussions" when such tricky > > issues come up. What's the rush of installing changes when you still > > didn't answer my questions, and we still are not sure these changes > > are correct? > > I'm confident that they are correct. Moreover, I'm also confident that the old code was incorrect, which is why the change was carried out. Both the C standard and modern C compilers agree. > > There's nothing strange or unusual that the 32-bit --with-wide-int configuration sees unexpected warnings when code is changed. You must have seen that many times before. It doesn't mean that there is anything wrong with the change; in this case it was just a somewhat pedantic GCC warning, quickly silenced. I get it that you are confident, but I want to be confident as well, and I'm not there yet. A discussion is not over until all of its parties say it is. By rushing to install changes while the discussion is still not over you create an unhealthy atmosphere where some people might conclude their opinions, questions, and doubts are ignored, and that doesn't contribute to the sense of cooperation towards common goals. > >>> It does, but LISP_WORD_TAG(type) is a 64=bit type with the only bits > >>> set above 32 bit, so how casting it to uintptr_t is TRT? > >> > >> Because XUNTAG is used to get the pointer part; we don't want the tag bits. > > > > Then just casting should do, no? Why the subtraction? > > Because when Lisp_Object is pointer-sized we need to remove the tag bits from the word. Only in the special configuration with a Lisp_Object that is larger than pointers can we simply cast away the tag bits. Those special configurations have telltale traits that can be used in cpp conditionals. IOW, we could have different definitions of XUNTAG for different configurations. It isn't unheard off, and other macros, including some that are involved in XUNTAG, do indeed have separate definitions for several configurations. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 15:04:46 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 19:04:46 +0000 Received: from localhost ([127.0.0.1]:48531 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhab0-0007Un-2Z for submit@debbugs.gnu.org; Sat, 16 Sep 2023 15:04:46 -0400 Received: from mail-lf1-x135.google.com ([2a00:1450:4864:20::135]:61647) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhaax-0007UZ-A7 for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 15:04:44 -0400 Received: by mail-lf1-x135.google.com with SMTP id 2adb3069b0e04-50305abe5f0so777752e87.2 for <65491@debbugs.gnu.org>; Sat, 16 Sep 2023 12:04:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1694891070; x=1695495870; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=pw8J4F1kfN7+JDt4k8l1d1HMqgHmbtp79NOLqjkFypQ=; b=jr7X0NxeCfiezpOVKt5HPlzITi+L15T55HwnTVbJRhMP7Z0jWMedmYENqc3t4jKcVi VdUL/ELqySwWuw7LTYGa0SEQa4DAlLJIVVTJ9LSE9jWkJ0vBMvkxyWVgURnx/816uw4M fAC+pz8+GEq1MXGBG6q+g2sNjVkzqNqAgjkIK96LQB2SNZyfxBDMhldHbcmPKIk5OR8K c4sy2jssIQTzjudAER8D+bSOU3qF9SHVJajRBxRX79C+afR1qlESnVX+IFcJDPPLQjnL GgFlIeCIWAuB5zWr36Lh/4ilg9QnhvARBP+nrixyA0zxOwh8gkJcKxbFO+TjLa5KjBPS lXWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694891070; x=1695495870; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=pw8J4F1kfN7+JDt4k8l1d1HMqgHmbtp79NOLqjkFypQ=; b=XIF7thx4o6R/yllDGEo78pf+CMNZazvLwoAqRWhWZkYmsPEK7NZ/y8wcR7DLuXV/xQ 7r5FVkYJRbAxDsM0WxgSEWhkGO7pNYwF494Kt7kku7d6IVc6nDqG7Xr50E+q4fXRJAE/ YPWtRUN16r2ZCng1Z+hx0oTcvaI9lOw6gOZ6xlSx+CSjGcCQ9M+0mYTN4gD0p71T6b3a V+EYBIjZ9awId6dF1UiAUlRU+iat6WRuc1oveczfh3k0RzQnDnytdOc2jOAk0xILsutw klnOpqpK4RFSUdfPrIcuUPCYoZOSTeBhupS0fBeixxdVNlCNbFBxxT5ERfxaqqM9jYnh 1+xg== X-Gm-Message-State: AOJu0YxkIVmKAgryBQzMCmYnRBdjG2CsLzbQ6F/DFAfEALs9km4fYntT 3NHupDC8ArfldxGG0/TlgqA= X-Google-Smtp-Source: AGHT+IHTOt2LaOK3rkh4TE0oii5v+BnQktq0uPmZakbxFW3QoY8wr8f1C6ESzdofX3h09xL7j0Gp4Q== X-Received: by 2002:ac2:5f69:0:b0:4f8:766f:8dc3 with SMTP id c9-20020ac25f69000000b004f8766f8dc3mr3850956lfc.32.1694891069848; Sat, 16 Sep 2023 12:04:29 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id q2-20020ac24a62000000b00501c673e780sm1118457lfp.84.2023.09.16.12.04.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 16 Sep 2023 12:04:29 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: <83o7i2gevi.fsf@gnu.org> Date: Sat, 16 Sep 2023 21:04:28 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <3E9D73A8-BF51-4DBE-87D2-AECEFEA32A0E@gmail.com> References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8758CDE8-40F1-48DB-9B94-38F771DC8C6C@gmail.com> <83o7i2gevi.fsf@gnu.org> To: Eli Zaretskii X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, eggert@cs.ucla.edu, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 16 sep. 2023 kl. 20.19 skrev Eli Zaretskii : > I get it that you are confident, but I want to be confident as well, > and I'm not there yet. That's fine, I'm in no rush. I'll be happy to answer any questions you = may have. It's natural that our perspectives are different. =46rom my point of = view I have a well-researched and carefully tested change with = everything indicating that it's safe and an improvement. But when the = first thing you see is a flood of warnings, it's quite understandable = that it is taken as a sign that something is wrong. > Those special configurations have telltale traits that can be used in > cpp conditionals. IOW, we could have different definitions of XUNTAG > for different configurations. It isn't unheard off, and other macros, > including some that are involved in XUNTAG, do indeed have separate > definitions for several configurations. Certainly, but we didn't need multiple definitions for XUNTAG before and = nothing has changed in that respect. However, if you think that it would make the code easier to understand = we could separate the code into two or more #if clauses, although most = of the time a single definition is easier to maintain. In any case, that would be a separate change. From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 15:46:55 2023 Received: (at 65491) by debbugs.gnu.org; 16 Sep 2023 19:46:56 +0000 Received: from localhost ([127.0.0.1]:48565 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhbFn-00009k-H3 for submit@debbugs.gnu.org; Sat, 16 Sep 2023 15:46:55 -0400 Received: from mail.cs.ucla.edu ([131.179.128.66]:43394) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhbFi-00009R-KL for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 15:46:53 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id C83253C00D189; Sat, 16 Sep 2023 12:46:37 -0700 (PDT) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id XqXnop6puLhO; Sat, 16 Sep 2023 12:46:36 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id F298C3C00D195; Sat, 16 Sep 2023 12:46:35 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu F298C3C00D195 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1694893596; bh=G4vG/kLPa2HCW9fSKGe21CxIoiI6W/rjh4rfs4wKPfM=; h=Message-ID:Date:MIME-Version:To:From; b=Loih5slQ80djcVNpL1sW82XHe08arLLEt8WFNkey+c4430hvNtJJ5QLN7T+IHXOG4 +mfDy2budhlLdTS2E2GTH7Z+EKZUCvK1e7wGxgTD3tzjB9ILknyU+QCmDLu2MvHaXN Uh9yFZXLeseoQSR9txwxSCfgsL92uBtW4MufTlp69KKnq56VKKsQMC1BADq2GzWH6T sx8+z89SSfkOSYgEYRE9O0bRR4Vx3Axm1cto46AhvLCs3ObJQWxLjleYZvpmIGG0vZ S0hz1S26gVS6/E/FxPhCAKXa5kbt8nmsmmD4MxsUyl6WBkVcSZpuu4nDnU3czWW00U 5tHbKL+i58NZg== X-Virus-Scanned: amavisd-new at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id fVBPbOIWSL53; Sat, 16 Sep 2023 12:46:35 -0700 (PDT) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id C5ABC3C00D194; Sat, 16 Sep 2023 12:46:35 -0700 (PDT) Message-ID: <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> Date: Sat, 16 Sep 2023 12:46:34 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Content-Language: en-US To: Eli Zaretskii , =?UTF-8?Q?Mattias_Engdeg=c3=a5rd?= References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> From: Paul Eggert Organization: UCLA Computer Science Department Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <83r0mygi4y.fsf@gnu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -1.5 (-) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.5 (--) On 2023-09-16 10:09, Eli Zaretskii wrote: > I also added Paul to the discussion, since he wrote most of the > involved macros. Mattias and I had a private email discussion about XUNTAG last month, and he's correct that Emacs's current code, which #defines XUNTAG(a, type, ctype) to ((ctype *) ((char *) XLP (a) - LISP_WORD_TAG (type))), violates the C standard due to calculating a pointer outside its containing object's bounds, whereas the patch P that you just reverted, which defines the same macro to ((ctype *) ((uintptr_t) XLP (a) - (uintptr_t) LISP_WORD_TAG (type))), does not have that particular undefined behavior. My own impression is that patch P is a net win, as it should make Emacs more reliable after likely future changes, for two reasons. First, the unpatched version's undefined behavior caused Emacs to crash when Mattias tried out what appeared to be an obvious change to the GC's vector handling. Had patch P been present, Emacs would not have crashed. Second, I vaguely suspect any attempt by Emacs to exploit recent memory safety improvements in Arm, Intel, etc. on GNU/Linux are more likely to work with patch P than without it. I suspect this because most other apps that tag pointers do it in the integer world (as patch P does), not in the pointer world (as Emacs currently does). The developments I have in mind are Arm MTE[1], Intel LAM[2], and (more speculatively) CHERI-RISC-V[3]. Although this is just a suspicion, I suspect I've thought about the issue more than anyone else has. I would not favor a two-pronged approach, in which patch P is present but is used optionally. This would likely cause more trouble than it'd cure, due to lack of testing of the extra combinations. Let's use just one approach and keep it simple and more testable. [1]: https://lwn.net/Articles/834289/ [2]: https://lwn.net/Articles/902094/ [3]: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/cheri-risc-v.html From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 16 23:03:12 2023 Received: (at 65491) by debbugs.gnu.org; 17 Sep 2023 03:03:13 +0000 Received: from localhost ([127.0.0.1]:48791 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhi40-0000Kz-GQ for submit@debbugs.gnu.org; Sat, 16 Sep 2023 23:03:12 -0400 Received: from sonic312-25.consmr.mail.ne1.yahoo.com ([66.163.191.206]:43066) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhi3y-0000Kk-8g for 65491@debbugs.gnu.org; Sat, 16 Sep 2023 23:03:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694919777; bh=DK20AyLUqH3ADCLol0zn3FbcQk4Ud4l9zsEZd5BeqKQ=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From:Subject:Reply-To; b=GISyLNYBAVXa+bAC1jznpBUhHcRqOcILNNDWTQoXuESN2jQsCUR1/aIIkJ71jp3gZE1rSdtyK/sNb9RvaZR5FTIuCCvro3mJbAKlu6zfmga/Q9qb42R5cFZ7MbQ7ELwZWfVkMKAOLnL2Zx9Dbm5mvnSM1Z0LjUHbmETsbibqD2tpic9ZiCZysKKSLOLu7NFFW4W5YGEH8tC4FyGd1dd3K/mU0VKj6EHodVPifdG4uUpkwk+2cC1vjFdAD/ZVrGEERqwcFCIHuNE5YMm2lo6YDfdgDkv1ahUz5cwJGOEBmpdnALi9lyNOaUTMMioAN9iUrquyY5Spb94Wml54k0VPcA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694919777; bh=/g3iGj2p3T7kdapuhAoQRgmkSdPF9JAIQs+0ehZWmao=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=qP2cWPZWjlXA9jZ7MRI6J2iQSLX+R4r2tBX+tyMqXSIaIgX1D2MD6pYJNnz7mTgi3QwvV6DCOTOxTVKJnb+sCE9CVF70h0ypenrucnZeY964MgysABnt/OanVWd5+TJneSeFMUZhPj8Hbx41BL7wzKYBl5cYW0cKdXGzMOCZTJheH76Yo+HK5hfC4YxzLzKYDaJdpStipBA5WRAUxF6YxCBe27w5fKoRrrwpyrjK65XJbl1n+7PLahi5KNMjp3S5kXGIrYS+RHnlj30NDVAVKJu6CL6UuBFjuMTD2kfqnMdgL/my/ox7Z6+jOILEDKGAsjIUZ/T8eygwoz3kP+hBtQ== X-YMail-OSG: nJIJVecVM1ku4vLc7OrO9jIJ5Mp1ZZ_OVKP8mMMDz0zXsVQThQRUVkyyddA.biQ 3Iyn1aKCKdWtYO.bAs454HC0jzDNiMzfwby4BiWn4MzJzJzqC2x6NsL6CaFFcinbQNJ3FsNWaUkZ 5eknrSy_LLat0ce8bUMozUJB3TT58JLVOiVJ.VflTBnCZnMWTJix9FzEw3oICEBKqHPy1vB1YQI6 .BfRrmcVXLL0Wex00kGOIkw2tJQEBd5lEGALKOXeHYyHjVH86E.TaIQa4Vz7P9Y01u00jUZJS4mf 5qez4rKe7u9E2DG3jwc._VYNdZghsyYL7X9yPN6eGCk3PD2eow.s3Ompig3G0mx9HC7KbNz6PbIc kWgQ16fgBL9R3TpShiGlYrvuPppw3deukW4fw6yZc0pqvB3i2jWIZNsQJNHlQu945ARkhpQ9hlQp rixi5RpvLvpOSc4qd3GUfal6XrqY3WVm6QuAttrBB3oFxrca3BBeGqrGl2c5tzwFlc76sj9YGf.r 7rbfzRHwSg6E8j.n7JKTXElOYNUyNerBfFu6I3V7odDkn2YdTCO_vMxR8RclrrZKyTTfILvG4j.v Ys6fR0hlIBmcn21es47JYEZCcO0Xku1AtD8bO_BaGbinoeoEgyDdNSNaG8vOdaZlKQsMohz7S2TM e1urRkto_qKO4Otlw.SjCijeI9KnMeML84W3Td2SgIF33G6A8Km0lzJrTpdYFGSwBVDe_HbVIWzT 6saJC_5ik6tJSCO23rWCqb4._BNfX559cupsPp6oMfdyQt34.XS6WmLqXN46WtRt72TnFObquxkj NwnRfTEiNkESVNlpWHrN6cglL4THN2ve2ERzLUOtydLf9PYKBHpviLvDOME.wRIE_BQ3a2zPr01v w2hZHrfCMjRh5Gig1_EAO12kdDdSmlQQImyiz_UKaUWT3wh6pUrX_55lHclJzxgvshOyZU8FNKrG bPdJMnOflqK6oR5oDaElnlqDvI1esh5sS.xT470q84iIT.NhsQ4xu5NB2bbuRNQrI400ZvGEvlvV V81oHECNH4xe9jD0Gdu7SpaTbt3Xyu2DJP4pQh_7HMdrUuj1P61OAXwmC2R.DWQcBrS.grlHBge_ 89GexuzkFyXRxU6Qdv5ON9cHMgYAFhajkoCDqAVgZqeroguv9TM3kaqf4pX7HbQ1lLx5gRQt48ln X80cfX4vto.4Bk6noeUJmUnyEyuC2Kt5X3mLUaOZ7XBMAM0JL80vp7TB7MeLzP2UkU0FwCYVeZKk 5yhpuqO0JITP3ZRo2w4tXAYNzil.OiTXyRTcl3UD8JwNzOi9u13Y6RXTaXFbHEwQezDfpqwb4jfH LGzDPqnM_V_bFwJIlG1QJUBDj7NucpABIF.QocyxyfkzPhiVEb5OR3GGxJd8sDK.UCppF5.1XuNr wsFzyyD8Fkm44QSmZnlLO61EY3EZKCup12QE.M6BkCcqpzwFAlDgdjK_c565aavgmQfWKMXichSC L3hso6Q6aIzyvB1OrYbwT5Rw01SXAbdjh0fpwrv6wp415kNsN0LHCBYbofXrSr3Fc9qnVZeug5iM Xqt0dzN8QyELAIO3wAA3i5lDm_H2tFz5I5EYiaRwSiBaubiSprIblFVBLM01KKTKZe8bEQPXofzL ftvYjAu_.f.coKkh2ELNa88aIdlz9mvxHh7WeMZIDHCE_55VwUU8iViRkSQ1nivsz2zQBplv4CLA Dpof0XWQDEYVvsKhtnyQt4lelH.bRzt3lw1gvbFksxbaGW9kDaoPwBSYXd7Xm4iS91ZqGBysZnuE uSqxQYE7vktl1GU1JPB90UAJTltGpJmhD7T6ngH7QPBBb3RHKcvNZX0AFZfCtMXm.gQH8bGBuDKm hugAa2fn_fhGfyQQPolqzD.MoOrUhYWQkowO8RaKvmmX61X1IJcZSiv8JGrXUlQfysE24xAiL3QK xdaoaYTNuN0av2wIIhPzD9WWw6TSLMELEEAYDjNzZy3Ud0o8BYlze8kKGmUCsVpN06PrlBA3hYxg EAhNL04eNGTAQtfqAN9tkiKee3FP5wM33_O9aN6p2FB370VHdAvV9vM3hzajsGE.AlX9VAdCg.7t ezcUkm1.CT4XIyGoil9pej1Nz5mjKbhn.Za7l_gc0PAsmVELxXYK2CGw3Rs152l15E5HUWNshH9s qtBDY.ssa47sXGFCp5EsHLdLBZesQzO0I9Bae4x8Z92_Cww63KgM3lXyuISRxjr_6e_JCmVPKdxf Bq8oliT8TbOmsrH2LT8ayCvg6IqKboFWlbM7bfAq1CUM5OG7.7ukVAYZAQ6pcljhDcq1Ru.pU X-Sonic-MF: X-Sonic-ID: ff4d7ede-2a63-4ad8-b882-bec0a0c3a137 Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.ne1.yahoo.com with HTTP; Sun, 17 Sep 2023 03:02:57 +0000 Received: by hermes--production-sg3-55c667b499-zxx5q (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2b7dec72eaba8ae0d65bfa0cb14b8994; Sun, 17 Sep 2023 03:02:54 +0000 (UTC) From: Po Lu To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: ("Mattias =?utf-8?Q?Engdeg=C3=A5rd=22's?= message of "Sat, 16 Sep 2023 19:03:33 +0200") References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83sf7egitx.fsf@gnu.org> Date: Sun, 17 Sep 2023 11:02:47 +0800 Message-ID: <871qexzelk.fsf@yahoo.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Mailer: WebService/1.1.21797 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Content-Length: 2362 X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Eli Zaretskii , monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Mattias Engdeg=C3=A5rd writes: > 16 sep. 2023 kl. 18.54 skrev Eli Zaretskii : > >> It does, but LISP_WORD_TAG(type) is a 64=3Dbit type with the only bits >> set above 32 bit, so how casting it to uintptr_t is TRT? > > Because XUNTAG is used to get the pointer part; we don't want the tag bit= s.=20 > >> Why did you need to change the original cast in the first place? > > The commit message tried to explain that, but in essence, the old code > untagged a Lisp_Object value by casting it to char *, then do pointer > arithmetic on that, and then cast the result to whatever pointer we > want. > > The C standard severely restricts pointer arithmetic: in particular, > for P+X where X is an integer and P is a pointer, P cannot be null > (nor will P+X, since both P and P+X must be pointers to objects in the > same array). > > This means that XUNTAG could never reliably untag a null pointer and > this did cause mayhem in some places. We have just been lucky not to > trigger it so far but I noticed when attempting to make some > innocent-looking changes. Within Standard C, the result of converting a pointer value to an integer and vice versa is also implementation defined behavior. GCC regards converting an integer derived from a pointer value back to a pointer as undefined behavior, should the resulting pointer point to an object outside that which the pointer value that gave rise to the original integer references. From `(gcc)Arrays and Pointers': When casting from pointer to integer and back again, the resulting pointer must reference the same object as the original pointer, otherwise the behavior is undefined. That is, one may not use integer arithmetic to avoid the undefined behavior of pointer arithmetic as proscribed in C99 and C11 6.5.6/8. So whether pointer or integer arithmetic is employed in XUNTAG makes no real difference to us, where undefined behavior is concerned, inasmuch as other compilers only afford us even more latitude. For example, Sun's C compiler documentation actively encourages using pointer arithmetic in lieu of integer arithmetic, and speaks nothing of pointer arithmetic between different objects being forbidden. And all other things being equal, I would rather see existing, time-tested, almost primordial code preserved intact. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 01:19:04 2023 Received: (at 65491) by debbugs.gnu.org; 17 Sep 2023 05:19:04 +0000 Received: from localhost ([127.0.0.1]:48866 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhkBT-000470-Qt for submit@debbugs.gnu.org; Sun, 17 Sep 2023 01:19:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45284) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhkBP-00046V-OU for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 01:19:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhkBB-0007c4-Rj; Sun, 17 Sep 2023 01:18:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=sb/x9mAO21Q1ThZAEyJwerAp4dBQ0A/zA8pioBj08Q8=; b=PQpUNl3vxp8p YA2XnckD2a3PC0fH6/v0lYtSh//xURN90PqOziwIYeKxL/thDt/9/HUTnx5QyYS6DStYw8lOvsTIe wBVF2giPm5nt0T5vCNXPdHsaqqdDqNp4ReLUmr4RufiAlSD1FL1KrShxJj+97PP+jv2XwGACBvhcS OZ7a71ncO4YiwKlWzV9MC8LzfwRIRf4xMg/2xmK0M84dPT8AhspBBE6IyxaTEhSJhpHnO0bKmrVc/ 92lBNwKI4PtllSeC0/00gJxNSVT9KEHv8UvWkWivOVxn2DFgao3yanE00r+QLa8bcg+KGsGCPobmR Kep8RrMzL7Zqk4iQFXeHEA==; Date: Sun, 17 Sep 2023 08:18:41 +0300 Message-Id: <83led5gyxa.fsf@gnu.org> From: Eli Zaretskii To: Paul Eggert In-Reply-To: <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> (message from Paul Eggert on Sat, 16 Sep 2023 12:46:34 -0700) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, mattias.engdegard@gmail.com, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sat, 16 Sep 2023 12:46:34 -0700 > Cc: 65491@debbugs.gnu.org, monnier@iro.umontreal.ca > From: Paul Eggert > > On 2023-09-16 10:09, Eli Zaretskii wrote: > > > I also added Paul to the discussion, since he wrote most of the > > involved macros. > > Mattias and I had a private email discussion about XUNTAG last month, > and he's correct that Emacs's current code, which #defines XUNTAG(a, > type, ctype) to ((ctype *) ((char *) XLP (a) - LISP_WORD_TAG (type))), > violates the C standard due to calculating a pointer outside its > containing object's bounds, whereas the patch P that you just reverted, > which defines the same macro to ((ctype *) ((uintptr_t) XLP (a) - > (uintptr_t) LISP_WORD_TAG (type))), does not have that particular > undefined behavior. First, may I ask that in the future such discussions be held on the list, or at least their summary be posted? You can see right here and now how this practice of discussing purely technical issues in private causes unnecessary friction and aggravation, let alone breakage. Next, even if this was discussed, due to our experience with changes in this area, it would have been prudent to post the proposed patch before installing it, so that it could be tested in all the supported configuration before risking such a fundamental breakage of the master branch. You personally do that all the time, and for good reasons, but Mattias has yet to learn that, it seems. More to the point: can you explain how this part ((uintptr_t) XLP (a) - (uintptr_t) LISP_WORD_TAG (type)) works in a 32-bit build --with-wide-int? AFAIU, XLP(a) yields a 32-bit value, and LISP_WORD_TAG has all of its low 32 bits zero, so why do we need the subtraction at all? Aren't we subtracting a zero from what XLP yields? It seems to me that in a 32-bit build with wide ints just #define XUNTAG(a, type, ctype) ((ctype *) XLP (a)) should be enough, since XLP yields a 'void *', no? > I would not favor a two-pronged approach, in which patch P is present > but is used optionally. This would likely cause more trouble than it'd > cure, due to lack of testing of the extra combinations. Let's use just > one approach and keep it simple and more testable. Not sure what you are alluding to here, but if by "two-pronged approach" you mean my suggestion to use cpp conditionals, then what I meant was to have a separate definition of XUNTAG for 32-bit builds with wide ints (which could still remove the undefined behavior), just without all the juggling of integer-to-pointer-and-back conversions, which completely obscure what should be a simple operation of extracting a pointer from an integer by masking some of its bits. Why do I need to fight with these macros each time I read this code, when the job it does is so simple? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 11:22:23 2023 Received: (at 65491) by debbugs.gnu.org; 17 Sep 2023 15:22:23 +0000 Received: from localhost ([127.0.0.1]:51175 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhtbK-0000f9-SS for submit@debbugs.gnu.org; Sun, 17 Sep 2023 11:22:23 -0400 Received: from mail.cs.ucla.edu ([131.179.128.66]:49096) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhtbD-0000eL-OG for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 11:22:17 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 022DE3C00D189; Sun, 17 Sep 2023 08:22:02 -0700 (PDT) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id G5kLHPnBVKSn; Sun, 17 Sep 2023 08:22:01 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id A32583C00D18A; Sun, 17 Sep 2023 08:22:01 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu A32583C00D18A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1694964121; bh=xN/mZDc41F7KyjDrb2Y2hQchxAYjHir7cIh9E//mz2Q=; h=Message-ID:Date:MIME-Version:To:From; b=MQ1AHq0bO4m/Yut3/wBe/0Ez83P8gpsWS+mfMUVdK4iYtMylUqeoUy2Gx/ure+Fy0 O4BaI1fHU4DH/mRNxUD6Dx0pCQZQWauGXLQzTRHK/kmcv2jMJ4t88mf/XeR76McFtz rgE9gEN39mINOSyRLg3bkdihtX5F2mZZy96nufzAvzUHFTcvHPYMxtTOfmhQ8fgfHY j+uAthbLVzfCrMwYchsCrXYWozCTC5hAW836NJm2ypb8+psKKFzIInRjzq9eWCGYSE pDOeuOJWjHjqCXT4UUaLEDiRntbbC68BLtd3/9O+zwkkMt/b7oBSNL9J+1Iea5xx+A 6GroFnvwOJWlw== X-Virus-Scanned: amavisd-new at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id FsLvt-811IIE; Sun, 17 Sep 2023 08:22:01 -0700 (PDT) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 7ADFD3C00D189; Sun, 17 Sep 2023 08:22:01 -0700 (PDT) Message-ID: Date: Sun, 17 Sep 2023 08:22:01 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 To: Eli Zaretskii References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> Content-Language: en-US From: Paul Eggert Organization: UCLA Computer Science Department Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <83led5gyxa.fsf@gnu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -1.5 (-) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, mattias.engdegard@gmail.com, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.5 (--) On 2023-09-16 22:18, Eli Zaretskii wrote: > It seems to me that in a 32-bit build with wide > ints just > > #define XUNTAG(a, type, ctype) ((ctype *) XLP (a)) > > should be enough, since XLP yields a 'void *', no? That would complicate the code unnecessarily. Let's not go there. Patch P works as-is, and it's simpler. Let's do that instead. > what I > meant was to have a separate definition of XUNTAG for 32-bit builds > with wide ints (which could still remove the undefined behavior), Yes, that's exactly what I suggest not to do. Why complicate the source code unnecessarily? And if we complicate it here, why not complicate it in similar ways in dozens of other places? I went through a lot of this when adding support for --with-wide-int in the first place, years ago. When doing so, I strove to avoid having multiple copies of the code whenever I could. And I pretty much succeeded: there are only two WITH_WIDE_INT conditionals in lisp.h (and only three in other source files, all introduced relatively recently by others to work around compiler bugs, and all which should be rewritten without the #if). It's an obvious win to have just one copy of the code instead of two, when one copy works and is just as efficient. As much as possible, --with-wide-int should not be a special case. We should not have "#ifdef WITH_WIDE_INT" scattered all over the place. Keep it simple. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 12:16:14 2023 Received: (at 65491) by debbugs.gnu.org; 17 Sep 2023 16:16:14 +0000 Received: from localhost ([127.0.0.1]:51260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhuRP-0004mV-3E for submit@debbugs.gnu.org; Sun, 17 Sep 2023 12:16:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36888) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhuRJ-0004lp-2G for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 12:16:09 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhuR0-0006vh-8B; Sun, 17 Sep 2023 12:15:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=NL9zBmXbMl4X8pFiVplqyroQDJwLwPXDGyZxwxoeiNs=; b=qlaD6YiTB6XA fuJHlQpIyO90JE2Xe/FNEuglx8kM4eC0bW4oinhZBIcHqngMwevhxpJFfQbsCs6BjOOndr2T/DNKh MtOAtx5zd32PKEv7fa9CYrSLgxIJVUaSPbOS/4IrN0ZfQWdE+0P1g89HkJW/RBhLCGKSs6SKdPCc5 eRQVfIZHfVkzBbBhY65XmFHexRzvyybg8asignKg/bHwRnPJBbuzTJNO86YhS6+pRwHGmwCDOMKUV DFdIcvY/esu/XyMjGCz8I2AVOQnjBAlJNPiYtT4cJMo4NKr7NSmjRmHTqRZC368zvKVPRCFlz93vs S4vRb6hg6HmzfocKlAMT1A==; Date: Sun, 17 Sep 2023 19:15:42 +0300 Message-Id: <83o7i0g4i9.fsf@gnu.org> From: Eli Zaretskii To: Paul Eggert In-Reply-To: (message from Paul Eggert on Sun, 17 Sep 2023 08:22:01 -0700) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, mattias.engdegard@gmail.com, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sun, 17 Sep 2023 08:22:01 -0700 > Cc: mattias.engdegard@gmail.com, 65491@debbugs.gnu.org, > monnier@iro.umontreal.ca > From: Paul Eggert > > On 2023-09-16 22:18, Eli Zaretskii wrote: > > > It seems to me that in a 32-bit build with wide > > ints just > > > > #define XUNTAG(a, type, ctype) ((ctype *) XLP (a)) > > > > should be enough, since XLP yields a 'void *', no? > > That would complicate the code unnecessarily. Let's not go there. Patch > P works as-is, and it's simpler. Let's do that instead. Even though the code is so completely obfuscated and hard to understand? Where do we draw the line between complications and code readability (and thus maintainability)? We don't want to depend on you personally each time we need to make some change in this stuff. > > what I > > meant was to have a separate definition of XUNTAG for 32-bit builds > > with wide ints (which could still remove the undefined behavior), > > Yes, that's exactly what I suggest not to do. Why complicate the source > code unnecessarily? It isn't "unnecessarily" in this case, IMO. > And if we complicate it here, why not complicate it in similar ways > in dozens of other places? Because in general I agree: we should not have several different implementations of a macro if a single one will do. But not at a price of such obfuscation, where even reasoning about the code requires very high level of understanding of the intricacies of C and what is and isn't UB (something that also changes with time). > I went through a lot of this when adding support for --with-wide-int in > the first place, years ago. When doing so, I strove to avoid having > multiple copies of the code whenever I could. And I pretty much > succeeded: there are only two WITH_WIDE_INT conditionals in lisp.h (and > only three in other source files, all introduced relatively recently by > others to work around compiler bugs, and all which should be rewritten > without the #if). I will be the last one to push for more WIDE_EMACS_INT conditions, but let's not forget that they are not the only conditions we have, okay? We also have USE_LSB_TAG and LISP_WORDS_ARE_POINTERS, and those are no easier (and maybe even harder) to figure out in each and every configuration. At least with WIDE_EMACS_INT all you need is to look at config.log, whereas with others you need to follow the convoluted ways of their definition in lisp.h and maybe err a few times. It will maybe make you laugh, but I frequently find it easier to fire up GDB and ask it to show me the values of these macros, to make sure I'm not making a mistake. This is a developer's plight we are better off without, don't you agree? > It's an obvious win to have just one copy of the code instead of two, > when one copy works and is just as efficient. As much as possible, > --with-wide-int should not be a special case. We should not have > "#ifdef WITH_WIDE_INT" scattered all over the place. Keep it simple. I agree in general, but the case of XUNTAG seems like this principle taken ad absurdum: subtraction with no fewer than 3 type-casts (not counting those in XLP and LISP_WORD_TAG) and a needless subtraction of zero, when a single type-cast should suffice! But if both you and Mattias still think a single definition of XUNTAG is better, all the downsides notwithstanding, I won't mount the barricades. Just let it be known that I agree under protest. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 12:37:46 2023 Received: (at 65491) by debbugs.gnu.org; 17 Sep 2023 16:37:47 +0000 Received: from localhost ([127.0.0.1]:51279 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhumI-0005Ip-Hc for submit@debbugs.gnu.org; Sun, 17 Sep 2023 12:37:46 -0400 Received: from mail.cs.ucla.edu ([131.179.128.66]:48546) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhumF-0005Ic-UT for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 12:37:45 -0400 Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 9558A3C00D189; Sun, 17 Sep 2023 09:37:30 -0700 (PDT) Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id MiZMCt3Z7Fjx; Sun, 17 Sep 2023 09:37:30 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.cs.ucla.edu (Postfix) with ESMTP id 33CD93C00D18A; Sun, 17 Sep 2023 09:37:30 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.cs.ucla.edu 33CD93C00D18A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=9D0B346E-2AEB-11ED-9476-E14B719DCE6C; t=1694968650; bh=oXRbmVxB2MP/ufPVgsECFuc9s0ugDBhBY5f0DEoJTpM=; h=Message-ID:Date:MIME-Version:To:From; b=YjsYfZHOfngAuyNp3GxMf1C8UMDfLgMmg1/zBfOJ45HNYXhTlOfE3Ibm38v7K4lfP bF9BBQ2pgDay4tZfTba++ioeffqhRsDfLFb1lJXYW4hArSQysapxQvqscnPJwl7b5c 0hUvIKGJbk0V6bZPxUZJUollVLdSY8CSCVf4q08JbB3qfGDfw1HGYEjJPSwEKcIUIa m8zYOmaZF9nmyfjVX6WXL0CLl4REkG3/4fms75T/Ad4qH6fobS3RvrRhzGJzzobER9 KJPASnmeSnMJjGaUFl6iwCU6Y5Y2BUk20p3bDhoBu7iVudtrb2UTenWKOaqEm5Tk1x hsCmsaNo9HNPQ== X-Virus-Scanned: amavisd-new at mail.cs.ucla.edu Received: from mail.cs.ucla.edu ([127.0.0.1]) by localhost (mail.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id DGYQ5l7eTRNZ; Sun, 17 Sep 2023 09:37:30 -0700 (PDT) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by mail.cs.ucla.edu (Postfix) with ESMTPSA id 0AA3E3C00D189; Sun, 17 Sep 2023 09:37:30 -0700 (PDT) Message-ID: Date: Sun, 17 Sep 2023 09:37:29 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Content-Language: en-US To: Eli Zaretskii References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> <83o7i0g4i9.fsf@gnu.org> From: Paul Eggert Organization: UCLA Computer Science Department Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <83o7i0g4i9.fsf@gnu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -1.5 (-) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, mattias.engdegard@gmail.com, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.5 (--) On 2023-09-17 09:15, Eli Zaretskii wrote: > Even though the code is so completely obfuscated and hard to > understand? If it's confusing, let's add a comment. That would be better than unnecessarily duplicating the actual code. > We also have USE_LSB_TAG and LISP_WORDS_ARE_POINTERS Yes, and we should minimize dependencies on those macros as well. The same principle applies to all these conditional flags - generally speaking the fewer uses of these macros, and the fewer distinct copies of the code, the better. We could even dump those two macros entirely. But this would be a bigger project, one I lack time for (though perhaps Mattias has time...). > I frequently find it easier to fire up GDB > and ask it to show me the values Oh, I do that all the time too, with both Emacs and other programs. It's OK. Why refuse help from GDB? From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 12:45:28 2023 Received: (at 65491) by debbugs.gnu.org; 17 Sep 2023 16:45:28 +0000 Received: from localhost ([127.0.0.1]:51290 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhutZ-0005U9-O6 for submit@debbugs.gnu.org; Sun, 17 Sep 2023 12:45:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34574) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhutW-0005Ts-UY for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 12:45:16 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhutI-0005sH-Oj; Sun, 17 Sep 2023 12:45:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Z9dAqtSVjw3x6bbkLSzlzYporC6vcI2yWHmIRVa/LYg=; b=nt6hI6oJ5zV4 DXI7iWnIYgEGNtJG66elfbDlZFTz1aWA5zHVtU5cDFHIPC0//jxQ7Ir20qcUEHN6+laAxb8IKQWlI B1EMSvR07Vcpoo74ph7mUo2vvuSaLpUBc2NNYSzrTLFKE04SbJvLDeoBTJDDA0miMY9+67cTEq7a2 q5y2p52SNvxmrmLM3eB2FNxuklsOj4HrNh0a5BGN2Yx0OA5Mew3GoyWKeJjvxo7olvDgK4w/Jdk81 fH2fPNMncQWcXWEkBxg+EzOalRCu6b3BR8VPSU4eZBx8iBKY9psneNaJ1ZbyW6MyQzrXeyI/5vUWA 4nvfk8pgvpxxiYtHpXOG9w==; Date: Sun, 17 Sep 2023 19:44:59 +0300 Message-Id: <83jzsog35g.fsf@gnu.org> From: Eli Zaretskii To: Paul Eggert In-Reply-To: (message from Paul Eggert on Sun, 17 Sep 2023 09:37:29 -0700) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> <83o7i0g4i9.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, mattias.engdegard@gmail.com, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Date: Sun, 17 Sep 2023 09:37:29 -0700 > Cc: mattias.engdegard@gmail.com, 65491@debbugs.gnu.org, > monnier@iro.umontreal.ca > From: Paul Eggert > > On 2023-09-17 09:15, Eli Zaretskii wrote: > > Even though the code is so completely obfuscated and hard to > > understand? > > If it's confusing, let's add a comment. That would be better than > unnecessarily duplicating the actual code. Feel free to add any comments you find useful, that is always welcome. But comments about complicated code have one downside: they need to be updated when the code changes, and people forget to do that... > > We also have USE_LSB_TAG and LISP_WORDS_ARE_POINTERS > > Yes, and we should minimize dependencies on those macros as well. My point is that XUNTAG depends on both of those as well. > > I frequently find it easier to fire up GDB > > and ask it to show me the values > > Oh, I do that all the time too, with both Emacs and other programs. It's > OK. Why refuse help from GDB? C code is supposed to be easily readable. Going to GDB means it isn't. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 13:02:42 2023 Received: (at 65491) by debbugs.gnu.org; 17 Sep 2023 17:02:42 +0000 Received: from localhost ([127.0.0.1]:51297 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhvAQ-0005y7-Fj for submit@debbugs.gnu.org; Sun, 17 Sep 2023 13:02:42 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:4909) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qhvAL-0005xq-3R for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 13:02:40 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 468DE440CE9; Sun, 17 Sep 2023 13:02:23 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1694970137; bh=2aSO7tgyHt23/IVZdN4mtLcWCoBQq5CnF0xQhXLgOBs=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=ehWUc9+KMeWdfQdgv/sNEIEvVbzJzQ04ducDUhffIpq98Vw9wxFwqv8uixDvHyIlU OL/Byg6qaE7x9Qoczc2UNfZqv6WAOxzqnlL5PKYh9gK6auAPffFqawYDLmUUbT8Sl0 +AhvG6i+Vx7uFKdWXJz4HJr8ykNdM6NK+1fFl3eZzvyKoaub//eTdDpNvVYNwFnHFI /7pVT2vxYSCpiaNO3aFn+9wXVLh71JkSsntckZIS60oBjLRCUhQDnYzNnnNILKbOHK 5VWA0DRndcVzT+wlQWZVWdWBc0IyLaCWqkHpDlbXWKVCmnNYWwZWImuZngPlJGUfmk 0hPC9wN6KC04g== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 6CEEB440C84; Sun, 17 Sep 2023 13:02:17 -0400 (EDT) Received: from pastel (unknown [104.247.237.102]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3C9621201D8; Sun, 17 Sep 2023 13:02:17 -0400 (EDT) From: Stefan Monnier To: Po Lu Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <871qexzelk.fsf@yahoo.com> (Po Lu's message of "Sun, 17 Sep 2023 11:02:47 +0800") Message-ID: References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83sf7egitx.fsf@gnu.org> <871qexzelk.fsf@yahoo.com> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Sun, 17 Sep 2023 13:02:15 -0400 MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.007 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Mattias =?windows-1252?Q?Engdeg=E5rd?= , Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Within Standard C, the result of converting a pointer value to an > integer and vice versa is also implementation defined behavior. FWIW, last I checked it's literally impossible to implement our conservative GC (or `malloc` for that matter) without relying on undefined and implementation defined behaviors in C. So the best we can do is to try and avoid those undefined behavior that compilers *do* use to bite in the rear. I still haven't seen any compiler that tries to make use of the implementation defined behavior of conversion from pointer to integer as a basis for optimization, so AFAIK we're still safe using those. Even converting them back to their original pointer (which is what we do with tag/untag pairs) is documented to be well-defined if you compile using GCC. In contrast the pointer arithmetic on NULL pointers appears to be something which compilers have started to (ab)use as an assumption for their optimizations. Hence the need to update our code. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 22:19:37 2023 Received: (at 65491) by debbugs.gnu.org; 18 Sep 2023 02:19:37 +0000 Received: from localhost ([127.0.0.1]:51714 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qi3rN-0004my-5p for submit@debbugs.gnu.org; Sun, 17 Sep 2023 22:19:37 -0400 Received: from sonic305-22.consmr.mail.ne1.yahoo.com ([66.163.185.148]:35753) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qi3rJ-0004mh-PB for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 22:19:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1695003560; bh=9FXOA5mOCsvyWB10RoRdD1wHwpaT36W4Ty8v4E8WUvE=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From:Subject:Reply-To; b=Ey2ITFSPhtUeJR8tWdgEQhdZKAwRr0xp2sLOdAlIrXZFzaRvqlbFT6TWObhvQszAsSRlrpw/Pw7G2/n0zVAGw0X/9hRWwgtb83yi0YZ2F6daNS0NVL3cbwnDtgB4m0HmjOD52Al3pq3HBUeg6O9xWBeTXXeB6UDKwB92pkczwJr0vHAlu1oRiNQShvmLJRmHvk9BXcQqeS17RApYHkoG4skqdmnki/Z+7jyvgjsypxIv07tNWRJfdwo/yA+sooIVe/8oQ5kpOrRUz0kLUDdziejUjYKT0GLGR3d2ofSZnCdUW4xos+dbJ0ACDw3IxC36Nx1XWxi96ThngwCj1tTRzw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1695003560; bh=X3qtCTQPy5RyBB689X47Cmt+X0AEUKUZCrjdNw/w4U6=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=OmNmAHmVANnDrfow25aqavK9AYCNjfmnD03a1Hr98V87fEFZycEEkpSr6osbYQxOd+5bZfF1PT3fnHekClb/NNHjr46QL6gjxyhzKW5kMhomSme9h3Ba/XmoOQpU4r2XPLovg80Mh0GjyyusnbB80gOrxZfsAxZGzBPgxqoIBI8I8mq3cHUfaF+hLnz3Wc1ueiown2r45zGtdZ725zpoupS12ciBKlW3+WYfiOvKRnxl5QkMgxjF4X7j4LjLixTNClpAxDuhKmRwuBkAjsL5oKA81dEUykMwDC1MrM5VgDuVTFe3BrxNQNkNjfs20l7JDloKlxeg16RGE2A0YtDq/w== X-YMail-OSG: ZtNKaawVM1kX2lO27TIsxpCZFHGeeSVVIm_LUAOtrXbcnt2UkHNgkCjM_OHAZVE CFq3F6hjRQykD0ie0J5udLT314cNUw7V5oIxgiQBL3ZBK0Zm4KXpk_YNXl2dCl0Dl9arflYbvbRQ ElTVPhoEMhS2Yy5KP3Kl14KkKtiANvh7.hLYonRlWubUqkE3WWkwLJw8jWpJbVBMul_wqwpmckbw 2FW1flzs4zK6CF9puQukOvPDJ8nuDMA49tIVTt1_s8v0pArgLtfmiMqFNCOtNxqml2kEPMxQU551 pz_SQ1vLvbF1J_m6HdaPzU8uZgQYB0h_BHHKoQeKT8HsN7NzOqK6mO4dgVEqXqtTyrGO1YbxCHK_ ajwcLnnFozvGwTy3crrnlqLT6jSLJUkeUZrwnb_4AJ7fvScpGNJrEW9lkpIsUy_B5sX8NtsQQ7YZ MkwnVzXSAjeu3eX7xghmMdaCrLBOTI71_KrLaCBQ7k4fUBRBTXMqW1DfsbDhESMJIiRGF0AzRk9Z 5BGYbwvxM3HaVIEniztzkWWOmn9gfSdQjm84tAbhfm.XAxzBHkCoY_MrvC1elRUp55Wx18vSSPTX HUETvoLr08AdopnMPQJHWkMDEedUgqUYxQJ0ym9Zt3Ml6f6bNGh5nC69yIGyJp.Q.PfEuvWeMV8X P9t6eEJPJIzOszw8dkYtM4C7A1tIQh3SR7EBsi9iTUx1KV2ehbQby8PtzmH2sc5gpH_2_4kYyiNb TLLZw8ZX1EE81RA9efkACt7FfCOZTT2e9LDMGMd6JyKgCXwF3L6sTYG1wEwpAgDxvTcscV4lWQWt 3DWG5xLSTH6s.NMNN2j3s1tiuT0nKf4nUiUfIB6boknEJ8ON.c_rSygEDPn2HvXBpT7As1AusS3i OvZrm8b4jbaCA.fsf4dit96x6XZlIxwGf_sewUjummjryvDNqi2.n5HQGY2E58AEDUXrM3jVmR6T cBn.GE2JXlCDpkLSEDqtSkXjtoUGGYT6TQg604iLUnKGZzxjxFKhmj0EeUTJD.d7Da30YuFcRsy7 DsdB2_XqJAv1v7i_OOd2NwW9in0903vPVMq_lS8xyLy.rrqx0fVFNOvB6sT8DL2Tu8yxb17u5SSA c4yNkeKJwab8eLNptPGhnio_V6i4a88EOJhh2w7C1632iZhBUGaXc1x_w3maGdqwd2Pdjvh7Cz7z l74ablBqDCA9DcCM8_SIVdnrmp11vo_f73KB57heuUyXY5F7JODcqnDNwraj320l7NrnYot31VDj X.b3wfnvJyjJRD._.w5ukLPKV_.p6DBaZDM9Hfyfr7Z5Vv64NOz7zrcxVd.MsSFaVCaM1FuF0s.b t5a6eu7MaOEqZ2s7bnCgYb3j1xzZ8PXg8MdN6cYMFhB9ME12Z.Sm.E1y8nl6vj3A70Xi9bvbo4DY EW.jK_vqafUIOChQTcrAHjHRQi5smWHXCl43FAmFOoeBx3K7FozV.qDvcYAxjmR2YulY4pOKg4S9 IgXuQjQgfnzsdmPN18Bi4.B.lHSLL.0Zys_IrxAg53Gb5sr_aShwPP9450sMvnI1_VRK4J7x0j6C e_MSlL05jhksaZz6Rpdis8Wcj4wfe8PLaHda7zbekWuvcBmY996at9KseQXjHObce93bbnQtrw5n HoSEhcaSArKzTmCVerOhTX81hU1JfyDCuyEUJXFmMjlFh.aMB7WEV3y4BT47DofLJgx6bmJGF1Xe K_4V_SayoKFNVWgxfH8cxNrRZauLllSwhPNhLD1Mvl08HFWOP0Er2rAB7a9Wf9GMc7HvDJvKvupg IbAcYKZ3yh5sA1SGycQsSNW3CAHZyIp.hi2VBoEoFl_bu6YTzNO3.u0DAkOONB.YwH3.FAx_8Bfv iXFrxgdNr_XmnrPcLQlZIAaci6vzJU_PhJqB5CTXeD2O5_fFo6F4SsJVMO4mDwLXGzyTmaDZR9A6 b1_5cKmgfzBN38iia8GzSGFtO193ArgjrKw0Z2ORrA8M1GGNZb41vdVudnCR37HjE7PEr3k7m_YN kRtwD5LgJqX4uV_dDORTwIJTd31byvpvHsc7JkdtsrNsFnBja7XO7o0rEoqyoWg0ZnaHYai37aCE RdcshApYl3doP3mgljHxxgU.eu_5XeQIpRFeWSgBrEzeYkqtKOnQPycUNQjJQOn5SOT4PaABosHT pzOiT9.lTfbXM5dZLlR9ZYIHyG7wLzakehAe2A8FIznmDKWVC1lBVqj3hmjoykjw9SuhUJFaGdcB jf6xxUEex1qjKczpNpUcifK_WnPO7FiLSeDPwd9qKkNBuZj69vJFG5feQnE2Lm5.lPzFC X-Sonic-MF: X-Sonic-ID: 6d33e53f-c152-453b-8210-9d2bafff087c Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.ne1.yahoo.com with HTTP; Mon, 18 Sep 2023 02:19:20 +0000 Received: by hermes--production-sg3-55c667b499-7t45b (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 3d33fddaff326e8c68172b91936e1586; Mon, 18 Sep 2023 02:19:14 +0000 (UTC) From: Po Lu To: Stefan Monnier Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: (Stefan Monnier's message of "Sun, 17 Sep 2023 13:02:15 -0400") References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83sf7egitx.fsf@gnu.org> <871qexzelk.fsf@yahoo.com> Date: Mon, 18 Sep 2023 10:19:09 +0800 Message-ID: <87jzsoxlya.fsf@yahoo.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Mailer: WebService/1.1.21797 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Content-Length: 683 X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Mattias =?utf-8?Q?Engdeg=C3=A5rd?= , Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Stefan Monnier writes: > Even converting them back to their original pointer (which is what we do > with tag/untag pairs) is documented to be well-defined if you compile > using GCC. Only if the consequent pointer designates the same object as the initial pointer (of which I see no scrutable definition within GCC's documentation.) But in practice, this doesn't matter, or so I thought. > In contrast the pointer arithmetic on NULL pointers appears to be > something which compilers have started to (ab)use as an assumption for > their optimizations. Hence the need to update our code. Hmm? Where in this thread was that fact established? Thanks. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 22:28:02 2023 Received: (at 65491) by debbugs.gnu.org; 18 Sep 2023 02:28:02 +0000 Received: from localhost ([127.0.0.1]:51719 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qi3zW-000537-6j for submit@debbugs.gnu.org; Sun, 17 Sep 2023 22:28:02 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:31227) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qi3zT-00052Z-5H for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 22:28:00 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id BD498442860; Sun, 17 Sep 2023 22:27:45 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1695004064; bh=mbmzy8mI8PqhKR8KYUrBSll0IUUyTvdWXHgeOlTak7s=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=jL9urIaZgm1Em84Qfiyt9eDen9m16ch22b0cgA6w17JQDEHVdJTxzaAqNdnTr8wl2 T1YD9/QzJuc6EUaq84/+9XU+YziwjCISvKh3ohEvBpX72NKVbAtpp9UGq+0KmyFedB DMpw3fcix2lEndbQBNqZsvHGrfo9CNEHzkQySoyBkX72YSspQkcYk5TSPJx9NqepaD /aS/4SCufRTYv8zmCsIU6U788Ap9GiX7KGwM8Rg8Ca0oCcn3eORiQSC2G1w6b7h9Vg 32G0qOpGu3CTbV/LqTsVXId+CDz/fGn/y27YVSrsI1xqpAnCtGL3KIc1JB+EVGMTaO 9HNJ0qGz5jIrw== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 7E93144284B; Sun, 17 Sep 2023 22:27:44 -0400 (EDT) Received: from pastel (unknown [45.72.220.249]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 51F821202C3; Sun, 17 Sep 2023 22:27:44 -0400 (EDT) From: Stefan Monnier To: Po Lu Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <87jzsoxlya.fsf@yahoo.com> (Po Lu's message of "Mon, 18 Sep 2023 10:19:09 +0800") Message-ID: References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83sf7egitx.fsf@gnu.org> <871qexzelk.fsf@yahoo.com> <87jzsoxlya.fsf@yahoo.com> Date: Sun, 17 Sep 2023 22:27:43 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.022 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Mattias =?windows-1252?Q?Engdeg=E5rd?= , Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> Even converting them back to their original pointer (which is what we do >> with tag/untag pairs) is documented to be well-defined if you compile >> using GCC. > Only if the consequent pointer designates the same object as the initial > pointer (of which I see no scrutable definition within GCC's > documentation.) IIUC the "if" here is talking about the fact that if the object is deallocated between the two casts, then you're on your own, of course. >> In contrast the pointer arithmetic on NULL pointers appears to be >> something which compilers have started to (ab)use as an assumption for >> their optimizations. Hence the need to update our code. > Hmm? Where in this thread was that fact established? IIUC Mattias bumped into this after he made a few "innocent looking" changes to the vector allocation code. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 17 23:08:41 2023 Received: (at 65491) by debbugs.gnu.org; 18 Sep 2023 03:08:42 +0000 Received: from localhost ([127.0.0.1]:51743 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qi4cr-0006OK-KP for submit@debbugs.gnu.org; Sun, 17 Sep 2023 23:08:41 -0400 Received: from sonic312-25.consmr.mail.ne1.yahoo.com ([66.163.191.206]:42340) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qi4cq-0006Nz-3n for 65491@debbugs.gnu.org; Sun, 17 Sep 2023 23:08:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1695006506; bh=evsDqaq8PUv2N4T0ZCibikkIxnXiuZSc0xbCx0cgsUk=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From:Subject:Reply-To; b=BHssxufJh1Y5Vi4P8uqmVZjYa90pR67SS41h0pm4a7xYXlO7i/saTHjUOXqh2rM/C3h7tQ2Ffaji6RDloO76Nz96HjnK1kgSSvzM0TgSye1uTr7D2Re6CJHYtuIEvb4TkNY8X+bu22WNA6wcBYorjlPANGGEJHA/4FI6qOzMlBr7+TrUTRMzGy4ZQMDTXVHU6F2TsmBR49EA/dHA3S+Nnt2ULlr5ovO7ixLAu2HvfOWEBXyYobQGPNF0l7g/geMqFA9u6acwEczdRrHUrWwydam5CCp0CcVOksn8Mj6wSlPAB7bbJRHd1hfu1cVegZU15KdVBBeOHngvx6VjQsBrlg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1695006506; bh=OiyTbTPogJ48jbvo0IxiC2gAnLhrhX1FrCDxxSmaOuF=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=K8KgjMHxCadJyXdaQyUuY9OUp0bEpEXyugxIAu4rV2h6fFpN/TOkq1c3j6C5sU4TIsAkeqlBmCkzzu3dYXh++5A6p1BUBVhuZBZW/eUH1pCR9ZuAvVjzVgQygyt6XKHwMYD9ZYr0GxMtoxRKCm+yEa1ki1aSRm2dqMSuqgAluJAZrOE6B/s6wbdYlzDbRARoOFsnOkhBZFhdbC6nAsiQQoqF70KD5KBl1qAvf67M03XUh+GYEqab9rW9PP5jIu1GRXHXd+KWKvYe/MohaDjVDofnhsSww2XJ5dPO5Yh3gZlQip51WZx3SiZ1qsrEu79kRZ7ibVBbDEDmkmQHthqvGQ== X-YMail-OSG: MV2B8NUVM1ngok2SLQtMD5J4LdK12xPLT2Cf9_CXUMYl1_nIOM9SnrJ_HwEcl24 cm4Tpit7GI0QOBV2E.xl6LYZ5ojsfvEXi4_eMMXshL5awjrnWgufa_dhlfwhiuhj0Gj9TQ09kIsf BKn0xasEsK2qcl6rz9qt8El4uW1f0SFp8iXkh5P8vZ.ZfDQPTgfhl.Pn1A4sbJypJiwn1U6.XnIs HxBf99uzj0Xh18EVb3dYlG1MZbz4E1QJcFXMxLGx9dYFio4PUBTE0iq49fMfClccYNhiUdYKp4nG Tq7.taI.BQhvRJxP4LhlUdK9W7yyfNx52nkmCJmm4trguAPT59Zw.SEeiEWRQlHgx3COwW12z_iI 5hPnGWALEeHU3mvulpSOjOX5nogC5H3R6u4kw8IVqr5Z5NRuS33wAd66p5LFWmjbsp0VJnVSyA4H 9VQpxwnUvT2Vc2OfyH1kIUp1WOQGv2LpVV.yrqV4tQDoJnRWmVKNURSAH.WvIir6H_qsEutKMsz7 zRfHyOuIXXNgbc4WXyp01dkz9BZYfFjVWi8f7hWKe1xYDTd5ttew0wQQTXO.sPy76DUxcpuWzTbT zkXLaKw5mKCaaPrHiaMfV4P44Aj65uZTGKRq9nvkTyBB5ctwbsuWyC3X4o31S1.bksqE8eBAmIgc nLaclpVqzR_24VNY3pFtiHMm3_DqFIXoRgGa_qrQ3TgRClCL_aDVQfFTaIdm9W9LmrHOvP21o1eb DN2dETO0hRP68uttRie6ToWq.Jyl4RvGJ8qJR9jaQq3Ek5hikZbE1yP1tTIUq_ySxUsJN_2ev2l5 8at.PfTmEvpFQvsTf97s57PT5X9l4HcbHfW7WWIoYbS1mihuawmxM07IQtf4Rlt2QPBHcvGhk.e0 JAIeMlWzl_QJTyqAkPqOyI6PraccRYpm3Wtn5VDKV32DyA2Cps5lkgBZHM5leXGBRrGN8e57zXT0 87O5t2A9.fqeE7Wa_NalQLvkDKe5BAIWgObsSTESRFoak3iM3b4ktPlu7QCAfgBAwCE2H0b2jpeg Mxn4CSlq0hwokKQ3MPynhHyjMpbwCsd.KAwHIFKs9ctndtdSiKeynavmWkQRrQQLZzOZP.gY3NgN cLsv6ZiYfWDpvLsLDnMwY6B2tRekkQaNl7IdWe.Ankt9INiMNGbwgBh0wzR6c.L9LIJaTkSovNTr lYIXTwKtv9Supm3t5vCh_7VNOBc441cyEEJ5tLWXzaf.gxtAxkIMg9TpnPLQdJUlsAj4DrSjIPBt RjhMVvhbyg5lGKVO6CENixDsyiS751Q2C7g7l4jWwXzlkJTzhyKz_B3WQagO.ZDzxvrTvWeOi7GX ukLGgRLmV7W0hjMzCD76TlSxoqQ3Bgp4jClajQO0DKOpxnBQUYZqiYAv11FeIER_pMb7LBJLy344 XQTtmUeBZaYfxldSjVYnR5Ps286q7XAmbwDRY6kOKKZIIk7oKIj1xjvjPUDuDcxjjzFAB8SUaRUS 9YxVX9eW.WAEr.mBTNv1lfKeraOrmoVbNaFIqEwuoBYc4EkvR_gJerUhDsMU61TI4Stw.cD.JK76 0mm7sEUuk3HftZcWU25z8UWfipaQaGh34kZ9OSHXQv7SvhYfWd4ANHVpb041Icdc3aJGyNzz7yOJ ziTTzhms_fg1KrEQ1sf0lp03nKyzSzUk6AxwUI.akgmoTYvs0r9DdfUDL7aovLHExzSLlCjHUrHV UkYYC7D5NFfseqimGaL2108C.Nk.WRf_Vxh5FLn0urHYWxsi1dPaVGLhVcOWexj8nMhkfEUXcgCC WrpDLR9K0o_3dFTlDJ.y5jzdudNVaTLylWwqjaGJBbF5Nyyd1vIAap9zuj7jlmJEI6sGCEKgC6_a gjGBucv2gBMYGvxwaWfzlKnoA3_ZJwwF4bBf7fWLzN97ht71lIckdsHSvADs1BM2LQvrLb4CsZFW tUg1ihxrc7yMJbki80PVqsVEAfzpzu93..vTUyYGUbyryobNXq6snRxpfj1aBVHif1tEf.q_il4N Nd7D0LAdACVVugWi5aU8uMpKJzfSX.QBJ8dTwjBcnoaamYTCAI5szZbWrkAp3gGBmvdFWnYrl0HW GGX9mxkt9uKTeVzxjgSXJTAncRwusnnUkXnTXHRbBvrPhmJsN.r5mts8s739Q.ke3ehgPGWkcC76 EnhJVth2.zGANsI0_D1hwdzjuiO4i_NWyfd3gGfxzcGWd_VygfJ8GhLRhUvfPAtouw7.NyM2_g2A pGL4qwYiroxMc22hMOfpTxhPh.okLS_ZHOMmE5Cl1eUXUp2Gxwq30INxH0kbvf.TFh3w- X-Sonic-MF: X-Sonic-ID: d246218b-974e-497a-9889-30e14c95ac1b Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.ne1.yahoo.com with HTTP; Mon, 18 Sep 2023 03:08:26 +0000 Received: by hermes--production-sg3-55c667b499-gg78d (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 95dd2425c22f76ba11f043122077d334; Mon, 18 Sep 2023 03:08:23 +0000 (UTC) From: Po Lu To: Stefan Monnier Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: (Stefan Monnier's message of "Sun, 17 Sep 2023 22:27:43 -0400") References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83sf7egitx.fsf@gnu.org> <871qexzelk.fsf@yahoo.com> <87jzsoxlya.fsf@yahoo.com> Date: Mon, 18 Sep 2023 11:08:16 +0800 Message-ID: <87fs3cxjof.fsf@yahoo.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Mailer: WebService/1.1.21797 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Content-Length: 1782 X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Mattias =?utf-8?Q?Engdeg=C3=A5rd?= , Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Stefan Monnier writes: >>> Even converting them back to their original pointer (which is what we do >>> with tag/untag pairs) is documented to be well-defined if you compile >>> using GCC. >> Only if the consequent pointer designates the same object as the initial >> pointer (of which I see no scrutable definition within GCC's >> documentation.) > > IIUC the "if" here is talking about the fact that if the object is > deallocated between the two casts, then you're on your own, of course. Nope, (gcc)Arrays and Pointers is quite unequivocal: * `The result of converting a pointer to an integer or vice versa (C90 6.3.4, C99 and C11 6.3.2.3).' A cast from pointer to integer discards most-significant bits if the pointer representation is larger than the integer type, sign-extends(1) if the pointer representation is smaller than the integer type, otherwise the bits are unchanged. A cast from integer to pointer discards most-significant bits if the pointer representation is smaller than the integer type, extends according to the signedness of the integer type if the pointer representation is larger than the integer type, otherwise the bits are unchanged. When casting from pointer to integer and back again, the resulting pointer must reference the same object as the original pointer, otherwise the behavior is undefined. That is, one may not use integer arithmetic to avoid the undefined behavior of pointer arithmetic as proscribed in C99 and C11 6.5.6/8. and directly cites the apposite portions of the Standard. > IIUC Mattias bumped into this after he made a few "innocent looking" > changes to the vector allocation code. I'll let Mattias fill us in. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 18 00:10:43 2023 Received: (at 65491) by debbugs.gnu.org; 18 Sep 2023 04:10:43 +0000 Received: from localhost ([127.0.0.1]:51794 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qi5at-0008LK-9s for submit@debbugs.gnu.org; Mon, 18 Sep 2023 00:10:43 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:52075) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qi5aq-0008L7-L2 for 65491@debbugs.gnu.org; Mon, 18 Sep 2023 00:10:41 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 36F838072D; Mon, 18 Sep 2023 00:10:27 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1695010226; bh=kz2EJuIURZAO2Sy9LFhAjYc56jYNZRkvFnEl6w76U6U=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=DpJPwn6xkAg35se0oEaKKrb9IxbZHfSFFUSRZqJOS7y6VMZ0iCnYDJ/ZfodY+AvTi E2YEJDB3iBe1pqRtMac79rLan8AK7XOcs8kYfMBtzNYigomNH57zVfAEFBN5ZjetVa HFZmEW6puNKeD6fEcwQWqTPAlXXgC4en78aNao4XDoVyodcNulNobZWTTa/gET3DE3 eXSsLm+4sGToUpIiwzQgw/ZKLCdSU5cF2HuZYKUtwNyhzQ6kKVu8qf2mJPHm/mrtZk DfuwywT1EEqEaq4hk/c5ne31EITgwGyBNFN190aSHO3CYsnHGNXD6fRqJ+vYDGWh/R XJY383ZC3zXoQ== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 1ED85802CD; Mon, 18 Sep 2023 00:10:26 -0400 (EDT) Received: from pastel (unknown [45.72.220.249]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E577D1202E6; Mon, 18 Sep 2023 00:10:25 -0400 (EDT) From: Stefan Monnier To: Po Lu Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors In-Reply-To: <87fs3cxjof.fsf@yahoo.com> (Po Lu's message of "Mon, 18 Sep 2023 11:08:16 +0800") Message-ID: References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83sf7egitx.fsf@gnu.org> <871qexzelk.fsf@yahoo.com> <87jzsoxlya.fsf@yahoo.com> <87fs3cxjof.fsf@yahoo.com> Date: Mon, 18 Sep 2023 00:10:25 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.019 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Mattias =?windows-1252?Q?Engdeg=E5rd?= , Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > When casting from pointer to integer and back again, the resulting > pointer must reference the same object as the original pointer, > otherwise the behavior is undefined. That is, one may not use > integer arithmetic to avoid the undefined behavior of pointer > arithmetic as proscribed in C99 and C11 6.5.6/8. Then they're (still) warning about a different situation than ours, i.e. the case where the integer is not the same, e.g. you cast to integer, do some arithmetic giving you a *different* number and then cast it back to a pointer. The way they write it means that even if the number is different it can still be "well"-defined (in the case where the pointer is still pointing to the same object). I'd guess this can happen if you the integer arithmetic ends up moving from one slot to another inside an array, for example. But in any case our TAG+UNTAG is simpler since (barring bugs) we always cast back the exact same integer, so the condition that "the resulting pointer must reference the same object as the original pointer" can only fail if the object was deallocated in the mean time, or if the integer was too small to contain the pointer (and both of those conditions should be true in our case, barring bugs). Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 18 12:11:08 2023 Received: (at 65491) by debbugs.gnu.org; 18 Sep 2023 16:11:08 +0000 Received: from localhost ([127.0.0.1]:54214 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiGq3-0004R9-NH for submit@debbugs.gnu.org; Mon, 18 Sep 2023 12:11:07 -0400 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]:44145) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiGpz-0004QT-L4 for 65491@debbugs.gnu.org; Mon, 18 Sep 2023 12:11:06 -0400 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-50300141a64so3672520e87.0 for <65491@debbugs.gnu.org>; Mon, 18 Sep 2023 09:10:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695053449; x=1695658249; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=fIbnGQM88diokKIgKiS+kV9pfHTC0uhmFoKqgeb3wUA=; b=KwZ7FJH5q1ETmHS/fc3lVaneGbBKdVRnG5SpswVHcWB8UWca3qjgAgcnvKma+jYjuH J2aAvY3EfmhuF2zHme+heBO0Xmes6MymF5HcBtuMJCgZlXK3c4941E/sGpc2qm/KhJqo 9jirgSExE5HvH+qVHH+beAWJT/6Xf/EBkOb+Th4MOpXJXBnygYaXsKpy0QznVWwqbt1S /tfd75TxEH6qpkBE3Wk2Q66TTJSr6AF40p54kSECcpJ0obTDq1xd0a2dvwKNoQYIV2jF l7HmlkZdYoiL8P4jXMvQvvypak2ByOwYXJOjEE8FrksckvdlM6MsEE97Yn0qb8t7nXbo 9zZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695053449; x=1695658249; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=fIbnGQM88diokKIgKiS+kV9pfHTC0uhmFoKqgeb3wUA=; b=R/tObmx3/EnvZnwZ3WGCyIrj7RMVdKp2CLm8C9OPqnPeyvwiDlrMPqGBrQXTFUAiq+ ErdQfzQed/1YknXLakO1E/HPpO51F69+82PTWgJKQlpkTJiINojzufdk7qrWXPeXH8fw 4KERfn7QZeaOEaX831+FFZMQJ62SV1K5ltY/rBCITR2Dr7Qy8DWV4sO5wZWgO1ubVZju IanpY/AKfgO0NyLaFOJQPDFx8RQ8X8ESDjWEz9ut80j2615jLdaTOXFSHzjknb9eE4Ll tCv0QC4egLEIYu/DRdH3uEjYRelUmK0/kKZ0pVmCag9/nkDMUrH/nYvV+fIMw65mM28P YOFw== X-Gm-Message-State: AOJu0Yx7myCoUmR4sJOEAxOqST9jjluTssv+EfnqXGGfwZXzuLW21GuJ r8fUgD1klvVKul2aj9XvWIA= X-Google-Smtp-Source: AGHT+IGeuxlyjJpIUbLO8R9BhB3HMtsSoChO8FwEk/WZ4XrRnF3aXPbBHazWplgTN2S765d4n6+1Sw== X-Received: by 2002:a19:655a:0:b0:503:985:92c4 with SMTP id c26-20020a19655a000000b00503098592c4mr3971220lfj.52.1695053448850; Mon, 18 Sep 2023 09:10:48 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id a28-20020ac2521c000000b005008cd93961sm1861258lfl.192.2023.09.18.09.10.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 18 Sep 2023 09:10:47 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: <83jzsog35g.fsf@gnu.org> Date: Mon, 18 Sep 2023 18:10:47 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <95C2D44B-BA1B-47F5-8279-B3A3B8F7EB39@gmail.com> References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> <83o7i0g4i9.fsf@gnu.org> <83jzsog35g.fsf@gnu.org> To: Eli Zaretskii X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Paul Eggert , monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 17 sep. 2023 kl. 18.44 skrev Eli Zaretskii : > Feel free to add any comments you find useful, that is always welcome. > But comments about complicated code have one downside: they need to be > updated when the code changes, and people forget to do that... Eli and Paul, thank you very much. I'm reapplying the patches (that we = now know shouldn't cause warnings), with added comments with respect to = the vanishing subtrahend in -with-wide-int builds. The conversion at the other end, TAG_PTR, suffers from the same = potential problem but in reverse, which is perhaps less obvious to the = casual C programmer but nevertheless a danger of the same kind, so we'd = better do something about it as well. I'll be back with a proposed = patch. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 18 13:14:11 2023 Received: (at 65491) by debbugs.gnu.org; 18 Sep 2023 17:14:11 +0000 Received: from localhost ([127.0.0.1]:54290 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiHp5-0000q7-0D for submit@debbugs.gnu.org; Mon, 18 Sep 2023 13:14:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46378) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiHp0-0000pk-QP for 65491@debbugs.gnu.org; Mon, 18 Sep 2023 13:14:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qiHom-0001dY-5w; Mon, 18 Sep 2023 13:13:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=sXqea6OxXFjCj2ru8CskxgMn4Vc2EFWtL70D1o7eOYk=; b=cYKuME245te4wcfJxqyC FKGo1eq9PJxOHfJ/8Yf+YANS7EhlfZJ2x+HeqnBOIG9HUG9UBkRZhJxzZj6hUy/KQ2Gtjxrx4TBrv axlz3HQCxdT9fxRUbj3hUqMHR1RsNvDYu4ODJMJOTjZjhINugd6EXkMuy8boDSCo+FUesW74IyguP oHdMbsxpzc+IvPYoUbBAUpJdqAagrGGxL73I33izaOY0ga8cak0PlGP5xI4Lmm7GGUDUbugvROiOU fUzPlz9a9AiD2stiz9s3OUwCSmw7MO75BeqZmvVoltGLzC6XeO4bSJqjhYwJtz1/4FHCqelZQy3Nr FyPArynEyiunEg==; Date: Mon, 18 Sep 2023 20:13:45 +0300 Message-Id: <83h6nre75i.fsf@gnu.org> From: Eli Zaretskii To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= In-Reply-To: <95C2D44B-BA1B-47F5-8279-B3A3B8F7EB39@gmail.com> (message from Mattias =?utf-8?Q?Engdeg=C3=A5rd?= on Mon, 18 Sep 2023 18:10:47 +0200) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> <83o7i0g4i9.fsf@gnu.org> <83jzsog35g.fsf@gnu.org> <95C2D44B-BA1B-47F5-8279-B3A3B8F7EB39@gmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, eggert@cs.ucla.edu, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Mattias Engdegård > Date: Mon, 18 Sep 2023 18:10:47 +0200 > Cc: Paul Eggert , > 65491@debbugs.gnu.org, > monnier@iro.umontreal.ca > > 17 sep. 2023 kl. 18.44 skrev Eli Zaretskii : > > > Feel free to add any comments you find useful, that is always welcome. > > But comments about complicated code have one downside: they need to be > > updated when the code changes, and people forget to do that... > > Eli and Paul, thank you very much. I'm reapplying the patches (that we now know shouldn't cause warnings), with added comments with respect to the vanishing subtrahend in -with-wide-int builds. Fine with me, but I still hope Paul will do good on his proposal to add comments... > The conversion at the other end, TAG_PTR, suffers from the same potential problem but in reverse, which is perhaps less obvious to the casual C programmer but nevertheless a danger of the same kind, so we'd better do something about it as well. I'll be back with a proposed patch. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 19 09:29:04 2023 Received: (at 65491) by debbugs.gnu.org; 19 Sep 2023 13:29:04 +0000 Received: from localhost ([127.0.0.1]:55632 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiamm-0006W6-Cu for submit@debbugs.gnu.org; Tue, 19 Sep 2023 09:29:04 -0400 Received: from mail-lf1-x12c.google.com ([2a00:1450:4864:20::12c]:49162) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qiamh-0006VN-8n for 65491@debbugs.gnu.org; Tue, 19 Sep 2023 09:28:59 -0400 Received: by mail-lf1-x12c.google.com with SMTP id 2adb3069b0e04-5031ccf004cso3168083e87.2 for <65491@debbugs.gnu.org>; Tue, 19 Sep 2023 06:28:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695130124; x=1695734924; darn=debbugs.gnu.org; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:sender:from:to:cc:subject:date:message-id:reply-to; bh=Wn5dotgV6xxJRxCnGMa4Nb8eHBkKFDx7cSfFFfiJ+08=; b=mlMlt3xQR1Iu+kQO3QwN+eqjYlP9ynJeQWaiZ1Xw0xlYrDvKC50Y0O0rt4UzsXIhNw SaT4DXgg+yXzWCOjxicpVyr9bxrHQWemdiGm17OvD3bGVLvp2cY2j38dmHfdPDs25M/G xM4wcJm8tlJHtf+PCkuN7kHRfg+GMDD/WRDe7q92yj1gucPTAw4bmzZM/pymTH8OhCtX JUTUcBgfSv65XyWsAUkgvKOOdsqt5C5wneROmXWd37Cy9/pwYtRs7jDFPh8dTMsVAlFC X0eC2rTXCoh9ZdKn/6UPQM1HHqi0SqgBEd34PuQK4B8buzrHUPBBVQPkXW6AiTd0xqq5 M+wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695130124; x=1695734924; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Wn5dotgV6xxJRxCnGMa4Nb8eHBkKFDx7cSfFFfiJ+08=; b=eQ6qIk0rNlgPl5QrYxRPSKPDwhGxx/p/XuNF8XdkxkMyWmkRcoj1p1T/W8UGiYt+AS xdotx7IdnHUzYO9OaV5XROIdgG8h4Dyz6Kw6bNT7huCkwP39fgQ5JIgVkPzVloF38Gpt 09nF4wq32xrQ84bOk8DAQGrln1nuG6myTkRugRpEMfoL9/DdKRkw8wPvgP852ibXdn+M EvuLCiLWy9AnINgdcmuF0yhYXhmXtV3bMH3/JMEtB5chw9GF5gZMBPkpfdBOzrh0L/YS 9Tama3qC4ARtIytsK9jmOeHfH8wxxQdO6v6LYpgNss2/JslP26ZyV3v4hDHPlBNC0vFC xAZA== X-Gm-Message-State: AOJu0YxMJmcfhsubG9JQJp+DXwSX6nMQvEUQOJRg11RZyQX1ULyAmlxi e51WkYL0flSCBT23TKxGNPU= X-Google-Smtp-Source: AGHT+IHU4bc+A/R6QhUbivfVivrbUpTJgI0SwmvjH8VDpzKzRZrfrCa0rooOvJyDvxf7R+PF96C4qg== X-Received: by 2002:ac2:5f79:0:b0:500:8fc1:8aba with SMTP id c25-20020ac25f79000000b005008fc18abamr9375772lfc.26.1695130124039; Tue, 19 Sep 2023 06:28:44 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id x27-20020ac259db000000b0050326307ed8sm498959lfn.102.2023.09.19.06.28.42 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Sep 2023 06:28:42 -0700 (PDT) From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= Message-Id: Content-Type: multipart/mixed; boundary="Apple-Mail=_F041D8D6-60C4-40A7-8567-C866BBB314B6" Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors Date: Tue, 19 Sep 2023 15:28:41 +0200 In-Reply-To: <95C2D44B-BA1B-47F5-8279-B3A3B8F7EB39@gmail.com> To: Eli Zaretskii References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> <83o7i0g4i9.fsf@gnu.org> <83jzsog35g.fsf@gnu.org> <95C2D44B-BA1B-47F5-8279-B3A3B8F7EB39@gmail.com> X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Paul Eggert , monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --Apple-Mail=_F041D8D6-60C4-40A7-8567-C866BBB314B6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > The conversion at the other end, TAG_PTR, suffers from the same = potential problem but in reverse, which is perhaps less obvious to the = casual C programmer but nevertheless a danger of the same kind, so we'd = better do something about it as well. I'll be back with a proposed = patch. Here it is. Very straightforward: just flip a cast and remove unused = definitions. --Apple-Mail=_F041D8D6-60C4-40A7-8567-C866BBB314B6 Content-Disposition: attachment; filename=0001-Don-t-use-pointer-arithmetic-for-pointer-tagging-bug.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0001-Don-t-use-pointer-arithmetic-for-pointer-tagging-bug.patch" Content-Transfer-Encoding: quoted-printable =46rom=209072db6ca24df50dc8dc66cd50ef638b8aac8a1b=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20=3D?UTF-8?q?Mattias=3D20Engdeg=3DC3=3DA5rd?=3D=20= =0ADate:=20Mon,=2018=20Sep=202023=2019:16:05=20+0200=0A= Subject:=20[PATCH]=20Don't=20use=20pointer=20arithmetic=20for=20pointer=20= tagging=20(bug#65491)=0A=0AThis=20makes=20for=20safer=20code=20when=20= tagging=20null=20pointers=20in=20particular,=0Asince=20pointer=20= arithmetic=20on=20NULL=20is=20undefined=20and=20therefore=20can=20be=0A= assumed,=20by=20the=20compiler,=20not=20to=20occur.=0A=0A*=20src/lisp.h=20= (untagged_ptr):=20Remove.=0A(TAG_PTR):=20Cast=20to=20uintptr_t=20instead=20= of=20untagged_ptr.=0A---=0A=20src/lisp.h=20|=2015=20+++------------=0A=20= 1=20file=20changed,=203=20insertions(+),=2012=20deletions(-)=0A=0Adiff=20= --git=20a/src/lisp.h=20b/src/lisp.h=0Aindex=20de6746f1c07..f0eea0c1f28=20= 100644=0A---=20a/src/lisp.h=0A+++=20b/src/lisp.h=0A@@=20-920,20=20= +920,11=20@@=20#define=20DEFUN_ARGS_7=09(Lisp_Object,=20Lisp_Object,=20= Lisp_Object,=20Lisp_Object,=20\=0A=20#define=20DEFUN_ARGS_8=09= (Lisp_Object,=20Lisp_Object,=20Lisp_Object,=20Lisp_Object,=20\=0A=20=09=09= =09=20Lisp_Object,=20Lisp_Object,=20Lisp_Object,=20Lisp_Object)=0A=20=0A= -/*=20untagged_ptr=20represents=20a=20pointer=20before=20tagging,=20and=20= Lisp_Word_tag=0A-=20=20=20contains=20a=20possibly-shifted=20tag=20to=20= be=20added=20to=20an=20untagged_ptr=20to=0A-=20=20=20convert=20it=20to=20= a=20Lisp_Word.=20=20*/=0A+/*=20Lisp_Word_tag=20is=20big=20enough=20for=20= a=20possibly-shifted=20tag,=20to=20be=0A+=20=20=20added=20to=20a=20= pointer=20value=20for=20conversion=20to=20a=20Lisp_Word.=20=20*/=0A=20= #if=20LISP_WORDS_ARE_POINTERS=0A-/*=20untagged_ptr=20is=20a=20pointer=20= so=20that=20the=20compiler=20knows=20that=20TAG_PTR=0A-=20=20=20yields=20= a=20pointer.=20=20It=20is=20char=20*=20so=20that=20adding=20a=20tag=20= uses=20simple=0A-=20=20=20machine=20addition.=20=20*/=0A-typedef=20char=20= *untagged_ptr;=0A=20typedef=20uintptr_t=20Lisp_Word_tag;=0A=20#else=0A= -/*=20untagged_ptr=20is=20an=20unsigned=20integer=20instead=20of=20a=20= pointer,=20so=20that=0A-=20=20=20it=20can=20be=20added=20to=20the=20= possibly-wider=20Lisp_Word_tag=20type=20without=0A-=20=20=20losing=20= information.=20=20*/=0A-typedef=20uintptr_t=20untagged_ptr;=0A=20typedef=20= EMACS_UINT=20Lisp_Word_tag;=0A=20#endif=0A=20=0A@@=20-943,7=20+934,7=20= @@=20#define=20LISP_WORD_TAG(tag)=20\=0A=20=0A=20/*=20An=20initializer=20= for=20a=20Lisp_Object=20that=20contains=20TAG=20along=20with=20PTR.=20=20= */=0A=20#define=20TAG_PTR(tag,=20ptr)=20\=0A-=20=20LISP_INITIALLY=20= ((Lisp_Word)=20((untagged_ptr)=20(ptr)=20+=20LISP_WORD_TAG=20(tag)))=0A+=20= =20LISP_INITIALLY=20((Lisp_Word)=20((uintptr_t)=20(ptr)=20+=20= LISP_WORD_TAG=20(tag)))=0A=20=0A=20/*=20LISPSYM_INITIALLY=20(Qfoo)=20is=20= equivalent=20to=20Qfoo=20except=20it=20is=0A=20=20=20=20designed=20for=20= use=20as=20an=20initializer,=20even=20for=20a=20constant=20initializer.=20= =20*/=0A--=20=0A2.32.0=20(Apple=20Git-132)=0A=0A= --Apple-Mail=_F041D8D6-60C4-40A7-8567-C866BBB314B6 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_F041D8D6-60C4-40A7-8567-C866BBB314B6-- From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 19 10:04:54 2023 Received: (at 65491) by debbugs.gnu.org; 19 Sep 2023 14:04:55 +0000 Received: from localhost ([127.0.0.1]:57597 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qibLS-0007vE-Hf for submit@debbugs.gnu.org; Tue, 19 Sep 2023 10:04:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52442) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qibLQ-0007uy-SV for 65491@debbugs.gnu.org; Tue, 19 Sep 2023 10:04:53 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qibLA-0006Nu-NY; Tue, 19 Sep 2023 10:04:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=WLXQiajrVzWLeq+XmjiTosNdLN2Ovl+nJbBTzKn/doM=; b=SMNZ6fR+bZ5oTOpKlnxA sCloW1K+p2sRoF2MdVDExokxDEilZQZJG+YgL982+uUipPjSxR+VarfTxUUylkFgGX/jSInVOwjX+ 9ZHJ1X5BICbtMznycKF/PRLaAh4dszkVa3z9sRYvI2BrcC5XeHpjVBAEG/x+Y6MJpTC7taW/ntjgr t4AizOmj+KxxVy2kr0lhF/oHlm1L8YURvjbsjX+7+USf2FMFU2ako3fH7kfJxTJaHcb7eMnBy1jB2 HxkGoLkntj9ULe8DawEHQHIJre1Jq6umOOBNH0oDKmAWmtBVWkjzCtKhpDyu09IEC1An97Tk1Y5dW /ca11JIUSFsjKA==; Date: Tue, 19 Sep 2023 17:04:39 +0300 Message-Id: <83h6nqcl8o.fsf@gnu.org> From: Eli Zaretskii To: Mattias =?utf-8?Q?Engdeg=C3=A5rd?= In-Reply-To: (message from Mattias =?utf-8?Q?Engdeg=C3=A5rd?= on Tue, 19 Sep 2023 15:28:41 +0200) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> <83o7i0g4i9.fsf@gnu.org> <83jzsog35g.fsf@gnu.org> <95C2D44B-BA1B-47F5-8279-B3A3B8F7EB39@gmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, eggert@cs.ucla.edu, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Mattias Engdegård > Date: Tue, 19 Sep 2023 15:28:41 +0200 > Cc: Paul Eggert , > 65491@debbugs.gnu.org, > monnier@iro.umontreal.ca > > Here it is. Very straightforward: just flip a cast and remove unused definitions. Builds okay on MS-Windows (with wide-ints) and on GNU/Linux (64-bit build). Thanks. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 19 10:06:16 2023 Received: (at 65491) by debbugs.gnu.org; 19 Sep 2023 14:06:16 +0000 Received: from localhost ([127.0.0.1]:57601 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qibMl-0007xn-WD for submit@debbugs.gnu.org; Tue, 19 Sep 2023 10:06:16 -0400 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]:50470) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qibMg-0007xS-QT for 65491@debbugs.gnu.org; Tue, 19 Sep 2023 10:06:14 -0400 Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-502fd1e1dd8so5604929e87.1 for <65491@debbugs.gnu.org>; Tue, 19 Sep 2023 07:06:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695132356; x=1695737156; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=1axo2Abtsv+4Zk3+Vx1OZO+PwsZVcdHrFfNrIPrXbZg=; b=I9YeXPeWyfOkdTN2W6vBDqItSIdmBOB1OzeGNp/BIfTOwoOjWnAIrvhrVrt3eiJk9Q wECzyzuPMjzTm56T1LWVLDZoNcJtFBtnW4pHDmOcCUvJa2xkLofiQvIkCet4tqhF5DZM fppJf3c4+Je1WnggveoTmloqfeulcuzBIR0j4ke/TEclScgmK7tR9JkEmJtdAP82KyXi 6Q4j9oqPG9f8pS6p4+y2u16aALp7vvVnwa7CfgV1z3dejzjqFGDD/oSPx5jo4TVbyzF4 FlmJT+N0htlfKSAx2pZnnY31c9mvOvXCOUXMGBhoAwjNmnKcl6tn9ng3UVeK8mSRKMNy ON6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695132356; x=1695737156; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=1axo2Abtsv+4Zk3+Vx1OZO+PwsZVcdHrFfNrIPrXbZg=; b=OdWObI8691CeyEGhVk+aiUvgBhuNfTVEmSuJlbiCFw6HOzVMICSWtt8UqT5zBrGVdG qezcpB20M9DcnlM6BrMceUB3Gs/3LHpMVL8dnZzW0iac+xFsibM+EINkT0CHDN9CH+zm j1ycFfTC0Tb+kmEn/6Ug6hyEU0IHXcIJY+t29SXCpDcD7tOTKzvTtL0QPW18yJKCntiU UY6M8oKY33NOx+pVLECmW/Wh10yyd7a0mSCJh9VcJhrOJRS5AeuGojMZC77PwJz37HRQ BcT4pWeGzOuAy7/QIvXAZRv+m3nBX3DsZu/SNX8kEOAtNyBXUUBC9bEizD8CCkU2Pyj3 LHAw== X-Gm-Message-State: AOJu0Yw+EOnncWTfcUzq+v2PVc6M1v+sI/cpgsSaDW0OdczdayYtD0Cy WHAukfZmdWK/Q34Lza8qaM8= X-Google-Smtp-Source: AGHT+IELAyCBsCysvwKHMMfoSyPJwDca/BXmU4ncCek6HXVgn6MwvWPPxkW7eIvsj3gydrBXoQEOnw== X-Received: by 2002:a05:6512:46d:b0:503:18c3:d874 with SMTP id x13-20020a056512046d00b0050318c3d874mr5495283lfd.45.1695132355749; Tue, 19 Sep 2023 07:05:55 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id m30-20020a056512015e00b004fbad09317csm2272152lfo.189.2023.09.19.07.05.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 19 Sep 2023 07:05:55 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: <83h6nqcl8o.fsf@gnu.org> Date: Tue, 19 Sep 2023 16:05:53 +0200 Content-Transfer-Encoding: 7bit Message-Id: <12E8D7EC-C8D3-453C-AAA1-549D0BB95A67@gmail.com> References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> <83v8cagkqv.fsf@gnu.org> <83ttrugkj2.fsf@gnu.org> <83r0mygi4y.fsf@gnu.org> <8af6fa1c-4873-bd6e-e896-ab5bb8d012a2@cs.ucla.edu> <83led5gyxa.fsf@gnu.org> <83o7i0g4i9.fsf@gnu.org> <83jzsog35g.fsf@gnu.org> <95C2D44B-BA1B-47F5-8279-B3A3B8F7EB39@gmail.com> <83h6nqcl8o.fsf@gnu.org> To: Eli Zaretskii X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, eggert@cs.ucla.edu, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 19 sep. 2023 kl. 16.04 skrev Eli Zaretskii : > Builds okay on MS-Windows (with wide-ints) and on GNU/Linux (64-bit > build). > > Thanks. Thanks for testing! Pushed to master. From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 25 12:07:02 2023 Received: (at 65491) by debbugs.gnu.org; 25 Sep 2023 16:07:02 +0000 Received: from localhost ([127.0.0.1]:46481 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qko6w-0000Eg-EM for submit@debbugs.gnu.org; Mon, 25 Sep 2023 12:07:02 -0400 Received: from mail-lj1-x229.google.com ([2a00:1450:4864:20::229]:46571) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qko6s-0000E0-Gm for 65491@debbugs.gnu.org; Mon, 25 Sep 2023 12:07:01 -0400 Received: by mail-lj1-x229.google.com with SMTP id 38308e7fff4ca-2c02e232c48so114614361fa.1 for <65491@debbugs.gnu.org>; Mon, 25 Sep 2023 09:06:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695658000; x=1696262800; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=Mt1jSXP4RPmpfrohS02UtPq6AeSrsb1WdaJdR/n1ZQg=; b=Lw0XYJyJsGev9o7LGAkLvstTfsreX00nybjh0TA8TWsYofOkQGZ8WLIpnRmY58evW8 cAneh4VIM/Lbc3QSbW0PZmRbQV/RbXSCvII7irVt6PlRBCw5mEmozarcoFHv5+ZV3f2H KzyujC+5NDqUuP+2Us6XHAbSC/Io1IDmwRPhusI7YUAnRJQNupQQ5z8zD7d/KS1VgZXV xz3WoRYAA14fU5tnZ339Nbl2QPy6CsdLnCi5kTn3SbzeRIJIYLUurhWX/NzbeUkvbEpt QyLXAHKslpcapkfX33gc6Dpc1ggeII6IfUL6/mjtYY/rhdGAJsC53QTmzxEzajXk3+4Y rSWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695658000; x=1696262800; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Mt1jSXP4RPmpfrohS02UtPq6AeSrsb1WdaJdR/n1ZQg=; b=if/nWsufeMfziN1wuqQ4JBP6g9mpfD5RyzLBnbbFMpXOB0RldijyxoVGANPx7Z0sW6 +wV+4xgEzi1Hx31h1/e1j2gDjTHkrgfaYpWWvta6CSp6xYZUp9NYK+RimtplagawPx56 XV7s0/ItW0oQfOxhBKSPTivLZ1j47JitGiR/9zu9lArFJeqErAjsNgEd9SsbmE2GuILK yrSx/odpKtsTgAOj/PPpB7AtAohtB3bBqxKe4tzdfqqMGonvCe6UcM0p3UaCPsW9WAeF JWbQ/hezFseYR0Gzd76rE+FaVXwKIBhEZDpgNGnfVN50fu8JTVMH6AMAaoT3PzuB5EQ/ vS3A== X-Gm-Message-State: AOJu0YzQCaiF0Rf4CDmP0F7+iDkARYrJ8zSRaTwzb+lTYfKZdLUhdUwY l1gMzs1CzTTjJI70YvE6DCw= X-Google-Smtp-Source: AGHT+IFkXRMJTTrkX59BZod+R3NvVjcuGonWctXAicg6zhPeqxf0CHBmkeiMBfkVnMNUqw1wdIXrRA== X-Received: by 2002:a2e:3c03:0:b0:2c0:1a80:dfb with SMTP id j3-20020a2e3c03000000b002c01a800dfbmr5416797lja.27.1695657998806; Mon, 25 Sep 2023 09:06:38 -0700 (PDT) Received: from smtpclient.apple (c188-150-165-235.bredband.tele2.se. [188.150.165.235]) by smtp.gmail.com with ESMTPSA id e1-20020a2e8ec1000000b002b9bf5b071bsm2269077ljl.20.2023.09.25.09.06.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 25 Sep 2023 09:06:38 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) Subject: Re: bug#65491: [PATCH] Improve performance allocating vectors From: =?utf-8?Q?Mattias_Engdeg=C3=A5rd?= In-Reply-To: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> Date: Mon, 25 Sep 2023 18:06:37 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <40613B9F-8EAA-42A9-9D42-929644A5A2C2@gmail.com> References: <6B2EDD07-AAEB-40E8-B369-F634296BD3D9@gmail.com> To: Stefan Monnier X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 65491 Cc: 65491@debbugs.gnu.org, Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > - Reducing the `vector_free_lists` array to its actually used first = half does improve performance a bit, even with subsequent scanning = speed-ups. > - The previously mentioned hack where scanning begins in the most = recently added bucket is surprisingly effective, often even more so than = a bitmap, but I'm wary of it being brittle. Need more measurements. These two changes were simple, fairly effective, and found to be = reasonably safe (in the sense that they are unlikely to make performance = worse), so I pushed them to master. This way we should be better off = even if no further progress is made. As mentioned, a bitmap is essentially as good as or better than the = last-used-index heuristic but I went with the latter because it involved = a lot less code. This can be changed. But it is possible to do better: size-homogeneous blocks, where each = block only houses vectors of a single size, have the major benefit that = live_small_vector_holding no longer needs to traverse half the block on = average but becomes is done in constant time. Sweeping, always = performance-critical since it traverses both living and dead objects, = becomes slightly faster. There is also no free-list array scanning at = all. Benchmarks suggest that larger homogeneous blocks (32 KiB, say) would be = beneficial, but we need to look at the potential extra fragmentation = caused by allocating a big block for each odd vector size.