From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 02:03:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 75845@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.173785696019406 (code B ref -1); Sun, 26 Jan 2025 02:03:02 +0000 Received: (at submit) by debbugs.gnu.org; 26 Jan 2025 02:02:40 +0000 Received: from localhost ([127.0.0.1]:53354 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbryy-00052u-7q for submit@debbugs.gnu.org; Sat, 25 Jan 2025 21:02:40 -0500 Received: from lists.gnu.org ([2001:470:142::17]:38902) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbryu-00052b-9k for submit@debbugs.gnu.org; Sat, 25 Jan 2025 21:02:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbryo-0005qZ-Ey for bug-gnu-emacs@gnu.org; Sat, 25 Jan 2025 21:02:30 -0500 Received: from mail-40133.protonmail.ch ([185.70.40.133]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbrym-0006sz-8a for bug-gnu-emacs@gnu.org; Sat, 25 Jan 2025 21:02:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1737856945; x=1738116145; bh=OZEA48SIi+ThEJXi+I2APTHlkWBcDtoF+wTHicINGT0=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector: List-Unsubscribe:List-Unsubscribe-Post; b=RAlWlRIWzQMxZ8Bjuu1D35svX9Tybt6b9D7JG9NTzsdyDnTau09+5iy7oHWUJcScs V2gGrSpdO7aGh1w7QAAoeCpIHn78Ayrwy3FVWXjmFFhiQqxk/cWg5AxurnSKALsg+I SNIpz//rfryXJhcAKPo69LJIeRfSmDJWDalu3g/c/eRDmXF+mshSnsXLF6ahqA+FwH dlPNpmpMTLplhO/wRuyXX9/f05Lc7/PtZhmlVU6/vB93clgnnTv1LPpsGCWAD2drkP h/q1jvC79pyYMqNDJeMJocFjFw5zTDhzyXRAW3AIfEO6q5d8NTSTzFChkpW5h1qi+E mvfJ26MrZry8g== Date: Sun, 26 Jan 2025 02:02:22 +0000 From: Pip Cet Message-ID: <87r04qpc2m.fsf@protonmail.com> Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 46221e7f570bfeb04b88ac3947e001b5c00dc602 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.70.40.133; envelope-from=pipcet@protonmail.com; helo=mail-40133.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) If the mapping function supplied to mapconcat modifies the sequence being mapped over, Emacs can crash. This seems easily fixable, but I wanted to file a bug to make sure we get it right: our current highly-optimized implementation of mapconcat will fail in many cases in which a naive implementation of mapconcat (in Lisp, say) would succeed. Some of those limitations are documented (the argument must be a sequence when the function is called), some aren't (the sequence must not become shorter or longer). My suggestion is to make the ordinary no-modification case work as fast as is possible under the constraint that extraordinary self-modifications should not result in crashes. They can and will result in unpredictable behavior, though. These crash: (let ((l (make-list 1000 nil))) (mapconcat (lambda (x) (ntake 1 l)) l)) (let ((s (string ?a 1000))) (mapconcat (lambda (x) (aset s 0 1000) (string = x)) s)) Patch will follow once this has a bug number. From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications References: <87r04qpc2m.fsf@protonmail.com> In-Reply-To: <87r04qpc2m.fsf@protonmail.com> Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 02:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 75845@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.173785907926509 (code B ref -1); Sun, 26 Jan 2025 02:38:02 +0000 Received: (at submit) by debbugs.gnu.org; 26 Jan 2025 02:37:59 +0000 Received: from localhost ([127.0.0.1]:53411 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbsX8-0006tS-EU for submit@debbugs.gnu.org; Sat, 25 Jan 2025 21:37:58 -0500 Received: from lists.gnu.org ([2001:470:142::17]:60192) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbsX4-0006sp-L6 for submit@debbugs.gnu.org; Sat, 25 Jan 2025 21:37:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbsWx-0001v7-Dv for bug-gnu-emacs@gnu.org; Sat, 25 Jan 2025 21:37:47 -0500 Received: from mail-4322.protonmail.ch ([185.70.43.22]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbsWu-0001gJ-AB for bug-gnu-emacs@gnu.org; Sat, 25 Jan 2025 21:37:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1737859061; x=1738118261; bh=f7yvlsW1EIyrggiJD3TBcjAWsgwzrj72ED/f5BbYOrg=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector: List-Unsubscribe:List-Unsubscribe-Post; b=kOoedIEhhn76tMmG5mKtH3VpXJt2zG637qw/qmlm7jPIfY7HdOU2f0riUmuaczjMi MuOeVIbPW1VL+eL7QksMF+9As7MANvKisLLU//HVIcZg6OgXaO85bSNU6Osdekg5x3 Iosdo39+PaCkzQPBxtMCpYa2K03J2X8EZOYsdRU0AWONhK5CzY0wcFJXqARs3Ri+RD o+96DSqTWChLWjE52y3LbEVy4epniVVYh4wJKa5klTDAaFwdGoq7h4HaKXpPf2tsn+ QAopnoGCNi81He3QJt9IEaNwwDc/E3UF6dbUSL20S4ZgHO6RpWJCdRXsVnENnQjHee plIE+ZCTTJelw== Date: Sun, 26 Jan 2025 02:37:37 +0000 From: Pip Cet Message-ID: <87ikq2pafv.fsf@protonmail.com> Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 1498d1f964813c7cc3e7d9314bbf51e3cb7865db MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.70.43.22; envelope-from=pipcet@protonmail.com; helo=mail-4322.protonmail.ch X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.049, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Pip Cet writes: > If the mapping function supplied to mapconcat modifies the sequence > being mapped over, Emacs can crash. > > This seems easily fixable, but I wanted to file a bug to make sure we > get it right: our current highly-optimized implementation of mapconcat > will fail in many cases in which a naive implementation of mapconcat (in > Lisp, say) would succeed. Some of those limitations are documented (the > argument must be a sequence when the function is called), some aren't > (the sequence must not become shorter or longer). > > My suggestion is to make the ordinary no-modification case work as fast > as is possible under the constraint that extraordinary > self-modifications should not result in crashes. They can and will > result in unpredictable behavior, though. > > These crash: > > (let ((l (make-list 1000 nil))) (mapconcat (lambda (x) (ntake 1 l)) l)) > > (let ((s (string ?a 1000))) (mapconcat (lambda (x) (aset s 0 1000) (strin= g x)) s)) > > Patch will follow once this has a bug number. This patch does the minimum work necessary; while the previous code attempted to return the "correct" length of a list which was modified, crashing in an eassert in mapconcat, the new code errors instead. As error is marked with ATTRIBUTE_COLD, this should result in better code. The string case is slightly trickier: we don't keep around an SDATA pointer, which is good, but we do remember byte and charpos index values, and those might get out of sync. So verify we're looking at a CHAR_HEAD when we need to, which is for multibyte strings. mapconcat itself is also fixed not to assert but to use whatever mapcar1 returns. >From cf861bc042650c968f2702d842ffa4a36f0105e9 Mon Sep 17 00:00:00 2001 From: Pip Cet Subject: [PATCH] Fix crashes in mapconcat etc. (bug#75845) This doesn't attempt to catch all cases in which the mapping function modifies the sequence, only those which would cause crashes. * src/fns.c (mapcar1): Error if we detect modifications by the mapping function. (mapconcat): Don't eassert that 'mapcar1' never returns a partially-filled vector, even though that is currently correct. --- src/fns.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/src/fns.c b/src/fns.c index 081ed2b9f51..8e8b9584509 100644 --- a/src/fns.c +++ b/src/fns.c @@ -3385,7 +3385,7 @@ mapcar1 (EMACS_INT leni, Lisp_Object *vals, Lisp_Obje= ct fn, Lisp_Object seq) for (ptrdiff_t i =3D 0; i < leni; i++) =09{ =09 if (! CONSP (tail)) -=09 return i; +=09 error ("list modified by mapping function"); =09 Lisp_Object dummy =3D calln (fn, XCAR (tail)); =09 if (vals) =09 vals[i] =3D dummy; @@ -3403,16 +3403,21 @@ mapcar1 (EMACS_INT leni, Lisp_Object *vals, Lisp_Ob= ject fn, Lisp_Object seq) } else if (STRINGP (seq)) { + ptrdiff_t i =3D 0; ptrdiff_t i_byte =3D 0; =20 - for (ptrdiff_t i =3D 0; i < leni;) + while (i_byte < SBYTES (seq)) =09{ =09 ptrdiff_t i_before =3D i; +=09 if (STRING_MULTIBYTE (seq) && !CHAR_HEAD_P (SREF (seq, i_byte))) +=09 error ("string modified by mapping function"); =09 int c =3D fetch_string_char_advance (seq, &i, &i_byte); =09 Lisp_Object dummy =3D calln (fn, make_fixnum (c)); =09 if (vals) =09 vals[i_before] =3D dummy; =09} + if (i < leni) +=09error ("string modified by mapping function"); } else { @@ -3473,8 +3478,7 @@ DEFUN ("mapconcat", Fmapconcat, Smapconcat, 2, 3, 0, =09 goto concat; =09} } - ptrdiff_t nmapped =3D mapcar1 (leni, args, function, sequence); - eassert (nmapped =3D=3D leni); + leni =3D mapcar1 (leni, args, function, sequence); =20 concat: ; ptrdiff_t nargs =3D args_alloc; --=20 2.47.1 From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 06:57:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Pip Cet , Stefan Monnier , Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Cc: 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.173787457716148 (code B ref 75845); Sun, 26 Jan 2025 06:57:02 +0000 Received: (at 75845) by debbugs.gnu.org; 26 Jan 2025 06:56:17 +0000 Received: from localhost ([127.0.0.1]:54020 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbwZ6-0004CO-GT for submit@debbugs.gnu.org; Sun, 26 Jan 2025 01:56:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42102) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbwZ3-0004C5-5n for 75845@debbugs.gnu.org; Sun, 26 Jan 2025 01:56:14 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbwYw-0001ym-Rv; Sun, 26 Jan 2025 01:56:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=7xXPlxc6bJw3BtIEB97mLUsl+Wr7LQ5KBHIXvGSxvkI=; b=Aof66tTTcyPZlb/3QwED rRpjHhR4qE4xtOF5GKvTjKievR/Mhj5GeJO/gEB25Nkry+DpMWs2H9KO1ZLkGhXXjo2HB461tIksG hYpsv9i+Ef4XnyR4ZS03koFoiov06CIm1rZmBUkVsDJ97+b1/qJ972wCLQaCW9wGb0BQO5jAWkZvE 9GY36qsE3t7L+479ARXGgLljnKo3VBoQcVc6aaxzG28ca9wI3OXpVl0gIwTYOTpeOQlxezFMMjWNl 2bbA9vR1ZfG4Sww5vfoedlMYrPS2OS+bjnNlonqwhF+HAM5N4E9u0QfGHkpaIDIO5ASzga4JifPsX +1LEqZRyl3qY5g==; Date: Sun, 26 Jan 2025 08:56:03 +0200 Message-Id: <86jzait664.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <87ikq2pafv.fsf@protonmail.com> (bug-gnu-emacs@gnu.org) References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sun, 26 Jan 2025 02:37:37 +0000 > From: Pip Cet via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > Pip Cet writes: > > > If the mapping function supplied to mapconcat modifies the sequence > > being mapped over, Emacs can crash. > > > > This seems easily fixable, but I wanted to file a bug to make sure we > > get it right: our current highly-optimized implementation of mapconcat > > will fail in many cases in which a naive implementation of mapconcat (in > > Lisp, say) would succeed. Some of those limitations are documented (the > > argument must be a sequence when the function is called), some aren't > > (the sequence must not become shorter or longer). > > > > My suggestion is to make the ordinary no-modification case work as fast > > as is possible under the constraint that extraordinary > > self-modifications should not result in crashes. They can and will > > result in unpredictable behavior, though. > > > > These crash: > > > > (let ((l (make-list 1000 nil))) (mapconcat (lambda (x) (ntake 1 l)) l)) > > > > (let ((s (string ?a 1000))) (mapconcat (lambda (x) (aset s 0 1000) (string x)) s)) > > > > Patch will follow once this has a bug number. > > This patch does the minimum work necessary; while the previous code > attempted to return the "correct" length of a list which was modified, > crashing in an eassert in mapconcat, the new code errors instead. As > error is marked with ATTRIBUTE_COLD, this should result in better code. > > The string case is slightly trickier: we don't keep around an SDATA > pointer, which is good, but we do remember byte and charpos index > values, and those might get out of sync. So verify we're looking at a > CHAR_HEAD when we need to, which is for multibyte strings. > > mapconcat itself is also fixed not to assert but to use whatever mapcar1 > returns. Thanks, adding Stefan and Mattias to the discussion, in the hope that they could review the patch and comment. > if (! CONSP (tail)) > - return i; > + error ("list modified by mapping function"); I think this error message is not explicit enough: it doesn't tell which list was modified by what function. mapcar and mapconcat are many times invoked by code that has little to do with the level which could make sense to the user, so we should try to provide as much contextual information as possible. Perhaps something like sequence arg to `mapcar' or `mapconcat' modified by its mapping function arg Better suggestions are welcome. > + if (STRING_MULTIBYTE (seq) && !CHAR_HEAD_P (SREF (seq, i_byte))) > + error ("string modified by mapping function"); AFAIU, this doesn't check whether the string was modified, it checks for modifications that invalidate the relation between i and i_byte, right? So I would suggest string arg of mapcar or mapconcat clobbered by its mapping function arg Because if we want to disallow _any_ modifications of the string, we'd need to use a different test (e.g., unibyte ASCII strings could also be modified by the mapping function). > + if (i < leni) > + error ("string modified by mapping function"); Why do we need this test? Does it prevent us from crashing in some cases? If that replaces the below: > @@ -3473,8 +3478,7 @@ DEFUN ("mapconcat", Fmapconcat, Smapconcat, 2, 3, 0, > goto concat; > } > } > - ptrdiff_t nmapped = mapcar1 (leni, args, function, sequence); > - eassert (nmapped == leni); > + leni = mapcar1 (leni, args, function, sequence); then can we have the test as eassert? From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 09:50:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= , Stefan Monnier , 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.173788495120285 (code B ref 75845); Sun, 26 Jan 2025 09:50:02 +0000 Received: (at 75845) by debbugs.gnu.org; 26 Jan 2025 09:49:11 +0000 Received: from localhost ([127.0.0.1]:54683 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbzGQ-0005H7-Dr for submit@debbugs.gnu.org; Sun, 26 Jan 2025 04:49:11 -0500 Received: from mail-4322.protonmail.ch ([185.70.43.22]:55943) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbzGE-0005GC-N8 for 75845@debbugs.gnu.org; Sun, 26 Jan 2025 04:49:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1737884932; x=1738144132; bh=ubWoe4sJX6Bz7gJsQOCLWG2HCQ050iRXh7c/f16DFbg=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=knh003FD9OI1pm7LJn9vRn0kqAW262d+Owf5bcpEjNlPsWYq3MDo0/uSic/pSB3Jt zLCicwDVuoVFcxNY/lUxzOyQeL2T2UNK4NszOnR066moB0Ai2O5qllb3F4yOeZoyY2 +nMD06LD/J7+iNJMocPDHBDoTa6N57vOfmqqbAHO5Oij6E/18930JUCTZJupMaPGUy Ouw5bWTF6yh7q2nI+BpwoPyNdkSACd9gg6Lb2oBgRiVVPgmyqB8oQM/GXbQVt3zmRD R/U1wK1Og2FV7kGuxEVZxA1EAwLWdIOnRnC9vvM7jtsdWLQbRQtMcu2qPQTJPOp749 7mG0gEDmYGSNw== Date: Sun, 26 Jan 2025 09:48:48 +0000 From: Pip Cet Message-ID: <87r04poqh9.fsf@protonmail.com> In-Reply-To: <86jzait664.fsf@gnu.org> References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> Feedback-ID: 112775352:user:proton X-Pm-Message-ID: c21864927890ec4671d3bfdd32e805b84ecd557d MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.1 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.1 (-) R"Eli Zaretskii" writes: >> Date: Sun, 26 Jan 2025 02:37:37 +0000 >> From: Pip Cet via "Bug reports for GNU Emacs, >> the Swiss army knife of text editors" >> >> Pip Cet writes: >> >> > If the mapping function supplied to mapconcat modifies the sequence >> > being mapped over, Emacs can crash. >> > >> > This seems easily fixable, but I wanted to file a bug to make sure we >> > get it right: our current highly-optimized implementation of mapconcat >> > will fail in many cases in which a naive implementation of mapconcat (= in >> > Lisp, say) would succeed. Some of those limitations are documented (t= he >> > argument must be a sequence when the function is called), some aren't >> > (the sequence must not become shorter or longer). >> > >> > My suggestion is to make the ordinary no-modification case work as fas= t >> > as is possible under the constraint that extraordinary >> > self-modifications should not result in crashes. They can and will >> > result in unpredictable behavior, though. >> > >> > These crash: >> > >> > (let ((l (make-list 1000 nil))) (mapconcat (lambda (x) (ntake 1 l)) l)= ) >> > >> > (let ((s (string ?a 1000))) (mapconcat (lambda (x) (aset s 0 1000) (st= ring x)) s)) >> > >> > Patch will follow once this has a bug number. >> >> This patch does the minimum work necessary; while the previous code ^^^^^^^ I don't think this situation is expected to be common enough to warrant a more specific error message, TBH, particularly not if it then fails to apply to all possible cases. My suggestion is to merge the two error messages to save some space, apply the patch, and make a mental note to include both mapconcat/mapcar1 and the error handling situation for further review when we have time. The rest of this email mostly consists of notes for that (future) discussion, except for this, which I've reordered: >> + if (i < leni) >> +=09error ("string modified by mapping function"); > > Why do we need this test? mapconcat assumed the wrong thing, I think other future callers might, as well. It's not obvious that mapcar1's return value can be different from its first argument, and given the semi-optimized state of this function I think future work is more likely. > Does it prevent us from crashing in some cases? Yes. Silly programmers like myself (see first suggested patch in another bug) may attempt to use out-of-bounds values in the array, which are uninitialized and would cause crashes. Also, a minor point in performance-sensitive code is that conditional branches are bad, but not if they call error(). Such branches will be optimized by GCC to be predicted not to call error in a static way, independently of the CPU's limited branch prediction cache. Here's the rest: >> attempted to return the "correct" length of a list which was modified, >> crashing in an eassert in mapconcat, the new code errors instead. As >> error is marked with ATTRIBUTE_COLD, this should result in better code. >> >> The string case is slightly trickier: we don't keep around an SDATA >> pointer, which is good, but we do remember byte and charpos index >> values, and those might get out of sync. So verify we're looking at a >> CHAR_HEAD when we need to, which is for multibyte strings. >> >> mapconcat itself is also fixed not to assert but to use whatever mapcar1 >> returns. > > Thanks, adding Stefan and Mattias to the discussion, in the hope that > they could review the patch and comment. Thanks! >> =09 if (! CONSP (tail)) >> -=09 return i; >> +=09 error ("list modified by mapping function"); > > I think this error message is not explicit enough: it doesn't tell > which list was modified by what function. mapcar and mapconcat are > many times invoked by code that has little to do with the level which > could make sense to the user, so we should try to provide as much > contextual information as possible. Perhaps something like > > sequence arg to `mapcar' or `mapconcat' modified by its mapping functio= n arg I like the idea of using the same message for both, reducing the memory requirement for the unused error string. (I may be unaware of a good error function to use in this case; I ended up not providing additional arguments because it seems unlikely to me that this problem, if really accidental, is something the programmer can fix without a Lisp backtrace, which indicates the situation more clearly). However, all functions using mapcar1 are affected by the fix, if not the current bug: mapcan and mapc are as well. Making a message more specific is bad if that makes it incorrect, and mapc is the most likely candidate for modifying its sequence argument. (I've never seen (mapc function "string") though, that I can recall). >> +=09 if (STRING_MULTIBYTE (seq) && !CHAR_HEAD_P (SREF (seq, i_byte))) >> +=09 error ("string modified by mapping function"); > > AFAIU, this doesn't check whether the string was modified, it checks > for modifications that invalidate the relation between i and i_byte, > right? So I would suggest Not even all of those, really, only if they happen to leave us looking at the middle of a multibyte sequence. Again, minimal effort. > Because if we want to disallow _any_ modifications of the string, we'd > need to use a different test (e.g., unibyte ASCII strings could also > be modified by the mapping function). I don't think we do want that. However, my minority opinion on the question of read-only objects is just that, so I'm not a good person to ask if you want to reintroduce such objects after most of them are removed when scratch/no-purespace is merged. > If that replaces the below: >> @@ -3473,8 +3478,7 @@ DEFUN ("mapconcat", Fmapconcat, Smapconcat, 2, 3, = 0, >> =09 goto concat; >> =09} >> } >> - ptrdiff_t nmapped =3D mapcar1 (leni, args, function, sequence); >> - eassert (nmapped =3D=3D leni); >> + leni =3D mapcar1 (leni, args, function, sequence); > > then can we have the test as eassert? I see no reason to set this trap for the next person who modifies mapcar1. If they see the return value is checked by some callers, they might miss the one caller which crashes. If they remove the return value, they might miss that the eassert, too, needs to be removed, making their job harder. If we want that eassert (why?), it belongs in mapcar1, not Fmapconcat. mapconcat in general seems to be semi-optimized to me, to be honest, and it makes the code very long and (apparently) bug-prone. Until we have better structures for growing strings, we could do worse than to call Fappend on the sequence argument and handle only the (simple) list case. If there are significant users of the vector case, we could add that, too, but strings and bool vectors add many lines of unnecessary code. (Bool vectors are obsolete, I'm not aware of any new users since I suggested obsoleting and removing them long ago. IMHO, that remains the right thing to do, but no new arguments for removing them have appeared, either. I'd do the change locally, but can't as bool vectors complicate alloc.c in so many places (which is why I want to remove them in the first place) that this would introduce too many merge conflicts.) Again, it's unclear to me which cases mapconcat has chosen to optimize, and why. I'm not proposing to optimize it for other cases at this point because I don't know what the criterion is, but I think we went overboard there supporting weird sequences instead of making the common case fast and reducing all other cases to it. I understand we can't go with the 7-line Lisp version of it, but maybe a compromise can be found. Pip From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 10:25:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Pip Cet Cc: mattiase@acm.org, monnier@iro.umontreal.ca, 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.173788707727362 (code B ref 75845); Sun, 26 Jan 2025 10:25:01 +0000 Received: (at 75845) by debbugs.gnu.org; 26 Jan 2025 10:24:37 +0000 Received: from localhost ([127.0.0.1]:54782 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tbzoj-00077F-3U for submit@debbugs.gnu.org; Sun, 26 Jan 2025 05:24:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56636) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tbzof-00076w-D1 for 75845@debbugs.gnu.org; Sun, 26 Jan 2025 05:24:35 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tbzoZ-0001aZ-1T; Sun, 26 Jan 2025 05:24:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=4xWjeXiXtvdiUJmoS+YK2BhJIryJ3wBmWMfQ0cCSHHk=; b=kYTrn4hbs1KfSZ4hyYds 4B0JdJHAHJhhtxJsRmF/dA0dqJml9uWOQgvxgeuz4jZy5xVSGe6dGBmgV5l4nimnVJmNCmSTIEZSH gT0LvBof7RevpjTEHOAvLhXpHW8kMZJ3rDGCfZt+y2egrl9TsF54UvU6HsOZ5/NMz30oIxK28D9TZ N3QZCfvQC2PCTqmN/ci8QBRA06LClrMMtAX/rAKcwrtkhAgKqtHk5HsbnBjc4hNAOmMCorCyCVGux ptr8vpgsg3A4y2Q9NDQ6a8QmtbalWiLF4i5fmz3wvmM/K6Ax1BlgKz+n1sb1p+1q4jEaOfjfp94kB 6ydA5UqvNQR9pg==; Date: Sun, 26 Jan 2025 12:24:23 +0200 Message-Id: <86ikq1swiw.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <87r04poqh9.fsf@protonmail.com> (message from Pip Cet on Sun, 26 Jan 2025 09:48:48 +0000) References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> <87r04poqh9.fsf@protonmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sun, 26 Jan 2025 09:48:48 +0000 > From: Pip Cet > Cc: Stefan Monnier , Mattias Engdegård , 75845@debbugs.gnu.org > > I don't think this situation is expected to be common enough to warrant > a more specific error message, TBH, particularly not if it then fails to > apply to all possible cases. I didn't want the message to be specific. I wanted it to explain which function signaled the error, so the bug could be easier to look for and spot. > My suggestion is to merge the two error messages to save some space, I'm okay with using a single error message. But let's make its text provide more context than just saying that some list was modified by some function. > > Does it prevent us from crashing in some cases? > > Yes. Silly programmers like myself (see first suggested patch in > another bug) may attempt to use out-of-bounds values in the array, which > are uninitialized and would cause crashes. No, I meant can we crash in the following code. If the caller then uses the value somewhere else, without validating it, it's a separate problem that should be perhaps dealt with at that other place. > > sequence arg to `mapcar' or `mapconcat' modified by its mapping function arg > > I like the idea of using the same message for both, reducing the memory > requirement for the unused error string. Then how about sequence arg to `mapcar' etc. modified by its mapping function arg > (I may be unaware of a good > error function to use in this case; I ended up not providing additional > arguments because it seems unlikely to me that this problem, if really > accidental, is something the programmer can fix without a Lisp > backtrace, which indicates the situation more clearly). The above just provides more context, to make it easier to look for the likely culprits. Arguments are not provided, just mentioned. > >> - ptrdiff_t nmapped = mapcar1 (leni, args, function, sequence); > >> - eassert (nmapped == leni); > >> + leni = mapcar1 (leni, args, function, sequence); > > > > then can we have the test as eassert? > > I see no reason to set this trap for the next person who modifies > mapcar1. If they see the return value is checked by some callers, they > might miss the one caller which crashes. If they remove the return > value, they might miss that the eassert, too, needs to be removed, > making their job harder. > > If we want that eassert (why?), it belongs in mapcar1, not Fmapconcat. That's what I meant: to make the test an assertion in mapcar1. From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 10:56:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= , Pip Cet , 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.17378889591243 (code B ref 75845); Sun, 26 Jan 2025 10:56:02 +0000 Received: (at 75845) by debbugs.gnu.org; 26 Jan 2025 10:55:59 +0000 Received: from localhost ([127.0.0.1]:54867 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tc0J4-0000Jz-OL for submit@debbugs.gnu.org; Sun, 26 Jan 2025 05:55:59 -0500 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:21954) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tc0J2-0000Jf-5t for 75845@debbugs.gnu.org; Sun, 26 Jan 2025 05:55:57 -0500 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 909624405F2; Sun, 26 Jan 2025 05:55:49 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1737888947; bh=pvqo61QjedU+V54+EKc5kD+VDRbvuqTkWdDJgu9Tm2E=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=MEQoiEtZCPn6g6ddXeQdMnA1TdiLEfHEtXqxBns+hTUQIz6geINWWhYbVsSzs1jT0 pt1q8RqmhrAdqTDRUvsWbIKYpYnMgiJjb8lMzEJAoSBOctEyE8Kg2NUxYCqXvhN1Tt RsxbY++HQo9fN5gPFXj500fcGfu8toXsD4n17Wp/XkJqPrfzK02Y25bQGpbov/c9e1 LazboV1S58f4OPHuzPRbkITuvdxj5UlFWzWM8f70oLEdctrccV0sQliDbwBPHqogMS n+H6FWxI4TqTLf991ktocUfrlG+rlFDk4hkTlyp6FkoYWsRw3wQps9fxlsTgnkO7aP 654vE7uSN9IoQ== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id AB6E64408FA; Sun, 26 Jan 2025 05:55:47 -0500 (EST) Received: from asado (dyn.144-85-147-102.dsl.vtx.ch [144.85.147.102]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B013112084B; Sun, 26 Jan 2025 05:55:46 -0500 (EST) From: Stefan Monnier In-Reply-To: <86jzait664.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 26 Jan 2025 08:56:03 +0200") Message-ID: References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> Date: Sun, 26 Jan 2025 05:55:44 -0500 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.117 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain T_SCC_BODY_TEXT_LINE -0.01 - X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Thanks, adding Stefan and Mattias to the discussion, in the hope that > they could review the patch and comment. The patch LGTM. >> if (! CONSP (tail)) >> - return i; >> + error ("list modified by mapping function"); > > I think this error message is not explicit enough: it doesn't tell > which list was modified by what function. mapcar and mapconcat are > many times invoked by code that has little to do with the level which > could make sense to the user, so we should try to provide as much > contextual information as possible. Perhaps something like > > sequence arg to `mapcar' or `mapconcat' modified by its mapping function arg Since this is in the list-part of the code, I'd prefer "list" rather than "sequence", since it's both shorter and more precise. But I'd capitalize that first word. > string arg of mapcar or mapconcat clobbered by its mapping function arg +1 for "clobbered" Stefan From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 12:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Stefan Monnier Cc: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= , Eli Zaretskii , 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.173789328415132 (code B ref 75845); Sun, 26 Jan 2025 12:09:02 +0000 Received: (at 75845) by debbugs.gnu.org; 26 Jan 2025 12:08:04 +0000 Received: from localhost ([127.0.0.1]:54999 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tc1Qp-0003vz-Dj for submit@debbugs.gnu.org; Sun, 26 Jan 2025 07:08:03 -0500 Received: from mail-10628.protonmail.ch ([79.135.106.28]:47273) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tc1Qm-0003vI-Un for 75845@debbugs.gnu.org; Sun, 26 Jan 2025 07:08:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1737893274; x=1738152474; bh=fhTeDPiiNTQLUbGi6bFCNdUodbwdgCobzZhZ8lK1k84=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=IrQs4xjXg1XYk9qebt3IDAdpoiMl9idoxG9dM9Chq3eaF6qaoJ1I9MnlM2FrtsB0d hAuQj3MQ07bfy0Rkm7i8HJyeF/0GMizbhHRFOcimWT8cWhU76iw2dPSMPmJH1TcSa5 BKQBikdSEYbcDpTUyGHRu/U6JCFEg7H0ujTn6N+xutAo6N8eZuLTIqMZunBLCcTm8h KAe0giuCOpV4RBR5JkDRk9KEjPBJCxuirkOAAZi3e4FfFW2/lh9b1Yz2kt/GHs2Q7g QxRJF+NB4IV+RhHz6Cd9oph56EKk9pUPw1DukjoWvZXj0wB2ZhKo70RBHzyOJueCTH B3Iy/q91UFC2w== Date: Sun, 26 Jan 2025 12:07:50 +0000 From: Pip Cet Message-ID: <87y0yxn5h5.fsf@protonmail.com> In-Reply-To: References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 1f0e4dd4556a887ae97821683db68e6b9603b1b6 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) "Stefan Monnier" writes: >> Thanks, adding Stefan and Mattias to the discussion, in the hope that >> they could review the patch and comment. > > The patch LGTM. Thanks. I'll wait for a decision to be made on which precise error to throw in this extremely rare situation. > >>> =09 if (! CONSP (tail)) >>> -=09 return i; >>> +=09 error ("list modified by mapping function"); >> >> I think this error message is not explicit enough: it doesn't tell FWIW, I started out by copying signal_error ("hash table test modifies table", obj); since that's the only similar case I could think of right away, and I thought this might not require additional thought. Upon reflection, my current suggestion is to define an error symbol vaguely equivalent to Java's ConcurrentModificationException, but with a less puzzling name (also, hopefully, this would happen much less often than that common exception). Having a single error symbol and documenting it once, for all cases, with general advice for what might have gone wrong and how to fix it (rewrite callbacks to be as pure as possible) doesn't seem harder than arguing about this specific corner case, but does provide more benefits. >> string arg of mapcar or mapconcat clobbered by its mapping function ar= g TBH, "string arg of ... mapconcat" sounds like it's referring to the separator string, to me. > +1 for "clobbered" +1 from me, too. "Concurrent modification exceptions" are puzzling to programmers partly because the name doesn't make it clear who was at fault. "Clobbered" clearly indicates that the modifying code was careless and is the part that needs to be fixed, and that we were lucky to detect this was attempted or happened. PUT_ERROR (Qdata_clobbered_error, "Lisp callback unexpectedly clobbered dat= a"); on the assumption that xsignal2 is the right thing to use here, and that the reference to the clobbering function would be the first and the data clobbered would be the second argument. No objections to reversing the string and arguments if that's clearer, or any rewording that makes sense to you. (There's a tiny white lie here: I don't think it's worth it to distinguish cases in which the clobberer was successful from those in which an illicit modification was prevented. Even if it's a single error for all cases, the combined probability isn't great enough to warrant two error symbols). As a tangent, I'd prefer providing format arguments in PUT_ERROR, on the understanding that expanding them is optional: omitting the format arguments would yield the current error message, and including them would provide additional output in situations where the user desires this. Pip From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 12:17:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Pip Cet Cc: mattiase@acm.org, monnier@iro.umontreal.ca, 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.173789378016845 (code B ref 75845); Sun, 26 Jan 2025 12:17:01 +0000 Received: (at 75845) by debbugs.gnu.org; 26 Jan 2025 12:16:20 +0000 Received: from localhost ([127.0.0.1]:55020 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tc1Yp-0004Na-AW for submit@debbugs.gnu.org; Sun, 26 Jan 2025 07:16:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58250) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tc1Ym-0004NG-8P for 75845@debbugs.gnu.org; Sun, 26 Jan 2025 07:16:17 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tc1Yc-00041z-N3; Sun, 26 Jan 2025 07:16:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=RA3bmHMJCqBwoJT1KGNWkD9xVrNmlr/L9hmj0vNKdwc=; b=JEMQcqoPpf4chdiqMq3R P7ssmzNDuLs+Exe71npAZni+7JZ2GUZz3vr3uXgXtDNByeeEtYQz9bzagize1eXceenMQ2OQO4sde Rpkrs++cAfMTl4l3pcK7o2HP1SKwHEaBLIi1oNMOMUriWeIvXQXEuSz4eIOygcq8pJEYPxC1wk0Am QaX9yXCUAHae2vZPX1yuvua9+yzhFmhS8POyO3uJzIc8SJ4eNtIZSD/T3EWh0LswJPdArFN0+C7a5 nb0ZE6wiFdtq8ISY4wmBd1elJBIk6onCxGNiRb7ytBgZpUhSzIa7OqYZCoheCqNdxytR67q6cDk3B GUPZdwMIda3Glg==; Date: Sun, 26 Jan 2025 14:16:03 +0200 Message-Id: <86bjvtsrcs.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <87y0yxn5h5.fsf@protonmail.com> (message from Pip Cet on Sun, 26 Jan 2025 12:07:50 +0000) References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> <87y0yxn5h5.fsf@protonmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sun, 26 Jan 2025 12:07:50 +0000 > From: Pip Cet > Cc: Eli Zaretskii , Mattias Engdegård , 75845@debbugs.gnu.org > > >> string arg of mapcar or mapconcat clobbered by its mapping function arg > > TBH, "string arg of ... mapconcat" sounds like it's referring to the > separator string, to me. If that is a concern, we could say First arg of mapcar etc. clobbered ... From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 26 Jan 2025 15:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Pip Cet Cc: mattiase@acm.org, monnier@iro.umontreal.ca, 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.173790590128699 (code B ref 75845); Sun, 26 Jan 2025 15:39:01 +0000 Received: (at 75845) by debbugs.gnu.org; 26 Jan 2025 15:38:21 +0000 Received: from localhost ([127.0.0.1]:57599 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tc4iL-0007So-Br for submit@debbugs.gnu.org; Sun, 26 Jan 2025 10:38:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40404) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tc4iJ-0007SZ-3z for 75845@debbugs.gnu.org; Sun, 26 Jan 2025 10:38:19 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tc4iC-0005Af-Pw; Sun, 26 Jan 2025 10:38:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=cr3Q8sc4QHWY/sx7ciOAn8tHckOYdGh85AXbO8s7i54=; b=Il3lSCkLXRaFzQtfKhCQ JAhD3H7lTNhWPMufheoqGTC5En2oWZYcEY3YvwrHPSPe7z1eS6G4oSMK1ZSjhZvJXmbVoODH3/nAF AbfOZI+iHyDqG2I4+GHcq4zmsCUUGQWNbXxXjDvVU1ldp5JefOS+a30nedXaIpPXveaC3D0FOPvXD j9F3ZDfmpfH4WfvZz2V0xroJuoniZwfEzrdRAAOQmLtbDThM/rmY/oq9a5v1xEK5xoL4hM5UyWfrz TY/icj/RKtkksMiNF7yaMeoZKwW4LxEd18a5uTqdihNAbj2gBlRk85J1ZsfaQOPdrVp1tlYT5es0V bzZV1tD4k60fAA==; Date: Sun, 26 Jan 2025 17:37:45 +0200 Message-Id: <86sep5r3g6.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <87y0yxn5h5.fsf@protonmail.com> (message from Pip Cet on Sun, 26 Jan 2025 12:07:50 +0000) References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> <87y0yxn5h5.fsf@protonmail.com> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sun, 26 Jan 2025 12:07:50 +0000 > From: Pip Cet > Cc: Eli Zaretskii , Mattias Engdegård , 75845@debbugs.gnu.org > > "Stefan Monnier" writes: > > >> Thanks, adding Stefan and Mattias to the discussion, in the hope that > >> they could review the patch and comment. > > > > The patch LGTM. > > Thanks. I'll wait for a decision to be made on which precise error to > throw in this extremely rare situation. I'd like also to hear from Mattias, if possible, before we install this. From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 08 Feb 2025 09:28:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: mattiase@acm.org Cc: pipcet@protonmail.com, monnier@iro.umontreal.ca, 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.173900687522071 (code B ref 75845); Sat, 08 Feb 2025 09:28:02 +0000 Received: (at 75845) by debbugs.gnu.org; 8 Feb 2025 09:27:55 +0000 Received: from localhost ([127.0.0.1]:38532 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tgh7z-0005js-0l for submit@debbugs.gnu.org; Sat, 08 Feb 2025 04:27:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58040) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tgh7x-0005je-6u for 75845@debbugs.gnu.org; Sat, 08 Feb 2025 04:27:54 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tgh7q-0005om-A5; Sat, 08 Feb 2025 04:27:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=M4FxqJGNYkksPc+SuQTSYmfqGMtJRtPeSmW4qNdiPRI=; b=INs3GwYxAyq6oXfPpDVO KCG3MA7Op87AT04h2pylOz8TMSeyq1fTCIS3G97ZFkZa28Elj/QAg190gijl/Wu8FJ5m6qRcWS4Yy dK6wpT9+A2VqzXurfZuwCrbTgbzR69UQ39nc8EGgVDyrfCOe1P18E+Ncyri9zm2URQhQ0dxmnbwJJ uHPkixsOtGmxFPSzM5uNfo5h5DDnsatlP9VexOckDzEnkfT3RNUh/u1zixV4qpZ/KO/FYT4h01clg FYQLEB0Ot5B61BloyUWECXyFQi13/XK44iEqKK7hs7Fxaf2nbxzTU7GzggNyc8Klb5KTJ5suDsTh6 oCbwMm8tITy/8w==; Date: Sat, 08 Feb 2025 11:27:40 +0200 Message-Id: <86seoo3hwz.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <86sep5r3g6.fsf@gnu.org> (message from Eli Zaretskii on Sun, 26 Jan 2025 17:37:45 +0200) References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> <87y0yxn5h5.fsf@protonmail.com> <86sep5r3g6.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ping! Mattias, any comments to thiws discussion and to the patch? > Cc: mattiase@acm.org, monnier@iro.umontreal.ca, 75845@debbugs.gnu.org > Date: Sun, 26 Jan 2025 17:37:45 +0200 > From: Eli Zaretskii > > > Date: Sun, 26 Jan 2025 12:07:50 +0000 > > From: Pip Cet > > Cc: Eli Zaretskii , Mattias Engdegård , 75845@debbugs.gnu.org > > > > "Stefan Monnier" writes: > > > > >> Thanks, adding Stefan and Mattias to the discussion, in the hope that > > >> they could review the patch and comment. > > > > > > The patch LGTM. > > > > Thanks. I'll wait for a decision to be made on which precise error to > > throw in this extremely rare situation. > > I'd like also to hear from Mattias, if possible, before we install > this. > > > > From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 08 Feb 2025 11:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: pipcet@protonmail.com, monnier@iro.umontreal.ca, 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.17390127128199 (code B ref 75845); Sat, 08 Feb 2025 11:06:02 +0000 Received: (at 75845) by debbugs.gnu.org; 8 Feb 2025 11:05:12 +0000 Received: from localhost ([127.0.0.1]:38811 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tgie8-00028B-Fv for submit@debbugs.gnu.org; Sat, 08 Feb 2025 06:05:12 -0500 Received: from mail-lf1-x133.google.com ([2a00:1450:4864:20::133]:59772) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tgie2-00021n-Qn for 75845@debbugs.gnu.org; Sat, 08 Feb 2025 06:05:10 -0500 Received: by mail-lf1-x133.google.com with SMTP id 2adb3069b0e04-54410d769f7so2912752e87.3 for <75845@debbugs.gnu.org>; Sat, 08 Feb 2025 03:05:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739012700; x=1739617500; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=eLEgQxixu+Rft9/Euz81RpHdSoFW9Z65XH+KFsOZNTk=; b=k1fJaGD9iB7mIrMcnw96ZcSJpsttCaa0yYbKiLYMxkGkAfNkRqGKovGgDtwsFoQ6sX BbdpITjPnDxZA3UbwYs3GbTw/CKUbxXlUzb4CO3tUYB6oyIvc11vEQkBcMER1kMBvHhd FMMLnwCNgH6+qamivO7nrTF5UZfEZ0U/gXpe89bHIRhgw2EVuSUlTDMAJobCTwsPzu1u KKDmvuzT2oTL08jULv3vyTqh2ALGN/boFlhPRIptaYG9Zd/XVRAYuDW4bk6+kKcXt8pn d5AejoWD0otKlOyPfYAOc9i9fF+qWQJBM5hM/isJgnWNupHGuIY1P+qFlqm4V+YzvYXh MVug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739012700; x=1739617500; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=eLEgQxixu+Rft9/Euz81RpHdSoFW9Z65XH+KFsOZNTk=; b=ROOxBx6Pt1qu1U56pqF7gknN4hDFY8xmyLaKCzpUNXVCvHIaBEjIddKmX9S3nNvlkB uQvfTkG/NXBZsvbgXVPebHeDyazO7fyrFww+F2HgZcKi37HEzxd69+1yHRwWpIFJSBeS w3PQCsfmQJem0SwzsHDKXOWgBAmXy8AGoi4QLV1igtQYfmr6cWWVF2dJ+JphenxXxYRO DMLWvQ1Q3KVBdoW3jwlRg/yXNsXkPmfe6fuarSWA5/O+F6aqotYW9yEFNxL3hytfz/x7 tOy9uGHaQXdk3iUi5eKZrN6Agn45VzDe/oBLw3G0YTTDk/OxjLqpY/RfP/hc5wwSl+rx KyeQ== X-Forwarded-Encrypted: i=1; AJvYcCWsPUotyasnj+jc1wXboLJhSK309xuHmIZBbpdczH+fHF/wLSzPVYUqhOzfClplYtNH/FLSkw==@debbugs.gnu.org X-Gm-Message-State: AOJu0YyR7KrdVW3BZUJJBDv6CCDFsb4n9fg5xU/KovZNCVZytbpUiHzf 1IiRqSMtoZmrcYpSVNz8nFiS4CpTWo3NpQe/Fky/1Pub5zDMj4/8 X-Gm-Gg: ASbGncshLeG6tBJ94z3g+LiCK+7YM9HvC0n7/LkdDKXjBfH4LYCcYKkM6d275V+2FRs Ab845QKAUrVH5wVZmbyObA0p5tGU4pIFvCvGXLAh02swnE1bXA+TOkfPdPYnNvp5UBd+dUcPhUv rGN6uYV9XUDzQx3JzquVaaPnGYVCTBG1IOnPem/3i9iWW5pUf/CEfRJrI93MsJdPOUCPU47j797 Zy8yvSx/ByZlzBDd2S3TXzL7dBGowPJH29guHubsRu4S2tlxzKkB+CQ+fACd0WqLGaDYZ9j0N9p gQ7PLTEFGDylqyZoKnM2I3DTOTWK1mpED++n5GlL1T2xvRDjreDpN1Z32CiYLVG7GWCWzCOZXVW aaTFkVw== X-Google-Smtp-Source: AGHT+IGnT/zQwblwL6asTLGl04aF9ROtf8Ogk+W4fOdT+r4/qh4EEJs5Lcbzpjz5B63qT7AWWaR6yQ== X-Received: by 2002:a05:6512:36d1:b0:53e:fa8b:8227 with SMTP id 2adb3069b0e04-54414ae6476mr1982071e87.45.1739012699994; Sat, 08 Feb 2025 03:04:59 -0800 (PST) Received: from smtpclient.apple (c188-150-186-155.bredband.tele2.se. [188.150.186.155]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-54506ef1733sm42581e87.1.2025.02.08.03.04.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 08 Feb 2025 03:04:59 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= In-Reply-To: <86seoo3hwz.fsf@gnu.org> Date: Sat, 8 Feb 2025 12:04:56 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <0ACBFFDF-6695-45E5-97E7-6F6A9612D6B4@gmail.com> References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> <87y0yxn5h5.fsf@protonmail.com> <86sep5r3g6.fsf@gnu.org> <86seoo3hwz.fsf@gnu.org> X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 8 feb. 2025 kl. 10.27 skrev Eli Zaretskii : >> I'd like also to hear from Mattias, if possible, before we install >> this. Sorry, I'm trying to catch up on my Emacs backlog after a hiatus. Will = get back to this shortly. Thank you for the reminder. From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 20 Feb 2025 18:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: Pip Cet , monnier@iro.umontreal.ca, 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.17400754178623 (code B ref 75845); Thu, 20 Feb 2025 18:17:02 +0000 Received: (at 75845) by debbugs.gnu.org; 20 Feb 2025 18:16:57 +0000 Received: from localhost ([127.0.0.1]:40461 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tlB6X-0002F1-3J for submit@debbugs.gnu.org; Thu, 20 Feb 2025 13:16:57 -0500 Received: from mail-lf1-x133.google.com ([2a00:1450:4864:20::133]:53349) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tlB6T-0002Dp-J3 for 75845@debbugs.gnu.org; Thu, 20 Feb 2025 13:16:54 -0500 Received: by mail-lf1-x133.google.com with SMTP id 2adb3069b0e04-546202d633dso1338275e87.2 for <75845@debbugs.gnu.org>; Thu, 20 Feb 2025 10:16:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740075407; x=1740680207; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=uQwf6iGYFPC5tC+o78AEXPaaX1n+H4ugMUcRuc3725E=; b=aNAKvdDVDMahmcoExxisySNPLjZDKSMz8j/VCwD55dahFyUzePahZaBj4rNjuX9djq lMW0uj5rw2VSDkpPS1gXMftlNbbVsF7fW2urvw3UVzSRno41H+M9Z5Kl6skey9kTVs9R N41AA/R0sWRP0MzKjw3EuI5Apjx20lzIRfswjjvxIv8etmT9udnpuVjF8jlnZUkzUDj5 uHVUNU+Qz9ahhi/h8B6FXwi11FUnat8bX4zhub/uUfAIy+v8Sx5XPP7g3tk1C4tBiNoL Sd+eUSNTvVu9EfPdD3WIWYqTFySz6gx7nLfBz4BLYvTuQjBr/oKw723djjtEfUJ900nr AU6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740075407; x=1740680207; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=uQwf6iGYFPC5tC+o78AEXPaaX1n+H4ugMUcRuc3725E=; b=H3LKxsft91j1i77FF2IjWPMg83wtLKBaXCIB8qPoVqYlMg6sdgrFPLKx2JYQQrPE/u E88g3r8CohDtLP016UvRNrnRShypIdQibaMf2R+P9xl81jLv88Y0w5CWPEQt+j09xXZw KmQ6UVwaOfy5TdG/b9Dt350PG+k6DHwuJ/vkK15TOezzOiEAOEgZbQYuNRCvOTsd7Uer t4b9D4ndOsJekcUoREV3tvul6QRlsDwloVIbDjd0rpsc5PPpv820/R6++8iPnYIp0Dzi NFBQ9+cWx6jv/GuUd+OdMwqmh6ORRPLrKMex55iN50RfOf7VoQrXsmej7SmnKEoMjHCe F41Q== X-Forwarded-Encrypted: i=1; AJvYcCU9sjSSYVED8YKqVCRu3WZhSfe9dMqL3I1ZJgaM209B2YjYJ3nZcRJ5XaclOr0qdop2y80eKg==@debbugs.gnu.org X-Gm-Message-State: AOJu0YynHxcCmkqpVDYwsYddj7FU2d2yN9gIssm0BF29Zfrs7bwbrxhz tPmKEbqjP+VdpAhVObL89ga7q317m12bYi79r8Hiw7gXlMfXrMyf X-Gm-Gg: ASbGncuASZuHUqL/jgNVXe9Tw+IrByuOoFACHvCNonDyPtAPbB3+34T/IShv6ku08p4 ncLr3A0cSOE7VyvAs/sndZiuZgFcamdc1qJ/jj9pdjxGrFmUrUfwVT1Guz3H1bBlJuUNS4Z7tbD nNYbt6advOddgT7w73uZO0W51jwRXZrBbIuIS7/nYc63r/RpqufUdkdr6S4GdilwyPrLk3Hoz1i mMqhqG9QsKgWT5GDqM684LiDR8JrF4RizA/WrQf4vh6abotGV0yFxqM/ydzZ0Im4S/S0M5rHRvp WmBbAF3+Wf8JIMytQB1/LXHXKrZvHSfZGBojC1NDd42HGQkPl1v3pODXRMu5RnDguyzmF50/cNx 1QJO1lQ== X-Google-Smtp-Source: AGHT+IESu6uf1Hu9yN4baG17EiiNN3+Pf54m5MeyleS7RMjcGgEJ8sW66CFHWGhaTcXME+1JZilgwg== X-Received: by 2002:a05:6512:acc:b0:546:2f7a:38ae with SMTP id 2adb3069b0e04-54838ee339dmr12333e87.3.1740075406753; Thu, 20 Feb 2025 10:16:46 -0800 (PST) Received: from smtpclient.apple (c188-150-186-155.bredband.tele2.se. [188.150.186.155]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-545d1b9c30fsm1701332e87.15.2025.02.20.10.16.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Feb 2025 10:16:46 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= In-Reply-To: <86sep5r3g6.fsf@gnu.org> Date: Thu, 20 Feb 2025 19:16:42 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <8CEC193B-264B-424A-BF1B-3829B96A23AB@gmail.com> References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> <87y0yxn5h5.fsf@protonmail.com> <86sep5r3g6.fsf@gnu.org> X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>>> Thanks, adding Stefan and Mattias to the discussion, in the hope = that >>>> they could review the patch and comment. Finally I got some time to look at this, and the patch looks fine, = largely. Now that `mapcar1` always returns its first argument (leni), the return = value can be eliminated. The change in the list loop is cheap and fine. The new tests and logic in the string loop are more expensive (5-10 % = overhead, from a quick test) but given that strings are less common in = calls to mapcar etc, it's probably not so bad. More significantly, these = checks will go away entirely when we abolish size-changing string = mutation -- mentioned elsewhere but there is no need for that issue to = be resolved before this bug. We should probably amend the documentation for mapcar & co with respect = to the restrictions on mutating the input sequence. (Common Lisp has = basically the same restrictions.) At least the manual would do well to = include a short notice about this; we do mention it for `maphash`. We could just contend ourselves with stating that the input sequence = shouldn't be mutated and that's it, but if you really want to spell it = out in detail (allowed to change the CARs but not the CDRs etc) then = that's fine. Again, strings are messy here because it depends on = internal representation, so maybe keep it simple. From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 21 Feb 2025 12:36:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Cc: Eli Zaretskii , monnier@iro.umontreal.ca, 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.17401413272499 (code B ref 75845); Fri, 21 Feb 2025 12:36:03 +0000 Received: (at 75845) by debbugs.gnu.org; 21 Feb 2025 12:35:27 +0000 Received: from localhost ([127.0.0.1]:56223 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tlSFZ-0000e9-Ui for submit@debbugs.gnu.org; Fri, 21 Feb 2025 07:35:26 -0500 Received: from mail-4316.protonmail.ch ([185.70.43.16]:13693) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1tlSFV-0000cn-ST for 75845@debbugs.gnu.org; Fri, 21 Feb 2025 07:35:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1740141315; x=1740400515; bh=jJCQwXf81p7zuagpqd+7FvZhltCHKasVhJThEg63UiI=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=BLAR/Ua3T0C2RjDc95KgqbhpYMamakp7PEJX4dFOaoRAo8ZnPG0UvSRZ3xo6Fhp1J URg3jPJ+ycK3jLw4VPgl7jhzzubKJJioj4yBdqUpyx2K9+gjK9f0TB0a/co9i1PdKy ax2ETWLwpTr3FyG8I6lFjHhoIwga29kM7Zf2n4+qDEIvPsoMxcfwa0X/O9zukIIyjF 4xJnAgOxI47NAT456WZiq/XWxWetSt2k2lXL92fUUjvFQb2lPXbTUYb4SyxbnsymBF rHSkpGI3Y4Fa67GGp8FVRVJBzzk9eSCM9bQppQlUBbMtvNvhSj7yXvu6Gw3VUs33FC zktM8zdL+VZEA== Date: Fri, 21 Feb 2025 12:35:10 +0000 From: Pip Cet Message-ID: <87ldtztr19.fsf@protonmail.com> In-Reply-To: <8CEC193B-264B-424A-BF1B-3829B96A23AB@gmail.com> References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> <87y0yxn5h5.fsf@protonmail.com> <86sep5r3g6.fsf@gnu.org> <8CEC193B-264B-424A-BF1B-3829B96A23AB@gmail.com> Feedback-ID: 112775352:user:proton X-Pm-Message-ID: e5c8edb9c3d429daf9142cd0951c53570a0116aa MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Mattias Engdeg=C3=A5rd writes: >>>>> Thanks, adding Stefan and Mattias to the discussion, in the hope that >>>>> they could review the patch and comment. > > Finally I got some time to look at this, and the patch looks fine, largel= y. Thanks! I'll probably apply something like it, but see below: eassert is now officially used in places where Lisp code can trigger it, so there's little point fixing eassert crashes in one place while new ones are being deliberately added in others. > Now that `mapcar1` always returns its first argument (leni), the return v= alue can be eliminated. Certainly, but this entire code is so strange that I don't really want to touch it more than necessary, and this particular redundancy seems potentially useful to me. (If we really want to optimize it further, let's get rid of the strange double loop in mapconcat first, which spreads the arguments then goes back over the list in reverse order to fill in the separators). > The change in the list loop is cheap and fine. > > The new tests and logic in the string loop are more expensive (5-10 % > overhead, from a quick test) but given that strings are less common in > calls to mapcar etc, it's probably not so bad. More significantly, I maintain that using strings as a sequence argument to the map* functions is so rare we should just turn them into a list or vector instead and drop the special-casing code. > these checks will go away entirely when we abolish size-changing ^^^^ I still think that's an "if". > string mutation -- mentioned elsewhere but there is no need for that I'm still not convinced abolishing string mutation is a good idea, but if we do, we should not special-case "size-changing" mutations, because how many bytes we use to encode each character is an implementation detail that shouldn't be exposed to Lisp. Indeed, the code becomes a lot simpler if we make all forms of string mutation allocate new SDATA blocks while keeping the old ones alive, but that needs GC changes. > We should probably amend the documentation for mapcar & co with > respect to the restrictions on mutating the input sequence. (Common > Lisp has basically the same restrictions.) At least the manual would > do well to include a short notice about this; we do mention it for > `maphash`. With maphash, this actually happens (it does, or did, in the nativecomp code) by accident. I'm not sure the same is true for sequences, where "don't crash" is probably the best we can hope for. That said, we've since introduced some new code which crashes because Lisp code misbehaves (608113628c2750b09b925b17c96a29b2dc9abc37, which easserts that a Lisp function returns an integer in the desired range), so there appears to be no consensus that Lisp code shouldn't cause eassert crashes at this point. My idea remains that one should be able to experiment with Lisp without running into eassert crashes (or ordinary non-eassert-mediated ones), but if there's no consensus supporting that, fixing mapcar while leaving readcharfun broken seems pointless. > We could just contend ourselves with stating that the input sequence > shouldn't be mutated and that's it, but if you really want to spell it I think that would be best. If you need fine-grained control, just don't use mapcar and open-code it instead. Pip From unknown Fri Sep 19 21:16:32 2025 X-Loop: help-debbugs@gnu.org Subject: bug#75845: mapconcat crashes for unreasonable self-modifications Resent-From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 21 Feb 2025 15:12:06 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 75845 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Pip Cet Cc: Eli Zaretskii , monnier@iro.umontreal.ca, 75845@debbugs.gnu.org Received: via spool by 75845-submit@debbugs.gnu.org id=B75845.174015067620313 (code B ref 75845); Fri, 21 Feb 2025 15:12:06 +0000 Received: (at 75845) by debbugs.gnu.org; 21 Feb 2025 15:11:16 +0000 Received: from localhost ([127.0.0.1]:58896 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tlUgK-0005Gp-BB for submit@debbugs.gnu.org; Fri, 21 Feb 2025 10:11:15 -0500 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]:53355) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1tlUg4-0005C3-Ml for 75845@debbugs.gnu.org; Fri, 21 Feb 2025 10:11:06 -0500 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-546202d633dso2333280e87.2 for <75845@debbugs.gnu.org>; Fri, 21 Feb 2025 07:10:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740150647; x=1740755447; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:from:to:cc:subject :date:message-id:reply-to; bh=bIzRFcoG6s7zjtontJCSen920HKPwkRxC1G+SplMXik=; b=LFvNS0vjmBifmrc3j0MsSmdrJ/D+/TZBBrlPSsi5mqxOlzzt4iPTA754QE/P9aEZgk emWerbjDhcK5qYzKcLzHMbdSBkeRb41eodjMoINSGqCSUtLJ5qY+sPcBrnzKdylAVFMz qMLlKVPV2dwYN7PB+pN/LSKv4JHv1C3NML7fYULziuI7t+EX/n/uu6grInqui9T3a3aR eoA1cIEuzNlFxov2JX1MgQExu/iIiu91zd0X7QAsX8hl9hf86qp3O8IVXnBNASVKwXrd l9K1c314kwE++LhyFeAMoHjt/lscutdnpb7HVut5AeDMTSk7efKZCzN/wyL0HMVMVK85 1CJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740150647; x=1740755447; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:sender:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=bIzRFcoG6s7zjtontJCSen920HKPwkRxC1G+SplMXik=; b=sGl+Xn+JHTmkEEabId/U+3UQqlMdnXZXYjoBr0EH7wZs7pGWDCuq023KHJx/9SpfJ3 HePp6jFjMFPsDUK7dclEE+TfDAL4n2EeSlYTn0UNrSCnMnOehZoPxstF63CQWWhLW2E3 MuWkNUJaL8CNJ1+sX8frYydovJ+zEZk/vN+IpeKApEKR7c9HT++G2LRpMJJqu0LulUDn hQKN5bFkCWAgOGOBh1QXtLYoeS4cx3aoYGAt9F/NL1KPDjaUYowBmR+lPlw+YrJL/FBn 5WGfJE6bMvNp7FBN6QlOXY+MF9lGB8WQuUzquQbZQqXPipep2hauMjrDPNhk3sKjCJwS ubBg== X-Forwarded-Encrypted: i=1; AJvYcCXzpJQhJHtmczRLi6ZvPWlbOUAiJ2d+eR8Htb5KiXGPpI8QnJxtgTtIooGr6q6SmCbVeYrz7A==@debbugs.gnu.org X-Gm-Message-State: AOJu0YyoOrFQdBGucVJBMww+mwC68OXpwcu9qrfpibSky7om0rtteGc2 8J77qe7Cg7I4mNF7KllMaCVe01WE+4A2KCOg0oo3R1J4Ye1C5xJ7 X-Gm-Gg: ASbGnctL5u9nA2k2muEPeNg8NZxp/XkjE8S6dNUOLqxTOelRurluOJhuGe2CStsvaVe 47eqsYwJpBgTxKfHfTGFfWa5QDGwEowiC25mgH5tfp6W9CTRt3cHZOMfiZ0p/4yJ30MVCm9sS6z YGUUhAGTRrQyNle+IcNbI6kqn3hhVT8vp9Edu/sCTllEW/3FW8dLBVpHTYmXwjohUJVnNl9uyXo k8/Q0uEta1dSymSdR8OYxkCJZ+rEndyJn3usVQVZcIyMC/2ZSpfl2T1nxoJdEAE/9Mb3/qmRlTZ 1HLn27uRwD3+jHdwtAkUBh6sUOtpxns8iP2wDqdPk/esq6dikW5whDqN58lw6WFVOc/KDdk3dRS ACQ2MX/MxokyNieZx X-Google-Smtp-Source: AGHT+IEuYy7nNaWnx/Bu7FQ2zV7Et1rIgT2qdmJ7EFc1Ln2QlPuN7tn2UwUW1CiChwtSKWVMtC3Qzw== X-Received: by 2002:a05:6512:3b23:b0:545:a2f:22ba with SMTP id 2adb3069b0e04-54838f590a5mr1598094e87.37.1740150646712; Fri, 21 Feb 2025 07:10:46 -0800 (PST) Received: from smtpclient.apple (c188-150-186-155.bredband.tele2.se. [188.150.186.155]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-545284e2817sm2387435e87.116.2025.02.21.07.10.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Feb 2025 07:10:46 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.15\)) From: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= In-Reply-To: <87ldtztr19.fsf@protonmail.com> Date: Fri, 21 Feb 2025 16:10:45 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <8EA363C3-3A4E-4FB8-9563-FF9768C31C3D@gmail.com> References: <87r04qpc2m.fsf@protonmail.com> <87ikq2pafv.fsf@protonmail.com> <86jzait664.fsf@gnu.org> <87y0yxn5h5.fsf@protonmail.com> <86sep5r3g6.fsf@gnu.org> <8CEC193B-264B-424A-BF1B-3829B96A23AB@gmail.com> <87ldtztr19.fsf@protonmail.com> X-Mailer: Apple Mail (2.3654.120.0.1.15) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) 21 feb. 2025 kl. 13.35 skrev Pip Cet : > (If we really want to optimize it further, let's get rid of the = strange > double loop in mapconcat first, which spreads the arguments then goes > back over the list in reverse order to fill in the separators). I don't think it's really a cache killer but will give it a look. > I maintain that using strings as a sequence argument to the map* > functions is so rare we should just turn them into a list or vector > instead and drop the special-casing code. I'll think about it. > I'm still not convinced abolishing string mutation is a good idea, Let's try to solve this bug in isolation. We're almost done! > With maphash, this actually happens (it does, or did, in the = nativecomp > code) by accident. I'm not sure the same is true for sequences, where > "don't crash" is probably the best we can hope for. Lisp shouldn't crash but it's impractical to be absolute about it. The = programming interface is so rich that it's almost impossible to prevent = crashes altogether, and performance considerations further reduce what = checks we want to perform. For instance, the user can easily crash Emacs = by running bad bytecode. But users expect more about basic Lisp = primitives like mapcar & co and since it seems that a reasonable effort = can prevent it, and it's not (too) expensive, let's try. Feel free to commit your patch, or post it here if you want us to poke = fun at it beforehand.