From unknown Thu Jun 19 14:04:38 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#78042 <78042@debbugs.gnu.org> To: bug#78042 <78042@debbugs.gnu.org> Subject: Status: 31.0.50; Improper sequence of `before/after-change-functions` calls Reply-To: bug#78042 <78042@debbugs.gnu.org> Date: Thu, 19 Jun 2025 21:04:38 +0000 retitle 78042 31.0.50; Improper sequence of `before/after-change-functions`= calls reassign 78042 emacs submitter 78042 Stefan Monnier severity 78042 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 24 11:39:24 2025 Received: (at submit) by debbugs.gnu.org; 24 Apr 2025 15:39:24 +0000 Received: from localhost ([127.0.0.1]:40326 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u7yfb-0006OP-TI for submit@debbugs.gnu.org; Thu, 24 Apr 2025 11:39:24 -0400 Received: from lists.gnu.org ([2001:470:142::17]:40290) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u7yfX-0006Nt-If for submit@debbugs.gnu.org; Thu, 24 Apr 2025 11:39:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u7yfR-0000St-MH for bug-gnu-emacs@gnu.org; Thu, 24 Apr 2025 11:39:13 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u7yfP-0003I5-RS for bug-gnu-emacs@gnu.org; Thu, 24 Apr 2025 11:39:13 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 1D48F442729; Thu, 24 Apr 2025 11:39:02 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745509140; bh=v1D+Zy+oFemunenFMN/o1MEKCbsFm443+1NeSLFUj0w=; h=From:To:Subject:Date:From; b=nq+GTkjF1sr24+LvOmeAcx1aF3tEZaRyaQZSG9KGVy48m8OAg8BJj2maZznAtAKpG By00lq/P4bhCCau9PmNGwr6xiILbDWNwHRjBXiE1LGMJ252eTB7h89dQ6dO3OtaBrq lmeXyAYRA6gsXp2IPlLfkWWg4LduYbR/5/KDB6DDoT+oznnWw9ntAgff1DUIZmf70p XWFxY7gRuvWO05Ebr2mnjo4oTt94mgNxPRdJv9C9w1hMy8uQhjsmcXfkZ0lJiwQvap ckNvgb8g1RBNAIdooLCgv0vmDUixs1PoNQYNRvA5niesnvUQk82FuTsh5DT7sXTOEU zBFg+RVDCPVMA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 9C53944272B; Thu, 24 Apr 2025 11:39:00 -0400 (EDT) Received: from pastel (104-195-208-18.cpe.teksavvy.com [104.195.208.18]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 7E4AA1206DB; Thu, 24 Apr 2025 11:39:00 -0400 (EDT) From: Stefan Monnier To: bug-gnu-emacs@gnu.org Subject: 31.0.50; Improper sequence of `before/after-change-functions` calls Message-ID: X-Debbugs-Cc: monnier@iro.umontreal.ca Date: Thu, 24 Apr 2025 11:38:59 -0400 MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.020 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Package: Emacs Version: 31.0.50 I just found another case where we break our promises w.r.t `before/after-change-functions`: src/emacs -Q src/regexp-emacs.c M-: (decode-coding-region 123 (point-max) 'windows-1252) RET [ The details of the file and buffer positions don't really matter, AFAIK. ] I first get a run of `before-change-functions` for 123...point-max, as expected at the beginning and a run of `after-change-functions` for 123...point-max, as expected at the end, but between the two I get another run of `before-change-functions` for 123..16497: #0 signal_before_change (start_int=start_int@entry=123, end_int=end_int@entry=16497, preserve_ptr=preserve_ptr@entry=0x0) at insdel.c:2157 #1 0x0000555555702c3a in prepare_to_modify_buffer_1 (start=start@entry=123, end=end@entry=16497, preserve_ptr=preserve_ptr@entry=0x0) at insdel.c:2024 #2 0x00005555557c9f36 in modify_text_properties (buffer=MISSING, buffer@entry=XIL(0x555555dfac0d), start=MISSING, end=MISSING) at textprop.c:85 #3 0x00005555557cb5cc in add_text_properties_1 (start=make_fixnum(123), end=make_fixnum(16497), properties=XIL(0x7fffffffce33), object=XIL(0x555555dfac0d), set_type=set_type@entry=TEXT_PROPERTY_REPLACE, destructive=destructive@entry=true) at textprop.c:1236 #4 0x00005555557cb9b8 in Fadd_text_properties (start=MISSING, start@entry=make_fixnum(123), end=MISSING, end@entry=make_fixnum(16497), properties=MISSING, object=MISSING, object@entry=XIL(0x555555dfac0d)) at textprop.c:1308 #5 0x00005555557cba12 in Fput_text_property (start=make_fixnum(123), end=end@entry=make_fixnum(16497), property=MISSING, property@entry=XIL(0x55b0), value=MISSING, value@entry=XIL(0x2aaa9984e428), object=XIL(0x555555dfac0d)) at textprop.c:1326 #6 0x0000555555644852 in produce_charset (coding=coding@entry=0x7fffffffd080, charbuf=charbuf@entry=0x555556a7ef88, pos=pos@entry=16497) at coding.c:7277 #7 0x00005555556448e8 in produce_annotation (coding=coding@entry=0x7fffffffd080, pos=16497, pos@entry=123) at coding.c:7320 #8 0x000055555564578f in decode_coding (coding=coding@entry=0x7fffffffd080) at coding.c:7421 #9 0x000055555564cc32 in decode_coding_object (coding=coding@entry=0x7fffffffd080, src_object=src_object@entry=XIL(0x555555dfac0d), from=from@entry=123, from_byte=from_byte@entry=123, to=to@entry=161558, to_byte=to_byte@entry=161558, dst_object=XIL(0x555555dfac0d)) at coding.c:8183 #10 0x000055555564e1c7 in code_convert_region (start=make_fixnum(123), end=make_fixnum(161558), coding_system=XIL(0x2aaa9984e428), dst_object=XIL(0x555555dfac0d), encodep=encodep@entry=false, norecord=norecord@entry=false) at coding.c:9510 There's probably also a corresponding run of `after-change-functions`, I haven't checked, but the problem is that the "final run of `after-change-functions` for 123...point-max is now "invalid" in the sense that it modifies a range of the buffer beyond the last run of `before-change-functions`, contrary to what we promise. - Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 24 21:35:25 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 01:35:25 +0000 Received: from localhost ([127.0.0.1]:43408 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u87yO-0002r4-KR for submit@debbugs.gnu.org; Thu, 24 Apr 2025 21:35:25 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:37909) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u87yJ-0002qf-DG for 78042@debbugs.gnu.org; Thu, 24 Apr 2025 21:35:22 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 4CCB08087F; Thu, 24 Apr 2025 21:35:12 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745544911; bh=BF4MChPgzBm7DYVP4gGxHfYFrwkZwDHoC6f+Hxp04Sw=; h=From:To:Subject:In-Reply-To:References:Date:From; b=TJ4LK2OEs1ms6soirmHhzuvq4BOR2IIIZUtQpQbZfUgRRRLchhIpUUcwIMHPFdZKk gad333sjnefpLhZdEnNIiZCXnfupvHN10onGu0FXMYKRn4VxZzhfD8+DyveX3lDvqx INvH7bfWL81S2G11ufWtKjSq1kpWs+yne7lFa1ddx54YMF1uHAW6vhpZoPRzpoAPcF DuxO7VMcxGIjlXNizjRz6sQ0zJ1QWi4a/bKvsirxUgyr/wMZRbhpr9OypQERzxkGHD M86ZTtPZlAU7rYnYPDOz0VYkB6T9eeGjhrTfpbj6AwGFII8YGRlt8GRFYmkkMG2Buc kWt+18vO5SCFw== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 0101E806F8; Thu, 24 Apr 2025 21:35:10 -0400 (EDT) Received: from alfajor (104-195-232-56.cpe.teksavvy.com [104.195.232.56]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id CB83B12026C; Thu, 24 Apr 2025 21:35:10 -0400 (EDT) From: Stefan Monnier To: 78042@debbugs.gnu.org Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls In-Reply-To: Message-ID: References: Date: Thu, 24 Apr 2025 21:35:09 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.065 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain Hmm... The code that runs those hooks is really littered at odd places, with no good/standard way to control when to run it and when not. I suggest the patch below (which includes a corresponding test). Any objection? Stefan --=-=-= Content-Type: text/x-diff; charset=iso-8859-1 Content-Disposition: inline; filename=decode-change-hook.patch Content-Transfer-Encoding: quoted-printable diff --git a/src/coding.c b/src/coding.c index 63b0dbeb18b..b4209986918 100644 --- a/src/coding.c +++ b/src/coding.c @@ -7363,6 +7363,7 @@ decode_coding (struct coding_system *coding) struct ccl_spec cclspec; int carryover; int i; + specpdl_ref count =3D SPECPDL_INDEX (); =20 USE_SAFE_ALLOCA; =20 @@ -7389,6 +7390,8 @@ decode_coding (struct coding_system *coding) =20 undo_list =3D BVAR (current_buffer, undo_list); bset_undo_list (current_buffer, Qt); + /* The caller should run *-change-functions over the whole region. = */ + specbind (Qinhibit_modification_hooks, Qt); } =20 coding->consumed =3D coding->consumed_char =3D 0; @@ -7501,7 +7504,7 @@ decode_coding (struct coding_system *coding) record_insert (coding->dst_pos, coding->produced_char); } =20 - SAFE_FREE (); + SAFE_FREE_UNBIND_TO (count, Qnil); } =20 =20 diff --git a/test/src/editfns-tests.el b/test/src/editfns-tests.el index 2553ad3ec2c..9a27c420f1e 100644 --- a/test/src/editfns-tests.el +++ b/test/src/editfns-tests.el @@ -498,10 +498,10 @@ sanity-check-change-functions-verbose (defvar sanity-check-change-functions-op nil) (defmacro sanity-check-change-functions-with-op (op &rest body) (declare (debug t) (indent 1)) - `(let ((sanity-check-change-functions-op ,op)) - (sanity-check--message "%S..." sanity-check-change-functions-op) + `(let ((sanity-check-change-functions-op (list ,op))) + (sanity-check--message "%S..." ,op) ,@body - (sanity-check--message "%S...done" sanity-check-change-functions-op))) + (sanity-check--message "%S...done" ,op))) =20 (defun sanity-check--message (&rest args) (if sanity-check-change-functions-verbose (apply #'message args))) @@ -530,6 +530,7 @@ sanity-check-change-functions-check-size (setq sanity-check-change-functions-buffer-size (buffer-size))))) =20 (defun sanity-check-change-functions-before (beg end) + (push `(BEFORE ,beg ,end) sanity-check-change-functions-op) (sanity-check--message "Before: %S %S" beg end) (unless (<=3D (point-min) beg end (point-max)) (sanity-check-change-functions-error @@ -540,6 +541,7 @@ sanity-check-change-functions-before (setq sanity-check-change-functions-end end)) =20 (defun sanity-check-change-functions-after (beg end len) + (push `(AFTER ,beg ,end ,len) sanity-check-change-functions-op) (sanity-check--message "After : %S %S (%S)" beg end len) (unless (<=3D (point-min) beg end (point-max)) (sanity-check-change-functions-error @@ -565,7 +567,7 @@ sanity-check-change-functions-after (defun sanity-check-change-functions-errors () (sanity-check-change-functions-check-size) (if sanity-check-change-functions-errors - (cons sanity-check-change-functions-op + (cons (reverse sanity-check-change-functions-op) sanity-check-change-functions-errors))) =20 (ert-deftest editfns-tests--before/after-change-functions () @@ -591,6 +593,24 @@ editfns-tests--before/after-change-functions (decode-coding-region beg (point) 'utf-8) (should (null (sanity-check-change-functions-errors))))) =20 + (let ((beg (point))) ;bug#78042 + (apply #'insert (make-list 5000 "hell\351 ")) + (sanity-check-change-functions-with-op 'DECODE-CODING-LARGE-REGION + (decode-coding-region beg (point) 'windows-1252) + (should-not (sanity-check-change-functions-errors)))) + + (let ((beg (point))) ;bug#78042 + (sanity-check-change-functions-with-op 'DECODE-CODING-INSERT + ;; The `insert' calls make sure we track the buffer-size + ;; so as to detect if `decode-coding-string' fails to run the + ;; `*-change-functions'. + (insert "<") + (decode-coding-string "hell\351 " 'windows-1252 nil (current-buffe= r)) + (forward-char 6) + (insert ">") + (should (equal "" (buffer-substring beg (point)))) + (should-not (sanity-check-change-functions-errors)))) + (sanity-check-change-functions-with-op 'ENCODE-CODING-STRING (encode-coding-string "=E9=E9=E9" 'utf-8 nil (current-buffer)) (should (null (sanity-check-change-functions-errors)))) --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 02:09:06 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 06:09:07 +0000 Received: from localhost ([127.0.0.1]:45810 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8CFG-0001ny-AN for submit@debbugs.gnu.org; Fri, 25 Apr 2025 02:09:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56198) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8CFD-0001nL-F9 for 78042@debbugs.gnu.org; Fri, 25 Apr 2025 02:09:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8CF7-0000o6-FI; Fri, 25 Apr 2025 02:08:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=v8AQYj8EyfvAyqdPJGdoYMgmyxJBPq2fKsSswaeEoyc=; b=q80lw63mz+uG HTwmk6ipWTu63Bs6N0G3kXpB5Ngw/VjOCXUs4hbgxQ9YIAtKFYa/Q6Ot9h5O4zbstme9pd1CS9i3F 9PY/1rjYGtF9cdvSwHNjH09XTXjW4hiePiR+npKFamEmj1hYNN8V5fXG2fQ8uepJPAEvykxuZ1ZEW jg0tQRQ1TXBOcGlY3+ocSPvgae7J2EiF/gFXZzQAJmkJax4NEH43/MxavbDKqrlTX3rKV0jjXY8Ti jGQsOtP6ZFagvNTlOd0/ExcHea/2ErX+qJ9b75geFEntW3aZKOAyq8sG6ZrnAmMcWARYyqJmDjE7W s6eW0uU9EBi1pf2p9Gt5Xw==; Date: Fri, 25 Apr 2025 09:08:53 +0300 Message-Id: <86plh0ydqi.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (bug-gnu-emacs@gnu.org) Subject: Re: bug#78042: 31.0.50; Improper sequence of `before/after-change-functions` calls References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: monnier@iro.umontreal.ca, 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: monnier@iro.umontreal.ca > Date: Thu, 24 Apr 2025 11:38:59 -0400 > From: Stefan Monnier via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > Package: Emacs > Version: 31.0.50 > > > I just found another case where we break our promises w.r.t > `before/after-change-functions`: > > src/emacs -Q src/regexp-emacs.c > M-: (decode-coding-region 123 (point-max) 'windows-1252) RET > > [ The details of the file and buffer positions don't really > matter, AFAIK. ] > > I first get a run of `before-change-functions` for 123...point-max, as > expected at the beginning and a run of `after-change-functions` for > 123...point-max, as expected at the end, but between the two I get > another run of `before-change-functions` for 123..16497: > > #0 signal_before_change > (start_int=start_int@entry=123, end_int=end_int@entry=16497, preserve_ptr=preserve_ptr@entry=0x0) at insdel.c:2157 > #1 0x0000555555702c3a in prepare_to_modify_buffer_1 > (start=start@entry=123, end=end@entry=16497, preserve_ptr=preserve_ptr@entry=0x0) at insdel.c:2024 > #2 0x00005555557c9f36 in modify_text_properties (buffer=MISSING, > buffer@entry=XIL(0x555555dfac0d), start=MISSING, end=MISSING) at textprop.c:85 > #3 0x00005555557cb5cc in add_text_properties_1 > (start=make_fixnum(123), end=make_fixnum(16497), properties=XIL(0x7fffffffce33), object=XIL(0x555555dfac0d), set_type=set_type@entry=TEXT_PROPERTY_REPLACE, destructive=destructive@entry=true) at textprop.c:1236 > #4 0x00005555557cb9b8 in Fadd_text_properties (start=MISSING, > start@entry=make_fixnum(123), end=MISSING, > end@entry=make_fixnum(16497), properties=MISSING, object=MISSING, > object@entry=XIL(0x555555dfac0d)) at textprop.c:1308 > #5 0x00005555557cba12 in Fput_text_property > (start=make_fixnum(123), end=end@entry=make_fixnum(16497), property=MISSING, > property@entry=XIL(0x55b0), value=MISSING, > value@entry=XIL(0x2aaa9984e428), object=XIL(0x555555dfac0d)) > at textprop.c:1326 > #6 0x0000555555644852 in produce_charset > (coding=coding@entry=0x7fffffffd080, charbuf=charbuf@entry=0x555556a7ef88, pos=pos@entry=16497) at coding.c:7277 > #7 0x00005555556448e8 in produce_annotation > (coding=coding@entry=0x7fffffffd080, pos=16497, pos@entry=123) > at coding.c:7320 > #8 0x000055555564578f in decode_coding (coding=coding@entry=0x7fffffffd080) > at coding.c:7421 > #9 0x000055555564cc32 in decode_coding_object > (coding=coding@entry=0x7fffffffd080, src_object=src_object@entry=XIL(0x555555dfac0d), from=from@entry=123, from_byte=from_byte@entry=123, to=to@entry=161558, to_byte=to_byte@entry=161558, dst_object=XIL(0x555555dfac0d)) > at coding.c:8183 > #10 0x000055555564e1c7 in code_convert_region > (start=make_fixnum(123), end=make_fixnum(161558), coding_system=XIL(0x2aaa9984e428), dst_object=XIL(0x555555dfac0d), encodep=encodep@entry=false, norecord=norecord@entry=false) at coding.c:9510 > > There's probably also a corresponding run of `after-change-functions`, > I haven't checked, but the problem is that the "final run of > `after-change-functions` for 123...point-max is now "invalid" in the > sense that it modifies a range of the buffer beyond the last run of > `before-change-functions`, contrary to what we promise. Sorry, I don't think I understand the problem. Is the problem that we have "nested" notifications, whereby you have before-change-functions A1 B1 before-change-functions A2 B2 after-change-functions A2 B2 after-change-functions A1 B1 IOW, _any_ situation where we have nested pairs of notifications is "breaking our promise"? If so, how can that be avoided in principle in Emacs, given its recursive nature, except by some mechanism which collects all the notifications and rearranges them to "keep our promise"? Or what am I missing? From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 03:32:49 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 07:32:49 +0000 Received: from localhost ([127.0.0.1]:46557 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8DYG-00007o-Pm for submit@debbugs.gnu.org; Fri, 25 Apr 2025 03:32:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48766) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8DYE-00007b-U5 for 78042@debbugs.gnu.org; Fri, 25 Apr 2025 03:32:47 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8DY7-0002vh-RM; Fri, 25 Apr 2025 03:32:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=pPyOdTgvBkcc6eySMmXnzc2PiBQH7mJ6SjDJn/nnAao=; b=D7pEcllDdVt3 JsUm5T//sjMdCJsBsTZQZP0goGHzlVWBe5HlzaV7SeKlShJFsFTDiZxvLpwxRyhafwNs/BLHY9XjE DcqjexFKfxk//qwLg3vmgvk8EkXrWNYCuZ1REt3P2KRZBiYUk6ifMpPFWtorkE/VW3bKDTbc/AmmO QD7uTZuMrcJYW4b/IYyFxZJNta/tBeyLUWY4PBectJ0xcbGR99JHkIELi0H4oYBvzaX8i4LR1BOCR VjLogjG50GSN677nUSiFjoZ2C+ePwCyv//PYgd//3T6cQ33zBK+v+sWB5k5BMOrAh4s+d8aDQnm5O 3rYD+72/yGK5pC1U9HFd9Q==; Date: Fri, 25 Apr 2025 10:32:32 +0300 Message-Id: <867c38y9v3.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (bug-gnu-emacs@gnu.org) Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls References: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Thu, 24 Apr 2025 21:35:09 -0400 > From: Stefan Monnier via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > The code that runs those hooks is really littered at odd places, with no > good/standard way to control when to run it and when not. > I suggest the patch below (which includes a corresponding test). > Any objection? > > > diff --git a/src/coding.c b/src/coding.c > index 63b0dbeb18b..b4209986918 100644 > --- a/src/coding.c > +++ b/src/coding.c > @@ -7363,6 +7363,7 @@ decode_coding (struct coding_system *coding) > struct ccl_spec cclspec; > int carryover; > int i; > + specpdl_ref count = SPECPDL_INDEX (); > > USE_SAFE_ALLOCA; > > @@ -7389,6 +7390,8 @@ decode_coding (struct coding_system *coding) > > undo_list = BVAR (current_buffer, undo_list); > bset_undo_list (current_buffer, Qt); > + /* The caller should run *-change-functions over the whole region. */ > + specbind (Qinhibit_modification_hooks, Qt); > } How do we make sure this requirement is heeded to? And how do we document this incompatible change in NEWS? From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 12:37:49 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 16:37:49 +0000 Received: from localhost ([127.0.0.1]:52037 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8M3f-0005My-QM for submit@debbugs.gnu.org; Fri, 25 Apr 2025 12:37:49 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:52855) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8M3Y-0005L6-QF for 78042@debbugs.gnu.org; Fri, 25 Apr 2025 12:37:46 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 63A4380911; Fri, 25 Apr 2025 12:37:35 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745599054; bh=9A8pF7AlRQo4u9I/4bvCxzTkbmwd1JTgtIZJJFW3e5U=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=PbGnPE8aLt2+6xjL8Kt8svcSSNepJYwKvV1ioOW/92Ppq8obBzyglNhInhF+OIe5L MBVIIEXt42wELlJrgXftDwz/XtHU9vBqqTdHASL5VGUW/imjf0/5WEmWYeEMtGXRwq 7vGeibf6mUUM5Zhx9i4zJ1557hDXs03fH3UXYqaG4zjSTKldZWj/fvqqKtGKglkO5q b5N43DDtSRjtPjbqG7feltxrqczVcwgmM4liO1tjxgD1PfVnHvgBMr7seKb4mVythF nrn4GR7mq0UjFmxOUhSJcaCyEwFywEqOSkO8nF8ildLT/JAegZ9Lgo+9F/nXoZqGum J2vbgbSwAJNOw== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id A02228061C; Fri, 25 Apr 2025 12:37:34 -0400 (EDT) Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 8E64B1204E6; Fri, 25 Apr 2025 12:37:34 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#78042: 31.0.50; Improper sequence of `before/after-change-functions` calls In-Reply-To: <86plh0ydqi.fsf@gnu.org> Message-ID: References: <86plh0ydqi.fsf@gnu.org> Date: Fri, 25 Apr 2025 12:37:34 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.205 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Sorry, I don't think I understand the problem. Is the problem that we > have "nested" notifications, whereby you have > > before-change-functions A1 B1 > before-change-functions A2 B2 > after-change-functions A2 B2 > after-change-functions A1 B1 > > IOW, _any_ situation where we have nested pairs of notifications is > "breaking our promise"? Yup: the `before-change-functions A2 B2` promises that there won't be any changes to the buffer outside of A2...B2 until the next `before-change-functions`. > If so, how can that be avoided in principle in Emacs, given its > recursive nature, except by some mechanism which collects all the > notifications and rearranges them to "keep our promise"? There is currently no attempt in the C code to solve this "by construction" or "by design". Instead we fix the offenders as we bump into them. Stefan From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 13:39:39 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 17:39:39 +0000 Received: from localhost ([127.0.0.1]:52459 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8N1W-0001cz-Es for submit@debbugs.gnu.org; Fri, 25 Apr 2025 13:39:38 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:57255) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8N1T-0001cg-Hw for 78042@debbugs.gnu.org; Fri, 25 Apr 2025 13:39:36 -0400 Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 04D4980911; Fri, 25 Apr 2025 13:39:30 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745602769; bh=gatoLUahRqEA4KEfiy0ICM9Kom1mi8WmXM5I6oBsVTQ=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=mBqPXg752lQvTDNjMRjxuD/SwfkNrRtb1jvArMEAOTp9sJobFJD7ubQxroeqFRl8o cQ/ra3VAMuUHCVB5aSzCC+BxhutQzTf61wTeiCviuKm6IXz+so9BcbySj5ZEnBFuQB hO7vC54GV+bui45c5LQQ5tjPqcvQkoOHixvf2jzR1sNvz58jt4cbk3YEJnk8qk5Y0F ioz207Xu1H+DyYQ8DhK3YihhrNTeB/NtmLWTQr54nTkNAtg5W9MjgduUAcgTwqMFKo nUsK3/H8fyDkRvFoMhjOm11BiC2GNjb36JbYxAU1wG9THEgTqnEedI/uWYytoX0Gqu 9hT38u8o0Dn4g== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 4142280400; Fri, 25 Apr 2025 13:39:29 -0400 (EDT) Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 3049F12039E; Fri, 25 Apr 2025 13:39:29 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls In-Reply-To: <867c38y9v3.fsf@gnu.org> Message-ID: References: <867c38y9v3.fsf@gnu.org> Date: Fri, 25 Apr 2025 13:39:28 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.203 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> diff --git a/src/coding.c b/src/coding.c >> index 63b0dbeb18b..b4209986918 100644 >> --- a/src/coding.c >> +++ b/src/coding.c >> @@ -7363,6 +7363,7 @@ decode_coding (struct coding_system *coding) >> struct ccl_spec cclspec; >> int carryover; >> int i; >> + specpdl_ref count = SPECPDL_INDEX (); >> >> USE_SAFE_ALLOCA; >> >> @@ -7389,6 +7390,8 @@ decode_coding (struct coding_system *coding) >> >> undo_list = BVAR (current_buffer, undo_list); >> bset_undo_list (current_buffer, Qt); >> + /* The caller should run *-change-functions over the whole region. */ >> + specbind (Qinhibit_modification_hooks, Qt); >> } > > How do we make sure this requirement is heeded to? The rest of the code in this function modifies the buffer without running any `before/after-change-functions` (except for things like charset properties, which is what I'm trying to suppress), so if a caller fails to run those hooks we'd know (and the above patch wouldn't make it worse). > And how do we document this incompatible change in NEWS? It's a bug fix, not an incompatible change. Stefan From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 14:39:57 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 18:39:57 +0000 Received: from localhost ([127.0.0.1]:52897 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8Nxt-0006FV-9a for submit@debbugs.gnu.org; Fri, 25 Apr 2025 14:39:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48732) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8Nxq-0006Es-1C for 78042@debbugs.gnu.org; Fri, 25 Apr 2025 14:39:54 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8Nxk-0006eD-Ja; Fri, 25 Apr 2025 14:39:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=E0Ua0ZyMaJHGSCZVPaYbBMI4Vj3oL9H5eCMVfwDSNMc=; b=g0Aha0x+O71u 4iId+BdKjL6gik4kPSFZW9K/vZDIoJmZjet5cf1pfv9tW+ucAfxqN53DlLjKRkbQSR2CqzPM4YQAo iEn2bm/i8ROmAaIPT6NCA1wf1e+3bHYzZ4JhxfFUmef/QFb0Y6ljWP9z6ytlxA2Far7t/QMw/pRuX 0kNx0iaG3j2B7fzJE8TClS5ghn7sgefaPl9O5p2qcis219YU69BbS+txYc+t5HpF5J0KQaWPFi4g0 PlQRyB28cU5HvcK5JgSuAp+OEqA9NZ3ou/vjEajT4WJVYsdTpP2uHj9K73CHXx+HylItFWYkOIiVj V0/noyq8Rd7lbd1ihVQ9FA==; Date: Fri, 25 Apr 2025 21:39:46 +0300 Message-Id: <86msc4w0el.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Fri, 25 Apr 2025 12:37:34 -0400) Subject: Re: bug#78042: 31.0.50; Improper sequence of `before/after-change-functions` calls References: <86plh0ydqi.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: 78042@debbugs.gnu.org > Date: Fri, 25 Apr 2025 12:37:34 -0400 > > > Sorry, I don't think I understand the problem. Is the problem that we > > have "nested" notifications, whereby you have > > > > before-change-functions A1 B1 > > before-change-functions A2 B2 > > after-change-functions A2 B2 > > after-change-functions A1 B1 > > > > IOW, _any_ situation where we have nested pairs of notifications is > > "breaking our promise"? > > Yup: the `before-change-functions A2 B2` promises that there won't be > any changes to the buffer outside of A2...B2 until the next > `before-change-functions`. > > > If so, how can that be avoided in principle in Emacs, given its > > recursive nature, except by some mechanism which collects all the > > notifications and rearranges them to "keep our promise"? > > There is currently no attempt in the C code to solve this "by > construction" or "by design". Instead we fix the offenders as we bump > into them. I submit that we will never be able to solve all of them. From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 14:56:55 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 18:56:55 +0000 Received: from localhost ([127.0.0.1]:53036 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8OEI-0007fO-U0 for submit@debbugs.gnu.org; Fri, 25 Apr 2025 14:56:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56340) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8OEG-0007el-Aa for 78042@debbugs.gnu.org; Fri, 25 Apr 2025 14:56:52 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8OEB-0000rl-0i; Fri, 25 Apr 2025 14:56:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=y5I74dhssiGwoCn0piwTSjfYM0ebJhHypKWml4FPajQ=; b=dKIkk/jecvAt NFPjGCXOAT/AIO/2uINRkaQQD9kmwKPWFcmncW2EkyUEVAA741hJPebt77cxMtCVmYwgHHcImKYnm Ueu6jQg7WWp7/ChVpJcwN/gvYxvYxQneBq7RFXBrdcMU2hs4JKIkMq2HVGy/ueYbsycVyuKh6a4hG soF3g8NGyhRj8/HvIYjmOK/pAST1zWQL2TwdtnuGV0eL1LzoX0wgsN0/tw9CEKZAnq/CznpPHNLIk bgABRweiwI2vOoMjt/6ZbN/126vktlU8ykQqcKdcGTTcD63JQpbgrsUpWTI0nmqg/qywrdROJK4/T G56vyBtzS2+EINSz6lXfZA==; Date: Fri, 25 Apr 2025 21:56:44 +0300 Message-Id: <86ecxgvzmb.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Fri, 25 Apr 2025 13:39:28 -0400) Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls References: <867c38y9v3.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: 78042@debbugs.gnu.org > Date: Fri, 25 Apr 2025 13:39:28 -0400 > > >> diff --git a/src/coding.c b/src/coding.c > >> index 63b0dbeb18b..b4209986918 100644 > >> --- a/src/coding.c > >> +++ b/src/coding.c > >> @@ -7363,6 +7363,7 @@ decode_coding (struct coding_system *coding) > >> struct ccl_spec cclspec; > >> int carryover; > >> int i; > >> + specpdl_ref count = SPECPDL_INDEX (); > >> > >> USE_SAFE_ALLOCA; > >> > >> @@ -7389,6 +7390,8 @@ decode_coding (struct coding_system *coding) > >> > >> undo_list = BVAR (current_buffer, undo_list); > >> bset_undo_list (current_buffer, Qt); > >> + /* The caller should run *-change-functions over the whole region. */ > >> + specbind (Qinhibit_modification_hooks, Qt); > >> } > > > > How do we make sure this requirement is heeded to? > > The rest of the code in this function modifies the buffer without > running any `before/after-change-functions` (except for things like > charset properties, which is what I'm trying to suppress), so if a caller > fails to run those hooks we'd know (and the above patch wouldn't make > it worse). I was asking how will we be able to make sure the caller does run the modification hooks. I don't quite understand what you mean by "we'd know". Know how and by what signs? > > And how do we document this incompatible change in NEWS? > > It's a bug fix, not an incompatible change. We now require callers to do something they didn't have to do before, or am I missing something? From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 15:12:28 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 19:12:28 +0000 Received: from localhost ([127.0.0.1]:53171 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8OTM-0000QE-Br for submit@debbugs.gnu.org; Fri, 25 Apr 2025 15:12:28 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:30440) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8OTJ-0000Pq-SJ for 78042@debbugs.gnu.org; Fri, 25 Apr 2025 15:12:26 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id F311144285D; Fri, 25 Apr 2025 15:12:19 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745608339; bh=BZe4DiSxTwZx4UuFKFE5zFvb3kVMdFGE4xiCLO0dEZA=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=de2payEAtRBZNcJOfpc6N4huc81f8l1YFQVz915O/a8SASbwwZIU8DNgKZTfky3u2 LNgVohyfh+LPXJz+pbisy97PVZTmyFMcWLYWQrAxQQ0XdnIi1XpS+RXDYih8Qvy910 GyQ2+lYd8bO0XJiHBtwMkb6WB0PLX4fzY4cHumaSBcYSgdGOiBbD5OsO4vvDg6ZfUm 7bUMHRSMonil7xOmM/AQ6SMWfjmT0yqWEy6gsnalKfbtBsURixZ9GliqbPh70mbkxT ZGCtbKQBPXaN6LZfoCVWtRxZrF1H1NmhZnqbg83J8vaZLWv4RM2Va1ksAcpiUuxSbL RGfs7OIiZbMLQ== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id EBC9F441B20; Fri, 25 Apr 2025 15:12:18 -0400 (EDT) Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id E0C3712041D; Fri, 25 Apr 2025 15:12:18 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#78042: 31.0.50; Improper sequence of `before/after-change-functions` calls In-Reply-To: <86msc4w0el.fsf@gnu.org> Message-ID: References: <86plh0ydqi.fsf@gnu.org> <86msc4w0el.fsf@gnu.org> Date: Fri, 25 Apr 2025 15:12:18 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.190 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> There is currently no attempt in the C code to solve this "by >> construction" or "by design". Instead we fix the offenders as we bump >> into them. > I submit that we will never be able to solve all of them. I'm pretty sure we can solve all the ones we find. But we may never find them all, indeed. My request for comments/objection was not about whether it is worth fixing this bug (there is no doubt about that on my side: it affects at least Eglot and CC-mode), but whether the approach I used is acceptable or if you have a better alternative to propose. Stefan From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 25 15:22:43 2025 Received: (at 78042) by debbugs.gnu.org; 25 Apr 2025 19:22:43 +0000 Received: from localhost ([127.0.0.1]:53238 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8OdG-00018T-ME for submit@debbugs.gnu.org; Fri, 25 Apr 2025 15:22:42 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:25087) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8OdE-00018C-77 for 78042@debbugs.gnu.org; Fri, 25 Apr 2025 15:22:40 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id BD1CE44285B; Fri, 25 Apr 2025 15:22:34 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745608953; bh=Gf+a12du+n1dzoTkH7zhp7y3V4vM9w+xc8iUcAASIaQ=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=oZUSkvsQdyQNtrwgBeKmYTP1mrvo1qDJrqAOXw3OUks+2nOO67ok/WaTwA+8m/0hn DCx245sQ9dhZgo6erbxFpW+sTS6YtXzRlyQCpL39PQ03Pci+vorumLZZn2JUN2skS2 xv8DIy/gpEQ+pVqbG1KJ+zl06h+ND7x0CokhYZfcyXgBlBEyY3ji3EVqY9kzW06nSE 5IYXN0LyZYlXjYkLk6r337zcCrzE7WB3YnTBzDNfMhultNEgJurT3WaApCZpfexHpv gUj2fPSqti7Lhk5vZZNtF66n6JaZms7vPm5T3Y/gs8VN0StsEP9yY1GEv0bnnz8dKo MFI4e/ikr957w== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id AE0A2442858; Fri, 25 Apr 2025 15:22:33 -0400 (EDT) Received: from lechazo (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id A17571200E4; Fri, 25 Apr 2025 15:22:33 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls In-Reply-To: <86ecxgvzmb.fsf@gnu.org> Message-ID: References: <867c38y9v3.fsf@gnu.org> <86ecxgvzmb.fsf@gnu.org> Date: Fri, 25 Apr 2025 15:22:32 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL 0.190 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> The rest of the code in this function modifies the buffer without >> running any `before/after-change-functions` (except for things like >> charset properties, which is what I'm trying to suppress), so if a caller >> fails to run those hooks we'd know (and the above patch wouldn't make >> it worse). > > I was asking how will we be able to make sure the caller does run the > modification hooks. The function where I made the modification inserts bytes into the buffer but without running the hooks. So it is *already* the case that the callers must run the hooks themselves. > I don't quite understand what you mean by "we'd know". Know how and by what signs? We would have noticed it because when those hooks are not run something *will* misbehave sooner or later, and for a common operation like `de/encode-coding-region`, it'll be sooner rather than later. I also know because I'm running my own Emacs with additional checks to detect when our change-functions break our promises, so that I can fix those bugs before they bite someone. >> > And how do we document this incompatible change in NEWS? >> It's a bug fix, not an incompatible change. > We now require callers to do something they didn't have to do before, > or am I missing something? Yes you're missing that the callers have always had to do that because in the usual case, there is no nested modifications (because the decode does not add any text-properties), and the function just inserts text into the buffer without running any change-functions. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 26 02:22:49 2025 Received: (at 78042) by debbugs.gnu.org; 26 Apr 2025 06:22:49 +0000 Received: from localhost ([127.0.0.1]:57099 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8Yw4-0006yU-UD for submit@debbugs.gnu.org; Sat, 26 Apr 2025 02:22:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41854) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8Yw0-0006xv-Oq for 78042@debbugs.gnu.org; Sat, 26 Apr 2025 02:22:45 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8Yvu-0006Qx-Ix; Sat, 26 Apr 2025 02:22:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=JCktL8oaPANIVHZlopdkF0VDeS5l9k6CWCjVw3jVb30=; b=nTslJpmsl9LH fIVWJM5ArGnLa8+tbJ9BYI5iOSw4KJFXUcY98vas/ngibl7iGdOngrtNLXLeD1fW/nQk4S7KB/wFy zSMtAv2PgIs2DioEFtgaw4ABQvKuvvi+jTu5Ny4oTOQliCB7ePic0rtAph5WVhpLtPYyMUd1wnf8v m9ra8bRIU35Unnpo55hurNGGBbpt3SmgTxEBWUk1/UUwf0wPdpEHfZPC7BGPlvwLeYLZNovCKsJAX xalPfjaZkewC2FQOx2foYpp5avNMyOQkIqLC/UIDxD3Ogm/1wpsWCHFWh3FZPETHjx+VsUdTi9WN1 SblOXVs2JxYZHj+GFJzUfg==; Date: Sat, 26 Apr 2025 09:22:32 +0300 Message-Id: <86bjsjwifr.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Fri, 25 Apr 2025 15:12:18 -0400) Subject: Re: bug#78042: 31.0.50; Improper sequence of `before/after-change-functions` calls References: <86plh0ydqi.fsf@gnu.org> <86msc4w0el.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: 78042@debbugs.gnu.org > Date: Fri, 25 Apr 2025 15:12:18 -0400 > > >> There is currently no attempt in the C code to solve this "by > >> construction" or "by design". Instead we fix the offenders as we bump > >> into them. > > I submit that we will never be able to solve all of them. > > I'm pretty sure we can solve all the ones we find. > But we may never find them all, indeed. > > My request for comments/objection was not about whether it is worth > fixing this bug (there is no doubt about that on my side: it affects at > least Eglot and CC-mode), but whether the approach I used is acceptable > or if you have a better alternative to propose. The only alternative I can think of is to be able to handle nested notifications, such as what happens in this case, as correct. What are the problems due to which we must have paired notifications without nesting? From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 26 02:43:21 2025 Received: (at 78042) by debbugs.gnu.org; 26 Apr 2025 06:43:21 +0000 Received: from localhost ([127.0.0.1]:57253 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8ZFx-0008Pp-3T for submit@debbugs.gnu.org; Sat, 26 Apr 2025 02:43:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34986) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8ZFu-0008PX-87 for 78042@debbugs.gnu.org; Sat, 26 Apr 2025 02:43:19 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8ZFo-0000Qb-22; Sat, 26 Apr 2025 02:43:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=j7AvdadheRdpHJIQdbGMs0/jhAe1gtOfbfNiVzFEgDM=; b=DjOvrTZpf9xM /WwfTpINNbFfFGa2DpJXyL4B1Yf76cOdWQOaoQqRGd8wEdQmOCYWa08kfIlF9uwEdtunvSFBxPcNu 8/oSPVKUglJao4DRNAEzF4Owr3egJGvdTXLQA2Wvxjvb2oVdi/nDUqh6+41ROQgahKIocOkP4uKww o6RBifvSGsr+nG81keovLzt/Myp2JkH8IE/ogcRB7tYFLk3spDGgN5Vn071nEbShPMsf6siTAXtqm lXecTccvAqAm+Oc/ZNV5ixw8lZtVgCRUMJUo5iUHlJrERzCiOgr2kdKQKdUw1yxk6x81+i+hkBjTM 5XLqq8vy05l17hAahaYBBA==; Date: Sat, 26 Apr 2025 09:43:06 +0300 Message-Id: <86a583whhh.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Fri, 25 Apr 2025 15:22:32 -0400) Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls References: <867c38y9v3.fsf@gnu.org> <86ecxgvzmb.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: 78042@debbugs.gnu.org > Date: Fri, 25 Apr 2025 15:22:32 -0400 > > >> The rest of the code in this function modifies the buffer without > >> running any `before/after-change-functions` (except for things like > >> charset properties, which is what I'm trying to suppress), so if a caller > >> fails to run those hooks we'd know (and the above patch wouldn't make > >> it worse). > > > > I was asking how will we be able to make sure the caller does run the > > modification hooks. > > The function where I made the modification inserts bytes into > the buffer but without running the hooks. > So it is *already* the case that the callers must run the hooks > themselves. I don't understand: if you made the changes (by removing the calls to modification hooks), then the function originally _was_ calling them, no? Anyway, at the time I audited the functions in coding.c and verified that the hooks were always called by some function in the code path. If we are changing that now, we need to repeat such auditing. > > I don't quite understand what you mean by "we'd know". Know how and by what signs? > > We would have noticed it because when those hooks are not run something > *will* misbehave sooner or later, and for a common operation like > `de/encode-coding-region`, it'll be sooner rather than later. > > I also know because I'm running my own Emacs with additional checks to > detect when our change-functions break our promises, so that I can fix > those bugs before they bite someone. The problem with such ad-hoc evidence is that it relies on the assumption that your code and the "usual" cases everyone runs do exercise all of the relevant code paths. Such assumptions are not very reliable in Emacs, IME. > >> > And how do we document this incompatible change in NEWS? > >> It's a bug fix, not an incompatible change. > > We now require callers to do something they didn't have to do before, > > or am I missing something? > > Yes you're missing that the callers have always had to do that because > in the usual case, there is no nested modifications (because the decode > does not add any text-properties), and the function just inserts text > into the buffer without running any change-functions. But the "usual-case" notification is about the entire decoded text, whereas the notifications your patch blocks are about smaller chunks of text for specific kinds of changes, and thus more fine-grained. So the hooks will need to do a more complex job now: they cannot rely on the fact that notifications for decoding are separate from notifications about text-property changes, so they will need to examine the entire decoded region of text and decide what to do with each chunk of it. So the existing hook functions might fail to work correctly after this change, and we must therefore call out this change in NEWS. Or maybe we should not install this at all? What are the problems these "nested" notifications cause, again? From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 26 12:35:57 2025 Received: (at 78042) by debbugs.gnu.org; 26 Apr 2025 16:35:57 +0000 Received: from localhost ([127.0.0.1]:34432 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8iVQ-000212-So for submit@debbugs.gnu.org; Sat, 26 Apr 2025 12:35:57 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:37142) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8iVO-00020Z-4K for 78042@debbugs.gnu.org; Sat, 26 Apr 2025 12:35:54 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id B91491000BC; Sat, 26 Apr 2025 12:35:48 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745685343; bh=j3IbCF8qFEmO4MywFMFGxY5zRisyVMWJo7PTgkC6rV4=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=EISSfosrp6O/VuU65eL+WDyMww55DZvHlMT1DZ7xt2brao9COSF5GbEGlx3hv4Ri7 ohhFqpkcKRVEKvptKYnmK8PFhRM51FFJpdzW3j1nuOBYvLxMcQeqrEfi3YmsDUeS/h uUuQ03XGiHt7w3jGqTCd8zgfhd5nJ89fmm4mZ75qYX2Xi2qMFKFibPDhtEuwTETFxx aH0MdUevPhOLNdsmSUneJ7upGzwz9b3sN0d/KHXOBBxA6yMuIYRanDuuAgEtEg0Fqd jt1edklDRZlbGpGMWJfTFQDIxfWGufUvieh5jnLst3qmVq+V6S9S8gANku70mYrUY5 yTGm15bgmCfrA== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id DFBC410002E; Sat, 26 Apr 2025 12:35:43 -0400 (EDT) Received: from alfajor (104-195-232-56.cpe.teksavvy.com [104.195.232.56]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B51611202C7; Sat, 26 Apr 2025 12:35:43 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#78042: 31.0.50; Improper sequence of `before/after-change-functions` calls In-Reply-To: <86bjsjwifr.fsf@gnu.org> Message-ID: References: <86plh0ydqi.fsf@gnu.org> <86msc4w0el.fsf@gnu.org> <86bjsjwifr.fsf@gnu.org> Date: Sat, 26 Apr 2025 12:35:43 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.216 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> My request for comments/objection was not about whether it is worth >> fixing this bug (there is no doubt about that on my side: it affects at >> least Eglot and CC-mode), but whether the approach I used is acceptable >> or if you have a better alternative to propose. > The only alternative I can think of is to be able to handle nested > notifications, such as what happens in this case, as correct. What > are the problems due to which we must have paired notifications > without nesting? The problem is that it would change the semantics of `after/before-change-functions`. E.g. code that needs to match the before with the after (e.g. CC-mode, Eglot, ...) would now need to be adjusted to keep a stack of "pending befores" and the C code would need to be reviewed to try and catch the few places where we currently run `after-change-functions` several times after running `before-change-functions` once. IOW, we'd trade one set of corner-case breakage for another. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 26 13:27:12 2025 Received: (at 78042) by debbugs.gnu.org; 26 Apr 2025 17:27:12 +0000 Received: from localhost ([127.0.0.1]:34634 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8jJ2-00052i-50 for submit@debbugs.gnu.org; Sat, 26 Apr 2025 13:27:12 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:30597) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8jIz-00052S-6q for 78042@debbugs.gnu.org; Sat, 26 Apr 2025 13:27:10 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 1253B4409A7; Sat, 26 Apr 2025 13:27:03 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745688421; bh=1jKBxwfcoyBtDD/afoJfjPU7dlK8l5kuYASQNAYrsU0=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=Mn/2mOVjvGU5GSGgD4CIIpF0dt+MDZBvldChDB2NSNAUdOKCz9ImWPVCYrMGYSBx/ eEY+2lGNjumAJwckd8pPv+KhYM7WZLlQbIYsXWfPr52eburcbV73zR/y0yvxlX4J3T 39g80cJ6ZY7G9ZRZkURGYf+fDK71T7x87yG//Yw/oTUNXwJGmqfKYYFKc2rJ6KgNwu ksH4yLqIaXne+ADe4fkHoKanb8ZkyQqn7SyI7vVnTIWnzoAE48kL+cEqcwMCRyCHtX ZWVc5XiNzxtVckHt7W2BD3KsMmaDkY8PT0cLRMSjlWY897kjFLzNmYSV/QVEC/Ckiy cut0q1YYz26+A== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id AEC66440A06; Sat, 26 Apr 2025 13:27:01 -0400 (EDT) Received: from alfajor (104-195-232-56.cpe.teksavvy.com [104.195.232.56]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 7F624120262; Sat, 26 Apr 2025 13:27:01 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls In-Reply-To: <86a583whhh.fsf@gnu.org> Message-ID: References: <867c38y9v3.fsf@gnu.org> <86ecxgvzmb.fsf@gnu.org> <86a583whhh.fsf@gnu.org> Date: Sat, 26 Apr 2025 13:27:00 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.018 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> The function where I made the modification inserts bytes into >> the buffer but without running the hooks. >> So it is *already* the case that the callers must run the hooks >> themselves. > > I don't understand: if you made the changes (by removing the calls to > modification hooks), then the function originally _was_ calling them, > no? Here's what currently happens: - the caller calls `before-change-functions`. - it calls `decode_coding`: - in all cases, if dst_object is a buffer, `decode_coding` inserts bytes into that buffer. - in no case does `decode_coding` take charge to run the `before/after-change-functions` needed for those insertions. - in some relatively rare cases, `decode_coding` adds some text-properties causing nested `before/after-change-functions` calls. This is always done on the just-inserted text, so it's *always* nested (aka redundant, and harmful). - the caller then calls `after-change-functions`. The patch I sent binds `Qinhibit_modification_hooks` to t during the execution of `decode_coding`, so it affects only nested calls to `before/after-change-functions` (which apply only to some cases and only to some parts of the inserted text), not the "normal" calls which are always performed outside of that function. > The problem with such ad-hoc evidence is that it relies on the > assumption that your code and the "usual" cases everyone runs do > exercise all of the relevant code paths. Such assumptions are not > very reliable in Emacs, IME. No, the evidence is simply that `decode_coding` does not itself do anything to run the change functions to reflect the text insertion. So the inhibition that my patch does can affect only those redundant nested runs. > But the "usual-case" notification is about the entire decoded text, > whereas the notifications your patch blocks are about smaller chunks > of text for specific kinds of changes, and thus more fine-grained. So > the hooks will need to do a more complex job now: they cannot rely on > the fact that notifications for decoding are separate from > notifications about text-property changes, so they will need to > examine the entire decoded region of text and decide what to do with > each chunk of it. No, any function placed on those hooks has to deal with a wild array of other cases and simply can never know whether they're called for text-property changes or anything else. Instead, (the coders of) those functions always have to expend a great effort to try and make as few assumptions as possible because those assumptions *always* end up being broken in one case or another. > Or maybe we should not install this at all? What are the problems > these "nested" notifications cause, again? I've seen a backtrace in CC-mode (tho I haven't yet been able to figure out how to reproduce it: it depends on the state of the CC-mode cache), it causes Eglot to throw its hands up and say "how well, Emacs messed up, let's start over", etc... I guess next time I'll just push my patch silently to avoid having to argue why we should fix our bugs, Stefan From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 26 14:41:31 2025 Received: (at 78042) by debbugs.gnu.org; 26 Apr 2025 18:41:31 +0000 Received: from localhost ([127.0.0.1]:34946 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8kSx-0000sA-1k for submit@debbugs.gnu.org; Sat, 26 Apr 2025 14:41:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56834) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8kSu-0000rl-Ts for 78042@debbugs.gnu.org; Sat, 26 Apr 2025 14:41:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8kSa-0004jX-NJ; Sat, 26 Apr 2025 14:41:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=rkiwaTL9XrsOq4Pmmb5D6+htu1mIgSVtgq/wAMffJEU=; b=ChlpTqPBsqFM ml8mlwPmHUDBhJwcQ4mrNMNwigYognArE22fO5G96ovEfYGLF2dV8iUfzZPRDID4F7tx7C31kVRg1 RsPb1j0+FFoSncixuC/E/RLNqV+s5hVma3wzhgtEUuI7umqGANILvrMz8JflOKr8QlpRJuJ9H9QtX s+0030N5M0Vsf360DuRux1c/aLFaRVsB7TSZ4eWBM/vZIVHiNfwYK+RGBU5CX7oCSJW8DME6ISrve qXgl1DpxtPXYUQGWlzsSu42d6yYjU+ZGjhFuO1ahU7wsjSsBzdfqxGF7PD8QJSnm8JPvrh8aTbHMY kfLH3c3liibrNQ7e2fUImQ==; Date: Sat, 26 Apr 2025 21:41:04 +0300 Message-Id: <86v7qqsr3z.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Sat, 26 Apr 2025 12:35:43 -0400) Subject: Re: bug#78042: 31.0.50; Improper sequence of `before/after-change-functions` calls References: <86plh0ydqi.fsf@gnu.org> <86msc4w0el.fsf@gnu.org> <86bjsjwifr.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: 78042@debbugs.gnu.org > Date: Sat, 26 Apr 2025 12:35:43 -0400 > > >> My request for comments/objection was not about whether it is worth > >> fixing this bug (there is no doubt about that on my side: it affects at > >> least Eglot and CC-mode), but whether the approach I used is acceptable > >> or if you have a better alternative to propose. > > The only alternative I can think of is to be able to handle nested > > notifications, such as what happens in this case, as correct. What > > are the problems due to which we must have paired notifications > > without nesting? > > The problem is that it would change the semantics of > `after/before-change-functions`. E.g. code that needs to match the > before with the after (e.g. CC-mode, Eglot, ...) would now need to be > adjusted to keep a stack of "pending befores" and the C code would need > to be reviewed to try and catch the few places where we currently run > `after-change-functions` several times after running > `before-change-functions` once. > > IOW, we'd trade one set of corner-case breakage for another. Except that making us prepared to handle nested notifications will solve the problem for good. IOW, fixing _that_ breakage has hope to fix the problem completely, whereas fixing each case we find as we go is just an endless rear-guard battle, and those can never be won. From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 26 14:43:52 2025 Received: (at 78042) by debbugs.gnu.org; 26 Apr 2025 18:43:53 +0000 Received: from localhost ([127.0.0.1]:34951 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8kVE-0000wR-Hw for submit@debbugs.gnu.org; Sat, 26 Apr 2025 14:43:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52218) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8kVB-0000wA-V3 for 78042@debbugs.gnu.org; Sat, 26 Apr 2025 14:43:50 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u8kV6-0005A8-Cp; Sat, 26 Apr 2025 14:43:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=u83s1gwNcF9o1KHC/XXRAbJdA36vWsFjeYd8PAmSUGI=; b=LcXniV4HTsjT OFLas85xIUhZC+P0upTT0Bim+Unnrv0HOKotJDDAjxApD5l8TYlcjW6KnWZOszlLfPflxKEEYpYo6 diYjHX22TBvsRozSSdLxbc2ndp9lmBkyiOclUeRAWwVLBP6IQJVnneIUnvFILw5Q0fuUG1Bx1H8nG 4/fiaWGlOnKAW/jJMAX3khKmKZvfl96T4pr8j4qgBHCJ7PGHcVX0KDkr3ecslkKnQkRzF8MiYPKKS 4UUTaWrwzXCbRwIpaKX+mbFhwlqfZVpFhVPhBiDHGWZjim72gt2HvfjTSt6uBnn6SiOk0wlyKCQkx Mi/lwdwjAJwe7Ttgj4pvmQ==; Date: Sat, 26 Apr 2025 21:43:39 +0300 Message-Id: <86tt6asqzo.fsf@gnu.org> From: Eli Zaretskii To: Stefan Monnier In-Reply-To: (message from Stefan Monnier on Sat, 26 Apr 2025 13:27:00 -0400) Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls References: <867c38y9v3.fsf@gnu.org> <86ecxgvzmb.fsf@gnu.org> <86a583whhh.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Stefan Monnier > Cc: 78042@debbugs.gnu.org > Date: Sat, 26 Apr 2025 13:27:00 -0400 > > I guess next time I'll just push my patch silently to avoid having to > argue why we should fix our bugs, That's not fair on so many levels that I don't even know where to start. But whatever, have it your way. From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 26 19:22:52 2025 Received: (at 78042) by debbugs.gnu.org; 26 Apr 2025 23:22:52 +0000 Received: from localhost ([127.0.0.1]:36172 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1u8orE-0003Ic-A9 for submit@debbugs.gnu.org; Sat, 26 Apr 2025 19:22:52 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:46541) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1u8orB-0003IE-Li for 78042@debbugs.gnu.org; Sat, 26 Apr 2025 19:22:50 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 49A4A1000BC; Sat, 26 Apr 2025 19:22:44 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1745709763; bh=k9Z62ewHHeqQ8uTZmtnAclRK5eMP7mjj+OeRkC+Weio=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=SzpUaHUH0VY0LJ6BpUqWl8NjTeaGN5nLXLW0zyg5+rP9yDuzJ9DCJZIAt4WjULh8p 2OOsBu+LcPowS2kS8mQY2wH+s9jZnwV0umjB1CrhUnNGhWVhx/1v+PF8JYJdtgWnaO Up5tASNgqV6hOvRur7g7jsmzfXzUbH5g3GTmwEOCwrQczYRoD1EeuqY5S3ZoK9rGx/ C7HK82n7fJUnjtpZaLgApvum0vq4tu6z/PiGSRDlY+pVe1vkbYheFgpQUemp/dZxvC 3JiNWSLrwU3IVMdzOF4r9mxRXvOEyJZaKNtHanWbW9QoZbiL7WaGMDLJaZVrhBt49Z 3RBBNNS+P+l4w== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 9912310002E; Sat, 26 Apr 2025 19:22:43 -0400 (EDT) Received: from alfajor (104-195-232-56.cpe.teksavvy.com [104.195.232.56]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 674D0120627; Sat, 26 Apr 2025 19:22:43 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#78042: 31.0.50; Improper sequence of `before/after-change-functions` calls In-Reply-To: <86v7qqsr3z.fsf@gnu.org> Message-ID: References: <86plh0ydqi.fsf@gnu.org> <86msc4w0el.fsf@gnu.org> <86bjsjwifr.fsf@gnu.org> <86v7qqsr3z.fsf@gnu.org> Date: Sat, 26 Apr 2025 19:22:42 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.215 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042 Cc: 78042@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) >> The problem is that it would change the semantics of >> `after/before-change-functions`. E.g. code that needs to match the >> before with the after (e.g. CC-mode, Eglot, ...) would now need to be >> adjusted to keep a stack of "pending befores" and the C code would need >> to be reviewed to try and catch the few places where we currently run >> `after-change-functions` several times after running >> `before-change-functions` once. >> >> IOW, we'd trade one set of corner-case breakage for another. > > Except that making us prepared to handle nested notifications will > solve the problem for good. I don't see why it'll be easier to find all the cases where we currently call `after-change-functions` several times. But maybe you're right. It'll still be a breaking change, so it might require introducing new hooks to replace `after/before-change-functions`. In the mean time, I'll keep fixing the problems I bump into. Stefan From debbugs-submit-bounces@debbugs.gnu.org Fri May 02 17:07:43 2025 Received: (at 78042-done) by debbugs.gnu.org; 2 May 2025 21:07:43 +0000 Received: from localhost ([127.0.0.1]:33974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uAxbi-0002vv-Kw for submit@debbugs.gnu.org; Fri, 02 May 2025 17:07:42 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:10094) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uAxbb-0002vV-PW for 78042-done@debbugs.gnu.org; Fri, 02 May 2025 17:07:40 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id BA9AB10006B; Fri, 2 May 2025 17:07:27 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1746220047; bh=mhodWLBDVLgq09QOtxjlM98N8n6k7reH8E9VgWfzNSc=; h=From:To:Subject:In-Reply-To:References:Date:From; b=IqPVTY/CqR1iftj9GM1K7MdMxYmFWuey1OPa0G4Dl4W/fjwA9gx69ui4DwwshaCBU 6aPRBjWos0sFE1TpQezFDCv27Zups0EZmHfrzFCuDXXbJI2U/AGxkqOvIZYWG1fiLC jB+j7NpObdJmIbP34TDDjD95alpgKZifptIYnmVdSoN+7lhY0GBit6eigOLCVPyEgG u9GNZbY4CAWXFAZJe4azd9kJpTC5AySsBJLJ0ETupaprExWe708ubPqoRPVFERZbcr 5puXAdUYwNkc9t7Ij3DkGWrCm7cBS1g6Ei8HU/zd8rWU/8HKGAoJDqAXxtrMZk5/ns eqFPTWWuyuDXQ== Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id EA69510002E; Fri, 2 May 2025 17:07:26 -0400 (EDT) Received: from pastel (104-195-232-56.cpe.teksavvy.com [104.195.232.56]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id C5F33120099; Fri, 2 May 2025 17:07:26 -0400 (EDT) From: Stefan Monnier To: 78042-done@debbugs.gnu.org Subject: Re: bug#78042: Improper sequence of `before/after-change-functions` calls In-Reply-To: Message-ID: References: Date: Fri, 02 May 2025 17:07:26 -0400 User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP AWL -0.227 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain X-SPAM-LEVEL: X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78042-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > The code that runs those hooks is really littered at odd places, with no > good/standard way to control when to run it and when not. > I suggest the patch below (which includes a corresponding test). > Any objection? Pushed, closing, Stefan From unknown Thu Jun 19 14:04:38 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 31 May 2025 11:24:14 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator