From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 08 Jun 2019 13:18:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 36136@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.155999987226722 (code B ref -1); Sat, 08 Jun 2019 13:18:01 +0000 Received: (at submit) by debbugs.gnu.org; 8 Jun 2019 13:17:52 +0000 Received: from localhost ([127.0.0.1]:53016 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hZbEF-0006wv-NB for submit@debbugs.gnu.org; Sat, 08 Jun 2019 09:17:51 -0400 Received: from lists.gnu.org ([209.51.188.17]:45304) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hZbEE-0006wo-1F for submit@debbugs.gnu.org; Sat, 08 Jun 2019 09:17:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53934) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hZbEB-0006WW-Op for bug-gnu-emacs@gnu.org; Sat, 08 Jun 2019 09:17:49 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hZbE9-00033Q-QZ for bug-gnu-emacs@gnu.org; Sat, 08 Jun 2019 09:17:47 -0400 Received: from colin.muc.de ([193.149.48.1]:51289 helo=mail.muc.de) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1hZbE6-0002un-0n for bug-gnu-emacs@gnu.org; Sat, 08 Jun 2019 09:17:43 -0400 Received: (qmail 62602 invoked by uid 3782); 8 Jun 2019 13:17:25 -0000 Received: from acm.muc.de (p2E5D5C7B.dip0.t-ipconnect.de [46.93.92.123]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 08 Jun 2019 15:17:24 +0200 Received: (qmail 5207 invoked by uid 1000); 8 Jun 2019 13:17:24 -0000 Date: Sat, 8 Jun 2019 13:17:24 +0000 Message-ID: <20190608131724.GA4643@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-Received-From: 193.149.48.1 X-Spam-Score: -1.6 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.6 (--) Hello, Emacs. The syntax-ppss cache is not invalidated when syntax-table text properties are set or cleared. This is because the invalidation function, syntax-ppss-flush-cache is invoked only as a before-change function, but typical (?all) syntax-table property changes happen when before-change-functions is inactive. This is a bug. In my debugging of a CC Mode scenario, a buffer change causes a syntax-table text property change at an earlier part of the buffer. This is to do with the change making previously non-matching C++ raw string identifiers match up. Font lock follows the syntax-ppss state, which spuriously records that the end part of the buffer is still in a string. Hence the non-string part of the buffer is still fontified with font-lock-string-face. Suggested fix: the functions set_properties, add_properties, remove_properties in textprop.c should check for changes to, specifically, syntax-table properties. When these changes are detected, a hook called something like syntax-table-props-change-alert-hook should be called (with some appropriate position parameters, tbd). syntax-ppss-flush-cache will be added to this hook. -- Alan Mackenzie (Nuremberg, Germany). From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 08 Jun 2019 20:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 36136@debbugs.gnu.org Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.156002620612084 (code B ref 36136); Sat, 08 Jun 2019 20:37:02 +0000 Received: (at 36136) by debbugs.gnu.org; 8 Jun 2019 20:36:46 +0000 Received: from localhost ([127.0.0.1]:53779 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hZi50-00038q-0L for submit@debbugs.gnu.org; Sat, 08 Jun 2019 16:36:46 -0400 Received: from colin.muc.de ([193.149.48.1]:35422 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1hZi4x-00038h-Te for 36136@debbugs.gnu.org; Sat, 08 Jun 2019 16:36:45 -0400 Received: (qmail 81245 invoked by uid 3782); 8 Jun 2019 20:36:40 -0000 Received: from acm.muc.de (p2E5D5C7B.dip0.t-ipconnect.de [46.93.92.123]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 08 Jun 2019 22:36:40 +0200 Received: (qmail 17829 invoked by uid 1000); 8 Jun 2019 20:36:39 -0000 Date: Sat, 8 Jun 2019 20:36:39 +0000 Message-ID: <20190608203639.GA28722@ACM> References: <20190608131724.GA4643@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190608131724.GA4643@ACM> User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Emacs. On Sat, Jun 08, 2019 at 13:17:24 +0000, Alan Mackenzie wrote: > The syntax-ppss cache is not invalidated when syntax-table text > properties are set or cleared. This is because the invalidation > function, syntax-ppss-flush-cache is invoked only as a before-change > function, but typical (?all) syntax-table property changes happen when > before-change-functions is inactive. > This is a bug. > In my debugging of a CC Mode scenario, a buffer change causes a > syntax-table text property change at an earlier part of the buffer. This > is to do with the change making previously non-matching C++ raw string > identifiers match up. Font lock follows the syntax-ppss state, which > spuriously records that the end part of the buffer is still in a string. > Hence the non-string part of the buffer is still fontified with > font-lock-string-face. > Suggested fix: the functions set_properties, add_properties, > remove_properties in textprop.c should check for changes to, > specifically, syntax-table properties. When these changes are detected, > a hook called something like syntax-table-props-change-alert-hook should > be called (with some appropriate position parameters, tbd). > syntax-ppss-flush-cache will be added to this hook. Here is a first draught of a fix to this bug. It creates a new abnormal hook, syntax-table-props-change-alert-functions, whose member functions are called each time a syntax-table text property is changed. syntax-ppss-flush-cache is inserted into this new hook at loading time for syntax.el, thus invalidating the syntax-ppss cache each time a syntax-table text property is changed. There is a consequential amendment to the doc string of inhibit-modification-hooks. This fixes the original problem. Comments? diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el index 9c6d5b5829..673d598de6 100644 --- a/lisp/emacs-lisp/syntax.el +++ b/lisp/emacs-lisp/syntax.el @@ -43,6 +43,10 @@ (eval-when-compile (require 'cl-lib)) +;; Ensure that the cache gets invalidated when `syntax-table' text +;; properties get set, even when `inhibit-modification-hooks' is non-nil. +(add-hook 'syntax-table-props-change-alert-functions #'syntax-ppss-flush-cache) + ;;; Applying syntax-table properties where needed. (defvar syntax-propertize-function nil diff --git a/src/insdel.c b/src/insdel.c index 85fffd8fd1..5e978d1b29 100644 --- a/src/insdel.c +++ b/src/insdel.c @@ -2360,9 +2360,11 @@ syms_of_insdel (void) Vcombine_after_change_calls = Qnil; DEFVAR_BOOL ("inhibit-modification-hooks", inhibit_modification_hooks, - doc: /* Non-nil means don't run any of the hooks that respond to buffer changes. + doc: /* Non-nil means don't run most of the hooks that respond to buffer changes. This affects `before-change-functions' and `after-change-functions', -as well as hooks attached to text properties and overlays. +as well as most hooks attached to text properties and overlays. +However the hook `syntax-table-props-change-alert-functions' gets run +regardless of the setting of this variable. Setting this variable non-nil also inhibits file locks and checks whether files are locked by another Emacs session, as well as handling of the active region per `select-active-regions'. */); diff --git a/src/textprop.c b/src/textprop.c index ae42c44185..050bb7b435 100644 --- a/src/textprop.c +++ b/src/textprop.c @@ -334,7 +334,10 @@ set_properties (Lisp_Object properties, INTERVAL interval, Lisp_Object object) record_property_change (interval->position, LENGTH (interval), XCAR (sym), XCAR (value), object); - } + if (EQ (sym, Qsyntax_table)) + CALLN (Frun_hook_with_args, Qsyntax_table_props_change_alert_functions, + make_fixnum (interval->position)); + } /* For each new property that has no value at all in the old plist, make an undo record binding it to nil, so it will be removed. */ @@ -346,6 +349,9 @@ set_properties (Lisp_Object properties, INTERVAL interval, Lisp_Object object) record_property_change (interval->position, LENGTH (interval), XCAR (sym), Qnil, object); + if (EQ (sym, Qsyntax_table)) + CALLN (Frun_hook_with_args, Qsyntax_table_props_change_alert_functions, + make_fixnum (interval->position)); } } @@ -400,7 +406,10 @@ add_properties (Lisp_Object plist, INTERVAL i, Lisp_Object object, { record_property_change (i->position, LENGTH (i), sym1, Fcar (this_cdr), object); - } + if (EQ (sym1, Qsyntax_table)) + CALLN (Frun_hook_with_args, Qsyntax_table_props_change_alert_functions, + make_fixnum (i->position)); + } /* I's property has a different value -- change it */ if (set_type == TEXT_PROPERTY_REPLACE) @@ -436,6 +445,9 @@ add_properties (Lisp_Object plist, INTERVAL i, Lisp_Object object, { record_property_change (i->position, LENGTH (i), sym1, Qnil, object); + if (EQ (sym1, Qsyntax_table)) + CALLN (Frun_hook_with_args, Qsyntax_table_props_change_alert_functions, + make_fixnum (i->position)); } set_interval_plist (i, Fcons (sym1, Fcons (val1, i->plist))); changed = true; @@ -470,9 +482,14 @@ remove_properties (Lisp_Object plist, Lisp_Object list, INTERVAL i, Lisp_Object while (CONSP (current_plist) && EQ (sym, XCAR (current_plist))) { if (BUFFERP (object)) - record_property_change (i->position, LENGTH (i), - sym, XCAR (XCDR (current_plist)), - object); + { + record_property_change (i->position, LENGTH (i), + sym, XCAR (XCDR (current_plist)), + object); + if (EQ (sym, Qsyntax_table)) + CALLN (Frun_hook_with_args, Qsyntax_table_props_change_alert_functions, + make_fixnum (i->position)); + } current_plist = XCDR (XCDR (current_plist)); changed = true; @@ -486,8 +503,13 @@ remove_properties (Lisp_Object plist, Lisp_Object list, INTERVAL i, Lisp_Object if (CONSP (this) && EQ (sym, XCAR (this))) { if (BUFFERP (object)) - record_property_change (i->position, LENGTH (i), - sym, XCAR (XCDR (this)), object); + { + record_property_change (i->position, LENGTH (i), + sym, XCAR (XCDR (this)), object); + if (EQ (sym, Qsyntax_table)) + CALLN (Frun_hook_with_args, Qsyntax_table_props_change_alert_functions, + make_fixnum (i->position)); + } Fsetcdr (XCDR (tail2), XCDR (XCDR (this))); changed = true; @@ -2319,6 +2341,20 @@ inherits it if NONSTICKINESS is nil. The `front-sticky' and Vtext_property_default_nonsticky = list2 (Fcons (Qsyntax_table, Qt), Fcons (Qdisplay, Qt)); + DEFVAR_LISP ("syntax-table-props-change-alert-functions", + Vsyntax_table_props_change_alert_functions, + doc: /* List of functions to call each time a `syntax-table' text property +gets added, removed, or changed. + +Each function is passed a single parameter, the buffer position of the +start of the property change. The current buffer is the one the +change has been made in. These functions get called even when +`inhibit-modification-hooks' is non-nil. + +Note that these functions may NOT themselves make any buffer changes, +including text property changes. */); + Vsyntax_table_props_change_alert_functions = Qnil; + interval_insert_behind_hooks = Qnil; interval_insert_in_front_hooks = Qnil; staticpro (&interval_insert_behind_hooks); @@ -2337,6 +2373,7 @@ inherits it if NONSTICKINESS is nil. The `front-sticky' and DEFSYM (Qrear_nonsticky, "rear-nonsticky"); DEFSYM (Qmouse_face, "mouse-face"); DEFSYM (Qminibuffer_prompt, "minibuffer-prompt"); + DEFSYM (Qsyntax_table_props_change_alert_functions, "syntax-table-props-change-alert-functions"); /* Properties that text might use to specify certain actions. */ -- Alan Mackenzie (Nuremberg, Germany). From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 09 Jun 2019 05:57:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Alan Mackenzie Cc: 36136@debbugs.gnu.org Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.15600598219029 (code B ref 36136); Sun, 09 Jun 2019 05:57:01 +0000 Received: (at 36136) by debbugs.gnu.org; 9 Jun 2019 05:57:01 +0000 Received: from localhost ([127.0.0.1]:54094 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hZqpB-0002LV-0C for submit@debbugs.gnu.org; Sun, 09 Jun 2019 01:57:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52733) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hZqp8-0002LI-KK for 36136@debbugs.gnu.org; Sun, 09 Jun 2019 01:57:00 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55113) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hZqp3-0001IJ-1t; Sun, 09 Jun 2019 01:56:53 -0400 Received: from [176.228.60.248] (port=2466 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hZqp2-0000Wu-Ea; Sun, 09 Jun 2019 01:56:52 -0400 Date: Sun, 09 Jun 2019 08:56:45 +0300 Message-Id: <83ef43cndu.fsf@gnu.org> From: Eli Zaretskii In-reply-to: <20190608203639.GA28722@ACM> (message from Alan Mackenzie on Sat, 8 Jun 2019 20:36:39 +0000) References: <20190608131724.GA4643@ACM> <20190608203639.GA28722@ACM> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Sat, 8 Jun 2019 20:36:39 +0000 > From: Alan Mackenzie > > > Suggested fix: the functions set_properties, add_properties, > > remove_properties in textprop.c should check for changes to, > > specifically, syntax-table properties. When these changes are detected, > > a hook called something like syntax-table-props-change-alert-hook should > > be called (with some appropriate position parameters, tbd). > > syntax-ppss-flush-cache will be added to this hook. > > Here is a first draught of a fix to this bug. I have no opinion about the issue and the idea of its proposed solution, but I do have some comments to the implementation. > diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el > index 9c6d5b5829..673d598de6 100644 > --- a/lisp/emacs-lisp/syntax.el > +++ b/lisp/emacs-lisp/syntax.el > @@ -43,6 +43,10 @@ > > (eval-when-compile (require 'cl-lib)) > > +;; Ensure that the cache gets invalidated when `syntax-table' text > +;; properties get set, even when `inhibit-modification-hooks' is non-nil. > +(add-hook 'syntax-table-props-change-alert-functions #'syntax-ppss-flush-cache) This means that whenever syntax.el is loaded (i.e., always, since syntax.el is preloaded), this hook will be non-nil. Is that a good idea? I mean, if we always call this function, why do this via a hook? > DEFVAR_BOOL ("inhibit-modification-hooks", inhibit_modification_hooks, > - doc: /* Non-nil means don't run any of the hooks that respond to buffer changes. > + doc: /* Non-nil means don't run most of the hooks that respond to buffer changes. > This affects `before-change-functions' and `after-change-functions', > -as well as hooks attached to text properties and overlays. > +as well as most hooks attached to text properties and overlays. > +However the hook `syntax-table-props-change-alert-functions' gets run > +regardless of the setting of this variable. If you go to such lengths, please be sure to mention that the new hook will NOT run if run-hooks is nil. > + if (EQ (sym, Qsyntax_table)) > + CALLN (Frun_hook_with_args, Qsyntax_table_props_change_alert_functions, > + make_fixnum (interval->position)); These calls make set/add/remove_properties slower. AFAIK, those functions are hotspots in Emacs, so please make these calls as efficient as possible. In particular, I would call run_hook_with_args directly, bypassing an extra Frun_hook_with_args function call. Better still, if we abandon the idea of doing this via a hook, calling what is now a hook function directly will be more efficient still. Also, what about the safety of this call? what if the hook signals an error or the user presses C-g while the hook runs? IOW, should you use safe_call or some of its variants, and should you inhibit QUIT? and if so, whether and how should you handle in the code the case when the hook does signal an error? > + DEFVAR_LISP ("syntax-table-props-change-alert-functions", > + Vsyntax_table_props_change_alert_functions, > + doc: /* List of functions to call each time a `syntax-table' text property > +gets added, removed, or changed. The first line of a doc string should be a complete sentence. > +Each function is passed a single parameter, the buffer position of the > +start of the property change. I think you also know the length of the text where you call the hook, so maybe pass that as well? > + These functions get called even when > +`inhibit-modification-hooks' is non-nil. But not when run-hooks is nil. > +Note that these functions may NOT themselves make any buffer changes, > +including text property changes. */); And how do we enforce this? > + Vsyntax_table_props_change_alert_functions = Qnil; This should have a comment saying the value is assigned in syntax.el, i.e. it is always non-nil in a dumped Emacs. From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 09 Jun 2019 14:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: 36136@debbugs.gnu.org Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.15600919533065 (code B ref 36136); Sun, 09 Jun 2019 14:53:02 +0000 Received: (at 36136) by debbugs.gnu.org; 9 Jun 2019 14:52:33 +0000 Received: from localhost ([127.0.0.1]:55187 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hZzBQ-0000nN-Su for submit@debbugs.gnu.org; Sun, 09 Jun 2019 10:52:33 -0400 Received: from colin.muc.de ([193.149.48.1]:31462 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1hZzBO-0000nD-8s for 36136@debbugs.gnu.org; Sun, 09 Jun 2019 10:52:30 -0400 Received: (qmail 99018 invoked by uid 3782); 9 Jun 2019 14:52:24 -0000 Received: from acm.muc.de (p2E5D56FA.dip0.t-ipconnect.de [46.93.86.250]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 09 Jun 2019 16:52:23 +0200 Received: (qmail 8897 invoked by uid 1000); 9 Jun 2019 14:52:23 -0000 Date: Sun, 9 Jun 2019 14:52:23 +0000 Message-ID: <20190609145223.GA6712@ACM> References: <20190608131724.GA4643@ACM> <20190608203639.GA28722@ACM> <83ef43cndu.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <83ef43cndu.fsf@gnu.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Eli. Thanks for criticising my proposed patch. On Sun, Jun 09, 2019 at 08:56:45 +0300, Eli Zaretskii wrote: > > Date: Sat, 8 Jun 2019 20:36:39 +0000 > > From: Alan Mackenzie > > > Suggested fix: the functions set_properties, add_properties, > > > remove_properties in textprop.c should check for changes to, > > > specifically, syntax-table properties. When these changes are detected, > > > a hook called something like syntax-table-props-change-alert-hook should > > > be called (with some appropriate position parameters, tbd). > > > syntax-ppss-flush-cache will be added to this hook. > > Here is a first draught of a fix to this bug. > I have no opinion about the issue and the idea of its proposed > solution, but I do have some comments to the implementation. [ .... ] > This means that whenever syntax.el is loaded (i.e., always, since > syntax.el is preloaded), this hook will be non-nil. Is that a good > idea? I mean, if we always call this function, why do this via a > hook? No, it's not a good idea. Mainly because of .... [ .... ] > Also, what about the safety of this call? what if the hook signals an > error or the user presses C-g while the hook runs? IOW, should you > use safe_call or some of its variants, and should you inhibit QUIT? > and if so, whether and how should you handle in the code the case when > the hook does signal an error? .... add-text-properties and friends are called from redisplay. So it would be particularly inconvenient for a hook function to throw an error here - some sophisticated error handling (like there must be for fontification-functions) would be needed. You're right. All that the hook function really needs to do is set some buffer local syntax-ppss variable indicating the maximum valid position. This doesn't need a hook, just a new variable in syntax.c and code there and in syntax-ppss to use it. [ All other comments noted, and to a large extent incorporated into the code.] So, I'll start again. Thanks! -- Alan Mackenzie (Nuremberg, Germany). From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 09 Jun 2019 18:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 36136@debbugs.gnu.org Cc: Stefan Monnier Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.156010556031966 (code B ref 36136); Sun, 09 Jun 2019 18:40:01 +0000 Received: (at 36136) by debbugs.gnu.org; 9 Jun 2019 18:39:20 +0000 Received: from localhost ([127.0.0.1]:55375 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ha2it-0008JW-Ea for submit@debbugs.gnu.org; Sun, 09 Jun 2019 14:39:19 -0400 Received: from colin.muc.de ([193.149.48.1]:49163 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1ha2ip-0008JK-4R for 36136@debbugs.gnu.org; Sun, 09 Jun 2019 14:39:16 -0400 Received: (qmail 59666 invoked by uid 3782); 9 Jun 2019 18:39:13 -0000 Received: from acm.muc.de (p2E5D56FA.dip0.t-ipconnect.de [46.93.86.250]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 09 Jun 2019 20:39:12 +0200 Received: (qmail 18017 invoked by uid 1000); 9 Jun 2019 18:39:12 -0000 Date: Sun, 9 Jun 2019 18:39:12 +0000 Message-ID: <20190609183912.GA17965@ACM> References: <20190608131724.GA4643@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190608131724.GA4643@ACM> User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sat, Jun 08, 2019 at 13:17:24 +0000, Alan Mackenzie wrote: > Hello, Emacs. > The syntax-ppss cache is not invalidated when syntax-table text > properties are set or cleared. This is because the invalidation > function, syntax-ppss-flush-cache is invoked only as a before-change > function, but typical (?all) syntax-table property changes happen when > before-change-functions is inactive. > This is a bug. > In my debugging of a CC Mode scenario, a buffer change causes a > syntax-table text property change at an earlier part of the buffer. This > is to do with the change making previously non-matching C++ raw string > identifiers match up. Font lock follows the syntax-ppss state, which > spuriously records that the end part of the buffer is still in a string. > Hence the non-string part of the buffer is still fontified with > font-lock-string-face. > Suggested fix: the functions set_properties, add_properties, > remove_properties in textprop.c should check for changes to, > specifically, syntax-table properties. When these changes are detected, > a hook called something like syntax-table-props-change-alert-hook should > be called (with some appropriate position parameters, tbd). > syntax-ppss-flush-cache will be added to this hook. > -- > Alan Mackenzie (Nuremberg, Germany). The following patch is simpler than my first proposal, following feedback from Eli. It works for me. Stefan, could you look at this, please? Make syntax-ppss react to changes in syntax-table text properties In particular, it trims its cache to the point of such a change. This fixes bug #36136. * src/textprop.c (syntax-propertize--done): New buffer local variable. (set_properties, add_properties, remove_properties): when a syntax-table text property is being changed, reduce syntax-propertize--done to the buffer position. * lisp/emacs-lisp/syntax.el (syntax-ppss--trim-cache): New function extracted from syntax-ppss-flush-cache. (syntax-ppss-flush-cache): Now only modifies syntax-propertize--done and syntax-ppss--done. (syntax-ppss): Calls syntax-ppss--trim-cache and sets syntax-propertize--done. diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el index 9c6d5b5829..6ca27d8e83 100644 --- a/lisp/emacs-lisp/syntax.el +++ b/lisp/emacs-lisp/syntax.el @@ -404,13 +404,10 @@ syntax-ppss-narrow (defvar-local syntax-ppss-narrow-start nil "Start position of the narrowing for `syntax-ppss-narrow'.") -(define-obsolete-function-alias 'syntax-ppss-after-change-function - #'syntax-ppss-flush-cache "27.1") -(defun syntax-ppss-flush-cache (beg &rest ignored) - "Flush the cache of `syntax-ppss' starting at position BEG." - ;; Set syntax-propertize to refontify anything past beg. - (setq syntax-propertize--done (min beg syntax-propertize--done)) - ;; Flush invalid cache entries. +(defun syntax-ppss--trim-cache (beg) + "Flush the cache specific to `syntax-ppss' from position BEG. + +This doesn't include the cache for `syntax-propertize'." (dolist (cell (list syntax-ppss-wide syntax-ppss-narrow)) (pcase cell (`(,last . ,cache) @@ -432,8 +429,16 @@ syntax-ppss-flush-cache ;; (unless cache ;; (remove-hook 'before-change-functions #'syntax-ppss-flush-cache t)) (setcar cell last) - (setcdr cell cache))) - )) + (setcdr cell cache))))) + +(define-obsolete-function-alias 'syntax-ppss-after-change-function + #'syntax-ppss-flush-cache "27.1") +(defun syntax-ppss-flush-cache (beg &rest ignored) + "Flush the cache of `syntax-ppss' starting at position BEG." + ;; Set syntax-propertize to refontify anything past beg. + (setq syntax-propertize--done (min beg syntax-propertize--done)) + ;; Flush invalid cache entries. + (setq syntax-ppss--done (min beg syntax-ppss--done))) ;;; FIXME: Explain this variable. Currently only its last (5th) slot is used. ;;; Perhaps the other slots should be removed? @@ -477,8 +482,10 @@ syntax-ppss running the hook." ;; Default values. (unless pos (setq pos (point))) + (syntax-ppss--trim-cache syntax-ppss--done) (syntax-propertize pos) ;; + (setq syntax-ppss--done (max syntax-ppss--done pos)) (with-syntax-table (or syntax-ppss-table (syntax-table)) (let* ((cell (syntax-ppss--data)) (ppss-last (car cell)) diff --git a/src/textprop.c b/src/textprop.c index ae42c44185..be9e34c889 100644 --- a/src/textprop.c +++ b/src/textprop.c @@ -334,6 +334,9 @@ set_properties (Lisp_Object properties, INTERVAL interval, Lisp_Object object) record_property_change (interval->position, LENGTH (interval), XCAR (sym), XCAR (value), object); + if (EQ (sym, Qsyntax_table) + && (interval->position < syntax_ppss__done)) + syntax_ppss__done = interval->position; } /* For each new property that has no value at all in the old plist, @@ -346,6 +349,9 @@ set_properties (Lisp_Object properties, INTERVAL interval, Lisp_Object object) record_property_change (interval->position, LENGTH (interval), XCAR (sym), Qnil, object); + if (EQ (sym, Qsyntax_table) + && (interval->position < syntax_ppss__done)) + syntax_ppss__done = interval->position; } } @@ -400,6 +406,9 @@ add_properties (Lisp_Object plist, INTERVAL i, Lisp_Object object, { record_property_change (i->position, LENGTH (i), sym1, Fcar (this_cdr), object); + if (EQ (sym1, Qsyntax_table) + && (i->position < syntax_ppss__done)) + syntax_ppss__done = i->position; } /* I's property has a different value -- change it */ @@ -436,6 +445,9 @@ add_properties (Lisp_Object plist, INTERVAL i, Lisp_Object object, { record_property_change (i->position, LENGTH (i), sym1, Qnil, object); + if (EQ (sym1, Qsyntax_table) + && (i->position < syntax_ppss__done)) + syntax_ppss__done = i->position; } set_interval_plist (i, Fcons (sym1, Fcons (val1, i->plist))); changed = true; @@ -470,9 +482,14 @@ remove_properties (Lisp_Object plist, Lisp_Object list, INTERVAL i, Lisp_Object while (CONSP (current_plist) && EQ (sym, XCAR (current_plist))) { if (BUFFERP (object)) - record_property_change (i->position, LENGTH (i), - sym, XCAR (XCDR (current_plist)), - object); + { + record_property_change (i->position, LENGTH (i), + sym, XCAR (XCDR (current_plist)), + object); + if (EQ (sym, Qsyntax_table) + && (i->position < syntax_ppss__done)) + syntax_ppss__done = i->position; + } current_plist = XCDR (XCDR (current_plist)); changed = true; @@ -486,8 +503,13 @@ remove_properties (Lisp_Object plist, Lisp_Object list, INTERVAL i, Lisp_Object if (CONSP (this) && EQ (sym, XCAR (this))) { if (BUFFERP (object)) - record_property_change (i->position, LENGTH (i), - sym, XCAR (XCDR (this)), object); + { + record_property_change (i->position, LENGTH (i), + sym, XCAR (XCDR (this)), object); + if (EQ (sym, Qsyntax_table) + && (i->position < syntax_ppss__done)) + syntax_ppss__done = i->position; + } Fsetcdr (XCDR (tail2), XCDR (XCDR (this))); changed = true; @@ -2319,6 +2341,11 @@ inherits it if NONSTICKINESS is nil. The `front-sticky' and Vtext_property_default_nonsticky = list2 (Fcons (Qsyntax_table, Qt), Fcons (Qdisplay, Qt)); + DEFVAR_INT ("syntax-ppss--done", syntax_ppss__done, + doc: /* Position up to which the `syntax-ppss' cache is valid. */); + syntax_ppss__done = -1; + Fmake_variable_buffer_local (intern ("syntax_ppss__done")); + interval_insert_behind_hooks = Qnil; interval_insert_in_front_hooks = Qnil; staticpro (&interval_insert_behind_hooks); -- Alan Mackenzie (Nuremberg, Germany). From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 12 Jun 2019 08:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Alan Mackenzie Cc: 36136@debbugs.gnu.org Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.15603286619326 (code B ref 36136); Wed, 12 Jun 2019 08:38:02 +0000 Received: (at 36136) by debbugs.gnu.org; 12 Jun 2019 08:37:41 +0000 Received: from localhost ([127.0.0.1]:60650 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1haylI-0002QL-QR for submit@debbugs.gnu.org; Wed, 12 Jun 2019 04:37:41 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:21165) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1haylG-0002Q1-3L for 36136@debbugs.gnu.org; Wed, 12 Jun 2019 04:37:39 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id AAB754436D8; Wed, 12 Jun 2019 04:37:31 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 663D64436D6; Wed, 12 Jun 2019 04:37:30 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1560328650; bh=daD8pdCXNnOhdII0PUuxwI9/WAIPIU/LDcq4AJhXjXM=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=J/6FWBwk07CQROYfvVRm+/w/FMsSxRMDtWxc0r1JBWqLAFW2rj3XLVryDHlo3stvQ IuEFGEk1nDWYpiWPsziwRkJGJEX6iOnGWtCy23vZtHsRA3PfZ27IopoffL5iRrpl8r EU26SOqDhVgr6ug+xoJx8DYPUL8QfqoFpryU6xYfiYl4fWnNJqpxWBrEfnffynxvyD 9Pmr6Ri7otmPLFKaJvrChChf3jnWhZX+L35PaBkW0mFEhXS0D0iG10k4ug01OmdsB3 OcM11u+iQbgCox0gq5DjujKHvGTcsdVARQL8fRR2vVY0JJ3bdgmF6fsN7+bM25CCXc JfQ2ZVn8NMkPg== Received: from alfajor (109-74-184-110.biz.static.gigbone.net [109.74.184.110]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 7769C12036E; Wed, 12 Jun 2019 04:37:29 -0400 (EDT) From: Stefan Monnier Message-ID: References: <20190608131724.GA4643@ACM> <20190609183912.GA17965@ACM> Date: Wed, 12 Jun 2019 04:37:25 -0400 In-Reply-To: <20190609183912.GA17965@ACM> (Alan Mackenzie's message of "Sun, 9 Jun 2019 18:39:12 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > The following patch is simpler than my first proposal, following > feedback from Eli. It works for me. > > Stefan, could you look at this, please? Here I am. > * src/textprop.c (syntax-propertize--done): New buffer local variable. > (set_properties, add_properties, remove_properties): when a syntax-table text > property is being changed, reduce syntax-propertize--done to the buffer > position. Hmm... I'm not too fond of adding ad-hoc support for specific text-properties in (set_properties, add_properties, remove_properties). > * lisp/emacs-lisp/syntax.el (syntax-ppss--trim-cache): New function extracted > from syntax-ppss-flush-cache. > (syntax-ppss-flush-cache): Now only modifies syntax-propertize--done and > syntax-ppss--done. > (syntax-ppss): Calls syntax-ppss--trim-cache and sets syntax-propertize--done. This part looks OK. I'm not sure if making the cache-flushing more lazy will be a win overall: it speeds up buffer modifications at the cost of slowing down syntax-ppss. To get back to the original problem: > This is because the invalidation function, syntax-ppss-flush-cache is > invoked only as a before-change function, but typical (?all) > syntax-table property changes happen when before-change-functions > is inactive. That's why it doesn't have "--" in its name: if you don't want to use syntax-propertize then you'll probably have to call that function by hand. I consider it as perfectly acceptable. Stefan From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 12 Jun 2019 10:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Stefan Monnier Cc: 36136@debbugs.gnu.org Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.156033451122283 (code B ref 36136); Wed, 12 Jun 2019 10:16:02 +0000 Received: (at 36136) by debbugs.gnu.org; 12 Jun 2019 10:15:11 +0000 Received: from localhost ([127.0.0.1]:60750 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hb0Hf-0005nL-5B for submit@debbugs.gnu.org; Wed, 12 Jun 2019 06:15:11 -0400 Received: from colin.muc.de ([193.149.48.1]:44286 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1hb0Hd-0005nA-Fj for 36136@debbugs.gnu.org; Wed, 12 Jun 2019 06:15:10 -0400 Received: (qmail 85848 invoked by uid 3782); 12 Jun 2019 10:15:05 -0000 Received: from acm.muc.de (p4FE15E42.dip0.t-ipconnect.de [79.225.94.66]) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 12 Jun 2019 12:15:03 +0200 Received: (qmail 5847 invoked by uid 1000); 12 Jun 2019 10:15:03 -0000 Date: Wed, 12 Jun 2019 10:15:03 +0000 Message-ID: <20190612101503.GA4587@ACM> References: <20190608131724.GA4643@ACM> <20190609183912.GA17965@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Wed, Jun 12, 2019 at 04:37:25 -0400, Stefan Monnier wrote: > > The following patch is simpler than my first proposal, following > > feedback from Eli. It works for me. > > Stefan, could you look at this, please? > Here I am. Hello! > > * src/textprop.c (syntax-propertize--done): New buffer local variable. > > (set_properties, add_properties, remove_properties): when a syntax-table text > > property is being changed, reduce syntax-propertize--done to the buffer > > position. > Hmm... I'm not too fond of adding ad-hoc support for specific > text-properties in (set_properties, add_properties, remove_properties). Neither am I, particularly. But the whole point of syntax-ppss, surely, is that it should work automatically, without users having to call syntax-ppss-flush-cache all the time. The root of the problem is that inhibit-modification-hooks is too blunt a tool. Setting it prevents running functions we want to run, just as much as ones we don't. I've just had another idea: we introduce a new property called something like dont-inhibit. When a function in before/after-change-functions has this property, it would run, regardless of inhibit-modification-hooks. Or possibly, we could introduce new hooks no-inhibit-before/after-change-functions. What do you think? > > * lisp/emacs-lisp/syntax.el (syntax-ppss--trim-cache): New function extracted > > from syntax-ppss-flush-cache. > > (syntax-ppss-flush-cache): Now only modifies syntax-propertize--done and > > syntax-ppss--done. > > (syntax-ppss): Calls syntax-ppss--trim-cache and sets syntax-propertize--done. > This part looks OK. > I'm not sure if making the cache-flushing more lazy will be a win > overall: it speeds up buffer modifications at the cost of slowing down > syntax-ppss. It will probably not make a great deal of difference either way. Buffer changes are frequent in Emacs, and so are calls to syntax-ppss in many major modes. > To get back to the original problem: > > This is because the invalidation function, syntax-ppss-flush-cache is > > invoked only as a before-change function, but typical (?all) > > syntax-table property changes happen when before-change-functions > > is inactive. > That's why it doesn't have "--" in its name: if you don't want to use > syntax-propertize then you'll probably have to call that function > by hand. I consider it as perfectly acceptable. Well, for CC Mode I'm going to have to do that anyway, since Emacs-26.x and earlier are already out there and aren't going to change. This is going to be tedious and error prone. But for the future, it would be nice if syntax-ppss could take note of all buffer changes, rather than just some of them. > Stefan -- Alan Mackenzie (Nuremberg, Germany). From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 12 Jun 2019 10:55:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Alan Mackenzie Cc: 36136@debbugs.gnu.org Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.156033688027776 (code B ref 36136); Wed, 12 Jun 2019 10:55:01 +0000 Received: (at 36136) by debbugs.gnu.org; 12 Jun 2019 10:54:40 +0000 Received: from localhost ([127.0.0.1]:60773 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hb0tp-0007Ds-PU for submit@debbugs.gnu.org; Wed, 12 Jun 2019 06:54:38 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:11419) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hb0tn-0007Da-BG for 36136@debbugs.gnu.org; Wed, 12 Jun 2019 06:54:36 -0400 Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id A9E1810089B; Wed, 12 Jun 2019 06:54:29 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 568301007B5; Wed, 12 Jun 2019 06:54:28 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1560336868; bh=q5Ha3AAJBxalW4hnix1VLaHETx+EUyiF3Kia8Oem7ec=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=RBu71Pxl5pB+wDd2uDacTBhcfwGNsdycZ56Q64typ6vUA3iqsTv+7fvOM5Cq6MKct GUKAbsuWdSkAjv+L2nnJNetir8NGSTjf4SN8go+0yF6V4UPheq4yyDxL5LbebiK9Rq bPSVZyi6hX82i5Iwy6hwJGOYtN4/HBYmTL+ycKxQb8FrCVEjiuHvPooPRw94M94wm8 /025yjWLmQ79vf7y9y12pwXnnri8wZE5S+fNLaJ24cBG4TD2ZVM5aMGNXVKVTwmJoy vazKHdfxVxnptf6WXW7GwxEyQcTIWqVdYfNH6Ex6Rbtq8ToxjnkbPK0IIR8zRUBA+H DHACe7Mz711/g== Received: from alfajor (cm-84.215.66.78.getinternet.no [84.215.66.78]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id B0F7A120C58; Wed, 12 Jun 2019 06:54:27 -0400 (EDT) From: Stefan Monnier Message-ID: References: <20190608131724.GA4643@ACM> <20190609183912.GA17965@ACM> <20190612101503.GA4587@ACM> Date: Wed, 12 Jun 2019 06:54:25 -0400 In-Reply-To: <20190612101503.GA4587@ACM> (Alan Mackenzie's message of "Wed, 12 Jun 2019 10:15:03 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >> Hmm... I'm not too fond of adding ad-hoc support for specific >> text-properties in (set_properties, add_properties, remove_properties). > Neither am I, particularly. But the whole point of syntax-ppss, surely, > is that it should work automatically, without users having to call > syntax-ppss-flush-cache all the time. `grep` seems to argue that "all the time" is an over statement. I see the following uses in Emacs's bundled files: - one call in lisp/elec-pair.el [ I think this one would likely be better handled some other way (e.g. by setting syntax-ppss-table), tho I haven't looked hard enough to have a firm opinion yet ]. - in before-change-functions, of course. - one call from syntax-propertize. - one call in font-lock-apply-syntactic-highlight (this is part of the font-lock-syntactic-keywords functionality marked obsolete in 24.1). - one call in fortran-line-length (not because the buffer is changed, but because the syntax-propertize-function is changed). Of these, only the one in syntax-propertize and the one in font-lock-apply-syntactic-highlight would be made unnecessary by your patch. I see a few more calls in the elpa.git files, but none of them seems affected either. > It will probably not make a great deal of difference either way. Buffer > changes are frequent in Emacs, and so are calls to syntax-ppss in many > major modes. That's also my impression (which is why I haven't bothered to make syntax-ppss-flush-cache lazy like your patch does, even tho I've been tempted to several times). > Well, for CC Mode I'm going to have to do that anyway, since Emacs-26.x > and earlier are already out there and aren't going to change. Indeed. [ unless you decide to make CC-mode use syntax-propertize, of course ;-) ] But I also agree that it shouldn't prevent us from looking for better solutions. > This is going to be tedious and error prone. I don't see why. Usually a single call does the trick (as seen above: either in the function that sets the syntax-table property, or in the function that takes care of refreshing things in response to a buffer modification). Stefan From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 13 Jun 2019 12:22:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Stefan Monnier Cc: 36136@debbugs.gnu.org Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.15604285185094 (code B ref 36136); Thu, 13 Jun 2019 12:22:01 +0000 Received: (at 36136) by debbugs.gnu.org; 13 Jun 2019 12:21:58 +0000 Received: from localhost ([127.0.0.1]:34679 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hbOju-0001K6-JA for submit@debbugs.gnu.org; Thu, 13 Jun 2019 08:21:58 -0400 Received: from colin.muc.de ([193.149.48.1]:47516 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1hbOjs-0001Jx-Fi for 36136@debbugs.gnu.org; Thu, 13 Jun 2019 08:21:57 -0400 Received: (qmail 33436 invoked by uid 3782); 13 Jun 2019 12:21:50 -0000 Received: from acm.muc.de (p4FE15E04.dip0.t-ipconnect.de [79.225.94.4]) by colin.muc.de (tmda-ofmipd) with ESMTP; Thu, 13 Jun 2019 14:21:50 +0200 Received: (qmail 25924 invoked by uid 1000); 13 Jun 2019 12:21:50 -0000 Date: Thu, 13 Jun 2019 12:21:50 +0000 Message-ID: <20190613122150.GA25866@ACM> References: <20190608131724.GA4643@ACM> <20190609183912.GA17965@ACM> <20190612101503.GA4587@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, Stefan. On Wed, Jun 12, 2019 at 06:54:25 -0400, Stefan Monnier wrote: > >> Hmm... I'm not too fond of adding ad-hoc support for specific > >> text-properties in (set_properties, add_properties, remove_properties). > > Neither am I, particularly. But the whole point of syntax-ppss, surely, > > is that it should work automatically, without users having to call > > syntax-ppss-flush-cache all the time. > `grep` seems to argue that "all the time" is an over statement. CC Mode has to call that function all the time, without any scare quotes. In particular, it has to call the thing before each font-locking action. It shouldn't have to; it doesn't even make any use of the package, and is thus messy programming indeed. The root of the problem is the implicit assumption made by syntax-ppss that programs would never set syntax-table properties. This assumption doesn't hold. [ .... ] > > It will probably not make a great deal of difference either way. Buffer > > changes are frequent in Emacs, and so are calls to syntax-ppss in many > > major modes. > That's also my impression (which is why I haven't bothered to make > syntax-ppss-flush-cache lazy like your patch does, even tho I've been > tempted to several times). > > Well, for CC Mode I'm going to have to do that anyway, since Emacs-26.x > > and earlier are already out there and aren't going to change. > Indeed. > [ unless you decide to make CC-mode use syntax-propertize, of course ;-) ] I don't think that would work without loss of functionality and loss of speed (to whatever degree), and even if it could, would be more work than it's worth. > But I also agree that it shouldn't prevent us from looking for > better solutions. What about my idea of giving a "no inhibit" property to functions on before/after-change-functions? This would be easy to implement, though might have unwanted unexpected consequences. > > This is going to be tedious and error prone. > I don't see why. Usually a single call does the trick (as seen above: > either in the function that sets the syntax-table property, or in the > function that takes care of refreshing things in response to a buffer > modification). Yes, you're right, it wasn't too bad. There are many functions in CC Mode which set the syntax-table property, so adapting each of them didn't come into consideration. I amended the existing cache invalidation routine to keep track of a syntax-table property position, and pass that position to syntax-ppss-flush-cache at the end of c-after-change, just before font-lock kicks in. But, as already said, I shouldn't have to do this. > Stefan -- Alan Mackenzie (Nuremberg, Germany). From unknown Fri Jun 13 11:36:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 13 Jun 2019 21:32:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Alan Mackenzie Cc: 36136@debbugs.gnu.org Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.156046149332504 (code B ref 36136); Thu, 13 Jun 2019 21:32:02 +0000 Received: (at 36136) by debbugs.gnu.org; 13 Jun 2019 21:31:33 +0000 Received: from localhost ([127.0.0.1]:36246 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hbXJk-0008SC-KU for submit@debbugs.gnu.org; Thu, 13 Jun 2019 17:31:32 -0400 Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:36962) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hbXJi-0008Rw-19 for 36136@debbugs.gnu.org; Thu, 13 Jun 2019 17:31:31 -0400 Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 81CEE4437E4; Thu, 13 Jun 2019 17:31:23 -0400 (EDT) Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 25AFD4437DE; Thu, 13 Jun 2019 17:31:22 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1560461482; bh=4g0Gi+DA5U+PkQVDvXgy/qS1IX13U/zSjkmNuwTY26E=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=PYfSxtzAoIMO+pDX6lGCQ97NzX975dYfRwj8VVBTDEjxeAcC4gx+L4JzMydMv7FG8 2bjbR3U+iJ8zg5+Gf7J/TkuQoXfNgiVbKmac3dcp5Mcv7ndE82DFm4up8HJNTdReXt tugv2nKt/46kVuumTQl6RMeSwI7LzxRiZ+tAr2+9pbv9Uo4B/nPSWOl1kmLhfGpN8t BKlvNjUZoV17oeVshDDmjp5yX9xbz+rbkkkQXLUxINfJ5lxmRhwgl+1KKvA49KPI7y L3fKHNqWd4ono+DN5eqkoAIm8JJ6EMq30rUF40M5+ExCGmdkHHThGJ0th7cykMJM42 gJvkZ3FaeELKA== Received: from alfajor (cm-84.215.66.78.getinternet.no [84.215.66.78]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 83842120892; Thu, 13 Jun 2019 17:31:21 -0400 (EDT) From: Stefan Monnier Message-ID: References: <20190608131724.GA4643@ACM> <20190609183912.GA17965@ACM> <20190612101503.GA4587@ACM> <20190613122150.GA25866@ACM> Date: Thu, 13 Jun 2019 17:31:19 -0400 In-Reply-To: <20190613122150.GA25866@ACM> (Alan Mackenzie's message of "Thu, 13 Jun 2019 12:21:50 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SPAM-INFO: Spam detection results: 0 ALL_TRUSTED -1 Passed through trusted hosts only via SMTP BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain X-SPAM-LEVEL: X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >> >> Hmm... I'm not too fond of adding ad-hoc support for specific >> >> text-properties in (set_properties, add_properties, remove_properties). >> > Neither am I, particularly. But the whole point of syntax-ppss, surely, >> > is that it should work automatically, without users having to call >> > syntax-ppss-flush-cache all the time. >> `grep` seems to argue that "all the time" is an over statement. > CC Mode has to call that function all the time, without any scare > quotes. IIUC the end of your message, you did find another spot where to call syntax-ppss-flush-cache, making it much less problematic, right? > The root of the problem is the implicit assumption made by syntax-ppss > that programs would never set syntax-table properties. This assumption > doesn't hold. AFAIK CC-mode is the only exception nowadays. >> But I also agree that it shouldn't prevent us from looking for >> better solutions. > > What about my idea of giving a "no inhibit" property to functions on > before/after-change-functions? I'm not convinced this won't introduce more problems than it intends to fix (it seems that it will likely lead to the subsequent introduction of a `no-really-I-do-mean-inhibit-all-this-time` etc...). > But, as already said, I shouldn't have to do this. I agree that it would be nice to invalidate the cache in a more "automatic" way. But it's a trade-off. Your new call to syntax-ppss-flush-cache brings the count of calls that we could eliminate this way to 3 (one of those being obsolete since 24.1 should likely disappear soon). That's still not very many. And those calls are pretty efficient I think compared to the more general code which catches every text-property change to see if it happens to change the `syntax-table` property. So the current situation is not ideal conceptually, but it's more efficient than the alternative both in terms of run-time complexity and in terms of code size. It's actually a pretty good deal. Especially since even with a more automatic approach, there'd still be other places where we need to call it manually anyway (e.g. fortran-line-length). Stefan From unknown Fri Jun 13 11:36:43 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Alan Mackenzie Subject: bug#36136: closed (Re: bug#36136: Acknowledgement (syntax-ppss fails to invalidate its cache on changes to syntax-table text properties)) Message-ID: References: <20190824193547.GA8747@ACM> <20190608131724.GA4643@ACM> X-Gnu-PR-Message: they-closed 36136 X-Gnu-PR-Package: emacs Reply-To: 36136@debbugs.gnu.org Date: Sat, 24 Aug 2019 19:36:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1566675362-19794-1" This is a multi-part message in MIME format... ------------=_1566675362-19794-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #36136: syntax-ppss fails to invalidate its cache on changes to syntax-tabl= e text properties which was filed against the emacs package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 36136@debbugs.gnu.org. --=20 36136: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D36136 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1566675362-19794-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 36136-done) by debbugs.gnu.org; 24 Aug 2019 19:35:53 +0000 Received: from localhost ([127.0.0.1]:43300 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i1bpJ-00058p-2j for submit@debbugs.gnu.org; Sat, 24 Aug 2019 15:35:53 -0400 Received: from colin.muc.de ([193.149.48.1]:23774 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1i1bpH-00058Z-6m for 36136-done@debbugs.gnu.org; Sat, 24 Aug 2019 15:35:51 -0400 Received: (qmail 52512 invoked by uid 3782); 24 Aug 2019 18:43:02 -0000 Received: from acm.muc.de (p4FE152C7.dip0.t-ipconnect.de [79.225.82.199]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 24 Aug 2019 20:43:01 +0200 Received: (qmail 8826 invoked by uid 1000); 24 Aug 2019 19:35:47 -0000 Date: Sat, 24 Aug 2019 19:35:47 +0000 To: 36136-done@debbugs.gnu.org Subject: Re: bug#36136: Acknowledgement (syntax-ppss fails to invalidate its cache on changes to syntax-table text properties) Message-ID: <20190824193547.GA8747@ACM> References: <20190608131724.GA4643@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 36136-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) tags 36136 + wontfix quit It has been decided not to fix this bug, since it doesn't affect many packages, and easy (though ugly) workarounds are available, principally by the affected packages calling syntax-ppss-flush-cache. -- Alan Mackenzie (Nuremberg, Germany). ------------=_1566675362-19794-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 8 Jun 2019 13:17:52 +0000 Received: from localhost ([127.0.0.1]:53016 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hZbEF-0006wv-NB for submit@debbugs.gnu.org; Sat, 08 Jun 2019 09:17:51 -0400 Received: from lists.gnu.org ([209.51.188.17]:45304) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hZbEE-0006wo-1F for submit@debbugs.gnu.org; Sat, 08 Jun 2019 09:17:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53934) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hZbEB-0006WW-Op for bug-gnu-emacs@gnu.org; Sat, 08 Jun 2019 09:17:49 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hZbE9-00033Q-QZ for bug-gnu-emacs@gnu.org; Sat, 08 Jun 2019 09:17:47 -0400 Received: from colin.muc.de ([193.149.48.1]:51289 helo=mail.muc.de) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1hZbE6-0002un-0n for bug-gnu-emacs@gnu.org; Sat, 08 Jun 2019 09:17:43 -0400 Received: (qmail 62602 invoked by uid 3782); 8 Jun 2019 13:17:25 -0000 Received: from acm.muc.de (p2E5D5C7B.dip0.t-ipconnect.de [46.93.92.123]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 08 Jun 2019 15:17:24 +0200 Received: (qmail 5207 invoked by uid 1000); 8 Jun 2019 13:17:24 -0000 Date: Sat, 8 Jun 2019 13:17:24 +0000 To: bug-gnu-emacs@gnu.org Subject: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Message-ID: <20190608131724.GA4643@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-Received-From: 193.149.48.1 X-Spam-Score: -1.6 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.6 (--) Hello, Emacs. The syntax-ppss cache is not invalidated when syntax-table text properties are set or cleared. This is because the invalidation function, syntax-ppss-flush-cache is invoked only as a before-change function, but typical (?all) syntax-table property changes happen when before-change-functions is inactive. This is a bug. In my debugging of a CC Mode scenario, a buffer change causes a syntax-table text property change at an earlier part of the buffer. This is to do with the change making previously non-matching C++ raw string identifiers match up. Font lock follows the syntax-ppss state, which spuriously records that the end part of the buffer is still in a string. Hence the non-string part of the buffer is still fontified with font-lock-string-face. Suggested fix: the functions set_properties, add_properties, remove_properties in textprop.c should check for changes to, specifically, syntax-table properties. When these changes are detected, a hook called something like syntax-table-props-change-alert-hook should be called (with some appropriate position parameters, tbd). syntax-ppss-flush-cache will be added to this hook. -- Alan Mackenzie (Nuremberg, Germany). ------------=_1566675362-19794-1-- From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 24 15:57:23 2019 Received: (at control) by debbugs.gnu.org; 24 Aug 2019 19:57:23 +0000 Received: from localhost ([127.0.0.1]:43307 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1i1cA6-0005fL-SL for submit@debbugs.gnu.org; Sat, 24 Aug 2019 15:57:23 -0400 Received: from colin.muc.de ([193.149.48.1]:21206 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1i1cA3-0005fB-Mx for control@debbugs.gnu.org; Sat, 24 Aug 2019 15:57:20 -0400 Received: (qmail 62621 invoked by uid 3782); 24 Aug 2019 19:04:29 -0000 Received: from acm.muc.de (p4FE152C7.dip0.t-ipconnect.de [79.225.82.199]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 24 Aug 2019 21:04:29 +0200 Received: (qmail 3824 invoked by uid 1000); 24 Aug 2019 19:57:15 -0000 Date: Sat, 24 Aug 2019 19:57:15 +0000 To: control@debbugs.gnu.org Subject: Re: bug#36136: Acknowledgement (syntax-ppss fails to invalidate its cache on changes to syntax-table text properties) Message-ID: <20190824195715.GA3814@ACM> References: <20190608131724.GA4643@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) tags 36136 + wontfix quit -- Alan Mackenzie (Nuremberg, Germany).