From unknown Sat Aug 16 23:47:50 2025 X-Loop: help-debbugs@gnu.org Subject: bug#22316: font-lock corrupts `end' position in jit-lock after-change function. Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 05 Jan 2016 21:46:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 22316 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 22316@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.145203032812513 (code B ref -1); Tue, 05 Jan 2016 21:46:01 +0000 Received: (at submit) by debbugs.gnu.org; 5 Jan 2016 21:45:28 +0000 Received: from localhost ([127.0.0.1]:39235 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aGZQ0-0003Fl-2d for submit@debbugs.gnu.org; Tue, 05 Jan 2016 16:45:28 -0500 Received: from eggs.gnu.org ([208.118.235.92]:41576) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aGZPz-0003FZ-D9 for submit@debbugs.gnu.org; Tue, 05 Jan 2016 16:45:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aGZPt-0004dX-0J for submit@debbugs.gnu.org; Tue, 05 Jan 2016 16:45:22 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:47691) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aGZPs-0004dR-US for submit@debbugs.gnu.org; Tue, 05 Jan 2016 16:45:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60780) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aGZPr-0006nJ-Qy for bug-gnu-emacs@gnu.org; Tue, 05 Jan 2016 16:45:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aGZPm-0004cb-Q7 for bug-gnu-emacs@gnu.org; Tue, 05 Jan 2016 16:45:19 -0500 Received: from mail.muc.de ([193.149.48.3]:31095) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aGZPm-0004cK-Gt for bug-gnu-emacs@gnu.org; Tue, 05 Jan 2016 16:45:14 -0500 Received: (qmail 81152 invoked by uid 3782); 5 Jan 2016 21:45:11 -0000 Received: from acm.muc.de (p548A4948.dip0.t-ipconnect.de [84.138.73.72]) by colin.muc.de (tmda-ofmipd) with ESMTP; Tue, 05 Jan 2016 22:45:10 +0100 Received: (qmail 5180 invoked by uid 1000); 5 Jan 2016 21:47:28 -0000 Date: Tue, 5 Jan 2016 21:47:28 +0000 Message-ID: <20160105214728.GA3602@acm.fritz.box> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.4 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.4 (----) Hello, Emacs. With the latest emacs-25 branch (committed 2016-01-05, ~21:15 +0000), do emacs -Q, and load (or type in) the follow AWK Mode file (M-x awk-mode, if required): ######################################################################## #!/usr/bin/gawk -f BEGIN { x = 4 } ######################################################################### Notice the fontification of "BEGIN". Type a space anywhere on the first line. The following happens: (i) The fontification on the "B" of "BEGIN" is erased instantly. (ii) Half a second (jit-lock-context-time) later, the fontification on "EGIN" is erased. The cause of the problem is in font-lock-extend-jit-lock-region-after-change. The sequence is as follows: (i) In AWK Mode, font-lock-extend-after-change-region-function is non-nil, and this function is called to set `beg' and `end', the bounds of the region to fontify. In the above case, this region is exactly line 1. (ii) f-l-extend-j-l-r-a-change adds 1 to end. The region (beg end) now covers line 1 and the "B" at the beginning of line 2. (iii) After-change fontification fontifies this spurious region, (beg end). Since "B" is "on its own", it appears to be an ordinary variable use, hence is fontified without a face. (iv) Half a second later, context fontification starts from `end', finds "EGIN" and fontifies this as an ordinary variable, without a face. The solution I propose is that should the region have been set by font-lock-extend-after-change-region-function, the `end' position should be accepted as is. Here is a patch to implement this. I will commit it soon to emacs-25 if I hear no objections. diff --git a/lisp/font-lock.el b/lisp/font-lock.el index 4a92069..3c1f01d 100644 --- a/lisp/font-lock.el +++ b/lisp/font-lock.el @@ -1302,15 +1302,18 @@ font-lock-extend-jit-lock-region-after-change (point-min)))) (when (< end (point-max)) (setq end - (if (get-text-property end 'font-lock-multiline) - (or (text-property-any end (point-max) - 'font-lock-multiline nil) - (point-max)) + (cond + ((get-text-property end 'font-lock-multiline) + (or (text-property-any end (point-max) + 'font-lock-multiline nil) + (point-max))) + ;; If `end' has been set by the function above, don't corrupt it. + (font-lock-extend-after-change-region-function end) ;; Rounding up to a whole number of lines should include the ;; line right after `end'. Typical case: the first char of ;; the line was deleted. Or a \n was inserted in the middle ;; of a line. - (1+ end)))) + (t (1+ end))))) ;; Finally, pre-enlarge the region to a whole number of lines, to try ;; and anticipate what font-lock-default-fontify-region will do, so as to ;; avoid double-redisplay. -- Alan Mackenzie (Nuremberg, Germany). From unknown Sat Aug 16 23:47:50 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Alan Mackenzie Subject: bug#22316: closed (Re: bug#22316: font-lock corrupts `end' position in jit-lock after-change function.) Message-ID: References: <20160108144405.63446.qmail@mail.muc.de> <20160105214728.GA3602@acm.fritz.box> X-Gnu-PR-Message: they-closed 22316 X-Gnu-PR-Package: emacs Reply-To: 22316@debbugs.gnu.org Date: Fri, 08 Jan 2016 14:45:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1452264302-3683-1" This is a multi-part message in MIME format... ------------=_1452264302-3683-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #22316: font-lock corrupts `end' position in jit-lock after-change function. which was filed against the emacs package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 22316@debbugs.gnu.org. --=20 22316: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D22316 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1452264302-3683-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 22316-done) by debbugs.gnu.org; 8 Jan 2016 14:44:09 +0000 Received: from localhost ([127.0.0.1]:42431 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aHYGu-0000wB-My for submit@debbugs.gnu.org; Fri, 08 Jan 2016 09:44:08 -0500 Received: from mail.muc.de ([193.149.48.3]:16310) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aHYGs-0000w2-J4 for 22316-done@debbugs.gnu.org; Fri, 08 Jan 2016 09:44:06 -0500 Received: (qmail 63447 invoked by uid 3782); 8 Jan 2016 14:44:05 -0000 Date: 8 Jan 2016 14:44:05 -0000 Message-ID: <20160108144405.63446.qmail@mail.muc.de> From: Alan Mackenzie To: 22316-done@debbugs.gnu.org Subject: Re: bug#22316: font-lock corrupts `end' position in jit-lock after-change function. Organization: muc.de e.V. In-Reply-To: X-Newsgroups: gnu.emacs.bug User-Agent: tin/2.3.1-20141224 ("Tallant") (UNIX) (FreeBSD/10.2-RELEASE-p7 (amd64)) X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 22316-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Bug fixed. -- Alan Mackenzie (Nuremberg, Germany). ------------=_1452264302-3683-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 5 Jan 2016 21:45:28 +0000 Received: from localhost ([127.0.0.1]:39235 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aGZQ0-0003Fl-2d for submit@debbugs.gnu.org; Tue, 05 Jan 2016 16:45:28 -0500 Received: from eggs.gnu.org ([208.118.235.92]:41576) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aGZPz-0003FZ-D9 for submit@debbugs.gnu.org; Tue, 05 Jan 2016 16:45:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aGZPt-0004dX-0J for submit@debbugs.gnu.org; Tue, 05 Jan 2016 16:45:22 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:47691) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aGZPs-0004dR-US for submit@debbugs.gnu.org; Tue, 05 Jan 2016 16:45:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60780) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aGZPr-0006nJ-Qy for bug-gnu-emacs@gnu.org; Tue, 05 Jan 2016 16:45:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aGZPm-0004cb-Q7 for bug-gnu-emacs@gnu.org; Tue, 05 Jan 2016 16:45:19 -0500 Received: from mail.muc.de ([193.149.48.3]:31095) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aGZPm-0004cK-Gt for bug-gnu-emacs@gnu.org; Tue, 05 Jan 2016 16:45:14 -0500 Received: (qmail 81152 invoked by uid 3782); 5 Jan 2016 21:45:11 -0000 Received: from acm.muc.de (p548A4948.dip0.t-ipconnect.de [84.138.73.72]) by colin.muc.de (tmda-ofmipd) with ESMTP; Tue, 05 Jan 2016 22:45:10 +0100 Received: (qmail 5180 invoked by uid 1000); 5 Jan 2016 21:47:28 -0000 Date: Tue, 5 Jan 2016 21:47:28 +0000 To: bug-gnu-emacs@gnu.org Subject: font-lock corrupts `end' position in jit-lock after-change function. Message-ID: <20160105214728.GA3602@acm.fritz.box> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie X-Primary-Address: acm@muc.de X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.4 (----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.4 (----) Hello, Emacs. With the latest emacs-25 branch (committed 2016-01-05, ~21:15 +0000), do emacs -Q, and load (or type in) the follow AWK Mode file (M-x awk-mode, if required): ######################################################################## #!/usr/bin/gawk -f BEGIN { x = 4 } ######################################################################### Notice the fontification of "BEGIN". Type a space anywhere on the first line. The following happens: (i) The fontification on the "B" of "BEGIN" is erased instantly. (ii) Half a second (jit-lock-context-time) later, the fontification on "EGIN" is erased. The cause of the problem is in font-lock-extend-jit-lock-region-after-change. The sequence is as follows: (i) In AWK Mode, font-lock-extend-after-change-region-function is non-nil, and this function is called to set `beg' and `end', the bounds of the region to fontify. In the above case, this region is exactly line 1. (ii) f-l-extend-j-l-r-a-change adds 1 to end. The region (beg end) now covers line 1 and the "B" at the beginning of line 2. (iii) After-change fontification fontifies this spurious region, (beg end). Since "B" is "on its own", it appears to be an ordinary variable use, hence is fontified without a face. (iv) Half a second later, context fontification starts from `end', finds "EGIN" and fontifies this as an ordinary variable, without a face. The solution I propose is that should the region have been set by font-lock-extend-after-change-region-function, the `end' position should be accepted as is. Here is a patch to implement this. I will commit it soon to emacs-25 if I hear no objections. diff --git a/lisp/font-lock.el b/lisp/font-lock.el index 4a92069..3c1f01d 100644 --- a/lisp/font-lock.el +++ b/lisp/font-lock.el @@ -1302,15 +1302,18 @@ font-lock-extend-jit-lock-region-after-change (point-min)))) (when (< end (point-max)) (setq end - (if (get-text-property end 'font-lock-multiline) - (or (text-property-any end (point-max) - 'font-lock-multiline nil) - (point-max)) + (cond + ((get-text-property end 'font-lock-multiline) + (or (text-property-any end (point-max) + 'font-lock-multiline nil) + (point-max))) + ;; If `end' has been set by the function above, don't corrupt it. + (font-lock-extend-after-change-region-function end) ;; Rounding up to a whole number of lines should include the ;; line right after `end'. Typical case: the first char of ;; the line was deleted. Or a \n was inserted in the middle ;; of a line. - (1+ end)))) + (t (1+ end))))) ;; Finally, pre-enlarge the region to a whole number of lines, to try ;; and anticipate what font-lock-default-fontify-region will do, so as to ;; avoid double-redisplay. -- Alan Mackenzie (Nuremberg, Germany). ------------=_1452264302-3683-1--