GNU bug report logs - #22761
Surprising interaction between font-lock, invisible text, and point (self-insert-command and insert behave differently)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 22761 in the body.
You can then email your comments to 22761 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Surprising interaction between font-lock, invisible text, and point
 (self-insert-command and insert behave differently)
Date: Sun, 21 Feb 2016 19:19:17 -0500

[Message part 1 (text/plain, inline)]

Hi,

I'm running into problems using font-lock to display certain strings as subscripts. The basic goal is to display ‘a__b’ as ‘ab’ with ‘b’ vertically offset. Using font-lock to make ‘__’ invisible and to add (display (raise -0.15)) to ‘b’ works fine. There is a strange interaction between self-insert-command and font-lock, however, and the problem does not happen if using ‘insert’ instead of ‘self-insert-command’.

The problem happens when editing ‘a_foo_b’ to replace ‘foo’ by ‘bar’ (to get ‘a_bar_b’). Removing ‘foo’ leaves ‘__’ in the buffer, which font-lock recognizes: the whole string gets displayed as ‘ab’. Even then, the point is still between the two underscores (it hasn't moved after font-lock added the invisible property to the underscores). Pressing ‘b’ to start inserting ‘bar’ works fine (I end up with ‘a_b_c’), except for one thing: the point gets moved after the second underscore. Thus, pressing ‘ar’ to complete ‘bar’ ends up inserting ‘a_b_arb’.

On the other hand, using ‘M-: (insert ?b)’ to insert the ‘b’ of ‘bar’ leaves the point in the right place.

To reproduce:

1. Open a buffer in fundamental-mode
2. Evaluate the following setup code:
    (progn
      (setq font-lock-defaults '(nil))
      (font-lock-add-keywords nil `((,(concat "[a-z]+\\(__\\)\\([a-z]+\\)")
                                     (1 '(face nil invisible 'subscript))
                                     (2 '(face nil display (raise -0.25))))))
      (add-to-invisibility-spec 'subscript)
      (make-local-variable 'font-lock-extra-managed-props)
      (add-to-list 'font-lock-extra-managed-props 'display)
      (add-to-list 'font-lock-extra-managed-props 'invisible)
      (font-lock-mode))
3. Insert the following text: before_between_after
4. Place the point after ‘between’; press <backspace> 7 times, to remove ‘between’ entirely.
5. Type ‘between’ (using the key sequence b e t w e e n)

Expected result: buffer contains before_between_after
Actual result: buffer contains before_b_etweenafter

On the other hand, the following protocol works fine:

1-4. Same as before
5. ‘M-: (insert ?b)’
6. Type ‘etween’ (using the key sequence e t w e e n)

Expected result: buffer contains before_between_after
Actual result: buffer contains before_between_after

Cheers,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #8 received at 22761 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 22761 <at> debbugs.gnu.org
Subject: Re: bug#22761: Surprising interaction between font-lock,
 invisible text, 
 and point (self-insert-command and insert behave differently)
Date: Mon, 22 Feb 2016 17:53:55 +0200

> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Sun, 21 Feb 2016 19:19:17 -0500
> 
> The problem happens when editing ‘a_foo_b’ to replace ‘foo’ by ‘bar’ (to get ‘a_bar_b’). Removing ‘foo’ leaves ‘__’ in the buffer, which font-lock recognizes: the whole string gets displayed as ‘ab’. Even then, the point is still between the two underscores (it hasn't moved after font-lock added the invisible property to the underscores). Pressing ‘b’ to start inserting ‘bar’ works fine (I end up with ‘a_b_c’), except for one thing: the point gets moved after the second underscore. Thus, pressing ‘ar’ to complete ‘bar’ ends up inserting ‘a_b_arb’.

Did you try binding global-disable-point-adjustment to a non-nil
value?

Message #11 received at 22761 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 22761 <at> debbugs.gnu.org
Subject: Re: bug#22761: Surprising interaction between font-lock, invisible
 text, and point (self-insert-command and insert behave differently)
Date: Mon, 22 Feb 2016 11:03:03 -0500

[Message part 1 (text/plain, inline)]

On 02/22/2016 10:53 AM, Eli Zaretskii wrote:
>> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com> Date: Sun,
>> 21 Feb 2016 19:19:17 -0500
>> 
>> The problem happens when editing ‘a_foo_b’ to replace ‘foo’ by
>> ‘bar’ (to get ‘a_bar_b’). Removing ‘foo’ leaves ‘__’ in the buffer,
>> which font-lock recognizes: the whole string gets displayed as
>> ‘ab’. Even then, the point is still between the two underscores (it
>> hasn't moved after font-lock added the invisible property to the
>> underscores). Pressing ‘b’ to start inserting ‘bar’ works fine (I
>> end up with ‘a_b_c’), except for one thing: the point gets moved
>> after the second underscore. Thus, pressing ‘ar’ to complete ‘bar’
>> ends up inserting ‘a_b_arb’.
> 
> Did you try binding global-disable-point-adjustment to a non-nil 
> value?

I imagine that it would work, but wouldn't it break other parts of Emacs? The mode to which I'm adding this subscripts feature also uses hideshow, for example, and I do want the point to be moved out of invisible sections then.

Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #14 received at 22761 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: 22761 <at> debbugs.gnu.org
Subject: Re: bug#22761: Surprising interaction between font-lock, invisible
 text, and point (self-insert-command and insert behave differently)
Date: Mon, 22 Feb 2016 18:38:41 +0200

> Cc: 22761 <at> debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
> Date: Mon, 22 Feb 2016 11:03:03 -0500
> 
> On 02/22/2016 10:53 AM, Eli Zaretskii wrote:
> >> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com> Date: Sun,
> >> 21 Feb 2016 19:19:17 -0500
> >> 
> >> The problem happens when editing ‘a_foo_b’ to replace ‘foo’ by
> >> ‘bar’ (to get ‘a_bar_b’). Removing ‘foo’ leaves ‘__’ in the buffer,
> >> which font-lock recognizes: the whole string gets displayed as
> >> ‘ab’. Even then, the point is still between the two underscores (it
> >> hasn't moved after font-lock added the invisible property to the
> >> underscores). Pressing ‘b’ to start inserting ‘bar’ works fine (I
> >> end up with ‘a_b_c’), except for one thing: the point gets moved
> >> after the second underscore. Thus, pressing ‘ar’ to complete ‘bar’
> >> ends up inserting ‘a_b_arb’.
> > 
> > Did you try binding global-disable-point-adjustment to a non-nil 
> > value?
> 
> I imagine that it would work, but wouldn't it break other parts of Emacs? The mode to which I'm adding this subscripts feature also uses hideshow, for example, and I do want the point to be moved out of invisible sections then.

If it works, perhaps you could bind disable-point-adjustment (not the
global variable) in the insertion commands.

In general, insertion in the middle of invisible text is tricky at
best.  So perhaps you should rethink how you handle this situation --
you could, for example, temporarily remove the entire invisible
portion, until the insertion is complete.

Message #17 received at 22761 <at> debbugs.gnu.org (full text, mbox):

From: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 22761 <at> debbugs.gnu.org
Subject: Re: bug#22761: Surprising interaction between font-lock, invisible
 text, and point (self-insert-command and insert behave differently)
Date: Mon, 22 Feb 2016 12:36:42 -0500

[Message part 1 (text/plain, inline)]

On 02/22/2016 11:38 AM, Eli Zaretskii wrote:
>> Cc: 22761 <at> debbugs.gnu.org From: Clément Pit--Claudel
>> <clement.pitclaudel <at> live.com> Date: Mon, 22 Feb 2016 11:03:03
>> -0500
>> 
>> On 02/22/2016 10:53 AM, Eli Zaretskii wrote:
>>>> From: Clément Pit--Claudel <clement.pitclaudel <at> live.com> Date:
>>>> Sun, 21 Feb 2016 19:19:17 -0500
>>>> 
>>>> The problem happens when editing ‘a_foo_b’ to replace ‘foo’ by 
>>>> ‘bar’ (to get ‘a_bar_b’). Removing ‘foo’ leaves ‘__’ in the
>>>> buffer, which font-lock recognizes: the whole string gets
>>>> displayed as ‘ab’. Even then, the point is still between the
>>>> two underscores (it hasn't moved after font-lock added the
>>>> invisible property to the underscores). Pressing ‘b’ to start
>>>> inserting ‘bar’ works fine (I end up with ‘a_b_c’), except for
>>>> one thing: the point gets moved after the second underscore.
>>>> Thus, pressing ‘ar’ to complete ‘bar’ ends up inserting
>>>> ‘a_b_arb’.
>>> 
>>> Did you try binding global-disable-point-adjustment to a non-nil
>>>  value?
>> 
>> I imagine that it would work, but wouldn't it break other parts of
>> Emacs? The mode to which I'm adding this subscripts feature also
>> uses hideshow, for example, and I do want the point to be moved out
>> of invisible sections then.
> 
> If it works, perhaps you could bind disable-point-adjustment (not
> the global variable) in the insertion commands.
> 
> In general, insertion in the middle of invisible text is tricky at 
> best.  So perhaps you should rethink how you handle this situation
> -- you could, for example, temporarily remove the entire invisible 
> portion, until the insertion is complete.

Thanks, this is a good idea. It will also be more intuitive for users, probably.
I tried to follow the strategy that prettify-symbols-unprettify-at-point uses, but I kept running into issues; using an overlay seems to work better. The issue described in my OP is still there (when the buffer text is ‘__’ and an ‘a’ is added in the middle (yielding ‘_a_’) the overlay is removed, and Emacs moves the point after the second underscore (probably before fontification kicks in and removes the invisible property?). Still I'm better off, because I can detect that case and protect against it using disable-point-adjustment.

Cheers,
Clément.

[signature.asc (application/pgp-signature, attachment)]

Message #20 received at 22761 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Clément Pit--Claudel <clement.pitclaudel <at> live.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 22761 <at> debbugs.gnu.org
Subject: Re: bug#22761: Surprising interaction between font-lock, invisible
 text, and point (self-insert-command and insert behave differently)
Date: Wed, 30 Oct 2019 16:55:30 +0100

Clément Pit--Claudel <clement.pitclaudel <at> live.com> writes:

> Still I'm better off, because I can detect that case and protect
> against it using disable-point-adjustment.

If I skim this thread correctly, I think the conclusion was that there
was nothing to fix here, so I'm closing this bug report.  Please reopen
if that's incorrect.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.