GNU bug report logs - #19145
24.4; prettify-symbols-mode inconsistent behavior

Previous Next

To reply to this bug, email your comments to 19145 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ken Mankoff <mankoff <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.4; prettify-symbols-mode inconsistent behavior
Date: Fri, 21 Nov 2014 12:40:12 -0500

I'm having issues with the new prettify-symbols-mode. I'm not sure if
this is a bug, or just a fact that the implementation is limited, in
which case this is a feature request for a more complete implementation.

Some symbols are sometimes not being treated correctly depending on what
characters follow. For example, I have the following setup for coding
Python:

    (prettify-symbols-mode t)
    (global-prettify-symbols-mode t)
    (add-hook 'python-mode-hook
              (lambda ()
                (push '("**2" . ?²) prettify-symbols-alist)
                (push '("_x" . ?ᵪ) prettify-symbols-alist)
                (push '("delta" . ?δ) prettify-symbols-alist)))


The issue may be somewhat subjective. For example, should foo_xx appear
with a subscript x and then a normal x? Or should it appear as I assume
you are reading it with no prettification? I would argue for the latter.
What about foo_x+2? Regardless, I've created a matrix of
prettifications, what I'd expect, and what happens.

| Characters | Expected               | Actual                        | Good? |
|------------+------------------------+-------------------------------+-------|
| foo_x      | subscript x            | subscript x                   | Y     |
| foo**2     | superscript 2          | superscript 2                 | Y     |
| delta      | delta symbol           | delta symbol                  | Y     |
| foo_x+     | subscript x            | No subscript                  | N     |
| foo_xi     | no subscript           | subscript                     | N     |
| foo_x[42]  | subscript              | subscript                     | Y     |
| foo_x**2   | subscript, superscript | no subscript, yes superscript | N     |
| foo**200   | no superscript         | superscript 2                 | N     |
| delta(42)  | delta symbol(42)       | symbol                        | Y     |
| delta+42   | symbol                 | symbol                        | Y     |
| delta**2   | symbol, superscript    | symbol, superscritp           | Y     |
| delta_x    | symbol, subscript      | no symbol                     | N     |

There are some inconsistencies, like why _x+ loses prettification, but
delta+ retains it, or why foo_x_x_x works, but delta_x does not.

Thanks,

  -k.

Message #8 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Ken Mankoff <mankoff <at> gmail.com>
Cc: 19145 <at> debbugs.gnu.org
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Fri, 21 Nov 2014 13:15:01 -0500

> Some symbols are sometimes not being treated correctly depending on what
> characters follow. For example, I have the following setup for coding
> Python:

>     (prettify-symbols-mode t)
>     (global-prettify-symbols-mode t)
>     (add-hook 'python-mode-hook
>               (lambda ()
>                 (push '("**2" . ?²) prettify-symbols-alist)
>                 (push '("_x" . ?ᵪ) prettify-symbols-alist)
>                 (push '("delta" . ?δ) prettify-symbols-alist)))

Can you try the patch below and see if it does what you want?


        Stefan


diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
index 5037020..475dd32 100644
--- a/lisp/progmodes/prog-mode.el
+++ b/lisp/progmodes/prog-mode.el
@@ -73,11 +73,13 @@ Regexp match data 0 points to the chars."
   ;; Check that the chars should really be composed into a symbol.
   (let* ((start (match-beginning 0))
 	 (end (match-end 0))
-	 (syntaxes (if (eq (char-syntax (char-after start)) ?w)
-		       '(?w ?_) '(?. ?\\)))
+	 (syntax-beg (if (eq (char-syntax (char-after start)) ?w)
+                         '(?w ?_) '(?. ?\\)))
+	 (syntax-end (if (eq (char-syntax (char-before end)) ?w)
+                         '(?w ?_) '(?. ?\\)))
 	 match)
-    (if (or (memq (char-syntax (or (char-before start) ?\s)) syntaxes)
-	    (memq (char-syntax (or (char-after end) ?\s)) syntaxes)
+    (if (or (memq (char-syntax (or (char-before start) ?\s)) syntax-beg)
+	    (memq (char-syntax (or (char-after end) ?\s)) syntax-end)
             ;; syntax-ppss could modify the match data (bug#14595)
             (progn (setq match (match-string 0)) (nth 8 (syntax-ppss))))
 	;; No composition for you.  Let's actually remove any composition

Message #11 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Ken Mankoff <mankoff <at> gmail.com>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 19145 <at> debbugs.gnu.org
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Fri, 21 Nov 2014 14:22:53 -0500

* On 2014-11-21 at 13:15, Stefan Monnier wrote:
>> Some symbols are sometimes not being treated correctly depending on
>> what characters follow. For example, I have the following setup for
>> coding Python:
>
>>     (prettify-symbols-mode t)
>>     (global-prettify-symbols-mode t)
>>     (add-hook 'python-mode-hook
>>               (lambda ()
>>                 (push '("**2" . ?²) prettify-symbols-alist)
>>                 (push '("_x" . ?ᵪ) prettify-symbols-alist)
>>                 (push '("delta" . ?δ) prettify-symbols-alist)))
>
> Can you try the patch below and see if it does what you want?
>
>
> diff --git a/lisp/progmodes/prog-mode.el b/lisp/progmodes/prog-mode.el
> index 5037020..475dd32 100644
> --- a/lisp/progmodes/prog-mode.el
> +++ b/lisp/progmodes/prog-mode.el
> @@ -73,11 +73,13 @@ Regexp match data 0 points to the chars."
>    ;; Check that the chars should really be composed into a symbol.
>    (let* ((start (match-beginning 0))
>  	 (end (match-end 0))
> -	 (syntaxes (if (eq (char-syntax (char-after start)) ?w)
> -		       '(?w ?_) '(?. ?\\)))
> +	 (syntax-beg (if (eq (char-syntax (char-after start)) ?w)
> +                         '(?w ?_) '(?. ?\\)))
> +	 (syntax-end (if (eq (char-syntax (char-before end)) ?w)
> +                         '(?w ?_) '(?. ?\\)))
>  	 match)
> -    (if (or (memq (char-syntax (or (char-before start) ?\s)) syntaxes)
> -	    (memq (char-syntax (or (char-after end) ?\s)) syntaxes)
> +    (if (or (memq (char-syntax (or (char-before start) ?\s)) syntax-beg)
> +	    (memq (char-syntax (or (char-after end) ?\s)) syntax-end)
>              ;; syntax-ppss could modify the match data (bug#14595)
>              (progn (setq match (match-string 0)) (nth 8 (syntax-ppss))))
>  	;; No composition for you.  Let's actually remove any composition


Much improved! Of my examples, only one case no longer works which is
delta_x or foo_x_x_x_x, for example.

  -k.
  

Message #14 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Ken Mankoff <mankoff <at> gmail.com>
Cc: 19145 <at> debbugs.gnu.org
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Sat, 22 Nov 2014 11:23:56 -0500

> Much improved! Of my examples, only one case no longer works which is
> delta_x or foo_x_x_x_x, for example.

This would seem to indicate that the syntax-table of the major-mode in
effect has marked the underscore character with "symbol syntax"
(denoted confusingly enough by an underscore character,
in the `C-u C-x =' help).

Such a setting is appropriate is "delta_x" is one identifier, but not if
it's supposed to be 3 elements (identifier "delta", infix "_", and
identifier "x").

So, is this setting correct (i.e. does your language treat "delta_x" as
a single identifier, and you're trying to prettify subparts of
identifiers)?


        Stefan

Message #17 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Ken Mankoff <mankoff <at> gmail.com>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 19145 <at> debbugs.gnu.org
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Sat, 22 Nov 2014 18:57:19 -0500

* On 2014-11-22 at 11:23, Stefan Monnier wrote:
>> Much improved! Of my examples, only one case no longer works which is
>> delta_x or foo_x_x_x_x, for example.
>
> This would seem to indicate that the syntax-table of the major-mode in
> effect has marked the underscore character with "symbol syntax"
> (denoted confusingly enough by an underscore character, in the `C-u
> C-x =' help).
>
> Such a setting is appropriate is "delta_x" is one identifier, but not
> if it's supposed to be 3 elements (identifier "delta", infix "_", and
> identifier "x").
>
> So, is this setting correct (i.e. does your language treat "delta_x"
> as a single identifier, and you're trying to prettify subparts of
> identifiers)?

I'm not sure. I'm just using python and elpy (and therefore
python-mode). I think underscores are normal characters, and a_b is a
generic variable name, three characters long. There is nothing special
about _ or _b (compared to LaTeX and LaTeX-mode, where there is
meaning).

Does this mean I should file an issue with python-mode, or more likely
that everything is working as it should, and I'm just running into an
edge case that can only be covered by a more complex implementation that
uses regexes?

  -k.
  

Message #20 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Ken Mankoff <mankoff <at> gmail.com>
Cc: 19145 <at> debbugs.gnu.org
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Mon, 24 Nov 2014 09:53:43 -0500

>> So, is this setting correct (i.e. does your language treat "delta_x"
>> as a single identifier, and you're trying to prettify subparts of
>> identifiers)?
> I'm not sure. I'm just using python and elpy (and therefore
> python-mode).

Duh!  Sorry, I somehow failed to see "python-mode" in your original
bug report.  So yes, you're trying to prettify subparts of identifiers,
and prettify-symbols-mode currently provides no support at all for that.

> that everything is working as it should, and I'm just running into an
> edge case that can only be covered by a more complex implementation that
> uses regexes?

Exactly.

for _x and **2, I think prettify-symbols-mode is probably not a good
solution anyway (because it won't extend to _xy or to **24).  You'd be
better off with a font-lock rule which just shifts the text up/down (you
might like to look at the way we do just that in text-mode.el), which
can work with any sequence of character rather than being limited to
those few characters which have a "superscript" form in Unicode.

But w.r.t "delta" in "delta_x" it would make a lot of sense for
prettify-symbols-mode to provide support for that.


        Stefan

Message #23 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Ted Zlatanov <tzz <at> lifelogs.com>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 19145 <at> debbugs.gnu.org, Ken Mankoff <mankoff <at> gmail.com>
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Tue, 25 Nov 2014 04:49:11 -0500

On Mon, 24 Nov 2014 09:53:43 -0500 Stefan Monnier <monnier <at> IRO.UMontreal.CA> wrote: 

SM> Duh!  Sorry, I somehow failed to see "python-mode" in your original
SM> bug report.  So yes, you're trying to prettify subparts of identifiers,
SM> and prettify-symbols-mode currently provides no support at all for that.
...
SM> But w.r.t "delta" in "delta_x" it would make a lot of sense for
SM> prettify-symbols-mode to provide support for that.

I don't think it would--I would keep `prettify-symbols-mode' strict. I
think Ken needs a different mode that's yet to be written:
`prettify-regex-mode'?

"delta_x" is an entirely different thing from "delta" in all the
programming languages I can think of, so I would not expect to see δ_x
all over the code just because I want to prettify deltas themselves. If
anything, it would make finding the real standalone deltas harder.

Ted

Message #26 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Ken Mankoff <mankoff <at> gmail.com>
Cc: 19145 <at> debbugs.gnu.org
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Tue, 25 Nov 2014 09:51:56 -0500

SM> But w.r.t "delta" in "delta_x" it would make a lot of sense for
SM> prettify-symbols-mode to provide support for that.
> I don't think it would--I would keep `prettify-symbols-mode' strict. I
> think Ken needs a different mode that's yet to be written:
> `prettify-regex-mode'?

By "provide support for that", I meant to provide the option to specify
symbols with regexps.


        Stefan

Message #29 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Ken Mankoff <mankoff <at> gmail.com>
To: Ted Zlatanov <tzz <at> lifelogs.com>
Cc: 19145 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Tue, 25 Nov 2014 09:53:53 -0500

* On 2014-11-25 at 04:49, Ted Zlatanov wrote:
> On Mon, 24 Nov 2014 09:53:43 -0500 Stefan Monnier <monnier <at> IRO.UMontreal.CA> wrote: 
>
> SM> Duh!  Sorry, I somehow failed to see "python-mode" in your original
> SM> bug report.  So yes, you're trying to prettify subparts of identifiers,
> SM> and prettify-symbols-mode currently provides no support at all for that.
> ...
> SM> But w.r.t "delta" in "delta_x" it would make a lot of sense for
> SM> prettify-symbols-mode to provide support for that.
>
> I don't think it would--I would keep `prettify-symbols-mode' strict. I
> think Ken needs a different mode that's yet to be written:
> `prettify-regex-mode'?

This mode exists - pretty-symbols-mode
https://github.com/drothlis/pretty-symbols

I just try to use official packages when possible, hence my recent
switch to prettify-symbols-mode.

  -k.
  

Message #32 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Ted Zlatanov <tzz <at> lifelogs.com>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 19145 <at> debbugs.gnu.org, Ken Mankoff <mankoff <at> gmail.com>
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Tue, 25 Nov 2014 10:17:41 -0500

On Tue, 25 Nov 2014 09:51:56 -0500 Stefan Monnier <monnier <at> IRO.UMontreal.CA> wrote: 

SM> But w.r.t "delta" in "delta_x" it would make a lot of sense for
SM> prettify-symbols-mode to provide support for that.
>> I don't think it would--I would keep `prettify-symbols-mode' strict. I
>> think Ken needs a different mode that's yet to be written:
>> `prettify-regex-mode'?

SM> By "provide support for that", I meant to provide the option to specify
SM> symbols with regexps.

But that would require some special adaptation because it wouldn't work
in reverse. IOW, how do you propose to preserve the current behavior of
the prettified symbol "delta" while also allowing for the regex "delta"?

Ted

Message #35 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Ken Mankoff <mankoff <at> gmail.com>
Cc: 19145 <at> debbugs.gnu.org
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Tue, 25 Nov 2014 12:36:28 -0500

> But that would require some special adaptation because it wouldn't work
> in reverse. IOW, how do you propose to preserve the current behavior of
> the prettified symbol "delta" while also allowing for the regex "delta"?

We could allow elements of the form (REGEXP CHARACTER PREDICATE)
instead of only (STRING . CHARACTER) in p-s-alist.


        Stefan

Message #38 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Ted Zlatanov <tzz <at> lifelogs.com>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 19145 <at> debbugs.gnu.org, Ken Mankoff <mankoff <at> gmail.com>
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Tue, 25 Nov 2014 13:53:59 -0500

On Tue, 25 Nov 2014 12:36:28 -0500 Stefan Monnier <monnier <at> IRO.UMontreal.CA> wrote: 

>> But that would require some special adaptation because it wouldn't work
>> in reverse. IOW, how do you propose to preserve the current behavior of
>> the prettified symbol "delta" while also allowing for the regex "delta"?

SM> We could allow elements of the form (REGEXP CHARACTER PREDICATE)
SM> instead of only (STRING . CHARACTER) in p-s-alist.

OK, that would work and be backwards compatible.  Do you want me to
attempt it?

Ted

Message #41 received at 19145 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Ken Mankoff <mankoff <at> gmail.com>
Cc: 19145 <at> debbugs.gnu.org
Subject: Re: bug#19145: 24.4; prettify-symbols-mode inconsistent behavior
Date: Tue, 25 Nov 2014 21:23:00 -0500

> OK, that would work and be backwards compatible.  Do you want me to
> attempt it?

Be my guest, yes,


        Stefan

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.