Subject: UTF-16 char display problems and the macOS "character palette"

From: Alan Third <[email-masked]>
Date: Sun, 24 Dec 2017 16:00:53 +0000

Hi, I’ve had a go at enabling the macOS character palette, which is
just a virtual keyboard that helps you to enter special characters,
emoji’s, etc.

It’s easy enough to bring it up (patch attached) but some special
characters are put into Emacs incorrectly. I think the problem is that
we have multi code‐point UTF‐16 characters, and when they are ‘typed’
into Emacs they are entered as individual 16 bit code‐points and are
therefore displayed as a series of blank spaces.

An example is '🢫' (RIGHTWARDS FRONT-TILTED SHADOWED WHITE ARROW). If I
enter it using C‐x 8 RET, it appears correctly, but if I use the
character palette it shows up as two blank spaces. Describe-char
reveals these to be HIGH SURROGATE-D83E and LOW SURROGATE-DCAB, in
that order.

I can’t work out if Emacs should be able to handle these multi
code‐point characters being entered from a ‘keyboard’ input or not. If
so, does anyone have any idea what I need to do?

(Another minor irritation is that some characters (like pointing
hands) seem to insert the desired character then follow up with
VARIATION SELECTOR-15. I assume this is supposed to tell us what
colour we want the hand? If so should it be displayed?)
-- 
Alan Third
>From ad16b98288abe91732217535e308ae445303ab59 Mon Sep 17 00:00:00 2001
From: Alan Third <[email-masked]>
Date: Sun, 24 Dec 2017 15:40:03 +0000
Subject: [PATCH] Add macOS character-palette

---
 lisp/term/ns-win.el |  8 ++++++++
 src/nsfns.m         | 14 ++++++++++++++
 src/nsterm.m        |  7 ++++++-
 3 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/lisp/term/ns-win.el b/lisp/term/ns-win.el
index d512e8e506..7955ae0cb0 100644
--- a/lisp/term/ns-win.el
+++ b/lisp/term/ns-win.el
@@ -144,6 +144,8 @@ global-map
 (define-key global-map [?\s-z] 'undo)
 (define-key global-map [?\s-|] 'shell-command-on-region)
 (define-key global-map [s-kp-bar] 'shell-command-on-region)
+;; The key-chord below is C-s-SPC
+(define-key global-map [C-s-268632064] 'ns-do-show-character-palette)
 ;; (as in Terminal.app)
 (define-key global-map [s-right] 'ns-next-frame)
 (define-key global-map [s-left] 'ns-prev-frame)
@@ -575,6 +577,12 @@ ns-do-emacs-info-panel
   (interactive)
   (ns-emacs-info-panel))
 
+(declare-function ns-show-character-palette "nsfns.m" ())
+
+(defun ns-do-show-character-palette ()
+  (interactive)
+  (ns-show-character-palette))
+
 (defun ns-next-frame ()
   "Switch to next visible frame."
   (interactive)
diff --git a/src/nsfns.m b/src/nsfns.m
index 05605bf657..402771e2f8 100644
--- a/src/nsfns.m
+++ b/src/nsfns.m
@@ -3135,6 +3135,19 @@ The position is returned as a cons cell (X . Y) of the
                            (pt.y - screen.frame.origin.y)));
 }
 
+DEFUN ("ns-show-character-palette",
+       Fns_show_character_palette,
+       Sns_show_character_palette, 0, 0, 0,
+       doc: /* Show the macOS character palette.  */)
+       (void)
+{
+  struct frame *f = SELECTED_FRAME ();
+  EmacsView *view = FRAME_NS_VIEW (f);
+  [NSApp orderFrontCharacterPalette:view];
+
+  return Qnil;
+}
+
 /* ==========================================================================
 
     Class implementations
@@ -3326,6 +3339,7 @@ - (NSString *)panel: (id)sender userEnteredFilename: (NSString *)filename
   defsubr (&Sns_frame_restack);
   defsubr (&Sns_set_mouse_absolute_pixel_position);
   defsubr (&Sns_mouse_absolute_pixel_position);
+  defsubr (&Sns_show_character_palette);
   defsubr (&Sx_display_mm_width);
   defsubr (&Sx_display_mm_height);
   defsubr (&Sx_display_screens);
diff --git a/src/nsterm.m b/src/nsterm.m
index 07ac8f978f..65a9aac4a7 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -6284,11 +6284,16 @@ flag set (this is probably a bug in the OS).
 - (void)insertText: (id)aString
 {
   int code;
-  int len = [(NSString *)aString length];
+  int len;
   int i;
 
   NSTRACE ("[EmacsView insertText:]");
 
+  if ([aString isKindOfClass:[NSAttributedString class]])
+      aString = [aString string];
+
+  len = [(NSString *)aString length];
+
   if (NS_KEYLOG)
     NSLog (@"insertText '%@'\tlen = %d", aString, len);
   processingCompose = NO;
-- 
2.14.3