GNU bug report logs - #57141
29.0.50; Most emoji not displaying in PGTK emacs

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 57141 in the body.
You can then email your comments to 57141 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.50; Most emoji not displaying in PGTK emacs
Date: Thu, 11 Aug 2022 14:10:25 +0000

I use emacs under wayland (sway) with PGTK and native compilation,
compiled from master. However, I found that in this configuration, at
least in my recent builds, unicode emoji are not properly displayed.
E.g. if I call `emoji-list', most emoji are not shown at all, except
very basic emoji like "Smileys > affection." If I insert a unicode
emoji, I just see a unicode placeholder. Calling `describe-char' on the
"Smileys > smiling" emoji from the *Emoji* buffer gives gives me:

               position: 65 of 2529 (3%), column: 0
               character: C-j (displayed as C-j) (codepoint 10, #o12, #xa)
               charset: ascii (ASCII (ISO646 IRV))
   code point in charset: 0x0A
               script: latin
               syntax:   	which means: whitespace
               to input: type "C-x 8 RET a" or "C-x 8 RET LINE FEED (LF)"
           buffer code: #x0A
               file code: not encodable by coding system nil
               display: no font available

I tried using a different emoji font via customizing the emoji face,
where I set the font family to "JoyPixels", but this resulted in emacs crashing.

I could not find an existing ticket with "emoji" in the subject that
fits my symptoms, but maybe I didn't look properly or this is a symptom
of another known bug. Or I just misconfigured something on my system
which results in problems with the coding system or something that
happens due to my compilation options. Anyway, I don't know what else to
do to fix this on my own, so I would be happy for hints of if this can
be fixed somehow.

Best regards,
Michael Eliachevitch
---

In GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.34, cairo version 1.17.6)
of 2022-08-11 built on e490
Repository revision: 89f51673792b13ae0d1b93d0bf8e35d452693c9c
Repository branch: makepkg
System Description: Arch Linux

Configured using:
'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
--localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
--with-modules --without-libotf --without-m17n-flt --without-gconf
--enable-link-time-optimization --with-native-compilation
--with-xinput2 --with-pgtk --without-xaw3d --with-sound=no
--with-xwidgets --without-gpm --without-compress-install
'--program-transform-name=s/\([ec]tags\)/\1.emacs/'
'CFLAGS=-march=native -mtune=generic -O3 -pipe -fno-plt -fexceptions
-Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
-fstack-clash-protection -fcf-protection'
LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
PNG RSVG SECCOMP SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP XIM
XWIDGETS GTK3 ZLIB

Important settings:
 value of $LC_ALL: C
 value of $LC_COLLATE: C.UTF-8
 value of $LC_CTYPE: en_US.UTF-8
 value of $LC_MESSAGES: en_US.UTF-8
 value of $LC_MONETARY: de_DE.UTF-8
 value of $LC_NUMERIC: en_US.UTF-8
 value of $LC_TIME: de_DE.UTF-8
 value of $LANG: en_US.UTF-8
 value of $XMODIFIERS: @im=fcitx
 locale-coding-system: nil

Major mode: Lisp Interaction

Minor modes in effect:
 tooltip-mode: t
 global-eldoc-mode: t
 eldoc-mode: t
 show-paren-mode: t
 electric-indent-mode: t
 mouse-wheel-mode: t
 tool-bar-mode: t
 menu-bar-mode: t
 file-name-shadow-mode: t
 global-font-lock-mode: t
 font-lock-mode: t
 blink-cursor-mode: t
 line-number-mode: t
 indent-tabs-mode: t
 transient-mark-mode: t
 auto-composition-mode: t
 auto-encryption-mode: t
 auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils emoji-labels
emoji generate-lisp-file multisession sqlite transient comp comp-cstr
warnings icons subr-x rx cl-seq format-spec edmacro kmacro eieio
byte-opt bytecomp byte-compile cconv eieio-core cl-macs gv cl-extra
help-mode cl-loaddefs cl-lib rmc iso-transl tooltip eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
faces cus-face macroexp files window text-properties overlay sha1 md5
base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads xwidget-internal dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
gtk pgtk lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 118945 18632)
(symbols 48 21096 1)
(strings 32 102059 1853)
(string-bytes 1 2486759)
(vectors 16 26117)
(vector-slots 8 1148486 18229)
(floats 8 39 31)
(intervals 56 744 0)
(buffers 992 13))

--
Michael Eliachevitch
Public PGP Key: https://keyoxide.org/hkp/546908c782383ad0e7d894ec1b8f95c8125dce31

Message #8 received at 57141 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
Cc: 57141 <at> debbugs.gnu.org
Subject: Re: bug#57141: 29.0.50; Most emoji not displaying in PGTK emacs
Date: Thu, 11 Aug 2022 19:06:49 +0300

> From: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
> Date: Thu, 11 Aug 2022 14:10:25 +0000
> 
> I use emacs under wayland (sway) with PGTK and native compilation,
> compiled from master. However, I found that in this configuration, at
> least in my recent builds, unicode emoji are not properly displayed.
> E.g. if I call `emoji-list', most emoji are not shown at all, except
> very basic emoji like "Smileys > affection." If I insert a unicode
> emoji, I just see a unicode placeholder. Calling `describe-char' on the
> "Smileys > smiling" emoji from the *Emoji* buffer gives gives me:

Could this be bug#57066?  Does the patch posted there fix the problem?

>                 position: 65 of 2529 (3%), column: 0
>                 character: C-j (displayed as C-j) (codepoint 10, #o12, #xa)
>                 charset: ascii (ASCII (ISO646 IRV))
>     code point in charset: 0x0A
>                 script: latin
>                 syntax:   	which means: whitespace
>                 to input: type "C-x 8 RET a" or "C-x 8 RET LINE FEED (LF)"
>             buffer code: #x0A
>                 file code: not encodable by coding system nil
>                 display: no font available

This is not an Emoji character, this is NBSP.

Do you have Color Note Emoji font installed?

Message #11 received at 57141 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
Cc: 57141 <at> debbugs.gnu.org
Subject: Re: bug#57141: 29.0.50; Most emoji not displaying in PGTK emacs
Date: Thu, 11 Aug 2022 21:32:01 +0300

> From: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
> Cc: 57141 <at> debbugs.gnu.org
> Date: Thu, 11 Aug 2022 17:13:32 +0000
> 
>                 position: 523 of 1964 (27%), restriction: <103-1965>, column: 0
>                 character: 😊 (displayed as 😊) (codepoint 128522, #o373012, #x1f60a)
>                 charset: unicode (Unicode (ISO10646))
>     code point in charset: 0x1F60A
>                 script: emoji
>                 syntax: w 	which means: word
>                 category: .:Base
>                 to input: type "C-x 8 RET 1f60a" or "C-x 8 RET SMILING FACE WITH SMILING EYES"
>             buffer code: #xF0 #x9F #x98 #x8A
>                 file code: #xF0 #x9F #x98 #x8A (encoded by coding system utf-8-unix)
>                 display: no font available
                           ^^^^^^^^^^^^^^^^^
This means Emacs cannot find a font for this character.  You need to
figure out why.

> > Do you have Color Note Emoji font installed?
> 
>  "Noto Color Emoji" is in the output of '(font-family-list) ` and it's also installed via my system package manager. Weirdly, if try to change the frame font to "Noto Color Emoji", which I did via the menu-bar "options->set default font", I get the error message
> 
>     set-frame-font: Font not available: #<font-spec nil nil Noto\ Color\ Emoji nil nil normal normal nil 10.0 nil nil nil nil>

This again means Emacs cannot use that font, for some reason.

Message #14 received at 57141 <at> debbugs.gnu.org (full text, mbox):

From: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 57141 <at> debbugs.gnu.org
Subject: Re: bug#57141: 29.0.50; Most emoji not displaying in PGTK emacs
Date: Thu, 11 Aug 2022 17:13:32 +0000

[Message part 1 (text/plain, inline)]

> This is not an Emoji character, this is NBSP.

Upps, my bad, obviously, seems that `(emoji-list)' skips those unrenderable emoji.
For describe-char on "😊", I get

--8<---------------cut here---------------start------------->8---
               position: 523 of 1964 (27%), restriction: <103-1965>, column: 0
               character: 😊 (displayed as 😊) (codepoint 128522, #o373012, #x1f60a)
               charset: unicode (Unicode (ISO10646))
   code point in charset: 0x1F60A
               script: emoji
               syntax: w 	which means: word
               category: .:Base
               to input: type "C-x 8 RET 1f60a" or "C-x 8 RET SMILING FACE WITH SMILING EYES"
           buffer code: #xF0 #x9F #x98 #x8A
               file code: #xF0 #x9F #x98 #x8A (encoded by coding system utf-8-unix)
               display: no font available

   Character code properties: customize what to show
   name: SMILING FACE WITH SMILING EYES
   general-category: So (Symbol, Other)
   decomposition: (128522) ('😊')

   There are text properties here:
   fontified            t
   wrap-prefix          ""
   ws-butler-chg        chg
--8<---------------cut here---------------end--------------->8---

I now also attached some screenshots of the emoji-list and also of the completion-buffer when I press <tab> in emoji-search, which shows the unicode placeholders that I get, so it might be more clear what my problem is.

> Do you have Color Note Emoji font installed?

"Noto Color Emoji" is in the output of '(font-family-list) ` and it's also installed via my system package manager. Weirdly, if try to change the frame font to "Noto Color Emoji", which I did via the menu-bar "options->set default font", I get the error message

   set-frame-font: Font not available: #<font-spec nil nil Noto\ Color\ Emoji nil nil normal normal nil 10.0 nil nil nil nil>

I get this for all other emoji fonts that I have installed, but maybe this is normal for an emoji font because setting it as font for the entire frame doesn't make much sense, but I don't know if this is not a hint of an underlying font-config issue.

> Could this be bug#57066?  Does the patch posted there fix the problem?

I agree this is likely to be an issue with harfbuzz, even though the symptoms of this bug look different from bug#57066, as I just don't get any rendered emoji instead of overlapping emoji (see attached screenshots). I will try this patch anyway after this mail.

I also saw bug#56789 which seemed similar in symptoms, but I just checked that my archlinux installation has harfbuzz 5.1.0, where the bug causing #56789 is should be fixed as far as I understood. I tried a clean rebuild to make sure I use that harfbuzz version and also I tried downgrading harfbuzz to 4.4.1 and rebuilding emacs and neither solved my rendering issues. I also tried the command

   hb-view  --output-file=foo.svg --font-size=13 \
   /usr/share/fonts/noto/NotoColorEmoji.ttf \
   -u 1f469,200d,2764,fe0f,200d,1f468

and it works, producing a nice SVG emoji image. So it's not that harfbuzz in general is broken.

Thanks for the help,
Michael

[emoji-list.png (image/png, attachment)]

[emoji-completion.png (image/png, attachment)]

Message #17 received at 57141 <at> debbugs.gnu.org (full text, mbox):

From: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 57141 <at> debbugs.gnu.org
Subject: Re: bug#57141: 29.0.50; Most emoji not displaying in PGTK emacs
Date: Thu, 11 Aug 2022 18:54:29 +0000

[Message part 1 (text/plain, inline)]

I figured out this issue, it was due to my custom font configuration in ~/.config/fontconfig/fonts.conf'. When deleting that file, emojis display and I can also set the face- and frame-fonts to emoji fonts. Probably it's just broken, but I think it's still possible that it's formally correct, but the hinting and aliasing rules that it specifies cause some bug down the line. Feel free to close this, though if anyone is interested, I attached the file.

I also tried applying the suggested patch from bug#57066, but saw no difference: Without the custom fonts.conf the emoji display works well whether the patch is applied or not . With the custom fonts.conf the patch doesn't fix the issue.

Thanks so much for the help Eli! I'm glad to have this finally solved as I'm seeing more and more emacs users and packages starting to use unicode emoji.

Cheers, Michael

[fonts.conf (text/plain, attachment)]

Message #22 received at 57141-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
Cc: 57141-done <at> debbugs.gnu.org
Subject: Re: bug#57141: 29.0.50; Most emoji not displaying in PGTK emacs
Date: Fri, 12 Aug 2022 08:38:11 +0300

> From: Michael Eliachevitch <m.eliachevitch <at> posteo.de>
> Cc: 57141 <at> debbugs.gnu.org
> Date: Thu, 11 Aug 2022 18:54:29 +0000
> 
> I figured out this issue, it was due to my custom font configuration in ~/.config/fontconfig/fonts.conf'. When deleting that file, emojis display and I can also set the face- and frame-fonts to emoji fonts. Probably it's just broken, but I think it's still possible that it's formally correct, but the hinting and aliasing rules that it specifies cause some bug down the line. Feel free to close this, though if anyone is interested, I attached the file.
> 
> I also tried applying the suggested patch from bug#57066, but saw no difference: Without the custom fonts.conf the emoji display works well whether the patch is applied or not . With the custom fonts.conf the patch doesn't fix the issue.
> 
> Thanks so much for the help Eli! I'm glad to have this finally solved as I'm seeing more and more emacs users and packages starting to use unicode emoji.

Thank you for telling us.  I'm therefore closing this bug.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.