From unknown Sat Jun 14 03:47:32 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#54032 <54032@debbugs.gnu.org> To: bug#54032 <54032@debbugs.gnu.org> Subject: Status: 29.0.50; Emoji display on Linux console switched to hexadecimal output Reply-To: bug#54032 <54032@debbugs.gnu.org> Date: Sat, 14 Jun 2025 10:47:32 +0000 retitle 54032 29.0.50; Emoji display on Linux console switched to hexadecim= al output reassign 54032 emacs submitter 54032 Aura Kelloniemi severity 54032 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 17 01:57:59 2022 Received: (at submit) by debbugs.gnu.org; 17 Feb 2022 06:57:59 +0000 Received: from localhost ([127.0.0.1]:49936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nKajm-0006Cb-Li for submit@debbugs.gnu.org; Thu, 17 Feb 2022 01:57:59 -0500 Received: from lists.gnu.org ([209.51.188.17]:60710) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nKajl-0006CU-68 for submit@debbugs.gnu.org; Thu, 17 Feb 2022 01:57:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:34162) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nKajk-0007Wd-TF for bug-gnu-emacs@gnu.org; Thu, 17 Feb 2022 01:57:56 -0500 Received: from smtp.sange.fi ([185.87.108.151]:51595) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nKaji-0005bD-1t for bug-gnu-emacs@gnu.org; Thu, 17 Feb 2022 01:57:56 -0500 Received: from 88-114-110-130.elisa-laajakaista.fi ([88.114.110.130] helo=solaria) by oiva.sange.fi with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1nKajX-00012v-G4 for bug-gnu-emacs@gnu.org; Thu, 17 Feb 2022 08:57:45 +0200 From: Aura Kelloniemi To: bug-gnu-emacs@gnu.org Subject: 29.0.50; Emoji display on Linux console switched to hexadecimal output Date: Thu, 17 Feb 2022 08:57:40 +0200 Message-ID: <87h78y56aj.fsf@sange.fi> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.7 (--) Received-SPF: pass client-ip=185.87.108.151; envelope-from=kaura.dev@sange.fi; helo=smtp.sange.fi X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.7 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.7 (--) Hello, on recent Emacs development repository builds, emoji characters are no more displayed on Linux console. Instead Emacs prints \UABCDEF hexadecimal codes. This is due to the commit 10c680551e899805a6de7360e9b65986fd87df72 which probably makes things better on some terminals. Reverting this commit fixed the issue for me, and emojis are again displayed as usual. Linux console is (sort of) capable of displaying emojis. Console font can be configured so that it has glyphs for emojis exactly the same way as for oth= er characters. Also, blind users using refreshable braile displays use Linux console to access Emacs. The braille terminal driver is able to detect the correct character code points even when Linux itself is not able to display them properly on the screen. This detection is done using the /dev/vcsu (virtual console screen unicode) character devices. For these reasons it is important that emacs outputs the real characters to the terminal on Linux console. Linux console has a terminal type string of "linux". lisp/term/linux.el contains already some Linux terminal specific code (which unfortunately assumes though that Linux has a default character set of Latin-1, which has never been true). My preferred solution to this problem would be to add and document a way to configure character display logic on TTYs more precisely. It would be great= to be able to control the terminal output of Unicode on grapheme cluster precision =E2=80=93 i.e. allow the user to define a function which translat= es code points/grapheme clusters into something that their terminal can display. I believe that Linux console is not the only terminal that behaves peculiar= ly when it comes to Unicode support. So this might benefit others than Linux VT users too. --=20 Aura In GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.31, c= airo version 1.17.4) of 2022-02-16 built on solaria Repository revision: e6e723bb4d300e6ceeeb12bf43bf3d54a6108cac Repository branch: makepkg System Description: Arch Linux Configured using: 'configure --prefix=3D/usr --sysconfdir=3D/etc --libexecdir=3D/usr/lib --localstatedir=3D/var --with-native-compilation --with-x-toolkit=3Dgtk3 --with-xft --with-wide-int --with-modules --with-gameuser=3D:games --with-sound=3Dalsa --with-cairo --with-harfbuzz --enable-link-time-optimization 'CFLAGS=3D-march=3Dnative -mtune=3Dnative = -O2 -pipe -fno-plt -fuse-ld=3Dgold -flto' CPPFLAGS=3D-D_FORTIFY_SOURCE=3D2 LDFLAGS=3D-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now' Configured features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP X11 XDBE XIM XPM GTK3 ZLIB Important settings: value of $LANG: fi_FI.UTF-8 locale-coding-system: utf-8-unix Major mode: Fundamental Minor modes in effect: telega-root-auto-fill-mode: t telega-active-locations-mode: t telega-patrons-mode: t gpm-mouse-mode: t leaf-key-override-global-mode: t shell-dirtrack-mode: t savehist-mode: t minibuffer-electric-default-mode: t icomplete-mode: t tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: linux auto-encryption-mode: t auto-compression-mode: t column-number-mode: t line-number-mode: t Load-path shadows: /home/aura/.config/emacs/elpa/transient-20220130.1941/transient hides /usr/= share/emacs/29.0.50/lisp/transient Features: (shadow sort company-oddmuse company-keywords company-etags etags fileloop xref project company-gtags company-dabbrev-code company-dabbrev company-fil= es company-clang company-capf company-cmake company-semantic company-template company-bbdb mail-extr textsec uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check shr pixel-fill kinsoku vterm bookmark face-rem= ap compile term disp-table ehelp vterm-module term/xterm xterm mule-util telega-obsolete telega telega-tdlib-events telega-webpage visual-fill-column telega-root telega-info telega-chat telega-modes image-mode exif telega-company telega-user telega-notifications notifications dbus telega-v= oip telega-msg telega-tme telega-sticker telega-i18n telega-vvnote bindat telega-ffplay telega-media telega-sort telega-filter telega-ins telega-fold= ers telega-inline telega-tdlib telega-util rainbow-identifiers org-element avl-tree generator org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint org-pcomplete org-list org-faces org-entities noutline outline org-version ob-emacs-lisp ob-core ob-eval org-table oc-bas= ic bibtex ol org-keys oc org-compat advice org-macs org-loaddefs dired-aux col= or ewoc telega-server telega-core telega-customize svg dom xml emacsbug sendma= il find-func cursor-sensor comp comp-cstr warnings rx cl-extra help-mode t-mou= se term/linux recentf tree-widget notmuch notmuch-tree notmuch-jump notmuch-he= llo notmuch-show notmuch-print notmuch-crypto notmuch-mua notmuch-message notmuch-draft notmuch-maildir-fcc notmuch-address notmuch-company notmuch-parser notmuch-wash diff-mode easy-mmode coolj notmuch-query goto-a= ddr thingatpt icalendar diary-lib diary-loaddefs cal-menu calendar cal-loaddefs notmuch-tag crm notmuch-lib notmuch-version notmuch-compat hl-line message yank-media rmc puny dired dired-loaddefs rfc822 mml mailabbrev mail-utils gmm-utils mailheader mm-view mml-smime mml-sec epa derived epg rfc6068 epg-config gnus-util text-property-search smime dig mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr company pcase server leaf-keywords leaf finder-inf package browse-url url url-proxy url-privacy url-expand url-methods url-history url-cookie url-dom= suf url-util mailcap url-handlers url-parse url-vars tramp tramp-loaddefs tramp= ver tramp-integration cus-edit pp wid-edit files-x tramp-compat shell pcomplete comint ansi-color ring parse-time iso8601 time-date ls-lisp format-spec auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache j= son map savehist minibuf-eldef keypad ido seq gv subr-x byte-opt bytecomp byte-compile cconv icomplete desktop frameset cl-loaddefs cl-lib cus-load i= nfo iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomm= ent text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-l= ang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button loadde= fs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget keymap hashtable-print-readable backquote threads dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty make-network-process native-compile emacs) Memory information: ((conses 16 899499 30380) (symbols 48 32992 10) (strings 32 222707 14412) (string-bytes 1 6442228) (vectors 16 114679) (vector-slots 8 1497371 73794) (floats 8 8158 532) (intervals 56 5228 1491) (buffers 992 15)) From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 17 02:52:22 2022 Received: (at 54032) by debbugs.gnu.org; 17 Feb 2022 07:52:22 +0000 Received: from localhost ([127.0.0.1]:50012 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nKbaP-0007cH-OY for submit@debbugs.gnu.org; Thu, 17 Feb 2022 02:52:22 -0500 Received: from eggs.gnu.org ([209.51.188.92]:36804) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nKbaO-0007c4-1t for 54032@debbugs.gnu.org; Thu, 17 Feb 2022 02:52:20 -0500 Received: from [2001:470:142:3::e] (port=44180 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nKbaF-00051F-3m; Thu, 17 Feb 2022 02:52:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=FOsGgkuZFCqN5AWZnGnOnMC+x4vqy7SPJ5DSerWvdXc=; b=mHvEfVVJqfapIAaIB4mr 4LUvth0WE/QLx+zeqyzQVIHHkLQ5zX8aO/5nEANBn04SZ3DlD9KnW3m73O1kckN2vKC2l91ApDkCB jmuNrCeL7SGX5EioxjrLeS4z4NIiudst/90V5xx5Es7xGmVZ/CRet4PEkm4AlPiJrZKJxJanl/Y3h XmAKuYb2Q3KCyevZuGYPBU8oHXI5lqhI1d54Tnun5favIng2VVm46JLpc7J47kJMpMO7/hut63Gxw q9MosnbzIHmRpzb+KNIIzT5ySSYpCZjexHSn2ezmmbuzY3RuIUKrQEWJlf+GM4zmDJ/ryAkOdfDFA Z5eXU3n0D7Rz3Q==; Received: from [87.69.77.57] (port=4134 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nKbaC-0003PR-Ui; Thu, 17 Feb 2022 02:52:10 -0500 Date: Thu, 17 Feb 2022 09:52:15 +0200 Message-Id: <83ilte0w28.fsf@gnu.org> From: Eli Zaretskii To: Aura Kelloniemi In-Reply-To: <87h78y56aj.fsf@sange.fi> (message from Aura Kelloniemi on Thu, 17 Feb 2022 08:57:40 +0200) Subject: Re: bug#54032: 29.0.50; Emoji display on Linux console switched to hexadecimal output References: <87h78y56aj.fsf@sange.fi> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54032 Cc: 54032@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Aura Kelloniemi > Date: Thu, 17 Feb 2022 08:57:40 +0200 > > on recent Emacs development repository builds, emoji characters are no more > displayed on Linux console. Instead Emacs prints \UABCDEF hexadecimal codes. > > This is due to the commit 10c680551e899805a6de7360e9b65986fd87df72 which > probably makes things better on some terminals. Reverting this commit fixed > the issue for me, and emojis are again displayed as usual. Thanks, but I think we need more detailed information to understand the problem. First, when you say "display emoji characters", which characters exactly does that allude to? Can you show specific examples of text that includes Emoji, which displayed the Emoji glyphs before the above commit, but not after it? In particular, are we talking about single codepoints in the Emoji block, or are we talking about Emoji sequences that involve more than one codepoint (and are supposed to display like a single Emoji glyph)? For each example, please show both the text and what you see on display for that text. Also, what is the value of auto-composition-mode? I think it should be the string "linux", in which case please try setting it to t and see if the display becomes better or worse. > Linux console is (sort of) capable of displaying emojis. Console font can be > configured so that it has glyphs for emojis exactly the same way as for other > characters. If that is the case, why did Emacs think the terminal cannot display these characters? Can you step with GDB inside terminal_glyph_code, when it is called for the first time in the Emacs session, and see whether the ioctl call we issue in calculate_glyph_code_table returns valid values for the Emoji codepoints? > Also, blind users using refreshable braile displays use Linux > console to access Emacs. The braille terminal driver is able to detect the > correct character code points even when Linux itself is not able to display > them properly on the screen. This detection is done using the /dev/vcsu > (virtual console screen unicode) character devices. Are you saying that the braille terminal driver will not respond to the ioctl call we issue in calculate_glyph_code_table? Is there any other method of knowing which characters are supported in that case? > For these reasons it is important that emacs outputs the real > characters to the terminal on Linux console. Outputting codepoints for which there are no glyphs produced unreadable display, since (AFAIU) the console displays them all as the "diamond" replacement character. Detecting the fact that a codepoint cannot be displayed allows us to produce something that at least can be interpreted, and allows the user to install optional features (such as those provided by latin1-disp.el) which will replace the characters that cannot be displayed by equivalent strings, for example ASCII strings. So we would like to keep the automatic detection of whether a given character can be displayed by the console, although it sounds like the current solution should be made more flexible and sophisticated in some way. > Linux console has a terminal type string of "linux". lisp/term/linux.el > contains already some Linux terminal specific code (which unfortunately > assumes though that Linux has a default character set of Latin-1, which has > never been true). That's just the default. We attempt to detect which characters can be displayed later on, when the functions I mentioned above are called during startup. > My preferred solution to this problem would be to add and document a way to > configure character display logic on TTYs more precisely. It would be great to > be able to control the terminal output of Unicode on grapheme cluster > precision – i.e. allow the user to define a function which translates code > points/grapheme clusters into something that their terminal can display. I'm not yet sure something like that would be needed. It will certainly slow down the display on the console, which is undesirable for obvious reasons. It is also too complex (not every Emacs user can write Lisp programs that play sophisticated games with characters and glyphs). I think we don't yet have a detailed enough understanding of the issue to discuss solutions, so I suggest to postpone this discussion until the questions I asked above are answered, and we have a good understanding of what is going on. The commit to which you point out was made based on reports from another user of the Linux console (albeit not about Emoji), and in that case the change had a positive effect. So the issue is not simple, and we need a good understanding of it before we devise a solution. Thanks.