GNU bug report logs - #23934
25.1.50; Incorrect rendering of cyrillic text on OS X

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 23934 in the body.
You can then email your comments to 23934 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alan Third <alan <at> idiocy.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 25.1.50; Incorrect rendering of cyrillic text on OS X
Date: Sun, 10 Jul 2016 16:45:51 +0100

[Message part 1 (text/plain, inline)]

When using the font "Menlo" Emacs displays the cyrillic text in the
hello file incorrectly. "Monaco" works fine. This affects both Mac and
NS ports.

Emacs -Q -fn Monaco

C-h h C-s russian RET

[monaco.png (image/png, attachment)]

[Message part 3 (text/plain, inline)]

Emacs -Q -fn Menlo

C-h h C-s russian RET

[menlo.png (russian, attachment)]

[Message part 5 (text/plain, inline)]

It appears that two characters are being drawn in the same space.

In GNU Emacs 25.1.50.1 (x86_64-apple-darwin15.5.0, NS appkit-1404.47 Version 10.11.5 (Build 15F34))
 of 2016-07-10 built on breton.holly.idiocy.org
Repository revision: 344eb6ee66e515c812139492990001764724e506
Windowing system distributor 'Apple', version 10.3.1404
Recent messages:
Mark set
Mark saved where search started
Contacting host: debbugs.gnu.org:80
Opening nndoc server on /var/folders/rs/0j9q5wvj5qv1h4p08j_dnbvw0000gn/T/gnus-temp-group-51168zC2-ephemeral...done
Error during redisplay: (widget-mouse-help #<window 58 on *Article nndoc+ephemeral:bug#3994*> #<overlay in no buffer> 575) signaled (wrong-type-argument stringp nil) [2 times]
mwheel-scroll: End of buffer [3 times]
mwheel-scroll: Beginning of buffer [13 times]
Mark set
previous-line: Beginning of buffer [2 times]
Making completion list...

Configured features:
JPEG RSVG NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8

Major mode: Debbugs

Minor modes in effect:
  diff-auto-refine-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  shell-dirtrack-mode: t
  global-flycheck-mode: t
  show-paren-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow emacsbug gnuplot-gui gnuplot info-look rect debug epa-file
mailalias smtpmail sendmail nnir thai-util thai-word lao-util view
flow-fill tabify vc-git cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs ibuf-ext ibuffer
ibuffer-loaddefs sh-script smie executable ob-gnuplot ob-dot ob-js
org-habit org-agenda image-file org-element org-rmail org-mhe org-irc
org-info org-gnus org-docview doc-view jka-compr image-mode org-bibtex
bibtex org-bbdb org-w3m org org-macro org-footnote org-pcomplete
org-list org-faces org-entities noutline outline org-version
ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys
ob-comint ob-core ob-eval org-compat org-macs org-loaddefs cal-menu
calendar cal-loaddefs linum magit-blame magit-stash magit-bisect
magit-remote magit-commit magit-sequence magit magit-apply magit-wip
magit-log magit-diff smerge-mode diff-mode magit-core magit-autorevert
autorevert filenotify magit-process magit-popup magit-mode magit-git
magit-section magit-utils git-commit help-fns radix-tree log-edit
pcvs-util with-editor async-bytecomp async tramp-sh tramp tramp-compat
tramp-loaddefs trampver shell pcomplete advice map misearch
multi-isearch shr-color color qp shr svg dom browse-url sort smiley
gnus-cite mail-extr gnus-async gnus-bcklg gnus-agent gnus-srvr
gnus-score score-mode nnvirtual nntp gnus-ml gnus-msg disp-table nndoc
gnus-cache gnus-dup gnus-art mm-uu mml2015 mm-view mml-smime smime dig
gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail
mail-source utf7 netrc nnoo parse-time gnus-spec gnus-int gnus-range
gnus-win gnus nnheader wid-edit mm-archive message dired dired-loaddefs
format-spec rfc822 mml mml-sec epa epg gnus-util rmail rmail-loaddefs
mailabbrev gmm-utils mailheader mail-utils network-stream starttls
url-cache crm debbugs-gnu add-log debbugs soap-client mm-decode
mm-bodies mm-encode url-http tls gnutls url-auth mail-parse rfc2231
rfc2047 rfc2045 ietf-drums url-gw nsm puny warnings rng-xsd rng-dt
rng-util xsd-regexp xml term/xterm xterm mm-util mail-prsvr unichar
exec-path-from-shell powershell-mode speedbar sb-image ezimage dframe
post derived cl abc-mode paredit go-eldoc go-mode url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util mailcap find-file ffap thingatpt etags xref project compile
comint ansi-color ring geiser flycheck find-func rx subr-x dash windmove
server edmacro kmacro paren wombat-theme use-package diminish bind-key
easy-mmode finder-inf autoinsert go-mode-autoloads info package
epg-config url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache url-vars seq byte-opt gv bytecomp
byte-compile cl-extra help-mode easymenu cconv cl-loaddefs pcase cl-lib
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/ns-win ns-win ucs-normalize term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev
obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote kqueue
cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 698126 81959)
 (symbols 48 56643 16)
 (miscs 40 5089 1029)
 (strings 32 139370 21892)
 (string-bytes 1 4623261)
 (vectors 16 89708)
 (vector-slots 8 2047057 36584)
 (floats 8 992 1205)
 (intervals 56 6204 1338)
 (buffers 976 62))

-- 
Alan Third

Message #8 received at 23934 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alan Third <alan <at> idiocy.org>
Cc: 23934 <at> debbugs.gnu.org
Subject: Re: bug#23934: 25.1.50; Incorrect rendering of cyrillic text on OS X
Date: Sun, 10 Jul 2016 19:04:40 +0300

> From: Alan Third <alan <at> idiocy.org>
> Date: Sun, 10 Jul 2016 16:45:51 +0100
> 
> When using the font "Menlo" Emacs displays the cyrillic text in the
> hello file incorrectly. "Monaco" works fine. This affects both Mac and
> NS ports.

How do we know it isn't a problem with that font?

I think the problem is triggered by displaying the u+0301 acute accent
above the а (to show the vowel with the accent).  Does Menlo has a
glyph for that codepoint?

Message #11 received at 23934 <at> debbugs.gnu.org (full text, mbox):

From: Alan Third <alan <at> idiocy.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 23934 <at> debbugs.gnu.org
Subject: Re: bug#23934: 25.1.50; Incorrect rendering of cyrillic text on OS X
Date: Sun, 10 Jul 2016 17:28:02 +0100

[Message part 1 (text/plain, inline)]

On Sun, Jul 10, 2016 at 07:04:40PM +0300, Eli Zaretskii wrote:
> > From: Alan Third <alan <at> idiocy.org>
> > Date: Sun, 10 Jul 2016 16:45:51 +0100
> > 
> > When using the font "Menlo" Emacs displays the cyrillic text in the
> > hello file incorrectly. "Monaco" works fine. This affects both Mac and
> > NS ports.
> 
> How do we know it isn't a problem with that font?
> 
> I think the problem is triggered by displaying the u+0301 acute accent
> above the а (to show the vowel with the accent).  Does Menlo has a
> glyph for that codepoint?

Yes, Emacs seems to think that the composed character takes up no
space. This may be no help whatsoever, but I've attached another image
showing a cyrillic a with acute alongside the latin version. The latin
one is not, afaik, composed, because I don't know how to do that. Both
have spaces round them.

I don't know how to check whether the font supports it.
Here's what describe-char says:

             position: 148 of 234 (63%), column: 2
            character: а (displayed as а) (codepoint 1072, #o2060, #x430)
    preferred charset: cyrillic-iso8859-5 (Right-Hand Part of ISO/IEC 8859/5 (Latin/Cyrillic): ISO-IR-144)
code point in charset: 0x50
               script: cyrillic
               syntax: w 	which means: word
             category: .:Base, L:Left-to-right (strong), Y:2-byte Cyrillic, c:Chinese, h:Korean, j:Japanese, y:Cyrillic
             to input: type "C-x 8 RET 430" or "C-x 8 RET CYRILLIC SMALL LETTER A"
          buffer code: #xD0 #xB0
            file code: #xD0 #xB0 (encoded by coding system utf-8-unix)
              display: composed to form "а́" (see below)

Composed with the following character(s) "́" using this font:
  mac-ct:-*-Menlo-normal-normal-normal-*-12-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 1 1072 870 7 0 7 7 1 nil]
  [0 1 769 646 7 2 6 10 -7 [-8 -1 -8]]

Character code properties: customize what to show
  name: CYRILLIC SMALL LETTER A
  general-category: Ll (Letter, Lowercase)
  decomposition: (1072) ('а')

There are text properties here:
  charset              cyrillic-iso8859-5
  face                 font-lock-string-face
  fontified            t

[back]

-- 
Alan Third

[Screen Shot 2016-07-10 at 17.18.31.png (image/png, attachment)]

Message #14 received at 23934 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alan Third <alan <at> idiocy.org>
Cc: 23934 <at> debbugs.gnu.org
Subject: Re: bug#23934: 25.1.50; Incorrect rendering of cyrillic text on OS X
Date: Sun, 10 Jul 2016 19:52:57 +0300

> Date: Sun, 10 Jul 2016 17:28:02 +0100
> From: Alan Third <alan <at> idiocy.org>
> Cc: 23934 <at> debbugs.gnu.org
> 
> > How do we know it isn't a problem with that font?
> > 
> > I think the problem is triggered by displaying the u+0301 acute accent
> > above the а (to show the vowel with the accent).  Does Menlo has a
> > glyph for that codepoint?
> 
> Yes, Emacs seems to think that the composed character takes up no
> space. This may be no help whatsoever, but I've attached another image
> showing a cyrillic a with acute alongside the latin version. The latin
> one is not, afaik, composed, because I don't know how to do that.

The Latin character is composed, because Emacs knows how to do that.
You can verify with "C-u C-x =".

Can you tell what does window-text-pixel-size return for the а́ part of
the text, both with Menlo and with Monaco?

> I don't know how to check whether the font supports it.
> Here's what describe-char says:
> 
>              position: 148 of 234 (63%), column: 2
>             character: а (displayed as а) (codepoint 1072, #o2060, #x430)
>     preferred charset: cyrillic-iso8859-5 (Right-Hand Part of ISO/IEC 8859/5 (Latin/Cyrillic): ISO-IR-144)
> code point in charset: 0x50
>                script: cyrillic
>                syntax: w 	which means: word
>              category: .:Base, L:Left-to-right (strong), Y:2-byte Cyrillic, c:Chinese, h:Korean, j:Japanese, y:Cyrillic
>              to input: type "C-x 8 RET 430" or "C-x 8 RET CYRILLIC SMALL LETTER A"
>           buffer code: #xD0 #xB0
>             file code: #xD0 #xB0 (encoded by coding system utf-8-unix)
>               display: composed to form "а́" (see below)
> 
> Composed with the following character(s) "́" using this font:
>   mac-ct:-*-Menlo-normal-normal-normal-*-12-*-*-*-m-0-iso10646-1
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The above line says that Menlo does have the glyph for the accent.
Emacs will not compose characters from different fonts.

> by these glyphs:
>   [0 1 1072 870 7 0 7 7 1 nil]
>   [0 1 769 646 7 2 6 10 -7 [-8 -1 -8]]

Can you show the corresponding data when you use the Monaco font?

Thanks.

Message #17 received at 23934 <at> debbugs.gnu.org (full text, mbox):

From: Alan Third <alan <at> idiocy.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 23934 <at> debbugs.gnu.org
Subject: Re: bug#23934: 25.1.50; Incorrect rendering of cyrillic text on OS X
Date: Sun, 10 Jul 2016 18:09:02 +0100

On Sun, Jul 10, 2016 at 07:52:57PM +0300, Eli Zaretskii wrote:
> > Date: Sun, 10 Jul 2016 17:28:02 +0100
> > From: Alan Third <alan <at> idiocy.org>
> > Cc: 23934 <at> debbugs.gnu.org
> > 
> > Yes, Emacs seems to think that the composed character takes up no
> > space. This may be no help whatsoever, but I've attached another image
> > showing a cyrillic a with acute alongside the latin version. The latin
> > one is not, afaik, composed, because I don't know how to do that.
> 
> The Latin character is composed, because Emacs knows how to do that.
> You can verify with "C-u C-x =".
> 
> Can you tell what does window-text-pixel-size return for the а́ part of
> the text, both with Menlo and with Monaco?

Using (window-text-pixel-size nil 3 5):

Menlo: (0 . 14)
Monaco: (7 . 16)

> > I don't know how to check whether the font supports it.
> > Here's what describe-char says:
> > 
> >              position: 148 of 234 (63%), column: 2
> >             character: а (displayed as а) (codepoint 1072, #o2060, #x430)
> >     preferred charset: cyrillic-iso8859-5 (Right-Hand Part of ISO/IEC 8859/5 (Latin/Cyrillic): ISO-IR-144)
> > code point in charset: 0x50
> >                script: cyrillic
> >                syntax: w 	which means: word
> >              category: .:Base, L:Left-to-right (strong), Y:2-byte Cyrillic, c:Chinese, h:Korean, j:Japanese, y:Cyrillic
> >              to input: type "C-x 8 RET 430" or "C-x 8 RET CYRILLIC SMALL LETTER A"
> >           buffer code: #xD0 #xB0
> >             file code: #xD0 #xB0 (encoded by coding system utf-8-unix)
> >               display: composed to form "а́" (see below)
> > 
> > Composed with the following character(s) "́" using this font:
> >   mac-ct:-*-Menlo-normal-normal-normal-*-12-*-*-*-m-0-iso10646-1
>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> The above line says that Menlo does have the glyph for the accent.
> Emacs will not compose characters from different fonts.
> 
> > by these glyphs:
> >   [0 1 1072 870 7 0 7 7 1 nil]
> >   [0 1 769 646 7 2 6 10 -7 [-8 -1 -8]]
> 
> Can you show the corresponding data when you use the Monaco font?

             position: 3 of 7 (29%), column: 2
            character: а (displayed as а) (codepoint 1072, #o2060, #x430)
    preferred charset: unicode (Unicode (ISO10646))
code point in charset: 0x0430
               script: cyrillic
               syntax: w 	which means: word
             category: .:Base, L:Left-to-right (strong), Y:2-byte Cyrillic, c:Chinese, h:Korean, j:Japanese, y:Cyrillic
             to input: type "C-x 8 RET 430" or "C-x 8 RET CYRILLIC SMALL LETTER A"
          buffer code: #xD0 #xB0
            file code: #xD0 #xB0 (encoded by coding system utf-8-unix)
              display: composed to form "а́" (see below)

Composed with the following character(s) "́" using this font:
  mac-ct:-*-Monaco-normal-normal-normal-*-12-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 1 1072 449 7 0 7 7 1 nil]
  [0 1 769 375 0 -6 -1 10 -7 nil]

Character code properties: customize what to show
  name: CYRILLIC SMALL LETTER A
  general-category: Ll (Letter, Lowercase)
  decomposition: (1072) ('а')

-- 
Alan Third

Message #22 received at 23934-done <at> debbugs.gnu.org (full text, mbox):

From: YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp>
To: Alan Third <alan <at> idiocy.org>
Cc: 23934-done <at> debbugs.gnu.org
Subject: Re: bug#23934: 25.1.50; Incorrect rendering of cyrillic text on OS X
Date: Mon, 11 Jul 2016 13:44:04 +0900

>>>>> On Sun, 10 Jul 2016 16:45:51 +0100, Alan Third <alan <at> idiocy.org> said:

> When using the font "Menlo" Emacs displays the cyrillic text in the
> hello file incorrectly. "Monaco" works fine. This affects both Mac and
> NS ports.

I've just pushed a fix to the trunk (commit
bfeda891a51178dd0032800ea4aef013c62bbe10).  Sorry, I forgot to add
Bug# in the commit log.

				     YAMAMOTO Mitsuharu
				mituharu <at> math.s.chiba-u.ac.jp

Message #25 received at 23934 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alan Third <alan <at> idiocy.org>
Cc: 23934 <at> debbugs.gnu.org
Subject: Re: bug#23934: 25.1.50; Incorrect rendering of cyrillic text on OS X
Date: Mon, 11 Jul 2016 17:40:46 +0300

> Date: Sun, 10 Jul 2016 18:09:02 +0100
> From: Alan Third <alan <at> idiocy.org>
> Cc: 23934 <at> debbugs.gnu.org
> 
> Using (window-text-pixel-size nil 3 5):
> 
> Menlo: (0 . 14)
> Monaco: (7 . 16)

It figures.

> > The above line says that Menlo does have the glyph for the accent.
> > Emacs will not compose characters from different fonts.
> > 
> > > by these glyphs:
> > >   [0 1 1072 870 7 0 7 7 1 nil]
> > >   [0 1 769 646 7 2 6 10 -7 [-8 -1 -8]]
                     ^
That "7" is IMO completely bogus.  All the fonts I tried, and also
Monaco you tried, give a zero there.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.