GNU bug report logs - #10895
Quirky behaviours with Arabic text

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10895 in the body.
You can then email your comments to 10895 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: sergei karhof <karhof21 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Quirky behaviours with Arabic text
Date: Mon, 27 Feb 2012 00:47:36 +0100

I want to report some quirky behaviours of Emacs (devel 24.0.93 version,
running under Windows 7), when dealing with Arabic text:

1) in some fonts (e.g. courier New), the vowel marks (harakaat) are
not displayed, despite the fact that the texts contains them. However,
if the cursor passes over the relevant text, moving in the
right-to-left direction, the vowel marks become visible. Curiously, if
the cursor passes back on the text, moving in the left-to-right
direction, the vowel marks disappear.
This problem manifests itself in different ways according to the font used.
With other fonts, the problem is only partial: only part of the
diacritic signs is visible, and by passing over them with the cursor
(from right to left), they become fully visible. When passing over the
text in the opposite direction, they again become half-visible.

2) the rendering of the vowel marks is poor even when they are
displayed. For instance, vowel marks partially overlap some of the
letters, e.g. the kasrah sign overlaps the two dots of the letter yaa.

3) vowel marks are not always in synch with the main text, so that
they sometimes appear in a displaced position (e.g. on the previous
letter). This is totally unacceptable, of course.

Bottomline: there are multiple problems with the visual rendering of
the diacritics.

Message #8 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: sergei karhof <karhof21 <at> gmail.com>
To: 10895 <at> debbugs.gnu.org
Subject: Screenshot #1
Date: Mon, 27 Feb 2012 11:15:41 +0100

[Message part 1 (text/plain, inline)]

See attached screenshot

[Screenshot 1.jpg (image/jpeg, attachment)]

Message #11 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: sergei karhof <karhof21 <at> gmail.com>
To: 10895 <at> debbugs.gnu.org
Subject: Screenshot #2
Date: Mon, 27 Feb 2012 11:16:24 +0100

[Message part 1 (text/plain, inline)]

[Screenshot 2.jpg (image/jpeg, attachment)]

Message #14 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: sergei karhof <karhof21 <at> gmail.com>
To: 10895 <at> debbugs.gnu.org
Subject: Screenshot #3
Date: Mon, 27 Feb 2012 11:16:55 +0100

[Message part 1 (text/plain, inline)]

[Screenshot 3.jpg (image/jpeg, attachment)]

Message #17 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: sergei karhof <karhof21 <at> gmail.com>
Cc: 10895 <at> debbugs.gnu.org
Subject: Re: bug#10895: Screenshot #1
Date: Mon, 27 Feb 2012 20:12:45 +0200

> Date: Mon, 27 Feb 2012 11:15:41 +0100
> From: sergei karhof <karhof21 <at> gmail.com>
> 
> See attached screenshot

Thanks.  Can you tell which one(s) of these 3 screenshots show correct
display?

Message #20 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Kenichi Handa <handa <at> m17n.org>, jasonr <at> gnu.org
Cc: sergei karhof <karhof21 <at> gmail.com>, 10895 <at> debbugs.gnu.org
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Mon, 27 Feb 2012 20:17:49 +0200

> Date: Mon, 27 Feb 2012 00:47:36 +0100
> From: sergei karhof <karhof21 <at> gmail.com>
> 
> I want to report some quirky behaviours of Emacs (devel 24.0.93 version,
> running under Windows 7), when dealing with Arabic text:

Thanks.

Jason and Kenichi, could you please take a look at this problem?  The
first question I'd ask is whether this is specific to Windows, or do
we have similar problems on GNU/Linux with libotf?

Sergei, could you please try invoking Emacs with the
`-xrm Emacs.fontBackend:gdi' command-line argument, and see if the
problem persists with the GDI backend?  That's assuming that GDI at
all supports Arabic shaping; I don't know enough about this to tell,
sorry.

Message #23 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: sergei karhof <karhof21 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: jasonr <at> gnu.org, 10895 <at> debbugs.gnu.org, Kenichi Handa <handa <at> m17n.org>
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Tue, 28 Feb 2012 17:26:45 +0100

[Message part 1 (text/plain, inline)]

On Mon, Feb 27, 2012 at 7:17 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:

> Sergei, could you please try invoking Emacs with the
> `-xrm Emacs.fontBackend:gdi' command-line argument, and see if the
> problem persists with the GDI backend?  That's assuming that GDI at
> all supports Arabic shaping; I don't know enough about this to tell,
> sorry.

I have just tried this, but it totally messes up the text, as you can
see from screenshot #5 here attached.

[#5.jpg (image/jpeg, attachment)]

Message #26 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: sergei karhof <karhof21 <at> gmail.com>
To: 10895 <at> debbugs.gnu.org
Subject: Bug: vowel marks in Arabic
Date: Tue, 28 Feb 2012 17:21:18 +0100

[Message part 1 (text/plain, inline)]

Here are two more screenshots:

#4a shows a text exactly as it was loaded from the file. The vowel
marks are totally missing, except for the word 'Allah', which for some
reason displays the vowel marks. The shaddah mark above that word, in
fact, is displayed twice, which is an error! You can see the word
'Allah' which is right at the left of the cursor. Those two marks
(shaped like the number 3, but rotated 180° degrees) above the word
are the shaddah. The reason for this anomaly might be that is some
systems the word Allah is a glyph on its own, which includes the
shaddah mark; if that was the case, this part of the problem could be
solved by removing any additional shaddah mark that might have been
added to that glyph. Anyhow, this was a digression.

The main problem is that the vowel marks are not displayed at all in
the other words.

#4b shows the same text, after the cursor has passed over them. This
is the nearest it comes to displaying the vowel marks correctly. But
again, they are out of synch with the letters, so the whole
vocalization is wrong!

Yet another two problems that I just noticed:
* the vertical space allotted to each line of text is not enough. You
can see from #4b that part of the vowel marks is cut off and thus
missing. This problem might be solved by increasing the inter-line
space, but wouldn't be nice if the proper inter-line space was
calculated and set automatically by the system?
* in #4b you can see, about 1cm at the left of the cursor, that two
vowel marks are overlapping in the same space, and of course they
shouldn't.

[#4a.jpg (image/jpeg, attachment)]

[#4b.jpg (image/jpeg, attachment)]

Message #29 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: Jason Rumney <jasonr <at> gnu.org>
To: sergei karhof <karhof21 <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 10895 <at> debbugs.gnu.org,
	Kenichi Handa <handa <at> m17n.org>
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Wed, 29 Feb 2012 22:54:41 +0800

sergei karhof <karhof21 <at> gmail.com> writes:

> On Mon, Feb 27, 2012 at 7:17 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
>> Sergei, could you please try invoking Emacs with the
>> `-xrm Emacs.fontBackend:gdi' command-line argument, and see if the
>> problem persists with the GDI backend?  That's assuming that GDI at
>> all supports Arabic shaping; I don't know enough about this to tell,
>> sorry.
>
> I have just tried this, but it totally messes up the text, as you can
> see from screenshot #5 here attached.

I would expect that. The GDI font backend has no support for font
shaping, which is definitely required for proper Arabic support.  What I
suspect is that Emacs is doing more in terms of supporting Arabic than
the uniscribe shaping engine expects.

Message #32 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jason Rumney <jasonr <at> gnu.org>
Cc: karhof21 <at> gmail.com, 10895 <at> debbugs.gnu.org, handa <at> m17n.org
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Wed, 29 Feb 2012 20:24:31 +0200

> From: Jason Rumney <jasonr <at> gnu.org>
> Cc: Eli Zaretskii <eliz <at> gnu.org>,  Kenichi Handa <handa <at> m17n.org>,  10895 <at> debbugs.gnu.org
> Date: Wed, 29 Feb 2012 22:54:41 +0800
> 
> What I suspect is that Emacs is doing more in terms of supporting
> Arabic than the uniscribe shaping engine expects.

Like what, for example?

Does anyone know of a good treatise for Arabic shaping with Uniscribe?

Message #35 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: sergei karhof <karhof21 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: handa <at> m17n.org, 10895 <at> debbugs.gnu.org, Jason Rumney <jasonr <at> gnu.org>
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Thu, 1 Mar 2012 11:27:40 +0100

>> What I suspect is that Emacs is doing more in terms of supporting
>> Arabic than the uniscribe shaping engine expects.
>
> Like what, for example?
>
> Does anyone know of a good treatise for Arabic shaping with Uniscribe?

I don't know much about these technical matters (I am just an
end-user), but I would like to point out that as for 'shaping' the
letters (if I understand correctly what it means), it is no issue at
all, because the letters AND the vowel marks are shaped correctly. The
problem is that the vowel marks are not displayed where they should,
that they are out of synch, and that there is not enough vertical
space to contain all the vowel marks.
So I guess that the underlying rendering engine might not be the
problem; maybe it's just the way the whole thing is implemented.

sergei

Message #38 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: Jason Rumney <jasonr <at> gnu.org>
To: sergei karhof <karhof21 <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 10895 <at> debbugs.gnu.org, handa <at> m17n.org
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Thu, 01 Mar 2012 22:49:33 +0800

sergei karhof <karhof21 <at> gmail.com> writes:

> I don't know much about these technical matters (I am just an
> end-user), but I would like to point out that as for 'shaping' the
> letters (if I understand correctly what it means), it is no issue at
> all, because the letters AND the vowel marks are shaped correctly. The
> problem is that the vowel marks are not displayed where they should,
> that they are out of synch, and that there is not enough vertical
> space to contain all the vowel marks.

Shaping in this context is about shaping whole words, not individual
letters. Positioning vowel marks correctly is part of that.

Message #41 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: sergei karhof <karhof21 <at> gmail.com>
Cc: 10895 <at> debbugs.gnu.org
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Mon, 17 Aug 2020 22:34:35 +0000

sergei karhof <karhof21 <at> gmail.com> writes:

> I want to report some quirky behaviours of Emacs (devel 24.0.93 version,
> running under Windows 7), when dealing with Arabic text:
>
> 1) in some fonts (e.g. courier New), the vowel marks (harakaat) are
> not displayed, despite the fact that the texts contains them. However,
> if the cursor passes over the relevant text, moving in the
> right-to-left direction, the vowel marks become visible. Curiously, if
> the cursor passes back on the text, moving in the left-to-right
> direction, the vowel marks disappear.
> This problem manifests itself in different ways according to the font used.
> With other fonts, the problem is only partial: only part of the
> diacritic signs is visible, and by passing over them with the cursor
> (from right to left), they become fully visible. When passing over the
> text in the opposite direction, they again become half-visible.
>
> 2) the rendering of the vowel marks is poor even when they are
> displayed. For instance, vowel marks partially overlap some of the
> letters, e.g. the kasrah sign overlaps the two dots of the letter yaa.
>
> 3) vowel marks are not always in synch with the main text, so that
> they sometimes appear in a displaced position (e.g. on the previous
> letter). This is totally unacceptable, of course.
>
> Bottomline: there are multiple problems with the visual rendering of
> the diacritics.

(That was 8 years ago.)

Do you still see this on a recent version of Emacs, such as the recently
released version 27.1?

Would this have been fixed by the recent addition of harfbuzz support?

Best regards,
Stefan Kangas

Message #44 received at 10895 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: karhof21 <at> gmail.com, 10895 <at> debbugs.gnu.org
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Tue, 18 Aug 2020 07:32:53 +0300

> From: Stefan Kangas <stefan <at> marxist.se>
> Date: Mon, 17 Aug 2020 22:34:35 +0000
> Cc: 10895 <at> debbugs.gnu.org
> 
> > 1) in some fonts (e.g. courier New), the vowel marks (harakaat) are
> > not displayed, despite the fact that the texts contains them. However,
> > if the cursor passes over the relevant text, moving in the
> > right-to-left direction, the vowel marks become visible. Curiously, if
> > the cursor passes back on the text, moving in the left-to-right
> > direction, the vowel marks disappear.
> > This problem manifests itself in different ways according to the font used.
> > With other fonts, the problem is only partial: only part of the
> > diacritic signs is visible, and by passing over them with the cursor
> > (from right to left), they become fully visible. When passing over the
> > text in the opposite direction, they again become half-visible.
> >
> > 2) the rendering of the vowel marks is poor even when they are
> > displayed. For instance, vowel marks partially overlap some of the
> > letters, e.g. the kasrah sign overlaps the two dots of the letter yaa.
> >
> > 3) vowel marks are not always in synch with the main text, so that
> > they sometimes appear in a displaced position (e.g. on the previous
> > letter). This is totally unacceptable, of course.
> >
> > Bottomline: there are multiple problems with the visual rendering of
> > the diacritics.
> 
> (That was 8 years ago.)
> 
> Do you still see this on a recent version of Emacs, such as the recently
> released version 27.1?
> 
> Would this have been fixed by the recent addition of harfbuzz support?

I think those problems were fixed long ago, but let's wait for the OP
to confirm, if possible.

Message #51 received at 10895-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: karhof21 <at> gmail.com, 10895-done <at> debbugs.gnu.org
Subject: Re: bug#10895: Quirky behaviours with Arabic text
Date: Thu, 1 Oct 2020 05:26:43 -0700

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Would this have been fixed by the recent addition of harfbuzz support?
>
> I think those problems were fixed long ago, but let's wait for the OP
> to confirm, if possible.

No further updates within 6 weeks, so I guess we will have to assume
this has been fixed.  I'm therefore closing this bug now.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.