GNU bug report logs - #5977
24.0.50; Lao HELLO is incorrectly displayed

Previous Next

Package: emacs;

Reported by: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>

Date: Mon, 19 Apr 2010 20:51:02 UTC

Severity: normal

Done: Eli Zaretskii <eliz <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 5977 in the body.
You can then email your comments to 5977 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#5977; Package emacs. (Mon, 19 Apr 2010 20:51:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Peter Dyballa <Peter_Dyballa <at> Freenet.DE>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Mon, 19 Apr 2010 20:51:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.0.50; Lao HELLO is incorrectly displayed
Date: Mon, 19 Apr 2010 22:50:11 +0200
[Message part 1 (text/plain, inline)]
Hello!

The Lao greetings are incorrectly rendered:

[GNU Emacs Hello Lao.png (image/png, inline)]
[Message part 3 (text/plain, inline)]


The font used is:

    xft:-unknown-DejaVu Sans Mono-normal-normal-normal-*-14-*-*-*-m-0- 
iso10646-1 (#x468)

Similar effects happen with:

    xft:-unknown-Code2000-normal-normal-normal-*-17-*-*-*-*-0- 
iso10646-1 (#xA30)


When I go forward or backward with the cursor keys in the greetings  
(also in the header) the direction of cursor movement is reversed at  
two or three spots. When I try to select them to be able to copy them  
they disappear.


In GNU Emacs 24.0.50.1 (powerpc-apple-darwin9.8.0, X toolkit, Xaw3d  
scroll bars)
 of 2010-04-17 on Latsche.local
Windowing system distributor `The X.Org Foundation', version  
11.0.10800000
configured using `configure  '--without-sound' '--without-dbus' '-- 
without-pop' '--without-gconf' '--with-x-toolkit=athena' '--enable- 
locallisppath=/Library/Application Support/Emacs/calendar23:/Library/ 
Application Support/Emacs' 'CFLAGS=-g -H -Wno-pointer-sign -pipe -fPIC  
-fno-common -mcpu=7450 -mtune=7450 -faltivec -fast' 'CPPFLAGS='  
'LDFLAGS=' 'CC=gcc-4.2' 'CPP=cpp-4.2''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: de_DE.UTF-8
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: de_DE.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  view-mode: t


--
Greetings

  Pete

Time is an illusion. Lunchtime, doubly so.


Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#5977; Package emacs. (Mon, 19 Apr 2010 23:17:02 GMT) Full text and rfc822 format available.

Message #8 received at 5977 <at> debbugs.gnu.org (full text, mbox):

From: Jason Rumney <jasonr <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977 <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Tue, 20 Apr 2010 07:15:49 +0800
Peter Dyballa <Peter_Dyballa <at> Freenet.DE> writes:

> The Lao greetings are incorrectly rendered:

This seems to be the case with all the scripts that use otf to display
since the bidi code was merged.




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#5977; Package emacs. (Tue, 20 Apr 2010 09:07:02 GMT) Full text and rfc822 format available.

Message #11 received at 5977 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977 <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Tue, 20 Apr 2010 12:06:33 +0300
> From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
> Date: Mon, 19 Apr 2010 22:50:11 +0200
> Cc: 
> 
> The Lao greetings are incorrectly rendered:

Does the problem go away if you type

    M-: (setq bidi-display-reordering nil) RET

?

> When I go forward or backward with the cursor keys in the greetings  
> (also in the header) the direction of cursor movement is reversed at  
> two or three spots.

I don't see cursor movement reversal on MS-Windows.  Can you tell what
characters are those where it happens, e.g., by counting the number of
C-f keystrokes from the beginning of the line?

Anyway, the problem is known: Lao (as well as quite a few other
scripts and display features in Emacs) use character compositions,
and the bidi display does not yet handle composed characters
correctly.  I need Handa-san's help in figuring out how to make
compositions work with bidi display, because I lack the necessary
knowledge of how support for character compositions is designed and
implemented, and the code is not documented enough, at least not for
me, to figure that out on my own.




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#5977; Package emacs. (Tue, 20 Apr 2010 09:16:01 GMT) Full text and rfc822 format available.

Message #14 received at 5977 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Jason Rumney <jasonr <at> gnu.org>
Cc: Peter_Dyballa <at> Freenet.DE, 5977 <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Tue, 20 Apr 2010 12:14:48 +0300
> From: Jason Rumney <jasonr <at> gnu.org>
> Date: Tue, 20 Apr 2010 07:15:49 +0800
> Cc: 5977 <at> debbugs.gnu.org
> 
> Peter Dyballa <Peter_Dyballa <at> Freenet.DE> writes:
> 
> > The Lao greetings are incorrectly rendered:
> 
> This seems to be the case with all the scripts that use otf to display
> since the bidi code was merged.

Are you saying that you don't see similar problems on MS-Windows?
Because I do.

The Lao script does not use any R2L characters (AFAICS), so the only
issue with the bidi code here is character compositions.  Other than
that, the characters are not reordered, so cursor motion should be
strictly left to right, as usual (although jumps are expected, due to
problems with composed characters).  All this code is in terminal
independent parts of the display engine, so it should affect all GUI
ports in the same way.  The only difference between Uniscribe and
libotf should be in how they render un-composed characters, such as
u+0EB5, that the bidi redisplay incorrectly puts on the screen.

Somebody who knows how character compositions work in Emacs, please
help me to DTRT with them in the bidi display!




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#5977; Package emacs. (Tue, 20 Apr 2010 10:29:01 GMT) Full text and rfc822 format available.

Message #17 received at 5977 <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977 <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Tue, 20 Apr 2010 12:28:11 +0200
Am 20.04.2010 um 11:06 schrieb Eli Zaretskii:

>> The Lao greetings are incorrectly rendered:
>
> Does the problem go away if you type
>
>    M-: (setq bidi-display-reordering nil) RET

YES! Many displayed strings change their shape afterwards.

>
> ?
>
>> When I go forward or backward with the cursor keys in the greetings
>> (also in the header) the direction of cursor movement is reversed at
>> two or three spots.
>
> I don't see cursor movement reversal on MS-Windows.  Can you tell what
> characters are those where it happens, e.g., by counting the number of
> C-f keystrokes from the beginning of the line?


When I start at the default minor-mode at the left-most character, the  
strings first change their look. First <right> position point on the  
third character. When I now invoke C-u C-x = on this character point  
goes left to the second character. Next <right> changes again the  
shape of the strings. Next <right> warps point to fifth character,  
next <right> back to fourth, next <right> to seventh, then comes  
<SPACE>, then warp to /, <SPACE>, first character of second string.  
Next <right> positions point on second character and one more <right>  
leaps to possibly fourth one, then fifth (neighbour), then jump to  
possibly seventh, eighth, last ninth, before it falls down on next  
line right of the small depressed Ω like character (see my screenshot).

With bidi-display-reordering set to nil no change in direction of  
movement happens. (Cursor also stops at final C-j of line. Next  
<right> does the line-feed and carriage-return.)

--
Greetings

  Pete

Time flies like an error – but fruit flies like a banana!
				- (almost) Groucho Marx





Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#5977; Package emacs. (Tue, 20 Apr 2010 12:19:01 GMT) Full text and rfc822 format available.

Message #20 received at 5977 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977 <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Tue, 20 Apr 2010 15:17:05 +0300
> Cc: 5977 <at> debbugs.gnu.org
> From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
> Date: Tue, 20 Apr 2010 12:28:11 +0200
> 
> When I start at the default minor-mode at the left-most character, the  
> strings first change their look. First <right> position point on the  
> third character. When I now invoke C-u C-x = on this character point  
> goes left to the second character. Next <right> changes again the  
> shape of the strings. Next <right> warps point to fifth character,  
> next <right> back to fourth, next <right> to seventh, then comes  
> <SPACE>, then warp to /, <SPACE>, first character of second string.  
> Next <right> positions point on second character and one more <right>  
> leaps to possibly fourth one, then fifth (neighbour), then jump to  
> possibly seventh, eighth, last ninth, before it falls down on next  
> line right of the small depressed Ω like character (see my screenshot).

Strange, I see none of this on MS-Windows.  Cursor positioning is also
device-independent code, so I'm probably missing something.





Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#5977; Package emacs. (Tue, 20 Apr 2010 23:17:01 GMT) Full text and rfc822 format available.

Message #23 received at 5977 <at> debbugs.gnu.org (full text, mbox):

From: Jason Rumney <jasonr <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Peter_Dyballa <at> Freenet.DE, 5977 <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Wed, 21 Apr 2010 07:16:27 +0800
Eli Zaretskii <eliz <at> gnu.org> writes:

> Are you saying that you don't see similar problems on MS-Windows?
> Because I do.

I haven't tried on Windows.  But on Linux, all the Indic scripts are
displaying incorrectly, some showing on two lines instead of one, and
cursor movement through them makes the display change - probably an
indication that Emacs thinks it is displaying something different than
what is actually displayed at that point.

> The Lao script does not use any R2L characters (AFAICS), so the only
> issue with the bidi code here is character compositions.

Yes, composition might be the common factor here, though I didn't see
any problem with Thai, which also uses composition but not
libotf/uniscribe.  I did not mean that it had anything to do with R2L,
just that the bidi merge was the point in time that this broke.





Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs <at> gnu.org:
bug#5977; Package emacs. (Wed, 21 Apr 2010 02:32:01 GMT) Full text and rfc822 format available.

Message #26 received at 5977 <at> debbugs.gnu.org (full text, mbox):

From: Kenichi Handa <handa <at> m17n.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Peter_Dyballa <at> Freenet.DE, 5977 <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Wed, 21 Apr 2010 11:32:58 +0900
Sorry for the late response on this matter.

In article <83fx2q5w86.fsf <at> gnu.org>, Eli Zaretskii <eliz <at> gnu.org> writes:

> Anyway, the problem is known: Lao (as well as quite a few other
> scripts and display features in Emacs) use character compositions,
> and the bidi display does not yet handle composed characters
> correctly.  I need Handa-san's help in figuring out how to make
> compositions work with bidi display, because I lack the necessary
> knowledge of how support for character compositions is designed and
> implemented, and the code is not documented enough, at least not for
> me, to figure that out on my own.

I've been using the branch for 23.2.  I've just build the
trunk code on GNU/Linus, and found that all characters
displayed by composition are incorrect.  But, at the moment,
I don't have a time to work on the trunk.

Here's a brief explanation about control flow.

At first, composition_compute_stop_pos is called in
compute_stop_pos and reseat_to_string to record where to
stop for composition handling in this member
     struct composition_it cmp_it;
of struct it.

Next, next_element_from_string and next_element_from_buffer
calls the macro CHAR_COMPOSED_P to check if the next element
should be composed.  CHAR_COMPOSED_P calls
composition_reseat_it which is the function to compose
character(s) and build a LGSTRING (lispy glyph string) that
carries all information about how to display that character
sequence (glyph-ids of a font, relative position, etc).
When a LGSTRING is built, it's cached and the ID of the
cached data is recorded in cmp_it (see above).

If composition_reseat_it successfully built a LGSTRING,
next_element_from_string and next_element_from_buffer call
next_element_from_composition.

next_element_from_composition sets it->what to
IT_COMPOSITION and setups it->cmp_it so that
x_draw_composite_glyph_string_foreground (called in
x_draw_glyph_string) can draw actual composition glyph(s).

---
Kenichi Handa
handa <at> m17n.org





Reply sent to Eli Zaretskii <eliz <at> gnu.org>:
You have taken responsibility. (Fri, 23 Apr 2010 18:32:01 GMT) Full text and rfc822 format available.

Notification sent to Peter Dyballa <Peter_Dyballa <at> Freenet.DE>:
bug acknowledged by developer. (Fri, 23 Apr 2010 18:32:02 GMT) Full text and rfc822 format available.

Message #31 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Fri, 23 Apr 2010 21:31:31 +0300
> Cc: 5977 <at> debbugs.gnu.org
> From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
> Date: Tue, 20 Apr 2010 12:28:11 +0200
> 
> 
> Am 20.04.2010 um 11:06 schrieb Eli Zaretskii:
> 
> >> The Lao greetings are incorrectly rendered:
> >
> > Does the problem go away if you type
> >
> >    M-: (setq bidi-display-reordering nil) RET
> 
> YES! Many displayed strings change their shape afterwards.

I think I fixed this (revno 100013), please see if the problem is
solved for you.

(The lines in HELLO for Arabic and Hebrew still display slightly
incorrectly, but that's expected, since the fixed code only supports
strict L2R scripts.  Doing that for bidirectional scripts is on my
TODO.)




Message #32 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Sat, 24 Apr 2010 12:03:22 +0200
[Message part 1 (text/plain, inline)]
Am 23.04.2010 um 20:31 schrieb Eli Zaretskii:

> I think I fixed this (revno 100013), please see if the problem is
> solved for you.


The GNU Emacs 24.0.50 with this fix still has problems with HELLO  
(Bzr-100013 in mode-line). The Lao greetings are different depending  
on whether the text cursor (point) is outside

[GNU Emacs Hello Lao outside.png (image/png, inline)]
[Message part 3 (text/plain, inline)]


or inside the text strings:

[GNU Emacs Hello Lao inside.png (image/png, inline)]
[Message part 5 (text/plain, inline)]


The second character in the two right words counts two, because when I  
start to press <RIGHT> I have to press it twice on this character to  
reach the third one, and from this it takes one <LEFT> to reach the  
second character and two <LEFT> to leave it, reaching the word  
beginning. At the end of the visible characters the cursor jumps to a  
completely unexpected position: early in the session it was the end of  
the Braille "Hello" (with column-number-mode enabled I still saw line  
#44 and column #48) greeting, which later became visibly one line  
above (line ending character of the Bengali greeting) while mode-line  
still showed (44,48). <UP> and <DOWN> put the text cursor on the Khmer  
or the Malayalam line endings. C-e somewhere in the Lao greetings puts  
the cursor to Bengali or Braille line ends – and C-a there puts the  
cursor on column #0 of line #44, i.e., the L of Lao.

When I click on the last but one character of the Lao greetings (44,  
46) the second Thai greeting changes its shape... (normal first,  
clicked last)

[GNU Emacs Hello Thai normal.png (image/png, inline)]
[Message part 7 (text/plain, inline)]

[GNU Emacs Hello Thai clicked.png (image/png, inline)]
[Message part 9 (text/plain, inline)]



The Arabic and Hebrew comments next to the Latin/English words  
"Arabic" resp. "Hebrew" have two ")" and their greetings are not lined  
up in a second (or right) column but touch the comments in parentheses.

Clicking into the Braille "Hello" changes the Burmese greeting below  
and the Burmese comment visibly... (normal first, clicked last)

[GNU Emacs Hello Burmese normal.png (image/png, inline)]
[Message part 11 (text/plain, inline)]

[GNU Emacs Hello Burmese clicked.png (image/png, inline)]
[Message part 13 (text/plain, inline)]


The XFT font frontend is used. Should I bug-report new Thai, Burmese,  
and Arabic/Hebrew HELLO bugs?

--
Greetings
                                 <]
  Pete       o        __o         |__    o           recumbo
    ___o    /I       -\<,         |o \  -\),-%       ergo sum!
___/\ /\___./ \___...O/ O____.....`-O-'-()--o_________________


Message #33 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Sat, 24 Apr 2010 13:36:29 +0300
> Cc: 5977-done <at> debbugs.gnu.org
> From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
> Date: Sat, 24 Apr 2010 12:03:22 +0200
> 
> 
> Am 23.04.2010 um 20:31 schrieb Eli Zaretskii:
> 
> > I think I fixed this (revno 100013), please see if the problem is
> > solved for you.
> 
> 
> The GNU Emacs 24.0.50 with this fix still has problems with HELLO  
> (Bzr-100013 in mode-line). The Lao greetings are different depending  
> on whether the text cursor (point) is outside or inside the text strings:

How did you place the cursor there?  If by clicking the mouse, please
see whether the problem disappears if you move cursor there by
keyboard (C-f etc.).

In general, cursor motion problems with composed characters are a
different issue altogether.  However, on MS-Windows this particular
problem does not exist if I move cursor by keyboard commands.  And if
I move cursor by clicking the mouse, I can only change the shape of a
different character, the 6th one in the first word of the Lao
greeting.

> At the end of the visible characters the cursor jumps to a
> completely unexpected position: early in the session it was the end of
> the Braille "Hello" (with column-number-mode enabled I still saw line
> #44 and column #48) greeting, which later became visibly one line
> above (line ending character of the Bengali greeting) while mode-line
> still showed (44,48).

AFAICS, <RIGHT> at the last character on a line positions cursor at the
end of the first line displayed in the window.  This is part of the
cursor motion problems with composed characters.  This part _is_
visible on MS-Windows, and I'm working on it as we speak.

> When I click on the last but one character of the Lao greetings (44,
> 46) the second Thai greeting changes its shape...

Yes, I see that as well.

> The Arabic and Hebrew comments next to the Latin/English words  
> "Arabic" resp. "Hebrew" have two ")" and their greetings are not lined  
> up in a second (or right) column but touch the comments in parentheses.

That's yet another problem, which I actually mentioned in my message:
character composition does not yet work for text reordered for
display.  Out of all HELLO scripts, only Arabic and Hebrew are
reordered, and they use character compositions because the invisible
RLM character should be composed with the opening parenthesis that
follows it.

I will fix this eventually, but it is more urgent to fix problems in
L2R scripts that don't need reordering, because these are used by the
absolute majority of users.

> Clicking into the Braille "Hello" changes the Burmese greeting below  
> and the Burmese comment visibly... (normal first, clicked last)

The two images you sent display identically on my system, and I myself
cannot try this because I don't have a Burmese font installed (are
there good ones that are free?).

> Should I bug-report new Thai, Burmese, and Arabic/Hebrew HELLO bugs?

The bugs are not new.  You can file the bugs, but if you do, please
file 3 separate reports:

  . character composition doesn't work with bidi-reordered text
  . mouse clicking HELLO changes some characters
  . cursor motion incorrect at end of line that ends in a composite
    character

Thanks.




Message #34 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Sat, 24 Apr 2010 13:44:33 +0200
Am 24.04.2010 um 12:36 schrieb Eli Zaretskii:

>>
>> The GNU Emacs 24.0.50 with this fix still has problems with HELLO
>> (Bzr-100013 in mode-line). The Lao greetings are different depending
>> on whether the text cursor (point) is outside or inside the text  
>> strings:
>
> How did you place the cursor there?  If by clicking the mouse, please
> see whether the problem disappears if you move cursor there by
> keyboard (C-f etc.).

I did by clicking with the mouse cursor. By text cursor movement no  
changes appear.

When I later click into one of the Lao greeting words, then the shape  
changes and changes back and it also happens that other characters can  
"swallow" cursor key movements. For example, when I click on (44, 34)  
than (44, 36) swallows events.

>
>> At the end of the visible characters the cursor jumps to a
>> completely unexpected position: early in the session it was the end  
>> of
>> the Braille "Hello" (with column-number-mode enabled I still saw line
>> #44 and column #48) greeting, which later became visibly one line
>> above (line ending character of the Bengali greeting) while mode-line
>> still showed (44,48).
>
> AFAICS, <RIGHT> at the last character on a line positions cursor at  
> the
> end of the first line displayed in the window.  This is part of the
> cursor motion problems with composed characters.  This part _is_
> visible on MS-Windows, and I'm working on it as we speak.

This effect is different in my X client. At the beginning of each line  
I can type C-e followed by <RIGHT>, and the text cursor goes to the  
next line below and is in column #0. This is what I expect. And I've  
never seen (most probably) different behaviour, which I would classify  
as a bug. From the Lao line C-e first leads to the end of the Braille  
line, later to the end of the Bengali line. Next <RIGHT> leads to M of  
Malayalam.


>> Clicking into the Braille "Hello" changes the Burmese greeting below
>> and the Burmese comment visibly... (normal first, clicked last)
>
> The two images you sent display identically on my system,

Yes, it's not easy to see, but it's there! And it does not happen  
every time...
The differences are slightly more space in the comment string between  
(22, 11) and (22, 12) and the greeting string between (22, 33) and  
(22, 34).

> and I myself cannot try this because I don't have a Burmese font  
> installed (are
> there good ones that are free?).

Yes, there are a few Unicode encoded. Besides UniBurma I have Sun- 
ExtA, Padauk (bold and regular), and of course Code2000! For me GNU  
Emacs chose
    xft:-unknown-UniBurma-normal-normal-normal-*-13-*-*-*-*-0- 
iso10646-1 (#x16)

which is both a TT and OT font installed (as the latter) in Mac OS X  
and provided to X11 via X server (mkfontscale/mkfontdir, xlsfonts) and  
libfontconfig.

Do you know the "Gallery of Unicode Fonts" at http://www.wazu.jp/? The  
"Unicode Font Guide For Free/Libre Open Source Operating Systems" at http://www.unifont.org/fontguide/ 
 might offers some resources.


Later I'll have the NS (Mac OS X/Cocoa/Aqua client) variant compiled  
to see how HELLO there works... And maybe also time to report (most  
likely not).

>
>> Should I bug-report new Thai, Burmese, and Arabic/Hebrew HELLO bugs?
>
> The bugs are not new.  You can file the bugs, but if you do, please
> file 3 separate reports:
>
>  . character composition doesn't work with bidi-reordered text
>  . mouse clicking HELLO changes some characters
>  . cursor motion incorrect at end of line that ends in a composite
>    character

When they're already known then I'll keep silent.

--
Greetings

  Pete

If we don't succeed, we run the risk of failure.
				– George W. Bush





Message #35 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Sat, 24 Apr 2010 17:34:17 +0200
Am 23.04.2010 um 20:31 schrieb Eli Zaretskii:

> I think I fixed this (revno 100013), please see if the problem is
> solved for you.


The NS variant, although rendering really beautifully, shows with Lao  
the same effects:

 • entering the string by cursor movement no change in shape
 • entering by clicking with mouse or trackpad leads to shape changes
 • entering by clicking with mouse or trackpad "creates" the same  
characters which swallow cursor movement
 • C-e somewhere on Lao line sends point to empty line over title  
"LANGUAGE (NATIVE NAME)...."
 • clicking on the last but one character of the Lao greetings the  
second Thai greeting changes its shape

Arabic and Hebrew greetings are well positioned, but right of ")" an  
empty box is displayed on Arabic line, representing RIGHT-TO-LEFT MARK  
at U+200F, and "{" on Hebrew line.

Clicking into the Braille "Hello" does not change the Burmese greeting  
below and the Burmese comment visibly but change the empty box on the  
Arabic line into "n" *only* when I click on a new/different than  
before character... The exactly same effect when clicking onto *any*  
character!

--
Greetings

  Pete

When in doubt, use brute force.
				– Ken Thompson





Message #36 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Sat, 24 Apr 2010 19:01:05 +0300
> Cc: 5977-done <at> debbugs.gnu.org
> From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
> Date: Sat, 24 Apr 2010 17:34:17 +0200
> 
> 
> Am 23.04.2010 um 20:31 schrieb Eli Zaretskii:
> 
> > I think I fixed this (revno 100013), please see if the problem is
> > solved for you.
> 
> 
> The NS variant, although rendering really beautifully, shows with Lao  
> the same effects:
> 
>   • entering the string by cursor movement no change in shape
>   • entering by clicking with mouse or trackpad leads to shape changes
>   • entering by clicking with mouse or trackpad "creates" the same  
> characters which swallow cursor movement
>   • C-e somewhere on Lao line sends point to empty line over title  
> "LANGUAGE (NATIVE NAME)...."

I don't see this effect on MS-Windows.  Can you try with revno 100027
or later?

> Clicking into the Braille "Hello" does not change the Burmese greeting  
> below and the Burmese comment visibly but change the empty box on the  
> Arabic line into "n" *only* when I click on a new/different than  
> before character... The exactly same effect when clicking onto *any*  
> character!

I don't see this, either.  Again, please try with revno 100027 or
later, if you haven't already.

Thanks.





Message #37 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Sun, 25 Apr 2010 01:24:22 +0200
Am 24.04.2010 um 18:01 schrieb Eli Zaretskii:

>> The NS variant, although rendering really beautifully, shows with Lao
>> the same effects:
>>
>>  • entering the string by cursor movement no change in shape
>>  • entering by clicking with mouse or trackpad leads to shape changes
>>  • entering by clicking with mouse or trackpad "creates" the same
>> characters which swallow cursor movement
>>  • C-e somewhere on Lao line sends point to empty line over title
>> "LANGUAGE (NATIVE NAME)...."
>
> I don't see this effect on MS-Windows.  Can you try with revno 100027
> or later?

The last effect is gone in the updated NS variant.

>
>> Clicking into the Braille "Hello" does not change the Burmese  
>> greeting
>> below and the Burmese comment visibly but change the empty box on the
>> Arabic line into "n" *only* when I click on a new/different than
>> before character... The exactly same effect when clicking onto *any*
>> character!
>
> I don't see this, either.  Again, please try with revno 100027 or
> later, if you haven't already.

This is fixed in the NS variant. Now three new things appeared:

 • pressing (possibly) anywhere in the table before the  
"space" (between (60, 43) and (60, 44)) in the second Thai greeting  
changes the shape of the text right of the slash: it becomes more like  
one word
 • Arabic and Bengali line can merge to one line
 • when clicking on Arabic characters (19, 14), (19, 41), (7, 35)  
create effects like merging of lines or merging of columns
 • clicking onto characters from South Asia or South East Asia  
anywhere (head or table) changes nearby characters

The X client is compiling...

--
Greetings

  Pete

Mac OS X is like a wigwam: no fences, no gates, but an apache inside.





Message #38 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Sun, 25 Apr 2010 11:01:35 +0200
Am 24.04.2010 um 18:01 schrieb Eli Zaretskii:

>> The NS variant, although rendering really beautifully, shows with Lao
>> the same effects:
>>
>>  • entering the string by cursor movement no change in shape
>>  • entering by clicking with mouse or trackpad leads to shape  
>> changes
>>  • entering by clicking with mouse or trackpad "creates" the same
>> characters which swallow cursor movement
>>  • C-e somewhere on Lao line sends point to empty line over title
>> "LANGUAGE (NATIVE NAME)...."
>
> I don't see this effect on MS-Windows.  Can you try with revno 100027
> or later?

In the X client the latter has changed. End of the line is on the same  
line. Still when clicking onto the first or third character of the  
left Lao greeting creates swallowing characters. Clicking into the  
strings produces shape changes.

>
>> Clicking into the Braille "Hello" does not change the Burmese  
>> greeting
>> below and the Burmese comment visibly but change the empty box on the
>> Arabic line into "n" *only* when I click on a new/different than
>> before character... The exactly same effect when clicking onto *any*
>> character!
>
> I don't see this, either.  Again, please try with revno 100027 or
> later, if you haven't already.

Subtle changes of the "space" between (22, 11)/(22, 12) and (22, 33)/ 
(22, 34) still exist when clicking on Braille characters. Also when  
clicking into Telugu greeting in header.

Clicking into the Arabic or Hebrew (NATIVE NAME) comments moves the  
coagulated greetings into the right column. Same when clicking into  
their and the left parts (thirds) of Tibetan greetings or (9, 33)  
Telugu example in the header.

When clicking into left half of Thai example in header the Hebrew line  
is restored as left and right table cell.

clicking on (11, 44), Korean 하, merges Arabic and Bengali lines below  
to one with roughly two columns, the right one far out.

"East Asia" line: Clicking on the Chinese BIG-5 example 早晨 makes  
either Arabic line be correct with left and right cell or shifts the  
greeting partly into the comment. The Chinese GB2312 example only  
corrects Arabic and additionally adds a small bit of space into  
Burmese comment and name, as described for clicking into Braille. The  
SPACEs and COMMA near Chinese examples also change display of Arabic.

On "CJK variety" line each element of this copy "GB(元气,开发),  
" (including the SPACE before the word BIG5) changes display of Arabic  
(mostly establishes the two column view).


Is this magic?

--
Greetings

  Pete

Atheism is a non prophet organization.





Message #39 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Sun, 25 Apr 2010 17:40:47 +0300
> Cc: 5977-done <at> debbugs.gnu.org
> From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
> Date: Sun, 25 Apr 2010 11:01:35 +0200
> 
> 
> Subtle changes of the "space" between (22, 11)/(22, 12) and (22, 33)/ 
> (22, 34) still exist when clicking on Braille characters. Also when  
> clicking into Telugu greeting in header.
> 
> Clicking into the Arabic or Hebrew (NATIVE NAME) comments moves the  
> coagulated greetings into the right column. Same when clicking into  
> their and the left parts (thirds) of Tibetan greetings or (9, 33)  
> Telugu example in the header.
> 
> When clicking into left half of Thai example in header the Hebrew line  
> is restored as left and right table cell.
> 
> clicking on (11, 44), Korean 하, merges Arabic and Bengali lines below  
> to one with roughly two columns, the right one far out.
> 
> "East Asia" line: Clicking on the Chinese BIG-5 example 早晨 makes  
> either Arabic line be correct with left and right cell or shifts the  
> greeting partly into the comment. The Chinese GB2312 example only  
> corrects Arabic and additionally adds a small bit of space into  
> Burmese comment and name, as described for clicking into Braille. The  
> SPACEs and COMMA near Chinese examples also change display of Arabic.
> 
> On "CJK variety" line each element of this copy "GB(元气,开发),  
> " (including the SPACE before the word BIG5) changes display of Arabic  
> (mostly establishes the two column view).
> 
> 
> Is this magic?

The changes on the Arabic and Hebrew lines are expected: there's
character composition there, and character compositions don't yet work
when text is reordered.  Depending on various redisplay optimizations
done on clicking, you can see all kinds of strange effects there, but
when you type "M-x redraw-display RET", you should again see those
lines in their initial (incorrect and unaligned) form.

The rest of the issues I don't understand.  I also don't see most of
them on MS-Windows, which is strange, since the code we are talking
about is implemented in the 100% device independent part of the
display engine.

Thanks.





Message #40 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Fri, 14 May 2010 12:41:36 +0300
> Cc: 5977-done <at> debbugs.gnu.org
> From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
> Date: Sun, 25 Apr 2010 11:01:35 +0200
> 
> 
> Am 24.04.2010 um 18:01 schrieb Eli Zaretskii:
> 
> >> The NS variant, although rendering really beautifully, shows with Lao
> >> the same effects:
> >>
> >>  • entering the string by cursor movement no change in shape
> >>  • entering by clicking with mouse or trackpad leads to shape  
> >> changes
> >>  • entering by clicking with mouse or trackpad "creates" the same
> >> characters which swallow cursor movement
> >>  • C-e somewhere on Lao line sends point to empty line over title
> >> "LANGUAGE (NATIVE NAME)...."
> >
> > I don't see this effect on MS-Windows.  Can you try with revno 100027
> > or later?
> 
> In the X client the latter has changed. End of the line is on the same  
> line. Still when clicking onto the first or third character of the  
> left Lao greeting creates swallowing characters. Clicking into the  
> strings produces shape changes.
> 
> >
> >> Clicking into the Braille "Hello" does not change the Burmese  
> >> greeting
> >> below and the Burmese comment visibly but change the empty box on the
> >> Arabic line into "n" *only* when I click on a new/different than
> >> before character... The exactly same effect when clicking onto *any*
> >> character!
> >
> > I don't see this, either.  Again, please try with revno 100027 or
> > later, if you haven't already.
> 
> Subtle changes of the "space" between (22, 11)/(22, 12) and (22, 33)/ 
> (22, 34) still exist when clicking on Braille characters. Also when  
> clicking into Telugu greeting in header.
> 
> Clicking into the Arabic or Hebrew (NATIVE NAME) comments moves the  
> coagulated greetings into the right column. Same when clicking into  
> their and the left parts (thirds) of Tibetan greetings or (9, 33)  
> Telugu example in the header.
> 
> When clicking into left half of Thai example in header the Hebrew line  
> is restored as left and right table cell.
> 
> clicking on (11, 44), Korean 하, merges Arabic and Bengali lines below  
> to one with roughly two columns, the right one far out.
> 
> "East Asia" line: Clicking on the Chinese BIG-5 example 早晨 makes  
> either Arabic line be correct with left and right cell or shifts the  
> greeting partly into the comment. The Chinese GB2312 example only  
> corrects Arabic and additionally adds a small bit of space into  
> Burmese comment and name, as described for clicking into Braille. The  
> SPACEs and COMMA near Chinese examples also change display of Arabic.
> 
> On "CJK variety" line each element of this copy "GB(元气,开发),  
> " (including the SPACE before the word BIG5) changes display of Arabic  
> (mostly establishes the two column view).
> 
> 
> Is this magic?

Please re-test with the latest trunk.  It looks like Handa-san's
changes in revno 100276 fixed at least some of these problems (those I
was able to see on MS-Windows).





Message #41 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Fri, 14 May 2010 12:25:25 +0200
Am 14.05.2010 um 11:41 schrieb Eli Zaretskii:

> It looks like Handa-san's
> changes in revno 100276 fixed at least some of these problems (those I
> was able to see on MS-Windows).


I'll do so! Yesterday, a holiday in Germany, I built an X client. It  
still has some problems with clicking into some "exotic" strings, with  
bidi (Arabic and Hebrew not positioned correctly and extra characters  
as described), and newly with Myanmar. It uses U+103A, U+103B, and U 
+103C – and I have no font which has glyphs in the range of U+103A...U 
+103F. Obviously these character are not used for composition, so open  
boxes are left – one having VIRAMA at U+1039 in it.

--
Greetings

             ~  O
  Pete       ~~_\\_/%
             ~  O  o





Message #42 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Fri, 14 May 2010 18:29:41 +0300
> Cc: 5977-done <at> debbugs.gnu.org
> From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
> Date: Fri, 14 May 2010 17:14:12 +0200
> 
> The explanation on the RTL MARK is taken from the OPEN BOX right of
> the (<Arabic>) comment. Right of the (<Hebrew>) comment different
> characters can appear: {, m, OPEN BOX (which here actually is TAB!).

I'm not sure I understand: are you saying that "C-u C-x =" does not
say that the character after the right paren in the Hebrew line is the
RLM, like it says in the Arabic line?

Thanks for testing.




Message #43 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Fri, 14 May 2010 17:14:12 +0200
[Message part 1 (text/plain, inline)]
Am 14.05.2010 um 11:41 schrieb Eli Zaretskii:

> Please re-test with the latest trunk.


It looks very good:

[Emacs.app HELLO.png (image/png, inline)]
[Message part 3 (text/plain, inline)]

The explanation on the RTL MARK is taken from the OPEN BOX right of  
the (<Arabic>) comment. Right of the (<Hebrew>) comment different  
characters can appear: {, m, OPEN BOX (which here actually is TAB!).  
These boxes or characters do not change size by the same amount (or  
not at all?) when I invoke text-scale-decrease or text-scale-increase.  
You can also see the boxes in the Myanmar texts.


--
Greetings

  Pete

Only useless documentation transcends the first two laws.
				– Arnold's Third Law of Documentation


Message #44 received at 5977-done <at> debbugs.gnu.org (full text, mbox):

From: Peter Dyballa <Peter_Dyballa <at> Freenet.DE>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5977-done <at> debbugs.gnu.org
Subject: Re: bug#5977: 24.0.50; Lao HELLO is incorrectly displayed
Date: Fri, 14 May 2010 18:30:26 +0200
Am 14.05.2010 um 17:29 schrieb Eli Zaretskii:

> I'm not sure I understand: are you saying that "C-u C-x =" does not
> say that the character after the right paren in the Hebrew line is the
> RLM, like it says in the Arabic line?

Right, it says TAB. The same is true in the X client. Mode-line  
contains (37,14) as line/column numbers.

In the X client some "exotic" strings change when I pick/click  
somewhere else. Similarly when I change the marked region by holding  
the mouse button (actually track pad) pressed and move the mouse (over  
the track pad). Particularly the Lao and Thai "Hello" strings change.  
For example:

Arabic "HELLO"	-> 2. Thai "Hello"
Greek (εη)	-> 2. Lao "Hello"
Greek (κ)	-> 2. Thai "Hello"

Only these three Greek characters have an effect.

--
Greetings

  Pete

If it does exist, it's out of date.
				– Arnold's Second Law of Documentation





bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sat, 12 Jun 2010 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 15 years and 9 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.