Subject: Re: bug#6998: 24.0.50; bidi: lines starting with neutral types have the wrong base direction?

From: Eli Zaretskii <[email-masked]>
Date: Mon, 13 Sep 2010 14:04:53 +0200

> Date: Sat, 11 Sep 2010 01:29:44 +0300
> From: Thamer Mahmoud <[email-masked]>
> Cc: 
> 
> I did some testing, and I found out that the differences between Emacs
> and other apps (fribidi, gedit, kwrite, etc) are explained by the other
> apps' use of "line-based reordering", while Emacs uses
> "paragraph-based reordering" (perhaps to avoid filled paragraphs
> having more than one direction).

More accurately, the "paragraph" in fribidi etc. is a single line,
because a linefeed is one of the paragraph separators.

Since this would produce non-sensical display in any human-readable
text that uses hard newlines, and because Emacs uses hard newlines in
just about every derivative of Text mode, the Emacs implementation of
the Unicode Bidirectional Algorithm uses what UAX#9 calls
``higher-level protocols'' to define what is the base embedding level
of a paragraph.  In Emacs, a paragraph is delimited by lines that are
entirely whitespace or empty.

This is explained in the "Bidirectional Editing" node of the Emacs
manual.

> So I guess this is not a bug per se.

Right, not a bug; a feature.

> However, I still see inconsistent rendering of the second example
> given above. But I'll file a more specific bug for that.

If you mean this example:

> * First
> [BLANK_LINE]
> ARABIC
> * Second
> What I expect:
> * First
> [BLANK_LINE]
>                                CIBARA
> * Second
> What is shown in Emacs:
> * First
> [BLANK_LINE]
>                                CIBARA
>                              Second *

Then it is also expected behavior: since there's no blank line between
"ARABIC" and "* Second", the latter is considered to belong to a
right-to-left paragraph, and rendered accordingly.  OTOH, "* First"
and "ARABIC" _are_ separated by a blank line, so they belong to
different paragraphs.

You can control the paragraph direction by inserting the LRM and RLM
characters in front of each paragraph.  Alternatively, you can force a
specific base direction on the entire buffer by setting the per-buffer
variable bidi-paragraph-direction.  This is also described in the
manual.