Bidirectional Wordprocessing With AbiWord

May 2, 2003

Tomas Frydrych <tomas@frydrych.uklinux.net>

Contents

Introduction

Controlling Bidirectional Ordering

Dominant Direction of Text

Explicit Direction Overrides

Direction Markers

Glyph Shaping

Introduction

Some languages, such as English, are written from left to right, while other languages, such as Arabic, from right to left. AbiWord can handle both directions of text, as well as their combinations -- AbiWord is a bidirectional wordprocessor.

Controlling Bidirectional Ordering

The bidirectional ordering of text in AbiWord is done closely following the Unicode Bidirectional Algorithm, which means that most of it is done automatically without the user's intervention. At the same time, the user has at his or her disposal three basic mechanisms that allow him or her to fine-tune the results. These are specifying dominant direction of text, overriding implicit directional properties of characters, and inserting direction markers.

Dominant Direction of Text

The same sequence of characters with different directional properties will look differently if it is assumed to be basically an left-to-right text with right-to-left text embedded, in it, or if it is understood to be basically a right-to-left text with left-to-right text emebedded. In AbiWord we refer to the basic direciton of text as the dominant direction (in the the Unicode documentation it is known as the base embedding level).

The dominant direction in AbiWord operates on four hierarchical levels, paragraph, section, document, and the program, all of which are controlled by the user.

Paragraph-Level Dominant Direction

The paragraph-level dominant direction can be set either by the Right-to-left dominant check box in the Format->Paragraph dialogue, or by using the equivalent button on the Extra toolbar. If the dominant direction is not set explicitly by the user, AbiWord will work it out from the rest of the dominant direction hierarchy, i.e., it will check if dominant direction is set explicitly for the section in which the paragraph is located, and if not, it will check the document level settings, finally resorting to program-level defaults.

Section-Level Dominant Direction

The section-level dominant direction is controlled by the Use RTL order check box of the Format->Columns dialogue. Apart from providing a default for any of the section paragraphs that do not have their dominant direction set explicitly, the section dominant direction controls how columns in multicolumn sections are ordered, it determines the dominant direction of text of footnotes inserted into the section, and the order of columns in tables. If section-level dominant direction is not given, AbiWord will derive it from the rest of the dominant direction hierarchy as described earlier.

Document-Level Dominant Direction

The document-level dominant direction is controlled from the Other tab of the Format->Document dialogue. It provides default dominant direction for all paragraphs and sections in the document that do not have explicit dominant direction set.

Program-Level Dominant Direction

The program-level value of dominant direction is hard-coded at compile time and it is the value to which AbiWord recourses if everything else fails. It is set to left-to-right unless the right-to-left is explicitly specified when the program is built.

Explicit Direction Overrides

In the Unicode character set each character is assigned certain directional properties which are then used by the Unicode Bidirectional Algorithm to order text. Thus, Hebrew or Arabic characters will automatically be treated as right-to-left, and English characters as left-to-right. In some circumstances, however, the user might require that characters are treated differently. In this case, AbiWord allows the user to specify that certain characters should be treated as left-to-right or right-to-left. This is done by selecting the characters in question and then applying the override using the and buttons.

Direction Markers

In the Unicode character set there are two special characters called direction markers, LRM (left to right direction marker) and RLM (right to left direction marker). When a Unicode text is reordered these characters behave as normal left-to-right and right-to-left characters, but when the text is displayed they are not shown. Their purpose is to allow small adjustments of the bidirectional order, particularly in combination with characters the direction of which depends on what precedes them and follows them. For example if a formula such as log(x) is embeded into a right to left text (represented here by capital letters), it will look like this:

GFE (log(x CBA

i.e., the closing parenthesis of the formula will be treated as an right-to-left character; this is a feature of the Unicode Bidirectional Algorithm. If the order the user wants is:

GFE log(x) CBA

it can be achieved by following the closing parenthesis with the LRM marker; the LRM and RLM markers are assigned to keyboard shortcuts Alt+Ctrl+> and Alt+Ctrl+< respectively.

The same result could, of course, be achieved by selecting the closing parenthesis and applying to it and explicit left-to-right override as described in the previous section. However, in simple situations like in this example, where only the direction of a single character needs to be altered, the direction markers provide a more convenient mechanism.

Another situation in which such markers come handy, is with phone numbers. For instance, if the phone number 123 456 789 is embedded into a right-to-left text, it will look like this:

FED 789 456 123 CBA

This is because the space between the three groups of numbers behaves as a right-to-left character. If what the user wants is

FED 123 456 789 CBA

all that is requires is that the LRM character is inserted just before typing in the first digit.

In certain circumstances AbiWord is capable of inserting these direction markers automatically based on the keyboard layout used (this is currently only supported under Windows). In order for this work, the user needs to first of all make sure that the option to change language when changing keyboard layout is turned on (Tools->Preferences->Language->Change Language When Changing Keyboard), and that also the bidirectional option auto-insert direction markers is turned on (Tools->Preferences->Language->). AbiWord will then follow all ), ], and } characters with a direction marker derived from the language applied to that character. For instance, if a ) character is set as being in Hebrew, it will be followed by a RLM marker but if it is set as being in English, it will be followed by a LRM marker. Similarly, it will precede all (, [, and { characters with a direction marker.

Glyph Shaping

Closely related to AbiWord's bidirectional capabilities is its ability to change visual appearance of certain glyphs depending on their context. This is essential for languages that use scripted alphabets, such as Arabic, in which each letter has up to four different shapes depending whether it stands alone, or at the beginning, in the middle or at the end of a word. Alongside this type of glyph shaping, AbiWord can also replace a sequence of two glyphs with a special ligature glyph where needed.

It should be noted that at present AbiWord uses a proprietary shaping engine of fairly limited capabilities, and that the shaping can only be done for languages for which the alternative glyph shapes have code points assigned in the Unicode character set (such as Arabic) but not for languages that rely solely on advanced font technologies for shaping (such as Syriac). Furthermore, AbiWord does not check whether the alternative glyph exists in the current font. This means that if the glyph shaping is turned on and the glyph is not found in the font, you will see a square or a circle in its place. In other words, glyph shaping will only work with proper fonts (the core MS TTF fonts Times New Roman and Arial have a reasonable Unicode coverage, and there are other TTF Unicode based fonts available). In future the proprietary shaping engine will be replace with a more powerful third-party library, possibly Pango.

An important thing to understand about the glyph shaping is that the the changes only take place in the visual plain, the characters that are contained in the document do not change in the process.

Glyph shaping is controlled from the Language tab of the Tools->Preference dialogue. There are two check boxes there: Determine glyph shapes from context and Use glyph shaping for Hebrew. The former of these is the master-switch that turns the glyph shaping engine on and off. When the second check box is checked, the shaping engine will shape also the five Hebrew letters that have a final form. Please note that this is not intended to be used for writing modern Hebrew and Yiddish documents, since in modern Hebrew and Yiddish the final forms are considered different characters (and because as I have explained above the shaping does not change the characters in a document, if nothing else, your spell-checker will not work).