Transliterator
is an abstract class that transliterates text from one format to another.
More...
#include <translit.h>
Inheritance diagram for Transliterator:
Public Types | |
typedef Transliterator ** | Factory (const UnicodeString &ID, Token context) |
A function that creates and returns a Transliterator. | |
Public Member Functions | |
virtual | ~Transliterator () |
Destructor. | |
virtual Transliterator * | clone () const |
Implements Cloneable. | |
virtual int32_t | transliterate (Replaceable &text, int32_t start, int32_t limit) const |
Transliterates a segment of a string, with optional filtering. | |
virtual void | transliterate (Replaceable &text) const |
Transliterates an entire string in place. | |
virtual void | transliterate (Replaceable &text, UTransPosition &index, const UnicodeString &insertion, UErrorCode &status) const |
Transliterates the portion of the text buffer that can be transliterated unambiguosly after new text has been inserted, typically as a result of a keyboard event. | |
virtual void | transliterate (Replaceable &text, UTransPosition &index, UChar32 insertion, UErrorCode &status) const |
Transliterates the portion of the text buffer that can be transliterated unambiguosly after a new character has been inserted, typically as a result of a keyboard event. | |
virtual void | transliterate (Replaceable &text, UTransPosition &index, UErrorCode &status) const |
Transliterates the portion of the text buffer that can be transliterated unambiguosly. | |
virtual void | finishTransliteration (Replaceable &text, UTransPosition &index) const |
Finishes any pending transliterations that were waiting for more characters. | |
virtual void | filteredTransliterate (Replaceable &text, UTransPosition &index, UBool incremental) const |
Transliterate a substring of text, as specified by index, taking filters into account. | |
int32_t | getMaximumContextLength (void) const |
Returns the length of the longest context required by this transliterator. | |
virtual const UnicodeString & | getID (void) const |
Returns a programmatic identifier for this transliterator. | |
const UnicodeFilter * | getFilter (void) const |
Returns the filter used by this transliterator, or NULL if this transliterator uses no filter. | |
UnicodeFilter * | orphanFilter (void) |
Returns the filter used by this transliterator, or NULL if this transliterator uses no filter. | |
void | adoptFilter (UnicodeFilter *adoptedFilter) |
Changes the filter used by this transliterator. | |
Transliterator * | createInverse (UErrorCode &status) const |
Returns this transliterator's inverse. | |
virtual UnicodeString & | toRules (UnicodeString &result, UBool escapeUnprintable) const |
Create a rule string that can be passed to createFromRules() to recreate this transliterator. | |
int32_t | countElements () const |
Return the number of elements that make up this transliterator. | |
const Transliterator & | getElement (int32_t index, UErrorCode &ec) const |
Return an element that makes up this transliterator. | |
UnicodeSet & | getSourceSet (UnicodeSet &result) const |
Returns the set of all characters that may be modified in the input text by this Transliterator. | |
virtual void | handleGetSourceSet (UnicodeSet &result) const |
Framework method that returns the set of all characters that may be modified in the input text by this Transliterator, ignoring the effect of this object's filter. | |
virtual UnicodeSet & | getTargetSet (UnicodeSet &result) const |
Returns the set of all characters that may be generated as replacement text by this transliterator. | |
virtual UClassID | getDynamicClassID (void) const =0 |
Returns a unique class ID polymorphically. | |
Static Public Member Functions | |
Token | integerToken (int32_t) |
Return a token containing an integer. | |
Token | pointerToken (void *) |
Return a token containing a pointer. | |
UnicodeString & | getDisplayName (const UnicodeString &ID, UnicodeString &result) |
Returns a name for this transliterator that is appropriate for display to the user in the default locale. | |
UnicodeString & | getDisplayName (const UnicodeString &ID, const Locale &inLocale, UnicodeString &result) |
Returns a name for this transliterator that is appropriate for display to the user in the given locale. | |
Transliterator * | createInstance (const UnicodeString &ID, UTransDirection dir, UParseError &parseError, UErrorCode &status) |
Returns a Transliterator object given its ID. | |
Transliterator * | createInstance (const UnicodeString &ID, UTransDirection dir, UErrorCode &status) |
Returns a Transliterator object given its ID. | |
Transliterator * | createFromRules (const UnicodeString &ID, const UnicodeString &rules, UTransDirection dir, UParseError &parseError, UErrorCode &status) |
Returns a Transliterator object constructed from the given rule string. | |
void | registerFactory (const UnicodeString &id, Factory factory, Token context) |
Registers a factory function that creates transliterators of a given ID. | |
void | registerInstance (Transliterator *adoptedObj) |
Registers an instance obj of a subclass of Transliterator with the system. | |
void | registerAlias (const UnicodeString &aliasID, const UnicodeString &realID) |
Registers an ID string as an alias of another ID string. | |
void | unregister (const UnicodeString &ID) |
Unregisters a transliterator or class. | |
StringEnumeration * | getAvailableIDs (UErrorCode &ec) |
Return a StringEnumeration over the IDs available at the time of the call, including user-registered IDs. | |
int32_t | countAvailableSources (void) |
Return the number of registered source specifiers. | |
UnicodeString & | getAvailableSource (int32_t index, UnicodeString &result) |
Return a registered source specifier. | |
int32_t | countAvailableTargets (const UnicodeString &source) |
Return the number of registered target specifiers for a given source specifier. | |
UnicodeString & | getAvailableTarget (int32_t index, const UnicodeString &source, UnicodeString &result) |
Return a registered target specifier for a given source. | |
int32_t | countAvailableVariants (const UnicodeString &source, const UnicodeString &target) |
Return the number of registered variant specifiers for a given source-target pair. | |
UnicodeString & | getAvailableVariant (int32_t index, const UnicodeString &source, const UnicodeString &target, UnicodeString &result) |
Return a registered variant specifier for a given source-target pair. | |
UClassID | getStaticClassID (void) |
Return the class ID for this class. | |
int32_t | countAvailableIDs (void) |
Return the number of IDs currently registered with the system. | |
const UnicodeString & | getAvailableID (int32_t index) |
Return the index-th available ID. | |
Protected Member Functions | |
Transliterator (const UnicodeString &ID, UnicodeFilter *adoptedFilter) | |
Default constructor. | |
Transliterator (const Transliterator &) | |
Copy constructor. | |
Transliterator & | operator= (const Transliterator &) |
Assignment operator. | |
virtual void | handleTransliterate (Replaceable &text, UTransPosition &pos, UBool incremental) const =0 |
Abstract method that concrete subclasses define to implement their transliteration algorithm. | |
void | setMaximumContextLength (int32_t maxContextLength) |
Method for subclasses to use to set the maximum context length. | |
void | setID (const UnicodeString &id) |
Set the ID of this transliterators. | |
Static Protected Member Functions | |
Transliterator * | createBasicInstance (const UnicodeString &id, const UnicodeString *canon) |
Create a transliterator from a basic ID. | |
void | _registerFactory (const UnicodeString &id, Factory factory, Token context) |
void | _registerInstance (Transliterator *adoptedObj) |
void | _registerAlias (const UnicodeString &aliasID, const UnicodeString &realID) |
void | _registerSpecialInverse (const UnicodeString &target, const UnicodeString &inverseTarget, UBool bidirectional) |
Register two targets as being inverses of one another. | |
int32_t | _countAvailableSources (void) |
Non-mutexed internal method. | |
UnicodeString & | _getAvailableSource (int32_t index, UnicodeString &result) |
Non-mutexed internal method. | |
int32_t | _countAvailableTargets (const UnicodeString &source) |
Non-mutexed internal method. | |
UnicodeString & | _getAvailableTarget (int32_t index, const UnicodeString &source, UnicodeString &result) |
Non-mutexed internal method. | |
int32_t | _countAvailableVariants (const UnicodeString &source, const UnicodeString &target) |
Non-mutexed internal method. | |
UnicodeString & | _getAvailableVariant (int32_t index, const UnicodeString &source, const UnicodeString &target, UnicodeString &result) |
Non-mutexed internal method. | |
Friends | |
class | TransliteratorParser |
class | TransliteratorIDParser |
class | TransliteratorAlias |
Transliterator
is an abstract class that transliterates text from one format to another.
The most common kind of transliterator is a script, or alphabet, transliterator. For example, a Russian to Latin transliterator changes Russian text written in Cyrillic characters to phonetically equivalent Latin characters. It does not translate Russian to English! Transliteration, unlike translation, operates on characters, without reference to the meanings of words and sentences.
Although script conversion is its most common use, a transliterator can actually perform a more general class of tasks. In fact, Transliterator
defines a very general API which specifies only that a segment of the input text is replaced by new text. The particulars of this conversion are determined entirely by subclasses of Transliterator
.
Transliterators are stateless
Transliterator
objects are stateless; they retain no information between calls to transliterate()
. (However, this does not mean that threads may share transliterators without synchronizing them. Transliterators are not immutable, so they must be synchronized when shared between threads.) This1 might seem to limit the complexity of the transliteration operation. In practice, subclasses perform complex transliterations by delaying the replacement of text until it is known that no other replacements are possible. In other words, although the Transliterator
objects are stateless, the source text itself embodies all the needed information, and delayed operation allows arbitrary complexity.
Batch transliteration
The simplest way to perform transliteration is all at once, on a string of existing text. This is referred to as batch transliteration. For example, given a string input
and a transliterator t
, the call
String result = t.transliterate(input);
will transliterate it and return the result. Other methods allow the client to specify a substring to be transliterated and to use Replaceable objects instead of strings, in order to preserve out-of-band information (such as text styles).
Keyboard transliteration
Somewhat more involved is keyboard, or incremental transliteration. This is the transliteration of text that is arriving from some source (typically the user's keyboard) one character at a time, or in some other piecemeal fashion.
In keyboard transliteration, a Replaceable
buffer stores the text. As text is inserted, as much as possible is transliterated on the fly. This means a GUI that displays the contents of the buffer may show text being modified as each new character arrives.
Consider the simple RuleBasedTransliterator
:
th>{theta}
t>{tau}
When the user types 't', nothing will happen, since the transliterator is waiting to see if the next character is 'h'. To remedy this, we introduce the notion of a cursor, marked by a '|' in the output string:
t>|{tau}
{tau}h>{theta}
Now when the user types 't', tau appears, and if the next character is 'h', the tau changes to a theta. This is accomplished by maintaining a cursor position (independent of the insertion point, and invisible in the GUI) across calls to transliterate()
. Typically, the cursor will be coincident with the insertion point, but in a case like the one above, it will precede the insertion point.
Keyboard transliteration methods maintain a set of three indices that are updated with each call to transliterate()
, including the cursor, start, and limit. Since these indices are changed by the method, they are passed in an int[]
array. The START
index marks the beginning of the substring that the transliterator will look at. It is advanced as text becomes committed (but it is not the committed index; that's the CURSOR
). The CURSOR
index, described above, marks the point at which the transliterator last stopped, either because it reached the end, or because it required more characters to disambiguate between possible inputs. The CURSOR
can also be explicitly set by rules in a RuleBasedTransliterator
. Any characters before the CURSOR
index are frozen; future keyboard transliteration calls within this input sequence will not change them. New text is inserted at the LIMIT
index, which marks the end of the substring that the transliterator looks at.
Because keyboard transliteration assumes that more characters are to arrive, it is conservative in its operation. It only transliterates when it can do so unambiguously. Otherwise it waits for more characters to arrive. When the client code knows that no more characters are forthcoming, perhaps because the user has performed some input termination operation, then it should call finishTransliteration()
to complete any pending transliterations.
Inverses
Pairs of transliterators may be inverses of one another. For example, if transliterator A transliterates characters by incrementing their Unicode value (so "abc" -> "def"), and transliterator B decrements character values, then A is an inverse of B and vice versa. If we compose A with B in a compound transliterator, the result is the indentity transliterator, that is, a transliterator that does not change its input text.
The Transliterator
method getInverse()
returns a transliterator's inverse, if one exists, or null
otherwise. However, the result of getInverse()
usually will not be a true mathematical inverse. This is because true inverse transliterators are difficult to formulate. For example, consider two transliterators: AB, which transliterates the character 'A' to 'B', and BA, which transliterates 'B' to 'A'. It might seem that these are exact inverses, since
"A" x AB -> "B"
"B" x BA -> "A"
where 'x' represents transliteration. However,
"ABCD" x AB -> "BBCD"
"BBCD" x BA -> "AACD"
so AB composed with BA is not the identity. Nonetheless, BA may be usefully considered to be AB's inverse, and it is on this basis that AB.getInverse()
could legitimately return BA.
IDs and display names
A transliterator is designated by a short identifier string or ID. IDs follow the format source-destination, where source describes the entity being replaced, and destination describes the entity replacing source. The entities may be the names of scripts, particular sequences of characters, or whatever else it is that the transliterator converts to or from. For example, a transliterator from Russian to Latin might be named "Russian-Latin". A transliterator from keyboard escape sequences to Latin-1 characters might be named "KeyboardEscape-Latin1". By convention, system entity names are in English, with the initial letters of words capitalized; user entity names may follow any format so long as they do not contain dashes.
In addition to programmatic IDs, transliterator objects have display names for presentation in user interfaces, returned by getDisplayName.
Factory methods and registration
In general, client code should use the factory method createInstance to obtain an instance of a transliterator given its ID. Valid IDs may be enumerated using getAvailableIDs()
. Since transliterators are mutable, multiple calls to createInstance with the same ID will return distinct objects.
In addition to the system transliterators registered at startup, user transliterators may be registered by calling registerInstance()
at run time. A registered instance acts a template; future calls to createInstance with the ID of the registered object return clones of that object. Thus any object passed to registerInstance()
must implement clone()
propertly. To register a transliterator subclass without instantiating it (until it is needed), users may call registerFactory. In this case, the objects are instantiated by invoking the zero-argument public constructor of the class.
Subclassing
Subclasses must implement the abstract method handleTransliterate()
.
Subclasses should override the transliterate()
method taking a Replaceable
and the transliterate()
method taking a String
and StringBuffer
if the performance of these methods can be improved over the performance obtained by the default implementations in this class.
Definition at line 242 of file translit.h.
|
A function that creates and returns a Transliterator. When invoked, it will be passed the ID string that is being instantiated, together with the context pointer that was passed in when the factory function was first registered. Many factory functions will ignore both parameters, however, functions that are registered to more than one ID may use the ID or the context parameter to parameterize the transliterator they create.
Definition at line 316 of file translit.h. |
|
Default constructor.
|
|
Copy constructor.
|
|
Destructor.
|
|
Non-mutexed internal method.
|
|
Non-mutexed internal method.
|
|
Non-mutexed internal method.
|
|
Non-mutexed internal method.
|
|
Non-mutexed internal method.
|
|
Non-mutexed internal method.
|
|
|
|
|
|
|
|
Register two targets as being inverses of one another. For example, calling registerSpecialInverse("NFC", "NFD", true) causes Transliterator to form the following inverse relationships:
NFC => NFD Any-NFC => Any-NFD NFD => NFC Any-NFD => Any-NFC (Without the special inverse registration, the inverse of NFC would be NFC-Any.) Note that NFD is shorthand for Any-NFD, but that the presence or absence of "Any-" is preserved. The relationship is symmetrical; registering (a, b) is equivalent to registering (b, a). The relevant IDs must still be registered separately as factories or classes. Only the targets are specified. Special inverses always have the form Any-Target1 <=> Any-Target2. The target should have canonical casing (the casing desired to be produced when an inverse is formed) and should contain no whitespace or other extraneous characters.
|
|
Changes the filter used by this transliterator.
If the filter is set to Callers must take care if a transliterator is in use by multiple threads. The filter should not be changed by one thread while another thread may be transliterating.
|
|
Implements Cloneable.
All subclasses are encouraged to implement this method if it is possible and reasonable to do so. Subclasses that are to be registered with the system using
|
|
Return the number of IDs currently registered with the system. To retrieve the actual IDs, call getAvailableID(i) with i from 0 to countAvailableIDs() - 1.
|
|
Return the number of registered source specifiers.
|
|
Return the number of registered target specifiers for a given source specifier.
|
|
Return the number of registered variant specifiers for a given source-target pair.
|
|
Return the number of elements that make up this transliterator. For example, if the transliterator "NFD;Jamo-Latin;Latin-Greek" were created, the return value of this method would be 3. If this transliterator is not composed of other transliterators, then this method returns 1.
|
|
Create a transliterator from a basic ID. This is an ID containing only the forward direction source, target, and variant.
|
|
Returns a This will be a RuleBasedTransliterator, if the rule string contains only rules, or a CompoundTransliterator, if it contains ID blocks, or a NullTransliterator, if it contains ID blocks which parse as empty for the given direction.
|
|
Returns a
The ID must be either a system transliterator ID or a ID registered using
|
|
Returns a
The ID must be either a system transliterator ID or a ID registered using
|
|
Returns this transliterator's inverse.
See the class documentation for details. This implementation simply inverts the two entities in the ID and attempts to retrieve the resulting transliterator. That is, if Subclasses with knowledge of their inverse may wish to override this method.
|
|
Transliterate a substring of text, as specified by index, taking filters into account. This method is for subclasses that need to delegate to another transliterator, such as CompoundTransliterator.
|
|
Finishes any pending transliterations that were waiting for more characters.
Clients should call this method as the last call after a sequence of one or more calls to
|
|
Return the index-th available ID. index must be between 0 and countAvailableIDs() - 1, inclusive. If index is out of range, the result of getAvailableID(0) is returned.
|
|
Return a StringEnumeration over the IDs available at the time of the call, including user-registered IDs.
|
|
Return a registered source specifier.
|
|
Return a registered target specifier for a given source.
|
|
Return a registered variant specifier for a given source-target pair.
|
|
Returns a name for this transliterator that is appropriate for display to the user in the given locale.
This name is taken from the locale resource data in the standard manner of the
If no localized names exist in the system resource bundles, a name is synthesized using a localized
|
|
Returns a name for this transliterator that is appropriate for display to the user in the default locale. See getDisplayName for details.
|
|
Returns a unique class ID polymorphically. This method is to implement a simple version of RTTI, since not all C++ compilers support genuine RTTI. Polymorphic operator==() and clone() methods call this method. Concrete subclasses of Transliterator must use the UOBJECT_DEFINE_RTTI_IMPLEMENTATION macro from uobject.h to provide the RTTI functions.
Implements UObject. |
|
Return an element that makes up this transliterator. For example, if the transliterator "NFD;Jamo-Latin;Latin-Greek" were created, the return value of this method would be one of the three transliterator objects that make up that transliterator: [NFD, Jamo-Latin, Latin-Greek]. If this transliterator is not composed of other transliterators, then this method will return a reference to this transliterator when given the index 0.
|
|
Returns the filter used by this transliterator, or
|
|
Returns a programmatic identifier for this transliterator.
If this identifier is passed to
|
|
Returns the length of the longest context required by this transliterator.
This is preceding context. The default implementation supplied by
Definition at line 1361 of file translit.h. |
|
Returns the set of all characters that may be modified in the input text by this Transliterator. This incorporates this object's current filter; if the filter is changed, the return value of this function will change. The default implementation returns an empty set. Some subclasses may override handleGetSourceSet to return a more precise result. The return result is approximate in any case and is intended for use by tests, tools, or utilities.
|
|
Return the class ID for this class. This is useful only for comparing to a return value from getDynamicClassID(). Note that Transliterator is an abstract base class, and therefor no fully constructed object will have a dynamic UCLassID that equals the UClassID returned from TRansliterator::getStaticClassID().
|
|
Returns the set of all characters that may be generated as replacement text by this transliterator. The default implementation returns the empty set. Some subclasses may override this method to return a more precise result. The return result is approximate in any case and is intended for use by tests, tools, or utilities requiring such meta-information.
|
|
Framework method that returns the set of all characters that may be modified in the input text by this Transliterator, ignoring the effect of this object's filter. The base class implementation returns the empty set. Subclasses that wish to implement this should override this method.
|
|
Abstract method that concrete subclasses define to implement their transliteration algorithm.
This method handles both incremental and non-incremental transliteration. Let
Implementations of this method should also obey the following invariants:
Subclasses may safely assume that all characters in [
This method is not for public consumption. Calling this method directly will transliterate [
|
|
Return a token containing an integer.
Definition at line 1372 of file translit.h. |
|
Assignment operator.
|
|
Returns the filter used by this transliterator, or
The caller must eventually delete the result. After this call, this transliterator's filter is set to
|
|
Return a token containing a pointer.
Definition at line 1378 of file translit.h. |
|
Registers an ID string as an alias of another ID string.
That is, after calling this function,
|
|
Registers a factory function that creates transliterators of a given ID.
|
|
Registers an instance
When After this call the Transliterator class owns the adoptedObj and will delete it.
|
|
Set the ID of this transliterators. Subclasses shouldn't do this, unless the underlying script behavior has changed.
Definition at line 1365 of file translit.h. |
|
Method for subclasses to use to set the maximum context length.
|
|
Create a rule string that can be passed to createFromRules() to recreate this transliterator.
|
|
Transliterates the portion of the text buffer that can be transliterated unambiguosly. This is a convenience method; see transliterate(Replaceable&, UTransPosition&, const UnicodeString&, UErrorCode&) const for details.
|
|
Transliterates the portion of the text buffer that can be transliterated unambiguosly after a new character has been inserted, typically as a result of a keyboard event. This is a convenience method; see transliterate(Replaceable&, UTransPosition&, const UnicodeString&, UErrorCode&) const for details.
|
|
Transliterates the portion of the text buffer that can be transliterated unambiguosly after new text has been inserted, typically as a result of a keyboard event.
The new text in
Upon return, values in
Typical usage of this method begins with an initial call with This method assumes that future calls may be made that will insert new text into the buffer. As a result, it only performs unambiguous transliterations. After the last call to this method, there may be untransliterated text that is waiting for more input to resolve an ambiguity. In order to perform these pending transliterations, clients should call finishTransliteration after the last call to this method has been made.
|
|
Transliterates an entire string in place. Convenience method.
|
|
Transliterates a segment of a string, with optional filtering.
|
|
Unregisters a transliterator or class. This may be either a system transliterator or a user transliterator or class. Any attempt to construct an unregistered transliterator based on its ID will fail.
|