Translation Tables

Introduction

There are two general scenarios in which translation tables are used to convert characters:

• In many contexts (such as in URLs, in HTML, in JSON, and so on), specific characters are disallowed and must be represented by escape sequences. In this case, it is necessary to convert the characters to or from the allowed set of characters.

• If you are reading from a source outside the database or writing to a destination outside the database, that entity may expect a different character set than InterSystems IRIS uses. In this case, it is necessary to convert the character encoding.

The “translation table” for a given context is actually a pair of tables. One table specifies how to convert from the default character set to the foreign character set (or to the foreign context), and other specifies how to convert in the other direction. In InterSystems IRIS, the convention is to refer to this pair of tables as a single unit that has an input mode and an output mode. Thus, there is an HTML translation table for managing conversions to and from HTML, and there is an CP1250 translation table for managing conversions to and from the CP1250 character set.

List of Tables

The following is a list of the InterSystems IRIS translation tables:

Output Escaping

This section indicates how specific translation tables convert characters in output mode:

For Unicode characters (characters above ASCII 255):

• The JSML and JSONML translation tables perform escaping, not described here.

• The URL translation table performs escaping, not described here.

• The URI translation table is irrelevant because URIs cannot contain characters above ASCII 255. If you attempt to use the URI translation table with such characters, the result is an <ILLEGAL VALUE> error. For example:

  USER>set x=$char(955)
   
  USER>w $ZCVT(x,"O","URI")
   
  W $ZCVT(x,"O","URI")
  ^
  <ILLEGAL VALUE>
  

  

• The HTML and XML translation tables do not perform escaping.

Sequential Character Conversion and Character Escaping

In some scenarios, you may want to perform two conversions: one to convert to a different character set, and another to perform character escaping. In such cases, the order of operations is important, and generally you need to convert to the applicable character set and then perform the escaping. In the reverse direction, it is necessary to perform the reverse conversions in the reverse order. An example best demonstrates this. Suppose that we start with a string that uses our local character set, and suppose that this string could potentially include Unicode characters. Suppose that we need to use this string within a URI. A URI can contain only ASCII characters, and within that set of characters, there are specific escape sequences for some characters. In this case, we can convert our string for use in a URI in two steps:

1. First convert the local representation to ASCII (the UTF-8 character set). For example, given our input string origstring:

    set utf8string = $ZCONVERT(origstring,"O","UTF8")
   

   

2. Then apply the character escaping:

    set final = $ZCONVERT(utf8string,"O","URI")
   

   

   The string final is safe to use within a URI.

To convert a URI back to our local character set, you perform the reverse operation:

1. Unescape the escaped characters:

    set unescaped=$ZCONVERT(uristring,"I","URI")
   

   

2. Convert from UTF–8 to your local representation:

    set local=$ZCONVERT(unescaped,"I","UTF8")
   

   

As explained above in the entry for the URI translation table, you can also convert directly, skipping the character set conversions; in this case, the $ZCONVERT function converts the character set for you.

Related APIs

For the currently available list of translation tables, refer to the XLTTablesOpens in a new tab property of %SYS.NLS.LocaleOpens in a new tab, as shown in the following example (with line breaks added):

USER>SET nlsoref=##class(%SYS.NLS.Locale).%New()
 
USER>WRITE $LISTTOSTRING(nlsoref.XLTTables,", ")
Unicode, RAW, BIN, SAME, UTF8, UnicodeLittle, UnicodeBig, URL, JS, JSML, JSON, JSONML, HTML, 
XML, XMLA, XMLC, CP1250, CP1251, CP1252, CP1253, CP1255, CP437, CP850, CP852, CP866, CP874, 
EBCDIC, Latin2, Latin9, LatinC, LatinG, LatinH, LatinT

Also, you can use %Net.CharsetOpens in a new tab to represent character sets within InterSystems IRIS. This class includes the following class methods:

• GetDefaultCharset() returns the default character set for the current InterSystems IRIS locale (see next heading).

  For example:

  USER>w ##class(%Net.Charset).GetTranslateTable("UTF8")
  UTF8
  

• GetTranslateTable() returns the name of the InterSystems IRIS translation table for a given input character set.

• TranslateTableExists() indicates whether the translation table for the given character set has been loaded.

For method signatures, see the class documentation for %Net.CharsetOpens in a new tab.

Related Concepts

InterSystems IRIS locale is a set of metadata that specify the user language, currency symbols, formats, and other conventions for a specific country or geographic region. A locale includes translation tables. InterSystems IRIS uses the phrase National Language Support (NLS) to refer collectively to the locale definitions and to the tools that you use to view and extend them.

External to the definition of any locale, a given InterSystems IRIS instance is configured to use specific translation tables, by default, for input/output activity. These are the default translation tables or default I/O tables. To see the current defaults, use %SYS.NLS.TableOpens in a new tab; see the class reference for detail.

See Also

• $zconvert

• Using System Classes for National Language Support

• %SYS.NLS.LocaleOpens in a new tab

• %SYS.NLS.TableOpens in a new tab