Skip to main content

Creating Data Transformations

This topic describes generally how to create and edit data transformations.

Other topics describe the syntax to use in data transformations, details for assign actions, and details for other kinds of actions.

You can also visit Online Learning to try Creating a Data TranformationOpens in a new tab yourself.

Important:

After a period of inactivity, the InterSystems Management Portal may log you out and discard any unsaved changes. Inactivity is the time between calls to the InterSystems IRIS server. Not all actions constitute a call to the server. For example, clicking Save constitutes a call to the server, but typing in a text field does not. Consequently, if you are editing a data transformation, but have not clicked Save for longer than Session Timeout threshold, your session will expire and your unsaved changes will be discarded. After a logout, the login page appears or the current page is refreshed. For more information, see Automatic Logout Behavior in the Management Portal.

Creating a Transformation

To create a transformation:

  1. Click New.

    If you are currently viewing a transformation and you have made changes but have not yet saved them, InterSystems IRIS® prompts you to confirm that you want to proceed (which will discard those changes).

    InterSystems IRIS then displays a dialog box where you can specify the basic information for the transformation.

  2. Specify some or all of the following information:

    • Package (required)—Enter a package name or click the arrow to select a package in the current namespace.

      Do not use a reserved package name; see Reserved Package Names.

    • Name (required)—Enter a name for your data transformation class.

    • Description—Enter an description for the data transformation; this becomes the class description.

    • Source Type and Source Class—Specifies the type of messages that this transformation will receive as input.

      Choose one of the following:

      Or click the search icon for Source Class and then select the class.

    • Source Document Type (applicable only if the messages are virtual documents)—Enter or choose the document type of the source messages. You can choose any type defined in the applicable schemas loaded into this namespace.

    • Target Type and Target Class—Specifies the type of messages that this transformation will generate as output. See the choices for Source Type and Source Class.

    • Target Document Type (applicable only if the messages are virtual documents)—Enter or choose the document type of the target messages. You can choose any type defined in the applicable schemas loaded into this namespace.

    Apart from Package and Name, you can edit all these details later.

  3. Specify details on the Transform tab. See Specifying Transformation Details.

Opening an Existing Transformation

To open a transformation:

  1. Click Open.

    If you are currently viewing a transformation and you have made changes but have not yet saved them, InterSystems IRIS prompts you to confirm that you want to proceed (which will discard those changes).

  2. Click the package that contains the transformation.

    Then click the subpackage as needed.

  3. Click the transformation class.

  4. Click OK.

Specifying Transformation Details

For a transformation, the Transform tab displays details that apply to the transformation as a whole. You may or may not have already specified some of these details. Other details can be edited only here. These details are as follows:

  • Name (read-only)—Complete package and class name of the data transformation class.

  • Create—Specifies how the transformation should create the target message. Choose one of the following:

    • new—Create a new object of the target class (and type, if applicable), before executing the elements within the data transformation. This is the default.

    • copy—Create a copy of the source object to use as the target object, before executing the elements within the transform.

    • existing—Use an existing object, provided by the caller of the data transformation, as the target object. See the following subsection.

  • Source Class—Specifies the type of messages that this transformation will receive as input. For details, see Creating a Transformation.

  • Source Document Type (applicable only if the messages are virtual documents)—Specifies the document type of the source messages.

  • Target Class—Specifies the type of messages that this transformation will generate as output. For details, see Creating a Transformation.

  • Target Document Type (applicable only if the messages are virtual documents)—Specifies the document type of the target messages.

  • Language—Specifies the language you will use in any expressions in this DTL. This should be objectscript.

  • Report Errors—Specifies whether InterSystems IRIS should log any errors that it encounters when executing this transform. If you select this option, InterSystems IRIS logs the errors as Warnings in the Event Log. InterSystems IRIS also returns a composite status code containing all errors as its return value. This option is selected by default.

  • Ignore missing source segments and properties—Specifies whether to ignore errors caused by attempts to get field values out of absent source segments of virtual documents or properties of objects. If you select this option, InterSystems IRIS suppresses these errors and does not call subtransforms where the named source is absent. This option is selected by default.

    You can precisely control the behavior by including tests and conditional logic branches to confirm that any required elements are present.

  • Treat empty repeating fields as null—Specifies whether InterSystems IRIS skips the following actions for repeating fields when the fields are empty:

    • foreach actions—If you select this option, InterSystems IRIS does not execute foreach actions on repeating fields that are empty.

    • assign actions—If you select this option, InterSystems IRIS does not execute assign actions on repeating fields if you use shortcut notation to indicate that both the source and target fields are repeating fields, and the source field is empty. For example, if the source.{PV1:AdmittingDoctor()} field is empty and you select this option, then InterSystems IRIS does not execute the following action:

      <assign value='source.{PV1:AdmittingDoctor()}' property='target.{PV1:AdmittingDoctor()}' action='set'.

      However, InterSystems IRIS does execute the following similar action since the target field is not a repeating field:

      <assign value='source.{PV1:AdmittingDoctor()}' property='target.{PV1:AdmittingDoctor(1)}' action='set' />

      This option is cleared by default.

  • Description—Description of the data transformation.

Using the Create existing Option

For Create, the existing option enables you to specify the target as an existing object, which results in a performance improvement. This option applies when you invoke a series of transformations programmatically (or perform other sequential processing). You would use this option in cases like the following scenario:

  • You have three transformations that you want to perform in sequence:

    1. MyApp.ADTTransform—Uses the new option for Create.

    2. MyApp.MRNTransform—Uses the existing option for Create.

    3. MyApp.LabXTransform—Uses the existing option for Create.

  • You invoke the transforms as follows:

    do MyApp.ADTTransform.Transform(message,.target)
do MyApp.MRNTransform(target,.newtarget)
do MyApp.LabXTransform(newtarget,.outmessage)

Editing Transformation Actions

This section describes generally how to add and edit the actions in a transformation. It includes the following subsections:

Other topics describe the syntax to use in data transformations, details for assign actions, and details for other kinds of actions.

Adding an Action

To add an action, you can always do the following:

  1. Optionally click a source or target property, depending on the kind of action you want to add.

  2. Select an action from the Add Action drop-down list in the ribbon bar.

  3. Edit the details for this action on the Action tab.

    If applicable, the property that you selected is shown in the Property field, for use as a starting point. Optionally, you can disable the action with the Disabled check box. If you disable a foreach or if action, all actions within the block are also disabled.

Other techniques are possible for assign actions, as discussed in Adding Assign Actions.

Editing an Action

To edit an action, first select it. To do so:

  • Click the corresponding row in the table below the diagram.

  • If the diagram displays the action, click the icon on the corresponding connector line, in the center divider.

    When you click on an item, it changes color, the connector line turns bold, and the source and target properties are highlighted in color. This means the connector is selected, as shown in the following figure.

    The highlighted source property connected to the highlighted target property by a bold connector.

Now edit the values on the Action tab. Optionally, you can disable the action with the Disabled check box. If you disable a foreach or if action, all actions within the block are also disabled.

Tip:

If you double-click a property in the diagram, InterSystems IRIS updates the currently selected action, if applicable. If you double-click a field in the source, then the editor interprets it as your wanting to set the value for the selected action. Similarly, if you double-click a target field, the editor interprets it as your wanting to set the target for the selected action.

Rearranging Actions

InterSystems IRIS executes the actions in the order they are listed in the table below the diagram.

To rearrange actions, you must use the table below the DTL diagram, as follows:

  1. Click the row corresponding to that action.

  2. Click one of the following icons in that row, as needed:

    Tool Description
    A button with an up arrow on it.
    		 Move the selected action up one position. If the action is the first action in a group, for example a for each or if block, then this moves the action up and out of the group.
    A button with a down arrow on it.
    		 Move the selected action down one position. If the action is the last action in a group, then this moves the action just after the group. For example, if the action is the last action in an if block, the action is moved right after the block. If the action is the last action in an if block just before the else, then this moves the action into the first position in the else block.
    A button with a left arrow on it.
    		 Move the selected action out of the current group, for example a for each or if block. This moves the action out of the current group to the position immediately before the group.
    A button with a right arrow on it.
    		 Move the selected action into the next group of actions, for example, a for each or if block.
    A red button with a white bar on it.
    		 Remove all the actions of the data transformation.
    A button with a red x on it.
    		 Remove the action in this row.

Working with Groups of Actions

You can gather actions into a display group by using the group action. Grouping actions helps organize them in the table below the diagram. The description that you define on the Action tab appears in the list to help you identify a group.

A group of actions, along with their description.

To move an action in or out of a group, select the action in the list and click (move into group) or (move out of group). To make the list more readable, you can collapse or expand groups as you review the list. To collapse a group and hide the actions it contains, click downward-pointing triangle next to the action name. To expand a group, click right-pointing triangle . You can also collapse and expand all groups at once using the downward-pointing triangle and right-pointing triangle buttons in the table’s Actions bar.

The Actions bar with the downward-pointing triangle highlighted

Note:

You can also expand and collapse blocks of actions created by if , for each , switch , and case actions.

Undoing a Change

To undo the previous change, click the Undo button .

Saving a Transformation

To save a transformation, do one of the following:

  • Click Save.

  • Click Save As. Then specify a new package, class name, and description and click OK.

  • Click Compile. This option saves the transformation and then compiles it.

Compiling a Transformation

To compile a transformation, click Compile. This option saves the transformation and then compiles it.

Building DTLs During a Migration

If you intend to migrate to InterSystems IRIS for Health from other vendors, and you have a body of existing source and target messages, you can streamline the process of creating the necessary DTLs:

  • You can automatically generate a starting DTL that performs the simpler transformations.

  • You can compare the target messages output from the generated DTL to the original target messages, quickly identifying message segments that require attention.

To build a post-migration DTL using an existing body of source messages and target messages, follow these steps:

  1. Store your source messages in a directory on the system where InterSystems IRIS for Health is installed. Store your target messages in a different directory on the same system. These directories should not contain anything other than the target messages or the source messages.

  2. In the Management Portal, navigate to Interoperability > Build > Generate DTL.

  3. On the DTL Generator page, click New, then fill out the DTL Generator Wizard form as follows:

    DTL Name

    Enter a name for the new DTL. The format for the name should be package.classname

    Use Existing DTL

    Click to perform the comparison of source and target messages based on a DTL that already exists rather than generating a new DTL to perform the comparison.

    Source Directory

    Enter the name of the directory where you stored the source messages.

    Source Document Type

    Enter the document type of the source messages.

    Document types are listed in the Management Portal at Interoperability > Interoperate > HL7 v2.x > HL7 Schema Structures. Choose version from the list on the left, and permissible document types are listed on the right. The format is category:doctype. For example, 2.1:ADT_A60.

    Target Directory

    Enter the name of the directory where you stored the target messages.

    Target Document Type

    Enter the document type of the target messages.

    Document types are listed in the Management Portal at Interoperability > Interoperate > HL7 v2.x > HL7 Schema Structures. Choose version from the list on the left, and permissible document types are listed on the right. The format is category:doctype. For example, 2.1:ADT_A60.

    Use Control ID

    Check this box if the control ID in position MSH:10 is unique. Using unique control IDs ensures accurate mapping between source messages and target messages.

    If you don’t use control ID, target records and source records must be processed in the same order to ensure accurate mapping between source and target.

    Rule Class

    Note:

    You will need to create a blank rule first, so that it appears within the Rule Class field. Once the DTL is generated, you will be able to add conditions to the rule. See the Rule Editor documentation for information about rules and instructions for creating them.

    Helper Class

    If you have extended the default helper class, choose the name of your extended class from this dropdown. For information about ways you might wish to extend the default helper class, HS.InteropTools.HL7.GenerateDTL.BuildHelperOpens in a new tab, see its description in the class reference.

    Pre-Transform

    Optionally, enter a comma-delimited list of transforms to execute before the generated transform.

  4. Click OK to generate the new DTL. The new DTL is created and applied to the original body of messages, generating a set of new target messages you can compare to the original target messages to identify and address issues with the transformation. A new row with the DTL name you specified appears in the table.

  5. In the table of DTLs on the left, click the name of your DTL. On the right, the following buttons appear:

    View Comparison

    Opens the Visual Comparison page where you can view the differences between the target messages generated by your new DTL and the original target messages.

    Reload Messages

    Loads all messages in the directories where you stored your source messages and target messages, making them available for DTL processing.

    Reprocess All

    Applies the DTL, which you may have edited, to all source messages that have been loaded, generating new target messages for comparison. This is useful to make sure that your changes to the DTL have not introduced new issues.

    Reprocess Issues

    Reapplies the DTL, which you may have edited, to the messages that had issues, generating a new set of generated target messages for comparison.

    Open DTL

    Opens the DTL editor.

    Halt Processing

    Stops message processing.

  6. Identify differences between the generated target messages and the original target messages by clicking View Comparison.

    The View Comparison page displays tables comparing the original target messages to the generated target messages.

    The View Comparison page

    • Each table depicts a single target message. You can navigate between messages by clicking the arrow buttons on either side of the message number displayed above the table.

    • Each row represents a single segment of the displayed message, lining up the original target message, position-by-position, directly above the generated target message.

    • The first cell of each row is the line number, and the second is the segment name.

    • Each cell from the third column onward represents a single position in its segment. You can learn the position number for a cell by looking at that cell’s tooltip.

    • Differences are highlighted.

    • The table at the bottom of the page displays the original source message that was transformed to create the target message under comparison.

  7. Address each issue by modifying your DTL. After fixing one or more issues, you can click Reprocess to apply the modified DTL and see the corrected output. See the topics included in Creating Data Transformations for instructions about how to address issues.

    You may find it useful to open the DTL generator on multiple browser tabs, one tab displaying the Visual Comparison page and another displaying the DTL editor, so that you can modify the DTL in one tab and view the results in the other.

  8. Go back and forth between editing the DTL and viewing the remaining differences until you are satisfied with the effectiveness and accuracy of your DTL.

Deleting a Transformation

To delete a transformation, you must use a different page, the Interoperability > List > Data Transformations page.

To delete a transformation:

  1. Click the row that displays its name.

  2. Click the Delete button.

  3. Click OK to confirm this action.

Syntax RulesNext section
Previous sectionIntroduction to DTL Tools
FeedbackOpens in a new tab