Skip to main content

Message Analyzer

The Message Analyzer is a command-line utility that allows you to:

  • Scan HL7 messages and compare them to a given document structure, deriving a new document structure that matches the messages. Most commonly, the derived document structure contains Z segments found in the scanned messages.

  • Validate messages against a given document structure, optionally updating the custom schema’s segments, data structures, and code tables to allow failing messages.

The Message Analyzer uses the following terminology:

Term Description
Document structure A HL7 message structure such as ADT_01. In the Management Portal, a schema’s document structures are listed on the DocType Structure tab.
HL7 configuration Collectively refers to the schemas, document structures, segments, data structures, and code tables that are defined in the Management Portal.
Library schema A base schema, for example 2.7.1, that is available in the Management Portal. A custom schema is based on one of these library schemas.

Running the Message Analyzer

To run the Message Analyzer:

  1. Open the InterSystems Terminal.

  2. Change to the namespace that contains or will contain the HL7 interoperability productions that route your HL7 messages. This is the namespace where the custom schema will be stored. For example, enter:

    set $namespace="MyRoutingNamespace"
    
  3. Enter:

    Do ##class(EnsLib.InteropTools.HL7.MessageAnalyzer).Interactive()
    

The Message Analyzer presents you with a series of prompts that lead you through the possible options. Entering ^ at any prompt goes back to the previous input or exits from the program, depending on the context. If only one option in a menu applies, the menu is not be shown and the option is automatically selected for you. When a prompt uses a <value> syntax, pressing Enter uses the value as the answer to the prompt.

Initial Setup

The first time you run the Message Analyzer, it prompts you for a workspace folder and information about the HL7 schema and document structure.

The next time you run the Message Analyzer, it skips this initial setup and starts with the main menu. If you need to switch the workspace folder, load different messages or use a different schema/document structure, choose the Setup Workspace option from the main menu.

Workspace Folder

When you run the Message Analyzer for the first time, it prompts you for a workspace folder:

Workspace folder:

Enter the name of a file-system folder (directory) that does not currently exist. This folder is used as a scratch workspace. During its initial setup, the Message Analyzer rejects an existing folder to avoid overwriting files. If you rerun the setup later, you can use the existing workspace or create a new one.

Source Folder

The following prompt asks you to specify the source file or folder:

Source file or folder:

Enter either a HL7 message file or a folder containing HL7 message files, which is a text file with each line representing a segment. A message file can contain any number of messages; the MSH segment is used to detect where a new message begins. The file can also contain a batch of messages.

Schema

The following prompt asks you to specify a schema:

Schema:

The entry must be one of the following:

  • An existing custom schema.

  • An existing library schema (for example, 2.4). Because this schema cannot be used directly, you are prompted for the name of a new custom schema to be based on the library schema.

  • A new custom schema. You are prompted for an existing library schema to base the custom version on.

When the Message Analyzer derives a new document structure, it makes changes to this custom schema based on the input messages. The Message Analyzer also updates the custom schema rather than a library schema when validating and modifying segments, data structures, and code tables to allow failing messages.

Document structure

The following prompt asks you to specify a document structure.

Existing document structure name, or a new one to be copied from another schema:

Enter the name of a document structure (e.g., ADT_A01) that exists in the custom schema or the name of a document structure which exists in some other schema. To choose from a list of document structures in the custom schema, enter the start of a document structure name, followed by ?. For example, entering A? might list ADT_A01 and ADT_A02. If a new document structure name is entered, you are prompted to specify a library schema where it exists so it can be copied to the custom schema.

When deriving a new document structure, the Message Analyzer compares the structure of the input HL7 messages to the specified document structure. When validating messages, the Message Analyzer compares the segments, data structures, and code tables of the specified document structure to those in the messages.

In the Management Portal, document structures are listed on the DocType Structure tab of the schema.

Main Menu

Once the Message Analyzer has been through its initial setup at least once, the main menu appears:

[1] Derive new document structures  
[2] Validate messages, optionally fixing the configuration  
[3] View history of fixes to the configuration (validation fixes only)  
[4] Setup Workspace

If you choose the Derive new document structures option, the Message Analyzer matches the input messages to the document structure specified during setup, and then derives a new document structure based on the structure of the messages.

If you choose the Validate messages, optionally fixing the configuration option, the Message Analyzer compares the input messages against the specified document structure, and then proposes changes to the schema that would allow failing messages to pass validation.

If you choose the View history of fixes to the configuration option, the Message Analyzer lists the changes that it has made to the custom schema.

If you choose the Setup Workspace option, the Message Analyzer re-runs the intitial setup process, allowing you to specify a new workspace, custom schema, or document structure.

Derive New Document Structures

The Derive new document structure option matches the input HL7 messages against the document structure specified during setup, and derives a new document structure based on the structure of the messages. For example, the derived document structure might include Z segments used by the messages.

Because the process of analyzing messages can be time-consuming, the Derive new document structures option prompts you to enter the percentage of messages to scan.

Enter a number between 1 and 100, or ^ to quit.

The results look like:

Begin document structure derivation run
Loading document structure definition
Match messages to document structure: ORU_R01
Derive new document structure
Match messages to derived document structure
Finished.
Summary Report for workspace /Users/MSmith/wf2temp
Schema is MySchema24
 ALL: 19 matched, 53 unmatched, derived 1 new document structures

If a new document structure is derived, the segments of the new document structure is displayed. In this case, you are asked whether you want to update the document structure in the custom schema.

Updating MySchema24:ORU_R01 Would you like to update the document structure in the database? (Y/N):

Validate Messages and Fix Configuration

The Validate messages, optionally fixing the configuration option of the Message Analyzer validates HL7 messages against the segments, data structures, and code tables of the custom schema. If these do not exist in the custom schema, then the Message Analyzer validates against the library schema on which the custom schema is based. Once validated, the Message Analyzer offers to change the custom schema based on the segments, data structures, and code tables of the input messages so the messages pass validation against the schema.

Workflow/2 - Validation
 [1] Select Field Validation [ ]
 [2] Select Component Validation [ ]
 [3] Select Code Table Validation [ ]
 [4] Start Validation
Enter one of the above options, ^ to go back a level:

Select one of the first three entries to indicate whether the Message Analyzer will perform that type of validation. If a menu line ends with "[ ]", the item will not be used in validation. If a menu line ends with "[X]", the item will be used in validation. When you have selected one or more entries, enter 4 to start validation.

Field validation checks whether the segments in the messages conform to segment definitions in the specified document structure. Component validation checks whether the data structures in the messages conform to the data structure definitions in the document structure. Code table validation checks whether the code tables in the messages match the code tables in the document structure.

When validation is complete, a report is shown:

Validating messages .. 
72 messages: 18-ok/54-failed (was 34/38) - 25%/75% (was 47%/53%) 
105 auto-fixable validation error(s) found

Fixing the Custom Schema

After validation, the Message Analyzer prompts you whether you want to make the possible fixes it has identified. If there is more than one possible fix, you are prompted to drill down to individually choose which fixes you want to make. When you choose a fix, it is not executed immediately, but rather is added to a queue. When you are done selecting fixes, you are prompted whether you want to accept all the fixes in the queue.

If you want to accept a subset of the possible fixes, use the prompts to identify the fixes you want to make, then enter ^ until you are prompted to apply the fixes.

Selecting a Category of Fixes

If you ran more than one type of validation (field, component, and code table), it is possible that the Message Analyzer found potential fixes for more than one category. In this case, you are prompted to specify what type of fixes you want to make. If the Message Analyzer found possible fixes in only one category (segment, data structure, or code table), this prompt is skipped.

Select a category of fixes to process
 [1] 70 code table(s)
 [2] 7 data structure(s)
 [3] 28 segment structure(s)
Enter one of the above options, ^ to go back a level:
Selecting a Structure, Segment, or Table

If there are possible fixes for multiple segments, data structures or code tables, you are prompted to select which fix you want to make. Once you select one of the fixes, it is removed from the list and you can select another fix. If you are done selecting fixes and do not want to accept them all, enter ^ until you are prompted to accept the fixes.

For example, suppose there are fixes for more than one data structure. You might see a prompt like:

Select a data structure to process
 [1] 2.4:CE coded element (1)
 [2] 2.4:HD hierarchic designator (1)
 [3] 2.4:ID coded value for HL7 defined tables (1)
 [4] 2.4:IS coded value for user-defined tables (1)
 [5] 2.4:MSG Message Type (1)
 [6] 2.4:PL person location (1)
 [7] 2.4:ST string data (1)
Enter one of the above options, ^ to go back a level:
Adding Fixes to the Queue

In most cases, each time you select a segment, data structure or code table, you are prompted whether you want to accept the fix. When you enter Y, the fix is not applied immediately; it is added to a queue of fixes that will be applied when you give a final confirmation. If there is more than one fix to the same code table, segment, or data structure, you are prompted to select which fix you want to make before adding it to the queue.

Making the Fixes

When you have added all the possible fixes to the queue or have indicated you are done adding fixes by using ^, you are prompted whether you want to apply the queued fixes to the custom schema. Items that are not already in the custom schema will be copied from the built-in library schema before the changes are made.

For example, you might be prompted to apply the following queue of changes to the custom schema:

Items queued for fixes to HL7 configuration:
 *Do you want to add code 'Z' to code table 2.4:1 (Administrative sex)
 *Do you want to add code 'EC' to code table 2.4:131 (!Copied from 2.5 - Contact Role)
 *Do you want to add 3 dummy components to data structure 2.4:CE (coded element)
 *Do you want to add 1 dummy components to data structure 2.4:IS (coded value for user-defined tables)
 *Do you want to increase the maximum size for field 2 (EncodingCharacters) in segment structure 2.4:MSH (Message Header) from 4 to 5
 *Do you want to make field 7 (DateTimeOfMessage) in segment structure 2.4:MSH (Message Header) optional

Fixes marked with a '*' apply to library items, which cannot be updated directly
Do you want to apply these fixes? (Y/N):
Understanding the Changes

Before the relevant fix can be made (for example, adding a new code to a code table), the Message Analyzer might need to:

  • Copy items (for example, code tables) to the custom schema.

  • Update custom items to reference other custom items instead of the library items they formerly referenced.

These changes are detailed in the update report which follows your confirmation that the fixes should be applied. Each copy, reference-update or principal change is detailed on its own line.

Lines beginning with Path: summarize updates to document structures, segment structures, data structures and code tables. In such a line, a + means that the item was copied to the custom schema; a * means that the given reference was updated.

For example, a path of ORU_R01 --> PID -*-> XTN+ -*-> 202+ means:

  • custom document structure ORU_R01

  • custom segment PID: reference to XTN was updated

  • data structure XTN: copied to custom schema, reference to 202 was updated

  • code table 202: copied to custom schema

The path does not describe the principal update target identified by the validation - in the example above this would be the adding of a code to the code table.

History of Fixes

When you select the View history of fixes to the configuration (validation fixes only) option from the main menu, you can generate a list of the changes the Message Analyzer has made to custom schema in your HL7 configuration. Select one of the options, then follow the prompts. You are given the option to write the history to a file.

Workflow/2 History of Configuration Changes
 [1] List all changes
 [2] List changes made today
 [3] List changes for a range of dates
Enter one of the above options, ^ to go back a level:

Setup Workspace

Choosing the Setup Workspace option from the main menu allows you to create a new workspace, import a new set of messages or change the schema and document structure. The Message Analyzer restarts — as if for the first time — and prompts you for the workspace, message source, schema, and document structure. If you enter a path to an existing workspace, you are asked whether you want to continue with its current setup or to clear it and copy a new set of messages into it.

FeedbackOpens in a new tab