Skip to main content

Defining Search Tables

This chapter describes generally how to define search tables for virtual documents. It includes the following topics:

Be sure to perform these tasks in the same namespace that contains your production. When you create search tables, do not use reserved package names; see “Reserved Package Names” in Developing Ensemble Productions.

Note:

Ensemble does not retroactively index messages that were received before you added the search table class.

Defining a Search Table Class

To define a search table class, use the following general procedure:

  • Create the class in the same namespace that contains your production. Also, do not use reserved package names; see “Reserved Package Names” in Developing Ensemble Productions.

  • Create a subclass (or a copy, as you prefer) of the default search table class used for your type of virtual document:

    Document Type Default Search Table Class Notes
    HL7 EnsLib.HL7.SearchTableOpens in a new tab Indexes a set of commonly needed properties; see “Properties That Are Indexed by Default” in the Ensemble HL7 Version 2 Development Guide.
    X12 EnsLib.EDI.X12.SearchTableOpens in a new tab Indexes the Identifier property, which corresponds to the X12 document ID.
    ASTM EnsLib.EDI.ASTM.SearchTableOpens in a new tab Indexes the Identifier property, which corresponds to the ASTM document ID.
    EDIFACT EnsLib.EDI.EDIFACT.SearchTableOpens in a new tab Indexes the Identifier property, which corresponds to the EDIFACT document ID.
    XML EnsLib.EDI.XML.SearchTableOpens in a new tab Indexes the name of the root element of the XML document.
  • In this subclass, include an XData block that defines the virtual properties as needed. The following subsection provides the details.

    This XData block does not need to include details about the virtual properties that are indexed by the superclass that you chose. For example, several of the search table classes index a property named Identifier. If you subclass any of those classes, your XData block does not need to include Identifier. You can improve search efficiency by identifying properties that are not selective. Properties are not selective when many messages have identical values for the property. When Ensemble is searching messages, it can be more efficient if it uses selective properties to limit the number of messages found before it tests the values of properties that are not selective. You can specify that a property is not selective by specifying Unselective="true" in the XData block.

  • If this search table class is mapped to multiple namespaces, compile it in each of those namespaces, to ensure that the metadata local to each namespace is up to date.

    Important:

    Search table metadata is located in the default global database for each Ensemble namespace; therefore, changes to a search table class do not update metadata in all namespaces to which the class is mapped. For this reason, it is necessary to recompile the search table class in each of these namespaces.

When you compile this class, Ensemble generates code that dynamically fetches the local metadata for each search table property and then caches the metadata if the process is running as an Ensemble host. If the property metadata is not present, as in the case where a mapped search table class does not have local metadata for a new property, the class still indexes all other properties and returns an error to indicate the metadata was not present. Similarly, when the message bodies are deleted, Ensemble removes the corresponding entries from the search table; no work is required on your part.

To use the search table class, specify it as a configuration option (called Search Table Class) for the applicable business host. When that business host processes messages, it uses the configured search table class to index those messages.

XData Details for a Search Table Class

When you create a search table class, your goal is to provide one search table entry for each virtual property that you want to search and filter in the Message Browser, Rules Editor, and other parts of the Management Portal. To do this, you include an XData block like the following stub within your search table class:

XData SearchSpec [ XMLNamespace="http://www.intersystems.com/EnsSearchTable" ]
{
<Items>
   <Item DocType="doctype1" PropName="name1" PropType="type1" StoreNulls="boolean" 
     Unselective="true">path1</Item>
   <Item DocType="doctype2" PropName="name2" PropType="type2" StoreNulls="boolean">path2</Item>
<  <Item DocType="doctype3" PropName="name3" PropType="type3" StoreNulls="boolean">path3</Item>
</Items>
}

Where:

  • path1, path2, path3, and so on are virtual property paths.

    Each of these is a string expression. This string expression may include the following components in any combination:

    • Literal characters within double quotes.

    • Virtual property syntax within {} or []. This resolves to the value of the specified field in the X12 document. Square brackets differ from curly brackets in that square brackets enclose a segment:field combination that does not require you to identify its containing document structure. For curly bracket syntax to resolve, the document structure must be known.

    • ObjectScript string operators, such as the underscore (_) for concatenation.

    • Functions, such as $PIECE or $EXTRACT.

  • doctype1, doctype2, doctype3, and so on are DocType identifiers. Each of these is a schema category name and document structure name, separated by a colon, as follows:

    category:structure
    

    If the category is missing, any schema is matched; if the structure is missing; any structure is matched. A value of "" (a blank string) matches any schema category and any document structure.

    This part of the <Item> specifies the DocType (or DocTypes) for which the given property path is to be indexed.

  • name1, name2, name3, and so on are virtual property names of your choice.

    This is the name that Ensemble will display in the Management Portal. Choose any string that you expect to be meaningful, when viewed with others in the list.

    If you assign the same name to different <Item> elements, this has a convenient additive effect: When the user selects this name for a search, all of the entries with the same name are searched.

  • type1, type2, type3, and so are optional type identifiers. Specify one of the following literal values:

    • String:CaseSensitive

    • String:CaseInsensitive

    • Integer

    • Numeric

    • Boolean

    • DateTime:ODBC

    • DateTime:HL7 (supported only for virtual documents that carry HL7, ASTM, or EDIFACT messages)

    String:CaseSensitive is the default.

  • boolean is an optional flag that controls what happens when a search encounters an empty field in the document. If this flag is true, Ensemble returns a valid pointer to an empty string. If this flag is false, Ensemble returns a Not Found status and a null pointer.

    Specify either 1 (which means true) or 0 (which means false). The default is 0.

  • Unselective="true" specifies that a property value typically does not select a small number of messages. Ensemble uses this information to search more efficiently. The default value is Unselective="false".

Important:

When Ensemble indexes virtual documents (thus adding to the search tables), it replaces any vertical bar (|) with a plus sign (+). Take this into consideration when you use the search table to search for content. For example, to search for a message that contains my|string, use my+string as the search criterion.

Example Search Table Class

The following example shows a search table class. The SearchSpec XData block contains the <Item> elements that define the search table.

Class Demo.HL7.MsgRouter.SearchTable Extends EnsLib.HL7.SearchTable
{

XData SearchSpec [ XMLNamespace="http://www.intersystems.com/EnsSearchTable" ]
{
<Items>
   <!-- Items that do not depend on DocType, indexing any HL7 message -->
   <Item DocType="" PropName="SendingFacilApp" >{1:4}_"|"_{1:3}</Item>
   <Item DocType="" PropName="RecvingFacilApp" >{1:6}_"|"_{1:5}</Item>
   <Item DocType="" PropName="MSHDateTime" PropType="DateTime:HL7" >{1:7}</Item>

   <!-- Get fields from named segments found in any HL7 message -->
   <Item DocType="" PropName="PatientName" >[PID:5]</Item>
   <Item DocType="" PropName="InsuranceCo" >[IN1:4]</Item>

   <!-- Get patient name from any HL7 message declared type ADT_A05 -->
   <Item DocType=":ADT_A05" PropName="PatientName" >{3:5}</Item>

   <!-- Get specific field from specific segment when the        -->
   <!-- HL7 message is assigned a specific DocType. Only in this -->
   <!-- case can you use names for segments, instead of numbers. -->
   <Item DocType="Demo.HL7.MsgRouter.Schema:ORM_O01 " PropName="ServiceId" >
     {ORCgrp().OBRuniongrp.OBRunion.OBR:UniversalServiceID.text}
   </Item>
   <Item DocType="2.3.1:ORU_R01 " PropName="ServiceId" >
     {PIDgrpgrp().ORCgrp(1).OBR:UniversalServiceID.text}
   </Item>
</Items>
}
} 

The XMLNamespace declaration (as shown in the preceding example) enables Studio to provide word completion as you type.

Defining Custom Search Table Classes

In some cases, the basic search table mechanism described in this chapter might not enable you to index messages as needed. In such cases, you can define and use custom search table classes.

The class can define two kinds of properties. Within this topic, these properties are called: standard properties (which are stored in the search table) and virtual properties (which are not stored in the search table but instead are retrieved at runtime). Either kind of property is either indexed or not. If you index a property, more disk space is consumed but queries for that property run more quickly. The Management Portal displays the indexed properties as a group above the non-indexed ones, so that users can select them appropriately.

To define a custom search table class, define a class as follows:

  • Extend Ens.CustomSearchTableOpens in a new tab.

    This class defines one standard class property, DocId, which is indexed.

  • Define additional class properties as needed, and add indices for these class properties. For example:

    Property Type As %String(COLLATION = "EXACT");
    
    Index Type On Type [ Type = bitmap ];
    

    Note that collection properties are not currently directly supported by the query generation mechanisms. For a collection property, use the GetVirtualPropertyList() method mechanism described below.

  • Optionally implement the GetPropertyList() method as needed.

    classmethod GetPropertyList(Output pIndexedProperties As %List, Output pProperties As %List) as %Status
    

    Where:

    • pIndexProperties is a $LISTBUILD list of standard properties that should be indexed.

    • pProperties is a $LISTBUILD list of standard properties to define.

    The purpose of this step is to indicate which class properties are to be used as standard properties (see the definitions before this list), as well as which of those should be indexed.

    By default, this method is generated, and Ensemble uses all class properties of the search table class as standard properties and indexes them all, except for private, internal, transient, and multidimensional properties.

    For virtual properties, implement GetVirtualPropertyList() instead (or in addition).

  • Optionally implement the GetVirtualPropertyList() method as needed.

    classmethod GetVirtualPropertyList(Output GetVirtualPropertyList As %List, 
                                       Output pVirtualProperties  As %List) 
                                       as %Status
    

    Where:

    • GetVirtualPropertyList is a $LISTBUILD list of virtual properties that should be indexed.

    • pVirtualProperties is a $LISTBUILD list of the virtual properties to define.

    The purpose of this step is to indicate which class properties are to be used as virtual properties, as well as which of those should be indexed.

    For standard properties, implement GetPropertyList() instead (or in addition).

  • If you implement GetVirtualPropertyList(), also implement the GetVirtualProperty() method. This method must return the value of a virtual property, given a document ID and a virtual property name:

    classmethod GetVirtualProperty(pDocID As %String, 
                                   pPropName As %String, 
                                   Output pPropValue As %String,
                                   ByRef pUserArgs) as %Status
    

    Where:

    • pDocID is the ID of a document in the custom search table.

    • pPropName is the name of a virtual property.

    • pPropValue is the value of that property.

    • pUserArgs specifies any arguments.

  • Implement the OnIndexDoc() method.

    ClassMethod OnIndexDoc(pDocObj As %Persistent, pSearchTable As Ens.CustomSearchTable) As %Status
    

    This method should specify how to populate a given row in the search table from properties in a supplied message.

For OnProcessCondition() and additional options, see the class reference for Ens.CustomSearchTableOpens in a new tab.

For an example, see Demo.CustomSearchTable.SampleOpens in a new tab.

Management of Search Tables

The class Ens.DocClassMapOpens in a new tab manages all the search tables (including custom search tables). It writes to and reads from a global (^Ens.DocClassMap). This global indicates, for each message class, which search tables contain data for it. Note that you should never edit this global directly.

The Ensemble classes use this class to remove search table entries when message bodies are deleted.

It should not normally be necessary to use this class directly. However, if the data in ^Ens.DocClassMap is lost or damaged, use the RebuildMap() method of this class to recreate the global. For details, see the class reference for Ens.DocClassMapOpens in a new tab.

Customizing Queries Used by the Management Portal

When users search for messages in the Message Viewer and the Message Bank Message Viewer pages in the Management Portal, Ensemble generates and then uses queries. In advanced cases, you can customize how Ensemble generates these queries. To do so, use the following general procedure:

  • Define a subclass of EnsPortal.MsgFilter.AbstractAssistantOpens in a new tab. For details, see the class reference for that class.

  • Set the name of the class into ^EnsPortal.Settings("MessageViewer","AssistantClass") for the Message Viewer or ^EnsPortal.Settings("MsgBankViewer","AssistantClass") for the Message Bank Viewer.

FeedbackOpens in a new tab