Skip to main content

Defining Search Tables for Virtual Documents

This page describes generally how to define search tables for virtual documents.

Be sure to perform these tasks in the same namespace that contains your production. When you create search tables, do not use reserved package names; see Reserved Package Names.

Note:

InterSystems IRIS® data platform does not retroactively index messages that were received before you added the search table class. You can use the BuildIndex() method of the search table class to index these pre-existing messages.

Defining a Search Table Class

To define a search table class, use the following general procedure:

  • Create the class in the same namespace that contains your production. Also, do not use reserved package names; see Reserved Package Names.

  • Create a subclass (or a copy, as you prefer) of the default search table class used for your type of virtual document:

    Document Type Default Search Table Class Notes
    X12 EnsLib.EDI.X12.SearchTableOpens in a new tab Indexes the Identifier property, which corresponds to the X12 document ID.
    EDIFACT EnsLib.EDI.EDIFACT.SearchTableOpens in a new tab Indexes the Identifier property, which corresponds to the EDIFACT document ID.
    XML EnsLib.EDI.XML.SearchTableOpens in a new tab Indexes the name of the root element of the XML document.
  • In this subclass, include an XData block that defines the virtual properties as needed. The following subsection provides the details.

    This XData block does not need to include details about the virtual properties that are indexed by the superclass that you chose. For example, several of the search table classes index a property named Identifier. If you subclass any of those classes, your XData block does not need to include Identifier. You can improve search efficiency by identifying properties that are not selective. Properties are not selective when many messages have identical values for the property. When InterSystems IRIS is searching messages, it can be more efficient if it uses selective properties to limit the number of messages found before it tests the values of properties that are not selective. You can specify that a property is not selective by specifying Unselective="true" in the XData block.

  • If this search table class is mapped to multiple namespaces, compile it in each of those namespaces, to ensure that the metadata local to each namespace is up to date.

    Important:

    Search table metadata is located in the default global database for each interoperability-enabled namespace; therefore, changes to a search table class do not update metadata in all namespaces to which the class is mapped. For this reason, it is necessary to recompile the search table class in each of these namespaces.

When you compile this class, InterSystems IRIS generates code that dynamically fetches the local metadata for each search table property and then caches the metadata if the process is running as an InterSystems IRIS host. If the property metadata is not present, as in the case where a mapped search table class does not have local metadata for a new property, the class still indexes all other properties and returns an error to indicate the metadata was not present. Similarly, when the message bodies are deleted, InterSystems IRIS removes the corresponding entries from the search table; no work is required on your part.

To use the search table class, specify it as a configuration option (called Search Table Class) for the applicable business host. When that business host processes messages, it uses the configured search table class to index those messages.

XData Details for a Search Table Class

When you create a search table class, your goal is to provide one search table entry for each virtual property that you want to search and filter in the Message Browser, Rules Editor, and other parts of the Management Portal. To do this, you include an XData block like the following stub within your search table class:

XData SearchSpec [ XMLNamespace="http://www.intersystems.com/EnsSearchTable" ]
{
<Items>
   <Item DocType="doctype1" PropName="name1" PropType="type1" StoreNulls="boolean" 
     Unselective="true">path1</Item>
   <Item DocType="doctype2" PropName="name2" PropType="type2" StoreNulls="boolean">path2</Item>
<  <Item DocType="doctype3" PropName="name3" PropType="type3" StoreNulls="boolean">path3</Item>
</Items>
}

Where:

  • path1, path2, path3, and so on are virtual property paths.

    Each of these is a string expression. This string expression may include the following components in any combination:

    • Literal characters within double quotes.

    • Virtual property syntax within {} or []. This resolves to the value of the specified field in the X12 document. Square brackets differ from curly brackets in that square brackets enclose a segment:field combination that does not require you to identify its containing document structure. For curly bracket syntax to resolve, the document structure must be known.

    • ObjectScript string operators, such as the underscore (_) for concatenation.

    • Functions, such as $PIECE or $EXTRACT.

  • doctype1, doctype2, doctype3, and so on are DocType identifiers. Each of these is a schema category name and document structure name, separated by a colon, as follows:

    category:structure
    

    If the category is missing, any schema is matched; if the structure is missing; any structure is matched. A value of "" (a blank string) matches any schema category and any document structure.

    This part of the <Item> specifies the DocType (or DocTypes) for which the given property path is to be indexed.

  • name1, name2, name3, and so on are virtual property names of your choice.

    This is the name that InterSystems IRIS will display in the Management Portal. Choose a string that you expect to be meaningful, when viewed with others in the list. A virtual property name cannot have spaces.

    If you assign the same name to different <Item> elements, this has a convenient additive effect: When the user selects this name for a search, all of the entries with the same name are searched.

  • type1, type2, type3, and so are optional type identifiers. Specify one of the following literal values:

    • String:CaseSensitive

    • String:CaseInsensitive

    • Integer

    • Numeric

    • Boolean

    • DateTime:ODBC

    • DateTime:HL7 (supported only for virtual documents that carry EDIFACT messages)

    String:CaseSensitive is the default.

  • boolean is an optional flag that controls what happens when a search encounters an empty field in the document. If this flag is true, InterSystems IRIS returns a valid pointer to an empty string. If this flag is false, InterSystems IRIS returns a Not Found status and a null pointer.

    Specify either 1 (which means true) or 0 (which means false). The default is 0.

  • Unselective="true" specifies that a property value typically does not select a small number of messages. InterSystems IRIS uses this information to search more efficiently. The default value is Unselective="false".

Important:

When InterSystems IRIS indexes virtual documents (thus adding to the search tables), it replaces any vertical bar (|) with a plus sign (+). Take this into consideration when you use the search table to search for content. For example, to search for a message that contains my|string, use my+string as the search criterion.

Defining Custom Search Table Classes

In some cases, the basic search table mechanism described in this page might not enable you to index messages as needed. In such cases, you can define and use custom search table classes.

The class can define two kinds of properties. Within this topic, these properties are called: standard properties (which are stored in the search table) and virtual properties (which are not stored in the search table but instead are retrieved at runtime). Either kind of property is either indexed or not. If you index a property, more disk space is consumed but queries for that property run more quickly. The Management Portal displays the indexed properties as a group above the non-indexed ones, so that users can select them appropriately.

To define a custom search table class, define a class as follows:

  • Extend Ens.CustomSearchTableOpens in a new tab.

    This class defines one standard class property, DocId, which is indexed.

  • Define additional class properties as needed, and add indexes for these class properties. For example:

    Property Type As %String(COLLATION = "EXACT");
    
    Index Type On Type [ Type = bitmap ];
    

    Note that collection properties are not currently directly supported by the query generation mechanisms. For a collection property, use the GetVirtualPropertyList() method mechanism described below.

  • Optionally implement the GetPropertyList() method as needed.

    classmethod GetPropertyList(Output pIndexedProperties As %List, Output pProperties As %List) as %Status
    

    Where:

    • pIndexProperties is a $LISTBUILD list of standard properties that should be indexed.

    • pProperties is a $LISTBUILD list of standard properties to define.

    The purpose of this step is to indicate which class properties are to be used as standard properties (see the definitions before this list), as well as which of those should be indexed.

    By default, this method is generated, and InterSystems IRIS uses all class properties of the search table class as standard properties and indexes them all, except for private, internal, transient, and multidimensional properties.

    For virtual properties, implement GetVirtualPropertyList() instead (or in addition).

  • Optionally implement the GetVirtualPropertyList() method as needed.

    classmethod GetVirtualPropertyList(Output GetVirtualPropertyList As %List, 
                                       Output pVirtualProperties  As %List) 
                                       as %Status
    

    Where:

    • GetVirtualPropertyList is a $LISTBUILD list of virtual properties that should be indexed.

    • pVirtualProperties is a $LISTBUILD list of the virtual properties to define.

    The purpose of this step is to indicate which class properties are to be used as virtual properties, as well as which of those should be indexed.

    For standard properties, implement GetPropertyList() instead (or in addition).

  • If you implement GetVirtualPropertyList(), also implement the GetVirtualProperty() method. This method must return the value of a virtual property, given a document ID and a virtual property name:

    classmethod GetVirtualProperty(pDocID As %String, 
                                   pPropName As %String, 
                                   Output pPropValue As %String,
                                   ByRef pUserArgs) as %Status
    

    Where:

    • pDocID is the ID of a document in the custom search table.

    • pPropName is the name of a virtual property.

    • pPropValue is the value of that property.

    • pUserArgs specifies any arguments.

  • Implement the OnIndexDoc() method.

    ClassMethod OnIndexDoc(pDocObj As %Persistent, pSearchTable As Ens.CustomSearchTable) As %Status
    

    This method should specify how to populate a given row in the search table from properties in a supplied message.

For OnProcessCondition() and additional options, see the class reference for Ens.CustomSearchTableOpens in a new tab.

For an example, see Demo.CustomSearchTable.Sample.

Management of Search Tables

The class Ens.DocClassMapOpens in a new tab manages all the search tables (including custom search tables). It writes to and reads from a global (^Ens.DocClassMap). This global indicates, for each message class, which search tables contain data for it. Note that you should never edit this global directly.

The InterSystems IRIS Interoperability classes use this class to remove search table entries when message bodies are deleted.

It should not normally be necessary to use this class directly. However, if the data in ^Ens.DocClassMap is lost or damaged, use the RebuildMap() method of this class to recreate the global. For details, see the class reference for Ens.DocClassMapOpens in a new tab.

You can also use the class Ens.VDoc.SearchTableOpens in a new tab to manage registered search table index properties. These are applied from Search Criteria dialogue of the Message Viewer to construct search table criteria message inquiries. An existing search table property may become obsolete when a deployed custom search table definition is modified. These obsolete index property entries continue to display in the Message Search table criteria until removed. Ens.VDoc.SearchTableOpens in a new tab includes methods for managing these properties:

  • DeleteProps() — method for deleting search table property indexes

  • DeleteObsoleteProps() — method for removing obsolete property indexes and truncating obsolete index search data

Removing obsolete properties allows maintenance of search tables without having to rebuild their entire indexes.

Customizing Queries Used by the Management Portal

When users search for messages in the Message Viewer and the Message Bank Message Viewer pages in the Management Portal, InterSystems IRIS generates and then uses queries. In advanced cases, you can customize how InterSystems IRIS generates these queries. To do so, use the following general procedure:

  • Define a subclass of EnsPortal.MsgFilter.AbstractAssistantOpens in a new tab. For details, see the class reference for that class.

  • Set the name of the class into ^EnsPortal.Settings("MessageViewer","AssistantClass") for the Message Viewer or ^EnsPortal.Settings("MsgBankViewer","AssistantClass") for the Message Bank Viewer.

FeedbackOpens in a new tab