First Look: InterSystems IRIS and UIMA

Creating and Invoking a UIMA Analysis PipelineCopy link to this section

InterSystems IRIS uses a functional index to create a UIMA analysis pipeline using InterSystems IRIS concepts without needing to worry about implementing Java interfaces. A functional index is a feature of the InterSystems IRIS database that allows a function to be executed when a record is inserted or updated in a table. In this case, the functional index is defined on a table column that contains the unstructured data that you want analyzed by the UIMA analysis pipeline. Setting up the pipeline is as easy as adding the location of the analysis engines’ descriptor files to the functional index definition.

Once the functional index is defined, InterSystems IRIS automatically feeds unstructured data into the UIMA analysis pipeline whenever new data is inserted or updated in the indexed table column. For example, if the functional index is defined on a column that contains reports, then a new report would be analyzed as soon as it is added to the table. Without this special functionality in InterSystems IRIS, you would need to send unstructured data through the pipeline programmatically in Java every time you wanted to analyze the data.

Annotation StoreCopy link to this section

By default, the results of a UIMA analysis pipeline are captured in verbose and cumbersome XML files. Because the UIMA standard does not provide a more sophisticated method of storing the annotations, InterSystems IRIS extends an UIMA analysis pipeline by using flexible, SQL-based storage to put the annotations in uniform, persistent tables for later retrieval. This storage system is called the Annotation Store.

This Annotation Store is created automatically the first time you compile the class that contains the functional index you defined to create the UIMA analysis pipeline. It is linked directly to the column in the original table that contains the unstructured data.

Architecturally, the Annotation Store is produced by adding a special analysis engine as the last component of a UIMA analysis pipeline. This happens automatically when you add a UIMA functional index to an InterSystems IRIS class. It’s also possible for a UIMA analysis pipeline developed outside of InterSystems IRIS to add this special analysis engine to the end of the pipeline to create an Annotation Store. Such an implementation is beyond the scope of this First Look.

You can also customize the Annotation Store using an XData block in the class that contains the functional index. For example, you can define additional columns and indices per table. You can also filter annotation types to keep them out of the Annotation Store.

InterSystems IRIS NLPCopy link to this section

InterSystems IRIS Natural Language Processing (NLP) is embedded into InterSystems IRIS® data platform and allows you to perform text analysis on unstructured text without any upfront knowledge of the subject matter. It does this by applying language-specific rules that identify semantic entities. Because these rules are specific to the language, not the content, InterSystems IRIS NLP can provide insight into the contents of texts without using a dictionary or ontology.

You can use InterSystems IRIS NLP as a UIMA analysis engine, generating UIMA annotations for NLP concepts and contexts. These annotations are fully compatible with UIMA annotations supplied by other UIMA analysis engines.

Before You BeginCopy link to this section

To get started, perform the following preliminary setup tasks:

1. Install the Java Runtime Environment.

2. Install InterSystems IRIS.

3. Create a new InterSystems IRIS namespace.

4. Start the Java Gateway.

Adding Class File with a UIMA Functional IndexCopy link to this section

You add an analysis engine to the UIMA analysis pipeline by defining a functional index for the table that contains the unstructured text. In this tour, you are adding the InterSystems IRIS NLP analysis engine to the pipeline.

In this part of the tour, you are creating a new class file. You can create the class file in your favorite text editor if you do not have the Atelier IDE set up.

1. Create a new file in Atelier or a text editor.

2. Copy and paste the following into the class file:

   Class Sample.MyData Extends %Persistent
   {
   Property MyText As %String;
   Index MyIndex On (MyText) As %UIMA.Index(AEDESCRIPTOR = "classpath:/com/intersystems/uima/annotator/iKnowEngine.xml");
   }

   Copy code to clipboard

   where:

   • MyText is the column of the Sample.MyData table that contains the unstructured text.

   • MyIndex is the UIMA functional index.

   • iKnowEngine.xml is the descriptor file for the InterSystems IRIS NLP analysis engine.

3. Save the file as sample.cls.

Compiling the Table ClassCopy link to this section

To automatically generate the Annotation Store, you simply compile the class that contains the functional index. If you created sample.cls in Atelier, simply compile the file.

If you made the changes in a text editor, use the InterSystems Terminal to load and compile the class.

To load and compile the class:

1. Open the InterSystems Terminal. For information about opening Terminal for your instance, see InterSystems IRIS Connection Information in InterSystems IRIS Basics: Connecting an IDE.

2. Switch to the namespace that you created for this demo. For example:

   set $namespace="samples"

3. Enter the following command to load the class file into the namespace:

   do $system.OBJ.Load("<sample-dir>\sample.cls")

   where <sample-dir> is the location where you saved the samples.cls class file.

4. Enter the following command to compile the Sample.MyData class that you pasted into sample.cls:

   do $system.OBJ.Compile("Sample.MyData")

Browsing the Annotation StoreCopy link to this section

Now that you have compiled the class with the UIMA functional index, you can browse the Annotation Store that was created to preserve the annotations generated by InterSystems IRIS NLP.

1. Open the Management Portal in your browser using the URL for your instance, as described in InterSystems IRIS Connection Information in InterSystems IRIS Basics: Connecting an IDE.

2. Switch to the Samples namespace using the link in the header.

   

3. Go to System Explorer > SQL.

4. Expand the Tables list in the left-hand pane.

   You can see the three tables of the Annotation Store. The naming convention of these tables corresponds to the table (Sample.MyData) that contains the unstructured text that was analyzed.

   • Sample_MyData.Type — Contains an overview of the Annotation Types used in this store.

   • Sample_MyData.Sofa — Contains sofas, which are the text objects that were analyzed by the UIMA analysis engine.

   • Sample_MyData.Annotation — Contains the annotations generated by the analysis engine.

   You can modify the functional index definition to create multiple annotation tables, and then channel the output into the right table based on the annotation type.

Sending New Text Through the Analysis PipelineCopy link to this section

The power of the UIMA analysis pipeline in InterSystems IRIS is that new unstructured text is automatically sent through the pipeline for analysis and the results added to the Annotation Store. Now that you’ve created the Annotation Store, you can see how new records added to the Sample.MyData table results in new entries being added to the Annotation Store.