Skip to main content

Custom Metrics

Important:

InterSystems has deprecatedOpens in a new tab InterSystems IRIS® Natural Language Processing (NLP). It may be removed from future versions of InterSystems products. The following documentation is provided as reference for existing users only. Existing users who would like assistance identifying an alternative solution should contact the WRCOpens in a new tab.

You can customize NLP to allow users to add their own custom metrics for entities, CRCs, CCs, paths, sentences and/or sources. These metrics can be populated for all sources in the domain, or for select sources. You can then query by these custom metric values.

Note:

Custom Metrics software is a Technology Preview. A Technology Preview is intended as a way of introducing and providing access to new software capabilities that InterSystems believes will be useful in enhancing the effectiveness of existing and future applications.

The capabilities listed here are ready for use by customers, but they are not yet complete in functionality and design. Customers who take advantage of these capabilities must understand:

  • InterSystems makes no backward compatibility guarantee with future updates;

  • Customers may incorporate these capabilities in deployed applications, but must first check with InterSystems to determine best course of action;

  • Customers who deploy these capabilities in their applications must commit to upgrading to the final released version.

InterSystems strongly encourages those who incorporate these items in their software to provide feedback on their experiences.

Implementing Custom Metrics

Implementation of custom metrics involves the following steps:

  1. Define a metric: To define one or more metrics, implement a subclass of %iKnow.Metrics.MetricDefinitionOpens in a new tab and add an XData block named “Metrics” that specifies the properties for one or more custom metrics.

  2. Define the metric computation: To compute the values of these metrics, implement the calculations in a %iKnow.Metrics.MetricBuilderOpens in a new tab subclass. This subclass should implement one or more of the Calculate***Metrics() methods to support calculating custom metrics for the corresponding target and type. For example, CalculateEntUniMetrics()Opens in a new tab calculates a metric targeting entities, applicable to the whole domain.

  3. Register the metric definition: To associate a metric definition with a particular domain, call the Register()Opens in a new tab method of the %iKnow.Metrics.MetricDefinition class to register your %iKnow.Metrics.MetricBuilderOpens in a new tab subclass as the "builder class" in the definition. This registers all metrics, their properties and targets, as defined in the “Metrics” XData block. Registering the metric definition configures any data structures required to store the metric values.

    If the domain was created as a subclass of %iKnow.DomainDefinitionOpens in a new tab and metrics were defined for the domain, you do not have to call the Register() method to register your metric definition.

  4. Build the metric values: To build the metric values using a %iKnow.Metrics.MetricBuilderOpens in a new tab implementation, call the Build()Opens in a new tab or BuildAll()Opens in a new tab method of the builder class. These methods build all applicable metrics, forwarding the actual metric calculations to the Calculate***Metrics() implementations. You can specify a filter to limit the custom metrics build to a defined subset of sources.

    Unless your %iKnow.Metrics.MetricBuilderOpens in a new tab subclass overrides the SUPPORTSMULTIPROCESS parameter (default = 1), the Build() method will divide the work over multiple processes, each instantiating separate MetricBuilder objects and processing batch after batch of entries using those builder objects. For source-indexed target types, these are batches of sources, for which all the target elements are processed by the same builder process.

  5. Optimize the build process: %iKnow.Metrics.MetricBuilderOpens in a new tab provides a number of callback utility methods, such as OnBeforeBuildMetrics()Opens in a new tab, OnBeforeBuildMetricsBatch()Opens in a new tab, OnBeforeBuildMetricsForSource()Opens in a new tab and the corresponding OnAfter***() methods that can help in optimizing performance by pre-calculating values for a whole batch, which are then selected in the individual Calculate***Metrics() methods. The individual Calculate***Metrics() methods return their values through an output parameter pValues, which is to be populated indexed by metric name (as a single MetricBuilder class can support and calculate multiple metrics in one pass): pValues("MyMetric") = [MyValue].

  6. Get the results: %iKnow.Metrics.MetricAPIOpens in a new tab provides a number of methods that you can use to interact with custom metrics registered in a particular domain. You can use these methods to return the custom metrics results.

    When Calculate***Metrics() methods don't return any values for a particular target element, the existing metric value is retained (if any). Returning "" for a value will overwrite (erase) the existing value.

For examples, refer to the Aviation.Metrics.Builder and the Aviation.Metrics.Definition classes.

Refer to A Note on Program Examples for details on the coding and data used in the examples in this book.

Types and Targets

  • A metric has a type which specifies whether the metric values apply within the context of the whole domain, or just one particular source or metadata group. The available values are $$$IKMTRTYPEDOMAIN, $$$IKMTRTYPESOURCE, and $$$IKMTRTYPEGROUP.

    $$$IKMTRTYPEGROUP metrics are valid in the context of a group of sources, as defined by a particular metadata field. Metric Builder classes for this type of metric can use the CurrentFieldName and CurrentFieldValue property values as part of their Calculate***Metrics() implementation.

  • A metric has one or more targets, which are the elements the custom metric applies to. The available values are $$$IKMTRENTITY, $$$IKMTRCRC, $$$IKMTRCC, $$$IKMTRPATH, $$$IKMTRSENTENCE, and $$$IKMTRSOURCE. The target is specified by which Calculate***Metrics() method you specify. For example, CalculateEntUniMetrics() applies to target $$$IKMTRENTITY; how it is applied depends upon the type.

Copying Metrics

You can copy metrics from one domain to another within the current namespace.

FeedbackOpens in a new tab