Monitoring InterSystems IRIS via REST

The metrics are returned in a text-based format, described in the Prometheus documentation (https://prometheus.io/docs/instrumenting/exposition_formats/Opens in a new tab). Each metric is listed on a single line with only one space, which separates the name from the value. Each unique metric is preceded by # HELP and # TYPE comment lines (described in https://prometheus.io/docs/instrumenting/exposition_formats/#comments-help-text-and-type-informationOpens in a new tab). Where applicable, InterSystems IRIS also includes a # UNIT comment line; this comment specifies the unit of measurement used for the metric. You can configure custom application metrics to supply these comment lines as well.

InterSystems IRIS metrics with their Sensor Group are listed in the table below. For more information about using Sensor Groups, see the ExcludeMetricsOpens in a new tab property.

In addition to the metrics described in the previous section, an InterSystems IRIS instance can also record metrics about active interoperability productions and include them in the output of the /metrics endpoint. The recording of these interoperability metrics is disabled by default. To enable it, you must perform the following steps for each interoperability production you want to monitor:

1. Open a Terminal session for the InterSystems IRIS instance running the production you want to monitor. If necessary, switch to the namespace associated with the production by executing the following command:

   set $namespace = "[interopNS]"
   

   

   where [interopNS] is the namespace name.

2. In the Terminal, execute the following command to enable the collection of metrics for the active production within the current namespace:

   do ##class(Ens.Util.Statistics).EnableSAMForNamespace()
   

   
   Note:

   If the recording of metrics is enabled for a namespace but the corresponding production is not active, the /metrics endpoint does not return any metrics.

   The Ens.Util.Statistics class provides methods for customizing the output of the /metrics endpoint. For example, invoking the method DisableSAMIncludeHostLabel will provide aggregated metrics for the entire production instead of providing them for each host individually.

   The metrics available after completing this step are described in the Basic Interoperability Metrics and Message Retention Metrics tables below.

3. To collect additional metrics for a production, enable activity monitoring by invoking the class method Ens.Util.Statistics.EnableStatsForProduction in the corresponding namespace using the Terminal. You must also add the Ens.Activity.Operation.Local business operation to the production. This process is detailed in Enabling Activity Monitoring.

   The additional metrics available after completing this step are described in the Activity Volume Metrics table below.

4. To collect additional HTTP transmission metrics for EnsLib.HTTP.OutboundAdapter or the EnsLib.SOAP.OutboundAdapter (used in productions), enable the reporting of HTTP metrics for the corresponding business operation by performing the following steps:

   1. Open the Management Portal for the InterSystems IRIS instance containing the web client you want to monitor.

   2. Select Interoperability and choose the namespace containing the web client.

   3. Select Configure > Production to open the Production Configuration page.

   4. Select the operation which uses the HTTP or SOAP outbound adapter.

   5. In the Alerting Control section of the Production Settings > Settings panel, select the Provide Metrics for SAM check box.

   6. Select Apply to save your settings.

   The additional metrics available after completing this step are described in the HTTP Metrics table below.

   Note:

   Currently, HTTP transmission metrics are only collected for business operations which invoke actors using the Queue style (not inProc). For more information on the difference between these invocation styles, see Defining a Business Operation Class.

InterSystems IRIS interoperability metrics are listed in the tables below. Metric names with a label appear here with line breaks to improve readability.

The Message Retention Metrics, described in the following table, can help you to determine if your production data is being purged appropriately. These metrics are tabulated by the Collect() method of the Ens.SAM.GeneralMetricsOpens in a new tab class, which can be called directly or scheduled for execution by the “Collect Interoperability Message Retention Metrics” task, Ens.Util.Tasks.MonitorMetricsOpens in a new tab.

You can use the DayBoundariesForCounts property to specify different day boundaries for message header count calculation cutoffs. The value of this property is a comma-separated list of the number of days to show message header counts, excluding oldest and 0. The default value is "56, 28, 14, 7, 1".

To call the API directly:

do ##class(Ens.SAM.GeneralMetrics).Collect(<boundaries>)

The boundaries parameter is optional, and provides the value for DayBoundariesForCounts.

InterSystems IRIS provides Activity Volume Metrics and HTTP Metrics (described in the two tables which follow) only if you perform additional activation steps as described at the beginning of this section.

To add custom application metrics to those returned by the /metrics endpoint:

1. Create a new class that inherits from %SYS.Monitor.SAM.AbstractOpens in a new tab.

2. Define the PRODUCT parameter as the name of your application. This can be anything except for iris, which is reserved for the InterSystems IRIS metrics.

3. Implement the GetSensors()Opens in a new tab method to define the desired custom metrics, as follows:

   • The method must contain one or more calls to the SetSensor()Opens in a new tab method. This method sets the name and value for an application metric. The values should be integers or floating point numbers to ensure compatibility with Prometheus.

     You can optionally define a label for the metric, though if you do, you must always define a label for that particular metric.

     Note:

     For best practices when choosing metric and label names, see Metric and Label Naming in the Prometheus documentation (https://prometheus.io/docs/practices/naming/Opens in a new tab).

   • The method can optionally include labels for a metric by invoking the SetSensorLabels() method. However: if you define a label for a metric, you must include that label every time you set that metric.

   • The method can optionally define HELP, TYPE, and UNIT information for a metric by invoking the SetSensorInfo() method. The output of the /metrics endpoint will include this information as comments. Refer to the Prometheus documentation for more information: https://prometheus.io/docs/instrumenting/exposition_formats/#comments-help-text-and-type-informationOpens in a new tab

   • The method must return $$$OK if successful.

   Important:

   A slow implementation of GetSensors() can negatively impact system performance. Be sure to test that your implementation of GetSensors() is efficient, and avoid implementations that could time out or hang.

4. Compile the class. An example is shown below:

   /// Example of a custom class for the /metric API
   Class MyMetrics.Example Extends %SYS.Monitor.SAM.Abstract
   {
   
   Parameter PRODUCT = "myapp";
   
   /// Collect metrics from the specified sensors
   Method GetSensors() As %Status
   {
      do ..SetSensor("my_counter",$increment(^MyCounter),"my_label")
      do ..SetSensor("my_gauge",$random(100))
      return $$$OK
   }
   
   }

   

5. Use the AddApplicationClass()Opens in a new tab method of the SYS.Monitor.SAM.ConfigOpens in a new tab class to add the custom class to the /metrics configuration. Pass as arguments the name of the class and the namespace where it is located.

   For example, enter the following in the Terminal from the %SYS namespace:

   %SYS>set status = ##class(SYS.Monitor.SAM.Config).AddApplicationClass("MyMetrics.Example", "USER")
   
   %SYS>w status
   status=1
   

   
   Note:

   When you upgrade your InterSystems IRIS system, you will need to redo this step.

6. Ensure that /api/monitor web application has the necessary Application Roles to access the custom metrics. For details on how to edit application roles, see Edit an Application: The Application Roles Tab.

   This step grants /api/monitor access to the data needed for the custom metric. For example, if the custom metric class is located in the USER database (protected by the %DB_USER resource), grant /api/monitor the %DB_USER role.

7. Review the output of the /metrics endpoint by pointing your browser to a URL with the following form, using the <baseURL> for your system: http://<baseURL>/api/monitor/metrics. The metrics you defined should appear after the InterSystems IRIS metrics, such as:

   [...]
   myapp_my_counter{id="my_label") 1
   myapp_my_gauge 92
   

The /metrics endpoint now returns the custom metrics you defined. The InterSystems IRIS metrics include an iris_ prefix, while your custom metrics use the value of PRODUCT as a prefix.