Skip to main content

Monitoring InterSystems IRIS Using SNMP

This topic describes the interface between InterSystems IRIS® data platform and SNMP (Simple Network Management Protocol). SNMP is a communication protocol that has gained widespread acceptance as a method of managing TCP/IP networks, including individual network devices, and computer devices in general. Its popularity has expanded its use as the underlying structure and protocol for many enterprise management tools. This is its main importance to InterSystems IRIS: a standard way to provide management and monitoring information to a wide variety of management tools.

SNMP is both a standard message format and a standard set of definitions for managed objects. It also provides a standard structure for adding custom-managed objects, a feature that InterSystems IRIS uses to define its management information for use by other applications.

Using SNMP with InterSystems IRIS

SNMP defines a client-server relationship where the client (a network management application) connects to a server program (called the SNMP agent) which executes on a remote network device or a computer system. The client requests and receives information from that agent. There are four basic types of SNMP messages:

  • GET – fetch the data for a specific managed object

  • GETNEXT – get data for the next managed object in a hierarchical tree, allowing system managers to walk through all the data for a device

  • SET – set the value for a specific managed object

  • TRAP – an asynchronous alert sent by the managed device or system

The SNMP MIB (Management Information Base) contains definitions of the managed objects. Each device publishes a file, also referred to as its MIB, which defines which portion of the standard MIB it supports, along with any custom definitions of managed objects. For InterSystems IRIS, this is the ISC-IRIS.mib file, located in the install-dir\SNMP directory.

InterSystems IRIS as a Subagent

The SNMP client connects to the SNMP agent which is listening on a well-known address, port 161. Since the client expects to connect on this particular port, there can only be one SNMP agent on a computer system. To allow access to multiple applications on the system, developers can implement master agents, which may be extended or connected to multiple subagents. InterSystems has implemented the InterSystems IRIS SNMP interface as a subagent, designed to communicate through an SNMP master agent.

Most operating systems that InterSystems IRIS supports provide an SNMP master agent which is extensible in some way to support multiple subagents. Many of these agents, however, implement their extensibility in a proprietary and incompatible manner. InterSystems IRIS implements its subagent using the Agent Extensibility (AgentX) protocol, an IETF-proposed standard as described in RFC 2741Opens in a new tab.

Some of the standard SNMP master agents support AgentX. If the SNMP master agent supplied by an operating system is not AgentX-compatible, you can replace it with the public domain Net-SNMP agent.

Note:

The exception is the Windows standard agent which does not support AgentX and for which the Net-SNMP version may not be adequate. For this exception, InterSystems supplies a Windows extension agent DLL, iscsnmp.dll, which handles the connection between the standard Windows SNMP service extension API and the InterSystems IRIS AgentX server.

Managing SNMP in InterSystems IRIS

Since SNMP is a standard protocol, the management of the InterSystems IRIS subagent is minimal. The most important task is to verify that the SNMP master agent on the system is compatible with the Agent Extensibility (AgentX) protocol (see InterSystems IRIS as a Subagent) and it is active and listening for connections on the standard AgentX TCP port 705. On Windows systems, the system automatically installs a DLL to connect with the standard Windows SNMP service. Verify that the Windows SNMP service is installed and started either automatically or manually.

Important:

Some SNMP master agents, notably Net-SNMP on Linux, do not enable AgentX by default and do not use TCP port 705 by default once they are enabled. For Net-SNMP you must modify the snmpd.conf file to enable communications with the InterSystems IRIS subagent. Recent versions of Net-SNMP also implement VACM (View-based Access Control Model) security and, by default, only allow access to the mib-2.system subtree; the InterSystems IRIS subagent starts and runs without error, but no SNMP requests are forwarded to InterSystems IRIS. You must expand the “views” defined in snmpd.conf to include the InterSystems IRIS MIB subtree.

Next, enable the monitoring service using the following steps:

  1. Navigate to the Services page in the Management Portal (System Administration > Security > Services).

  2. Click the %Service_Monitor service.

  3. Select the Service enabled check box and click Save.

  4. Return to the list of services page and ensure that the %Service_Monitor service is enabled.

Finally, configure the InterSystems IRIS SNMP subagent to start automatically at InterSystems IRIS startup using the following steps:

  1. Navigate to the Monitor Settings page in the Management Portal (System Administration > Configuration > Additional Settings > Monitor).

  2. Select Yes for the Start SNMP Agent at System Startup setting and click Save.

  3. When you edit this setting, the InterSystems IRIS end of the SNMP interface immediately stops and starts.

You can also start and stop the InterSystems IRIS SNMP subagent manually or programmatically using the ^SNMP routine:

Do start^SNMP(<port>,<timeout>)
Do stop^SNMP

where <port> is the TCP port for the connection (default is 705) and <timeout> is the TCP port read timeout value (default is 20 seconds). Until the <timeout> value is reached, InterSystems IRIS logs any problems encountered while establishing a connection or answering requests in the SNMP.LOG file in the install-dir\mgr directory.

Note:

When the SNMP master agent is restarted, it may be necessary to manually restart the InterSystems IRIS SNMP subagent using the ^SNMP routine, as described in the foregoing.

SNMP Troubleshooting

The InterSystems IRIS subagent (running the ^SNMP routine) depends on the correct installation and configuration of the SNMP master agent supplied by the operating system. As noted in InterSystems IRIS as a Subagent, there are two main ways in which the ^SNMP routine communicates with this master agent:

  • Primarily, ^SNMP uses the AgentX protocol on TCP port 705.

  • On Windows, ^SNMP uses a Windows extension agent DLL installed as iscsnmp.dll.

Detailed instructions for configuring the SNMP agent should be supplied with the operating system, and system managers should take some time to understand how to do this. The following are some basic guidelines and tips for troubleshooting if problems are encountered in getting InterSystems IRIS to communicate with the SNMP agent.

All Systems

  • Make sure the SNMP agent is working independently of InterSystems IRIS and you can at least query the mib-2.system tree for general system information. If this fails, on Windows check the Windows SNMP Service; on UNIX®/Linux see if the SNMP daemon (snmpd) is running.

  • If you can successfully query the SNMP system information but not the InterSystems IRIS MIB, then check for a background process in InterSystems IRIS running the ^SNMP routine. Try starting it using the $$start^SNMP() function. If the routine starts but does not continue running, check for errors in the messages.log and SNMP.log log files in the InterSystems IRIS install-dir/mgr directory. On Windows, iscsnmp.dll logs any errors it encounters in Windows\System32\snmpdbg.log (on a 64–bit Windows system, the file is in the SysWOW64 subdirectory).

  • Make sure the InterSystems IRIS %Service_Monitor service is enabled.

  • More information can be logged to the SNMP.log file if you set ^SYS(“MONITOR”,”SNMP”,”DEBUG”)=1 in the %SYS namespace and restart the ^SNMP InterSystems IRIS subagent process. This logs details about each message received and sent.

Windows Systems

  • Not all Windows versions install the Windows SNMP service by default. You may need to do this as an additional step. Make sure the Security tab of the Properties dialog for the service has at least a public community with READ rights. To send SNMP traps, you must define a Community Name and Destination on the Trap tab of the properties dialog.

  • InterSystems IRIS expects the SNMP service to be installed before you install InterSystems IRIS, so it can add iscsnmp.dll to the proper Registry keys. Once InterSystems IRIS is installed, the SNMP service must be restarted so that it properly loads iscsnmp.dll and can find and communicate with the new InterSystems IRIS instance.

    Note:

    If InterSystems IRIS is installed before the SNMP service, iscsnmp.dll cannot be properly registered, and you must use the set myStatus=$$Register^SNMP() function to do this after the Windows SNMP service is installed. Once this is done the SNMP service must be restarted.

  • On Windows, the $$start^SNMP() function only signals the SNMP service, and the InterSystems IRIS ^SNMP process is actually started by a callback from the SNMP service into InterSystems IRIS. It may take a few seconds for the process to start, and a few more seconds before it can respond to queries.

UNIX® Systems

Many UNIX operating systems (IBM AIX®) do not support the AgentX protocol at this time. If your system does not support AgentX, you must install a separate SNMP agent which supports AgentX, such as Net-SNMP.

Linux and macOS with Net-SNMP

  • AgentX support is not enabled by default, and the default port is not 705. You must modify the snmpd.conf file and add master agentx and agentXSocket TCP:localhost:705, or use snmpd -x TCP:localhost:705 on the command line.

  • Basic system information like syslocation, syscontact and sysservices must be defined in snmpd.conf to enable successful startup of the snmpd daemon.

  • Recent versions of Net-SNMP also implement VACM (View-based Access Control Model) security and, by default, allow access to the mib-2.system subtree only; as a result, the InterSystems IRIS subagent starts and runs without error, but no SNMP requests are forwarded to InterSystems IRIS. You must expand the “views” defined in snmpd.conf to include the InterSystems IRIS MIB subtree.

  • To send SNMP traps, you must define a destination using the trapsink parameter in snmpd.conf, for example trapsink 192.16.61.36 public.

InterSystems IRIS MIB Structure

All of the managed object data available through the InterSystems IRIS SNMP interface is defined in the InterSystems IRIS MIB file, ISC-IRIS.mib, which is located in the install-dir\SNMP directory. Typically an SNMP management application must load the MIB file for the managed application to understand and appropriately display the information. Since this procedure varies among applications, consult your management application documentation for the appropriate way to load the InterSystems IRIS MIB.

The specific data defined in the InterSystems IRIS MIB is documented in the file itself and, therefore, is not repeated here. However, it may be valuable to understand the overall structure of the InterSystems IRIS MIB tree, especially as it relates to multiple instances on the same system.

Note:

The best way to view the MIB tree is to load the MIB into a management application or MIB browser. These tools display the MIB as a tree with object IDs (OIDs), matching text representations of the objects, and descriptions of the objects.

SNMP defines the Structure of Management Information (SMI), a specific, hierarchical tree structure for all managed objects, which is detailed in RFC 1155Opens in a new tab. Each managed object is named by a unique object identifier (OID), which is written as a sequence of integers separated by periods, for example: 1.3.6.1.2.1.1.1. The MIB translates this dotted integer identifier into a text name.

The standard SNMP MIB defines many standard managed objects. To define application-specific extensions to the standard MIB, as InterSystems IRIS does, an application uses the enterprise branch which is defined as:

iso.org.dod.internet.private.enterprises (1.3.6.1.4.1)

The Internet Assigned Numbers Authority (IANA) assigns each organization a private enterprise number as the next level in the hierarchy. For InterSystems IRIS this is 16563, which represents intersystems.

Below this, InterSystems IRIS implements its enterprise private subtree as follows:

  • The level below intersystems is the “product” or application ID level. For InterSystems IRIS this is .4 (iscIris). This serves as the MIB module identity.

  • The next level is the “object” level, which separates data objects from notifications. For InterSystems IRIS, these are .1 (irisObjects) and .2 (irisTraps). By convention, the intersystems tree uses a brief lowercase prefix added to all data objects and notification names. For InterSystems IRIS this is iris.

  • The next level is the “table” or group level. All data objects are organized into tables, even if there is only one instance or “row” to the table. This serves to organize the management data objects into groups. This is also necessary to support multiple InterSystems IRIS instances on one machine. All tables use the InterSystems IRIS instance name as the first index of the table. The tables may also have one or more additional indexes.

  • The next level, which is always .1, is the “conceptual row” for the table (as required by the SNMP SMI).

  • Finally, the individual data objects contained in that table, including any that are designated as indexes.

  • The notifications (traps) are defined as individual entries at the same hierarchical level as the “table”. For more information, see InterSystems IRIS SNMP Traps.

  • InterSystems IRIS-specific auxiliary objects sent via notifications (traps) are defined as individual entries at the same hierarchical level as the “table”. For more information, see InterSystems IRIS SNMP Traps.

For example, encode the size of a database as:

1.3.6.1.4.1.16563.4.1.3.1.6.4.84.69.83.84.1

This translates to:

iso.org.dod.internet.private.enterprises.intersystems.isciris.irisObjects
irisDBTab.irisDBRow.irisDBSize.TEST(instname).1(DBindex)

Extending the InterSystems IRIS MIB

Application programmers can add managed object definitions and extend the MIB for which the InterSystems IRIS subagent provides data. This is not intended to be a complete MIB editor or SNMP toolkit; rather, it is a way to add simple application metrics that you can browse or query through SNMP.

Note:

The objects must follow the basic InterSystems IRIS SNMP structure, there is limited support for SNMP table structures (only integer-valued indexes are supported), and SNMP traps are not created (see the %Monitor.AlertOpens in a new tab class). A basic understanding of SNMP structure of management information is helpful.

To create these objects do the following:

  1. Create InterSystems IRIS object definitions in classes that inherit from the %Monitor.AdaptorOpens in a new tab class. See the InterSystems Class Reference for details about adding managed objects to the %Monitor package.

  2. Execute an SNMP class method to enable these managed objects in SNMP and create a MIB definition file for management applications to use. The method to accomplish this is MonitorTools.SNMP.CreateMIB().

See the MonitorTools.SNMPOpens in a new tab class documentation for details of the CreateMIB() method parameters.

The method creates a branch of the private enterprise MIB tree for a specific application defined in the %Monitor database. In addition to creating the actual MIB file for the application, the method also creates an internal outline of the MIB tree. The InterSystems IRIS subagent uses this to register the MIB subtree, walk the tree for GETNEXT requests, and reference specific objects methods for gathering the instance data in GET requests.

All the managed object definitions use the same general organization as the InterSystems IRIS enterprise MIB tree, that is: application.objects.table.row.item.indices. The first index for all tables is the InterSystems IRIS application ID. All applications must register with the IANA to obtain their own private enterprise number, which is one of the parameters in the CreateMIB() method.

To disable the application in SNMP, use the MonitorTools.SNMP.DeleteMIB() method. This deletes the internal outline of the application MIB, so the InterSystems IRIS subagent no longer registers or answers requests for that private enterprise MIB subtree.

For an example of defining a user Monitor class, see Sample User-Defined SNMP Monitor Class.

InterSystems IRIS SNMP Traps

In addition to the object data and metrics available through SNMP queries, InterSystems IRIS can send asynchronous alerts or SNMP traps. The following table describes the InterSystems IRIS-specific SNMP traps.

InterSystems IRIS SNMP Notification Objects (Traps)
Trap Name (Number) Description
irisStart (1) The InterSystems IRIS instance has been started.
irisStop (2) The InterSystems IRIS instance is in the process of shutting down.
irisDBExpand (3) An InterSystems IRIS database has expanded successfully.
irisDBOutOfSpace (4) Future expansion of an InterSystems IRIS database may be limited; there is not enough free space on the file system for 10 more expansions or there is less than 50 MB of free space.
irisDBStatusChange (5) The read/write status of an InterSystems IRIS database has been changed.
irisDBWriteFail (6) A write to an InterSystems IRIS database has failed. It includes the InterSystems IRIS error code for the failed write.
irisWDStop (7) The write daemon for an InterSystems IRIS instance has stalled.
irisWDPanic (8) The write daemon for an InterSystems IRIS instance has entered “panic” mode; that is, the write daemon is out of buffers and must write database blocks directly to disk without first committing them to the Write Image Journal (WIJ) file.
irisLockTableFull (9) The lock table for an InterSystems IRIS instance is full, which causes subsequent Locks to fail.
irisProcessFail (10) A process has exited InterSystems IRIS abnormally (due to an access violation). For detailed information, see the messages.log file.
irisECPTroubleDSrv (11) A connection to this ECP Data Server for an InterSystems IRIS database has encountered a serious communication problem. For detailed information, see the messages.log file.
irisECPTroubleASrv (12) A connection from this ECP Application Server to a remote InterSystems IRIS database has encountered a serious communication problem. For detailed information, see the messages.log file.
irisAuditLost (13) InterSystems IRIS has failed to record an Audit event. The most likely cause is a problem with space for the Audit database, which requires operator assistance.
irisLoggedError (14) A “severe ” error has been logged in the messages.log file. This trap includes the error message defined in irisSysErrorMsg.
irisLicenseExceed (15) A request for a license has exceeded the number of licenses currently available or allowed.
irisEventLogPost (16) An entry posted in the Interoperability Event Log.
irisAppAlert (100) This is a generic trap that can be used by InterSystems IRIS applications to generate alerts via SNMP. For detailed information about how this trap can be used, see the %Monitor.Alert.External class method.

The following table describes the InterSystems IRIS-specific auxiliary objects that can be sent in the traps described in the preceding table.

InterSystems IRIS-specific Auxiliary Objects Sent in Traps
Auxiliary Object Name (Number) Description
irisDBWriteError (1) The InterSystems IRIS-specific error code for a failed database write. Possible values are: <DATABASE>, <DISKHARD>, <BLOCKNUMBER>, <FILEFULL> or <DATABASE MAP LABEL>.
irisApp (2) A short text string (maximum of 20 characters) that identifies the application that generated (or was the source of) an irisAppAlert trap.
irisAppSeverity (3) A code that indicates the severity of the problem for a irisAppAlert trap. The code can be 0 (info), 1 (warning), 2 (severe), or 3 (fatal).
irisApptext (4) A text string description (maximum of 1024 characters) of the problem, error, or event that caused the irisAppAlert trap.

Sample User-Defined SNMP Monitor Class

This section describes an example of how to define a user Application Monitor class (see Application Monitor) that you can query via SNMP. The Application Monitor includes only properties with %Monitor data types in the SNMP data.

Example Sample Class

The following is the sample class for this example:

Class SNMP.Example Extends %Monitor.Adaptor
{

  /// Give the application a name. This allows you to group different 
  /// classes together under the same application level in the SNMP MIB.
  /// The default is the same as the Package name.
  Parameter APPLICATION = "MyApp";

  /// This groups a set of properties together at the "table" level of the
  /// SNMP MIB hierarchy. The default is the Class name.
  Parameter GROUPNAME = "MyTable";

  /// An integer metric counter
  Property Counter1 As %Monitor.Integer;

  /// Another integer metric counter
  Property Counter2 As %Monitor.Integer;

  /// A status indicator as a string data type
  Property Status As %Monitor.String;

  /// The method is REQUIRED. It is where the Application Monitor
  /// calls to collect data samples, which then get picked up by the
  /// ^SNMP server process when requested.
  Method GetSample() As %Status
  {
    set ..Counter1=$r(200)
    set ..Counter2=200+$r(100)
    set n=$r(4)
    set ..Status=$s(n=1:"Crashed",n=2:"Warning",n=3:"Error",1:"Normal")
    Quit $$$OK
  }

}

Before compiling this class in a user namespace, InterSystems IRIS must load supporting classes into the namespace; these classes are required to store the data samples for SNMP.

To load the classes, run ^%SYSMONMGR, as described in Using ^%SYSMONMGR to Manage Application Monitor, and do the following:

  • Select option 2, Manage Monitor Classes.

  • Select option 3, Register Monitor System Classes.

When you compile the sample class, it creates the SNMP.Sample.Example class to store the sample data.

Important:

Do not delete generated sample classes explicitly; if you select both the Application Monitor and generated sample classes for deletion, the sample class routines remain although the monitor class routines are deleted, which causes an error. To ensure that all sample class routines are properly removed, delete only the Application Monitor class that generated it; when you delete the monitor class both the monitor class and generated sample class, as well as related routines for both classes, are deleted. For example, to delete a sample class (for example, SNMP.Sample.Example), use the Management Portal to delete the monitor class from which it is generated (that is, SNMP.Example).

Run ^%SYSMONMGR to activate the sample class and start the Application Monitor to collect samples:

  1. Select option 2, Manage Monitor Classes.

  2. Select option 1, Activate/Deactivate a Monitor Class.

  3. To see an numbered list of registered Monitor Classes, enter ?.

  4. Enter the number of Monitor Class you want to activate; for example, to activate a user-defined class named SNMP.Example, enter the number next to the class name.

  5. Select option 6, Exit (to return to the Application Monitor main menu).

  6. Select option 1, Manage Application Monitor.

  7. Select option 1, Start Application Monitor.

  8. Select option 5, Exit (to return to the Application Monitor main menu).

  9. Select option 6, Exit (to exit from Application Monitor main menu).

Note:

For information about configuring and using Application Monitor, see Application Monitor.

Example of Creating a User MIB

To create the SNMP MIB, run the MonitorTools.SNMP:CreateMIB method from the %SYS namespace. See the MonitorTools.SNMPOpens in a new tab class documentation for details.

The input parameters for the method are similar to the following:

CreateMIB("MyApp","USER",99990,1,"mycorp","myapp","mc","MC-MYAPP","Unknown",1)

Important:

Do not use 99990 as the Enterprise ID for production; each organization should register with the IANA for their own ID.

USER>set $namespace = "%SYS"
 
%SYS>Do ##class(MonitorTools.SNMP).CreateMIB("MyApp","USER",99990,1,"mycorp",
"myapp","mc","MC-MYAPP","Unknown",1)
 
 
Create SNMP structure for Application - MyApp
 
     Group - MyTable
          Counter1 = Integer
          Counter2 = Integer
          Status = String
 
 
Create MIB file for MyApp
 
     Generate table MyTable
          Add object Counter1
          Add object Counter2
          Add object Status
 
 
%SYS>

This creates the MC-MYAPP.MIB file in your default directory (install-dir\mgr\User), which you can load into your SNMP management application.

Note:

You may need to restart the SNMP master agent and the InterSystems IRIS ^SNMP service on your system before each recognizes this MIB.

--
-- MIB file generated for mcMyApp product.
--
-- Sep 16, 2008
--

MC-MYAPP DEFINITIONS ::= BEGIN

IMPORTS

  MODULE-IDENTITY, OBJECT-TYPE, NOTIFICATION-TYPE,
  Counter32, Gauge32, Integer32
  FROM SNMPv2-SMI
  DisplayString
  FROM SNMPv2-TC
  enterprises
  FROM RFC1155-SMI
  cacheSysIndex
  FROM ISC-IRIS;


mcMyApp MODULE-IDENTITY
    LAST-UPDATED "200809161700Z"
    ORGANIZATION "mycorp"
    CONTACT-INFO "
                    Unknown"
    DESCRIPTION ""
    ::= { mycorp 1 }

mycorp OBJECT IDENTIFIER ::= { enterprises 16563 }

myappObjects OBJECT IDENTIFIER ::= { mcMyApp 1 }

--
-- Begin tables
--

-- Table myappMyTable

myappMyTable         OBJECT-TYPE
       SYNTAX        SEQUENCE OF myappMyTableR
       MAX-ACCESS    not-accessible
       STATUS        current
       DESCRIPTION
       ""
   ::= { myappObjects 1 }

myappMyTableR        OBJECT-TYPE
       SYNTAX        myappMyTableR
       MAX-ACCESS    not-accessible
       STATUS        current
       DESCRIPTION
       "Conceptual row for MyTable table."
       INDEX  { cacheSysIndex }
   ::= { myappMyTable 1 }

myappMyTableR ::=
       SEQUENCE {
        myappCounter1   Integer32
        myappCounter2   Integer32
        myappStatus   DisplayString
       }

myappCounter1        OBJECT-TYPE
       SYNTAX        Integer32
       MAX-ACCESS    read-only
       STATUS        current
       DESCRIPTION
       ""
       ::= { myappMyTableR 1 }

myappCounter2        OBJECT-TYPE
       SYNTAX        Integer32
       MAX-ACCESS    read-only
       STATUS        current
       DESCRIPTION
       ""
       ::= { myappMyTableR 2 }

myappStatus          OBJECT-TYPE
       SYNTAX        DisplayString
       MAX-ACCESS    read-only
       STATUS        current
       DESCRIPTION
       "Status"
       ::= { myappMyTableR 3 }

--  End of MyTable table

myappTraps  OBJECT IDENTIFIER ::= { mcMyApp 2 }

-------------------------------------------------------------
 END

FeedbackOpens in a new tab