Skip to main content

History Monitor

The History Monitor maintains a historical database of performance and system usage metrics. Its primary purposes are to:

  • Provide a performance baseline and help with analysis of performance issues.

  • Help analyze system usage over time for capacity planning.

This database is defined in the SYS.History class package and kept in the %SYS namespace. All of the details of the database structure are published there and the data is accessible through SQL or the normal persistent object access. The class documentation in SYS.History also contains descriptions of all the individual properties, methods and queries that are available.

The data is generally organized into performance (see SYS.History.PerformanceOpens in a new tab) and system usage (see SYS.History.SystemUsageOpens in a new tab) data. The performance metrics are intended to be sampled at short intervals (by default, 30 seconds), and the system usage data at longer intervals (by default, 5 minutes). At the beginning of each day, the individual interval samples are summarized into hourly and daily tables as averages, maximums, minimums, standard deviation, medians and totals. You can select which, if any, of the summary functions are kept for each metric class. The interval and hourly data may be purged automatically after a defined number of days (by default, seven (7) days and 60 days, respectively); the daily summary is intended for long-term analysis and can be purged manually.

Base Metrics

All of the collected metrics are defined in four %SerialObject classes in SYS.History. These same classes are used as the basis for the Interval, Hourly, and Daily databases, so all of the properties are defined as %Numeric types to allow for decimal values in the summaries.

The performance related metrics are defined in:

  • SYS.History.PerformanceOpens in a new tab — The properties in this class are general performance metrics like global references and routine calls.

    Note:

    These properties are all “counter” types and the interval data is collected as deltas, which represent the change in the counter over the last interval. When this data is summarized into hourly and daily values, the data is normalized to per-second rates

  • SYS.History.WriteDaemonOpens in a new tab — The properties in this class describe the performance of write daemon cycles. The system automatically keeps track of the last 20 write daemon cycles, and the History Monitor stores the data for the cycles that occurred in each interval. Typically, there are multiple cycles within each interval.

The system usage metrics are defined in:

  • SYS.History.SystemUsageOpens in a new tab — The properties in this class track how busy the system is but do not tend to change as quickly or dramatically as the performance data, such as the number of processes in InterSystems IRIS® data platform and license information.

  • SYS.History.DatabaseOpens in a new tab — This class tracks the database growth, file size and free space, for each local database.

Collecting Data

To begin collecting data, you must do the following:

See Using ^%SYSMONMGR to Manage Application Monitor and Start/Stop System Monitor for information about these procedures.

The detailed interval collection of data is defined in two persistent classes:

The corresponding %Monitor classes must be activated in Application Monitor in order to collect data and build the history data:

System Monitor, including Application Monitor, starts by default in the %SYS namespace when the InterSystems IRIS instance starts. You can configured other startup namespaces, however. The %Monitor classes are provided by default only in %SYS, but can be added to other configured startup namespaces using ^%SYSMONMGR.

Summaries

The %Monitor.System.HistoryPerfOpens in a new tab and %Monitor.System.HistorySysOpens in a new tab classes, as executed by Application Monitor, also create the hourly and daily summaries at the end of each day. The summaries are defined as the persistent classes SYS.History.HourlyOpens in a new tab and SYS.History.DailyOpens in a new tab; they include all four of the base classes as embedded objects.

For each metric property, the system may calculate the average, maximum (high-water mark), standard deviation, minimum, median, or total for each hour and for the whole day. The summary functions are selectable (or may be disabled) for each base class (SYS.History.PerformanceOpens in a new tab, SYS.History.WriteDaemonOpens in a new tab, SYS.History.SystemUsageOpens in a new tab, or SYS.History.DatabaseOpens in a new tab) and for each summary period class, using the SetSummary() method of each of the base classes. By default, the History Monitor calculates average, maximum and standard deviation for each class for both hourly and daily summaries.

Note:

The counter properties of the SYS.History.PerformanceOpens in a new tab class are normalized to per second rates for these calculations (except Total).

Purging Data

After creating the summaries, Application Monitor automatically purges the interval and hourly databases. The default is seven (7) days for interval data and 60 days for hourly data, but these may be changed using the SetPurge() method in SYS.History.PerfDataOpens in a new tab and SYS.History.HourlyOpens in a new tab classes. The SYS.History.DailyOpens in a new tab data is not automatically purged, but can be done manually using the SYS.History.Daily:Purge()Opens in a new tab method.

Accessing the Data

Since the database is defined as persistent classes, the data is available using standard SQL or persistent object access. Using the SQL browser in the Management Portal is a quick and easy way to see the various SQL schemas/tables that are created, including the individual property values.

There are several basic queries implemented in each of the persistent classes in SYS.History (SYS.History.PerfDataOpens in a new tab, SYS.History.SysDataOpens in a new tab, SYS.History.HourlyOpens in a new tab, and SYS.History.DailyOpens in a new tab) that can be used to access the individual tables for a date range; for more information about the queries, see the class reference documentation.

The are also several Export() methods provided for each persistent class so that the individual tables can be exported to files in CSV format, suitable for use with a spreadsheet such as Microsoft Excel. In particular, the SYS.History.PerfData:Export()Opens in a new tab method creates a file that is very similar in format to that created by the ^mgstat utility (for more information, see Monitoring Performance Using ^mgstat).

Adding User-Defined Metrics

You can add user-defined metrics to the History Monitor (SYS.History package):

  1. Create a class, or multiple classes, that inherit from SYS.History.AdaptorOpens in a new tab and add %Numeric properties to define the metrics.

    Note:

    User-written classes must be in the %SYS namespace, and should begin with “Z” or “z” to prevent naming conflicts with system classes and problems during upgrades.

  2. Code the Sample()Opens in a new tab method to instantiate the class and provide periodic values for each property. This method is called when the interval data is collected.

  3. When you compile your class, it is added as an embedded object to an interval persistent class in SYS.History. You can choose where and when it is collected using the INTERVALOpens in a new tab parameter provided in SYS.History.AdaptorOpens in a new tab class. This selects which interval class it is added to and which %Monitor class does the collection, as shown in the following table:

    INTERVAL Selected Interval Class Used %Monitor Class Used
    “User” (default) SYS.History.User %Monitor.System.HistoryUserOpens in a new tab
    “UserPerf” SYS.History.UserPerf %Monitor.System.HistoryPerfOpens in a new tab
    “UserSys” SYS.History.UserSys %Monitor.System.HistorySysOpens in a new tab

    Selecting “UserPerf” or “UserSys” lets you collect data at the same interval and with the same timestamp as SYS.History.PerfDataOpens in a new tab or SYS.History.SysDataOpens in a new tab, which makes it easier to correlate your data with the system data. “User” gives you a choice of a third (unrelated) time interval.

    Note:

    There are several parameters in the SYS.History.AdaptorOpens in a new tab class that provide options for how properties are collected and summarized; for more information, see the SYS.History.AdaptorOpens in a new tab class reference documentation.

  4. User-defined classes are also added as embedded objects to the SYS.History.UserHourly and SYS.History.UserDaily summary classes. The user-defined metrics are summarized and automatically purged just like the system metrics.

    Important:

    User-defined metric classes become embedded objects in persistent data. You should not change definitions after data collection has started: deleting objects can result in orphaned data; re-defining existing classes or properties can cause already stored data to be misinterpreted.

    However, because of the schema evolution feature, you can safely add new objects and properties. See Schema Evolution.

FeedbackOpens in a new tab