Caché Monitoring Guide
Monitoring Caché Using the Management Portal
[Back] [Next]
   
Server:docs2
Instance:LATEST
User:UnknownUser
 
-
Go to:
Search:    

You can monitor many aspects of your Caché instance starting at the System Dashboard of the Management Portal. From the dashboard you can view performance indicators and then, for selected indicators, navigate to more detailed information. This chapter describes the following monitoring tasks:

See Caché System Monitoring Tools in the “Using Caché Monitor” chapter of this guide for an overview of general Caché instance monitoring tools.
Monitoring System Dashboard Indicators
The [Home] > [System Dashboard] page of the Management Portal displays real-time status of key system performance indicators in the following categories. Each category is described in one of the tables that follow.
In most cases, you can click any of these indicators to display a description of the metric in the bottom detail box at the lower left corner of the page.
System Performance Indicators
Indicator Definition
Globals/Second Most recently measured number of global references per second.
Global Refs Number of global references since system startup.
Global Sets Number of global Set and Kill operations since system startup.
Routine Refs Number of routine loads and saves since system startup.
Logical Requests Number of logical block requests since system startup.
Disk Reads Number of physical block read operations since system startup.
Disk Writes Number of physical block write operations since system startup.
Cache Efficiency Most recently measured cache efficiency (Global references / (physical reads + writes)).
In the description box, click the details link to display the [Home] > [System Usage] page in the bottom detail box. See the Monitoring System Performance section for details.
ECP and Shadowing Indicators
Indicator Definition
Application Servers Summary status of ECP (Enterprise Cache Protocol) application servers connected to this system.
Application Server Traffic Most recently measured ECP application server traffic in bytes per second.
Data Servers Summary status of ECP data servers to which this system is connected.
Data Server Traffic Most recently measured ECP data server traffic in bytes per second.
Shadow Source Summary status of shadow connections on this data source.
Shadow Server Summary status of shadows configured on this shadow server.
For more information on the first four indicators, the ECP indicators, see the Configuring Distributed Systems chapter of the Caché Distributed Data Management Guide.
In the description box for the last two indicators, the shadow indicators, click the details link to display the [Home] > [Shadow Servers] page. For more information on shadowing, see the Shadowing chapter of the Caché Data Integrity Guide.
System Time Indicators
Indicator Definition
System Up Time Elapsed time since this system was started.
Last Backup Date and time of last system backup.
You can run backups or view the backup history from the [Home] > [Backup] page. For more information on developing a backup plan, see the Backup and Restore chapter of the Caché Data Integrity Guide.
System Usage Indicators
Indicator Definition
Database Space Indicates whether there is a reasonable amount of disk space available for database files. Clicking details displays the [Home] > [Databases] page.
Journal Space Indicates whether there is a reasonable amount of disk space available for journal files. Clicking details displays the [Home] > [Journals] page.
Journal Entries Number of entries written to the system journal. Clicking details displays the [Home] > [Journals] page.
Lock Table Current status of the system Lock Table. Clicking details displays the [Home] > [Manage Locks] page.
Write Daemon Current status of the system Write daemon.
Transactions Current status of open local and remote (ECP) transactions. If there are no open transactions, status is Normal; status may also be Warning (if the duration of the longest open local or remote transaction is greater than 10 minutes) and Troubled (if greater than 20 minutes). Clicking details displays the [Home] > [Transactions] page.
Processes Most recent number of running processes. Clicking details displays the [Home] > [Processes] page.
CSP Sessions Most recent number of CSP sessions. Clicking details displays the [Home] > [CSP Sessions] page.
Most Active Processes Running processes with highest amount of activity (number of commands executed). Clicking details displays the [Home] > [Processes] page.
For more information on any of these topics, click Help under the InterSystems logo on the portal page displayed when you click for more details.
Errors and Alerts Indicators
Indicator Definition
Serious Alerts Number of serious alerts that have been raised. Clicking details displays the [Home] > [System Logs] > [View Console Log] page.
Application Errors Number of application errors that have been logged. Clicking details displays the [Home] > [System Logs] > [View Application Error Log] page.
See the Monitoring Log Files section in this chapter for more details.
Licensing Indicators
Indicator Definition
License Limit Maximum allowed license units for this system.
Current License Use License usage as a percentage of available license units.
Highest License Use Highest license usage as a percentage of available license units.
In the description box, click the details link to display the [Home] > [License Usage] > [License Activity Summary] page. For more information on licensing, see the Managing Caché Licenses chapter of the Caché System Administration Guide
Task Manager Upcoming Tasks
Indicator Definition
Upcoming Tasks Lists the next five tasks scheduled to run.
Task Name of the upcoming task.
Time Time the task is scheduled to run.
Status Task status—one of: scheduled, completed, running.
In the description box, click the details link to display the [Home] > [Task Manager] > [View Upcoming Tasks] page. For details on the Task Manager, see the Using the Task Manager section of the “Managing Caché” chapter of the Caché System Administration Guide.
Monitoring System Performance
System performance metrics are described in the following tables:
System Usage Table
To view the system usage statistics, navigate to the [Home] > [System Usage] page.
System Usage Statistics
Statistic Definition
Global references (all) Logical count of accesses to globals, including Sets, Kills, $Data, $Order, $Increment, $Query, and global references in expressions.
Global update references Logical count of global references that are Set, Kill, or $Increment operations.
Routine calls Number of calls to a routine.
Routine buffer loads and saves Total number of routine loads and saves as a result of ZLoad, ZSave, and running routines. (In a well-tuned environment, this number increases slowly, since most routine loads are satisfied by the routine cache memory without accessing the disk. Each routine load or save transfers up to 32 KB of data (64 KB for Unicode).)
Logical block requests Number of database blocks read by the global database code. (In a well-tuned environment, many of these reads are satisfied without disk access.)
Block reads Number of physical database blocks read from disk for both global and routine references.
Block writes Number of physical database blocks written to disk for both global and routine references.
WIJ writes Number of blocks written to the write image journal file.
Journal entries Number of journal records created—one for each database modification (Set , Kill, etc.) or transaction event (TStart, TCommit) or other event that is saved to the journal.
Journal block writes Number of 64-KB journal blocks written to the journal file.
Routine lines Number of routine lines executed since system startup.
Last update Date and time stamp of the displayed statistics.
See the Gathering Global Activity Statistics with ^GLOSTAT chapter for an alternative method of monitoring these statistics.
Generic (Shared) Memory Heap Usage
To view the Caché’s generic memory heap (gmheap) usage, referred to on this page as shared memory heap (SMH) usage, navigate to the [Home] > [System Usage] page, and click the Shared Memory Heap Usage link.
To change the size of the generic memory heap or gmheap (sometimes known as the shared memory heap or SMH), from the home page of the management portal choose System Administration then Configuration then Additional Settings then Advanced Memory; see Advanced Memory Settings in the “Caché Additional Configuration Settings” chapter of the Caché Additional Configuration Settings Reference for more information.
The column headings in the table on this page refer to the following:
Generic Memory Heap Usage
Identifier Definition
Miscellaneous Memory usage for miscellaneous purposes.
Audit System Memory usage for system auditing.
Classes Instantiated Memory usage for the class hash table and control blocks.
Event System Memory usage for the event system.
Global Mapping Memory usage for global mapping and subscript-level mapping (SLM).
License Upgrades Memory usage for license upgrades.
Lock Table Memory usage for the lock system.
National Language Support Memory usage for National Language Support (NLS) tables.
Performance Monitor Memory usage for the Caché Performance Monitor (^PERFMON).
Process Table Memory usage for the Process ID (PID) table.
Routine Buffer in Use Table Memory usage for routine buffer-in-use tables.
Security System Memory usage for the security system.
Shadowing Memory usage for shadowing.
Shared Library Memory usage for shared libraries.
TTY Hash Table Memory usage for TTY hash tables.
DB Name & Directory Memory usage for database names and directories.
iKnow Language Model Data Memory usage for iKnow language models.
ECP Memory usage for ECP.
Expand Daemon Memory usage for expanding daemons.
IPQ Memory usage for IPQ.
Total
Total for each column.
Available SMT & GST Available memory in the static memory table (SMT) and general string table (GST).
Total SMT & GST Total allocated and used memory in the static memory table (SMT) and general string table (GST).
Total SMH Pages Used Total directly allocated generi/shared memory heap (SMH) and string table allocated SMH memory, together with the total used/available memory in the static memory table (SMT) and the general string table (GST); the number of 64-KB pages is displayed parenthetically.
Monitoring Locks
Caché locks are created when a Caché process issues a LOCK command on a Caché local variable or global variable, as long as the entity is not already locked by another process. Entities need not exist in the database to lock them. The article Locking and Concurrency Control discusses Caché locks in detail.
To display locks system-wide, navigate to the [Home] > [View Locks] page from the System Operations section of the Management Portal home page. To delete selected locks system-wide, navigate to the [Home] > [Manage Locks] page from the System Operations section of the Management Portal home page. In both cases, the displayed lock table lists one row for each held lock and for each waiting lock request, identifying the owner. A single row may identify multiple locks held by an owner on the same entity. For example, holding an incremented lock or holding both a Shared lock and an Exclusive lock. If more than one process holds a lock on the same entity, each owner has its own row.
The Lock Table has the following column entries.
Column Heading Definition
Owner The process ID of the process holding or waiting for the lock. Contains the client system name if it is a remote lock.
ModeCount Lock mode and lock increment count. If the lock count is 1 the count is not displayed. For a list of ModeCount values, refer to the Lock Management chapter of Using Caché ObjectScript.
Reference Lock reference string of the lock item (does not include the database name).
Directory The database location of the lock item.
System The system name of where the lock is located, if it is the local system the column is blank.
Routine The routine line currently being executed by the process holding or waiting for the lock.
Remove Manage Locks only: If this lock is removable, this option along with the Remove all locks for process option (for local locks) or the Remove all locks from remote client option (for remote locks) appears in the row. Click the appropriate option to remove the lock. remove all locks for the process, or remove all locks from the remote client. If a lock you are removing is part of an open transaction, you are warned before confirming the removal.
In most cases, the only time you need to remove locks is as a result of an application problem.
For a more in-depth description of the LOCK command and its features, see the LOCK entry of the Caché ObjectScript Reference.
You may need to enlarge the size of the lock table if your system uses a large number of locks. You can do this using the Management Portal:
  1. Navigate to the [Home] > [Configuration] > [Advanced Memory Settings] page.
  2. In the locksiz (locksiz) row, click Edit.
  3. In the locksiz box, update the amount of memory allocated on your system for locks (in bytes), and click OK.
    The minimum is 65536; the maximum value depends on the value of gmheap (Generic Memory Heap Size). Increase the heap size if you need more room for the lock table. Caché rounds up the value to the next multiple of 64 KB. The default range is from 655361769472.
  4. Click Save and restart Caché for this information to take effect.
For more detailed information and alternative ways to manage locks, see the Lock Management chapter of Using Caché ObjectScript.
Monitoring Log Files
Several log files are available in the system manager directory, typically install-dir\mgr.
console log
Caché reports general messages, startup/shutdown, license, and network errors, certain operating system errors, and the success or failure of jobs started remotely from other systems through an operator console facility, which writes them to the console log, install-dir\mgr\cconsole.log by default. Caché System Monitor also writes notifications to the console log.
On Caché Windows-based systems, all console messages are sent to the console log file, the name of which is configurable. On Caché UNIX® or OpenVMS platform systems, you can configure console messages to be sent to the console log file, the console terminal, or both. See the console parameter in the Caché Parameter File Reference and Advanced Memory Settings in the Caché Additional Configuration Setting Reference for information about console log configuration.
The size of the cconsole.log file is monitored by System Monitor. The file grows until it reaches the configured maximum size (default 5 MB), at which point it is renamed to console.log.old.date_time, any existing console.log.old.date_time file is deleted, and a new console.log is created. The maximum number of megabytes used by the console log is therefore twice the configured maximum. To configure the maximum console log size, navigate to the [Home] > [Configuration] > [Startup Settings] page of the Management Portal and update the MaxConsoleLogSize setting.
Note:
If you have trouble starting Caché, use any text editor or text viewer to view the console log.
alerts log
Caché Monitor scans the console log at regular intervals for entries of the configured minimum severity and generates corresponding notifications, which it writes to the alerts log, install-dir\mgr\alerts.log, by default. Caché Monitor can be configured to send email notifications instead; see the Using Caché Monitor chapter of this guide for more information.
System Monitor log
Status messages about the functioning of Caché System Monitor (see the chapter Using Caché System Monitor in this guide) are written to the System Monitor log, install-dir\mgr\SystemMonitor.log.
The size of the SystemMonitor.log file is monitored by System Monitor. The file grows until it reaches the maximum size of 5 MB, at which point it is renamed to SystemMonitor.log.old, overwriting any existing SystemMonitor.log.old file, and a new SystemMonitor.log is created. The maximum number of megabytes used by the System Monitor log is therefore 10 MB. .
initialization log
The initialization log, cboot.log, contains information about the initialization of the Caché instance.
journal history log
The journal history log, journal.log, contains a list of all journal files maintained by the Caché instance and is used by all journal-related functions, utilities, and APIs to locate journal files. See the Journaling chapter of the Caché Data Integrity Guide for information about journaling.
These .log files are plain text files and can be viewed using any text editor or viewer.
To view the console log or the System Monitor log using the Management Portal, navigate to the [Home] > [System Logs] page and select [Console Log] or [System Monitor Log].
Note:
If the console log or System Monitor log is larger than 1 MB, only the most recent 1 MB portion is displayed by the Management Portal. Click the Show entire file link to display the entire file, which may require some time if the file is very large.
You can also view log information about application errors or ODBC errors by selecting [Application Error Log] or[xDBC Error Log] from the [Home] > [System Logs] page.
The contents of the Caché system error log, or SYSLOG, cannot be viewed in the management portal. To view entries in the Caché system error log, open a Terminal window and execute the ^SYSLOG routine in the %SYS namespace.