Using the Mirror Monitor

To display the Mirror Monitor, navigate to the System Operation > Mirror Monitor page on any mirror member.

On a failover member, the Mirror Monitor contains the following buttons and sections:

The View Mirror Journal Files button lets you view and search through the member’s mirror journal files or nonmirror journal files; see View Journal Files for more information.
The Stop Mirror On This Member button (backup only) temporarily stops mirroring on the backup, as described in Stopping Mirroring on Backup and Async Members.
One of two buttons to demote the backup to DR async, leaving the mirror with a single failover member; see Demoting the Backup to DR Async for more information.
Mirror Failover Member Information lists the member name, superserver address and mirror private address of each failover member (see Mirror member network addresses for information about these addresses).
Arbiter Connection Status shows the address (hostname and port) of the arbiter if one is configured, the current failover mode, and the status of the member’s arbiter connection, as follows:
- Both failover members are connected to the arbiter
- Only this member is connected to the arbiter
- This member is not connected to the arbiter (if the connection has been lost or no arbiter is configured)
See Automatic Failover Mechanics for information about the arbiter and the meaning of this connection information.

Note:

When a failover member loses contact with the arbiter a severity 2 message is sent to the messages log. If the member fails to rebuild the connection to the arbiter, another severity 2 message is logged.
Mirror Member Status provides the member type and status, journal transfer status, dejournaling status of each mirror member, as described in Mirror Member Journal Transfer and Dejournaling Status. This table also shows the X.509 DNs of members if configured.
Mirrored Databases lists information about each mirrored database on the current member, including its name and directory as well as its status and dejournaling status, as described in Mirrored Database Status. Mirrored Databases also lets you perform several operations on one or more databases.

Note:

The Mirrored Databases list includes only databases that are included in the mirror on the current member. For example, the databases listed on different reporting async members may be different as they may have different sets of mirrored databases.

On an async member, the Mirror Monitor contains the following buttons and sections:

The View Journal Files button (DR asyncs only) lets you view and search through the member’s mirror journal files or nonmirror journal files; see View Journal Files for more information.
The Stop Mirror On This Member button (DR asyncs only) temporarily stops mirroring on the async, as described in Stopping Mirroring on Backup and Async Members.
The Promote to Failover Member button (DR asyncs only) promotes a DR async to failover member; see Promoting a DR Async Member to Failover Member for information about this operation and its uses.
The section below the buttons displays the member name, async type, and X.509 DN (if configured) of the member.
Mirrors this async member belongs to provides information about each mirror a reporting async member belongs to and the member’s status, journal transfer status, and dejournaling status in that mirror, as described in Mirror Member Journal Transfer and Dejournaling Status. Each line includes a Details link to display information about the members of that mirror and a Stop Mirroring On This Member link to cause the async member to stop mirroring for that mirror, as described in Stopping Mirroring on Backup and Async Members
Mirror Member Status provides the type, status, journal transfer latency, and dejournal latency of all members of the mirror selected in Mirrors this async member belongs to, including the current async member.
If the async member belongs to a single mirror (which is the case with all DR asyncs), that mirror is displayed in this section by default; if the member belongs to more than one mirror, this section and the Mirrored Databases for MIRROR section below it do not appear until you click the Details link for one of the mirrors listed in Mirrors this async member belongs to section.
Mirrored Databases for MIRROR section is as described for failover members, with the same operations available. For reporting asyncs, the databases displayed are those in the mirror selected in Mirrors this async member belongs to and displayed in Mirror Member Status.

Mirror Member Journal Transfer and Dejournaling Status

When an InterSystems IRIS instance belongs to a mirror, its member type and status, journal transfer status, and dejournaling status are displayed by the Mirror Monitor and the ^MIRROR routine Status Monitor option, as described in Monitoring Mirrors.

The following tables describe the possible types and statuses displayed; the first shows statuses specific to particular member types, while the statuses in the second apply to all member types.

Type	Status	Description
Failover	Primary	Current primary.
Failover	Backup	Connected to primary as backup.
Failover	In Trouble	As primary, in a trouble state due to lost connection with the backup; see Automatic Failover Mechanics for complete information about the varying circumstances under which the primary can enter a temporary or indefinite trouble state.
Disaster Recovery	Connected	Connected to primary as async.
Read-Only Reporting	Connected	(as above)
Read-Write Reporting	Connected	(as above)
Indeterminate	Not Initialized	The member is not initialized (the mirror configuration is not yet loaded).

Status (any type)	Description
Transition	In a transitional state that will soon change when initialization or another operation completes; this status prompts processes querying a member’s status to query again shortly. When there is no operating primary, a failover member can report this status for an extended period while it retrieves and applies journal files in the process of becoming primary; if there is another failover member that is primary, the status is Synchronizing instead.
Synchronizing	Starting up or reconnecting after being stopped or disconnected, retrieving and applying journal files in order to synchronize the database and journal state before becoming Backup or Connected.
Waiting	Unable to complete an action, such as becoming Primary, Backup, or Connected; will retry indefinitely, but user intervention may be required. See messages log for details.
Stopped	Mirroring on member stopped indefinitely by user and will not start automatically; see messages log for details.
Crashed	Mirror no longer running due to unexpected condition; see messages log for details.
Error	An unexpected error occurred while fetching the member’s status.
Down	Displayed on other members for a member that is down or inaccessible.

Note:

Mirror member Type and Status can also be obtained using the $SYSTEM.Mirror.GetMemberType()Opens in a new tab and $SYSTEM.Mirror.GetMemberStatus(Opens in a new tab) methods. Some combinations of Type and Status not listed above are reported by these calls, as follows:

Not Member and Not Initialized—Instance is configured as not a mirror member.
Read-Only or Read-Write Reporting and M/N Status—Instance is an async member of several mirrors; supply the mirrorname argument to get Status for a particular mirror.

For backup and async mirror members, Journal Transfer indicates whether a mirror member has the latest journal data from the primary and, if not, how far behind journal transfer is, while Dejournaling indicates whether all of the journal data received from the primary has been dejournaled (applied to the member’s mirrored databases) and, if not, how, how far behind dejournaling is. The following tables describe the possible statuses for these fields displayed by the Mirror Monitor and ^MIRROR. (These fields are always N/A for the primary.)

Journal Transfer	Description
Active (backup only)	The backup has received the latest journal data from and is synchronized with the primary. (See in Backup Status and Automatic Failover for more information about Active backup status.) Note that the backup can be Active even if its Dejournaling status is not Caught up; as long as the backup has all the needed journal files, they can be dejournaled even after it has lost its connection to the primary.
Caught up	On the backup, indicates that the backup has received the latest journal data from the primary, but is not fully synchronized in that the primary is not waiting for it to acknowledge receipt of journal data. This status is often transient, as when the backup reconnects to the mirror. On an async, indicates that the async has received the latest journal data from and is synchronized with the primary.
time behind	The member is a specific amount of time behind the primary, with time representing the amount of time elapsed between the timestamp of the last journal block the member received and the current time.
Disconnected on time	The member was disconnected from the primary at the specified time.

As noted, Active in the Journal Transfer field indicates that the backup has received all journal data from and is synchronized with the primary, and is therefore capable of taking over from the primary during failover without contacting the primary’s ISCAgent to obtain additional journal data.

Dejournaling	Description
Caught up	All journal data received from the primary has been dejournaled (applied to the member’s mirrored databases).
time behind	Some journal data received from the primary has not yet been dejournaled, with time representing the amount of time elapsed between the timestamp of the last dejournaled journal block and the last journal block received from the primary.
Disconnected on time	The member was disconnected from the primary at the specified time.
Warning! Some databases need attention.	At least one mirrored database is not in a normal state; databases should be checked.
Warning! Dejournaling is stopped.	Dejournaling has been stopped by an operator or because of an error; see Managing Database Dejournaling.

Caught Up in the Dejournaling field for an Active backup failover member or Caught Up in both the Dejournaling field and the Journal Transfer field for an async member indicates that the member has received the most recent journal data from the primary and applied the most recent global operations contained in that data. If the member is not caught up, the amount of time elapsed since generation of the most recent journal data or writing of the most recent operation on the primary is displayed instead.

Incoming Journal Transfer Rate

Below the mirror member status list on backup and async members, the rate at which journal data has arrived from the primary since the last time the Mirror Monitor was refreshed is displayed under Incoming Journal Transfer Rate for This Member.

When the Mirror Monitor page is first loaded, this area displays the text --- (will be displayed on refresh). When the page is next refreshed, the information displayed depends on whether the incoming journal data is compressed (see Mirror Traffic Compression), as follows:

If the journal data is not compressed, the incoming journal data rate is provided in kilobytes (KB) per second, for example:
42345 KB/s (22s interval)
If the incoming journal data is compressed, the display includes the incoming compressed data rate, the incoming journal (uncompressed) data rate, and the ratio of the latter to the former, for example:
14288 KB/s network; 39687 KB/s journal; ratio 2.78:1 (143s interval)

Mirrored Database Status

Important:

On backup and DR async members, the Missing Mirrored Databases Report on the Mirror Monitor page alerts you to any mirrored databases that are present on the primary but not on the current member. This is very important, as the backup, or a DR async if promoted to backup, cannot successfully take over in the event of a primary outage if it does not have the full set of mirrored databases. The full mirror database name of each missing database is listed. The Missing Mirrored Databases Report is not displayed if there are no missing databases.

On all members, the Mirrored Databases list on the Mirror Monitor page displays one of the following statuses for each database listed:

Status	Description
Normal (primary only)	The mirrored database is writable (if not a read-only database) and global updates are being journaled.
Dejournaling (backup and async)	The database has been activated and caught up and the mirror is applying journal data to the database.
Needs Catchup	The database has been activated but not caught up yet; the user-initiated Catchup operation is needed.
Needs Activation	The database has not been activated yet; the user-initiated Activate and Catchup operations are needed.
Catchup Running	The user-initiated Catchup operation is running on the database.
Dejournaling Stopped	Dejournaling has been stopped by an operator or an error; see Stopping Mirroring on Backup and Async Members and Managing Database Dejournaling.
Database Dismounted	The database is dismounted.
Orphaned (backup and async)	The mirrored database does not exist on the primary.
Obsolete	The mirrored database is obsolete and should be removed from the mirror.

On the primary, the Next Record to Dejournal column contains N/A if the status of the database is Normal. Otherwise, the column includes the following:

Time is the timestamp at the beginning of the next journal record to be applied to the database, or Current if this matches the primary's current journal position.
FileName is the name of the mirror journal file containing the next journal record to be applied.
Offset is position within the journal file of the beginning of the next journal record to be applied.

The status of a database and the operations related to it (Activate and Catchup) are discussed in Activating and Catching Up Mirrored Databases; the operations are available in the drop-down below the list. You can also use the dropdown to mount dismounted databases (but not to dismount mounted databases). You can use the Remove link or select Remove from the drop-down to remove a listed database from the mirror; see Remove Mirrored Databases from a Mirror for more information.

Using the ^MIRROR Status Monitor