Promoting a DR Async Member to Failover Member

Promoting a DR Async Member to Failover Member

A disaster recovery (DR) async mirror member can be promoted to failover member, replacing a current failover member if two are configured or joining the current member if there is only one. For example, when one of the failover members will be down for a significant period due to planned maintenance or following a failure, you can temporarily promote a DR async to take its place (see Temporary Replacement of a Failover Member with a Promoted DR Async). During true disaster recovery, when both failover members have failed, you can promote a DR to allow it to take over production as primary failover member, accepting the risk of some data loss; see Manual Failover to a Promoted DR Async During a Disaster for more information.

When a DR async is promoted to failover member, it is paired, if possible, with the most recent primary as failover partner; when this cannot be done automatically, you are given the option of choosing the failover partner. Following promotion, the promoted member communicates with its failover partner's ISCAgent as any failover member does at startup, first to obtain the most recent journal data, then to become primary if the failover partner is not primary, or to become backup if the failover partner is primary. The promoted member cannot automatically become primary unless it can communicate with its failover partner to obtain the most recent journal data.

When promoting a DR async to failover member, there are several important considerations to bear in mind:

• Depending on the location of the DR async, network latency between it and the failover partner may be unacceptably high. See Network Latency Considerations for information about latency requirements between the failover members.

• When the DR async becomes a failover member, the failover member compression setting is applied, rather than the async member compression setting as before the promotion (see Mirror Traffic Compression for information about these settings). Depending on the network configuration, you may need to adjust the failover member compression setting, as described in Editing or Removing a Failover Member, for optimal mirror function.

• When a mirror private network is used to connect the mirror private addresses of the failover members, as described in Sample Mirroring Architecture and Network Configurations, a DR async that is not connected to this network should be promoted only to function as primary, and this should be done only when no other failover member is in operation. If a DR async is promoted when a primary is in operation but does not have access to the primary’s mirror private address, it cannot become backup; it will, however, be able to obtain journal data from the primary’s agent and become primary with the most recent journal data when the primary is shut down.

• If a mirror VIP is in use, and the promoted DR async is not on the VIP subnet, some alternative means must be used to redirect user connections to the promoted DR should it become primary; for example, manually updating the DNS name to point to the DR async’s IP instead of the VIP, or configuring one of the mechanisms discussed in Redirecting Application Connections Following Failover.

In some disaster recovery situations, however, the promoted DR async cannot contact any existing failover member’s agent. When this is the case, you have the option of promoting the DR with no failover partner, as described under Promotion With Partner Selected by User in this section. This means that the DR can become primary only, using only the journal data it already has and any more recent journal data that may be available on other connected mirror members, if any. When this happens, the new primary may not have all the journal data that has been generated by the mirror, and some application data may be lost. If you restart a former failover partner while a DR async promoted in this manner is functioning as primary, it may need to be rebuilt; see Rebuilding a Mirror Member for more information. Be sure to see the DR promotion procedure later in this section for details.

When possible, the promoted DR async’s failover partner is selected automatically, as follows:

• When there is a running primary failover member, the primary is automatically selected as failover partner; the promoted member obtains the most recent journal data from it and becomes the backup. If InterSystems IRIS is running on the current backup, that member is simultaneously demoted to DR async; if it is not, the member is demoted to DR async when InterSystems IRIS is restarted.

• When InterSystems IRIS is not running on any failover member but the ISCAgents on both failover members (or one if there is only one) can be contacted, the most recent primary is automatically selected as failover partner and the promoted member obtains the most recent journal data from it and becomes the primary. When InterSystems IRIS is restarted on the former primary, it automatically becomes the backup; when InterSystems IRIS is restarted on the former backup, it automatically becomes a DR async.

When InterSystems IRIS is not running on any failover member and at least one ISCAgent cannot be contacted, the promotion procedure informs you of which agents cannot be contacted and gives you the option of choosing a failover partner. To avoid the possibility of data loss, you should select the failover member that was last primary, even if its agent cannot be contacted. The results differ depending on the selection you make and ISCAgent availability, as follows:

• If the agent on the partner you select can be contacted, the promoted DR async obtains the most recent journal data from it and then becomes primary. When InterSystems IRIS is restarted on the partner, it automatically becomes backup.

• If the agent on the partner you select cannot be contacted, the promoted DR async does not become primary until the partner’s agent can be contacted and the most recent journal data obtained. At any time before the partner’s agent becomes available, however, you can, force the promoted member to become primary (as described in Manual Failover to a Promoted DR Async During a Disaster) without obtaining the most recent journal data; some application data may be lost as a consequence.

• If you choose no failover partner, the promoted DR async attempts to obtain the most recent available journal data from all other connected async mirror members before becoming primary. Because there may not be any connected members with more recent journal data than the promoted DR async, some application data may be lost.

  When you make this choice, you have the option of setting the no failover state on the promoted DR async so that it will prepare to become primary, including obtaining journal data from other connected members, but not become primary until you clear no failover. This allows you to perform any additional verification you wish and to bring additional members online, if possible, to potentially make more journal data available before allowing the promoted DR async to become primary.

  Note:

  Messages about successful and unsuccessful attempts to contact mirror members to review their journal data, as well as successful and unsuccessful attempts to retrieve recent data when it is identified, are posted in the messages log.

  Caution:

  Do not restart InterSystems IRIS on a former failover member whose ISCAgent was down when the DR async was promoted until you have set ValidatedMember=0 in the [MirrorMember] section of the Configuration Parameter File for the InterSystems IRIS instance, as described in the DR promotion procedure that follows.

When the failover partner is not selected automatically, the following rules apply:

• Any former failover member that is not selected as partner becomes a DR async member when InterSystems IRIS is restarted.

• On any former failover member whose agent could not be contacted at the time the DR async was promoted, you must at earliest opportunity and before restarting InterSystems IRIS instance set ValidatedMember=0 in the [MirrorMember] section of the Configuration Parameter File for the InterSystems IRIS instance (see [MirrorMember]). This instructs the InterSystems IRIS instance to obtain its new role in the mirror from the promoted DR async, rather than reconnecting to the mirror in its previous role. The ^MIRROR routine lists the failover member(s) on which this change is required.

To promote a DR async member to failover member, do the following:

1. On the DR async member that you are promoting to failover member, navigate to the System Operation > Mirror Monitor page to display the Mirror Monitor.

2. Click the Promote to Failover Member button at the top of the page.

3. Follow the instructions provided by the resulting dialog boxes. In the simplest case, this involves only confirming that you want to proceed with promotion, but it may include selecting a failover partner or no partner, as described earlier in this section.

4. If a VIP is configured for the mirror, the promoted DR async must have a network interface on the VIP’s subnet to be able to acquire the VIP in the event of becoming primary (due to manual failover or to a later outage of the primary while operating as backup).

   • If the DR async has exactly one interface on the VIP’s subnet, the procedure automatically selects this interface.

   • If the DR async has more than one interface on the VIP’s subnet, the procedure asks you to choose an interface.

   • If the DR async does not have an interface on the VIP’s subnet, the promotion procedure warns you that this is the case and asks you to confirm before proceeding. If you go ahead with the procedure and promote the DR async, you will have to make take manual steps to allow users and applications to connect to the new primary, for example updating the DNS name to point to the DR async’s IP instead of the VIP.

5. When a former failover member’s agent is available at the time a DR async is promoted, it automatically sets ValidatedMember=0 in the [MirrorMember] section of the Configuration Parameter File for the InterSystems IRIS instance (see [MirrorMember]). This instructs the InterSystems IRIS instance to obtain its new role in the mirror from the promoted DR async, rather than reconnecting to the mirror in its previous role.

   If a former failover member’s agent cannot be contacted at the time of promotion, this change cannot be made automatically. Therefore, at the earliest opportunity and before InterSystems IRIS is restarted on any former failover member whose agent could not be contacted at the time of promotion, you must manually set ValidatedMember=0 by editing the Configuration Parameter File for the InterSystems IRIS instance. The instructions list the former failover member(s) on which this change must be made.

   Caution:

   Restarting InterSystems IRIS on a mirror member whose agent was down at the time of DR async promotion without first setting ValidatedMember=0 may result in both failover members simultaneously acting as primary.