Caché High Availability Guide
Using HP Serviceguard with Caché
[Back] [Next]
   
Server:docs1
Instance:LATEST
User:UnknownUser
 
-
Search:    

Caché can be configured as a resource controlled by HP Serviceguard. This appendix highlights the key portions of the configuration of Serviceguard including how to incorporate the custom Caché resource. Refer to your HP documentation and consult with HP on all cluster configurations.

When using Caché in a high availability environment controlled by HP Serviceguard:
  1. Install the hardware and operating system according to your vendor recommendations for high availability, scalability and performance; for more information, see Hardware Configuration.
  2. Configure HP Serviceguard to control a simple shared disk resource with a virtual IP address and test that common failures are detected and the cluster continues operating; for more information, see HP-UX and HP Serviceguard Configuration.
  3. Install Caché according to the guidelines in this appendix and test connectivity to the database from your application; for more information, see Install Caché in the Cluster.
  4. Test disk failures, network failures, and system crashes. Test and understand your application’s response to such failures; for more information, see Test and Maintenance.
Hardware Configuration
Configure the hardware according to best practices for the application. In addition to adhering to the recommendations of HP and your hardware vendor, consider the following:
HP-UX and HP Serviceguard Configuration
Prior to installing the Caché-based application, follow the recommendations described below when configuring HP-UX and Serviceguard. These recommendations assume a two-node cluster with quorum disk.
HP-UX
When you configure HP-UX, ensure that:
HP Serviceguard
This appendix assumes you are using HP Serviceguard version 11.18 or later.
Note:
The disk monitor service is available with HP Serviceguard version 11.20 and the cmvolmond daemon.
Caché is configured as an external module in a failover package whose configuration typically has the following attributes:
package_type                  failover
auto_run                      yes
node_fail_fast_enabled        yes
failover_policy               configured_node
failback_policy               manual
local_lan_failover_allowed    yes
The procedures in this appendix assume the following configuration choices:
In general, do the following:
  1. Edit the cache.sh script’s inst variable to be equal to the Caché instance being controlled.
  2. Copy the cache.sh file to the /etc/cmcluster/<pkg>/ directory, where <pkg> is the package name (for example: /etc/cmcluster/cachepkg/).
  3. Ensure the permissions, owner, and group allow this script to be executable by the Serviceguard daemons.
Note:
InterSystems provides a sample HP Serviceguard service script (cache.sh) as part of a development installation of Caché. The sample script is located in the dev/cache/HAcluster/HP/ subdirectory under the Caché installation directory. It is not necessary to do a development installation in the cluster; copy the dev/cache/HAcluster/HP/cache.sh file to all cluster members from any developer’s installation, then modify as described in the previous procedure.
Install Caché in the Cluster
This section describes the additional steps required when installing Caché in a shared-disk cluster environment; they assume a two-node cluster that has never had Caché installed:
Note:
For information about upgrading Caché in an existing failover cluster, see Upgrading a Cluster in the “Upgrading Caché” chapter of the Caché Installation Guide.
Installing a Single Instance of Caché in the Cluster
Installing a single instance of Caché is most common in production environments; it is not common in test or development clusters.
Note:
If you might install additional instances of Caché to run in the cluster at some point in the future, follow the procedure described in Installing Multiple Instances of Caché in the Cluster.
To install a single instance of Caché and your application:
  1. Mount the entire set of disks required for Caché and the application to run (for installation, data files, journal files, and any other disk required for the application in use) on one node.
  2. Create a link from /usr/local/etc/cachesys to the shared disk. This forces the Caché registry and all supporting files to be stored on the shared disk resource. A good choice is to use a ./usr/local/etc/cachesys/ subdirectory under your intended install directory.
    Assuming Caché will be installed in the /cacheprod/cachesys/ subdirectory, create the following directories and links on the node that has mounted the shared disk:
    mkdir –p /cacheprod/cachesys/usr/local/etc/cachesys
    mkdir –p /usr/local/etc/
    ln –s /cacheprod/cachesys/usr/local/etc/cachesys/ /usr/local/etc/cachesys
  3. Run Caché cinstall on the node with the mounted disks, ensuring the required users and groups (either default or custom) are available on all nodes in the cluster, and that they all have the same UID and GIDs.
  4. Stop Caché and dismount the shared disks.
  5. Configure the volume group and file system sections of your cluster configuration. Ensure the fs_opts for your journal file systems match current file system mount recommendations.
  6. Configure the virtual IP address ensuring the application, all interfaces, any ECP clients, etc. use the virtual IP address as configured here to connect to the Caché resource.
  7. Start the package and relocate the package to the second node.
    Note:
    At this point there is no control of Caché in the cluster configuration.
  8. On the second node in the cluster, create the link in /usr/bin/etc/ and the links in /usr/bin for ccontrol and csession:
    mkdir -p /usr/local/etc/
    ln –s /cacheprod/cachesys/usr/local/etc/cachesys/ /usr/local/etc/cachesys
    
    ln –s /usr/local/etc/cachesys/ccontrol /usr/bin/ccontrol 
    ln –s /usr/local/etc/cachesys/csession /usr/bin/csession
  9. Test starting and stopping Caché via the script:
    cd /etc/cmcluster/cachepkg/
    ./cache.sh start
    ./cache.sh stop
  10. Add the Caché service to be monitored. The service script acts as the monitor of the Caché instance:
    service_name          cachesvc
    service_cmd           "/etc/cmcluster/cachepkg/cache.sh monitor"
  11. Add the Caché module by adding the path of the cache.sh script as an external_script:
    external_script        /etc/cmcluster/cachepkg/cache.sh
    Note:
    Serviceguard appends a stop or a start parameter to the cache.sh script as appropriate.
  12. Test manual package start, stop, and relocation.
Installing Multiple Instances of Caché in the Cluster
Installing multiple instance of Caché is most common in test and development clusters; it is not common in production environments.
Note:
For information about strategies for deploying multiple instances of Caché in HP Serviceguard clusters, please contact the InterSystems Worldwide Response Center (WRC).
To allow for installing multiple instances of Caché and your application, follow these steps during initial cluster configuration:
  1. Mount the entire set of disks required for Caché and the application to run (for installation, data files, journal files, and any other disk required for the application in use) on one node.
  2. Run Caché cinstall on the node with the mounted disks, ensuring the required users and groups (either default or custom) are available on all nodes in the cluster, and that they all have the same UID and GIDs.
  3. The /usr/local/etc/cachesys directory and all its files must be available to all nodes at all times. Therefore, after Caché is installed on the first node, copy /usr/local/etc/cachesys to each node in the cluster, as follows:
    cd /usr/local/etc/
    scp –p cachesys node2:/usr/local/etc/
    Note:
    Keep the Caché registries on all nodes in sync using ccontrol create and/or ccontrol update; for example:
    ccontrol update CACHEPROD directory=/cacheprod/cachesys/ versionid=2012.2.4.500.0
  4. Verify that ownerships and permissions on this cachesys directory and its files are identical on all nodes.
  5. Stop Caché and dismount the shared disks.
  6. Configure the volume group and file system sections of your cluster configuration. Ensure the fs_opts for your journal file systems match current file system mount recommendations.
  7. Configure the virtual IP address ensuring the application, all interfaces, any ECP clients, etc. use the virtual IP address as configured here to connect to the Caché resource.
  8. Start the package and relocate the package to the second node.
  9. On the second node in the cluster, create links in /usr/bin for ccontrol and csession:
    ln –s /usr/local/etc/cachesys/ccontrol /usr/bin/ccontrol 
    ln –s /usr/local/etc/cachesys/csession /usr/bin/csession
  10. Test starting and stopping Caché via the script:
    cd /etc/cmcluster/cachepkg/
    ./cache.sh start
    ./cache.sh stop
  11. Add the Caché service to be monitored. The service script acts as the monitor of the Caché instance:
    service_name          cachesvc
    service_cmd           "/etc/cmcluster/cachepkg/cache.sh monitor"
  12. Add the Caché module by adding the path of the cache.sh script as an external_script:
    external_script        /etc/cmcluster/cachepkg/cache.sh
    Note:
    Serviceguard appends a stop or a start parameter to the cache.sh script as appropriate.
  13. Test manual package start, stop, and relocation.
Special Considerations
Consider the following for your applications:
Test and Maintenance
Upon first setting up the cluster, be sure to test that failover works as planned. Any time changes are made to the operating system, its installed packages, the disk, the network, Caché, or your application, be sure to test that failover continues to work as expected.
In addition to the topics described in this section, you should contact the InterSystems Worldwide Response Center (WRC) for assistance when planning and configuring your HP Serviceguard service to control Caché. The WRC can check for any updates to the cache.sh script, as well as discuss failover and HA strategies.
Typical Full Scale Testing Must Go Beyond a Controlled Service Relocation
While service relocation testing is necessary to validate that the package configuration and service scripts are all functioning properly, be sure to also test response to simulated failures.
Be sure to test failures such as:
Testing should include a simulated or real application load, as follows:
Keep Patches and Firmware Up to Date
Avoid known problems by adhering to a patch and update schedule.
Use Caché Monitoring Tools
Use the Caché console log, the Caché Monitor and the Caché System Monitor to be alerted to problems with the database that may not be caught by the cluster software. (See the chapters Monitoring Caché Using the Management Portal, Using the Caché Monitor and Using the Caché System Monitor in the Caché Monitoring Guide for information about these tools.)