Caché Installation Guide
Calculating System Parameters for OpenVMS
[Back] [Next]

This document is part of Installing Caché on OpenVMS.” It explains how you can calculate the best parameter values for your system.

For optimal Caché performance, you need to calculate proper values for certain Caché system parameters. These values allow you to determine whether you need to adjust certain OpenVMS system parameters. The values you choose should minimize swapping and paging that require disk access, and thus improve system performance.
Review this section carefully and calculate the proper values for both your operating system and Caché before proceeding. Use the OpenVMS Parameter Calculator and the tables provided here to record the current and calculated values for your system parameters. You can then refer to these tables when you install Caché. After your system is running, you may need to adjust these values to gain optimal performance.
InterSystems recommends enabling hyper-threading on all Poulson (Itanium 9500 series)-based or Intel Xeon-based processors. InterSystems recommends disabling hyper-threading for older Itanium processors. Please consult the InterSystems Worldwide Response Center (WRC) if you have questions about your specific server and platform.
If you are not already familiar with the memory organization at your operating system level, consult the appropriate system documentation and read the overviews of process and physical memory allocation provided in the next few sections:
Once you are familiar with OpenVMS memory organization, use the following sections to tune the parameters for Caché:
OpenVMS Process Memory Organization
An OpenVMS process is any entity that can be scheduled. It may be interactive or run as a batch process. If the OpenVMS process creates subprocesses, that process and all its subprocesses together are called a job.
Many operating system tasks are OpenVMS processes. All processes, whether they are interactive, batch, or operating system-specific, share the structures that this section discusses. OpenVMS can run several processes concurrently. Depending on the size of your computer, OpenVMS could be running between 4 and 1,000 or more simultaneous processes.
Each process uses a share of physical memory, where it stores certain process-specific operating system information, any code the process is using, and any data that it uses or generates. For example, if a programmer edits a routine with an OpenVMS editor, OpenVMS loads the editor into memory, allocates space for the editor's buffers and variables and loads all or a portion of what is being edited into memory.
Shared Memory and Caché
Sometimes code or data is available to more than one process. The editor is one example of this type of code. Another example is the code for Caché and some of the data created with Caché.
Any piece of code or data that several processes can share is stored in a shared section of memory. Shared sections are created by the first user of a system, or during system initialization. They are then made available to subsequent users of that system rather than being duplicated and stored with each individual process.
Caché system code, global buffers, and routine buffers are stored in shared sections of memory.
How OpenVMS Measures Process Memory
OpenVMS measures the code and data area that a process requires in pages. Shared sections are also measured in pages, sometimes called shared or global pages.
On OpenVMS systems, page size is not fixed. Normally, it is 8192 bytes. Each page is divided into pagelets that are 512 bytes long.
OpenVMS Page Organization
Sometimes a process needs to access many more pages than can fit within the physical memory allocated to the process. When this situation arises, OpenVMS stores the extra pages on disk. The operating system keeps only the most active, or most recently used, pages in memory. The total of all the pages of a process, whether they are stored in memory or on disk, is called the virtual memory set of the process. The pages currently stored in memory are called the physical working set of the process, often called working set. The status of each page is kept in the process's working set list.
OpenVMS keeps track of free pages available to processes in a free page list. It also keeps a modified page list, which tracks pages that have been changed by a process and must be written to disk before being made available on the free page list.
Sometimes a process needs access to a page not currently in its physical working set. When this happens, OpenVMS searches two places for that page. First, OpenVMS looks in the modified and free page lists, because the page might still be available in physical memory. If the page does not appear on either of these lists, OpenVMS then retrieves the page from disk.
When OpenVMS must retrieve a page, the process has incurred a page fault. When the new page is in the modified or free page list, the page fault is called a soft page fault. When the page is only available on disk, the page fault is called a hard page fault. While OpenVMS retrieves the page from disk, the process becomes inactive. It remains in a page fault wait state until OpenVMS has successfully retrieved the necessary page. The length of time that this procedure requires varies according to the size of the disk, its rotation speed, and the load on the system.
When OpenVMS retrieves the new page, it places one of the current working set pages in the modified or free page list. OpenVMS chooses a page that the process has not used recently.
You can minimize soft page faults by setting the OpenVMS process parameters WSDEFAULT, WSQUOTA, and WSEXTENT appropriately for each process.
The Caché SETQUOTA utility lets you modify the default values of quota parameters for jobbed processes, including the network daemons and job servers. See Modifying Quotas for Jobbed Processes with SETQUOTA for information on using SETQUOTA.
OpenVMS Physical Memory Allocation
In OpenVMS, you use the SYSGEN utility to set system limits on process memory use. You can then set process-by-process (user-by-user) limits with the AUTHORIZE utility.
The following table describes the OpenVMS system parameters that affect the memory available to the working set of a process.
OpenVMS Working Set Parameters
Parameter Definition
WSDEFAULT Initial size of the working set in 512-byte pagelets.
WSQUOTA Maximum guaranteed amount of physical memory, in pagelets, that OpenVMS allocates to a given process. WSQUOTA is limited by the value of the SYSGEN parameter WSMAX. If WSQUOTA is larger than WSMAX, the user only has access to the amount of memory in WSMAX.
WSEXTENT Indicates the maximum amount of physical memory which the process can own. The amount WSEXTENT less WSQUOTA is called “borrowed” memory, which is only allocated to the process if the system's current free list size exceeds the amount specified in the parameter BORROWLIM.
WSINC The number of pagelets by which the working set size of a process is increased when it needs to grow.
WSDEC The number of pagelets by which the working set size of a process is decreased when it needs to be reduced in size.
BORROWLIM Minimum number of pagelets that must be in the free page list before a process can borrow pages in WSINC amounts up to WSEXTENT pages.
PFRATH (Page Fault Rate High) Indicates the number of page faults that must occur within the period specified by AWSTIME, before OpenVMS makes an “automatic working set adjustment” to increase the working set by WSINC.
PFRATL (Page Fault Rate Low) If the number of page faults is less than this value, OpenVMS makes an “automatic working set adjustment” to decrease the working set by WSDEC, but not below the point set by AWSMIN.
While parameters have units of pagelets, OpenVMS allocates memory in pages. For this reason, the parameter values you choose should be even multiples of the number of pagelets per page. OpenVMS automatically rounds these parameters if you do not follow this guideline.
Default Memory Allocation
The following steps summarize the algorithm OpenVMS uses to determine the amount of physical memory available to each process. Each capitalized term represents a system parameter that OpenVMS uses in allocating physical memory.
  1. The process begins executing an image. OpenVMS allocates to it the number of pagelets specified by the quota parameter WSDEFAULT.
    Page Allocation:
  2. If the process has a higher page fault rate than PFRATH, OpenVMS allocates more pagelets to it. OpenVMS allocates these pagelets in increments of WSINC until the process has WSQUOTA total pagelets.
    Page Allocation:
  3. If the process continues to have a high page fault rate and the free list size exceeds BORROWLIM, OpenVMS will continue to allocate pagelets in WSINC blocks until the free list size is inadequate or WSEXTENT is reached.
    Page Allocation:
  4. If the process’s page fault rate drops below PFRATL, OpenVMS removes pages from the working set in increments of WSDEC until the page fault rate exceeds PFRATL.
  5. When the process exits the image, it loses these additional pages. For example, consider a monthly batch payroll job. When you run it in July, it begins execution at WSDEFAULT, and gains pages until it reaches WSEXTENT. When you run it in August, it again begins execution at the default value of WSDEFAULT.
See your OpenVMS documentation for more on the OpenVMS page allocation scheme.
Process Control of Memory Allocation
OpenVMS provides several facilities to alter working set parameters on a process-by-process or user-by-user basis. The following table summarizes these methods:
Working Set Control
Type of Process Method of Working Set Control
Users The AUTHORIZE utility lets you change the default working set on a user-by-user basis.
Interactive Processes The DCL command SET WORKING_SET allows interactive processes to change working set parameters.
Batch Processes The SUBMIT command qualifiers for working sets allow batch processes to alter physical memory allocations.
Batch Queues You can use the DCL command INITIALIZE/QUEUE to alter working set parameters for batch queues.
Keeping Memory Free for New Processes
Memory is finite. Every time a new process begins under OpenVMS, it takes WSDEFAULT pages away from the total number of available pages. The remaining memory is available for the free page list and for WSINC additions to each process's allocation.
Heavily loaded systems can run so many concurrent processes that the number of pages remaining for use in the free page list becomes very small. OpenVMS includes a parameter called FREELIM that sets a lower limit for the number of pages in the free page list.
The size of the free and modified page lists directly affects the ratio of hard page faults to soft page faults. Hard page faults cause a process to experience a resource wait state, resulting in slow execution. For better performance, it is important to tune the system to minimize hard page faults. While soft faults are less expensive, an excessive soft page fault rate can also lead to poor performance.
Whenever OpenVMS detects that the size of the free page list falls below FREELIM, OpenVMS starts a mechanism to bring the free list size up to FREEGOAL. To achieve this, if all other mechanisms are inadequate, OpenVMS may move the entire working set of an inactive process out to disk. This procedure is called swapping. The process placed on disk is swapped out. When the process is brought back into memory, it is swapped in. Swapping has a detrimental effect on overall system performance.
Allocation for Shared Sections
OpenVMS uses a slightly different approach for allocating memory for shared code and data. The GBLSECTIONS parameter sets the number of shared sections to be allocated when the system is started. The GBLPAGES parameter sets the number of global page table entries. Every group of 128 page entries requires 4 bytes of resident memory. In addition, OpenVMS uses the GBLPAGFIL parameter to set the maximum number of page file blocks available for global pages.
How Caché Uses OpenVMS Memory
Caché uses both shared memory and memory private to each process when running on OpenVMS. It uses the following techniques to increase performance:
Balancing Memory Locking and Paging
All Caché code is shared, and can be physically locked in memory. The same is true for the global and routine buffer pools.
If a routine is larger than the locked portion of a routine buffer, only a portion of the routine in the routine buffer is locked in memory. The remainder of the routine is loaded into unlocked physical memory.
Locking shared data and routines in memory allows better response time since memory access is quicker than disk access. As a result, the more globals and routines that are kept in memory, the better. Memory, however, is a finite resource. The more global and routine buffers that are allocated, the less memory is available for OpenVMS processes. When less memory is available, more OpenVMS paging occurs.
The goal, therefore, is to choose a number of global and routine buffers that lets you keep enough globals and routines in memory without negatively affecting OpenVMS paging. This document provides guidelines for selecting these values.
Using Process-Private Space to Reduce Paging
Every Caché process maps to the shared memory sections, but it also has access to a private area of memory called process-private space. This private area includes variables, arrays, stacks, and other data structures that belong to a particular process.
A portion of this private area of memory may be locked into the process working set to reduce paging. None of the private area, however, is physically locked in memory.
Using Resident Memory
The OpenVMS platform makes use of the Reserved Memory Registry, a mechanism for reserving memory within memory-resident sections. The two features that Caché supports, memory-resident global sections and shared page tables, are always used as a pair.
The advantages of using a memory-resident global section mapped via a shared page table for the shared memory requirement are:
Caché uses one shared memory section; it is either a traditional global section, a memory-resident section, or a reserved (also resident) named memory section. Caché uses a memory-resident section, with shared page tables, any time the total shared memory requirement is greater than 5MB.
Calculating OpenVMS Parameters
The accompanying OpenVMS Parameter Calculator offers good starting values for both Caché and OpenVMS parameters. Later, if your system is not functioning as well as you would like, you can adjust these parameters to achieve optimal performance. The following sections describe the process of determining OpenVMS parameters:
You may want to print out the tables that follow so that you can record the relevant values.
Determine Parameter Calculator Input Values
This section helps you find appropriate values for the fields that control the number of processes and the number of global and routine buffers used by Caché. You need to determine these values, so that you can use the Parameter Calculator to compute both Caché and OpenVMS parameters accurately.
During installation you can set these and other Caché parameters, or you can retain default values. If you find later that your system needs tuning, you can adjust the values at any time using the Management Portal. You need to restart Caché for most parameter changes to take effect.
As you determine the values for these fields, also record them in the Input to OpenVMS Parameter Calculator Table:
A bug exists in the OpenVMS operating system when attempting to allocate resident global buffers if insufficient space is available. To prevent Caché from hanging at startup, it is best to calculate the specific memory needs of your system and manually enter these values on the Memory and Startup page (System Administration > Configuration > System Configuration > Memory and Startup) of the Management Portal or in the configuration (cache.cpf) file itself.
Determine Number of Processes
Before you can determine how many global and routine buffers Caché should allocate at startup, you must first determine the maximum number of Caché processes that run on your system at one time.
Every user in Caché constitutes one Caché process. Every Caché Job command creates a process. You should consider any process that appears on the %SS system status display, except the system processes Garbage Collector and Write daemon. As a rule of thumb, use the number of processes called for in your Caché license for this value; it cannot exceed your license limit.
Enter the value in the Number of Processes field in the parameter calculator.
Determine Size of Routine Cache
Each routine buffer holds one and only one routine at a time. Many processes can share a routine buffer. Thus, when a process switches from one Caché routine to another, it merely accesses the other routine in another buffer. When no process is currently accessing a buffer, the buffer is returned to the pool of free buffers, and a new routine can be loaded into the empty buffer as it is required by another Caché process. If all buffers are occupied when Caché needs to load a routine, it chooses the least recently used (LRU) buffer, not the least frequently used (LFU) buffer. Allowing more buffers enhances performance.
The nature of the application influences the appropriate number of buffers. If many users access a small number of routines, a relatively small number of buffers will suffice for optimum response time. Conversely, an application with a large number of routines will benefit from a large number of buffers.
The parameter calculator starts with a default value for the number of routine buffers. This default is based upon a medium-sized site. You can enter a different value for this parameter in the Routine Buffer Pool field in the parameter calculator based on the specifics of your site.
If you prefer to enter a value in this field with an MB unit of measure, this value is used for the Memory for Routine Cache (MB) field in the Calculation Result section and also in calculations involving routine buffers; it is not recalculated.
Determine Number of Global Buffers Per Process
The parameter calculator assumes the global buffers are 8 KB in size. The calculator starts with recommended “rule of thumb” values for clustered and nonclustered systems.
If your system is memory-poor, you can reduce this value. Do not, however, decrease it below a multiple of 32 KB per process.
Using more global buffers helps the performance of most sites. You can use the statistics produced by the ^GLOSTAT utility to determine if adding more global buffers will reduce disk access and thereby improve performance.
Record Current OpenVMS Parameter Values
Every system has memory needs other than those for Caché. For example, there are memory requirements for computer language layered products as well as for OpenVMS itself. To analyze these requirements for your system, use the following procedures to display the values of the indicated parameters prior to installing Caché.
If you are performing an upgrade, shut down Caché to get accurate numbers.
  1. Run the OpenVMS SYSGEN utility to display the current value of each parameter listed in the OpenVMS System Parameter Values Table.
    SYSGEN> SHOW parameter-name 
  2. Run the OpenVMS SYSGEN utility to display the current value of the process parameters listed in the OpenVMS Process Parameter Values Table.
    Alternatively, you may run the OpenVMS AUTHORIZE utility to display the current value of the user authorization file (UAF) parameters for each user account that uses Caché:
    UAF> show user-name
    You must then set the UAF records for all of the user accounts individually once the calculator returns the minimum recommendations for these values.
Use the Caché OpenVMS Parameter Calculator
This section describes how to use the Caché OpenVMS Parameter Calculator. This tool is an interactive HTML document that provides up-to-date calculations for the OpenVMS parameters required by Caché. The default values provide reasonable numbers for a medium sized installation. InterSystems experts are available to provide additional help configuring and tuning Caché for your site.
Perform the following steps to retrieve parameter values:
  1. Choose whether you are calculating parameters for a clustered system. If you select Yes, you receive both the cluster master values and values for any additional nodes; otherwise, the tool calculates values for a single node.
  2. As input to the OpenVMS Parameter Calculator use the values you determined in the previous section for the following fields:
    Input to OpenVMS Parameter Calculator
    Parameter User-supplied value
    Number of Processes  
    Routine Buffer Pool  
    Global Buffers per Process  
  3. To calculate the necessary parameter values, open the Caché OpenVMS Parameter Calculator in a browser running on a system that is hosting a Caché instance and Caché Management Portal by entering the following URL manually. If you are reading this document in a browser, you can probably open it in a new tab.
    Substitute the Caché instance’s webserver port number if it is not using 57772, the default webserver port.
  4. Fill in the values returned by the calculator in the appropriate tables in the following section, Analyzing the Calculation Results.
Analyzing the Calculation Results
The results from the calculator are suggested values; treat them as a guideline. You may need to adjust settings to balance and maximize system performance, depending on your site’s hardware and software configuration. The following sections describe the categories of calculator output values:
Update Caché Parameters
The following values are for Caché parameters that you can update during the installation or by using the Management Portal from a web server after you install Caché.
Output Caché Parameters
Parameter Calculated value
Number of Global Buffers  
Memory for Database Cache (MB)  
Memory for Routine Cache (MB)  
Update the memory cache settings using the Memory and Startup page (System Administration > Configuration > System Configuration > Memory and Startup) of the portal. Click Manually for the Configure Memory Settings option, so that you can enter the values recommended for your system.
Update OpenVMS System Parameters
The system parameter values computed by the OpenVMS Parameter Calculator are the amounts you need to add to your current OpenVMS system to make it ready to run Caché. Caché will consume this much memory, so evaluate your system to make sure that there is the proper amount of memory available.
Use this table to help you combine the required amounts returned by the calculator with your current system values. Update these parameters by using the SET command of the OpenVMS SYSGEN utility.
OpenVMS System Parameter Values
Parameter Current value Required Caché amount Resulting value
The parameter calculator also provides the following values:
Update Resident Memory
If the total required shared memory is more than 5 MB (true of most installations), Caché automatically allocates all memory from resident memory. Caché does not require you to reserve a named memory-resident section, but be sure to update the GBLPAGES and GBLPAGFIL parameters as the calculator indicates. Although this memory is not pageable, OpenVMS requires sufficient GBLPAGFIL and still deducts GBLSECTION pages.
In OpenVMS, reserved sections can be either named sections that you create with SYSMAN and have a fixed size or resident sections, which include reserved sections, but can also be “anonymous.”
The following are the advantages of reserving a named section:
The drawback to creating a named reserved section is that if you want to increase the size, you may have to restart your OpenVMS system to reconfigure the amount of space reserved for the memory-resident global section. You can reserve this space when the system starts (via SYSMAN and AUTOGEN) or you can allocate it dynamically. If you do not reserve it, when Caché requests it, there may be insufficient space available and the request fails. If Caché attempts to use a memory-resident global section and it cannot, it allocates the global buffer pool out of a non–memory-resident global section.
Use the Resident Memory value if you want to Reserve a Named Memory-Resident Section for Caché.
Reserve a Named Memory-Resident Section
  1. Use the value returned by the OpenVMS Parameter Calculator for Resident Memory as the reserved memory size.
  2. Use SYSMAN to reserve a section of memory and supply a name for the named section. You can use any name that consists of alphanumeric characters (and the underscore) and is not longer than 43 characters. The syntax for SYSMAN is:
    MCR SYSMAN ! run sysman
    SYSMAN> RESERVED_MEMORY ADD "Resident_Memory_Name" - 
  3. Run AUTOGEN to process the Reserved Memory Registry data file which contains the information to establish your memory-resident global section. After the system is restarted, the SYSMAN command displays the shared memory section you reserved:
    MCR SYSMAN ! run sysman
  4. Update the configuration file by using the Management Portal that connects to your OpenVMS Caché instance from a web server:
    1. In the useresidentmem (ResidentMemorySectionName) setting row, click Edit.
    2. Enter the name you reserved in the SYSMAN utility (Resident_Memory_Name in the example in step 2) in the Value box and click Save.
    Alternately, you can modify the Caché configuration file (cache.cpf) to use the reserved section; set the reserved memory parameter in the [config] section so that it reads:
    useresidentmem=<resident memory section name>
    The <resident memory section name> is the name of the section you specified in SYSMAN when you reserved it. For the example in step 2, the setting is as follows:
    If you name this section, you must include the name in the configuration file.
  5. Restart Caché. If it cannot use the reserved memory section, Caché displays a message and stores an error code in the Caché system error log (see Caché System Error Log in the “Monitoring Caché Using the Management Portal” chapter of the Caché Monitoring Guide for more information).
If you use resident memory to map your global buffer pool, you may be able to reduce some of the system parameters, in particular, WSMAX. Typically OpenVMS comes with a WSMAX default value much larger than is required for running Caché.
Update OpenVMS Process Parameters
InterSystems recommends minimum values for some PQL process quota parameters. This is because Caché maintains a table, GJOBQ, which holds default values for many of these parameters. You must have authorized these values for Caché processes to be able to use the values in the GJOBQ table.
You do not need to calculate values for these parameters. The minimum suggested values appear in the OpenVMS Parameter Calculator. Enter these values in the Recommended minimum value column in the following table.
OpenVMS Process Parameter Values
Parameter Current value Recommended minimum value
If your current value for any of these parameters is less than the recommended minimum value, you should adjust the values during installation using the SET command of the OpenVMS SYSGEN utility or the OpenVMS AUTHORIZE utility for each user account.
Modifying Quotas for Jobbed Processes with SETQUOTA
The SETQUOTA utility lets you modify the default values of quota parameters for jobbed processes, including the network daemons and job servers. These parameters affect system performance. SETQUOTA can be set to run automatically during multiuser startup, or interactively after startup. If the default quota values are not providing good performance, use SETQUOTA in one of the ways described here to experiment with other values until you find the combination that provides the best performance.
Setting Quotas During Startup
SETQUOTA sets three quotas when it runs automatically during instance startup:
To change the default quota values for the next startup, you must set the appropriate node(s) of the ^SYS("QUOTAS") global. SETQUOTA modifies job quota tables according to the current values in this global, as shown in the following table:
To change quotas for these processes
Set this node
Network daemons
  • The first job server created by STU or ZSTU during startup
  • Any job that does not use a job server
  • Any additional job servers created during STU or ZSTU
  • All jobbed processes
In each case, quota can be any of the quota parameters in following table.
Parameter Value
For example, to change the startup value of the quota ASTLM from 40 (the default) to 80, you would issue the following command:
Modifying Quotas System-Wide After Startup
To change quotas once Caché is running, use the APPLY^SETQUOTA entry point, which lets you revise quotas for all jobbed processes, including any job servers after the first one created during startup. This is useful when you want to modify quotas without shutting down Caché.
For example, to change quotas for the network daemons when you activate the network after startup, you must first issue two commands to modify the process-level quota table, as follows:
Then activate the network.
Resetting Quotas to Default Values
The DEFAULT^SETQUOTA entry point lets you reset quotas to the default values. You can use this entry point to reset either the system or process quota table.
To reset all quotas system-wide to their default values, issue the following command:
To reset all quotas for your process and any future jobs it spawns, issue the following command:
Existing jobs are not affected when you reset quotas.
Displaying Current Quotas
The DISPLAY^SETQUOTA entry point lets you display the current quota values in both the system and process tables. Display these values before and after modifying them, so that you will know which values are providing the best performance.
To display the system-wide quota table, issue the following command:
To display the process-level quota table, issue the following command: