Managing Caché Memory

The goal of memory planning and management is to provide enough memory to all of the entities that use it under all normal operating circumstances. This section discusses the following:

Important:

Correct sizing and allocation of system memory are critical in maximizing the performance and availability of Caché and your application. Before installing Caché, be sure to review the guidelines that follow, calculate your initial memory requirements, and determine whether your system has sufficient memory.

For additional information on allocating memory, see Calculating System Parameters for UNIX®, Linux, and macOS and Shared Memory Allocation on Windows in the chapter “Installing Caché on Windows”. For an in-depth look at Caché memory planning and allocation by an InterSystems senior technology architect, see InterSystems Data Platforms and Performance Part 4 - Looking at MemoryOpens in a new tab on InterSystems Developer Community.

Sizing System Memory for Caché

Generally, there are four main consumers of memory on a server hosting a Caché instance. At a high level, you can calculate the amount of physical memory required by simply adding up the requirements of each of the items on the following list:

Operating system, including the file system cache
Installed applications
Caché and application processes
Caché is process-based. If you look at the operating system statistics while your application is running, you will see numerous processes running as part of Caché.
Caché shared memory, which includes
- The database and routine caches
- The generic memory heap (gmheap)
- Other shared memory structures

For the best possible performance, all four of these should be maintained in physical (system) memory under all normal operating conditions. Virtual memory and mechanisms for using it such as swap space and paging are important because they enable the system to continue operating during a transient memory capacity problem, but the highest priority is to include enough physical memory to avoid the use of virtual memory.

Calculating Initial Memory Requirements

Of course, every application is different and any given system may require a series of operational adjustments to optimize memory use. However, the following list provides general guidelines to use as a basis in sizing memory for your application. Benchmarking and performance load testing the application will further influence your estimate of the ideal memory sizing and parameters.

Important:

If you have not configured sufficient physical memory on a Linux system and thus regularly come close to capacity, you run the risk that the out of memory killer may misidentify long-running Caché processes that touch a lot of memory in normal operation, such as the write daemon and CSP server processes, as the source of the problem and terminate them. This will result in an outage of the Caché instance and require crash recovery at the subsequent startup. Disabling the out of memory killer is not recommended, however, as this safety mechanism keeps your operating system from crashing when memory runs short, giving you a chance to intervene and restore Caché to normal operation. The recommended way to avoid this problem is to configure enough physical memory to avoid any chance of the out of memory killer coming into play. (For a detailed discussion of process memory in Caché, see Caché Process Memory.)

General guidelines for sizing memory for your application are as follows:

System memory to provision (install in a physical server or allocate to a virtual server)
Start with 4 to 8 GB per CPU core (physical or virtual). This core count does not include any threads such as Intel HyperThreading (HT) or IBM Simultaneous Multi-Threading (SMT). So, for example, if you have an IBM AIX LPAR with 8 cores allocated, the calculation would be 4-8 GB * 8 = 32 to 64 GB of total RAM allocated to that LPAR, even with SMT-4 enabled (which would appear as 32 logical processors).
Shared memory to allocate within Caché
- On servers with less than 64 GB of RAM, allocate
  - 50% of total memory to the database cache
  - 256 MB minimum to the routine cache
  - 256 MB minimum to the generic memory heap
- On servers with more than 64 GB of RAM, allocate
  - 70% of total memory to the database cache
  - 512 MB minimum to the routine cache
  - 384 MB minimum to gmheap
Swap space or page file to configure
As a general guideline, configure the smaller of a) 25 to 50% of your physical memory or b) 32 GB as virtual memory. As previously noted, swapping and paging degrade performance and should come into play only when transient memory capacity problems require it. Further, you should configure alerts to notify operators when the system uses virtual memory so they can take immediate action to avoid more severe consequences.

Note:

When large and huge pages are configured, as is highly recommended, Caché shared memory segments are pinned in physical memory and never swapped out; for more information, see Large and Huge Pages.

Note:

If you are configuring a data server in an ECP cluster, see Memory Use on Large ECP SystemsOpens in a new tab in the “Developing Distributed Applications” chapter of the Caché Distributed Data Management Guide for important information about adjustments to database cache sizes that may be necessary.

Vertically Scaling for Memory

Performance problems in production systems are often due to insufficient memory for application needs. Adding memory to the server hosting one or more Caché instances lets you allocate more to the database cache, the routine cache, generic memory, or some combination. A database cache that is too small to hold the workload’s working set forces queries to fall back to disk, greatly increasing the number of disk reads required and creating a major performance problem, so this is often a primary reason to add memory. Increases in generic memory and the routine cache may also be helpful under certain circumstances.

Large and Huge Pages

Where supported, the use of large and huge memory pages can be of significant performance benefit and is highly recommended, as described in the following:

IBM AIX® — The use of large pages is highly recommended, especially when configuring over 16GB of shared memory (the sum of the database cache, the routine cache, and the generic memory heaps, as discussed in Calculating Initial Memory Requirements).
By default, when large pages are configured, the system automatically uses them in memory allocation. If shared memory cannot be allocated in large pages, it is allocated in standard (small) pages. However, you can use the memlock parameter for finer-grained control over large pages.
For more information, see Configuring Large and Huge Pages in this chapter and memlock in the Configuration Parameter File Reference.
Linux (all distributions) — The use of static huge pages (2MB) when available is highly recommended for either physical (bare metal) servers or virtualized servers. Using static huge pages for the Caché shared memory segments yields an average CPU utilization reduction of approximately 10-15% depending on the application.
By default, when huge pages are configured, Caché attempts to provision shared memory in huge pages on startup. If there is not enough space, Caché reverts to standard pages and orphans the allocated huge page space, potentially causing system paging. However, you can use the memlock parameter to control this behavior and fail at startup if huge page allocation fails.
For more information, see Configuring Large and Huge Pages in this chapter and memlock in the Configuration Parameter File Reference.
Windows
The use of large pages is recommended to reduce page table entry (PTE) overhead.
By default, when large pages are configured, Caché attempts to provision shared memory in large pages on startup. If there is not enough space, Caché reverts to standard pages. However, you can use the memlock parameter to control this behavior and fail at startup if large page allocation fails.
For more information, see Shared Memory Allocation on Windows in the “Installing Caché on Windows” chapter and memlock in the Configuration Parameter File Reference.

Allocating Memory Within Caché

There are two primary steps involved in configuring the way in which a Caché instance uses memory, described in the following sections:

The first action, allocating memory for routine and database caches, determines memory available to hold code and data. The second action, configuring gmheap, determines memory available for all other purposes. These, taken both separately and together, are important factors in the performance and functioning of the instance. For guidelines for initial memory allocations within Caché, see Calculating Initial Memory Requirements.

Two other memory settings are described in the section:

Other Caché Memory Settings

Important:

Some changes on the Memory and Startup page require an instance restart and some do not. If you modify a setting that requires a restart and save your changes, none of the changes take effect until you restart Caché, even those that by themselves do not require a restart. If a restart is required, the message Modification saved. You must restart system for the new values to take effect. displays. After you close the page, the warning message does not appear again to remind you that a restart is required, and it is therefore best to restart the instance immediately.

Allocating Memory to the Routine and Database Caches

To allocate memory for routine and database caches,

On the Management Portal, navigate to the Memory and Startup page (System Administration > Configuration > System Configuration > Memory and Startup).
Select Manually.

Important:

When Caché is first installed, memory for routine and database caches is set, by default, to be Automatically allocated. With this default, Caché allocates a conservative fraction of the available physical memory for the database cache, not to exceed 1 GB. This setting is not appropriate for production use. Before deploying the system for production use or before performing any tests or benchmarking intended to simulate production use, you must manually create an appropriate memory allocation for the database cache (typically as much memory as possible after taking into account the needs of application and operating system processes) by selecting Manually and following the procedures described in this section.

Allocating Memory to the Routine Cache

Memory Allocated for Routine Cache (MB) — The routine cache specifies the system memory allocated for caching server code.

Caché takes the total amount of memory you allocate for the routine cache and creates buffers of different sizes according to this formula: half the total memory in 64 KB buffers, three-eighths in 16 KB buffers, and one-eighth in 4 KB buffers. For example, if you allocate 500 MB, Caché creates 3906 64 KB buffers (250 MB), 11718 16 KB buffers (187.5 MB), and 15,625 4 KB buffers (62.5 MB). These groups of buffers are sometimes called pools, as in “the 16 K buffer pool”.

Bear in mind the following points regarding the routine cache memory allocation:

For typical production instances, a good starting allocation is 350-400 MB. However, the ideal allocation for a given application depends on may factors, and adjustment may be necessary to optimize performance.
The maximum number of buffers that Caché allocates to any pool is 65,529. The format for Caché routines does not allow more than 32,768 characters for literal strings regardless of the setting for the maximum routine size.
The minimum allocation is 35 MB; the instance will allocate this much even if you specify less. The minimum number of buffers in a pool is 430.

Note:

You can also allocate memory to the routine cache using the routines setting in the cache.cpf file; for more information, see routines in the Caché Parameter File Reference.

Allocating Memory to the Database Cache

Memory Allocated for [blocksize] Database Cache (MB) — The database cache specifies the system memory allocated for buffering data; this is also called creating global buffers. The database cache and the memory allocated to it are sometimes referred to as the global buffer pool.

Enter a separate allocation for each enabled database block size listed. The 8K block size is required and is listed by default. To enable more database block sizes (16K, 32K, 64K), use the DBSizesAllowed setting on the Startup Settings page (System Administration > Additional Settings > Startup); see DBSizesAllowed in the Caché Additional Configuration Settings Reference for more information.

Both block size and the maximum number of buffers available have implications for performance. To determine how many global buffers Caché will create for databases with a particular block size, divide the allocation for a block size by the block size; the smaller the block size, the larger the number of global buffers that will be created for databases with that block size. For guidelines for selecting the appropriate block sizes for your applications, see “Large Block Size Considerations” in the “Configuring Caché” chapter of the Caché System Administration Guide.

Important:

Note:

You can also allocate memory to the database cache using the globals setting in the cache.cpf file; for more information, see globals in the Caché Parameter File Reference.

Configuring Generic Memory Heap (gmheap)

You can configure gmheap on the Advanced Memory page (System Administration > Configuration > Additional Settings > Advanced Memory).

gmheap — The generic memory heap (also known as the shared memory heap) determines the memory available to Caché for purposes other than the routine and database caches.

To see details of used and available memory for gmheap, use the Shared Memory Heap Usage page (System Operation > System Usage page; click the Shared Memory Heap Usage link).

For more information, see gmheap in the “Advanced Memory Settings” section of the Caché Additional Configuration Settings Reference and also Generic (Shared) Memory Heap Usage in the “Monitoring Caché Using the Management Portal” chapter of the Caché Monitoring Guide.

Other Caché Memory Settings

Other memory settings that you can change on the Memory and Startup page are:

Maximum per Process Memory (KB) — The maximum memory allocation for a process for this Caché instance. The default is 262144 KB. The allowed range is 128 KB to 2147483647 KB.

Note:

It is not necessary to reset this value unless you have set it lower than its default (262144 KB). If you receive <STORE> errors, increase the size.

This amount of process private memory, which is used for symbol table allocation and various other memory requirements (for example I/O device access structures and buffers), is allocated in increasing extents as required by the application until the maximum is reached. The initial allocation is 128 KB. Once this memory is allocated to the process, it is not deallocated until the process exits.
If you select the Enable Long Strings check box, Caché allocates a large string stack to handle long strings for each process.