Caché Installation Guide
Calculating System Parameters for UNIX® and Linux
[Back] [Next]
   
Server:docs1
Instance:LATEST
User:UnknownUser
 
-
Search:    

This document is part of the Caché Installation Guide for UNIX® and Linux. This document explains how you can calculate the best parameters for your system. It is divided into two sections:

For optimal Caché performance, you need to calculate proper values for certain Caché system parameters. These values allow you to determine whether you need to adjust certain system level parameters. The values you choose should minimize swapping and paging that require disk accesses, and thus improve system performance.
Review this section carefully and calculate the proper values for both your operating system and Caché before proceeding. Use the tables provided here to record the current and calculated values for your system level parameters. You can then refer to these tables when you install Caché. After your system is running, you may need to adjust these values to gain optimal performance.
If you are not already familiar with the memory organization at your operating system level, consult the appropriate system documentation.
Determine Memory and Disk Requirements
This section outlines the basic system requirements for most systems. Because these requirements vary by platform, consult your platform documentation for additional information. The specific requirements include the following:
Calculate Memory Requirements
Use the breakdown of memory usage shown in the following table to calculate the memory your system needs for Caché.
UNIX® Memory Requirements
Components Memory Requirements
Operating system 1800 KB (operating system dependent)
Caché 842 KB
Global database cache 8 KB per buffer
Routine cache 32 KB per routine buffer
User overhead 1024 KB per process
Network (if present) 300 KB per port for each network system process (DMNNET and RECEIVE). Caché ports have two DMNNET system processes per port. In addition, there is a network shared memory requirement, which depends on the number of ports and the number of remote hosts configured. For a basic system, this requirement is about 304 KB.
By default, Caché automatically allocates shared memory, including routine buffers and global buffers, to a total of one-eighth of the system available shared memory space. If you plan to run large applications or support large numbers of users, tune the system according to the following formula:
            (number of routine buffers)*32 KB
            + (number of global buffers)*(block size)
            + 4 MB
            ___________________________________
            = Shared memory needed
For applications where load growth is reflected in the number of simultaneous direct Caché sessions, the memory demand to accommodate the processes increases as the computing power increases. For example, a system that is upgraded from 4 to 8 cores would be capable of supporting a much larger number of sessions (that is, processes). Since each process consumes memory, it might be necessary to increase physical memory.
Note:
The amount of memory per process may vary depending on the application and can be larger than the default value recommended in the UNIX® Memory Requirements table.
For configurations dedicated to servers with a limited number of processes (for example, ECP Data Server or Ensemble), an increase in the load does not necessarily involve a greater number of processes. Therefore, a larger load on a more powerful system may not require more memory for processes.
Support for Huge Memory Pages for Linux
Using pages of much larger size than the default saves memory by saving space in PTE tables. To allocate Huge Pages on Linux, do the following:
  1. Check the status.
    /proc/meminfo contains huge pages information. By default, no huge pages are allocated. Default huge page size is 2 MB. For example:
    HugePages_Total:     0
    HugePages_Free:      0
    HugePages_Rsvd:      0
    Hugepagesize:     2048 KB
  2. Change the number of huge pages.
    You can change the system parameter directly: For example, to allocate 2056 Huge Pages, execute:
    # echo 2056 > /proc/sys/vm/nr_hugepages
    Note:
    Alternatively, you can use sysctl(8) to change it:
    # sysctl -w vm.nr_hugepages=2056  
    Huge pages must be allocated contiguously, which may require a reboot. Therefore, to guarantee the allocation, as well as to make the change permanent, do the following:
    1. Enter a line in /etc/sysctl.conf file:
      echo "vm.nr_hugepages=2056" >> /etc/sysctl.conf  
    2. Reboot the system.
    3. Verify meminfo after reboot; for example:
      [root woodcrest grub]# tail -4 /proc/meminfo
      HugePages_Total:  2056
      HugePages_Free:   2056
      HugePages_Rsvd:      0
      Hugepagesize:     2048 KB
  3. Verify the use of Huge Pages by Caché.
    When Caché is started, it reports how much shared memory was allocated; for example, a message similar to the following is displayed (and included in the cconsole.log file):
    Allocated 266MB shared memory (huge pages): 189MB global buffers, 24MB routine buffers
    The amount of memory available in Huge Pages should be greater than the total amount of shared memory to be allocated; if it is not greater, Huge Pages are not used.
    Note:
    To prevent Caché from starting if huge pages are configured but cannot be allocated, set the memlock parameter to 7 (see memlock in the Caché Parameter File Reference).
    It is not advisable to specify HugePages_Total much higher than the shared memory amount because the unused memory will not be available to other components.
    When Caché is configured to lock the shared memory segment in memory to prevent paging, Huge Pages can provide the required increase in the maximum size that may be locked into memory, as described in the Locked-in Memory section of the Red Hat Linux Platform Notes in this appendix.
Support for Large (16 MB) and Huge (16 GB) Pages on IBM AIX®
In addition to 4-KB and 64-KB pages, IBM AIX® supports 16-MB (“large”) pages and 16-GB (“huge”) pages. You must configure IBM AIX® to use these page sizes and configure the number of pages of each page size; IBM AIX® does not automatically change the number of configured large or huge pages based on demand.
Pages of these sizes should be configured only in high-performance environments because memory allocated to large pages can be used only for large pages, and the memory allocated to huge pages can be used only for huge pages.
To allocate large and huge pages, users must have the CAP_BYPASS_RAC_VMM and CAP_PROPAGATE capabilities or have root authority.
By default on IBM AIX®, Caché specifies the SHM_LGPAGE option together with the SHM_PIN option; this indicates to the operating system that large pages or huge pages are to be used if configured.
Note:
If you want to use a small-page size, and still have the pin option, use the memlock option; this allocates shared pages without the Large Page/PIN option, and the shared memory is pinned after the allocation.
Configuring Large Pages for IBM AIX®
Configure large pages using the vmo command as follows:
vmo -r -o lgpg_regions=<LargePages> -o lgpg_size=<LargePageSize>
where <LargePages> specifies the number of large pages to reserve, and <LargePageSize> specifies the size, in bytes, of the hardware-supported large pages.
Note:
On systems that support dynamic Logical PARtitioning (LPAR), you can omit the -r option to dynamically configure large pages without a system reboot.
For example, the following command configures 1 GB of large pages:
# vmo -r -o lgpg_regions=64 -o lgpg_size=16777216
Once you have configured large pages, run the bosboot command to save the configuration in the boot image. After the system comes up, enable it for pinned memory using the following vmo command:
vmo -o v_pinshm=1
Configuring Huge Pages for IBM AIX®
Configure huge pages via a system’s Hardware Management Console (HMC). To configure the number of huge pages on a system, under the Managed System’s Properties menu, select Show Details in the Advanced Options field of the Memory tab.
Important:
You must power off the entire managed system to change the number of huge pages on the system.
Once a managed system has been configured with huge pages, you can assign those pages to partitions by changing a partition’s profile.
Note:
To prevent Caché from starting if huge pages are configured but cannot be allocated, set the memlock parameter to 7 (see memlock in the Caché Parameter File Reference).
Calculate Swap Space
The amount of swap space available on your system should never be less than the amount of real memory plus 256 KB.
With this minimum in mind, InterSystems recommends the following value as the minimum amount of swap space needed for Caché:
                       ((# of processes + 4)† * (1024 KB))‡  
                     +           total global buffer space  
                     +          total routine buffer space  
                     _____________________________________
                     =                  Minimum swap space
† Add 4 to the # of processes for the Caché Control Process, the Write daemon, the Garbage Collector, and the Journal daemon. Also add 1 for each slave Write daemon. The # of processes must include all user and jobbed processes which might run concurrently. If you are running networking, add 1 for the RECEIVE system process plus the number of DMNNET daemons you have running (2 per port).
‡ The 1024 KB number is approximate. It is based on the current size of the Caché executable and grows with the partition size you allocate to each Caché process. On most systems, provide only as much swap space as necessary. However, some systems require you to provide swap space for the worst case. Under these conditions, you need to increase this number to as high as 1.5 MB, depending on the partition size you specify.
Be sure to confirm that your UNIX® system permits the amount of swap space you require. For specific information about swap space on your system, consult your UNIX® operating system manual.
Solaris Swap Space
To calculate swap space for the Solaris platform:
swap –l
Example:
>swap –l
swapfile                   dev     swaplo blocks   free
/dev/dsk/c0t2d0s0         136,0      16    526304   526304
/dev/dsk/c0t2d0s1         136,1      16    2101184 2101184
IBM AIX® Swap Space
To display swap space for IBM AIX®:
lsps –a
Page Space  Physical Volume   Volume Group    Size %Used  
Active  Auto  Type
hd6         hdisk2            rootvg                   512 MB      72     
yes   yes    lv
HP-UX Swap Space
To display swap space for HP-UX:
swapinfo (3M)
# /usr/sbin/swapinfo
            KB      KB      KB   PCT    START/         KB
TYPE      AVAIL    USED    FREE  USED   LIMIT RESERVE  PRI  NAME
dev      524288  138260  386028   26%       0       -    1  /dev/vg00/lvol2
reserve       -   78472  -78472
memory   195132  191668    3464   98%
Calculate Disk Requirements
In addition to the swap space you just calculated, you need disk space for the following items:
Although you do not need to remove any installation files after completing the installation procedure, you can do so if you are short on disk space. The installation program tells you how much space can be saved, and asks if you want to delete the installation files.
Determine Number of Global Buffers
Caché supports the following maximum values for the number of global buffers:
Set your values to less than the maximum number of buffers.
For more information, see globals in the “config” appendix of the Caché Parameter File Reference and Memory and Startup Settings in the “Configuring Caché” chapter of the Caché System Administration Guide.
Determine Number of Routine Buffers
Caché supports the following maximum value for the number of routine buffers:
65,535
Set your values to less than this maximum number of buffers.
For more information, see routines in the “config” appendix of the Caché Parameter File Reference and Memory and Startup Settings in the “Configuring Caché” chapter of the Caché System Administration Guide.
Determine Maximum Number of Users
The maximum users allowed by Caché is the lowest of the following values:
For more information, see Determining License Capacity and Usage in the “Managing Caché Licensing” chapter of the Caché System Administration Guide.
Determine Maximum Database Size
The ulimit parameter in UNIX® determines the maximum file size available to a process. For the Caché Manager group, the value of ulimit should either be unlimited or as large as the largest database you may have.
For more information, see Configuring Databases in the “Configuring Caché” chapter of the Caché System Administration Guide.
Configure UNIX® Kernel Parameters
The following sections describe issues related to tuning and performance on various UNIX® platforms:
Set Values for Tunable UNIX® Parameters
Caché uses a configurable number of semaphores, in sets whose size you define. The parameters SEMMNI, SEMMNS, and SEMMSL reflect the number of semaphores per set and the total number of semaphores Caché uses. The UNIX®/Linux parameters that govern shared memory allocation are SHMMAX, SHMMNI, SHMSEG, and SHMALL. Caché uses shared memory and allocates one segment of shared memory; the size of this segment depends on the area set aside for global buffers and routine buffers. It uses the following formula to determine the segment's minimum size:
                       space required for routine buffers
                     +  space required for global buffers
                     +                               4 MB
                    _____________________________________
                     =         Shared memory segment size
If you are distributing your data across multiple computers, Caché allocates a second segment; by default, there is no memory allocated for the second segment. (If you plan to use distributed data, contact your vendor or InterSystems support for configuration guidelines.) You can alter NBUF and NHBUF according to other system requirements. Because Caché does all its own disk buffering, you should keep NBUF and NHBUF small. The following table lists the most common names of the UNIX® parameters that you may need to change, the minimum value InterSystems recommends for each parameter, and a brief description of each. Verify that your parameter values are set to at least the minimum value. Certain parameters may not be implemented on all platforms or may be referred to differently. Refer to platform-specific tuning notes for more information.
Tunable UNIX® Parameters
Kernel Parameter Recommended Minimum Value Definition
CDLIMIT Number of bytes in largest virtual volume Maximum size of a file.
MSGMAX 2 KB Maximum message size, in bytes.
MSGMNI Number of Caché instances x 3; each Caché instance uses three message queues Maximum number of uniquely identifiable message queues that may exist simultaneously.
NOFILES 35 Number of open files per process.
SEMMNI Product of SEMMNI and SEMMSL must be greater than the # of user processes + 4 Number of semaphore identifiers in the kernel; this is the number of unique semaphore sets that can be active at any one time.
SEMMNS 128 or ... Total number of semaphores in the system. User processes include jobbed processes and all other semaphores required by other software.
Number of processes expected to run. If the process table might expand, use a larger number to provide for expansion.
SEMMSL See SEMMNI Maximum number of semaphores per identifier list.
SHMALL 60 KB or ... Maximum total shared memory system-wide. Units should be in KB. 1000 represents the MCOMMON shared region.
1000 + total global buffer space+ total routine buffer space *
SHMMNI 3 Maximum number of shared memory identifiers system-wide.
SHMSEG 3 Number of attached shared memory segments per process.
SHMMAX 60 KB or ... Maximum shared memory segment size in KB.
1000 + total global buffer space+ total routine buffer space
* This is the minimum value for SHMALL required for Caché UNIX®. You must also take into account any other applications that use shared memory. If you are unsure of other shared memory use, calculate SHMALL as SHMSEG multiplied by SHMMAX, in pages; this larger value suffices in all cases.
Important:
Enough swap space must be created to support the memory allocated, unless the operating system documentation explicitly states otherwise. On certain operating systems (Solaris, for example) Caché creates locked shared memory segments, which are not pageable but still may need swap space.
Adjust Maximum File Size
The hard limit for the maximum file size (RLIMIT_FSIZE) on any system running Caché must be unlimited. Set the value to unlimited on the operating system before installing. Make sure that the limit is set to unlimited for both the root user and the user who will run Caché.  Caché also sets the process soft limit to RLIMIT_FSIZE in its daemons to prevent I/O errors.
Important:
Caché will not install or start up if RLIMIT_FSIZE is not set to unlimited.
See the operating system documentation for your platform for instructions on how to set the system hard limit for the maximum file size, RLIMIT_FSIZE.
Platform Configuration Issues
The following sections contain configuration issues for individual UNIX®/Linux platforms. For more information, consult the system documentation for your platform.
HP-UX Platform Notes
This topic includes the information on the following adjustments:
Use the HP System V IPC Shared-Memory Subsystem to update parameters. See the HP System V Inter-Process Communication Mechanisms online documentation page for additional information. To change a value, perform the following steps:
  1. Enter the /usr/sbin/sam command to start the System Administration Manager (SAM) program.
  2. Double-click the Kernel Configuration icon.
  3. Double-click the Configurable Parameters icon.
  4. Double-click the parameter you want to change and enter the new value in the Formula/Value field.
  5. Click OK.
  6. Repeat these steps for all of the kernel configuration parameters that you want to change.
  7. When you are finished setting all of the kernel configuration parameters, select Process New Kernel from the Action menu.
The HP-UX operating system automatically reboots after you change the values for the kernel configuration parameters.
HP-UX Release 11i Parameters
HP-UX release 11i does not implement the CDLIMIT and NOFILES parameters. However, you can tune the values of the ulimit and maxfiles parameters instead.
Important:
If you tune maxfiles and maxfiles_lim, ensure that the values you set reflect the actual needs of your Caché system. Caché closes all possible open file descriptors when starting a new process via the Job command; setting a high value for these parameters may cause unnecessary close operations which can impact job start performance.
HP-UX Key Kernel Tunable Parameters
HP recommends that only parameters that are different from the OS default value be included in the /stand/system file. HP further recommends that changes to the /stand/system file be made only using the kctune command and that parameters not explicitly mentioned here be explicitly set to default values, as follows:
When the version of the release to which a parameter applies is not explicitly mentioned, the recommendation is valid for all versions of HP-UX 11i v2 and 11i v3.
HP-UX Key Kernel Tunable Parameters
Parameter Value Notes
dbc_max_pct 1 1–2GB Percentage of physical memory
dbc_min_pct 1 <dbc_max_pct> Same as dbc_max_pct
fcache_fb_policy 2 1 Flush behind policy
hires_timeout_enable 1 Requires hi-res timer patches
lcpu_attr 0 Disable hyperthreading
nkthread 30100 Threads allowed to run simultaneously (nproc+100)
nproc 30000
30000=maximum for 11i v2
60000=maximum for 11i v3
maxuprc 28000 Maximum procs/user
nclist 8292 Number of cblocks for pty and tty – min 8292
nfile 3 5600066 System-wide open files
nstrpty 600 Maximum streams-based ptys
nstrtel 600 Maximum telnet device files
o_sync_is_o_dsync 3 1 Enables translation of O_SYNC to O_DSYNC
process_id_max 2 31000 Maximum PID number
scsi_max_qdepth 3    
Set appropriate value for storage subsystem:
  • For EVA, <= 2048/[max luns-per-controllerport].
  • For XP, <= (1024, 2048, or 4096)/[max luns-per-controller-port], depending on model.
semmsl 128 Semaphores/ID
semmni 512 System-wide semaphore IDs: set to 50 non-Cache semaphore Ids
semmns (semmsl * semmni) 65536 System wide semaphores > Caché # processes
shmmax ½ phys mem Shared memory
swchunk 16384 Swap chunk size
vps_ceiling 512 Maximum system-selectable page size
Table Notes
The following notes apply to the “HP-UX Key Kernel Tunable Parameters” table:
1 This parameter is for 11i v2 only.
2 This parameter is for 11i v3 only.
3 This parameter is obsolete in 11i v3.
HP-UX Network Parameters
The following parameters should be inserted in /etc/rc.config.d/nddconf to support gigabit ethernet:
TRANSPORT_NAME[0]=tcp
NDD_NAME[0]=tcp_recv_hiwater_def
NDD_VALUE[0]=262144

TRANSPORT_NAME[1]=tcp
NDD_NAME[1]=tcp_xmit_hiwater_def
NDD_VALUE[1]=262144

TRANSPORT_NAME[2]=tcp
NDD_NAME[2]=tcp_recv_hiwater_lfp
NDD_VALUE[2]=262144

TRANSPORT_NAME[3]=tcp
NDD_NAME[3]=tcp_
xmit_hiwater_lfp
NDD_VALUE[3]=262144

TRANSPORT_NAME[4]=sockets
NDD_NAME[4]=socket_
udp_rcvbuf_default
NDD_VALUE[4]=262144

TRANSPORT_NAME[5]=sockets
NDD_NAME[5]=socket_
udp_sndbuf_default
NDD_VALUE[5]=65535
Hyper-threading (HT) Technology
InterSystems recommends enabling hyper-threading on all Poulson (Itanium 9500 series)-based or Intel Xeon-based processors. InterSystems recommends disabling hyper-threading for older Itanium processors. Please consult the InterSystems Worldwide Response Center (WRC) if you have questions about your specific server and platform.
IBM AIX® Platform Notes
The default settings of several IBM AIX® parameters can adversely affect performance. The settings and recommendations are detailed for the following:
I/O Pacing Parameters
IBM AIX® implements an I/O pacing algorithm that may hinder Caché write daemons. In IBM AIX® 5.2 and IBM AIX® 5.3, I/O pacing is automatically enabled when using HACMP clustering; beginning in IBM AIX® 6.1, however, I/O pacing is enabled on all systems and the default high-water mark is set higher than in earlier releases.
If write daemons are slowing or stalling, you may have to adjust the high-water mark; for information, see the “Using Disk-I/O Pacing” section of the IBM AIX® Performance Management Guide at the following IBM web page:http://publib.boulder.ibm.com/infocenter/systems/scope/aix/topic/com.ibm.aix.prftungd/doc/prftungd/disk_io_pacing.htm.
Important:
Beginning in IBM AIX® 6.1, you should not have to make any high-water mark adjustments.
If you have questions about the impact to your system, however, contact the InterSystems Worldwide Response Center (WRC) or your IBM AIX® supplier before making any changes. These recommendations are independent of Caché versions and apply to both JFS and Enhanced JFS (JFS2) file systems.
File System Mount Option
Although different mount options may improve performance for some workloads, InterSystems recommends the concurrent I/O (cio) mount option for file systems that contain only CACHE.DAT files.
Note:
Non-Caché workloads that benefit from file system caching (for example, operating system-level backups and/or file copies) are slowed by the cio mount option.
For JFS2 file systems that contain only journal files, cio is strongly recommended. For information, see UNIX® File System Recommendations in the “Journaling” chapter of the Caché Data Integrity Guide.
To improve recovery speed using the CACHE.WIJ file after a hard shutdown or system crash, InterSystems recommends a mount option that includes file system buffering (for example, rw) for the file system that contains the CACHE.WIJ file.
For information about mount options, see the IBM AIX® Commands Reference at the following IBM web page: http://publib.boulder.ibm.com/infocenter/systems/scope/aix/topic/com.ibm.aix.cmds/doc/aixcmds3/mount.htm.
Memory Management Parameters
The number of file systems and the amount of activity on them can limit the number of memory structures available to JFS or JFS2, and delay I/O operations waiting for those memory structures.
To monitor these metrics, issue a vmstat -vs command, wait two minutes, and issue another vmstat -vs command. The output looks similar to the following:
# vmstat -vs
              1310720 memory pages
              1217707 lruable pages
               144217 free pages
                    1 memory pools
               106158 pinned pages
                 80.0 maxpin percentage
                 20.0 minperm percentage
                 80.0 maxperm percentage
                 62.8 numperm percentage
               764830 file pages
                  0.0 compressed percentage
                    0 compressed pages
                 32.1 numclient percentage
                 80.0 maxclient percentage
               392036 client pages
                    0 remote pageouts scheduled
                    0 pending disk I/Os blocked with no pbuf
                 5060 paging space I/Os blocked with no psbuf
              5512714 filesystem I/Os blocked with no fsbuf
               194775 client filesystem I/Os blocked with no fsbuf
                    0 external pager filesystem I/Os blocked with no fsbuf
If you see an increase in the following parameters, increase the values for better Caché performance:
When increasing these parameters from the default values:
  1. Increase the current value by 50%.
  2. Check the vmstat output.
  3. Run vmstat twice, two minutes apart.
  4. If the field is still increasing, increase again by the same amount; continue this step until the field stops increasing between vmstat reports.
Important:
Change both the current and the reboot values, and check the vmstat output regularly because I/O patterns may change over time (hours, days, or weeks).
See the following IBM web pages for more detailed information:
IBM AIX® Tunable Parameters
None of the following listed parameters requires tuning because each is dynamically adjusted as needed by the kernel. See the appropriate IBM AIX® operating system documentation for more information.
The following table lists the tunable parameters for the IBM pSeries AIX® 5.2 operating system.
IBM AIX® Interprocess Communication Tunable Parameters
Parameter Purpose Dynamic Values
msgmax Specifies maximum message size. Maximum value of 4 MB
msgmnb Specifies maximum number of bytes on queue. Maximum value of 4 MB
msgmni Specifies maximum number of message queue IDs. Maximum value of 4096
msgmnm Specifies maximum number of messages per queue. Maximum value of 524288
semaem Specifies maximum value for adjustment on exit. Maximum value of 16384
semmni Specifies maximum number of semaphore IDs. Maximum value of 4096
semmsl Specifies maximum number of semaphores per ID. Maximum value of 65535
semopm Specifies maximum number of operations per semop() call. Maximum value of 1024
semume Specifies maximum number of undo entries per process. Maximum value of 1024
semvmx Specifies maximum value of a semaphore. Maximum value of 32767
shmmax Specifies maximum shared memory segment size. Maximum value of 256 MB for 32-bit processes and 0x80000000u for 64-bit
shmmin Specifies minimum shared-memory-segment size. Minimum value of 1
shmmni Specifies maximum number of shared memory IDs. Maximum value of 4096
maxuproc
maxuproc, which specifies the maximum number of processes than can be started by a single non-root user, is a tunable parameter that can be adjusted as described in this subsection.
If this parameter is set too low then various components of the operating system can fail as more and more users attempt to start processes; these failures include loss of CSP pages, background tasks failing, etc. Therefore, you should set the maxuproc parameter to be higher than the maximum number of processes that might be started by a non-root user (including interactive users, web server processes, and anything that might start a process).
Note:
Do not set the value excessively high because this value protects a server from a runaway application that is creating new processes unnecessarily; however, setting it too low causes unexplained problems.
Intersystems suggests that you set maxuproc to be double your expected maximum process count which gives a margin of error but still provides protection from runaway processes. For example, if your system has 1000 interactive users and often runs 500 background processes, then a value of at least 3000 would be a good choice.
The maxuproc value can be examined and changed either from the command line or from the smit/smitty administrator utilities, both as root user, as follows:
If you increase the value of maxuproc, the change is effective immediately. If you decrease the value of maxuproc, the change does not take effect until the next system reboot. In both cases the change persists over system reboots.
Red Hat Linux Platform Notes
This topic includes the information on the following adjustments:
Shared Memory Limit
The default shared memory limit (shmmax) on Linux platforms is 32 MB. This value is too small for Caché, but it can be changed in the proc file system without a restart.
For example, to allow 128 MB, type the following command:
$ echo 134217728 >/proc/sys/kernel/shmmax
You can put this command into a startup script.
Alternatively, you can use sysctl(8), if available, to control this parameter. Look for a file called /etc/sysctl.conf and add a line similar to the following:
kernel.shmmax = 134217728
This file is usually processed at startup, but sysctl can also be called explicitly later.
Important:
The msgmni parameter may also be set too low if you are running more than one instance of Caché on a machine. As stated in the Tunable UNIX® Parameters table, set this value to three times the number of instances of Caché that run simultaneously on your system.
Other parameters are sufficiently sized for a Caché application. To view the values of other parameters, look in the files /usr/src/linux/include/asm-xxx/shmparam.h and /usr/src/linux/include/linux/sem.h.
For more information, reference The proc File System chapter of the Red Hat Enterprise Linux 4: Reference Guide.
Locked-in Memory
On Linux platforms, if you configure Caché to lock the shared memory segment in memory to prevent paging, you must increase the maximum size that may be locked into memory (the memlock parameter). The default value is 32 KB. View the current value using the ulimit command.
For example, to display all current limits:
bash$ ulimit -a 
core file size (blocks, -c) unlimited 
data seg size ( KBytes, -d) unlimited 
file size (blocks, -f) unlimited 
pending signals (-i) 1024 
max locked memory (KBytes, -l) 32 <---------- THIS ONE 
max memory size (KBytes, -m) unlimited 
open files (-n) 1024 
pipe size (512 bytes, -p) 8 
POSIX message queues (bytes, -q) 819200 
stack size ( KBytes, -s) 10240 
cpu time (seconds, -t) unlimited 
max user processes (-u) 49000 
virtual memory ( KBytes, -v) unlimited 
file locks (-x) unlimited 
To display only memlock, use the -l option:
bash$ ulimit -l 
32 
If you have privileges, you can alter the value directly using the ulimit command; however, it is better to update the memlock parameter in the /etc/security/limits.conf file. If the memlock limit is too low, Linux reports a ENOMEM - "Not enough memory" error, which does not make the cause obvious. The actual memory is allocated; it is the lock that fails.
For more information, see memlock in the Caché Parameter File Reference.
Note:
You can achieve the same effect by using Linux Huge Pages for Caché shared memory. See the section Support for Huge Memory Pages for Linux in this chapter for more information.
Adjustments for Large Number of Concurrent Processes
Make the following adjustments if you are running a system that requires a large number of processes or telnet logins.
  1. In the /etc/xinetd.d/telnet file, add the following line:
    instances = unlimited
    
  2. In the /etc/xinetd.conf file, add or change the instances setting to:
    instances = unlimited
    
  3. After you make these modifications, restart the xinetd services with:
    # service xinetd restart
  4. The default pty (pseudo terminal connection) limit is 4096. If this is not sufficient, add or change the maximum pty line in the /etc/sysctl.conf file. For example:
    kernel.pty.max=10000
Dirty Page Cleanup
On large memory systems (for example, 8GB or larger), when doing numerous flat-file writes (for example, Caché backups or file copies), you can improve performance by adjusting the following parameters, which are located in proc/sys/vm/:
You can set these variables by adding the following to your /etc/sysctl.conf file:
vm.dirty_background_ratio=5 
vm.dirty_ratio=10
These changes force the Linux pdflush daemon to write out dirty pages more often rather than queue large amounts of updates that can potentially flood the storage with a large burst of updates.”
Oracle Solaris Platform Notes
Depending on the size of the database cache your deployment requires, it may be necessary to increase shared memory kernel parameters. See the Solaris Tunable Parameters Reference Manual for specific information on Solaris tunable parameters.
The Solaris 10 release no longer uses the /etc/system mechanism to tune the IPC shared memory parameters. These allocations are now automatic or configured through the resource controls mechanism.
If you try to use /etc/system on Solaris 10, you may receive the following message:
* IPC Shared Memory
*
* The IPC Shared Memory module no longer has system-wide limits.
* Please see the "Solaris Tunable Parameters Reference Manual" for
* information on how the old limits map to resource controls and
* the prctl(1) and getrctl(2) manual pages for information on
* observing the new limits.
See “Chapter 6 Resource Controls (Overview)” of the System Administration Guide: Oracle Solaris Containers-Resource Management and Oracle Solaris Zones on the Oracle web site for detailed information on using the rctladm, prctl, and projects commands to set Solaris 10 parameters.
The following subsections summarize several important configuration and tuning guidelines for a reliable, performing deployment of Caché on the Solaris ZFS filesystem:
Note:
The recommended minimum Solaris version is Solaris 10 10/08, which contains several key patches related to ZFS.
General ZFS Settings
You can adjust general ZFS parameters in the /etc/system file as follows:
Note:
Updates to the /etc/system file take effect after the next reboot.
ZFS Pool Configuration and Settings
For detailed information about the relevant commands for creating and administering pools and filesystems, see the Solaris ZFS Administration Guide. In addition, you should do the following:
Miscellaneous Solaris Settings
If a driver encounters a request larger than the maximum size of physical I/O requests (maxphys), it breaks the request into maxphys-size chunks. This value does not need to be specified on Solaris SPARC implementations, but it should be explicitly configured on Solaris x64 systems.
To configure the maximum size of physical I/O requests, add the following line to the /etc/system file:
set maxphys=1048576
SUSE Linux Platform Notes
This topic includes the information on the following adjustments:
Shared Memory Limits
The default shared memory limits (shhmax and shmall) on SUSE Linux 32-bit platforms are too small for Caché, and can be changed in the proc file system without a restart.
Caché uses shared memory for database buffers, global buffers, routine buffers, as well as license use. If the machine is being used only for Caché, InterSystems recommends setting the shared memory to approximately half the total memory. For more information, see the subsections of Determine Memory and Disk Requirements in this appendix, and Determining License Capacity and Usage in the “Managing Caché Licensing” chapter of the Caché System Administration Guide.
Note:
The recommendations to change the shared memory limits do not apply to SUSE Linux 64-bit systems.
For example, to allow 512 MB, type the following commands:
#sets shmall and shmmax shared memory
echo 536870912 >/proc/sys/kernel/shmall     #Sets shmall to 512 MB
echo 536870912 >/proc/sys/kernel/shmmax     #Sets shmmax to 512 MB
You can put these commands into a script that is run at startup. The SUSE Linux product documentation recommends you put the commands in the /etc/init.d/boot.local script file.
You can change the settings for the system memory user limits by modifying a file called /etc/profile.local. Add lines similar to the following:
#sets user limits (ulimit) for system memory resources
ulimit -v 512000     #set virtual (swap) memory to 512 MB 
ulimit -m 512000     #set physical memory to 512 MB
In this same file, you can permanently change the values for the PATH and CLASSPATH parameters by adding lines similar to the following:
#sets env values PATH and CLASSPATH
export PATH=$PATH:/usr/cache/bin:/path/to/j2sdk/bin:/.
export CLASSPATH=
      $CLASSPATH:/cache/dev/java/lib/JDK16/cachedb.jar:/path/to/otherjar/file:/.
Important:
To avoid the risk of losing your changes during system upgrades, do not change the /etc/profile file.
Locked-in Memory
On Linux platforms, if you configure Caché to lock the shared memory segment in memory to prevent paging, you must increase the maximum size that may be locked into memory by updating the memlock parameter in the /etc/security/limits.conf file. See the Locked-in Memory section of the Red Hat Linux Platform Notes in this appendix for information about updating the memlock parameter.