Skip to main content

Preparing to Install InterSystems IRIS

InterSystems IRIS® data platform supports multiple methods of deployment, and runs on many different platforms.

The Installation Guide describes the process of installing InterSystems IRIS from an installation kit. InterSystems IRIS can also be deployed in containers, as described in Running InterSystems Products in ContainersOpens in a new window, and provides several methods for automated deployment, which are described in Deploying InterSystems IRIS in InterSystems IRIS Basics: Connecting an IDE. If you are using one of these methods, consult the documentation listed in the latter.

Before installing, confirm that InterSystems IRIS supports your server or development platform. Review the InterSystems IRIS Data Platform Supported Platforms document for the current release, which describes what technologies are supported for each release.

Installation Planning Considerations

There are a few different ways to configure a new installation of InterSystems IRIS. When the installation runs, it prompts for the following information to decide which configuration to install:

The following section can be performed before or after installing InterSystems IRIS:

Installation Directory

The installation directory is the directory in which an InterSystems IRIS instance is installed. Throughout the documentation, this is referred to as install-dir. Once InterSystems IRIS is installed, it is impossible to change the installation directory.

There are several restrictions to what you can specify as the install-dir. The directory must be a fully resolved physical path, containing no symbolic links. The name of the directory can only use characters in the US ASCII character set, and cannot contain a caret (^). Also, InterSystems IRIS cannot be installed into a directory that is:

  • a UNC (non-local) path.

  • at the root level of a drive (such as C:\).

  • anywhere under the \Program Files directory.

If unspecified during installation, install-dir uses a default value. This default varies by platform, installation type, and user choice, as shown in the following table:

Platform
Installation Type
Default Directory
Windows
attended C:\InterSystems\Iris (or IrisN when multiple instances exist), unless installing user specifies otherwise.
unattended C:\InterSystems\Iris (or IrisN when multiple instances exist), unless INSTALLDIR property specifies otherwise.
UNIX®, Linux, macOS
attended No default; installing user must specify.
Do not choose the /home directory, any of its subdirectories, or the /usr/local/etc/irissys directory.
unattended No default; ISC_PACKAGE_INSTALLDIR parameter required.

Setup Type

During installation, you can choose which components of InterSystems IRIS you would like to install. The options are:

  • Development — Includes the InterSystems IRIS Database Engine (User Database, SQL Gateway, Server Monitoring Tools), Studio, all supported language bindings, and xDBC (ODBC and JDBC) drivers. Select this option if you plan to use this instance to perform both client and server tasks.

  • Server — Includes InterSystems IRIS Database Engine (User database, SQL Gateway, Server monitoring tools) and Web Gateway. Select this option if you plan to use this instance as an InterSystems IRIS database server which can be accessed by InterSystems IRIS clients.

  • Custom — Allows user to specify specific components to install or uninstall. Select this option if you to install or remove specific InterSystems IRIS components.

On Windows, you can choose from these two additional setup types:

  • Client — Includes Studio, and the xDBC (ODBC and JDBC) drivers. Select this option if you plan to use this instance as a client to an InterSystems IRIS database server on this or another computer.

  • Web Server — Includes Web Gateway (IIS, Apache 2.0, Apache 2.2). Select this option if you want to install only those parts of InterSystems IRIS that are required for the Web Gateway.

The following table identifies which components are installed for each setup type. When performing a custom installation, you may include only individual components from a group.

Components Installed by Setup Type
Component Group Components Development Server Client Web
InterSystems IRIS Database Engine (InterSystems IRIS Server)
Server Monitoring Tools
User database
SQL Gateway
Agent Service (ISCAgent)
Apache FOP (Formatting Objects Processor)
InterSystems IntegratedML*
X*
X*
   
InterSystems IRIS launcher (on Windows Only)  
X
X
X
 
Studio (on Windows only)  
X
 
X
 
xDBC
ODBC Driver
Java Database Connectivity
X
 
X
 
InterSystems IRIS Application Development
Java Binding for InterSystems IRIS
InterSystems IRIS shared library support
.NET Binding for InterSystems IRIS
Threaded Server Libraries
Other Samples
X
     
Documentation
PDF Documentation
Online Documentation
X
X
   
Web Gateway
CSP for IIS
CSP for Apache 2.0.x
CSP for Apache 2.2.x
 
X
 
X

* By default, IntegratedML is not included. To install IntegratedML, you must select during a custom installation.

Preexisting CSP Gateway

If the CSP Gateway is already installed on your system when you install the Web Gateway, the installer automatically upgrades the CSP Gateway to the Web Gateway. However, the installer creates a new copy of the CSP.ini file. To preserve the CSP.ini file from the CSP Gateway, perform the following steps:

  1. Create a backup of the CSP.ini file, located in install-dir/CSP/bin.

  2. Execute the Web Gateway installer.

  3. Restore the backup of CSP.ini.

Character Width Setting

You must choose between 8-bit or Unicode (16-bit) character width for your installation. An 8-bit instance cannot process data in 16-bit format, while a Unicode instance can process both 8-bit and 16-bit data.

If you need your instance to store and process data in a language that uses only Unicode character sets, such as Japanese, you must select Unicode. If the base character set for the locale your instance uses is based on the Latin-1 character set, ISO8859-1, you can select 8-bit, but bear in mind:

  • If you select 8-bit, the data stored by your instance is portable only to 8-bit locales based on the same character set.

  • If you select Unicode, the data stored by the instance is not portable to an 8-bit instance.

Port Numbers

A standard installation sets the following port numbers for an InterSystems IRIS instance:

  • Superserver port number1972. If taken, then 51773 or the first available subsequent number.

  • Web server port number52773 or the first available subsequent number

You can assign different port numbers during a new installation by performing a Custom installation (see Setup Type). You cannot enter a port number greater than 65535.

For information about setting the port numbers on an already-installed instance of InterSystems IRIS, see Set Port Numbers in the “Using Multiple Instances of InterSystems IRIS” chapter of the System Administration Guide.

Security Setting

An InterSystems IRIS installation has three initial security configurations: Minimal, Normal, or Locked Down. Minimal is only available for InterSystems IRIS installations. If you select Normal or Locked Down, the installer prompts for additional information, such as a password and account information.

For more information, see Initial InterSystems Security Settings.

Install the VS Code - ObjectScript development

Visual Studio Code (VS Code) is a free source code editor made by Microsoft for Windows, Linux and macOS. The InterSystems ObjectScript extension for Visual Studio Code enables you to use VS Code to connect to an InterSystems IRIS server and develop code in ObjectScript. An alternative to VS Code - ObjectScript is Studio, which installs alongside InterSystems IRIS on Windows-based operating systems.

For information on downloading and using VS Code - ObjectScript, see https://intersystems-community.github.io/vscode-objectscript/Opens in a new window.

Memory Planning and Management

Memory management is a very important factor in maximizing the performances of an InterSystems IRIS deployment. Read the following planning considerations:

Minimum Disk Space Requirements

The installation kit must be available, either on your computer or on a network, to perform an installation. InterSystems recommends you have at least 1.5 GB of disk space available before installing InterSystems IRIS. The installation procedure confirms that sufficient disk space is available before installing.

Configuring Huge and Large Pages

Where supported, using huge or large memory pages can be of significant performance benefit and is highly recommended.

Configuring Huge Pages on Linux

On Linux platforms with sufficient huge pages available, the InterSystems IRIS shared memory segment is allocated from the huge page pool. Using huge pages saves memory by saving space in page tables, and allows InterSystems IRIS to lock in its shared memory segment so its pages are not be paged out. InterSystems recommends the use of huge pages on systems hosting InterSystems IRIS under most circumstances.

The default memory page size on Linux systems is 4 KB. Most current Linux distributions include an option for huge pages, a memory page size of 2 MB or 1 GB depending on system configuration. When huge pages are configured, InterSystems IRIS automatically uses them in memory allocation.

Important:

With the 2.6.38 kernel, some Linux distributions have introduced transparent huge pages (THP) to automate the creation, management, and use of huge pages. However, THP does not handle the shared memory segments that make up the majority of the InterSystems IRIS memory allocated, and can cause memory allocation delays at runtime that may affect performance, especially for applications that have a high rate of job or process creation. For these reasons, InterSystems recommends that THP be disabled on all systems hosting InterSystems IRIS. For more detailed information on this topic, see Linux Transparent huge pages and the impact to InterSystems IRISOpens in a new window on InterSystems Developer Community.

To configure huge pages on Linux:

  1. Check the status.

    /proc/meminfo contains huge pages information. By default, no huge pages are allocated. Default Huge Page size is 2 MB. For example:

    HugePages_Total:     0
    HugePages_Free:      0
    HugePages_Rsvd:      0
    Hugepagesize:     2048 KB
    Copy code to clipboard
  2. Change the number of huge pages.

    You can change the system parameter directly: For example, to allocate 2056 huge pages, execute:

    # echo 2056 > /proc/sys/vm/nr_hugepages
    Copy code to clipboard
    Note:

    Alternatively, you can use sysctl(8) to change it:

    # sysctl -w vm.nr_hugepages=2056  
    Copy code to clipboard

    Huge pages must be allocated contiguously, which may require a reboot. Therefore, to guarantee the allocation, as well as to make the change permanent, do the following:

    1. Enter a line in /etc/sysctl.conf file:

      echo "vm.nr_hugepages=2056" >> /etc/sysctl.conf  
      Copy code to clipboard
    2. Reboot the system.

    3. Verify meminfo after reboot; for example:

      [root woodcrest grub]# tail -4 /proc/meminfo
      HugePages_Total:  2056
      HugePages_Free:   2056
      HugePages_Rsvd:      0
      Hugepagesize:     2048 KB
      Copy code to clipboard
  3. Verify the use of huge pages by InterSystems IRIS.

    When InterSystems IRIS is started, it reports how much shared memory was allocated; for example, a message similar to the following is displayed (and included in the messages.log file):

    Allocated 3580MB shared memory: 3000MB global buffers, 226MB routine buffers
    Copy code to clipboard

    The amount of memory available in huge pages should be greater than the total amount of shared memory to be allocated; if it is not greater, huge pages are not used.

    Note:

    It is not advisable to specify HugePages_Total much higher than the shared memory amount because the unused memory are not be available to other components.

Configuring Large Pages on IBM AIX®

AIX supports multiple page sizes: 4 KB, 64 KB, 16 MB, and 16 GB. Use of 4 KB and 64 KB pages is transparent to InterSystems IRIS. In order for InterSystems IRIS to use 16 MB large pages, you must configure them within AIX. AIX does not automatically change the number of configured large or huge pages based on demand. Currently, InterSystems IRIS does not use 16 GB huge pages.

Large pages should be configured only in high-performance environments because memory allocated to large pages can be used only for large pages.

By default, when large pages are configured, the system automatically uses them in memory allocation. If shared memory cannot be allocated in large pages then it is allocated in standard (small) pages. For finer grain control over large pages, see memlock in the Configuration Parameter File Reference.

Configure large pages on AIX using the vmo command as follows:

vmo -r -o lgpg_regions=<LargePages> -o lgpg_size=<LargePageSize>

where <LargePages> specifies the number of large pages to reserve, and <LargePageSize> specifies the size, in bytes, of the hardware-supported large pages.

Note:

On systems that support dynamic Logical PARtitioning (LPAR), you can omit the -r option to dynamically configure large pages without a system reboot.

For example, the following command configures 1 GB of large pages:

# vmo -r -o lgpg_regions=64 -o lgpg_size=16777216
Copy code to clipboard

Once you have configured large pages, run the bosboot command to save the configuration in the boot image. After the system comes up, enable it for pinned memory using the following vmo command:

vmo -o v_pinshm=1
Copy code to clipboard

However, if memlock=64, vmo -o v_pinshm=1 is not required. For more information on memlock, see memlock in the Configuration Parameter File Reference.

Configuring Large Pages on Windows

InterSystems recommends enabling InterSystems IRIS to allocate its memory as large pages on Windows systems, which provides more efficient use of memory and paging space. To do so, grant the Windows “Lock Pages in Memory” (SELockMemory) privilege to the Windows user account used to run InterSystems IRIS, which allows InterSystems IRIS to request its memory as large pages. For finer grain control over memory allocation at startup, see memlock in the Configuration Parameter File Reference.

Note:

If InterSystems IRIS is running under the default SYSTEM account, it has the “Lock Pages in Memory” privilege by default.

When you restart InterSystems IRIS while using large pages, you typically also need to restart Windows to guarantee that all configured memory is allocated. If startup is unable to allocate the full amount of configured memory, it attempts to start up with less memory and may or may not use large pages. You can check the actual memory allocated by reviewing the most recent InterSystems IRIS startup in the messages log.

Additional Memory Resources

Correctly sizing system memory is critical for maximizing the performance and availability of InterSystems IRIS and your application. InterSystems IRIS memory usage is discussed in the Memory Overview section of the “Vertical Scaling” chapter in Scalability Guide, and the Calculating Memory Requirements and Allocation section provides guidelines for calculating initial memory requirements so you can determine whether your system has sufficient memory.

After installation, monitoring how InterSystems IRIS allocates memory is an important performance factor. If InterSystems IRIS is not allocating memory efficiently, you can adjust memory settings for processes, routine and database caches, and the generic heap. For information about how to perform these memory-related tasks, see the Memory and Startup Settings section of the “Configuring InterSystems IRIS” chapter in System Administration Guide.

File System Recommendations

This section contains recommendations for setting up file systems for InterSystems IRIS. Review the subsections that apply to your platform:

For suggestions about which supported file system to use for each platform, see Supported File Systems in the online InterSystems IRIS Data Platform Supported Platforms document for this release.

General File System Recommendations

In the interests of performance and recoverability, InterSystems recommends a minimum of four separate file systems for InterSystems IRIS to host:

  1. Installation files, executables, and system databases (including, by default, the write image journal, or WIJ, file)

  2. Database files (and optionally the WIJ)

  3. Primary journal directory

  4. Alternate journal directory

In addition, you can add another separate file system to the configuration for the WIJ file which, by default, is created in the install-dir\mgr directory. Ensure that such a file system has enough space to allow the WIJ to grow to its maximum size. For more information on the WIJ, see the “Write Image Journaling and Recovery” chapter in Data Integrity Guide.

Note:

On UNIX®, Linux, and macOS platforms, /usr/local/etc/irissys is the InterSystems IRIS registry directory and therefore must be on a local filesystem.

In the event of a catastrophic disk failure that damages database files, the journal files are a key element in recovering from backup. Therefore, you should place the primary and alternate journal directories on storage devices that are separate from the devices used by database files and the WIJ. (Journals should be separated from the WIJ because damage to the WIJ could compromise database integrity.) Since the alternate journal device allows journaling to continue after an error on the primary journal device, the primary and alternate journal directories should also be on devices separate from each other. For practical reasons, these different devices may be different logical units (LUNs) on the same storage array; the general rule is the more separation the better, with separate sets of physical drives highly recommended. See Journaling Best Practices in the “Journaling” chapter of the InterSystems IRIS Data Integrity Guide for more information about separate journal storage.

The journal directories and the WIJ directory are not configured during installation. For information on changing them after you install InterSystems IRIS, see Configuring Journal Settings in the InterSystems IRIS Data Integrity Guide.

InterSystems does not support the use of symbolic links for database directories.

Note:

Current storage arrays, especially SSD/Flash-based arrays, do not always allow for the type of segregation recommended in the preceding. When using such a technology, consult and follow the storage vendor’s recommendations for performance and resiliency.

Buffered I/O vs. Direct I/O

In general, most operating systems offer two distinct I/O strategies:

  • Buffered I/O — The operating system caches reads and writes. This is the normal mode of operation, unless a program (such as InterSystems IRIS) or user specifies otherwise.

  • Direct I/O, Concurrent I/O, or Unbuffered I/O — Reads and writes bypass the operating system cache, providing better performance and scalability characteristics for some InterSystems IRIS files.

InterSystems IRIS takes advantage of Direct, Concurrent or Unbuffered I/O if available. The choice of file system and mount options may require additional attention depending on the type of files, as follows:

Journal Files

Some platforms have specific recommendations to use direct or concurrent I/O mount options for optimal performance, as documented in Supported File Systems in the online InterSystems IRIS Data Platform Supported Platforms document for this release. On other platforms, InterSystems IRIS uses direct I/O automatically for journal files as appropriate and no special consideration is required.

Databases (IRIS.DAT files)

InterSystems IRIS uses its own database cache, so buffering at the operating system level is not advantageous for database files. Be sure to allocate sufficient space for the database cache, and also review the details of the optimal use of direct I/O for your platform.

  • Linux — InterSystems IRIS uses direct I/O for database files.

    If using the VxFS file system, this can be overridden by mounting the file system for concurrent I/O with the cio mount option.

  • macOS — InterSystems IRIS uses buffered I/O for database files.

  • Windows — InterSystems IRIS uses unbuffered I/O for database files.

  • IBM AIX — InterSystems IRIS uses concurrent I/O for database files, regardless of whether the cio file system mount option is used.

    Note:

    On AIX, in unusual configurations in which an external command is used to read a database file while InterSystems IRIS is running, the external command may fail because the file is opened for concurrent I/O by InterSystems IRIS. An example is performing an external backup using the cp command instead of a more sophisticated backup or snapshot utility. Mounting the file system with the cio option resolves this by forcing all programs to open files with concurrent I/O.

Installation files and executables

InterSystems IRIS uses buffered I/O for these files. On platforms that support the direct I/O mount option, InterSystems still recommends using buffered I/O for these files.

External application files and streams

Applications that use external files typically benefit from those files being located on a buffered file system.

noatime Mount Option

Generally, it is advisable to disable updates to the file access time when this option is available. This can typically be done using the noatime mount option on various file systems.

Storage Recommendations

Many storage technologies are available today, from traditional magnetic spinning HDD devices to SSD and PCIe Flash based devices. In addition, multiple storage access technologies include NAS, SAN, FCoE, direct-attached, PCIe, and virtual storage with hyper-converged infrastructure.

The storage technology that is best for your application depends on application access patterns. For example, for applications that predominantly involve random reads, SSD or Flash based storage would be an ideal solution, and for applications that are mostly write intensive, traditional HDD devices might be the best approach.

The sections that follow provide guidelines as general suggestions. Specific storage product providers may specify separate and even contradictory best practices that should be consulted and followed accordingly. This section discusses:

Storage Connectivity

This section contains considerations about storage area network (SAN) and network-attached storage (NAS).

SAN Fibre Channel

Use multiple paths from each host to the SAN switches or storage controllers. The level of protection increases with multiple HBAs to protect from a single card failure, however a minimum recommendation is to use at least a dual-port HBA.

To provide resiliency at the storage array layer, an array with dual controllers in either an active-active or active-passive configuration is recommended to protect from a storage controller failure, and to provide continued access even during maintenance periods for activities such as firmware updates.

If using multiple SAN switches for redundancy, a good general practice is to make each switch a separate SAN fabric to keep errant configuration changes on a single switch from impacting both switches and impeding all storage access.

Network Attached Storage (NAS)

With 10Gb Ethernet commonly available, for best performance 10Gb switches and host network interface cards (NICs) are recommended.

Having dedicated infrastructure is also advised to isolate traffic from normal network traffic on the LAN. This helps ensure predictable NAS performance between the hosts and the storage.

Jumbo frame support should be included to provide efficient communication between the hosts and storage.

Many network interface cards (NICs) provide TCP Offload Engine (TOE) support. TOE support is not universally considered advantageous. The overhead and gains greatly depend on the server’s CPU for available cycles (or lack thereof). Additionally, TOE support has a limited useful lifetime because system processing power rapidly catches up to the TOE performance level of a given NIC, or in many cases exceeds it.

Storage Configuration

The storage array landscape is ever-changing in technology features, functionality, and performance options, and multiple options provide optimal performance and resiliency for InterSystems IRIS. This section provides general best practice guidelines for optimal InterSystems IRIS performance and data resiliency.

In the past, RAID10 was recommended for maximum protection and performance. However, storage controller capacities, RAID types and algorithm efficiencies, and controller features such as inline compression and deduplication provide more options than ever before. Your application’s I/O patterns should help you decide with your storage vendor which storage RAID levels and configuration provide the best solution.

Where possible, it is best to use block sizes similar to that of the file type. While most storage arrays have a lower limit on the block size that can be used for a given volume, you can approach the file type block size as closely as possible; for example, a 32KB or 64KB block size on the storage array is usually a viable option to effectively support IRIS.DAT files with 8KB block format. The goal here is to avoid excessive/wasted I/O on the storage array based on your application’s needs.

The following table is provided as a general overview of storage I/O within an InterSystems IRIS installation.

I/O Type
When
How
Notes
Database reads, mostly random
Continuous by user processes
User process initiates disk I/O to read data
Database reads are performed by daemons serving web pages, SQL queries, or direct user processes
Database writes, ordered but non-contiguous
Approx. every 80 seconds or when pending updates reach threshold percentage of database cache, whichever comes first
Database write daemons
(8 processes)
Database writes are performed by a set of database system processes known as write daemons. User processes update the database cache and the trigger (time or database cache percent full) commits the updates to disk using the write daemons. Typically expect anywhere from a few MBs to several GBs that must be written during the write cycle depending on update rates.
WIJ writes, sequential
Approx. every 80 seconds or when pending updates reach threshold percentage of database cache, whichever comes first
Database master write daemon (1 process) The WIJ is used to protect physical database file integrity from system failure during a database write cycle. Writes are approximately 256KB each in size.
Journal writes, sequential
Every 64KB of journal data or 2 seconds, or sync requested by ECP or application
Database journal daemon (1 process)
Journal writes are sequential and variable in size from 4KB to 4MB. There can be as low as a few dozen writes per second to several thousand per second for very large deployments using ECP and separate application servers.

Bottlenecks in storage are one of the most common problems affecting database system performance. A common error is sizing storage for data capacity only, rather than allocating a high enough number of discrete disks to support expected Input/Output Operations Per Second (IOPS).

I/O Type
Average Response Time
Maximum Response Time
Notes
Database block size random read (non-cached)
<=6 ms
<=15 ms
Database blocks are a fixed 8KB, 16KB, 32KB, or 64KB—most reads to disk are not cached because of large database cache on the host.
Database block size random write (cached)
<=1 ms
<2 ms
All database file writes are expected to be cached by the storage controller cache memory.
4KB to 4MB journal write (without ECP)
<=2 ms
<=5 ms
Journal writes are sequential and variable in size from 4KB to 4MB. Write volume is relatively low when no ECP application servers are used.
4KB to 4MB journal write (with ECP)
<=1 ms
<=2 ms
Journal synchronization requests generated from ECP impose a stringent response time requirement to maintain scalability. The synchronization requests issue can trigger writes to the last block in the journal to ensure data durability.

Please note that these figures are provided as guidelines, and that any given application may have higher or lower tolerances and thresholds for ideal performance. These figures and I/O profiles are to be used as a starting point for your discussions with your storage vendor.

Platform-Specific Preparations

The following sections contain platform-specific considerations. Review the following sections that apply to your intended installation platform:

Supported Platforms and Components

For information about supported technologies for a release version of InterSystems IRIS, including supported operating systems and web servers, see the Supported Technologies chapter of the online InterSystems Supported Platforms document for the release.

To achieve optimal journal performance and ensure journal data integrity when there is a system crash, InterSystems recommends various file systems and mount options for journal files. For specific platform details see the UNIX® File System Recommendations section of the “Journaling” chapter of the InterSystems IRIS Data Integrity Guide.

Note:

With each instance, InterSystems IRIS installs a private web server and a private Web Gateway to serve CSP pages to ensure proper operation of the Management Portal. The private web server ensures that:

  • the Management Portal runs out of the box.

  • development environments are provided with an out-of-the-box testing capability.

The private web server is not supported for any other purpose. You should not use the private web server for deployments of http-based applications, including CSP and SOAP over http or https; instead, you must install and deploy one of the supported web servers.

The private web server configuration is preserved through upgrades.

On Windows, the private web server Windows service name is “Web Server for instance name”. InterSystems IRIS installs the web server into the install-dir\httpd directory. It is uninstalled when you uninstall the corresponding InterSystems IRIS instance.

Maximum User Process Recommendations

Ensure that the maximum user processes is set high enough to allow all InterSystems IRIS processes for a given user, as well as other default processes, to run on the system.

AIX® Platform Notes

The default settings of several AIX® parameters can adversely affect performance. The settings and recommendations are detailed for the following:

I/O Pacing Parameters

AIX® implements an I/O pacing algorithm that may hinder InterSystems IRIS write daemons. In AIX® 5.2 and AIX® 5.3, I/O pacing is automatically enabled when using HACMP clustering; beginning in AIX® 6.1, however, I/O pacing is enabled on all systems and the default high-water mark is set higher than in earlier releases.

If write daemons are slowing or stalling, you may have to adjust the high-water mark; for information, see the “Using Disk-I/O Pacing” section of the AIX® Performance Management Guide at the following IBM web page:http://publib.boulder.ibm.com/infocenter/systems/scope/aix/topic/com.ibm.aix.prftungd/doc/prftungd/disk_io_pacing.htmOpens in a new window.

Important:

Beginning in AIX® 6.1, you should not have to make any high-water mark adjustments.

If you have questions about the impact to your system, however, contact the InterSystems Worldwide Response Center (WRC)Opens in a new window or your AIX® supplier before making any changes. These recommendations apply to both JFS and Enhanced JFS (JFS2) file systems.

File System Mount Option

Different mount options may improve performance for some workloads.

Note:

Non-InterSystems IRIS workloads that benefit from file system caching (for example, operating system-level backups and/or file copies) are slowed by the cio mount option.

To improve recovery speed using the IRIS.WIJ file after a hard shutdown or system crash, InterSystems recommends a mount option that includes file system buffering (for example, rw) for the file system that contains the IRIS.WIJ file.

For information about mount options, see the AIX® Commands Reference at the following IBM web page: http://publib.boulder.ibm.com/infocenter/systems/scope/aix/topic/com.ibm.aix.cmds/doc/aixcmds3/mount.htmOpens in a new window.

Memory Management Parameters

The number of file systems and the amount of activity on them can limit the number of memory structures available to JFS or JFS2, and delay I/O operations waiting for those memory structures.

To monitor these metrics, issue a vmstat -vs command, wait two minutes, and issue another vmstat -vs command. The output looks similar to the following:

# vmstat -vs
              1310720 memory pages
              1217707 lruable pages
               144217 free pages
                    1 memory pools
               106158 pinned pages
                 80.0 maxpin percentage
                 20.0 minperm percentage
                 80.0 maxperm percentage
                 62.8 numperm percentage
               764830 file pages
                  0.0 compressed percentage
                    0 compressed pages
                 32.1 numclient percentage
                 80.0 maxclient percentage
               392036 client pages
                    0 remote pageouts scheduled
                    0 pending disk I/Os blocked with no pbuf
                 5060 paging space I/Os blocked with no psbuf
              5512714 filesystem I/Os blocked with no fsbuf
               194775 client filesystem I/Os blocked with no fsbuf
                    0 external pager filesystem I/Os blocked with no fsbuf
Copy code to clipboard

If you see an increase in the following parameters, increase the values for better InterSystems IRIS performance:

  • pending disk I/Os blocked with no pbuf

  • paging space I/Os blocked with no psbuf

  • filesystem I/Os blocked with no fsbuf

  • client filesystem I/Os blocked with no fsbuf

  • external pager filesystem I/Os blocked with no fsbuf

When increasing these parameters from the default values:

  1. Increase the current value by 50%.

  2. Check the vmstat output.

  3. Run vmstat twice, two minutes apart.

  4. If the field is still increasing, increase again by the same amount; continue this step until the field stops increasing between vmstat reports.

Important:

Change both the current and the reboot values, and check the vmstat output regularly because I/O patterns may change over time (hours, days, or weeks).

See the following IBM web pages for more detailed information:

AIX® Tunable Parameters

AIX® Interprocess Communication Tunable Parameters

The following table lists the tunable parameters for the IBM pSeries AIX® 5.2 operating system. None of the following listed parameters require tuning because each is dynamically adjusted as needed by the kernel. See the appropriate AIX® operating system documentationOpens in a new window for more information.

Parameter Purpose Dynamic Values
msgmax Specifies maximum message size. Maximum value of 4 MB
msgmnb Specifies maximum number of bytes on queue. Maximum value of 4 MB
msgmni Specifies maximum number of message queue IDs. Maximum value of 4096
msgmnm Specifies maximum number of messages per queue. Maximum value of 524288
semaem Specifies maximum value for adjustment on exit. Maximum value of 16384
semmni Specifies maximum number of semaphore IDs. Maximum value of 4096
semmsl Specifies maximum number of semaphores per ID. Maximum value of 65535
semopm Specifies maximum number of operations per semop() call. Maximum value of 1024
semume Specifies maximum number of undo entries per process. Maximum value of 1024
semvmx Specifies maximum value of a semaphore. Maximum value of 32767
shmmax Specifies maximum shared memory segment size. Maximum value of 256 MB for 32-bit processes and 0x80000000u for 64-bit
shmmin Specifies minimum shared-memory-segment size. Minimum value of 1
shmmni Specifies maximum number of shared memory IDs. Maximum value of 4096
maxuproc

maxuproc, which specifies the maximum number of processes than can be started by a single nonroot user, is a tunable parameter that can be adjusted as described in this subsection.

If this parameter is set too low then various components of the operating system can fail as more and more users attempt to start processes; these failures include loss of CSP pages, background tasks failing, etc. Therefore, you should set the maxuproc parameter to be higher than the maximum number of processes that might be started by a nonroot user (including interactive users, web server processes, and anything that might start a process).

Note:

Do not set the value excessively high because this value protects a server from a runaway application that is creating new processes unnecessarily; however, setting it too low causes unexplained problems.

Intersystems suggests that you set maxuproc to be double your expected maximum process count which gives a margin of error but still provides protection from runaway processes. For example, if your system has 1000 interactive users and often runs 500 background processes, then a value of at least 3000 would be a good choice.

The maxuproc value can be examined and changed either from the command line or from the smit/smitty administrator utilities, both as root user, as follows:

  • From the command line, view the current setting:

    # lsattr -E -l sys0 -a maxuproc
    Copy code to clipboard

    then modify the value:

    # chdev -l sys0 -a maxuproc=NNNNNN
    Copy code to clipboard

    where NNNNNN is the new value.

  • From the administrator utility smit (or smitty) choose System Environments > Change / Show Characteristics of Operating System > Maximum number of PROCESSES allowed per user.

If you increase the value of maxuproc, the change is effective immediately. If you decrease the value of maxuproc, the change does not take effect until the next system reboot. In both cases the change persists over system reboots.

Required C/C++ Runtime Libraries

You must ensure that the required C/C++ runtime is installed on your IBM AIX® system before installing InterSystems IRIS.

InterSystems IRIS for AIX is compiled using the IBM XL C/C++ for AIX 16.1.0 compiler. If the system on which you are installing InterSystems IRIS does not have the corresponding version of the runtime already installed, you must install it.

See https://www.ibm.com/support/home/Opens in a new window for more information.

Use of Raw Ethernet

In order to use Raw Ethernet, an IBM AIX® machine must have the DLPI (Data Link Provider Interface) packages installed. If the machine does not have the DLPI packages, obtain them from your IBM provider and create DLPI devices through the following procedure:

  1. Log in as root.

  2. In the PSE drivers section of the /etc/pse.conf file, uncomment the four lines that refer to the DLPI drivers.

  3. Save the file.

  4. Restart the computer.

If the DLPI devices are not installed, the EthernetAddress() method of the %SYSTEM.INetInfoOpens in a new window class returns a null string rather than information about the Ethernet device.

Red Hat Linux Platform Notes

This topic includes the information on the following adjustments:

Shared Memory Limit

The default shared memory limit (shmmax) on Linux platforms is 32 MB. This value is too small for InterSystems IRIS, but it can be changed in the proc file system without a restart. The new memory limit remains in effect until you restart the Red Hat Linux system.

For example, to allow 128 MB, type the following command:

$ echo 134217728 >/proc/sys/kernel/shmmax
Copy code to clipboard

You can put this command into a startup script.

Alternatively, you can use sysctl(8), if available, to set this parameter permanently. Add a line to the/etc/sysctl.conf similar to the following:

kernel.shmmax = 134217728
Copy code to clipboard

This file is usually processed at startup, but sysctl can also be called explicitly later.

Important:

The msgmni parameter may also be set too low if you are running more than one instance of InterSystems IRIS on a machine. Set this value to three times the number of instances of InterSystems IRIS that run simultaneously on your system.

Other parameters are sufficiently sized for an InterSystems IRIS application. To view the values of other parameters, look in the files /usr/src/linux/include/asm-xxx/shmparam.h and /usr/src/linux/include/linux/sem.h.

For more information, reference “The proc File SystemOpens in a new window” chapter of the Red Hat Enterprise Linux 4: Reference Guide.

Locked-in Memory

On Linux platforms, if shared memory is allocated in huge pages, the pages are automatically locked in memory and no further action is required. See the Configuring Huge Pages on Linux section in this chapter for information about allocating huge pages.

If not using huge pages, you can configure InterSystems IRIS to lock the shared memory segment in memory to prevent paging. This is described in the LockSharedMemory section of the “memlock” entry in the Configuration Parameter File Reference.

Otherwise, you must increase the maximum size that may be locked into memory. The default value is 32 KB. View the current value using the ulimit command.

For example, to display all current limits:

bash$ ulimit -a 
core file size (blocks, -c) unlimited 
data seg size ( KBytes, -d) unlimited 
file size (blocks, -f) unlimited 
pending signals (-i) 1024 
max locked memory (KBytes, -l) 32 <---------- THIS ONE 
max memory size (KBytes, -m) unlimited 
open files (-n) 1024 
pipe size (512 bytes, -p) 8 
POSIX message queues (bytes, -q) 819200 
stack size ( KBytes, -s) 10240 
cpu time (seconds, -t) unlimited 
max user processes (-u) 49000 
virtual memory ( KBytes, -v) unlimited 
file locks (-x) unlimited 
Copy code to clipboard

To display only max-locked memory, use the -l option:

bash$ ulimit -l 
32 
Copy code to clipboard

If you have privileges, you can alter the value directly using the ulimit command; however, it is better to update the memlock parameter in the /etc/security/limits.conf file. If the memlock limit is too low, Linux reports a ENOMEM - "Not enough memory" error, which does not make the cause obvious. The actual memory is allocated; it is the lock that fails.

For more information, see memlock in the Configuration Parameter File Reference.

Adjusting for Large Number of Concurrent Processes

Make the following adjustments if you are running a system that requires a large number of processes or telnet logins.

  1. In the /etc/xinetd.d/telnet file, add the following line:

    instances = unlimited
    
    Copy code to clipboard
  2. In the /etc/xinetd.conf file, add or change the instances setting to:

    instances = unlimited
    
    Copy code to clipboard
  3. After you make these modifications, restart the xinetd services with:

    # service xinetd restart
    Copy code to clipboard
  4. The default pty (pseudo terminal connection) limit is 4096. If this is not sufficient, add or change the maximum pty line in the /etc/sysctl.conf file. For example:

    kernel.pty.max=10000
    Copy code to clipboard

Dirty Page Cleanup

On large memory systems (for example, 8GB or larger), when doing numerous flat-file writes (for example, InterSystems IRIS backups or file copies), you can improve performance by adjusting the following parameters, which are located in proc/sys/vm/:

  • dirty_background_ratio — Maximum percentage of active that can be filled with dirty pages before pdflush begins to write them. InterSystems recommends setting this parameter to 5.

  • dirty_ratio — Maximum percentage of total memory that can be filled with dirty pages before processes are forced to write dirty buffers themselves during their time slice instead of being allowed to do more writes. InterSystems recommends setting this parameter to 10

You can set these variables by adding the following to your /etc/sysctl.conf file:

vm.dirty_background_ratio=5 
vm.dirty_ratio=10

These changes force the Linux pdflush daemon to write out dirty pages more often rather than queue large amounts of updates that can potentially flood the storage with a large burst of updates.”

Using Kerberos

To use Kerberos on the Red Hat Linux platform, you must install the krb5-devel package in addition to the krb5-libs package. Installing krb5-devel establishes the required symbolic links for using Kerberos. The package is required for production environments, not only development environments. See the Red Hat NetworkOpens in a new window web site for more information about these components.

SUSE Linux Platform Notes

This topic includes the information on the following adjustments:

Shared Memory Limits

The default shared memory limits (shhmax and shmall) on SUSE Linux 32-bit platforms are too small for InterSystems IRIS, and can be changed in the proc file system without a restart.

InterSystems IRIS uses shared memory for database buffers, global buffers, routine buffers, as well as license use. If the machine is being used only for InterSystems IRIS, InterSystems recommends setting the shared memory to approximately half the total memory. For more information, see the “Vertical Scaling” chapter of the Scalability Guide, as well as Memory and Startup Settings in the “Configuring InterSystems IRIS” chapter and Determining License Capacity and Usage in the “Managing InterSystems IRIS Licensing” chapter of the System Administration Guide.

Note:

The recommendations to change the shared memory limits do not apply to SUSE Linux 64-bit systems.

For example, to allow 512 MB, type the following commands:

#sets shmall and shmmax shared memory
echo 536870912 >/proc/sys/kernel/shmall     #Sets shmall to 512 MB
echo 536870912 >/proc/sys/kernel/shmmax     #Sets shmmax to 512 MB
Copy code to clipboard

You can put these commands into a script that is run at startup. The SUSE Linux product documentation recommends you put the commands in the /etc/init.d/boot.local script file.

You can change the settings for the system memory user limits by modifying a file called /etc/profile.local. Add lines similar to the following:

#sets user limits (ulimit) for system memory resources
ulimit -v 512000     #set virtual (swap) memory to 512 MB 
ulimit -m 512000     #set physical memory to 512 MB
Copy code to clipboard

In this same file, you can permanently change the values for the PATH and CLASSPATH parameters by adding lines similar to the following:

#sets env values PATH and CLASSPATH
export PATH=$PATH:/usr/iris/bin:/path/to/j2sdk/bin:/.
export CLASSPATH=
      $CLASSPATH:/iris/dev/java/lib/JDK18/intersystems-jdbc-3.0.0.jar.
Copy code to clipboard
Important:

To avoid the risk of losing your changes during system upgrades, do not change the /etc/profile file.

Locked-in Memory

On Linux platforms, if shared memory is allocated in huge pages, they are automatically locked in memory and no further action is required. See the Configuring Huge Pages on Linux section in this chapter for information about allocating huge pages.

If not using huge pages, you can configure InterSystems IRIS to lock the shared memory segment in memory to prevent paging. This is described in the LockSharedMemory section of the “memlock” entry in the Configuration Parameter File Reference.

Otherwise, you must increase the maximum size that may be locked into memory. See the Locked-in Memory section of the Red Hat Linux Platform Notes in this chapter for instructions.

Using Kerberos

To use Kerberos on the SUSE Linux platform, you must install the krb5-devel package in addition to the krb5-libs package. Installing krb5-devel establishes the required symbolic links for using Kerberos. The package is required for production environments, not only development environments. See the SUSE documentationOpens in a new window web site for more information about these components.

Ubuntu Platform Notes

This topic includes the information on the following adjustments:

Semaphore Deletion Setting

Under some circumstances, the OS may delete an instance’s semaphores when the instance owner connects to an Ubuntu host, for example using SSH. To prevent this, edit the /etc/systemd/logind.conf file and change RemoveIPC=yes (the default) to RemoveIPC=no.

Updating to a newer version of Ubuntu may revert RemoveIPC to the default value of yes. After updating Ubuntu, be sure to change RemoveIPC to avoid unwanted semaphore deletion.

FeedbackOpens in a new window