CSP Gateway Configuration Guide
Introduction to the CSP Gateway
The CSP Gateway provides the communications layer between the hosting web server and Caché when you call a Caché Server Page.
Who Should Read this Document
The Caché installation includes scripts that perform Web Server and CSP Gateway configuration for common Web Servers and operating systems.
In most cases, installing Caché according to the usual Caché instructions and installing a typical, supported web server provides a system that works with the CSP Gateway without the need to consult this document.
However, if you have an atypical web server architecture or you are an advanced user who wants to get the best out of your environment, you might want to use this document. This document describes the details of procedures for configuring a web server and the CSP Gateway to connect to Caché. It also describes how to use services that the CSP Gateway provides.
Documentation on Caché Server Pages can be found in the following places:
Conventions Used in this Document
When following steps in this document, change paths to match your own installation.
Lines terminated with a back-slash (\) are continued to the next line.
For example, enter the following line, as shown in this document:
Init fn=load-modules shlib=CSPn3.dll \
Init fn=load-modules shlib=CSPn3.dll funcs=csp_term
Gateway components and physical installation paths
Later sections in this guide describe how the Gateway components should be configured with all supported web servers. The installation paths for components should be regarded as examples rather than taken literally. Also, the InterSystems installers create and maintain separate Gateway installations for the internal (private) web server and any third-party web server that might be present on the same host. In this context ‘third-party web server’ refers to a web server that is not part of the software installed by InterSystems.
The precise installation location of Gateway components is not particularly critical provided:
The physical installation paths match those given in the hosting web server configuration where appropriate.
The security settings, in relation to required access for individual components, are adjusted appropriately. This is particularly important for Gateway components that are accessed directly by the web server since web servers are usually locked down to the extent that the files they are able to access (and executables that can be run) are carefully controlled. It should be borne in mind that security considerations will also be important for any Gateway configuration (and log) files that are accessed by Gateway binaries that are themselves bound to the web server core executable.
The security policy of the hosting web server is respected. Some web servers notably those shipped with Secure Linux (SELinux) and OpenVMS (HPSWS) are configured such that it is not possible for them to access files that lie outside their own file system. This restriction will clearly have an impact on where certain web-server-facing Gateway components can be installed.
There are four types of Gateway component to consider.
Binaries to be loaded by the web server (API based extensions).
This includes Windows DLLs, UNIX Shared Objects and OpenVMS Shareable Images:
The physical location where these are installed should match the corresponding configuration directives in the hosting web server configuration. This includes directives indicating which third-party modules should be loaded. The web server requires permission to read and load these modules. Modules named CSP*
require permission to read and write to the Gateway configuration and log files (CSP.ini
). These are usually created in the same location as the Gateway binaries.
When considering access control for these modules, bear in mind that it is the web server worker processes
that need to be able to access the modules together with any dependent configuration and log files. For example, in the case of Apache, the server is usually started with superuser permissions but the worker processes that actually serve web requests run with a much lower level of authority (as indicated by the User and Group directives in the Apache configuration file). It is the User and Group specified for the worker processes that should be granted permission to load the Gateway modules and (where appropriate) the ability to read and write to the configuration and log files (CSP.ini
Executables to be called by the web server (CGI modules). (Not all configurations require these executables.)
The physical location where these are installed should match the corresponding configuration directives in the hosting web server configuration. This will include directives indicating which web requests should be processed by these CGI modules.
The worker processes of the hosting web server require execute permission for these modules. There are no further dependencies.
Static files to be returned by the web server.
With current CSP Gateway configurations, CSP is often configured to serve static files directly from Cache as opposed to having the web server return them. This section does not apply to such configurations.
Images (such as created-with-csp.gif
, and so on)
The worker processes of the hosting web server require Read permissions for these files.
The CSP Network Service Daemon (NSD)
Not all configurations require this facility.
This can be installed anywhere and the web server does not need to be aware of its physical location since communication between these two points is over TCP (usually port 7038).
The NSD requires permission to read and write to the Gateway configuration and log files (CSP.ini
), which are usually created in the same location.
For security reasons, do not
install this module in a location that is accessible by the web server. This module should not share a location with the modules listed in steps 1, 2 or 3. Many web server configurations described in this document explicitly exclude this module from the list of accessible files that can be accessed by the web server. However, it is much safer to physically install the NSD elsewhere in the file system.
Gateway Cache and Permanent Storage
Cached content is stored in files of type .dat
in the Gateway's temp
directory, placed by the install script directly beneath the Gateway's installation directory. For example, in a typical IIS installation this is in: C:\inetpub\CSPGateway\temp
. The location needs full read/write/delete permissions for the hosting web server worker processes.
The CSP Gateway provides high-performance connectivity solutions for Microsoft, Oracle (formerly Sun and Netscape), Apache, and Nginx web servers. In addition to these solutions, connectivity to Caché through the Common Gateway Interface (CGI) is available for all supported Operating Systems.
Both the Microsoft and the Oracle line of web servers support a multithreaded API which allows extensions, in the form of dynamically bound libraries, to be made to the web server’s core functionality. Current versions of the CSP Gateway make full use of these APIs in order to bring high-performance web connectivity to the Caché system. The Windows version of Apache also operates in an exclusively multithreaded mode and, as such, can also take advantage of the CSP Gateway implemented as a dynamically bound library. The Apache-based web server supplied for OpenVMS (HP Secure Web Server) is also a predominantly multithreaded server.
The UNIX® versions of Apache are architecturally different from the Microsoft Windows based web servers in that they are not exclusively multithreaded. Apache version 1.3 under UNIX does not use threads under any circumstances and operates exclusively in accordance with the traditional multi-process server model. Apache version 2 (and later) is implemented using a hybrid model made up of threads and multiple processes. In this model, each UNIX process is effectively a multithreaded server in its own right. The Oracle web server can operate as either an exclusively multithreaded server or as a hybrid multithreaded and multi-process server.
The Apache web server publishes a proprietary API in addition to supporting extensions implemented as CGI modules. Extra functionality can be added to Apache by means of user-defined modules (compiled C programs). In fact, a large part of Apache’s core functionality is implemented as a set of modules. You can add modules to Apache by one of two methods. First, the source to the module can be compiled directly into the Apache core. This option arguably offers the best performance but, unfortunately, involves reconfiguring and rebuilding the web server. As an alternative to building the module source directly into the Apache core, current versions of Apache (1.3 onwards) support extensions implemented as dynamically linked libraries. This facility allows you to take advantage of the high performance of Apache modules without the need to physically build the module into the core of Apache. The CSP module is distributed as a Windows Dynamic Link Library (DLL), a UNIX® Dynamic Shared Object (DSO), and as an OpenVMS shareable image. . UNIX® Shared Objects and OpenVMS shareable images are conceptually similar to a Windows Dynamic Link Library (DLL) and are linked at run time. The overhead involved in linking to a library at run time is very low on modern operating systems.
A more recent addition to the set of web servers supported by CSP is Nginx. Unlike other web servers, Nginx is based on an asynchronous event-driven architecture. With the event-driven architecture, notifications or signals are used to mark the initiation and completion of each individual operation. A consequence of this design is that while web requests are being processed, resources can be temporarily released and used by other operations. Resources can be allocated and released dynamically and are only associated with the processing of a web request while they actually required. This leads to a highly optimized use of memory and CPU. The asynchronous nature of this architecture results in threads executing concurrently without blocking each other, thus further enhancing the sharing of resources that might otherwise be associated with a thread waiting on a blocking operation. Nginx is supplied with an API to allow extensions, such as CSP, to be added to its core functionality. However, unlike other web servers, extension modules must be built into the web server core at compilation time. Nginx does not support dynamically loaded extension modules.
All supported web servers can be extended to support CSP through modules that are dynamically loaded by the hosting web server. It is expected that most CSP installations will take advantage of these high-performance web connectivity solutions. An alternative architecture in which the functionality of the CSP Gateway is implemented as a stand-alone executable, operating in its own process and not directly connected to a web server is also provided. This version of the CSP Gateway is known as the Network Service Daemon
(NSD). In this context, the NSD is responsible for providing the CSP Gateway’s core functionality and maintaining persistent connections to Caché. The web server communicates with the web server via small modules of which there are two types: modules that work to the hosting web server’s proprietary API and modules implemented as CGI executables. The NSD-based architecture is therefore used in cases where there is either a requirement to either extend the web server by means of the CGI standard or in cases where it is desirable to disengage the functionality of the CSP Gateway from that of the hosting web server.
Configuring the Web Server and the CSP Gateway
The Caché installation performs web server and CSP Gateway configuration for common web servers and operating systems.
After installing Caché and the CSP Gateway, consult the sections in this book relevant to your system to map file extensions for your system. The appendices in this book have configuration information for atypical CSP Gateway configurations.
To install the CSP Gateway on a remote server (that is, a system that is not running an instance of Caché), you can use one of two methods. On the remote server, you can
On UNIX platforms only, you can run the standalone CSPGateway
installation script. The script asks for information about the remote Caché server: name, address, port, and optional password. The script automatically configures csp.ini
based on this information.
Configuring the CSP Gateway for Ensemble
The CSP Gateway provides web connectivity for both the Caché and Ensemble product lines. The Gateway components are the same in both cases. A Gateway installation provided with a Caché installation can provide web connectivity to an Ensemble installation and vice versa.
For simplicity, this document is written for use with Caché installations. However, with the exception of default path names, the instructions contained in this document apply equally to Ensemble. The default Ensemble names replace Cache
Gateway Management Module configuration
Gateway architectures that work directly to a hosting web server’s API typically consist of two modules: A Management Module (for example, CSPmsSys.dll
) and a runtime module (for example, CSPms.dll
). The runtime Module is responsible for processing requests for CSP files and the Management Module provides the Gateway’s Management interface. In the CSP Gateway, the runtime Module assumes responsibility for loading and routing management requests to the management module. All requests for the CSP Gateway (CSP and management) are processed by the runtime Module. The Management Module must be installed in the same location as the runtime Module
Files of type .csp
are processed in Caché by CSP. All other files (static files) can be served by the web server or CSP. CSP can serve any type of file that is placed in the CSP applications path (including static files). Setting up CSP to serve static files simplifies the web server configuration for CSP applications because you, thus, do not need to create aliases in the web server configuration to represent the locations where an application’s static files are held. Setting up CSP to serve static files resolves issues of contention when a single (that is, common) web server serves two different versions of Caché, each requiring different versions of certain static files (for example, hyperevent broker components).
To have CSP serve static files for a particular CSP application, place the static files in the CSP application’s file system in the correct location relative to the CSP files that make up the application (not in the web server’s own documents file system). (Note that if you are serving files containing Unicode text, CSP uses the BOM to determine the correct encoding to use. The BOM must be present in Unicode text files.)
Consult the sections in this book for your platform.
To run Zen-based applications, you must enable the Serve Files
option and properly configure your web server. See the section Static Files
in Using CSP
for more information.
Serving Static Files from Caché
You can configure web servers and Gateway installations so that Caché assumes responsibility for serving static files. The Caché Management Portal and the Zen samples are supplied configured for Caché to serve all components in the application. However, it is still possible to configure the web server so that it retains responsibility for serving statics.
Hybrid Multi-Process/Multi-Threaded Web Server Architecture
The Gateway contains enhanced support for the hybrid multi-process/multi-threaded web server architecture. Apache version 2.x.x under UNIX is an example of a web server implemented according to this architecture.
With earlier builds of the Gateway, the stand-alone modules (CSPa2*[Sys].so
) were supported but each and every hosting web server worker process would bind to and create an independent instance of the Gateway. In other words, each worker process would load its own instance of the Gateway which would, in turn, load its own copy of the running configuration, manage its own connection table and form cache. The main observable symptom of this behavior being the variability in the Gateway System Status form each time Refresh
was selected. Each refresh of the Status form would result in the status for just the worker process and Gateway instance that happened to be processing the request for the status form. Also, state-aware mode (preserve mode 1) could not be supported in the multi-process architecture since this mode of operation depends on being able to route all requests for a session to the same Caché process which, in turn, means that all connections to Caché must be held in a single multithreaded process.
An architecture in which all Gateway resources are duplicated across all web server processes is unnecessary and wasteful.
The core Gateway resources are now held in the shared memory sector. All web server worker processes hare a common running configuration, connection table and form cache. The Gateway System Status form shows the status for the whole web server installation instead of just that of a single worker process. The status form’s connection table includes an extra column with the web server process ID with respect to each connection to Caché.
Finally, state-aware sessions are supported in the multi-process architecture. Although the connection pool (to Caché) is distributed amongst several web server processes, the Gateway uses an InterProcess Communications (IPC) protocol to route requests for state-aware sessions to the correct hosting process in the web server environment.
The Gateway is supplied with the new Caché based Gateway Registry. All web server and Gateway installations are registered with Caché as they connect. The registry contains the infrastructure to allow Caché code to interact with connected Gateway installations for the purpose of reading and writing the configuration and monitoring the system status and Event Log.
Enable Sticky Sessions on Hardware Load Balancer on High Availability Solutions
For High Availability solutions running over CSP, InterSystems recommends that you use a hardware load balancer for load balancing and failover. InterSystems requires that you enable sticky session support in the load balancer; this guarantees that -- once a session has been established between a given instance of the gateway and a given application server -- all subsequent requests from that user run on the same pair. This configuration assures that the session ID and server-side session context are always in sync; otherwise, it is possible that a session is created on one server but the next request from that user runs on a different system where the session is not present, which results in runtime errors (especially with hyperevents, which require the session key to decrypt the request). See your load balancer documentation for directions on how to enable sticky session support.
It is possible to configure a system to work without sticky sessions but this requires that the CSP session global be mapped across all systems in the enterprise and can result in significant lock contention so it is not recommended.
Enable Script to Reactivate Gateway Configuration
You can enable an external (non-Cache based) script to reactivate the Gateway’s configuration.
Scripts should add the following line (case sensitive) to the SYSTEM section of the Gateway configuration file:
The Gateway caretaker daemon checks the RELOAD flag approximately every minute and, if correctly set, reloads and reactives its configuration and removes the flag from the file. The following message is written to the Event Log after a successful reload operation:
Gateway Configuration Reloaded and Reactivated
Private Web Server and Caché Management Portal
The Management Portal Apache server is self-contained and configured to listen on a non-standard TCP port (something other than the usual, well known, HTTP server port of 80). It does not interfere with any other web server installation operating on the same host.
To access the Management Portal, enter the following URL (which resolves to the port number on your private web server for the current Caché instance):
The minimal Apache server used for the Management Portal is often referred to as the Private Web Server.
A minimal build of the Apache web server is supplied for the purpose of running the Management Portal. This server is known as the Private Web Server (PWS) and is built and configured to meet the management needs of InterSystems server products and is configured to only connect to Cache and Ensemble. The options selected to create the PWS are not, in general, suitable for production use. In particular, security is minimal and the configuration deployed is generally unsuitable for applications for which a high volume of HTTP requests is anticipated. Testing (by InterSystems) of the PWS only covers the use of this server for managing Caché and Ensemble. However many developers find it useful to use the PWS for testing their own CSP and Zen applications.
When installing Caché and Ensemble, this private version of Apache is installed to ensure that:
The Management Portal runs out of the box.
An out-of-the-box testing capability is provided for development environments.
The private Apache web server is not supported for any other purpose.
The PWS is responsible for supporting the management portals for Caché, Ensemble, and HealthShare Foundation. However, customers are not required to use this web server to manage InterSystems products: customers may run the various management portals through a web server of their own choosing.
Finally, the PWS is self contained and configured to listen on a non-standard TCP port (something other than the usual, well known, HTTP server port of 80). It will not interfere with any other web server installation operating on the same host.
The PWS is available for UNIX, Linux, Mac OS X and Windows, but not available for OpenVMS. See the section Building the Private Web Server
for more information.
The entry point for the Management Portal is normally via the following CSP path and file:
Building the Private Web Server
The (default) full Apache server is usually created with the following sequence of commands:
The minimal Apache build is typically created as follows:
./configure --prefix=/usr/cachesys/httpd --with-port=57772 \
--enable-mods-static="log_config mime alias unixd authz_core" \
--enable-so --without-gdbm --without-ndbm \
--without-berkeley-db --with-included-apr --with-expat=builtin \
Notice that many of the services that are normally required for a production grade installation are excluded.
While this server can be used to host other CSP applications it is strongly recommended that a full, independent web server installation is used for this purpose. It should be remembered that any changes made to the configuration of the Management Portal Apache installation are overwritten when the hosting Caché installation is upgraded.
The Management Portal Apache installation uses the following CSP Gateway modules for communicating with Caché:
: InterSystems does not supply an Apache installation for OpenVMS systems. For these systems, the Management Portal can be either hosted by a web server (and CSP Gateway) installation operating on a remote, non-OpenVMS, platform or through the Apache web server supplied by Hewlett-Packard. This latter server, known as HP Secure Web Server (SWS), is based on Apache. Specifically, SWS version 2.2 is based on Apache version 2.0.63.
Managing the Private Web Server
Under normal operational conditions, the Management Portal Web Server for a particular instance of Caché is started when Caché is started and closed down when Caché is closed down. Occasionally it may be necessary to restart the Management Portal Web Server without disrupting the corresponding Caché server. For example, a web server restart is necessary if a configuration change is made to the web server (httpd.conf
Use the following commands to start and stop the Management Portal Web Server.
Start the Management Portal Web Server:
<cache-install-dir>\httpd\bin\httpd -k start -n <instname>httpd -c "Listen <port>"
Stop the Management Portal Web Server:
<cache-install-dir>\httpd\bin\httpd -k stop -n <instname>httpd
Caché instance name: CACHE
TCP port for Apache: 57772
Start the Management Portal Web Server:
Stop the Management Portal Web Server:
TCP port for Apache: 8972
Limitations of the Private Web Server
This section discusses the differences between the configuration of the PWS and that of a typical production grade Apache installation.
Windows-based Apache installations use a special multithreaded form of the Apache Multi-Processing Module (MPM) which is better suited to the way the operating system is optimized. Therefore, the behavior of the PWS under Windows is similar to that of a production grade Apache build as far as the ability to handle concurrent load is concerned.
If high availability and production-grade security is a requirement, or threre is a need to integrate with other sources of web information, or a need for a high degree of control over the web server, a separate production-grade build of Apache is recommended - ideally operating on its own server. If, on the other hand, low volumes of HTTP traffic are expected, and there are limited demands for high availability and security, then the PWS may be suitable for deployment under these circumstances.
The PWS defaults to using the Apache Group’s prefork Multi-Processing Module (MPM). This is a non-threaded server model: the number of requests that can be concurrently served is directly related to the number of Apache worker processes in the pool.
The PWS is configured to occupy the smallest possible footprint by allowing a maximum of two worker processes to be created for the pool. The following settings will be found in the Apache configuration (httpd.conf
) for the PWS:
By contrast, the default Apache configuration for a production grade build is usually as follows:
This configuration allows Apache to create 5 worker processes at start-up, increasing to a maximum of 256 as the concurrent load increases. Because of these differences in configuration, the performance of the PWS is noticeably inferior to that of a production grade Apache build. This performance deficit becomes more noticeable as the concurrent load increases. However, it is possible to change the configuration of the PWS to match that of a full Apache installation (shown above). Apache must be completely restarted after changing these parameters.