Introduction to the CSP Gateway
The CSP Gateway provides the communications layer between the hosting web server and Caché when you call a Caché Server Page.
Who Should Read this Document
The Caché installation includes scripts that perform Web Server and CSP Gateway configuration for common Web Servers and operating systems.
In most cases, installing Caché according to the usual Caché instructions and installing a typical, supported web server provides a system that works with the CSP Gateway without the need to consult this document.
However, if you have an atypical web server architecture or you are an advanced user who wants to get the best out of your environment, you might want to use this document. This document describes the details of procedures for configuring a web server and the CSP Gateway to connect to Caché. It also describes how to use services that the CSP Gateway provides.
Documentation on Caché Server Pages can be found in the following places:
Using Caché Server Pages — describes how to create CSP pages
CSP HTML Tag Reference — a reference to all CSP tags
Caché Server Pages Quick Start Tutorial — gets you started
CSP Web Applications Tutorial — an in-depth tutorial
CSP Web Gateway Documentation — online help on configuring the CSP Gateway is available in this book in the CSP Gateway Management page in the Management Portal System Administration > Configuration > CSP Gateway Management.
CSP Gateway Configuration Guide — when you install Caché, the CSP Gateway is installed automatically and works for most sites. If you need to configure the CSP Gateway manually, use this document. The following two topics in this guide contain procedures on configuring the CSP Gateway:
The other chapters contain information relating to particular web servers and are outlined in the following section.
Conventions Used in this Document
The default installation directory for Caché is documented in the Default Caché Installation Directory section of the Caché Installation Guide. This guide refers to the default using the variable install-dir. Examples may use C:\cachesys or C:\cache-install-dir\ as the installation directory.
When following steps in this document, change paths to match your own installation.
Lines terminated with a back-slash (\) are continued to the next line.
For example, enter the following line, as shown in this document:
Init fn=load-modules shlib=CSPn3.dll \ funcs=csp_term
Init fn=load-modules shlib=CSPn3.dll funcs=csp_term
Gateway components and physical installation paths
Later sections in this guide describe how the Gateway components should be configured with all supported web servers. The installation paths for components should be regarded as examples rather than taken literally. Also, the InterSystems installers create and maintain separate Gateway installations for the internal (private) web server and any third-party web server that might be present on the same host. In this context ‘third-party web server’ refers to a web server that is not part of the software installed by InterSystems.
The precise installation location of Gateway components is not particularly critical provided:
The physical installation paths match those given in the hosting web server configuration where appropriate.
The security settings, in relation to required access for individual components, are adjusted appropriately. This is particularly important for Gateway components that are accessed directly by the web server since web servers are usually locked down to the extent that the files they are able to access (and executables that can be run) are carefully controlled. It should be borne in mind that security considerations will also be important for any Gateway configuration (and log) files that are accessed by Gateway binaries that are themselves bound to the web server core executable.
The security policy of the hosting web server is respected. Some web servers – notably that shipped with Secure Linux (SELinux) – are configured such that it is not possible for them to access files that lie outside their own file system. This restriction will clearly have an impact on where certain web-server-facing Gateway components can be installed.
There are four types of Gateway component to consider.
Binaries to be loaded by the web server (API based extensions).
This includes Windows DLLs and UNIX Shared Objects:
CSPms[Sys].dll CSPn3[Sys].(dll|so|exe) CSPa*[Sys].(dll|so) mod_csp*.(so|exe)
The physical location where these are installed should match the corresponding configuration directives in the hosting web server configuration. This includes directives indicating which third-party modules should be loaded. The web server requires permission to read and load these modules. Modules named CSP* require permission to read and write to the Gateway configuration and log files (CSP.ini, CSP.log). These are usually created in the same location as the Gateway binaries.
When considering access control for these modules, bear in mind that it is the web server worker processes that need to be able to access the modules together with any dependent configuration and log files. For example, in the case of Apache, the server is usually started with superuser permissions but the worker processes that actually serve web requests run with a much lower level of authority (as indicated by the User and Group directives in the Apache configuration file). It is the User and Group specified for the worker processes that should be granted permission to load the Gateway modules and (where appropriate) the ability to read and write to the configuration and log files (CSP.ini and CSP.log).
Executables to be called by the web server (CGI modules). (Not all configurations require these executables.)
The physical location where these are installed should match the corresponding configuration directives in the hosting web server configuration. This will include directives indicating which web requests should be processed by these CGI modules.
The worker processes of the hosting web server require execute permission for these modules. There are no further dependencies.
Static files to be returned by the web server.Note:
With current CSP Gateway configurations, CSP is often configured to serve static files directly from Cache as opposed to having the web server return them. This section does not apply to such configurations.
Java applets (such as CSPBroker.class, CSPBroker.jar, and so on)
Images (such as created-with-csp.gif, and so on)
The worker processes of the hosting web server require Read permissions for these files.
The CSP Network Service Daemon (NSD)Note:
Not all configurations require this facility.
This can be installed anywhere and the web server does not need to be aware of its physical location since communication between these two points is over TCP (usually port 7038).
The NSD requires permission to read and write to the Gateway configuration and log files (CSP.ini and CSP.log), which are usually created in the same location.Note:
For security reasons, do not install this module in a location that is accessible by the web server. This module should not share a location with the modules listed in steps 1, 2 or 3. Many web server configurations described in this document explicitly exclude this module from the list of accessible files that can be accessed by the web server. However, it is much safer to physically install the NSD elsewhere in the file system.
Cached content is stored in files of type .dat in the Gateway's temp directory, placed by the install script directly beneath the Gateway's installation directory. For example, in a typical IIS installation this is in: C:\inetpub\CSPGateway\temp. The location needs full read/write/delete permissions for the hosting web server worker processes.
Supported Web Servers
More detailed information on supported web servers can be found in the section “Supported Web Servers” in the online InterSystems Supported Platforms document for this release. The following table summarizes the web servers discussed in this document.
|Operating System||Web Servers|
|Microsoft Windows||Microsoft — Internet Information Services (IIS)|
The CSP Gateway provides high-performance connectivity solutions for Microsoft, Oracle (formerly Sun and Netscape), Apache, and Nginx web servers. In addition to these solutions, connectivity to Caché through the Common Gateway Interface (CGI) is available for all supported Operating Systems.
Both the Microsoft and the Oracle line of web servers support a multithreaded API which allows extensions, in the form of dynamically bound libraries, to be made to the web server’s core functionality. Current versions of the CSP Gateway make full use of these APIs in order to bring high-performance web connectivity to the Caché system. The Windows version of Apache also operates in an exclusively multithreaded mode and, as such, can also take advantage of the CSP Gateway implemented as a dynamically bound library.
The UNIX® versions of Apache are architecturally different from the Microsoft Windows based web servers in that they are not exclusively multithreaded. Apache version 1.3 under UNIX does not use threads under any circumstances and operates exclusively in accordance with the traditional multi-process server model. Apache version 2 (and later) is implemented using a hybrid model made up of threads and multiple processes. In this model, each UNIX process is effectively a multithreaded server in its own right. The Oracle web server can operate as either an exclusively multithreaded server or as a hybrid multithreaded and multi-process server.
The Apache web server publishes a proprietary API in addition to supporting extensions implemented as CGI modules. Extra functionality can be added to Apache by means of user-defined modules (compiled C programs). In fact, a large part of Apache’s core functionality is implemented as a set of modules. You can add modules to Apache by one of two methods. First, the source to the module can be compiled directly into the Apache core. This option arguably offers the best performance but, unfortunately, involves reconfiguring and rebuilding the web server. As an alternative to building the module source directly into the Apache core, current versions of Apache (1.3 onwards) support extensions implemented as dynamically linked libraries. This facility allows you to take advantage of the high performance of Apache modules without the need to physically build the module into the core of Apache. The CSP module is distributed as a Windows Dynamic Link Library (DLL), and as a UNIX® Dynamic Shared Object (DSO). UNIX® Shared Objects are conceptually similar to a Windows Dynamic Link Library (DLL) and are linked at run time. The overhead involved in linking to a library at run time is very low on modern operating systems.
A more recent addition to the set of web servers supported by CSP is Nginx. Unlike other web servers, Nginx is based on an asynchronous event-driven architecture. With the event-driven architecture, notifications or signals are used to mark the initiation and completion of each individual operation. A consequence of this design is that while web requests are being processed, resources can be temporarily released and used by other operations. Resources can be allocated and released dynamically and are only associated with the processing of a web request while they actually required. This leads to a highly optimized use of memory and CPU. The asynchronous nature of this architecture results in threads executing concurrently without blocking each other, thus further enhancing the sharing of resources that might otherwise be associated with a thread waiting on a blocking operation. Nginx is supplied with an API to allow extensions, such as CSP, to be added to its core functionality. However, unlike other web servers, extension modules must be built into the web server core at compilation time. Nginx does not support dynamically loaded extension modules.
All supported web servers can be extended to support CSP through modules that are dynamically loaded by the hosting web server. It is expected that most CSP installations will take advantage of these high-performance web connectivity solutions. An alternative architecture in which the functionality of the CSP Gateway is implemented as a stand-alone executable, operating in its own process and not directly connected to a web server is also provided. This version of the CSP Gateway is known as the Network Service Daemon (NSD). In this context, the NSD is responsible for providing the CSP Gateway’s core functionality and maintaining persistent connections to Caché. The web server communicates with the web server via small modules of which there are two types: modules that work to the hosting web server’s proprietary API and modules implemented as CGI executables. The NSD-based architecture is therefore used in cases where there is either a requirement to either extend the web server by means of the CGI standard or in cases where it is desirable to disengage the functionality of the CSP Gateway from that of the hosting web server.
CSP Gateway Standalone Installers
In addition to installing the CSP Gateway as part of the Caché install, you can install the CSP Gateway independently using a standalone installer. The standalone installers for UNIX® and Windows are available on the WRC distribution site under Components.
The CSP Gateway Windows standalone installer is provided as an executable and the UNIX standalone installer is provided as a command line script.
The defaults for the Windows standalone installer assume that you are using the IIS web server, and the defaults for the UNIX standalone installer assume you are using the Apache web server. If you have an atypical web server architecture, or want to customize your environment, you should read this document carefully to understand the configuration options.
Configuring the Web Server and the CSP Gateway
The Caché installation performs web server and CSP Gateway configuration for common web servers and operating systems.
After installing Caché and the CSP Gateway, consult the sections in this book relevant to your system to map file extensions for your system. The appendices in this book have configuration information for atypical CSP Gateway configurations.
To install the CSP Gateway on a remote server (that is, a system that is not running an instance of Caché), you can use one of two methods. On the remote server, you can
Run the Caché installation script and, on the Setup Type page, select Web Server only or
On UNIX platforms, you can run the standalone CSPGateway installation script. The script asks for information about the remote Caché server: name, address, port, and optional password. The script automatically configures csp.ini based on this information.
For more information on configuring a remote web server, see the section Using Caché Server Pages with a Remote Web Server in this book. After installing the CSP Gateway, consult the sections in this book to map file extensions for your system.
To prevent runtime errors, for High Availability configurations running over CSP, InterSystems recommends that you use a hardware load balancer with sticky session support enabled. For more information, see Enable Sticky Sessions on Hardware Load Balancer on High Availability Solutions.
Configuring the CSP Gateway for Ensemble
The CSP Gateway provides web connectivity for both the Caché and Ensemble product lines. The Gateway components are the same in both cases. A Gateway installation provided with a Caché installation can provide web connectivity to an Ensemble installation and vice versa.
For simplicity, this document is written for use with Caché installations. However, with the exception of default path names, the instructions contained in this document apply equally to Ensemble. The default Ensemble names replace Cache with Ensemble.
Gateway Management Module configuration
Gateway architectures that work directly to a hosting web server’s API typically consist of two modules: A Management Module (for example, CSPmsSys.dll) and a runtime module (for example, CSPms.dll). The runtime Module is responsible for processing requests for CSP files and the Management Module provides the Gateway’s Management interface. In the CSP Gateway, the runtime Module assumes responsibility for loading and routing management requests to the management module. All requests for the CSP Gateway (CSP and management) are processed by the runtime Module. The Management Module must be installed in the same location as the runtime Module
File Types Served by CSP
Files of type .csp, .cls and .zen are processed in Caché by CSP. All other files (static files) can be served by the web server or CSP. CSP can serve any type of file that is placed in the CSP applications path (including static files). Setting up CSP to serve static files simplifies the web server configuration for CSP applications because you, thus, do not need to create aliases in the web server configuration to represent the locations where an application’s static files are held. Setting up CSP to serve static files resolves issues of contention when a single (that is, common) web server serves two different versions of Caché, each requiring different versions of certain static files (for example, hyperevent broker components).
To have CSP serve static files for a particular CSP application, place the static files in the CSP application’s file system in the correct location relative to the CSP files that make up the application (not in the web server’s own documents file system). (Note that if you are serving files containing Unicode text, CSP uses the BOM to determine the correct encoding to use. The BOM must be present in Unicode text files.)
Consult the sections in this book for your platform.
To run Zen-based applications, you must enable the Serve Files option and properly configure your web server. See the section Static Files in Using CSP for more information.
Serving Static Files from Caché
You can configure web servers and Gateway installations so that Caché assumes responsibility for serving static files. The Management Portal and the Zen samples are supplied configured for Caché to serve all components in the application. However, it is still possible to configure the web server so that it retains responsibility for serving statics.
Hybrid Multi-Process/Multi-Threaded Web Server Architecture
The Gateway contains enhanced support for the hybrid multi-process/multi-threaded web server architecture. Apache version 2.x.x under UNIX is an example of a web server implemented according to this architecture.
With earlier builds of the Gateway, the stand-alone modules (CSPa2*[Sys].so) were supported but each and every hosting web server worker process would bind to and create an independent instance of the Gateway. In other words, each worker process would load its own instance of the Gateway which would, in turn, load its own copy of the running configuration, manage its own connection table and form cache. The main observable symptom of this behavior being the variability in the Gateway System Status form each time Refresh was selected. Each refresh of the Status form would result in the status for just the worker process and Gateway instance that happened to be processing the request for the status form. Also, state-aware mode (preserve mode 1) could not be supported in the multi-process architecture since this mode of operation depends on being able to route all requests for a session to the same Caché process which, in turn, means that all connections to Caché must be held in a single multithreaded process.
An architecture in which all Gateway resources are duplicated across all web server processes is unnecessary and wasteful.
The core Gateway resources are now held in the shared memory sector. All web server worker processes hare a common running configuration, connection table and form cache. The Gateway System Status form shows the status for the whole web server installation instead of just that of a single worker process. The status form’s connection table includes an extra column with the web server process ID with respect to each connection to Caché.
Finally, state-aware sessions are supported in the multi-process architecture. Although the connection pool (to Caché) is distributed amongst several web server processes, the Gateway uses an InterProcess Communications (IPC) protocol to route requests for state-aware sessions to the correct hosting process in the web server environment.
The Gateway is supplied with the new Caché based Gateway Registry. All web server and Gateway installations are registered with Caché as they connect. The registry contains the infrastructure to allow Caché code to interact with connected Gateway installations for the purpose of reading and writing the configuration and monitoring the system status and Event Log.
Enable Sticky Sessions on Hardware Load Balancer on High Availability Solutions
For High Availability solutions running over CSP, InterSystems recommends that you use a hardware load balancer for load balancing and failover. InterSystems requires that you enable sticky session support in the load balancer; this guarantees that -- once a session has been established between a given instance of the gateway and a given application server -- all subsequent requests from that user run on the same pair. This configuration assures that the session ID and server-side session context are always in sync; otherwise, it is possible that a session is created on one server but the next request from that user runs on a different system where the session is not present, which results in runtime errors (especially with hyperevents, which require the session key to decrypt the request). See your load balancer documentation for directions on how to enable sticky session support.
It is possible to configure a system to work without sticky sessions but this requires that the CSP session global be mapped across all systems in the enterprise and can result in significant lock contention so it is not recommended.
Enable Script to Reactivate Gateway Configuration
You can enable an external (non-Cache based) script to reactivate the Gateway’s configuration.
Scripts should add the following line (case-sensitive) to the SYSTEM section of the Gateway configuration file:
The Gateway caretaker daemon checks the RELOAD flag approximately every minute and, if correctly set, reloads and reactivates its configuration and removes the flag from the file. The following message is written to the Event Log after a successful reload operation:
Gateway Management Gateway Configuration Reloaded and Reactivated
Private Web Server and Management Portal
The Management Portal Apache server is self-contained and configured to listen on a non-standard TCP port (something other than the usual, well known, HTTP server port of 80). It does not interfere with any other web server installation operating on the same host.
To access the Management Portal, enter the following URL (which resolves to the port number on your private web server for the current Caché instance):
The minimal Apache server used for the Management Portal is often referred to as the Private Web Server.
A minimal build of the Apache web server is supplied for the purpose of running the Management Portal. This server is known as the Private Web Server (PWS) and is built and configured to meet the management needs of InterSystems server products and is configured to only connect to Cache and Ensemble. The options selected to create the PWS are not, in general, suitable for production use. In particular, security is minimal and the configuration deployed is generally unsuitable for applications for which a high volume of HTTP requests is anticipated. Testing (by InterSystems) of the PWS only covers the use of this server for managing Caché and Ensemble. However many developers find it useful to use the PWS for testing their own CSP and Zen applications.
When installing Caché and Ensemble, this private version of Apache is installed to ensure that:
The Management Portal runs out of the box.
An out-of-the-box testing capability is provided for development environments.
The private Apache web server is not supported for any other purpose.
For deployments of http-based applications, including CSP, Zen, and SOAP over http or https, you should not use the private web server for any application other than the Management Portal; instead, you must install and deploy one of the supported web servers (for information, see the section “Supported Web Servers” in the online InterSystems Supported Platforms document for this release).
The PWS is responsible for supporting the management portals for Caché, Ensemble, and HealthShare Foundation. However, customers are not required to use this web server to manage InterSystems products: customers may run the various management portals through a web server of their own choosing.
Finally, the PWS is self contained and configured to listen on a non-standard TCP port (something other than the usual, well known, HTTP server port of 80). It will not interfere with any other web server installation operating on the same host.
The PWS is available for UNIX, Linux, Mac OS X and Windows. See the section Building the Private Web Server for more information.
The entry point for the Management Portal is normally via the following CSP path and file:
Building the Private Web Server
The (default) full Apache server is usually created with the following sequence of commands:
./configure --prefix=<install-dir> make make install
The minimal Apache build is typically created as follows:
./configure --prefix=/usr/cachesys/httpd --with-port=57772 \ --with-pcre=$srcdir/pcre \ --enable-mods-static="log_config mime alias unixd authz_core" \ --disable-ssl \ --enable-so --without-gdbm --without-ndbm \ --without-berkeley-db --with-included-apr --with-expat=builtin \ --with-mpm=prefork --disable-shared make make install
Notice that many of the services that are normally required for a production grade installation are excluded.
While this server can be used to host other CSP applications it is strongly recommended that a full, independent web server installation is used for this purpose. It should be remembered that any changes made to the configuration of the Management Portal Apache installation are overwritten when the hosting Caché installation is upgraded.
The Management Portal Apache installation uses the following CSP Gateway modules for communicating with Caché:
Windows: CSPa2.dll and CSPa2Sys.dll
UNIX®: CSPa2.so and CSPa2Sys.so
Managing the Private Web Server
Under normal operational conditions, the Management Portal Web Server for a particular instance of Caché is started when Caché is started and closed down when Caché is closed down. Occasionally it may be necessary to restart the Management Portal Web Server without disrupting the corresponding Caché server. For example, a web server restart is necessary if a configuration change is made to the web server (httpd.conf).
Use the following commands to start and stop the Management Portal Web Server.
Start the Management Portal Web Server:
<cache-install-dir>\httpd\bin\httpd -k start -n <instname>httpd -c "Listen <port>"
Stop the Management Portal Web Server:
<cache-install-dir>\httpd\bin\httpd -k stop -n <instname>httpd
Caché installed in: C:\cachesys
Caché instance name: CACHE
TCP port for Apache: 57772
C:\cachesys\httpd\bin\httpd -k start -n CACHEhttpd -c "Listen 57772"
C:\cachesys\httpd\bin\httpd -k stop -n CACHEhttpd
Start the Management Portal Web Server:
<cache-install-dir>/httpd/bin/httpd -d <cache-install-dir>/httpd -c "Listen <port>"
Stop the Management Portal Web Server:
kill `cat <cache-install-dir>/httpd/logs/httpd.pid`
On AIX®, LD_LIBRARY_PATH must include the /cache-install-dir/bin directory in order to manually run httpd.conf.
Caché installed in: /usr/cachesys
TCP port for Apache: 8972
/usr/cachesys/httpd/bin/httpd -d /usr/cachesys/httpd -c "Listen 8972"
kill `cat /usr/cachesys/httpd/logs/httpd.pid`
You can also use the following methods of the class Config.Startup:
Limitations of the Private Web Server
This section discusses the differences between the configuration of the PWS and that of a typical production grade Apache installation.
Windows-based Apache installations use a special multithreaded form of the Apache Multi-Processing Module (MPM) which is better suited to the way the operating system is optimized. Therefore, the behavior of the PWS under Windows is similar to that of a production grade Apache build as far as the ability to handle concurrent load is concerned.
If high availability and production-grade security is a requirement, or there is a need to integrate with other sources of web information, or a need for a high degree of control over the web server, a separate production-grade build of Apache is recommended - ideally operating on its own server. If, on the other hand, low volumes of HTTP traffic are expected, and there are limited demands for high availability and security, then the PWS may be suitable for deployment under these circumstances.
The PWS defaults to using the Apache Group’s prefork Multi-Processing Module (MPM). This is a non-threaded server model: the number of requests that can be concurrently served is directly related to the number of Apache worker processes in the pool.
The PWS is configured to occupy the smallest possible footprint by allowing a maximum of two worker processes to be created for the pool. The following settings will be found in the Apache configuration (httpd.conf) for the PWS:
MinSpareServers 1 MaxSpareServers 2
By contrast, the default Apache configuration for a production grade build is usually as follows:
StartServers 5 MinSpareServers 2 MaxSpareServers 20 ServerLimit 256 MaxClients 256
This configuration allows Apache to create 5 worker processes at startup, increasing to a maximum of 256 as the concurrent load increases. Because of these differences in configuration, the performance of the PWS is noticeably inferior to that of a production grade Apache build. This performance deficit becomes more noticeable as the concurrent load increases. However, it is possible to change the configuration of the PWS to match that of a full Apache installation (shown above). Apache must be completely restarted after changing these parameters.