New and Enhanced Features for InterSystems IRIS 2021.2
This article describes the new and enhanced features in the 2021.2 release of InterSystems IRIS®, which is a Continuous Delivery (CD) release. This is the first CD release to be available on server platforms, in addition to an OCCI-compliant container format.
Enhancing Developer Experience
InterSystems IRIS 2021.2 introduces Python fully integrated into the kernel, making Python a full peer to ObjectScript. Almost anything that you can do in ObjectScript, you can now also do in Python, including defining class methods. You can also interleave Python and ObjectScript, including directly calling Python libraries from ObjectScript without writing any Python code. Python provides access to many thousands of high-quality pre-built libraries, which can speed development and lower your maintenance cost. Python developers who are not familiar with ObjectScript can start developing without learning a new language.
Any InterSystems IRIS object can be created and accessed with Embedded Python:
Objects implemented in embedded Python are treated the same as objects implemented in ObjectScript.
In Embedded Python you have full and direct access from Python objects to ObjectScript objects and from ObjectScript objects to Python objects.
Embedded Python has full access to globals, which are accessed as normal Python objects. You can use InterSystems IRIS persistence to store objects in the database, making them available in future sessions until the objects are explicitly deleted.
Embedded Python augments the InterSystems IRIS Python SDK, which includes client libraries and the external Python gateway.
For an introduction to embedded Python, see the Embedded Python Overview.
Embedded Python is designed to run with whatever version of Python you have installed on your machine.
If you are running Microsoft Windows and do not have Python already installed, the InterSystems IRIS installation kit installs it for you.
Many flavors of UNIX or Linux come with Python installed. If you need to install it, use the version recommended for your operating system by your package manager, for example:
macOS: Install Python 3.9 using Homebrew (https://firstname.lastname@example.org)
Ubuntu: apt-get install python3
Red Hat Enterprise Linux or Oracle Linux: yum install python3
SUSE: zypper install python3
If you get an error that says “Failed to load python,” it means that you either don’t have Python installed or an unexpected version of Python is installed on your system. Install it or reinstall it using one of the above methods.
On a UNIX-based system, you may want to install Python packages with the pip3 command. If you do not have pip3 installed already, install the package python3-pip with your system’s package manager.
Interoperability Productions in Python
In this release, you can develop interoperability productions in Python using the Production EXtension (PEX) framework. This gives you the choice of developing productions in Python, Java, .NET, or ObjectScript. You can easily combine production components developed in different languages. You can develop in the language you are familiar with even if the other production components were developed in a different language. You can use Python with PEX to create new protocol adapter, perform complex analysis or calculations, and to create persistent messaging and long-running business processes. For more information, see Developing Production Components with External Languages.
Visual Studio Code ObjectScript Extension Pack Updates
The Visual Studio Code ObjectScript Extension Pack is available from the Visual Studio Code download page and has the following enhancements that make developing code faster and easier:
Integrated documentation — hover-over in-line documentation, browse class hierarchies, and preview custom class documentation.
Server-side source improvements — search and support for many client-side web application workflows.
Debugging — inspect properties of objects and improved reliability.
The extension pack includes the ObjectScript extension and the Language Server extension. For more information, see the VSCode ObjectScript Extension documentation https://intersystems-community.github.io/vscode-objectscript/.
Make SQL Queries With Minimal Code in Interoperability Productions
Enhancing Analytics and AI
The SQL LOAD DATA command loads data from a CSV file or JDBC source into an SQL table efficiently. This allows you to easily populate a table with well-validated data. You can refine the command with COLUMNS and VALUES clauses similar to an INSERT statement and override default behavior with the USING clause, similar to how it’s used in IntegratedML. For details see LOAD DATA in the SQL Reference.
Adaptive Analytics Enhancements
In this release, Adaptive Analytics added two additional features:
Validation of InterSystems Reports as a client of Adaptive Analytics — now our customers can provide reports using InterSystems Reports with the same data model that is shared with PowerBI, Tableau and other Business Intelligence tools.
Import of InterSystems IRIS Business Intelligence (BI) cubes — we have the ability to export an InterSystems IRIS BI cube definition and import it as a virtual cube in Adaptive Analytics. Please note that there are caveats with this – some things cannot be exported from a cube such as Cube Relationships and cubes based on data connectors – see the Adaptive Analytics documentation for more information.
Enhancing Cloud and Operations
This release contains cloud connectors that make it easier for you to manage InterSystems IRIS applications in Amazon Web Services and use connectors to access services. This release has the following adapters:
Inbound and outbound adapters for S3 (Amazon Simple Storage Service)
Outbound adapter for Cloudwatch (Amazon monitoring service)
Outbound adapter for SNS (Amazon Simple Notification Service) for messaging
This release makes it easier to deploy and manage InterSystems IRIS in Kubernetes with the following new InterSystem Kubernetes Operator (IKO) features:
IKO can deploy and manage InterSystems System Alert and Monitoring (SAM) and the InterSystems API Manager (IAM) with your InterSystems IRIS cluster. This makes it easier to administer and scale your system.
IIKO can deploy locked down InterSystems IRIS and InterSystems Web Gateway containers.
IKO can deploy InterSystems Web Gateway containers with Nginx as well as Apache web servers.
IKO can include ephemeral as well as persistent volumes in deployments.
Enhancing Speed, Scale, and Security
Online Shard Rebalancing
InterSystems IRIS Sharding distributes data and its associated workload across multiple nodes. Even data distributions offer near linear scalability for analytical query workloads. Therefore, if the data volume or workload increases, you may add additional data nodes to meet your performance goals. After adding a data node, a rebalancing operation can be used to re-distribute the older data across all available nodes, which enhances the elasticity of sharded clusters. Starting in this release, this rebalancing happens online: users can continue to query and update the data while rebalancing is running.
Adaptive SQL Optimizer
The InterSystems IRIS SQL Optimizer leverages table statistics to derive the best query plan for each user-submitted statement and uses an efficient query cache to reuse the generated code. When those statements include parameters, the values submitted at runtime may provide opportunities for faster execution using an alternative query plan. The new Runtime Plan Choice (RTPC) infrastructure introduced with this release ensures InterSystems IRIS SQL takes advantage of such opportunities efficiently. RTPC scans for the use of outlier values and efficiently estimates the selectivity of range conditions based on more detailed table statistics. This leads to more adaptive query planning and significant savings in execution time and I/O for many real-world datasets.
In addition, InterSystems IRIS now uses block-level sampling rather than full or row-based scanning to gather the table statistics used by the optimizer. This efficient algorithm enables gathering statistics (such as by using the TUNE TABLE command) for even the largest tables with billions of rows within seconds. Also, InterSystems IRIS SQL will now gather table statistics on-the-fly when a table has none to ensure appropriate query plans.
Saving on Storage
In this release, stream and journal compression can significantly reduce storage needed for your InterSystems IRIS deployment:
Stream compression – is now on by default for all globals-based stream classes, with no application change required. Existing data remains readable and will be compressed upon the next write. Experiments with real-world data have indicated compression ratios ranging from 30% for short texts to 80% and more for XML and other document types.
Journal compression — compresses inactive journal files immediately after journal switch. Rollback and roll forward are executed directly from the compressed format. This significantly reduces the storage requirements for this vital part of InterSystems IRIS data integrity strategy. See Journaling Best Practices for more information.
TLS 1.3 Support (OpenSSL 1.1.1)
With this version, InterSystems IRIS includes support for OpenSSL 1.1.1 and fully supports TLS 1.3. With TLS 1.3 users will see faster performance among other improvements, such as cutting the encryptions latency in half. This is accomplished by eliminating an entire round trip from the handshake process.
Beginning with InterSystems IRIS 2021.2 we will no longer ship OpenSSL libraries on UNIX but depend on the OS to provide those. One benefit of this change is that updates to the OpenSSL library no longer requires a new installation of InterSystems IRIS but can be performed with the usual OS updates. For more information on this new approach, see Relationship of TLS Version to Operating System and Its Version. Because InterSystems products require access to the OS provided OpenSSL library, the product will now perform a check during the installation and the startup of an instance. The call can also be manually invoked. See Installing the Required Dependencies for details.
The change to not ship OpenSSL libraries also triggered an adjustment we needed to make for kits. Every kit is specific to a major version of OpenSSL (OpenSSL 1.1.1 is the major version, minor versions are indicated by a letter following the major version, such as OpenSSL 1.1.1f).
On Windows, the kit does install the OpenSSL library.
If the correct version of OpenSSL is not installed on your UNIX system, the installation will not succeed. You must install OpenSSL and then reinstall InterSystems IRIS. For example, on MacOS, you can install OpenSSL using Homebrew, see https://email@example.com.
New ^TRACE Utility
This release introduces a new tool for tracing raw events from one or more processes. Existing utilities such as %SYS.MONLBL and PERFMON track mostly the same event types but immediately generate a report formatted for a specific type of analysis. The new ^TRACE tool captures these events in a more generic file format and allows interactive navigation and summarization of the captured information through a command-line interface or API. Supported event types include, but are not limited to global sets and kills, physical writes, network requests, cache hits and reads, and various journal events. Information captured for these events includes the routine line and call stack, as well as the full global reference where applicable. This offers a single interface for a broader set of performance analysis tasks.
Other Enhancements and Efficiency Improvements
In each release, InterSystems makes many efficiency improvements and minor enhancements. This release includes:
Compact double support for external clients, including JDBC, .NET clients, Python, and IRISNative.
In this release, the security tables now have an embedded version number, which allows finer access over allowable imports. You can export security tables from version 2021.1 and then import them to this version. For details, see ^SECURITY.
This release updates the Log4j library to version 2.17.0.
This release updates the node.js library to version 14.
New Platforms for Continuous Delivery Releases of InterSystems IRIS
InterSystems IRIS 2021.2 is a continuous delivery release of InterSystems IRIS. There are two streams of InterSystems IRIS releases:
Continuous delivery releases — these releases provide access to new features and are ideal for developing and deploying applications that are continuously being updated and can take immediate advantage of new features.
Extended maintenance releases — these releases are less frequent than the continuous delivery releases but provide the increased stability of maintenance releases. These releases are ideal for large enterprise applications where the ease of getting fixes in maintenance releases is more important than getting early access to new features.
Both CD and EM releases are provided on all supported platforms (see Supported Platforms Guide) including UNIX, Windows, the cloud platforms, and the OCCI-compliant container format. Previous CD releases were only available on cloud platforms and the OCCI-compliant container format.
In addition to providing fully-supported releases, InterSystems provides access to prerelease software for developers who want to get an early look at new features.