New and Enhanced Features for InterSystems IRIS 2021.1
This topic describes the new and enhanced features in the 2021.1 release of InterSystems IRIS® data platform, which is an extended maintenance (EM) release. Some of these features were also available in the continuous delivery (CD) releases since 2020.1, the previous EM release.
With InterSystems IRIS 2021.1, customers can deploy InterSystems IRIS Adaptive Analytics, an add-on product that extends InterSystems IRIS to deliver greater ease of use, flexibility, scalability, and efficiency to analytics end users regardless of their business intelligence (BI) tools of choice. It enables defining an analytics-friendly business model and transparently accelerates analytic query workloads that run against this model by autonomously building and maintaining interim data structures in the background.
Other enhancements for Analytics use cases include:
SQL users querying InterSystems IRIS directly now have access to SQL standard Window Functions for easily expressing complex aggregations in a single query. This enables pushing data-intensive calculations typical for reporting and BI use cases closer to the data and improves both net performance and simplicity.
InterSystems IRIS embedded BI capability will experience measurable performance improvements thanks to enhancements to its native MDX query engine.
Enhancing Developer Experience
InterSystems IRIS 2021.1 introduces a new Python and R Gateway to run code in those languages out-of-process or on a different server as needed. The Python Gateway supports virtual environments, meaning that each Gateway can use its own version of Python to maximize developer flexibility. All gateways now support starting automatically on first use, as well as simple and secure re-entrant connections. This means, for example, that external code can transparently reach back into IRIS for data access without having to explicitly open a new connection and provide user credentials. The .NET Gateway now supports .NET Core 2.1 (first in 2020.3).
New in InterSystems IRIS 2021.1 is the ability to define external stored procedures in SQL. This enables developers to leverage code written in Java, Python or .NET from SQL, using simple SQL syntax. The external code is invoked transparently through the corresponding Gateways. This release also adds SQL user-defined aggregate functions with CREATE AGGREGATE and DROP AGGREGATE.
This release also adds a number of significant enhancements and extensions to the client APIs, including a fully native Python client SDK that runs on all platforms supporting Python, and support for $list and $order operations in all four Native API languages (Python, Java, C# and Node.JS). The ODBC driver has been enhanced with additional T-SQL support. XEP adds support for deferred indexing and indexes can be built as a background process (first in 2020.3).
Java developers can now take advantage of Java SE 11 LTS. InterSystems tests and supports both the Oracle OpenJDK and AdoptOpenJDK implementations of the standard for all of its Java-based components. See the corresponding section in the Supported Technologies list for more detail (first in 2020.4).
The InterSystems IRIS JDBC driver now fully supports Connection Pooling for efficient managing of database connections from your Java applications. See the Connection Pooling section in the JDBC documentation for more detail (first in 2020.4).
For ObjectScript developers, the VSCode-ObjectScript Version 1.0 is available. VSCode-ObjectScript is an open source extension for the VSCode IDE to enable practical development of ObjectScript applications for InterSystems IRIS. For details, see the VS Code InterSystems ObjectScript documentation.
This release has support for Spark 2.4.4 (first in 2020.3).
IntegratedML Machine Learning
This release that includes IntegratedML, a new feature that brings “best of breed” machine learning to analysts and developers via simple and intuitive SQL syntax (first in 2020.3). Developers can now easily train and deploy powerful predictive models from within IRIS, right where their data lives. For details, see Using IntegratedML and Learn IntegratedML in InterSystems IRIS.
For this release:
Community Edition containers are also available from Docker Hub.
Kits (and container tarballs) are available from the WRC Software Distribution site.
Full installation kits are provided for a subset of server platforms on the WRC. When using the installer, you must specify a custom install and select the IntegratedML option to install it on your system.
This release provides the following enhancements to the deployment and operations experience, both in the cloud and on-premises:
The InterSystems Kubernetes Operator (IKO) packages IRIS-specific knowledge and best practices into an easy-to-use, automated tool for provisioning and operating dynamic clusters. Starting with 2021.1, IKO also supports deploying InterSystems System Alerting & Monitoring (SAM).
This release include asynchronous mirroring support for sharded clusters (first in 2020.3). Users can now configure mirroring (synchronous or asynchronous) on an existing cluster, or fail over the entire cluster to the set of asynchronous mirror members in another data center in Disaster Recovery scenarios. See the corresponding section in the Scalability Guide for more details (first in 2020.4).
The InterSystems SQL syntax has been extended with a set of new commands for managing and configuring your database from a SQL prompt. This enables users with just JDBC or ODBC access to perform most administrative tasks without requiring access to the System Management Portal or an ObjectScript terminal prompt. It includes common tasks such as building indices and managing frozen plans. For details, see “BUILD INDEX,” “FREEZE PLANS,” “PURGE CACHED QUERIES,” “CREATE INDEX,” and new options in “SET OPTION” (first in 2020.4).
You can now manage Work Queues from the System Management Portal (first in 2020.3).
The newly available iris-lockeddown container is a security-hardened container image that implements many security best practices, offering peace of mind for customers deploying sensitive applications in complex environments. Users of the Web Gateway container will be pleased to see improvements to its default configuration.
Starting with 2021.1, InterSystems IRIS is now available for ARM platforms, both as full kits and pre-packaged containers. This enables customers to deploy their applications to cost-efficient hardware platforms, both physical and in the cloud. For more information, refer to the Supported Platforms guide.
This release simplifies the deployment of InterSystems Reports, the new reporting capability for InterSystems IRIS (first in 2020.4). As part of a closer integration, InterSystems Reports now uses the same user accounts as InterSystems IRIS for managing, building and executing reports. In addition, all configuration and management data for InterSystems Reports uses InterSystems IRIS if the setup scripting is used. A script to complete the initial configuration of InterSystems IRIS Report Server for on-prem deployments and a docker-compose file for Docker deployments of the Reports Server are both available as part of this release.
With InterSystems IRIS 2021.1, customers can deploy InterSystems API Manager (IAM) 2.3, which includes many enhancements broadening the reach of this crucial component in a modern API-centric environments.
There are the following other interoperability enhancements:
Improved X12 error handling (First in 2020.2).
This release adds support for a new "foreach" action, which can be used within Routing Rules used for segmented virtual documents (ASTM, EDIFACT and X12). The foreach action is supported in the Rule Type "Segmented Virtual Document Message Routing Rule". The foreach action can loop over repeating segments in the virtual document and nested loops are supported. This enables developers to build rules that match certain conditions regardless of the position of a segment within a repeating group. For details, see About Actions (first in 2020.4).
Block-level Compression Reduces the Overall Storage Footprint (Experimental Feature)
Block-level compression reduces the overall storage footprint (amount of disk required). Depending on the data cardinality (extent of repetition in data values), compression can reduce storage consumption significantly. Sparsely populated data (lots of zeros and spaces in the data) compress much better.
Block-level compression is an experimental feature in InterSystems IRIS 2020.2. This means that it is not supported for production. The feature is well tested, and compression can provide substantial space savings; however there may be an impact on performance. In some cases performance may increase as less data needs to be moved from disk to memory; in others it could decrease because of the computation to do compression and decompression. Space savings and performance impact will depend on the application and platform.
InterSystems is looking for feedback around the space saving and performance overhead seen on customers' real systems — in test environments. Please contact the Worldwide Response Center (WRC) if you are measuring this in your environment.
Currently, the three compression types provided are zlib, zstd, and lz4. Only data and big string blocks are compressed, although this could be extended to other block types in the future. A given database block is compressed only if the compression will allow freeing at least one 4KB chunk of space on disk. Block compression is only supported on Linux systems that support sparse files.
Other Enhancements and Efficiency Improvements
In each release, InterSystems makes many efficiency improvements and minor enhancements. This release includes these improvements:
You can now use Proof Key for Code Exchange (PKCE) with OAuth authentication. PKCE enables you to securely perform the OAuth exchange from public clients and mitigates the threat of having the authorization code intercepted. PKCE is supported in both the OAuth clients and servers.
The configuration and utility functions in $SYSTEM.SQL have now been organized thematically in subclasses such as %SYSTEM.SQL.Functions, %SYSTEM.SQL.Schema and %SYSTEM.SQL.Stats.Runtime, making their signature and behavior more consistent across the board. The old entry points in %SYSTEM.SQL have been deprecated but are still available for backwards compatibility (first in 2020.3).
You can now use Transact-SQL through JDBC. Please see the Transact-SQL Migration Guide for more on hosting Transact-SQL applications on InterSystems IRIS (first in 2020.3).
Node.js Native API now includes the List class. See “Native API Quick Reference for Node.js” (first in 2020.3).
Java Messaging Service (JMS) adapter is able to connect to a broader range of servers. (first in 2020.3)
InterSystems IRIS on Linux has been enhanced to use Asynchronous I/O for writes to database files, as it always has on all UNIX® and Windows platforms (first in 2020.3). This is coupled with automatic use of direct I/O instead of buffered I/O. This change optimizes the disk I/O characteristics for database files in the following ways:
Improves application responsiveness at higher scaling levels by more fairly sharing I/O bandwidth with database reads and journal writes.
Improves integrity check performance by allowing integrity check to read multiple blocks asynchronously.
Improves effectiveness of asynchronous reads performed by $prefetchon.
InterSystems SQL saw a number of performance enhancements that will speed up many different types of queries when upgrading to this release. A complex customer benchmark composed of millions of queries ran 6% faster on 2020.4 and 2021.1 compared to earlier releases.
This release introduces a new algorithm for estimating field selectivity when gathering table statistics. This improves the ability of the SQL optimizer to choose the fastest query plan for any given SQL statement (first in 2020.4).
For a number of applicable scenarios, the SQL engine will now use a kernel-level iterator to read through temporary results. This simplifies generated code and can speed up certain steps in a query plan, such as sorting, by up to 40% (first in 2020.4).
Recent updates to browser security have changed handling of third-party cookies. These updates use the SameSite attribute to reduce the risk of cross-site request forgery (CSRF) attacks, unauthorized access to data, and other possible security issues. Chrome (starting with v.84) enforces stricter rules for SameSite behavior, and these rules can cause issues with existing websites and web applications. These issues may include login problems, the login page being displayed repeatedly, and page elements not displaying properly. In previous versions, you could not modify the SameSite attribute; hence web applications running on these versions may have such issues.
This release (first in 2020.4) solves these problems by setting the SameSite attribute for cookies and by allowing you to set change the default setting; however, you may need to modify your code to customize values setting the session cookie scope and the user cookie scope. Additionally, if you are using “SameSite=None”, you must ensure that your web application can support secure HTTPS connections. For details, see About the SameSite Attribute.
This release improves performance on newly installed systems where the database cache size has not been configured. Under most circumstances, you should carefully configure cache sizes and Configure Huge and Large Pages for optimal system performance. Configuring cache sizes is especially important for live production systems, systems with heavy loads, and systems with multiple instances. See Memory Usage Changes for Global and Routine Buffers for details. (first in 2020.3)