New in HealthShare Health Connect 2024.1

SMART Scopes

InterSystems FHIR Server now supports SMART on FHIR OAuth2 access scopes using “CRUDS” (create/read/update/delete/search) style permissions. This enhancement aligns the application with the latest standards in healthcare interoperability and security, ensuring seamless integration with electronic health record systems and better data protection.

To learn more, refer to the HL7 documentation on scopes for using SMART to delegate access: https://hl7.org/fhir/smart-app-launch/scopes-and-launch-context.html#scopes-for-requesting-fhir-resourcesOpens in a new tab

For backward compatibility, InterSystems continues to support the 1.0 FHIR scopes.

JsonAdvSQL

InterSystems is excited to announce the release of JsonAdvSQL, a new FHIR search strategy designed to enhance performance and compliance. This release addresses critical limitations in the previous JSON strategy and introduces significant improvements.

JsonAdvSQL represents a significant step forward in FHIR search capabilities, offering faster, more compliant, and scalable solutions. This upgrade is poised to greatly enhance user experience and operational efficiency in managing healthcare data.

Some of the features of the enhanced conformance JsonAdvSQL provides are as follows:

• Multi-resource compartment searches with wildcards and _type

• Full support for _include / _revinclude and wildcards and :iterate

• Canonical searches now include version support

• New modifiers/prefixes and improved handling of implicit ranges

• Enhanced capability in text, identifier, and URI searches

• Better handling of Date/Quantity/Number search parameters, including new prefixes like ‘sa’, ‘eb’, and ‘ap’

• Improved expression of SearchInclude/SearchRevInclude wildcards and system vs. type level operations in CapabilityStatement

New FHIR Object Model

HealthShare Health Connect 2024.1 introduces a new FHIR Object Model to enhance the developer experience when working with FHIR data in ObjectScript. Key features of the new FHIR Object Model include:

• Static Class Representation: A package of %RegisteredObject classes that closely parallel the structure and naming conventions of FHIR resources and data elements. This provides better tooling support like IntelliSense, auto-completion, and documentation pop-ups when working with FHIR data structures.

• Convenience Methods: The model classes include convenient methods to allocate nested structures and collections on demand, following the naming conventions of the FHIR specification. This allows developers to synthesize FHIR payloads more easily, driven by IDE suggestions.

• Abstract Set API Support: The FHIR model classes support a subset of the Abstract Set API, providing a uniform coding pattern across dynamic (%DynamicAbstractObject) and static (%RegisteredObject) data models when working with FHIR data.

• Data Conversion: The classes include methods to convert between the static class model and the dynamic %DynamicAbstractObject representation, facilitating interoperability between the two models.

• Query Engine: The Abstract Set API integration allows developers to query FHIR data structures using the JSON Path Language (JPL), without the need for indexing or persisting the data.

The new FHIR Object Model aims to reduce the cognitive burden of working with the extensive FHIR data structures, providing better tooling support, consistent coding patterns, and efficient querying capabilities, ultimately enhancing the developer experience when building FHIR-based applications in HealthShare Health Connect. It is fully compatible with VS Code.

Interoperability Usage Metrics

InterSystems introduces Interoperability Usage Metrics in this release. With the substantial volume of data flowing through the interoperability engine, gaining insights into how the platform is utilized has become imperative for efficient operation and resource planning. Interoperability now provides a centralized monitor for counting the number of used interfaces for a given deployment. Users can access a granular breakdown of interface usage by date and namespace, this enables a more detailed understanding of how the system is utilized over specific periods and within specific contexts. The Interoperability Usage Metrics are conveniently accessible via API, facilitating the creation of comprehensive reports. API access empowers users to integrate usage data seamlessly into their reporting systems.

HL7 Production Validator

It can be expensive to bring a new interface live and hard to trust machine-generated parts of your logic (for example, logic produced by the HL7 Productivity Toolkit), especially when you lack a good testing facility. Interface verification is one of the major blockers for adopting new releases due to the cost and effort it requires. To ease the burden of upgrading systems, there is a need to have verification and trusted transformation logic to ensure correct system behavior.

To solve this common issue, InterSystems introduces the HL7 Production Validator to the 2024.1 release. The HL7 Production Validator is a terminal based utility that extracts HL7 Service headers, messages, and Operation messages to a temporary database. This production data is copied to your upgraded instance (the “target system”) and reconstituted for comparison against the original instance (the “source system”). Differences are evaluated based on recorded changes to the messages; these differences are represented to the user such that they can easily identify if the behavior is the same and identify where adjustments may be needed.

The HL7 Production Validator is particularly beneficial to customers who are migrating from HealthShare HealthConnect to InterSystems IRIS for Health or upgrading their current HealthShare Health Connect instance.

Using vectors in ObjectScript

As part of the new Vector Search capability (see related note in the Enhancing Analytics and AI section), InterSystems is making the $vector ObjectScript language feature available to all developers. InterSystems introduced $vector as an internal-only feature in 2022.2 to address the need for a new datatype and corresponding set of operators to work with large arrays of elements of a single datatype, and its highly efficient encoding format and use of SIMD functions for bulk operations contributed to the order-of-magnitude performance improvements brought by Columnar Storage. The Vector Search capability introduced with this release offers another use case for working with this kind of datatype, notably to capture embeddings and to perform fast bulk operations on the entire vector at once. Given the feature addresses two distinct use cases in modern computing challenges well, InterSystems has decided to make the feature available to all ObjectScript developers.

You can find full documentation of the new syntax and the current set of operators in the ObjectScript reference.

At first sight, you may think $vector has a lot of overlap with $list and wonder which one to use. They are, however, distinct features that server different use cases. In general, if you are dealing with a single datatype and always access and operate on data elements in bulk, you may fit the $vector use case.

Support for JSON_TABLE

HealthShare Health Connect 2024.1 introduces support for the JSON_TABLE function according to the SQL standard. This function, intended for use in the FROM clause of a query, maps JSON values to columns in a relational table, enabling you to query JSON data in the context of SQL and providing very powerful capabilities in combination with lateral joins.

SQL Development

The SQL standard includes a WITH clause that helps in breaking complex nested queries down into small blocks, similar to how you’d use SQL views, but restricted to a single statement. In 2024.1, you can now use such Common Table Expressions (CTEs) in SELECT statements to add more structure and transparency to your queries. Many tools that generate SQL statement can also use CTEs.

As with each release, HealthShare Health Connect 2024.1 includes a number of enhancements that improve query performance:

• When executing queries, any intermediate results that can be shared between different worker threads are now cached. This saves significantly on I/O when running complex queries on large datasets.

• Space efficiency for tables that use columnar storage where all table columns meet the requirements for pure columnar storage has been optimized.

• The Adaptive Parallel Execution mechanism introduced in prior releases now applies to a broader set of queries, including those that employ complex conditions, and virtually eliminates the cases in which traditional parallel subqueries would be used in the background.

• The set of cases where filter and other query predicates for statements involving foreign tables can be pushed down to a remote database has been expanded. This makes foreign tables more performant by limiting the amount of data that gets passed back to your instance of HealthShare Health Connect.

Vector Search

Vector Search is a new experimental feature in HealthShare Health Connect 2024.1 that provides the core functionality for semantic and vector-powered search of structured data, the cornerstone of generative AI applications, such as those implementing the Retrieval Augmented Generation (RAG) pattern. Vector search in 2024.1 comprises a new VECTOR SQL datatype and a set of related functions, including TO_VECTOR (which converts strings to vectors) and both VECTOR_DOT_PRODUCT and VECTOR_COSINE (which compute similarity between vectors).

In a typical application, specialized statistical language models called “embedding models” (smaller versions of LLMs like ChatGPT) convert sequences of text (or images, if the embedding model has been trained to process images) into embedding vectors, dense numeric arrays that specify a point in high-dimensional “latent embedding” space. Vectors are then stored in a SQL table column and the similarity functions are used to compare them to an input or query vector, the result of applying the same embedding model to the query string. The results of that search are the most similar vectors, and this can be a higher quality search capability than traditional “keyword” search because the embedding models have been trained to capture the “semantics” of the text. Future releases of InterSystems IRIS will increase the speed of search and introduce SQL syntax that abstracts the process of converting stored text as well as queries into vectors thereby providing a convenient way to implement powerful “semantic search” over data in InterSystems IRIS without having to deal with the underlying vectors directly.

Business Intelligence Improvements

InterSystems continues to improve the performance of InterSystems IRIS BI. In this release, the Analyzer user interface’s filter list management has been enhanced. In addition, this release includes:

• Handling of deeply nested AND/OR conditions in the Advanced filters in Analyzer. This change introduces a means for computing structures consisting of deeply nested CROSSJOIN and %OR functions in MDX slicer clauses.

• Enhanced APIs to allow accessing pivot table and dashboard metadata to allow for other visualization user interfaces to access that stored data.

Multi-Volume Databases

As customer databases grow, so do database files. To avoid those files becoming unmanageably large, or hit hard filesystem limits, HealthShare Health Connect now supports splitting your database across multiple physical “volumes” transparently. This new capability is easily configured: for any database, you can now configure a threshold size. When your initial .DAT file is about to hit this size and new global data needs to be written, HealthShare Health Connect will transparently create a new “database volume” file and start writing new data to that file. When that volume hits the threshold, another file is created, and so on.

There is no impact on applications and code accessing the data, as they continue to see the full database's content, no matter how many volumes it might be spread across. Furthermore, you can configure which directory the next volume should be created in, and when needed, you can rearrange database volumes across directories as a maintenance operation. Combined with planned work to increase the overall maximum database size, this will ensure your data remains easy to manage, well into the petabyte range.

Fast Online Backup

InterSystems for a long time has offered two main backup options: External Backup and Online Backup. With External Backup, users can freeze the write daemon for a brief period of time such that external solutions can create a consistent snapshot of all database, journal and other files. Those external solutions may track what changed versus the previous snapshot to produce an incremental snapshot that is much smaller than a full copy. In Online Backup, HealthShare Health Connect tracks which blocks changed, and can produce incremental backups while the system remains fully online (with only a very brief pause in writing).

This release introduces a new mechanism for incremental and full backups to the Online Backup feature, writing separate backup files per database using parallel processes. For large systems with many databases, this provides very significant improvements in the overall time it takes to perform a backup. The new Backup.Online class is the entry point for the new capability.

Minimum Supported CPU Models

HealthShare Health Connect 2024.1 now has set a minimum CPU instruction set policy for Intel and AMD (amd64/x86_64) processors. InterSystems now requires all CPUs to have the AVX and BMI instructions, which are generally available on the following CPU architectures:

• For Intel processors: Haswell and up

• For AMD processors: Steamroller and up

InterSystems is taking advantage of newer instructions to improve product performance. HealthShare Health Connect 2024.1 takes advantage of the AVX instruction to reliably speed up vector operations.

Operating System Updates

HealthShare Health Connect 2024.1 will support Ubuntu 24.04 shortly after the OS is generally available.

MacOS Sonoma is added as a supported version of MacOS.