New and Enhanced Features for Caché 2017.2

This section includes:

Parallel Dejournaling For Mirroring and Journal Restore

In this release, the throughput of dejournaling is improved in Mirroring and Journal Restore. This is aimed at improving the scalability of mirrored systems that incur high rates of database updates.

Dejournaling is the process of applying records from journal files to the databases. Prior to this release, a single job performed all of the database updates in a dejournal session, aided by a series of prefetcher jobs. In this release, multiple updater jobs (up to four of them) can work in parallel to apply records to different databases. This feature is used if there are sufficient CPUs and shared memory heap available .

In Mirroring, use of parallel dejournaling can be restricted by member type with a new configuration setting. By default, the feature is off for Reporting Async members. If reports include data from multiple databases that is in the process of being updated by dejournaling, the differences in timing of updates to different databases could result in more variability in the results of those reports than in previous versions. The character of that variability is not substantially different than what could be expected in any report run against changing data, so it is expected that most reporting applications could use this feature with no negative impact.

For details on parallel dejournaling, see “Configuring Parallel Dejournaling” in the “Mirroring” chapter of the Caché High Availability Guide.

For journal restore, parallel dejournaling is not used if certain non-default options are selected. In particular, it is not used in conjunction with the option to abort after any error, nor with the option to journal applied updates. For more information, see “Restore Globals From Journal Files Using ^JRNRESTO” in the “Journaling” chapter of the Caché Data Integrity Guide.

Neither shadowing nor system startup make use of parallel dejournaling.

Finally, this release includes two improvements that apply regardless of whether parallel dejournaling is used. This release includes internal improvements to dejournal prefetching efficiency and it limits the memory consumption for the "dejournal queue" to 50MB per updater; prior releases may have used substantially more memory for the dejournal queue.

DeepSee New Features

DeepSee Folder Manager — To simplify the deployment of DeepSee components, there is a new Folder Manager option to export related items. When exporting to a container class, this new option will not only export the selected items but also other items that are related to the selected items. Related items for a dashboard include the pivot tables and termlists used by the dashboard. Related items for a pivot table include named filters, pivot variables, and shared calculated members.

Prior to this release the Folder Manager always used the server file system for exporting/importing files. Now folder manager provides the option of using the local file system or the server file system

Dashboard Filters — Named filters can now be used as the default value for dashboard filters.

iFind and iKnow New Features

iFind now supports cooccurrence search, allowing you to look for records where the search terms appear close to one another, but not necessarily in a particular order, as with the more strict positional search option. Use square brackets around a comma-separated list of search terms to enable cooccurrence search, optionally including a range within which the search terms need to occur. For example, the query [Boston, New York, 5] will look for records where “Boston” and “New York” appear within at most 5 positions of one another.

This change also includes a couple of performance improvements for building iKnow domains, especially when leveraging systems with high core counts. Depending on your hardware and dataset, you may see improvements from 10 to 30% in overall processing time.

There are a few extensions to the iKnow REST APIs. For example, you can now retrieve just the result count or just the count along with full results for most endpoints querying domain data.

SQL Enhancements

This release includes the following SQL enhancements:

SQL Query Auditing

This release adds the ability to audit the execution of SQL queries. There are three new system events for auditing SQL queries:

%System/%SQL/DynamicStatement — for Dynamic SQL Queries
%System/%SQL/EmbeddedStatement — for Embedded SQL Queries
%System/%SQL/XDBCStatement - for xDBC Queries

To enable auditing for these events, go to the System Audit Events portal page by selecting System Administration > Security > Auditing > Configure System Events.

Query Optimizations

SQL queries have several improved optimizations. The query optimizer now considers outlier selectivity when calculating the selectivity of join conditions, resulting in improved plans for some queries. Outer joins can now take advantage of all optimizations available to inner joins. In particular, outer join conditions that could only partially be satisfied by an index no longer require the construction of a temporary file, often significantly improving query performance.

Optional ANSI SQL Operator Precedence

There is a new option to set SQL operator precedence to the ANSI SQL standard instead of the left-to-right CACHE SQL operator precedence. CACHE SQL operator precedence is still the default. You can change SQL operator precedence to ANSI SQL via an API or in the General SQL Settings portal page. Select System Administration > Configuration > SQL and Object Settings > General SQL Settings.

Frozen Plan Evolution

Previous releases of Caché included the ability to freeze SQL query plans and to automatically freeze query plans when you upgrade to a new version. If a new release provided improved optimization for a query plan, it would not be applied when you upgraded. With this release, Caché identifies any query with a frozen plan that might benefit from new optimizations and flags queries where the frozen plan may benefit from a new optimization. On the SQL Statement index page, these queries are identified in the New Plan column. You can then unfreeze these plans to take advantage of the new optimizations.

Where to Find Information on New Features in Atelier, the Eclipse-Based IDE

Atelier, the Eclipse-Based IDE for Caché, is available on an independent release cycle from Caché. Consequently, new features are described in the Atelier documentation provided with each new Atelier release. The Atelier IDE brings together the powerful and popular Eclipse development environment and the InterSystems Caché database. Atelier allows you to develop Caché applications using a modern file-based IDE on a client system. Atelier handles uploading the application to the Caché server where it can be run and debugged.

The focus of future development will be on the new Eclipse based IDE. Studio will remain an option to install and developers can continue to develop code with it. However, it will be treated as a maintenance product and will not see new functionality added as we move forward with Atelier. Some minor bugs may not be addressed either depending on resources required versus the severity of the issue.

Atelier is available as a separate download in addition to Caché or Ensemble. You can choose to install either a stand-alone Rich Client Platform (RCP) application, or a plug-in that can be added to an existing Eclipse installation. Users of the RCP application can add additional Eclipse plug-ins. Atelier uses the Eclipse auto-update mechanism to help users get the latest changes. For information on downloading Atelier and for the Atelier documentation, see http://www.intersystems.com/atelierOpens in a new tab, the Atelier home page.

Other Items of Note

In addition, many more minor improvements and corrections are also included. In particular, if you are upgrading an existing installation, please review the detailed list of changes in the Upgrade Checklist.

Areas of improvement include:

XML Schema Wizard provides an option allowing cascading delete of subclasses when the instance of the superclass is deleted.
Caché Nodejs supports node 7.
New option to force CSP buffer to flush even if the buffer is not yet filled.