This chapter provides the following information for Caché 2017.2:
New and Enhanced Features for Caché 2017.2
This section includes:
Parallel Dejournaling For Mirroring and Journal Restore
In this release, the throughput of dejournaling is improved in Mirroring and Journal Restore. This is aimed at improving the scalability of mirrored systems that incur high rates of database updates.
Dejournaling is the process of applying records from journal files to the databases. Prior to this release, a single job performed all of the database updates in a dejournal session, aided by a series of prefetcher jobs. In this release, multiple updater jobs (up to four of them) can work in parallel to apply records to different databases. This feature is used if there are sufficient CPUs and shared memory heap available .
In Mirroring, use of parallel dejournaling can be restricted by member type with a new configuration setting. By default, the feature is off for Reporting Async members. If reports include data from multiple databases that is in the process of being updated by dejournaling, the differences in timing of updates to different databases could result in more variability in the results of those reports than in previous versions. The character of that variability is not substantially different than what could be expected in any report run against changing data, so it is expected that most reporting applications could use this feature with no negative impact.
For details on parallel dejournaling, see “Configuring Parallel Dejournaling” in the “Mirroring” chapter of the Caché High Availability Guide.
For journal restore, parallel dejournaling is not used if certain non-default options are selected. In particular, it is not used in conjunction with the option to abort after any error, nor with the option to journal applied updates. For more information, see “Restore Globals From Journal Files Using ^JRNRESTO” in the “Journaling” chapter of the Caché Data Integrity Guide.
Neither shadowing nor system startup make use of parallel dejournaling.
Finally, this release includes two improvements that apply regardless of whether parallel dejournaling is used. This release includes internal improvements to dejournal prefetching efficiency and it limits the memory consumption for the "dejournal queue" to 50MB per updater; prior releases may have used substantially more memory for the dejournal queue.
DeepSee New Features
DeepSee Folder Manager — To simplify the deployment of DeepSee components, there is a new Folder Manager option to export related items. When exporting to a container class, this new option will not only export the selected items but also other items that are related to the selected items. Related items for a dashboard include the pivot tables and termlists used by the dashboard. Related items for a pivot table include named filters, pivot variables, and shared calculated members.
Prior to this release the Folder Manager always used the server file system for exporting/importing files. Now folder manager provides the option of using the local file system or the server file system
Dashboard Filters — Named filters can now be used as the default value for dashboard filters.
iFind and iKnow New Features
iFind now supports cooccurrence search, allowing you to look for records where the search terms appear close to one another, but not necessarily in a particular order, as with the more strict positional search option. Use square brackets around a comma-separated list of search terms to enable cooccurrence search, optionally including a range within which the search terms need to occur. For example, the query [Boston, New York, 5] will look for records where “Boston” and “New York” appear within at most 5 positions of one another.
This change also includes a couple of performance improvements for building iKnow domains, especially when leveraging systems with high core counts. Depending on your hardware and dataset, you may see improvements from 10 to 30% in overall processing time.
There are a few extensions to the iKnow REST APIs. For example, you can now retrieve just the result count or just the count along with full results for most endpoints querying domain data.
This release includes the following SQL enhancements:
SQL Query Auditing
This release adds the ability to audit the execution of SQL queries. There are three new system events for auditing SQL queries:
%System/%SQL/DynamicStatement — for Dynamic SQL Queries
%System/%SQL/EmbeddedStatement — for Embedded SQL Queries
%System/%SQL/XDBCStatement - for xDBC Queries
To enable auditing for these events, go to the System Audit Events portal page by selecting System Administration > Security > Auditing > Configure System Events.
SQL queries have several improved optimizations. The query optimizer now considers outlier selectivity when calculating the selectivity of join conditions, resulting in improved plans for some queries. Outer joins can now take advantage of all optimizations available to inner joins. In particular, outer join conditions that could only partially be satisfied by an index no longer require the construction of a temporary file, often significantly improving query performance.
Optional ANSI SQL Operator Precedence
There is a new option to set SQL operator precedence to the ANSI SQL standard instead of the left-to-right CACHE SQL operator precedence. CACHE SQL operator precedence is still the default. You can change SQL operator precedence to ANSI SQL via an API or in the General SQL Settings portal page. Select System Administration > Configuration > SQL and Object Settings > General SQL Settings.
Frozen Plan Evolution
Previous releases of Caché included the ability to freeze SQL query plans and to automatically freeze query plans when you upgrade to a new version. If a new release provided improved optimization for a query plan, it would not be applied when you upgraded. With this release, Caché identifies any query with a frozen plan that might benefit from new optimizations and flags queries where the frozen plan may benefit from a new optimization. On the SQL Statement index page, these queries are identified in the New Plan column. You can then unfreeze these plans to take advantage of the new optimizations.
Where to Find Information on New Features in Atelier, the Eclipse-Based IDE
Atelier, the Eclipse-Based IDE for Caché, is available on an independent release cycle from Caché. Consequently, new features are described in the Atelier documentation provided with each new Atelier release. The Atelier IDE brings together the powerful and popular Eclipse development environment and the InterSystems Caché database. Atelier allows you to develop Caché applications using a modern file-based IDE on a client system. Atelier handles uploading the application to the Caché server where it can be run and debugged.
The focus of future development will be on the new Eclipse based IDE. Studio will remain an option to install and developers can continue to develop code with it. However, it will be treated as a maintenance product and will not see new functionality added as we move forward with Atelier. Some minor bugs may not be addressed either depending on resources required versus the severity of the issue.
Atelier is available as a separate download in addition to Caché or Ensemble. You can choose to install either a stand-alone Rich Client Platform (RCP) application, or a plug-in that can be added to an existing Eclipse installation. Users of the RCP application can add additional Eclipse plug-ins. Atelier uses the Eclipse auto-update mechanism to help users get the latest changes. For information on downloading Atelier and for the Atelier documentation, see http://www.intersystems.com/atelier, the Atelier home page.
Other Items of Note
In addition, many more minor improvements and corrections are also included. In particular, if you are upgrading an existing installation, please review the detailed list of changes in the Upgrade Checklist.
Areas of improvement include:
XML Schema Wizard provides an option allowing cascading delete of subclasses when the instance of the superclass is deleted.
Caché Nodejs supports node 7.
New option to force CSP buffer to flush even if the buffer is not yet filled.
Caché 2017.2 Upgrade Checklist
The purpose of this chapter is to highlight those features of Caché 2017.2 that, because of their difference in this version, affect the administration, operation, or development activities of existing systems.
Those customers upgrading their applications from earlier releases are strongly urged to read the upgrade checklist for the intervening versions as well. This document addresses only the differences between 2017.1 and 2017.2.
If you are upgrading from Caché 2008.2 or an earlier version, you cannot upgrade directly to Caché 2017.2, but must first upgrade to a version between Caché 2009.1 and Caché 2016.2. For details, see Supported Upgrade Paths in the Caché Installation Guide.
The upgrade instructions listed at the beginning of this document apply to this version.
This section contains information of interest to those who are familiar with administering prior versions of Caché and wish to learn what is new or different in this area for version 2017.2. The items listed here are brief descriptions. In most cases, more complete descriptions are available elsewhere in the documentation.
This section details changes that have an effect on the way the system operates.
Changes to Permissions Required for Studio Access to a Namespace
In this release, Studio must have a login role that allows database access to the default database for the namespace in order to connect to a namespace. If Studio does not have such a role, it will get an error when it attempts to connect to the namespace. In previous releases, Studio access was allowed if the login role had any SQL privilege to the default database for the namespace.
Feature Tracker Enabled by Default for New Installs
Feature Tracker is a software utility in Caché that gathers statistics on software module usage. The statistics track whether or not software modules are present and used in a given Caché instance. Feature Tracker sends this information via https to InterSystems weekly. These statistics help us plan development and support. The information gathered does not include any application data. In previous releases, Feature Tracker was disabled by default. In this release, Feature Tracker is enabled by default for new installs. If you are upgrading an existing installation, the upgrade preserves the existing state of Feature Tracker.
If you wish to disable Feature Tracker, you suspend the task that runs it. For details, see “How to Deactivate Feature Tracker” in the Caché System Administration Guide.
Parallel Dejournaling Changes the Order of Dejournaling
This release includes parallel dejournaling, which makes it possible for up to four jobs to perform the dejournaling. A database is always dejournaled by a single job, so parallel dejournaling does not impact the order in which a single database is dejournaled. But it can change the order in which one database is dejournaled relative to the dejournaling in another database. This is because databases being updated by separate dejournaling jobs are likely to be at slightly different places in the dejournaling sequence; for example, database A may contain updates made on the primary at 11:45:30 when database B is only up to the updates from 11:45:28. For details, see “Configuring Parallel Dejournaling” in the “Mirroring” chapter of the Caché High Availability Guide.
Changes to Journal Buffer Pool Size
In this release, the journal buffer pool size is configurable and the default has been increased to 64MB. In previous releases, the default was 8MB on non-Unicode systems and 16MB on Unicode systems. An increase in this buffer pool size can improve efficiency, but it also increases the amount of data that may be lost if a system crash occurs.
This section holds items of interest to users of specific platforms.
There are no known platform-specific changes.
This section contains information of interest to those who have designed, developed and maintained applications running on prior versions of Caché.
The items listed here are brief descriptions. In most cases, more complete descriptions are available elsewhere in the documentation.
The following classes were present in the previous version and have been removed in this release:
The following methods were present in the previous version and have been removed in this release:
The following properties were present in the previous version and have been removed in this release:
No parameters have been removed in this release that were present in the previous version.
No indices have been removed in this release that were present in the previous version.
The following methods have different signatures in this version of Caché:
old method: Build (pDataBaseName:%String) As (none)
new method: Build (pDataBaseName:%String) As %Status
old method: SaveData (What, File, Username1, Password1, Username2, Password2, Description) As %String
new method: SaveData (What, File, Username1, Password1, Username2, Password2, Description, KeyLen) As %String
old method: DrawLogContent (filename) As %Status
new method: DrawLogContent (tmp) As %Status
old method: %GetKPIValue (pKPIName:%String, *pValue:%String, pKPIProperty:%String="", pSeries:%String="", &pFilters:%String, pCellContext:%String="", &pCacheKey:%String, *pPctComplete:%Integer, pParentQueryKey:%String="") As %Status
new method: %GetKPIValue (pKPIName:%String, *pValue:%String, pKPIProperty:%String="", pSeries:%String="", &pFilters:%String, pCellContext:%String="", &pCacheKey:%String, *pPctComplete:%Integer, pParentQueryKey:%String="", *pKPIStatus:%Status) As %Status
old method: getFilterInfo (fnames, fvalues) As (none)
new method: getFilterInfo (fnames, fvalues, flabels) As (none)
old method: %ExecuteModelInternal (pInput:%DeepSee.PMML.ModelInput, *pOutput:%DeepSee.PMML.ModelOutput) As %Status
new method: %ExecuteModelInternal (&pInput:%DeepSee.PMML.ModelInput, *pOutput:%DeepSee.PMML.ModelOutput) As %Status
old method: CalculateXBeta (pObservation:%DeepSee.PMML.ModelInput, *pXBeta, *pBestTarget:%String="", *pBestScore:%Double="", pAddZero:%String="") As %Status
new method: CalculateXBeta (&pObservation:%DeepSee.PMML.ModelInput, *pXBeta, *pBestTarget:%String="", *pBestScore:%Double="", pAddZero:%String="") As %Status
old method: %PrepareObject (pQuery:%DeepSee.Query.query) As %Status
new method: %PrepareObject (pQuery:%DeepSee.Query.query, &pVariables) As %Status
old method: %GetDimensionMembers (pCubeName:%String, pSpec:%String, pContext:%List, *pMembers, pMaxMembers:%Integer=100, *pMemberClass:%String, &pRelatedFilters, pCalcMode:%Integer=0, pSearchKey:%String="") As %Status
new method: %GetDimensionMembers (pCubeName:%String, pSpec:%String, pContext:%String="", *pMembers, pMaxMembers:%Integer=100, *pMemberClass:%String, &pRelatedFilters, pCalcMode:%Integer=0, pSearchKey:%String="") As %Status
old method: %OnNew (pName:%String="", pCode:%String="", pLocation:%String="", pData:%String="", pInnerException:%Exception.AbstractException=$$$NULLOREF, pStack:%String="") As %Status
new method: %OnNew (pName:%String="", pCode:%String="", pLocation:%String="", pData:%String="", pInnerException:%Exception.AbstractException=$$$NULLOREF, pStack:%String) As %Status
old method: GetGlobalSizeBySubscript (Directory:%String, StartNode:%String, EndNode:%String="", &Size:%String=0) As (none)
new method: GetGlobalSizeBySubscript (Directory:%String, StartingNode:%String, EndingNode:%String="", &Size:%String=0) As (none)
old method: CompileAll (Flags:%String=0, IO:%String=$p, &Count:%Integer, &Errors:%Integer, MultiCompile:%Integer=1, Journal:%Integer=1, KeepSource:%Boolean=1) As %Status
new method: CompileAll (Flags:%String=0, IO:%String=$p, &Count:%Integer, &Errors:%Integer, MultiCompile:%Integer, Journal:%Integer, KeepSource:%Boolean=1) As %Status
old method: CompileSelected (Mask:%String="*.*", Flags:%String=0, IO:%String=$p, &Count:%Integer, &Errors:%Integer, MultiCompile:%Integer=1, Journal:%Integer=1, KeepSource:%Boolean=1) As %Status
new method: CompileSelected (Mask:%String="*.*", Flags:%String=0, IO:%String=$p, &Count:%Integer, &Errors:%Integer, MultiCompile:%Integer, Journal:%Integer, KeepSource:%Boolean=1) As %Status
old method: SQLClassname (qh:%Library.SQLProcContext, table:%String) As %Library.String
new method: SQLClassname (qh:%Library.SQLProcContext, table:%String(MAXLEN=257)) As %Library.String
old method: SetSQLStats (flag:%Library.Integer=1) As %Library.Integer
new method: SetSQLStats (flag:%Library.Integer=0) As %Library.Integer
old method: dumpSVGNode (e, svgDoc, src, intro, coda, NSPrefix, maxWidth) As (none)
new method: dumpSVGNode (e, svgDoc, src, intro, coda, NSPrefix, maxWidth, maxHeight) As (none)
old method: extractXSLFOSource (svg, intro, coda, maxWidth) As (none)
new method: extractXSLFOSource (svg, intro, coda, maxWidth, maxHeight) As (none)
old method: GetSRC (filetype:%String, url:%String, sslConfig:%String="", username:%String="", password:%String="") As %String
new method: GetSRC (filetype:%String, url:%String, sslConfig:%String="", sslCheckServerId:%Boolean, username:%String="", password:%String="") As %String
old method: SaveLast (filetype, url, sslConfig="") As (none)
new method: SaveLast (filetype, url, sslConfig="", sslCheckServerId) As (none)
old method: Sample (DaemonID:%Integer) As SYS.Stats.Resource
new method: Sample (DaemonID:%Integer=1) As SYS.Stats.WriteDaemon
Class Compiler Changes
Hard Errors in delete() are Reported with %Status Value
In previous releases, if the delete() method in the class compiler encountered a hard error, it would throw the error to the caller. In this release, these errors are handled by the delete() method, which reports the error to the caller with a %Status value.
Change Default to Display Mode in DeepSee Listing Selection
In previous releases, if a DeepSee field was not explicitly declared %EXTERNAL(field) or %INTERNAL(field), DeepSee would display its internal value in the listing output. In this release DeepSee will use its external value. To have the internal value used, you must wrap the field in %INTERNAL().
Java and Gateway Changes
Do not use JMS Gateway as Message Listener
This release removes the JMSGateway class from the Java Gateway. This class was intended as an example on how to use the JMS listener in Ensemble. If you have used this class to implement a JMS listener, you should replace this with the mechanism demonstrated in the EnsLib.JavaGateway.JMSTest class.
Object Gateway Load^%apiGTW Does Not Make err Variable Public
In previous releases, the call to Load^%apiGTW would deliberately make the variable err public so that it could be used in the Management Portal and other InterSystems APIs. In this release, it does not automatically make it public but instead returns a status value. If your code depended on this previous behavior, you must explicitly define err as a public variable by calling:
.NET Language Bindings
Replace SysList, SysListUtil, and CacheListRO Classes Used With ODBCx APIs
If you use the SysList, SysListUtil, and CacheListRO Classes directly with the ODBCx APIs, replace them with other classes that provide similar functionality. Although the classes have been deprecated, they have not yet been deleted. If you use these classes but not use them with any other ADO APIs, you can defer replacing them until it is convenient. But if you use them with the ODBCx APIs, you must replace them before using with this release.
%RunJava Utility Class Removed
In this release, the %RunJava utility class has been removed. If you call methods in this class, you must replace them with calls to the utility methods in %Net.Remote.Service.
Limit Number of Subscripts for Indirection
In this release, applications are limited to 254 subscripts for indirection. It is unlikely that there is any existing code doing indirection with more than 254 subscripts. Such code would create nodes that are not easily accessible.
Changes in API to Create QuickStatement
The QuickStatement class syntax has changed and the class was moved from cache-jdbc to cache-db jar. The new syntax is
QuickStatement qs = QuickStatement.createQuickStatement((CacheConnection) connection);
If you use the previous syntax, “QuickStatement qs = (QuickStatement) rs.getObject("**QuickStatement**");”, you must update your code to the new syntax.
Changes in %Date LogicalToOdbc and LogicalToDisplay
In previous releases if a timestamp value was passed through %Date LogicalToOdbc or LogicalToDisplay, it was not altered. As part of a fix to support dates prior to December 31, 1840, this behavior has changed. In this release if a timestamp value was passed through %Date LogicalToOdbc or LogicalToDisplay, it removes the time portion of the value and returns only the date portion. If your application logic depends on the previous behavior, you may need to modify your code. For example, if you query data in Display or Odbc mode from a table linked to Oracle and use a function like CAST(field AS DATE) or TO_DATE(...), Oracle returns a datetime value. In this release, Caché converts it to a DATE.
Enforce REFERENCES Privileges for Foreign Keys
In previous releases, it was possible to create a foreign key constraint through a DDL statement without holding the REFERENCES privilege for the referenced table. In this release, for a user to define a foreign key constraint that references table T, the user must hold the REFERENCE privilege on table T, or the REFERENCE privilege on the column(s) of table T that are being referenced by the foreign key. If an application creates foreign keys without holding REFERENCES privileges, it will encounter errors.
Change in oref Numbers Ordering
In this release, we have improved the efficiency of processing OREFs. As a consequence, the OREF number assignments, which were previously assigned in the order created are now assigned in an independent way. Consequently, if your code relies on the way in OREF numbers were assigned, you must modify the logic. The previous ordering was an implementation detail and was not a documented feature. For example, if OREF string values are used to index a Caché multidimensional array, the subscript order of the OREF string values will no longer have any relation to the order in which those OREF values were created. If an application is using OREF value order to control order of execution or order of data handling, then that application will need to be modified to use other techniques, like $INCREMENT or $SEQUENCE, to generate the desired ordering.
Similarly, this change affects the order of items in relationships (for example, the order of children within a parent). Note, however, that it is not supported to rely on the order of items in relationships.
Console Log Changes
In some cases, long console log messages that were broken into two messages in previous releases will now be written as a single message. If you have code that parses for specific messages in the console log, you should update it.
SOAP Wizard Includes Option to Generate Shorter Array Names
In previous releases, the default type name for an array item includes both the item name and type name or the key name and type name, even when the names are identical. In this release, by default, if the item name and type name are the same, only one is included in the default array type name. Similarly, if the key name and type name are identical, only one is included in the default array type name. For example, in previous releases, an XML schema (or a WSDL for a web service) might include something like the following:
<element minOccurs="0" name="PropName" type="s01:ArrayOfSimpleObjectSimpleObject" xmlns:s01="mytypes"/>
As of this release, the schema or WSDL will instead include this:
<element minOccurs="0" name="PropName" type="s01:ArrayOfSimpleObject" xmlns:s01="mytypes"/>
In some cases, you can revert to the previous default behavior:
The AllowRedundantArrayName property of %XML.Schema allows the old form in an XML schema. If true, the previous longer type names are generated.
The ALLOWREDUNDANTARRAYNAME parameter of a web service class allows the old form of array name in the XML schemas in the WSDL for the web service. If this parameter is true, then the previous longer type names are generated.
There is no way to restore the previous default of long array names, if you are generating the schema by using the XMLSchema() method of %XML.Adaptor. The schema would need to be created using the %XML.Schema class.
Change to Invoking Web Methods Using HTTP Requests
In previous releases you could call a Caché SOAP web service directly without making an HTTP SOAP request. This shortcut avoids using a SOAP client but it bypasses the security available using HTTP SOAP requests. This shortcut is used by the catalog and test pages that Caché generates for a SOAP web service class. InterSystems recommends that you not use this shortcut but always use a SOAP web client to generate an HTTP SOAP request to call Caché SOAP web services. If you have code that uses the %SOAP.WebServiceInvoke class in a URL to invoke a SOAP web service, InterSystems recommends that you replace this with an HTTP SOAP request generated by a SOAP client.
Although it is not recommended, you can continue to use this shortcut mechanism, but you must explicitly enable this access. To use the %SOAP.WebServiceInvoke class and the soap_method query parameter, you must:
Enable %SOAP.WebServiceInfo and WebServiceInvoke with commands such as:
set ^SYS("Security","CSP","AllowClass",webapplicationname,"%SOAP.WebServiceInfo")=1 set ^SYS("Security","CSP","AllowClass",webapplicationname,"%SOAP.WebServiceInvoke")=1Copy code to clipboard
Where webapplicationname is the web application name with a trailing slash, for example, "/csp/mynamespace/".
Ensure that the user accessing the CSP page has USE permission for the %Development resource.
XSLT2 Transformation Returns Error Status for Fatal Errors
In previous releases, when an XSLT2 transformation encountered a fatal error, it returned $$$OK. In this release, it correctly returns an error status. If your code relies on the default error handling, you may have to modify it to handle this change. If you have overridden the default with custom error handling, the custom error handling will continue to work as it did in previous releases.
Change in XML Output for Property with Element Qualified
In previous releases, if a class with ELEMENTQUALFIED=1 had a property which is a literal datatype with ELEMENTQUALIFIED, the property was not output with a namespace specified. This change corrects this, and, in Caché 2017.2 and later releases, the property is output by XMLExport or %XML.Writer with a namespace qualification.
For example, in previous releases, the XML output for a property would be:
In Caché 2017.2, the XML output for this property is:
<s01:MyProp xmlns="" xmlns:s01="http://my.namespace.com/test">