Upgrade Checklist for Caché 2018.1
The purpose of this chapter is to highlight those features of Caché 2018.1 that, because of their difference in this version, affect the administration, operation, or development activities of existing systems.
Customers upgrading their applications from earlier releases are strongly urged to read the upgrade checklist for the intervening versions as well. This document addresses only the differences between 2017.2 and 2018.1.
Before upgrading Caché, read “Upgrading Caché” in the Caché Installation Guide for instructions. If you are upgrading from a version earlier than Caché 2012.1, pay special attention to Supported Upgrade Paths in the Caché Installation Guide.
The maintenance release Caché 2018.1.5 introduced the following upgrade checklist items:
The maintenance release Caché 2018.1.4 introduced the following upgrade checklist items:
The maintenance release Caché 2018.1.3 introduced the following upgrade checklist items:
The maintenance release Caché 2018.1.1 introduced the following upgrade checklist items:
This section contains information of interest to those who are familiar with administering prior versions of Caché and wish to learn what is new or different in this area for version 2018.1. The items listed here are brief descriptions. In most cases, more complete descriptions are available elsewhere in the documentation.
Increase in Memory Usage (Version 2018.1.1 and later)
Maintenance release 2018.1.1 provides performance and scalability enhancements for large-scale systems. These enhancements increase the amount of memory allocated for global buffer metadata by 64 bytes per buffer on Intel systems and by 128 bytes per buffer on IBM Power systems. For example, with 8K buffer sizes, the shared memory allocated for a global buffer increases by 0.75% on Intel systems and by 1.5% on IBM Power systems. There is also an increase in per-process memory usage for systems using large numbers of subscript level mappings.
Allocating Space for Write Image Journaling (WIJ) Files
This release contains a new parameter, Target size for the WIJ, in the system configuration journal settings. Setting this target size ensures that disk space is allocated for the WIJ early in the start-up process. If sufficient space is not allocated early and there is not enough available space for the WIJ, the system may encounter problems. Allocating space for WIJ is an advanced configuration setting. If you encounter issues with this, contact the InterSystems Worldwide Resource Center.
Changes to JRNDUMP Utility Navigation
The JRNDUMP utility has improved navigation. If you have a script that responds to JRNDUMP prompts, you should check the script to ensure that it handles the changed navigation.
This section holds items of interest to users of specific platforms.
Windows Upgrade Installation Does Not Create USER Namespace if Previously Deleted
In previous releases, if, on a Windows system, you upgraded Caché where the USER namespace had been deleted, the installation would create a new USER namespace. In this release, the installation will not create a USER namespace if it was previously deleted from the instance. This is typically the desired behavior, but, if you have a script that automatically deletes the USER namespace after an upgrade, you may need to modify the script. The Windows install now matches the behavior of Caché installations on other platforms, that is, if the instance does not have a USER namespace, the upgrade does not create one.
This section contains information of interest to those who have designed, developed and maintained applications running on prior versions of Caché.
The items listed here are brief descriptions. In most cases, more complete descriptions are available elsewhere in the documentation.
No classes have been removed in this release that were present in the previous version.
The following methods were present in the previous version and have been removed in this release:
No properties have been removed in this release that were present in the previous version.
No parameters have been removed in this release that were present in the previous version.
No indices have been removed in this release that were present in the previous version.
The following methods have different signatures in this version of Caché:
2017.2: modifiedAddress () As (none)
2018.1: modifiedAddress (NewVal, Type, useZen) As (none)
2017.2: modifiedAgentAddress () As (none)
2018.1: modifiedAgentAddress (NewVal, Type, useZen) As (none)
2017.2: exportPDF (printMultiple, preserveTempFiles) As (none)
2018.1: exportPDF (printMultiple, preserveTempFiles, filename) As (none)
2017.2: printSVGContent (svgFrameId, parms, svgContent) As (none)
2018.1: printSVGContent (svgFrameId, parms, svgContent, filename) As (none)
2017.2: setupPASV (&Device:%String) As %Boolean
2018.1: setupPASV (&Device:%String, *ServerName:%String) As %Boolean
2017.2: setupPORT (&Device:%String) As %Boolean
2018.1: setupPORT (&Device:%String, *ServerName:%String) As %Boolean
2017.2: OutputHeaders () As (none)
2018.1: OutputHeaders () As %Status
2017.2: Test (host:%String, username:%String, password:%String, dir:%String="/etc", spec:%String="", dotFiles:%Boolean=1) As %Status
2018.1: Test (host:%String, username:%String, password:%String, dir:%String="/etc", spec:%String="", dotFiles:%Boolean=1, &t) As %Status
2017.2: GetUserId () As %String
2018.1: GetUserId (PID:%String, JobNumber:%String) As %String
Debugger on AIX Breaks After True Post-Conditional
When you are stepping through a routine on an AIX system and a command post-conditional is true, the debugger generates an additional <BREAK> before executing the command.
The cterm application now accepts terminal input directly as Unicode instead of ANSI characters. This change should not impact terminal usage and behavior, but it is possible that there may be differences in some locales. This change allows Caché to support additional languages.
DeepSee Cubes Require Recompile (Version 2018.1.1 and later)
After upgrading to Caché 2018.1.1, you must recompile all DeepSee cubes. The 2018.1.1 Maintenance Release contains a fix to the %DeepSee.Generator that requires this recompile. If you build a cube without recompiling, it may cause an error.
Complex MDX Queries Are Rejected (Version 2018.1.1 and later)
In previous releases, a complex MDX query with nested %OR setfunctions could produce erroneous results. This release detects the constructs, blocks them, and issues an error. It is possible to produce the intended logical result with a simpler construct. You should rewrite these queries so that any %OR contains only AND terms.
Recompile Measure-Specific Listings
This release contains a correction to handle listing filters within compound cubes. You must recompile measure-specific listings to get this fix.
The %DeepSee.Utils:%GetCubeMeasures() method should return the caption as the second item in the output list. For calculated measures it was returning the name. In this release, it correctly returns the caption. If your code depended on the previous incorrect behavior, you should modify it.
Language Binding Changes
ID Notation in XEP (Version 2018.1.1 and later)
In C# classes used for XEP, developers can specify an id annotation. In previous releases, this could cause bulk updates to fail. To avoid this problem, Caché now generates id_property. In some cases, schemas with the generated id annotation may encounter a problem. If you encounter this problem, remove the annotation and the behavior will be unchanged from the previous version.
Issue Using C++ Binding with Output Redirection
In order to perform output redirection, the C++ binding stores and resets the $ZR value. If $ZR points to a database that the user does not have write access, this encounters a protect error.
Changes to HttpRequest Security
In this release, %Net.HttpRequest is enhanced to support SPNEGO, Kerberos and NTLM authentication, as well as Basic authentication, for HTTP clients. This new authentication process for %Net.HttpRequest changes the existing %Net.HttpRequest support for Basic authentication. In previous releases, setting the Username property will as a side-effect return a Basic Authorization header. This behavior will only continue for HTTP 1.0 responses. For HTTP 1.1 responses an unsolicited Authorization header will never be sent unless the InitiateAuthentication property is set. If the current Basic authentication support is used, it will result in a 301 status response with a WWW-Authenticate header indicating Basic and possibly other schemes. HttpRequest then sends another request with the proper Authorization header. Existing code will continue to work but there will be an extra round-trip
SQL Count Function Returns BIGINT
The SQL COUNT function now returns a BIGINT instead of an INT. This has the following impact:
The type of value returned by COUNT() has been changed from INTEGER to BIGINT.
The LENGTH reported to xDBC for COUNT() has changed from 4 to 8.
The PRECISION reported to xDBC for COUNT() has changed from 18 to 19.
System Statistics Changes (Version 2018.1.1 and Later)
To improve performance of large-scale systems, the statistics reported by the system utilities have changed. In most cases this will not cause any compatibility issues; however, if your code explicitly examines the statistics output for specific data, you may have to update your code to adjust for these changes. The following minor changes are visible:
The ^GLOSTAT utility no longer offers detailed block type statistics and no longer asks the question "Should detailed statistics be displayed for each block type? No =>". Additionally a bug is corrected in ^GLOSTAT which was present since 2016.1.0 and caused remote reads and remote cache efficiency to be incorrect (usually zero).
The Management Portal System Dashboard is enhanced to show private global references and updates and WIJ writes, in the same way that ^GLOSTAT does. The detailed block type statistics page, 'Disk and Buffer Statistics', is no longer available.
The ability to zero statistics (from ^GLOSTAT and the Management Portal) is not available. This capability is not compatible with the more efficient statistics collection. Most counters are now 64-bit and timed collection is available, so there is no need to zero the counters.
The class SYS.Stats.Disk, which accesses the detailed block type statistics, reports all as zero; the class reference is updated to reflect this. The class SYS.Stats.Global, which accesses the main global statistics, continues to function as before and is enhanced to include write image journal writes as a property called 'WIJWrites'. Additionally, the internal class SYS.Metrics was previously exposed (unintentionally) as a public API; it is now properly marked as internal-only and it no longer reports values for detailed block type statistics.
The WS-Monitoring and BMC Patrol facilities that report the detailed block type statistics, report them zero or null. BMC Patrol, WMI, and SNMP monitoring facilities lock counts (total, success, and fail) also report as zero.
The cstat -g1 is updated to include more fields that were previously only available in ^GLOSTAT displays. Some statistics that were available via the cstat -c option are no longer available.
%Monitor.Process, MONLBL, PERFMON and similar performance monitoring tools are unaffected by this change and maintain their own counters.
New Compatibility Changes in Caché and Ensemble 2018.1.5
SQL — System ID and IDENTITY Datatypes 2018.1.4 Change Reverted to %Integer
In maintenance release 2018.1.4, the datatype of system-generated ID properties and IDENTITY fields were changed from %Integer to BIGINT. This change in maintenance release 2018.1.5 reverts that change and returns to the previous behavior where the type is %Integer. If your code expects these ID fields or IDENTITY columns to be of type BIGINT, you need to modify your code. This change is only an issue if you are upgrading from 2018.1.4 to 2018.1.5 or later.
CSP — Improved Handling of SameSite HTTP Option
Recent updates to browser security have changed handling of third-party cookies. These updates use the SameSite attribute to reduce the risk of cross-site request forgery (CSRF) attacks, unauthorized access to data, and other possible security issues. Chrome (starting with v.84) enforces stricter rules for SameSite behavior, and these rules can cause issues with existing websites and web applications. These issues may include login problems, the login page being displayed repeatedly, and page elements not displaying properly. In previous versions, Caché did not support modifications to the SameSite attribute; hence web applications running on these versions may have such issues.
This release solves these problems by setting the SameSite attribute for cookies and by allowing you to set change the default setting; however, you may need to modify your code to customize values setting the session cookie scope and the user cookie scope. Additionally, if you are using “SameSite=None”, you must ensure that your web application can support secure HTTPS connections.
Ensemble Adapters — QueueWaitTime Changes
In this release Ensemble changed the method it uses to calculate queue wait time and includes the amount of time the active message is waiting. In some cases this could mean that waiting for a message will exceed the queue wait time, where in previous releases the wait would not have exceeded the limit.
Zen Mojo — Classes That Use Highcharts Libraries Need to Update References
If you are using Zen Mojo classes that use Highcharts 4.0.1 or Highcharts 4.0.4 libraries, you must update your code to use Highcharts 8.0.4.
New Compatibility Changes in Caché and Ensemble 2018.1.4
The maintenance release Caché 2018.1.4 introduced the following upgrade checklist items:
CSP Gateway Installer — How the Gateway Installer Treats an Existing CSP.ini File
In environments where a third party Web Server is running remotely, the Web Gateway installer tried to modify some settings in CSP.ini which could result in unwanted changes in terms of service accessibility. A review of the configuration after each upgrade was always recommended. In this release the Web Gateway installer will not modify an existing CSP.ini in any way. For details on configuring the web server and Web Gateway, see “Configuring the Web Server and Web Gateway.” This change first appeared in maintenance release 2018.1.4.
CSP Gateway — Gateway and Private Web Server WebSockets Changes
In the maintenance release 2018.1.4, the Web Gateway's implementation of WebSockets with Apache now uses the wakeable pollset feature of the Apache Portable Runtime (APR) library. Because of this dependency, this release of the CSP Gateway will cause problems in two scenarios.
On Apache 2.2, WebSockets will no longer work. Any attempt to use WebSockets with Apache 2.2 will result in immediate closure of the connection and an error message in the Event Log saying that WebSockets are not supported in the current build of the CSP Gateway. To avoid this issue, customers should upgrade to Apache 2.4. The reason for removing WebSockets support from Apache 2.2 builds of the CSP Gateway is that some releases of Apache 2.2 are linked to an old version of APR that does not support wakeable pollsets.
On Oracle Solaris, if Apache 2.4 was built or configured to link to APR 1.6.3 or earlier, then WebSocket messages cannot be sent or received reliably. This affects the Private Web Server as well, which, as of 2018.1.4, links to APR 1.5.2. This is due to a Solaris-specific bug in the wakeable pollset functions in APR, detailed at https://bz.apache.org/bugzilla/show_bug.cgi?format=multiple&id=61786. To avoid this issue, customers on Solaris should upgrade their builds of Apache to a version that is linked to APR 1.6.4 or higher. A quick way to check the version of APR is to view the CSP Gateway Event Log and read the header in one of the entries. For example, the following header snippet indicates APR 1.5.2:
>>> Time: Tue Jan 28 11:25:44 2020; RT Build: 1801.1631m (solaris/apapi:srv=2.4.20/apr=1.5.2/apu=1.5.4/mpm=worker)
This change first appeared in maintenance release 2018.1.4.
CSP Gateway — Change to How Web Sockets Use Licenses
There are two ways in which this change to how web sockets use licenses may impact custom code.
In some cases, web sockets were not counted correctly for licenses. If your application was approaching the license limit, this change could increase the number of licenses being used.
In order to not continue to consume a license after a CSP session times out, for SharedConnection=1 web sockets, if the CSP session times out then the web socket times out and cannot be used. In previous releases, the web socket would continue to be active.
DeepSee — Timing Changes in ResultSet
This changes the way that the %DeepSee.ResultSet operates during concurrent executions to preserve the integrity of results. Applications using the %DeepSee.ResultSet where the optimistic concurrency worked correctly before this fix may start noticing results are not ready when going to read them. If this is occurring, make sure the results are complete by checking for:
before attempting to read results. This change first appeared in maintenance release 2018.1.4.
DeepSee — %dsCellContext in KPI/Plugin Code Changes
This change fixed the cell context passed to a %KPI plugin's %dsCellContext environment variable. Applications that directly apply the contents of %dsCellContext to an MDX query will not see a change in behavior. But applications which parse the contents of %dsCellContext to be used in further action should verify that the custom code still handles the original MDX clauses. This change first appeared in maintenance release 2018.1.4.
Ensemble — HL7 Changes to Events for Unexpected Segment Names
In order to see information about unexpected HL7 Segment names one must now set ^Ens.Debug("TraceCat","parse") to 1 — this replaces possible event log warnings. Message validation can still be used to enforce schema segment names. This change first appeared in maintenance release 2018.1.4.
Object Library — %Net.SSH.Session.Disconnect() Method Should be Called with Error Checking
To avoid a problem the %Net.SSH.Session.Disconnect() method checks to see if the disconnection can be done successfully. If it cannot, it returns the “Session in use” error and does not perform the disconnection. Consequently, if you call Disconnect(), you should handle this possible error in your code.
SQL — IFNULL Function May Require Explicit CAST
A problem has been corrected with the SQL IFNULL function where an argument to IFNULL was not properly converted from Display or Odbc format to logical format if the argument was a host variable. This does create a slight compatibility issue with a statement like:
Select IFNULL(DOB,'null', DOB) from Sample.Person
Where DOB is type DATE.
For this statement, IFNULL assumes all input arguments are of type DATE, but obviously 'null' is a string. Caché does not support an implicit conversion from string to null for this case.
To fix this problem, you should use CAST to convert 'null' to DATE:
Select IFNULL(DOB,cast('null' as DATE), DOB) from Sample.Person
This change first appeared in maintenance release 2018.1.4.
SQL — System ID Datatypes are Now BIGINT
The datatype of system-generated ID properties and fields is now BIGINT, where in previous releases it had been of type INTEGER. This is also true for corresponding reference fields. If your code expects these ID fields to be of type INTEGER, you need to modify your code. This change first appeared in maintenance release 2018.1.4. Note that this change is reverted in 2018.1.5.
New Compatibility Changes in Caché and Ensemble 2018.1.3
In rare cases, minor compatibility changes are introduced in a maintenance release. Caché 2018.1.3 contains the following compatibility changes:
Security — LDAP Changes
The fields which govern which LDAP server you connect to for authentication have been moved from the Security.System class to the Security.LDAPConfigs class. If you use the Security.System set of APIs to modify your LDAP settings, you will need to update your code to use the new Security.LDAPConfigs class which contain the same LDAP properties. The set of LDAP properties in the Security.System class have been deprecated, and will no longer affect the operation of the system, and will eventually be removed. If you upgrade an existing system which uses LDAP Authentication, then the properties are automatically moved to the new Security.LDAPConfigs class, and the system will continue to run normally after the upgrade without needing to change anything.
If you are currently using "Allow Operating System LDAP Authorization" for terminal sessions, and turn on "Allow multiple security domains", then you will no longer be automatically authenticated against the LDAP server, since you no longer know which server your operating system process was authenticated against. You will now be prompted for a username and password. The one exception to this is if the environment variable ISC_LDAP_CONFIG is defined to point to an existing LDAP configuration, then that LDAP server will be used for your authorization.
Since the domains database is no longer used, it is removed from the security database. All the methods in Security.Domains have been deprecated, and when called perform no action.
In this change, when "Allow multiple security domains" is enabled, it relaxes the check that when a user is created in the security database that the domain piece must already exist in the Security.Domains database. So now you can create a Caché password user like firstname.lastname@example.org, and the domain example.com does not need to previously exist in the Security.Domains database.
Security — Login Rule Change Audit Event Removed
The %System/%Security/LoginRuleChange audit event has been removed. If you had code which modified this specific event using Security.Events(), it will not longer have any affect on the system, and the Modify() method will return an error. It is unlikely that user code accessed this event.
System — Maximum Frame Depth Reduced on UNIX 32-bit systems
To fix a process termination when a <FRAMESTACK> error was encountered when handling a previous <FRAMESTACK> error, the change reduced the maximum frame depth for Unicode systems. This change is unlikely to cause problems unless your code created conditions that were very close to the previously allowed frame depth. This change only impacts UNIX 32-bit systems.
Language Bindings.ActiveX — Action Required to Fix Lock Problem
If your system has run into the ActiveX Lock problem described in Fix ActiveX locking issue in the Caché & Ensemble Maintenance Release Changes and if your system has these locks that have not been freed, you should execute the following kill command in all effected namespaces after installing this change:
Installation — Upgrading a Windows System Will Not Re-create USER Namespace if it was Deleted
In previous releases, if you upgraded a Windows system and the USER namespace had been deleted from the instance, the installation would create a new USER namespace. With this change, if the USER namespace has been deleted from the instance, the upgrade installation will not create a new USER namespace.
System — Support Longer Strings in Legacy %ZEN.Auxiliary.jsonProvider
In previous releases, the %ZEN.Auxiliary.jsonProvider would produce a <MAXSTRING> error if the JSON properties were larger than 32K or 3.8M (with long strings enabled). This change supports these longer strings. However, to fix this issue, the returned %ZEN.proxyobject may contain streams. Consequently, you should check the return object for streams before accessing it.
Ensemble Production — Custom Subclasses of EnsLib.HTTP.GenericMessage May Require Changes
EnsLib.HTTP.GenericMessage now explicitly defined XMLNAME parameter as well as XMLTYPE. Any custom sub classes of EnsLib.HTTP.GenericMessage need to account for XMLNAME paramter now being inherited and override as required.
Ensemble Production — New X12 Validation May Impact Previously Ignored Flags
The behavior for existing X12 validation flags in routers is unchanged. However, in the unlikely event that you have specified extra validation flags that were not previously valid, the behavior may change. If you specified a nonexistent validation flag, it would have been ignored, but if the flag now matches a new validation flag, the specified validation will be performed.
For a list of the new validation flags, see “Validation” in Routing X12 Documents in Productions.
Ensemble Production — Changes in Message Bank Storage for HTTP Messages
HTTP, REST and SOAP Generic Messages are now banked as XML Export projection and not as the unique case for Ens.StreamContainer. It is possible to resend Generic messages and Ens.StreamContainer messages from the Message Bank.
Ensemble Production — Ens.Utl.XML.SecuritySignature Changes
Ens.Util.XML.SecuritySignature no longer extends %XML.Security.Signature. Any use of Ens.Util.XML.SecuritySignature other than ValidateSAML() will need to be changed to use %XML.Security.Signature.
Ensemble Production — Developers Using Studio Need Additional Privilege
Developers using Studio may need to be granted execute privilege on stored procedures: Ens_Config.Production_Extent.
Ensemble Production — SOAP Outbound Adapter Changes
The EnsLib SOAP Outbound Adapter clears the HTTPHeaders of the %Client property after each Invoke, InvokeMethod or InvokeWithSOAPBody call. If not using SOAP Wizard generated classes that lead to %Client property being re-initialized before each SOAP call then it may be necessary to set again any http headers that were expected to be maintained in custom code.