Skip to main content

Caché® and Ensemble® Change Notes (2018.1.9)

This document is meant to help you assess the impact of upgrading to the Caché and Ensemble 2018.1.9 maintenance release. It lists the changes since 2018.1.0, grouped by category.

Important:

For information about upgrading an installed instance of Caché or Ensemble, and procedures that you may need to follow, see the “Upgrading Caché” chapter in the Caché Installation Guide.

InterSystems News, Alerts, and Advisories

From time to time, InterSystems publishes items of immediate importance to users of our software. These include alerts, mission-critical issues, important updates, fixes, and release announcements. You can obtain the most current list at https://www.intersystems.com/support-learning/support/product-news-alerts/Opens in a new tab. InterSystems recommends that you check this list periodically to obtain the latest information on these issues.

Incompatibilities in This Maintenance Release

DP-11573: SQL Import/Export wizard now restricts file extensions

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2018.1.5

With this change, the SQL Import/Export wizard ([System Explorer] -> [SQL] -> [Wizards] -> [Data Import/Export]) will only allow the user to import from or export to files on the server with an allowed extension. Allowed extensions are: .txt, .csv. Previously any file on the system could be used. When importing from or exporting to a file on the client system, filenames are not restricted.

DP-11630: Eliminate on unload behavior from svgWidgetProvider and lock down behavior of RemoveFile() call

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP
Platforms: All
Version: 2018.1.0

As of Chrome 80 synchronous HTTP events will no longer be honored as part of page exit processing. Previously this widget made use of such events to perform implicit server-side clean up of temporary files created during the PDF generation process. This feature is no longer supported.

All temp files used by this widget reside in the instanceHome/mgr/Temp directory, which is cleaned up periodically by normal maintenance processes. Applications that do not wish the temp files to linger until regular clean-up may explicitly remove individual files with the RemoveFile() method supplied by this widget. Note however, that RemoveFile() will only remove temp files created by this widget during the current CSP session and is not a generic utility.

DP-11979: Web sockets with SharedConnection=1 can now time out

NOTE: This item may require a change to code, configuration, or operation.

Category: Web Gateway
Platforms: All
Version: 2018.1.4

In previous releases, a web socket connection with SharedConnection=1 would never time out. With this change, this connection will time out if it is inactive for the CSP session timeout.

DP-12747: SSLCC_Protocol parameter changes for TLS 1.3

NOTE: This item may require a change to code, configuration, or operation.

Category: Web Gateway
Platforms: All
Version: 2018.1.5

To allow for TLS 1.3, there are changes to the way you access the SSLCC_Protocol parameter. If you are using an external script or the Web Gateway registry methods (particularly %CSP.Mgr.GatewayMgr.GetServerParams or %CSP.Mgr.GatewayMgr.SetServerParams) to read/write the SSLCC_Protocol parameter in CSP.ini, you must adapt your code to use the SSLCC_Protocol_Min and SSLCC_Protocol_Max parameters instead.

DP-13047: Support FIPS mode for libcrypto.so.1.1 in Red Hat Linux 8.2

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: Linux
Version: 2018.1.5

On Red Hat Linux, you can only use libcrypto.so.1.1 FIPS mode on Red Hat Linux version 8.2 and later. You cannot use this mode on earlier versions.

DP-13698: Changes to TCP TLS in FIPSMode

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2018.1.5

When FIPS mode is enabled, TCP TLS will use operating system's libraries and the TLS1.1 and earlier protocols won't be supported. This change should not impact existing code but, if your existing TCP TLS code uses FIPSMode, you should test it to ensure that there are no subtle behavior changes.

DP-285619: Change class names for storage classes

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Library
Platforms: All
Version: 2018.1.9

Updated the following storage class names:

%CacheStorage -> %Storage.Persistent
%CacheSQLStorage -> %Storage.SQL
%CacheSerialState -> %Storage.Serial
%CacheShardStorage -> %Storage.Shard
%Compiler.Storage.Cache -> %Compiler.Storage.Persistent
%Compiler.Storage.CacheExtent -> %Compiler.Storage.Extent
%Compiler.Storage.CacheSerial -> %Compiler.Storage.Serial
%Compiler.Storage.CacheSQL -> %Compiler.Storage.SQL
%Compiler.Storage.Extent -> %Compiler.Storage.CustomExtent
%Compiler.Storage.Serial -> %Compiler.Storage.CustomSerial
%Compiler.Storage.Generator.Cache -> %Compiler.Storage.Generator.Persistent

Added conversion of the old to the new name in the class dictionary upgrade. This is performed automatically in %Dictionary class save, and logic for UDL import now also performs this dictionary upgrade.

The original names have been removed. If customer classes subclass these old names they should update to the new names.

DP-288880: Change in queue wait time calculation

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2018.1.5

To improve the efficiency of the Ens.MonitorService, this release changes the way the queue wait time is calculated. For business hosts with a pool size of 1, the amount of time the active message has taken to be processed is now counted s part of the queue waiting time. In rare cases, this could change how InterSystems IRIS behaves in responding to the QueueWaitTime alert setting.

DP-404283: Changes to enabling %ZEN.Dialog classes

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2018.1.6

In this release, %ZEN.Dialog.* classes cannot be run in web applications unless the web application is enabled for analytics, the namespace is enabled for interoperability productions, or it is explicitly enabled for the web application. To explicitly enable %ZEN.Dialog.* classes, enter the following:

Set ^SYS("Security","CSP","AllowPrefix",application-name,"%ZEN.Dialog.")=1

Also, with this change, any web application that is enabled for analytics or interoperability will have %ZEN.Dialog.* classes enabled by default.

DP-405034: %Net.SMTP checks server identities by default

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Library
Platforms: All
Version: 2018.1.6

This change fixes a bug where the SSLCheckServerIdentity property in %Net.SMTP defaulted to false, but was documented as defaulting to true. The property now defaults to true and is in alignment with documentation. This means that, when connecting to an SSL/TLS secured web server, %Net.SMTP will check that the certificate server name matches the DNS name used to connect to the server and fail if they don't match. This is the behavior specified in RFC 2818 section 3.1.

As a result, when connecting to an SSL/TLS secured web server, the default behavior of %Net.SMTP will now be to fail if the certificate server name does not match the DNS name used to connect to the server.

This change is unlikely to cause compatibility issues, but it is possible that when using %Net.SMTP to send messages you might have issues connecting to an SSL/TLS enabled server. If this happens, and you understand the security trade-offs, you can set SSLCheckServerIdentity to 0 to restore the previous behavior.

DP-407254: Update interaction between delegated and password authentication

NOTE: This item may require a change to code, configuration, or operation.

Category: System
Platforms: All
Version: 2018.1.0

This fixes an issue with authentication ordering when delegated and password authentication are enabled. If your user authentication includes both delegated authentication and password authentication, you can choose either to call or not call ZAUTHENTICATE for password users. Typically, ZAUTHENTICATE is only used for delegated users and not for password users.

But if you want your ZAUTHENTICATE routine to also be called for password users, you should check the <b>Always try Delegated Authentication</b> in the <b>System Administration > Security > System Security > Authentication/Web Sessions Options</b> page in the Management Portal or by using the ^SECURITY utility (System Parameter Setup, Edit authentication options).

The default for <b>Always try Delegated Authentication</b> is No and ZAUTHENTICATE is not called for password users.

DP-413293: Zen Reports: do not allow use of $DATASOURCE URL parameter by default

NOTE: This item may require a change to code, configuration, or operation.

Category: Compatibility Features
Platforms: All
Version: 2018.1.0

Previously ZEN Reports would allow the report's datasource to be specified at runtime via the $DATASOURCE URL Parameter. If this is not defined, then we will use the DATASOURCE class parameter, and if that isn't defined we will use the ReportDefinition to generate the XML data. The ReportDefinition is the standard way to specify report data.

With this change, Zen Reports will no longer use the $DATASOURCE URL parameter by default. Developers are strongly encouraged to use the DATASOURCE class parameter to specify an external datasource if desired. If a report does require the $DATASOURCE URL, you can re-enable the previous behavior on a per-report or per-application basis by setting the parameter:

Parameter USEURLDATASOURCE = 1; 

in either the report class, or in the report's Application class. Setting this value is discouraged.

DP-416562: Correct WebSocket handling of low-level errors

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP Server
Platforms: All
Version: 2018.1.0

This fix corrects a bug in the error handling for web socket connections. If an error was encountered before the CSP server dispatched to the WebSocket class, then the error would be ignored. Now the CSP server will call the Error() method in the WebSocket class. Similarly, for authentication failures, the server will call Login(). Note that by default, Login() just calls Error(). 

The websocket will also now abort the connection if OnPreServer() returns an error. Previously errors were ignored.

DP-416563: Do not allow navigation to %-CSP pages from the /csp/sys/oauth2/ application

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP Server
Platforms: All
Version: 2018.1.0

This change modifies the default configuration to disallow access to %-CSP pages from the /csp/sys/oauth2/ application.

DP-417168: Record Mapper and EDI Document Viewer file view permissions check

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2018.1.0

The Interoperability Record Mapper and EDI Document viewer management portal pages enforce holding the privilege %Ens_ViewFileSystem:USE before allowing the user to select a file using the file select dialog popup. This change adds the same check before a file is opened.

DP-422560: HTML encode REST error messages

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2018.1.9

Previously, error messages returned from REST APIs were not HTML encoded. This issue has been corrected.

DP-5315: EnsLib.HTTP.GenericMessage now explicitly defines XMLNAME parameter as well as XMLTYPE

NOTE: This item may require a change to code, configuration, or operation.

Category: Ensemble
Platforms: All
Version: 2018.1.3

With this change, Ens.StreamContainer and its subclasses can be deserialized from XML. Ens.StreamContainer and subclasses EnsLib.HTTP.GenericMessage,EnsLib.SOAP.GenericMessage,EnsLib.REST.GenericMessage and Ens.MFT.StreamContainer can now be deserialized from XML. This means for example they can be resent from the Message Bank or tested in the DTL editor.

As part of the correction, EnsLib.HTTP.GenericMessage now explicitly defines XMLNAME parameter as well as XMLTYPE. If you have defined any custom subclasses, you need to modify them to account for this change; override the XMLNAME parameter as required for your code.

DP-6736: Maximum frame depth reduced on 32-bit Unicode

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: UNIX®
Version: 2018.1.3

This change fixes a problem caused if a second <FRAMESTACK> error is encountered when TRY/CATCH catches a <FRAMESTACK> and tries to create the error object. This condition lead to the process terminating. This was an issue only on non-Windows 32-bit Unicode platforms.

This change is unlikely to cause problems unless your code created conditions that were very close to the previously allowed frame depth.

This was an issue only on non-Windows 32-bit Unicode platforms.

DP-6920: New X12 validation may impact previously ignored flags

NOTE: This item may require a change to code, configuration, or operation.

Category: Ensemble
Platforms: All
Version: 2018.1.3

This change adds element-level validation for X12 documents.

The behavior for existing X12 validation flags in routers is unchanged. However, in the unlikely event that you have specified extra validation flags that were not previously valid, the behavior may change. If you specified a nonexistent validation flag, it would have been ignored, but if the flag now matches a new validation flag, the specified validation will be performed.

For a list of the new validation flags, see Validation in Routing X12 Documents in Productions.

DP-7204: Fix a number of WebSocket thread-safety issues

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP.Gateway
Platforms: All
Version: 2018.1.4

Fix a number of WebSocket thread-safety issues that resulted in occasional lost messages and dropped connections.

If you are using Nginx with the default dynamic modules configuration (using CSPx.so and CSPxSys.so), note that we are now deprecating this configuration. The dynamic modules configuration will still function as before, but everyone using the Web Gateway with Nginx is urged to rebuild and reconfigure Nginx to work with the Network Service Daemon (NSD) build of the Web Gateway instead. The NSD now supports WebSockets when used with Nginx, so migration to the NSD is now doable if you have a WebSocket application.

If you are already using Nginx with the NSD, you must still rebuild Nginx and edit its configuration to apply this change.

Migration instructions if you are already using Nginx with the NSD

  1. Install an updated version of the Web Gateway.
  2. Rebuild Nginx using the updated version of ngx_http_csp_module.c. See https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCGI_ux#GCGI_nginx_unnsd for instructions on how to do this.
  3. Add the CSPNSD_pass directive to the Nginx configuration file to indicate the hostname/IP and port where the NSD is listening. For example, if your configuration file contains
    location /csp
    { CSP On; }
    
    and the NSD is listening at 127.0.0.1:7038 (the default), then change the configuration block to
    location /csp { CSP On; CSPNSD_pass 127.0.0.1:7038; }
    
  4. Start up Nginx and the NSD.
Migration instructions if you are using Nginx with dynamic modules (CSPx.so, CSPxSys.so)
  1. Install an updated version of the Web Gateway.
  2. Rebuild Nginx using ngx_http_csp_module.c instead of ngx_http_csp_module_sa.c. See https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCGI_ux#GCGI_nginx_unnsd for instructions on how to do this.
  3. Remove the CSPModulePath directive from the Nginx configuration.
  4. Add the CSPNSD_pass directive to the Nginx configuration file to indicate the hostname/IP and port where the NSD is listening. For example, if your configuration file contains location /csp { CSP On; } and the NSD is listening at 127.0.0.1:7038 (the default), then change the configuration block to

    location /csp

    { CSP On; CSPNSD_pass 127.0.0.1:7038; }

  5. Start up Nginx and the NSD. See https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCGI_atypical_windows#GCGI_usingnsd and https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCGI_atypical_unix#GCGI_usingnsdunix for more information on how to manage the NSD.

This change also adds a few additional Nginx configuration directives that you should be aware of when migrating, and you should review whether their default values are acceptable for your workloads. If the default values are not acceptable, then specify your own value in the proper location{} block in the Nginx configuration file.

Syntax: CSPNSD_response_headers_maxsize size;
Default: CSPNSD_response_headers_maxsize 8k;
Context: location, if in location
Description: The maximum size of the HTTP headers in a response. An error is returned to the web client if a response header exceeds this size.

Syntax: CSPNSD_connect_timeout time; Default: CSPNSD_connect_timeout 300s; Context: location, if in location Description: Timeout for connecting to the NSD when a request is received from the web client.

Syntax: CSPNSD_send_timeout time; Default: CSPNSD_send_timeout 300s; Context: location, if in location Description: Timeout for a single send operation to the NSD when sending a request. The timeout is set only between two successive write operations, not for the transmission of the whole request.

Syntax: CSPNSD_read_timeout time; Default: CSPNSD_read_timeout 300s; Context: location, if in location Description: Timeout for a single read operation from the NSD when reading a response. The timeout is set only between two successive read operations, not for the transmission of the whole response.

An example configuration that uses all the new directives is the following.

location /csp

{ CSP On; CSPNSD_pass 127.0.0.1:7038; CSPNSD_response_headers_maxsize 8k; CSPNSD_connect_timeout 300s; CSPNSD_send_timeout 300s; CSPNSD_read_timeout 300s; }

DTB739: Add stronger concurrency protection to %DeepSee.ResultSet

NOTE: This item may require a change to code, configuration, or operation.

Category: DeepSee
Platforms: All
Version: 2018.1.4

This changes the way that the %DeepSee.ResultSet operates during concurrent executions to preserve the integrity of results. This provides much deeper inspection of the primary components of a particular query, requiring that each writer to the query cache secure a lock on the query key and all axis keys before being granted access to change anything in either the results or axis caches.

DTB818: Store initial axis text in axis cache for KPI/MDX context reference

NOTE: This item may require a change to code, configuration, or operation.

Category: DeepSee
Platforms: All
Version: 2018.1.4

The cell context passed to a %KPI plugin's %dsCellContext environment variable could lose useful information due to the fact that it was rebuilt from the pre-processed list of members that actually exist in the cube data. This variable now contains the original requested MDX text for the slicer.

PLATFORMS-XALANC12: Update xalan_c to version 1.12

NOTE: This item may require a change to code, configuration, or operation.

Category: Platforms
Platforms: All
Version: 2018.1.0

This change updates xalan_c to version 1.12.

SDK048: SQL Import/Export wizard now restricts file extensions

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2018.1.5

With this change, the SQL Import/Export wizard ([System Explorer] -> [SQL] -> [Wizards] -> [Data Import/Export]) will only allow the user to import from or export to files on the server with an allowed extension. Allowed extensions are: .txt, .csv. Previously any file on the system could be used. When importing from or exporting to a file on the client system, filenames are not restricted.

SGM031: Support SameSite for CSP session and user cookies

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP
Platforms: All
Version: 2018.1.5

When a server sets an HTTP(S) cookie, it can include an attribute called SameSite that affects the scope in which the cookie is sent. The options are as follows:

  • None - Send cookie with cross-site requests.
  • Lax - Send cookie with safe, top-level cross-site navigation.
  • Strict - Do not send cookie with cross-site requests.

Recent updates to browser security have changed handling of third-party cookies. These updates use the SameSite attribute to reduce the risk of cross-site request forgery (CSRF) attacks, unauthorized access to data, and other possible security issues. Chrome (starting with v.84) enforces stricter rules for SameSite behavior, and these rules can cause issues with existing websites and web applications. These issues may include login problems, the login page being displayed repeatedly, and page elements not displaying properly.

Before this change, it was not possible to modify the SameSite attribute, and web applications running on these versions may have such issues. This change allows you specify default values for the SameSite attribute, which will thereby prevent third-party cookie issues form arising.

With this change, the product will explicitly set SameSite both for CSP session cookies and any cookie set via the %CSP.Response.SetCookie() method.

    Two new fields have been added for customizing web applications:
  • Session Cookie Scope
  • User Cookie Scope
  • These fields are displayed in Management Portal's Edit Web Application window, alongside "Use Cookie for Session" and "Session Cookie Path."

    Session Cookie Scope controls the default SameSite value for session cookies associated with the given web application. User Cookie Scope controls the default SameSite value for user-defined cookies created with %CSP.Response.SetCookie(). Session Cookie Scope and User Cookie Scope can be set to None, Lax, or Strict. System web applications and new or upgraded user applications default to Strict for both new fields.

    The %CSP.Response.SetCookie() method also accepts SameSite as an argument, which overrides the default value. You can use this new argument to exercise more fine-grained control over your cookies.

    You should configure grouped applications to use the same Session Cookie Scope setting. Additionally, if you are using "SameSite=None", the web application needs to support secure (HTTPS) connections.

    Atelier

    CDS3103: Prevent Atelier from auditing entries for read-only database

    Category: Atelier
    Platforms: All
    Version: 2018.1.3

    Atelier could generate <PROTECT> error audit entries for a read-only databases. This change corrects this problem.

    Backup/Restore

    DP-411054: Free suspended write daemon properly when backup is aborted

    Category: Backup/Restore
    Platforms: All
    Version: 2018.1.0

    Before this fix, aborting a running backup could leave the write daemon suspended, leading to a system hang.  This occurred when aborting in all but the last pass of backup.

    HYY2345: Avoid access violation in VSS writer

    Category: Backup/Restore
    Platforms: All
    Version: 2018.1.3

    This change addresses an issue that could cause access violation in VSS writer.

    HYY2383: Allow backups greater than 16TB

    Category: Backup/Restore
    Platforms: All
    Version: 2018.1.5

    Addressed an issue that resulted in failing to create a Caché online backup that is greater than 16TB.

    The problem is present in all releases.

    HYY2406: Properly restore selected databases from multiple backups after SFN change

    Category: Backup/Restore
    Platforms: All
    Version: 2018.1.5

    Under some circumstances a restore of multiple backups could fail if a source database being restored had different System File Numbers (SFNs) in the backups. This change corrects this problem.

    RJF437: Fix online backup restore for databases with more than 2**31 blocks

    Category: Backup/Restore
    Platforms: All
    Version: 2018.1.5

    Restoring an online backup of a database with 2^31 blocks or more (16TB+ for standard 8KB block size) would result in a corrupted database with blocks of the global not allocated in the map blocks. This change ccrrects this issue.

    RJF438: Fix false <DATABASE> error generating online backups of 29TB+ database

    Category: Backup/Restore
    Platforms: All
    Version: 2018.1.5

    Generating an online backup of a database that's more than approximately 29TB in size would incorrectly fail with <DATABASE>. This only applies to 8KB block size databases. Databases with 16KB block sizes or higher are unaffected. This problem has existed since Caché 4.1.

    Note: if there are any legacy databases with 4KB block sizes they would be affected at smaller sizes (~3.5TB), but 4KB databases are no longer supported.

    RJF457: Improve error cleanup in DBACK

    Category: Backup/Restore
    Platforms: All
    Version: 2018.1.5

    In some cases, ctrl-c or other errors in the ^BACKUP/^DBACK utility could leave the system unable to take another backup. In rare cases this could even leave the write daemon suspended inadvertently.

    SML2663: Fix backup task looping when alias is defined for null device

    Category: Backup/Restore
    Platforms: All
    Version: 2018.1.1

    Under certain circumstances if an alias is defined for the null device, backup could loop and fill the disk drive. This change corrects this problem.

    Business Intelligence

    DP-13432: Standardize usage of %CreateParameterNonce and %GetParameterNonce with FILTERNAMES and FILTERVALUES for Excel exports and PDF prints

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.6

    Listing and portlet exports to Excel now display the correct name for the filter chosen.

    DP-14043: Fix queries in which members that contain commas in their names are excluded

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.6

    The analyzer search box allows for excluding one or more members. Excluding multiple members that contain commas in their names now returns the correct answer.

    DP-20641: Ensure that the correct ("query",0) node for the new query key is formed

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.6

    Certain queries that use a combination of %FILTER, NOW, NONEMPTYCROSSJOIN, and NOT, that previously hung, now work as expected.

    DP-290333: Test for event creation success in TaskMaster.CreateTaskGroup

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.9

    This change prevents the creation of UPDATEFACESTEMP type tasks with names longer than 32 characters that produce naming conflicts when attempting to create system events. Previously, this issue occured when 100,000 %DeepSee.TaskMaster task groups had been created on a particular system.

    DP-292462: Correct test for %GetQueryStatus error to avoid unnecessary messages in the log

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.6

    Recent changes to the %DeeSee.ResultSet were causing extraneous messages in the DeepSee log such as:

      ResultSet GetQueryStatus no status found
    
    For the majority of cases these messages did not communicate relevant information and so are now suppressed in those situations.

    DP-404982: Solves a problem where %FixBuildErrors can report that errors are fixed, when they are not

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.6

    When a cube has an OnProcessFact method, %DeepSee.Utils:%FixBuildErrors can handle reference errors improperly. It reports success and deletes ^DeepSee.BuildErrors whether or not the record is fixed. This change fixes the problem.

    DP-405378: Correct subquery queryKey bookkeeping in compound cubes

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.6

    This change corrects a problem where a crossjoin containing an ordering function in the second (inner) term might lose track of the correct order of compound cube results.

    DP-406811: Invalidate cache buckets containing fact deletions triggered by buildRestriction

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.0

    If cube has buildRestriction and the source row is updated to be removed from the cube based on that restriction, then the relevant query cache will now be marked invalid and results will be recalculated after a cube synchronize.

    DP-407877: Apply user-defined minimum cell widths for table widgets

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.0

    This change makes it possible to define minumum cell widths in the pivot table widget.

    DP-417737: Correct vulnerability in Analytics User Portal

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.0

    This change corrects a vulnerability in the Analytics User Portal. The vulnerability would allow authenticated users with access to the User Portal page to execute user-specified ObjectScript commands on the server using their current security context. There are limitations; the string would be converted to uppercase before being executed, and it cannot contain any periods. There are also length limitations. 

    This vulnerability has been corrected.

    DP-418010: Fix detection of stale cell cache for cube relationships

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.0

    A query involving cached cell results derived from related cube subqueries could produce incorrect results if any of those related cubes were updated with new dimension members that were inserted into any dimension used in that query.

    DP-426867: Fix check for reusable joinindex entry in %CreateJoinIndex

    Category: Business Intelligence
    Platforms: All
    Version: 2018.1.9

    This change corrects an issue that occassionally would prevent the system from recalculating a join index after synchronizing data.

    Class Compiler

    AJP022: Treat classes in read only databases as up to date

    Category: Class Compiler
    Platforms: All
    Version: 2018.1.5

    This change fixes a problem that could cause a compile error for HL7 message classes when a Rebuild All is performed.

    MAK4926: Protect temp global that stores generated method logic with locks

    Category: Class Compiler
    Platforms: All
    Version: 2018.1.5

    Add locking protection around temp global that stores codemode=generator or objectgenerator method logic in the class compiler. Logic which uses this information from SQL could be called in the middle of another process killing this data leading to <UNDEFINED> error in getcode^%qadadt on the ^CacheTempClsCode global in rare cases.

    Compatibility Features

    DP-413293: Zen Reports: do not allow use of $DATASOURCE URL parameter by default

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Compatibility Features
    Platforms: All
    Version: 2018.1.0

    Previously ZEN Reports would allow the report's datasource to be specified at runtime via the $DATASOURCE URL Parameter. If this is not defined, then we will use the DATASOURCE class parameter, and if that isn't defined we will use the ReportDefinition to generate the XML data. The ReportDefinition is the standard way to specify report data.

    With this change, Zen Reports will no longer use the $DATASOURCE URL parameter by default. Developers are strongly encouraged to use the DATASOURCE class parameter to specify an external datasource if desired. If a report does require the $DATASOURCE URL, you can re-enable the previous behavior on a per-report or per-application basis by setting the parameter:

    Parameter USEURLDATASOURCE = 1; 

    in either the report class, or in the report's Application class. Setting this value is discouraged.

    Containers

    DP-422624: use machid 77/78 for dockerubuntu platforms when HSCONTAINERS build option is specified

    Category: Containers
    Platforms: All
    Version: 2018.1.9

    When the build option HSCONTAINERS is specified in the build specification, dockerbuntux64 build will use machid 77 and dockerubuntuarm64 kernel will use machid 78.

    CSP

    DP-11630: Eliminate on unload behavior from svgWidgetProvider and lock down behavior of RemoveFile() call

    NOTE: This item may require a change to code, configuration, or operation.

    Category: CSP
    Platforms: All
    Version: 2018.1.0

    As of Chrome 80 synchronous HTTP events will no longer be honored as part of page exit processing. Previously this widget made use of such events to perform implicit server-side clean up of temporary files created during the PDF generation process. This feature is no longer supported.

    All temp files used by this widget reside in the instanceHome/mgr/Temp directory, which is cleaned up periodically by normal maintenance processes. Applications that do not wish the temp files to linger until regular clean-up may explicitly remove individual files with the RemoveFile() method supplied by this widget. Note however, that RemoveFile() will only remove temp files created by this widget during the current CSP session and is not a generic utility.

    MAK4708: Fix UNDEFINED problem in %CSP.Mgr.GatewayRegistryImpl:GetUnique method

    Category: CSP
    Platforms: All
    Version: 2018.1.4

    Fix UNDEFINED problem in %CSP.Mgr.GatewayRegistryImpl:GetUnique method

    MAK4766: When a REST/SOAP application fails login with no username/password, audit login failure

    Category: CSP
    Platforms: All
    Version: 2018.1.3

    When a REST/SOAP application fails login with no username/password, add an audit entry for the login failure. Before this change the code would assume that the login page would be displayed which is the case for CSP applications so nothing would be logged. Now it detectss REST/SOAP and adds an audit entry.

    MAK4890: Do not remove CacheUserName when redirecting to password change page

    Category: CSP
    Platforms: All
    Version: 2018.1.3

    When you login with CSP and we redirect to the password change page we were removing the CacheUserName %request parameter which should be retained so the user can see who they logged in as and so who they need to change the password for.

    MAK4934: Audit a license failure in CSP

    Category: CSP
    Platforms: All
    Version: 2018.1.2

    If a CSP request fails because it can not get a license, audit the failure.

    MAK5007: Honor request to not free gateway cache on a %CSP.Routine:Compile call

    Category: CSP
    Platforms: All
    Version: 2018.1.2

    In some cases the setting to not free the gateway cache was ignored and the cache was freed after a call to %CSP.Routine:Compile. The setting to retain and not free the cache is ^%SYS("CSP","DisableGatewayCacheClear")=1. This change corrects this problem.

    MAK5010: Protect against %request.Data("Error:FullURL",1) not being defined in %CSP.PasswordChange.cls

    Category: CSP
    Platforms: All
    Version: 2018.1.3

    This change protects against %request.Data("Error:FullURL",1) not being defined in %CSP.PasswordChange.cls which would result in an UNDEFINED error.

    MAK5013: Correct problem where CSP was calling OnStartSession callback function on every request in a session

    Category: CSP
    Platforms: All
    Version: 2018.1.1

    The OnStartSession callback function should be called when a CSP session is created. Under certain circumstances, it was being called for every request in the session rather than just at the session creation time. This change corrects this problem.

    MAK5043: Correct CSP parsing of lines longer than 32000 characters

    Category: CSP
    Platforms: All
    Version: 2018.1.3

    CSP assumed that all lines were less than 32000 characters in length and would add a CR/LF for any line longer than this length. Now the code checks if the eol token is read before adding a CR/LF. CSP is updated to use longer reads as long strings are supported now.

    MAK5051: Do not apply cookie path matching for stream server requests

    Category: CSP
    Platforms: All
    Version: 2018.1.9

    If CSP gets a request for say /csp/samples/page.csp where the /csp/samples application has a cookie path of '/csp/samples' but the cookie it gets has a path of just '/csp' then we detect this mismatch and generate a new sessionId for this request automatically, but do not do this for static pages since the session will not be reused. Under certain conditions this functionality was not working when serving streams. This change corrects this problem.

    MAK5090: Correct %request.Data("IRISLoginPage") parameter showing up when it should not

    Category: CSP
    Platforms: All
    Version: 2018.1.4

    Correct problem which caused %request to contain 'IRISLoginPage' setting for all POST requests.

    MAK5128: Allow CSP Gateway 'ClearCache' command to be run as long as you have "%Development" resource

    Category: CSP
    Platforms: All
    Version: 2018.1.4

    When you compile a CSP/ZEN page we may generate static .js files automatically. As part of this, we clear the CSP gateway cache for these static files so it will request new versions of these from the server. The clearing of this cache was only allowed if you had '%Admin_Manage' resource, this has been changed to allow '%Development' resource too because if you can recompile the class you should be able to clear the cache for this file.

    MAK5245: Do not treat web socket licenses like REST/SOAP

    Category: CSP
    Platforms: All
    Version: 2018.1.5

    Web socket licenses need to be treated like CSP licenses rather than REST/SOAP licenses.

    MAK5262: Make SharedConnection=0 web sockets honor the %session.EndSession flag to terminate the current session

    Category: CSP
    Platforms: All
    Version: 2018.1.5

    When a SharedConnection=0 web socket completes the logic was not checking the %session.EndSession flag which if we allows the user to request this session is deleted once the current request finishes. Now we will check for this flag and terminate the session if it is set to provide a simple way for web socket code to say it is done with this session.

    MAK5263: Correct failure to release CSP license when calling ##class(%CSP.Session).%DeleteId(id)

    Category: CSP
    Platforms: All
    Version: 2018.1.5

    When you manually delete a CSP session by calling ##class(%CSP.Session).%DeleteId(sessionId) the license associated with this was not being released. This is now corrected.

    MAK5366: In cases where we generate a new CSP session automatically make sure SecureSessionCookie session property is initialized

    Category: CSP
    Platforms: All
    Version: 2018.1.6

    When CSP gets a request with an invalid HMAC in the session token we automatically get a new sessionId and construct a new session and continue with the request. This new session initialization logic when the original one was invalid did not initialize the SecureSessionCookie property correctly so this was set to 0 even if %request.Secure is true. This prevented SameSite=none session cookies from being sent to the client. This change corrects the problem.

    MXT2080: Support CORS for CSP pages which are not subclasses of %CSP.REST

    Category: CSP
    Platforms: All
    Version: 2018.1.1

    This enhancement supports CORS for CSP pages which are not subclasses of %CSP.REST. In previous releases, CORS was only supported for subclasses of %CSP.REST.

    CORS processing for CSP pages that do not inherit from %CSP.REST is provided by the CSP login page; that is, subclasses of %CSP.Login which are assigned as the login page for a CSP application. To turn on CORS assign the application's login page to be a subclass %CSP.Login that has the HandleCorsRequest parameter = 1. In addition, OnHandleCorsRequest and/or OnHandleOptionsRequest methods may be overridden in order to override the default CORS response for the application.

    SGM021: Return error status if asynchronous %CSP.WebSocket Write fails

    Category: CSP
    Platforms: All
    Version: 2018.1.5

    Modified the %CSP.WebSocket.Write() method so that it returns an error status if an asynchronous web socket has been dropped at the system level. Several new CSP error codes have also been introduced (error codes 7907-7915).

    SGM031: Support SameSite for CSP session and user cookies

    NOTE: This item may require a change to code, configuration, or operation.

    Category: CSP
    Platforms: All
    Version: 2018.1.5

    When a server sets an HTTP(S) cookie, it can include an attribute called SameSite that affects the scope in which the cookie is sent. The options are as follows:

    • None - Send cookie with cross-site requests.
    • Lax - Send cookie with safe, top-level cross-site navigation.
    • Strict - Do not send cookie with cross-site requests.

    Recent updates to browser security have changed handling of third-party cookies. These updates use the SameSite attribute to reduce the risk of cross-site request forgery (CSRF) attacks, unauthorized access to data, and other possible security issues. Chrome (starting with v.84) enforces stricter rules for SameSite behavior, and these rules can cause issues with existing websites and web applications. These issues may include login problems, the login page being displayed repeatedly, and page elements not displaying properly.

    Before this change, it was not possible to modify the SameSite attribute, and web applications running on these versions may have such issues. This change allows you specify default values for the SameSite attribute, which will thereby prevent third-party cookie issues form arising.

    With this change, the product will explicitly set SameSite both for CSP session cookies and any cookie set via the %CSP.Response.SetCookie() method.

      Two new fields have been added for customizing web applications:
    • Session Cookie Scope
    • User Cookie Scope
    • These fields are displayed in Management Portal's Edit Web Application window, alongside "Use Cookie for Session" and "Session Cookie Path."

      Session Cookie Scope controls the default SameSite value for session cookies associated with the given web application. User Cookie Scope controls the default SameSite value for user-defined cookies created with %CSP.Response.SetCookie(). Session Cookie Scope and User Cookie Scope can be set to None, Lax, or Strict. System web applications and new or upgraded user applications default to Strict for both new fields.

      The %CSP.Response.SetCookie() method also accepts SameSite as an argument, which overrides the default value. You can use this new argument to exercise more fine-grained control over your cookies.

      You should configure grouped applications to use the same Session Cookie Scope setting. Additionally, if you are using "SameSite=None", the web application needs to support secure (HTTPS) connections.

      SGM037: Correct error logic for CSP CheckPermissions failures

      Category: CSP
      Platforms: All
      Version: 2018.1.5

      When a permission check failed, CSP methods would quit with '0' instead of a properly formatted status code. This caused erroneous error counts and reports when CSP page re-compilation was attempted in Studio without proper permissions. Error messages were also written to the current device regardless of whether the 'displaylog' or 'displayerror' flags were set. This change corrects both issues: the CSP methods now return proper statuses for permission failures so the errors can be counted and reported correctly, and the extraneous message have been removed.

      CSP / ZEN

      DP-423252: fix check for CSPCHD URL token

      Category: CSP / ZEN
      Platforms: All
      Version: 2018.1.9

      This change corrects logic that checked whether a CSP session ID can be passed in the URL of a CSPCHD token.

      DP-425916: If recurse is disabled for REST web-application then requests with "/" in them return 404

      Category: CSP / ZEN
      Platforms: All
      Version: 2018.1.9

      This change ensure that the recurse option for CSP/Zen Web Applications - which determines if pages in subdirectories of the physical path can be accessed via the Web Application - cannot be accidentally set when creating a REST application, as was previously possible. The result of this caused requests with a "/" in them to return a 404 error.

      DP-427903: Fix regression in %ZEN.Controller error handler

      Category: CSP / ZEN
      Platforms: All
      Version: 2018.1.9

      This change fixes an issue that could result in an error in the InvokeInstanceMethod() method of the %ZEN.Controller class.

      CSP Server

      DP-402100: %CSP.WebSocket:OpenServer() should quit 0 for a non-existent WebSocketID

      Category: CSP Server
      Platforms: All
      Version: 2018.1.6

      This change updates the %CSP.WebSocket:OpenServer() method to return an error status if no data is found for a given WebSocket ID. Previously, this method would return $$$OK even though it failed to find and open the web socket.

      DP-404562: Adds unique error codes to %CSP.WebSocket

      Category: CSP Server
      Platforms: All
      Version: 2018.1.6

      This change replaces generic error usage in the %CSP.WebSocket class with new error codes. WebSocket behavior is unchanged otherwise.

      DP-406217: Allow CSP pages to specify exact redirect URL

      Category: CSP Server
      Platforms: All
      Version: 2018.1.6

      Typically when performing a browser-side redirect using %response.Redirect, the target URL will be transformed to an absolute URL before being sent to the client. The system may also add or modify the CSPToken, CSPCHD, and other URL parameters if the target URL is a CSP/ZEN page

      This change introduces a new property: %response.UseExactRedirect. If this is set to 1, then the server will not perform these transformations; the URL specified in %response.Redirect will be used exactly as specified by the user. This only affects the current request, and does not affect redirects using %response.ServerSideRedirect. The default is 0.

      Note that RFC 2616 initially defined the Location header to require an absolute URI (see section 14.30), but this was later relaxed by RFC 7231 (see Section 7.1.2, and Appendix B). Setting %response.UseExactRedirect=1 will allow a CSP Application to redirect to a relative URL. This generally should not be relevant because by default we will translate the URL to the equivalent absolute URL, but may be helpful in some unusual cases.

      DP-411189: %Studio.General:ConstructCSPSession initializes SecureSessionCookie properly

      Category: CSP Server
      Platforms: All
      Version: 2018.1.9

      If https is in use, the %Studio.General.ConstructCSPSession() now sets SecureSessionCookie to true.

      DP-416562: Correct WebSocket handling of low-level errors

      NOTE: This item may require a change to code, configuration, or operation.

      Category: CSP Server
      Platforms: All
      Version: 2018.1.0

      This fix corrects a bug in the error handling for web socket connections. If an error was encountered before the CSP server dispatched to the WebSocket class, then the error would be ignored. Now the CSP server will call the Error() method in the WebSocket class. Similarly, for authentication failures, the server will call Login(). Note that by default, Login() just calls Error(). 

      The websocket will also now abort the connection if OnPreServer() returns an error. Previously errors were ignored.

      DP-416563: Do not allow navigation to %-CSP pages from the /csp/sys/oauth2/ application

      NOTE: This item may require a change to code, configuration, or operation.

      Category: CSP Server
      Platforms: All
      Version: 2018.1.0

      This change modifies the default configuration to disallow access to %-CSP pages from the /csp/sys/oauth2/ application.

      DP-421886: Do not reuse existing CSP Session during authentication

      Category: CSP Server
      Platforms: All
      Version: 2018.1.9

      This change ensures that a new CSP session is created when logging into a CSP application with a new username.

      In addition, if the "Prevent login CSRF attack" option is enabled for a web application, a session token cannot be passed via the URL.

      CSP.Gateway

      ALE3199: Stop IIS service during upgrade

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.3

      Fixed a problem where IIS service (w3svc) was not stopped when CSPSILENTRESTART property was set (which is set by default). Added an entry to installation log file to indicate when IIS service is being stopped.

      ALE3359: Do not modify pre-existing CSP.ini for external web server

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.4

      In environments where a third party Web Server is running remotely, the CSP/Web Gateway installer tried to modify some settings in CSP.ini which could result in unwanted changes in terms of service accessibility. A review of the configuration after each upgrade was always recommended. With this change the Web Gateway installer will not modify an existing CSP.ini in any way. For details on configuring the web server and Web Gateway, see Configuring the Web Server and Web Gateway.

      AOD008: Fix CSPFileTypes * directive not working in root of httpd.conf

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.3

      When the "CSPFileTypes *" directive was placed at the root of the Apache server configuration file outside a <Location> or <Directory> tag, the Web Gateway would fail to serve any files, leading to an HTTP 404 error. This issue has been fixed so that "CSPFileTypes *" is recognized at the root of the Apache configuration file.

      AOD010: Fix <SUBSCRIPT> error in processing multipart/form-data POST request

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.3

      Improve the parsing of the "name" and "filename" parameters in the Content-Disposition header in POST requests of type multipart/form-data. This fixes a <SUBSCRIPT> error that would sometimes occur when the filename was Base64-encoded.

      AOD024: Ensure that Web Gateway registry methods do not hang when called from a non-default server

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.4

      This change fixes a problem where, if the Web Gateway is configured to use multiple servers for load balancing and failover and receives a Gateway registry request from a server that is not the default server, the request would hang.

      AOD025: Fix 7-second delay in establishing a WebSocket connection after disabling registry method

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.4

      The Web Gateway registry methods can be disabled by placing the line REGISTRY_METHODS=Disabled in the [SYSTEM] section of CSP.ini. This change eliminates a 7-second delay in the establishment of each WebSocket connection after registry methods were disabled.

      AOD032: Ensure CSP.log has correct owner and group when created or modified by Apache parent process

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.4

      Fix a permissions problem where if the Apache parent process creates CSP.log, the Apache worker processes are unable to write to it.

      AOD041: Allow the max response buffer size to be decreased to 8K

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.4

      The Web Gateway buffers response data from the instance. The maximum response buffer size (default 128K) can be manually adjusted using the MAX_RESPONSE_BUFFER_SIZE parameter in the SYSTEM section of the Web Gateway configuration file. It was found that on some systems, a small response buffer leads to faster page loading. The MAX_RESPONSE_BUFFER_SIZE parameter used to enforce a lower limit of 32K. The lower limit has been reduced to 8K, and if a smaller value is specified, the Web Gateway will use 8K as the max response buffer size.

      Example usage in CSP.ini:

         [SYSTEM]
         MAX_RESPONSE_BUFFER_SIZE=8K
      

      The web server must be restarted in order for changes to this parameter to take effect.

      AOD062: Allow upload of size 0 files

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.4

      This change fixes an issue where the Web Gateway treated size 0 file uploads as though no file was specified. Size 0 files can now be uploaded.

      AOD064: Correct a regression in the handling of virtual hosts

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.4

      Correct a regression where the Web Gateway failed to recognize application configurations specific to an Apache virtual host. For example, if the Web Gateway configuration defined an application path /csp/user and another application path //virtualhostname/csp/user, then the Web Gateway would never use the //virtualhostname/csp/user configuration. All requests via //virtualhostname/csp/user were instead incorrectly served via the /csp/user application. The handling of virtual hosts in the Web Gateway has been fixed.

      AOD074: Clean up connection table when Apache worker process terminates abnormally

      Category: CSP.Gateway
      Platforms: UNIX®
      Version: 2018.1.5

      Before this change, when an Apache worker process terminated abnormally on Unix, its entries would remain in the Web Gateway connection table until the web server was restarted. Abnormal process termination is now handled more gracefully, and the connection table is cleared of defunct entries corresponding to processes that no longer exist at the time the next worker process is started.

      AOD081: Correct a regression in the handling of virtual hosts

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.5

      Correct a regression where the Web Gateway would sometimes route a request to an incorrect Apache virtual host. For example, if the Web Gateway configuration defined an application path //virtualhostname1/csp/ and another application path //virtualhostname2/csp/, then a request to //virtualhostname1/csp/ would sometimes be routed to the Caché server for //virtualhostname2/csp/.

      CMT1587: Improve the performance of Gateway installations working to large configurations (many application paths defined)

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.3

      This enhancement aims to improve the performance of Gateway installations working to large configurations. In this context a 'large configuration' is one containing many application paths defined in the Gateway configuration and/or the Caché configuration.

      With previous builds large configurations resulted in a noticeable degradation in CSP performance. In order to address this problem, the Gateway now uses an improved internal layout (in the shared memory sector) for holding the configuration. Also, a better indexing mechanism and search algorithm has been developed to enable the Gateway to quickly identify the running configuration for a particular application path.

      Apart from improved performance, the only visible change is the message that's recorded (in the Event Log) when more memory is required to be reserved in the shared memory sector for holding the configuration.

      Previously the message recorded was:

      Configuration Error: Insufficient space in the configuration buffer
      Configuration block Size: A; Size of configuration block to insert: B; Space available: C (Consider setting CONFIG_BUFFER_SIZE=X (or higher) in the [SYSTEM] section of CSP.ini)
      

      The new message is:

      Configuration Error: Insufficient space in the configuration buffer  
      Consider setting MAX_APPLICATIONS=X (or higher) in the [SYSTEM] section of CSP.ini  
      

      The MAX_APPLICATIONS parameter is the total number of paths defined in the Gateway plus the number of paths defined in each participating Cache installation. The Gateway (that is, the hosting web server) must be completely restarted after modifying this parameter.

      CMT1591: Ensure that the 'SSL/TLS Cipher Suites' (SSLCC_Cipher_Suites) configuration parameter can be updated via the Gateway Registry methods

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that the 'SSL/TLS Cipher Suites' (SSLCC_Cipher_Suites) configuration parameter can be updated via the Gateway Registry methods.

      For example:

      Kill properties Set properties("SSLCC_Cipher_Suites")="ALL:"
      set status = gateway.SetServerParams("LOCAL", .properties) 
      

      CMT1592: Ensure that the 'Connection Security Level' (Connection_Security_Level) configuration parameter can be updated via the Gateway Registry methods

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that the 'Connection Security Level' (Connection_Security_Level) configuration parameter can be updated via the Gateway Registry methods.

      For example:

      Kill properties Set properties("Connection_Security_Level")=10 // SSL/TLS
      Set status = gateway.SetServerParams("LOCAL", .properties) 
      

      Before this change it was not possible to set the Security Level to SSL/TLS (10). The correct values representing each security level are listed below.

      #define CSP_H_SECURITY_LEVEL0             "Password"
      #define CSP_H_SECURITY_LEVEL1             "Kerberos"
      #define CSP_H_SECURITY_LEVEL2             "Kerberos with Packet Integrity"
      #define CSP_H_SECURITY_LEVEL3             "Kerberos with Encryption"
      #define CSP_H_SECURITY_LEVEL10            "SSL/TLS"
      

      CMT1593: Write extended error information to Event Log if the Gateway fails to load a DLL under Windows

      Category: CSP.Gateway
      Platforms: Windows
      Version: 2018.1.2

      This enhancement will write extended error information to the Event Log if the Gateway fails to load a DLL under Windows AND the the Log Level is set to 'e'.

      The information recorded will include the reason why a particular library could not be loaded and also record the path to the location where the Gateway was looking.

      For example, if the 'cconnect' library cannot be found, a series of messages similar to those shown below will be recorded.

      >>> Time: Sat Jun 03 11:34:44 2017; RT Build: 1801.1636 (win64/iis/mod:srv=7.5); Log-Level: 0; Gateway-PID: 3700; Gateway-TID: 3212  
       Unable to load the following library - code: 126 - The specified module could not be found.  
       C:/Inetpub/CSPGateway/cconnect64.dll 
      >>> Time: Sat Jun 03 11:34:44 2017; RT Build: 1801.1636 (win64/iis/mod:srv=7.5); Log-Level: 0; Gateway-PID: 3700; Gateway-TID: 3212  
       Unable to load the following library - code: 193 - %1 is not a valid Win32 application.  
       C:/Inetpub/CSPGateway/cconnect.dll  
      >>> Time: Sat Jun 03 11:34:44 2017; RT Build: 1801.1636 (win64/iis/mod:srv=7.5); Log-Level: 0; Gateway-PID: 3700; Gateway-TID: 3212  
       Unable to load the following library - code: 126 - The specified module could not be found.  
       C:/Inetpub/CSPGateway/bin/cconnect64.dll  
      ...
      >>> Time: Sat Jun 03 11:34:44 2017; RT Build: 1801.1636 (win64/iis/mod:srv=7.5); Log-Level: 0; Gateway-PID: 3700; Gateway-TID: 3212  
       Initialization  
       Information: The CCONNECT library is not present on this system (This library is used for the optional Kerberos and SSL/TLS based security between the Gateway and Cache)  
      

      CMT1600: Avoid re-encrypting Microsoft DPAPI-encrypted passwords after a fault on decrypting an existing (encrypted) password

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change avoids re-encrypting Microsoft DPAPI-encrypted passwords after a fault on decrypting an existing (encrypted) password.

      There were cases where the Gateway attempted to re-encrypt aan already encrypted password after an error decrypting a password. The Gateway now applies further checks to determine whether or no it is dealing with an encrypted value and if it is, will simply report the error in the Event Log.

      CMT1601: Respond gracefully when Caché returns a formal HTTP error code (and message) in response to the Gateway checking the integrity of a connection and the associated instance process

      Category: CSP.Gateway
      Platforms: Windows
      Version: 2018.1.2

      With this change the Gateway responds gracefully when Caché returns a formal HTTP error code (and message) in response to the Gateway checking the integrity of a connection and the associated instance process.

      For example, if an invalid password is submitted (or the Gateway is attempting to connect to a non-existent user account) Caché will respond with a message similar to that shown below:

         HTTP/1.0 403 Forbidden  
         Content-type: text/html  
         Connection: closed  
         Password authentication failed  
         ERROR #838: User cm does not exist 
      

      The Gateway doesn't always recognize HTTP messages that are returned in response to internal validation/handshake commands which, in turn can lead to confusing error messages being recorded. For example:

         WARNING: Incorrectly formatted value for $ZVersion  
         CacheSP: chd=1  
      

      Or:

         cspTestConnection(mode=0, context=14, response_size=33): Response
         CacheSP: es=1;p=0;chd=1;ato=60;
      

      Or:

         New Value for $ZVersion (Old value: '/csp')
         Cache for Windows (x86-64) 2015.2.1 (Build 705_0_17102U) Mon Mar 6 2017 16:28:18 EST
      

      Or:

         Gateway response cache
         The CSP Gateway has detected a server version change for configuration '4) 2015.2.1 (Build 705_0_17102U) Mon Mar 6 2017 16:28:18 EST'.  Clearing the response cache
      

      The Gateway now correctly processes these messages and records a more explanatory error message in the log. For example:

         Connection Validation Failed: mode=0; context=11;  
         HTTP/1.0 403 Forbidden  
         Content-type: text/html  
         Connection: closed  
         Password authentication failed  
         ERROR #838: User cm does not exist
      

      CMT1616: Ensure that a UNIX signal involved in worker process close-down is not recorded as an error condition when terminating IPC Server threads

      Category: CSP.Gateway
      Platforms: UNIX®
      Version: 2018.1.2

      This change ensures that a UNIX signal involved in worker process close-down is not recorded as an error condition when terminating IPC Server threads supporting state-aware connectivity.

      For example, this change will prevent the following spurious 'errors' being recorded when the hosting web server terminates a worker process.

         Error : cspIPCService : Signal=1; Phase=1
      

      CMT1619: Correct a fault formulating the final copy of the HTTP response headers prior to dispatch to the hosting web server

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects a fault in the function responsible for formulating the final copy of the HTTP response headers prior to dispatch to the hosting web server. This fault could result in a security violation being assumed/detected by IIS, leading to a failure of the request and the hosting Application Pool closing down (in order to protect the web server).

      One symptom of this problem is the following error being recorded in the Event Log:

         Invalid parameter detected in function at line 0 
      

      CMT1621: Secure access to the Gateway Registry methods

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change secures access to the Gateway Registry methods.

      Registry Methods have been divided into two categories:

      1. Read orientated operations (e.g. GetDefaultParams()).
        Users must have one of the following roles assigned: %All, %Manager or %Operator.

      2. Update oriented operations (e.g. SetDefaultParams()).
        Users must have one of the following roles assigned: %All or %Manager.

      For example, if a user with just the %Operator role assigned were to invoke the method to change the Gateway's Default Configuration Parameters the following error message would be returned.

      Insufficient privileges to invoke the 'SetDefaultParams' method
      

      CMT1628: Modify the Gateway Management forms so that they automatically redirect to the 'login' form after the user session times-out

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      The Gateway Management forms have been modified such that they will now automatically redirect to the 'login' form after the user session times-out. Previous versions would remain on the current form (after timeout) until the next user action, after which control would go back to the login form. Of course, this change only applies to Gateway installations for which the Management forms have been password protected.

      CMT1633: Ensure that the Gateway does not attempt to resize its run time configuration and internal indices after a web server worker process rotation

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that the Gateway does not attempt to resize its run time configuration and internal indices after a web server worker process rotation. With previous releases, this would occasionally happen - particularly (though not exclusively) when connecting to mirrored configurations after a web server worker rotation event. This, in turn, would lead to a mismatch between process-local indices and the running configuration held in the Gateway's shared memory sector.

      Apart from request failures and failures to connect to Cache, the following error messages in the Event Log (Linux systems) were symptomatic of this issue:

         Backtrace (SIGSEGV) : cspDaemonListenerErrorTrap  
            /scratch3/cm/apache2231/csp/CSPa22.so(cspDaemonListenerErrorTrap+0xdb) [0x7f0f9d01a9ab]  
            /lib64/libpthread.so.0() [0x310120f4c0]  
            /scratch3/cm/apache2231/csp/CSPa22.so(csprtDaemonListener+0x2cb) [0x7f0f9d01ad8f]  
            /lib64/libpthread.so.0() [0x31012077e1]  
            /lib64/libc.so.6(clone+0x6d) [0x3100ae153d]  
      

      And/or:

         Backtrace (SIGSEGV) : cspDaemonErrorTrap  
            /scratch3/cm/apache2231/csp/CSPa22.so(cspDaemonErrorTrap+0x1cd) [0x7f6fea974c1c]  
            /lib64/libpthread.so.0() [0x310120f4c0]  
            /scratch3/cm/apache2231/csp/CSPa22.so(csprtSysManager+0x7420) [0x7f6fea9f9d4d]  
            /scratch3/cm/apache2231/csp/CSPa22.so(csprt+0x4316) [0x7f6fea9c493a]  
            /scratch3/cm/apache2231/csp/CSPa22.so(+0x6ede6) [0x7f6fea9b9de6]  
            /scratch3/cm/apache2231/bin/httpd(ap_run_handler+0x5b) [0x43f499]  
            /scratch3/cm/apache2231/bin/httpd(ap_invoke_handler+0x16f) [0x43fd8a] 
         etc .........
      

      CMT1634: Correct a Memory Access Violation that's reported if the Gateway Management URL (instead of the RunTime URL) is used to obtain a Gateway System Status report

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects a Memory Access Violation that's reported if the Gateway Management URL (instead of the RunTime URL) is used to obtain a Gateway System Status report.

      For example:

      Incorrect URL: http://localhost/csp/bin/Systems/Module.cxw?CSPSYS=1

      Correct URL: http://localhost/csp/bin/RunTime/Module.cxw?CSPSYS=1

      CMT1640: Ensure that content (to be cached) is not written to permanent storage until it is confirmed that there is enough space

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that content (to be cached) is not written to permanent storage until it is confirmed that there is enough space in the master index to hold the reference for the cached file. A failure to do this in previous releases could lead to to an accumulation of 'orphaned' files in the Gateway's /temp directory. This happened because the Gateway can only clear down cached physical files that it is aware of that is. those for which there is a corresponding entry in the master index of cached files that is held in the Gateway's shared memory sector.

      Also, in cases where the whole content of a cached entity can fit into a single cache block, the Gateway will not write the payload to permanent storage, but instead include it in the data block used to hold the master index entry.

      CMT1643: Correct a fault in the cconnect library trace facility.

      Category: CSP.Gateway
      Platforms: Windows
      Version: 2018.1.2

      This change corrects a fault in the cconnect library trace facility. This change utilizes a new function provided by the cconnect library (CconnectSetTraceFile()).

      This function allows the Gateway to supply the path and file name of the trace file to be used (usually cconnect.log), rather than opening this file within the Gateway code and passing the (pointer) reference for the open file, to the cconnect library. There are problems (notably under Windows) with passing pointers to open files over function/library boundaries. These problems will be avoided by enabling the cconnect library to completely own the management of the trace file within its own code base.

      CMT1645: Ensure that the command line '-encrypt_password' facility generates the same 'hash' as that which would otherwise be generated by the Gateway Management suite

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that the command line '-encrypt_password' facility (provided by CSPnsd) generates the same 'hash' as that which would otherwise be generated by the Gateway Management suite.

      Example:

         # ./CSPnsd -encrypt_password=0:SYS && echo
         1Phmog51gaVNwReQh0rKvtmGBlRw
      

      This change corrects a fault in the mechanism which led to a mismatch between encrypted passwords generated from within the Gateway Management forms and those generated from the command line.

      CMT1646: Ensure that the the Gateway configuration file (CSP.ini) is correctly updated when modifying passwords using the 'Security.Users.Modify()' method

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that the the Gateway configuration file (CSP.ini) is correctly updated when modifying passwords using the 'Security.Users.Modify()' method.

      Example: Set properties("Password") = "new" Set status = ##Class(Security.Users).Modify("CSPSystem", .properties)

      With previous versions of this method, unwanted trailing data was left at the end of the CSP.ini file if the updated file happened to be shorter in length than the original.

      CMT1648: Protect against a memory access violation that can occur if GZIP compression is enabled but the ZLIB initialization function (deflateInit2) fails

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change protects against a memory access violation (SIGSEGV) that can occur if GZIP compression is enabled but the ZLIB initialization function (deflateInit2) fails.

      Under these circumstances the following message will be written to the Gateway Event Log:

         GZIP Initialization Error  
         Initialization of the ZLIB library failed (deflateInit2) gzip_err = -2  
      

      CMT1649: Ensure that the 'Maximum Connections per Session' throttle applies across all worker processes in a multi-process web server

      Category: CSP.Gateway
      Platforms: UNIX®
      Version: 2018.1.2

      This change ensures that the 'Maximum Connections per Session' throttle applies across all worker processes in a multi-process web server. For example: Apache using a 'prefork' MPM under UNIX.

      With previous releases, the 'Maximum Connections per Session' throttle was applied independently to each and every worker process resulting in a single client potentially consuming more connections that it should.

      When the throttle is exceeded, the following message will be written to the Event Log (log level 'e'):

         Connection Allocation Error  
         Maximum number of connections in use for session ID: FCJotTP9SP (Server: CACHE20172; Maximum Connections per Session: 6)  
      

      Affected requests will be queued for as long as the the 'Queued Request Timeout' timeout will allow, after which the Gateway will return a 'Server Busy' status and the following message will be written to the Event Log:

         Server Availability Error
         All channels to the Server are busy: Please try later 
       

      CMT1655: Rework the signal handler for Access Violation (SIGSEGV) errors to avoid hangs in Apache

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change reworks the signal handler for Access Violation (SIGSEGV) errors to avoid hangs in Apache. Also avoided will be defunct (or zombie) processes appearing in the process listings after a SIGSEGV event.

      CMT1656: Correct a fault that could result in a Memory Access Violation (SIGSEGV) occurring when Apache recycles a worker process UNIX systems).

      Category: CSP.Gateway
      Platforms: UNIX®
      Version: 2018.1.2

      This change corrects a fault that could result in a Memory Access Violation (SIGSEGV) occurring when Apache recycles a worker process UNIX systems).

      When this fault occurs the following function call back-trace will typically be written to the Gateway Event Log:

         Backtrace (SIGSEGV) : cspDaemonErrorTrap  
         /opt/cspgateway/bin/CSPa24.so(cspDaemonErrorTrap+0x1c6) [0x7fa334803346]  
         /lib64/libpthread.so.0(+0xf100) [0x7fa340d02100]  
         /opt/cspgateway/bin/CSPa24Sys.so(cspsmSemaphoreDestroy+0x32) [0x7fa3340f1b57]  
         /opt/cspgateway/bin/CSPa24Sys.so(cspsmCloseDown+0x361) [0x7fa3340df9b2]  
         /opt/cspgateway/bin/CSPa24Sys.so(cspsys_x_closedown+0x11) [0x7fa33410752d]  
         /opt/cspgateway/bin/CSPa24.so(csprtUnInitialize+0x118) [0x7fa334800ef5]  
         /opt/cspgateway/bin/CSPa24.so(csprtCloseDown+0x8b2) [0x7fa3347fe076
      

      CMT1658: Correct a fault that led to the occasional failure of WebSocket connections under Nginx1.12.2.

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects a fault that led to the occasional failure of WebSocket connections under Nginx 1.12.2.

      Apart from a failure to initialize a WebSocket, this fault resulted in the following message being recorded in the Nginx error log:

         [alert] 30754#0: *40 zero size buf in writer t:0 r:0 f:0 0000000000000000 00000000020D0E30-00000000020D0E30 0000000000000000 0-0
      

      CMT1659: Ensure that the Gateway version/build information is included in the SERVER_SOFTWARE CGI Environment Variable for Nginx Servers

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that the Gateway version/build information is included in the SERVER_SOFTWARE CGI Environment Variable for Nginx Servers. Testing revealed that this information was often missing for Nginx hosted Gateway installations.

      Example:

         SERVER_SOFTWARE   nginx/1.12.2 CSP-Universal/2018.2.0.999.0-1802.1667-16
         

      CMT1677: Correct a fault that led to spurious event log messages warning about absenct HTTP request method

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects a fault that led to spurious event log messages (incorrectly) warning about the absence of an HTTP request method.

      For example:

         WARNING: The REQUEST_METHOD is absent for this request (CSP_DELETE_VARS)
         HTTP_ACCESS_CONTROL_REQUEST_METHOD,HTTP_ORIGIN,HTTP_ACCESS_CONTROL_REQUEST_HEADERS
      

      CMT1681: Protect against an infinite loop occurring when the Gateway traverses its forms cache

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change aims to protect against an infinite loop occurring when the Gateway traverses its forms cache.

      CMT1682: Correct a fault that made it impossible to download files of type DLL and EXE over HTTP

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change correct a fault that made it impossible to download files of type DLL and EXE over HTTP in a CSP application.

      CMT1683: Extend the mechanism for interrupting Caché processes to include a 'reason' code.

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change extends the mechanism for interrupting Caché processes to include a 'reason' code.

      The Gateway will send a 'reason for interrupting' code to Caché with the 'interrupt' message. This code will be included in the call to interrupt a process:

      $System.Util.SendInterrupt(ProcessID,Reason)

      The following 'reason' codes will be recorded.

         1 Client disconnected while waiting for a response (e.g. browser closed).
         2 Request timed-out ('Server Response Timeout' exceeded).
         3 Interrupt request dispatched from the Gateway Management forms ('System Status' form).
         4 Client disconnected while sending the request payload.
         5 Protocol error detected between the Gateway and Caché.
      

      CMT1687: Correct a fault in the initialization of the Shared Memory block for the Nginx Gateway solution

      Category: CSP.Gateway
      Platforms: UNIX®
      Version: 2018.1.2

      This change corrects a fault in the initialization of the Shared Memory block for the Nginx Gateway solution - known as the 'Universal Gateway Modules'. Problems caused by this fault were only evident in UNIX-based Nginx installations in which multiple worker processes were used.

      For example, in nginx.conf:

         worker_processes  3;
      

      One visible manifestation of this problem was the premature timeout of the Gateway Management forms.

      CMT1688: Correct a potential race condition in the functionality that initializes Gateway 'Server' connections

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects a potential race condition in the functionality that initializes Gateway 'Server' connections (the reserved connections responsible for serving Gateway registry functionality and websockets operating in 'shared connection' mode).

      While no problem has occurred that could be attributed to this fault, the potential for a race condition theoretically exists for configurations that connect to multiple Caché or InterSystems IRIS servers.

      CMT1689: Improve the mechanisms for closing down the threads responsible for hosting the 'Server' connections and listening for incoming commands

      Category: CSP.Gateway
      Platforms: UNIX®
      Version: 2018.1.2

      This change improves the mechanisms for closing down the threads responsible for hosting the 'Server' connections and listening for incoming commands from Caché and InterSystems IRIS. Server connections are responsible for serving Gateway Registry methods in Caché and InterSystems IRIS and websockets operating in 'shared connection' mode.

      The previous mechanism for closing down these threads was rather crude and it is possible that a failure to cleanly (and gracefully) terminate these threads has been responsible for crashes in Apache/UNIX installations. The new scheme will gracefully interrupt 'Server' listener threads so that they can clean up their resources and cleanly terminate. The core 'closedown' function managing this process will only force the threads to terminate if, for whatever reason, orderly termination is not possible.

      CMT1692: Ensure that the correct HTTP response status code is saved in the Nginx 'access' log records

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that the correct HTTP response status code is saved in the Nginx 'access' log records. It was recently noticed that all CSP requests were being recorded as having a response status code of '000'. The CSP module will now insert the status code supplied by Caché or InterSystems IRIS in the appropriate Nginx data block.

      CMT1701: Modify the procedure that searches for the Gateway's 'temp' directory so that the IIS 'temp' directory does not get used by default

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change modifies the procedure that searches for the Gateway's 'temp' directory so that the IIS 'temp' directory does not get used by default. An earlier change introduced a scheme whereby the Gateway's files could be placed in a location other than that hosting the binaries. The intention was to allow a better security scheme to be applied to these files.

      The 'alternative' location for the Gateway's temp directory is '../temp' relative to the directory holding the Gateway binaries. The problem for IIS installations that place the Gateway binaries in '/inetpub/CSPgateway/' is that this alternative location resolves to '/inetpub/temp/' which is the 'temp' directory used by IIS.

      For IIS, the Gateway will now attempt to use the following paths for its 'temp' directory (and in the order show):

      • ./temp/ (i.e. /inetpub/CSPGateway/temp/)
      • ../csptemp/ (i.e. /inetpub/csptemp/)
      • ../temp/ (i.e. /inetpub/temp/)

      The 'temp' directory is where the Gateway holds its repository of cached response files.

      CMT1702: Correct a regression in the construction and normalization of the PATH_TRANSLATED CGI Environment Variable.

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects a regression in the construction and normalization of the PATH_TRANSLATED CGI Environment Variable.

      After the improvements to allow the processing of CSP URLs of any length, this variable was found to be constructed as follows (for example) ...

      C:\inetpub\CSPGateway\samples\inspector.csp\csp\samples\inspector.csp

      ... instead of the expected form ...

      C:\inetpub\CSPGateway\csp\samples\inspector.csp

      CMT1704: Allow the '%response.AllowOutputFlush' facility to be set in the HTTP Response Headers

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This enhancement will allow the '%response.AllowOutputFlush' facility to be set in the HTTP Response Headers. This will make the 'output flush' facility accessible to older Caché systems for which the '%response.AllowOutputFlush' is not defined.

      Example:

         
         

      CMT1705: Reduce the risk of interference when versions of the same library are loaded both by the CSP Gateway and a hosting Apache installation

      Category: CSP.Gateway
      Platforms: UNIX®
      Version: 2018.1.2

      This change reduces the risk of interference between versions of the same library that are loaded both by the Web Gateway and a hosting Apache installation under Linux. For example, the GZIP library (libz.so) will often be loaded both by the Gateway and Apache. Loading two copies of a library into one process can sometimes cause conflicts (leading to crashes), particularly where different versions of the library are used.

      CMT1709: Support oversize lines in the HTTP response headers

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change allows the support of oversize lines in the HTTP response headers (for example, 'Redirect' lines targeting long URLs). The Gateway will now dynamically resize its response headers buffer up to 8K as required.

      Note: Bear in mind that other parts of HTTP infrastructure may apply their own limits to the length of URLs. Using Apache, the code in the 'Test Hints' (a 4K URL) will render successfully (on redirection) with Firefox, Chrome and Microsoft Edge. It will, however, be blocked by the default IIS (v10) configuration.

      CMT1710: Introduce greater flexibility into the security restrictions applied to the Web Gateway Registry and Management methods

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change introduces greater flexibility into the security restrictions applied to the Web Gateway Registry and Management methods.

      Rather than checking for '%Operator' and '%Manager' roles, the access gateway to these methods instead checks if the user has 'Use' access to the '%Admin_Operate' and '%Admin_Manage' resources, respectively. This allows Systems Administrators to create their own finely-tuned user roles for the various Operator and Manager functions.

      CMT1712: Upgrade the version of ZLIB to v1.2.11 (from v1.2.3)

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change represents an upgrade of the ZLIB (GZIP) library to v1.2.11 (from v1.2.3).

      CMT1713: Correct a fault that can lead to CSP requests hanging if the Gateway run-time module does not have write-access to its configuration file (CSP.ini).

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.1

      This change corrects a fault that can lead to CSP requests hanging if the Gateway run-time module does not have write-access to its configuration file (CSP.ini).

      CMT1715: Move the 'Refresh' and 'Clear' links to the top of the 'View HTTP Trace' form

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This enhancement moves the 'Refresh' and 'Clear' links to the top of the 'View HTTP Trace' form. These links were previously located at the bottom of the form which led to difficulties with managing oversize log/trace files where the Administrator had no option but to wait for the whole report to load before it could be cleared.

      CMT1716: Introduce paging to the 'View HTTP Trace' form

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This enhancement introduces paging to the 'View HTTP Trace' form. Previous versions of this form would attempt to list all HTTP request/response records found in the current Event Log file. This led to the form locking up for oversize Event Log files. The new version will only attempt to list a 1000 HTTP records at a time in the left hand frame. Browsers are able to cope with this number of records. As with the main Event Log listing, and where multiple pages of records are involved, 'More' and 'Top' links will be provided as appropriate.

      Finally, the auto-refresh facility has been removed from the 'View HTTP Trace' listing as unexpected page refreshes can be a nuisance when studying the HTTP data. A listing 'Refresh' link is available at the top of the form so that the contents can be refreshed at a time convenient to the Administrator.

      CMT1717: Adapt the Gateway so that it can cope with oversize header blocks for components within multi-part request messages

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      The Gateway has been modified such that it can cope with oversize header blocks (greater than 1K) for individual components within multi-part request messages. With this change, header blocks up to 2K in size can be accommodated.

      CMT1718: Increase the capacity of the buffering algorithm used to receive multi-part request messages

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      For this change the buffering algorithm used to receive and process multi-part request messages has had its capacity increased. With these improvements, component header blocks of up to 6K can be accommodated.

      The baseline input buffer for multi-part request messages has been increased from 4K to 8K - a size more appropriate for parsing multi-part SOAP messages.

      CMT1725: Corrected a fault that resulted in requests for static file names containing non-ASCII characters (Umlauts etc...) not being processed

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.1

      This change corrects a fault that resulted in requests for static file names containing non-ASCII characters (Umlauts etc...) not being processed.

      For example: http://localhost:80/csp/user/caché.jpg

      Requests such as the one above would return an 'HTTP Error 404.0 - Stream Not Found' error.

      CMT1726: Protect against an issue resulting from double forward-slash sequences (//) in the URL path

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change protects against an issue resulting from double forward-slash sequences (//) in the URL path.

      For example: http://localhost:57773//csp/sys/UtilHome.csp

      Notice the '//' sequence introducing the /csp... path part of the URL.

      To be more specific, this problem only affects the Caché and InterSystems IRIS Management Portals running over Apache web servers. The observed problem when running over this infrastructure is the portal home page being repeatedly refreshed, with each refresh consuming a new license unit. Eventually the license becomes exhausted.

      Although not typically used, URLs constructed in this way are theoretically valid: see https://tools.ietf.org/html/rfc3986#section-3.3

      However, the way the above URL is processed differs between IIS and Apache. With IIS the path is passed to CSP 'as is'. CSP sees '//csp/sys/UtilHome.csp' and (correctly) returns 'File Not Found' (HTTP 404).

      Apache, on the other hand, tries to be helpful and removes, what it sees as, the surplus '/' character. In this case, CSP sees '/csp/sys/UtilHome.csp' and the page is served. On the browser side, however, all the relative links embedded in the form still have the extra '/' included in the URLs and this causes the application to loop: The home page UtilHome.csp, followed by a hyperlink (an AJAX call), followed by the home page etc ... repeats until the browser is forced down or the license is exhausted.

      The remedy implemented here is to modify the Apache interface such that it behaves the same way as IIS. In other words, if an unexpected '//' sequence is detected in the raw request data then the unparsed path will be used instead of the (usual) parsed version and the Gateway will handle 'first line' URL parsing.

      CMT1731: Correct the documentation for the Web Gateway's 'Connection Security Level' configuration parameter.

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects the documentation for the Web Gateway's 'Connection Security Level' configuration parameter.

      For SSL/TLS based connectivity to the database, the documentation indicates the following setting:

         Connection Security Level = 11
      

      This should be:

         Connection Security Level = 10
      
      Note: The Web Gateway will still use TLS if this parameter is set to the documented value of 11. However, the Gateway Management forms will set the more correct value of 10 for TLS.

      CMT1735: Correct a fault in buffer initialization for the function that retrieves CGI Environment Variables from the Nginx web server

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects a fault in buffer initialization for the function that retrieves CGI Environment Variables from the Nginx web server. This fault led to 'non-defined' environment variables not being correctly acknowledged as being 'non-defined' by the CSP Gateway.

      This change is in the distributed source file: ngx_http_csp_module_sa.c (module build 19).

      CMT1736: Correct a fault in the processing of the 'Sec-WebSocket-Protocol' HTTP request header (for WebSockets) under Nginx

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change corrects a fault in the processing of the 'Sec-WebSocket-Protocol' HTTP request header (for WebSockets) under Nginx. With previous builds, the Gateway was sending a value for this header in the response in cases where the client was not specifying a particular 'WebSocket Protocol'. It was found that Firefox was the only browser able to tolerate this breach of protocol. Other browsers (notably IE, Edge and Chrome) simply rejected the request for a WebSocket connection.

      Reference: https://tools.ietf.org/html/rfc6455

      CMT1737+CMT1738: Protect against a potential DoS vulnerability caused by the 'hang' period applied to repeated login failures

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change updates the hang time for invalid logins. If a user enters invalid credentials (either invalid username or invalid password), the new behavior is as follows:

      The new behavior is the same for both a username who exists in the security user database, and for a username who does not exist in the security user database.

      For any type of login for services, Terminal, console, telnet, bindings, or $system.Security.Login(), the system will hang for 2 seconds before the user is allowed to re-enter credentials. The system will also hang for the same two seconds if some other login error is detected (e.g. service disabled, insufficient privileges).

      For a web-based CSP type login (CSP, SOAP, REST), an error is immediately returned to the user. Then:

      • For a CSP application login, the user receives an "Access Denied" error.
      • For a REST/SOAP login, the user receives an HTTP/1.1 401 Unauthorized message.

      The user can immediately enter and submit new credentials. There is no "hang 2" implemented here to force a wait. (This is because the CSP server processes are already running and are pooled across connections; it is undesirable to have them to hang and unable to process other requests.) If the system invalid login retry limit is set to 0 (disabled), the above behavior continues for each successive invalid login. If the system invalid login retry limit is not 0, and the system invalid login retry limit is reached, then the behavior changes as follows:

      • For CSP application login, the user receives a "Service unavailable" for this and subsequent invalid logins.
      • For REST/SOAP login the user receives a "HTTP/1.1 503 Service Unavailable" message.

      The user can immediately re-enter their credentials, but that information will be ignored for the next 2 seconds and the service unavailable messages will persist until the two seconds have elapsed, even if valid credentials are entered. Note that audit records will not be written at this point, nor will an existing user record be updated.

      CMT1745: Create extra Registry indices when a Gateway instance is simultaneously supporting both TLS and non-TLS traffic

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change ensures that the extra Registry indices are created when a Gateway instance is simultaneously supporting both TLS and non-TLS traffic through the same web server.

      The problem addressed here is that Registry information is keyed by the instance host name (CSPIHN) field which is made up of the web server host name and listening port, the latter of which will be different for TLS (usually 443) and non-TLS traffic (usually 80).

      In practice it was found that one Registry stream would work, and successfully respond to requests, and one would not. This change will ensure that each instance host name will refer back to the current 'Server connection'(1) for the associated web server process regardless of whether it was initially created, and indexed against, the TLS or non TLS stream.

      For example, for a mixed TLS/non-TLS configuration in which the first request was over TLS (the first request always triggers the creation of the Gateway 'Server' process), the Registry records would look something like the following:

       
      ^cache.Temp.SysMetricsGWClient("COM","DESKTOP-NQL2POG:443:3NG.6fe.1011GpRwO",0,"$J")=24860
      ^cache.Temp.SysMetricsGWClient("COMX","127.0.0.1:443")="DESKTOP-NQL2POG:443"
      ^cache.Temp.SysMetricsGWClient("COMX","127.0.0.1:80")="DESKTOP-NQL2POG:443"
      

      Note: 'Server Connections' are the special Gateway connections reserved for the purpose of serving Registry requests.

      CMT1746: Transmit 'Registry Disabled' flag to Instance and adapt Registry Methods so that they return immediately when Registry infrastructure is disabled

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change will transmit the 'Registry Disabled' flag to Caché and InterSystems IRIS (as %request.RegistryMethods). The Registry Methods have been adapted so that they will return immediately when the Registry infrastructure is disabled.

      This change applies to the following setting in the Gateway configuration:

         [SYSTEM]
         REGISTRY_METHODS=Disabled
      

      With previous versions, the methods would hang and timeout when accessed during a time at which the registry was disabled.

      CMT1748: Increase the size of the buffer allocated to the Instance 'configuration changed' timestamp

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change increases the size of the buffer allocated to the Caché and InterSystems IRIS 'configuration changed' timestamp. Lack of buffer space for this data element resulted in mirror-aware Gateway configurations not performing reliably (seemingly random failure of requests, for example).

        New timestamp format (for example): 64980,72829.772386293:11/28/2018
        Old timestamp format:               64980,72829.772386293
      

      CMT1751: Check the integrity of responses to HTTP OPTIONS requests

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This change checks the integrity of responses to HTTP OPTIONS requests. This enhancement forms the basis of a requirement to prevent such requests from taking an additional license.

      One check that is applied is that responses to OPTIONS requests (from Caché and InterSystems IRIS) should not contain an entity body. This is currently the case but the HTTP standard does mandate that responses can potentially include a body, but, at the present time, there are no conditions under which a body should be generated.

      From RFC 2616: https://www.ietf.org/rfc/rfc2616.txt

      If the OPTIONS request includes an entity-body (as indicated by the presence of Content-Length or Transfer-Encoding), then the media type MUST be indicated by a Content-Type field. Although this specification does not define any use for such a body, future extensions to HTTP might use the OPTIONS body to make more detailed queries on the server. A server that does not support such an extension MAY discard the request body.

      CMT1752: Add 'HTTP OPTIONS' requests to the set of CSP resources that do not require a license

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.2

      This enhancement adds 'HTTP OPTIONS' requests to the set of CSP resources that do not require an Caché and InterSystems IRIS license.

      These requests will no longer incur a licensing cost provided that a valid license key with greater than one unit is already deployed and the Web Gateway is capable of checking the integrity of the generated response.

      CMT1757: Ensure that an array used in the Application Path configuration block is initialized properly

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.3

      This change ensures that one of the arrays used in the Application Path configuration block is initialized properly. This fault is possibly responsible for a number of memory access violations.

      CRE-2302: Upgrade web server to Apache 2.4.46

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.5

      Internal web server (Apache httpd) is upgraded to version 2.4.46.

      DP-402802: Fix segfaults caused by accessing gateway management pages on AIX

      Category: CSP.Gateway
      Platforms: AIX
      Version: 2018.1.5

      This change fixes a problem where accessing the gateway management pages on AIX could produce a segment fault. This problem only occurred with small stack sizes.

      DP-7204: Fix a number of WebSocket thread-safety issues

      NOTE: This item may require a change to code, configuration, or operation.

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.4

      Fix a number of WebSocket thread-safety issues that resulted in occasional lost messages and dropped connections.

      If you are using Nginx with the default dynamic modules configuration (using CSPx.so and CSPxSys.so), note that we are now deprecating this configuration. The dynamic modules configuration will still function as before, but everyone using the Web Gateway with Nginx is urged to rebuild and reconfigure Nginx to work with the Network Service Daemon (NSD) build of the Web Gateway instead. The NSD now supports WebSockets when used with Nginx, so migration to the NSD is now doable if you have a WebSocket application.

      If you are already using Nginx with the NSD, you must still rebuild Nginx and edit its configuration to apply this change.

      Migration instructions if you are already using Nginx with the NSD

      1. Install an updated version of the Web Gateway.
      2. Rebuild Nginx using the updated version of ngx_http_csp_module.c. See https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCGI_ux#GCGI_nginx_unnsd for instructions on how to do this.
      3. Add the CSPNSD_pass directive to the Nginx configuration file to indicate the hostname/IP and port where the NSD is listening. For example, if your configuration file contains
        location /csp
        { CSP On; }
        
        and the NSD is listening at 127.0.0.1:7038 (the default), then change the configuration block to
        location /csp { CSP On; CSPNSD_pass 127.0.0.1:7038; }
        
      4. Start up Nginx and the NSD.
      Migration instructions if you are using Nginx with dynamic modules (CSPx.so, CSPxSys.so)
      1. Install an updated version of the Web Gateway.
      2. Rebuild Nginx using ngx_http_csp_module.c instead of ngx_http_csp_module_sa.c. See https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCGI_ux#GCGI_nginx_unnsd for instructions on how to do this.
      3. Remove the CSPModulePath directive from the Nginx configuration.
      4. Add the CSPNSD_pass directive to the Nginx configuration file to indicate the hostname/IP and port where the NSD is listening. For example, if your configuration file contains location /csp { CSP On; } and the NSD is listening at 127.0.0.1:7038 (the default), then change the configuration block to

        location /csp

        { CSP On; CSPNSD_pass 127.0.0.1:7038; }

      5. Start up Nginx and the NSD. See https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCGI_atypical_windows#GCGI_usingnsd and https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCGI_atypical_unix#GCGI_usingnsdunix for more information on how to manage the NSD.

      This change also adds a few additional Nginx configuration directives that you should be aware of when migrating, and you should review whether their default values are acceptable for your workloads. If the default values are not acceptable, then specify your own value in the proper location{} block in the Nginx configuration file.

      Syntax: CSPNSD_response_headers_maxsize size;
      Default: CSPNSD_response_headers_maxsize 8k;
      Context: location, if in location
      Description: The maximum size of the HTTP headers in a response. An error is returned to the web client if a response header exceeds this size.
      

      Syntax: CSPNSD_connect_timeout time; Default: CSPNSD_connect_timeout 300s; Context: location, if in location Description: Timeout for connecting to the NSD when a request is received from the web client.

      Syntax: CSPNSD_send_timeout time; Default: CSPNSD_send_timeout 300s; Context: location, if in location Description: Timeout for a single send operation to the NSD when sending a request. The timeout is set only between two successive write operations, not for the transmission of the whole request.

      Syntax: CSPNSD_read_timeout time; Default: CSPNSD_read_timeout 300s; Context: location, if in location Description: Timeout for a single read operation from the NSD when reading a response. The timeout is set only between two successive read operations, not for the transmission of the whole response.

      An example configuration that uses all the new directives is the following.

      location /csp
      

      { CSP On; CSPNSD_pass 127.0.0.1:7038; CSPNSD_response_headers_maxsize 8k; CSPNSD_connect_timeout 300s; CSPNSD_send_timeout 300s; CSPNSD_read_timeout 300s; }

      SDK116: Ensure that Web Gateway SERVER connections are properly shut down

      Category: CSP.Gateway
      Platforms: All
      Version: 2018.1.5

      This change corrects a defect that could cause server processes associated with the Web Gateway SERVER connection to remain after the associated web server worker has been shut down. Over time this would result in an increasing number of processes sitting in the EVTW state running the %SYS.cspServer3 routine.

      Cube

      SGM016: Support HTTPS from the Windows Cube

      Category: Cube
      Platforms: Windows
      Version: 2018.1.5

      This devchange adds an option to use HTTPS for links to the Management Portal from the Windows Cube menu. This new option is displayed in the Server Manager as a Yes/No column and can be enabled or disabled for a server connection via the HTTPS checkbox in the Add or Edit window.

      Debugging

      CDS3169: Fix Studio debugging for unprivileged user

      Category: Debugging
      Platforms: All
      Version: 2018.1.4

      Users without privileges to the system database could get an error when trying to start a second debug session with Studio.

      CDS3244: Debugger does not work with NodeNameInPid

      Category: Debugging
      Platforms: All
      Version: 2018.1.5

      This change fixes an issue where if the NodeNameInPid compatibility setting was enabled, a Studio debug session would get an error when starting.

      DeepSee

      DTB630: Fix problem with shared calculated member in compound cube queries

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Compound cube queries where cubes within the compound cube and the compound cube itself reference a shared calculated member in the query text could produce an incorrect value. This change corrects this problem.

      Note: Consolidating the values of the calculated member from multiple cubes only has a meaningful answer for some calculations. For example, for a calculated member that defines different literals in each cube there is no meaningful way to aggregate these calculations.

      DTB631: Allow %FixBuildErrors to resolve IDs of records that have been removed

      Category: DeepSee
      Platforms: All
      Version: 2018.1.3

      If a record in a cube's sourceClass was removed after encountering a build error, the %DeepSee.Utils:%FixBuildErrors repair method would encounter another error and so that ID would never be removed from the list until another full build. A removal of a source record is now considered a means of "fixing" the error.

      DTB632: Do not present aggregate override for calculate measures in level options dialog

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Aggregate overrides do not make sense with calculated measures. This change removes the option to set this from the Level Options dialog.

      DTB643: Retrieve searchable measure indices directly from SQL via %BITMAPCHUNK function

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      This optimization enhancement creates searchable measure condition indices directly in the SQL query resultset. Use of this optimization can be managed using ^DeepSee.EngineSettings("searchableMeasureSqlChunk"). To preserve compatibility, this optimization is not available for computed SQL dimensions that invoke stored procedures via a CALL statement.

      DTB646: Improve source control behavior in portal pages

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      This change improves source control in the DeepSee UI so that it prevents editing and submission of the loaded document back to the server. In previous versions, the write locks on the source controlled file would work, so allowing editing could lead to the local copy being out of sync with source control. If this occurred a message indicating this would be presented on the next load of the document. This change ensures that the page refreshes the writable state of the in-memory document whenever the source control menu is used to change the source control state. This avoids the problem leading to being out of sync with source control.

      The UI has the following changes:

      • In the Architect the various buttons now respond to the current editable state of the document.

      • Users that do not have write privileges will not be presented the source control buttons.

      • Users that do not have write privileges in the Architect will have the New, Save, Compile, and Build

      • buttons disabled.

      DTB663: Use asynchronous axis processing when searchable measures are in play

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Searchable measures in Analyzer could time out during the resolution of the searchable measure. This is now one of the conditions that engages the asynchronous axis processing.

      DTB669: Look at all pieces of %OR keys when examining dependsOn index in %Intersect

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Complex filters that included homogeneous %OR sets intersected a level that it dependsOn could incorrectly return no results. This is corrected.

      DTB671: Set font style attributes in SVG nodes so that Batik can properly consume the output

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Some fonts would not print correctly in chart widget legends when using SVG printing. This change corrects this so a font that is installed and Zen enabled using

      Set ^zenNavigator.UserFontListCSV = customFont

      in the namespace serving the dashboard will now print in the widget legend.

      DTB681: Fix problem where date range in slicer returns all results instead of null set

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Time ranges can use the special [NOW +/- offset]

      syntax to define ranges and members that do not exist in the data. When this was used as the slicer, it could have the effect of logging an empty set on the slicer axis, equivalent to an empty slicer. This shows all results instead of the null results that would be correct.

      This change now places a single member of the requested set of time members if the entire set contains no intersection with the set of facts in the cube.

      DTB693: Improve heterogeneous %OR axis construction when containing nested intersection functions

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      This fixes some problems that came up when handling complex heterogeneous %OR clauses in MDX queries. A particular focus is slicers of the form:

      %OR({ CROSSJOIN(A,B), CROSJOIN(C,D) })

      DTB694: Fix <UNDEFINED> error caused by a complex filter

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Under certain circumstances a complex DeepSee filter could cause a query to fail with an <UNDEFINED> error. This change corrects this problem.

      DTB695: Convert CROSSJOIN with two %OR arguments into an explicit tuple in axis processing

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      An MDX statement using %OR functions as the arguments of a crossjoin could end up OR'ing those arguments if the crossjoin itself was itself nested within a %OR. An example of this could look like:

      %OR( { CROSSJOIN( %OR({A,B}), %OR({C,D}) ),...})

      The handling of this statement is improved to emulate an explicit tuple, for example:

      %OR( { ( %OR({A,B}), %OR({C,D}) ),...})

      which is handled correctly.

      DTB696: Limit payload testing to POST requests in DeepSee REST DataServer

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      DeepSee REST requests sent to the

      /api/deepsee/v1/Data/*

      services using CORS would fail validation. This is corrected.

      DTB697: Do not flatten %OR(<tuple>) query object in pre-processing

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Construction of a slicer that contains an %OR could produce incorrect results if that %OR contained a tuple that also had a %OR as one of its terms, for example:

      %OR({([GenD].[H1].[Gender].&[Male],%OR({[AgeD].[H1].[Age Bucket].&[10 to 19],[AgeD].[H1].[Age Bucket].&[20 to 29]}))})

      Removing the outer %OR from the slicer, for example:

      ([GenD].[H1].[Gender].&[Male],%OR({[AgeD].[H1].[Age Bucket].&[10 to 19],[AgeD].[H1].[Age Bucket].&[20 to 29]}))

      would produce the correct result with an equivalent statement. Depending on how the query is created it is not always possible to control the presence of the outermost %OR, and this is now corrected even in the event that the %OR is added internally.

      DTB698: Improve compressed date range handling when building queries for KPI plugins

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      KPI Plugins rely on the results of a special DRILLTHROUGH or DRILLFACTS query constructed using the complete cell context for a single cell. Filters that included date ranges which could benefit from time folding could end up passing on the incorrect cell context and would subsequently compute their value from an incorrect SQL resultset. This is corrected.

      DTB700: Reject deeply nested %OR/tuple query constructions in MDX queries

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      There is a problem where the MDX engine cannot reliably process complex heterogeneous %OR statements that include more than two nested levels of %OR setFunctions and operations that represent a logical AND. Before this change, these constructs would be accepted but the results were not correct. This change now tests for these constructions and rejects them during query execution.

      Note: By design queries using highly complex OR/AND constructions will now see those rejected. This is to prevent unpredictable results. You should rewrite any queries that are rejected as a result of this change. Any of these blocked queries can be rewritten and flattened so that any %OR within them contains only AND terms.

      DTB716: Make sure pivotTable query re-initializes properly after previous error in axis processing

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      A query that encounters an error in axis building for a related cube subquery would fail to recognize that problem in the axis initialization if reloaded in the pivotTable component. This adjusts the record keeping in the axis cache to ensure that the query re-executes.

      DTB719: Avoid impacting other unrelated tasks when canceling DeepSee query

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Canceling a query in the DeepSee Resultset affected background tasks assigned to other DeepSee Agent operations. This includes operations that are not query related and directed to different cubes than the query being canceled. This change tracks the task subgroups and corrects this problem.

      DTB724: Preserve original case of the member declaration in calculatedMember.%ToString()

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      The call to reproduce the string representation of a calculated member would not always preserve the original case of the declared name of that member. In some instances the text is provided in upper case and in others it is in lower case. This change corrects this problem.

      DTB725: Fix TaskMaster agents handling task cancellation when actively working on a task

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      When canceling a task group assigned to %DeepSee.TaskMaster agents, only the tasks that had yet to be picked up by an agent were removed. Any tasks that were in process were expected to complete and move to other task groups. This could cause problems if a query with a very long-running background task was canceled and then attempted again before the background task completed. This could cause errors due to two processes attempting to work on contents of the same cache.

      With this change each of the agents working on active tasks for a particular group being canceled are sent external interrupts to shut down the remaining work that has no further use.

      DTB731: Widen Architect Details pane to accommodate all fields without a scrollbar

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      A horizontal scrollbar was showing up in the Architect's Details pane in order to access only a few hidden pixels. This pane is widened in order to eliminate the need for the scrollbar altogether.

      DTB737: Add validation to the RegistryMap object before saving to class

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      This introduces a means of adding object validation to the %DeepSee.CubeManager.RegistryMap:SaveToClass method. In this release the validation prevents any duplicate group names in the RegistryMap before saving the object to the class.

      DTB739: Add stronger concurrency protection to %DeepSee.ResultSet

      NOTE: This item may require a change to code, configuration, or operation.

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      This changes the way that the %DeepSee.ResultSet operates during concurrent executions to preserve the integrity of results. This provides much deeper inspection of the primary components of a particular query, requiring that each writer to the query cache secure a lock on the query key and all axis keys before being granted access to change anything in either the results or axis caches.

      DTB741: Do not call %CreateAgents for single-threaded %SynchronizeCube

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      The call to %DeepSee.Utils:%SynchronizeCube would create background agent processes even if the method were executed in its default pAsync=0 mode. This is corrected so agents are only created if they are needed.

      DTB742: Protect processing of tuple children from being called with a null parent

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      In some cases when a related member is one of the members of a tuple, it could throw a <SUBSCRIPT> error should that related member resolve to NO MEMBER in the related subquery. This is corrected.

      DTB743: Introduce user setting to toggle appearance of calculated members in searchBox filters

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Calculated members can be filed under existing cube dimensions. There is now a namespace-wide setting Admin > Settings > General Tab > [Show Calculated Members in Filters] allowing the DeepSee administrator to decide whether or not these calculated members appear in filters.

      This setting has no effect on member lists for virtual dimensions that are created specifically by the definition of a calculated member.

      DTB745: Preserve selected rowSpec through drill-down operations

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      The chooseRowSpec control was not fully compatible with drill down. This change accounts for the effects of this control type when interacting with the pivot's drilldown.

      The expected behavior is now:

      • When the row level is set using chooseRowSpec, drilling down functions correctly from the chosen point of reference and drilling back up will ultimately land on the chosen level.
      • If chooseRowSpec is used to change the start level while drilled down, it will clear the drill down state and start back at the top with the latest choice.

      DTB751: Prevent name collision with registered groups when generating natural group names in Cube Manager

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      In the Cube Registry users are allowed to pick any name they like for the registered groups. If one or more of those names used the naming convention Group <integer> an unregistered group could receive the same generated name and then it would not be possible to register that group. This is now corrected so that all natural groups will be assigned an unused <integer> if name collisions are detected between the generated unregistered natural groups and existing registered groups.

      If a Registered group name is changed by the user in the registry to match one of the unregistered group names, the natural group name will be updated to avoid collision when the registry is saved to the server.

      DTB752: Tighten search restriction application in memberData:%GetMembers

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      DeepSee filters were not fully restricting to the desired search term in some cases. This tightens the generated SQL to more accurately limit member lists.

      DTB756: More explicit grouping of SQL WHERE clause terms in %GetMembers

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      In some cases order of operations was getting confused when there were many AND and OR operations being added to the WHERE clause in alternating order. This change adds more parentheses grouping as the SQL WHERE clause is being created from MDX FILTER terms to keep the order of operations straight.

      DTB758: Add test of %reduce in memberData:%GetMembers to prune member list

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Basic filters that fetch all members of a level will not provide options that have no facts associated with them.

      DTB761: Accept a literal null as a valid axis label in %GetOrdinalLabel, %GetOrdinalKey

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      When reading the headers in a DeepSee resultset using the APIs %DeepSee.ResultSet:%GetOrdinalLabel and %DeepSee.ResultSet:%GetOrdinalKey, literal null strings would be ignored and cause an inaccurate label depth count in the return of a particular ordinal axis position. This change corrects this problem.

      DTB767: Maintain continuity of %DeepSee.ResultSet object throughout each REST request

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      REST services were executing the business logic to serve the request in such a way that would trigger the detail listing cleanup before assembling the response payload. This is corrected.

      DTB768: For compound DRILLTHROUGH, do not close subquery resultsets until master resultset is closed

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Compound cubes were unable to execute the underlying DRILLTHROUGH SQL queries when restrictions were involved. This is corrected.

      DTB770: Consult nonempty nodes in the results cache when using %CELL offset functions

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      The details of execution for how a resultset arrives at the final axis members could affect the results of the %CELL functions. These details included

      • Whether a related cube filter or a local filter was used
      • Whether a member provided restriction context in a filter or an axis provided that context
      These %CELL functions now return the same results for the same filtering context regardless of the order that context is derived or which part of the query contributes to it.

      DTB775: Traverse %KPI contents using chain axis structure when accumulating user arguments

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      The user-defined arguments that could affect behavior of plugins were not getting passed down to the callback code. This is corrected.

      DTB782: Implement clone clientmethod in queryChunk component

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      A Zen exception could occur when rapidly clicking the Toggle listing control on a widget, particularly if the extra clicks occur while the widget is still waiting for the current result. This change corrects this problem.

      DTB785: Always wait for joinindex task group if it was created in %ResolveRelationships

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      It was possible for the DeepSee TaskMaster agents to queue up join index processing that might never complete on a very busy system. This is corrected.

      DTB788: Address some issues in resultset when items are dispatched to the background

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Queries being executed through the pivotTable component could encounter background errors due to the new way resultsets manage query keys and cache initialization. This seemed to be limited to queries which dispatched the axes processing to background agents.

      DTB797: Use parameter nonce for FILTERVALUES in Excel export of MDX queries

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      The Excel export's FILTERVALUES parameter now uses the nonce functionality to shorten the overall length of the URL.

      DTB807: Preserve literal null in related cube axis when transferring to local cube

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      In some cases a given DeepSee query axis can get replaced with a literal null when filing the information into the cube's axis cache. If this occurs in a related cube subquery, that literal null could potentially cause problems when resolving to the context of the calling cube. In particular, this change addresses - A literal value that is produced in the subquery is now properly filed in the calling cube axis cache as a literal. - A literal that is a non-OR'ed member of a cell intersection will nullify the Join Index of that address. - %OR processing of an axis now understands the meaning of a literal null as the member of an ORSET

      DTB810: Print blank row in PDF table when DeepSee listing contains no results

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      If a DeepSee listing has no data and attempted to print to PDF, it would cause an error attempting to render the PDF document. This will now produce an empty listing with all of the other information (Titles, filters, etc.) intact.

      DTB813: Briefly disable listing controls to prevent double toggle during user double-click

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      When a listing control is clicked, it will now get disabled for 500ms, and then get reactivated. This is to prevent a double click from being interpreted as two separate listing toggle requests.

      DTB814: Introduce more robust SQL tokenizer and parser for managing listing field and orderBy lists

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      This change introduces a more complete tokenizer and parser for analyzing the SELECT and ORDER BY lists that can be defined in DeepSee listing definitions. It also restores the support of the more complex functions that can be used in SQL SELECT terms within DeepSee listing definitions. The parsing supports the use of the special DeepSee listing macros ($$$TEXT, iknow macros, and $$$PMML).

      DTB815: Fix DeepSee SQL parser issues

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      This change fixes an issue when selecting a new property from the tree in the FieldList dialog. This also corrects another issue where a class-defined CAPTION of the property would erroneously get added to the property when the dialog was constructing an ORDER BY list.

      DTB816: Enable autorefresh in PMML model tester page

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      The PMML Model Tester page stopped working when the default for autorefresh of the page was changed. This is corrected.

      DTB817: Create additional join index representing the query slicer in %ResolveRelationships

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Some related cube queries could return inconsistent results if they contained multiple references to the same related cube in both the slicer and one or more axes due to a problem in the join index. The query could return different results depending on whether or not certain other queries were run first. This is corrected.

      DTB818: Store initial axis text in axis cache for KPI/MDX context reference

      NOTE: This item may require a change to code, configuration, or operation.

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      The cell context passed to a %KPI plugin's %dsCellContext environment variable could lose useful information due to the fact that it was rebuilt from the pre-processed list of members that actually exist in the cube data. This variable now contains the original requested MDX text for the slicer.

      DTB819: Clear header field when item is removed in field list dialog

      Category: DeepSee
      Platforms: All
      Version: 2018.1.3

      When a field list is deleted in the Field List dialog, the 'Edit Header' text box is cleared as well as the 'Edit Field' text box.

      DTB820: Stabilize axis and results keys for each instance of query and axis objects

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Since pre-processing occurs for computational optimization purposes, multiple request for keys could lead to a difference between the key calculated for locking purposes and the key being worked on in the cache. This key is now only calculated if it has not been set in the object, otherwise it is used as a lookup to retrieve the stored "keyText".

      DTB822: Change syntax for control timer call to be compatible with Internet Explorer 11

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      The initial protection against double-clicking using a timer was not compatible with Internet Explorer 11. This is changed to use an equivalent syntax that is compatible.

      DTB823: Avoid redundant refresh when waiting for pivotTable results

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      The pivot table component could enter a loop and refresh more than necessary while waiting for results. This change reduces the likelihood that multiple refreshes will be initiated each time the query status is polled.

      DTB829: Process pivotTable column headers in correct order to manage stationary headers

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Some arrangements of column headers in a pivot table could result in incorrect placement of the stationary headers. An example of this is seen in the query:

      SELECT NON EMPTY {[PatGrpD].[H1].[Patient Group].Members,NONEMPTYCROSSJOIN([HomeD].[H1].[ZIP].Members,[GenD].[H1].[Gender].Members)} ON 0,NON EMPTY [BirthD].[H1].[Year].Members ON 1 FROM [PATIENTS]
      

      When scrolling the rightmost inner headers were traveling with the data table. This is corrected.

      DTB832: Set literal null to axis cache when MDX PERIODSTODATE finds an empty member list

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      It was possible for the MDX engine to enter an infinite loop when MDX clause that included a set function wrapped in an aggregate function, and that set function returned no members. An example is the aggregate AGGREGATE(PERIODSTODATE([BirthDate].[H1].[Year],[BirthDate].[H1].[Month].&[NOW+12]) If PERIODSTODATE finds no data, then the AGGREGATE would fail to process properly. The null return is now explicitly recorded for the surrounding aggregate to avoid confusion in processing.

      DTB834: Initialize custom dashboard controls according to declared type

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      User are able to create custom control components for use on DeepSee dashboard widgets. One option for this is to extend Zen components to customize their behavior in a user class. In this extension case it is useful to have the custom control initialized with the same settings that the original receives so that any functionality that is not customized still functions the same way when used in the same context.

      This change enables this feature. To make use of this feature, the custom component must define a property type that matches the type of an existing control component (eg: searchBox, select, text, etc.). This will enable the additional initialization of that control immediately after the custom control is instantiated.

      DTB835: Remove UI-specific limits when exporting MDX queries to Excel

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      There is some query manipulation in the DeepSee Analyzer which limits results in order to prevent long wait times during query exploration. With these changes when a user exports to Excel from the Analyzer, these limits are automatically removed no matter what choices the UI has made at that moment. This change does not affect PDF printing.

      DTB836: Add timestamp and job information to log of DeepSee build errors

      Category: DeepSee
      Platforms: All
      Version: 2018.1.3

      DeepSee build errors now include the $H timestamp and $Job number under the subscript

        ^DeepSee.BuildErrors(CubeName,ID,"info") = $LB($H,$J)
      

      These will also be printed when calling

        Do ##class(%DeepSee.Utils).%PrintBuildErrors(CubeName)
      

      A recompile of the cube definition class is required in order for the new logging to take effect. The APIs

      • %FixBuildErrors
      • %PrintBuildErrors

      are compatible with cubes compiled both before and after this change is in place.

      DTB842: Improve bookkeeping when rendering headers in pivotTable

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      This resolves two header alignment issues in the pivotTable:

      1. If a crossjoin on rows is drilled into by either double-click or dropping a new level on a member, the fixed column headers could become misaligned with the underlying data table. In some cases this could expose the template headers that are created with the table. This could make it appear as if there are duplicate headers. In other cases the headers would scroll with the table and then detach right before the horizontal scroll completes its tracking. This is not a problem if the original table does not have a crossjoin on rows.

      2. If columns are defined with mixed complexity such that there are nested headers to the left of at least one header that spans 2 or more columns, the placement of lower rows could have the incorrect offset.

      DTB846: Confirm there are actually rows to display before calculating nub table contents

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      When a table has a crossjoin on rows but is filtered so that there is no data, the nub table in the upper left will now be matched to the row headers.

      DTB847: Resolve named parameters before generating query key

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      %MDX subqueries using the %PARM reference that was meant to respond to axis context would not always properly apply the context. When this happened the subquery results would show the same result corresponding to the default parameter value no matter where it was in the resultset. This change fixes the problem.

      DTB851: Examine advanced filter graph to attempt to avoid AND/OR depth greater than 2 in comparison statements

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      To protect the integrity of results, structures with deeply nested %OR/CROSSJOIN calculations are limited to reduced, depth 2 logical statements (OR's of AND's or AND's of OR's).

      The Avanced filter editor added a hidden %OR whenever the condition used a comparison that resulted in a set. For example Age > 20 Would produce an MDX FILTER() statement that assumes the result is a set and then that set is wrapped in %OR for performance purposes. This now only adds the %OR if the current AND/OR depth is less than 2. Visually, this means the tree depth in the Advanced Filter Editor will graphically match the logical depth of the resulting MDX statement if the depth is at or above the threshold for tripping the error An %OR statement has depth > 2 and cannot be processed without an invisible %OR around any of the conditions.

      DTB853: Store MDX with $variables resolved as original query text

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      MDX pivot variables get resolved before calculating the query key, but the original request filed in the cache would include the unresolved variable references, which would cause %PrepareKey in background processing to fail.

      Since these are query wide the filed request text needs to include the specific request context. This resolved text is now filed so that %PrepareKey will pick it up and it will be executable even when the variables are out of scope in the background processing.

      DTB857: Correct compound cube cache lookup failure in pivotTable component

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      The process of folding compound cube results into the primary query cache was not compatible with recent changes to %PrepareKey. When failure occurred, a pivotTable component would fail to look up the correctly calculated results from the cache. This change prevents an overwrite of the the original query key text, unique to compound cubes, so that %PrepareKey can find the correct results.

      DTB858: Enhance efforts to fetch filter caption in pivotTable.getFilterInfo

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      If an Analyzer user created a single-member filter by directly dragging that member to the Filters, that filter was not getting set into the filter table for exports. This is corrected.

      DTB863: Correct error handling behavior in KPI plugin caching

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      KPI caching behavior is now tightened up in two areas: - If a KPI encounters a runtime error, it will always try again - If a plugin delivers results for an MDX query, that query cache is marked "mustCompute" to communicate that the MDX cache must check the KPI cache to verify results

      If the plugin design is such that if different results are expected due to environmental changes that might alter expected cell results for identical MDX queries, the plugin class *must* set the parameter Parameter FORCECOMPUTE = 1; This will instruct the MDX engine to recompute the KPI results regardless of the cache contents, providing the custom code to dynamically respond to the current environmental context.

      DTB868: Fix pivotTable monitoring of background query processing to prevent incomplete reporting

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Pivot tables requesting axes-only queries in the background could fail to properly determine the state of the resultset on the server and decide to quit retesting the results, perpetually displaying one of the various incomplete messages. The polling of the asynchronous resultset is now improved to prevent this hang in the user interface.

      A possible race condition between the primary query and asynchronous axis tasks is also resolved using the available locking APIs.

      DTB869: Allow full length spec strings to be returned from %GetSpecForAxisNode

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Listing queries executed with filters that contained very long lists of members could end up returning more rows than the original count. This is corrected.

      DTB874: Protect reference to %DeepSee.ResultSet:%Query in %Execute

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      A call to %DeepSee.ResultSet:%Execute automatically switches the prepared query object to use %UseAgents = 0 mode. This was causing a runtime exception that is now caught and returned as a status code.

      DTB878: Fix typo in node reference number for current members in buried axis levels

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      If CURRENTMEMBER referenced a level that is not the innermost level of the axis, the CURRENTMEMBER could fail to calculate properly. This is corrected.

      DTB888: Include %UpdatePendingResults in wait loop for DeepSee REST services

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      The DeepSee REST services would not always return the final result for asynchronous KPIs if instructed to wait using WAIT:1 (the default if not provided by the user). The wait logic now checks for the presence of pending results for these asynchronous KPIs and attempts to update results until all results are returned or the timeout is reached.

      DTB905: Update pivotTable query text whenever a state change occurs

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Actions can now be invoked in the Analyzer with the current query described by the pivotTable object regardless of whether or not that table has actually been executed.

      DTB912: Allow calls to system methods when a Cube Group's update plan is set to "Manual"

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Allow use of the APIs %DeepSee.CubeManager.RegistryMapGroup:BuildCube %DeepSee.CubeManager.RegistryMapGroup:SynchronizeCube if the cube registry settings declare UpdatePlan="Manual" for the cubes in a given registered group.

      The automated system tasks will still ignore the "Manual" setting as before.

      DTB919: Normalize pivot variables with $$$LOWER in REST dispatch

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Setting pivot variables in the REST service

        /Data/PivotExecute/
      

      via the VARIABLES object in the POST command would not actually apply the variable setting unless the variable name was entered in all lower case. This is corrected so that variables are properly identified regardless of casing entered in the request.

      If the variable name is in all lower case, it would still be applied via the REST call.

      DTB926: Account for NO MEMBER to nullify AND branches in slicer branch processing

      Category: DeepSee
      Platforms: All
      Version: 2018.1.5

      In some cases a slicer restriction that defined a No Data resultset would not be correctly interpreted and would instead have the effect of no slicer at all. This change tightens the bookkeeping on the restrictions that can be known to nullify the resulset prior to examination of the indices.

      DTB936: Protect Resultset %OnClose from <SUBSCRIPT> error

      Category: DeepSee
      Platforms: All
      Version: 2018.1.5

      Clearing the request nodes in a resultset's cache could sometimes log a <SUBSCRIPT> error. This is corrected.

      DTB947: Fix %MDX subquery with

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      This change fixes a problem where queries using a %MDX subquery with the %CONTEXT parameter and a transient calculated member defined by a WITH clause could fail to return results.

      A workaround for instances without this change is to rewrite any %MDX subqueries failing in this way to include a repeat of the WITH clause defined in the primary query.

      DTB952: Correct typo in Intersect with %NOT keys

      Category: DeepSee
      Platforms: All
      Version: 2018.1.5

      Crossjoins between members of levels linked by a dependsOn index, where the second term in the crossjoin is marked with a %NOT operator, could fail to find results. This is corrected.

      DTB955: Add special treatment for %Search references in %GetSpecForAxisNode

      Category: DeepSee
      Platforms: All
      Version: 2018.1.5

      When executing an MDX query with a %KPI function using %CONTEXT on visible axes, an adhoc SQL %Search in the filtering clauses might cause the %KPI to fail with a parsing error. This is corrected.

      MES470: Improve performance of queries with cube relationships

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      This change improves performance of building join indexes for queries involving cube relationships.

      PFS013: Remove fact from cube if update reason = 2

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Restore behavior of removing fact from cube if update reason = 2. A previous change DTB422 in Caché 2016.1.2 removed this behavior and only allowed you to remove a fact by deleting the record from the source table (the ID does not exist in the source class).

      This change allows you to remove a fact from a cube under the following two conditions:

      1) If the ID does not exist in the source class

      2) If the update reason = 2

      PFS014: Properly parse Custom Listing Tree Items for add selected item button

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      When adding a field to a custom listing by using the + button, the field will get added to the listing using the same parsing logic as dragging+dropping and double clicking.

      PFS020: Utilize $auto in Scorecard columns with display=label

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      A Scorecard column that has display=label and label=$auto will now dynamically update the column header to reflect the data.

      PFS022: Account for cube versioning in %GetDimensionMembers

      Category: DeepSee
      Platforms: All
      Version: 2018.1.5

      If cube versioning is active, %GetDimensionMembers now populates the member array with members from either the active version or the specified version

      PFS024: Update pivot table controller name in Analyzer on initial save

      Category: DeepSee
      Platforms: All
      Version: 2018.1.3

      The pivot table controller name is now updated during the initial save. This means that locally stored calculated members will be immediately available.

      PFS027: Apply member restrictions when there are multiple branches

      Category: DeepSee
      Platforms: All
      Version: 2018.1.4

      Member restriction logic in %GetMembers has been improved. Member restriction was not happening as expected in certain cases where advanced logic caused multiple filter branches. This has been corrected and is most visible within the SearchBox component

      PFS028: Improve handling of calculated members in compound subqueries

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      Cube defined calculated members are rewritten for compound subqueries. This rewrite will replace elements that do not exist in the cube the subquery is being run against.

      PFS032: Use source of property when determining SQLColumnName in Field List dialog

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      The Field List dialog will now pass through the source of the property instead of the type of the property when determining the SQL column name.

      PFS039: Add Shared Dimension local overrides to time levels

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Shared time dimensions now pass local override information to levels, preventing cases where errors would be thrown when properties did not exist in both source classes

      PFS042: Generate correct subquery when using operators in advanced filters against related dimensions

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      When a related dimension is used with an operator in an advanced filter, the subqueries were not getting generated properly. With this fix, the subqueries will be generated correctly and errors will no longer be thrown.

      PFS051: Build remote spec for subqueries with $$$UPPER

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Sometimes an axis could generate different subqueries against related dimensions based on the original query text. Now subqueries will always generated the related spec with the upper case version of the spec.

      PFS052: Fix error in %CanonizeRelationKey caused by advanced filter that selects many members

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Using an operator in an advanced filter against a related dimension where the operator causes many members to be selected will no longer throw an error.

      PFS064: Fix error in %MDX and %KPI timefolded queries

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Using %MDX and %KPI functions within MDX with timefolded queries did not produce the correct results. This change corrects this problem.

      PFS072: Do not allow multiple items to be selected when searchBox multiselect=false

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      In the Advanced Filter, when populating a condition like <MEMBER> IS <VALUE>, <VALUE> can only be a single member. When using "Search" in the searchBox component, selecting a member while a different member that does not appear in the search results was already selected allowed for multiple values to be selected. The Advanced filter was not designed to handle this case, so an error will be displayed when trying to execute the query using this item.

      PFS075: Match tIntersectIndex with LabelNode info instead of OrdinalKey

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Queries that use dimensions from a depth 2+ relationship and a filter that has a key that is not the resolved key in a query will no longer throw undefined errors.

      PFS076: Construct proper related spec when resolving a current member against a related cube

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      When using CurrentMember against a depth 2+ related dimension, the related spec was getting generated improperly, which could produce wrong results if the Null Replacement for the relationship were defined. This has been corrected.

      PFS080: Build correct axis when subquery produces no results

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      When a subquery against a related cube did not have any results, the axis for the base cube was not getting generated properly. This is now corrected.

      PFS081: Handle complex %OR specs in Deep Relationships

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Using %OR() or some other specs in a query with related members of depth 2+ on both columns and rows would trigger an optimization that did not properly handle the %OR(). This is now corrected.

      PFS083: When building related spec for subquery, add full spec context

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      When running queries against related dimensions with depth > 1, the full spec context was not getting generated properly. At depth > 3, the full spec context was gone and it would be impossible to reference the correct member (at depth 2,3 it was likely to reference the correct member, but possible to reference the wrong member). This is corrected.

      PFS084: Check moveEnabled before adding dragHandler

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      %ZEN.Component.dragGroup:normalize() now checks if moveEnabled is true before adding the dragHandler.

      PFS089: Map null replaced members in Deep Relationship queries

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      Prior to this change, Deep Relationship queries would throw an undefined error if there was a null replaced member. Null replaced members are now handled properly.

      PFS100: On chart redraw, do not apply selected style to line if markers are used

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      During a redraw of a chart (caused by events like resizing), if a line with markers had a marker selected, the redraw would apply the selected marker style to the entire line. This no longer occurs.

      PFS105: Improve performance of asynchronous synchronize by deleting facts in background

      Category: DeepSee
      Platforms: All
      Version: 2018.1.1

      When running an asynchronous synchronize, deletes are now passed through to be handled by the background process instead of being handled by the main process. This improves the performance of an asynchronous synchronize when many facts are deleted. This change requires you to recompile all DeepSee cubes after an upgrade. If you do not recompile the DeepSee cubes, some cubes will cause an error when they are built.

      PFS108: In Excel export, do not convert empty string to date

      Category: DeepSee
      Platforms: All
      Version: 2018.1.3

      There was a case during Excel export that an empty string would be exported as "1840-12-31", this has been corrected. The cell is now exported with an empty value

      PFS109: Handle nested %OR in %GetFiltersForRelationship

      Category: DeepSee
      Platforms: All
      Version: 2018.1.2

      There was a case where nested %OR functions were not getting added to subqueries properly. This was causing more members than expected to be in the results. Now, the subqueries are getting the proper filter applied and the expected members are being returned.

      DeepSee.User Interface

      DTB609: Improve lookup of KPI display value for URL settings

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      Keys passed into a dashboard via the URL were not getting properly translated in the filter display for KPIs, particularly when passing in multiple and/or excluded values. The handling of filter keys now supports all documented filter values:

      • A single member "&[keyval]" where keyval is the key for the member. See "Key Values" in the DeepSee MDX Reference.
      • A range of members "&[keyval1]:&[keyval2]"
      • A set of members "{&[keyval1],&[keyval2],&[keyval3]}"
      • All members of the level except for a specified single member "%NOT &[keyval]"
      • All members of the level except for a specified subset "%NOT{&[keyval1],&[keyval2],&[keyval3]}"

      DTB675: Properly log folder name for nested folders when rendering User Portal folder list

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      If DeepSee folder items were organized in a folder structure where a primary folder contains only subfolders and no items directly, then collapsing that primary folder would still display the subfolders on screen in the User Portal Home page. This is corrected so that collapsing the top folder hides everything filed below it.

      DTB709: Provide datasource cube extension to searchBox in Advanced Filter dialog

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.3

      The searchBox in the Advanced Filter dialog would not display dimension members if the cube name contains a '.' character. While use of this character is discouraged in logical cube names, it is not rejected and users experienced a regression when this dialog switched to using the searchBox to display its members. This regression is corrected.

      DTB711: Commit navigator's dashboard category to page property

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      The DashboardViewer navigator was not properly committing changes to the dashboard's Category setting and so save would not write the change to the definition. This is corrected.

      DTB733: Change name parsing for shared calculated member deletion

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      A shared calculated member name containing a '.' character could be saved and edited, but could not be deleted. This is corrected.

      DTB744: Adjust calculation of printed element positioning when applying SVG word wrapping

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      SVG printing was not properly positioning text that had been word-wrapped in the original html element. This change tunes the SVG word-wrapping algorithm so that printed text ends up where it is supposed to be.

      DTB747: Reapply AddWidgetNames if sourcecontrol internally reloads a dashboard

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.2

      When a loaded dashboard automatically filled in empty widget names on load, it would not properly reapply those names if the source control hooks reloaded the original definition. The same process is now used to regenerate the names when using source control.

      DTB750: Account for Cube Registry row removal when firing rowClick

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      If a group were removed in the Cube Registry using the red 'x' and another group was immediately added it could throw a Zen exception. This is corrected.

      DTB755: Protect check of component.disabled when editing Architect dialogs

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      A Zen exception could occur when trying to open certain property editing dialogs in the DeepSee Architect. This is corrected.

      DTB765: Support commas in HTML select elements for choosing DeepSee dimensions

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      When defining a display name for dimensions or measures, the inclusion of a comma would cause the select elements in the Advanced filter editor and Calculated Member editors to split a single display name into two lines. This is corrected.

      DTB784: Provide source pivot's measure settings during Excel export from widgets

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      Excel export from DeepSee widgets was not respecting the pivot's settings as defined in the Measure Options. This is corrected.

      DTB793: Initialize output variables in %ParseMemberSpec

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.2

      Filters could produce an Invalid Filter in some cases using multiple selections in deep relationships. This addresses a case where faulty parsing information caused a malformed subquery to be built.

      DTB796: Replace KPI Excel Export FILTER parameters with nonce values in the URL

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      This shortens the URL created in an Excel export of KPIs via a dashboard widget by employing a nonce for each of the FILTER terms that are passed to the Excel export reporting page.

      DTB809: Fix signature mismatch

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      A error could be thrown for MDX time ranges used in certain contexts. This is corrected.

      DTB812: Freeze pivot table row/column headers while scrolling

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.2

      When viewing a large pivot table that requires scrolling either horizontally or vertically, the row/column headers now remain in place so the context is easily referenced at any place in the table.

      DTB828: Correct pivotTable's onclick behavior when paging

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.3

      The onclick action for a multi-page pivot table would only reliably send the correct context from the first page. This change corrects the onclick context for all the later pages.

      DTB830: Use widget's print settings when exporting to Excel

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      The Title, Subtitle and Show Date print settings stored with widget definitions will now be applied to Excel export.

      DTB831: Add DeepSee macros for setting literal axis nodes

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      This provides a common macro for setting axis nodes that are literals

      $$$DeepSeeLitAxisNode(%parent,%label,%value) --> $LB("lit",1,1,%parent,%label,%value)

      There is also a second macro that can set a literal null

      $$$DeepSeeNullAxisNode(%parent) --> $$$DeepSeeLitAxisNode(%parent,"","")

      DTB833: Do not attempt autosave if dashboard definition cannot be modified

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      On some browsers the combination of dashboard autosave and NOMODIFY=1 in the URL settings could throw an alert This dashboard is read only and cannot be modified. The page now checks for write access before attempting the autosave.

      DTB841: Fix call to control timeout make sure the correct control calls setDisabled

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      The call to disable a listing control for a short time after toggling could fail to call re-enable for the correct control. This is corrected.

      DTB848: Do not process expression when loading into Advanced Filter Editor

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      The expression being loaded into the Advanced Filter Editor was over-processed when being converted to a graphical display, causing some referential keys (for example, NOW) to be resolved into concrete keys.

      DTB850: Correct offset calculation for column totals header

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      Corrections to column header alignments corrupted the positioning of the summary header. This corrects that positioning.

      DTB852: Reduce size of resize handle below the Analyzer's controller table

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      There is an invisible grab handle below the controller table (containing the Rows, columns, and other definitions) which is available for users to vertically resize that space. The bar was tall enough that it could interfere with the mouse cursor interacting with the filters placed in the filter bar. This is reduced in height so the mouse clears the handle before it is over the searchBox's magnifying glass.

      DTB861: Correct summary column header alignment issues that occur during pivotTable paging

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      Pivot tables with multiple pages of results and column summaries turned on could experience problems with alignment issues. This was most likely to occur when the rows contained a single level as the first member of the set on rows, and a crossjoin as the second member of that set, for example:

        SELECT 
          NON EMPTY [PatGrpD].[H1].[Patient Group].Members ON 0,
          NON EMPTY {[BirthD].[H1].[Date].Members,NONEMPTYCROSSJOIN([AllerD].[H1].[Allergies].Members,[AgeD].[H1].[Age Group].Members)} ON 1 
        FROM [PATIENTS]
      
      This issue is corrected.

      DTB873: Verify query is complete before displaying No Results in pivotTable

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      Certain queries that invoke the background axis processing in the pivotTable could return No Results (6) after a reset. These same queries would return full results after a reload of the pivotTable. The table now has an extra confirmation that the results are complete before making this particular decision that it has completed with no results.

      DTB877: Execute listings after background query completes

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      Listings executed under %ExecuteAsynch could fail to fetch the actual results due to a timing issue. This is corrected.

      DTB881: Remove breakout of cell width search when calculating width of listing headers

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      The pivot table's listing code has some special considerations that were needed for older Internet Explorer browsers. One of these interfered with the calculation of the column widths.

      DTB916: Use nonces for export parameters that are commonly many characters

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.4

      Several parameters in the URL communication used for PDF and Excel exports in Analyzer and the Dashboard widgets can have a character length much longer than anticipated which in turn can break the URL length limit on Internet Explorer in a variety of different combinations. This encodes many of these parameters using a numeric nonce that removes the length from the URL to make errors in printing in Internet Explorer much less common.

      This applies to all Excel exports and the PDF print cases not supported by SVG printing.

      The workaround is to use any other browser.

      DTB973: Protect property type check in %CreateControls

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.5

      A widget on a dashboard which used a custom control could throw a <PROPERTY DOES NOT EXIST> error on dashboard load if that custom control does not have a defined property 'type'. This is corrected.

      PFS004: Show meaningful message when a dimension has too many members to display in Analyzer Member Tree

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      If a level is expanded in Analyzer with more than 10,000 members, the message "Too many members to display" is now shown.

      PFS026: Dynamically update Legend Title

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.2

      If the Legend Title is not explicitly defined, the Legend Title will now be dynamically updated as the pivot sources change. This is true for the set/choose spec controls as well as the set/choose datasource controls.

      PFS087: Print pivot table drilldown labels to PDF

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      Prior to this change, trying to print a pivot table to PDF while in a drilled down state, the row label text would not be displayed. This has been corrected

      PFS091: Always update level caption when drawing pivot table

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      Prior to this change, it was possible that a level on a pivot table retained the old caption after replacing that level with a new one. When drawing the pivot table, the caption is now always updated, which will be reflected in the label updating in cases when it did not previously.

      PFS092: Prevent failure printing pivot table to PDF

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      Prior to this change, printing a pivot table to PDF could fail if a cell were clicked before printing. This has been corrected.

      PFS095: Ok button now closes Image Upload dialog after saving

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      Prior to this change, the OK button would not close the dialog after the Save button had been clicked. This has been corrected

      PFS096: Cast searchBox NOW offset as int to prevent concat when we want addition

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      When loading the searchBox component members, it was possible to see the calendar load to an unexpected month when using NOW+<offset>. This has been corrected.

      PFS101: Use disabled property to better control source property/expression editing in architect

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      The Source Property and Source Expression fields in Architect will now be more strictly controlled by the radio buttons. When the radio button is selected for either the Source Property or Source Expression, the text input for the other option will be disabled and the text will be cleared.

      PFS102: Build searchBox list of values before any action is taken

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      There were certain cases where a searchBox could forget the current value when performing specific actions after the initial load of the searchBox. The state of the searchBox is now updated before any action is performed.

      PFS103: Fix multi cell drillthrough filter generation

      Category: DeepSee.User Interface
      Platforms: All
      Version: 2018.1.1

      There was an issue preventing the correct filter from being generated when viewing a listing across multiple cells from different contexts. This has been corrected.

      Ensemble

      DP-5315: EnsLib.HTTP.GenericMessage now explicitly defines XMLNAME parameter as well as XMLTYPE

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      With this change, Ens.StreamContainer and its subclasses can be deserialized from XML. Ens.StreamContainer and subclasses EnsLib.HTTP.GenericMessage,EnsLib.SOAP.GenericMessage,EnsLib.REST.GenericMessage and Ens.MFT.StreamContainer can now be deserialized from XML. This means for example they can be resent from the Message Bank or tested in the DTL editor.

      As part of the correction, EnsLib.HTTP.GenericMessage now explicitly defines XMLNAME parameter as well as XMLTYPE. If you have defined any custom subclasses, you need to modify them to account for this change; override the XMLNAME parameter as required for your code.

      DP-6920: New X12 validation may impact previously ignored flags

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      This change adds element-level validation for X12 documents.

      The behavior for existing X12 validation flags in routers is unchanged. However, in the unlikely event that you have specified extra validation flags that were not previously valid, the behavior may change. If you specified a nonexistent validation flag, it would have been ignored, but if the flag now matches a new validation flag, the specified validation will be performed.

      For a list of the new validation flags, see Validation in Routing X12 Documents in Productions.

      JGM589: Correct Ensemble recommendation for password recording of SSLConfig settings

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      The Ensemble class documentation for the following classes EnsLib.EMail.InboundAdapter, EnsLib.EMail.OutboundAdapter, EnsLib.HTTP.OutboundAdapter,EnsLib.Telnet.OutboundAdapter,EnsLib.TCP.DuplexAdapter,EnsLib.TCP.Common stated that if a private key password is required for the SSL configuration then it can be appended to the SSLConfig name in the setting after a |. This is not recommended. It is recommended to use the Private Key Password property of the SSL Configuration and the call documentation has been amended accordingly.

      JGM611: Support specifying local interface for EnsLib HTTP Outbound Adapter

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      This change adds a configuration Setting called LocalInterface to the Ensemble HTTP Outbound Adapter. This setting controls binding of the Adapter's TCP connection(s) to a specific local network interface.

      The default value for this setting is empty which means use any interface and which is compatible with the previous behavior of the Adapters.

      JGM637: Creation of temporary and secondary databases for interoperable namespaces to trap remote default db

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      When creating an interoperable enabled namespace in a non HS instance 2 extra databases are created - one for temporary non journalled data and a Secondary one for storing passwords. The code would trap an error if the default database was remote. The code now checks if the default database is remote and reports in the creation log that it is not creating the new database(s) on the remote instance. that is:

      1. Examining if Ensemble temporary global mapping already configured
        1. Creating Ensemble temporary Data location
        2. Default Globals DB for Namespace is remote. Not creating Temporary Database
      2. Examining if secondary global mapping already configured
        1. Creating Seconday Data location
        2. Default Globals DB for Namespace is remote. Not creating Secondary Database

      JGM640: Ens.Alarm process to trap missing data

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      The Ens.Alarm process could loop indefinitely if persisted alarm data was missing. This has now been corrected.

      JGM643: HTTP, REST and SOAP Generic Messages are now stored in the message bank using XML projection

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      When the Message Bank Operation banks HTTP, REST and SOAP Generic Messages the HttpHeaders array were not being included in the banked messages. This was because all sub classes of Ens.StreamContainer were banked as Ens.StreamContainer and not as an XML export. HTTP, REST and SOAP Generic Messages are now banked as XML Export projection so that the properties can be examined on the Message Bank.

      It is now possible to resend Generic messages and Ens.StreamContainer messages from the Message Bank using the related change JGM798.

      JGM712: EnsLib ValidateSAML to allow checking for unsigned Assertion element

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      The utility api Ens.Util.XML.SecuritySignature provides some validation for the saml:Assertion element. It required that the assertion element was signed. It is now possible to carry validation of a stream containing an unsigned Assertion element but having an element that contains the assertion to be signed. There are 2 new options: u to require an unsigned Assertion and s that when used with u requires the unsigned assertion to be wrapped by a signed element.

      Note this does not check the schema.

      Check signatures and expiration as specified by pValSpec. This does not validate the XML schema used for the SAML token.

      pValSpec Specifies types of Assertion validation to perform:

      • t - must contain a signed token.
      • a - token must contain a signed Assertion. If not found the error text is "No Assertion".
      • u - token must contain an unsigned Assertion. If not found the error text is "No Unsigned Assertion". If both a and u are specified then either a signed or unsigned assertion needs to be present.
      • s - combine with u - if unsigned assertions exist the s requires them be a children of signed elements. Note: The Assertion might be wrapped in a structure that does not follow from schema.
      • r - require Assertions to contain both NotBefore and NotOnOrAfter time conditions.
      • v - verify Assertion signature and, if present, NotBefore/NotOnOrAfter conditions. If option 'u' is specified and 'v' NotBefore/NotOnOrAfter conditions will also be checked.
      • o - validate other signed nodes within the assertion such as TimeStamp. Signed reference elements with attribute name of ID or Id will be searched for.

      Set pClockSkew to the desired number of seconds or to -1 to prevent NotBefore/NotOnOrAfter condition checking.

      To carry out schema validation of the input stream create an instance of %XML.Reader, setting the appropriate properties for validation and pass in as optional parameter pXMLReader.

      The ValidateSAML method also has 2 new optional parameters:

      • The 7th parameter is an optional output of the information gathered during the reading of the input stream concering signed and unsigned nodes.
      • The 8th parameter is an option instance of %XML.Reader that the method will use instead of creating one. This allows the caller to create an instance of %XML.Reader and setting the desired properties for schema validation and pass in as optional parameter pXMLReader.

      JGM739: EnsLib SOAP Outbound Adapter to reset HTTP Headers after success or failure but only if setting SuperSession

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      The Ensemble SOAP Outbound adapter would clear its soap client's HTTP Headers after a successful send.

      If the adapter could not send and the host was set to retry the HTTP Headers would not be cleared. This led to a problem if Send Super Session was enabled since for each retry a Super Session header would be added.

      This has been resolved by clearing the HTTP Headers after success or failure.

      JGM748: Increase length of event log trace Text property

      Category: Ensemble
      Platforms: All
      Version: 2018.1.5

      When recording an event log entry the text information was trunccated to 1200 characters. This has now been extended to 32000.

      JGM765: Increase allowed length of Complex Record Map names

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      Complex Record Maps names (see Using the Complex Record Mapper) were limited to 50 characters. The limit is now increased to 200.

      JGM778: Modify %IO.FIleStream::NewTempFilename algorithm

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      The %IO.FIleStream::NewTempFilename algorithm might return the same filename to concurrent jobs. The algorithm has been been modified to use the %Stream.FileBinary::NewFileName() algorithm.

      JGM782: Interoperability Studio requires users to have execute privilege stored procedure Ens_Config.Production_Extent

      Category: Ensemble
      Platforms: All
      Version: 2018.1.3

      Developers using Studio in namespaces with interoperability productions enabled require execute privilege on the stored procedures Ens_Config.Production_Extent.

      This is granted to the roles %EnsRole_Developer and %EnsRole_Administrator.

      JGM783: Correct Interoperability monitoring's possible repeated sending of Queue Wait alerts for a host

      Category: Ensemble
      Platforms: All
      Version: 2018.1.5

      The Ens.MonitorService checks queue wait time for chosen host items. If a queue wait alert is sent for a host item (either by detection from active message or message at the front of the queue) it was possible for the monitor to send another alert for the host item before the queue wait is cleared (that is below threshold wait).

      This is now corrected.

      Note: if the production is stopped then the Ens.MonitorService might send a 'repeated' alert for an item before the queue wait is cleared because the in-memory cache of items alerted is cleared when the Ens.MonitorService stops.

      JGM911: Interoperability Record Map File and FTP services to be able to skip invalid records

      Category: Ensemble
      Platforms: All
      Version: 2018.1.5

      The EnsLib.RecordMap.Service.FileService and EnsLib.RecordMap.Service.FTPService stop processing a source file if a record fails validation, for example a field is too long. A new setting is provided "Fatal Errors" which is similar to that in the batch services. In the EnsLib.RecordMap.Service.FileService and EnsLib.RecordMap.Service.FTPService processing the new FileErrors setting can be either "Any" or "ParseOnly and is as follows:

      An error saving an individual Record, such as a validation error, will be treated as fatal and end the processing of the message when the setting is "Any". This is the current behaviour.

      If "ParseOnly" is selected, errors when saving individual Records will not be treated as fatal, and parsing of the message will continue after logging an error log and skipping the errored record.

      If AlertOnError is enabled, an alert will be sent for a save error when "ParseOnly" is selected.

      The default is "Any".

      The error reported if ParseOnly indicates the position in the stream for the invalid record.

      Ensemble Java Bindings

      MC2563: Add JMS Inbound and Outbound Adapters and Business Service and Operation

      Category: Ensemble Java Bindings
      Platforms: All
      Version: 2018.1.2

      This change adds the capability to send and receive messages using the Java Messaging Service (JMS) in productions. The feature includes the following:

      • EnsLib.JMS.Service Business Service
      • EnsLib.JMS.InboundAdapter
      • EnsLib.JMS.Operation Business Operation
      • EnsLib.JMS.OutboundAdapter
      • EnsLib.JMS.Message class for messages

      The jar files for this feature are available in:

      install-dir\dev\java\lib\JDK18\cache-enslib-jms-2.0.0.jar

      The following client development files are also available:

      install-dir\dev\java\jms\proxy-classes.xml

      install-dir\dev\java\jms\wljmsclient.jar

      install-dir\dev\java\jms\wlthint3client.jar

      Javadocs documentation for the Java classes is available in:

      install-dir\dev\java\doc\cache-enslib-jms\index.html

      Ensemble.Adapter

      JGM633: Expose FTP Setting CommandTranslateTable to FTP adapters

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.5

      The FTP adapters now expose the setting CommandTranslateTable. ( See http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GNET_ftp_command_translate_table).

      The default is empty string to keep current behavior and only applies to FTP not SFTP

      This setting is under a new section called FTP Settings.

      For FTP Protocol the translate table to use for the command channel, specifically for the filename/pathnames. Normally this should not be specified in which case if the ftp server supports UTF-8 then we will use that for the filename/pathnames, if the server does not support UTF-8 then we will use RAW mode and just read the bytes as sent.

      The values shown in the display list are the internal table names.

      JGM719: QueueWaitTime to also check active message time for single job hosts

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.5

      For items with a single job the amount of time the active message is active is also checked against the queue wait time - this is to allow for the case where there is no queue.

      When an item moves from queue to active or vice versa we account for the updated TimeProcessed and thus will still remember the message as being delayed.

      The number of seconds a message at the front of the queue may have waited since being queued before an alert is triggered.

      For items with a single job the amount of time the active message is active is also checked against the queue wait time - this is to allow for the case where there is no queue.

      Only one alert will be raised per host item per sequential trigger of the queue wait threshold. Note that this alert will be sent even if AlertOnError is False.

      Zero means no alerts of this type will be sent.

      JGM728: QueueWaitTime to alert if Active message delay detected after queue wait detected

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.5

      For items with a single job the amount of time the active message is active is also checked against the queue wait time -- this is to allow for the case where there is no queue.

      The Monitor Service attempts to not to re-alert on queue wait alert if it has already alerted for a queue wait delay and the queue has remained in delay state.

      The Monitor Service now also log an Information log when a queue is detected as no longer being delayed.

      JGM764: Clearing of Queue Wait alert flag is based on percentage of the specified queue wait alert

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.5

      If a Queue Wait alert has been triggered then the clearing of the known delay happens when the queue delay time for the item at the head of the queue is now based on being less than 80% of the Queue Wait Alert time setting. This is to prevent false re-alerting as a queue is drained.

      The information message states:

      Queue Wait: Resetting queue wait alert for config item 'toSelf'. It now has a delay less than 8 seconds (permitted alert delay of 10) or an empty queue.

      The text identifying if the alert is due to Host being disabled has been amended to aid text searching with the word QueueWait added:

      QueueWaitAlert: Message Header Id '14143' queued for config item 'toSelf' with priority 'Async' has been queued for more than 10 seconds (QueueWait Host is disabled) (alert request ID=26)

      The change is made to the alert identifying if the queue delay is in the active message:

      QueueWaitAlert: Message Header Id '14161' queued for config item 'toSelf' with priority 'Async' has been queued for more than 10 seconds (QueueWait Message is active) (alert request ID=28)

      JGM767: Correct TCP Framed adapter reading data longer than 910286 characters

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.3

      The EnsLib TCP Framed Adapter read inbound data in chunks of no more than 910286 characters at a time. If the inbound data is more than this length (including framing characters) and the ending framing characters are not included in a 'chunked' read then the last character read was duplicated. This is now corrected.

      JGM799: EnsLib.HTTP.Service to let Web Gateway set Content-Length or send chunked

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.3

      The EnsLib.HTTP.Service would set the %response Contentlength in OnPage() if there was a response object returned by ProcessInput. It was possible that it could set the wrong length depending on encoding.

      The EnsLib.HTTP.Service now does not set the %response.ContentLength.

      As a result the Web Gateway will either send the response using chunked transfer encoding, or calculate and add a content-length header. The default is for the Web Gateway to use chunked transfer encoding, where a content-length header is unnecessary and is not added. If the response is small enough for the Web Gateway to fit into its internal response buffer, then the content-length header is added automatically.

      It is still possible to set %response.ContentLength in a subclass.

      JGM801: Correct ASTM TCP Adapter handling of EOTOPTIONAL when one message and EOT delayed

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.3

      The ASTM TCP Adapter supports receiving consecutive ASTM E1394 messages not separated by ENQ/EOT control characters but just run together on the wire over a TCP connection when the service class parameter EOTOPTIONAL is defined as 1.

      If only one ASTM E1394 message between the ENQ and EOT control characters was sent and there was a delay before the EOT control character was sent the adapter might report the error "Received unexpected ASTM ENQ ACK character: Ascii <EOT>"

      This is now corrected.

      JGM837: Interoperable SOAP Outbound Adapter to be able to set Write Timeout

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.5

      The underlying HTTP request code in %Net.HttpRequest allows specifying a timeout value for writes to the web server.

      This change exposes a new setting in the EnsLib.SOAPP.OutboundAdapter "Write Timeout" (localized) which is used to specify the timeout value for writes to the web server. The default is the same as current behavior - namely to wait for an unlimited time.

      JGM938: Expose SFTP Session RemoteCharset and LocalCharset to the Interoperability FTP adapter

      Category: Ensemble.Adapter
      Platforms: Windows
      Version: 2018.1.5

      The underlying SFTP classes used by the Interoperability SFTP adpaters now implement two properties for the encoding of filenames: RemoteCharset and LocalCharset.

      The introduction of these properties changed the default behavior since SSH draft working group standards give filename representation in the UTF-8 character set.

      The ability to set these 2 properties is now exposed in the Interoperability FTP adapters as

      SFTP Server Character Set (SFTPRemoteCharset)

      Character set for file names used by the remote server. Defaults to UTF8. Set to empty string for no translation on the character sets for filenames. This setting is used to set the RemoteCharset property in the %Net.SSH.Session object.

      SFTP Local Character Set

      Character set used by the local system for filename encoding. For Windows the default is the empty setting which will leave local filenames as Unicode For UNIX the default is to convert to UTF8 This setting is used to set the LocalCharset in the %Net.SSH.Session object

      JGM940: Interoperability MQ Series Inbound and Outbound adapter to support username and password

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.5

      Interoperability MQ Series Inbound and Outbound adapter to support username and password authentication as provided by the core MQ classes.

      The username and password are obtained from the exposed Credentials setting.

      MC2560: EnsLib.JavaGateway.Service to report error from CmdLineForJava

      Category: Ensemble.Adapter
      Platforms: All
      Version: 2018.1.3

      While testing JMS, we encountered a problem on a machine where the Java version is wrong.

      EnsLib.JavaGateway.Service:RunJava calls %Net.Remote.Service:CmdLineForJava. %Net.Remote.Service:CmdLineForJava notices this problem, and returns an error with correct description, stating the Java version is wrong.

      However, in EnsLib.JavaGateway.Service:RunJava, after the call to %Net.Remote.Service:CmdLineForJava fails, it ignores the error returned, and just returns $$$EnsSystemError to its caller.

      The result of this is that the error we see in Ensemble Event Log is: Failed to start the Java Gateway server: ERROR <Ens>ErrException: -- logged as '-' number - @''

      EnsLib.JavaGateway.Service now returns the more useful error message that's returned from CmdLineForJava.

      Ensemble.BPL

      JGM581: Correct BPL Group selected items As a Scope

      Category: Ensemble.BPL
      Platforms: All
      Version: 2018.1.4

      In the Ensemble BPL editor when grouping selected items as a Scope the selected items would be lost on compile. This is now corrected.

      Ensemble.DICOM Adapter

      KDS338: Ensure Transfer Syntax Sub-Item is Included in Presentation Context Item of A-ASSOCIATE-AC

      Category: Ensemble.DICOM Adapter
      Platforms: All
      Version: 2018.1.4

      Each Presentation Context Item within a DICOM A-ASSOCIATE-AC includes fields for item-type, item-length, presentation contex-ID, result/reason code, and the transfer syntax sub-item. Although the transfer syntax sub-item value is not significant unless the Result/Reason code is 0 for acceptance, it is still expected to be present. However, we had been failing to include it when Result/Reason was not acceptance. This change ensures that we do include it even when the transfer-syntax-name field is empty.

      KDS339: Send Correct Reason Information For Rejected Presentation Contexts in A-ASSOCIATE-AC

      Category: Ensemble.DICOM Adapter
      Platforms: All
      Version: 2018.1.4

      One of the fields in each Presentation Context item within a DICOM A-ASSOCIATE-AC is the result/reason. This indicates whether the Presentation Context was accepted, and also the reason if it was not. Previously, we had set this value to 3 (abstract-syntax-not-supported) for any Presentation Context which was not accepted. Now, if the problem is not the abstract syntax, but rather that there is no match among the transfer syntaxes for it, then we will set the value appropriately, so that it is 4 (transfer-syntax-not-supported).

      Ensemble.DTL

      JGM657: Correct error with DTL FunctionWizard if function does not have parameters

      Category: Ensemble.DTL
      Platforms: All
      Version: 2018.1.1

      A JavaScript error would be thrown when selecting a function without parameters from the DTL Function Wizard. This is now corrected.

      Ensemble.Editors

      JGM915: Correct Interoperability BPL Editor validation of scripts entered in Value fields

      Category: Ensemble.Editors
      Platforms: All
      Version: 2018.1.4

      The mechanism used by the BPL Editor to validate Objectscript values in a BPL's Value fields was incorrect. The mechanism has been removed.

      Ensemble.HL7/EDI

      JGM685: Correct X12 parsing BIN & BDS segments*

      Category: Ensemble.HL7/EDI
      Platforms: All
      Version: 2018.1.3

      The parsing of X12 BIN & BDS segments was failing. This is now corrected.

      JGM856: HL7 Parser Event Log Warning replaced by TraceCat entry when segment name pattern unexpected

      Category: Ensemble.HL7/EDI
      Platforms: All
      Version: 2018.1.4

      The HL7 Parser would create an event log warning for every segment where the segment name was not either 3 upper case letters, 2 upper case letters followed by a number or the letter Z followed by 1 to 5 characters (one of which could be the underscore).

      The warning is no longer recorded. Instead the check will be made and an event log Trace created instead when ^Ens.Debug("TraceCat","parse") or ^Ens.Debug("TraceCat") is set to true.

      Note validation of the message is not affected.

      JSL5241: Fix XMLVDOC failure when importing schema into system with non-standard collation

      Category: Ensemble.HL7/EDI
      Platforms: All
      Version: 2018.1.1

      Under certain circumstances importing an XML virtual document schema can fail on a system with a nonstandard collation. These schemas will not work correctly. This change corrects this error. With this change Ensemble imports the schema correctly, but the change does not correct schemas that were imported before the change.

      KDS341: Avoid Quitting Out of Wild Assign in DTL Due to Missing Source if IgnoreMissingSource is True

      Category: Ensemble.HL7/EDI
      Platforms: All
      Version: 2018.1.4

      In DTL, there is special handling in assign actions for when both the property (target) and value (source) are virtual document property paths. If the value's property path includes a pair of empty parentheses, then it is regarded as "wild" and the DTL will loop over each index in fetching values. If the property's property path also includes a pair of empty parentheses, then each of the values is assigned to the corresponding index in the property; otherwise, each of the values gets assigned in order to the property, resulting in the property being assigned the value of the last index within the value's property path. Either way, there is always the possibility that there is no segment defined at some index in the value's property path. Prior to this change, this would always result in the DTL quitting once it reached the missing segment even though there were other indices after it that were defined. After this change, the DTL only quits at this point if IgnoreMissingSource is not turned on. If IgnoreMissingSource is turned on, it continues going through the remaining indexes.

      KDS439: Avoid Deleting Segment Global for Open Documents

      Category: Ensemble.HL7/EDI
      Platforms: All
      Version: 2018.1.5

      In certain circumstances, when a virtual document (HL7, EDIFACT, ASTM, or X12) was cloned, but then the document was either closed or deleted, the segment data for the clone document would also be deleted. This would result in Invalid Oref errors if someone tried to access the clone's segments. This change fixes that problem.

      Ensemble.ManagementPortal

      JGM580: Correct Ensemble Message Bank Resending of XML serialized messages

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.4

      When a serialized XML enabled class is resent from the Message Bank it was possible for the correlation to original class to fail if the character encoding was not the expected. This is now corrected.

      JGM707: Correct JS error in Managed Alerts Viewer page on refresh

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.2

      When the Managed Alerts Viewer was set to automatically refresh a JavaScript error could be thrown. This is now corrected

      JGM743: Correct Ensemble generation of Production Documentation

      Category: Ensemble.ManagementPortal
      Platforms: Windows
      Version: 2018.1.2

      The creation of production PDF documentation on Windows encountered an error due to using a banner image. This is now corrected.

      JGM752: Do not delete uncompiled Rule in Rule Editor if not created in that page session

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.1

      The Rule Editor will delete an uncompiled rule on leaving the page or on issuing a Save As. This could lead to deleting a rule that existed before entering the Rule Editor page. This is now corrected.

      JGM790: Correct UNDEFINED when exporting all event logs

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.2

      When exporting more than one page of event log entries an UNDEFINED error would be thrown. This is now corrected.

      JGM795: Improve efficiency of Interoperability Data Lookup Tables page

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.4

      Interoperability Data Lookup Tables page was making a round trip to the server for each value displayed. This is now corrected. The inefficiency could lead to failure to display the page for large tables.

      JGM844: Allow changing Web Application for non Health Interoperability Management Portal pages

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.4

      Non Health Interoperability Management Portal pages (such as Configure, Production) for a given namespace can now be set to use a different web application when linking from the main System Management Portal.

      To do this create a Web Application that is a copy of the Web Application created for the namespace. Set the global ^%SYS("Ensemble","InstalledNamespace",)=

      JGM849: Correct size of Enterprise Monitor Status portal page table size

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.4

      The calculation of the table in the Enterprise Monitor Status page was incorrect and the end of the table might not be visible using the vertical scroll bar depending on the number of clients and the page size. This is now corrected

      JGM864: Interoperability Enterprise Monitor Status Page to show a maximum of 1000 clients (up from 100)

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.4

      The interoperability Enterprise Monitor Page showed a maximum of 100 clients and the paging buttons of Next and Previous did not work.

      This change increases the maximum number of clients to show to be 1000 and removes the non functioning Next and Previous buttons

      JGM866: Optimize Event log query used by Interoperability Production Monitor

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.4

      The Interoperability Production Monitor shows top 10 most recent event log entries. The query to find these has been improved to avoid possible delays when there are a large number of event log entries.

      KDS433: Add Empty Option to Settings Dropdown For Non-Required Properties On Production Config Page

      Category: Ensemble.ManagementPortal
      Platforms: All
      Version: 2018.1.3

      If a configuration item in a production includes a setting for which the underlying class property has a VALUELIST (and possibly a DISPLAYLIST), then that setting's value appears on the Production Configuration page in a combobox. Prior to this change, the only options that could be selected in this combobox were the ones listed in the VALUELIST/DISPLAYLIST. Now, if the property is not marked as Required, the empty value will also be added to the options which can be selected as that setting's value.

      External Languages

      DP-288152: Introduce support for Node.JS version 10.x.x

      Category: External Languages
      Platforms: All
      Version: 2018.1.6

      This change represents support for Node.JS version 10.x.x. This is a Long Term Support (LTS) release.

      Node.js v10.0.0 was released in April 2018 and ships with V8 version 6.6.346.24

      The 'magic module number' to ensure binary compatibility between releases: 64 (see: https://nodejs.org/en/download/releases/).

      DP-7886: Introduce support for Node.JS version 12.x.x

      Category: External Languages
      Platforms: All
      Version: 2018.1.6

      This change supports Node.JS version 12.x.x.

      Gateways - ODBC

      DP-404515: Use correct data length for SQLGetData SQL_C_WCHAR bufsize 0

      Category: Gateways - ODBC
      Platforms: All
      Version: 2018.1.6

      Fixes a bug where calling SQLGetData twice, the first time with buffer length 0, caused the second call to return an incorrect data length for wide-char (SQL_C_WCHAR) queries.

      Global Module

      DP-422520: Fix process private global referencing the wrong blocks on big-endian (AIX) unicode systems

      Category: Global Module
      Platforms: AIX
      Version: 2018.1.9

      Previously, on unicode installations on a big-endian platform, references to a process private global could access data in IRISTEMP that are not part of the global's tree. This issue has been resolved.

      JO3169: Correction to not fetch 4 bytes for 3 byte global names

      Category: Global Module
      Platforms: All
      Version: 2018.1.5

      A rare issue which could lead to an access violation from a global reference with a 3 character global name has been resolved.

      RJF423: Fix database compaction could bring purged block into buffers

      Category: Global Module
      Platforms: All
      Version: 2018.1.4

      Under rare conditions, database compaction or defragmentation (but not GCOMPACT), could cause a global reference or update in another process to get unexpected results (update could be lost or reference could return old data). This problem has existed since the inception of database compaction.

      RJF424: Fix database degradation caused by global compaction adding a pointer level

      Category: Global Module
      Platforms: All
      Version: 2018.1.3

      Under certain rare conditions, GCOMPACT, database compaction or defragmentation could lead to database degradation. This problem was introduced in Caché 2016.2.

      IDEs

      DP-292447: Properly check document timestamp in %Atelier.v1.Utils.General:TS()

      Category: IDEs
      Platforms: All
      Version: 2018.1.6

      Ensure that the IDE uses the same logic as Studio when checking the timestamp of a document. This only affects users who use server-side source control.

      DP-292469: Add new "udl-multiline" option to "format" URL parameter on GET /doc/ endpoint

      Category: IDEs
      Platforms: All
      Version: 2018.1.6

      Add "udl-multiline" option for GET /doc/ "format" URL parameter. Passing ?format=udl-multiline will return the document with method arguments formatted on multiple lines. Also bumped API version to 4 to reflect that the schema changed.

      DP-292510: Add support for "location" on POST /action/index endpoint

      Category: IDEs
      Platforms: All
      Version: 2018.1.6

      Add support for location mapping between documents when doing a POST /action/index request. To map the location in a document to the corresponding location in "other" documents (like the generated INT for a class), append the label+offset to the end of the document name separated by a colon (i.e. [ "%Activate.Enum.cls" ] becomes [ "%Activate.Enum.cls:method+5" ]) and all "other" documents in the response body will contain the document name and corresponding offset separated by a colon:

      "others": [
          "%Activate.Enum.1.INT:+11"
          
      ]

      DP-400052: Properly return SQL results containing braces in POST /action/query endpoint

      Category: IDEs
      Platforms: All
      Version: 2018.1.6

      This change fixes a bug where SQL result strings that contain braces or curly braces cause the POST /action/query endpoint to report a parsing error.

      DP-405736: Fix Uncomment Line off-by-one error in Studio

      Category: IDEs
      Platforms: All
      Version: 2018.1.6

      This change fixes a bug wherein Studio's Edit > Advanced > Uncomment Line and Uncomment Block commands would fail to uncomment the last selected line.

      DP-417920: Correct access to the Atelier debug agent

      Category: IDEs
      Platforms: All
      Version: 2018.1.9

      This change fixes an earlier change that ensures that the Atelier debug agent is dispatched to from a REST context. Before this fix, it was possible to receive the error ERROR #5916: Illegal Web Request [zOnPreServer+7^%Atelier.v5.XDebugAgent.1:%SYS]

      iKnow

      JDN274: Handle Windows newline (\r\n) pattern correctly on Linux platforms

      Category: iKnow
      Platforms: Windows
      Version: 2018.1.1

      When text files are read in binary mode, the Windows end-of-line pattern '\r\n' is not translated to the newline pattern '\n'. The iKnow engine handled this double byte pattern as a newline, but only on Windows platforms. Handling such text files on Linux machines resulted therefore in possibly slightly different behavior, leading to different linguistic output. This change corrects this problem. All platforms treat the newline pattern in the same way.

      Installation

      ALE3190: Set correct version string in registry in custom install with all features selected

      Category: Installation
      Platforms: Windows
      Version: 2018.1.1

      This change solves a problem where Windows install was not setting product version string in custom install with all features selected.

      ALE3239: do not re-create USER namespace in Windows upgrade

      Category: Installation
      Platforms: Windows
      Version: 2018.1.3

      Windows upgrade will no longer re-create USER namespace if it was deleting in the original instance.

      ALE3278: Preserve "System Manager Machines" parameter in the Web Gateway on upgrade

      Category: Installation
      Platforms: Windows
      Version: 2018.1.2

      Windows upgrade will no longer reset "System Manager Machines" parameter in the Web Gateway on upgrade to "*.*.*.*".

      ALE3286: Statically link openssl libraries in private web server

      Category: Installation
      Platforms: UNIX®
      Version: 2018.1.3

      Apache httpd on lnxrhx64 will have openssl libraries linked statically.

      ALE3294: Copy files if they are the same version as existing files in WebGateway install on Windows

      Category: Installation
      Platforms: Windows
      Version: 2018.1.2

      Web Gateway install on Windows will replace existing CSPGateway files if they are the same version as kit files. In previous cases, it was not necessary to replace the files if they were the same version, but in going from CSP Gateway to Web Gateway it is required.

      ALE3295: Install Core 2.1 packages on UNIX

      Category: Installation
      Platforms: UNIX®
      Version: 2018.1.2

      .NET Core 2.1 nupkg files will be installed on UNIX into the [INSTALLDIR]/dev/dotnet/bin/Core21 directory.

      ALE3318: Add CacheServices group permissions to Property key on upgrade

      Category: Installation
      Platforms: Windows
      Version: 2018.1.4

      On upgrade when service is run from a defined user account install will add CacheServices group write permissions for Properties registry key.

      ALE3337: Install check that instance name, directory and port numbers are not in use by InterSystems IRIS instances

      Category: Installation
      Platforms: UNIX®,Windows
      Version: 2018.1.4

      Windows and UNIX install will check that instance name and directory are not taken by InterSystem IRIS instances. It will also set the default values for instance name, installation directory and port number to values which are not taken by any instances.

      ALE3343: Install 32-bit SSL libraries into ssl32 directory in ODBC standalone installer

      Category: Installation
      Platforms: Windows
      Version: 2018.1.4

      The 32-bit ODBC standalone installer will install libeay32.dll and sslaey32.dll into ssl32 directory to match installation location in full server kit.

      ALE3346: Identify unattended install on Windows as custom install

      Category: Installation
      Platforms: Windows
      Version: 2018.1.5

      Unattended install on Windows will always identify as custom install type.

      ALE3369: Create Web Gateway INI files if needed before atempting to modify them

      Category: Installation
      Platforms: UNIX®
      Version: 2018.1.4

      UNIX Web Gateway install will create CSP.ini, CSP.log and CSPRT.ini if they don't exist before making any modifications to CSP.ini.

      CDS3063: Prompt for database encryption key during upgrade installation

      Category: Installation
      Platforms: Windows
      Version: 2018.1.2

      When an instance with a database encryption key was upgraded, there was a "Data is missing" error and the encryption key was not activated when the instance was started at the end of the upgrade. The instance will now prompt for the encryption key file at the end of the upgrade.

      CRE-3678: Installation changes for IntegratedML

      Category: Installation
      Platforms: All
      Version: 2018.1.0

      With this change, there are installation kits that include the IntegratedML libraries and other kits that do not. For the kits that do include these libraries, note that IntegratedML is not installed by default, but is rather part of the Custom installation.

      If you are using IntegratedML and you upgrade, be sure to select the appropriate kit and also to use the Custom installation option.

      DP-288841: Update installation for Entity Framework

      Category: Installation
      Platforms: Windows
      Version: 2018.1.0

      This change modifies how Entity Framework is installed. Now the installer uses Microsoft.VisualStudio.Setup.Configuration to query the available configuration and write necessary information into the registry.

      DP-412668: New ISCAgent standalone installer

      Category: Installation
      Platforms: Windows
      Version: 2018.1.0

      This change provides a new ISCAgent installer for Windows. Functionality will be the same as current ISCAgent installer except for the following changes:

      • It will accept normal Windows Installer command line options for logging, unattended and silent install, similar to what InterSystems IRIS installer is using. For example, to run silent install one would use command "ISCAgent_<version>.exe /qn".
      • It will install VC++ redistributables in both full UI and unattended install.
      • Dialogs will have the same look as InterSystems IRIS installer dialogs.

      DP-421225: Remove call to ^mdbmsins

      Category: Installation
      Platforms: All
      Version: 2018.1.0

      This change fixes an error that arose when upgrading from Cache 2018.1.6 to 2018.1.8.

      DP-426599: Macro for %Installer.Role.Resources changed to evaluate variables

      Category: Installation
      Platforms: All
      Version: 2018.1.9

      This change allows the use of variable replacement in the Resources property for a role created using %Installer.Role.

      TSL807: Health Connect UNIX silent install - default to Normal security

      Category: Installation
      Platforms: UNIX®
      Version: 2018.1.3

      Previously, an unattended installation of Health Connect would default to minimal security on Linux if normal/locked down was not specified. This would result in a broken instance.

      Now a silent install provides the same security level options as a manual installation. If no password is specified, it will fail immediately with:

      Password must be defined in Normal or Locked Down security installation.

      Interoperability

      DP-11171: Ensure Interoperability Portal pages make file select popups active for the current tab

      Category: Interoperability
      Platforms: All
      Version: 2018.1.6

      Certain Interoperability Portal pages use a popup to make an operating system file selection. Some of these popups might not have been set to be always the active window for the tab which could lead to a browser reporting an error and the user not seeing an intended message. This is now corrected.

      DP-13401: Remove synchronous call when unloading Interoperability Message Resend pages

      Category: Interoperability
      Platforms: All
      Version: 2018.1.6

      The Interoperability Message Resend page used synchronous calls during unload. This is no longer permitted in certain browsers. The Resend page code has been changed to remove dependency on synchronous call.

      DP-20991: Interoperability SFTP adapter to always reset underlying SSH session if error connecting

      Category: Interoperability
      Platforms: All
      Version: 2018.1.6

      The Intereoperability (S)FTP adapter when encountering a connect error using the SFTP protocol now resets the underlying SSH Session if that had been successful.

      DP-288880: Change in queue wait time calculation

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Interoperability
      Platforms: All
      Version: 2018.1.5

      To improve the efficiency of the Ens.MonitorService, this release changes the way the queue wait time is calculated. For business hosts with a pool size of 1, the amount of time the active message has taken to be processed is now counted s part of the queue waiting time. In rare cases, this could change how InterSystems IRIS behaves in responding to the QueueWaitTime alert setting.

      DP-288903: Correct Record Map reading of input data when record terminator is set to None

      Category: Interoperability
      Platforms: All
      Version: 2018.1.0

      The Record Maps with no record terminator were leading to an error processing the input file using the various record map file services. This is now corrected.

      DP-402919: Correct XML VDoc DOM setting for preceding sibling having one child

      Category: Interoperability
      Platforms: All
      Version: 2018.1.6

      Creating XML VDoc may not create desired output due to use of cached data to aid in finding the Index when the preceding sibling only has one child and the current sibling has more than one. This is now corrected

      DP-402935: Corrects framed TCP adapter not functioning with single end framing character

      Category: Interoperability
      Platforms: All
      Version: 2018.1.6

      The inability to process 'framed' messages terminated using a single terminating character by the EnsLib.TCP.Framed.PassthroughService and EnsLib.TCP.Framed.PassthroughOperation, which use EnsLib.TCP.FramedCommon via EnsLib.TCP.FramedInboundAdapter and  EnsLib.TCP.FramedOutboundAdapter has been corrected.

      DP-404404: Improve Use of Process Private Globals Created for Tracking Open HL7 Segments

      Category: Interoperability
      Platforms: All
      Version: 2018.1.9

      InterSystems IRIS keeps track of HL7 sections in a private process global to avoid deleting virtual document segments prematurely. This change fixes the management of these process private globals.

      DP-408979: FIX - Purge task logs errors to production for DICOM I/O Log entries

      Category: Interoperability
      Platforms: All
      Version: 2018.1.0

      In a DICOM production, when ArchiveIO is enabled, multiple I/O log entries can point to the same EnsLib.DICOM.Document object. The Event Log would fill with error messages when purging with AllTypes and MessageBodies=1 (defined when creating the Purge Task) because EnsLib.DICOM.Document:%OnDelete() would call $$$EnsError and write to the Event Log for any duplicate deletes.

      To prevent this, %OnDelete() still composes and returns the error status, but it now uses $$$ERROR instead and lets the caller decide whether to record or ignore it. (The purge task ignores it.)

      DP-410236: PATCH verb supported by EnsLib.HTTP.OutboundAdapter

      Category: Interoperability
      Platforms: All
      Version: 2018.1.0

      The EnsLib.HTTP.OutboundAdapter now supports the PATCH verb and passes it to %Net.HttpRequest.

      The new adapter methods are Patch(), PatchFormDataArray(), and PatchURL().

      DP-410391: Modify setting 'raw' XML to a property path to aid determination of order

      Category: Interoperability
      Platforms: All
      Version: 2018.1.0

      There was a previous correction that corrected the output order for a single property path in a sequence where a prior array sibling had more than one element. This correction lead to a possible change in output order when 'raw' XML (for example from a source XML VDoc complex property) was set using a Property path syntax. The latter change is now corrected.

      A new transient property of EnsLib.EDI.XML.Document is introduced to allow the prior behavior. The property is ByPassPropCacheForMixed.

      When setting 'raw' XML to a Property path (as opposed to a DOM path), this flag allows setting the data directly to the internal DOM storage. While this might improve performance, the output may have nodes in a different order than desired when combining with single property path sets.

      This setting is provided for backwards compatibility. The default is to set to the internal intermediary Property storage.

      DP-415142: Delete Unsent HL7 Messages Created in Business Services

      Category: Interoperability
      Platforms: All
      Version: 2018.1.9

      Previously, sending HL7 messages in a batch with the Individual option would send each child message (starting with MSH), but not the parent message, which would never be deleted, even when a purge was performed. This issue has been resolved.

      DP-415411: Project FullSize property to SQL for all EDI document types

      Category: Interoperability
      Platforms: All
      Version: 2018.1.0

      This change projects the FullSize property to SQL for X12, EDIFACT, and ASTM.  The property had already existed for all three (plus HL7), and the projection was already included for HL7.

      DP-416568: Modifications to how productions start/stop/update as _Ensemble

      Category: Interoperability
      Platforms: All
      Version: 2018.1.0

      This change modifies how the implementation of the internal wrapper methods that allow Interoperability Production Host Items to run as InterSystems IRIS user _Ensemble.

      DP-416858: Add optional use of Access Token for SASL XOAUTH support in EnsLib email adapters

      Category: Interoperability
      Platforms: All
      Version: 2018.1.0

      The email adapters now provide a new setting for a configured OAuth2 Application to use for SASL XOAUTH2. The adapters will use the associated Access Token if the application is authorized.

      The new settings include OAuth2ApplicationName, which is an optional OAuth2 Client Configuration Application name to use. If specified, this indicates that OAuth 2 is to be used and the name is used in the Authorization and Access Token retrieval process.

      DP-417168: Record Mapper and EDI Document Viewer file view permissions check

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Interoperability
      Platforms: All
      Version: 2018.1.0

      The Interoperability Record Mapper and EDI Document viewer management portal pages enforce holding the privilege %Ens_ViewFileSystem:USE before allowing the user to select a file using the file select dialog popup. This change adds the same check before a file is opened.

      DP-423129: [Interoperability] Correct error for Email Adapter using SASL XOAUTH when no credentials set

      Category: Interoperability
      Platforms: All
      Version: 2018.1.9

      Previously, use of a SASL XOAUTH configuration with the interoperability email outbound adapter would generate an error if a Credentials setting was not required. This issues is now corrected.

      DP-8412: Interoperability SFTP adapter sending of file streams to account for translation table

      Category: Interoperability
      Platforms: All
      Version: 2018.1.6

      The Interoperability SFTP adapter could send file streams without accounting for the stream's translation table if the source stream was of type %IO.FileStream. This is now corrected.

      JDBC

      DP-407658: Pass timeout as argument to socket.connect()

      Category: JDBC
      Platforms: All
      Version: 2018.1.6

      socket.connect(SocketAddress) does not use the timeout set inside the socket object, but in turn calls socket.connect(SocketAddress, 0) with an infinite timeout. This change passes the timeout as an argument to connect(SA, timeout).

      DP-422091: SQL SERVER: Correct timeout for stored procedure execution

      Category: JDBC
      Platforms: All
      Version: 2018.1.9

      Previously, stored procedures that were called via JDBC would only time out if the execution of the procedure took too long, not the fetching of the procedure's data (if it returned a result set). This issue has been corrected.

      Journaling

      DP-411849: Enable large reads for scanning forward for open transactions in journal restore

      Category: Journaling
      Platforms: All
      Version: 2018.1.0

      This change improves performance of scanning forward for open transactions at startup, failover, and journal restore.

      DP-412800: Fix the cache line alignment for journal buffer descriptors (jrniodb)

      Category: Journaling
      Platforms: All
      Version: 2018.1.0

      This corrects a bug with journal buffer descriptor structures not being properly aligned to begin on a cache line boundary.  In some configuration and version combinations this could lead to high amounts of contention and gate scalability on large NUMA systems.  

      Whether a particular version and configuration combination will be susceptible prior to this fix is not easy to predict.  irisstat -j1 shows the offset of journal buffer descriptors in the left hand column labelled "jrniodb".  When properly aligned these offsets end in 0x00 or 0x80 (128 byte alignment).  Other alignments, which are no longer possible with this bug fix, may show a degradation in journal throughput scaling (esp. on large NUMA systems): 0x70/0xf0 is known to be particularly problematic.

      DP-417229: Ensure that ^STU has access to system data

      Category: Journaling
      Platforms: All
      Version: 2018.1.0

      This change ensures that ^STU has access to system data. Before this change, it was possible for %ZSTART code (which calls ^STU) to fail occasionally because the _Ensemble user and the Security.System 'SYSTEM' object could not be found.

      HYY2100: Improve performance of journaling

      Category: Journaling
      Platforms: All
      Version: 2018.1.2

      This change provides substantial improvements to journaling performance.

      HYY2282: Address an issue with dejournal prefetchers that could cause system to hang

      Category: Journaling
      Platforms: All
      Version: 2018.1.1

      With certain System events (e.g. "logout") enabled for Auditing, an exiting dejournal prefetcher may get stuck in HALT with swcheck set, which could cause the system to freeze because some process (such as database truncation) may set switch 13 before waiting for all the jobs with swcheck set to finish.

      The change addressed the issue that caused exiting dejournal prefetchers to hang in HALT.

      HYY2283: Address an issue that could cause rollback or dejournaling to fail

      Category: Journaling
      Platforms: All
      Version: 2018.1.1

      This change addresses an issue that could cause rollback or dejournaling to fail related to journal switches.

      HYY2285: delegate jrnstop task to journal daemon

      Category: Journaling
      Platforms: All
      Version: 2018.1.3

      (This change and ALE3167 together address the stated prodlog.)

      With this change, the user job that runs ^JRNSTOP to stop journaling no longer needs the write access to journal files.

      HYY2296: suppress messages when new prefetchers failed to start due to limit

      Category: Journaling
      Platforms: All
      Version: 2018.1.3

      Addressed an issue that could result in spurious informative messages like "Mirror Prefetch exited due to halt command executed" from dejournaling tasks such as mirroring, journal restore or shadowing.

      HYY2307: fix error in journal restore following backup restore

      Category: Journaling
      Platforms: All
      Version: 2018.1.3

      Addressed an issue that could cause journal restore following backup restore to fail with an error like this:

      [***ERROR: InitShare+17^JRNRESTF *dest() Subscript 1 is ""]

      if journal updates are restored to the original location (i.e., without database redirection).

      The problem is present in 2017.2.0 and later.

      HYY2326: Release lock on journal purging after Purge^JRNUTIL failed to access %SYS

      Category: Journaling
      Platforms: All
      Version: 2018.1.2

      This change corrects an issue that could prevent the system administrator from performing journal purge after a user without sufficient privilege for journal purging attempted to perform one and failed.

      HYY2374: Ensure journal restore API abort on journal or database error as requested

      Category: Journaling
      Platforms: All
      Version: 2018.1.4

      Addressed an issue that caused journal restore via class API (Journal.Restore) not to abort on database errors as specified by the user (by setting property AbortOnDatabaseError=1), nor to abort on journal error by default.

      The problem is present in 2011.1 and later.

      HYY2386: Address "bad global reference" issue with dejournaling in the event of SMH Surrender

      Category: Journaling
      Platforms: All
      Version: 2018.1.4

      This change addressed an issue triggered by SMH Surrender that could cause dejournaling to generate "Bad global reference" error or even in rare cases cause data corruption. It affected dejournaling with multiple updaters.

      The problem is present in 2017.2.x and later releases.

      HYY2390: Ensure updates to journal directory block are written

      Category: Journaling
      Platforms: All
      Version: 2018.1.4

      Addressed an issue on mirror primary that could cause sets/kills to a database created after a mirror database is deleted.

      The problem is believed to be present in 2011.1.1 and later.

      HYY2396: address a case of journal log corruption

      Category: Journaling
      Platforms: All
      Version: 2018.1.5

      Addressed an issue that could result in journal log corruption in certain circumstances (for example, when a backup task runs at the same as journal switch). Mirror journal logs are also vulnerable, although to a lesser extent and not affected by backup tasks.

      JournalingGroup2019: Multiple changes and corrections related to journaling and mirroring

      Category: Journaling
      Platforms: All
      Version: 2018.1.3

      Various issues associated with journaling and mirroring have been addressed. The changes associated with these issues are SML2776, SML2781, SML2782, SML2783, SML2785, JO2990, JO3117, JO3137, JO3140, JO3141, RJF391, RJF392, HYY2362, HYY2364, and HYY2373.

      Kernel

      DP-12473: Improve mirror async journal write strategy

      Category: Kernel
      Platforms: Windows
      Version: 2018.1.6

      Improve mirror journal transfer throughput. This change improves responsiveness to synchronous commit transactions with an active backup mirror member. This improvement can be an order of magnitude in some configurations, particularly on Windows.

      DP-13007: Fix unexpected mirror journal log file corruption.

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      This change fixed an unexpected mirror journal log file corruption when mirror member is becoming primary. The cause of the problem is the header of mirror journal log file has a wrong title for "Edition=", it has "EDITION=" and caused the corruption detected.

      DP-13047: Support FIPS mode for libcrypto.so.1.1 in Red Hat Linux 8.2

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Kernel
      Platforms: Linux
      Version: 2018.1.5

      On Red Hat Linux, you can only use libcrypto.so.1.1 FIPS mode on Red Hat Linux version 8.2 and later. You cannot use this mode on earlier versions.

      DP-13698: Changes to TCP TLS in FIPSMode

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Kernel
      Platforms: All
      Version: 2018.1.5

      When FIPS mode is enabled, TCP TLS will use operating system's libraries and the TLS1.1 and earlier protocols won't be supported. This change should not impact existing code but, if your existing TCP TLS code uses FIPSMode, you should test it to ensure that there are no subtle behavior changes.

      DP-13748: Fix access violation after <STORE>

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      When a $STACK() function encountered a <STORE> error, the system attempted to return a null string and continue. But in some cases the environment after the <STORE> error was invalid and caused the process to terminate abnormally. The function will now throw a <STORE> error instead of trying to continue.

      DP-289134: Upgrade zlib to 1.2.11

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      Built-in zlib library has been upgraded to Version 1.2.11.

      DP-290856: Increase maximum name length for %File.CopyFile

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      The %File.CopyFile() method failed if the destination was a directory and the source file name length was greater than 100. This change increases the limit to the operating system maximum.

      DP-402381: Prevent ##CLASS(SYS.Journal.History).Purge() from changing group ID of journal.log

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      Under certain conditions including upgrades and purging journal files, the group ID or permission of journal.log can change, and that results in a future failure. This change corrects the problem.

      DP-402481: Collates after ]] can exceed the string stack

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      Under some circumstances you couldd get a STRINGSTACK error in an expression with a couple of Collates operators, with long strings disabled and fairly long operands whose value required that they be encoded onto the string stack. This change corrects the problem.

      DP-403474: Fixes testWebSocketStress.py failures for shared WebSockets

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      This change removes a race condition between the WebSocket handshake and the data request. Prior to this change, the server would occasionally fail to process the request, and the socket would time out waiting for data.

      DP-405568: Write out directory block in a timely manner

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      This change addressed an issue that caused creating or mounting a database to take a long time when journal activity is heavy.

      DP-407036: Fix possible lock table corruption when some jobs in convert wait state and denied

      Category: Kernel
      Platforms: All
      Version: 2018.1.0

      This change corrects an issue in which application servers appeared to lose their connections to the dataserver instance. Attempts to access globals from remote database(s) on these application servers would hang. The issue has been corrected.

      DP-407260: Stop logging messages for normal operation.

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      This change eliminated logging some messages that indicated the system was operating correctly.

      DP-408315: Validate global correctly in getGlobalSysSfn^%SYS.SECURITY

      Category: Kernel
      Platforms: All
      Version: 2018.1.0

      With this change, getGlobalSysSfn^%SYS.SECURITY validates global references in a more robust way.

      DP-408829: Integrity check must not read global blocks into global buffers

      Category: Kernel
      Platforms: All
      Version: 2018.1.6

      Prior to this correction, under extremely rare circumstances, integrity check run on an active system could cause a global reference or update in another process running concurrently to get unexpected results. This problem has existed since at least Caché 2011.1, though it's never been reported in the field.

      Note that a side effect of this correction is that integrity check no longer loads pointer blocks into the database cache as it checks them, whereas it generally did in the past (but almost never loaded data blocks into the buffer pool).  Therefore integrity check may do a few percent more reads, and may have somewhat less of an impact on other jobs running on the system.

      DP-409717: Ignore buffer size change for client/server TCP connection

      Category: Kernel
      Platforms: All
      Version: 2018.1.0

      If client code in an xDBC client/server application issued a USE command which changed the device buffer size, the application process would fail with an access violation.

      DP-411991: Retry configuring large page for AIX when the errno is EBUSY.

      Category: Kernel
      Platforms: AIX
      Version: 2018.1.0

      On AIX system, retry configuring large page for AIX when the errno is EBUSY.

      DP-412701: Update OpenSSL source codes from version 1.0.2zb to 1.0.2zd.

      Category: Kernel
      Platforms: All
      Version: 2018.1.7

      Update OpenSSL source code from version 1.0.2zb to 1.0.2zd. In addition to being used in the kernel, the OpenSSL library is used in the Apache-based private web server for any https connections, in the ODBC client for TLS-based connectivity, and in the PKI service.

      DP-412811: Increase max KMIP Key ID to 188 (LTO5)

      Category: Kernel
      Platforms: All
      Version: 2018.1.0

      The InterSystems IRIS KMIP interface module was previously limited to 37 character Key IDs when retrieving a list of keys from the KMIP server.  This was due to the use of a length constant in the KMIP library that wasn't appropriate.

      The max key length has been updated to 188 to comply with LTO5 in the KMIP spec.

      DP-413143: Upgrade zlib to 1.2.12

      Category: Kernel
      Platforms: All
      Version: 2018.1.0

      This change upgrades zlib to version 1.2.12.

      DP-414173: Require %All for open 63 of IRISTEMP

      Category: Kernel
      Platforms: All
      Version: 2018.1.0

      Generally holding a database resource gives you the ability to use open 63 to attach a view buffer to that database. Utilities such as %GSIZE use this ability.

      For the IRISTEMP database, which holds the process-private globals for all users, this change adds a refinement to protect other users' process-private globals from being viewed. Now you need to hold %All in order to use open 63 for the IRISTEMP database.

      It is unlikely but perhaps someone has some odd application where they do this. In order to run our utilities which use open 63, one would have to hold %All. This means if you wanted to use %GSIZE to see the size of the (non-process private) globals in IRISTEMP, you need to hold %ALL.

      DP-414224: Handle error() call in Windows mailbox thread

      Category: Kernel
      Platforms: Windows
      Version: 2018.1.0

      Under very unusual conditions, a $VIEW(-1,pid) of another process could cause that process to terminate with an exception.

      DP-414650: Fix error that could cause ECP daemon not finding lock entry for delock.

      Category: Kernel
      Platforms: All
      Version: 2018.1.0

      This change fixes an error in ECP that would sometimes result in a message like the following: Could not find lock entry for delock request, tid=70000, token=6f3261, sfn=3, lock=^ER1DCS(921208056)

      DP-415121: Update OpenSSL source codes from version 1.0.2zd to 1.0.2zf.

      Category: Kernel
      Platforms: All
      Version: 2018.1.0

      This change updates OpenSSL source codes from version 1.0.2zd to 1.0.2zf.

      DP-421863: Fix deactivation of data element encryption key corrupting db/jrn encryption key schedule on AIX

      Category: Kernel
      Platforms: AIX
      Version: 2018.1.9

      Previously, on AIX machines with POWER8 or later processors that use database or journal encryption simultaneously with certain data element encryptions, deactivating a data element encryption key could corrupt the key schedule used for database and journal encryption, which could corrupt database and journal files. This issue has been resolved.

      DP-422311: Improve performance of subscript indirection with extended global

      Category: Kernel
      Platforms: All
      Version: 2018.1.9

      This change improves the performance of delimiter checking in subscripts.

      DP-6736: Maximum frame depth reduced on 32-bit Unicode

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Kernel
      Platforms: UNIX®
      Version: 2018.1.3

      This change fixes a problem caused if a second <FRAMESTACK> error is encountered when TRY/CATCH catches a <FRAMESTACK> and tries to create the error object. This condition lead to the process terminating. This was an issue only on non-Windows 32-bit Unicode platforms.

      This change is unlikely to cause problems unless your code created conditions that were very close to the previously allowed frame depth.

      This was an issue only on non-Windows 32-bit Unicode platforms.

      Language Bindings Java and .NET XEP

      WAL385: Handle attempt to delete nonpersistent XEP class without error

      Category: Language Bindings Java and .NET XEP
      Platforms: All
      Version: 2018.1.1

      The Java EventPersister.deleteExtent and the .NET EventPersister.DeleteExtent methods provide a way to delete the extent of a class generated via XEP. In previous relesases, an error was thrown if the specified class was not persistent. With this change, the methods only attempt to delete an extent if the class is persistent. Otherwise, they ignore the class and return success.

      Language Bindings.ActiveX

      WAL508: Fix ActiveX locking issue

      Category: Language Bindings.ActiveX
      Platforms: All
      Version: 2018.1.2

      In some cases, ActiveX locks were not freed on the server. Attempting to call the obj.sys_UnlockId(Me.RecordIDctrl.Text) method when using CacheActiveX.dll resulted in a runtime error. This is now fixed.

      If your system has these locks that have not been freed, you should execute the following kill command in all effected namespaces after installing this change:

      kill ^ISC.oddMETA in all effected namespaces

      Language Bindings.C++

      DVU3562: Prevent second disconnect from cpp_binding

      Category: Language Bindings.C++
      Platforms: All
      Version: 2018.1.4

      This change fixes a problem where a pointer was set to 0 after call to CconnectClose completed, as result is was possible to attempt another operation on the same pointer from another thread.

      Language Bindings.Gateway

      MC2884: Fix problem that could cause proxy objects to be deleted

      Category: Language Bindings.Gateway
      Platforms: All
      Version: 2018.1.5

      This change fixes a problem where some proxy objects could lose their final reference count (hence be destroyed) during a Y9 loop.

      MC2950: Fix multiple problems in XSLT handling error conditions

      Category: Language Bindings.Gateway
      Platforms: All
      Version: 2018.1.5

      This change combines a series of fixes for XSLT. These fixes are:

      • This fix corrects a problem in handling an error condition. Without this change the error would not be reported and incorrect results could be returned.
      • This fix corrects a problem where there are multiple warning conditions. Without this change, incorrect result could be returned.
      • This fix corrects a problem where an XSLT business operation could encounter a "Compiled style sheet not found" error when the style sheet was accessible. This change improves the thread safety of the code.
      • This fix improves the way XSLT handles fatal errors.

      Language Bindings.Java

      MC2479: %Net.Remote.Proxy to convert OREF to string when inserting to Gateway.Proxies

      Category: Language Bindings.Java
      Platforms: All
      Version: 2018.1.3

      Without converting OREF to string, Gateway.Proxies holds a reference count on the OREF, which causes %OnClose to never be executed. This change converts the OREF to a string when inserting into Gateway.Proxies.

      SDK031: Java Binding: correct oref handling for collections of Objects

      Category: Language Bindings.Java
      Platforms: All
      Version: 2018.1.5

      This change corrects a problem that could result in the following errors while using the Java Binding with collections of objects (ArrayOfObjects, ListOfObjects):

      com.intersys.objects.ObjectClosedException: Attempt to access previously closed object with previous oref 3 object: com.intersys.cache.jbind.JBindCacheObject@ff855004[-123]

      com.intersys.objects.CacheServerException: <INVALID OREF>popFrame+144^%SYS.BINDSRV

      Language Bindings.Net

      WAL456: Fix Single Character Persistence in .NET XEP

      Category: Language Bindings.Net
      Platforms: All
      Version: 2018.1.1

      Prior to this change, if a .NET class had a char of Character valued property a roundtrip would not return the matching data. The property would be projected as %Library.String but instead of a string an integer would be stored. This is now fixed.

      WAL461: Update Bindings Wizard to Include Option for Skipping/Including System APIs

      Category: Language Bindings.Net
      Platforms: All
      Version: 2018.1.3

      In previous releases, the Bindings Wizard would always default to not generating system APIs (that is, most methods that come from %Library.Persistent). This change adds a new checkbox to the wizard to control this setting -- if you want to generate system APIs such as the Extent query, uncheck the box labeled "Skip generation of system APIs".

      Language Bindings.Net.ADO

      DPV5499: Correct DBSRV DirectDialect for non-resultset statement

      Category: Language Bindings.Net.ADO
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected with the ADO.NET Managed Provider when using the SQLDialect connection feature and executing a non-resultset statement (like an INSERT). The symptom was the client process would hang waiting on information from the server that the server never sent.

      JCN2002: Update default SSL support to include TLS1.1 and TLS1.2

      Category: Language Bindings.Net.ADO
      Platforms: All
      Version: 2018.1.4

      The SslProtocols.Default to create our SslStream was by default was not supporting TLS 1.1 and 1.2. This change just updates the default setting that can be supported by our ADO.Net provider connection.

      TLS1.1 and 1.2 will only be supported in DotNet 4.5 version and up.

      This change also applies to DotNetGateway 4.5 which now supports TLS 1.1 and 1.2.

      Core protocols supported are SslProtocols.Tls, SslProtocols.Tls11, and SslProtocols.Tls12.

      JCN2003: Add SSLProtocol specification to ConnectionString

      Category: Language Bindings.Net.ADO
      Platforms: All
      Version: 2018.1.4

      Different versions of DotNet support a variety of SSLProtocols and there needs to be a way to specify the exact protocol. DotNet 2.0 and 4.0 support SSL3 and TLS1 by default, while DotNet 4.5 supports SSL3, TLS1, TLS11, TLS12 by default, and CORE supports TLS1, TLS11, TLS12 by default. A new keyword for the ConnectionString can take the name of the enums in System.Security.Authentication.SslProtocols, which are as follows from the DotNet source:

              //
              // Summary:
              //     No SSL protocol is specified.
              None = 0,
              //
              // Summary:
              //     Specifies the SSL 2.0 protocol. SSL 2.0 has been superseded by the TLS protocol
              //     and is provided for backward compatibility only.
              Ssl2 = 12,
              //
              // Summary:
              //     Specifies the SSL 3.0 protocol. SSL 3.0 has been superseded by the TLS protocol
              //     and is provided for backward compatibility only.
              Ssl3 = 48,
              //
              // Summary:
              //     Specifies the TLS 1.0 security protocol. The TLS protocol is defined in IETF
              //     RFC 2246.
              Tls = 192,
              //
              // Summary:
              //     Specifies that either Secure Sockets Layer (SSL) 3.0 or Transport Layer Security
              //     (TLS) 1.0 are acceptable for secure communications
              Default = 240,
              //
              // Summary:
              //     Specifies the TLS 1.1 security protocol. The TLS protocol is defined in IETF
              //     RFC 4346.
              Tls11 = 768,
              //
              // Summary:
              //     Specifies the TLS 1.2 security protocol. The TLS protocol is defined in IETF
              //     RFC 5246.
              Tls12 = 3072
      

      The value specified with the keyword can be either the name such as "...SSLPROTOCOL=Tls12;..." or the integer value such as "...SSLPROTOCOL=3072;..."

      If you specify the keyword SSLPROTOCOL the language bindings turn on SSL by default and you do not have to separately specify the keyword "SSL". Using "SSL" alone will turn on the default protocol behavior listed above.

      SDK067: Close connection in response to ThreadAbortException

      Category: Language Bindings.Net.ADO
      Platforms: All
      Version: 2018.1.4

      If an ADO request was interrupted by a ThreadAbortException while waiting for data from the server, the socket could be left in an inconsistent state. Now we handle this case by closing the connection.

      WAL509: Fix NeedsReset Bookkeeping for ADO Connection Pooling

      Category: Language Bindings.Net.ADO
      Platforms: All
      Version: 2018.1.3

      If customer C# code using ADO.NET and connection pooling neglected to close connections when finished with them, incorrect data could be inserted for streams (among other problems) because that connection would be recycled without being reset. This is now fixed. Users should close connections, but even if a pooled connection is not closed, it will always be reset before being recycled.

      Language Bindings.Net.ObjectBinding

      WAL465: Improve Relationship Reference Bookkeeping in .NET Binding Server

      Category: Language Bindings.Net.ObjectBinding
      Platforms: All
      Version: 2018.1.1

      Prior to this corrections, relationships and the bindings caching mechanism could run into problems for:

      A -&g; C <- B

      where objects A and B are both pointing to C. The code:

      1. Opened object A

      2. Accesses A.C

      3. Opened Object B

      4. Accessed B.C

      5. Closed and reopened A, getting the prior swizzled version of A (another process made changes to A, these were not seen).

      The relationships broke the bindings caching mechanism, but this is now accounted for.

      WAL504: Remove Out of Date Proxy Classes from .NET Binding Samples

      Category: Language Bindings.Net.ObjectBinding
      Platforms: All
      Version: 2018.1.2

      The .NET Binding samples included proxy classes. This is a problem for a few reasons:

      1. The proxies can go out of date, meaning the samples appear unusable (although proxies just need to be regenerated).

      2. Customers need to generate proxies for their own projects anyway.

      This change removes the proxy classes from the samples.

      WAL524: Improve Cleanup in Related Object Reference Bookkeeping for Bindings Server

      Category: Language Bindings.Net.ObjectBinding
      Platforms: All
      Version: 2018.1.3

      There was a problem in OREF reuse that could be exposed by particular related class structures. This issue first appeared in Caché 2018.1.1. This change fixes the problem.

      Language Bindings.Net.XEP

      WAL354: Fix remote method/function calls in .NET XEP

      Category: Language Bindings.Net.XEP
      Platforms: All
      Version: 2018.1.4

      Prior to this change .NET XEP CallX functions (such as persister.CallClassMethod and persister.CallFunction) were throwing errors. This is now fixed.

      WAL410: Fix ObjectScript projection for generated ID property

      Category: Language Bindings.Net.XEP
      Platforms: All
      Version: 2018.1.1

      In C# classes used for XEP, developers can specify an Id annotation that causes an Id property to be assigned a generated value.

      [Id(generated = true)]
      public System.Int64? id_property;
      
      Before this change bulk inserts for classes with this type of ID could fail. This is now fixed.

      In some cases, schemas with the generated id annotation may encounter a problem. If you encounter this problem, remove the annotation and the behavior will be unchanged from the previous version.

      Language Bindings.Nodejs

      CMT1693: Reinstate the ability to exchange raw 8-bit string data

      Category: Language Bindings.Nodejs
      Platforms: Windows
      Version: 2018.1.3

      This change reinstates the ability to exchange raw 8-bit string data with Cach&eacurte; and InterSystems IRIS. As the Node.js/V8 API has developed it has become the case that most string functionality has been geared towards either UTF8 or 2-Byte Unicode.

      It remains the case that the default character encoding scheme for iris.node is UTF8, but in addition to UTF16, the module will now recognize the following character sets:

      • ANSI
      • ASCII (the 7-bit set)
      • ISO-8859-1
      • Windows-1252

      For example, using one of the above character sets, it is possible to send strings containing character values in the range 128 to 255 without them becoming encoded as UTF8.

         var connection = db.open{path:path, username: username, password: password,namespace: namespace, charset: "ISO-8859-1"};
         var data = String.fromCharCode(128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149)
         var result = db.set("MyGlobal", 1, data);
      

      Finally, there is a performance benefit in using one of the above character sets.

      CMT1694: Ensure that the iris.node module can accept data passed to Caché and InterSystems IRIS from JavaScript buffers.

      Category: Language Bindings.Nodejs
      Platforms: All
      Version: 2018.1.3

      This cchange ensures that the iris.node module can accept data passed to Caché and InterSystems IRIS from JavaScript buffers. This enhancement will facilitate passing binary data to the database. For example:

         var data = new Buffer(10);
         for (n = 0; n < 10; n ++) {
            data[n] = n + 1;
         }
         var result = db.set("MyGlobal", 1, data);
       

      CMT1743: Correct problems and inconsistencies encountered on returning Unicode text from Class Methods

      Category: Language Bindings.Nodejs
      Platforms: All
      Version: 2018.1.3

      This change corrects a number of problems and inconsistencies encountered on returning Unicode text from Class Methods.

      For complete interchangeability between API and TCP based connectivity to Caché and InterSystems IRIS, all COS functions and methods should return non-7-bit ASCII text as UTF8 encoded regardless of whether the encoding scheme specified in cache.node and iris.node is UTF8 or UTF16.

      For example:

      ClassMethod MyClassMethod() As %String
      {
         set result = [Unicode text]
         quit $zcvt(result, "O", "UTF8")
      }
      

      CMT1744: Correct a regression in the structure of the JSON object returned by the invoke_method() call

      Category: Language Bindings.Nodejs
      Platforms: All
      Version: 2018.1.3

      This change corrects a regression in the structure of the JSON object returned by the invoke_method() call.

      CMT1753: Correct a regression in the optimized version of the Global API methods

      Category: Language Bindings.Nodejs
      Platforms: All
      Version: 2018.1.3

      This change corrects a regression in the optimized version of the Global API methods (such db.set() and db.get()).

      Recent regression testing revealed that these methods would occasionally throw unexpected exceptions in some Node.js scripts.

      CMT1754: Correct a regression in the optimized version of the Global db.get() method

      Category: Language Bindings.Nodejs
      Platforms: All
      Version: 2018.1.3

      This change corrects a regression in the optimized version of the Global db.get() method.

      Recent regression testing revealed that the the (optimized) get() method did not correctly encode the output as UTF8 when this coding scheme was in force.

      CMT1760: Correct a small memory leak in the language bindings for Caché Objects

      Category: Language Bindings.Nodejs
      Platforms: All
      Version: 2018.1.3

      A previous change introduced a template and container based approach for binding to the Object Classes with a view to hiding the mechanics of maintaining instances on the InterSystems IRIS side (the 'oref' values etc ...):

          var cclass = db.cache_class("MyClass");
          var result = cclass.MyMethod({arguments ...});
      
      Repeated calls to the above construct could result in an ever-increasing amount of heap memory being consumed by the Node.js process. A memory leak was suspected and this change addresses a leak in the node add-on.

      Licensing

      AAR027: Fix access violation during license upgrade

      Category: Licensing
      Platforms: All
      Version: 2018.1.5

      A rare defect present since 2015.1 has been resolved where upgrading to a license key authorizing fewer users than the currently active key could crash and block new jobs from starting, requiring an instance restart to fix.

      RFD2048: Fix error trap in License Monitor

      Category: Licensing
      Platforms: All
      Version: 2018.1.3

      This change fixes error handling of log file in License Manager code, preventing unnecessary restarts of License Server.

      RFD2071: Limit excessive retries in JOB command for license failures

      Category: Licensing
      Platforms: All
      Version: 2018.1.4

      This change just fixes the retry timer on Linux so that we don't get the excessive number in the "License limit exceeded %n times since %date" message. Note that if a JOB fails to start for lack of available license units, the JOB waits a second and starts another child process. This happens until the timeout expires (or until it succeeds if there is no timeout).

      Lock

      SML2762: Handle more than 10 ECP client waiters in ECP data server

      Category: Lock
      Platforms: All
      Version: 2018.1.3

      This change fixes a possible lock table corruption in data server when there are more than 10 app servers connect to the data server.

      Management Portal

      CDS3006: Provide better error message if new Client Application does not exist

      Category: Management Portal
      Platforms: All
      Version: 2018.1.3

      When defining a new Client Application in the Portal Security Management, if the application executable does not exist there would be an <UNDEFINED> error. This change provides a more useful error message.

      DP-402588: Correct handling of Port field in Object Gateways SMP configuration page

      Category: Management Portal
      Platforms: All
      Version: 2018.1.0

      This change corrects a bug in the way that the Management Portal treated the Port field in Object Gateways configuration page.

      DP-404817: Correctly enable Management Portal breadcrumbs even when browser rejects cookies

      Category: Management Portal
      Platforms: All
      Version: 2018.1.0

      This change fixes a bug in which clicking the breadcrumb links in the Management Portal would require the user to log in again if their browser was set up to reject cookies. The scope of this fix is limited and does not include Analytics and Interoperability pages.

      DP-405655: Enable breadcrumbs for Analytics and Interoperability pages when cookies are disabled

      Category: Management Portal
      Platforms: All
      Version: 2018.1.0

      A previous change enabled the breadcrumb links in the Management Portal for all pages except those associated with Analytics & Interop applications, even in the case when cookies are disabled. This change updates the remaining pages.

      DP-413000: Correct XSS vulnerabilities in Import and Export CSP pages

      Category: Management Portal
      Platforms: All
      Version: 2018.1.0

      This change corrects XSS vulnerabilities in the UtilExpGlobalExport.csp and UtilExpImportLocal.csp CSP pages.

      SAM519: Replace legacy call to non-existent EscapeHTML method with $ZConvert in CSP.UI.SQL.Error

      Category: Management Portal
      Platforms: All
      Version: 2018.1.4

      This change replaces a legacy call to EscapeHTML method (which is no longer method of this object)in %CSP.UI.SQL.Error with a direct call to $ZConvert()

      SAM522: Correct status code trapping in CSP.UI.SYSTEMExpResultsPage

      Category: Management Portal
      Platforms: All
      Version: 2018.1.3

      This change corrects a latent defect where an uninitialized variable was silently tripping a Try-Catch and prematurely aborting the initialization of new namespaces.

      SAM528: Suppress loss-of-connectivity pop-ups in Management Portal

      Category: Management Portal
      Platforms: All
      Version: 2018.1.2

      This change revises the behavior of Management Portal to allow optional suppression of JavaScript alert() pop-ups when connectivity to the server is lost.

      This change introduces the global ^%SYS("Portal","DisableConnectivityPopup"). If this is left undefined or set to zero, the previous behavior is unchanged and loss-of-connectivity pop-ups are not suppressed. If set to 1, then errors resulting from a hyperevent that return a status code of 0 (not a real code, internal trap) or 400 will be routed to the less intrusive no-popups reporting logic. This change addresses the problem where Management Portal pages with background heartbeats displaying errors up during system restarts. Other conditions causing 400 and 500 series errors will continue to display the pop-ups since these conditions should stop client side execution and may require the user to respond.

      SDK074: Allow Period (.) in Web Application name

      Category: Management Portal
      Platforms: All
      Version: 2018.1.4

      This change modifies the Web Application definition to allow the period (.) character for the name field.

      YSD3545: SMP - Do not reuse Audit filter unless the previous search is successful

      Category: Management Portal
      Platforms: All
      Version: 2018.1.4

      This change is made to prevent a filter being reused when it has previously been timed out. If the filter has been timed out then the Filter should be reset to the defaults.

      YSD3559: SMP - Fix Mirroring for Async member and Monitor

      Category: Management Portal
      Platforms: All
      Version: 2018.1.1

      The following changes were made:

      The "Update network addresses" message, when available, is now a link. Clicking it will take you to the dialog where you can modify the network addresses.

      The "Edit Async" page is modified to include the validate failure logic.

      YSD3577: Ensure doEdit() in Edit Async Mirror can handle certain failure conditions

      Category: Management Portal
      Platforms: All
      Version: 2018.1.1

      This change prevents an undefined error from the Editing Async mirror page. The mirror set can be edited if the local member in this mirror set has not failed or if it has failed it was because of TIMEOUT.

      YSD3589: SMP - Fixed problem copying role in system management portal loses SQL privileges permissions

      Category: Management Portal
      Platforms: All
      Version: 2018.1.4

      In the management portal, the System Administration -> Security -> Roles -> Create New Role

      menu takes you to the Edit Roles page. If you select a role from the "Copy from" drop down list, the SQL Privileges were not copied. After this change, the SQL Privileges are copied.

      YSD3619: Mirroring fixes for when cloned system is detected

      Category: Management Portal
      Platforms: All
      Version: 2018.1.1

      This change ensures that the links for "Remove local mirror configuration" and "Join this mirror as a new member" behave properly whether user has or has not the resources required to perform the action. When they do not, an error message is displayed. When they do, the appropriate dialog is displayed.

      When the "Remove" or "Join" is performed and the save is successful, the dialog is closed and the calling page, Edit Async page or the Monitor page, will be refreshed with new data. When the "Remove" or "Join" is performed and the save returned an error, the error message is displayed in an alert box. Then when OK is pressed, the dialog will be closed and the calling page, Edit Async page or the Monitor page, will be refreshed.

      YSD3630: Mirror - Fix an error checking the SSL verification status

      Category: Management Portal
      Platforms: All
      Version: 2018.1.2

      This change fixes an error checking SSL verification status. The result of this error was that a mirror could be allowed to join the set even if the verification failed. This error was first encountered in Caché 2018.1.1.

      YSD3646: Correct escape for URL on %CSP.UI.System.ExpResultPage

      Category: Management Portal
      Platforms: All
      Version: 2018.1.2

      This change corrects the URL used in var refreshpage.

      YSD3678: Fixed ProdLog #160406: [med] When creating a new Namespace, iKnow/Text Analytics is not enabled even though "Analytics" box is checked

      Category: Management Portal
      Platforms: All
      Version: 2018.1.5

      This project fixes the following problems:

      1. In the past, when a new namespace is created and we create a new Web Application for it, sometimes when DeepSee is enabled, iKnow is not. Since we have combined the two into one "Analytics", if one is enabled then the other should too.

      2. From the Web Application edit UI. When DeepSee is enabled and iKnow is not, the checkbox "Analytics" is checked. This is incorrect. It should be unchecked because the logic is that if one of them is disabled then both should be disabled. User must check it to enable both.

      3. From the Web Application edit UI. When the "Analytics" checkbox is changed, we were saving the flag to Security.Applications, but we did not call the server method to actually enable or disable the feature for the application. This was wrong. The following should be called:

        Do EnableDeepSee^%SYS.cspServer(pProxy.PID, +Properties("DeepSeeEnabled"))
        Do EnableIKnow^%SYS.cspServer(pProxy.PID,+Properties("DeepSeeEnabled"))
        

        where Properties("DeepSeeEnabled") is the value of the "Analytics" checkbox (1=enable; 0=disable)

      In terms of the iKnow menu being disabled from the Management Portal, the following flag was used to determine the state: ##class(%iKnow.UI.Dialog.selectNamespace).%IsiKnowEnabled($namespace) the above changes will ensure that the value for this call is accurately returned.

      Migration

      DPV5295: FM2Class: Correct mapping of indices when the field also has an "A"-type index (FM2Class 2.35)

      Category: Migration
      Platforms: All
      Version: 2018.1.1

      This corrects a problem with the FM2Class mapper utility where an index might not be defined in the mapped class if the field also has an "A"-type index that was not defined in the class.

      DPV5352: FM2Class: Correct references to fields that were omitted because of RECURSION="NONE"

      Category: Migration
      Platforms: All
      Version: 2018.1.1

      This correction is FM2Class version 2.36. A problem has been corrected when using the setting Recursion = 0 or Recursion = "NONE" where the creation of index maps might get an <UNDEFINED> error attempting to define an index map for a field that was not mapped because of the recursion=0 setting.

      DPV5412: FM2Class: Revert duplicate field name handing behavior to v2.36 behavior

      Category: Migration
      Platforms: All
      Version: 2018.1.2

      In FM2Class version 2.37, a minor change was made to the naming convention for fields which ended up with duplicate SqNames in the class definition. This cause a compatibility issue with some customers who replied on the naming convention of the previous behavior. This change reverts the naming convention to the way it way prior to version 2.37.

      This is FM2Class version 2.39.

      DPV5459: FM2Class: Support new FileMan 22.2 datatypes, including TIME, YEAR, and UNIVERSAL TIME

      Category: Migration
      Platforms: All
      Version: 2018.1.3

      This is FM2Class version 2.40

      The FM2Class utility now supports mapping the new datatypes supported by FileMan v22.2. For more information on these datatypes, see

      VA FileMan 22.2 User Manual published by the United States Department of Veteran Affairs. This section includes some information from the User Manual.

      The types supported are described as follows:

      BOOLEAN Fields

      A field defined as a BOOLEAN data type can have only two entry values: YES or NO. The internal values of the BOOLEAN DATA TYPE is set to 1 for YES and 0 for NO.

      BOOLEAN fields first appeared in Fm2Class v2.35.

      LABEL REFERENCE Fields

      A field defined as a LABEL REFERENCE data type is designed to store a tag and routine entry of the format, TAG^ROUTINE. It is stored as a free-text field.

      LABEL REFERENCE fields map internally as %Library.String.

      TIME Fields

      A field defined as a TIME data type can accept many of the date/time entries, but only stores the TIME portion.

      For example, if the external value is 15:09:43, the internal value is 150943.

      TIME fields map internally as %Library.FilemanTime (a new datatype class - see following).

      YEAR Fields

      A field defined as a YEAR data type can accept many of the date entries, but only stores the YEAR portion.

      For example, if the external value is 2016, the internal value is 3160000.

      YEAR fields map internally as %Library.FilemanYear (a new datatype class - see following).

      UNIVERSAL TIME Fields

      A field defined as a UNIVERSAL TIME data type can accept many of the date/time entries and stores the date/time in a format with the local time and includes an indicator showing the offset from Universal Time. The first 14 characters of the internal storage of the UNIVERSAL TIME data type are exactly like the current DATE/TIME data type that includes seconds. The three characters in position 15, 16, and 17 indicate the UTC time offset in five (5) minute increments. In the example following: (440-500)/12=-5, this is a negative five hour offset from UTC.

      For example, if the external value is JAN 6,2016@08:03:36 (UTC-5:00), then the internal value is 3160106.080336440.

      UNIVERSAL TIME fields map internally as %Library.FilemanTimestampUTC (a new datatype class - see below).

      FT POINTER Fields

      A field defined as a FT POINTER data type works similar to the POINTER data type, but internally stores the free text that was returned from the pointed-to value.

      For example, if the external value is PATCH,USER, the internal value is PATCH,USER.

      FT POINTER fields map internally as %Library.String

      FT DATE Fields

      A field defined as a FT DATE data type works similar to the DATE/TIME data type, but internally stores the free text that was input by the user to determine the date.

      For example, if the external value is T-1, then the internal value is T-1.

      FT DATE fields map internally as %Library.String.

      RATIO Fields

      A field defined as a RATIO data type is designed to accept two numbers with a colon “:” between the two numbers. It is formatted and stored like a mathematical ratio. For example, the external and internal value could be 1:7.

      RATIO fields map internally as %Library.String

      New datatype classes:

      %Library.FilemanTime:

      Custom Time datatype designed to handle internal FILEMAN format Time (HHMMSS).

      A field defined as a TIME data type can accept many of the date/time entries, but only stores the TIME portion.

      Example:
      External: 15:09:43 
      Internal: 150943 
      ClientDataType = TIME, OdbcType = TIME, SqlCategory = STRING
      Parameter COLLATION = "STRING";
      

      %Library.FilemanYear:

      Custom DATE data type designed to convert FILEMAN format YEAR fields.

      A field defined as a YEAR data type can accept many of the date entries, but only stores the YEAR portion.

      Example:

      External: 2016 
      Internal: 3160000 
      ClientDataType = VARCHAR, OdbcType = VARCHAR, SqlCategory = STRING
      Parameter COLLATION = "STRING";
      

      %Library.FilemanTimeStampUTC:

      Custom TimeStamp datatype designed to handle internal FileMan UNIVERSAL TIME datatype (CYYMMDD.HHMMSSZZZ).

      This data type projects proper VARCHAR/STRING metadata to DISPLAY and ODBC Client software. The conversion methods of this datatype assume a full FIleMan 22.2 run-time environment is installed, and that the DUZ(2) variable is defined and DUZ(2) references an INSTITUTION that includes defined COUNTRY and LOCATION TIMEZONE values.

      ClientDataType = VARCHAR, OdbcType = VARCHAR, SqlCategory = STRING
      Parameter COLLATION = "STRING";
      

      The main purpose of these datatypes is to provide internal (Logical) and external (Display/Odbc) values for the type. They are not set up with the SQL engine to be compatible with other date/time/timestamp types. For example: %FilemanTime will not compare properly with a %Time type in a query because the logical values of the two types are different. The logical value for these three types are strings, and they use STRING collation to force the compared value to be a string. If you want to do something like compare a %FilemanTime field with the current time, you must first convert the FM TIME field to a %Time format so the comparison is valid. An example of this is:

      SELECT ... FROM ... WHERE CAST(%external(FM_TIME_FIeld) AS TIME) > CURRENT_TIME

      Mirroring

      DP-403641: Prevent database from missing data when CatchupDB failed reading journal directory block

      Category: Mirroring
      Platforms: All
      Version: 2018.1.5

      This change prevents the database from missing data when CatchupDB fails reading journal directory block.

      DP-403917: Address a data integrity issue caused by dejournaling

      Category: Mirroring
      Platforms: All
      Version: 2018.1.5

      Addressed a data integrity issue in mirror dejournaling, caused by discrepancy between dejournal reader's private and shared data. In some circumstances changes in environment between dejournaling sessions could lead to a difference between the reader's private data and the shared data. This difference could cause a data integrity issue.

      DP-421683: Fix job command when jobbing from a mirror process running in background

      Category: Mirroring
      Platforms: All
      Version: 2018.1.9

      This change fixes an issue where a backup mirror member, that has deferred licensing, that was demoted by the primary member would return an error and remain in a stopped state.

      JO2818+YSD3541: Validate local network addresses at mirror startup to detect cloned systems

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      When a mirror member instance starts up, its network address is validated by attempting to contact the instance at the configured mirror private address. If a different instance is contacted at the configured address, the local instance was very likely copied from another host, for example through backup and restore. If no instance is contacted, it may be that the network configuration of the local instance's host has changed and its mirror network addresses need updating. The Management Portal and the ^MIRROR routine both provide instructions for resolving these errors. For more information, see Resolving Network Address Validation Errors in the High Availability Guide.

      JO3065: Add /IOTABLE="RAW" when opening a TCP device for mirroring

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Previously changing the default translation for TCP devices from RAW caused a problem with mirroring. This has been resolved.

      JO3107: Restore ability to force a member to become the primary when the other member's agent is up but directory is not accessible

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      Restore the ability to use Force Become Primary in the situation when the agent for the other mirror member is reachable but the installation directory is not accessible.

      JO3111: Mirror Server dmns (send and ack) need to cooperate when cleaning up shared memory (mirrorsvr_share_free)

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      A relatively rare problem where the mirror might enter a permanent trouble state when the connection to an async member has been lost has been resolved.

      JO3114: Fix rare case where a crash during a Catchup operation causes journal data to be skipped

      Category: Mirroring
      Platforms: All
      Version: 2018.1.2

      If a system crashes during a Catchup operation, there was a small window where it was possible that some journal data would be skipped. This change corrects this problem. This problem was introduced in Caché 2017.2. If you have a Caché 2017.2 instance that crashed during a Catchup operation, contact the InterSystems Worldwide Resource Center for help in determining if you have encountered this problem.

      JO3117: Remove code that lets a backup take over if it thinks the primary went down within the trouble timeout

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      This fixes a problem where the mirror backup may incorrectly decide the primary has gone down within the trouble timeout period and take over without all of the journal data which has been committed to databases on the primary. In this case when the primary restarts it will not be able to rejoin the mirror because its databases will be out of sync.

      JO3136: Non-dmn jobs ignore GFDBINACTIVEMIRROR while a datbase is dismounting (DISMINPROG is set)

      Category: Mirroring
      Platforms: All
      Version: 2018.1.4

      A problem has been fixed where if a mirrored database was dismounted while a mirror member was in the synchronizing state applying journal files at startup it was possible for the mirror process to generate a core file from an exception.

      JO3179: Don't send data to asyncs which hasn't been sent to a backup when in trouble state

      Category: Mirroring
      Platforms: All
      Version: 2018.1.5

      A problem has been resolved where a DR or Reporting async member could connect to a primary in a trouble state and retrieve data which is not present after the mirror fails over resulting in the async member being out of sync and needing to be rebuilt. The problem does not occur for asyncs which are connected to the primary at the time of the trouble.

      RJF391: Fix "bad global reference" in mirror dejournaling due to dejournal stack corruption

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      Prior to these fixes mirror dejournaling could, in rare cases, get a "bad global reference" error, or much rarer still, apply an update to an incorrectly (to the wrong global or with a bad value).

      SML2618: Allow InterSystems IRIS instance to join Caché/Ensemble mirror set

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      When an InterSystems IRIS instance joins a Caché/Ensemble mirror set while the Caché/Ensemble systems do not have InterSystems IRIS ISCAgent installed, the InterSystems IRIS with this change will be able to join the mirror set when the Caché/Ensemble mirror set is in UNICODE version. If the Caché/Ensemble is non-UNICODE then the InterSystems IRIS joins the mirror set but can't connect to primary, while the primary won't be able to add this new InterSystems IRIS member in its mirror set. Uses have to manually add the InterSystems IRIS member in the mirror set through ^MIRROR or SMP utility.

      SML2626: Validate network address in Management Portal when primary modifying them

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      The Management Portal was not validating the network address when the administrator changed it. This change corrects this problem.

      SML2641: Ensure the the mirror manager master daemon does not wait for the ECP server

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Under certain circumstances, the mirror master daemon could wait for a nonresponsive ECP server. This change corrects this problem.

      SML2642: Fix an unwanted message after failing to join mirror

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Under some circumstances, when the system failed to join the mirror set, a misleading message telling the administrator to authorize the mirror on the primary was displayed. This change corrects this problem.

      SML2653: Fix failure to become primary even after force primary

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Under certain circumstances a mirror would fail to become primary and the administrator could not force the system to become primary. This change corrects this problem.

      SML2654: Ensure that last journal file sent is not purged prematurely

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Under rare circumstances when the disk is full, a mirror could purge the last file sent when an async member had not yet received it. This change corrects this problem.

      SML2655: Fix CatchupDB in backup member when the mirror set has only one backup member

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Under certain circumstances, such as caused by copying cache.dat from one system to another, a mirror could not immediately catch up when becoming primary. This change corrects this problem.

      SML2656: Fix failure to restore with long database path and mirror database name in backup file

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Restoring a file with a long database path and mirror database name could fail. This change corrects this problem.

      SML2664: Fix rare case where member is promoted to master but other mirror members do not recognize this

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Under rare conditions, a member is successfully promoted to master but the other mirror members do not recognize this promotion. This change corrects this problem.

      SML2665: Ensure messages from Display Mirror Configuration are consistent with Manage 'Allow Parallel Dejournaling'

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      In Display Mirror Configuration, the Management Portal displayed the internal values of the setting and in managing 'Allow Parallel Dejournaling', the Management Portal displayed the kinds of mirror members selected. This change ensures that the Management Portal displays the kinds of mirror members selected in both situations.

      SML2666: Activate AllowParallelDejournaling changes in backup/async members when primary changed it

      Category: Mirroring
      Platforms: All
      Version: 2018.1.1

      Activating AllowParallelDejournaling did not automatically activate this on backup/async members. This change ensures that when AllowParallelDejournaling is activated on the primary, it is also activated on backup/async members.

      SML2700: Fix CatchupDB in backup member when the mirror set has only one backup member

      Category: Mirroring
      Platforms: All
      Version: 2018.1.2

      Under certain circumstances if a backup member needed to catch up, it could not become the primary mirror because the database would not be marked as active. This change corrects this problem.

      SML2720: Add SYS.Mirror.VerifyMirrorSSLCertificates() public API.

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      This enhancement adds a new public API, SYS.Mirror.VerifyMirrorSSLCertificates(), to verify the Mirror SSL certificates across instances.

      SML2736: Allow Caché/Ensemble to join/connect IRIS mirror set

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      For Caché/Ensemble instances to join/connect to IRIS mirror set, all the Caché/Ensemble instances need to have this change installed.

      SML2756: Support join mirror to a machine with same instance names in InterSystems IRIS and Caché/Ensemble registries

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      This change allows a Caché/Ensemble instance to join mirror to a machine which has two instances with the same name on the InterSystems IRIS and Caché/Ensemble registries when the instance it intends to join is an InterSystems IRIS instance.

      SML2765: Fix failure in backup member when primary setup and enabled SSL

      Category: Mirroring
      Platforms: All
      Version: 2018.1.3

      This change added a parameter for ValidateFailoverPartner^MIRRORMGR() when the MIRRORMGR daemon of backup member received shutdown message due to SSL update, and the daemon in backup member sync mirror configuration with primary and call ValidateFailoverPartner() without passing this new forceBecomePrimary parameter and caused the error.

      TRW1700: Check for minimum instance directory length in ISCAgent REMCTRL

      Category: Mirroring
      Platforms: All
      Version: 2018.1.5

      The ISCAgent did not correctly handle a request for mirroring remote control for an instance with a 0-length name. This change corrects this issue.

      Monitoring

      DP-13739: Handle multiple threads in Windows SNMP extension agent

      Category: Monitoring
      Platforms: Windows
      Version: 2018.1.6

      This change fixes a rare problem where heavy SNMP traffic could cause errors, which typically are seen as header errors.

      RFD1926: Count buffers/cache memory on Linux as "free" memory

      Category: Monitoring
      Platforms: All
      Version: 2018.1.5

      For Linux, the memory shown by the "free" command as used for 'buffers' and 'cache' is made available to applications as needed. These will now be added to the available memory when calculating System Monitor alerts for "Paging" (which use Sensors for PhysicalMemory and PageSpace).

      RFD2001: Allow SNMP user extensions

      Category: Monitoring
      Platforms: All
      Version: 2018.1.2

      This change corrects a problem that caused problems with SNMP user extensions.

      RFD2010: Fix time reported for Xecute commands by MONLBL

      Category: Monitoring
      Platforms: All
      Version: 2018.1.1

      MONLBL was including the time to execute the line after the Xecute command when reporting the elapsed time. This change corrects this problem.

      Networking

      DP-420509: Update OpenSSL source codes from version 1.0.2zf to 1.0.2zg.

      Category: Networking
      Platforms: All
      Version: 2018.1.0

      Update OpenSSL source codes from version 1.0.2zf to 1.0.2zg.

      DP-422021: Fix Windows irisconnect parsing of Unix-style LF line endings

      Category: Networking
      Platforms: UNIX®
      Version: 2018.1.9

      This change resolves a parsing issue when using the cconnect (on Cache) or irisconnect (on InterSystems IRIS) commands on Windows systems.

      Networking.ECP

      GK1380: Unexpected database error on reverse $Query of SLM mapped global across ECP.

      Category: Networking.ECP
      Platforms: All
      Version: 2018.1.1

      A rare issue has been resolved where reverse $query raised database error on a SLM mapped global across ECP.

      GK1405: Fix hung ECP server dmn cleanup rtn that may leave the clean up job in an invalid state

      Category: Networking.ECP
      Platforms: All
      Version: 2018.1.3

      In very rare conditions, when the ECP cleanup rtns detects that an ECP server dmn is hung it aborts the cleanup. The cleanup sets up some temporary storage for dmn cleaning, but aborting the cleanup leaves the process in an invalid state.

      Leaving it in that state may raise an unexpected system error later during halting.

      GK1410: Fix <UNDEFINED> error during startup of journal rollback when it tries to resume a previous rollback

      Category: Networking.ECP
      Platforms: All
      Version: 2018.1.3

      This change fixes <UNDEFINED> error during startup of journal rollback when it tries to resume a previous rollback.

      GK1432: Modified the ECP server dmn to clear the switch if it's going to hibernate

      Category: Networking.ECP
      Platforms: All
      Version: 2018.1.5

      In a rare condition, if the ECP server write/process dmn had to hibernate, it didn't clear the global access Switch, and if there was a major background activity such as DB compaction or data move, the server hung/dead-locked forever. This change corrects this issue.

      OAuth

      DP-416533: /csp/sys/oauth2/ web application should not require %All

      Category: OAuth
      Platforms: All
      Version: 2018.1.0

      This change modifies the /csp/sys/oauth2/ web application so that it does not run with the %All role. The %All role will no longer be added to the application roles. If the web application already exists this role, the application is updated to no longer use this role.

      Object

      DMC1167: SSH: Correct various issues and work around libssh2 bug

      Category: Object
      Platforms: All
      Version: 2018.1.4

      This change corrects a number of issues have been discovered in the SSH library:

      1. Memory leak that can occur when SFTP close or shutdown operations block (this may be platform dependent).
      2. Edge cases around session disconnection.
      3. Preventing double authentication attempts (which will hang until the timeout expires).
      4. A bug in libssh2 (see fix here https://github.com/libssh2/libssh2/pull/439) that:
        1. This could cause sporadic key exchange errors on the second session.

      DMC1168: Disable RANDOM_PADDING in libssh2 for Windows

      Category: Object
      Platforms: Windows
      Version: 2018.1.4

      The libssh2 library has some logic that allows for random padding to be added to each transport message sent to the remote SSH server, as per RFC4253 (see following), however it seems that a number of SSH server implementations do not support this. On these servers, this results in a bad key exchange (KEX) which further results in connection failure. The failure can manifest itself in different ways because libssh2 ends up with a bad remote encryption key and thus is unable to properly decode the KEX portion of the connection. Usually, the error would be LIBSSH2_ERROR_OUT_OF_BOUNDARY which indicates that the server's indicated max packet size has been garbled by the bad decrypt and is thus out of range.

      RFC4253 states:

      > Each packet is in the following format:
      > 
      >       uint32    packet_length
      >       byte      padding_length
      >       byte[n1]  payload; n1 = packet_length - padding_length - 1
      >       byte[n2]  random padding; n2 = padding_length
      >       byte[m]   mac (Message Authentication Code - MAC); m = mac_length
      > 
      >       [...]
      > 
      >       random padding
      >          Arbitrary-length padding, such that the total length of
      >          (packet_length || padding_length || payload || random padding)
      >          is a multiple of the cipher block size or 8, whichever is
      >          larger.  There MUST be at least four bytes of padding.  The
      >          padding SHOULD consist of random bytes.  The maximum amount of
      >          padding is 255 bytes.
      

      The libssh2 library contains some logic to implement this in transport.c, but there's no mechanism to enable it (via the configure script). However, on the Windows platforms there's a header file that #defines RANDOM_PADDING for an unrelated function that also turning on random padding for Windows only.

      Object Compiler

      DP-405035: Perform Immediate unlock of %oddSQLLock in FinalizeODBCForeignKeyInfo^%ocsSQLRTDict

      Category: Object Compiler
      Platforms: All
      Version: 2018.1.6

      During class compilation, the logic in FinalizeODBCForeignKeyInfo^%ocsSQLRTDict acquires a lock on the public variable %oddSQLLock in order to prevent multiple processes from entering a piece of code at the same time. For a normal class compilation this lock has not presented a problem in the past, but for the CREATE TABLE ... AS SELECT ... feature, the class compilation could not reside within a transaction that could take some time.  This causes other class compilations to fail with a lock timeout error. This change corrects the problem.

      DP-405405: Correct %Save when parent ID is reference to multi-property ID

      Category: Object Compiler
      Platforms: All
      Version: 2018.1.6

      A problem has been corrected when a class has a parent-child relationship and the parent class has an IDKey index on a property that is a reference to a persistent class and the referenced class has an IDKey index on two or more properties.  

      This change corrects issues with %SaveData, %ComposeOid, %DeleteData, %ExistsId, %PhysicalAddress, %LockId, and %OnDetermineClass.

      DP-415160: Correction for deferred compile mode and embeddedSQL

      Category: Object Compiler
      Platforms: All
      Version: 2018.1.0

      This change corrects a problem introduced in version 2018.1.6 in which where embedded SQL in a class method using deferred compile mode was reporting an error that the table did not exist when the class was compiled.  The SQL compilation is not supposed to occur until run-time, but a problem was introduced where the compilation was performing the table name lookup at compile-time and reporting the table was missing if it was not defined yet.

      Object Gateway

      MKM696: Fixed %Net.Remote.Gateway %Connect when SSL config supplied.

      Category: Object Gateway
      Platforms: All
      Version: 2018.1.4

      The %Connect method of %Net.Remote.Gateway should now work with any valid combination of host and SSL configuration. Previously, it only worked if the host was the empty string, which was converted to "127.0.0.1" by the method.

      WAL367: Remove Unused List of Proxies From Object Gateway Code

      Category: Object Gateway
      Platforms: All
      Version: 2018.1.5

      This change fixed a memory leak in the Java Gateway.z

      Object Library

      DMC1146: CacheSSH - Timeout on failed network connections during SFTP Put and Get operations

      Category: Object Library
      Platforms: All
      Version: 2018.1.4

      During SFTP Put and Get operations, an infinite timeout was used to block on SFTP I/O. If the remote server stopped responding, then this could block indefinitely until the network connection was torn down and the socket got an error.

      Both Put and Get should be using the timeout that can be set on the SSH session via the `SetTimeout()` method.

      NOTE: The default timeout is 30 seconds, so for very slow links or heavily loaded servers, this might need to be increased.

      DMC1147: Allow MQ interface to authenticate with username/password

      Category: Object Library
      Platforms: All
      Version: 2018.1.5

      The MQ interface did not support a mechanism to allow a username / password authentication when connecting to the MQ series system. There are two new properties on the MQ object:

      • Username
      • Password

      If specified, these are passed into the MQCONNECTION via ClientSecurityParameters, if they are not specified, then the ClientSecurityParameters option is omitted which is the current behavior.

      DMC1150: Fix deallocation problem in SSH client when session disconnected with channel or SFTP still open

      Category: Object Library
      Platforms: All
      Version: 2018.1.4

      if the `%Net.SSH.Session` object was disconnected by forcibly called Disconnect() while either an SSH channel was still open (for example, via an XDEV from a Execute() command), or an SFTP object was open, then when those objects were cleaned up there could be a fatal error. The exact circumstances varied, on some platforms, nothing would happen, on other platforms this caused a SIGSEGV error. This change corrects the issue. Note that if you call the Disconnect() method, you should check for an error status since it may return the "Session in use" error.

      DP-10882: Let %Date use the process/system sliding window settings for converting 2 year dates

      Category: Object Library
      Platforms: All
      Version: 2018.1.6

      Remove the hardcoded sliding window settings for %Date:DisplayToLogical conversion so that it will use the process or system wide settings which is in line with how the other methods in %Date convert years.

      DP-13669: Improve SFTP filename character set handling

      Category: Object Library
      Platforms: UNIX®
      Version: 2018.1.5

      The logic used for narrowing filenames passed from a Unicode instance of InterSystems IRIS to libssh2 used the RemoteCharset property to allow the user to customize how the translations occurred. However, on Windows this logic ultimately ends up using WideCharToMultiByte() (locally) which doesn't give us the proper control over the charset on the remote side. For a PSFTP(a common Windows SFTP server) using CP1252, a Windows client would not be able to properly convert the character sets no matter what value of RemoteCharset was used (even "" which means "no conversion").

      It also turns out that remote SFTP server versions >3 do use UTF8 as per the specification, so we need to not do any conversions other than from UTF8 in this case.

      This change corrects these issues.

      DP-285619: Change class names for storage classes

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Object Library
      Platforms: All
      Version: 2018.1.9

      Updated the following storage class names:

      %CacheStorage -> %Storage.Persistent
      %CacheSQLStorage -> %Storage.SQL
      %CacheSerialState -> %Storage.Serial
      %CacheShardStorage -> %Storage.Shard
      %Compiler.Storage.Cache -> %Compiler.Storage.Persistent
      %Compiler.Storage.CacheExtent -> %Compiler.Storage.Extent
      %Compiler.Storage.CacheSerial -> %Compiler.Storage.Serial
      %Compiler.Storage.CacheSQL -> %Compiler.Storage.SQL
      %Compiler.Storage.Extent -> %Compiler.Storage.CustomExtent
      %Compiler.Storage.Serial -> %Compiler.Storage.CustomSerial
      %Compiler.Storage.Generator.Cache -> %Compiler.Storage.Generator.Persistent
      

      Added conversion of the old to the new name in the class dictionary upgrade. This is performed automatically in %Dictionary class save, and logic for UDL import now also performs this dictionary upgrade.

      The original names have been removed. If customer classes subclass these old names they should update to the new names.

      DP-404893: Fix filename truncation problem in SFTP Rename() operation

      Category: Object Library
      Platforms: All
      Version: 2018.1.6

      The {{%Net.SSH.SFTP:Rename()}} function appeared to occasionally truncate the filename for the "new" name. This problem only occurred if the new filename was longer than the old.

      DP-405034: %Net.SMTP checks server identities by default

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Object Library
      Platforms: All
      Version: 2018.1.6

      This change fixes a bug where the SSLCheckServerIdentity property in %Net.SMTP defaulted to false, but was documented as defaulting to true. The property now defaults to true and is in alignment with documentation. This means that, when connecting to an SSL/TLS secured web server, %Net.SMTP will check that the certificate server name matches the DNS name used to connect to the server and fail if they don't match. This is the behavior specified in RFC 2818 section 3.1.

      As a result, when connecting to an SSL/TLS secured web server, the default behavior of %Net.SMTP will now be to fail if the certificate server name does not match the DNS name used to connect to the server.

      This change is unlikely to cause compatibility issues, but it is possible that when using %Net.SMTP to send messages you might have issues connecting to an SSL/TLS enabled server. If this happens, and you understand the security trade-offs, you can set SSLCheckServerIdentity to 0 to restore the previous behavior.

      DP-407220: Rework SSH handling of unicode character sets on Windows

      Category: Object Library
      Platforms: Windows
      Version: 2018.1.0

      The Unicode character set processing on Windows was being subverted by some built-in conversion code in ./shared/System/ that was trying to apply the current Windows code-page to the conversion when we had already done the character set conversion (to UTF8) inside of InterSystems IRIS, resulting in garbled characters.

      DP-408482: Fix "minExclusive" and "maxExclusive" that were not correctly created from xsd

      Category: Object Library
      Platforms: All
      Version: 2018.1.6

      If a schema has exclusive maxes and mins, then %XML.Util.SchemaReader converts these to inclusive MAXVAL and MINVAL fields. However, this conversion assumed whole-number precision, which is incorrect if fractionDigits is specified. With this change, the SchemaReader checks for fractionDigits and computes maxes and mins that match that degree of precision. If fractionDigits is unspecified, it uses whole-number precision.

      DP-408594: Correct cookie sharing across subdomains in %Net.HttpRequest

      Category: Object Library
      Platforms: All
      Version: 2018.1.6

      The cookie sharing code between subdomains in %Net.HttpRequest was not compliant with RFC6265. Specifically when doing to sub.mydomain.com if a cookie is set by the server with a 'domain=mydomain.com' this cookie was not being sent to the server when a subsequent request was made to my domain.com.

      Also refreshed the list of root public domains from publicsuffix.org as the data was out of date.

      DP-409587: When calling OutputToDevice on a global stream with a specific length set 'AtEnd' property correctly

      Category: Object Library
      Platforms: All
      Version: 2018.1.6

      When using %Stream.GlobalBinary, %Stream.GlobalCharacter, %Stream.TmpCharacter, %Stream.TmpBinary and calling 'OutputToDevice' passing in a specific length to output if the length exactly matches the size of the stream we were not setting the AtEnd property to true so a loop such as:

      While 'stream.AtEnd {
        Set sc=stream.OutputToDevice(100)
      }
      

      Could end up looping forever if the stream was exactly 100 characters long.

      DP-414767: Fix host key crash in FIPS mode

      Category: Object Library
      Platforms: All
      Version: 2018.1.0

      When running in FIPS mode, making an SSH connection could crash because FIPS specifically disallows the traditional MD5 hashes and the libssh2 function returns a NULL pointer that the code was not checking for.

      DP-414946: Update %Net.SSH.Session to support SHA1 and SHA256 hashes for host keys

      Category: Object Library
      Platforms: All
      Version: 2018.1.0

      With this change, %Net.SSH.Session now supports SHA1 and SHA256 SSH host keys. The Connect() method accepts an optional hostkey argument, which is the hash of the public key of the remote host. Before this change, this was the MD5 hash; now the SHA1 and SHA256 are also accepted. The length of this hash is used internally to determine the type of hash.

      You can also pass a new, optional fourth argument to Connect() to indicate the hash type to use. Accepted values are "MD5" "SHA1" and "SHA256".

      If an invalid hash value is given, then a new error ($$$SSHInvalidHostKey) is returned. This error is separate from the $$$SSHHostKeyMismatch error which occurs if the hostkey argument doesn't match the remote system.

      For example, you can ask for a SHA256 hash via the fourth argument:

      USER>s s=##class(%Net.SSH.Session).%New()
      

      USER>s sc=s.Connect("gresham",22,,"SHA256")

      USER>w s.HostKey 73D7A8B93EC2FD286ACF5B86C1D91ECDA9EF8625D7CC7B20E9C889DBDF73D699

      But if you ask for a SHA1 hash, you'll get that instead.

      For another example, we are passing the SHA256 host key. The fourth argument for Connect() indicates the kind of hash that HostKey property should use, not the type of hash used in argument 3 (which is determined by its length).

      USER>s sc=s.Connect("gresham",22,"73D7A8B93EC2FD286ACF5B86C1D91ECDA9EF8625D7CC7B20E9C889DBDF73D699","SHA1")
      

      USER>w s.HostKey C41F99C85DB2425FF9C25977CA3AB7617BE2A92B

      JMK007: Fix SAML response validation

      Category: Object Library
      Platforms: All
      Version: 2018.1.5

      This change fixes a SAML 2.0 response document that fails validation.

      MAK4765: If %Net.HttpRequest gets a READ error when reading the server response report this as an error %Status

      Category: Object Library
      Platforms: All
      Version: 2018.1.3

      If %Net.HttpRequest gets a READ error when reading the server response report this as an error %Status where as before it would return $$$OK.

      MAK4842: Improve %File:CreateDirectoryChain to avoid permissions issue on Windows

      Category: Object Library
      Platforms: Windows
      Version: 2018.1.3

      On Windows, if a user does not have permission on a directory but does have permissions on a subdirectory and you called ##class(%File).CreateDirectoryChain to create a new directory in the subdirectory, it would fail. You do have the ccorrect permission on it, but it would fail when it attempted to see if the parent directory existed, but you do not have permissions on the parent directory. Now it scan backwards to avoid this problem.

      MAK5203: Apply timeout on HTTP proxy 'CONNECT' read

      Category: Object Library
      Platforms: All
      Version: 2018.1.5

      Using %Net.HttpRequest with a proxy connection that did not return any data we were not respecting the OpenTimeout value correctly and could wait indefinitely for a response.

      SOH676: Fix Typo in %ZEN.Auxiliary.altJSONProvider method

      Category: Object Library
      Platforms: All
      Version: 2018.1.3

      The implementation of the %AbstractListToAET method in the %ZEN.Auxiliary.altJSONProvider class contained a typo. The call on the %New() method was incorrectly typed as "%SNew". This change fixes this problem.

      ObjectScript

      CDS3080: Fix rare compiler crash caused by CONTINUE statement

      Category: ObjectScript
      Platforms: All
      Version: 2018.1.1

      Under rare conditions, depending on the underlying operating system memory layout and the level of nesting in the routine, the ObjectScript compiler could get an access violation when compiling a CONTINUE statement. This change corrects this problem.

      CDS3089: Fix $COMPILE() problem in large routine

      Category: ObjectScript
      Platforms: All
      Version: 2018.1.5

      Issuing $COMPILE() in a large routine (over 64K of object code) could cause various errors or access violations after return from the $COMPILE().

      CDS3111: Fix <STRINGSTACK> compiling string concatenation

      Category: ObjectScript
      Platforms: All
      Version: 2018.1.3

      Compiling a long string concatenation expression in direct mode or XECUTE, when some of the pieces contain Unicode characters, could result in a <STRINGSTACK> error. This change corrects this problem.

      CDS3113: Fix access violation with indirection in new job

      Category: ObjectScript
      Platforms: All
      Version: 2018.1.3

      This change fixes a possible access violation if the first use of the ObjectScript compiler in a new job is for name indirection that resolves to a subscripted name with a special variable as the subscript.

      DP-403666: Fix legacy NodeJS to properly handle exceptions thrown in user code

      Category: ObjectScript
      Platforms: All
      Version: 2018.1.6

      This change ensures that the legacy Node.js binding will properly handle exceptions thrown by user code. Previously, exceptions that were not of the type %Exception.SystemException would result in inconsistent behavior, including a '' response, a <MAXSTRING> error, or a crash. 

      Currently the server reports errors by sending the numerical code associated with the ObjectScript error to the client as a single byte. The client client has logic to reconstruct the error string, which is then reported to the application. 

      Now, if user code throws any exception other than a SystemException, the client will report a <THROW> error, with the associated message "The application encountered an unexpected exception.". This change also ensures that the server will not return an error code greater than 255. A SystemException should not use a value this high, but if one is encountered, the server will send the error code 255 to the client, which will be interpreted as a <SYNTAX> error. 

      Note: This change only affects connections using TCP. Connections using the callin API do not report unhandled exceptions correctly. Applications using the callin interface should not directly invoke code that could raise an exception.

      DP-409576: Workaround Gnu mktime() bug

      Category: ObjectScript
      Platforms: All
      Version: 2018.1.0

      This change corrects a bug in which $ZDATETIME($H,-2) is sometimes off by an hour on some Linux-based systems on the day the Daylight Saving Time change occurs.  

      This change modifies the InterSystems IRIS implementation of $ZDT(htime,-2) in a way that works around the underlying C run-time library.

      Platforms

      PLATFORMS-WIN11: Add support for Windows 11

      Category: Platforms
      Platforms: Windows
      Version: 2018.1.7

      This release adds support for Windows 11.

      PLATFORMS-XALANC12: Update xalan_c to version 1.12

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Platforms
      Platforms: All
      Version: 2018.1.0

      This change updates xalan_c to version 1.12.

      REST

      MXT2179: Set CurrentAuthenticationScheme for Basic authentication in HttpRequest

      Category: REST
      Platforms: All
      Version: 2018.1.1

      The CurrentAuthenticationScheme for Basic authentication in an HttpRequest was being set to "". This change ensures that it is set to "Basic".

      Security

      CTW001: Add auditing for %SYS.X509Credentials Class

      Category: Security
      Platforms: All
      Version: 2018.1.3

      This change enables security auditing of CUD operations to %SYS.X509Credentials objects.

      DMC1163: Don't hard-code SSPI token size; determine max size from provider

      Category: Security
      Platforms: All
      Version: 2018.1.4

      During the SSPI login process, we had allocated token buffers of 8000 bytes. In some cases, it appears that tokens can be larger than this and in these cases, login would fail. The correct approach is to call QuerySecurityPackage and look at the cbMaxToken field to determine the maximum token buffer size for that provider (Kerberos in this case).

      DP-20972: Add OAuth functionality to %Net.SMTP and %Net.POP3

      Category: Security
      Platforms: All
      Version: 2018.1.0

      %Net.SMTP and %Net.POP3 now support XOAUTH2 as an authentication method. %Net.POP3.Connect() now accepts an access token as a parameter and %Net.Authenticator (which is used by %Net.SMTP for SMTP AUTH) now has an access token property and allows XOAUTH2 to be included in its mechanism list. Users who wish to use XOAUTH2 for POP should pass an access token to %Net.POP3.Connect() and users who wish to use XOAUTH2 for SMTP should set the access token property of %Net.Authenticator and specify XOAUTH2 as the mechanism to be used.

      Usage notes: - If %Net.POP3.Connect() is passed an access token it will attempt to use XOAUTH2 regardless of whether or not a password is supplied. Therefore, users who do not wish to use XOAUTH2 should be careful that they do not set the new AccessToken parameter that comes after Password. - %Net.Authenticator will only use XOAUTH2 if its mechanism is set to be XOAUTH2 - Users who don't wish to use XOAUTH2 need not make any changes.

      DP-400092: When copying users, allow password to change

      Category: Security
      Platforms: All
      Version: 2018.1.6

      This change fixes a problem where if you copy a kerberos user in the Management Portal you get an error.

      DP-400093: When copy user fails in Management Portal, audit it correctly

      Category: Security
      Platforms: All
      Version: 2018.1.6

      This change corrects a problem where copying a user in the Management Portal would fail, but the audit record indicated that it succeeded.

      DP-403670: Allow setting Dispatch Class for Web Application using ^SECURITY

      Category: Security
      Platforms: All
      Version: 2018.1.6

      This change modifies the ^SECURITY routine to allow users to configure the Dispatch Class for a Web Application.

      When using a Dispatch Class many of the Web Application settings are no longer relevant. Users will not be prompted for these settings when a Dispatch class is specified. This matches the behavior in the Management Portal.

      DP-404024: Provide way to limit folders available from ZEN fileSelect dialog

      Category: Security
      Platforms: All
      Version: 2018.1.0

      This change adds the ability to limit the set of folders that can be viewed by the {{%ZEN.Dialog.fileSelect}} component. Folders can be added to the list by running {{##class(%CSP.Portal.Utils).AddFileSelectDir(dir)}} or {{##class(%CSP.Portal.Utils).RemoveFileSelectDir(dir)}}. The list of configured directories can be retrieved with {{##class(%CSP.Portal.Utils).GetAllowedFileSelectDirs(.dir)}}. Internally, the values are stored in {{^%SYS("Portal","RestrictDirs")}}. If no directories are configured, all directories are visible (this is the current behavior).

      When configured a user should not be able to navigate to a directory outside the set of configured directories and their descendants.

      DP-404140: Support configurable Diffie Hellman key sizes

      Category: Security
      Platforms: All
      Version: 2018.1.6

      This change adds a configuration item to the server-side SSL configurations specifying the minimum size of Diffie Hellman keys.

      DP-404283: Changes to enabling %ZEN.Dialog classes

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Security
      Platforms: All
      Version: 2018.1.6

      In this release, %ZEN.Dialog.* classes cannot be run in web applications unless the web application is enabled for analytics, the namespace is enabled for interoperability productions, or it is explicitly enabled for the web application. To explicitly enable %ZEN.Dialog.* classes, enter the following:

      Set ^SYS("Security","CSP","AllowPrefix",application-name,"%ZEN.Dialog.")=1

      Also, with this change, any web application that is enabled for analytics or interoperability will have %ZEN.Dialog.* classes enabled by default.

      DP-404857: Use correct defaults for OAuth2 client JWT algorithms

      Category: Security
      Platforms: All
      Version: 2018.1.6

      This change corrects the default values for the OAuth2 client IDToken and Userinfo algorithms.

      Previously the IDToken, Userinfo and Access Token algorithms would default to using the corresponding value in the Server's JWT settings page, if the client value was "". Note that these values cannot be set to "" in the Management Portal, but they can be configured manually, or via dynamic registration from a third-party OAuth2 client. A previous fix changed this logic to instead default to the server values if the client value was "none".

      Neither behavior is correct. The IDToken and Userinfo algorithm settings are defined in the Open ID Connect Dynamic Registration specification https://openid.net/specs/openid-connect-registration-1_0.html.

      See the definition for:

      userinfo_signed_response_alg
      userinfo_encrypted_response_alg
      userinfo_encrypted_response_enc
      id_token_encrypted_response_alg
      id_token_encrypted_response_enc
      

      The specification defines what the value of these fields should be if omitted by the client ("none" in most cases.) The specification does not allow for the server to substitute another value arbitrarily.

      This change modifies the logic to not apply the Server JWT settings to the IDToken or Userinfo algorithms. They will still apply to the Access Token algorithms. These settings are not part of the OpenID Connect standard.

      Note that the Request algorithm settings (from the client JWT settings page) are defined in the OpenID connect spec, but they were never overridden by the server settings, so the logic relating to them is unchanged.

      DP-405517: Preserve CSPCHD token in OAuth2 redirects

      Category: Security
      Platforms: All
      Version: 2018.1.6

      This change corrects a problem that would prevent OAuth2 applications using Authorization Code or Implicit authorization flow from interfacing correctly with CSP applications when cookies are disabled. When cookies are disabled the CSP session token is included in the CSPCHD URL parameter. Previously, this would be removed when redirecting back to the application after successful authorization, which resulted in the session being lost. This has been corrected. Now OAuth2.Response uses the %response.UseExactRedirect flag to prevent any translation of the specified redirect URL.

      In order for the redirect to work, the application should call ..Link() on the redirect URL when calling GetAuthorizationCodeEndpoint() or GetImplicitEndpoint():

       set url=##class(%SYS.OAuth2.Authorization).GetAuthorizationCodeEndpoint("MyApp", scope, ..Link(myRedirectURI),,,.sc) 
      

      This will ensure that the CSPCHD token is included in the URI. This is only necessary if this is a CSP/ZEN application and cookies are disabled.

      DP-409573: Do not use CSP sessions for OAuth2 Client pages

      Category: Security
      Platforms: All
      Version: 2018.1.6

      This change corrects the OAuth2.Response and OAuth2.JWTServer CSP pages to not use persistent CSP sessions. These pages are served via the /csp/sys/oauth2/ application on OAuth2 Client systems.

      DP-411575: Update Apache httpd to 2.4.52

      Category: Security
      Platforms: All
      Version: 2018.1.6

      This change updates Apache httpd used as private web server to version 2.4.52.

      DP-415141: Return status correctly in %SYS.Audit:Import()

      Category: Security
      Platforms: All
      Version: 2018.1.0

      This change fixes a bug in the Import() method of %SYS.Audit, where an error status code was not being returned for an error condition.

      DP-416839: Reset SSL cache on failed connection to LDAP

      Category: Security
      Platforms: All
      Version: 2018.1.9

      This change fixes a bug in which it was possible to get an "UNKNOWN CA" error when logging in, when using LDAP to authenticate.

      This change provides a new option the method %SYS.LDAP:SetOption. The option is described in the class reference.

      For the example of "LDAP host names" parameter, to specify multiple host names, separate the names by spaces. If the LDAP server is configured to use a particular port, you can specify it by appending ":portname" to the host name; typical usage is not to specify a port and to let the LDAP functions use the default port, such as:

      ldapserver.example.com
      ldapserver.example.com ldapbackup.example.com
      

      If you have multiple replicated domain servers on your network like

      ldapserver.example.com
      ldapserver1.example.com
      ldapserver2.example.com
      ldapserver3.example.com
      

      You can specify the domain "example.com" as your host name. LDAP will perform a DNS query asking for the addresses of all the matching ldap servers, and then automatically select one to connect to.

      DP-418534: Set Content-Type header correctly in OAuth2.JWTServer

      Category: Security
      Platforms: All
      Version: 2018.1.0

      This change modifies OAuth2.JWTServer to set the Content-Type header in the OnPreHTTP() Method instead of in the OnPage() method. 

      This change prevents a <PROTECT> error.

      DP-422560: HTML encode REST error messages

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Security
      Platforms: All
      Version: 2018.1.9

      Previously, error messages returned from REST APIs were not HTML encoded. This issue has been corrected.

      DP-423994: correct /csp/sys/oauth2 application update logic

      Category: Security
      Platforms: All
      Version: 2018.1.9

      This change removes unnecessary entries in the ApplicationChange audit log by correcting an issues in the CreateCSPApplication() method of the OAuth2.Server.Configuration class.

      DP-426920: SMP: properly escape username

      Category: Security
      Platforms: All
      Version: 2018.1.9

      This change properly escapes the $username value displayed in the Management Portal and the Analytics UserPortal.

      DPV5443: SQL SECURITY: [GLOBAL] PRIVATE TEMP TABLES no longer allow xDBC login access to a namespace

      Category: Security
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected where a user could connect via xDBC to a namespace even though they had no privileges on any SQL objects in the namespace except for a GLOBAL PRIVATE TEMPORARY TABLE or a PRIVATE TEMPORARY TABLE, which all users implicitly have access to because the owner of such temporary tables is _PUBLIC. Now the user must have some SQL privilege on an SQL object other than a [GLOBAL] PRIVATE TEMPORARY TABLE.

      JMK017: Removing Expired Certs from AllCA.cer

      Category: Security
      Platforms: All
      Version: 2018.1.5

      This change removes expired certificates from AllCA.cer. Before this change, these certificates were not removed but could not be used.

      MXT2128: Do not return invalid redirect_uri error to the redirect_uri

      Category: Security
      Platforms: All
      Version: 2018.1.3

      With this change OAuth2 does not return invalid redirect_uri error to the redirect_uri. Instead it returns the error to the authorization code requester.

      MXT2145: OAuth2: Allow unexpected methods in "token_endpoint_auth_methods_supported" property during discovery

      Category: Security
      Platforms: All
      Version: 2018.1.3

      OAuth2: This change allows unexpected methods in "token_endpoint_auth_methods_supported" property during discovery.

      MXT2211: Make sure to updateJWKS on each client even if client secret is not set

      Category: Security
      Platforms: All
      Version: 2018.1.3

      Make sure to updateJWKS on each client even if client secret is not set. After executing "Update JWKS" on the server definition screen, the public jwks for the authroization server must be updated for each client even if the client does not have a client secret.

      SML2880: Enforcing FIPS mode to match the OS setting in Red Hat 8.2

      Category: Security
      Platforms: All
      Version: 2018.1.5

      Red Hat 8.2 can be booted with FIPS enabled or disabled, and this affects the openssl libraries that need to be loaded. In order to be consistent with the operating system behavior, you must ensure that the FIPS setting matches whether the system was booted with FIPS enabled or disabled. If the setting does not match, then Caché will not start. You must edit FIPSMode in the Configuration Parameter File (CPF) to match the operating system setting and then restart Caché.

      STC2606: Update Group Model for LDAP authorizations

      Category: Security
      Platforms: All
      Version: 2018.1.3

      This change updates the group model for LDAP auhtorizations. For details, see "About LDAP Groups and Caché" in the Caché Security Administration Guide.

      STC2661: Add multiple LDAP domains

      Category: Security
      Platforms: All
      Version: 2018.1.3

      This change adds the capability to authenticate to more than one LDAP server. When the system is initially installed, a default LDAP configuration is created for you based on your default domain. You must then use the Management Portal to modify the default settings of the LDAP configuration which was created for you, such as a valid LDAP Search Username and password combination.

      If you wish to authenticate to multiple LDAP servers, you must first turn on the feature "Allow multiple security domains". When you do this, all usernames in the user database are modified to be in the format username@defaultdomain.com, and any username prompts must be entered as username@defaultdomain.com.

      Now to authenticate against a different LDAP server, you must create a new LDAP configuration in the Management Portal which points to a different server. For example, you may have a LDAP configuration called example.com which points to an LDAP server example.internal.com. To authenticate against this LDAP server, you would need to enter the username myuser@example.com. Now if you want to authenticate against a second LDAP server, you would create a new LDAP configuration called example.org, with a the second LDAP server called examaple.internal.org. To authenticate against this server, you would use a username of user@example.org. Note that when creating a new LDAP configuration, there is an option to copy an existing LDAP configuration which makes adding new configurations simple, especially if you have similar LDAP structures and the same search username/password on all your servers.

      Once you create a new LDAP configuration, you can easily test your settings by clicking the Test LDAP Authentication button and entering a username@ldapconfig/password.

      STC2669: Add LDAP configuration display to Management Portal

      Category: Security
      Platforms: All
      Version: 2018.1.3

      LDAP configurations can now be displayed from the Operator menu in the Management Portal.

      STC2671: Return email and mobile phone from LDAP to the User Record

      Category: Security
      Platforms: All
      Version: 2018.1.3

      When using the built in LDAP authentication, the LDAP email address, mobile phone, and mobile phone provider are now returned and stored in the user record.

      STC2675: Security.Domains database no longer used

      Category: Security
      Platforms: All
      Version: 2018.1.3

      When you have multiple security domains enabled on your system, it is no longer necessary to create a domain for your user before you can enter him into the user database. Previously you would have to go to the mgt port and create a domain "example.com" before you could enter a user in the user database from that domain, for example jimsmith@example.com.

      Now you can directly create a username jimsmith@example.com without creating the domain name first.

      Since the domains database is no longer used, it is removed from the security database. All the methods in Security.Domains have been deprecated, and when called perform no action.

      STC2686: Remove %System/%Security/LoginRuleChange event

      Category: Security
      Platforms: All
      Version: 2018.1.3

      The %System/%Security/LoginRuleChange audit event has been removed.

      STC2699: LDAP updates for operating system authentication

      Category: Security
      Platforms: All
      Version: 2018.1.3

      The following changes were made to LDAP authentication.

      1. The first time an operating system LDAP authentication user logs in, they will be prompted for a username/password. Once they successfully log in, subsequent logins will not be prompted for.
      2. For Active Directory LDAP servers, if the account is set to expire in the future, that date is now stored in the user record in the security database. If we are disconnected from the LDAP server, and using LDAP credentials cache authentication, the user will not be able to log in if we are past the expiration date.
      3. The security scan task will now check all the LDAP users in the security database against the LDAP server and if the account is expired/pwchange set/disabled/ or deleted on the LDAP server, the account is deleted from the Security Database.
      4. The Security.Users:Detail() query now included the Flags parameter. This makes it possible to examine a record and determine if it is an LDAP user.
      5. The LDAP authentication checks PWDCHANGE/AccountExpires/Disabled before performing a bind and retrieving the users groups which speeds up authentication failures for these items.
      6. If the user has the %All role and cannot authenticate properly, the test displays all the group information for the user. Previously the test would stop after the Authentication failure. If the account has an expiration date, it is now displayed in the test output.

      STC2713: Increase size of CreateUsername in Security.Users

      Category: Security
      Platforms: All
      Version: 2018.1.3

      This change increases the size of the CreateUsername property in the Security.Users class to 128 characters. Previously it was 50.

      STC2772: Add audit event %SYSTEM/%SQL/PrivilegeFailure

      Category: Security
      Platforms: All
      Version: 2018.1.3

      There is a new Audit Event added to the Audit table called %SYSTEM/%SQL/PrivilegeFailure.

      STC2775: Continue to log message once certificate expires

      Category: Security
      Platforms: All
      Version: 2018.1.3

      Previously the system monitor would log when a SSL certificate was going to expire up to 31 days before it was due to expire. Once the certificate expired, it would stop logging this. It now continues to log that it has expired.

      STC2832: Increase size of $username field to handle long domains

      Category: Security
      Platforms: All
      Version: 2018.1.3

      rnames can now be up to 160 characters long. If you have Multiple Domains enabled, the username consists of the domain appended to the username, for example user@example.com.

      STC2835: LDAP InstanceId default now uses "_" rather than ":"

      Category: Security
      Platforms: All
      Version: 2018.1.3

      LDAP InstanceId default now uses "_" rather than ":" because the ":" character is not a valid character for the sAMAccountname field on an active directory server.

      STC2901: 1-argument form of $SYSTEM.Security.Login() must be a Caché user or Kerberos user

      Category: Security
      Platforms: All
      Version: 2018.1.3

      In order to successfully login a user with $SYSTEM.Security.Login(Username), the user must be an Caché password user or kerberos user, and not LDAP, Delegated or other.

      STC2960: Don't write to audit file while dismounted

      Category: Security
      Platforms: All
      Version: 2018.1.3

      Attempting to erase or encrypt/decrypt the audit database, dismounts the audit database. This change suppresses the attempt to create an audit record for this action, since it cannot be written to the dismounted database. After operation completes, audit events for the process are restored to their previous values. trw158

      STC2991: Update LDAP test

      Category: Security
      Platforms: All
      Version: 2018.1.3

      This change updates this test. Note that this test only checks to see if LDAP is set up correctly for the system so that the instance can connect to the LDAP server and perform authentication checks for the entered user. It does not perform any authorizations or permission checks to determine if the user can actually successfully log into the system. The end user may need to add additional permissions to the user for them to successfully log in. If the "Test LDAP" succeeds for the entered user, but the user cannot actually log in, the audit record should be checked for the login failure.

      STC3050: Add message to install for KMIP server encryption

      Category: Security
      Platforms: All
      Version: 2018.1.5

      When upgrading a system which uses database encryption keys from a KMIP server, you must set the database encryption option to

      "Unattended key activation with a KMIP server"

      and select the keys on the KMIP server you want to activate using the ^EncryptionKey routine.

      Once you do this you can perform the upgrade.

      STC3093: Reduce journal data when user logs into using LDAP or delegated authentication

      Category: Security
      Platforms: All
      Version: 2018.1.5

      In previous releases journal data would be generated when the user data was modified by a login even if the roles were not changing. This change avoids this excess journal data and only generates journal data if the user roles change.

      WDS732: Increase maximum number of activated database encryption keys to 256

      Category: Security
      Platforms: All
      Version: 2018.1.1

      The maximum number of activated database encryption keys was 4. This change increase the maximum number of activated database encryption keys to 256. It slso increases the maximum number of keys in an encryption key file to 256.

      WDS738: Prevent Windows ERROR_SHARING_VIOLATION when renaming files in CVEncrypt

      Category: Security
      Platforms: Windows
      Version: 2018.1.1

      If CACHE.DAT is renamed, the system could display an ERROR_SHARING_VIOLATION message. This change corrects this problem.

      WDS741: Fix ability to save startup options with KMIP

      Category: Security
      Platforms: All
      Version: 2018.1.1

      Under certain conditions you could not save the startup options with ^SECURITY when enabling encryption of the audit database. This change corrects this problem.

      WDS745: Improve detection of AES hardware instructions

      Category: Security
      Platforms: All
      Version: 2018.1.1

      On some platforms the AES-NI instructions were not being detected. This change ensures that the AES instructions will be detected on those platforms if they are present.

      Shadowing

      HYY2291: Addressed a rare case of shadow database discrepancy

      Category: Shadowing
      Platforms: UNIX®,Windows
      Version: 2018.1.3

      Addressed an issue that could result in data discrepancy between shadow and source databases in the rare circumstance where the source server of shadowing has 256 (or a multiple of) databases mounted in a short period of time.

      The issue affected shadowing in all supported releases for Windows and Unix.

      HYY2340: Addressed an issue with the query that lists incomplete transactions on a shadow

      Category: Shadowing
      Platforms: All
      Version: 2018.1.3

      This change addressed an issue that cause the query SYS.Shadowing.Shadow:IncompeteTransactions not to list any incomplete transactions.

      HYY2405: Address an issue of shadowing interoperability

      Category: Shadowing
      Platforms: All
      Version: 2018.1.5

      Addressed an issue that caused shadowing to fail when the database server runs on InterSystems IRIS 2019.4 or higher while the shadow server runs on a lower version, including any Caché version.

      The shadow server should be updated with this change.

      With this change, journal incompatibility between the database server and the shadow server is reported in messages.log (or cconsole.log) on the shadow server like the following:

      The file, file path, which is in journal format version 12, is not readable on current system, which has journal version 11

      The error used to be reported like the following:

      SHADOW SERVER (djk): <ZEXIT>connect+45^SHDWCLI;ERROR #1021: Database server and shadow server have incompatible journal versions: version 12 on database server vs. version 11 on shadow server

      Note that on InterSystems IRIS systems 2019.4 or higher (with HYY2255), journal files are created with journal version 11 by default, unless their SFN tables consist of multiple journal blocks. Both versions of journal files can be read on those InterSystems IRIS systems, but only version 11 is readable on InterSystems IRIS or Caché systems without HYY2255.

      Sharding

      JMM994: Cause Perl and Python bindings to return correct timestamp values in all cases

      Category: Sharding
      Platforms: All
      Version: 2018.1.4

      This change fixes a problem in which Perl and Python bindings returned wrong timestamp values in a few specific cases.

      Source File Control API

      BES044: Add new "udl-multiline" option to "format" URL parameter on GET /doc/ endpoint

      Category: Source File Control API
      Platforms: All
      Version: 2018.1.6

      Add "udl-multiline" option for GET /doc/ "format" URL parameter. Passing ?format=udl-multiline will return the document with method arguments formatted on multiple lines. Also bumped Atelier API version to 4 to reflect that the schema changed.

      BES045: Add support for "location" on POST /action/index endpoint

      Category: Source File Control API
      Platforms: All
      Version: 2018.1.6

      Add support for location mapping between documents when doing a POST /action/index request. To map the location in a document to the corresponding location in "other" documents (like the generated INT for a class), append the label+offset to the end of the document name separated by a colon (i.e. [ "%Activate.Enum.cls" ] becomes [ "%Activate.Enum.cls:method+5" ]) and all "other" documents in the response body will contain the document name and corresponding offset separated by a colon:

      "others": [
          "%Activate.Enum.1.INT:+11"
      ]
      

      BES046: Properly check document timestamp in %Atelier.v1.Utils.General:TS()

      Category: Source File Control API
      Platforms: All
      Version: 2018.1.6

      Ensure that Atelier API uses the same logic as Studio when checking the timestamp of a document. This only affects users who use server-side source control.

      BES047: Properly return SQL results containing braces in POST /action/query endpoint

      Category: Source File Control API
      Platforms: All
      Version: 2018.1.6

      Fix an issue where SQL result strings that contain braces or curly braces cause the POST /action/query endpoint to report a parsing error.

      SQL

      AK1024: Fix rare case where %ID was not handled correctly in sorts and selections

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      In very rare cases GROUP BY %ID or WHERE conditions on %ID were not processed correctly when a bitmap index was used in certain single-table queries. This change corrects this problem.

      DP-11573: SQL Import/Export wizard now restricts file extensions

      NOTE: This item may require a change to code, configuration, or operation.

      Category: SQL
      Platforms: All
      Version: 2018.1.5

      With this change, the SQL Import/Export wizard ([System Explorer] -> [SQL] -> [Wizards] -> [Data Import/Export]) will only allow the user to import from or export to files on the server with an allowed extension. Allowed extensions are: .txt, .csv. Previously any file on the system could be used. When importing from or exporting to a file on the client system, filenames are not restricted.

      DP-12403: Fix null check for COUNT to account for collation

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      This change fixes a bug to ensure that null values of collated fields are counted correctly. Specifically, we now ensure (for example) that:

         COUNT(%SQLUPPER null) = 0
      
      whereas before this expression would return 1. This is because the previous not null check did not check against the collated representation of null. That is, we would check that value '= "" even though the appropriate check is value '= " " since %SQLUPPER(null) = " ".

      DP-12816: Transform COUNT(field) to COUNT(Collation(field))

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      Queries of the form:

         SELECT COUNT(field)
      
      would previously not use an index on the field unless that index also stored the field data. This is because the query optimizer did not recognize that it did not need to retrieve the field data in order to execute the query. After this change, queries of this form get transformed to:
         SELECT COUNT(Collation(field))
      
      which triggers optimizations that avoid the problem described above.

      DP-13438: Correct <UNDEFINED> determining fields needed for view privileges

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      A problem has been corrected where an <UNDEFINED> error might occur when compiling a SELECT query that selects an explicit field and * from a view.

      DP-13787: Don't loop on chunks in agfrag if ID needed

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      In very rare cases GROUP BY %ID or WHERE conditions on %ID were not processed correctly when a bitmap index was used in certain single-table queries. This change corrects the problem.

      DP-279480: Fix problem with not able to remove SQL Privileges when Preferred Language is not English

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      This change fixed a problem so that on the Security/Users/User edit page, SQL Privileges tab, the Remove link is properly built when it should be in all languages.

      DP-281816: Correct types reported for several scalar and aggregate functions

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      The following changes have been made for scalar functions:

      {fm MOD(expr1,expr2)}
      
      This function used to return type NUMERIC. Now it will return type DOUBLE if the type of expr1 is DOUBLE, otherwise it will return NUMERIC

      The aggregate functions STDDEV(expr), STDDEV_POP(expr), STDDEV_SAMP(expr), VARIANCE(expr), VAR_POP(expr), and VAR_SAMP(expr) These functions were supposed to return type NUMERIC, but that was not always the case. Now they will return type DOUBLE if the type of expr is DOUBLE, otherwise they will return NUMERIC.

      {fn TRUNCATE(numeric-expr,scale)}
      
      TRUNCATE returns the same data type as numeric-expr.

      DP-402646: <SUBSCRIPT>hasmeth+8^%qaqcpr ^oddEXTR("") error when compiling an SQL query

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      Checking for conversion methods on virtual fields led to <SUBSCRIPT> errors. This change checks whether the current field being handled is a virtual field and if so skips checking for the specified conversion method.

      DP-403645: Fix bad SQL results on parallel query that segments on a $double field that could miss a row

      Category: SQL
      Platforms: All
      Version: 2018.1.5

      When running a parallel SQL query on a table where the plan has picked a column that is in IEEE numeric format, and the data in this column is in InterSystems IRIS double format and not IEEE format which could happen if the data is manually imported outside of Objects/SQL access then rounding errors could result in the query missing a row of data. This change corrects this problem.

      DP-404801: Use noagf to identify unaggregated fields in `having` clause

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      Under certain conditions, SQL`having` conditions containing fields were incorrectly leading to queries that should result in only one row were resulting in more than one row. This change corrects the problem.

      DP-405944: Fix collation for %PARALLEL on a numeric column that could cause a rows to be skipped

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      When splitting a global into chunks to process in parallel we detect if the last node is a numeric value we added one to it to avoid $double rounding errors. This can cause problems with numeric values because although 1.1]]0.1 is true "1.1"]]"0.1" is false. The reason being that "0.5" is not a canonical number due to the leading '0' character where as "1.1" is a canonical number so for example:

      set a("0.1")="string zero point one"
      set a(0.1)="numeric zero point one"
      set a("1.1")="string one point one"
      
      zw a
      a(.1)="numeric zero point one"
      a(1.1)="string one point one"
      a("0.1")="string zero point one"
      

      The result of this is if the last range in a parallel query was a string value with a leading '0' which can be converted into a number between 0 and 1 e.g. "0.1" when we added 1 to this to ensure we do not get $double rounding problems we created a string value "1.1" which collates before the end of the global i.e. "0.1". This means rows from a query are not processed when the query is run in parallel. This change fixes the problem.

      DP-407453: Fix a caching problem for a DISTINCT query with a SELECT DISTINCT view in FROM

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      This change fixes a caching problem that causes some UNION ALL queries to run slowly. This problem impacted Runtime Plan Choice (RTPC) query optimization.

      DP-408527: Collect light weight SQL stats on read-write mirror members

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      This change collects light weight SQL stats on read-write mirror members.

      DP-409827: Close loophole where SQL privileges could be dropped

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change corrects an issue where a user at the terminal prompt could make a call to an API in %SYS.SQLSEC and if the correct arguments were passed in, could delete a user's SQL privileges.

      DP-409848: Fix Logic with Phrases in iFind

      Category: SQL
      Platforms: All
      Version: 2018.1.6

      This change fixes previously erroneous logic related to iFind positional search (and as a result, iFind co-occurrence search) which caused the results of all positional search terms OR'd together in a node, except for the last term, to be ignored. For example, in a table with the following records in its MyText field (which has any non-minimal iFind index on it):

      ||MyText||
      |The quick brown fox jumped over the lazy dog|
      |Colorless green ideas sleep furiously|
      

      The following query:

      SELECT MyText FROM MyTable WHERE %ID %FIND search_index(MyTextIdx, '"lazy dog" OR "green ideas"')
      

      would only return the second row, and would not include results that contained the "lazy dog" positional search term. Following this change, all positional terms in an OR node are accounted for. This change also makes a minor update to prevent unnecessary checks of positional search strings that only contain the first word in a positional search term.

      DP-411647: $System.SQL.Util.SetOption("SQLSECURITY") checks resource earlier

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      DP-412841: Correct query compilation error (IsAStream^%qaqcct)

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change fixes an SQL query compilation error. The query referred to a virtual collated field, and the error included this:

      <SUBSCRIPT>IsAStream+15^%qaqcct ^||%sql.smd(1,"")
      

      DP-413390: SQL PRIV: Correct SQL Admin CQ privilege checking

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      A problem has been corrected where it might be possible for a user to execute an existing cached query that requires the %NOCHECK, %NOLOCK, %NOINDEX, %NOTRIGGER, or %NOJOURN SQL Admin privilege without the user having the SQL Admin privilege.

      DP-413782: Return results in the correct selectmode for UNION %PARALLEL queries

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change fixes a problem in which UNION %PARALLEL queries would only return results in logical mode.

      DP-416924: Correct query with HAVING or ORDER BY with implicit join

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change corrects an error in which a query could return duplicate records if the query included a HAVING clause or included an ORDER BY clause that contained an implicit join.

      DP-417280: Correct %ValidateIndices utility for tables with row-level security

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This correction fixes the %ValidateIndices utility method for classes that use row-level security.  The general flaw in the utility is that an index was traversed and when an index value was found that belonged to a row the user did not have privileges to read, the utility thought the row did not exist.  If the utility was called with pFix=1, the utility would then delete the row.

      %ValidateIndices has been updated to allow the utility to validate all rows in the table regardless if the caller has the privilege to read all rows from the table or not.  If the class/table uses row-level security, the display of any issues found will obscure the error messages so that no data in the table is displayed to the caller of %ValidateIndices.

      DP-417432: Correct <SUBSCRIPT> error for parallel query on table with %SPACE collation

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change corrects a <SUBSCRIPT> error that occurred when running a parallel query on a table with %SPACE collation.

      DP-417498: Fix collation of scalar subqueries

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change fixes the collation of scalar subqueries during view recursion.

      Before this change, it was possible to have different query plans for equivalent queries that differ only in trivial ways.

      DP-418158: Disable auto-parallel if query contains constant and mode is %Runtime/%FDBMS

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change disables the auto-parallel mode for a query that contains a constant and that is run in either the %Runtime or %FDBMS mode. Such queries, when run in parallel, did not return expected results.

      DP-418248: Improve performance of queries with outer-join conditions involving multiple view tables

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change improves the performance of queries with outer-join conditions involving multiple view tables.

      DP-418975: Fix errors running tune table while purging cached queries in the background

      Category: SQL
      Platforms: All
      Version: 2018.1.0

      This change corrects an issue that occurred when running tune table while purging cached queries in the background. The issue caused errors like the following:

      ERROR #5521: SQLError: SQLCODE=-400 %msg=Fatal error occurred::
      There are 1 errors in routine %sqlcq.USER.1
      ...
      
      ERROR #5521: SQLError: SQLCODE=-400 %msg=Fatal error occurred::
      Error executing code for 
      Getting sample of rows from data blocks ...
      

      DP-421268: SQL: Correct bad temp routine name when NodeNameInPid = TRUE

      Category: SQL
      Platforms: All
      Version: 2018.1.9

      Previously, NodeNameInPid was 1,  the name of the temp routine created for RunTempCode was invalid because $JOB was used in the routine name. This issue has been resolved.

      DP-422716: Escalate privilege before calling $system.Config.Modifybbsiz from sql internal %qaq* rtn

      Category: SQL
      Platforms: All
      Version: 2018.1.9

      Previously, ODBC users required the %Admin_Manage resource, which is not normally assigned to such users, to call $SYSTEM.Config.Modifybbsiz(). This issue has been resovled.

      DP-425405: Add DISTINCT to the correlated non-rowid fields of streamed EXISTS view

      Category: SQL
      Platforms: All
      Version: 2018.1.9

      Previously, in queries that used an EXISTS subquery, extra rows would incorrectly be returned if a correlated field contained the same values in the subquery. This change corrects this behavior by adding the DISTINCT keyword in such queries to all correlated, non-RowID fields.

      DP-425964: Update the logic that computes the row threshold for switching to use PPGs for hybrid temp-files

      Category: SQL
      Platforms: All
      Version: 2018.1.9

      Previously, the system overestimated the row threshold for switching to use PPGs for hybrid temporary files, causing SQL processing using too much memory in some cases when hybrid temp-files are used. The logic used to calculate this threshold has been updated to avoid the processing.

      DPV5074: Do not run TuneTable on a table mapped to a readonly database

      Category: SQL
      Platforms: All
      Version: 2018.1.3

      When running TUneTable, if the class that projected the table originates in a database mounted readonly, the tuning of the table will not occur. You will see a message like:

      Table 'Ens_ServiceRegistry_External.Action' is mapped to a readonly datababase. No tuning will be performed.

      DPV5093: Correct FormatToLogical input conversion for IFNULL variable arguments

      Category: SQL
      Platforms: All
      Version: 2018.1.4

      A problem has been corrected with the SQL IFNULL function where an argument to IFNULL was not properly converted from Display or Odbc format to logical format if the argument was a host variable. For example, suppose the following IFNULL function:

      IFNULL(:in, CURRENT_DATE, :in)

      Here the assumption is the type of :in is a DATE, but the input value for :in was not being converted from an external DATE format to a logical DATE format.

      DPV5122: Default type of ID column is now %Library.BigInt

      Category: SQL
      Platforms: All
      Version: 2018.1.4

      A change has been made to the datatype for system generated ID properties/fields. The type of the ID property/field used to be %Integer, now the type of the ID is %BigInt. This allows for tables with large amounts of data to exceed ID values of 2,147,483,647.

      This also means a reference field, where the field references a table that has a system generated ID value, will also report as type BIGINT.

      The type of a system generated childsub field has also been changed to %Library.BigInt.

      This change was reverted in Caché 2018.1.5.

      DPV5292: IDENTITY columns without a specified datatype will use type BIGINT instead of INTEGER

      Category: SQL
      Platforms: All
      Version: 2018.1.4

      When an IDENTITY column is created via DDL, the datatype is optional. If omitted, the type of the field/property created is now BIGINT / %Library.BigInt(MINVAL=1). Prior to this change the field / property was defined as type INTEGER.

      This change was reverted in Caché 2018.1.5.

      DPV5304: Corrections for SQL Auditing - use updated SQL Audit Event IDs

      Category: SQL
      Platforms: All
      Version: 2018.1.4

      In previous releases, the following audit events had issues:

        %System/%SQL/DynamicStatement
        %System/%SQL/XDBCStatement
        %System/%SQL/EmbeddedStatement
      
      This change ensures that these audit events work correctly.

      DPV5496: Correct TuneTable setting of Selectivity and AverageFieldSize for serial properties

      Category: SQL
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected with the TuneTable utility where in some cases running TuneTable would not store the Selectivity and AverageFIeldSize in the class definition properly for serial sub-fields.

      DPV5502: Correct %SQL.Migration.Import.CopyData issue where ^CacheStream never gets cleaned up

      Category: SQL
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected where using %SQL.Migration.Import.CopyData() method to copy a table's data via the ODBC gateway leaves temporary stream data defined in ^CacheStream if the data being copied contains stream fields.

      DPV5516: Correct code generation of NextCode when global name contains the characters "NEXT"

      Category: SQL
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected when compiling a class using %Storage.SQL that includes NextCode, and the map uses a global name that contains the characters "NEXT". The logic to replace the pseudo label NEXT was mangling the global name.

      DPV5539: Correct %Date behavior when ZDateNull compatibility setting is 1

      Category: SQL
      Platforms: All
      Version: 2018.1.3

      Caché SQL now supports the Caché configuration setting ZDateNULL=ON. Dates outside of the range 01/01/1841 through 12/30/2098 will evaluate to null when used in SQL statements.

      DPV5570: Correct <PROTECT> error when printing result sets from Management Portal

      Category: SQL
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected where printing a result set from the Management Portal SQL query execution could result in a process stuck in a loop encountering a <PROTECT> error.

      DPV5790: Correct SQL function used as argument to CALL PROC: CALL PROC(...,function(...),...)

      Category: SQL
      Platforms: All
      Version: 2018.1.5

      A problem has been corrected where an SQL CALL <procedure> statement was attempting to use a function as an argument. For example:

      &sql(:ret = CALL SQLUser.Concat1(:Arg1,:Arg2,Concat2(:Arg3,:Arg4,'Z')))

      DPV5809: IDENTITY columns without a specified datatype revert to previous type of INTEGER

      Category: SQL
      Platforms: All
      Version: 2018.1.5

      In Caché/Ensemble 2018.1.4 the behavior of IDENTITY columns without a specified datatype changed. Previously, these had been treated as INTEGER. In 2018.1.4 DPV5292 changed the behavior to treat them as BIGINT. This change reverts the 2018.1.4 change. In 2018.1.5 and future versions IDENTITY columns without a specified datatype are treated as INTEGER.

      DPV5811: Default type of ID column reverts to %Integer

      Category: SQL
      Platforms: All
      Version: 2018.1.5

      In Caché/Ensemble 2018.1.4 the default type of ID column changed. Previously, this had been treated as %Integer. In 2018.1.4 DPV5122 changed the behavior to treat it as %Library.BigInt. This change reverts the 2018.1.4 change. In 2018.1.5 and future versions the default type of ID column is %Integer.

      DTB953: Correct construction of %Execute statements with multiple parameters to respect class query defaults

      Category: SQL
      Platforms: All
      Version: 2018.1.5

      Class queries invoked by the %ZEN.Component.querySource which have parameters initialized with defaults in the definition signature could have those defaults overridden by null strings. This is corrected.

      This problem can be worked around by supplying the appropriate defaults in the pInfo object being supplied to %ZEN.Component.querySource:%CreateResultSet.

      MAK5029: Do not record light weight SQL query stats on a mirrored system unless it is the primary

      Category: SQL
      Platforms: All
      Version: 2018.1.2

      In a mirrored environment when a system is not the primary there was potential for data to be recorded in the CACHE database that was never cleaned up and slowly used more and more database space. This change prevents this unlimited expansion.

      A consequence of this is that lightweight SQL query statistics are not reported from a mirrored non-primary system. If you use a mirrored non-primary system for reporting use of SQL, the lightweight stats will not include these reporting queries.

      SDK048: SQL Import/Export wizard now restricts file extensions

      NOTE: This item may require a change to code, configuration, or operation.

      Category: SQL
      Platforms: All
      Version: 2018.1.5

      With this change, the SQL Import/Export wizard ([System Explorer] -> [SQL] -> [Wizards] -> [Data Import/Export]) will only allow the user to import from or export to files on the server with an allowed extension. Allowed extensions are: .txt, .csv. Previously any file on the system could be used. When importing from or exporting to a file on the client system, filenames are not restricted.

      TRW1564: Ignore %NOINDEX hint in relevant cases of nnlf null analysis

      Category: SQL
      Platforms: All
      Version: 2018.1.5

      The use of the %NOINDEX hint could cause incorrect results when applied to a condition on a subquery. This has been corrected.

      TRW1586: Fix <UNDEFINED> errors caused by IN conditions on lists of composite row IDs

      Category: SQL
      Platforms: All
      Version: 2018.1.3

      Some queries with IN conditions on lists of composite row IDs could trigger <UNDEFINED> errors. This change corrects this issue.

      SQL.DDL

      DPV5460: Correct ALTER TABLE when adding a column with COMPUTEONCHANGE

      Category: SQL.DDL
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected where the class definition is not updated properly when ALTER TABLE is used to add a computed field to a table with a COMPUTEONCHANGE clause.

      DPV5810: SQL DDL: Correct alter table add column not null with default

      Category: SQL.DDL
      Platforms: All
      Version: 2018.1.5

      A problem has been corrected where a statement like:

      alter table ABC add column My_TS timestamp not null default CURRENT_TIMESTAMP

      Would fail to add the column of data existed in the table, and no error was returned.

      It could also be a problem if the default value was GETDATE(), GETUTCDATE(), or SYSDATE.

      SQL.GateWay

      DPV5825: SQL GATEWAY: Correct error reporting for getRows() call when connection disrupted

      Category: SQL.GateWay
      Platforms: All
      Version: 2018.1.5

      A problem has been corrected where a network connection disruption could occur during a JDBC gateway query and the query may not return an error despite not having returned all the rows in the query.

      JCN1995: Make PrepareHandle signature match between interface.h and main.cpp

      Category: SQL.GateWay
      Platforms: Windows
      Version: 2018.1.3

      There was a difference between the signature of PrepareHandle method in main.cpp and the same signature in interface.h. This difference could cause a signal 6 error on UNIX platforms, which causes an application termination. The difference did not have an impact on Windows platforms. This change makes the signatures the same in both files.

      TRW1618: Fix errors INSERT / UPDATE joined tables for due to quoting

      Category: SQL.GateWay
      Platforms: All
      Version: 2018.1.5

      SQL Gateway table labels for JOINs in subqueries of INSERT and UPDATE statements now respect the "Do not use delimited identifiers by default" setting. Without this change column prefixes might not match which caused an error.

      SQL.JDBC

      DPV5377: SQL Gateway: Allow for inserts into Oracle® database view to not use Auto Generated Keys logic

      Category: SQL.JDBC
      Platforms: All
      Version: 2018.1.2

      A problem has been corrected where an INSERT into an external table in an Oracle database might fail if the external table does not support returning auto generated keys. One example of this is the insert into an external table that is defined as a view in Oracle and has an INSTEAD OF trigger. In this case, Oracle does not allow the RETURNING clause on the insert statement. This returning clause is added when RETURN_GENERATED_KEYS is specified for the JDBC statement.

      When the table is linked to Caché, there is no way for Caché to tell the table does not support auto-generated keys. To solve this problem, this change implements a class parameter that will determine when the compiled insert code should attempt to get auto-generated keys or not.

      The new class parameter is:

      /// Determines if INSERT statements for this external table attempt to retrieve auto-generated keys.  Set this to 0 if this external table does not support auto generated keys.
      Parameter EXTERNALGENERATEDKEYS = 1;
      

      The default for this parameter is 1, so if the parameter is not defined, auto generated keys will still be supported upon INSERT.

      The Link Table Wizard will automatically generate this parameter with a value of 1 for JDBC gateway connections.

      If the external table does not support auto-generated keys, and you need to perform inserts into this table, you should modify this class definition and change the value of the EXTERNALGENERATEDKEYS to 0 and recompile the class.

      DVU3588: Fix parser tokeniser crash when using "order" as a table name

      Category: SQL.JDBC
      Platforms: All
      Version: 2018.1.3

      If 'order' was used as a table name at the end of statement, the parser tokeniser was assuming that more data should follow. This change corrects this issue.

      JCN1876: Fix problem in readAhead logic leaving outstanding message

      Category: SQL.JDBC
      Platforms: All
      Version: 2018.1.3

      Under certain circumstances, SQL.JDBC would not consume a message. This could trigger an out of sequence error condition. This change corrects this problem.

      JCN2045: Add hasStreamParameters to server side cache

      Category: SQL.JDBC
      Platforms: All
      Version: 2018.1.1

      Preserve hasStreamParameters in the server side CachedPrepare structure for reuse in a subsequent statement.

      Statement message processing depends on many factors, one of which is "does the statement contain a stream in one of the parameters." If a statement has executed once, we cache the information about the statement for reuse, so when the statement is called again, it makes the performance faster. The statement variable "hasStreamParameters" was introduced to speed decision making when sending statements to the server, but because the variable was not preserved in the cache, the repeated statement did not have access to the information about the stream column. This led to missing a message exchange with the server causing the message sequence error.

      This problem can only happen with stream parameters in a statement that is executed from the cache. This problem is corrected.

      JCN2046: Fix potential hang in JDBC batch insert

      Category: SQL.JDBC
      Platforms: All
      Version: 2018.1.1

      Under certain circumstances, a JDBC batch insert could cause a hang. This change corrects this problem.

      JCN2058: Proper String conversion for BigDecimal $list types

      Category: SQL.JDBC
      Platforms: All
      Version: 2018.1.3

      There were a couple of inconsistent cases where $list types 6 and 7 were not being returned consistently as BigDecimal. This could cause rounding errors if they were being cast to Java Floats which are single precision doubles. This change corrects this issue.

      JCN2166: Fix overflow error using DBList.grabNegLong()

      Category: SQL.JDBC
      Platforms: All
      Version: 2018.1.4

      DBList.grabNegLong() which in turn calls grabNegInt() if the length of the data in the type 7 $list is 5 bytes. The problem is that the negative value is not accounted for and the attempt to convert the int overflows and does not properly report a negative long.

      This appears to happen only for converting a type 7 $list value to a double or long. getObject, getLong and getDouble may also be affected when pulling data from a numeric SQLType.

      This issue displays incorrect data in the jdbc client, but does not cause any database corruption. The data is displayed correctly as a string or BigDecimal value for the SQLType numeric.

      -4294967296 to -2147483649 is the range of numbers affected if you negate the scale on the numeric and possibly long values.

      SQL.ODBC

      DVU3798: JDBC and ADO.Net: Fix calculation of required buffer length

      Category: SQL.ODBC
      Platforms: All
      Version: 2018.1.4

      This change corrects a problem in the calculation of required buffer length in JDBC and ADO.Net.

      JCN1996: Fix access violation using SQLUnicodeTypes flag with SQLGetData

      Category: SQL.ODBC
      Platforms: All
      Version: 2018.1.1

      This change corrects an access violation in processing the rarely used SQLUnicodeTypes flag.

      JCN2038: Fix rare error converting .NET timestamp to %TimeStamp

      Category: SQL.ODBC
      Platforms: All
      Version: 2018.1.1

      In rare cases a multi-threaded application could store an incorrect timestamp with more digits of precision than present in the .NET timestamp. This change corrects this problem.

      SGM007: Correct multibyte to server character conversion

      Category: SQL.ODBC
      Platforms: All
      Version: 2018.1.4

      This change fixes a problem in the ODBC client driver's multibyte to server character conversion. The problem occurred when a client connected to an 8-bit instance and bound a query parameter with multibyte character values. With this fix, the ODBC driver correctly accounts for the application side's variable character length when converting multibyte strings to server-side single-byte-character strings.

      SQL.Query Optimizing

      HSU239: Fix the selectivity estimation for IN condition when BIAS is off

      Category: SQL.Query Optimizing
      Platforms: All
      Version: 2018.1.3

      The SQL query optimizer was incorrectly applying the selectivity of an outlier value for a field in its calculation of the selectivity of an IN condition on that field. This change corrects this problem.

      HSU285: Fix nested inner join query that returned wrong number of records

      Category: SQL.Query Optimizing
      Platforms: All
      Version: 2018.1.1

      Under certain conditions an inner join subquery would return more records than should be selected. This change corrects this problem.

      SQL.Query Processing

      AK983: Fix rare case in which parallelized query returns incorrect results

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.3

      Fixed a problem where in rare cases, a parallelized query would return incorrect results. This problem first occurred in version 2016.2.0.

      DPV5043: Correct <MAXNUMBER> error for %STARTSWITH/LIKE '1E1234' value

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.2

      A problem has been corrected where some queries might encounter a <MAXNUMBER> error at run time. These queries use a LIKE clause where the LIKE pattern is a host variable and the argument value is a string of the form "1E1234" and the number after the "E" is larger than 308.

      This correction is to the SQL generated code, and it requires a recompilation of any embedded SQL and a purge of cached queries in order to correct the statement having this issue on your system.

      DPV5452: Correct ORDER BY %ID DESC query when %ID is single field reference to table with a single field %String IdKey

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected when order by %id desc is performed on a table where the table has a single field idkey that is a reference to a table which is also a single field idkey and the type of the idkey property in the references table is %String. For example, with these two classes:

      Class Test.Class Extends (%Persistent, %Populate) 
      { 
         Property Name As %String; 
         Property MyId As Test.OtherClass; 
         Index PK On MyId [ IdKey, PrimaryKey, Unique ]; 
      } 
      
      Class Test.OtherClass Extends (%Persistent, %Populate) 
      { 
         Property MyId As %String; 
         Index PK On MyId [ IdKey, PrimaryKey, Unique ]; 
      } 
      
      The following query returned no results
      select %Id,Name from Test.Class order by %Id desc 
      
      This has been corrected.

      DPV5467: Correct DATE(<timestamp>) and CAST(<timestamp> AS DATE) when <timestamp> is a subclass of %TimeStamp

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected with the DATE() function and the CAST(... AS DATE) function when the argument to the function has a type that is a subclass of %TimeStamp, %PosixTime, %FilemanDate, %FilemanTime or %String. Similar fixes have also been made for other forms of the CAST function.

      DPV5505: Correct query results with LIKE and a LoopInitValue on an index map

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected where a query using LIKE '...%' or %STARTSWITH against a table using %Storage.SQL and that will use an index map that specifies a LoopInitValue might not return the correct results.

      DPV5576: Correct frozen plan usage if statement is from version earlier than 2017.1 and uses TRIM function

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.3

      A problem has been corrected where a query using the SQL TRIM function might return an incorrect value for TRIM if the statement is using a frozen plan and the plan was frozen using a version of Caché/Ensemble prior to 2017.1.

      DPV5679: Correct <SUBSCRIPT>SavePlanar^%qaqplansave on INSERT statement with virtual OR fields

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.4

      A problem has been corrected where the compilation of a complex INSERT ... SELECT statement may fail with an <SUBSCRIPT> error.

      MAK5003: Fix light weight stats leaving redundant nodes in the data structure

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.2

      The SQL light weight statistics get aggregated from a process into a central area and then from this central area into the SQL statement index where they are available to be viewed. The aggregator from the central area to the SQL statement index logic was leaving redundant nodes in the data structure. These redundant nodes take up space and the aggregator needs to $order over these to get to the statistics that actually have data in them. This change removes these redundant nodes and avoids the space overhead and the CPU cycles needed to constantly skip over these.

      MAK5325: Trap errors thrown due to frozen plan inconsistencies and re-plan query

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.5

      If frozen plan uses an index that has been dropped Caché reported an error. With this change the query plan is recreated without using the index.

      TRW1634: Check mt("c") constant conditions for references requiring evaluation

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.1

      Subquery correlated field references to constant expressions could trigger <UNDEFINED> errors. This change corrects this error.

      TRW1645: Fix identification of row-level security predicates and propagation of complex logs in OR transformations

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.3

      This change corrects a problem where some parallel queries against tables with row-level security could produce a runtime error.

      TRW1651: Correct outer join queries with TOP and DISTINCT or GROUP BY

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.3

      This change corrects a problem where some outer join queries with TOP in combination with DISTINCT or GROUP BY could return incorrect null-padded results.

      TRW1683: Properly declare INSET predicate to cancel outer joins

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.4

      This change corrects a conditon where combining arrow syntax and FOR SOME %ELEMENT predicates could result in incorrectly null padded rows.

      TRW1687: Retain null checking for unique subscript looping in the RHS of a LEFT OUTER JOIN

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.4

      Queries performing outer joins with ON clauses referencing values that may be null could perform unnecessary work.

      TRW1690: Don't add %VID's to streamed view SELECT lists created during ON clause transformation

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.4

      This change corrects a condition where some queries performing outer joins on views could perform poorly relative to earlier versions of Caché.

      TRW1702: Always regenerate modata after ^%qaqpagg

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.5

      COUNT(*) queries with nested subquery views or complex outer joins could fail to compile with an <UNDEFINED> error on an asl(..."train"...) node. This change corrects this issue.

      TRW1704: Assume list collection fields are not "short"

      Category: SQL.Query Processing
      Platforms: All
      Version: 2018.1.5

      DISTINCT or GROUP BY on a list collection field could result in a <SUBSCRIPT> runtime error if the combined length of all values in the field for a row exceeded ~500 characters. This change corrects this issue.

      SQL.SQLFiler/TableCompiler

      DPV5328: SQL Filer: Correct generation of Normailze and ValidateField methods when methods contain MPP commands

      Category: SQL.SQLFiler/TableCompiler
      Platforms: All
      Version: 2018.1.1

      A problem has been corrected in the class compiler when a property has a Normalize or IsValid member method that includes macro preprocessor directives like #IF 0 ... #ELSE ... #ENDIF. In some cases, this might have caused bad code generation, and the class would fail to compile.

      Studio

      AJP018: Do not generate [ Language = objectscript ] at the class level for Zen/DeepSee classes

      Category: Studio
      Platforms: All
      Version: 2018.1.5

      This change fixes a problem that could cause compilation failure for some generated classes.

      DVU3699: Fix occasional Studio crash when Output/Find in Files pane set to auto hide

      Category: Studio
      Platforms: All
      Version: 2018.1.1

      When Output and/or Find in Files pane is set to auto hide, Studio can crash while streaming output if output includes Unicode characters. This change corrects this problem.

      MAK4774: When scanning a package to see if any entries are visible if we do not find any after 2,000 entries assume something is visible

      Category: Studio
      Platforms: All
      Version: 2018.1.4

      When scanning a package to see if any entries are visible if we do not find any after 2,000 entries assume something is visible rather than assuming nothing is visible in this package. This could result in some directories showing up in the Studio open dialog but when you click on these to go into this directory nothing is displayed, but this is better than the converse where the directory itself does not show up even though you know there are items of interest inside this.

      MAK4930: Support HTTPS connections when creating a Studio template session

      Category: Studio
      Platforms: All
      Version: 2018.1.3

      If your internal web server only accepts HTTPS requests Studio, it failed to create a CSP template session. With this change if ^%SYS("WebServer","Protocol")="https" the CSP template creation logic will turn on HTTPS when making the request. As the %Net.HttpRequest class needs an SSL configuration in order to make an HTTPS connection this can be specified by setting the global ^%SYS("WebServer","SSLConfiguration") to the SSL configuration name.

      You can also specify a default SSL configuration name for all %Net.HttpRequest objects using the ^%SYS("HttpRequest","SSLConfiguration") global setting, however the ^%SYS("WebServer","SSLConfiguration") is specific to creating CSP template sessions from Studio where as this other global will apply to any %Net.HttpRequest object that is created on this system.

      System

      ALE3355: Build xerces without curl

      Category: System
      Platforms: All
      Version: 2018.1.4

      Xerces will not include curl support on all platforms.

      CDS2974: Correct size of global buffer area for VIEW validation

      Category: System
      Platforms: All
      Version: 2018.1.3

      In some circumstances, seen primarily on a Ubuntu instance, a valid VIEW command in system code could throw a <COMMAND> error. This change corrects this issue.

      CDS3049: ^%ETN will preserve state of <STORE> handling

      Category: System
      Platforms: All
      Version: 2018.1.1

      Calling LOG^%ETN or BACK^%ETN after a <STORE> error would change the state of the memory, causing a second <STORE> error. This change corrects this problem.

      CDS3053: Issue proper error for <_CALLBACK SYNTAX>

      Category: System
      Platforms: All
      Version: 2018.1.3

      When incorrect callback syntax is used (a variable name with an underscore), the error <_CALLBACK SYNTAX> will be issued instead of <FUNCTION>.

      CDS3060: Support long strings for symbol table save/restore

      Category: System
      Platforms: All
      Version: 2018.1.3

      The system code that saves and restores the symbol table for Weblink applications will now support strings larger than 32767 characters.

      CDS3067: Fix <UNIMPLEMENTED> caused by low memory and argument passed by reference to args... formal argument

      Category: System
      Platforms: All
      Version: 2018.1.2

      If an argument is passed by reference to a formal argument like args... and the partition memory becomes full there could be an <UNIMPLEMENTED> error. This change corrects this problem.

      CDS3068: Fix access violation when formal args... array is greatly increased

      Category: System
      Platforms: All
      Version: 2018.1.2

      If an argument is passed by reference to a formal argument like args... and many args(n) entries are added in the subroutine, there could be an access violation. This change corrects this problem.

      CDS3086: Fix $ListValid() that could cause access violation

      Category: System
      Platforms: All
      Version: 2018.1.1

      In some rare circumstances an invalid string passed to $ListValid() could cause a memory access violation. This change corrects this problem.

      CDS3092: Avoid string stack overflow with $Query() and $Name()

      Category: System
      Platforms: All
      Version: 2018.1.3

      This change fixes an issue where in rare cases a $Query() or $Name() (including the internal query used by the Merge command) on a global with an extended reference could result in an access violation.

      CDS3096: Improve performance of CSP and other background processes

      Category: System
      Platforms: All
      Version: 2018.1.3

      A problem has been fixed that caused some background processes, particularly CSP processes, to have worse performance than an identical foreground process.

      CDS3108: Fix access violation with large MultiValue arrays

      Category: System
      Platforms: All
      Version: 2018.1.3

      This change fixes an issue where a MultiValue program with a large array, DIM(n) with n>8187, could get an access violation.

      CDS3118: Avoid premature end of Weblink symbol table restore

      Category: System
      Platforms: All
      Version: 2018.1.3

      This change fixes an issue where in some cases if a variable to be restored already exists, the Weblink restore operation could end prematurely without restoring everything, but with a success return code.

      CDS3119: Large command count from $V(-1,pid) is wrong

      Category: System
      Platforms: All
      Version: 2018.1.4

      A long-running process that accumulates a very large number of commands would see a non-numeric value from $VIEW(-1,pid) which is used in some queries.

      CDS3176: Correct rare object code problem on big endian systems

      Category: System
      Platforms: All
      Version: 2018.1.3

      Under a very rare circumstance involving the structure of a routine's object code, the routine could not be run on a system that has different endian than the system that compiled the routine. The result could be an access violation or <NOLINE> errors. The change fixes this problem and applies to big endian systems only.

      CDS3182: Prevent loop with TRY/CATCH and <STORE> errors

      Category: System
      Platforms: All
      Version: 2018.1.4

      When processing nested <STORE> errors with TRY/CATCH there could be a condition that results in the process going into an uninterruptible loop. That condition will now result in process termination with <STORE> <ERRTRAP>.

      CDS3209: Correct issue with ValidateRoutineBuffers()

      Category: System
      Platforms: All
      Version: 2018.1.5

      An issue has been corrected where a rare timing condition could cause routine buffer corruption if the method $SYSTEM.Util.ValidateRoutineBuffers() was run during routine activity.

      CDS3215: Handle $TEXT() error properly

      Category: System
      Platforms: All
      Version: 2018.1.5

      In some cases when $TEXT() throws an error caught by $ETRAP there could be an access violation that terminates the process.

      CDS3219: Eliminate possible access violations during job start

      Category: System
      Platforms: All
      Version: 2018.1.4

      It was possible for a ProcessQuery to see an invalid $ROLES structure if it happened at just the wrong time during the initialization of a new job. This condition could cause access violations. This change corrects this problem.

      CDS3250: Process created by JOB command could crash if output device is closed

      Category: System
      Platforms: All
      Version: 2018.1.5

      If a JOB command specifies an output device and no input device, and the new process closes the output device and then tries to write to it, the process would fail with an access violation.

      CDS3257: Do not pass LD_LIBRARY_PATH to external programs

      Category: System
      Platforms: All
      Version: 2018.1.5

      The loader library path environment variable will not be passed to processes created with $ZF(-1), $ZF(-2), $ZF(-100), or CPIPE.

      CDS3273: MERGE in low memory conditions could fail

      Category: System
      Platforms: All
      Version: 2018.1.5

      A MERGE command into a local variable when $STORAGE is low could result in silent failure or a later memory access violation.

      CDS3287: Correct memory error during compilation

      Category: System
      Platforms: All
      Version: 2018.1.5

      This change corrects an issue where compiling ObjectScript code containing a concatenation of mixed ASCII and Unicode strings could cause a memory error.

      DMC1172: Support Ubuntu 20.04 LTS

      Category: System
      Platforms: All
      Version: 2018.1.5

      This change adds InterSystems IRIS platform support for Ubuntu 20.04 LTS.

      DMC1178: Fix MQ username / password login

      Category: System
      Platforms: All
      Version: 2018.1.5

      To avoid a problem, this change ensures that when authenticating with username / password set the connection Version to 5.

      DMC1179: Improve SFTP filename character set handling

      Category: System
      Platforms: Windows
      Version: 2018.1.5

      The logic used for narrowing filenames passed from a Unicode instance to libssh2 used the `RemoteCharset` property to allow the user to customize how the translations occurred. However, on Windows this logic ultimately ends up using WideCharToMultiByte() (locally) which doesn't give the proper control over the charset on the remote side. This worked on UNIX because UNIX tends to be UTF8, but for a PSFTP(a common Windows SFTP server) using CP1252 a Windows client would not be able to properly convert the character sets no matter what value of `RemoteCharset` was used (even "" which means "no conversion").

      Remote SFTP server versions >3 do use UTF8 as per the specification, so we do not do any conversions other than from UTF8 in this case.

      DP-402582: Prevent the ECP client write dmn form timing out

      Category: System
      Platforms: All
      Version: 2018.1.6

      If it takes too long for the ECP server to recover (for example because of too many jrn files to scan), this change will prevent the ECP client write dmn from timing out and preventing it from recovering.   Without this change the ECP app-server may not recover properly when there is a mirror failover or DB-server restart when recovery takes more than 2 minutes.

      DP-405455: Restore open flags for Windows async I/O

      Category: System
      Platforms: Windows
      Version: 2018.1.6

      This change reverses the negative performance impact of a previous change on async I/O on certain Windows systems. It appears that the effects of that change are noticeable on spinning drives although it may be related to certain versions of Windows. Aynch I/O is used to write incremental backup output files, databases and the wij, by mirroring and by the prefetch code.

      DP-405464: Update user record with login info less frequently

      Category: System
      Platforms: All
      Version: 2018.1.0

      When a user successfully logs in, their user record is updated with the following information:

      InvalidLoginAttempts - set to 0
      LoginDateTime - set to $zts
      LoginService - Service the user logged in from
      LoginDevice - Device the user logged in on
      

      However, on a system where REST calls are being repeatedly executed by a user, updating the user record with this information becomes a bottleneck (e.g. hundreds of logins per second). It also generates excessive journal records.

      Now when a user logs in, the information is only updated if the length of time from the last login is > 2 seconds, or if the previous login attempt was a login failure.

      This should significantly decrease the amount of data being put in the journal file.

      Note that the audit record is still written for a successful login.

      DP-405710: Don't consider the journal dmn waiting for a mirror buffers as hung

      Category: System
      Platforms: All
      Version: 2018.1.6

      This change avoids a case where the backup mirror member could force down the primary if the primary is stuck waiting on the backup while sending journal data.

      DP-405781: Call delegated authentication even for normal logins

      Category: System
      Platforms: All
      Version: 2018.1.6

      A previous change removed the call to delegated authentication before calling authentication. This change restores this call.

      DP-407254: Update interaction between delegated and password authentication

      NOTE: This item may require a change to code, configuration, or operation.

      Category: System
      Platforms: All
      Version: 2018.1.0

      This fixes an issue with authentication ordering when delegated and password authentication are enabled. If your user authentication includes both delegated authentication and password authentication, you can choose either to call or not call ZAUTHENTICATE for password users. Typically, ZAUTHENTICATE is only used for delegated users and not for password users.

      But if you want your ZAUTHENTICATE routine to also be called for password users, you should check the <b>Always try Delegated Authentication</b> in the <b>System Administration > Security > System Security > Authentication/Web Sessions Options</b> page in the Management Portal or by using the ^SECURITY utility (System Parameter Setup, Edit authentication options).

      The default for <b>Always try Delegated Authentication</b> is No and ZAUTHENTICATE is not called for password users.

      DP-408426: Eliminate an unnecessary error about journal switch

      Category: System
      Platforms: All
      Version: 2018.1.6

      It's unnecessary for ^JRNSWTCH or %SYS.Journal.System:RollToNextFile() to fail when journal switch is in progress.

      DP-408747: Fix HealthShare Security Domain Functionality

      Category: System
      Platforms: All
      Version: 2018.1.6

      This change is to handle the required HealthShare Security Domain Functionality.

      DP-408869: Fix %Library.GlobalEdit:GetGlobalSizeBySubscript() and KillRange() for Begin ranges like X("A")*NEXT:X("C")

      Category: System
      Platforms: All
      Version: 2018.1.6

      Ranges similar to X("A")*NEXT:X("C") were sometimes including data under the ^X("A") node. What it should do is start looking at data after all the ^A("A") nodes, for example, ^A("B") and later. With this change it performs a $query() and gets the next subscript at this level (for example ^A("B"), and changes the start range to this. This updated range is then used to calculate the size. DataMove uses this method, and the end result was that it could see MBToCopy values higher than what was actually copied. This also would cause the estimated time to finish to be longer than what the copy actually took.

      For example:

      %SYS>F i=1:1:1200 s ^A("A",i)=$j(i,30)
      s x=##Class(%GlobalEdit).GetGlobalSizeBySubscript($zu(12),"A(""A"""_$c(1)_"*NEXT)","A(""C"")",.s)
      

      Before the change, the behavior would be:

      %SYS>zw s
      s=.06
      s("Blocks",1)=1
      s("Blocks",2)=6
      s("Blocks","BigStrings")=0
      s("Blocks","Total")=7
      s("MB",1)="0.01"
      s("MB",2)="0.05"
      s("MB","BigStrings")="0.00"
      s("MB","Total")=.06
      

      After the change, the behavior would be:

      %SYS>zw s
      s=0
      s("Blocks",1)=0
      s("Blocks",2)=0
      s("Blocks",3)=0
      s("Blocks","BigStrings")=0
      s("Blocks","Total")=0
      s("MB",1)="0.00"
      s("MB",2)="0.00"
      s("MB",3)="0.00"
      s("MB","BigStrings")="0.00"
      s("MB","Total")=0
      

      And with the following:

      %SYS>F i=1:1:1200 s ^A("A",i)=$j(i,30)
      s x=##Class(%GlobalEdit).KillRange($zu(12),"A(""A"""_$c(1)_"*NEXT)","A(""C"")")
      

      Before the change, all nodes under ^A("A") would be deleted. After the change, all the nodes would continue to exist.

      DP-410605: Fix User security record being corrupted

      Category: System
      Platforms: All
      Version: 2018.1.6

      When a process calls $SYSTEM.Security.Login() to log in as another user, if an error occurs in the span of a couple of lines of the method, the original user record can be overwritten by the new user record. This would cause the original user to no longer be able to log in.

      This was seen when a stored procedure which as one of its tasks uses $SYSTEM.Security.Login() to change itself to the _SYSTEM user to do some "system stuff". It just so happened that the connection had a timeout set, and an <ALARM> error was generated at just the wrong spot in the code when the user credentials were getting changed. Normally an error could never happen in this section of the code during normal operation, but when an alarm is set by the server code, an asynchronous error could occur causing the failure.

      This change prevents the old user record from being overwritten by the new user record in the case of an error.

      Note that the window here is very small and unlikely to occur.

      DP-410986: Fix memory leak caused by LDAP searching with pages

      Category: System
      Platforms: All
      Version: 2018.1.0

      Calling ##Class(%SYS.LDAP).GetNextPages() could cause a memory leak when searching. This problem could be seen using the LDAP interoperability Business Operations. This change corrects the problem.

      DP-418812: Mount databases for directio on Unix when wdwrite_asyncio_max is non-zero

      Category: System
      Platforms: UNIX®
      Version: 2018.1.0

      This change fixes a bug where a database which had been dismounted and remounted would not have direct I/O enabled on a system where the write daemon is using async I/O (the default configuration).

      DP-419512: Restore large page support on newer Windows versions

      Category: System
      Platforms: Windows
      Version: 2018.1.0

      This change restores large page support on newer Windows versions. Specifically, it corrects how InterSystems IRIS calls the Windows MapViewOfFile() function. (This function was changed starting with Windows 10 version 1703 and later Windows Server versions.)

      DP-421333: Avoid timeout errors in callin mirror routines when deferred licensing is on

      Category: System
      Platforms: All
      Version: 2018.1.9

      Previously, when an instance was using deferred user identification—by calling $SYSTEM.License.DeferUserIdentification(1)— and set a mirror to NOFAILOVER, there could be timeout error. This issue has been corrected.

      DP-423100: fix lookup of right link bdb in irisstat -b64

      Category: System
      Platforms: All
      Version: 2018.1.9

      This change increases the efficiency of the irisstat -b64 option.

      DP-426214: support web server url prefix usage pre-2023.2

      Category: System
      Platforms: All
      Version: 2018.1.9

      The change ensures that all URLs generated by InterSystems IRIS will include the configured WebServerURLPrefix, even if WebServerName is not set.

      GK1346: Changed routine load polling interval

      Category: System
      Platforms: All
      Version: 2018.1.4

      There's was a 200ms polling interval when a routine buffer is discovered being loaded by some other job. Reduced that interval to 1 ms.

      GK1356: Fixed a routine cache corruption

      Category: System
      Platforms: All
      Version: 2018.1.3

      In rare conditions, if for any reason a large routine loader fails to load one of the large routine chunks, it may corrupt the routine hash table. This change corrects this rare problem.

      GK1390: Corrected execution context when returning form a routine or method in a limited memory environment

      Category: System
      Platforms: All
      Version: 2018.1.3

      In very rare condition when returning from a method or a routine and the process has limited free memory, the execution context was unpredictable. This change corrects this issue.

      GK1404: Fixed a rtn cache cleanup loop

      Category: System
      Platforms: All
      Version: 2018.1.3

      This change fixes a condition where in a rare condition the class and routine invalidation could get in an infinite loop during the cache cleanup on namespace switch or configuration change.

      GK1418: Protect against unexpected object context switch

      Category: System
      Platforms: All
      Version: 2018.1.0

      This change corrects rare conditions where if there is an unexpected and unhandled error during destruction, the active class and object may not get restored correctly.

      JLC2167: Ensure that a useful message is displayed after failure starting emergency administrator access mode

      Category: System
      Platforms: All
      Version: 2018.1.1

      A failure starting the emergency administrator access mode displayed a low-level error message which did not provide guidance on correcting the failure. This change ensures that a useful message is displayed under this failure condition.

      JLC2191: Fix CPU count for AIX

      Category: System
      Platforms: AIX
      Version: 2018.1.3

      This change corrects a problem where, on some AIX systems, the number of CPU chips displayed by $System.CPU.Dump() and other system functions was the physical number of chips instead of the number allocated to the logical partition.

      JLC2212: Remove the uppercase translation of the German sharp S character

      Category: System
      Platforms: All
      Version: 2018.1.3

      This change removes the uppercase translation of the German sharp S character that had been introduced in Caché 2018.1. It reverts the translation to the previous behavior, which is to assign the uppercase of this character to itself.

      JLC2316: Fix ppc64 crash due to bad pointer in despnt->valpnt (AL Clsetstr)

      Category: System
      Platforms: All
      Version: 2018.1.5

      This change fixes a rare problem affecting POWER/PowerPC (ppc64) platforms that could cause a crash in low-memory situations.

      JO3121: Preserve journal files required for "delayed" transaction rollback operations

      Category: System
      Platforms: All
      Version: 2018.1.3

      A problem has been resolved where in the rare instance that a user suspended transaction rollback at startup, the system did not correctly preserving the journal files required to restart the rollback operation.

      JO3145: Prevent overflow in calculating number of mb removed from database by truncation

      Category: System
      Platforms: All
      Version: 2018.1.5

      Corrected overflow issues related to reporting the number of MB freed by a database truncation operation (return unused space at end).

      JO3151: Avoid exceptions due to errors during global operations

      Category: System
      Platforms: All
      Version: 2018.1.4

      A problem has been resolved which in rare circumstances, if an error was thrown during a global operation, it could result in database degradation during subsequent sets.

      JO3158: Resolve trouble with rollback skipping updates from a prior journal file on the primary mirror member

      Category: System
      Platforms: All
      Version: 2018.1.4

      A problem has been resolved where rollback could fail in a mirrored database if the database was brought over from another system and activated and the rollback required reading an earlier journal file than the current one. The window of vunerability exists until the system is restarted. The rollback failure leaves a message similar to the following in the console log:

      01/28/20-18:03:48:802 (7864) 1 [Utility.Event] Skipped rolling back 1 update(s) in c:\intersystems\irisa\mgr\journal\MIRROR-TESTLA-20200128.016 @270216
      01/28/20-18:03:48:830 (7864) 1 [Utility.Event] Total 1 update(s) were skipped during rollback of transaction beginning at address 270200 of c:\intersystems\irisa\mgr\journal\MIRROR-TEST-20200128.016. 
      

      This can occur if the transaction was not fully lock-protected from concurrent changes, or if it was partially rolled back already, or if it was partially duplicated in another file due to journal switch.

      JO3160: Correct $O(^$GLOBAL()) to return all results in an implied namespace

      Category: System
      Platforms: All
      Version: 2018.1.4

      Corrected $O(^$GLOBAL(...)) so it no longer stops (returns a null string) after returning the 1st global name when run in an implied namespace.

      JO3162: Write dmn error handling corrections for UNIX/writev()

      Category: System
      Platforms: UNIX®
      Version: 2018.1.5

      A rare problem has been resolved where an apparently successful retry after a write error could result in database degradation of the blocks involved. The problem was restricted to UNIX systems which are not using async database I/O. A signal that this may have occurred is a "SERIOUS DISK WRITE ERROR" messages in the console log followed by "Retry succeeded, write daemon continuing."

      RJF329+JO3013: Enhancements to provide improved performance on large scale systems

      Category: System
      Platforms: All
      Version: 2018.1.1

      This change includes the following enhancements that improve performance on large scale systems:

      • Improved performance handling mappings with a large number of subscripts.

      • Reduced resources required to track statistics.

      • Reduced contention managing global buffers

      These changes increase the amount of memory allocated for global buffer metadata by 64 bytes per buffer on Intel systems and by 128 bytes per buffer on IBM Power systems. For example, with 8K buffer sizes, the shared memory allocated for a global buffer increases by 0.75% on Intel systems and by 1.5% on IBM Power systems.

      These enhancements cause minor changes in statistics displayed by utilities and the Management Portal.

      RJF332: Fix free count recorded in integrity check (from Mgmt Portal, Task or $$CheckList^Integrity)

      Category: System
      Platforms: All
      Version: 2018.1.1

      A problem introduced in Caché 2018.1 caused integrity check to incorrectly report free blocks. This is corrected. An example of the incorrect output follows (note the last line)

      Summary of blocks in /cache/iris/mgr/user/
      

      357 Pointer Level blocks 2856kb (10% full) 36,586 Data Level blocks 285MB (88% full) 1,436 Big String blocks 11MB (83% full) # = 524 38,442 Total blocks 300MB (87% full) 0.020284 Free blocks 0kb

      RJF366: Relax global directory name check in $view(,-5) to improve REPAIR

      Category: System
      Platforms: All
      Version: 2018.1.3

      This change improves REPAIR in the case where bad global names occur. This change relaxes the global name check in $view(,-5).

      RJF410: Fix some sign bit issues for databases with more than two billion blocks (compaction, defrag, and cstat)

      Category: System
      Platforms: All
      Version: 2018.1.4

      This change corrects a number of issues related to running database management utilities on databases with more than two billion (2**31) blocks. This affected database compaction, truncation, defragmentation, and cstat. Data integrity was not compromised, but the utilities might not function properly.

      RJF452: Fix unnecessary delay reading incremental bitmaps in the write daemon on Windows

      Category: System
      Platforms: Windows
      Version: 2018.1.5

      A problem has been corrected on Windows where the write daemon could waste an excessive amount of time while reading incremental bitmap blocks from disk. In extreme cases with very large databases after a burst of updates distributed widely across data ranges, this could last long enough to cause the system to pause due to write daemon inactivity being detected. Systems that have never used online backup could not experience this problem because the databases are marked to not track incremental changes. This is visible in irisstat -a0 -m1: databases with the NBK flag set are immune.

      SML2644: Fix a timing issue on starting a job while Caché is shutting down

      Category: System
      Platforms: All
      Version: 2018.1.1

      Under certain circumstances shutting down the system could cause an error if a job was being started. This change eliminates this error.

      SML2660: Fix $zu(49) for databases with large size (16+ TB).

      Category: System
      Platforms: All
      Version: 2018.1.3

      SML2697: Fix access violation when releasing shared memory

      Category: System
      Platforms: All
      Version: 2018.1.2

      This change fixes an access violation when releasing shared memory. This violation typically occurred during start-up on systems with very limited memory.

      SOH631: Clear out pregex cache on job servers

      Category: System
      Platforms: All
      Version: 2018.1.4

      The $LOCATE and $MATCH functions save the most recent regular expression object so that it will not have to be recreated in programs that repeatedly search using the same regular expression. The memory allocated to this saved object was not being released when a job was halted. If the halted job is a Job Server job then when that job server was reused the memory allocated to the saved regular expression object was lost.

      This problem has been fixed. The InterSystems kernel has been modified so that the saved regular expression object is closed whenever a Job Server job is halted.

      SOH652: Fix Race Conditions sending MBXs between MBX readers and main line

      Category: System
      Platforms: Windows
      Version: 2018.1.4

      Certain messages that are sent between processes are received in a thread or signal handler responsible for receiving mail box (MBX) buffers. In many cases, these MBX requests can be directly replied to by the receiving thread/handler. In a few complex cases the MBX buffer must be transferred to the main line ObjectScript interpreter for processing. These transfers of MBX buffers within a process are done asynchronously and execution by the ObjectScript interpreter is delayed until the interpreter is in a state suitable for handing asynchronous requests. In a few situations on only the Windows platforms this synchronization of a MBX message buffer transfer between the MBX message receiving thread and the main-line thread running the ObjectScript interpreter could encounter "glitches" or "race conditions". In heavily loaded Windows systems many calls to the %SYS.QueryProccess methods could cause the process to crash. This change is designed to fix such glitches when they occur on a Windows platform.

      Examples of asynchronous MBX messages handled by the ObjectScript interpreter include:

      • Attaching an ObjectScript debugger to a job at the same instant the job is starting could cause that job to crash. With this change, the system now ignores messages that attempt to connect to a debugger when the MBX buffer is received before the new job is initialized.
      • Sending an MBX message requesting that a process execute some specific ObjectScript code could result in incorrect, "glitchy" behavior when the MBX buffer is received during process start up or if it is received when the program debugging state is being modified.
      • Other MBX messages that are asynchronously transferred between the MBX message receiving thread and the main-line ObjectScript interpreter thread are MBX requests that lock data base files and MBX requests that contain broadcast messages.

      In all these examples this change modifies the interface between the Windows thread that receives MBX messages and the main-line thread that executes the MBX requests. The "glitches" and "race conditions." have been eliminated. This description also covers the fixes in SOH594 and SOH654.

      SOH706: Eliminate MBX stall during SSLread/SSLwrite

      Category: System
      Platforms: All
      Version: 2018.1.4

      Whenever a process was waiting for a SSLread or SSLwrite operation to complete there was a slowdown in performance of certain %SYS.ProcessQuery methods that were querying that process because the process queries had to wait for any pending SSLread/SSLwrite operations to complete an iteration of a time-out loop. The methods now stop waiting for an SSLread/SSLwrite whenever another process sends a process query request. After completing the process query request the methods go back to waiting for the SSLread/SSLwrite to complete.

      SOH716: Fix problem in %BeginChangeTracking() %EndChangeTracking() methods

      Category: System
      Platforms: All
      Version: 2018.1.5

      In class %ZEN.Controller, the %BeginChangeTracking() and the %EndChangeTracking() methods did not work correctly when there were a large number of objects to be tracked. This has been fixed.

      SOH722: Add GRETRELEASE to eregex.c modules

      Category: System
      Platforms: All
      Version: 2018.1.5

      An in-use global buffer is not released before doing a regular expression match in Caché. Therefore, a long running regular expression match can prevent the write daemon from running. Regular expression matching in Caché will now release any held global buffer before doing the regular expression evaluation.

      STC2817: Fix installing into directory whose name has special symbols, such as the equals sign

      Category: System
      Platforms: All
      Version: 2018.1.1

      Installing Caché could create extra copies of the data files if the install directory has a special symbol in its path. This change corrects this problem.

      System.I/O

      CDS3167: Fix Windows csession extra output character

      Category: System.I/O
      Platforms: Windows
      Version: 2018.1.3

      This change corrects the output from csession on Windows, which included one extraneous character at the end of the session.

      Tasks

      RFD2108: Fix Task Manager message for task over run

      Category: Tasks
      Platforms: All
      Version: 2018.1.5

      Fixes problem with false "previous task is still running" messages.

      Terminal

      CDS3076: Reliability updates for Terminal

      Category: Terminal
      Platforms: All
      Version: 2018.1.3

      This change fixes some problems in Terminal that could cause a hung process or access violation, primarily during the start or end of a session.

      CDS3124: Fix Kerberos terminal hang after Control-C

      Category: Terminal
      Platforms: Windows
      Version: 2018.1.3

      This change fixes an issue where a Windows terminal using a Kerberos connection could stop writing output after a Control-C interrupt.

      CDS3132: Add Windows terminal escape sequences for screen size and title

      Category: Terminal
      Platforms: Windows
      Version: 2018.1.3

      The Windows terminal now supports these escape sequences:

      ESC [ 1 t  - restore window
      ESC [ 2 t  - minimize window
      ESC [ 11 t - report window state
      ESC [ 8;rows;columns t  - set window size
      ESC [ 18 t  - report window size
      
      The window state is reported in $ZB of the following READ command as
      normal: ESC [ 1 t
      minimized: ESC [ 2 t
      
      The window size is reported in $ZB of the following READ command as
      ESC [ 8;rows;columns t
      

      If rows or columns is 0 in the set command, the current value is not changed.

      Range of values supported is rows: 10-120, columns: 10-160

      One consequence of changing the size is that the scroll-back buffer is cleared by the reset. Another is that for larger row values the font size is decreased to make the window fit on the screen.

      This sequence is now supported to set the window title:

      OSC 2; title ST
      

      OSC, Operating System Command, is the 7-bit sequence ESC ] or the 8-bit character $C(157).

      ST, String Terminator, is the 7-bit sequence ESC \ or the 8-bit character $C(156).

      The maximum length of the title is 80 characters.

      Tools

      DP-11651: Correctly differentiate between operating system commands in SystemPerformance

      Category: Tools
      Platforms: All
      Version: 2018.1.5

      A problem has been corrected where ^SystemPerformance diagnostic reports could contain missing or incorrect performance data from OS-level commands (such as irisstat, sar, and iostat).

      DP-12272: Check essential requirements during SystemPerformance initialization and abort on failure

      Category: Tools
      Platforms: All
      Version: 2018.1.5

      This change ensures that SystemPerformance will abort, and report an error, if initialization finds that any essential requirements are not met. Currently these requirements are that the defined log directory is usable, and that the logged in user has enough privileges (Read/Write for the ^IRIS.SystemPerformance global).

      DP-13388: Add Linux 'lscpu' command output to CPU section of ^SystemPerformance

      Category: Tools
      Platforms: UNIX®
      Version: 2018.1.6

      The system performance tool collects 'lscpu' as well as the contents of /proc/cpuinfo, both of which contain information about CPUs on the system. It's only collected once per run.

      DP-13400: Collect multiple 'tasklist' outputs in ^SystemPerformance on Windows

      Category: Tools
      Platforms: Windows
      Version: 2018.1.6

      Before this change, tasklist was only collected once at the start of the profile run. Now it's collected 4 times at even intervals.

      DP-270035: Use unique output file names in SystemPerformance to avoid collision when multiple reports are running concurrently

      Category: Tools
      Platforms: All
      Version: 2018.1.5

      A defect has existed since the inception of ^SystemPerformance (and ^pButtons in Caché/Ensemble), where some operating system information could fail to be gathered if two or more reports were running concurrently. This defect has been corrected.

      DP-278280: Fix handling of $ZR value in KillRange

      Category: Tools
      Platforms: All
      Version: 2018.1.9

      This change fixes a problem in %Library.GlobalEdit.KillRange() that prevented it from sometimes missing the first node in a range of nodes.

      DP-278341: Fix REPAIR calculation of blnextpntlen et al, and display them consistently after edits

      Category: Tools
      Platforms: All
      Version: 2018.1.6

      Fixed some issues around calculation and display of pointer references when editing database blocks with ^REPAIR. Also improved some error messages in REPAIR.

      DP-278352: Better errors returned from DMREPAIR when top pointer can't be found in global directory

      Category: Tools
      Platforms: All
      Version: 2018.1.6

      The error reporting for REPAIR functions that traverse the tree has been improved in the case that the global directory entry can't be found. The functions in question include "Pointer Tree" as well as the "UP" and "LEFT" navigation functions and others. Prior to this change they could produce <BLOCKNUMBER> or "0 is not a block number". Now they report "Global [global_name] not found."

      DP-278364: REPAIR only asks about down pointer big string bit when editing bottom pointer nodes

      Category: Tools
      Platforms: All
      Version: 2018.1.6

      In REPAIR, when editing nodes of a pointer block the prompt "Does that block contain any big string nodes?" will only appear if the block being edited has bottom pointer type. The prompt was irrelevant/misleading in other cases.

      DP-278617: Update %Library.GlobalEdit:GetGlobalSizeBySubscript() when no data in range

      Category: Tools
      Platforms: All
      Version: 2018.1.9

      This change updates %Library.GlobalEdit.GetGlobalSizeBySubscript() to handle the special case were no data is in the range.

      DP-278763: Handle quote in subscript for KillRange

      Category: Tools
      Platforms: All
      Version: 2018.1.9

      In this change, %Library.GlobalEdit.KillRange() now properly handles a quote appearing in the starting subscript.

      DP-278823: %GlobalEdit:KillRange can't handle certain subscripts

      Category: Tools
      Platforms: All
      Version: 2018.1.9

      This change ensures that %Library.GLobalEdit.KillRange() correctly trims a "|" character from a namespace or directory in the subscript of a global name.

      DP-402896: Fix <UNDEFINED> error in ^REPAIR deleting first node from pointer block

      Category: Tools
      Platforms: All
      Version: 2018.1.6

      Starting in Caché 2018.1.0, attempting to use ^REPAIR to delete the first node of a pointer block would fail with an <UNDEFINED> error.  That is corrected.

      DP-405779: SystemPerformance - new section "irisstat -R" containing a single snapshot of routine buffers

      Category: Tools
      Platforms: All
      Version: 2018.1.6

      SystemPerformance reports on all platforms will now include a new section named "irisstat -R".  This section contains a single sample of the irisstat command using the -R1 flag, which dumps routine buffers.

      DP-418892: Reset REDEBUG and ECP debug flags in ^SystemCheck when an error occurs while collecting irisstats

      Category: Tools
      Platforms: All
      Version: 2018.1.9

      The REDEBUG and ECP debug flags, which are set by ^SystemCheck for the duration of irisstat snapshot collection, are reset to their original values if an error occurs during execution.

      SOH724: Fix Telnet freezes with SSL connections

      Category: Tools
      Platforms: Windows
      Version: 2018.1.5

      This change corrects a problem where applications using Windows telnet connections that include SSL encryption could freeze or the Telnet terminal would only echo one character at a time. This problem first occurred in Caché 2018.1.4.

      Utilities

      AAR026: Correctly differentiate between OS-level commands in ^pButtons

      Category: Utilities
      Platforms: All
      Version: 2018.1.5

      A problem has been corrected where ^pButtons diagnostic reports could contain missing or incorrect performance data from OS-level commands (such as cstat, sar, iostat, etc.).

      AAR028: Check essential requirements during ^pButtons initialization and abort on failure

      Category: Utilities
      Platforms: All
      Version: 2018.1.5

      ^pButtons will abort, and report an error, if initialization finds that any essential requirements are not met. Currently these requirements are that the defined log directory is usable, and that the logged in user has enough privileges (Read/Write for the ^pButtons global).

      JO3131: Fix <UNDEFINED> from MISMATCH WIJ block viewer with Unicode strings

      Category: Utilities
      Platforms: All
      Version: 2018.1.4

      A problem where ^STURECOV could report an <UNDEFINED> error from REPAIR when displaying the blocks involved in resolving a "mismatched wij" trouble has been resolved.

      RFA030: Caché Buttons: List total and available disk space on all drives on Windows

      Category: Utilities
      Platforms: Windows
      Version: 2018.1.5

      Added a listing of total and available space for all disk drives on Windows platforms.

      Sample output:

      Volume      Size         Free Space     % Free
        C:      150.00 GB       55.35 GB       36.9
        D:      771.75 GB      429.10 GB       55.6
        E:       30.00 GB       21.80 GB       72.7
      

      RFD2050: Fix handling of remote databases for GLOBUFF

      Category: Utilities
      Platforms: All
      Version: 2018.1.3

      This change corrects handling of remote database in ^GLOBUFF utility.

      RJF324: Fix CheckList^Integrity returning an additional and spurious "Zwait" error if there were no globals found

      Category: Utilities
      Platforms: All
      Version: 2018.1.1

      If CheckList^Integrity found no globals to process (perhaps due to a global directory problem), the error status it returns could include a spurious "<Zwait>" error. This change corrects this problem.

      RJF383: Fix integrity check <NULL VALUE> and misleading type 8 errors

      Category: Utilities
      Platforms: All
      Version: 2018.1.4

      If while checking the chain of pointer blocks at a given level, a pointer block was fond to have an unexpected type (transiently or not), integrity check could generate misleading errors:

      • On the bottom pointer level, It could fail to record a usable error and instead generate <NULL VALUE>
      • On any pointer level, it could report this as a type 8 error "Pointer block is degraded and can't be parsed" (or possibly another misleading error type)

      These problems are corrected.

      RJF455: Fix SYS.Database.ReturnUnusedSpace() return value for large databases

      Category: Utilities
      Platforms: All
      Version: 2018.1.5

      When the "Return Unused Space" function is used for a database and the new size is greater than 4TB (regardless of block size), the new size could be reported incorrectly in the user interface or in the return value of the ReturnUnusedSpace() method of SYS.Database.

      SML2646: Fix errors when recompiling user's classes during upgrade

      Category: Utilities
      Platforms: All
      Version: 2018.1.1

      When upgrading a system, the upgrade could enounter problems recompiling user classes because the class compiler was not handling the dependencies correctly. This change corrects this problem and user classes will be recompiled correctly on upgrade.

      SML2647: Fix $SYSTEM.INetInfo.CheckAddressExist() in Windows if the Windows directory is not 'C:\Windows'

      Category: Utilities
      Platforms: Windows
      Version: 2018.1.1

      On Windows systems, $SYSTEM.INetInfo.CheckAddressExist() could fail with a nonstandard Windows directory. This change corrects this problem.

      SML2683: Fix process info not refreshed after viewing variables in JOBEXAM.

      Category: Utilities
      Platforms: All
      Version: 2018.1.3

      STC2631: Remove EMS Audit events

      Category: Utilities
      Platforms: All
      Version: 2018.1.3

      This change removes the two EMS audit events EMSChange and EMSError. These events were created for a feature that was never implemented. This change also increments the security tables version to 2018.1.

      STC2707: Add Audit entries for Job commands and Task Manager jobs

      Category: Utilities
      Platforms: All
      Version: 2018.1.3

      This change adds the following four new audit events have been added to the system:

      %Security/%Login/TaskStart - Audit processes started by the Task Manager
      %Security/%Login/TaskEnd
      %Security/%Login/JobStart - Audit processes started with the JOB command
      %Security/%Login/JobEnd
      

      VS Code

      DP-424221: Save and restore IO redirection state when calling the async Atelier endpoints

      Category: VS Code
      Platforms: All
      Version: 2018.1.9

      This change corrects an issue where I/O redirection code called for asynchronous endpoints did not save and restore the previous state, resulting in proper error messages when a class that has errors in it is compiled.

      Web

      DP-283006: Avoid SUBSCRIPT error when %RoutineMgr:GetOther called with no routine

      Category: Web
      Platforms: All
      Version: 2018.1.9

      Previously, calling the GetOther() method of the %RoutineMgr class with no routine name would throw a

      DP-6869: Do not apply cookie path matching for stream server requests

      Category: Web
      Platforms: All
      Version: 2018.1.9

      This change detects stream server requests and disables cookie path matching when serving streams by looking at the 'service' type of the request.

      Web Gateway

      DP-10980: Fix an out-of-bounds array access in cspInheritCacheWebFile()

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.0

      Previously, there was an issue in which certain calls attempted to access out-of-bounds data. This has been corrected.

      DP-11979: Web sockets with SharedConnection=1 can now time out

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.4

      In previous releases, a web socket connection with SharedConnection=1 would never time out. With this change, this connection will time out if it is inactive for the CSP session timeout.

      DP-12747: SSLCC_Protocol parameter changes for TLS 1.3

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.5

      To allow for TLS 1.3, there are changes to the way you access the SSLCC_Protocol parameter. If you are using an external script or the Web Gateway registry methods (particularly %CSP.Mgr.GatewayMgr.GetServerParams or %CSP.Mgr.GatewayMgr.SetServerParams) to read/write the SSLCC_Protocol parameter in CSP.ini, you must adapt your code to use the SSLCC_Protocol_Min and SSLCC_Protocol_Max parameters instead.

      DP-13035: Correct a regression in the handling of virtual hosts

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.5

      This change corrects a regression where the Web Gateway would sometimes route a request to an incorrect Apache virtual host. For example, if the Web Gateway configuration defined an application path //virtualhostname1/csp/ and another application path //virtualhostname2/csp/, then a request to //virtualhostname1/csp/ would sometimes be routed to the InterSystems IRIS server for //virtualhostname2/csp/.

      DP-13740: Ensure that Web Gateway SERVER connections are properly shut down

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.5

      This change resolves an issue that could cause server processes associated with a Web Gateway server connection to remain running after the associated web server worker has been shut down.

      DP-21243: In cases where we generate a new CSP session automatically make sure SecureSessionCookie session property is initialized

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.6

      When CSP gets a request with an invalid HMAC in the session token it automatically gets a new sessionId and construct a new session and continues with the request. This new session initialization logic when the original one was invalid did not initialize the SecureSessionCookie property correctly so this was set to 0 even if %request.Secure is true. This prevented SameSite=none session cookies from being sent to the client.

      DP-402068: Revamp management of thread life cycles

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.6

      Fix random crashes that would sometimes occur when connections to InterSystems IRIS were closed down or when web server worker processes were closed down.

      DP-403826: Fix memory leak when Web Gateway fails to log in to CSP server

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.6

      This change fixes a memory leak that occurs when the Web Gateway fails to log in to the CSP server, causing high memory usage if many login failures occur.

      DP-404812: Do not delete httpd.pid if ##class(Config.Startup).StopWebServer() can't stop the httpd process

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.6

      Fixes the handling of a case where Config.Startup:StopWebServer() would delete the httpd.pid file but fail to stop the Private Web Server process. This could occur if, for example, the instance user has the %All role, but the OS-level user who logged in does not have permission to stop the process. The process would then continue running until killed manually at the OS level, since the instance needs the httpd.pid file to find and kill the process during shutdown. Now, if the process fails to terminate, the httpd.pid file is left in place so that InterSystems IRIS can still find and clean up the old process later.

      DP-405020: Prevent configuring invalid CSP server names and application paths

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.6

      In the Server Access and Application Access configuration pages, it was possible to add a CSP server or application name containing special words or characters that would corrupt the Web Gateway configuration. For example, the name "system" is reserved by the Web Gateway, and creating a CSP server named "system" led to corruption of the configuration. The Web Gateway now validates CSP server and application names and rejects those containing reserved words or characters.

      DP-406096: FIX - Don't send file information when no file selected

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.0

      When attempting to upload a file, the Web Gateway previously treated the "no-file-selected" case the same as an empty file. This has been corrected.

      DP-410864: Prevent client error in delivering CSP/WebGateway cached content

      Category: Web Gateway
      Platforms: All
      Version: 2018.1.0

      Build nnn.1823

      The Web Gateway caches the first HTTP response returned for a given page or resource. If that response is gzip compressed and a separate client requests the same resource, the Web Gateway would return the compressed response even if the client doesn't accept gzip encoding.

      With this change, the Web Gateway now checks whether the client can accept the cached encoding, and if it can't, the Web Gateway retrieves a decompressed copy from the server. The gzipped format stays in the cache, and Web Gateway skips caching the decompressed copy. This conserves cache space and the performance gains from having gzip enabled. (If a gzipped response was in the cache, a majority of clients likely support that format.)

      It's possible for a non-gzipped response to end up in the cache first, and in that case the Web Gateway will still return that non-compressed response even for clients that accept gzip encoding. However, that at least doesn't violate the HTTP protocol; the client should always be able to accept non-encoded content.

      Web Services

      BES020: Make SOAP Client respect system-wide ProxyTunnel setting

      Category: Web Services
      Platforms: All
      Version: 2018.1.4

      Made change to %SOAP.WebClient.DoSOAPRequest() so SOAP client respects the system-wide ProxyTunnel setting.

      JGM757: Do not construct Basic authentication token in %Net.HttpRequest if Username is blank

      Category: Web Services
      Platforms: All
      Version: 2018.1.1

      When making %Net.HttpRequest calls it was possible for a FRAMESTACK error to be thrown if Basic authentication was required but no Username was specified. This is now corrected. Also prevented multiple sends if Basic scheme and username/password failed authentication.

      JGM836: Expose HTTP Request's WriteTimeout as a property of %SOAP.WebClient

      Category: Web Services
      Platforms: All
      Version: 2018.1.5

      The underlying HTTP request code in %Net.HttpRequest allows specifying a timeout value for writes to the web server.

      This change allows %SOAP.WebClient instances to set this property via a property named HttpWriteTimeout.

      The convention in the %SOAP.WebClient class is that properties that begin with Http are passed through to the %Net.HttpRequest instance used by the class.

      If HttpWriteTimeout is not set the value from the HttpRequest object (%Net.HttpRequest) WriteTimeout property will be used which has a default of -1.

      -1 means it will wait for an unlimited time for the remote server to accept the written data.

      Change it to another value to specify the timeout in seconds.

      The minimum value accepted is 2 seconds.

      MXT2243: Allow entity resolver and sax flags to be specified for WSDL reader and XML schema reader

      Category: Web Services
      Platforms: All
      Version: 2018.1.3

      Two new properties are added to both %SOAP.WSDL.Reader and %XML.Utils.SchemaReader to allow additional control of the SAX parser calls:

      /// This property should be set to a combination of flags (see %occSAX.inc for details) if
      /// the default behavior of the parser is required to be modified.
      /// The $$$SAXVALIDATIONPROHIBITDTDS flag is added by default in order to not allow DTDs.
      Property SAXFlags As %Integer [ InitialExpression = {$$$SAXFULLDEFAULT+$$$SAXVALIDATIONPROHIBITDTDS} ];
      
      /// This property should be set to an instance of %XML.SAX.EntityResolver OR a user-defined
      /// subclass IF the default EntityResolver is not required.
      Property EntityResolver As %XML.SAX.EntityResolver;
      

      SAM566: Correct URL Escaping in SOAP Server-side Redirect

      Category: Web Services
      Platforms: All
      Version: 2018.1.5

      This fix properly accounts for the situation where a classname containing a percent sign coincidentally followed by two valid hex digits was (formerly) being misinterpreted as an ASCII character code.

      Web Services.WS-Security

      SAM575: Relax c14n encoding restrictions on SOAP input

      Category: Web Services.WS-Security
      Platforms: All
      Version: 2018.1.5

      The c14n Canonicalization standard supports variations that either strip comments, or require them to be formatted according to certain whitespace expectations. Previously our processing of SOAP signature would summarily reject packets that claimed the use of the #WithComments extension. This change relaxes that restriction, allowing packets to claim that they are using the #WithComments variation on the base algorithm and to treat processing of that form as if it were the base algorithm (compliance with canonical form rules is not verified). Outgoing SOAP packets are unchanged, we only support the base algorithm, where all comment tags are stripped from the XML document prior to encryption and signing.

      SAM580: Correct mixed character encoding issue in encrypted SOAP packets

      Category: Web Services.WS-Security
      Platforms: All
      Version: 2018.1.4

      This change corrects an issue where higher order Unicode characters nested within partially encrypted or signed SOAP requests could result SAX parsing errors during decryption and processing.

      XML

      JN1751: Correct Escaping of Query Parameters in SAX Parser ParseURL()

      Category: XML
      Platforms: All
      Version: 2018.1.3

      The SAX parser's ParseURL method accepts URL encoded or plain URLs as a convenience for our users. This change corrects a problem where certain characters such as & and = were causing the query part of the URL to be corrupted.

      Zen

      SAM512: Add looping structure to calls to ZU(177,3) in EndChangeTracking of Zen Controller

      Category: Zen
      Platforms: All
      Version: 2018.1.3

      This change corrects an issue where change tracking for page updates could overflow if more than 8191 (2 to the 13th minus 1) Orefs are touched between refresh cycles.

      SAM530: Correct tablePane confusion between doubleClick and toggle behavior

      Category: Zen
      Platforms: All
      Version: 2018.1.3

      This change corrects a problem where, depending on the speed of clicking multiple times on the same row of a table (when in certain operating modes) caused the table to confuse double-click behavior with toggle selection behavior, incorrectly leaving visual artifacts on screen as a result of aborted toggling attempts.

      SAM557: Create EDGE-specific override to default layout tables in %ZEN.LayoutManager

      Category: Zen
      Platforms: All
      Version: 2018.1.3

      This change adds special handling for default vertical layouts under Microsoft EDGE browser to address its current, non-standard rendering behavior.

      Zen Reports

      JSL5237: Enhancement: ZR: allow adjusting page format

      Category: Zen Reports
      Platforms: All
      Version: 2018.1.1

      The report and section elements of a ZEN Report have a new property, pageNumberFormat.

      JSL5242: Fix string appearing at end of HTML reports

      Category: Zen Reports
      Platforms: All
      Version: 2018.1.1

      Under certain conditions an incorrect text would be appended to the html output. This change corrects this problem.

      For Additional Help

      If you need assistance with evaluating how upgrading to this maintenance release will affect your applications, systems, or related plans, please contact the InterSystems Worldwide Support Center:

      • Phone:  +1.617.621.0700
      • Fax:  +1.617.734.9391
      • Email:  support@intersystems.com

      Current release notes (and complete product documentation) can be found online at https://docs.intersystems.com.

      FeedbackOpens in a new tab