InterSystems IRIS for Health™ Upgrade Checklist and Change Notes (2021.1.2)

DP-402897: Fix warning exporting widgets to Excel by exporting pivot table as CSV

Category: Business Intelligence
Platforms: All
Version: 2021.1.1

This change fixes a problem where exporting pivot tables from Analyzer/Pivot Widgets as XLS and then opening in Excel, they throw a security warning.

To avoid this warning and improve integration with other data visualization tools, XLS export is replaced with CSV export.

DP-404426: Fixes inherited cubes that do not show sourceClass list in Architect

Category: Business Intelligence
Platforms: All
Version: 2021.1.1

If a cube (abstract or not) using ^DeepSee.Overrides for setting a source class with the explicit setting sourceClass="" is then inherited, the Architect was showing the inherited content of the cube but was not able to populate the source class tree on the left pane.

This change makes the sourceClass override aply during any cube model load wthin the Architect if the sourceClass is null.

DP-404982: Solves a problem where %FixBuildErrors can report that errors are fixed, when they are not

Category: Business Intelligence
Platforms: All
Version: 2021.1.1

When a cube has an OnProcessFact method, %DeepSee.Utils:%FixBuildErrors can handle reference errors improperly. It reports success and deletes ^DeepSee.BuildErrors whether or not the record is fixed. This change fixes the problem.

DP-405378: Correct subquery queryKey bookkeeping in compound cubes

Category: Business Intelligence
Platforms: All
Version: 2021.1.1

This change corrects a problem where a crossjoin containing an ordering function in the second (inner) term might lose track of the correct order of compound cube results.

DP-405956: Fixes errors in Mini-Analyzer and Analyzer drag and drop and enabling filters

Category: Business Intelligence
Platforms: All
Version: 2021.1.1

In some cases, dragging and dropping in the Mini-Analyzer caused JavaScript errors and in other cases enabling and disabling filters would not work in the Analyzer and Mini-Analyzer. This change corrects those problems.

DP-406477: Fix problem where compile fails if parent cube has no named FactNums attribute

Category: Business Intelligence
Platforms: All
Version: 2021.1.1

In versions with selective build enabled by default, cubes created in Architect will normally have the namedFactNums attribute set to true (and factNumber attributes with default values for appropriate model elements. However, cubes imported from an older version and compiled in Studio or Terminal may not have any such attributes (and may compile and build successfully despite this).

This leads to a problem when child cubes are newly created in Architect on version with selective build, and inherit from parent cubes created on an older version. These child cubes have namedFactNums = "true" by default*, but as the parent cube's model elements have no factNumber, the child cube will fail to compile.

It's possible to work around this either by manually resolving the discrepancy in Studio or by opening the parent cube in Architect and saving it there, which enables selective build for the cube and adds namedFactNums and factNumber to the cube class.

This change fixes the problem.

DP-407899: Correct handling of !NO MEMBER in MDX filters

Category: Business Intelligence
Platforms: All
Version: 2021.1.1

In version 2021.1, MDX queries return no results if they include a filter clause that excludes (using %NOT) a member that does not exist in the level being filtered on. For example, the following query returns no results when there is no &[] member in the [HOMED].[H1].[ZIP] level:

SELECT NON EMPTY [HomeD].[H1].[ZIP].Members ON 1 FROM [PATIENTS] %FILTER [HOMED].[H1].[ZIP].&[<null>].%NOT

SELECT NON EMPTY [HomeD].[H1].[ZIP].Members ON 1 FROM [PATIENTS] %FILTER [HOMED].[H1].[ZIP].&[32006].%NOT

It is more common to filter on members that do exist, of course, but filtering on members that do not exist should work correctly, as the set of members in a level can change over time and should be able to do so without breaking existing pivots. In particular, a customer may well wish to exclude null values without knowing whether their data will contain any at a given time.

The problem occurs in version 2021.1. It does not occur in version 2019.1.0 or in Ensemble 2018.1.2.

DP-405632: Update host configuration for Ubuntu 20.04

Category: Cloud
Platforms: UNIX®
Version: 2021.1.1

Changes between Ubuntu 18.04 and 20.04 may potentially cause network issues. This change deals with this two of these changes:

1. On some configurations, "iptables" and "iptables-save" have moved from /sbin to /usr/sbin.

   This may have been causing issues reaching the outside network; we now check both locations.

2. resolv.conf now includes option "trust-ad" by default.

   This may have been causing Weave Net to crash (see [https://github.com/weaveworks/weave/issues/3868]); we now remove this option.

DP-402030: Fix CMQL queries fail on first execute after conversion to IRIS

Category: Compatibility Features
Platforms: All
Version: 2021.1.1

Fixes a problem where if a CMQL query has been executed on a Caché instance, and then the instance is converted to InterSystems IRIS, the query will fail on the first execute on the converted instance, but will run afterwards.

DP-402536: Fix an error where Caché Migration Feature stream properties would be set to null

Category: Compatibility Features
Platforms: All
Version: 2021.1.1

This change fixes a problem that would cause stream properties to be set to null when the value is assigned using the IRISActiveX library. The problem only affects properties using the Caché Migration Feature stream types (such as %Library.GlobalCharacterStream) and only when the property is explicitly set.

DP-403621: Support NodeJS version 14

Category: Compatibility Features
Platforms: All
Version: 2021.1.1

This change adds the headers and build scripts to support building the legacy node.js binding for node version 14.x.x.

DP-403622: SDK205Support NodeJS version 16

Category: Compatibility Features
Platforms: All
Version: 2021.1.1

This change adds the headers and build scripts to support building the legacy node.js binding for node version 16.x.x.

DP-404220: Fix old Node.js support methods without a return value

Category: Compatibility Features
Platforms: All
Version: 2021.1.1

This change updates the old NodeJS binding to be able to call ObjectScript methods that do not return a value. Previously, attempting to call such a method would result in a <COMMAND> error when run via the API connectivity, or a <SYNTAX> error when run using TCP connectivity.

Now invoking a method that does not return a value will result in undefined being returned to the javascript code.

Note: If a client without this change connects to a server with this change using TCP, any method call that should return undefined will actually return an empty string (''). If you have a newer client connecting to an older server, then you will still get a <COMMAND> error (that is the same behavior as before this change.)

DP-404837: Prevent crashes in legacy Node.js template interface

Category: Compatibility Features
Platforms: All
Version: 2021.1.1

This change corrects defects in the legacy Node.js binding that could cause crashes when using the class template interface.

This change corrects two problems.

1. When using the class template interface to invoke a method or a class method that returns an object value, and the first argument to the method was longer than 256 bytes, there would be a buffer overflow that could cause memory corruption and potentially a crash. This problem has been corrected.

2. When caling IrisClass() to generate a class template, if the connection is using TCP connectivity the class metadata retrieved from the server via GetClass^%SYS.nodejsServer) could be corrupted, which could result in a crash.

DP-402100: %CSP.WebSocket:OpenServer() should quit 0 for a non-existent WebSocketID

Category: CSP Server
Platforms: All
Version: 2021.1.1

This change updates the %CSP.WebSocket:OpenServer() method to return an error status if no data is found for a given WebSocket ID. Previously, this method would return $$$OK even though it failed to find and open the web socket.

DP-404562: Adds unique error codes to %CSP.WebSocket

Category: CSP Server
Platforms: All
Version: 2021.1.1

This change replaces generic error usage in the %CSP.WebSocket class with new error codes. WebSocket behavior is unchanged otherwise.

DP-406217: Allow CSP pages to specify exact redirect URL

Category: CSP Server
Platforms: All
Version: 2021.1.1

Typically when performing a browser-side redirect using %response.Redirect, the target URL will be transformed to an absolute URL before being sent to the client. The system may also add or modify the CSPToken, CSPCHD, and other URL parameters if the target URL is a CSP/ZEN page

This change introduces a new property: %response.UseExactRedirect. If this is set to 1, then the server will not perform these transformations; the URL specified in %response.Redirect will be used exactly as specified by the user. This only affects the current request, and does not affect redirects using %response.ServerSideRedirect. The default is 0.

Note that RFC 2616 initially defined the Location header to require an absolute URI (see section 14.30), but this was later relaxed by RFC 7231 (see Section 7.1.2, and Appendix B). Setting %response.UseExactRedirect=1 will allow a CSP Application to redirect to a relative URL. This generally should not be relevant because by default we will translate the URL to the equivalent absolute URL, but may be helpful in some unusual cases.

DP-408957: Ignore cache clearing errors during CSP compile

Category: CSP Server
Platforms: All
Version: 2021.1.1

This change corrects the following error that occurs sometimes when compiling a CSP page in Studio.

Compilation started on 10/19/2021 16:36:55 with qualifiers 'k'
Compiling file /csp/user/menu.csp
Compiling class csp.menu
Compiling routine csp.menu.1
Detected 2 errors during compilation in 0.046s.
ERROR #7913: Unable to complete client request
  > ERROR #6084: Unknown errors detected, but no error code reported

DP-405725: Zen Reports do not output duplicate ID for table elements

Category: Data Reporting
Platforms: All
Version: 2021.1.1

This change corrects a bug that could result in an error when including certain display components directly in a table, if the component defines an ID attribute. In these cases Zen Reports would produce XSLFO with elements with duplicate ID's, which would result in an error from FOP like:

SEVERE: Exception
org.apache.fop.apps.FOPException: org.apache.fop.fo.ValidationException: Property ID "line" (found on "fo:table") previously used; ID values must be unique within a document! (See position 1861:-1)

DP-407946: Execute queries using %ExecuteAsynch when preparing for printing/exporting

Category: Data Reporting
Platforms: All
Version: 2021.1.1

When exporting pivotTable results, the %DeepSee.UI.MDXExcel and %DeepSee.UI.MDXPDF pages now use the multithreaded %ExecuteAsynch by default to execute the queries in the background. This can be suppressed with a URL parameter ASYNC=0 when opening the page, which will default to the prior behavior. Omission of this parameter will produce the new default behavior.

DP-402388: Fixes problem where Business Operation cannot connect to Java Gateway

Category: Gateways - Java
Platforms:
Version: 2021.1.1

EnsLib.JavaGateway.Service::ConnectGateway is deprecated but unfortunately, in the process, the pDebug As %Boolean 3rd parameter was removed This change restores the old method signature to EnsLib.JavaGateway.Service::ConnectGateway() in order to fix the backward incompatibility.

DP-407804: Correct Java ClassPath when directory contains spaces

Category: Gateways - Java
Platforms: All
Version: 2021.1.1

This changes fixes a condition where the Java Gateway could fail to start if there are spaces in the classpath directory.

DP-407653: Correct IRISConnect.isValid handling of read timeouts

Category: Gateways - JDBC
Platforms: All
Version: 2021.1.1

IRISConnect.isValid(readTimeout) is a public API that returns true if the connection is active, and false if it is disconnected. The API sets the read timeout for the socket to a duration in seconds and sends a ping message for the server to respond to.

The problem with the code is that if the read does timeout, it does not mean the connection is dead, just that it hasn't responded, and the exception code should reset the read timeout of the socket back to the jdbc default value (either network timeout that was set or infinite). Another problem in this case is that there is a pending request for query results that has been requested from the server, but has not been read. For example, in one case the query was artificially delayed by 20 seconds, and the read timeout is resetting how long we wait for a query timeout (which is controlled on the server side, not the client), and the client gives up with a read timeout consuming the response to the query (even before the ping is attempted).

This change adds a check to return true if we are processing a query read-ahead, otherwise to ping the connection with the timeout requested in isValid().

DP-408799: Correct JDBC gateway query code generation to close cursor

Category: Gateways - JDBC
Platforms: All
Version: 2021.1.1

A customer reported a problem where they have a class query of type %SQLQuery projected as a procedure.  The SQL query in this class query is against a linked table, through the JDBC gateway.  They are calling the cached query using:

   set rs = ##class(%ResultSet).%New()

When the UCQ oref was destructed, the SQL Close call was not made because i%CursorState was not defined, so the process did not realize an SQL cursor statement had been executed.  When the SQL Close did not occur, _%JDBCGateway.removeStatement(i%hstmt)_ was not called, which means the statement was not removed from the Java process and memory in the Java process would continue to build up.   This change allows the UCQ's %Destruct method to close the SQL cursor.

DP-404515: Use correct data length for SQLGetData SQL_C_WCHAR bufsize 0

Category: Gateways - ODBC
Platforms: All
Version: 2021.1.1

Fixes a bug where calling SQLGetData twice, the first time with buffer length 0, caused the second call to return an incorrect data length for wide-char (SQL_C_WCHAR) queries.

DP-404634: Allows users to drill down into OREF's when debugging in VSCode

Category: IDEs
Platforms: All
Version: 2021.1.1

See https://github.com/intersystems-community/vscode-objectscript/issues/272.

DP-405497: Fixes input Japanese being garbled when running method with parameters in debug launch mode

Category: IDEs
Platforms: All
Version: 2021.1.1

See https://github.com/intersystems-community/vscode-objectscript/issues/570.

DP-405539: Fixes problem where launch of class method not accepting path with Windows drive

Category: IDEs
Platforms: Windows
Version: 2021.1.1

See https://github.com/intersystems-community/vscode-objectscript/issues/654.

DP-405736: Fix Uncomment Line off-by-one error in Studio

Category: IDEs
Platforms: All
Version: 2021.1.1

This change fixes a bug wherein Studio's Edit > Advanced > Uncomment Line and Uncomment Block commands would fail to uncomment the last selected line.

DP-407521: Fix Studio so that it follows HTTPS configuration setting

Category: IDEs
Platforms: All
Version: 2021.1.1

SDK112 (DPP-388) updated Studio to use the web server configuration defined in the client registry, when applicable. However, it did not check whether HTTPS was enabled but instead always used "http" in these cases. This could prevent features such as Studio's Tools > Add-Ins > SOAP Wizard from connecting to the server, if HTTPS was required and the client registry was in use. With this devchange, Studio now selects "https" or "http" for the URL based on the client's HTTPS setting.

DP-409553: Fix document auto-import when using %Studio.SourceControl.ISC

Category: IDEs
Platforms: All
Version: 2021.1.1

This change addresses two vscode-objectscript GitHub issues related to the source control hooks' auto-import feature.

Open a document under server side source control in VS Code. Edit this document which asks to checkout the document, select 'yes' and save/compile it. Under a separate source control tool you can see the file associated with this document is now checked out and modified. Revert this file in the separate source control tool.

In VS Code there is no indication the file on disk associated with this document was changed. In Studio the timestamp check it performs on a regular basis and when the document gets focus will trigger a source control call that detects the associated file is modified and if the document is not modified in Studio will cause the document to be reloaded from the file automatically and the view inside Studio will be refreshed. Additionally if the file in Studio was modified you will be prompted if you wish to revert your edits back to the version of the file on disk.

This change corrects the behavior of VS Code to recognize that the file is no long writeable.

DP-412668: New ISCAgent standalone installer

Category: Installation
Platforms: Windows
Version: 2021.1.1

This change provides a new ISCAgent installer for Windows. Functionality will be the same as current ISCAgent installer except for the following changes:

• It will accept normal Windows Installer command line options for logging, unattended and silent install, similar to what InterSystems IRIS installer is using. For example, to run silent install one would use command "ISCAgent_<version>.exe /qn".
• It will install VC++ redistributables in both full UI and unattended install.
• Dialogs will have the same look as InterSystems IRIS installer dialogs.

DP-410849: Gateways section for Ubuntu disables usesharedmem

Category: IntegratedML
Platforms: Ubuntu
Version: 2021.1.1

For initial installations, the usesharedmemory CPF flag is disabled for Ubuntu 20.04, which has a shared memory problem. If the system is an upgrade, no change is made to this setting.

DP-402935: Corrects framed TCP adapter not functioning with single end framing character

Category: Interoperability
Platforms: All
Version: 2021.1.1

The inability to process 'framed' messages terminated using a single terminating character by the EnsLib.TCP.Framed.PassthroughService and EnsLib.TCP.Framed.PassthroughOperation, which use EnsLib.TCP.FramedCommon via EnsLib.TCP.FramedInboundAdapter and  EnsLib.TCP.FramedOutboundAdapter has been corrected.

DP-406031: Fix Java Gateway hang

Category: Interoperability
Platforms: All
Version: 2021.1.1

This change fixes a Java Gateway hang by normalizing the Y0 response message structure

Y0 response message structure before this change (protocol<=61):

• piece #1: server protocol number

• piece #2: server protocol number

• piece #3: boolean flag indicates if server supports shared memory or not. Only present if client has requested shared memory.

Y0 response message structure with this change (protocol>=63):

• piece #1: server protocol number

• piece #2: server protocol number

• piece #3: boolean flag indicates if server supports shared memory or not. This piece is always present even if client didn't request shared memory.

• piece #4: External Language Server name under which the running server is defined.

DP-410350: Avoid Possible loss of one attribute on setting another attribute in XMLVDoc

Category: Interoperability
Platforms: All
Version: 2021.1.1

In an XML VDoc element, if 10 or more attributes had been set, and one attribute's value was then set to another value, then, on output, one of the other attributes would not be included.  This change fixes that problem so that no attributes are lost when another one is set, even if there are a large number of attributes.

DP-407658: Pass timeout as argument to socket.connect()

Category: JDBC
Platforms: All
Version: 2021.1.1

socket.connect(SocketAddress) does not use the timeout set inside the socket object, but in turn calls socket.connect(SocketAddress, 0) with an infinite timeout. This change passes the timeout as an argument to connect(SA, timeout).

DP-402510: Fix problem where background debugging loops when reading from the null device

Category: Kernel
Platforms: UNIX®
Version: 2021.1.1

If a background process is changed to foreground with ^%PMODE and attempts to read from its principal device which is the default null device, the process could loop. This change fixes this problem that was discovered developing a debugging utility.

DP-403205: Fix two-argument $GET() of array that returns wrong value if Undefined(2) is set

Category: Kernel
Platforms: All
Version: 2021.1.1

If a process has set $SYSTEM.Process.Undefined(2), a two-argument $GET() on VAR(SUB) where SUB is undefined would return a null string instead of the expected second argument of $GET(). This change fixes the problem.

DP-403473: Do not use Windows short path names

Category: Kernel
Platforms: Windows
Version: 2021.1.1

The %File method NormalizeFilenameWithSpaces() is used to apply quotes to a file name that contains spaces, so it can be passed as an argument to a shell command. In some cases on Windows, it would convert the name to a Windows short file name instead of adding quotes. This did not always result in a valid file reference. The Windows short file names are no longer necessary, so the method now always returns the original name and adds quotes if it contains spaces.

DP-403566: Fix compilation error that could cause core dumps

Category: Kernel
Platforms: All
Version: 2021.1.2

Change DP-403184, in fixing a compilation error, introduced a different compilation error which could cause mismatches between compiled and interpreted expressions with Undefined = 2 and could cause core dumps. This change fixes the new error.

DP-403623: Fixes <BREAK> error and inconsistent behavior, sometimes resulting in a core

Category: Kernel
Platforms: All
Version: 2021.1.1

If there is an uncaught <COMMAND> error because $XECUTE() quits without returning a value, a GOTO command at the error prompt had unpredictable results. This change corrects the problem.

DP-404235: Restrict users permitted to perform clone and merge operations

Category: Kernel
Platforms: All
Version: 2021.1.1

Two issues were identified with the clone and merge operations:

1) Clone and merge can be invoked by the IRIS stop/start user. We now protect against this by verifying that the current userID corresponds to root or the IRIS owner

2) Clone and merge were being run as root. These operations are now being run with the current userID and groupID

While addressing these issues, a third was found:

3) Values for userID and groupID were being overwritten during a restart. Assignment of these variables now occurs only once, in code common to both start and restart.

DP-404683: Allows messages greater than 32KB to be passed through $system.Event.Signal/WaitMsg

Category: Kernel
Platforms: All
Version: 2021.1.1

The 2 argument version of $system.Event.Signal() and the corresponding $system.Event.WaitMsg() were previously limited to 16K Unicode characters or 32K 8-bit characters. This would result in a <STORE> error and caused problems with the debugger when trying to inspect objects that exceeded this.

This limit has been removed and it is now possible to pass strings as large as InterSystems IRIS will allow (3.8M characters currently). Any values that exceed 32KB are now allocated on the Shared Memory Heap (SMH); note that passing large values this way can increase the amount of overall GMHEAP that needs to be allocated.

Usage Notes

Obviously, heavy use of $system.Event.Signal() / $system.Event.WaitMsg() with large message bodies could deplete the available SMH and cause <STORE> errors. InterSystems recommends that any application that expects to have any more than a few large messages "outstanding" (that is a queue depth greater than 1) should rather use another medium for storing the message bodies such as globals -- Productions do not use the 2-argument version of $system.Event.Signal() and thus won't be susceptible to any additional use of the SMH.

However, for the debugger, although it can pass large messages, it works in a request/response way. There typically will not be more than one potentially large message pending (per session).

DP-405614: Scan ALTPID for speed and scalability of JOBs, sessions, and threads entering instance

Category: Kernel
Platforms: All
Version: 2021.1.1

The time to enter InterSystems IRIS is improved, particularly for very large systems with many thousands of active jobs.  This applies to the JOB command as well as normal session startup, or callin threads.  This also reduces contention between multiple jobs entering at the same time so the total system-wide throughput of the JOB command or session startup is increased.

DP-406339: Fixes Java Gateway fail caused by edge-case problem in $zobjexport(16)

Category: Kernel
Platforms: All
Version: 2021.1.1

When $zobjexport(16) had a byte array of exactly 32727 bytes, it would encounter an error that could cause the Java Gateway to fail. This change corrects the problem.

DP-406491: Make VariableList Query re-entrant

Category: Kernel
Platforms: All
Version: 2021.1.1

The underlying code that implemented the VariableList query of %SYS.ProcessQuery was not re-entrant. If the query was interrupted by a JOBEXAM from another process the original query would not be complete. It is now safe and will give correct results if interrupted.

A new method is added to %SYS.ProcessQuery to get the variable list directly without running a query.

Set numvars=##class(%SYS.ProcessQuery).GetVariableList(InternalStack,.varArray)

numvars is the total number of variables returned.
varArray is the number of nodes (subscripts) returned.
varArray(1) is the first node, varArray(2) is the second, and so on.

Each node is:
$ListBuild($ListBuild(varname,$Data(varname)), $ListBuild(varname,$Data(varname)), ...)

The names are not sorted, except private variables follow public variables. Private variable names are prefixed with a tilde character (~). Because only defined variables are listed, the $Data value is always non-zero.

DP-406788: Fix problem where two application servers held exclusive lock during ECP recovery

Category: Kernel
Platforms: All
Version: 2021.1.1

Fix missing locks to be uploaded to data server during ECP recovery. This problem started since version 2020.1 which expanded network buffer for lock requests.

DP-406859: Fix tcpaccept() method to handle timout correctly on Windows platform

Category: Kernel
Platforms: Windows
Version: 2021.1.1

Under some circumstances, netcp tcpaccept() method was not handling the time interval correctly and was waiting longer than specified. This change corrects the problem.

DP-407260: Stop logging messages for normal operation.

Category: Kernel
Platforms: All
Version: 2021.1.1

This change eliminated logging some messages that indicated the system was operating correctly.

DP-408034: Allow range ABC.* to work in DataMove

Category: Kernel
Platforms: All
Version: 2021.1.1

This change allows range such as ABC.* to work in DataMove

DP-408829: Integrity check must not read global blocks into global buffers

Category: Kernel
Platforms: All
Version: 2021.1.1

Prior to this correction, under extremely rare circumstances, integrity check run on an active system could cause a global reference or update in another process running concurrently to get unexpected results. This problem has existed since at least Caché 2011.1, though it's never been reported in the field.

Note that a side effect of this correction is that integrity check no longer loads pointer blocks into the database cache as it checks them, whereas it generally did in the past (but almost never loaded data blocks into the buffer pool).  Therefore integrity check may do a few percent more reads, and may have somewhat less of an impact on other jobs running on the system.

DP-412701: Update OpenSSL source codes from version 1.0.2zb to 1.0.2zd.

Category: Kernel
Platforms: All
Version: 2021.1.2

Update OpenSSL source code from version 1.0.2zb to 1.0.2zd. In addition to being used in the kernel, the OpenSSL library is used in the Apache-based private web server for any https connections, in the ODBC client for TLS-based connectivity, and in the PKI service.

DP-403498: Add license DisplayName for international characters

Category: Licensing
Platforms: Windows
Version: 2021.1.1

Use DisplayName from [License] section, if it exists, in place of Customer Name for displays of license key. This will be encoded in the file as UTF-8, and then converted to IRIS strings where necessary. The DisplayName will be used for all Portal licensing pages, and will appear in command line startup where we print "licensed for use exclusively by". For command line startup we use the original UTF8 string, so the terminal must be able to display this properly.

genkey is modified to accept an optional /DisplayName argument (similar to /LicenseID).

Note that DisplayName will be limited to 128 bytes (not characters). Also, even though the DisplayName is in the [License] section it is NOT protected by the checksum in the key. This is to maintain compatibility of keys with previous versions (both IRIS kernel and License Manager). However, customers should not modify this string, as it will make IRIS think it's a different key, even though the significant aspects of the key are the same.

Removed MachineID from Portal License Key page as part of this change, as it hasn't really been used in a long time.

DP-405634: Fix handling of DisplayName in license comparison

Category: Licensing
Platforms: All
Version: 2021.1.1

Even with a properly loaded and activated license, a warning message was displayed in the License Key portal page: "The license key in MGR will not take effect until it is activated or the system is restarted." The change fixes the problem.

DP-405405: Correct %Save when parent ID is reference to multi-property ID

Category: Object Compiler
Platforms: All
Version: 2021.1.1

A problem has been corrected when a class has a parent-child relationship and the parent class has an IDKey index on a property that is a reference to a persistent class and the referenced class has an IDKey index on two or more properties.  

This change corrects issues with %SaveData, %ComposeOid, %DeleteData, %ExistsId, %PhysicalAddress, %LockId, and %OnDetermineClass.

DP-406105: Prevent UNDEFINED error due to conflict with UCQ and re-entry class compile

Category: Object Compiler
Platforms: All
Version: 2021.1.1

Using the %Studio.Project:InstallFromGbl function it was possible to get an UNDEFINED error looking for qstruct("deleteextent") if the class compiler was called due to UCQ while installing a package.

DP-403980: Remove IRIS.TempNodes<$job> global when process exits

Category: Object Library
Platforms: All
Version: 2021.1.1

Remove entries for globals like IRIS.TempNodes<$job> when a process exits so these do not clutter lists of globals such as in %GD.

DP-405062: Reduce number of WQM workers we start until they are needed

Category: Object Library
Platforms: All
Version: 2021.1.1

Reduce the number of worker jobs we create in a WQM category so they are only started when needed rather than starting the 'MaxActiveWorkers' number immediately on the first use of a work queue within this category.

DP-406986: Increase partition memory used by WQM daemon process to the same as WQM worker jobs

Category: Object Library
Platforms: All
Version: 2021.1.1

Increase partition memory used by WQM daemon process to the same as WQM worker jobs. Previously it was using the system default partition limit but all WQM worker/daemon jobs should be consistent in their partition memory limit.

DP-407307: Reduce CPU use of WQM daemon process and allow it to go idle more quickly

Category: Object Library
Platforms: All
Version: 2021.1.1

The WQM daemon process which manages the worker jobs was consuming too much CPU on systems with very large process tables. This has been corrected.

Also the WQM daemon will poll the workers every 0.1 second when the WQM is active but when inactive it will slow down to a 1 second poll rate. This transition to inactive was 2 minutes from the last time it was active. This has been changed to 1 minute from the last active time. It can also become inactive in less than 1 minute if there are no WQM groups in existence.

DP-407332: The %SYS.Task.ResourceCleanup task should run all mirror members and not just the primary

Category: Object Library
Platforms: All
Version: 2021.1.1

The ResourceCleanup task which is run by the task manager was set to run only on mirror primary systems but there are resources that need cleaning on none-primary systems so we have updated this to run on all systems.

DP-407848: Allow removing a class when ^oddDEF does not exist

Category: Object Library
Platforms: All
Version: 2021.1.1

Support deleting a class when internal components have got out of sync.

DP-408482: Fix "minExclusive" and "maxExclusive" that were not correctly created from xsd

Category: Object Library
Platforms: All
Version: 2021.1.1

If a schema has exclusive maxes and mins, then %XML.Util.SchemaReader converts these to inclusive MAXVAL and MINVAL fields. However, this conversion assumed whole-number precision, which is incorrect if fractionDigits is specified. With this change, the SchemaReader checks for fractionDigits and computes maxes and mins that match that degree of precision. If fractionDigits is unspecified, it uses whole-number precision.

DP-408594: Correct cookie sharing across subdomains in %Net.HttpRequest

Category: Object Library
Platforms: All
Version: 2021.1.1

The cookie sharing code between subdomains in %Net.HttpRequest was not compliant with RFC6265. Specifically when doing to sub.mydomain.com if a cookie is set by the server with a 'domain=mydomain.com' this cookie was not being sent to the server when a subsequent request was made to my domain.com.

Also refreshed the list of root public domains from publicsuffix.org as the data was out of date.

DP-409587: When calling OutputToDevice on a global stream with a specific length set 'AtEnd' property correctly

Category: Object Library
Platforms: All
Version: 2021.1.1

When using %Stream.GlobalBinary, %Stream.GlobalCharacter, %Stream.TmpCharacter, %Stream.TmpBinary and calling 'OutputToDevice' passing in a specific length to output if the length exactly matches the size of the stream we were not setting the AtEnd property to true so a loop such as:

While 'stream.AtEnd {
  Set sc=stream.OutputToDevice(100)
}

Could end up looping forever if the stream was exactly 100 characters long.

DP-404353: Correct %OnAddToSaveSet callcount parameter callback on %Save of persistent object

Category: Object Storage
Platforms: All
Version: 2021.1.1

After a previous change the %OnAddToSaveSet 'callcount' parameter which should report the number of times this callback was called for this class was always being passed in as '1' even for cases where the callback was being called multiple times. If the user was using this parameter to decide if we should call %AddToSaveSet on the current instance this failure could result in <FRAMESTACK> errors.

DP-403184: Fixes cores and errors when compiler optimizes expressions

Category: ObjectScript
Platforms: All
Version: 2021.1.1

The ObjectScript compiler can optimize expressions like $SELECT(1:23) to the constant 23. But in some situations that optimization could result in incorrect object code leading to access violations or errors. This change corrects the problem.

DP-403374: Fixes complex expression that could generate bad code (#10)

Category: ObjectScript
Platforms: All
Version: 2021.1.1

In the ObjectScript code $DATA(^global)=expr where expr is a complex expression that the compiler knows will evaluate to zero, it may generate invalid object code that will result in an access violation. This change corrects the problem.

DP-403666: Fix legacy NodeJS to properly handle exceptions thrown in user code

Category: ObjectScript
Platforms: All
Version: 2021.1.1

This change ensures that the legacy Node.js binding will properly handle exceptions thrown by user code. Previously, exceptions that were not of the type %Exception.SystemException would result in inconsistent behavior, including a '' response, a <MAXSTRING> error, or a crash. 

Currently the server reports errors by sending the numerical code associated with the ObjectScript error to the client as a single byte. The client client has logic to reconstruct the error string, which is then reported to the application. 

Now, if user code throws any exception other than a SystemException, the client will report a <THROW> error, with the associated message "The application encountered an unexpected exception.". This change also ensures that the server will not return an error code greater than 255. A SystemException should not use a value this high, but if one is encountered, the server will send the error code 255 to the client, which will be interpreted as a <SYNTAX> error. 

Note: This change only affects connections using TCP. Connections using the callin API do not report unhandled exceptions correctly. Applications using the callin interface should not directly invoke code that could raise an exception.

DP-404003: Fix <SYSTEM> error after compiling complex expression

Category: ObjectScript
Platforms: All
Version: 2021.1.1

When a complex boolean expression containing $Select() is optimized by the routine compiler, invalid object code could result in a <SYSTEM> error. This change corrects the problem.

DP-404209: Fixes Equals optimization that could be wrong

Category: ObjectScript
Platforms: All
Version: 2021.1.1

When the ObjectScript compiler optimizes a complex Equals comparison involving $Select() in which it knows what the result of each side will be, it could produce an incorrect result. This change corrects the problem.

DP-404552: Fix compiler optimization of $Extract() with $Select()

Category: ObjectScript
Platforms: All
Version: 2021.1.1

If the third argument of $Extract() is a $Select() that can be optimized to a numeric constant, bad object code could be compiled, producing unpredictable results. This change fixes the problem.

DP-405828: Fix $Data(^global) optimization

Category: ObjectScript
Platforms: All
Version: 2021.1.1

When $Data(^global) is followed by a $Select expression that the compiler can optimize to a constant, the compiler could generate invalid object code causing an error or access violation. This change corrects the problem.

DP-406903: Fix variable=$SELECT() optimization problem

Category: ObjectScript
Platforms: All
Version: 2021.1.1

The specific syntax in a routine, but not in an Xecute,

  Set local=$Select(1:$Select(0:expression,1:expression2))

or

Set local=$Select(1:-$Select(0:expression1,1:expression3))

could be compiled into incorrect object code under certain conditions. This change fixes the problem. These conditions include:

• If the expression2 can be optimized by the compiler to a numeric constant between 0 and 255 inclusive.

• If the expression3 can be optimized by the compiler to a numeric constant between -255 and 0 inclusive.

• If the 0 condition for expression1 is an expression which the compiler knows will evaluate to 0.

DP-402305: Deleting OAuth 2.0 clients now deletes the access tokens issued to those clients

Category: Security
Platforms: All
Version: 2021.1.1

Deleting an OAuth 2.0 client configuration or client description now deletes all access tokens issued to that client. Since deleting an OAuth 2.0 server configuration deletes all client descriptions on that server, deleting a server configuration now also deletes all access tokens issued by that server.

DP-403670: Allow setting Dispatch Class for Web Application using ^SECURITY

Category: Security
Platforms: All
Version: 2021.1.1

This change modifies the ^SECURITY routine to allow users to configure the Dispatch Class for a Web Application.

When using a Dispatch Class many of the Web Application settings are no longer relevant. Users will not be prompted for these settings when a Dispatch class is specified. This matches the behavior in the Management Portal.

DP-404140: Support configurable Diffie Hellman key sizes

Category: Security
Platforms: All
Version: 2021.1.1

This change adds a configuration item to the server-side SSL configurations specifying the minimum size of Diffie Hellman keys.

DP-404341: OAuth2 Introspection response should use 'token_type' claim, not 'token_type_hint'

Category: Security
Platforms: All
Version: 2021.1.1

a previous change modified the OAuth2 server Introspection endpoint to accept the standard "token_type_hint" parameter instead of the "token_type" parameter, but it also incorrectly modified the response to include a claim named "token_type_hint" This should be "token_type".

See [RFC 7662|https://tools.ietf.org/html/rfc7662], sections "2.1 - Introspection Request" and "2.2 - Introspection Response".

DP-404857: Use correct defaults for OAuth2 client JWT algorithms

Category: Security
Platforms: All
Version: 2021.1.1

This change corrects the default values for the OAuth2 client IDToken and Userinfo algorithms.

Previously the IDToken, Userinfo and Access Token algorithms would default to using the corresponding value in the Server's JWT settings page, if the client value was "". Note that these values cannot be set to "" in the Management Portal, but they can be configured manually, or via dynamic registration from a third-party OAuth2 client. A previous fix changed this logic to instead default to the server values if the client value was "none".

Neither behavior is correct. The IDToken and Userinfo algorithm settings are defined in the Open ID Connect Dynamic Registration specification https://openid.net/specs/openid-connect-registration-1_0.html.

See the definition for:

userinfo_signed_response_alg
userinfo_encrypted_response_alg
userinfo_encrypted_response_enc
id_token_encrypted_response_alg
id_token_encrypted_response_enc

The specification defines what the value of these fields should be if omitted by the client ("none" in most cases.) The specification does not allow for the server to substitute another value arbitrarily.

This change modifies the logic to not apply the Server JWT settings to the IDToken or Userinfo algorithms. They will still apply to the Access Token algorithms. These settings are not part of the OpenID Connect standard.

Note that the Request algorithm settings (from the client JWT settings page) are defined in the OpenID connect spec, but they were never overridden by the server settings, so the logic relating to them is unchanged.

DP-405517: Preserve CSPCHD token in OAuth2 redirects

Category: Security
Platforms: All
Version: 2021.1.1

This change corrects a problem that would prevent OAuth2 applications using Authorization Code or Implicit authorization flow from interfacing correctly with CSP applications when cookies are disabled. When cookies are disabled the CSP session token is included in the CSPCHD URL parameter. Previously, this would be removed when redirecting back to the application after successful authorization, which resulted in the session being lost. This has been corrected. Now OAuth2.Response uses the %response.UseExactRedirect flag to prevent any translation of the specified redirect URL.

In order for the redirect to work, the application should call ..Link() on the redirect URL when calling GetAuthorizationCodeEndpoint() or GetImplicitEndpoint():

 set url=##class(%SYS.OAuth2.Authorization).GetAuthorizationCodeEndpoint("MyApp", scope, ..Link(myRedirectURI),,,.sc) 

This will ensure that the CSPCHD token is included in the URI. This is only necessary if this is a CSP/ZEN application and cookies are disabled.

DP-406140: Ensure CSV Export button appears when switched on in Widget Settings

Category: Security
Platforms: All
Version: 2021.1.1

When you go into a dashboard, and go to Widget Settings for a pivot table widget, there are switches for Excel Export and CSV Export. When both switches are on, the export button appears with Excel, CSV, and CSV with Headers options. With just the Excel switch on, the export button appears and exports to Excel as before. However, when just the CSV switch is on, no button appears. This change corrects the problem.

DP-409573: Do not use CSP sessions for OAuth2 Client pages

Category: Security
Platforms: All
Version: 2021.1.1

This change corrects the OAuth2.Response and OAuth2.JWTServer CSP pages to not use persistent CSP sessions. These pages are served via the /csp/sys/oauth2/ application on OAuth2 Client systems.

DP-411575: Update Apache httpd to 2.4.52

Category: Security
Platforms: All
Version: 2021.1.1

This change updates Apache httpd used as private web server to version 2.4.52.

DP-411748: Fix 32-bit ODBC driver crashes on Windows Server 2022

Category: Security
Platforms: Windows
Version: 2021.1.1

This change fixes a problem in which irisconnect would crash if the user attempted to test an SSL connection without defining the SSL/TLS configuration in the registry or SSLdef.ini. Now irisconnect gracefully reports an error if the configuration is not found.

DP-405684: Fix %SYSTEM.Sharding.GetOption to correctly recognize MasterNamespace argument

Category: Sharding
Platforms: All
Version: 2021.1.1

This change fixes a problem in which the $SYSTEM.Sharding method GetOption(), if executed in a namespace other than the one specified in its MasterNamespace argument, failed to set $NAMESPACE to the specified namespace, causing it to return incorrect results.

DP-406362: Generate correct shard-local global names for sharded classes when USEEXTENTSET=0

Category: Sharding
Platforms: All
Version: 2021.1.1

This change fixes a problem in which some sharded classes had shard-local global names which did not correctly map to the shard-local databases. This resulted in data being incorrectly stored in the master database rather than in shard-local databases, causing queries to return incorrect results (each row was returned as many times as the shard count).

DP-406794: Fix method used for Management Portal sharded table data distribution query

Category: Sharding
Platforms: All
Version: 2021.1.1

This change fixes the %SQL.Manager.Catalog method ShardTableDataDistributionExecute, to cause it to return correct per-shard row counts for sharded tables.

DP-407540: Allow specifying master namespace in %SYSTEM.Cluster Attach methods

Category: Sharding
Platforms: All
Version: 2021.1.1

This change fixes a problem in which specifying the master namespace, rather than the cluster namespace, in the cluster URL passed to a %SYSTEM.Cluster AttachAsXXXNode method resulted in the error "ERROR #9600: $SYSTEM.Cluster methods cannot be used on cluster that has been modified using $SYSTEM.Sharding API". With this change, when specifying a cluster URL for node 1 of a sharded cluster, it is permitted to specify the master namespace (default "IRISDM") as an alternative to specifying the cluster namespace (default "IRISCLUSTER").

This fixes an issue found using the Management Portal. However, note that while that issue relates to attaching drasync node to a cluster using Management Portal, the problem, as well as this fix, applies to direct use of the %SYSTEM.Cluster API, as well as to indirect use via Management Portal, and applies to all node types, not just mirrored, drasync nodes.

DP-402424: Fixes problem where the results of TuneTable execution on Management Portal aren't displayed correctly

Category: SQL
Platforms: All
Version: 2021.1.1

Depending on the number of records, Selectivity and Extentsize of running TuneTable on Management Portal did not appear correctly. This change corrects the problem.

DP-402500: Fix incorrect purge of UCQ for embedded SQL

Category: SQL
Platforms: All
Version: 2021.1.1

This change fixes a problem where any UCQ (universal cached query) for embedded SQL in any namespace was purged unexpectedly when the system task 'Update SQL query statistics' was executed the first time in a day. These are now only be purged if the caller was removed and the caller is no longer in the routine buffer cache.

DP-403030: Fix problem where last line of statement could be incorrectly read as a comment

Category: SQL
Platforms: All
Version: 2021.1.1

This change fixes a problem executing SQL statements with comments with "--" prefix

If a query statement run in a client contains comments such as in this example:

---GROUP BY Patient HAVING COUNT(ID) BETWEEN 4 and 6

or any other text preceded by the "--" characters, the query won't return any results and the client never gets a response from the server.

DP-403832: Fixes error during UCQ execution might be masked by post UCQ <UNDEFINED>

Category: SQL
Platforms: All
Version: 2021.1.1

In UCQ ensure we do not mask an error running query itself with an UNDEFINED error because the variables we are expecting to copy into the users SQL values are not defined.

DP-405479: Support for cross-site unique identifiers in FM2Class

Category: SQL
Platforms: All
Version: 2021.1.1

This change is version 2.41 of FM2Class.

There is a new setting, which you can define directly in the global for now, to define the datatype class for the IEN:

set ^%SYS("sql","fm2class","IENType")=“XX.AAA"

This class name can be anything you want.  If this class name is not defined, the default %Library.String is used. For example, you could choose a name something like PackageName.IENType.  Use whatever package name you want, and just make sure it does not collide with any existing classes. If the class does not exist, FM2Class will create it for you (which is what should happen the first time you run FM2Class with this version of the mapper). So there is nothing to do other than define this global to the name of the class you want to use.   Then run FM2Class like you normally would.   Here is an example of the data returned (in Display or ODBC mode, Logical mode returns the on-disk IEN value).

  
[SQL]SQL:Hosp1>>select ien, SPOUSES_EMPLOYERS_STATE from Hosp1.patient_2 where SPOUSES_EMPLOYERS_STATE is not null
13. select ien, SPOUSES_EMPLOYERS_STATE from Hosp1.patient_2 where SPOUSES_EMPLOYERS_STATE is not null
 
IEN SPOUSES_EMPLOYERS_STATE
20 36
210 36
312 26
357 11
380 11
426 36
428 36
743 24
 
8 Rows(s) Affected
statement prepare time(s)/globals/cmds/disk: 0.0500s/35,128/173,001/2ms
          execute time(s)/globals/cmds/disk: 0.0008s/1,787/11,655/0ms
                          cached query class: %sqlcq.Hosp1.cls17
---------------------------------------------------------------------------
[SQL]SQL:Hosp1>>set selectmode odbc
 
selectmode = odbc
[SQL]SQL:Hosp1>>select ien, SPOUSES_EMPLOYERS_STATE from Hosp1.patient_2 where SPOUSES_EMPLOYERS_STATE is not null
14. select ien, SPOUSES_EMPLOYERS_STATE from Hosp1.patient_2 where SPOUSES_EMPLOYERS_STATE is not null
 
IEN SPOUSES_EMPLOYERS_STATE
20|664 36|664
210|664 36|664
312|664 26|664
357|664 11|664
380|664 11|664
426|664 36|664
428|664 36|664
743|664 24|664
 
8 Rows(s) Affected
statement prepare time(s)/globals/cmds/disk: 0.0001s/4/137/0ms
          execute time(s)/globals/cmds/disk: 0.0014s/1,787/14,342/0ms
                          cached query class: %sqlcq.Hosp1.cls17

DP-405703: Fix fastInsert to support DSTIME

Category: SQL
Platforms: All
Version: 2021.1.1

A problem has been corrected where a class defined to use DSTIME was not properly setting the ^OBJ.DSTIME global when an INSERT was performed over xDBC and the fastInsert feature was used.

DP-405825: Correct lock issues on cached queries and statement index

Category: SQL
Platforms: All
Version: 2021.1.1

This change fixes a rare issue with the various APIs in %SYS.SQLSRV for Statement Index IUD and in Prepare of the dynamic statement cached query lookup/creation.

When using the WQM and the Sharding mechanism, there are times when an <EXTERNAL INTERRUPT> error is sent to shards to stop processing the query (typically because we have received all of the rows required for the table already). Depending on the query, this <EXTERNAL INTERRUPT> can happen multiple times. The <EXTERNAL INTERRUPT> error caused problems when the interrupt occurred after the LOCK, but before the set of gotlock.  When gotlock not TRUE, the global never got unlocked in the API.

This change ensures the global is unlocked in the no-error case and in the error case.

DP-405835: Disable external interrupt errors around sharding query construction logic

Category: SQL
Platforms: All
Version: 2021.1.1

In the code run on a shard system to build the shard local query, this change disables getting EXTERNAL INTERRUPT errors so we can ensure the query is always built.

DP-405944: Fix collation for %PARALLEL on a numeric column that could cause a rows to be skipped

Category: SQL
Platforms: All
Version: 2021.1.1

When splitting a global into chunks to process in parallel we detect if the last node is a numeric value we added one to it to avoid $double rounding errors. This can cause problems with numeric values because although 1.1]]0.1 is true "1.1"]]"0.1" is false. The reason being that "0.5" is not a canonical number due to the leading '0' character where as "1.1" is a canonical number so for example:

set a("0.1")="string zero point one"
set a(0.1)="numeric zero point one"
set a("1.1")="string one point one"

zw a
a(.1)="numeric zero point one"
a(1.1)="string one point one"
a("0.1")="string zero point one"

The result of this is if the last range in a parallel query was a string value with a leading '0' which can be converted into a number between 0 and 1 e.g. "0.1" when we added 1 to this to ensure we do not get $double rounding problems we created a string value "1.1" which collates before the end of the global i.e. "0.1". This means rows from a query are not processed when the query is run in parallel. This change fixes the problem.

DP-406231: Apply conditions in shfls() for override plan

Category: SQL
Platforms: All
Version: 2021.1.1

Fixed a rare bug for a query in which GROUP BY <g> is used, no aggregates are used, there are some ungrouped fields <sf> that appear in the SELECT-list or the HAVING clause, there are WHERE conditions on the <sf> fields (and possibly also on the <g> fields), and there is a map that has the <g> fields as leading subscripts, and has all the <sf> fields as later subscripts.  Prior to this change, WHERE conditions on the <sf> fields were not applied, possibly resulting in incorrect extra output rows.

Note that while the SQL Standard does not allow non-grouped fields to be used (unaggregated) in the SELECT or HAVING clause, this bug is possible even for SQL Standard queries, for example, if the <g> fields functionally determine all the <sf> fields, that is. if the <g> fields constitute a UNIQUE key.  In that case the SQL Standard query:

SELECT <g>,<sf> FROM t WHERE cond(<sf>) GROUP BY <g>,<sf>

would be internally converted to the query:

SELECT <g>,<sf> FROM t WHERE cond(<sf>) GROUP BY <g>

and the bug could occur.

DP-406409: Correct handling of <EXTERNAL INTERRUPT> errors in SQL runtime logic

Category: SQL
Platforms: All
Version: 2021.1.1

A problem has been corrected where an SQL statement that was interrupted by a "Cancel" via xDBC, SMP, or via $SYSTEM.Util.SendInterrupt(PID) did not always properly release locks and/or end transactions.

Two new SQLCODE values have been introduced by this change:
SQLCODE=-457, SQL statement execution interrupted
SQLCODE=-458, Server request interrupted

Prior to this change, in the xDBC Server, the same SQLCODE was used for an <ALARM> (timeout), -450, as for an <EXTERNAL INTERRUPT> (A Cancel). Now -450 is used for Timeout, and -458 for Cancel.

This change also corrects a problem where the server process would not always terminate if the server process was Terminated with a <RESJOB> error.

DP-406534: Correct fastSelect <UNDEFINED> error in Prepare when all fields come from index subscripts

Category: SQL
Platforms: All
Version: 2021.1.1

The problem is exposed as an <UNDEFINED>output+66^%qaqcct error during Prepare of a SELECT statement via xDBC.

The problem was fastSelect was used for a case where it should not have been.  The query plan simply loops on index subscripts and outputs the row, there is no going back to the master map and no map data field retrieval. This change corrects the problem.

DP-406629: Fix 'opg' set-looping code-generation for IN

Category: SQL
Platforms: All
Version: 2021.1.1

Fixed a bug when GROUP BY and aggregates are used, when the plan loops over subscripts with the GROUP BY fields, and the plan also loops on a subsequent subscript that has an IN condition (or an equivalent OR condition).  Such a query would incorrectly return 0 rows.

DP-407157: Analysis of query grouping accounts for shard key collation

Category: SQL
Platforms: All
Version: 2021.1.1

Queries containing GROUP BY on sharded tables with non-integer shard keys could perform redundant re-grouping of results from each shard. This change corrects that behavior.

DP-407232: Correct hash for %XSQL.DSI.TempTable and %XSQL.DSL.Cursor CQ implementations

Category: SQL
Platforms: All
Version: 2021.1.1

A problem has been corrected where the hash used for a %XSQL.DSI.TempTable or %XSQL.DSI.Cursor cached query implementation was not defined with the same hash as it was looked up upon Prepare. For example, suppose a call is made to:

set oref=
##class(%XSQL.DSI.TempTable).%Prepare("CREATE PRIVATE TEMPORARY TABLE...")

This type of private temporary table is used for statement-level triggers and caused a new cached implementation of the PTT to be generated for every insert/update/delete executed on the table (if there is a statement-level trigger defined for the event). This led to degraded performance and bloat of the size of the statement caches.

DP-407319: Prevent <UNDEFINED> error by not applying special distinct aggregate processing

Category: SQL
Platforms: All
Version: 2021.1.1

This change prevents an <UNDEFINED> error caused by applying special distinct aggregate processing when group matches the shard key ("gruplup").

DP-407368: Correct endian ordering issue checking routine header signature against stack information

Category: SQL
Platforms: All
Version: 2021.1.1

On big endian systems it was possible we would fail to build a universal cached query correctly as we thought another process had already modified the UCQ. This would be reported as errors such as:

ERROR #5821: Cannot instantiate query: 'SQLCODE = -400'

We have corrected the check and also added %msg to this error message so make debugging similar issues in future simpler.

DP-407384: Fix case where query optimizer is incorrectly using the normal selectivity

Category: SQL
Platforms: All
Version: 2021.1.1

This change corrects the behavior of prmIncOutlier and checks the parameter code includes outlier behavior.

DP-407453: Fix a caching problem for a DISTINCT query with a SELECT DISTINCT view in FROM

Category: SQL
Platforms: All
Version: 2021.1.1

This change fixes a caching problem that causes some UNION ALL queries to run slowly. This problem impacted Runtime Plan Choice (RTPC) query optimization.

DP-407554: Handle DISTINCT BY() in sharding query correctly

Category: SQL
Platforms: All
Version: 2021.1.1

In some cases when the sharding key and gruplup matched, the DISTINCT BY clause would not be handled correctly and extra rows could be returned. This change corrects the problem.

DP-407796: Correct <PARAMETER> error with some embedded SQL UCQ executions

Category: SQL
Platforms: All
Version: 2021.1.1

A problem has been corrected where the execution of an embedded SQL statement that included a host variable that is a property object reference. A problem with the code generation of the Universal Cached Query was putting too many arguments in the call to the %New() method of the UCQ.

DP-408028: Escalate roles to post remote event

Category: SQL
Platforms: All
Version: 2021.1.1

Posting events from shard worker nodes to the system that initiated the query requires the API escalate privileges to avoid getting PROTECT errors such as:

<PROTECT>postRemoteEvent+31^%SYS.WorkQueueMgr 

DP-408257: Fix race condition in SQL parallel union processing

Category: SQL
Platforms: All
Version: 2021.1.1

When running SQL 'union %parallel' or 'union all %parallel' it was possible in rare cases to get an error that 'Worker job with IPQ exited unexpectedly' due to a race condition in the parallel logic. This is corrected now.

DP-408385: Correct procedure privilege check for schema level privileges on 404 check

Category: SQL
Platforms: All
Version: 2021.1.1

A problem has been corrected where a call to an SQL procedure from xDBC might not work correctly if the user received the EXECUTE privilege on the procedure through a schema-level privilege.

DP-408513: Cleanup invalid cached query classes

Category: SQL
Platforms: All
Version: 2021.1.1

Make code which builds cached query classes more resilient to existing invalid classes.

DP-408527: Collect light weight SQL stats on read-write mirror members

Category: SQL
Platforms: All
Version: 2021.1.1

This change collects light weight SQL stats on read-write mirror members.

DP-409182: Kill iDj fast i%var variables in SQL Close

Category: SQL
Platforms: All
Version: 2021.1.1

When SQL uses fast i%var variables in cached query definitions, there were property values used by the SQL statement that were not killed in SQL CLOSE.  Most of the time this is okay, but a customer reported an error with their application because they had some {iDj} variables defined in SQL Storage that held an oref, and the expectation was this oref would be destructed when the SQL statement closed.  Any {iDj} map variables that are code generated using i%vars in the cached query are now killed during close.

DP-409582: Correct %XSQL.DSI.Cursor implementation

Category: SQL
Platforms: All
Version: 2021.1.1

The %XSQL.DSI.Cursor implementation was broken by a previous change, this change fixes this so queries generated by %XSQL.DSI.Cursor now act as normal cached queries.

DP-409838: FM2Class v2.42 - Improve performance of WipeClass

Category: SQL
Platforms: All
Version: 2021.1.1

This is version 2.42 of FM2Class.

This change improved the performance of FM2Class for the WipeClass function.  This is the code called to clean up the class definition when re-mapping a class and using the RetainClass=1 option.

DP-409848: Fix Logic with Phrases in iFind

Category: SQL
Platforms: All
Version: 2021.1.1

This change fixes previously erroneous logic related to iFind positional search (and as a result, iFind co-occurrence search) which caused the results of all positional search terms OR'd together in a node, except for the last term, to be ignored. For example, in a table with the following records in its MyText field (which has any non-minimal iFind index on it):

||MyText||
|The quick brown fox jumped over the lazy dog|
|Colorless green ideas sleep furiously|

The following query:

SELECT MyText FROM MyTable WHERE %ID %FIND search_index(MyTextIdx, '"lazy dog" OR "green ideas"')

would only return the second row, and would not include results that contained the "lazy dog" positional search term. Following this change, all positional terms in an OR node are accounted for. This change also makes a minor update to prevent unnecessary checks of positional search strings that only contain the first word in a positional search term.

DP-411896: Fix query reconstruction and condition evaluation for queries with FOR SOME %ELEMENT

Category: SQL
Platforms: All
Version: 2021.1.2

Before this change, a query compilation issue could occur if a query included the WHERE FOR SOME %ELEMENT (%Key=? AND %VALUE =?) clause.

DP-412145: Purge of UCQ queries was not taking out a lock

Category: SQL
Platforms: All
Version: 2021.1.1

There was a bug in the logic that scans for universal cached queries that no longer have a 'parent' routine and that can be thus purged. This bug resulted in purging the query without taking out a lock on the query (thus enabling another process from building a query with the same name at the same time, resulting in an inconsistent state of the universal cached queries). This change corrects the problem.

DP-406019: Fix TQSL error handling leading and trailing {...} braces

Category: SQL Dialects
Platforms: All
Version: 2021.1.1

When preparing the SQL text, this change removes any leading/trailing {...} braces and white space. The TSQL parser does not handle the braces.

DP-406039: Implement bit operator &, |, and ^ for SQL context in TSQL

Category: SQL Dialects
Platforms: All
Version: 2021.1.1

A problem has been corrected where the TSQL bit operators &, |, and ^ were not working properly in a TSQL SQL statement.  They were working in procedures context, but not SQL context.

DP-406274: Correct argument in procedure that causes Error #5001 Invalid Argument name in TSQL

Category: SQL Dialects
Platforms: All
Version: 2021.1.1

A problem has been corrected in IRIS TSQL support when calling a procedure with positioned parameter and omitting one of the parameter values.  For example:

exec my_proc 114209,'a',,'R1'

DP-405455: Restore open flags for Windows async I/O

Category: System
Platforms: Windows
Version: 2021.1.1

This change reverses the negative performance impact of a previous change on async I/O on certain Windows systems. It appears that the effects of that change are noticeable on spinning drives although it may be related to certain versions of Windows. Aynch I/O is used to write incremental backup output files, databases and the wij, by mirroring and by the prefetch code.

DP-405710: Don't consider the journal dmn waiting for a mirror buffers as hung

Category: System
Platforms: All
Version: 2021.1.1

This change avoids a case where the backup mirror member could force down the primary if the primary is stuck waiting on the backup while sending journal data.

DP-406111: Convert ^CacheMsg to ^IRIS.Msg during Caché to InterSystems IRIS conversion

Category: System
Platforms: All
Version: 2021.1.1

The correct namespace was not being set before the conversion of the global. The code now sets the implied namespace to the database being converted, and then set it back to %SYS after done.

DP-406818: Correct reading of terminal script files with Japanese characters

Category: System
Platforms: All
Version: 2021.1.1

Terminal script files that contained Japanese characters, or any Unicode characters encoded with UTF8 or otherwise, were not read correctly. The encoded characters are now presented correctly, as bytes, as input to InterSystems IRIS where they can be processed by, for example, $ZCVT(input,"I","UTF8").

DP-406855: Remove unnecessary test from new password hash algorithm

Category: System
Platforms: All
Version: 2021.1.1

An unnecessary test in the code which rehashes the password could cause LDAP users to be unable to login after an upgrade to 2021.1. This change removed the test.

DP-407609: Correct DataMove issues

Category: System
Platforms: All
Version: 2021.1.1

This change fixes the following issues:

1. Setting the MB per minute rate to a very low value (say 1mb) would allow the DataMove to run with no limit set. This was due to the value being rounded to 0, where 0=unlimited.

2. Owner and group on the datamoveworkdb IRIS.DAT was not being set. This change sets to the same value of the MGR database. Previously this could cause a PROTECT error.

3. If data being moved contained a "|" in the subscript, a <SYNTAX> error would occur during the activate phase. This was due to a line of code using $p(node,"|",3) rather than $p(node,"|",3,*) which truncated the subscript after the "|".

4. If the system was forced down or crashed while a DataMove was running, the new DataMove mappings would be lost when the DataMove was later activated after the system restarted. This change preserves the mappings on a restart.

DP-408045: Better error message for DataMove unexpected error

Category: System
Platforms: All
Version: 2021.1.1

This change added more detail to an error which reports unexpected errors.

DP-408050: Fix undefined error when specifying maxmbpermin in DataMove

Category: System
Platforms: All
Version: 2021.1.1

This change fixes an undefined error when specifying the MaxMBPerMin property when calling the DataMove.API:Create() method.

DP-408426: Eliminate an unnecessary error about journal switch

Category: System
Platforms: All
Version: 2021.1.1

It's unnecessary for ^JRNSWTCH or %SYS.Journal.System:RollToNextFile() to fail when journal switch is in progress.

DP-408580: Allow DataMove to work with globals with characters greater than $c(122) and less than or equal to $c(255)

Category: System
Platforms: All
Version: 2021.1.1

This change updated the $$globnext^NSPMAP function to support valid characters in global names up to $c(255). The DataMove.Data:Generate() method now allows for global name to start with characters up to $c(255)

DP-408674: Fix DataMove range error on 8-bit system

Category: System
Platforms: All
Version: 2021.1.1

ENU8 and ENUW collations differ for the list of valid characters for global names. This change updated the code to use $zname(char,3) to get the list of valid characters in the loaded collation, and to use this list to generate the next valid global name.

DP-408747: Fix HealthShare Security Domain Functionality

Category: System
Platforms: All
Version: 2021.1.1

This change is to handle the required HealthShare Security Domain Functionality.

DP-408751: Remove DataMove error tests which are not errors

Category: System
Platforms: All
Version: 2021.1.1

DataMove tries to set some O/S permissions on the work database. If it can't (possibly because it is running with lower permissions), it should log a message and continue running successfully. It wasn't clearing the error status after logging the message so the DataMove would stop. It now clears the error status and continue without stopping.

DP-408869: Fix %Library.GlobalEdit:GetGlobalSizeBySubscript() and KillRange() for Begin ranges like X("A")*NEXT:X("C")

Category: System
Platforms: All
Version: 2021.1.1

Ranges similar to X("A")*NEXT:X("C") were sometimes including data under the ^X("A") node. What it should do is start looking at data after all the ^A("A") nodes, for example, ^A("B") and later. With this change it performs a $query() and gets the next subscript at this level (for example ^A("B"), and changes the start range to this. This updated range is then used to calculate the size. DataMove uses this method, and the end result was that it could see MBToCopy values higher than what was actually copied. This also would cause the estimated time to finish to be longer than what the copy actually took.

For example:

%SYS>F i=1:1:1200 s ^A("A",i)=$j(i,30)
s x=##Class(%GlobalEdit).GetGlobalSizeBySubscript($zu(12),"A(""A"""_$c(1)_"*NEXT)","A(""C"")",.s)

Before the change, the behavior would be:

%SYS>zw s
s=.06
s("Blocks",1)=1
s("Blocks",2)=6
s("Blocks","BigStrings")=0
s("Blocks","Total")=7
s("MB",1)="0.01"
s("MB",2)="0.05"
s("MB","BigStrings")="0.00"
s("MB","Total")=.06

After the change, the behavior would be:

%SYS>zw s
s=0
s("Blocks",1)=0
s("Blocks",2)=0
s("Blocks",3)=0
s("Blocks","BigStrings")=0
s("Blocks","Total")=0
s("MB",1)="0.00"
s("MB",2)="0.00"
s("MB",3)="0.00"
s("MB","BigStrings")="0.00"
s("MB","Total")=0

And with the following:

%SYS>F i=1:1:1200 s ^A("A",i)=$j(i,30)
s x=##Class(%GlobalEdit).KillRange($zu(12),"A(""A"""_$c(1)_"*NEXT)","A(""C"")")

Before the change, all nodes under ^A("A") would be deleted. After the change, all the nodes would continue to exist.

DP-408945: Increase timeout for starting DataMove processes

Category: System
Platforms: All
Version: 2021.1.1

On busy systems, the job command can actually fail within the timeout period. When the job command fails, the entire DataMove is halted.

This change updated the code to change the timeout to 60 seconds, and to log retries if the job fails. (up to five retries before we fail)

The change also updated the code to calculate the JRNMB field to properly calculate the amount of journal left to apply for each range when journal file compression is on.

DP-408983: Fix DataMove undefined when changing rate

Category: System
Platforms: All
Version: 2021.1.1

DataMove could get an undefined error if the max rate is changed after the copy is done, but before changes are activated. this change eliminates the undefined error in this case.

DP-409045: Update DataMove metrics more often

Category: System
Platforms: All
Version: 2021.1.1

DataMove metrics should be updated every 5 seconds or so. There was a problem where creating new DataMove processes pre-empted the update of the metrics fields causing them to skip updating the fields. Normally this is not a problem, but when there are a lot of moves (hundreds) where the amount of data being moved is low (a few mb) new processes were being created almost continuously, causing the updates to be skipped for potentially long periods of time. This change updated the algorithm so that the update of the metrics is done before determining to start another copy process.

DP-409094: Update mappings for DataMove on Japanese systems

Category: System
Platforms: All
Version: 2021.1.1

Japanese locales have no identifier characters in the range $c(128)-$c(255). This change updated the global mapping for DataMove to provide an endpoint character of $c(255) when calling $$globnext^NSPMAP("%z"). This now returns "%z"_$c(255) as the ending range.

DP-409618: Check DataMove stop flag more often

Category: System
Platforms: All
Version: 2021.1.1

To improve responsiveness this change checks the DataMove stop flag more frequently.

DP-409970: Fixed a bug in DataMove for Japanese systems

Category: System
Platforms: All
Version: 2021.1.1

This change enables DataMove to work with Japanese systems and other systems that have data containing characters above $CHAR(128).

DP-410409: Recalculate DataMove blocks copied on a crash

Category: System
Platforms: All
Version: 2021.1.1

This change improves the reporting of the number of blocks copied if DataMove is interrupted.

DP-410429: Corrects a DataMove issue in the Japanese locale

Category: System
Platforms: All
Version: 2021.1.1

This change corrects an error in DataMove that could occur when working with data in the Japanese locale

DP-410605: Fix User security record being corrupted

Category: System
Platforms: All
Version: 2021.1.1

When a process calls $SYSTEM.Security.Login() to log in as another user, if an error occurs in the span of a couple of lines of the method, the original user record can be overwritten by the new user record. This would cause the original user to no longer be able to log in.

This was seen when a stored procedure which as one of its tasks uses $SYSTEM.Security.Login() to change itself to the _SYSTEM user to do some "system stuff". It just so happened that the connection had a timeout set, and an <ALARM> error was generated at just the wrong spot in the code when the user credentials were getting changed. Normally an error could never happen in this section of the code during normal operation, but when an alarm is set by the server code, an asynchronous error could occur causing the failure.

This change prevents the old user record from being overwritten by the new user record in the case of an error.

Note that the window here is very small and unlikely to occur.

DP-410986: Fix memory leak caused by LDAP searching with pages

Category: System
Platforms: All
Version: 2021.1.1

Calling ##Class(%SYS.LDAP).GetNextPages() could cause a memory leak when searching. This problem could be seen using the LDAP interoperability Business Operations. This change corrects the problem.

DP-407082: Increase terminal window size to make all output visible

Category: Terminal
Platforms: All
Version: 2021.1.1

Output to the top line or right-most column of the terminal window could be overlaid by the menu bar or scroll bar. This change makes the window slightly larger so all the output is visible.

DP-403494: Correct JavaExe() result when JAVA_HOME contains spaces

Category: Tools
Platforms: Windows
Version: 2021.1.1

On Windows, if the JAVA_HOME environment variable is a path name with spaces, $System.Util.JavaExe() could return an invalid path. This change corrects the problem.

DP-405779: SystemPerformance - new section "irisstat -R" containing a single snapshot of routine buffers

Category: Tools
Platforms: All
Version: 2021.1.1

SystemPerformance reports on all platforms will now include a new section named "irisstat -R".  This section contains a single sample of the irisstat command using the -R1 flag, which dumps routine buffers.

DP-407065: GetInternalStackLevel of %SYS.ProcessQuery returns NEW levels

Category: Tools
Platforms: All
Version: 2021.1.1

A second by-reference argument is added to the GetInternalStackLevel method of %SYS.ProcessQuery: GetInternalStackLevel(stack, newlevels)

When there are NEW commands in a routine, each NEW command creates a new internal stack level without increasing $STACK. GetInternalStackLevel returns the highest internal stack level for the given $STACK. If there are other internal levels created by NEW commands within the given $STACK, they are returned in the newlevels argument as a $LIST of values. There may be different variables defined at each of those levels and the same variable may have different values at each level.

DP-407279: Handle recursion of $Compile() that could cause a segfault

Category: Tools
Platforms: All
Version: 2021.1.1

There could be an address exception if a Source Control class has a method that causes recursion into the class loader/compiler. This change corrects the problem.

DP-402068: Revamp management of thread life cycles

Category: Web Gateway
Platforms: All
Version: 2021.1.1

Fix random crashes that would sometimes occur when connections to InterSystems IRIS were closed down or when web server worker processes were closed down.

DP-403826: Fix memory leak when Web Gateway fails to log in to CSP server

Category: Web Gateway
Platforms: All
Version: 2021.1.1

This change fixes a memory leak that occurs when the Web Gateway fails to log in to the CSP server, causing high memory usage if many login failures occur.

DP-404812: Do not delete httpd.pid if ##class(Config.Startup).StopWebServer() can't stop the httpd process

Category: Web Gateway
Platforms: All
Version: 2021.1.1

Fixes the handling of a case where Config.Startup:StopWebServer() would delete the httpd.pid file but fail to stop the Private Web Server process. This could occur if, for example, the instance user has the %All role, but the OS-level user who logged in does not have permission to stop the process. The process would then continue running until killed manually at the OS level, since the instance needs the httpd.pid file to find and kill the process during shutdown. Now, if the process fails to terminate, the httpd.pid file is left in place so that InterSystems IRIS can still find and clean up the old process later.

DP-405020: Prevent configuring invalid CSP server names and application paths

Category: Web Gateway
Platforms: All
Version: 2021.1.1

In the Server Access and Application Access configuration pages, it was possible to add a CSP server or application name containing special words or characters that would corrupt the Web Gateway configuration. For example, the name "system" is reserved by the Web Gateway, and creating a CSP server named "system" led to corruption of the configuration. The Web Gateway now validates CSP server and application names and rejects those containing reserved words or characters.

DP-409791: In WQM detect workers that do not run any commands as inactive

Category: Work Queue Manager
Platforms: All
Version: 2021.1.1

The work queue manager keeps track of if worker jobs are active or not so it can detect deadlock situations and automatically start additional workers. This logic did not work in the case of a worker jobs that has done callout and this callout logic was sitting idle. This change corrects the problem.