Skip to main content

InterSystems IRIS® Upgrade Checklist and Change Notes (2021.1.3)

This document is meant to help you assess the impact of upgrading to the InterSystems IRIS 2021.1.3 maintenance release. It contains the upgrade checklist items since the last extended maintance (EM) release, 2020.1.0. It lists fixed issues. It lists other changes since 2021.1.0, grouped by category.

InterSystems News, Alerts, and Advisories

From time to time, InterSystems publishes items of immediate importance to users of our software. These include alerts, mission-critical issues, important updates, fixes, and release announcements. You can obtain the most current list at https://www.intersystems.com/support-learning/support/product-news-alerts/Opens in a new tab. InterSystems recommends that you check this list periodically to obtain the latest information on these issues.

Upgrade Checklist (Developers)

DP-12437+PLUS: IRISNative and Gateway Changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Language Bindings - Native
Platforms: All
Version: 2021.1.0

This release contains the following changes to the language bindings IRISNative API and to the gateway:

  • Changes to Native SDKs and IRISList :

    • If the underlying value is the empty string, getObject returns an empty string or zero-length byte array. If the underlying value is UNDEFINED, getObject returns a null.
    • For numeric types, if the underlying value is an empty string, get returns a 0 value.
    • getObject returns a byte array for type 1 (ASCII).
    If you use this feature, you will have to refactor and recompile your code. These changes have the following consequences:
    • Empty string is not converted to $char(0) and back as it was done in previous releases. Empty string is left as empty string. Note that this is different from the SQL semantics for empty strings.
    • An attempt to use empty string or empty byte[] as a subscript generates a <SUBSCRIPT> error.
    • No changes in setting null. set(null,) and set ("",) both result in empty string on the server.
    • Getting undefined value will result in null.
    • Getting any numeric (int, double) value from "" returns 0. For methods and functions, the return value is null.
    • getByte from "" returns empty byte[0] , getString from "" results in empty string.
    • GetObject from type 1 results in byte[0] in both the IRIS and IRISList classes.
    • GetDateTime from empty string returns null, where in previous releases it was throwing an exception.
    • getIRISList returns a valid object of length 0 for values 02 01, and returns a null object for 01.
    • Iterator returns an empty string if subscript is not found, where in previous releases a null was returned.
  • Projection of byte array in method arguments — In previous releases, byte arrays were projected as strings, but in this release they are projected as a proxy object. You can convert the proxy object to a %GlobalBinaryStream object (first in 2020.3).
  • Parameter order in IRISNative iris function and iris procedure change — In the previous version, there was a difference in the order of the parameters between the documentation and the implementation. In this release, the implementation has been updated to match the documentation (first in 2020.3). The parameter order is as follows:

    iris.function(functionName,routineName,*args)

    iris.procedure(procedureName,routineName,*args)

    Where the implementation had the routineName parameter before the functionName parameter and the procedureName parameter. If you have coded these methods to the previous order, you must update the code. See function() and procedure() for details.

DP-12771: Configuration API changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Tools
Platforms: All
Version: 2021.1.0

If you are using the %Net.Remote.Object gateway APIs to configure Object gateways, you must now use the Config.Gateways APIs. The properties associated with the old values LineRecallEntries and LineRecallBuffer in the Config.config class have been removed. If you were setting these properties via the Config.Config APIs, you will now need to update the "history" property rather than these.

DP-13503+PLUS: ICM can be used only with InterSystems API Manager (IAM) 1.5 and later

NOTE: This item may require a change to code, configuration, or operation.

Category: Cloud
Platforms: All
Version: 2021.1.0

With this change, ICM can only be used with the InterSystems API Manager (IAM) version 1.5 or later.

If you were using the Rancher or Weave Scope monitoring solutions with a previous ICM version, with this version you need to deploy these manually or use generic ICM commands.

The parameters in CPF have changed from the beta version. This change only impacts beta users of ICM.

Replace: 

[Startup]
  EnableECP=1

with:

[Actions]
ModifyService:Name=%service_ecp,Enabled=1

Replace:

 [Startup]
  EnableIAM=1

with: 

[Actions]
ModifyApplication:Name=/api/iam,Enabled=1
ModifyUser:Name=IAM,Enabled=1

DP-13572: JAR file change

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways - Java
Platforms: All
Version: 2021.1.0

If your Java application has dependencies on the Gateway JAR, you must change them to be dependent on the JDBC JAR.

DP-13931: Minor write daemon changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: UNIX®
Version: 2021.1.0

These are minor changes unlikely to impact code. In the %SYS.ProcessQuery, $SYSTEM.Process, SYS.Process classes, the property Routine has the value "AUXWD" rather than "SWRTDMN" for the auxiliary write daemons. For the property JobType, which is documented as taking a value from %syPidtab.inc, the definition in that include file is renamed from $$$SLWDTYPE to $$$AUXWDTYPE, and the numeric value is unchanged. In UNIX® systems, process names are changed from SWD1, SWD2, ... to AUXWD1, AUXWD2,...

DP-13935: Return value change for OneWay EnsLib.HTTP.GenericService

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

Before this change, for any custom code that subclassed EnsLib.HTTP.GenericService and had enabled OneWay, an "HTTP/1.1 200 OK" would be returned to the caller. This is now corrected to ensure returning an "HTTP/1.1 202 Accepted" to the caller.

DP-13977+PLUS: Decimal type changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways
Platforms: All
Version: 2021.1.0

In this release, $decimal is mapped to Decimal . NET and BigDecimal in Java and the scale is preserved. In previous versions, decimal was mapped to other types and scale was lost.

in ADO.Net, if a decimal value was set as proxy by a previous version, it will not be accessible by a simple get in this and later versions.

In Java, if BigDecimal is set using a generic conversion to byte array, a simple get cannot decode it in this and later versions.

DP-14007: SELECT with INTO clause changes when SQLCODE'=0

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2021.1.0

This release contains a change to the behavior of an embedded SELECT statement with an INTO clause. If the execution of the SELECT statement results in SQLCODE=100, or in the case of a FETCH that results in SQLCODE=100, any INTO variables in the statement will be set to null (""). The variable will be set to "" when SQLCODE=100 or SQLCODE<0 is returned.

DP-20597: ISCAgent now started automatically

NOTE: This item may require a change to code, configuration, or operation.

Category: Cloud
Platforms: All
Version: 2021.1.0

ISCAgent is now started automatically and is started on port 2188 by default in containers. You can disable ISCAgent with "--ISCAgent false". You can control the port used with the new syntax "--ISCAgentPort <port>" instead of the previous syntax "--before /home/irisowner/irissys/startISCAgent.sh <port>".

DP-20621: ^JCONVERT changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2021.1.0

^JCONVERT now creates and processes common journal files in the enhanced format, which supports long strings. It has a new prompt to control whether common journal files are created in the enhanced format or in the previous format. If you need to create common journal files that can be read by ^%JREAD or ^JCONVERT from a previous version, you must specify that you do not want the enhanced format.

DP-20650: %CSP.Page changes impact viewing web page from Studio

NOTE: This item may require a change to code, configuration, or operation.

Category: IDEs
Platforms: All
Version: 2021.1.0

Classes that extend %CSP.Page, override the CSPURL or LOCATION parameters, and explicitly URL encode these parameters may be directed to the wrong web page when viewing the web page from Studio. This is not the case for any system classes, and the URL encoding these parameters is not a documented feature. This can be corrected by not explicitly URL encoding these parameters.

DP-275329: Task termination change

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2021.1.0

In previous releases, tasks that were terminated by shutdown were marked as suspended. With this change, they are not marked as suspended by default. If you want tasks terminated by shutdown to be marked as suspended, set the new SuspendTerminated property.

DP-400051: Production export includes custom base schemas

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

The Interoperability Production Code Export will now include custom base schemas if appropriate. The query EnumerateVDocSchemas now also returns the Base value.

DP-401894: Improve storage efficiency by performing kills in ascending order

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2021.1.0

This change increases the storage efficiency of the database following a KILL command — especially when purging large quantities of data from a database in ascending order. In the past programmers sometimes did KILLs in descending order when doing large purges of data. To improve storage efficiency, you should change such programs to do kills in ascending order as doing kills in ascending order will now almost always result in better efficiency and better performance. There should be less need to run the GCOMPACT utility to improve storage efficiency following large purges of data.

This change does cause a 128 byte increase in the shared memory allocation per global buffer. If you are using the 8k default buffer size, this is a 1.5% increase.

DP-401903: Update of the Japanese language model

NOTE: This item may require a change to code, configuration, or operation.

Category: NLP
Platforms: All
Version: 2021.1.0

For Japanese text, parentheses () and their content may get indexed differently from previous versions, depending on the use of Katakana and number of characters within. While doing so is not strictly required, users are recommended to reindex their NLP domains with Japanese text to ensure uniformity of the output.

DP-402073: Must recompile cubes to see detail listings

NOTE: This item may require a change to code, configuration, or operation.

Category: Business Intelligence
Platforms: All
Version: 2021.1.0

As part of a change to improve MDX query performance, this change stores certain cube-related data in a different location. This change requires that you recompile all cube classes that were created on a version prior to this change. If you do not recompile these classes, detail listings on those cubes will return no results.

DP-402085: OAuth — change to AddOct method

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Library
Platforms: All
Version: 2021.1.0

The rarely used method %OAuth2.JWKS.AddOct() no longer base64url decodes client secrets. If you make use of this method but still need to base64url decode the secrets they pass in, you should base64url decode it before calling the method.

DP-402912: Fix problem where errors within ZWRITE command are not raised to caller's error trap

NOTE: This item may require a change to code, configuration, or operation.

Category: ObjectScript
Platforms: All
Version: 2021.1.1

This change fixes a problem where errors within ZWRITE command are not raised to caller's error trap.

Now ZWRITE throws some errors that were ignored before, but they are serious show-stopper errors that should not happen in a normal environment.

DP-403547: OAuth server now returns 401 for invalid credentials instead of 200

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2021.1.0

Prior to this change, InterSystems IRIS returned a status of 200 whether or not the user login to authorize OAuth was successful. If the login wasn't successful, the login page was displayed again with status of 200. If the login was successful, the permissions page was displayed with status of 200.

InterSystems IRIS now returns a status of 401 when the user login was unsuccessful. Customers working with OAuth, especially those who implement custom OAuth login pages, should be aware of this change.

DP-404204: Upgrading containers and file ownership changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Cloud
Platforms: Cloud
Version: 2021.1.1

The volume mounted for durable %SYS must now be writeable by user 51773 (irisowner), because cloning into that directory now happens as irisowner instead of root. When it isn't writeable, startup will fail. The cloud distribution now uses a non-root install, not a traditional installation. All files are now owned by 51773:51773 (aka irisowner:irisowner). The installation directory has not changed (still /usr/irissys). 

Before upgrading, users (excluding those using ICM or IKO) MUST chown their durable %SYS directory ($ISC_DATA_DIRECTORY) so that all files are owned by irisowner, for example: 

chown -R 51773:51773 $ISC_DATA_DIRECTORY

This will most likely need to be done as root. IKO and ICM perform this ownership change automatically. 

For locked down images, the installation directory ($ISC_PACKAGE_INSTALLDIR) has changed from /home/irisowner/IRIS to /usr/irissys.

Upgrades with IRIS-lockeddown and IRISHealth-lockeddown between versions pre- and post- this change are not supported and will not succeed.

DP-404238: Limit use of PasswordHash to initial configuration

NOTE: This item may require a change to code, configuration, or operation.

Category: Cloud
Platforms: Cloud
Version: 2021.1.1

PasswordHash makes it easy to change system-wide passwords during the initial system configuration. The password change is effective before the instance is initialized. In previous releases, you could also use this feature to make system-wide password changes once the system was in use and no longer had the initial passwords. With this change, PasswordHash can only be used during the initial system configuration and cannot be used later to change passwords.

Specifically, PasswordHash will only be processed if the existing hash matches a known, fixed value used to set the initial password.

DP-404256: CSP pages not backed up by internal routine versioning

NOTE: This item may require a change to code, configuration, or operation.

Category: System
Platforms: All
Version: 2021.1.0

This is a change to an undocumented feature, routine versioning, that some customers may be using. If you are using routine versioning, with this change CSP pages will not be backed up to the ^rBACKUP global.

DP-404334: Quote $ZF() shell commands on Windows

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: Windows
Version: 2021.1.1

Windows shell commands must be enclosed in quotes if there are quoted arguments along with a quoted command path. Those quotes will now be added by $ZF(-1), $ZF(-2), and $ZF(-100,"SHELL").

DP-405034: %Net.SMTP checks server identities by default

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Library
Platforms: All
Version: 2021.1.1

This change fixes a bug where the SSLCheckServerIdentity property in %Net.SMTP defaulted to false, but was documented as defaulting to true. The property now defaults to true and is in alignment with documentation. This means that, when connecting to an SSL/TLS secured web server, %Net.SMTP will check that the certificate server name matches the DNS name used to connect to the server and fail if they don't match. This is the behavior specified in RFC 2818 section 3.1.

As a result, when connecting to an SSL/TLS secured web server, the default behavior of %Net.SMTP will now be to fail if the certificate server name does not match the DNS name used to connect to the server.

This change is unlikely to cause compatibility issues, but it is possible that when using %Net.SMTP to send messages you might have issues connecting to an SSL/TLS enabled server. If this happens, and you understand the security trade-offs, you can set SSLCheckServerIdentity to 0 to restore the previous behavior.

DP-407501: 32-bit processes and external language servers

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways - .NET
Platforms: All
Version: 2021.1.1

External language servers that do not have "Exec 32" checked will change from executing as a 32-bit process to executing as a 64-bit process when started from the Management Portal. This will only affect you if you are loading external assemblies compiled for x86 and did not check the Exec 32 box of the server definition.

DP-407729: FIX: NOT %INLIST and NOT IN disparity when there is a NULL in the list

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2021.1.3

This change makes the behavior of NOT %INLIST $listbuild(x,NULL,:z) consistent with the behavior of NOT IN (x, NULL, :z).

DP-409100: Run object destructors when a .Net application quits

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2021.1.3

Actions performed by an object's %OnClose method, such as unlocking a record in a persistent class, were not being performed in a timely manner for a client/server application using .Net, JDBC, etc.

DP-410050: Correct a rare bug on class initialization

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2021.1.3

This change corrects a very rare bug that could cause the wrong version of a class to be loaded into memory. This could lead to increased memory usage if class components are not being used. Otherwise, it only changes the time at which they are loaded into memory.

DP-413048: Remove Linux capability checking from InterSystems IRIS containers

NOTE: This item may require a change to code, configuration, or operation.

Category: Cloud
Platforms: Linux
Version: 2021.1.3

This change removes the no-longer-necessary Linux capability checking from InterSystems IRIS containers, to avoid incompatibilities with new versions of Docker.

This doesn't immediately affect ICM users because an older Docker version is set in Samples/defaults.json.

DP-413293: Zen Reports: do not allow use of $DATASOURCE URL parameter by default

NOTE: This item may require a change to code, configuration, or operation.

Category: Compatibility Features
Platforms: All
Version: 2021.1.3

Previously ZEN Reports would allow the report's datasource to be specified at runtime via the $DATASOURCE URL Parameter. If this is not defined, then we will use the DATASOURCE class parameter, and if that isn't defined we will use the ReportDefinition to generate the XML data. The ReportDefinition is the standard way to specify report data.

With this change, Zen Reports will no longer use the $DATASOURCE URL parameter by default. Developers are strongly encouraged to use the DATASOURCE class parameter to specify an external datasource if desired. If a report does require the $DATASOURCE URL, you can re-enable the previous behavior on a per-report or per-application basis by setting the parameter: 

Parameter USEURLDATASOURCE = 1;

in either the report class, or in the report's Application class. Setting this value is discouraged.

DP-413775: Change a few DataMove APIs

NOTE: This item may require a change to code, configuration, or operation.

Category: DataMove
Platforms: All
Version: 2021.1.3

Several of the DataMove API's have been changed to add a new first parameter "Name" in the DataMove.API class as follows: 

MapInitialize(Name as %String, Namespaces As %String)
MapGlobalsCreate(Name as %String, Namespace As %String, GblName As %String, ByRef Properties As %String)
MapGlobalsDelete(Name as %String, Namespace As %String, GblName As %String) As %Status
MapGlobalsModify(Name as %String, Namespace As %String, GblName As %String, ByRef Properties As %String)

Customer will need to update their scripts to add this new parameter.

DP-416562: Correct WebSocket handling of low-level errors

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP Server
Platforms: All
Version: 2021.1.3

This fix corrects a bug in the error handling for web socket connections. If an error was encountered before the CSP server dispatched to the WebSocket class, then the error would be ignored. Now the CSP server will call the Error() method in the WebSocket class. Similarly, for authentication failures, the server will call Login(). Note that by default, Login() just calls Error(). 

The websocket will also now abort the connection if OnPreServer() returns an error. Previously errors were ignored.

DP-416563: Do not allow navigation to %-CSP pages from the /csp/sys/oauth2/ application

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP Server
Platforms: All
Version: 2021.1.3

This change modifies the default configuration to disallow access to %-CSP pages from the /csp/sys/oauth2/ application.

DP-417168: Record Mapper and EDI Document Viewer file view permissions check

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.3

The Interoperability Record Mapper and EDI Document viewer management portal pages enforce holding the privilege %Ens_ViewFileSystem:USE before allowing the user to select a file using the file select dialog popup. This change adds the same check before a file is opened.

DP-10720: Remove unused environment variables

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2021.1.0

Some environment variables that were not used by SQL or the class compiler have been removed. These were not used by the product, but if you have referenced them, you should remove them from your application. As of this change, the following variables are no longer defined: 

envRoutineSizeGet
envUseOldJavaGeneratorGet
envUseJavaGenerator1Get
envUseJavaGenerator2Get
envUseJavaGenerator3Get
envUseSortGet
envTimeChangedGet
The following variables are still defined but return an error if you try to update the setting: 
envDefaultOHandleClassNameGet
envDefaultSerialStateClassNameGet
envDefaultStorageClassNameGet

DP-10835: New version of XEP

NOTE: This item may require a change to code, configuration, or operation.

Category: Language Bindings - Native
Platforms: All
Version: 2021.1.0

This change provides a new version of XEP support, which includes enhanced indexing and a new underlying client/server communication method. Consequently, you cannot use an XEP from a previous version of the product with this version as the server and you cannot use this version' XEP with a previous version as the server. In addition, there may be some minor code changes that you need to make to any code that called the previous version of XEP support. See XEP Requirements and Configuration for details. For example, there is the following change:

XEP updateObject() method — the updateObject() method will now throw an exception if the object to be updated does not exist in the extent of the class. If your application depends on updateObject reverting to insert when the object does not exist then you will need to update the application to catch the exception and properly invoke the store() method to insert a new object.

DP-10957: System defaults setting now checks values when specified

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

In previous releases, invalid values in the default system settings would not be checked until the values were applied in a production. This change ensures that system default settings are valid when specified. If you have invalid system default settings that were saved by a previous release, saving them in this release removes the values. Settings are validated by calling the IsValid() method of the appropriate class. If no IsValid() method is defined, then no validation is performed.

DP-11014+PLUS: Remove ability to connect to Caché from .NET Provider or ODBC

NOTE: This item may require a change to code, configuration, or operation.

Category: ODBC
Platforms: All
Version: 2021.1.0

In order to enable new features, neither the .NET Provider nor ODBC supports connections to Caché instances in this release. The .NET Provider can only connect to InterSystems IRIS instances version 2019.1 and later.

DP-11192: Error checking on timeout settings

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

This release adds error checking to the timeout settings in the production configuration page. If the requested timeout interval exceeds 1 hour, the page treats it as an error and does not allow the setting. Since it is extremely unlikely that you would want to delay the instance shutdown for an hour, this is most likely an error. However, if you really need such an extended time to wait for the production component to stop, you can still set shutdown to be longer than an hour using a script that calls an Ens.Director method.

DP-11269: Changes to TuneTable and TuneSchema defaults

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2021.1.0

With this change, the second argument of $SYSTEM.SQL.TuneTable and $SYSTEM.SQL.TuneSchema (update) now has a default=1.

Before this change, the default was 0 for update. Calling $SYSTEM.SQL.TuneTable(tablename) and $SYSTEM.SQL.TuneSchema(schemaname) without specifying a value for the second argument will result in a change of behavior from previous versions.

DP-11323: Make PEX callback methods abstract

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

The PEX class hierarchy has changes to reduce errors when users overload methods rather than override them. With this change, all methods that should be overridden by user code are marked as abstract. If you define a subclass and do not override these methods, you will get a compile error. Before this change, the compilation would succeed but the user callback code would not execute correct because the default code would be called instead. The documentation stated that these methods were abstract and needed to be overridden but this was not enforced by the code.

DP-11351: %Persistent IDENTIFIEDBY parameter marked as Deprecated

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Storage
Platforms: All
Version: 2021.1.0

The %Library.Persistent IDENTIFIEDBY parameter is deprecated. If you have used IDENTIFIEDBY to define relationships, you should replace it with a parent/child relationship.

DP-11557: XML VDoc character escape changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

In XML VDoc, when you use setValueAt() to set the value at a node, you can specify a value that consists of mixed content (that is, a value that consists of a mix of element and text nodes). This change ensures that if a < (left-angle) character appears without a closing XML /> or </, then the < is treated as normal text and is XML escaped (replaced by <).

Before this change, this would have been treated as mixed content and would have caused an error. If you have code to handle this expected error, you should examine this case.

See Using Mixed Content When Setting Paths.

DP-11573: SQL Import/Export wizard now restricts file extensions

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2021.1.0

With this change, the SQL Import/Export wizard ([System Explorer] -> [SQL] -> [Wizards] -> [Data Import/Export]) will only allow the user to import from or export to files on the server with an allowed extension. Allowed extensions are: .txt, .csv. Previously any file on the system could be used. When importing from or exporting to a file on the client system, filenames are not restricted.

DP-11630: Eliminate on unload behavior from svgWidgetProvider and lock down behavior of RemoveFile() call

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP
Platforms: All
Version: 2021.1.0

As of Chrome 80 synchronous HTTP events will no longer be honored as part of page exit processing. Previously this widget made use of such events to perform implicit server-side clean up of temporary files created during the PDF generation process. This feature is no longer supported.

All temp files used by this widget reside in the instanceHome/mgr/Temp directory, which is cleaned up periodically by normal maintenance processes. Applications that do not wish the temp files to linger until regular clean-up may explicitly remove individual files with the RemoveFile() method supplied by this widget. Note however, that RemoveFile() will only remove temp files created by this widget during the current CSP session and is not a generic utility.

DP-11796: PEX library changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

With this change, we use a different serialization library and we no longer use the jackson library. Consequently, your production settings should point to the intersystems-utils-3.1.0.jar provided and not to the jackson library. If your code references the jackson library, you must update it or install the jackson library.

DP-11827: 32-bit ODBC driver removed on UNIX®

NOTE: This item may require a change to code, configuration, or operation.

Category: ODBC
Platforms: UNIX®
Version: 2021.1.0

With this change release, the 32-bit ODBC drivers libirisodbcu35.so and odbcgatewayu.so have been removed on UNIX® systems. If you have used these drivers, you should use the 64-bit equivalents libirisodbcur6435.so and odbcgatewayur64.so.

If you are switching over to the 64-bit unixODBC client driver (libirisodbcu6435.so), you should also make sure you are using the 64-bit unixODBC driver manager and that your applications are built against 64-bit unixODBC. If your app or driver manager is 32-bit and is trying to use the 64-bit driver, this could cause unexpected problems. This change only impacts the 32-bit ODBC drivers built against the unixODBC driver manager. The 32-bit iODBC-based drivers are still included in the installation.

DP-11838: No longer modify existing CSP.ini for external web server

NOTE: This item may require a change to code, configuration, or operation.

Category: Web Gateway
Platforms: All
Version: 2021.1.0

Before this change, in environments where a third party web server is running remotely, the Web Gateway installer tried to modify some settings in CSP.ini which could result in unwanted changes in terms of service accessibility. A review of the configuration after each upgrade was always recommended.

With this change, the Web Gateway installer will not modify an existing CSP.ini in any way.

DP-11979: Web sockets with SharedConnection=1 can now time out

NOTE: This item may require a change to code, configuration, or operation.

Category: Web Gateway
Platforms: All
Version: 2021.1.0

In previous releases, a web socket connection with SharedConnection=1 would never time out. With this change, this connection will time out if it is inactive for the CSP session timeout.

DP-11994+PLUS: API Cleanup

NOTE: This item may require a change to code, configuration, or operation.

Category: JDBC
Platforms: All
Version: 2021.1.0

Typically, developers use the standard JDBC APIs, which are unchanged in this release, but some proprietary public classes and methods have moved or become more restricted in access permissions. These accessible proprietary classes and methods may have been used by customers in their applications, and will need to be updated.

For example, if your code is accessing directly the members of the ConnectionParameters class, you must update your code and use the getter and setter methods.

DP-12017: SELECT with INTO clause changes

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2021.1.0

This change modifies the behavior of an embedded SELECT statement with an INTO clause. If the execution of the SELECT statement results in SQLCODE=100, or in the case of a FETCH that results in SQLCODE=100, any INTO variables in the statement will be set to null ("") if they are defined. If the INTO variable was undefined prior to the execution of the SELECT, and the SELECT does not set the variable (or any previous FETCH does not set the variable for cursor selects), the variable will remain undefined. If the variable was defined prior to the SELECT, or the SQL SELECT or FETCH sets the variable, the variable will be set to "" when SQLCODE=100 or SQLCODE<0 is returned.

This change might result in some applications requiring updating in order to avoid application errors, or incorrect results returned.

DP-12116: Contents tab of Message Viewer now limits output for XML-enabled objects

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

To improve efficiency in displaying XML-enabled objects, large XML objects may not be fully serialized in the message viewer Contents tab. To see the fully serialized value, you must now use the Full Contents link.

An XML comment will be displayed if the contents are truncated.

DP-12215: GLOSTAT changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2021.1.0

As part of a project to improve the performance of counters, certain GLOSTAT statistics have been changed:

  • %SYS.ProcessQuery property 'DataBlockWrites' and the equivalent $zu(67,38,pid) now counts any database block queued for writing by this process, whereas it previously counted only blocks internally typed as data blocks. This more accurately reflects the write load induced by this process than before (without adding the unnecessary complication of an additional statistic).
  • The undocumented $zu(190,6,0) and $zu(190,6,7) functions to zero system statistics are removed and now throw <FUNCTION>.
  • The iris stat -p32 columns formerly labelled 'rfcnt' and 'bfhit' are now labelled 'phyrd' and 'wdqbk' and they reflect, for the process in that row, the number of database reads and database blocks queued for writing respectively.

DP-12268: Changes for the arbiter container with ISCAgent using nondefault port

NOTE: This item may require a change to code, configuration, or operation.

Category: Cloud
Platforms: All
Version: 2021.1.0

The arbiter is an independent system hosting ISCAgent for mirror members. By default, ISCAgent uses port 2188. Before this change, to specify a nondefault port, you used syntax like the following: 

$IRISSYS/startISCAgent.sh 2000
With this change, you must instead use -p as follows: 
$IRISSYS/startISCAgent.sh -p 2000
Using arguments other than -p will be ignored and the container will run with port 2188.

DP-12307: Asynchronous I/O for database writes on Linux and UNIX®

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: UNIX®
Version: 2021.1.0

With this change, the product on Linux uses asynchronous I/O for writes to database files, as it always has on all UNIX® and Windows platforms. This is coupled with automatic use of direct I/O instead of buffered I/O. This change optimizes the disk I/O characteristics for database files, allowing higher scaling on busy systems without compromising application responsiveness. Of course, synchronization still occurs at key points to guarantee data integrity.

This change may require attention in certain configurations as file system buffering may have allowed the product to perform acceptably with an under-configured database cache in previous versions. Check that database cache is sized appropriately on your system.

Additionally, the product on Linux and all UNIX® platforms now uses asynchronous I/O for writing to the write image journal to provide optimal I/O characteristics. This change, however, requires no special attention.

For details, see Buffered I/O vs. Direct I/O.

DP-12412+DP-13124: Better handling of X12 documents

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

There are several changes in how X12 documents are handled:

  • Change handling poorly formatted documents — improved handling of certain poorly formatted documents means that the issues with these documents are logged in the Event Log and then handled by sending an appropriate Reply document, rather than these documents resulting in the service failing and documents not being deleted. If you are relying on service failures to identify flaws in the X12 document, you should look at the Reply document instead.
  • Validation change — this change now includes code values in validations that were previously ignored. If you are using validation with the flags r, u, or t, then some errors that would have been missed in previous versions will now be caught. This change also adds a new segment, CTX-11, to the validation style schema for HIPAA_5010:999. This does not impact any code for setting the CTX-5() segment, but does mean that if a user has code for getting values from the CTX segments, and the document they are inspecting has a Business Unit Identifier, then that CTX segment will have to be accessed using path 2000().2100().CTX-11 instead of the 2000().2100().CTX-5() which is used for Segment Context. This makes no difference in either setting or getting values using the new style schema.
  • Change in allowed order of segments — this change allows segments to be in any order permitted by the standard rather than requiring them to be in the specific order as listed in the standard. This allows valid documents to pass validation where in previous releases, they would have failed. This change should not require any change in your code unless you were using the wrong property path to get the value from some segment to work around this incorrect behavior; in that case you need to switch to using the correct property paths.

DP-12524: Change default of SKIPMESSAGEHISTORY in Ens.BusinessProcess to true

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

With this change, if custom business process code accesses the internal %MessagesSent or %MessagesReceived array, then the class parameter SKIPMESSAGEHISTORY needs to be overwritten and set to 0.

This MessagesSent and MessagesReceived logging information is separate from the Ens.MessageHeader objects that are created for the requests sent and responses received by a business process. The Ens.MessageHeader objects are not controlled by the SKIPMESSAGEHISTORY parameter.

DP-12548: Modify string handling in Java Native and IRISList

NOTE: This item may require a change to code, configuration, or operation.

Category: Language Bindings - Native
Platforms: All
Version: 2021.1.0

This change modifies how the Java Native API handles strings in method and function calls, get() methods, and IRISList.

  • An empty string will not be converted to $c(0) and back. Empty string will be empty string.
  • An attempt to use empty string or empty byte[] as subscript will generate <SUBSCRIPT> error.
  • No changes in setting null. set(null,) and set ("",) both will be resulted in empty string on server.
  • Getting undefined value will result in null.
  • get any numeric (int, double) value from "" return 0. It will return null for Methods/Functions/
  • getByte from "" return empty byte[0], getString from "" will result in empty string.
  • getObject from type 1 will result in byte[0] in both - IRIS and IRISList.
  • getDateTime from empty string will return null, before it was throwing an exception.
  • getIRISList will return a valid object of length==0 for values 02 01, null object for 01.
  • Iterator will return empty string if subscript is not found, before null was returned.

DP-12559: Activity monitoring changes

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2021.1.0

The activity monitoring pages now distinguish between days based on the combination of UTC and local time. This means when querying the Ens_Activity_Data.Days table for a specific TimeSlotUTC or TimeSlot, it is necessary to check if there are 2 rows per unique TimeSlotUTC or TimeSlot: One row when TimeSlot is different from TimeSlotUTC and one row when TimeSlot is the same as TimeSlotUTC. The actual value is the sum of these two rows.

DP-12577: Memory usage changes for global and routine buffers

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: UNIX®,macOS
Version: 2021.1.0

This change improves performance on newly installed systems where the database cache size has not been configured. Under most circumstances, you should carefully configure cache sizes and plan your memory management for optimal system performance. Configuring cache sizes is especially important for live production systems, systems with heavy loads, and systems with multiple instances.

However, when users initially evaluate the product, they typically run it without configuring the database cache size. The new behavior in allocating memory on systems where cache size has not been configured is:

  • When database cache (global buffers) is unspecified in the configuration, the instance auto-selects 25% of total physical memory for database cache. Previously, the calculation used 12.5% with an upper limit of 1GB (so effectively a flat 1GB for most systems). There is now no upper limit imposed. Additionally, this changes the calculation on Windows systems to be based on physical memory rather than "available" memory so that the behavior is consistent across restarts (and also consistent with Linux and UNIX® platforms).

    While this may result in a large allocation, if the instance isn't used to load a large amount of data, then most of this memory is not touched and, in most operating system environments, will not occupy physical memory. If the operating system has enough large pages available for this memory allocation, InterSystems IRIS will use them, and thus will occupy physical memory. In this case, however, that large page memory is known to be available. Control over use of large pages is available via the memlock configuration parameter.

  • On startup, if database cache is unspecified, a message "Global buffer setting requires attention. Auto-selected 25% of total memory." will be emitted.
  • Leaving routine cache (routine buffers) unspecified in the configuration no longer means a minimal amount routine cache (36MB) but instead is taken as 10% of the amount of global buffers with a minimum of 80MB and maximum of 1020MB. (More precisely, it's the number of global buffers multiplied by 8KB multiplie by 10%, so the simple 10% applies as long as only 8KB-size global buffers are used). This applies regardless of whether the database cache was configured or auto-selected as described above. In this way, most systems may be able to select only database cache size and leave routine cache size unaltered.

    • See Memory Management and Scaling for InterSystems IRIS for more information about memory management.

    DP-12603: Corrections to datetime functions to disallow invalid input

    NOTE: This item may require a change to code, configuration, or operation.

    Category: SQL
    Platforms: All
    Version: 2021.1.0

    Before this change, some invalid inputs to datetime functions would not cause an error but would return a value. Now these errors are caught and returned as errors. If your code does not handle errors in these functions, you should add error handling.

    DP-12609: Changes to creating interoperability namespace with %Installer

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Interoperability
    Platforms: All
    Version: 2021.1.0

    Before this change, if you used the %Installer.Installer class to make an interoperability production enabled namespace, it would set the node to a fixed portal URL. It now sets it to the empty string, which is the typically desired behavior. If you wish to set the node to a fixed portal URL, you can do this with the following: 

    Set ^%SYS("Ensemble","InstalledNamespace",tNSUpper)=""

    DP-12631: JDBCGatewayPort change

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Cloud
    Platforms: All
    Version: 2021.1.0

    If you were overriding the JDBCGatewayPort parameter using the JSON field "JDBCGatewayPort", you should delete that parameter and instead override the parameter directly in the custom CPF file (pointed to by "UserCPF"). For example: 

    [Gateways]
%JDBC Server=JDBC,127.0.0.1,53773

    DP-12694: RecordMap error handling changes

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Interoperability
    Platforms: All
    Version: 2021.1.0

    The error detection for the RecordMap service is now stricter with the ParseOnly flag selected. Before this change, errors would be ignored with ParseOnly even if the AlertOnError flag was enabled. With this change, if the AlertOnError flag is enabled and a record fails validation, then an alert is sent even if ParseOnly is selected.

    This change also corrects the situation when previously no batch would be sent if the FatalErrors setting was equal to ParseOnly and the last record failed to save and there was at least one good record in the batch.

    DP-12727: X12 schema changes

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Interoperability
    Platforms: All
    Version: 2021.1.0

    This release includes additional keyfields in X12 schemas. In some places, unnecessary schemas or incorrect loops have been removed. If your code relies on the existence of these incorrect elements of the schema, you must revise it.

    DP-12747: SSLCC_Protocol parameter changes for TLS 1.3

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Web Gateway
    Platforms: All
    Version: 2021.1.0

    To allow for TLS 1.3, there are changes to the way you access the SSLCC_Protocol parameter. If you are using an external script or the Web Gateway registry methods (particularly %CSP.Mgr.GatewayMgr.GetServerParams or %CSP.Mgr.GatewayMgr.SetServerParams) to read/write the SSLCC_Protocol parameter in CSP.ini, you must adapt your code to use the SSLCC_Protocol_Min and SSLCC_Protocol_Max parameters instead.

    DP-12762: Newly supported connection pooling for JDBC

    NOTE: This item may require a change to code, configuration, or operation.

    Category: JDBC
    Platforms: All
    Version: 2021.1.0

    Before this change, connection pooling was not a supported feature. This change a major revision to the connection pooling code and API, and this feature is now fully supported. If your code called the previous unsupported version, you should update it to ensure that it reflects the revised implementation.

    DP-12897: Method signature changes in OAuth2.Server.Session

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Security
    Platforms: All
    Version: 2021.1.0

    This change includes a change to the signatures of the GetUser() and Login() methods of the OAuth2.Server.Session class. If you have manually subclassed this class and overloaded these methods, you should be aware that the system may attempt to pass them one additional (string) parameter when dispatching to user code from the core methods GetUser and Login (respectively) of OAuth2/Server/Auth

    DP-12945: .NET version change

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Interoperability
    Platforms: All
    Version: 2021.1.0

    The Interoperability .NET Gateway now supports Core 2.1 version and the default value is now 4.5 instead of 2.0. If your application is dependent on the default value and needs the 2.0 version, you must explicitly specify it.

    DP-13043: Remove support for H2OSA provider

    NOTE: This item may require a change to code, configuration, or operation.

    Category: IntegratedML
    Platforms: All
    Version: 2021.1.0

    The H2OSA provider is no longer supported. If you have used it with a previous release, you must switch to a different provider.

    DP-13047: Support FIPS mode for libcrypto.so.1.1 in Red Hat Linux 8.2

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Kernel
    Platforms: Linux
    Version: 2021.1.0

    On Red Hat Linux, you can only use libcrypto.so.1.1 FIPS mode on Red Hat Linux version 8.2 and later. You cannot use this mode on earlier versions.

    DP-13055: %ETN could change null subscript setting

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Kernel
    Platforms: All
    Version: 2021.1.0

    In this release, the $ZREFERENCE variable, which contains the last global referenced, can have a different value after running %ETN error trap utility than it had in the same context in previous releases. If your code is parsing the $ZREFERENCE value to determine where the error information was written, you must modify your code to determine this information using the return value from LOG^%ETN or BACK^%ETN, which is a $LIST of the first two subscripts in the ^ERRORS global.

    DP-13063: Change default superserver port

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Cloud
    Platforms: All
    Version: 2021.1.0

    In this and future releases, the default SuperServer port is 1972. If this port is not available, then the default port will be 51773 or the next free port after this. This change was first in release 2020.3. In releases before 2020.3, the default SuperServer port was 51773.

    If you are upgrading a system using the InterSystems Cloud Manager (ICM) and relied upon 51773, you may need to add the following override to your JSON defaults: 

    "SuperServerPort": "51773"

    Note:If you are upgrading an instance of an earlier release with full installation kits on a server platform, this change does not impact you. The SuperServer port from the instance is preserved through the upgrade.

    DP-13164: Change to IRISCallableStatement.getBytes(String)

    NOTE: This item may require a change to code, configuration, or operation.

    Category: JDBC
    Platforms: All
    Version: 2021.1.0

    To improve consistency between IRISCallableStatement.getBytes(String) and IRISCallableStatement.getBytes(Int), the behavior has changes. If you are calling IRISCallableStatement.getBytes(parameterName) and depend on the details of the previous behavior, you should replace the method call with a call to outputParameterList.getByteArray(parameters.getListOffset(parameterName)).

    DP-13182: Sites using Nginx must rebuild and reconfigure

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Web Gateway
    Platforms: UNIX®
    Version: 2021.1.0

    If you use Nginx with the Web Gateway you must follow these steps to rebuild Nginx. If you do not use Nginx with the Web Gateway, you can ignore this issue.

    If you are using Nginx with the default dynamic modules configuration (using CSPx.so and CSPxSys.so), note that we are now deprecating this configuration. The dynamic modules configuration will still function as before, but if you use the Web Gateway with Nginx you should rebuild and reconfigure Nginx to work with the Network Service Daemon (NSD) build of the Web Gateway instead. The NSD now supports WebSockets when used with Nginx, you can now migrate to the NSD even if you have a WebSocket application.

    If you are already using Nginx with the NSD, you must still rebuild Nginx and edit its configuration to apply this change.

    The steps you must follow are different if you are already using NSD or if you are migrating from dynamic module configuration.

    If you are already using NSD:

    1. Install an updated version of the Web Gateway.
    2. Rebuild Nginx using the updated version of ngx_http_csp_module.c. See Nginx Web Servers for instructions on how to do this.
    3. Add the CSPNSD_pass directive to the Nginx configuration file to indicate the hostname/IP and port where the NSD is listening. For example, if your configuration file contains:

      location /csp {
    CSP On;
}
      and the NSD is listening at 127.0.0.1:7038 (the default), then change the configuration block to: 
      location /csp {
    CSP On;
    CSPNSD_pass 127.0.0.1:7038;
}
    4. Start up Nginx and the NSD.

    If you are migrating to using Nginx with NSD from using Nginx with dynamic modules (CSPx.so, CSPxSys.so):

    1. Install an updated version of the Web Gateway.
    2. Rebuild Nginx using ngx_http_csp_module.c instead of ngx_http_csp_module_sa.c. See Nginx Web Servers for instructions on how to do this.
    3. Remove the CSPModulePath directive from the Nginx configuration.
    4. Add the CSPNSD_pass directive to the Nginx configuration file to indicate the hostname/IP and port where the NSD is listening. For example, if your configuration file contains:

      location /csp {
    CSP On;
}
      and the NSD is listening at 127.0.0.1:7038 (the default), then change the configuration block to: 
      location /csp {
    CSP On;
    CSPNSD_pass 127.0.0.1:7038;
}
    5. Start up Nginx and the NSD. See Using the Network Service Daemon (NSD) and Using the NSD on UNIX, Linux, macOS for more information on how to manage the NSD.

    This change also adds a few additional Nginx configuration directives that you should be aware of when migrating, and you should review whether their default values are acceptable for your workloads. If the default values are not acceptable, then specify your own value in the proper location{} block in the Nginx configuration file. These directives are:

    • CSPNSD_response_headers_maxsize -- The maximum size of the HTTP headers in a response. An error is returned to the web client if a response header exceeds this size.
      • Syntax: CSPNSD_response_headers_maxsize size;
      • Default value: 8k
      • Context: location, if in location
    • CSPNSD_connect_timeout -- Timeout for connecting to the NSD when a request is received from the web client.
      • Syntax: CSPNSD_connect_timeout time;
      • Default value: 300s
      • Context: location, if in location
    • CSPNSD_send_timeout -- Timeout for a single send operation to the NSD when sending a request. The timeout is set only between two successive write operations, not for the transmission of the whole request.
      • Syntax: CSPNSD_send_timeout time;
      • Default value: 300s
      • Context: location, if in location
    • CSPNSD_read_timeout -- Timeout for a single read operation from the NSD when reading a response. The timeout is set only between two successive read operations, ot for the transmission of the whole response.
      • Syntax: CSPNSD_read_timeout time;
      • Default value: 300s
      • Context: location, if in location

    An example configuration that uses all the new directives is the following: 

    location /csp {
    CSP On;
    CSPNSD_pass 127.0.0.1:7038;
    CSPNSD_response_headers_maxsize 8k;
    CSPNSD_connect_timeout 300s;
    CSPNSD_send_timeout 300s;
    CSPNSD_read_timeout 300s;
}

    DP-13195: Remove extern symbol zfedll from include files

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Kernel
    Platforms: UNIX®
    Version: 2021.1.0

    There may be an unexpected incompatibility with any DLL that contains a source file that includes any of the following 4 statements: 

    #include iris-cdzf.h
#include iris-callin.h
#include cdzf.h
#include callin.h
    Including any of these #include files formerly defined a extern symbol with the name zfedll. The purpose of this symbol is not documented and it is assumed that no other code in the DLL will reference this symbol. The extern symbol zfedll is no longer defined and any DLL that makes an unauthorized reference to the zfedll symbol will now get a link-time error reporting that the symbol is not defined.

    DP-13358: Re-implement UNIX XDEVshm

    NOTE: This item may require a change to code, configuration, or operation.

    Category: xDBC
    Platforms: UNIX®
    Version: 2021.1.0

    In order to fix problems in the previous implementation, the UNIX XDEVshm code has been rewritten. Although the basic functionality is unchanged, there may be minor changes in behavior. Consequently, if you use XDEVshm on the UNIX platform, you should test your code to ensure that it is not sensitive to these minor changes.

    DP-13459: Studio now uses web server configuration from client registry

    NOTE: This item may require a change to code, configuration, or operation.

    Category: IDEs
    Platforms: All
    Version: 2021.1.0

    In previous releases, Studio ignored the web server configuration from the client registry and always queried the server for this information. With this change, Studio first queries the client registry for WebServerAddress, WebServerPort and WebServerInstanceName. If these are not defined in the client registry, Studio will query the server.

    However, if these are defined in the client registry but have incorrect values, you must correct them for Studio to function correctly.

    DP-13501: Enforce valid cube name in cube compile

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Business Intelligence
    Platforms: All
    Version: 2021.1.0

    Before this change, cube names that violated the documented rules could be successfully compiled though they may have caused other errors. With this change, these invalid cube names cause a compile error and must be corrected to successfully compile the cube.

    DP-13514: Replace legacy terminology

    NOTE: This item may require a change to code, configuration, or operation.

    Category: NLP
    Platforms: All
    Version: 2021.1.0

    This release adopts new terminology replacing master/slave with head/tail and BlackLists with SkipLists. In most cases, existing API calls will continue to work though you can update to the new terminology, but, if your code is explicitly checking for specific names in column heads or class reference entries or if you are using the Web Service projections of the %iKnow query APIs, you will need to adjust your code to account for these changes.

    DP-13552: Handle string values in $SYSTEM.Event.Wait()

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Kernel
    Platforms: All
    Version: 2021.1.0

    In previous releases, a $SYSTEM.Event.Wait() with a string value would cause no timeout. With this change, the string is converted to an integer and that is used as the timeout value. If the string is a nonnumeric string, it is treated as a zero timeout.

    DP-13698: Changes to TCP TLS in FIPSMode

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Kernel
    Platforms: All
    Version: 2021.1.0

    When FIPS mode is enabled, TCP TLS will use operating system's libraries and the TLS1.1 and earlier protocols won't be supported. This change should not impact existing code but, if your existing TCP TLS code uses FIPSMode, you should test it to ensure that there are no subtle behavior changes.

    DP-13700: $SYSTEM.SQL not using SQLCODE in some contexts

    NOTE: This item may require a change to code, configuration, or operation.

    Category: SQL
    Platforms: All
    Version: 2021.1.0

    The error information formerly provided in SQLCODE from $SYSTEM.SQL.Statement ClearAll(), ClearRelation(), ClearSchema(), ClearStatement(), FreezeAll(), FreezeSchema(), FreezeRelation(), FreezeStatement(), UnfreezeAll(), UnfreezeSchema(), UnfreezeRelation(), and UnfreezeStatement() is now included in the return status. The error information provided by SQLCODE from $SYSTEM.SQL FreezePlans() and ClearStatistics() can be found in the Errors parameter. SQLCODE has been removed from the PublicList in these contexts.

    DP-13734: Handle gateway timeout correctly

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Web Gateway
    Platforms: All
    Version: 2021.1.0

    When the Gateway times out, the associated InterSystems IRIS processes should terminate. In some cases, this would not happen and the processes would continue. With this change, these processes are terminated correctly. If your application depends on this behavior to continue past this timeout, you should adapt your code to handle this situation.

    DP-13783: ZEN: Very large integers now serialized as JSON strings

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Compatibility Features
    Platforms: All
    Version: 2021.1.0

    To avoid problems with languages such as JavaScript that cannot use double precision floating point numbers to store integers and cannot handle a value greater than 2^53-1, these values are now serialized as JSON quoted strings. The behavior with values less than 2^53 is unchanged.

    DP-13850: Map selectability changes for mapped tables

    NOTE: This item may require a change to code, configuration, or operation.

    Category: SQL
    Platforms: All
    Version: 2021.1.0

    Before this change, if you set map selectability for a table definition that was mapped into another namespace, the map selectability setting was changed for the table definition in all namespaces. With this change, you only set the map selectivity for a single namespace.

    DP-13874: Return HTTP error codes for redirects to custom error pages

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Web Gateway
    Platforms: All
    Version: 2021.1.0

    This changes the status code returned when Web Gateway redirects to a custom error page. If your web application expects HTTP status 200 for these custom error redirections, you need to update it to handle 5xx codes.

    DP-13937: Remove truncation to 150 characters on INFORMATION.SCHEMA.STATEMENTS:Statement column

    NOTE: This item may require a change to code, configuration, or operation.

    Category: SQL
    Platforms: All
    Version: 2021.1.0

    In previous releases, an ORDER BY STATEMENT on INFORMATION.SCHEMA.STATEMENTS:Statement column was automatically truncated to 150 characters. This automatic truncation has been removed to allow you to use more characters. However, in some cases having no truncation can cause a subscript error. To avoid this you should explicitly specify a truncation that will catch significant data, but will not be unlimited.

    DP-14014: Must recompile cubes that invoke UPDATEFACTSTEMP

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Business Intelligence
    Platforms: All
    Version: 2021.1.0

    If you have any cubes that invoke the UPDATEFACTSTEMP task, you must recompile them before the next build or background synchronization.

    DP-20560: Deprecate the 'l' flag and /lock qualifier in class compilation

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Object Compiler
    Platforms: All
    Version: 2021.1.0

    With this change, the class compiler always uses a lock when compiling or loading to ensure that the operation is not interrupted by other events; consequently, the 'l' flag and the /lock qualifier are deprecated and will be ignored. This change should not impact any existing user code or procedures other than by avoiding errors in the operation.

    DP-20566: Improve error handling in relationship queries

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Object Compiler
    Platforms: All
    Version: 2021.1.0

    This release improves the class compiler error handling of SQL relationship queries. Some errors that were not reported are now reported; consequently applications may behave differently when compiled or executed. But these new error messages can identify errors in relationship queries that should be fixed to achieve the intended results.

    DP-9922+PLUS: IRISNative and Gateway Changes

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Language Bindings - Object
    Platforms: All
    Version: 2021.1.0

    This release contains the following changes to the language bindings IRISNative API and to the gateways:

    • Access to the IRISReference class — you now have to use the accessor methods getValue() and setValue() to access the value. Previously, you could access the value directly
    • Object returned as IRISList — if a user method returns an object of IRISList, the InterSystems IRIS side now gets a $list literal. Previously, the InterSystems IRIS side would get a proxy object of %Net.Remote.Object. If your code is expecting an object of %Net.Remote.Object, you need to change it. There were significant problems including inefficiency with the previous behavior.
    • Private classes — in this release you cannot access private Java or .NET classes using public properties or methods. You must change the private classes to be public in Java or .NET if you want to continue using those properties and methods. This change fixes some errors and inconsistencies accessing private classes with public properties or methods.
    • %Net.Remote.Object property name change — in this release the %Net.Remote.Object internal Gateway property is renamed to %gateway. This is an internal property and should not be used, but if your code accesses it, you should change it to use the new name.
    • Empty strings — in this release, the language bindings map an empty string in the external language to an empty string in InterSystems IRIS. In previous releases, these were mapped to $c(0).

    DPP-847: Removal of deprecated Java Business Hosts -- use PEX instead

    NOTE: This item may require a change to code, configuration, or operation.

    Category: Interoperability
    Platforms: All
    Version: 2021.1.0

    This change removes Java Business Hosts, a feature that was deprecated in release 2019.3. PEX replaces the Java Business Hosts feature. For information on migrating existing Java Business Hosts productions to use PEX, see the community article Migrate from Java Business Host to PEX. For a description of PEX, see PEX: Developing Productions with Java and .NET.

    MAK5205: Changes to when SQL statement indexes are Removed

    NOTE: This item may require a change to code, configuration, or operation.

    Category: SQL.Query Processing
    Platforms: All
    Version: 2021.1.0

    Previously when all routines/classes that contained a SQL statement were removed, the statement index entry was also removed. This meant that all lightweight SQL statistics were also removed when the routine/class that contained the SQL statement was removed. It is often very useful to have access to this historic information and in addition if the statement was frozen it was not removed, so this change modifies the behavior to always keep the SQL statement index entry (and lightweight SQL statistics) when the containing routine/class is deleted.

    A new button has also been added to SQL statement index page in the system management portal called 'Clean stale' which will run a task to remove all statement index entries that are not frozen where the containing routine/class is no longer present or no longer contains this SQL statement.

    This change does not impact how your code is executed, but in order to remove the index entries, you have to use the Management Portal.

    SDK048: SQL Import/Export wizard now restricts file extensions

    NOTE: This item may require a change to code, configuration, or operation.

    Category: SQL
    Platforms: All
    Version: 2021.1.0

    With this change, the SQL Import/Export wizard ([System Explorer] -> [SQL] -> [Wizards] -> [Data Import/Export]) will only allow the user to import from or export to files on the server with an allowed extension. Allowed extensions are: .txt, .csv. Previously any file on the system could be used. When importing from or exporting to a file on the client system, filenames are not restricted.

    SGM031: Support SameSite for CSP session and user cookies

    NOTE: This item may require a change to code, configuration, or operation.

    Category: CSP
    Platforms: All
    Version: 2021.1.0

    When a server sets an HTTP(S) cookie, it can include an attribute called SameSite that affects the scope in which the cookie is sent. The options are as follows:

    • None - Send cookie with cross-site requests.
    • Lax - Send cookie with safe, top-level cross-site navigation.
    • Strict - Do not send cookie with cross-site requests.

    Recent updates to browser security have changed handling of third-party cookies. These updates use the SameSite attribute to reduce the risk of cross-site request forgery (CSRF) attacks, unauthorized access to data, and other possible security issues. Chrome (starting with v.84) enforces stricter rules for SameSite behavior, and these rules can cause issues with existing websites and web applications. These issues may include login problems, the login page being displayed repeatedly, and page elements not displaying properly.

    Before this change, it was not possible to modify the SameSite attribute, and web applications running on these versions may have such issues. This change allows you specify default values for the SameSite attribute, which will thereby prevent third-party cookie issues form arising.

    With this change, the product will explicitly set SameSite both for CSP session cookies and any cookie set via the %CSP.Response.SetCookie() method.

      Two new fields have been added for customizing web applications:
    • Session Cookie Scope
    • User Cookie Scope

      • These fields are displayed in Management Portal's Edit Web Application window, alongside "Use Cookie for Session" and "Session Cookie Path."

      Session Cookie Scope controls the default SameSite value for session cookies associated with the given web application. User Cookie Scope controls the default SameSite value for user-defined cookies created with %CSP.Response.SetCookie(). Session Cookie Scope and User Cookie Scope can be set to None, Lax, or Strict. System web applications and new or upgraded user applications default to Strict for both new fields.

      The %CSP.Response.SetCookie() method also accepts SameSite as an argument, which overrides the default value. You can use this new argument to exercise more fine-grained control over your cookies.

      You should configure grouped applications to use the same Session Cookie Scope setting. Additionally, if you are using "SameSite=None", the web application needs to support secure (HTTPS) connections.

      TRW1706: Collation changes that can impact query processing

      NOTE: This item may require a change to code, configuration, or operation.

      Category: SQL.Query Processing
      Platforms: All
      Version: 2021.1.0

      Concatenations of expressions that are coercible to a common collation (for example, string constants or parameters concatenated with commonly collated fields) will now be collated as the common collation. This change could lead to conditions that previously evaluated to false for certain values now evaluate to true. For example, the following condition will evaluate to true: 

      ' ' || field LIKE 'A'

      where field has a value 'a'. It is unlikely that any code is dependent on having a false value for this condition.

      Upgrade Checklist (Administrators)

      DP-13887: FIPS mode must match operating system FIPS mode

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Security
      Platforms: All
      Version: 2021.1.0

      If you attempt to start InterSystems IRIS in FIPS mode, the operating system must support FIPS mode and have been booted with FIPS mode enabled. Otherwise, InterSystems IRIS will not start and you must either enable FIPS mode in the operating system or not enable it in InterSystems IRIS.

      DP-13951: New metrics in %Monitor.Process

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Monitoring
      Platforms: All
      Version: 2021.1.0

      This change adds new metrics from %Monitor.Process, and the counter index numbers have changed. The new metrics are MONBLKWAIT, MONBLKWAKE, MONMEMALLOC, and MONMEMFREE.

      DP-401580: New External Language Servers page replaces gateway pages

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Management Portal
      Platforms: All
      Version: 2021.1.0

      In the management portal, the new External Language Servers page replaces the following pages:

      • Object Gateways
      • JDBC Gateways
      • XSLT Gateways

      To access the new page, click System Administration > Configuration > Connectivity > External Language Servers. For details on the new page, see Managing External Server Connections.

      DP-403425: Installer leaves duplicate ODBC driver

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Installation
      Platforms: Windows
      Version: 2021.1.0

      To allow both Caché and InterSystems IRIS to both exist on a system, the installer should not remove the Caché ODBC driver. On Windows, a new InterSystems IRIS install may result in a duplicate entry for "IRIS ODBC Driver" in add/remove programs applet. You can safely uninstall the older "IRIS ODBC Driver".

      DP-404283: Changes to enabling %ZEN.Dialog classes

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Security
      Platforms: All
      Version: 2021.1.1

      In this release, %ZEN.Dialog.* classes cannot be run in web applications unless the web application is enabled for analytics, the namespace is enabled for interoperability productions, or it is explicitly enabled for the web application. To explicitly enable %ZEN.Dialog.* classes, enter the following: 

      Set ^SYS("Security","CSP","AllowPrefix",application-name,"%ZEN.Dialog.")=1

      Also, with this change, any web application that is enabled for analytics or interoperability will have %ZEN.Dialog.* classes enabled by default.

      DP-407254: Update interaction between delegated and password authentication

      NOTE: This item may require a change to code, configuration, or operation.

      Category: System
      Platforms: All
      Version: 2021.1.1

      This fixes an issue with authentication ordering when delegated and password authentication are enabled. If your user authentication includes both delegated authentication and password authentication, you can choose either to call or not call ZAUTHENTICATE for password users. Typically, ZAUTHENTICATE is only used for delegated users and not for password users.

      But if you want your ZAUTHENTICATE routine to also be called for password users, you should check the <b>Always try Delegated Authentication</b> in the <b>System Administration > Security > System Security > Authentication/Web Sessions Options</b> page in the Management Portal or by using the ^SECURITY utility (System Parameter Setup, Edit authentication options).

      The default for <b>Always try Delegated Authentication</b> is No and ZAUTHENTICATE is not called for password users.

      DP-408131: Accessing install directory on Windows from ODBC requires additional privileges

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Installation
      Platforms: Windows
      Version: 2021.1.3

      Users in "Authenticated Users" group no longer are able to write files into the installation directory in locked down install without additional privileges.

      DP-413960: Limit PERFMON collection (index) numbers

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Monitoring
      Platforms: All
      Version: 2021.1.3

      Limit all PERFMON collections to 64k. Before this change, use of collections >65535 would silently fail; now an error is returned.

      MIRMONTASK-2021: Removal of Mirror Monitor Launch Task

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Management Portal
      Platforms: All
      Version: 2021.1.0

      In the Task Manager, the Mirror Monitor Launch Task task has been removed. This task is applicable only for HealthShare installations and is not needed for InterSystems IRIS for Health.

      TLS-2021: Removal of TLS 1.3

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Security
      Platforms: All
      Version: 2021.1.0

      InterSystems IRIS 2020.2 through 2020.4 contained the new TLS 1.3 security libraries. On some platforms, we discovered that there were conflicts with older versions of the libraries installed by the operating system or other applications. Consequently, we have removed TLS 1.3 from this release. We will include TLS 1.3 support in a future release in a way that will not conflict with older versions of the library that are installed on the system.

      DP-11245: Passwords now use SHA-2 hashes

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Security
      Platforms: All
      Version: 2021.1.0

      With this change, the product uses SHA-2 hashes for passwords instead of SHA-1; consequently, you cannot export a user with an SHA-2 password hash and import it into an earlier version of the product that does not use SHA-2 hashes for passwords. You can import a user with an SHA-1 password hash into this release or any release that supports SHA-2 hashes.

      DP-11381: Remove _PUBLIC ownership from generated listing tables

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.0

      Listing tables are now generated so that they require explicitly assigned SELECT privileges.

      For field list listings in a given cube, this will mean users must have SELECT privileges on both the source table for the cube and the generated <cube class>.Listing table for that cube.

      For custom listings, SELECT privileges of user tables might be required to run the custom listing. If the custom listing uses the special $$$RESTRICT token, SELECT privileges on the <cube class>.Listing table will also be required.

      DP-12141: Cannot use old versions of cvendian and cvencrypt

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Security
      Platforms: All
      Version: 2021.1.0

      The maximum length of encryption ID is now increased from 63 to 64, and the encryption ID should not contain wide characters.

      As a result, you cannot use cvendian and cvencrypt utilities from previous versions on encrypted databases. Use the versions of cvendian and cvencrypt provided with this release.

      DP-12790: Restore default superserver port number to 1972

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Installation
      Platforms: UNIX®
      Version: 2021.1.0

      In this and future releases, the default SuperServer port is 1972. If this port is not available, then the default port will be 51773 or the next free port after this. In previous releases, the default SuperServer port was 51773.

      Note: If you are upgrading an instance of an earlier release with full installation kits on a server platform, this change does not impact you. The SuperServer port from the instance is preserved through the upgrade.

      Fixed Security Issues

      DP-282954

      CWE: CWE-208: Observable Timing Discrepancy
      Severity: Medium
      CVSS Score: 5.3
      Version: 2021.1.3

      FIXED: An unauthenticated actor may have been able to use a timing discrepancy to determine existing usernames for the instance.

      DP-406999

      CWE: CWE-20: Improper Input Validation
      Severity: Medium
      CVSS Score: 6.3
      Version: 2021.1.3

      FIXED: Due to improper input validation, a user with access to terminal may be able to execute arbitrary commands with %All privilege.

      DP-408556

      CWE: CWE-287: Improper Authentication
      Severity: Medium
      CVSS Score: 5.3
      Version: 2021.1.3

      FIXED: SSL/TLS clients performing hostname checking may send some of the message payload contents before the hostname check occurs, potentially disclosing information to a server that would later fail the hostname check.

      DP-415197

      CWE: CWE-84: Improper Neutralization of Encoded URI Schemes in a Web Page
      Severity: Medium
      CVSS Score: 4.3
      Version: 2021.1.3

      FIXED: An attacker may be able to craft a malicious link to the class reference documentation application to execute arbitrary JavaScript.

      DP-416493

      CWE: CWE-250: Execution with Unnecessary Privileges
      Severity: Medium
      CVSS Score: 6.3
      Version: 2021.1.3

      FIXED: Due to a code path that executes with unnecessary privileges, a user with developer privileges may be able to execute arbitrary commands with %All privilege.

      DP-416789

      CWE: CWE-459: Incomplete Cleanup
      Severity: Medium
      CVSS Score: 6.3
      Version: 2021.1.3

      FIXED: In rare circumstances, a user may be able to use administrative options such as NOCHECK, %NOLOCK, %NOINDEX, %NOTRIGGER, or %NOJOURN on SQL queries that they are otherwise authorized to run without those options.

      DP-417169

      CWE: CWE-862: Missing Authorization
      Severity: Medium
      CVSS Score: 4.3
      Version: 2021.1.3

      FIXED: An authenticated user would be able to use particular Ensemble/Interoperability application pages to view the file system contents and files themselves without proper permissions.

      DP-417272

      CWE: CWE-285: Improper Authorization
      Severity: Medium
      CVSS Score: 4.2
      Version: 2021.1.3

      FIXED: In rare circumstances, a user may be able to run a cached query for which they did not have privileges.

      DP-417313

      CWE: CWE-200: Exposure of Sensitive Information to an Unauthorized Actor
      Severity: Medium
      CVSS Score: 4.3
      Version: 2021.1.3

      FIXED: JWKS values could be logged in plaintext in ^ISCLOG data.

      ADO.NET

      DP-412178: Increase data type sizes to prevent "index out of bounds" exceptions

      Category: ADO.NET
      Platforms: All
      Version: 2021.1.3

      When writing data types to the wire, the code validate that the current size of the buffer can record all of the data for that data type. The buffer needs to be able to hold the entire contents of the value, as well as information about the length and type of the value.

      Before this change, in certain cases (time stamp, time, and date), the code did not validate that the buffer could hold information about the length and type. The result was an index out of bounds exception when the information to be recorded was longer than the buffer size.

      Backup/Restore

      DP-411054: Free suspended write daemon properly when backup is aborted

      Category: Backup/Restore
      Platforms: All
      Version: 2021.1.3

      Before this fix, aborting a running backup could leave the write daemon suspended, leading to a system hang.  This occurred when aborting in all but the last pass of backup.

      DP-413772: Allow ExternalThaw to cancel a pending ExternalFreeze request

      Category: Backup/Restore
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue that caused ##class(Backup.General).ExternalThaw() to fail to interrupt a hanging ##class(Backup.General).ExternalFreeze().

      Business Intelligence

      DP-402897: Fix warning exporting widgets to Excel by exporting pivot table as CSV

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.1

      This change fixes a problem where exporting pivot tables from Analyzer/Pivot Widgets as XLS and then opening in Excel, they throw a security warning.

      To avoid this warning and improve integration with other data visualization tools, XLS export is replaced with CSV export.

      DP-404426: Fixes inherited cubes that do not show sourceClass list in Architect

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.1

      If a cube (abstract or not) using ^DeepSee.Overrides for setting a source class with the explicit setting sourceClass="" is then inherited, the Architect was showing the inherited content of the cube but was not able to populate the source class tree on the left pane.

      This change makes the sourceClass override aply during any cube model load wthin the Architect if the sourceClass is null.

      DP-404982: Solves a problem where %FixBuildErrors can report that errors are fixed, when they are not

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.1

      When a cube has an OnProcessFact method, %DeepSee.Utils:%FixBuildErrors can handle reference errors improperly. It reports success and deletes ^DeepSee.BuildErrors whether or not the record is fixed. This change fixes the problem.

      DP-405033: Run AuditQueryCode after asynch queries complete

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      With this change, %DeepSee.ResultSet:%RunQueryAuditCode() is run at the end of %ExecuteAsynch() after agents executing cells have completed their tasks.

      As a result, the result set's %Statistics property now has full statistics on the query's performance at the point where %RunQueryAuditCode() is executed.

      DP-405378: Correct subquery queryKey bookkeeping in compound cubes

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.1

      This change corrects a problem where a crossjoin containing an ordering function in the second (inner) term might lose track of the correct order of compound cube results.

      DP-405579: FIX - When Selective cube build is enabled two or more levels on time dimension cannot have different factNames

      Category: Business Intelligence
      Platforms: Windows
      Version: 2021.1.3

      When Selective cube build is enabled two or more levels on time dimension cannot have factName

      DTB918 tightened check on factNames.

      Consider following class: 

      Class wrc946213.data Extends %Persistent

      

      
{ Property SomeDate As %Date; }

      And corresponding cube: 

      <cube sourceClass="wrc946213.data" namedFactNums="true" ...>
 <dimension name="SomeDate" sourceProperty="SomeDate" ... >
 <hierarchy name="H1" disabled="false" hidden="false">
 <level name="SomeDateYear" disabled="false" factName="DxSomeDateYear" timeFunction="Year" ...>
 </level>
 <level name="SomeDateMonthYear" disabled="false" factName="DxSomeDateMonthYear" timeFunction="MonthYear" ... >
 </level>
 </hierarchy>
 </dimension>
 </cube>

      Compilation of this cube fails with the following error: 

      Compiling class wrc946213.cube
 ERROR #5001: Alias mismatch when filing 'DxSomeDate' : '' : 'DxSomeDateMonthYear'
 > ERROR #5490: Error running generator for method '%GetModel:wrc946213.cube' 
 ERROR: %DeepSee.CubeDefinition.cls(%GetModel) of generated code compiling subclass 'wrc946213.cube'
 > ERROR #5030: An error occurred while compiling class 'wrc946213.cube'
 Compiling routine wrc946213.cube.1

      Checked on InterSystems IRIS for Windows (x86-64) 2021.2.0L (Build 475U) Tue Mar 9 2021 01:18:47 EST

      DP-405956: Fixes errors in Mini-Analyzer and Analyzer drag and drop and enabling filters

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.1

      In some cases, dragging and dropping in the Mini-Analyzer caused JavaScript errors and in other cases enabling and disabling filters would not work in the Analyzer and Mini-Analyzer. This change corrects those problems.

      DP-406477: Fix problem where compile fails if parent cube has no named FactNums attribute

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.1

      In versions with selective build enabled by default, cubes created in Architect will normally have the namedFactNums attribute set to true (and factNumber attributes with default values for appropriate model elements. However, cubes imported from an older version and compiled in Studio or Terminal may not have any such attributes (and may compile and build successfully despite this).

      This leads to a problem when child cubes are newly created in Architect on version with selective build, and inherit from parent cubes created on an older version. These child cubes have namedFactNums = "true" by default*, but as the parent cube's model elements have no factNumber, the child cube will fail to compile.

      It's possible to work around this either by manually resolving the discrepancy in Studio or by opening the parent cube in Architect and saving it there, which enables selective build for the cube and adds namedFactNums and factNumber to the cube class.

      This change fixes the problem.

      DP-406811: Invalidate cache buckets containing fact deletions triggered by buildRestriction

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      If cube has buildRestriction and the source row is updated to be removed from the cube based on that restriction, then the relevant query cache will now be marked invalid and results will be recalculated after a cube synchronize.

      DP-407899: Correct handling of !NO MEMBER in MDX filters

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.1

      In version 2021.1, MDX queries return no results if they include a filter clause that excludes (using %NOT) a member that does not exist in the level being filtered on. For example, the following query returns no results when there is no &[] member in the [HOMED].[H1].[ZIP] level: 

      SELECT NON EMPTY [HomeD].[H1].[ZIP].Members ON 1 FROM [PATIENTS] %FILTER [HOMED].[H1].[ZIP].&[<null>].%NOT
      The problem is the same whether the member is &[<null>] or some other member (for example &[test]) that does not exist in the level. %NOT works as expected when the member exists - this query returns correct results:

      SELECT NON EMPTY [HomeD].[H1].[ZIP].Members ON 1 FROM [PATIENTS] %FILTER [HOMED].[H1].[ZIP].&[32006].%NOT

      It is more common to filter on members that do exist, of course, but filtering on members that do not exist should work correctly, as the set of members in a level can change over time and should be able to do so without breaking existing pivots. In particular, a customer may well wish to exclude null values without knowing whether their data will contain any at a given time.

      The problem occurs in version 2021.1. It does not occur in version 2019.1.0 or in Ensemble 2018.1.2.

      DP-408372: Shorten hang in Resultset:%OutputJSON to improve Analyzer experience

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      A wait in the status polling has been reduced to reduce the overall wait time in the pivotTable and other %DeepSee.ResultSet-related components.

      Most notably this helps the Analyzer move into its managed polling state sooner and removes an initial ~1s stall.

        ---- DTB905 added the follow code to %DeepSee.Component.pivotController:updateState():

      pivot.SetCurrentQueryText('current',pivot.SynthesizeMDX()); pivot.SetCurrentQueryText('resolved',pivot.SynthesizeMDX());

      This can cause dramatically increased execution times. For example, when running the following query: SELECT NON EMPTY HEAD(NONEMPTYCROSSJOIN([AllerD].[H1].[Allergies].Members,[AgeD].[H1].[Age].Members),2000,SAMPLE) ON 1 FROM [PATIENTS]

      We can see that almost 4 seconds is spent running this code (1 second per zenmethod call):

      !image-2021-09-28-15-38-54-880.png!

      If we run a similar query, but instead of crossjoining the two members, putting Age on rows performs fine. Each of the 4 zenmethod calls are taking between 10-20ms.

      At this point, it seems like an odd interaction with Zen being amplified by the additional calls in DTB905.

      DP-408448: Export pivot table to Excel without re-executing query

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      Currently, %DeepSee.UI.MDXExcel:OnPage() executes the query whose results are to be exported to Excel. Given that we do not plan to implement DP-406077 in the near future, Mike Senatore would like us to avoid that execution in any case where we are exporting a pivot table or pivot widget that has recently been successfully executed. He has found cases where after a pivot successfully loads, attempting to export it to Excel takes several minutes and ultimately results in a Web Gateway timeout. If we were able to generate the Excel file based on data extracted from the existing ResultSet object, it would likely prevent this problem.

      DP-409762: Fix internal logic used by %OR

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      This change modifies the internal logic (in GetSpecForAxisNode()) used by %OR so that it can apply across a wider variety of constructs.

      Before this change, %GetFiltersForCellRange returned incorrect results in some cases.

      DP-411270: Restore support for direct-to-Excel MDX export via API

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      The change that made is possible to reopen an MDX result set during Excel export broke the MDX-text based %DeepSee.Utils API. This change restores the API functionality.

      DP-412648: Improve handling of AGGREGATE in crossjoins

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      Before this change, two calculated members used in a crossjoin could lose track of the dividing line between two virtual members when the AGGREGATE function is used to create one or both of those members. This fix supports the rule that if it is possible to determine the Aggregate is defined to look like a member, it will be converted to a %OR to truly behave like a singular member. This will not apply when the Aggregate defines a measure to apply to target set, in which case the function is acting like a metric.

      DP-415407: Protect %SetAncestorLockKeys from undefined argument

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      Protect %DeepSee.ResultSet:%SetAncestorLockKeys from an <UNDEFINED> error when no argument is supplied.

      DP-417737: Correct vulnerability in Analytics User Portal

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      This change corrects a vulnerability in the Analytics User Portal. The vulnerability would allow authenticated users with access to the User Portal page to execute user-specified ObjectScript commands on the server using their current security context. There are limitations; the string would be converted to uppercase before being executed, and it cannot contain any periods. There are also length limitations. 

      This vulnerability has been corrected.

      DP-418010: Fix detection of stale cell cache for cube relationships

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      A query involving cached cell results derived from related cube subqueries could produce incorrect results if any of those related cubes were updated with new dimension members that were inserted into any dimension used in that query.

      DP-419305: Fix folder delete in FolderManager

      Category: Business Intelligence
      Platforms: All
      Version: 2021.1.3

      Correct a regression that prevents folder deletion in the Folder Manager due to reporting a new error message.

      Cloud

      DP-404204: Upgrading containers and file ownership changes

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Cloud
      Platforms: Cloud
      Version: 2021.1.1

      The volume mounted for durable %SYS must now be writeable by user 51773 (irisowner), because cloning into that directory now happens as irisowner instead of root. When it isn't writeable, startup will fail. The cloud distribution now uses a non-root install, not a traditional installation. All files are now owned by 51773:51773 (aka irisowner:irisowner). The installation directory has not changed (still /usr/irissys). 

      Before upgrading, users (excluding those using ICM or IKO) MUST chown their durable %SYS directory ($ISC_DATA_DIRECTORY) so that all files are owned by irisowner, for example: 

      chown -R 51773:51773 $ISC_DATA_DIRECTORY

      This will most likely need to be done as root. IKO and ICM perform this ownership change automatically. 

      For locked down images, the installation directory ($ISC_PACKAGE_INSTALLDIR) has changed from /home/irisowner/IRIS to /usr/irissys.

      Upgrades with IRIS-lockeddown and IRISHealth-lockeddown between versions pre- and post- this change are not supported and will not succeed.

      DP-404238: Limit use of PasswordHash to initial configuration

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Cloud
      Platforms: Cloud
      Version: 2021.1.1

      PasswordHash makes it easy to change system-wide passwords during the initial system configuration. The password change is effective before the instance is initialized. In previous releases, you could also use this feature to make system-wide password changes once the system was in use and no longer had the initial passwords. With this change, PasswordHash can only be used during the initial system configuration and cannot be used later to change passwords.

      Specifically, PasswordHash will only be processed if the existing hash matches a known, fixed value used to set the initial password.

      DP-405632: Update host configuration for Ubuntu 20.04

      Category: Cloud
      Platforms: UNIX®
      Version: 2021.1.1

      Changes between Ubuntu 18.04 and 20.04 may potentially cause network issues. This change deals with this two of these changes:

      1. On some configurations, "iptables" and "iptables-save" have moved from /sbin to /usr/sbin.

        This may have been causing issues reaching the outside network; we now check both locations.

      2. resolv.conf now includes option "trust-ad" by default.

        This may have been causing Weave Net to crash (see [https://github.com/weaveworks/weave/issues/3868]); we now remove this option.

      DP-411646: In durable sys upgrades, always set httpd.conf's user and group

      Category: Cloud
      Platforms: All
      Version: 2021.1.3

      The User and Group fields in httpd.conf now will always be updated during durable %SYS upgrades to reflect the appropriate values for the new InterSystems IRIS installation. Both User and Group will be set to the group that owns the install directory's httpd.conf file.

      DP-413048: Remove Linux capability checking from InterSystems IRIS containers

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Cloud
      Platforms: Linux
      Version: 2021.1.3

      This change removes the no-longer-necessary Linux capability checking from InterSystems IRIS containers, to avoid incompatibilities with new versions of Docker.

      This doesn't immediately affect ICM users because an older Docker version is set in Samples/defaults.json.

      Compatibility Features

      DP-402030: Fix CMQL queries fail on first execute after conversion to InterSystems IRIS

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.1

      Fixes a problem where if a CMQL query has been executed on a Caché instance, and then the instance is converted to InterSystems IRIS, the query will fail on the first execute on the converted instance, but will run afterwards.

      DP-402536: Fix an error where Caché Migration Feature stream properties would be set to null

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.1

      This change fixes a problem that would cause stream properties to be set to null when the value is assigned using the IRISActiveX library. The problem only affects properties using the Caché Migration Feature stream types (such as %Library.GlobalCharacterStream) and only when the property is explicitly set.

      DP-403621: Support NodeJS version 14

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.1

      This change adds the headers and build scripts to support building the legacy node.js binding for node version 14.x.x.

      DP-403622: SDK205Support NodeJS version 16

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.1

      This change adds the headers and build scripts to support building the legacy node.js binding for node version 16.x.x.

      DP-404220: Fix old Node.js support methods without a return value

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.1

      This change updates the old NodeJS binding to be able to call ObjectScript methods that do not return a value. Previously, attempting to call such a method would result in a <COMMAND> error when run via the API connectivity, or a <SYNTAX> error when run using TCP connectivity.

      Now invoking a method that does not return a value will result in undefined being returned to the javascript code.

      Note: If a client without this change connects to a server with this change using TCP, any method call that should return undefined will actually return an empty string (''). If you have a newer client connecting to an older server, then you will still get a <COMMAND> error (that is the same behavior as before this change.)

      DP-404837: Prevent crashes in legacy Node.js template interface

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.1

      This change corrects defects in the legacy Node.js binding that could cause crashes when using the class template interface.

      This change corrects two problems.

      1. When using the class template interface to invoke a method or a class method that returns an object value, and the first argument to the method was longer than 256 bytes, there would be a buffer overflow that could cause memory corruption and potentially a crash. This problem has been corrected.

      2. When caling IrisClass() to generate a class template, if the connection is using TCP connectivity the class metadata retrieved from the server via GetClass^%SYS.nodejsServer) could be corrupted, which could result in a crash.

      DP-406886: $SYSTEM.Security.Login() can be called for VISM client

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.3

      This change ensures that $SYSTEM.Security.Login() can be called from a VISM application.

      DP-413293: Zen Reports: do not allow use of $DATASOURCE URL parameter by default

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.3

      Previously ZEN Reports would allow the report's datasource to be specified at runtime via the $DATASOURCE URL Parameter. If this is not defined, then we will use the DATASOURCE class parameter, and if that isn't defined we will use the ReportDefinition to generate the XML data. The ReportDefinition is the standard way to specify report data.

      With this change, Zen Reports will no longer use the $DATASOURCE URL parameter by default. Developers are strongly encouraged to use the DATASOURCE class parameter to specify an external datasource if desired. If a report does require the $DATASOURCE URL, you can re-enable the previous behavior on a per-report or per-application basis by setting the parameter: 

      Parameter USEURLDATASOURCE = 1;

      in either the report class, or in the report's Application class. Setting this value is discouraged.

      DP-414415: Pass original data in storage when using parameternonce to communicate parameters between pages

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.3

      %DeepSee.Utils:%CreateParmeterNonce is used to store information in a global with a nonced ID assigned to it so that it can be communicated to exports without risk of violating the max length of browser URLs. The pages which receive this use %DeepSee.Utils:%GetParameterNonce to retrieve the information from temporary storage.

      There is no need for this communication to contain any sort of translation, so the $ZCVT(tText,"I","UTF8") is removed from the store procedure. Any translation that might be necessary can and should be done by the functions actually writing the data to files if it is deemed necessary.

      This also makes sure the deepseeCSVProvider uses UTF-8 encoding when creating files so these characters are correctly written to the .csv file.

      DP-415588: Update Apache Ant from 1.7.1 to 1.10.12

      Category: Compatibility Features
      Platforms: All
      Version: 2021.1.3

      This change updates Apache Ant from 1.7.1 to 1.10.12 (the latest version) to resolve 2 high severity and 4 medium severity security vulnerabilities. Ant is still used to build severl modules that are part of Zen Reports, and that are included in the installation under the lib directory.

      CSP Server

      DP-402100: %CSP.WebSocket:OpenServer() should quit 0 for a non-existent WebSocketID

      Category: CSP Server
      Platforms: All
      Version: 2021.1.1

      This change updates the %CSP.WebSocket:OpenServer() method to return an error status if no data is found for a given WebSocket ID. Previously, this method would return $$$OK even though it failed to find and open the web socket.

      DP-404562: Adds unique error codes to %CSP.WebSocket

      Category: CSP Server
      Platforms: All
      Version: 2021.1.1

      This change replaces generic error usage in the %CSP.WebSocket class with new error codes. WebSocket behavior is unchanged otherwise.

      DP-406217: Allow CSP pages to specify exact redirect URL

      Category: CSP Server
      Platforms: All
      Version: 2021.1.1

      Typically when performing a browser-side redirect using %response.Redirect, the target URL will be transformed to an absolute URL before being sent to the client. The system may also add or modify the CSPToken, CSPCHD, and other URL parameters if the target URL is a CSP/ZEN page

      This change introduces a new property: %response.UseExactRedirect. If this is set to 1, then the server will not perform these transformations; the URL specified in %response.Redirect will be used exactly as specified by the user. This only affects the current request, and does not affect redirects using %response.ServerSideRedirect. The default is 0.

      Note that RFC 2616 initially defined the Location header to require an absolute URI (see section 14.30), but this was later relaxed by RFC 7231 (see Section 7.1.2, and Appendix B). Setting %response.UseExactRedirect=1 will allow a CSP Application to redirect to a relative URL. This generally should not be relevant because by default we will translate the URL to the equivalent absolute URL, but may be helpful in some unusual cases.

      DP-408957: Ignore cache clearing errors during CSP compile

      Category: CSP Server
      Platforms: All
      Version: 2021.1.1

      This change corrects the following error that occurs sometimes when compiling a CSP page in Studio. 

      Compilation started on 10/19/2021 16:36:55 with qualifiers 'k'
Compiling file /csp/user/menu.csp
Compiling class csp.menu
Compiling routine csp.menu.1
Detected 2 errors during compilation in 0.046s.
ERROR #7913: Unable to complete client request
  > ERROR #6084: Unknown errors detected, but no error code reported

      DP-416562: Correct WebSocket handling of low-level errors

      NOTE: This item may require a change to code, configuration, or operation.

      Category: CSP Server
      Platforms: All
      Version: 2021.1.3

      This fix corrects a bug in the error handling for web socket connections. If an error was encountered before the CSP server dispatched to the WebSocket class, then the error would be ignored. Now the CSP server will call the Error() method in the WebSocket class. Similarly, for authentication failures, the server will call Login(). Note that by default, Login() just calls Error(). 

      The websocket will also now abort the connection if OnPreServer() returns an error. Previously errors were ignored.

      DP-416563: Do not allow navigation to %-CSP pages from the /csp/sys/oauth2/ application

      NOTE: This item may require a change to code, configuration, or operation.

      Category: CSP Server
      Platforms: All
      Version: 2021.1.3

      This change modifies the default configuration to disallow access to %-CSP pages from the /csp/sys/oauth2/ application.

      Data Reporting

      DP-405725: Zen Reports do not output duplicate ID for table elements

      Category: Data Reporting
      Platforms: All
      Version: 2021.1.1

      This change corrects a bug that could result in an error when including certain display components directly in a table, if the component defines an ID attribute. In these cases Zen Reports would produce XSLFO with elements with duplicate ID's, which would result in an error from FOP like: 

      SEVERE: Exception
org.apache.fop.apps.FOPException: org.apache.fop.fo.ValidationException: Property ID "line" (found on "fo:table") previously used; ID values must be unique within a document! (See position 1861:-1)

      DP-407946: Execute queries using %ExecuteAsynch when preparing for printing/exporting

      Category: Data Reporting
      Platforms: All
      Version: 2021.1.1

      When exporting pivotTable results, the %DeepSee.UI.MDXExcel and %DeepSee.UI.MDXPDF pages now use the multithreaded %ExecuteAsynch by default to execute the queries in the background. This can be suppressed with a URL parameter ASYNC=0 when opening the page, which will default to the prior behavior. Omission of this parameter will produce the new default behavior.

      DataMove

      DP-413775: Change a few DataMove APIs

      NOTE: This item may require a change to code, configuration, or operation.

      Category: DataMove
      Platforms: All
      Version: 2021.1.3

      Several of the DataMove API's have been changed to add a new first parameter "Name" in the DataMove.API class as follows: 

      MapInitialize(Name as %String, Namespaces As %String)
MapGlobalsCreate(Name as %String, Namespace As %String, GblName As %String, ByRef Properties As %String)
MapGlobalsDelete(Name as %String, Namespace As %String, GblName As %String) As %Status
MapGlobalsModify(Name as %String, Namespace As %String, GblName As %String, ByRef Properties As %String)

      Customer will need to update their scripts to add this new parameter.

      DP-413825: Verify namespace is initialized for DataMove

      Category: DataMove
      Platforms: All
      Version: 2021.1.3

      With this change, DataMove first verifies that the namespace is initialized, so that DataMove does not remove any existing mappings.

      Gateways - .NET

      DP-407501: 32-bit processes and external language servers

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Gateways - .NET
      Platforms: All
      Version: 2021.1.1

      External language servers that do not have "Exec 32" checked will change from executing as a 32-bit process to executing as a 64-bit process when started from the Management Portal. This will only affect you if you are loading external assemblies compiled for x86 and did not check the Exec 32 box of the server definition.

      DP-414366: Correct closing of Gateway

      Category: Gateways - .NET
      Platforms: All
      Version: 2021.1.3

      This change corrects a bug in which when a variable containing a Gateway is left around until after a new Gateway is created, the delayed destruction of the first Gateway can potentially close the device that's used by the second Gateway.

      Gateways - Java

      DP-402388: Fixes problem where Business Operation cannot connect to Java Gateway

      Category: Gateways - Java
      Platforms:
      Version: 2021.1.1

      EnsLib.JavaGateway.Service::ConnectGateway is deprecated but unfortunately, in the process, the pDebug As %Boolean 3rd parameter was removed This change restores the old method signature to EnsLib.JavaGateway.Service::ConnectGateway() in order to fix the backward incompatibility.

      DP-407804: Correct Java ClassPath when directory contains spaces

      Category: Gateways - Java
      Platforms: All
      Version: 2021.1.1

      This changes fixes a condition where the Java Gateway could fail to start if there are spaces in the classpath directory.

      DP-411028: %Net.Remote.Gateway %OnClose() method no longer calls %Disconnect()

      Category: Gateways - Java
      Platforms: All
      Version: 2021.1.3

      In %Net.Remote.Gateway, the %OnClose() no longer calls %Disconnect(). The code had sometimes caused both sides to try to close down the connection at the same time, resulting in a hang.

      With this change, if the Gateway connection is not properly closed, %Net.Remote.Gateway does not simulate a %Disconnect.

      This change also has the added benefit that if the external side initiates a close, then you can easily kill the gateway object without trying to do another reverse close.

      Gateways - JDBC

      DP-407653: Correct IRISConnect.isValid handling of read timeouts

      Category: Gateways - JDBC
      Platforms: All
      Version: 2021.1.1

      IRISConnect.isValid(readTimeout) is a public API that returns true if the connection is active, and false if it is disconnected. The API sets the read timeout for the socket to a duration in seconds and sends a ping message for the server to respond to.

      The problem with the code is that if the read does timeout, it does not mean the connection is dead, just that it hasn't responded, and the exception code should reset the read timeout of the socket back to the jdbc default value (either network timeout that was set or infinite). Another problem in this case is that there is a pending request for query results that has been requested from the server, but has not been read. For example, in one case the query was artificially delayed by 20 seconds, and the read timeout is resetting how long we wait for a query timeout (which is controlled on the server side, not the client), and the client gives up with a read timeout consuming the response to the query (even before the ping is attempted).

      This change adds a check to return true if we are processing a query read-ahead, otherwise to ping the connection with the timeout requested in isValid().

      DP-408799: Correct JDBC gateway query code generation to close cursor

      Category: Gateways - JDBC
      Platforms: All
      Version: 2021.1.1

      A customer reported a problem where they have a class query of type %SQLQuery projected as a procedure.  The SQL query in this class query is against a linked table, through the JDBC gateway.  They are calling the cached query using: 

         set rs = ##class(%ResultSet).%New()
       

      When the UCQ oref was destructed, the SQL Close call was not made because i%CursorState was not defined, so the process did not realize an SQL cursor statement had been executed.  When the SQL Close did not occur, _%JDBCGateway.removeStatement(i%hstmt)_ was not called, which means the statement was not removed from the Java process and memory in the Java process would continue to build up.   This change allows the UCQ's %Destruct method to close the SQL cursor.

      DP-413359: JDBCGateway to load third-party JAR files into system class loader

      Category: Gateways - JDBC
      Platforms: All
      Version: 2021.1.3

      This change forces all JDBC third party JAR files to be loaded into system class loader. This will resolve the problem of unchecked memory leak.

      DP-413446: Fix bugs in [Gateway] section conversion

      Category: Gateways - JDBC
      Platforms: All
      Version: 2021.1.3

      This change fixes a couple of bugs that could occur upon upgrades. These bugs affected the conversion of the [Gateway] section of the CPF file.

      Gateways - ODBC

      DP-404515: Use correct data length for SQLGetData SQL_C_WCHAR bufsize 0

      Category: Gateways - ODBC
      Platforms: All
      Version: 2021.1.1

      Fixes a bug where calling SQLGetData twice, the first time with buffer length 0, caused the second call to return an incorrect data length for wide-char (SQL_C_WCHAR) queries.

      DP-412481: FIX - Ciphersuites config not sticky

      Category: Gateways - ODBC
      Platforms: All
      Version: 2021.1.3

      Investigate behavior reported in the parent issue and address any product defect(s) identified. Urgency is that this string of issues need to be fully resolved.

      --

      The netssl code was loading in the Ciphersuites from the registry key, but there was a typo in that segment of code that overwrote the start of the Ciphersuites string with "", the default value for CA_PATH if CA_PATH is not specified (which it typically is not). Fixing the typo so that the pparam defaults are written to the pparam strings as they should be instead of the sparam strings. It looked like this typo was only for the registry code; SSLdefs code looked OK.

      Gateways - Python

      DP-406468: External Server - fix <UNDEFINED> error encountered when starting a Python gateway

      Category: Gateways - Python
      Platforms: All
      Version: 2021.1.3

      This change fixes a bug in %Net.Remote.ObjectGateway that resulted in an <UNDEFINED> error when starting a Python gateway.

      Global Module

      DP-410457: SQL: Clear ^rINDEXEXT("F") when rebuilding extent index

      Category: Global Module
      Platforms: All
      Version: 2021.1.3

      There are some class/sql metadata defined in the DATA/GLOBALS databases of a namespace, the contents of ^rINDEXEXT.  The customer upgrades their code by dropping in a new DATA/GLOBALS database, but this causes ^rINDEXEXT to be either missing or out of date.  There is a mechanism we have in place for such cases, to call $SYSTEM.OBJ.RebuildExtentIndex(0,1). 

      However, $SYSTEM.OBJ.RebuildExtentIndex(0,1) was not first clearing out ^rINDEXEXT(F"), so some old FKey metadata was still defined after rebuilding the index.  This has been corrected.

      IDEs

      DP-404634: Allows users to drill down into OREF's when debugging in VSCode

      Category: IDEs
      Platforms: All
      Version: 2021.1.1

      See https://github.com/intersystems-community/vscode-objectscript/issues/272.

      DP-405497: Fixes input Japanese being garbled when running method with parameters in debug launch mode

      Category: IDEs
      Platforms: All
      Version: 2021.1.1

      See https://github.com/intersystems-community/vscode-objectscript/issues/570.

      DP-405539: Fixes problem where launch of class method not accepting path with Windows drive

      Category: IDEs
      Platforms: Windows
      Version: 2021.1.1

      See https://github.com/intersystems-community/vscode-objectscript/issues/654.

      DP-405736: Fix Uncomment Line off-by-one error in Studio

      Category: IDEs
      Platforms: All
      Version: 2021.1.1

      This change fixes a bug wherein Studio's Edit > Advanced > Uncomment Line and Uncomment Block commands would fail to uncomment the last selected line.

      DP-407521: Fix Studio so that it follows HTTPS configuration setting

      Category: IDEs
      Platforms: All
      Version: 2021.1.1

      SDK112 (DPP-388) updated Studio to use the web server configuration defined in the client registry, when applicable. However, it did not check whether HTTPS was enabled but instead always used "http" in these cases. This could prevent features such as Studio's Tools > Add-Ins > SOAP Wizard from connecting to the server, if HTTPS was required and the client registry was in use. With this devchange, Studio now selects "https" or "http" for the URL based on the client's HTTPS setting.

      DP-409553: Fix document auto-import when using %Studio.SourceControl.ISC

      Category: IDEs
      Platforms: All
      Version: 2021.1.1

      This change addresses two vscode-objectscript GitHub issues related to the source control hooks' auto-import feature.

      Open a document under server side source control in VS Code. Edit this document which asks to checkout the document, select 'yes' and save/compile it. Under a separate source control tool you can see the file associated with this document is now checked out and modified. Revert this file in the separate source control tool.

      In VS Code there is no indication the file on disk associated with this document was changed. In Studio the timestamp check it performs on a regular basis and when the document gets focus will trigger a source control call that detects the associated file is modified and if the document is not modified in Studio will cause the document to be reloaded from the file automatically and the view inside Studio will be refreshed. Additionally if the file in Studio was modified you will be prompted if you wish to revert your edits back to the version of the file on disk.

      This change corrects the behavior of VS Code to recognize that the file is no long writeable.

      DP-413625: Corrects an access problem for server-side editing from VSCode

      Category: IDEs
      Platforms: All
      Version: 2021.1.3

      Before this change, when a user wanted to enable server-side editing from VSCode, the user had to run this command for each namespace: 

      GRANT EXECUTE ON %Library.RoutineMgr_StudioOpenDialog TO %Developer
      With this change, that step is no longer necessary.

      DP-416045: Correct Studio bug with View Web Page for a class that had a fully qualified LOCATION parameter

      Category: IDEs
      Platforms: All
      Version: 2021.1.3

      This change fixes a bug in which the View Web Page option did not work for a class whose LOCATION parameter is a full path with the scheme prepended (http:// or https://).

      DP-416194: Show calculated properties as "Calculated" instead of nonprintable characters

      Category: IDEs
      Platforms: All
      Version: 2021.1.3

      With this change, IDEs show calculated properties as "Calculated" instead of as nonprintable characters.

      Installation

      CRE-3678: Installation changes for IntegratedML

      Category: Installation
      Platforms: All
      Version: 2021.1.1

      With this change, there are installation kits that include the IntegratedML libraries and other kits that do not. For the kits that do include these libraries, note that IntegratedML is not installed by default, but is rather part of the Custom installation.

      If you are using IntegratedML and you upgrade, be sure to select the appropriate kit and also to use the Custom installation option.

      DP-408131: Accessing install directory on Windows from ODBC requires additional privileges

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Installation
      Platforms: Windows
      Version: 2021.1.3

      Users in "Authenticated Users" group no longer are able to write files into the installation directory in locked down install without additional privileges.

      DP-410894: Handle bad mappings in CPF file during conversion

      Category: Installation
      Platforms: All
      Version: 2021.1.3

      While converting the CPF file during a Caché to InterSystems IRIS conversion, the code now updates the [Map.*] sections and replaces any references to the CACHE* system databases to the corresponding InterSystems IRIS* system databases.

      In one scenario, there was a line that was not converted correctly, causing the conversion to fail.

      DP-412668: New ISCAgent standalone installer

      Category: Installation
      Platforms: Windows
      Version: 2021.1.1

      This change provides a new ISCAgent installer for Windows. Functionality will be the same as current ISCAgent installer except for the following changes:

      • It will accept normal Windows Installer command line options for logging, unattended and silent install, similar to what InterSystems IRIS installer is using. For example, to run silent install one would use command "ISCAgent_<version>.exe /qn".
      • It will install VC++ redistributables in both full UI and unattended install.
      • Dialogs will have the same look as InterSystems IRIS installer dialogs.

      DP-414705: remove old files in upgrade on macOS before copying new version

      Category: Installation
      Platforms: macOS
      Version: 2021.1.3

      macOS does not always refresh executable signature cache when files are overwritten by "cp" command. As a workaround, the installer will remove old version before copying a new one on this platform.

      DP-414750: Update Xerces to 3.2.3

      Category: Installation
      Platforms: All
      Version: 2021.1.3

      xerces_c version used by InterSystems IRIS build is upgraded from 3.2.2 to 3.2.3.

      IntegratedML

      DP-410849: Gateways section for Ubuntu disables usesharedmem

      Category: IntegratedML
      Platforms: Ubuntu
      Version: 2021.1.1

      For initial installations, the usesharedmemory CPF flag is disabled for Ubuntu 20.04, which has a shared memory problem. If the system is an upgrade, no change is made to this setting.

      IntegratedML - Core

      DP-420361: H5PY fix. Pins H5PY to version 3.1 when possible, and uses version 3.7 when the platform python version does not support older transitive dependencies.

      Category: IntegratedML - Core
      Platforms: All
      Version: 2021.1.3

      Pins H5PY to version 3.1 when possible, and uses version 3.7 when the platform python version does not support older transitive dependencies (i.e. numpy).

      Interoperability

      DP-402578: FIX standardize private/internal method names for all external languages

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      In order to use the same method calls from InterSystems IRIS to our PEX components in all supported external languages, the method names need to be the same across all supported languages for PEX. This change adds underscores to internal/private methods for PEX components in all three languages (Python, Java, DotNet).

      DP-402935: Corrects framed TCP adapter not functioning with single end framing character

      Category: Interoperability
      Platforms: All
      Version: 2021.1.1

      The inability to process 'framed' messages terminated using a single terminating character by the EnsLib.TCP.Framed.PassthroughService and EnsLib.TCP.Framed.PassthroughOperation, which use EnsLib.TCP.FramedCommon via EnsLib.TCP.FramedInboundAdapter and  EnsLib.TCP.FramedOutboundAdapter has been corrected.

      DP-406031: Fix Java Gateway hang

      Category: Interoperability
      Platforms: All
      Version: 2021.1.1

      This change fixes a Java Gateway hang by normalizing the Y0 response message structure

      Y0 response message structure before this change (protocol<=61):

      • piece #1: server protocol number

      • piece #2: server protocol number

      • piece #3: boolean flag indicates if server supports shared memory or not. Only present if client has requested shared memory.

      Y0 response message structure with this change (protocol>=63):

      • piece #1: server protocol number

      • piece #2: server protocol number

      • piece #3: boolean flag indicates if server supports shared memory or not. This piece is always present even if client didn't request shared memory.

      • piece #4: External Language Server name under which the running server is defined.

      DP-408463: Improve error message when Production Export encounters non-existent classes

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      When a production references a class that does not exist, and the user attempts to export the production code using the "Export" button under the "Action" tab of Production settings, the web page pops up the error "ERROR #6308: Item '<non-existent class>' is invalid or does not have any data to export, skipping this item."

      Previously, however, the non-existent item was not being skipped in the export and the export was failing entirely. This change updates the export logic to actually skip non-existent items while allowing the export to succeed.

      It also fixes a related bug where failures to export would not close the export popup, but subsequent attempts to export would silently fail until the popup was closed and reopened.

      DP-408979: FIX - Purge task logs errors to production for DICOM I/O Log entries

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      In a DICOM production, when ArchiveIO is enabled, multiple I/O log entries can point to the same EnsLib.DICOM.Document object. The Event Log would fill with error messages when purging with AllTypes and MessageBodies=1 (defined when creating the Purge Task) because EnsLib.DICOM.Document:%OnDelete() would call $$$EnsError and write to the Event Log for any duplicate deletes.

      To prevent this, %OnDelete() still composes and returns the error status, but it now uses $$$ERROR instead and lets the caller decide whether to record or ignore it. (The purge task ignores it.)

      DP-410236: PATCH verb supported by EnsLib.HTTP.OutboundAdapter

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      The EnsLib.HTTP.OutboundAdapter now supports the PATCH verb and passes it to %Net.HttpRequest.

      The new adapter methods are Patch(), PatchFormDataArray(), and PatchURL().

      DP-410350: Avoid Possible loss of one attribute on setting another attribute in XMLVDoc

      Category: Interoperability
      Platforms: All
      Version: 2021.1.1

      In an XML VDoc element, if 10 or more attributes had been set, and one attribute's value was then set to another value, then, on output, one of the other attributes would not be included.  This change fixes that problem so that no attributes are lost when another one is set, even if there are a large number of attributes.

      DP-410391: Modify setting 'raw' XML to a property path to aid determination of order

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      There was a previous correction that corrected the output order for a single property path in a sequence where a prior array sibling had more than one element. This correction lead to a possible change in output order when 'raw' XML (for example from a source XML VDoc complex property) was set using a Property path syntax. The latter change is now corrected.

      A new transient property of EnsLib.EDI.XML.Document is introduced to allow the prior behavior. The property is ByPassPropCacheForMixed.

      When setting 'raw' XML to a Property path (as opposed to a DOM path), this flag allows setting the data directly to the internal DOM storage. While this might improve performance, the output may have nodes in a different order than desired when combining with single property path sets.

      This setting is provided for backwards compatibility. The default is to set to the internal intermediary Property storage.

      DP-411008: Fix invalid error message popups when exporting production from the SMP

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      Fixed a bug where attempting to export valid non-class files like PTD files failed and caused an error popup when exporting from productions.

      DP-415411: Project FullSize property to SQL for all EDI document types

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      This change projects the FullSize property to SQL for X12, EDIFACT, and ASTM.  The property had already existed for all three (plus HL7), and the projection was already included for HL7.

      DP-416568: Modifications to how productions start/stop/update as _Ensemble

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      This change modifies how the implementation of the internal wrapper methods that allow Interoperability Production Host Items to run as InterSystems IRIS user _Ensemble.

      DP-416858: Add optional use of Access Token for SASL XOAUTH support in EnsLib email adapters

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      The email adapters now provide a new setting for a configured OAuth2 Application to use for SASL XOAUTH2. The adapters will use the associated Access Token if the application is authorized.

      The new settings include OAuth2ApplicationName, which is an optional OAuth2 Client Configuration Application name to use. If specified, this indicates that OAuth 2 is to be used and the name is used in the Authorization and Access Token retrieval process.

      DP-417168: Record Mapper and EDI Document Viewer file view permissions check

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Interoperability
      Platforms: All
      Version: 2021.1.3

      The Interoperability Record Mapper and EDI Document viewer management portal pages enforce holding the privilege %Ens_ViewFileSystem:USE before allowing the user to select a file using the file select dialog popup. This change adds the same check before a file is opened.

      JDBC

      DP-407658: Pass timeout as argument to socket.connect()

      Category: JDBC
      Platforms: All
      Version: 2021.1.1

      socket.connect(SocketAddress) does not use the timeout set inside the socket object, but in turn calls socket.connect(SocketAddress, 0) with an infinite timeout. This change passes the timeout as an argument to connect(SA, timeout).

      DP-412831: SQL SERVER: Correction for stream columns in Static ResultSets

      Category: JDBC
      Platforms: All
      Version: 2021.1.3

      This is a correction for a customer-reported issue in InterSystems IRIS.

      A problem has been corrected when executing a query through JDBC that is using a Static ResultSet and the ResultSet has one or more stream columns.  The error reported was <INVALID OREF>.

      Journaling

      DP-410180: Do not change journal directory when unrelated journal property changes

      Category: Journaling
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue that caused journaling to change back to the primary directory when a journal property change unrelated to journal directories is activated.

      DP-411849: Enable large reads for scanning forward for open transactions in journal restore

      Category: Journaling
      Platforms: All
      Version: 2021.1.3

      This change improves performance of scanning forward for open transactions at startup, failover, and journal restore.

      DP-412800: Fix the cache line alignment for journal buffer descriptors (jrniodb)

      Category: Journaling
      Platforms: All
      Version: 2021.1.3

      This corrects a bug with journal buffer descriptor structures not being properly aligned to begin on a cache line boundary.  In some configuration and version combinations this could lead to high amounts of contention and gate scalability on large NUMA systems.  

      Whether a particular version and configuration combination will be susceptible prior to this fix is not easy to predict.  irisstat -j1 shows the offset of journal buffer descriptors in the left hand column labelled "jrniodb".  When properly aligned these offsets end in 0x00 or 0x80 (128 byte alignment).  Other alignments, which are no longer possible with this bug fix, may show a degradation in journal throughput scaling (esp. on large NUMA systems): 0x70/0xf0 is known to be particularly problematic.

      DP-417229: Ensure that ^STU has access to system data

      Category: Journaling
      Platforms: All
      Version: 2021.1.3

      This change ensures that ^STU has access to system data. Before this change, it was possible for %ZSTART code (which calls ^STU) to fail occasionally because the _Ensemble user and the Security.System 'SYSTEM' object could not be found.

      DP-419192: Abort dejournaling upon unknown journal record types

      Category: Journaling
      Platforms: All
      Version: 2021.1.3

      This change addresses an issue with dejournaling that could cause mirror database discrepancy when the primary member runs a newer version of InterSystems IRIS than the failover or async member and the primary journal file contains journal records not recognized by the older version of InterSystems IRIS.

      Kernel

      DP-402510: Fix problem where background debugging loops when reading from the null device

      Category: Kernel
      Platforms: UNIX®
      Version: 2021.1.1

      If a background process is changed to foreground with ^%PMODE and attempts to read from its principal device which is the default null device, the process could loop. This change fixes this problem that was discovered developing a debugging utility.

      DP-403205: Fix two-argument $GET() of array that returns wrong value if Undefined(2) is set

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      If a process has set $SYSTEM.Process.Undefined(2), a two-argument $GET() on VAR(SUB) where SUB is undefined would return a null string instead of the expected second argument of $GET(). This change fixes the problem.

      DP-403473: Do not use Windows short path names

      Category: Kernel
      Platforms: Windows
      Version: 2021.1.1

      The %File method NormalizeFilenameWithSpaces() is used to apply quotes to a file name that contains spaces, so it can be passed as an argument to a shell command. In some cases on Windows, it would convert the name to a Windows short file name instead of adding quotes. This did not always result in a valid file reference. The Windows short file names are no longer necessary, so the method now always returns the original name and adds quotes if it contains spaces.

      DP-403566: Fix compilation error that could cause core dumps

      Category: Kernel
      Platforms: All
      Version: 2021.1.2

      Change DP-403184, in fixing a compilation error, introduced a different compilation error which could cause mismatches between compiled and interpreted expressions with Undefined = 2 and could cause core dumps. This change fixes the new error.

      DP-403623: Fixes <BREAK> error and inconsistent behavior, sometimes resulting in a core

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      If there is an uncaught <COMMAND> error because $XECUTE() quits without returning a value, a GOTO command at the error prompt had unpredictable results. This change corrects the problem.

      DP-404235: Restrict users permitted to perform clone and merge operations

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      Two issues were identified with the clone and merge operations:

      1) Clone and merge can be invoked by the InterSystems IRIS stop/start user. We now protect against this by verifying that the current userID corresponds to root or the InterSystems IRIS owner

      2) Clone and merge were being run as root. These operations are now being run with the current userID and groupID

      While addressing these issues, a third was found:

      3) Values for userID and groupID were being overwritten during a restart. Assignment of these variables now occurs only once, in code common to both start and restart.

      DP-404334: Quote $ZF() shell commands on Windows

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Kernel
      Platforms: Windows
      Version: 2021.1.1

      Windows shell commands must be enclosed in quotes if there are quoted arguments along with a quoted command path. Those quotes will now be added by $ZF(-1), $ZF(-2), and $ZF(-100,"SHELL").

      DP-404370: Handle net.ipv4.tcp_syncookies=0 on Linux for /CLOSELISTEN option.

      Category: Kernel
      Platforms: Linux
      Version: 2021.1.3

      This change adds the correct handling for net.ipv4.tcp_syncookies=0 on Linux for /CLOSELISTEN option.

      DP-404683: Allows messages greater than 32KB to be passed through $system.Event.Signal/WaitMsg

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      The 2 argument version of $system.Event.Signal() and the corresponding $system.Event.WaitMsg() were previously limited to 16K Unicode characters or 32K 8-bit characters. This would result in a <STORE> error and caused problems with the debugger when trying to inspect objects that exceeded this.

      This limit has been removed and it is now possible to pass strings as large as InterSystems IRIS will allow (3.8M characters currently). Any values that exceed 32KB are now allocated on the Shared Memory Heap (SMH); note that passing large values this way can increase the amount of overall GMHEAP that needs to be allocated.

      Usage Notes

      Obviously, heavy use of $system.Event.Signal() / $system.Event.WaitMsg() with large message bodies could deplete the available SMH and cause <STORE> errors. InterSystems recommends that any application that expects to have any more than a few large messages "outstanding" (that is a queue depth greater than 1) should rather use another medium for storing the message bodies such as globals -- Productions do not use the 2-argument version of $system.Event.Signal() and thus won't be susceptible to any additional use of the SMH.

      However, for the debugger, although it can pass large messages, it works in a request/response way. There typically will not be more than one potentially large message pending (per session).

      DP-405614: Scan ALTPID for speed and scalability of JOBs, sessions, and threads entering instance

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      The time to enter InterSystems IRIS is improved, particularly for very large systems with many thousands of active jobs.  This applies to the JOB command as well as normal session startup, or callin threads.  This also reduces contention between multiple jobs entering at the same time so the total system-wide throughput of the JOB command or session startup is increased.

      DP-406339: Fixes Java Gateway fail caused by edge-case problem in $zobjexport(16)

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      When $zobjexport(16) had a byte array of exactly 32727 bytes, it would encounter an error that could cause the Java Gateway to fail. This change corrects the problem.

      DP-406491: Make VariableList Query re-entrant

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      The underlying code that implemented the VariableList query of %SYS.ProcessQuery was not re-entrant. If the query was interrupted by a JOBEXAM from another process the original query would not be complete. It is now safe and will give correct results if interrupted.

      A new method is added to %SYS.ProcessQuery to get the variable list directly without running a query. 

      Set numvars=##class(%SYS.ProcessQuery).GetVariableList(InternalStack,.varArray)

      numvars is the total number of variables returned.
      varArray is the number of nodes (subscripts) returned.
      varArray(1) is the first node, varArray(2) is the second, and so on.

      Each node is:
      $ListBuild($ListBuild(varname,$Data(varname)), $ListBuild(varname,$Data(varname)), ...)

      The names are not sorted, except private variables follow public variables. Private variable names are prefixed with a tilde character (~). Because only defined variables are listed, the $Data value is always non-zero.

      DP-406788: Fix problem where two application servers held exclusive lock during ECP recovery

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      Fix missing locks to be uploaded to data server during ECP recovery. This problem started since version 2020.1 which expanded network buffer for lock requests.

      DP-406859: Fix tcpaccept() method to handle timout correctly on Windows platform

      Category: Kernel
      Platforms: Windows
      Version: 2021.1.1

      Under some circumstances, netcp tcpaccept() method was not handling the time interval correctly and was waiting longer than specified. This change corrects the problem.

      DP-407036: Fix possible lock table corruption when some jobs in convert wait state and denied

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue in which application servers appeared to lose their connections to the dataserver instance. Attempts to access globals from remote database(s) on these application servers would hang. The issue has been corrected.

      DP-407260: Stop logging messages for normal operation.

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      This change eliminated logging some messages that indicated the system was operating correctly.

      DP-407397: Make sure the pidtab expansion in STU occurs before running user code

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change ensures that the pidtab expansion in STU occurs before the user code (e.g. SYSTEM^%ZSTART) is executed.

      DP-407503: Reset mirror debug flags when ^SystemCheck finishes

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change resets the mirror debug flags when ^SystemCheck is finished.

      DP-407957: Don't allow details from another error in <PROTECT> $ZERROR

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      Some information in $ZERROR from a <SUBSCRIPT> error could persist in the $ZERROR of a subsequent, unrelated <PROTECT> error.

      DP-408014: Corrected routine counters used by ^mrstat

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      Corrected a couple of routine counters used by ^mrstat. It's a minor fix, rtn size dependent counters weren't being added to correct buckets.

      It affected the following rtn counters ("RMed from proc LRU...", "RMed from shared LRU...", "added to proc LRU...")

      DP-408034: Allow range ABC.* to work in DataMove

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      This change allows range such as ABC.* to work in DataMove

      DP-408315: Validate global correctly in getGlobalSysSfn^%SYS.SECURITY

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      With this change, getGlobalSysSfn^%SYS.SECURITY validates global references in a more robust way.

      DP-408618: Fix $system.CPU core count; enable hyperthreading properly

      Category: Kernel
      Platforms: Linux
      Version: 2021.1.3

      Correct the core count (and threads per core and cores per chip) reported in $system.CPU for Linux as well as the hyperthreading determination.

      DP-408775: Get processor frequency on AIX without needing security permissions

      Category: Kernel
      Platforms: AIX
      Version: 2021.1.3

      This change modifies the implementation of $system.CPU.Refresh() so that you can run this method as the non-root user and still get the processor frequency.

      DP-408829: Integrity check must not read global blocks into global buffers

      Category: Kernel
      Platforms: All
      Version: 2021.1.1

      Prior to this correction, under extremely rare circumstances, integrity check run on an active system could cause a global reference or update in another process running concurrently to get unexpected results. This problem has existed since at least Caché 2011.1, though it's never been reported in the field.

      Note that a side effect of this correction is that integrity check no longer loads pointer blocks into the database cache as it checks them, whereas it generally did in the past (but almost never loaded data blocks into the buffer pool).  Therefore integrity check may do a few percent more reads, and may have somewhat less of an impact on other jobs running on the system.

      DP-409100: Run object destructors when a .Net application quits

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      Actions performed by an object's %OnClose method, such as unlocking a record in a persistent class, were not being performed in a timely manner for a client/server application using .Net, JDBC, etc.

      DP-409717: Ignore buffer size change for client/server TCP connection

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      If client code in an xDBC client/server application issued a USE command which changed the device buffer size, the application process would fail with an access violation.

      DP-410050: Correct a rare bug on class initialization

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change corrects a very rare bug that could cause the wrong version of a class to be loaded into memory. This could lead to increased memory usage if class components are not being used. Otherwise, it only changes the time at which they are loaded into memory.

      DP-411991: Retry configuring large page for AIX when the errno is EBUSY.

      Category: Kernel
      Platforms: AIX
      Version: 2021.1.3

      On AIX system, retry configuring large page for AIX when the errno is EBUSY.

      DP-412571: irisHLL (HyperLogLog) windows estimate divergence fix

      Category: Kernel
      Platforms: Windows
      Version: 2021.1.3

      Fix HLL windows estimate divergence (to max 64 bit int) by rewriting windows clzl function.

      DP-412701: Update OpenSSL source codes from version 1.0.2zb to 1.0.2zd.

      Category: Kernel
      Platforms: All
      Version: 2021.1.2

      Update OpenSSL source code from version 1.0.2zb to 1.0.2zd. In addition to being used in the kernel, the OpenSSL library is used in the Apache-based private web server for any https connections, in the ODBC client for TLS-based connectivity, and in the PKI service.

      DP-412811: Increase max KMIP Key ID to 188 (LTO5)

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      The InterSystems IRIS KMIP interface module was previously limited to 37 character Key IDs when retrieving a list of keys from the KMIP server.  This was due to the use of a length constant in the KMIP library that wasn't appropriate.

      The max key length has been updated to 188 to comply with LTO5 in the KMIP spec.

      DP-412948: Correct display for the GBLOCKCOPY monitor

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change corrects the output display of the GBLOCKCOPY monitor.

      DP-413104: Fix job hang when unlocking large amount of owned remote locks.

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      Fix job hang when unlocking large amount of owned remote locks.

      DP-413143: Upgrade zlib to 1.2.12

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change upgrades zlib to version 1.2.12.

      DP-414224: Handle error() call in Windows mailbox thread

      Category: Kernel
      Platforms: Windows
      Version: 2021.1.3

      Under very unusual conditions, a $VIEW(-1,pid) of another process could cause that process to terminate with an exception.

      DP-414650: Fix error that could cause ECP daemon not finding lock entry for delock.

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change fixes an error in ECP that would sometimes result in a message like the following: Could not find lock entry for delock request, tid=70000, token=6f3261, sfn=3, lock=^ER1DCS(921208056)

      DP-414757: Prevent dead lock on MISC resource during pidtab expansion

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change prevents a dead lock on MISC resource during pidtab expansion.

      DP-414766: Reset mirror debug in ^SystemCheck when error occurred while running.

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change resets the mirror debug flag in the ^SystemCheck routing when error occurs.

      DP-414973: Avoid missing journal entry in primary when mirrored is re-mounted.

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      Avoid missing journal entry in primary when mirrored is re-mounted.

      DP-415121: Update OpenSSL source codes from version 1.0.2zd to 1.0.2zf.

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change updates OpenSSL source codes from version 1.0.2zd to 1.0.2zf.

      DP-415977: Support /ENV argument for CPIPE open

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change adds a /ENV argument to the CPIPE Open command. It specifies environment variables to be set in the new process. There are two ways to specify the values - explicitly or in an array. The examples show two names but there can be any number. The explicit list must be enclosed in parentheses.

      Explicitly: 

        /ENV=(name1:value1,name2:value2)
      In an array: 
        Set arr(name1)=value1
  Set arr(name2)=value2
  /ENV=arr...

      DP-416121: Fixed an issue in ECP that resulted in <WIDE CHAR> error when using $ORDER in reverse

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue in ECP that sometimes resulted in a <WIDE CHAR> error when using $ORDER in reverse.

      DP-416283: Support /ENV on $ZF(-100)

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change adds /ENV to the $ZF(-100) flags argument. It specifies environment variables to be set in the new process. There are two ways to specify the values - explicitly or in an array. The explicit list must be enclosed in parentheses.

      This is similar to /ENV for CPIPE (DP-415977) but subtly different for explicit values. With CPIPE the name and value are expressions - they can be string literals (quoted strings) or variable names or any expression. With $ZF(-100), because the entire flags argument is a string, the name and value are always treated as strings, not variable names. If they contain characters that would interfere with parsing the rest of the string, such as colon or comma, they must be enclosed in quotes, which must be doubled because they are within the flags string.

      The format is /ENV=(name:value) There can be multiple name:value pairs separated by a comma.

      This example will run the command with variable MYNAME equal to "Tom" and MYARG equal to "comma,". 

      do $ZF(-100,"/ENV=(MYNAME:Tom,MYARG:""comma,"")",command)
      An alternate format is /ENV=var... where var is an array. The subscript is the environment variable name and the node value is the value to be set.

      This example produces the same result as the previous one. 

      set arr("MYNAME")="Tom"
set arr("MYARG")="comma,"
do $ZF(-100,"/ENV=arr...",command)

      DP-416662: Fix check for $C(num)=0 when num#256=48

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue on x64 platforms. Before this change, set var=$C(num),result=var=0 would set result=1 for all num values where num#256=48.

      DP-418023: Correct variable states in %ETN or %STACK

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change corrects an error that cause the error trap %ETN, or the %STACK utility, to omit the data value for a variable, given a particular order of NEW commands in the execution stack.

      Before this change, doing a $$LOG^%ETN() stack dump could sometimes die, resulting in an incomplete dump in the ^ERRORS global.

      DP-419462: Global iterator: Correct compact unicode/vector big string value handling and don't cache big strings by default

      Category: Kernel
      Platforms: All
      Version: 2021.1.3

      This change correct compact unicode/vector big string value handling and also no longer caches big strings by default.

      Licensing

      DP-403498: Add license DisplayName for international characters

      Category: Licensing
      Platforms: Windows
      Version: 2021.1.1

      Use DisplayName from [License] section, if it exists, in place of Customer Name for displays of license key. This will be encoded in the file as UTF-8, and then converted to InterSystems IRIS strings where necessary. The DisplayName will be used for all Portal licensing pages, and will appear in command line startup where we print "licensed for use exclusively by". For command line startup we use the original UTF8 string, so the terminal must be able to display this properly.

      genkey is modified to accept an optional /DisplayName argument (similar to /LicenseID).

      Note that DisplayName will be limited to 128 bytes (not characters). Also, even though the DisplayName is in the [License] section it is NOT protected by the checksum in the key. This is to maintain compatibility of keys with previous versions (both the InterSystems IRIS kernel and License Manager). However, you should not modify this string, as it will make InterSystems IRIS think it is a different key, even though the significant aspects of the key are the same.

      Removed MachineID from Portal License Key page as part of this change, as it has not been used in a long time.

      DP-405634: Fix handling of DisplayName in license comparison

      Category: Licensing
      Platforms: All
      Version: 2021.1.1

      Even with a properly loaded and activated license, a warning message was displayed in the License Key portal page: "The license key in MGR will not take effect until it is activated or the system is restarted." The change fixes the problem.

      DP-411783: Allow change in IPNEUTRAL for upgrade from no license

      Category: Licensing
      Platforms: All
      Version: 2021.1.3

      Active license upgrades do not allow changes in the IPNEUTRAL feature bit, as you would then have a confusing mixture of different user License IDs. But we should allow an upgrade for a license download from the License Server.

      Management Portal

      DP-404817: Correctly enable Management Portal breadcrumbs even when browser rejects cookies

      Category: Management Portal
      Platforms: All
      Version: 2021.1.3

      This change fixes a bug in which clicking the breadcrumb links in the Management Portal would require the user to log in again if their browser was set up to reject cookies. The scope of this fix is limited and does not include Analytics and Interoperability pages.

      DP-413589: Support paging for results tables produced by %CSP.UI.Component.SQLTables

      Category: Management Portal
      Platforms: All
      Version: 2021.1.3

      This adds the ability for the SQLTables component to produce paged results, which reduces the rendering burden of huge HTML tables and improves user experience.

      DP-417081: Update Database page to sort correctly

      Category: Management Portal
      Platforms: All
      Version: 2021.1.3

      This change corrects the Database page to display data correctly when you sort by available space.

      Mirroring

      DP-412174: Changing async member type between DR and Reporting now restarts dejournaling

      Category: Mirroring
      Platforms: All
      Version: 2021.1.3

      This correction ensures that when you change the async member type between DR and Reporting, that change also restarts dejournaling.

      Before this correction, in such a scenario, dejournaling was not restarted and consequently, multiple updater jobs were left running.

      DP-412459: Allow users to correct the "ERR" failure in ProcessValidationFailure^MIRROR.

      Category: Mirroring
      Platforms: All
      Version: 2021.1.3

      This change allows users to correct the "ERR" failure in ProcessValidationFailure^MIRROR.

      DP-415939: Allow users to delete all mirror journal files when deleting a mirror set configuration.

      Category: Mirroring
      Platforms: All
      Version: 2021.1.3

      This change allows users to delete all mirror journal files when deleting a mirror set configuration.

      DP-418792: Fix 30 seconds delay on primary sending journal data to async members

      Category: Mirroring
      Platforms: All
      Version: 2021.1.3

      This change removes an unnecessary 30 second delay on the primary member sending journal data to async members.

      Monitoring

      DP-402130: PERFMON/MONLBL events for undefined globals, no-op kills, and name level locks

      Category: Monitoring
      Platforms: All
      Version: 2021.1.3

      Three new metrics are added to the PERFMON / MONLBL / %Monitor.* suite. The PERFMON names for them are: 

      29.) RefNoGlo: searches for non-existent global
30.) KillNoData: global kills finding nothing to kill
44.) LockName: name-level locks

      These events may be of interest in optimizing application code as they represent sources of overhead that may be avoidable. Note: not every reference to a non-existent global results in a RefNoGlo event; RefNoGlo represents only those that required a global directory search or those that were received by an ECP server.

      Note that the names for these metrics are slightly different within %Monitor.Process.

      DP-407883: Fix ^%SYS.MONLBL for selected processes, Pause -> Clear -> Resume

      Category: Monitoring
      Platforms: All
      Version: 2021.1.3

      This change corrects a problem in %SYS.MONLBL when used with a list of selected processes. If you used Pause, and then Clear, and then Resume, the list of selected processes was lost.

      DP-408095: Fix handling of Pause/Resume for MONLBL

      Category: Monitoring
      Platforms: All
      Version: 2021.1.3

      This change fix handling of the Pause/Resume options for selected processes in MONLBL, and related issues for PIDs which are 'mapped' (PSTATMAP bit) vs. 'selected' (PSTATFLAG).

      This change also fixes a check in checkstats() which incorrectly caused processes which are not 'selected' to set PBRMON and map to shared memory (even though they would not report any data once the collection was Resumed).

      DP-412460: Fix monitoring for 'selected' processes in MONLBL

      Category: Monitoring
      Platforms: All
      Version: 2021.1.3

      Before this fix, a session to monitor 'selected' processes in %SYS.MONLBL could allow other processes to attach to shared memory, and then not notify them to detach. The result was that an error could occur after you stopped and restarted MONLBL. Specifically the error was "Shared memory for a previous Monitor run still allocated."

      DP-413960: Limit PERFMON collection (index) numbers

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Monitoring
      Platforms: All
      Version: 2021.1.3

      Limit all PERFMON collections to 64k. Before this change, use of collections >65535 would silently fail; now an error is returned.

      Networking

      DP-420509: Update OpenSSL source codes from version 1.0.2zf to 1.0.2zg.

      Category: Networking
      Platforms: All
      Version: 2021.1.3

      Update OpenSSL source codes from version 1.0.2zf to 1.0.2zg.

      OAuth

      DP-416533: /csp/sys/oauth2/ web application should not require %All

      Category: OAuth
      Platforms: All
      Version: 2021.1.3

      This change modifies the /csp/sys/oauth2/ web application so that it does not run with the %All role. The %All role will no longer be added to the application roles. If the web application already exists this role, the application is updated to no longer use this role.

      The CSPURL parameter has also been added to the following pages, so that they cannot be executed from other /csp/sys/ applications:

    • OAuth2.Response.cls

    • OAuth2.PostLogoutRedirect.cls

    • OAuth2.JWTServer.cls

      • Object Compiler

      DP-405405: Correct %Save when parent ID is reference to multi-property ID

      Category: Object Compiler
      Platforms: All
      Version: 2021.1.1

      A problem has been corrected when a class has a parent-child relationship and the parent class has an IDKey index on a property that is a reference to a persistent class and the referenced class has an IDKey index on two or more properties.  

      This change corrects issues with %SaveData, %ComposeOid, %DeleteData, %ExistsId, %PhysicalAddress, %LockId, and %OnDetermineClass.

      DP-406105: Prevent UNDEFINED error due to conflict with UCQ and re-entry class compile

      Category: Object Compiler
      Platforms: All
      Version: 2021.1.1

      Using the %Studio.Project:InstallFromGbl function it was possible to get an UNDEFINED error looking for qstruct("deleteextent") if the class compiler was called due to UCQ while installing a package.

      DP-406736: Correct error where OutlierSelectivity property clause caused compilation errors

      Category: Object Compiler
      Platforms: All
      Version: 2021.1.3

      Before this change, strings in serial outlier selectivities were parsed incorrectly. For example, given a serial field F1 with an outlier value of "OTL" and an outlier selectivity of .8 would be encoded as: 

         <outlierselectivity>F1:.8:"OTL"<outlierselectivity>
      The value "OTL" would then be parsed incorrectly when projecting to SQL, resulting (usually) in a null value or (rarely, e.g. if the value began with "_") in a compilation error. This change ensures that serial outlier selectivities are parsed correctly.

      DP-416677: Correct error when opening a %Dictionary class

      Category: Object Compiler
      Platforms: All
      Version: 2021.1.3

      This change corrects an error in which the system did not release a lock on a %Dictionary package class. The error occurred when the class dictionary version for the class was out of date and the attempt to automatically upgrade the version failed.

      Object Library

      DP-403980: Remove IRIS.TempNodes<$job> global when process exits

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      Remove entries for globals like IRIS.TempNodes<$job> when a process exits so these do not clutter lists of globals such as in %GD.

      DP-405034: %Net.SMTP checks server identities by default

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      This change fixes a bug where the SSLCheckServerIdentity property in %Net.SMTP defaulted to false, but was documented as defaulting to true. The property now defaults to true and is in alignment with documentation. This means that, when connecting to an SSL/TLS secured web server, %Net.SMTP will check that the certificate server name matches the DNS name used to connect to the server and fail if they don't match. This is the behavior specified in RFC 2818 section 3.1.

      As a result, when connecting to an SSL/TLS secured web server, the default behavior of %Net.SMTP will now be to fail if the certificate server name does not match the DNS name used to connect to the server.

      This change is unlikely to cause compatibility issues, but it is possible that when using %Net.SMTP to send messages you might have issues connecting to an SSL/TLS enabled server. If this happens, and you understand the security trade-offs, you can set SSLCheckServerIdentity to 0 to restore the previous behavior.

      DP-405062: Reduce number of WQM workers we start until they are needed

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      Reduce the number of worker jobs we create in a WQM category so they are only started when needed rather than starting the 'MaxActiveWorkers' number immediately on the first use of a work queue within this category.

      DP-406986: Increase partition memory used by WQM daemon process to the same as WQM worker jobs

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      Increase partition memory used by WQM daemon process to the same as WQM worker jobs. Previously it was using the system default partition limit but all WQM worker/daemon jobs should be consistent in their partition memory limit.

      DP-407199: Make detection of WQM daemon not running more resilient

      Category: Object Library
      Platforms: All
      Version: 2021.1.3

      Calling a work queue manager API will automatically start the work queue manager daemon, if it was not already running. This change shortens a time it takes to check whether the work queue manager daemon is running.

      DP-407220: Rework SSH handling of unicode character sets on Windows

      Category: Object Library
      Platforms: Windows
      Version: 2021.1.3

      The Unicode character set processing on Windows was being subverted by some built-in conversion code in ./shared/System/ that was trying to apply the current Windows code-page to the conversion when we had already done the character set conversion (to UTF8) inside of InterSystems IRIS, resulting in garbled characters.

      DP-407307: Reduce CPU use of WQM daemon process and allow it to go idle more quickly

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      The WQM daemon process which manages the worker jobs was consuming too much CPU on systems with very large process tables. This has been corrected.

      Also the WQM daemon will poll the workers every 0.1 second when the WQM is active but when inactive it will slow down to a 1 second poll rate. This transition to inactive was 2 minutes from the last time it was active. This has been changed to 1 minute from the last active time. It can also become inactive in less than 1 minute if there are no WQM groups in existence.

      DP-407332: The %SYS.Task.ResourceCleanup task should run all mirror members and not just the primary

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      The ResourceCleanup task which is run by the task manager was set to run only on mirror primary systems but there are resources that need cleaning on none-primary systems so we have updated this to run on all systems.

      DP-407848: Allow removing a class when ^oddDEF does not exist

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      Support deleting a class when internal components have got out of sync.

      DP-408482: Fix "minExclusive" and "maxExclusive" that were not correctly created from xsd

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      If a schema has exclusive maxes and mins, then %XML.Util.SchemaReader converts these to inclusive MAXVAL and MINVAL fields. However, this conversion assumed whole-number precision, which is incorrect if fractionDigits is specified. With this change, the SchemaReader checks for fractionDigits and computes maxes and mins that match that degree of precision. If fractionDigits is unspecified, it uses whole-number precision.

      DP-408574: In %Net.HttpRequest check server name certificate before we send any HTTP payload

      Category: Object Library
      Platforms: All
      Version: 2021.1.3

      In %Net.HttpRequest when we form a new https connection and SSLCheckServerIdentity property is on (the default) we verify the servers identity after connecting and performing the ssl handshake. As the ssl handshake needs to be performed before we have access to the server certificate this check was being done after the HTTP payload from the client was sent to the server (or at least the first packet was sent) and so this would send potentially sensitive client information to a server which has an invalid server certificate. To avoid this problem when SSLCheckServerIdentity is on and we are forming a new https connection we will flush the output immediately before any content is sent and verify the server name before moving on to sending the payload.

      Note that this will cause slower performance on the initial request due to this extra round trip to the server and forcing the https handshake to occur before we send the payload. To mitigate this overhead it is suggested clients use the same %Net.HttpRequest object for subsequent requests to the same server as this will reuse the existing SSL TCP channel and so will not need to revalidate the server name.

      DP-408594: Correct cookie sharing across subdomains in %Net.HttpRequest

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      The cookie sharing code between subdomains in %Net.HttpRequest was not compliant with RFC6265. Specifically when doing to sub.mydomain.com if a cookie is set by the server with a 'domain=mydomain.com' this cookie was not being sent to the server when a subsequent request was made to my domain.com.

      Also refreshed the list of root public domains from publicsuffix.org as the data was out of date.

      DP-409587: When calling OutputToDevice on a global stream with a specific length set 'AtEnd' property correctly

      Category: Object Library
      Platforms: All
      Version: 2021.1.1

      When using %Stream.GlobalBinary, %Stream.GlobalCharacter, %Stream.TmpCharacter, %Stream.TmpBinary and calling 'OutputToDevice' passing in a specific length to output if the length exactly matches the size of the stream we were not setting the AtEnd property to true so a loop such as: 

      While 'stream.AtEnd {
  Set sc=stream.OutputToDevice(100)
}

      Could end up looping forever if the stream was exactly 100 characters long.

      DP-414548: Avoid duplicate rebuild of routine/class index

      Category: Object Library
      Platforms: All
      Version: 2021.1.3

      This change removes redundant steps in the logic that builds the routine/class indexes.

      The change also increases the timeout for the routine/class index when doing a rebuild from 5s to 120s, as an index build can take a short while and other processes need to wait for the first process to complete.

      DP-414767: Fix host key crash in FIPS mode

      Category: Object Library
      Platforms: All
      Version: 2021.1.3

      When running in FIPS mode, making an SSH connection could crash because FIPS specifically disallows the traditional MD5 hashes and the libssh2 function returns a NULL pointer that the code was not checking for.

      DP-416814: Fix direct reference to %RuntimeMode in %ScrollableResultSet

      Category: Object Library
      Platforms: All
      Version: 2021.1.3

      This change corrects a refactoring that changed ..RuntimeMode to (incorrect) i%RuntimeMode.

      DP-417775: Allow globals to be killed in HALT cleanup even if GlobalKillEnabled=0

      Category: Object Library
      Platforms: All
      Version: 2021.1.3

      This change ensures that the system cleans up globals correctly (without <PROTECT> errors) during the global cleanup logic in HALT handling, even if the CPF file has GlobalKillEnabled=0.

      Object Storage

      DP-404353: Correct %OnAddToSaveSet callcount parameter callback on %Save of persistent object

      Category: Object Storage
      Platforms: All
      Version: 2021.1.1

      After a previous change the %OnAddToSaveSet 'callcount' parameter which should report the number of times this callback was called for this class was always being passed in as '1' even for cases where the callback was being called multiple times. If the user was using this parameter to decide if we should call %AddToSaveSet on the current instance this failure could result in <FRAMESTACK> errors.

      DP-416355: Avoid UNDEFINED error in %RemoveFromSaveSet() for item not in save set

      Category: Object Storage
      Platforms: All
      Version: 2021.1.3

      Before this change, when calling %RemoveFromSaveSet for a class that is not in the save set (for example, a transient property), the result could be an <UNDEFINED> error. Now %RemoveFromSaveSet() detects such cases and skips removing items that are not in the set of objects to save.

      ObjectScript

      DP-402912: Fix problem where errors within ZWRITE command are not raised to caller's error trap

      NOTE: This item may require a change to code, configuration, or operation.

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      This change fixes a problem where errors within ZWRITE command are not raised to caller's error trap.

      Now ZWRITE throws some errors that were ignored before, but they are serious show-stopper errors that should not happen in a normal environment.

      DP-403184: Fixes cores and errors when compiler optimizes expressions

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      The ObjectScript compiler can optimize expressions like $SELECT(1:23) to the constant 23. But in some situations that optimization could result in incorrect object code leading to access violations or errors. This change corrects the problem.

      DP-403374: Fixes complex expression that could generate bad code (#10)

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      In the ObjectScript code $DATA(^global)=expr where expr is a complex expression that the compiler knows will evaluate to zero, it may generate invalid object code that will result in an access violation. This change corrects the problem.

      DP-403666: Fix legacy NodeJS to properly handle exceptions thrown in user code

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      This change ensures that the legacy Node.js binding will properly handle exceptions thrown by user code. Previously, exceptions that were not of the type %Exception.SystemException would result in inconsistent behavior, including a '' response, a <MAXSTRING> error, or a crash. 

      Currently the server reports errors by sending the numerical code associated with the ObjectScript error to the client as a single byte. The client client has logic to reconstruct the error string, which is then reported to the application. 

      Now, if user code throws any exception other than a SystemException, the client will report a <THROW> error, with the associated message "The application encountered an unexpected exception.". This change also ensures that the server will not return an error code greater than 255. A SystemException should not use a value this high, but if one is encountered, the server will send the error code 255 to the client, which will be interpreted as a <SYNTAX> error. 

      Note: This change only affects connections using TCP. Connections using the callin API do not report unhandled exceptions correctly. Applications using the callin interface should not directly invoke code that could raise an exception.

      DP-404003: Fix <SYSTEM> error after compiling complex expression

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      When a complex boolean expression containing $Select() is optimized by the routine compiler, invalid object code could result in a <SYSTEM> error. This change corrects the problem.

      DP-404209: Fixes Equals optimization that could be wrong

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      When the ObjectScript compiler optimizes a complex Equals comparison involving $Select() in which it knows what the result of each side will be, it could produce an incorrect result. This change corrects the problem.

      DP-404552: Fix compiler optimization of $Extract() with $Select()

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      If the third argument of $Extract() is a $Select() that can be optimized to a numeric constant, bad object code could be compiled, producing unpredictable results. This change fixes the problem.

      DP-405828: Fix $Data(^global) optimization

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      When $Data(^global) is followed by a $Select expression that the compiler can optimize to a constant, the compiler could generate invalid object code causing an error or access violation. This change corrects the problem.

      DP-406440: Raise error when going to Unix implied namespace with null character

      Category: ObjectScript
      Platforms: UNIX®
      Version: 2021.1.3

      This change causes InterSystems IRIS to raise an error when a user on a Unix build attempts to go to an implied namespace containing a null character.

      DP-406903: Fix variable=$SELECT() optimization problem

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.1

      The specific syntax in a routine, but not in an Xecute, 

        Set local=$Select(1:$Select(0:expression,1:expression2))

      or 

      Set local=$Select(1:-$Select(0:expression1,1:expression3))

      could be compiled into incorrect object code under certain conditions. This change fixes the problem. These conditions include:

      • If the expression2 can be optimized by the compiler to a numeric constant between 0 and 255 inclusive.

      • If the expression3 can be optimized by the compiler to a numeric constant between -255 and 0 inclusive.

      • If the 0 condition for expression1 is an expression which the compiler knows will evaluate to 0.

      DP-409576: Workaround Gnu mktime() bug

      Category: ObjectScript
      Platforms: All
      Version: 2021.1.3

      This change corrects a bug in which $ZDATETIME($H,-2) is sometimes off by an hour on some Linux-based systems on the day the Daylight Saving Time change occurs.  

      This change modifies the InterSystems IRIS implementation of $ZDT(htime,-2) in a way that works around the underlying C run-time library.

      ODBC

      DP-417170: Correct fastSelect with UNION query and literals in select list

      Category: ODBC
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue where a SELECT query over xDBC that is a UNION query and the first leg of the UNION query has literal values in the SELECT list might return incorrect results.

      Security

      DP-20972: Add OAuth functionality to %Net.SMTP and %Net.POP3

      Category: Security
      Platforms: All
      Version: 2021.1.3

      %Net.SMTP and %Net.POP3 now support XOAUTH2 as an authentication method. %Net.POP3.Connect() now accepts an access token as a parameter and %Net.Authenticator (which is used by %Net.SMTP for SMTP AUTH) now has an access token property and allows XOAUTH2 to be included in its mechanism list. Users who wish to use XOAUTH2 for POP should pass an access token to %Net.POP3.Connect() and users who wish to use XOAUTH2 for SMTP should set the access token property of %Net.Authenticator and specify XOAUTH2 as the mechanism to be used.

      Usage notes: - If %Net.POP3.Connect() is passed an access token it will attempt to use XOAUTH2 regardless of whether or not a password is supplied. Therefore, users who do not wish to use XOAUTH2 should be careful that they do not set the new AccessToken parameter that comes after Password. - %Net.Authenticator will only use XOAUTH2 if its mechanism is set to be XOAUTH2 - Users who don't wish to use XOAUTH2 need not make any changes.

      DP-402305: Deleting OAuth 2.0 clients now deletes the access tokens issued to those clients

      Category: Security
      Platforms: All
      Version: 2021.1.1

      Deleting an OAuth 2.0 client configuration or client description now deletes all access tokens issued to that client. Since deleting an OAuth 2.0 server configuration deletes all client descriptions on that server, deleting a server configuration now also deletes all access tokens issued by that server.

      DP-403670: Allow setting Dispatch Class for Web Application using ^SECURITY

      Category: Security
      Platforms: All
      Version: 2021.1.1

      This change modifies the ^SECURITY routine to allow users to configure the Dispatch Class for a Web Application.

      When using a Dispatch Class many of the Web Application settings are no longer relevant. Users will not be prompted for these settings when a Dispatch class is specified. This matches the behavior in the Management Portal.

      DP-404024: Provide way to limit folders available from ZEN fileSelect dialog

      Category: Security
      Platforms: All
      Version: 2021.1.3

      This change adds the ability to limit the set of folders that can be viewed by the {{%ZEN.Dialog.fileSelect}} component. Folders can be added to the list by running {{##class(%CSP.Portal.Utils).AddFileSelectDir(dir)}} or {{##class(%CSP.Portal.Utils).RemoveFileSelectDir(dir)}}. The list of configured directories can be retrieved with {{##class(%CSP.Portal.Utils).GetAllowedFileSelectDirs(.dir)}}. Internally, the values are stored in {{^%SYS("Portal","RestrictDirs")}}. If no directories are configured, all directories are visible (this is the current behavior).

      When configured a user should not be able to navigate to a directory outside the set of configured directories and their descendants.

      DP-404140: Support configurable Diffie Hellman key sizes

      Category: Security
      Platforms: All
      Version: 2021.1.1

      This change adds a configuration item to the server-side SSL configurations specifying the minimum size of Diffie Hellman keys.

      DP-404283: Changes to enabling %ZEN.Dialog classes

      NOTE: This item may require a change to code, configuration, or operation.

      Category: Security
      Platforms: All
      Version: 2021.1.1

      In this release, %ZEN.Dialog.* classes cannot be run in web applications unless the web application is enabled for analytics, the namespace is enabled for interoperability productions, or it is explicitly enabled for the web application. To explicitly enable %ZEN.Dialog.* classes, enter the following: 

      Set ^SYS("Security","CSP","AllowPrefix",application-name,"%ZEN.Dialog.")=1

      Also, with this change, any web application that is enabled for analytics or interoperability will have %ZEN.Dialog.* classes enabled by default.

      DP-404341: OAuth2 Introspection response should use 'token_type' claim, not 'token_type_hint'

      Category: Security
      Platforms: All
      Version: 2021.1.1

      a previous change modified the OAuth2 server Introspection endpoint to accept the standard "token_type_hint" parameter instead of the "token_type" parameter, but it also incorrectly modified the response to include a claim named "token_type_hint" This should be "token_type".

      See [RFC 7662|https://tools.ietf.org/html/rfc7662], sections "2.1 - Introspection Request" and "2.2 - Introspection Response".

      DP-404857: Use correct defaults for OAuth2 client JWT algorithms

      Category: Security
      Platforms: All
      Version: 2021.1.1

      This change corrects the default values for the OAuth2 client IDToken and Userinfo algorithms.

      Previously the IDToken, Userinfo and Access Token algorithms would default to using the corresponding value in the Server's JWT settings page, if the client value was "". Note that these values cannot be set to "" in the Management Portal, but they can be configured manually, or via dynamic registration from a third-party OAuth2 client. A previous fix changed this logic to instead default to the server values if the client value was "none".

      Neither behavior is correct. The IDToken and Userinfo algorithm settings are defined in the Open ID Connect Dynamic Registration specification https://openid.net/specs/openid-connect-registration-1_0.html.

      See the definition for: 

      userinfo_signed_response_alg
userinfo_encrypted_response_alg
userinfo_encrypted_response_enc
id_token_encrypted_response_alg
id_token_encrypted_response_enc

      The specification defines what the value of these fields should be if omitted by the client ("none" in most cases.) The specification does not allow for the server to substitute another value arbitrarily.

      This change modifies the logic to not apply the Server JWT settings to the IDToken or Userinfo algorithms. They will still apply to the Access Token algorithms. These settings are not part of the OpenID Connect standard.

      Note that the Request algorithm settings (from the client JWT settings page) are defined in the OpenID connect spec, but they were never overridden by the server settings, so the logic relating to them is unchanged.

      DP-405517: Preserve CSPCHD token in OAuth2 redirects

      Category: Security
      Platforms: All
      Version: 2021.1.1

      This change corrects a problem that would prevent OAuth2 applications using Authorization Code or Implicit authorization flow from interfacing correctly with CSP applications when cookies are disabled. When cookies are disabled the CSP session token is included in the CSPCHD URL parameter. Previously, this would be removed when redirecting back to the application after successful authorization, which resulted in the session being lost. This has been corrected. Now OAuth2.Response uses the %response.UseExactRedirect flag to prevent any translation of the specified redirect URL.

      In order for the redirect to work, the application should call ..Link() on the redirect URL when calling GetAuthorizationCodeEndpoint() or GetImplicitEndpoint(): 

       set url=##class(%SYS.OAuth2.Authorization).GetAuthorizationCodeEndpoint("MyApp", scope, ..Link(myRedirectURI),,,.sc)

      This will ensure that the CSPCHD token is included in the URI. This is only necessary if this is a CSP/ZEN application and cookies are disabled.

      DP-406140: Ensure CSV Export button appears when switched on in Widget Settings

      Category: Security
      Platforms: All
      Version: 2021.1.1

      When you go into a dashboard, and go to Widget Settings for a pivot table widget, there are switches for Excel Export and CSV Export. When both switches are on, the export button appears with Excel, CSV, and CSV with Headers options. With just the Excel switch on, the export button appears and exports to Excel as before. However, when just the CSV switch is on, no button appears. This change corrects the problem.

      DP-409573: Do not use CSP sessions for OAuth2 Client pages

      Category: Security
      Platforms: All
      Version: 2021.1.1

      This change corrects the OAuth2.Response and OAuth2.JWTServer CSP pages to not use persistent CSP sessions. These pages are served via the /csp/sys/oauth2/ application on OAuth2 Client systems.

      DP-411575: Update Apache httpd to 2.4.52

      Category: Security
      Platforms: All
      Version: 2021.1.1

      This change updates Apache httpd used as private web server to version 2.4.52.

      DP-411748: Fix 32-bit ODBC driver crashes on Windows Server 2022

      Category: Security
      Platforms: Windows
      Version: 2021.1.1

      This change fixes a problem in which irisconnect would crash if the user attempted to test an SSL connection without defining the SSL/TLS configuration in the registry or SSLdef.ini. Now irisconnect gracefully reports an error if the configuration is not found.

      DP-415141: Return status correctly in %SYS.Audit:Import()

      Category: Security
      Platforms: All
      Version: 2021.1.3

      This change fixes a bug in the Import() method of %SYS.Audit, where an error status code was not being returned for an error condition.

      DP-416839: Reset SSL cache on failed connection to LDAP

      Category: Security
      Platforms: All
      Version: 2021.1.3

      This change fixes a bug in which it was possible to get an "UNKNOWN CA" error when logging in, when using LDAP to authenticate.

      This change provides a new option the method %SYS.LDAP:SetOption. The option is described in the class reference.

      For the example of "LDAP host names" parameter, to specify multiple host names, separate the names by spaces. If the LDAP server is configured to use a particular port, you can specify it by appending ":portname" to the host name; typical usage is not to specify a port and to let the LDAP functions use the default port, such as: 

      ldapserver.example.com
ldapserver.example.com ldapbackup.example.com

      If you have multiple replicated domain servers on your network like 

      ldapserver.example.com
ldapserver1.example.com
ldapserver2.example.com
ldapserver3.example.com

      You can specify the domain "example.com" as your host name. LDAP will perform a DNS query asking for the addresses of all the matching ldap servers, and then automatically select one to connect to.

      DP-418534: Set Content-Type header correctly in OAuth2.JWTServer

      Category: Security
      Platforms: All
      Version: 2021.1.3

      This change modifies OAuth2.JWTServer to set the Content-Type header in the OnPreHTTP() Method instead of in the OnPage() method. 

      This change prevents a <PROTECT> error.

      Sharding

      DP-405684: Fix %SYSTEM.Sharding.GetOption to correctly recognize MasterNamespace argument

      Category: Sharding
      Platforms: All
      Version: 2021.1.1

      This change fixes a problem in which the $SYSTEM.Sharding method GetOption(), if executed in a namespace other than the one specified in its MasterNamespace argument, failed to set $NAMESPACE to the specified namespace, causing it to return incorrect results.

      DP-406362: Generate correct shard-local global names for sharded classes when USEEXTENTSET=0

      Category: Sharding
      Platforms: All
      Version: 2021.1.1

      This change fixes a problem in which some sharded classes had shard-local global names which did not correctly map to the shard-local databases. This resulted in data being incorrectly stored in the master database rather than in shard-local databases, causing queries to return incorrect results (each row was returned as many times as the shard count).

      DP-406794: Fix method used for Management Portal sharded table data distribution query

      Category: Sharding
      Platforms: All
      Version: 2021.1.1

      This change fixes the %SQL.Manager.Catalog method ShardTableDataDistributionExecute, to cause it to return correct per-shard row counts for sharded tables.

      DP-407540: Allow specifying master namespace in %SYSTEM.Cluster Attach methods

      Category: Sharding
      Platforms: All
      Version: 2021.1.1

      This change fixes a problem in which specifying the master namespace, rather than the cluster namespace, in the cluster URL passed to a %SYSTEM.Cluster AttachAsXXXNode method resulted in the error "ERROR #9600: $SYSTEM.Cluster methods cannot be used on cluster that has been modified using $SYSTEM.Sharding API". With this change, when specifying a cluster URL for node 1 of a sharded cluster, it is permitted to specify the master namespace (default "IRISDM") as an alternative to specifying the cluster namespace (default "IRISCLUSTER").

      This fixes an issue found using the Management Portal. However, note that while that issue relates to attaching drasync node to a cluster using Management Portal, the problem, as well as this fix, applies to direct use of the %SYSTEM.Cluster API, as well as to indirect use via Management Portal, and applies to all node types, not just mirrored, drasync nodes.

      DP-410108: Check mt("unary") before creating a new collated f node

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      Before nufun method calls nufld method to create a new entry (374 here), we need to check mt("unary"), if there indeed exists an entry for this fn(field), we should use the value in that entry instead of creating a new one. In our example,mt("v",374) should not be created and the expected value for mt("v",362) should be 

      mt("v",362) = "n^^369"

      DP-412863: Update VerifyShards() to check inter-shard ECP connections

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This change fixes a bug in $system.Sharding.VerifyShards(), related to ECP connections.

      DP-413773: Avoid reporting negative global references count for sharded queries

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This changes fixes a problem in which a negative global references count was sometimes reported for sharded queries, in $SYSTEM.SQL.Shell() or in Management Portal.

      DP-415310: Pass %NOINDEX query hint to shards

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This change fixes a problem in which the %NOINDEX query hint was not passed to shard-local queries, causing it to be ignored when executing sharded queries.

      DP-416531: Don't set properties fields to "" for defaults when mapping globals

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This change causes sharding logic, when setting up a properties array for a call to ##class(Config.MapGlobals), to not set any properties for which the default is intended, rather than setting such properties to "".

      DP-416775: Fix shared lock leak in $system.Sharding.VerifyShards()

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This change fixes a problem in which VerifyShards() could sporadically fail to release a shared lock on a shard's entry in the sharding metadata global.

      DP-417495: VerifyShards: don't set master backup address when called recursively

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This change fixes a problem in which VerifyShards failed with a <SUBSCRIPT> error, if called in a mirrored sharded cluster, in which the master node has no backup failover member.

      DP-417989: Sharding only activate namespace once after series of mapping operations

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This change fixes a problem in which, when a master namespace contains many user-defined mappings, it could take an unacceptably long time for sharding to propagate them to shards.

      DP-418033: Disable sharding debug tracing in Backup.ShardedCluster methods

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This change fixes a problem in which a sharded cluster could become hung in a quiesced state if the Quiesce, JournalCheckpoint, or ExternalFreeze methods of Backup.ShardedCluster were called while sharding debug tracing was enabled.

      DP-419674: Prevent redundant VerifyShards message for mirrored shards

      Category: Sharding
      Platforms: All
      Version: 2021.1.3

      This change fixes a problem in which $system.Sharding.VerifyShards() mistakenly sent a second, redundant message to mirrored shards, in cases where this was unnecessary.

      SQL

      DP-402424: Fixes problem where the results of TuneTable execution on Management Portal aren't displayed correctly

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      Depending on the number of records, Selectivity and Extentsize of running TuneTable on Management Portal did not appear correctly. This change corrects the problem.

      DP-402500: Fix incorrect purge of UCQ for embedded SQL

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change fixes a problem where any UCQ (universal cached query) for embedded SQL in any namespace was purged unexpectedly when the system task 'Update SQL query statistics' was executed the first time in a day. These are now only be purged if the caller was removed and the caller is no longer in the routine buffer cache.

      DP-403030: Fix problem where last line of statement could be incorrectly read as a comment

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change fixes a problem executing SQL statements with comments with "--" prefix

      If a query statement run in a client contains comments such as in this example:

      ---GROUP BY Patient HAVING COUNT(ID) BETWEEN 4 and 6

      or any other text preceded by the "--" characters, the query won't return any results and the client never gets a response from the server.

      DP-403832: Fixes error during UCQ execution might be masked by post UCQ <UNDEFINED>

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      In UCQ ensure we do not mask an error running query itself with an UNDEFINED error because the variables we are expecting to copy into the users SQL values are not defined.

      DP-405479: Support for cross-site unique identifiers in FM2Class

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change is version 2.41 of FM2Class.

      There is a new setting, which you can define directly in the global for now, to define the datatype class for the IEN: 

      set ^%SYS("sql","fm2class","IENType")=“XX.AAA"

      This class name can be anything you want.  If this class name is not defined, the default %Library.String is used. For example, you could choose a name something like PackageName.IENType.  Use whatever package name you want, and just make sure it does not collide with any existing classes. If the class does not exist, FM2Class will create it for you (which is what should happen the first time you run FM2Class with this version of the mapper). So there is nothing to do other than define this global to the name of the class you want to use.   Then run FM2Class like you normally would.   Here is an example of the data returned (in Display or ODBC mode, Logical mode returns the on-disk IEN value). 

        
[SQL]SQL:Hosp1>>select ien, SPOUSES_EMPLOYERS_STATE from Hosp1.patient_2 where SPOUSES_EMPLOYERS_STATE is not null
13. select ien, SPOUSES_EMPLOYERS_STATE from Hosp1.patient_2 where SPOUSES_EMPLOYERS_STATE is not null
 
IEN SPOUSES_EMPLOYERS_STATE
20 36
210 36
312 26
357 11
380 11
426 36
428 36
743 24
 
8 Rows(s) Affected
statement prepare time(s)/globals/cmds/disk: 0.0500s/35,128/173,001/2ms
          execute time(s)/globals/cmds/disk: 0.0008s/1,787/11,655/0ms
                          cached query class: %sqlcq.Hosp1.cls17
---------------------------------------------------------------------------
[SQL]SQL:Hosp1>>set selectmode odbc
 
selectmode = odbc
[SQL]SQL:Hosp1>>select ien, SPOUSES_EMPLOYERS_STATE from Hosp1.patient_2 where SPOUSES_EMPLOYERS_STATE is not null
14. select ien, SPOUSES_EMPLOYERS_STATE from Hosp1.patient_2 where SPOUSES_EMPLOYERS_STATE is not null
 
IEN SPOUSES_EMPLOYERS_STATE
20|664 36|664
210|664 36|664
312|664 26|664
357|664 11|664
380|664 11|664
426|664 36|664
428|664 36|664
743|664 24|664
 
8 Rows(s) Affected
statement prepare time(s)/globals/cmds/disk: 0.0001s/4/137/0ms
          execute time(s)/globals/cmds/disk: 0.0014s/1,787/14,342/0ms
                          cached query class: %sqlcq.Hosp1.cls17

      DP-405703: Fix fastInsert to support DSTIME

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      A problem has been corrected where a class defined to use DSTIME was not properly setting the ^OBJ.DSTIME global when an INSERT was performed over xDBC and the fastInsert feature was used.

      DP-405825: Correct lock issues on cached queries and statement index

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change fixes a rare issue with the various APIs in %SYS.SQLSRV for Statement Index IUD and in Prepare of the dynamic statement cached query lookup/creation.

      When using the WQM and the Sharding mechanism, there are times when an <EXTERNAL INTERRUPT> error is sent to shards to stop processing the query (typically because we have received all of the rows required for the table already). Depending on the query, this <EXTERNAL INTERRUPT> can happen multiple times. The <EXTERNAL INTERRUPT> error caused problems when the interrupt occurred after the LOCK, but before the set of gotlock.  When gotlock not TRUE, the global never got unlocked in the API.

      This change ensures the global is unlocked in the no-error case and in the error case.

      DP-405835: Disable external interrupt errors around sharding query construction logic

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      In the code run on a shard system to build the shard local query, this change disables getting EXTERNAL INTERRUPT errors so we can ensure the query is always built.

      DP-405944: Fix collation for %PARALLEL on a numeric column that could cause a rows to be skipped

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      When splitting a global into chunks to process in parallel we detect if the last node is a numeric value we added one to it to avoid $double rounding errors. This can cause problems with numeric values because although 1.1]]0.1 is true "1.1"]]"0.1" is false. The reason being that "0.5" is not a canonical number due to the leading '0' character where as "1.1" is a canonical number so for example: 

      set a("0.1")="string zero point one"
set a(0.1)="numeric zero point one"
set a("1.1")="string one point one"

      zw a
a(.1)="numeric zero point one"
a(1.1)="string one point one"
a("0.1")="string zero point one"

      The result of this is if the last range in a parallel query was a string value with a leading '0' which can be converted into a number between 0 and 1 e.g. "0.1" when we added 1 to this to ensure we do not get $double rounding problems we created a string value "1.1" which collates before the end of the global i.e. "0.1". This means rows from a query are not processed when the query is run in parallel. This change fixes the problem.

      DP-406231: Apply conditions in shfls() for override plan

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      Fixed a rare bug for a query in which GROUP BY <g> is used, no aggregates are used, there are some ungrouped fields <sf> that appear in the SELECT-list or the HAVING clause, there are WHERE conditions on the <sf> fields (and possibly also on the <g> fields), and there is a map that has the <g> fields as leading subscripts, and has all the <sf> fields as later subscripts.  Prior to this change, WHERE conditions on the <sf> fields were not applied, possibly resulting in incorrect extra output rows.

      Note that while the SQL Standard does not allow non-grouped fields to be used (unaggregated) in the SELECT or HAVING clause, this bug is possible even for SQL Standard queries, for example, if the <g> fields functionally determine all the <sf> fields, that is. if the <g> fields constitute a UNIQUE key.  In that case the SQL Standard query:

      SELECT <g>,<sf> FROM t WHERE cond(<sf>) GROUP BY <g>,<sf>

      would be internally converted to the query:

      SELECT <g>,<sf> FROM t WHERE cond(<sf>) GROUP BY <g>

      and the bug could occur.

      DP-406409: Correct handling of <EXTERNAL INTERRUPT> errors in SQL runtime logic

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      A problem has been corrected where an SQL statement that was interrupted by a "Cancel" via xDBC, SMP, or via $SYSTEM.Util.SendInterrupt(PID) did not always properly release locks and/or end transactions.

      Two new SQLCODE values have been introduced by this change:
      SQLCODE=-457, SQL statement execution interrupted
      SQLCODE=-458, Server request interrupted

      Prior to this change, in the xDBC Server, the same SQLCODE was used for an <ALARM> (timeout), -450, as for an <EXTERNAL INTERRUPT> (A Cancel). Now -450 is used for Timeout, and -458 for Cancel.

      This change also corrects a problem where the server process would not always terminate if the server process was Terminated with a <RESJOB> error.

      DP-406534: Correct fastSelect <UNDEFINED> error in Prepare when all fields come from index subscripts

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      The problem is exposed as an <UNDEFINED>output+66^%qaqcct error during Prepare of a SELECT statement via xDBC.

      The problem was fastSelect was used for a case where it should not have been.  The query plan simply loops on index subscripts and outputs the row, there is no going back to the master map and no map data field retrieval. This change corrects the problem.

      DP-406629: Fix 'opg' set-looping code-generation for IN

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      Fixed a bug when GROUP BY and aggregates are used, when the plan loops over subscripts with the GROUP BY fields, and the plan also loops on a subsequent subscript that has an IN condition (or an equivalent OR condition).  Such a query would incorrectly return 0 rows.

      DP-407157: Analysis of query grouping accounts for shard key collation

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      Queries containing GROUP BY on sharded tables with non-integer shard keys could perform redundant re-grouping of results from each shard. This change corrects that behavior.

      DP-407232: Correct hash for %XSQL.DSI.TempTable and %XSQL.DSL.Cursor CQ implementations

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      A problem has been corrected where the hash used for a %XSQL.DSI.TempTable or %XSQL.DSI.Cursor cached query implementation was not defined with the same hash as it was looked up upon Prepare. For example, suppose a call is made to: 

      set oref=
##class(%XSQL.DSI.TempTable).%Prepare("CREATE PRIVATE TEMPORARY TABLE...")
      The first call created a cached query for the implementation of the private temporary table. A second call to %Prepare with the same table definition would fail to find the cached implementation because of a hash mismatch between when the PTT was defined and when it was looked up.

      This type of private temporary table is used for statement-level triggers and caused a new cached implementation of the PTT to be generated for every insert/update/delete executed on the table (if there is a statement-level trigger defined for the event). This led to degraded performance and bloat of the size of the statement caches.

      DP-407319: Prevent <UNDEFINED> error by not applying special distinct aggregate processing

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change prevents an <UNDEFINED> error caused by applying special distinct aggregate processing when group matches the shard key ("gruplup").

      DP-407368: Correct endian ordering issue checking routine header signature against stack information

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      On big endian systems it was possible we would fail to build a universal cached query correctly as we thought another process had already modified the UCQ. This would be reported as errors such as: 

      ERROR #5821: Cannot instantiate query: 'SQLCODE = -400'

      We have corrected the check and also added %msg to this error message so make debugging similar issues in future simpler.

      DP-407384: Fix case where query optimizer is incorrectly using the normal selectivity

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change corrects the behavior of prmIncOutlier and checks the parameter code includes outlier behavior.

      DP-407453: Fix a caching problem for a DISTINCT query with a SELECT DISTINCT view in FROM

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change fixes a caching problem that causes some UNION ALL queries to run slowly. This problem impacted Runtime Plan Choice (RTPC) query optimization.

      DP-407554: Handle DISTINCT BY() in sharding query correctly

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      In some cases when the sharding key and gruplup matched, the DISTINCT BY clause would not be handled correctly and extra rows could be returned. This change corrects the problem.

      DP-407729: FIX: NOT %INLIST and NOT IN disparity when there is a NULL in the list

      NOTE: This item may require a change to code, configuration, or operation.

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change makes the behavior of NOT %INLIST $listbuild(x,NULL,:z) consistent with the behavior of NOT IN (x, NULL, :z).

      DP-407796: Correct <PARAMETER> error with some embedded SQL UCQ executions

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      A problem has been corrected where the execution of an embedded SQL statement that included a host variable that is a property object reference. A problem with the code generation of the Universal Cached Query was putting too many arguments in the call to the %New() method of the UCQ.

      DP-408028: Escalate roles to post remote event

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      Posting events from shard worker nodes to the system that initiated the query requires the API escalate privileges to avoid getting PROTECT errors such as: 

      <PROTECT>postRemoteEvent+31^%SYS.WorkQueueMgr
      DP-408033: Server-side corrections for CALL %SQL_Util.CSV(...)

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      I was looking at the issue reported in  [#DP-2293]and discovered the <UNDEFINED> error mentioned in the description of the Jira.  This error has been corrected.  I also found a run-time <UNDEFINED> error in %SQL.CSVResultSet.%SendODBC method.  This too has been corrected.

      DP-408257: Fix race condition in SQL parallel union processing

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      When running SQL 'union %parallel' or 'union all %parallel' it was possible in rare cases to get an error that 'Worker job with IPQ exited unexpectedly' due to a race condition in the parallel logic. This is corrected now.

      DP-408385: Correct procedure privilege check for schema level privileges on 404 check

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      A problem has been corrected where a call to an SQL procedure from xDBC might not work correctly if the user received the EXECUTE privilege on the procedure through a schema-level privilege.

      DP-408513: Cleanup invalid cached query classes

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      Make code which builds cached query classes more resilient to existing invalid classes.

      DP-408527: Collect light weight SQL stats on read-write mirror members

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change collects light weight SQL stats on read-write mirror members.

      DP-409182: Kill iDj fast i%var variables in SQL Close

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      When SQL uses fast i%var variables in cached query definitions, there were property values used by the SQL statement that were not killed in SQL CLOSE.  Most of the time this is okay, but a customer reported an error with their application because they had some {iDj} variables defined in SQL Storage that held an oref, and the expectation was this oref would be destructed when the SQL statement closed.  Any {iDj} map variables that are code generated using i%vars in the cached query are now killed during close.

      DP-409582: Correct %XSQL.DSI.Cursor implementation

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      The %XSQL.DSI.Cursor implementation was broken by a previous change, this change fixes this so queries generated by %XSQL.DSI.Cursor now act as normal cached queries.

      DP-409641: Correct resolution of Global Private Table name when statement is cached

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      A problem has been corrected when using GLOBAL PRIVATE TEMPORARY tables and inserting, updating, or deleting data in the GPT table when the INS/UPD/DEL statement is cached.

      DP-409665: Correct SQL compiler to re-throw interrupt or hard errors

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      Added error handling and dection robustness to SQL compiling, and UCQ and dynamic query construction.

      Avoid a situation where a query is being compiled but never alllowed to finish. We now let the compile complete and then throw an EXTERNAL INTERRUPT error.

      DP-409827: Close loophole where SQL privileges could be dropped

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue where a user at the terminal prompt could make a call to an API in %SYS.SQLSEC and if the correct arguments were passed in, could delete a user's SQL privileges.

      DP-409838: FM2Class v2.42 - Improve performance of WipeClass

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This is version 2.42 of FM2Class.

      This change improved the performance of FM2Class for the WipeClass function.  This is the code called to clean up the class definition when re-mapping a class and using the RetainClass=1 option.

      DP-409848: Fix Logic with Phrases in iFind

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      This change fixes previously erroneous logic related to iFind positional search (and as a result, iFind co-occurrence search) which caused the results of all positional search terms OR'd together in a node, except for the last term, to be ignored. For example, in a table with the following records in its MyText field (which has any non-minimal iFind index on it): 

      ||MyText||
|The quick brown fox jumped over the lazy dog|
|Colorless green ideas sleep furiously|

      The following query: 

      SELECT MyText FROM MyTable WHERE %ID %FIND search_index(MyTextIdx, '"lazy dog" OR "green ideas"')

      would only return the second row, and would not include results that contained the "lazy dog" positional search term. Following this change, all positional terms in an OR node are accounted for. This change also makes a minor update to prevent unnecessary checks of positional search strings that only contain the first word in a positional search term.

      DP-410823: Restore default of tab delimiter on %DisplayFormatted method

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      A previous change [DP-21280] modified the default delimiter for outputting result data to a file from the SQL shell to be a comma instead of the tab character. This had the unintended side effect of modifying behavior on users who were calling %DisplayFormatted manually. This restores the previous default delimiter of tab character.

      DP-411647: $System.SQL.Util.SetOption("SQLSECURITY") checks resource earlier

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      DP-411681: Support LAST_DAY() scalar function in query reconstruction

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      A problem has been corrected some queries that run in parallel (or against a sharded table) and use the LAST_DAY(...) scalar function. Before this change, such queries could a query compilation error.

      DP-411780: Correct scoping of public SQL variables %ROWCOUNT, %ROWID, and %msg for embedded SQL in methods

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This is a fix for a bug reported by a customer.

      When embedded SQL is defined in a class method that uses procedure blocks, we make the variables used by the embedded SQL statement public within the method and add code to NEW them at the top of the method.  This includes internal public variables SQLCODE, %ROWID, %ROWCOUNT, and %msg. 

      However, with the implementation of Universal Cached Queries, only SQLCODE was made public and NEW'd.  %ROWCOUNT, %ROWID, and %msg were missing from the NEW list. This oversight has been corrected.

      DP-411812: Correct owner for CREATE OR REPLACE VIEW

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      The DDL command CREATE OR REPLACE VIEW now sets the view's owner to the user that executes the statement.

      DP-411896: Fix query reconstruction and condition evaluation for queries with FOR SOME %ELEMENT

      Category: SQL
      Platforms: All
      Version: 2021.1.2

      Before this change, a query compilation issue could occur if a query included the WHERE FOR SOME %ELEMENT (%Key=? AND %VALUE =?) clause.

      DP-412117: Fix query reconstruction error for %parallel queries with FOR SOME %ELEMENT (...OR...)

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This changes corrects a query reconstruction problem for %parallel queries that included FOR SOME %ELEMENT ( ... OR ... ).

      DP-412145: Purge of UCQ queries was not taking out a lock

      Category: SQL
      Platforms: All
      Version: 2021.1.1

      There was a bug in the logic that scans for universal cached queries that no longer have a 'parent' routine and that can be thus purged. This bug resulted in purging the query without taking out a lock on the query (thus enabling another process from building a query with the same name at the same time, resulting in an inconsistent state of the universal cached queries). This change corrects the problem.

      DP-412627: Fix worker query reconstruction of LIKE ... ESCAPE ... conditions

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change fixes a problem in parallel queries with LIKE ... ESCAPE ... conditions. The problem was that the ESCAPE part was ignored in the reconstructed worker query.

      DP-412848: Check if subqueries have correlated fields when determine IsStreamless

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      There are several methods to determine if one mt("v") entry is streamless or not:

      isStreamless^%qaqpag

      isStreamless^%qaqpre

      isExpressionStreamless^%qaqpqo

      It is obvious that if this entry is an _x_ or _t_ nodes, it is streamless; else if this entry is an _f_ node, it is not streamless. But for _q_ node, which is a query block, before this dev change, sometimes we use $p(3) of this _q_ node, however, $p(3) may not be populated at that time, for example, in the bug Jira, we call isStreamless^%qaqpre with $p(3) of this _q_ node ="," although there are two correlated fields. So we need to look into mt("p") for this query block, if $p(4), which is a list (separated by ",") of correlated fields is empty, then we can say this is a streamless expression, otherwise, this is not a streamless expression.

      for example: 

      WHERE age=(select max(age) from Sample.Person)
       this is a streamless expression, but just like the query in the bug Jira DP-411666: 
      FAC.ID = ( SELECT TOP 1 FAC2.ID
FROM temp_Assets.Classification FAC2
WHERE
FAC2.Entity = FAC.Entity AND FAC.Type = FAC2.Type
AND FAC2.BusinessDateFrom<='2015-05-01'
ORDER BY FAC2.BusinessDateFrom DESC )
      this query block contains FAC.Entity and FAC.Type as the correlated fields, this is not a streamless expression

      DP-412869: Fix tune table to correctly mark outliers when outlier selectivity and sample size are both small

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      With this change, tune table now correctly marks outliers when outlier selectivity and sample size are both small.

      DP-412965: Support %SQL.Statement Prepare/ExecDirect without privilege checking

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      When preparing dynamic SQL statements via %SQL.Statement, there is now an option to not perform privilege checking.

      The %Prepare() method for %SQL.Statement now takes an optional second argument called checkPriv.  If checkPriv=0, SQL privileges are not checked on the statement.  The default value for checkPriv is 1.

      For example: 

      USER>set stmt=##class(%SQL.Statement).%New()
USER>set sc=stmt.%Prepare("DELETE FROM T",0) // No privileges checked
USER>set stmt2=##class(%SQL.Statement).%New()
USER>set sc=stmt2.%Prepare("DELETE FROM T") // Privilege is checked
      There is a new class method in %SQL.Statement called %ExecDirectNoPriv.  This is the exact same behavior and interface as %ExecDirect() except that no privileges are checked.

      Note, the %ExecDirectNoPriv() method may not be called directly in application mode. If it is called directly in application mode, it will immediately return "" and the statement will not be executed.  An example of attempting to call this directly in application mode would be something like: 

      iris terminal MyInstance "##class(%SQL.Statement).%ExecDirectNoPriv(,\"truncate table T\")"
      This change provides application code with an option to use its own security checking when using dynamic queries.  Suppose you do not want users to be able to execute adhoc queries against table T, but within your application, you do want users to be able to run method M where method M internally runs a SELECT query against table T.  You don't want to use embedded SQL within your method because your code needs the flexibility of Dynamic SQL.  You can now develop your code in method M that Prepares/Executes the query against table T without privilege checking.

      DP-413134: Prevent <PROTECT> errors from running SQL on some mirror members

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      There are various functions in %SYS.SQLSRV that file information into the SQL Statement Index. These functions do typically locking of the global nodes prior to filing.  Beforet is fix, it was possible for these functions were report <PROTECT> errors when running %SQL.Statement SQL statements on a mirrored database that was mounted as read-only and was a non-async member.

      These <PROTECT> errors occurred on the lock of the SQL statement entry, and then again on the unlock of the entry in the catch code.

      DP-413390: SQL PRIV: Correct SQL Admin CQ privilege checking

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      A problem has been corrected where it might be possible for a user to execute an existing cached query that requires the %NOCHECK, %NOLOCK, %NOINDEX, %NOTRIGGER, or %NOJOURN SQL Admin privilege without the user having the SQL Admin privilege.

      DP-413511: Correct string concatenation in subqueries

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue where string concatenation in a subquery was not working.

      DP-413782: Return results in the correct selectmode for UNION %PARALLEL queries

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change fixes a problem in which UNION %PARALLEL queries would only return results in logical mode.

      DP-413800: Correct code generation for FIRST_VALUE(col)

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      Before this change, the system generated incorrect code for the FIRST_VALUE function. Specifically, the code was inappropriate for string values.

      DP-413802: Update SQL window function SUM to throw -37 error when used over a stream field

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This fix updates the SQL window function SUM to behave consistently with ordinary aggregates in the way that it handles streams.

      DP-413975: Throw -37 error for SQL window function that uses an unsupported aggregation over a stream field

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change ensures that InterSystems IRIS throws an error when a query attempts to use an SQL window function with an unsupported aggregation over a stream field.

      DP-415661: Handle TO_NUMBER() correctly

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This query corrects an issue affecting queries that use TO_NUMBER(). Before this correction, such queries returned incorrect results, unless the query also included the %NOINDEX option.

      DP-416476: Correct query compilation error "too many arguments/subscripts"

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue in which a query against a very large table might fail to compile with an error #1031: Too many arguments/subscripts.  This issue can happen if more than 1024 $list slots are specified for a single storage node, and the query selects fields that are 1024 or more storage slots apart.

      DP-416479: Add XMLELEMENT, XMLATTRIBUTES and XMLFOREST support to the query regeneration logic

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change adds support for XMLELEMENT, XMLATTRIBUTES, and XMLFOREST functions to the query reconstruction logic used by regular %PARALLEL processing (not Adaptive Parallel Processing).

      DP-416681: Add privilege checking to %SYSTEM.java.SQL and %SYSTEM.python.SQL class methods

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      With this change, the following class methods require the %Admin_ExternalLanguageServerEdit resource in order to execute:

      • %SYSTEM.java.SQL.addPath()
      • %SYSTEM.java.SQL.closeGateway()
      • %SYSTEM.python.SQL.addPath()
      • %SYSTEM.python.SQL.closeGateway()

      This means %Admin_ExternalLanguageServerEdit is also required to call the following stored procedures:

      • %SYSTEM_java.add_Path(...)
      • %SYSTEM_java.close_Gateway()
      • %SYSTEM_python.add_Path(...)
      • %SYSTEM_python.close_Gateway()

      DP-416691: Change HOUR function back to taking first piece of $zt output

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      The HOUR function takes some time value and outputs only the hour portion of that time value. For logical %Date/$HOROLOG time values, we convert the time portion of the logical value to HH:MM:SS format with $ZTIME. A previous update that added proper validation of the time input (to allow error throwing in the case of invalid input) accidentally changed the output from %Date/$HOROLOG input values to take the second piece of the $ZTIME output, when the first piece should be taken.

      DP-416924: Correct query with HAVING or ORDER BY with implicit join

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects an error in which a query could return duplicate records if the query included a HAVING clause or included an ORDER BY clause that contained an implicit join.

      DP-417080: Correct type returned for TO_NUMBER() scalar function

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      A problem has been corrected where the type reported by a TO_NUMBER(...) function was not a numeric type.

      TO_NUMBER(_val_) will now return the following type: 

       
VARCHAR, VARBINARY, TIME --> NUMERIC
BIT -> TINYINT
DATE ->INTEGER
TIMESTAMP, POSIXTIME -> BIGINT
all others -> type of val

      DP-417280: Correct %ValidateIndices utility for tables with row-level security

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This correction fixes the %ValidateIndices utility method for classes that use row-level security.  The general flaw in the utility is that an index was traversed and when an index value was found that belonged to a row the user did not have privileges to read, the utility thought the row did not exist.  If the utility was called with pFix=1, the utility would then delete the row.

      %ValidateIndices has been updated to allow the utility to validate all rows in the table regardless if the caller has the privilege to read all rows from the table or not.  If the class/table uses row-level security, the display of any issues found will obscure the error messages so that no data in the table is displayed to the caller of %ValidateIndices.

      DP-417432: Correct <SUBSCRIPT> error for parallel query on table with %SPACE collation

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects a <SUBSCRIPT> error that occurred when running a parallel query on a table with %SPACE collation.

      DP-417498: Fix collation of scalar subqueries

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change fixes the collation of scalar subqueries during view recursion.

      Before this change, it was possible to have different query plans for equivalent queries that differ only in trivial ways.

      DP-417615: Correct query compilation error

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue that caused the following error when compiling a query: 

      <SUBSCRIPT>initemp+2^%qaqcasl *mt() Subscript 2 is ""

      DP-417933: Fix max length, precision, and scale metadata for window functions

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects the metadata that is generated for queries using window functions.

      DP-418002: Correct issue with NOT IN

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects a regression in query performance related to queries using the NOT IN clause. In some cases, a <DIVIDE> error was also possible.

      DP-418158: Disable auto-parallel if query contains constant and mode is %Runtime/%FDBMS

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change disables the auto-parallel mode for a query that contains a constant and that is run in either the %Runtime or %FDBMS mode. Such queries, when run in parallel, did not return expected results.

      DP-418248: Improve performance of queries with outer-join conditions involving multiple view tables

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change improves the performance of queries with outer-join conditions involving multiple view tables.

      DP-418271: DDL: Corrections for LOAD BULK DATA statement

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects a problem where LOAD BULK DATA was reporting 0 for SELECT COUNT ( * ) FROM T after loading data into table T.

      DP-418360: Make log file writeable in WriteToLogFile()

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change ensures that if you run tune table with a log file specified, the log file is writeable and is generated correctly.

      DP-418476: DDL: Create temporary tables in process-private globals

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue in which queries like 

      CREATE GLOBAL TEMPORARY TABLE table ... AS SELECT ..
      Incorrectly created regular tables instead of process-private globals.

      DP-418751: Handle system with 'GlobalKillEnabled=0' in resource cleanup

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      Before this change, the resource cleanup logic that we run periodically to remove globals/semaphores/events could get a <PROTECT> error if the system has 'GlobalKillEnabled=0' in the configuration. Now, the resource cleanup logic is able to successfully work in all cases.

      DP-418975: Fix errors running tune table while purging cached queries in the background

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue that occurred when running tune table while purging cached queries in the background. The issue caused errors like the following: 

      ERROR #5521: SQLError: SQLCODE=-400 %msg=Fatal error occurred::
There are 1 errors in routine %sqlcq.USER.1
...

      ERROR #5521: SQLError: SQLCODE=-400 %msg=Fatal error occurred::
Error executing code for 
Getting sample of rows from data blocks ...
      DP-419115: Correct error in NULLIF

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      After this change, NULLIF will always explicitly compare the first argument with the second argument.

      DP-419869: Avoid OREF cycles causing growth of memory footprint

      Category: SQL
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue in which a killed %ResultSet was still using memory and resources.

      SQL Dialects

      DP-406019: Fix TQSL error handling leading and trailing {...} braces

      Category: SQL Dialects
      Platforms: All
      Version: 2021.1.1

      When preparing the SQL text, this change removes any leading/trailing {...} braces and white space. The TSQL parser does not handle the braces.

      DP-406039: Implement bit operator &, |, and ^ for SQL context in TSQL

      Category: SQL Dialects
      Platforms: All
      Version: 2021.1.1

      A problem has been corrected where the TSQL bit operators &, |, and ^ were not working properly in a TSQL SQL statement.  They were working in procedures context, but not SQL context.

      DP-406274: Correct argument in procedure that causes Error #5001 Invalid Argument name in TSQL

      Category: SQL Dialects
      Platforms: All
      Version: 2021.1.1

      A problem has been corrected in InterSystems IRIS TSQL support when calling a procedure with positioned parameter and omitting one of the parameter values.  For example:

      exec my_proc 114209,'a',,'R1'

      SQL Search

      DP-405493: SQL Search now properly handles search strings with both escaped and non-escaped wildcards

      Category: SQL Search
      Platforms: All
      Version: 2021.1.3

      This change expands wildcard functionality so that users can now mix escaped and unescaped wildcards in the same search string without all wildcard characters defaulting to unescaped. Users can now also toggle ? wildcards to be escaped (and treated as a literal), whereas previously all ? characters were treated as wildcards. Additionally, the use of ? wildcards in their character functionality (as opposed to just word functionality) has been extended to positional search. Changes are made as necessary to positional, co-occurrence, and synonym search, as well as transformations, in order to properly support wildcards across search functionalities.

      DP-418450: Correct <UNDEFINED> on xDBC query prepare for table with functional index

      Category: SQL Search
      Platforms: All
      Version: 2021.1.3

      This change corrects an issue that caused an <UNDEFINED> error during the Prepare step for a SELECT statement of the form: 

      select * from khoj.Relationship where %ID %FIND search_index("relationiFind", 'SUC')
      The table has a functional index.  The <UNDEFINED> error occurred when attempting to determine if the query can use fastSelect or not.  This has been corrected.

      System

      DP-405455: Restore open flags for Windows async I/O

      Category: System
      Platforms: Windows
      Version: 2021.1.1

      This change reverses the negative performance impact of a previous change on async I/O on certain Windows systems. It appears that the effects of that change are noticeable on spinning drives although it may be related to certain versions of Windows. Aynch I/O is used to write incremental backup output files, databases and the wij, by mirroring and by the prefetch code.

      DP-405710: Don't consider the journal dmn waiting for a mirror buffers as hung

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change avoids a case where the backup mirror member could force down the primary if the primary is stuck waiting on the backup while sending journal data.

      DP-406111: Convert ^CacheMsg to ^IRIS.Msg during Caché to InterSystems IRIS conversion

      Category: System
      Platforms: All
      Version: 2021.1.1

      The correct namespace was not being set before the conversion of the global. The code now sets the implied namespace to the database being converted, and then set it back to %SYS after done.

      DP-406818: Correct reading of terminal script files with Japanese characters

      Category: System
      Platforms: All
      Version: 2021.1.1

      Terminal script files that contained Japanese characters, or any Unicode characters encoded with UTF8 or otherwise, were not read correctly. The encoded characters are now presented correctly, as bytes, as input to InterSystems IRIS where they can be processed by, for example, $ZCVT(input,"I","UTF8").

      DP-406855: Remove unnecessary test from new password hash algorithm

      Category: System
      Platforms: All
      Version: 2021.1.1

      An unnecessary test in the code which rehashes the password could cause LDAP users to be unable to login after an upgrade to 2021.1. This change removed the test.

      DP-407254: Update interaction between delegated and password authentication

      NOTE: This item may require a change to code, configuration, or operation.

      Category: System
      Platforms: All
      Version: 2021.1.1

      This fixes an issue with authentication ordering when delegated and password authentication are enabled. If your user authentication includes both delegated authentication and password authentication, you can choose either to call or not call ZAUTHENTICATE for password users. Typically, ZAUTHENTICATE is only used for delegated users and not for password users.

      But if you want your ZAUTHENTICATE routine to also be called for password users, you should check the <b>Always try Delegated Authentication</b> in the <b>System Administration > Security > System Security > Authentication/Web Sessions Options</b> page in the Management Portal or by using the ^SECURITY utility (System Parameter Setup, Edit authentication options).

      The default for <b>Always try Delegated Authentication</b> is No and ZAUTHENTICATE is not called for password users.

      DP-407609: Correct DataMove issues

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change fixes the following issues:

      1. Setting the MB per minute rate to a very low value (say 1mb) would allow the DataMove to run with no limit set. This was due to the value being rounded to 0, where 0=unlimited.

      2. Owner and group on the datamoveworkdb IRIS.DAT was not being set. This change sets to the same value of the MGR database. Previously this could cause a PROTECT error.

      3. If data being moved contained a "|" in the subscript, a <SYNTAX> error would occur during the activate phase. This was due to a line of code using $p(node,"|",3) rather than $p(node,"|",3,*) which truncated the subscript after the "|".

      4. If the system was forced down or crashed while a DataMove was running, the new DataMove mappings would be lost when the DataMove was later activated after the system restarted. This change preserves the mappings on a restart.

      DP-408045: Better error message for DataMove unexpected error

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change added more detail to an error which reports unexpected errors.

      DP-408050: Fix undefined error when specifying maxmbpermin in DataMove

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change fixes an undefined error when specifying the MaxMBPerMin property when calling the DataMove.API:Create() method.

      DP-408426: Eliminate an unnecessary error about journal switch

      Category: System
      Platforms: All
      Version: 2021.1.1

      It's unnecessary for ^JRNSWTCH or %SYS.Journal.System:RollToNextFile() to fail when journal switch is in progress.

      DP-408580: Allow DataMove to work with globals with characters greater than $c(122) and less than or equal to $c(255)

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change updated the $$globnext^NSPMAP function to support valid characters in global names up to $c(255). The DataMove.Data:Generate() method now allows for global name to start with characters up to $c(255)

      DP-408674: Fix DataMove range error on 8-bit system

      Category: System
      Platforms: All
      Version: 2021.1.1

      ENU8 and ENUW collations differ for the list of valid characters for global names. This change updated the code to use $zname(char,3) to get the list of valid characters in the loaded collation, and to use this list to generate the next valid global name.

      DP-408747: Fix HealthShare Security Domain Functionality

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change is to handle the required HealthShare Security Domain Functionality.

      DP-408751: Remove DataMove error tests which are not errors

      Category: System
      Platforms: All
      Version: 2021.1.1

      DataMove tries to set some O/S permissions on the work database. If it can't (possibly because it is running with lower permissions), it should log a message and continue running successfully. It wasn't clearing the error status after logging the message so the DataMove would stop. It now clears the error status and continue without stopping.

      DP-408869: Fix %Library.GlobalEdit:GetGlobalSizeBySubscript() and KillRange() for Begin ranges like X("A")*NEXT:X("C")

      Category: System
      Platforms: All
      Version: 2021.1.1

      Ranges similar to X("A")*NEXT:X("C") were sometimes including data under the ^X("A") node. What it should do is start looking at data after all the ^A("A") nodes, for example, ^A("B") and later. With this change it performs a $query() and gets the next subscript at this level (for example ^A("B"), and changes the start range to this. This updated range is then used to calculate the size. DataMove uses this method, and the end result was that it could see MBToCopy values higher than what was actually copied. This also would cause the estimated time to finish to be longer than what the copy actually took.

      For example: 

      %SYS>F i=1:1:1200 s ^A("A",i)=$j(i,30)
s x=##Class(%GlobalEdit).GetGlobalSizeBySubscript($zu(12),"A(""A"""_$c(1)_"*NEXT)","A(""C"")",.s)

      Before the change, the behavior would be: 

      %SYS>zw s
s=.06
s("Blocks",1)=1
s("Blocks",2)=6
s("Blocks","BigStrings")=0
s("Blocks","Total")=7
s("MB",1)="0.01"
s("MB",2)="0.05"
s("MB","BigStrings")="0.00"
s("MB","Total")=.06

      After the change, the behavior would be: 

      %SYS>zw s
s=0
s("Blocks",1)=0
s("Blocks",2)=0
s("Blocks",3)=0
s("Blocks","BigStrings")=0
s("Blocks","Total")=0
s("MB",1)="0.00"
s("MB",2)="0.00"
s("MB",3)="0.00"
s("MB","BigStrings")="0.00"
s("MB","Total")=0

      And with the following: 

      %SYS>F i=1:1:1200 s ^A("A",i)=$j(i,30)
s x=##Class(%GlobalEdit).KillRange($zu(12),"A(""A"""_$c(1)_"*NEXT)","A(""C"")")

      Before the change, all nodes under ^A("A") would be deleted. After the change, all the nodes would continue to exist.

      DP-408945: Increase timeout for starting DataMove processes

      Category: System
      Platforms: All
      Version: 2021.1.1

      On busy systems, the job command can actually fail within the timeout period. When the job command fails, the entire DataMove is halted.

      This change updated the code to change the timeout to 60 seconds, and to log retries if the job fails. (up to five retries before we fail)

      The change also updated the code to calculate the JRNMB field to properly calculate the amount of journal left to apply for each range when journal file compression is on.

      DP-408983: Fix DataMove undefined when changing rate

      Category: System
      Platforms: All
      Version: 2021.1.1

      DataMove could get an undefined error if the max rate is changed after the copy is done, but before changes are activated. this change eliminates the undefined error in this case.

      DP-409045: Update DataMove metrics more often

      Category: System
      Platforms: All
      Version: 2021.1.1

      DataMove metrics should be updated every 5 seconds or so. There was a problem where creating new DataMove processes pre-empted the update of the metrics fields causing them to skip updating the fields. Normally this is not a problem, but when there are a lot of moves (hundreds) where the amount of data being moved is low (a few mb) new processes were being created almost continuously, causing the updates to be skipped for potentially long periods of time. This change updated the algorithm so that the update of the metrics is done before determining to start another copy process.

      DP-409094: Update mappings for DataMove on Japanese systems

      Category: System
      Platforms: All
      Version: 2021.1.1

      Japanese locales have no identifier characters in the range $c(128)-$c(255). This change updated the global mapping for DataMove to provide an endpoint character of $c(255) when calling $$globnext^NSPMAP("%z"). This now returns "%z"_$c(255) as the ending range.

      DP-409618: Check DataMove stop flag more often

      Category: System
      Platforms: All
      Version: 2021.1.1

      To improve responsiveness this change checks the DataMove stop flag more frequently.

      DP-409768: Adjustments to LOG^%ETN(err)

      Category: System
      Platforms: All
      Version: 2021.1.3

      When LOG^%ETN(arg) is called when the 'arg' string contains (but does not start with) either <STORE> or <INTERRUPT> then problems can occur.   Such 'arg' values do not indicate a system <STORE> or <INTERRUPT> signal (probably indicate just a comment about such a signal.) But these error strings were being processed as if they were such a system signal.  

      With this change, the value of 'arg' will only be assumed to be a <STORE> signal when the error string begins with those characters.   Similarly, an 'arg' string containing <INTERRUPT> will not be assumed to an <INTERRUPT> signal.   A trap to ^%ETN, FORE^%ETN or BACK^ETN will be assumed to be an <INTERRUPT> signal only when the $ZERROR string starts with <INTERRUPT>.

      DP-409970: Fixed a bug in DataMove for Japanese systems

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change enables DataMove to work with Japanese systems and other systems that have data containing characters above $CHAR(128).

      DP-410409: Recalculate DataMove blocks copied on a crash

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change improves the reporting of the number of blocks copied if DataMove is interrupted.

      DP-410429: Corrects a DataMove issue in the Japanese locale

      Category: System
      Platforms: All
      Version: 2021.1.1

      This change corrects an error in DataMove that could occur when working with data in the Japanese locale

      DP-410605: Fix User security record being corrupted

      Category: System
      Platforms: All
      Version: 2021.1.1

      When a process calls $SYSTEM.Security.Login() to log in as another user, if an error occurs in the span of a couple of lines of the method, the original user record can be overwritten by the new user record. This would cause the original user to no longer be able to log in.

      This was seen when a stored procedure which as one of its tasks uses $SYSTEM.Security.Login() to change itself to the _SYSTEM user to do some "system stuff". It just so happened that the connection had a timeout set, and an <ALARM> error was generated at just the wrong spot in the code when the user credentials were getting changed. Normally an error could never happen in this section of the code during normal operation, but when an alarm is set by the server code, an asynchronous error could occur causing the failure.

      This change prevents the old user record from being overwritten by the new user record in the case of an error.

      Note that the window here is very small and unlikely to occur.

      DP-410986: Fix memory leak caused by LDAP searching with pages

      Category: System
      Platforms: All
      Version: 2021.1.1

      Calling ##Class(%SYS.LDAP).GetNextPages() could cause a memory leak when searching. This problem could be seen using the LDAP interoperability Business Operations. This change corrects the problem.

      DP-411930: DataMove creation step contained in a transaction

      Category: System
      Platforms: All
      Version: 2021.1.3

      The DataMove creation step, which involves setting multiple globals, is now contained in a transaction. If any error occurs during the creation of the DataMove, the partial creation is rolled back. The user can then correct the parameters passed to the DataMove.API:Generate() function, and then retry the creation. Previously the user could get a error when trying to access the partially created DataMove.

      DP-414797: Allow [Gateways] section to be removed from CPF file

      Category: System
      Platforms: All
      Version: 2021.1.3

      This change makes [Gateways] an optional section of the CPF file, simplifying the upgrade code.

      DP-418812: Mount databases for directio on Unix when wdwrite_asyncio_max is non-zero

      Category: System
      Platforms: UNIX®
      Version: 2021.1.3

      This change fixes a bug where a database which had been dismounted and remounted would not have direct I/O enabled on a system where the write daemon is using async I/O (the default configuration).

      DP-419512: Restore large page support on newer Windows versions

      Category: System
      Platforms: Windows
      Version: 2021.1.3

      This change restores large page support on newer Windows versions. Specifically, it corrects how InterSystems IRIS calls the Windows MapViewOfFile() function. (This function was changed starting with Windows 10 version 1703 and later Windows Server versions.)

      Tasks

      DP-417244: Add check for EndDate="" to ImportTasks method.

      Category: Tasks
      Platforms: All
      Version: 2021.1.3

      This change modifies ImportTasks() to prevent importing a Task that has EndDate="" (i.e., no EndDate).

      Terminal

      DP-407082: Increase terminal window size to make all output visible

      Category: Terminal
      Platforms: All
      Version: 2021.1.1

      Output to the top line or right-most column of the terminal window could be overlaid by the menu bar or scroll bar. This change makes the window slightly larger so all the output is visible.

      Tools

      DP-403494: Correct JavaExe() result when JAVA_HOME contains spaces

      Category: Tools
      Platforms: Windows
      Version: 2021.1.1

      On Windows, if the JAVA_HOME environment variable is a path name with spaces, $System.Util.JavaExe() could return an invalid path. This change corrects the problem.

      DP-405779: SystemPerformance - new section "irisstat -R" containing a single snapshot of routine buffers

      Category: Tools
      Platforms: All
      Version: 2021.1.1

      SystemPerformance reports on all platforms will now include a new section named "irisstat -R".  This section contains a single sample of the irisstat command using the -R1 flag, which dumps routine buffers.

      DP-407065: GetInternalStackLevel of %SYS.ProcessQuery returns NEW levels

      Category: Tools
      Platforms: All
      Version: 2021.1.1

      A second by-reference argument is added to the GetInternalStackLevel method of %SYS.ProcessQuery: GetInternalStackLevel(stack, newlevels)

      When there are NEW commands in a routine, each NEW command creates a new internal stack level without increasing $STACK. GetInternalStackLevel returns the highest internal stack level for the given $STACK. If there are other internal levels created by NEW commands within the given $STACK, they are returned in the newlevels argument as a $LIST of values. There may be different variables defined at each of those levels and the same variable may have different values at each level.

      DP-407279: Handle recursion of $Compile() that could cause a segfault

      Category: Tools
      Platforms: All
      Version: 2021.1.1

      There could be an address exception if a Source Control class has a method that causes recursion into the class loader/compiler. This change corrects the problem.

      DP-413093: Make DataMove more mirror aware

      Category: Tools
      Platforms: All
      Version: 2021.1.3

      The following updates have been made to DataMove:

      Restrictions: 1) Journaling must be turned on for the system. 2) The freeze on journal error switch must be set 3) When creating a DataMove, data can only be moved from a local database to another database, or from a mirrored database to a mirrored database. You cannot move data data from a local database to a mirrored database, or from a mirrored database to a local database. 4) You cannot have a DataMove which mixes local and mirrored databases. 5) In a DataMove which contains mirrored databases, you can only run the DataMove on the Primary node. You cannot run it on the failover, DR, async or reporting nodes.

      New functionality:

      1) A DataMove can be run on a failover, DR, async or reporting node if the data to be moved is not in a mirrored database. 2) If a DataMove is running on the primary, and the failover or DR node becomes the primary, the datamove is stopped. If the node which the DataMove was running on becomes the primary again, the DataMove will resume where it left off.

      Web Gateway

      DP-402068: Revamp management of thread life cycles

      Category: Web Gateway
      Platforms: All
      Version: 2021.1.1

      Fix random crashes that would sometimes occur when connections to InterSystems IRIS were closed down or when web server worker processes were closed down.

      DP-403826: Fix memory leak when Web Gateway fails to log in to CSP server

      Category: Web Gateway
      Platforms: All
      Version: 2021.1.1

      This change fixes a memory leak that occurs when the Web Gateway fails to log in to the CSP server, causing high memory usage if many login failures occur.

      DP-404812: Do not delete httpd.pid if ##class(Config.Startup).StopWebServer() can't stop the httpd process

      Category: Web Gateway
      Platforms: All
      Version: 2021.1.1

      Fixes the handling of a case where Config.Startup:StopWebServer() would delete the httpd.pid file but fail to stop the Private Web Server process. This could occur if, for example, the instance user has the %All role, but the OS-level user who logged in does not have permission to stop the process. The process would then continue running until killed manually at the OS level, since the instance needs the httpd.pid file to find and kill the process during shutdown. Now, if the process fails to terminate, the httpd.pid file is left in place so that InterSystems IRIS can still find and clean up the old process later.

      DP-405020: Prevent configuring invalid CSP server names and application paths

      Category: Web Gateway
      Platforms: All
      Version: 2021.1.1

      In the Server Access and Application Access configuration pages, it was possible to add a CSP server or application name containing special words or characters that would corrupt the Web Gateway configuration. For example, the name "system" is reserved by the Web Gateway, and creating a CSP server named "system" led to corruption of the configuration. The Web Gateway now validates CSP server and application names and rejects those containing reserved words or characters.

      DP-410864: Prevent client error in delivering CSP/WebGateway cached content

      Category: Web Gateway
      Platforms: All
      Version: 2021.1.3

      Build nnn.1823

      The Web Gateway caches the first HTTP response returned for a given page or resource. If that response is gzip compressed and a separate client requests the same resource, the Web Gateway would return the compressed response even if the client doesn't accept gzip encoding.

      With this change, the Web Gateway now checks whether the client can accept the cached encoding, and if it can't, the Web Gateway retrieves a decompressed copy from the server. The gzipped format stays in the cache, and Web Gateway skips caching the decompressed copy. This conserves cache space and the performance gains from having gzip enabled. (If a gzipped response was in the cache, a majority of clients likely support that format.)

      It's possible for a non-gzipped response to end up in the cache first, and in that case the Web Gateway will still return that non-compressed response even for clients that accept gzip encoding. However, that at least doesn't violate the HTTP protocol; the client should always be able to accept non-encoded content.

      Web Services

      DP-410616: Correct XML namespace handling when ELEMENTQUALIFIED property parameter is 0

      Category: Web Services
      Platforms: All
      Version: 2021.1.3

      This change corrects a problem with the XML namespace handling when an XML enabled class with {{ELEMENTQUALIFIED=1}} contains a property with the property parameter {{ELEMENTQUALIFIED=0}}. The problem would result in the property being qualified with the paren't class's namespace instead of no namespace. For example: 

      Class dp.Message Extends (%RegisteredObject, %XML.Adaptor) [ ProcedureBlock ]{
Parameter ELEMENTQUALIFIED = 1;
Parameter NAMESPACE = "urn:sdk-test/n1";
Property MessageBody As dp.n2.MessageBody(ELEMENTQUALIFIED = 0, XMLNAME = "MessageBody") [ Required ];
} 
      Class dp.n2.MessageBody Extends (%RegisteredObject, %XML.Adaptor) [ ProcedureBlock ]{
Parameter ELEMENTQUALIFIED = 0;
Parameter NAMESPACE = "urn:sdk-test/n2";
Property Time As %String(MAXLEN = "", XMLNAME = "Time") [ Required, SqlFieldName = _Time ];
Property MessageText As %String(MAXLEN = "", XMLNAME = "MessageText") [ Required ];
}
      Would produce XML of the form:  {code:xml} <?xml version="1.0" encoding="UTF-8"?> <Message xmlns="urn:sdk-test/n1"> <MemberID xmlns="">ME</MemberID> <MessageBody xmlns:s01="urn:sdk-test/n2"> <Time>NOW</Time> <MessageText>Hello</MessageText> </MessageBody> </Message> instead of: {code:xml} <?xml version="1.0" encoding="UTF-8"?> <Message xmlns="urn:sdk-test/n1"> <MemberID xmlns="">ME</MemberID> <MessageBody xmlns="" xmlns:s01="urn:sdk-test/n2"> <Time>NOW</Time> <MessageText>Hello</MessageText> </MessageBody> </Message>

      Note that {{xmlns=""}} is missing from the original {{<MessageBody>}} element. This problem only affected complex types; scalar (%DataType) values were projected correctly.

      Work Queue Manager

      DP-409294: Do not add messages.log entry for expected external interrupt error

      Category: Work Queue Manager
      Platforms: All
      Version: 2021.1.3

      With this change, the Work Queue Manager no longer adds entries to messages.log for expected interrupt messages.

      DP-409791: In WQM detect workers that do not run any commands as inactive

      Category: Work Queue Manager
      Platforms: All
      Version: 2021.1.1

      The work queue manager keeps track of if worker jobs are active or not so it can detect deadlock situations and automatically start additional workers. This logic did not work in the case of a worker jobs that has done callout and this callout logic was sitting idle. This change corrects the problem.

      DP-411313: Fix licensing issue with Worker jobs

      Category: Work Queue Manager
      Platforms: All
      Version: 2021.1.3

      This change makes sure that Work Queue Manager Worker jobs do not take a license in %ZSTART when deferred licensing is used.

      DP-412650: Correct WQM worker job termination

      Category: Work Queue Manager
      Platforms: All
      Version: 2021.1.3

      When using the Work Queue Manager and the Shard Queue Manager if the group being used is no longer needed, the system signals all workers in the group to stop (to avoid using CPU and other resources).

      There was a bug in the cancellation logic that would result in work being processed instead of being canceled, resulting also in greater use of CPU and other resources. Potentially extra worker jobs would be created, which could exhaust the operating system process table.

      The change corrects the error.

      DP-414171: Avoid <MAXSTRING> when truncating output

      Category: Work Queue Manager
      Platforms: All
      Version: 2021.1.3

      Before this change, if a Work Queue Manager task produced very long output of around 3.5Mb and also returned a long %Status code return value, it was possible to get a <MAXSTRING> error when serializing this data.

      With this change, we explicitly take into account the length of the %Status code and leave sufficient space for this to avoid getting <MAXSTRING> errors.

      DP-414996: Truncate output if there are 16-bit wide chars

      Category: Work Queue Manager
      Platforms: All
      Version: 2021.1.3

      This change truncate Work Queue Manager output if there are 16-bit wide chars. This change avoids <MAXSTRING> error on big output.

      DP-415039: Do not report DISCONNECT errors in WQM as alerts

      Category: Work Queue Manager
      Platforms: All
      Version: 2021.1.3

      Before this fix, if the work queue manager got an async DISCONNECT error during some parts of processing, it would log these as errors to messages.log file when they are to be ignored. Now we detect and ignore these errors.

      DP-415819: Fix <UNDEFINED> error in Work Queue Manager code

      Category: Work Queue Manager
      Platforms: All
      Version: 2021.1.3

      This change fixes an undefined error in WaitForComplete^%SYS.WorkQueueMgr. Before this fix, if a work queue was deleted, checking ##class(%SYSTEM.Event).Count() threw an <UNDEFINED> error.

      For Additional Help

      If you need assistance with evaluating how upgrading to this maintenance release will affect your applications, systems, or related plans, please contact the InterSystems Worldwide Support Center:

      • Phone:  +1.617.621.0700
      • Fax:  +1.617.734.9391
      • Email:  support@intersystems.com

      Current release notes (and complete product documentation) can be found online at https://docs.intersystems.com.

      FeedbackOpens in a new tab