InterSystems IRIS® Upgrade Checklist and Change Notes (2023.1.1)

DP-419513: Mark procedure block methods as 'private'

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Compiler
Platforms: All
Version: 2023.1.0

This change improves the generation of ObjectScript code to ensure that code is always run in the correct class context. For more details, see the "Improvements to how IRIS classes are generated and called" post in the Developer Community.

DP-420288: fix unix irispython to work inside a venv

NOTE: This item may require a change to code, configuration, or operation.

Category: Embedded Python
Platforms: UNIX®
Version: 2023.1.0

This change fixes the Unix irispython command to work inside a virtual environment and requires a new set of symbolic links.

The virtual environment now needs the following symbolic links: python3 -> [InterSystems IRIS install directory]/bin/irispythonvenv python3venv -> /usr/bin/python3 [or the path that contains your machine's installation of Python]

DP-422560: HTML encode REST error messages

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2023.1.1

Previously, error messages returned from REST APIs were not HTML encoded. This issue has been corrected.

DP-247608: Provide access to captions and descriptions of members using inherited cube methods

NOTE: This item may require a change to code, configuration, or operation.

Category: Business Intelligence
Platforms: All
Version: 2023.1.0

If a level within a cube utilizes predefined members by using the <member> tags, the displayName and description attributes were not available through the utilities

 Cube.Class:%GetMemberCaption
 Cube.Class:%GetMemberDescription

As an example, adding a displayName and caption to the Comments dimension in the HoleFoods sample:

<dimension name="Comments" type="computed" dimensionClass="SQL" hasAll="false">
 <hierarchy name="H1">
  <level name="Comments" factName="MxComment">
   <member name="Complaints" 
     displayName="Complaints Member Caption"  
     description="Complaints Member Description"
     spec="WHERE $$$FACT %CONTAINSTERM('complained') OR $$$FACT %CONTAINSTERM('ranted')"/>
   <member name="Compliments" 
     spec="WHERE $$$FACT %CONTAINSTERM('happy')"/>
  </level>
 </hierarchy>
</dimension>

SAMPLES>Set cubeClass=##class(%DeepSee.Utils).%GetCubeClass("HOLEFOODS") 
SAMPLES>w $zobjclassmethod(cubeClass, "%GetMemberCaption", "Comments", "H1", "Comments", "Complaints")
 Complaints Member Caption
SAMPLES>w $zobjclassmethod(cubeClass, "%GetMemberDescription", "Comments", "H1", "Comments", "Complaints")
 Complaints Member Description

DP-405720: Return Intelligent Auth Challenges with "WWW-Authenticate" Header

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2023.1.0

With this change, by default InterSystems IRIS REST applications set the "WWW-Authenticate" header to "Basic" on a 401 response.

DP-406452: For READ UNCOMMITTED, build and run separate queries

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

Improve the generated code for embedded SQL and %SQL.Statement dynamic SQL by building separate queries based on the partition's isolation mode. If the partition is in READ UNCOMMITTED, we can build parallel queries which can be a major performance improvement. We also do not need to generate runtime logic checking the isolation mode and then re-reading data for READ COMMITTED in the default READ UNCOMMITTED queries.

As part of this change, we added logic so we can find frozen plans for these queries if they were frozen before the partition isolation mode was added to the query.

The isolation mode is added with a SQL comment option that looks like this:

/*#OPTIONS { "IsolationLevel":0 } */

Two compatibility issues should be noted:

1) This change may cause queries to be run in parallel automatically based on the auto-parallel heuristics. If a query does not specify a strict ordering of results (no ORDER BY, for example) then SQL is allowed to return results in any order it wishes and in this case running a query in parallel will cause the order of results to be different on each run of the query. If the user needs the results in a specific order they should include an ORDER BY clause in the query or add the %NOPARALLEL keyword to the query.

2) For %SQL.Statement with this change, the process isolation level will be determined when we prepare the query and not when we execute. This is different from the previous behavior where the process isolation level was taken from the query execution time. Customers do not normally change process isolation level with queries already prepared so it is unlikely this will cause problems, but it is a change in behavior.

DP-407729: FIX: NOT %INLIST and NOT IN disparity when there is a NULL in the list

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

This change makes the behavior of NOT %INLIST $listbuild(x,NULL,:z) consistent with the behavior of NOT IN (x, NULL, :z).

DP-409303: Convert TIME_STRUCT to POSIX

NOTE: This item may require a change to code, configuration, or operation.

Category: ODBC
Platforms: All
Version: 2023.1.0

On insert TIME_STRUCT into POSIX field (1093), the ODBC driver will convert TIME_STRUCT to POSIX with default date 1-1-1900.

DP-409401: Increase maximum local array subscript length from 511 to 32767

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2023.1.0

The maximum length of a local variable subscript is increased from 511 encoded bytes to 32767 (the limits are described in detail in the documentation). Attempting to set such a subscript on a prior version would fail with a error.

The maximum length of a global subscript is unchanged at 511 encoded bytes, so when using arbitrarily long subscripts on local variables, it is easier than it was in prior versions to generate a local variable that cannot be merged into a global -- the MERGE command would stop with a error upon encountering such a subscript. Note: this condition was possible in prior versions because differences in local and global encoding rules can result in a differing number of encoded bytes when an identical string is used as either a local or global.

DP-409705: Refresh .NET versions

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways - .NET
Platforms: All
Version: 2023.1.0

This change refreshes the .NET Versions to align with the versions that are still in support by Microsoft. We have removed versions that are no longer in support by Microsoft (.NET Framework 2.0, 4.0 and 4.5, and .NET Core 1.0 and 2.1).

We have also added support for two versions: .NET Framework 3.5 and 4.6.2. 

This change also updates the assembly versions so they match the new versions we support.

The new assemblies will be under a different path, so projects that use the path to the dll location under the InterSystems IRIS install location will need to update the path to correspond to the new versions. For example, a previous path would be:

<InterSystems IRIS install location>\dev\dotnet\bin\v4.5\InterSystems.Data.IRISClient.dll

That location will no longer exist under the new installation, and should be changed to:

<InterSystems IRIS install location>\dev\dotnet\bin\v4.6.2\InterSystems.Data.IRISClient.dll

In terms of compatibility between versions, the new 4.6.2 version is backwards compatible and the applications will run on systems that have any .NET Framework 4.x installed.

However, .NET Framework is not forwards-compatible, so if customers have applications that target .NET Framework 4.5, they cannot use one of our .NET Framework 4.6.2 libraries as a dependency. Their options are:

• Change the target framework of their application to be at least 4.6.2. .NET Framework 4.5 has been out of support by Microsoft so this will also ensure users are using a supported language version.
• Use the .NET Framework 3.5 version of our library. Users might lose access to certain features/functionality that was introduced in version 4.0.
• Use old versions of our library that target 4.5. These will not contain the latest bug fixes or functionality, but users will not need to modify the dependencies of their applications.

DP-410079: Make Maximum_Server_Connections a flat limit

NOTE: This item may require a change to code, configuration, or operation.

Category: Web Gateway
Platforms: All
Version: 2023.1.0

This change makes Maximum Server Connections a flat limit instead of a per-process limit. The strategy is to implement a throttle in cspTCPIPOpenSession, tallying connections on the same server in addition to the session-ID-based tally. This change also makes it so that the Maximum Server Connections configuration field is displayed and used even in the non-threaded/multi-process case. Additionally, processes with status Server and Private now count toward the total server connection tally.

Customers who have multi-process web server architectures (e.g., Apache Prefork MPM) may need to adjust their Maximum Server Connection settings to accommodate more connections. The Web Gateway does not take responsibility for the underlying web server configuration. To support the same number of InterSystems IRIS server connections as before, simply multiply the original maximum connection limit by the web server's maximum number of processes. (In Web Gateway Management, see Server Configuration -> Maximum Server Connections.)

DP-410183: Fix XML generation when a class extends another class in a different XML namespace

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Library
Platforms: All
Version: 2023.1.0

This change corrects a problem with XML generation when a class extends a class in a different XML namespace. Specifically, if a class A in namespace N1 with ELEMENTQUALIFIED=0 extends a class B in namespace N2 also with ELEMENTQUALIFIED=0, and class B contains a property of type C in namespace N2, when you generate XML for an instance of class A, previously the element for Class C would be qualified with namespace N2, when it should in fact be unqualified.

For example:

Class dp.ActualType Extends dp.n1.BaseType
{
Parameter ELEMENTQUALIFIED = 0;
Parameter NAMESPACE = "urn:sdk-test/n2";
Parameter XMLNAME = "ActualType";
Property ActualProp As %String(MAXLEN = "", XMLNAME = "ActualProp") [ Required ];
}

Class dp.n1.BaseType Extends (%RegisteredObject, %XML.Adaptor)
{
Parameter ELEMENTQUALIFIED = 0;
Parameter NAMESPACE = "urn:sdk-test/n1";
Parameter XMLNAME = "BaseType";
Parameter XMLSEQUENCE = 1;
Property BaseProp As dp.n1.BasePropType(XMLNAME = "BaseProp") [ Required ];
}

Class dp.n1.BasePropType Extends (%RegisteredObject, %XML.Adaptor)
{
Parameter ELEMENTQUALIFIED = 0;
Parameter NAMESPACE = "urn:sdk-test/n1";
Parameter XMLNAME = "BasePropType";
Parameter XMLSEQUENCE = 1;
Property Tag As %String(MAXLEN = "", XMLNAME = "Tag") [ Required ];
}

<?xml version="1.0" encoding="UTF-8"?>
<s01:ActualType xmlns:s01="urn:sdk-test/n2">
    <s02:BaseProp xmlns:s02="urn:sdk-test/n1">
        <Tag></Tag>
    </s02:BaseProp>
    <ActualProp>myvalue</ActualProp>
</s01:ActualType>

<?xml version="1.0" encoding="UTF-8"?>
<s01:ActualType xmlns:s01="urn:sdk-test/n2">
    <BaseProp xmlns:s02="urn:sdk-test/n1">
        <Tag></Tag>
    </BaseProp>
    <ActualProp>myvalue</ActualProp>
</s01:ActualType>

DP-410199: ##class(%DynamicAbstractObject).%FromJSON("") returns "" instead of an error

NOTE: This item may require a change to code, configuration, or operation.

Category: JSON
Platforms: All
Version: 2023.1.0

A call on ##class(%DynamicAbstractObject).%FromJSON("") will return the empty string, "", instead of throwing an exception for "Premature end of data".   However, %FromJSON(arg) will still throw that error if (1) arg is a %Stream containing no additional characters; if (2) arg is a file name referencing a %File that contains no characters; or if (3) arg is an ObjectScript string containing only white space characters or is otherwise an incomplete JSON string.

Note:  a call on DynamicObject.%ToJSON() that does not signal an error will always return an ObjectScript string containing at least two characters, e.g., [] or {}.

Any code expecting %FromJSON("") to throw an error will need to be modified to special check for "" being passed as an argument to %FromJSON(arg).

DP-410225: Add more fields for audit log entry in structured logging

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2023.1.0

Before this change, when the LOGDMN process scanned the audit log, it persisted only the following audit event fields to the log file: EventSource,EventType,GroupName,Namespace

With this change, the LOGDMN process 1) includes all the fields as seen in the details of ^SECURITY for an audit log record and 2) improves audit log scanning process to one pass for better performance.

Log entry parsing, parameter matching, and value extraction might break if your code relies on the previous behavior.

DP-410526: Optimize REST dispatch handling & modify DispatchMap()

NOTE: This item may require a change to code, configuration, or operation.

Category: REST
Platforms: All
Version: 2023.1.0

The code for dispatching requests in %CSP.REST was quite inefficient, particularly for Dispatch classes with a large number of routes. This change refactors the dispatch code to be more efficient and changes the signature of DispatchMap() method.

Note that dispatch classes with a large number of routes will always be relatively inefficient because we need to evaluate the Regex for each route in order to determine proper functioning. In general, routes early in the UrlMap will be executed more efficiently than routes later in the map.

DP-411232: New %DROP_UNOWNED SQL admin privilege

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

In previous releases, if a user held the %DROP_TABLE or %DROP_VIEW administrative privilege, the user would be able to drop a table or view that the user did not own. In this release, in order to drop a table or view, the user must either be the owner of the table or view or must hold the %DROP_UNOWNED administrative privilege (which is new in this release).

DP-411644: Improve SplitSize^%GSIZE() to more evenly split on SLM mapped global

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2023.1.0

Improve SplitSize^%GSIZE() to more evenly split on subscript-level mapped global.

For a global mapped to remote database through ECP, the data server needs to have this new feature in order to process the request from app server with this new feature.

DP-412103: SQL STRING(...) function now correctly handles nulls and empty strings

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

This change modifies the SQL STRING(...) function code generator to work correctly for any number of parameter values, including NULL and the empty string (which is represented internally as $char(0)), and to generate more efficient code.

Prior to this change, the SQL STRING(...) function would return a NULL value ("") if ANY of the parameters are NULL, which was incorrect. In this scenario, the NULL value is meant to be treated as an empty-string ('') if other parameters are present. 

Now the logic of SQL STRING(...) is in line with the Sybase SQL Anywhere definition, and will return the concatenation of all parameters, treating NULL parameters as the empty-string ('') as in the following example:

SELECT STRING('This', ' ', NULL, 'now', ' ', 'works!') INTO :testdata
SELECT STRING('This', ' ', '', 'now', ' ', 'works!') INTO :testdata

With this change, each of these SQL statements set the testdata variable equal to "This now works!"

DP-412300: Add %NOJOURN SQL Admin priv & disable transactions with %NOJOURN SQL keyword

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

With this change, users must now hold the %NOJOURN SQL Admin privilege in order to use the %NOJOURN keyword in INSERT [ OR UPDATE ], UPDATE, DELETE, BUILD INDEX, or fast insert statements. Also, the %NOJOURN flag for INSERT [ OR UPDATE ], UPDATE, DELETE, and fast insert statements now explicitly turns off transactions. Transactions are implicitly skipped when journaling is disabled, but this change optimizes mirrored environments because mirrors must keep journalling enabled so that data can be propagated to backups. Also, this change fixes a bug where %NOLOCK SQL Admin priv was not checked when %NOLOCK is used in BUILD INDEX statements.

DP-412389: New ^mtemp format for ^%STACK; new way to extract local variable values

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Library
Platforms: All
Version: 2023.1.0

Some of the internal data formats used in a ^mtemp(%msub) stack dump and used in a ^ERRORS(date,index) stack dump have changed. See the class reference documentation of ExamStackByPid() method of the %SYS.ProcessQuery class.

Also, there is some change to output (especially output of objects) in the ^%ERN and ^%STACK displays of stack dumps. The same data is still there but the data formatting is different in some cases. This debugging display is generated interactively to the programmer who should not have problems with the modified formats.

New routines $$VGetn^%STACK(index,level,variable) and $$VGetn^%ERN(date,index,level,variable) provide a supported way to extract local variable values out of stack dumps in the ^mtemp and ^ERRORS globals. These new routines work with both the old and new format stack dumps.

DP-412418: In %RoutineMgr:StudioOpenDialog query for /mapped=0 use the data database for globals not the routine database

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Library
Platforms: All
Version: 2023.1.0

The %RoutineMgr:StudioOpenDialog query, which is used by a large number of interfaces, turns patterns into lists of items in the InterSystems IRIS system. One of the pattern extensions it supports is '.gbl' to reference globals. When you called this with for example '\*.gbl' and the qualifier /mapped=0 (which is normally the default) it would search for globals in the routine database but it should have been searching the default data database as well for globals as routines/classes are found from patterns like '\*.mac' or '\*.cls'.

With this change, the query will result in *.gbl finding globals in the data database for /mapped=0 where as before it would only find ones in the routine database. It is unlikely customers are relying on this behavior.

DP-412625: Interoperability XML virtual document schema correction when collapsing single child generation

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2023.1.0

When a schema has a generation of single properties, the collapse mechanism could lead to the final child node not being represented properly in the schema. This is now corrected.

DP-412629: Improve XML Virtual Document support for 'any' element

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2023.1.0

Before this change, when setting XML data to an 'any' element of an XML Virtual Document, the 'any' tag would be included in the output.

Also when using a sub-transform, the XML representation used to set to a target property of element type 'any' would not include the top level. This is also corrected. The correction is to the EnsLib.EDI.XML.Document SetSubDocument() method.  This method inserts the XML representation of pSubDocument into the target document at the path specified by pPropertyPath. The XML representation is the GetValueAt using "/1" as the property path except if the target Property Path where the sub document is to be set is of type "any" then the XML representation of pSubDocument retrieved  will be the full document including top element i.e. path "/"

DP-412853: Add %FromJSONFile(); do not load files in %FromJSON()

NOTE: This item may require a change to code, configuration, or operation.

Category: Object Library
Platforms: All
Version: 2023.1.0

Previously ##class(%DynamicAbstractObject).%FromJSON() would accept three types of inputs:

• a stream object
• a JSON formatted string
• a filename on the server containing JSON

This has been fixed by creating a new method that expects an input string containing a filename:

ClassMethod %FromJSONFile(str) As %DynamicAbstractObject

The %FromJSON() method no longer loads a file. If the input is a stream, the method works as before.

DP-413011: Prevent some datatype combinations in CASE value expressions

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

With this change, certain combinations of data types in CASE statements are no longer permitted and will throw errors instead of yielding potentially unexpected behavior. CASE statements may still combine ODBC numeric types (BIGINT, INTEGER, DOUBLE, NUMERIC, SMALLINT, TINYINT, BIT). Statements may also combine only GUID and VARBINARY with each other. An example of an invalid combination would be TIME with a numeric value, as in:

CASEWHEN case1 THEN MyTimeFieldWHEN case2 THEN MyIntegerFieldEND

CASEWHEN case1 THEN CAST(MyTimeField AS INT)WHEN case2 THEN MyIntegerFieldEND

DP-413023: Require double quotes around delimited identifiers

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

This change prevents the parser from accepting a non-quoted delimited identifier of the form <number><simpleidentifier> as <number>  <simpleidentifier>. Before this change, the parser would accept this as a constant field with identifier <simpleidentifier> and value <number>. With this change, the parser will now throw an error.

DP-413048: Remove Linux capability checking from InterSystems IRIS containers

NOTE: This item may require a change to code, configuration, or operation.

Category: Cloud
Platforms: Linux
Version: 2023.1.0

This change removes the no-longer-necessary Linux capability checking from InterSystems IRIS containers, to avoid incompatibilities with new versions of Docker.

This doesn't immediately affect ICM users because an older Docker version is set in Samples/defaults.json.

DP-413293: Zen Reports: do not allow use of $DATASOURCE URL parameter by default

NOTE: This item may require a change to code, configuration, or operation.

Category: Compatibility Features
Platforms: All
Version: 2023.1.0

Previously ZEN Reports would allow the report's datasource to be specified at runtime via the $DATASOURCE URL Parameter. If this is not defined, then we will use the DATASOURCE class parameter, and if that isn't defined we will use the ReportDefinition to generate the XML data. The ReportDefinition is the standard way to specify report data.

With this change, Zen Reports will no longer use the $DATASOURCE URL parameter by default. Developers are strongly encouraged to use the DATASOURCE class parameter to specify an external datasource if desired. If a report does require the $DATASOURCE URL, you can re-enable the previous behavior on a per-report or per-application basis by setting the parameter:

Parameter USEURLDATASOURCE = 1; 

in either the report class, or in the report's Application class. Setting this value is discouraged.

DP-413402: SYSTEM: correct some Security.Datatype.* ClientDataType and OdbcType settings

NOTE: This item may require a change to code, configuration, or operation.

Category: ODBC
Platforms: All
Version: 2023.1.0

The OdbcType for the following system datatype classes has been updated to be VARCHAR, where before it was an Integer: * Security.Datatype.Authentication * Security.Datatype.BooleanYN * Security.Datatype.LDAPFlags * Security.Datatype.Permission * Security.Datatype.PrivateKeyType * Security.Datatype.Protocol * Security.Datatype.ResourceType * Security.Datatype.SSLType * Security.Datatype.ServiceCapabilities * Security.Datatype.TLSVersion

DP-413544: Change ZBREAK so %Destruct stepping is ON by default

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2023.1.0

When debug stepping is enabled and an object reference is closed, the debugger will now step into the %Destruct and %OnClose methods. Previously this stepping was disabled by default. This stepping may be turned off with the command ZBREAK /NOSTEP:DESTRUCT.

DP-413564: Correct Interoperability Reply Code Action D triggering Alert On Error if retried

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2023.1.0

The Interoperability Reply Code Action D would only trigger an Alert On Error if there had not been a prior action [R(etry)]. This is now corrected. If for example the Reply Code Action is E=RD and Alert On Error is enabled, then an alert will be sent for the Retry and also the Disabling if the failure timeout is reached.

DP-413732: Modify %UnitTest.Common.INC to replace PTools-specific macros with symmetric general-purpose macros

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

This change deprecates all PTools-specific macros in the %UnitTest.Common.INC file. The file still contains the macros (to facilitate any transition), but the macros will be removed in a future release. If you have used the PTools-specific macros, update your code to use the general-purpose ones.

It's also important to note that there are two macros that have the same name between the  PTools-specific macros and the general-purpose macros, so this change renames the following PTools-specific macros:

FROM:
  /// Duplicate MACRO name changed to help transition from DEPRECATED MACROs
  #DEFINE UTDevLog $$$getCurrentUTPToolsDevLog
  #DEFINE UTDevLogQ $$$getCurrentUTPToolsDevLogQ


TO:
  /// 'UTDevLog' & 'UTDevLogQ' replaced by 'UTPToolsDevLog' & 'UTPToolsDevLogQ'
  /// to help transition from the Duplicate & DEPRECATED MACROs
  #DEFINE UTPToolsDevLog $$$getCurrentUTPToolsDevLog
  #DEFINE UTPToolsDevLogQ $$$getCurrentUTPToolsDevLogQ

DP-413836: Correct %Next() of %SQL.ClassQueryResultSet to set %SQLCODE and %Message properties

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

Before this change, the %Next() method of %SQL.ClassQueryResultSet would not set %SQLCODE and %Message. Now, when the %Next() method receives a bad status code from the Fetch() method of a class query, it will accurately set the %SQLCODE and %Message properties.

In some cases, a class query's Fetch may now result in %SQLCODE being set to reflect a failure whereas it may have previously failed silently.

DP-414056: Prevent end user naming of Interoperability Host Names with reserved leading underscore

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2023.1.0

With this change, host names cannot start with an underscore (_) character. If a production includes hosts whose names have the leading underscore, there is a compilation error when you compile the production class.

DP-414622: Require %Development:USE privilege for DDL statements that invoke external language code

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

There are a number of DDL statements in which a user can call ObjectScript or Python (and in some cases Java and DotNet) code. The following shows an example:

CREATE TABLE Test.MyTable(MyField VARCHAR(100), MyOtherField INT COMPUTECODE OBJECTSCRIPT \{set {MyOtherField} = +$h / 2})

Before this code, there was no check that the user executing such DDL statements had %Development:USE privileges.

With this change, a user who does not have %Development:USE permissions will encounter a privilege violation (SQLCODE -99) error and the DDL will fail. The DDL statements affected are CREATE METHOD/PROCEDURE/FUNCTION/QUERY/TRIGGER, and CREATE/ALTER TABLE where the user specifies ObjectScript DEFAULT or COMPUTECODE for a column.

The only scenarios in which a user without proper permissions can still execute such DDL statements are as follows:

• The DDL is executed via embedded SQL (embedded SQL does not do privilege checks)
• The code being run explicitly specifies no privilege checking, for example, a %SQL.Statement prepared with nocheck-priv or run with %ExecDirectNoPriv().

DP-414666: Check for %Service_SQL/%Service_Object/%Service_Native resources in the server

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

There is a new system Resource called %Service_Native. The system-defined roles %Developer and %Manager have the USE permission on this resource by default. %Service_Native controls whether the user can issue Native API calls via Java, .NET, Python, and Node.js. In order to use the Native API, the user must have the %Service_Native:USE permission.

This change also categorizes server functions into three groups:  SQL, Object, and Native. A user that connects to the InterSystems IRIS Server requires:

• The %Service_SQL resource in order to execute SQL functions with a database driver
• The %Service_Object resource in order to execute Object functions
• The %Service_Native resource in order to execute Native/SYSIO functions

DP-414670: Update PTools/SQLStats to report number of commands rather than number of lines

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

This change updates PTools/SQLStats to report the number of commands executed, rather than the number of lines executed. The number of commands is a better reflection of the amount of work that is being performed via InterSystems IRIS routine invocations.

DP-414746: SQL SERVER: Remove obsolete and deprecated functions from DBSRV

NOTE: This item may require a change to code, configuration, or operation.

Category: xDBC Server
Platforms: All
Version: 2023.1.0

With this change, early InterSystems IRIS XEP client versions may no longer work. XEP customers will have to upgrade to the newer XEP client version.

Specifically, this change removes support for the following obsolete and deprecated functions from %SYS.DBSRV:

• All XEP functions have been removed except XW (Bulk Fetch). All other functions have been replaced with SYSIO function calls.
• RS - Retrieve stream (ODBC old implementation)
• CV - Compare TImestamp

DP-414811: Add JWTAudience field to OAuth2 clients

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2023.1.0

This change adds the ability for an InterSystems IRIS OAuth2 client to manually configure the value to use in the "aud" header when using the private_key_jwt and client_secret_jwt authentication methods, as well as for the JWT Authorization grant type. 

This can be done with the new JWTAudience property in the OAuth2.Client class. This can be configured via the "Audience" field in the OAuth2 Client Configuration page in the Management Portal.

For new OAuth2 client configurations, this will default to use the Authorization server's token endpoint. If it is not defined (e.g., for existing configurations) it will continue to use the Authorization server's issuer endpoint. 

NOTE: InterSystems IRIS Authorization servers prior to DP-414485 require clients to use the issuer endpoint. This means that in order to for newly created InterSystems IRIS OAuth2 clients running on a system with this change to authenticate with an InterSystems IRIS OAuth2 Server without DP-414485, the user will need to manually configure the JWT Audience.

DP-415695: Make %PosixTime default display value same as ODBC

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

Before this change, the Display value for %PosixTime values defaulted to the default date and time formats for the InterSystems IRIS system.  Now the default date format is 3 and the default time format is 1, which matches the ODBC format and matches the Display format of %TimeStamp. 

You can always revert to the previous behavior by setting the DATEFORMAT=-1 and TIMEFORMAT=-1 in the type parameters of the %PosixTime property definition.

DP-415945: Interoperability Java and DotNet Gateway services modifications for core api change

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2023.1.0

The classes EnsLib.JavaGateway.Service and EnsLib.DotNetGateway.Service have a new exposed setting in the Production Configuration page of "External Language Server Name" (property %gatewayName).

The value of External Language Server Name is a name given in the Management Portal at System > Configuration > External Language Servers

For an EnsLib.JavaGateway.Service, it can be %Java Server or a custom Java type entry. (Note that when used with a SQL Business Service or Operation that uses JDBC, it is still a Java type entry and not JDBC type)

For an EnsLib.DotNetGateway.Service, it can be %DotNet Server or a custom .NET type.

  Once External Language Server Name is specified, it supersedes the settings in Server and Port and the associated gateway configuration settings such as classpath , heartbeat, use passphrase and JVM.

Note it is now not possible to start a gateway that uses a passphrase without using an External Language Server Name.   The superclass EnsLib.JavaGateway.Common used by Services and Operations that use the gateway now identifies if the associated Java Gateway specifies an External Language Server Name  and uses that value rather than the Server and port values to connect to the relevant proxy gateway. 

It is intended that there will be no need for EnsLib.JavaGateway.Service and EnsLib.DotNetGateway.Service items at a later date.

DP-416085: Interoperability DotNet Gateway .Net version options update

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways - .NET
Platforms: All
Version: 2023.1.0

The EnsLib.DotNetGateway.Service provides a list of available .NET frameworks. These have been updated to:

• Framework 4.6.2
• Framework 4.5
• Framework 3.5
• Framework 2.0
• Core 2.1
• .NET 5.0
• .NET 6.0

Not all binaries may be available. The default is Framework 4.6.2.

DP-416318: Use new readiness script, waitReady.sh

NOTE: This item may require a change to code, configuration, or operation.

Category: ICM
Platforms: All
Version: 2023.1.0

With this change, we use a new readiness script. The readiness script determines when InterSystems IRIS has reached a given state (e.g. "running"), which is especially useful during cluster deployment, where it serves multiple purposes:

• Enforcing startup/shutdown order. For example, in a mirror, don't configure the backup until the primary is up (likewise for ECP client/server).
• Detecting when InterSystems IRIS has become unavailable. For example, this can be part of Kubernetes readiness probe.

With this change, we are now using a new readiness script called waitReady.sh.

DP-416353: Check required resource in Config.Gateway.Delete()

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways - .NET
Platforms: All
Version: 2023.1.0

With this change, in order to delete a gateway, you must have the permission %Admin_ExternalLanguageServerEdit:USE.

DP-416356: Remove support of transient gateway definition in XSLT

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways - XSLT
Platforms: All
Version: 2023.1.0

With this change, it is no longer possible to start the XSLT Gateway using a transient gateway definition specified by command line arguments that override the predefined definition.

DP-416562: Correct WebSocket handling of low-level errors

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP Server
Platforms: All
Version: 2023.1.0

This fix corrects a bug in the error handling for web socket connections. If an error was encountered before the CSP server dispatched to the WebSocket class, then the error would be ignored. Now the CSP server will call the Error() method in the WebSocket class. Similarly, for authentication failures, the server will call Login(). Note that by default, Login() just calls Error(). 

The websocket will also now abort the connection if OnPreServer() returns an error. Previously errors were ignored.

DP-416563: Do not allow navigation to %-CSP pages from the /csp/sys/oauth2/ application

NOTE: This item may require a change to code, configuration, or operation.

Category: CSP Server
Platforms: All
Version: 2023.1.0

This change modifies the default configuration to disallow access to %-CSP pages from the /csp/sys/oauth2/ application.

DP-416898: Mark internal methods in %Net.Remote.Service with Internal keyword

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways - .NET
Platforms: All
Version: 2023.1.0

This change marks many methods in %Net.Remote.Service with the Internal keyword. You should scan your code for uses of %Net.Remote.Service and make sure that it calls only methods that are visible.

DP-417168: Record Mapper and EDI Document Viewer file view permissions check

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2023.1.0

The Interoperability Record Mapper and EDI Document viewer management portal pages enforce holding the privilege %Ens_ViewFileSystem:USE before allowing the user to select a file using the file select dialog popup. This change adds the same check before a file is opened.

DP-417230: Fixed maximum length of monetary amount in HIPAA_5010 X12 validation schemas

NOTE: This item may require a change to code, configuration, or operation.

Category: Interoperability
Platforms: All
Version: 2023.1.0

This change reduces the maximum length for each 782 element (monetary amount) in our HIPAA_5010 validation schemas. Now this maximum length is 10.  It also fixes length validation for decimal elements to not count either the "-" sign or "." if present. As a consequence, a document which had previously passed SNIP validation now fails when this element is too long.

This change brings the schemas into compliance with section B.1.1.3.1.2 of the Implementation Guides for the HIPAA_5010 schemas.

Note, this change does not address the possibility of implied places for cents because we do not provide validation on maximum value.

DP-417351: Change internal repressention of JSON literals; require recompile

NOTE: This item may require a change to code, configuration, or operation.

Category: Kernel
Platforms: All
Version: 2023.1.0

This change modifies the internal representation of JSON literals.

ObjectScript routines/methods compiled before InterSystems IRIS 2022.1 and using numeric values in JSON constructors must either be recompiled in the more up-to-date InterSystems IRIS version or must be executed on an InterSystems IRIS version that includes this change.

DP-418513: Remove Server property from toDao() method and Config.Gateways class

NOTE: This item may require a change to code, configuration, or operation.

Category: Gateways - .NET
Platforms: All
Version: 2023.1.0

This change removes the Server property from the Config.Gateways class.

DP-418706: Update some CONVERT style code behaviors to match MS SQL Server

NOTE: This item may require a change to code, configuration, or operation.

Category: SQL
Platforms: All
Version: 2023.1.0

This change will correct the behavior of CONVERT in InterSystems IRIS to output dates in the format as prescribed by MS SQL server, in order to ensure our function is compatible with MS SQL as stated in the documentation.

Similarly, for code 126, the CONVERT() function now generates output in the format yyyy-mm-ddThh:mi:ss.mmm.

DP-417383: Better defaults for shared memory heap and lock table

NOTE: This item may require a change to code, configuration, or operation.

Category: System
Platforms: All
Version: 2023.1.0

Two parameters in the [config] section of the Configuration Parameter File (CPF) have changed:   gmheap=0 is the new default and is appropriate for most systems (including production systems) where no specific need for a specialized setting has been identified.  A setting of 0 allows the system to make inferences about overall system size and pick a reasonable value, between 300MB and 2GB. Non-zero values are strictly honored and can be set lower than 300MB or higher than 2GB.    locksiz=0 is the new default.  It is limited only by the size of the shared memory heap given by the gmheap parameter.  On upgrade, the current locksiz parameter is automatically set to 0.  If a non-zero value is specified, it specifies the exact number of bytes to serve as an upper limit on lock table memory within the gmheap.

DP-408952: Link to Rule Editor from SMP

NOTE: This item may require a change to code, configuration, or operation.

Category: Graphical Editor
Platforms: All
Version: 2023.1.0

With this change, the Rule Editor is now linked in the Management Portal under Interoperability. This change requires users to add the /ui prefix as a possible forwarding path to the custom web server.

DP-412390: Require minor OS version in installs on Red Hat and SUSE

NOTE: This item may require a change to code, configuration, or operation.

Category: Platforms
Platforms: UNIX®
Version: 2023.1.0

Installation on Red Hat 7 x64 will require minor version 9 or above.

Installation on Red Hat 8 x64 and ARM64 will require minor version 2 or above.

Installation on SUSE 15 x64 will require minor version 3 or above.

DP-412620: Reduce memory contention in ^PERFMON and add options

NOTE: This item may require a change to code, configuration, or operation.

Category: Monitoring
Platforms: All
Version: 2023.1.0

This change reduces overhead of using ^PERFMON on larger systems. It also adds options to memory usage to reduce contention:

• Use multiple 'tables' for the Global, Routine, and Database tables. This should help with general counter 'contention' issues on a very busy system.
• Disable counting of MONLINES. Addresses a specific contention issue for this counter which can get updated a lot for routines.
• Disable counting of 'other' slot for the Process table. This reduces contention when all Process slots are full and new Processes would all get assigned to 'other'.

Internally, the recording of stats is now automatically distributed among multiple tables, and then the reporting of the stats aggregates the multiple tables. This change can use a lot more shared memory now, so there is a prompt to the regular ^PERFMON startup that shows you how much of gmheap would be used, and asks if you want to continue. 

There are options to disable the counting of MONLINES (aka RtnLine) in the PERFMON startup, the $$Start^PERFMON API, and the class API in %Monitor.Manager.

Also added parameters to $$Collect^PERFMON() for routines and globals so that you can collect for larger numbers of routines/globals. Modified defaults for table sizes to be consistent.

DP-412937: Audit log improvements

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2023.1.0

The audit log has been corrected as follows, for better usability:

• Background tasks started in the Portal are now audited with a TaskStart and TaskEnd audit record. Previously there was just a JobStart and JobEnd logged, but it did not specify what task was being run.
• Started and stopping of %SYS.WorkQueue jobs are excluded from the audit log.
• Starting and stopping of %Service_WebGateway jobs are excluded from the audit log.

DP-413465: Return meaningful error from ##class(%SYS.Journal.History).GetHeader()

NOTE: This item may require a change to code, configuration, or operation.

Category: Journaling
Platforms: All
Version: 2023.1.0

In the case where the journal log contains no or bad version info, ##class(%SYS.Journal.History).GetHeader() (and its direct or indirect callers such as ##class(Journal.Restore).CheckJournalIntegrity()) now returns a meaningful error such as "Journal log <path> missing version", which is also compatible with the existing $System.Status facility. The old return value in this case, "0v", is obscure in meanings and incompatible with the existing $System.Status facility.

DP-413960: Limit PERFMON collection (index) numbers

NOTE: This item may require a change to code, configuration, or operation.

Category: Monitoring
Platforms: All
Version: 2023.1.0

Limit all PERFMON collections to 64k. Before this change, use of collections >65535 would silently fail; now an error is returned.

DP-414613: SECURITY: Require new %Secure_Native Resource:USE for Native API calls

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2023.1.0

There is a new system Resource called %Service_Native.  The system-defined roles %Developer and %Manager have the USE permission on this resource by default.

%Service_Native controls whether the user can issue Native API calls via Java, .NET, Python, and Node.js.  In order to use the Native API, the user must have the %Service_Native:USE permission.

DP-414881: Require xlC runtime libraries 16.1.0.9+ on AIX

NOTE: This item may require a change to code, configuration, or operation.

Category: Installation
Platforms: AIX
Version: 2023.1.0

With this change, the InterSystems IRIS installer on AIX will require  xlC.rte and libc++.rte version 16.1.0.9 or above.

DP-415808: CPF file changes related to gateways

NOTE: This item may require a change to code, configuration, or operation.

Category: CPF
Platforms: All
Version: 2023.1.0

This change updates the Gateways section of the CPF file to reorganize the fields for each gateway type, and to add a resource field.

For each gateway type, the CPF file no longer has heartbeat properties. For all types except Remote, there is a new default Resource field. The UsePassphrase property has been removed.

DP-416117: Enable more minimal installation

NOTE: This item may require a change to code, configuration, or operation.

Category: Installation
Platforms: Windows
Version: 2023.1.0

With this change, the Windows installation will have a new feature under Development with the name "Other Development Libraries" with internal name "development_other". All components under Development that are not assigned to any sub-feature are now contained in "Other Development Libraries".

With this change, the installer can create a more minimal installation.

DP-416498: Allow CPF properties to be set to ""

NOTE: This item may require a change to code, configuration, or operation.

Category: CPF
Platforms: All
Version: 2023.1.0

A bug has been fixed where in previous versions, when setting a CPF property to a null value, if there was a default value for the property, the default value would be used instead of setting it to null.

For example, when passing the Properties array to Config.Databases.Create:

set Properties("MountAtStartup")=""

Starting in 2022.1.1, we will actually set the property equal to the empty string. For required properties, this will throw an error similar to:

ERROR #5659: Property 'Config.Databases::MountAtStartup(1@Config.Databases,ID=)' required

If you encounter this issue after upgrading, simply remove the setting of the Properties array to "", or explicitly set it to the desired value as in the following example:

set Properties("MountAtStartup")=0

DP-417320: Enforce SOAP/REST licensing

NOTE: This item may require a change to code, configuration, or operation.

Category: Licensing
Platforms: All
Version: 2023.1.0

In previous versions, the product did not enforce the SOAP/REST licensing rules.

With this version, we are now enforcing those licensing rules. Each authenticated request will be licensed as a concurrent user (with multiple connections allowed) only when providing both SOAP and REST services. Unauthenticated requests (i.e. $Username = "UnknownUser") will be counted as independent user connections and subject to a 10-second minimum connection time.

DP-417392: Change "iris merge" return status to follow UNIX® standard

NOTE: This item may require a change to code, configuration, or operation.

Category: CPF
Platforms: All
Version: 2023.1.0

This change modifies how "iris merge" handles these cases:

1) If the merge file does not exist. In this case, we report an error and return exit status 0. This applies specifically to this command:

iris merge IRIS <file>

iris: Unable to find/open file     

 2) If the merge file does exist and the merge is successful, we report success and return exit status 0. This applies specifically to these commands:

iris merge IRIS <file>
iris merge IRIS

IRIS Merge completed successfully

With these changes, the merge commands match the UNIX® standard.

DP-417758+PLUS: Rename and move mirroring and sharding actions

NOTE: This item may require a change to code, configuration, or operation.

Category: CPF
Platforms: All
Version: 2023.1.0

With these changes, the names are now "ConfigMirror" and "ConfigShardedCluster".

Before these changes, mirroring and sharding configuration parameters resided in Startup section:

[Startup]
MirrorMember=
MirrorPrimary=
MirrorSetName=
ShardClusterURL=
ShardMasterRegexp=
ShardMirrorMember=
ShardRegexp=
ShardRole=

[Actions]
ConfigMirror:Name=,Member=,Primary=
ConfigShardedCluster:ClusterURL=,Role=,Member=,MasterRegexp=,Regexp=

MirrorMember --> Member
MirrorPrimary --> Primary
MirrorSetName --> Name
ShardClusterURL --> ClusterURL
ShardMasterRegexp --> MasterRegexp (default is -0$)
ShardRegexp --> Regexp (default is -[0-9]+$)
ShardMirrorMember --> Member
ShardRole --> Role

This change also adds the classmethod SYS.ICM.ShardingCheck() to allow applications to determine if sharding configuration is complete.

PLATFORMS-CENTOS-7: Centos 7 no longer supported as a developer platform

NOTE: This item may require a change to code, configuration, or operation.

Category: Platforms
Platforms: All
Version: 2023.1.0

In this release, Centos 7 is no longer supported as a developer platform.

PLATFORMS-ICU-691: ICU 69.1 & Xerces 3.2

NOTE: This item may require a change to code, configuration, or operation.

Category: Platforms
Platforms: All
Version: 2023.1.0

In this release, ICU has been updated to version 69.1. Also the current Xerces version is now 3.2.

PLATFORMS-POWERPC: PowerPC chip sets no longer supported

NOTE: This item may require a change to code, configuration, or operation.

Category: Platforms
Platforms: AIX
Version: 2023.1.0

This release drops support for the older PowerPC chip sets. For IBM AIX, the product is now supported only on POWER Systems (POWER 7 and higher).

PLATFORMS-RH7: Remove RH7 support

NOTE: This item may require a change to code, configuration, or operation.

Category: Platforms
Platforms: UNIX®
Version: 2023.1.0

This release removes support for Red Hat 7.x.

PLATFORMS-UBUNTU-1804: Remove Ubuntu 18.04 support

NOTE: This item may require a change to code, configuration, or operation.

Category: Platforms
Platforms: UNIX®
Version: 2023.1.0

This release removes support for Ubuntu 18.04.

DP-422560: HTML encode REST error messages

NOTE: This item may require a change to code, configuration, or operation.

Category: Security
Platforms: All
Version: 2023.1.1

Previously, error messages returned from REST APIs were not HTML encoded. This issue has been corrected.