Extended Database Size

Enabling Extended Database Size

There is a system wide setting to allow Extended Database Size. When this setting is ON, as a database expands within 100 GB of the original limit, it will be automatically updated to the extended format. This setting is OFF by default so that a) the system will not generate database files that cannot be mounted by older systems and b) an integrity check can be run on all existing databases first as described below. Once any older-version ECP Application Servers, Mirror Members, or other systems that may need to mount this system's databases are upgraded, and after an integrity check on all existing databases has shown no errors, it is recommended to turn this setting ON.

Additional details on the Extended Database Size setting:

• When the setting is OFF, the system can still mount an extended format database, but it won't generate any anew.

• When this setting is ON, as a database expands within 100 GB of the original limit, it will be automatically updated to the extended format. This is true whether that expansion is due to explicitly setting the Current Size, or due to normal database expansion.

• Setting the Maximum Size of a database is restricted to the original limit unless either the setting is ON or the database in question is already using the extended format. When the setting is ON and Maximum Size is set greater than the original limit, it will be updated to use the extended format immediately. This can be used to generate an extended format database with a small size for test purposes, but otherwise it is recommended to leave the Maximum Size unset and allow databases to be updated to extended format only as size demands it.

The Extended Database Size is accessible as the property ExtendedDatabaseSize in the [Miscellaneous] section of the CPF file, the Config.MiscellanousOpens in a new tab class, and the Compatibility page of the Management Portal (System Administration > Configuration > Additional Settings > Compatibility).

Integrity Check

Bytes in the header of each database block unused by default are now used to store high-order bits of the right-link block number when the Extended Database Size is turned on. Unused bytes by default are set to zero when blocks are initialized and they are never modified. It is therefore expected - excluding anomalies such as cosmic rays or other sources of extremely isolated storage corruption - that these bytes are always zero on existing databases. However, in versions prior to InterSystems IRIS 2026.1, integrity check did not check the value of these bytes to confirm this expectation.

Integrity check now checks the value of those unused bytes as well another unused byte that's reserved for future use. It reports any errors as type 40 "Unused bytes in block header were unexpectedly nonzero", or type 41 "Unused bytes in block header of bigstring block #NNN were unexpectedly nonzero." In the exceedingly unlikely case that this error is found, the affected blocks can be corrected easily. Alternatively, the check of unused blocks in integrity check can be disabled temporarily and integrity check can be re-run to confirm that the database is otherwise okay, deferring corrective action until a more convenient time. Some action should be taken promptly, though, as it is possible for this error to mask a more serious error that would otherwise be reported by integrity check.

If errors of type 40 or 41 are detected by integrity check:

• To correct the blocks, run the following command from the %SYS namespace:

  Do ZeroUnusedBytes^REPAIR(dir,$listbuild(blk1,blk2,etc))
  

  

  Pass dir as the database directory and the list of affected blocks from the integrity check.

  It takes no user input but does output status messages. If called as a function, it returns 1 on success (or if the blocks didn't require repair) or 0 on error. Note that this uses switch 10 to pause users very briefly while performing the repair (similar to that experienced by namespace activation) and therefore may be best scheduled off peak hours. Rerun the integrity check again after this repair, as the repaired error could have masked additional errors in some circumstances.

• To temporarily disable the check of unused bytes in integrity check, from the %SYS namespace:

  Do UnusedBytesOn^Integrity(0)
  

  

  Replacing the 0 argument with a 1 turns it back on. If called as a function it returns the prior value, and omitting the argument returns the value without changing the setting.  The setting is automatically reverted on the next startup.

• Call the Worldwide Response Center if you need help determining the best course of action.

Runtime Safeguard

Until you choose to enable extended database sizes, having a non-zero value in those previously unused bytes will not cause problems. When a database file has not yet been updated to the extended format, the system zeroes the high bytes of the right link in memory upon reading from the database file to ensure that it is safe to adopt a version with this feature without taking special action for existing databases.

If this condition is found during normal database access, a warning will be logged to the messages.log:

[Database.LinkHighRead] Read of database /path/to/db/ block NNNN found a non-zero value (X) in the high bytes of the right link but this database does not have extended sizes enabled. Run integrity check and correct any errors before enabling extended database sizes.

Note that this warning is not logged if the block is read by utilities such as integrity check, ^%GSIZE, ^REPAIR, or by the VIEW command, though the bytes are zeroed nonetheless to prevent any problems. When the block in memory has bytes zeroed in this way, that information is tracked in memory so that integrity check can still report on the error.

It is possible for the problem in the block to be self-correcting if it later gets modified or excised from the tree by database updates. Even if that were to happen however, integrity check must be run to detect any other affected blocks.

If a database were updated to the extended format but had non-zero values in the high bytes of the right link, the error would be reported by integrity check as an incorrect right link, and could lead to an error such as <DATABASE> or <BLOCKNUMBER>. If the database had a non-zero value in the reserved unused byte, it would still be reported as type 40 or 41, "Unused bytes ... unexpectedly nonzero".

Interaction with Database Properties

There are several properties of a database for which the new limit is relevant.  The behavior when a large size is selected is tailored to each property:

• Size may be set higher than original limit if Extended Database Size is ON, or if the database already has the extended format.

• Maximum Size is recommended to be left at zero to allow unlimited expansion driven by demand. A non-zero value higher than the original limit may be set if Extended Database Size is ON or if the database already has the extended format, and the database will be converted to extended format. This can be used for test purposes to generate an extended format database without requiring it to be extremely large, but otherwise it is not recommended to be used as a way to prematurely update a database to extended format.

• Expansion Size is recommended to be left at zero to allow the system to select an expansion size. The limit for Expansion Size remains at the smaller original limit in all cases.

• New Volume Threshold may be set to a value higher than the original limit if and only if the database already has extended format.

• New Global Growth / Pointer Block settings are recommended to be left unchanged. The limit for this setting is given by the format.

• The BlockFormat property in the SYS.DatabaseOpens in a new tab class has the value 2 if the database has the original format and 3 if it has the extended format. This is also displayed annotated as Base or Extended in ^DATABASE option 9 and the Management Portal’s database details page.

Note that the Management Portal interface prevents saving the properties if they are out of bounds whereas the SYS.DatabaseOpens in a new tab class and ^DATABASE utility generally silently reduce values that are out of bounds. (An exception is the New Global Growth / Pointer Block where a value over the limit is treated as an error).