Skip to main content

Configuring Journaling

This page describes how to enable and configure journaling.

Enabling Journaling

By default, journaling is enabled for the InterSystems IRIS® data platform databases IRISSYS, IRISAUDIT, and USER. You can enable or disable journaling on each database from the Local Databases page of the Management Portal (System Administration > Configuration > System Configuration > Local Databases). Click Edit on the row corresponding to the database and click Yes or No in the Global Journal State box.

The default setting of the journal state for new databases is Yes. When you first mount a database from an earlier release of InterSystems IRIS, the value is set to Yes, regardless of the previous setting for new globals and regardless of the previous settings of individual globals within that database.

You can change the global journal setting for a database on a running system. If you do this, InterSystems IRIS warns you of the potential consequences and audits the change if auditing is enabled.

Journal File Names and Rollover

A journal file’s name consists of an optional user-defined prefix, followed by base name consisting of the date and time it is created (in the format yyyymmdd), a period (.), and then a three-digit suffix used to incrementally number the journal files created during one calendar day. When a journal file fills, the system automatically switches to a new one with the same prefix and base name but with the suffix increased by one. The base name changes only if a new calendar day begins while the journal file is in use.

For example, if the first journal file that is active on April 27, 2023 is named 20230427.001. When it fills, the system starts a new one called 20230427.002; this process is known as journal file rollover. If midnight passes and the date changes while the journal file is in use, however, it is renamed 20230428.001.

Journaling Best Practices

The following are some important points to consider when planning and configuring journaling:

  • Journal files are stored in both a primary journal directory and an alternate journal directory (for use if the primary directory becomes unwriteable for any reason).

    In the interests of both performance and recoverability, InterSystems recommends placing the primary and alternate journal directories on storage devices that are separated from the devices used by databases and the write image journal (WIJ), as well as separated from each other. For practical reasons, these different devices may be different logical unit numbers (LUNs) on the same storage area network (SAN), but the general rule is the more separation the better, with separate sets of physical drives highly recommended. The major benefits of this separation between database/WIJ storage and primary and alternate journal storage include the following:

    • Isolating journal directories from failures that may compromise the databases or WIJ ensures that journal files will be available for use in restoring the database after such a failure.

    • Separating primary and alternate journal directories ensures that when an outage occurs on the device on which the primary directory is located, journaling can continue.

    • Separating journal I/O paths is a key factor in achieving the I/O concurrency that most applications require.

    For simplicity and convenience, InterSystems IRIS installation creates the directory install-dir\Mgr\journal, configures it as both the primary and alternate journal directory, and creates in it the first journal file for the default journaled databases. InterSystems recommends, however, that you identify and prepare separate storage devices for the primary and alternate journal directories and reconfigure these settings (as described in Configuring Journal Settings) as soon as possible after installation.

    Note:

    Journal files should always be backed up along with database files, as described in Backup and Restore. Consider replicating journal files offsite for disaster recovery purposes, enabling recovery from a failure involving multiple storage devices at the primary data center. InterSystems IRIS mirroring, disk-level replication, or other mechanisms can be used for this purpose.

  • Verify that journaling is enabled for all databases (other than those that contain only transient data).

    Important:

    Be sure to read Consequences of Not Journaling Databases for important information about limits to the recovery of non-journaled databases.

  • Consider setting the journal Freeze on error option to Yes. If a failure causes the system to be unable to write to both the primary and the alternate journal devices, this setting causes the system to freeze, making it unavailable to users and ensuring that no data is lost. Alternatively, you can set Freeze on error option to No, which lets the system continue and leads to journaling being disabled, keeping the system available but compromising data integrity and recoverability. See Journal I/O Errors for more information about Freeze on error.

  • Do not purge a journal file unless it was closed prior to the last known good backup, as determined by the backup validation procedure. Set the number of days and the number of successful backups after which to keep journal files appropriately.

  • To ensure optimal performance during a journal restore, consider increasing the size of the shared memory heap (gmheap); see Restore Journal Files for more information.

  • Ensure that journal file compression is enabled to minimize the amount of space taken up by journal files. This option is controlled by the CompressFiles CPF setting, and is enabled by default for new InterSystems IRIS installations.

  • Consider also archiving journal files. When you specify an archive target for journaling, the journal file is copied to that location when the file reaches the maximum size and a new journal file is started. You can also configure journaling so that the original journal file is purged after it has been copied to the archive.

Configuring Journal Archive Targets

To configure archive targets for InterSystems IRIS journal files (which you can use when configuring journal settings), navigate to the Archive Targets page of the Management Portal (System Administration > Configuration > System Configuration > Archive Targets).

On this page, you can view, edit, or delete an archive target.

This page lists any existing archive targets. The fields are as follows:

  • Name—Unique name for the archive target, for use when archiving files.

  • Type—Specifies how a file is to be copied to this target location. This can be one of the following

    • s3, in the case of an S3 location.

    • rsync, in the case of an on-premises location. Note that if the location is a Windows directory, the system actually uses robocopy, but this page lists the type as rsync for simplicity.

  • Location—Specifies the actual directory. The format depends on whether the location is an S3 location or is on premises.

    • For S3, Location has the following format:

      s3://directoryname/

      For example:

      s3://test-dir/journal-archives/

    • For UNIX®, Location can have either of the following formats: 

      /directoryname/

      Or:

      server:/directoryname/

    • For Windows, Location can have either of the following formats: 

      \\server\share\directoryname\

      Or:

      drive:\directoryname\

To create an archive target, click Create New Archive Target, specify values as given above, and click Save. Note that when you create or edit an archive target, the choices for Type are AWS S3 and rsync or robocopy. In the latter case, the system will use the applicable option based on the format of the given location.

To edit an archive target, click the Edit link for the target, modify values as needed, and click Save.

To delete an archive target, click the Delete link for the target, and then click OK to confirm.

Configuring Journal Settings

To configure InterSystems IRIS journaling, navigate to the Journal Settings page of the Management Portal (System Administration > Configuration > System Configuration > Journal Settings).

You can edit the following settings:

  • Primary journal directory — Enter the name of a directory in which to store the journal files. The directory name may be up to 214 characters long.

  • Secondary journal directory — Enter the name of an alternate directory for journaling to use if the current directory becomes unwritable for any reason. (You can also manually switch journal directories, as described in Switch Journal Directories.) The directory name may be up to 214 characters long.

    Important:

    InterSystems recommends placing the primary and alternate journal directories on storage devices that are separated from the devices used by databases and the write image journal (WIJ), as well as separated from each other; see Journaling Best Practices for more information.

  • Start new journal file every — Enter the number of megabytes for the maximum size of the journal file after which the journal file switches. The default size is 1024 MB. The maximum size is restricted by 4096 MB – 1 – size of the journal buffer (see jrnbufs, below). Using the default journal buffer size of 64 MB, the maximum size of a journal buffer is 4031 MB. As the minimum size of the journal buffer is 16 MB, the absolute maximum size of a journal file is 4079 MB.

  • Journal File Prefix (optional) — Enter an alphanumeric prefix for journal file names.

  • Archive journal files (optional) — Select a value from the dropdown list in To Archive Target or click Create New Archive Target... and define a new archive target.

  • When to purge journal files — Specify when to purge journal files that are not needed by the system.

  • For this set of options, if you have specified an archive target, the page displays the optional check box As soon as they are copied to the archive. If you select this check box, the next two options are ignored (and are grayed out).

    If you do not specify an archive target, or if you do not select As soon as they are copied to the archive, then you can set either or both of the following two options. If you enter nonzero values for both settings, purging occurs when a journal file meets whichever of the two conditions occurs first. If you set 0 (zero) for one and not the other, purging is determined by the nonzero setting. If both are 0, the automatic purging of journal files (and journal history) is disabled.

    • After this many days — Enter the number of days after which to purge (valid values: 0-100).

      Note:

      When you set the number of days after which to purge journal files, the last journal file from the day before the purge limit is also retained. For example, if After this many days is set to 1 and the purge is run at midnight on April 1, the last journal file created on March 30 is retained along with those created on March 31.

    • After this many successive successful backups — Enter the number of consecutive successful backups after which to purge (valid values: 0-10).

      This includes an online backup, an external backup using $$BACKUP^DBACK("","E"), or the use of the Backup.General.ExternalSetHistory()Opens in a new tab method to add to the backup history.

    Note:

    If After this many days is set to 0 (no time-based purge) and After this many successive successful backups is set to 1, journal files are not purged until there have been two successful backups; that is, there must be two successful backups for the “successive” criterion to be met.

    You can also update these settings using the ^JRNOPTS routine or by selecting option 7, Edit Journal Properties, from the ^JOURNAL routine menu. See Update Journal Settings Using ^JRNOPTS for details.

    Note:

    Journal files are sometimes retained even if they meet the criteria of the purge setting. When this happens, the event is recorded in the messages log and the reason (for example, that the journal file contains open transactions) is provided.

  • Freeze on error — This setting controls the behavior when an error occurs in writing to the journal. The default is No (unselected). See Journal I/O Errors for a detailed explanation of this setting.

    Note:

    When an InterSystems IRIS instance is the primary failover member of a mirror (see Mirroring), the instance’s Freeze on error configuration is automatically overridden to freeze all journaled global updates when a journal I/O error occurs, regardless of the current setting. If the current setting is No, behavior reverts to this setting when the instance is no longer a primary failover member.

  • Journal Web Session — This setting controls whether or not Web Server Page session journaling is enabled. The default is No (unselected).

  • Compress journal files — This setting controls whether or not to compress journal files. The default is Yes (selected). See CompressFiles for more information.

    Note:

    When both journal compression and journal encryption (see Journal Encryption Using ENCRYPT^JOURNAL) are active, journal data is always compressed before being encrypted; encrypted data is never compressed.

  • Write image journal entry — Enter the location of the write image journal (WIJ) file. See Write Image Journaling for a detailed explanation of this setting.

  • Target size for the wij (MB) (0=not set) — Enter the target size for the WIJ file.

Note:

All of the settings on this page are included in the instance’s iris.cpf file. For information about the journal settings, see [Journal] for information about the WIJ settings, see Write Image Journal (WIJ) Settings, as well as targwijsz and wijdir).

You are not required to restart InterSystems IRIS after changing most of these settings (except where indicated), but any change causes a new journal file to begin.

There are two additional configuration settings affecting journaling, as follows:

  • jrnbufs — Specifies the amount of memory allocated to journal buffers; the default is 64 MB, the maximum is 1024 MB, and the minimum is 16 MB for Unicode instances and 8 MB for 8-bit instances. Increasing this setting (see jrnbufsOpens in a new tab) means increasing the amount of journal data that can be held in memory, which improves journaling performance, but increases the maximum amount of journal data that could be lost in the event of a system failure because it was written to the buffer after the last journal sync (see The Journal Write Cycle). Because this setting and the Start new journal file setting (described above) have a combined size limit of 4 GB, increasing jrnbufs can potentially reduce the journal file size.

    To change the jrnbufs setting, navigate to the Advanced Memory Settings page of the management portal (System Administration > Configuration > Additional Settings > Advanced memory). The jrnbufs setting can also be changed by editing the iris.cpf file; for more information, see jrnbufs in the Configuration Parameter File Reference.

  • SynchCommit — Specifies when the TCOMMIT command requests that journal data involved in a transaction be flushed to disk: when this setting is true, TCOMMIT does not complete until the journal data write operation completes; when it is false (the default), TCOMMIT does not wait for the write operation to complete.

    To change the SynchCommit setting, navigate to the Compatibility Settings page of the management portal (System Administration > Configuration > Additional Settings > Compatibility). For more information on SynchCommit, see SynchCommit and TCOMMIT.

See Also

Next sectionBasic Journaling Operations
Previous sectionJournaling Overview
FeedbackOpens in a new tab