Skip to main content

Journaling Utilities

InterSystems IRIS® data platform provides several utilities to perform journaling tasks. The ^JOURNAL utility provides menu choices to run some common journaling utilities, which you can also run independently. There are also several other journaling utilities, which you run from the %SYS namespace.

In the following sections the sample procedures show C:\MyIRIS as the InterSystems IRIS installation directory.

Perform Journaling Tasks Using ^JOURNAL

The following example shows the menu available by invoking the ^JOURNAL routine; the full menu is not repeated in subsequent examples.

%SYS>Do ^JOURNAL
 
 1) Begin Journaling (^JRNSTART)
 2) Stop Journaling (^JRNSTOP)
 3) Switch Journal File (^JRNSWTCH)
 4) Restore Globals From Journal (^JRNRESTO)
 5) Display Journal File (^JRNDUMP)
 6) Purge Journal Files (PURGE^JOURNAL)
 7) Edit Journal Properties (^JRNOPTS)
 8) Activate or Deactivate Journal Encryption (ENCRYPT^JOURNAL())
 9) Display Journal status (Status^JOURNAL)
10) -not available-
11) -not available-
12) Journal catch-up for mirrored databases (MirrorCatchup^JRNRESTO)
13) Switch Journaling to Secondary Directory (SWDIR^JOURNAL)

Option?
Note:

Option 11) Manage pending or in progress transaction rollback (Manage^JRNROLL) is displayed if pending or in-progress transaction rollbacks are encountered when you run the ^STURECOV (at system startup) or ^MIRROR (at primary mirror member startup) routine; for more information, see Manage Transaction Rollback Using Manage^JRNROLL.

Enter the appropriate menu number option to start that particular routine. Press Enter without entering an option number to exit the utility. The following subsections describe the options available through the ^JOURNAL utility:

Start Journaling Using ^JRNSTART

To start journaling, run ^JRNSTART or enter 1 at the Option prompt of the ^JOURNAL menu, as shown in the following examples.

Example of running ^JRNSTART directly:

%SYS>Do ^JRNSTART

Example of starting journaling from the ^JOURNAL menu:

%SYS>Do ^JOURNAL
 
 1) Begin Journaling (^JRNSTART)
 ...
Option? 1

If journaling is running when you select this option, a message similar to the following is displayed:

Already journaling to C:\MyIRIS\mgr\journal\20231113.001 

Stop Journaling Using ^JRNSTOP

To stop journaling, run ^JRNSTOP or enter 2 at the Option prompt of the ^JOURNAL menu, as shown in the following examples.

Note:

When the Freeze on error flag (see Configure Journal Settings) is set to “Yes,” stopping journaling is allowed (although there is a risk of data loss) and does not cause the instance to freeze.

Example of running ^JRNSTOP directly:

%SYS>Do ^JRNSTOP
 
Stop journaling now? No => Yes


Example of stopping journaling from the ^JOURNAL menu:

%SYS>Do ^JOURNAL
 
 ...
 2) Stop Journaling (^JRNSTOP)
 ...
Option? 2
Stop journaling now? No => Yes

If journaling is not running when you select this option, you see a message similar to the following:

Not journaling now.

Switch Journal Files Using ^JRNSWTCH

To switch the journal file, run ^JRNSWTCH or enter 3 at the Option prompt of the ^JOURNAL menu, as shown in the following example:

%SYS>Do ^JOURNAL
 
 ...
 3) Switch Journal File (^JRNSWTCH)
 ...
Option? 3
Switching from: C:\MyIRIS\mgr\journal\20231113.002
To:             C:\MyIRIS\mgr\journal\20231113.003

The utility displays the name of the previous and current journal files.

Switch Journaling Directories Using SWDIR^JOURNAL

To switch journaling directories, assuming a secondary directory is configured as described in Configuring Journal Settings, run SWDIR^JOURNAL or enter 13 at the Option prompt of the ^JOURNAL menu, as shown in the following example:

%SYS>Do ^JOURNAL
 
 ...
13) Switch Journaling to Secondary Directory (SWDIR^JOURNAL)

Option? 3
Option? 13
Journaling to \\remote\MyIRIS\journal_secondary\MIRROR-MIRRORONE-20230720.007

The utility displays the name of the current journaling directory and journal file following the switch.

Restore Globals From Journal Files Using ^JRNRESTO

The InterSystems IRIS ^JRNRESTO routine is used after a database is restored from backup to return it to its state immediately prior to a failure by applying database updates from journal files. This is called a journal restore, and the process of applying the updates is called dejournaling. A journal restore dejournals all journal records created between the creation of the backup and the failure. For example, if the database was backed up early Tuesday morning and crashed on Wednesday afternoon, after you restore the Tuesday backup, you can restore updates from the journal files created on Tuesday and Wednesday.

If sufficient system resources are available, up to 16 dejournaling jobs can perform the updates in parallel within a journal restore operation (see System Requirements for Parallel Dejournaling). This increases the performance of the operation and is called parallel dejournaling. Multiple updaters can update the same database simultaneously, but not the same global, which allows for higher throughput when updates are applied to a small number of databases while ensuring that the data within each global is logically consistent.

Parallel dejournaling is enabled only when the host system has sufficient CPUs to support it and the InterSystems IRIS instance has enough shared memory heap (gmheap) available to allocate for this purpose. In practice, parallel dejournaling will not be used in journal restores on most InterSystems IRIS instances unless gmheap is increased. The number of parallel dejournaling jobs can never exceed the size of the gmheap divided by 200 MB; for example, to support four dejournaling jobs running in parallel, gmheap must be at least 800 MB. (Even if you do not have enough memory available to support parallel dejournaling, dejournaling throughput may improve if you increase the size of the shared memory heap from the default.)

Note:

To change the size of the shared memory heap or gmheap (sometimes known as the shared memory heap or SMH), navigate to the Advanced Memory Setting page (System Administration > Configuration > Additional Settings > Advanced Memory); see Memory and Startup Settings for more information.

Parallel dejournaling is also used by InterSystems IRIS mirroring; for more information, see Configuring Parallel Dejournaling.

^JRNRESTO restores only to databases whose journal state is Yes at the time of the journal restore. The first time it encounters each database, the routine checks and records its journal state. The restore process skips journal records for databases whose journal state is No. If no databases are marked as being journaled, the routine asks if you wish to terminate the restore; you can then change the database journal state to Yes on specific databases and restart ^JRNRESTO.

Note:

The journal state of a database at the time of restore determines what action is taken; InterSystems IRIS stores nothing in the journal about the current journal state of the database when a given journal record is written. This means that changes to databases whose journal state is Yes are durable, but changes to other databases may not be. InterSystems IRIS ensures physical consistency, but not necessarily application consistency, if transactions involve databases whose journal state is No.

^JRNRESTO lets you make several decisions about the journal restore, as follows:

  • Restore global updates to databases in the InterSystems IRIS instance in which you are running the routine, or to databases in another InterSystems IRIS instance. You can choose to restore updates for all globals to all databases in the current instance, or to select individual databases in the current or another instance and optionally specify which globals to restore to each.

  • Restore mirror journal files to a mirrored database (catch up a mirrored database) or to a nonmirrored database. On a mirror member, you are prompted to indicate whether you are catching up a mirrored database, as noted in the following procedure; if so, the procedure is redirected to the MirrorCatchup^ entry point to ^JRNRESTO (see Restore Journal to Mirrored Database Using MirrorCatchup^JRNRESTO).

  • Apply existing journal filters (see Filter Journal Records Using ^ZJRNFILT) to the restore.

  • Select a range of journal files to restore from.

  • Disable journaling of updates during the restore to make the operation faster.

  • Choose what to do when an error occurs, as follows:

    • Continue with the journal restore despite database-related problems, for example, a target database cannot be mounted or an error occurs while applying an update. With this setting, updates to the affected database(s) are skipped; these databases may not be consistent with the other databases following the operation, or may contain inconsistent globals (although the logically consistency of the data within each global is guaranteed) and will need to be recovered separately.

    • Abort the journal if an update would have to be skipped due to a database-related problem. With this setting, databases and the globals they contain will be consistent with each other as of the record that caused the restore to be aborted. Parallel dejournaling is disabled with this setting.

Caution:

If you use journal restore scripts based on prompts, you should update the scripts because some prompts may have changed since the last release.

To restore global updates from journal files:

  1. Run the ^JRNRESTO routine in the %SYS namespace, then press <Enter> at the Restore the Journal? prompt to continue.

  2. If you are running the routine on a mirror member, the following prompt is displayed.

    Catch-up mirrored databases? No =>
    
    
    • If you are restoring mirror journal files to a mirrored database in the same mirror in which the mirror journal files were created, enter yes; the procedure is redirected to the MirrorCatchup^ entry point to ^JRNRESTO (see Restore Journal to Mirrored Database Using MirrorCatchup^JRNRESTO).

    • If you are restoring mirror journal files to a nonmirrored database, or are not restoring mirror journal files, enter no or <Enter> and continue to use the procedure described here.

  3. If you have existing journal filters (see Filter Journal Records Using ^ZJRNFILT), specify whether you want to use them:

    Use current journal filter (ZJRNFILT)? 
    Use journal marker filter (MARKER^ZJRNFILT)? 
    
    
  4. Choose whether you want to restore all journaled globals to all databases in the current InterSystems IRIS instance, or to specify one or more databases and optionally specify which globals to restore to each.

    Process all journaled globals in all directories? 
    
    
    • Enter Yes if you want to restore all globals to all databases in the current instance.

    • Enter No or <Enter> if you want to restore only selected databases in the current or another instance. Then do the following:

      • Indicate whether the journal files were created under a different operating system from that of the current system. This is important because the directory paths you specify for the databases you want to restore must exactly match the paths in the journal files, which are in canonical form. If you respond with No, ^JRNRESTO puts the directory paths you enter into canonical form for the current operating system so they will match those in the journal files. If you respond with Yes, ^JRNRESTO does not canonicalize the paths you enter, because the canonical form in the journal files is different from the canonical form on the current system. In the latter case, you must take care to enter directory paths in the canonical form appropriate to the operating system of the journal files to ensure that they will match.

        For example:

        • if you are working on a Windows system and enter No at this prompt, then enter the path c:\intersystems\iris\mgr\user, ^JRNRESTO automatically canonicalizes this to c:\intersystems\iris\user\ to match journal files created on a Windows system.

        • if you are working on a Unix® system and enter Yes at this prompt because the journal files were created on a Windows system, you must be sure to enter the canonical form of the path, c:\intersystems\iris\mgr\user\, to ensure that it matches the journal files, because ^JRNRESTO cannot canonicalize it for you.

      • Specify each database you want to restore by entering its directory path; this indicates the source database from which the journal records were taken. Press <Enter> at the Redirect to directory prompt to indicate that source and target are the same and restore global updates to the source database. If you are restoring to a different database, for example because you have restored the source database from backup to a different system, enter the directory path of the target database.

        If you are restoring mirror journal files to a nonmirrored database, at the Directory to restore prompt, you can do either of the following:

        • Enter directory path of the source database and then either <Enter> or the directory path of the target nonmirrored database, as described in the foregoing.

        • Enter the full, case-sensitive mirror database name of the source mirrored database, for example,:mirror:JLAP:MIRRORDB, which can be found using the List mirrored databases option on the Mirror Status menu of the ^MIRROR utility, and then specify the directory path of the target nonmirrored database.

        Note:

        If you are restoring mirror journal files to a mirrored database, you will not have reached this point in the procedure; see Restore Journal to Mirrored Database Using MirrorCatchup^JRNRESTO.

      • For each database you specify, either confirm that you want to restore updates for all globals or enter one or more globals to restore.

      • When you have entered all the databases, press <Enter> at the Directory to restore prompt, then confirm the list of specified databases and globals.

      For example:

      Process all journaled globals in all directories? no
      Are journal files imported from a different operating system? No => No
       
      Directory to restore [? for help]: c:\intersystems\test23\mgr\user\
      Redirect to Directory: c:\intersystems\test23\mgr\user\
       => --> c:\intersystems\test23\mgr\user\
      Process all globals in c:\intersystems\test23\mgr\user\? No => yes
       
      Directory to restore [? for help]: c:\intersystems\test23\mgr\data\
       Redirect to Directory: c:\intersystems\test23\mgr\data\
       => --> c:\intersystems\test23\mgr\data\
      Process all globals in c:\intersystems\test23\mgr\data\? No => no
      
      Global ^Survey.LastName
      Global ^Survey.ID
      Global ^
      
      Directory to restore [? for help]:
      
      Processing globals from the following datasets:
       1. c:\intersystems\test23\mgr\user\   All Globals
       2. c:\intersystems\test23\mgr\data\   Selected Globals:
                ^Survey.LastName
                ^Survey.ID
      
      Specifications correct? Yes =>  Yes
      
      
      Note:

      If you are redirecting two or more databases to the same directory, you must make the same global selection — that is, either enter yes to process all globals, or no and then the same list of globals to process – for all of these databases. If you try to restore multiple databases to a single directory and the global selections are not all the same, the utility gives you the opportunity to either change your database redirection and global selections or cancel the operation.

  5. Specify the journal files to restore from, which should be from the same InterSystems IRIS instance as the source databases you are restoring, by specifying the correct journal history log (see Journal History Log).

    • Enter yes or <Enter> at the prompt to use the journal history log of the current InterSystems IRIS instance to identify the journal files to process. For example, if you entered yes at the Process all journaled globals in all directories? prompt at the start of the process, enter yes here to restore all databases in the current instance from the current instance’s journal files.

    • If you entered no at the Process all journaled globals in all directories? prompt and then specified databases in another InterSystems IRIS instance, enter no here to specify the journal history log and journal file directory path of that instance, or files copied from that instance, so that the databases can be restored from that instance’s journal files.

      Important:

      If you are using a journal history log from another InterSystems IRIS instance, you must use a copy of the file, not the actual log.

      For example,

      If you have a copy of the journal history log file from the 
      instance where the journal files were created, enter its full path below;
      otherwise, press ENTER and continue.
      Journal history log: c:\InterSystems\IRIS23_journals\journal.log
      
      Specify the location of the journal files to be processed
      Directory of the journal files:  c:\InterSystems\IRIS23_journals\journal\
      Directory of the journal files: 
      
      
  6. Specify the range of journal files you want to process. Bear in mind the following:

    • If InterSystems IRIS switched to multiple journal files since the restored backup, you must restore the journal files in order from the oldest to the most recent. For example, if you have three journal files to restore, 20230214.001, 20230214.002, and 20230215.001, you must restore them in the following order:

      20230214.001

      20230214.002

      20230215.001

    • When you back up with online backup, information about the oldest journal file required for transaction rollback during restore is displayed at the beginning of the third and final pass and stored in the backup log. See Backup and Restore for more information.

    Respond to the prompts as follows:

    • If you entered yes at the earlier prompt about whether this instance creates journal files, or answered no and then specified the journal history log and journal file location of another instance, you can enter the pathnames of the first and last journal files to process. You can also enter ? at either prompt to see a numbered list of the files in the specified location, then enter the numbers of the files, for example:

      Specify range of files to process
      Enter ? for a list of journal files to select the first and last files from
      First file to process:  ?
       
      1) c:\intersystems\iris2\mgr\journal\20230212.001
      2) c:\intersystems\iris2\mgr\journal\20230213.001
      3) c:\intersystems\iris2\mgr\journal\20230214.001
      4) c:\intersystems\iris2\mgr\journal\20230214.002
      5) c:\intersystems\iris2\mgr\journal\20230215.001
      6) c:\intersystems\iris2\mgr\journal\20230216.001
      7) c:\intersystems\iris2\mgr\journal\20230217.001
      8) c:\intersystems\iris2\mgr\journal\20230217.002
      
      First file to process:  5 c:\intersystems\iris2\mgr\journal\20230215.001
      Final file to process:
        c:\intersystems\20231316mar14\mgr\journal\20230217.002 => 
      
      Prompt for name of the next file to process? No => no
      
      
    • If you entered no at the earlier prompt about whether this instance creates journal files and did not specify a journal history log, processing continues with prompts that attempt to identify the specific journal files you want to process. For example:

      Journal history log:
      Specify range of files to process (names in YYYYMMDD.NNN format)
       
      from:     <20230212.001> [?] => 20230215.001
       
      through:  <20230217.001> [?] => 20230217.002
       
      Prompt for name of the next file to process? No => no
       
      Provide or confirm the following configuration settings:
       
      Journal File Prefix: [?] =>
       
      Files to dejournal will be looked for in:
           c:\intersystems\iris\mgr\journal\
      in addition to any directories you are going to specify below, UNLESS
      you enter a minus sign ('-' without quotes) at the prompt below,
      in which case ONLY directories given subsequently will be searched
      
      Directory to search: {return when done} -
           [Directory search list is emptied]
      Directory to search: {return when done} c:\intersystems\iris2\mgr\journal
      Directory to search: {return when done}
      Here is a list of directories in the order they will be searched for files:
           c:\intersystems\iris2\mgr\journal\
      
      
  7. Process the journal files:

    Prompt for name of the next file to process? No => No 
    The following actions will be performed if you answer YES below:
     
    * Listing journal files in the order they will be processed
    * Checking for any missing journal file on the list ("a broken chain")
     
    The basic assumption is that the files to be processed are all
    currently accessible. If that is not the case, e.g., if you plan to
    load journal files from tapes on demand, you should answer NO below.
    Check for missing journal files? Yes => Yes
    
    
  8. If one or more journal files within the range you specified are missing, you are given the opportunity to abort the operation. If you do not, or if no files are missing, the process proceeds with an opportunity to check journal integrity before starting the restore:

    Journal files in the order they will be processed:
    1. c:\intersystems\iris2\mgr\journal\20230215.001
    2. c:\intersystems\iris2\mgr\journal\20230216.001
    3. c:\intersystems\iris2\mgr\journal\20230217.001
    4. c:\intersystems\iris2\mgr\journal\20230217.002
    
    While the actual journal restore will detect a journal integrity problem
    when running into it, you have the option to check the integrity now
    before performing the journal restore. The integrity checker works by
    scanning journal files, which may take a while depending on file sizes.
    Check journal integrity? No => No
    
    
  9. If the current journal file is included in the restore, you must switch journaling to another file, and are prompted to do so:

    The journal restore includes the current journal file.
    You cannot do that unless you stop journaling or switch
         journaling to another file.
    Do you want to switch journaling? Yes => yes
    Journaling switched to c:\intersystems\iris2\mgr\journal\20230217.003
    
  10. Next, choose whether to disable journaling of updates during the restore to make the operation faster.

    You may disable journaling of updates for faster restore for all
    databases other than mirrored databases. 
    Do you want to disable journaling the updates? Yes => yes
    Updates will NOT be journaled
    
    
    Important:

    If you do not disable journaling of updates during the restore, parallel dejournaling will not be used to increase performance, as described at the beginning of this section.

    If journaling is disabled but database updates continue, you cannot use the last good journals to do a manual restore unless you can assure either of the following:

    • You know exactly what will be updated and can control what is restored to the satisfaction of the application.

    • You have restored the database(s) involved from the last backup and accept that after applying the journals you will have lost the data written when journaling was off.

    InterSystems recommends that you run the following commands after completing the journal restore under these circumstances to verify that the object IDs are not out of sync; only IDs that are found to be out of sync are reported in the array, errors:

    Do CheckIDCounters^%apiOBJ(.errors)
    zwrite errors
    
  11. After confirming or changing the default behavior on error for the restore job, confirm the restore to begin:

    Before we job off restore daemons, you may tailor the behavior of a
    restore daemon in certain events by choosing from the options below:
     
         DEFAULT:    Continue despite database-related problems (e.g., a target
         database cannot be mounted, error applying an update, etc.), skipping
         updates to that database. Affected database(s) may not be self-consistent
         and will need to be recovered separately
     
         ALTERNATE:  Abort if an update would have to be skipped due to a
         database-related problem (e.g., a target database cannot be mounted,
         error applying an update, etc.). Databases will be left in a
         self-consistent state as of the record that caused the restore to be
         aborted. Parallel dejournaling will be disabled with this setting  
    
         DEFAULT:    Abort if an update would have to be skipped due to a
         journal-related problem (e.g., journal corruption, some cases of missing
         journal files, etc.)
     
         ALTERNATE:  Continue despite journal-related problems (e.g., journal
         corruption, some missing journal files, etc.), skipping affected updates
     
    Would you like to change the default actions? No => No
      
    Start the restore? Yes =>
    
    Important:

    If you choose to abort due to database-related problems, parallel dejournaling will not be used to increase performance, as described at the beginning of this section.

  12. The progress of the journal restore is displayed at intervals, and when the job is complete a list of databases updated by the restore is displayed:

    c:\MyIRIS1\mgr\journal\20230406.001
     35.73%  70.61% 100.00%
    c:\MyIRIS1\mgr\journal\20230406.002
     35.73%  70.61% 100.00%
    c:\MyIRIS1\mgr\journal\20230406.003
     100.00%
    [Journal restore completed at 20230407 02:25:31]
     
    The following databases have been updated:
    
    1. c:\MyIRIS1\mgr\source22\
    2. c:\MyIRIS1\mgr\source23\
    3. c:\MyIRIS1\mgr\irislocaldata\
    4. c:\MyIRIS1\mgr\irislib\
    5. c:\MyIRIS1\mgr\iristemp\
    
    The following databases have been skipped:
    
    1. /bench/user/InterSystems/IRIS/162/
    2. /scratch1/user/InterSystems/IRIS/p750.162/mgr/
    3. /scratch1/user/InterSystems/IRIS/p750.162/mgr/irislocaldata/
    4. /scratch1/user/InterSystems/IRIS/p750.162/mgr/irislib/
    5. /scratch1/user/InterSystems/IRIS/p750.162/mgr/iristemp/
    6. /scratch1/user/InterSystems/IRIS/p750.162/mgr/user/
    
  13. If there are any open transactions, you will be prompted at the end of the restore to roll back the incomplete transactions:

    Rollback incomplete transactions? No =>
    

    This prompt will only appear if there are any incomplete transactions in the any of the databases being restored. See Rolling Back Incomplete Transactions for more information.

Rolling Back Incomplete Transactions

Restoring the journal will also prompt you to roll back incomplete transactions if there are any. Ensure that users have completed all transactions so that the restore does not attempt to roll back active processes.

To ensure that transactions are all complete before you restore your backup and clear the journal file, InterSystems strongly recommends the following:

  • If you need to roll back transactions for your own process, the process must halt or use the TROLLBACK command.

  • If you need to roll back transactions system-wide, shut down InterSystems IRIS and restart it to ensure that no users are on the system.

Restoring Mirror Journal Files

You can restore mirror journal files to either mirrored or nonmirrored databases. If you are restoring to a mirrored database, see step 2 of the procedure in Restore Globals From Journal Files Using ^JRNRESTO and Restore Journal to Mirrored Database Using MirrorCatchup^JRNRESTO. If you are restoring to a nonmirrored database, see step 4 of the procedure in Restore Globals From Journal Files Using ^JRNRESTO.

Filter Journal Records Using ^ZJRNFILT

InterSystems provides a journal filter mechanism to manipulate the journal file. The journal filter program is a user-written routine called ^ZJRNFILT whose format is shown below. This is called by the InterSystems IRIS journal restore program, ^JRNRESTO, and ensures that only selected records are restored. Create the ^ZJRNFILT routine using the following format:

ZJRNFILT(jidsys,dir,glo,type,restmode,addr,time)

Argument Type Description
jidsys input

Two components separated by a comma: jid (job ID) and remsysid (for ECP only).

Pass jidsys to the %SYS.Journal.Record.GetRealPIDSYSinFilterOpens in a new tab method (as shown in the “^ZJRNFILT Examples” below) to identify the PID (process ID) that generated the journal.

dir input Full pathname of the directory containing the IRIS.DAT file to be restored, as specified in the journal record.
glo input Global in journal record.
type input Command type in journal record (S for Set, K for Kill).
addr input Address of the journal record.
time input Time stamp of the record (in $horolog format). This is the time the journal buffer is created, not when the Set or Kill operation occurs, so it represents the earliest this particular operation could have happened.
restmode output

0 - do not restore record.

1 - restore record.

^ZJRNFILT Considerations

Consider the following when using ^ZJRNFILT:

  • If the startup routine (^STU) calls ^JRNRESTO, it does not call the filter routine under any circumstances.

  • Journal restore only calls the journal filter (^ZJRNFILT) if it exists. If it does exist, the restore procedure prompts you to confirm the use of the filter in the restore process.

  • If you answer yes to use the journal filter, for every record in the journal file to restore, the routine calls the journal filter ^ZJRNFILT with the indicated input arguments to determine whether to restore the current record.

  • You can use any logic in your ^ZJRNFILT routine to determine whether or not to restore the record. Return confirmation through the output restmode argument.

  • If you are using the directory name, dir, in the ^ZJRNFILT routine logic, specify the full directory pathname.

  • The entire global reference is passed to ^ZJRNFILT for use in program logic.

  • When the journal restore process completes, it prompts you to confirm whether to rename the ^ZJRNFILT routine or delete it. If you choose to rename the filter, the utility renames it ^XJRNFILT and deletes the original ^ZJRNFILT.

  • The restore process aborts with an appropriate error message if any errors occur in the ^ZJRNFILT routine.

^ZJRNFILT Examples

Two globals, ^ABC and ^XYZ, are journaled. While journaling is turned on, the following code is executed, and the journal file records the Set and Kill operations for these globals:

 For I=1:1:500 Set ^ABC(I)=""
 For I=1:1:500 Set ^XYZ(I)=""
 For I=1:1:100 Kill ^ABC(I)
  1. To restore all records for ^ABC only, the ^ZJRNFILT routine looks like this:

    ZJRNFILT(jidsys,dir,glo,type,restmode,addr,time)    /*Filter*/
     Set restmode=1                                  /*Return 1 for restore*/
     If glo["XYZ" Set restmode=0                     /*except when it is ^XYZ*/
     Quit
     ; 
  2. To restore all records except the kill on ^ABC, the ^ZJRNFILT routine looks like this:

    ZJRNFILT(jidsys,dir,glo,type,restmode,addr,time)    /*Filter*/
     Set restmode=1                                  /*Return 1 for restore*/
     If glo["^ABC",type="K" Set restmode=0           /*except if a kill on ^ABC*/
     Quit
     ;
    
  3. In some cases (for example, when the jid is a PID or on a mirror member), remsysid is not the actual ECP system ID. In these cases, use the %SYS.Journal.Record.GetRealPIDSYSinFilterOpens in a new tab method to return the real ECP system ID as well as the real PID.

    To pull the real PID (and ECP system PID) in a filter, the ^ZJRNFILT routine looks like this:

    ZJRNFILT(jidsys,dir,glo,type,restmode,addr,time) ;
     SET restmode=0 ;test only
     SET pid=##class(%SYS.Journal.Record).GetRealPIDSYSinFilter(jidsys,.ecpsysid)
     DO ##class(%SYS.System).WriteToConsoleLog($SELECT(pid="":"jid="_+jidsys,1:"pid="_pid)_",ecpsysid="_ecpsysid)
     QUIT
  4. To restore all records after a specific time, the ^ZJRNFILT routine looks like this:

    ZJRNFILT(jidsys,dir,glo,type,restmode,addr,time) ;
     New zdh 
     Set zdh=$zdatetimeh("08/14/2015 14:18:31") ;in $H format as variable 'time'
     If time>zdh Set restmode=1 Quit
     If time<zdh Set restmode=0 Quit
     If $p(time,",",2)<$p(zdh,",",2) {
      Set restmode=0
     } Else {
        Set restmode=1
     }
     Quit
    

Display Journal Records Using ^JRNDUMP

To display the records in the journal file, enter 5 at the Option prompt of the ^JOURNAL menu or run ^JRNDUMP as shown in the following example:

  1. Run the ^JRNDUMP utility from the system manager namespace by entering:

    %SYS>DO ^JRNDUMP
    
       Journal                 Directory & prefix
     
       20231113.001            C:\MyIRIS\Mgr\Journal\
       20231113.002 [JRNSTART] C:\MyIRIS\mgr\journal\
       20231113.003            C:\MyIRIS\mgr\journal\
       20231113.004            C:\MyIRIS\mgr\journal\
       20231114.001            C:\MyIRIS\mgr\journal\
       20231115.001            C:\MyIRIS\mgr\journal\
       20231115.002            C:\MyIRIS\mgr\journal\
       20231115.003            C:\MyIRIS\mgr\journal\
    >  20231115.004            C:\MyIRIS\mgr\journal\
     
    
    
  2. The routine displays a list of journal files. A greater-than sign (>) appears to the left of the currently selected file followed by a prompt:

    Pg(D)n,Pg(U)p,(N)ext,(P)rev,(G)oto,(E)xamine,(Q)uit =>
    
    

    Use these options to navigate to the journal file you wish to locate:

    • If the instance is a mirror member, enter M to limit the list to mirror journal files only. (For information about mirror journal files, see Journal Files and Journal History Log and Mirror Synchronization.)

    • Enter D or U to page through the list of journal files.

    • Enter N or P to move the > to the desired journal file.

    • Enter G to directly enter the full pathname of the journal file to display.

    • Enter E to display the contents of the selected journal file.

    • Enter I to display information about the selected journal file and, optionally, a list of databases from the journal.

    • Enter Q or <Enter> to quit the routine.

  3. When you enter I, when you accept the currently selected journal file or specify a different one, information like the following is displayed:

    Journal: C:\MyIRIS\mgr\journal\20231113.003
    File GUID: 97734819-CA75-4CB1-9C3E-74D294784D23
    Max Size: 1073741824
    Time Created: 2023-11-13 10:44:52
    File Count: 22
    Min Trans: 22,3497948
    Prev File: C:\MyIRIS\mgr\journal\20231113.002
    Prev File GUID: 8C5D3476-F12C-4258-BF6C-7423876653A4
    Prev File End: 0
    Next File: C:\MyIRIS\mgr\journal\20231113.004
    Next File GUID: 4F4D20B1-D38C-473E-8CF0-4D04C6AF90B0
    
    (D)atabases,(Q)uit =>
    

    Min Trans is the file count and offset of the minimal transaction position, that is, any open transaction must have started at or later than that point.

    If the selected file is a mirror journal file, addition information is displayed.

    Entering Q at the prompt at the bottom returns you to the journal file list. Enter D to display database information like the following:

    Journal: C:\MyIRIS\mgr\journal\20231113.003
      sfn  Directory or Mirror DB Name
    ==========================================================================
        0  C:\MyIRIS\mgr\
        1  C:\MyIRIS\mgr\irislib\
        2  C:\MyIRIS\mgr\iristemp\
        3  :mirror:MIR:MIRTEST
        5  C:\MyIRIS\mgr\irislocaldata\
        6  C:\MyIRIS\mgr\user\
     
    (P)rev,(N)ext,(Q)uit =>
    

    Enter Q to return to the journal file information display.

  4. After you enter G or E, the utility displays the journal file name and begins listing the contents of the file by offset address. For example:

    Journal: C:\MyIRIS\mgr\journal\20230330.002
       Address   Proc ID Op Directory        Global & Value
    ===============================================================================
        131088      2980 S  C:\MyIRIS\mgr\  SYS("shdwcli","doctest","remend") = 1+
        131156      2980 S  C:\MyIRIS\mgr\  SYS("shdwcli","doctest","end") = 1013+
        131220      2980 S  C:\MyIRIS\mgr\  SYS("shdwcli","doctest","jrnend") = 1+
    ...
    
    
  5. At the bottom of the current listing page is information about the journal file and another prompt:

    Last record:     573004;   Max size: 1073741824
    (N)ext,(P)rev,(G)oto,(F)ind,(E)xamine,(Q)uit =>
    
    

    Use these options to navigate to the journal record you wish to display:

    • Enter N or P to display the next or previous page of addresses.

    • Enter G to move the display to a particular address.

    • Enter F to search for a particular string within the journal file.

    • Enter E to enter the address of a journal record and display its contents.

    • Enter Q to return to the list of journal files.

  6. After entering E or G, enter an address at the prompt. The E option displays the contents of the journal record at or near the address you entered; the G option displays the page of journal records starting at that location.

    For either option, the utility locates the record that is the closest to the offset address you specify; it does not need to be a valid address of a journal record. Also, you may enter 0 (zero) to go to the beginning of the journal file, or enter -1 to go to the end of the journal file.

  7. You may browse through a display of the journal records using N or P to display the next or previous journal record contents, respectively. When you are finished displaying records, enter Q at the prompt to return to the list of journal records.

There are different types of journal records:

  • The journal header is 8192 bytes long. It appears once at the start of every journal file. The ^JRNDUMP utility does not display the journal header record.

  • Journal data records.

  • Journal markers

The following is a sample journal file data record as displayed by ^JRNDUMP. The example shows how a Set command is recorded. The new value is recorded, but not the old value, because the Set occurred outside a transaction:

Journal: C:\MyIRIS\mgr\journal\20230119.004
 
Address:                 233028
Type:                    Set
In transaction:          No
Process ID:              4836
ECP system ID:           0
Time stamp:              60284,53240
Collation sequence:      5
Prev address:            232984
Next address:            0
 
Global:    ^["^^C:\MyIRIS\mgr\"]ABC
New Value: 2
 
 
(N)ext,(P)rev,(Q)uit =>

In a transaction, the old value is also recorded, to allow transaction rollback, as seen in this second example:

Journal: C:\MyIRIS\mgr\journal\20231115.004
 
Address:                 204292
Type:                    Set
In transaction:          Yes
Process ID:              458772
ECP system ID:           0
Time stamp:              60584,52579 - 11/15/2023 14:36:19
Collation sequence:      5
Prev address:            204224
Next address:            204372 

Global:    ^["^^C:\MyIRIS\mgr\"]ABC
New Value: 5
Old Value: 2
 
 
(N)ext,(P)rev,(Q)uit =>

The following is an example of a journal marker record created by an incremental backup:

Journal: C:\MyIRIS\mgr\journal\20231115.004
 
Address:                 210848
Type:                    JrnMark
Marker ID:               -1
Marker text:             NOV 15 2023;03:14PM;Incremental
Marker seq number:       1
Prev marker address:     0
Time stamp:              60584,52579 - 11/15/2023 14:36:19
Prev address:            210744
Next address:            210940
 
 
 
(N)ext,(P)rev,(Q)uit =>

The following table describes each field in the journal data record.

Journal Data Record Fields Displayed by ^JRNDUMP
Field Description
Address Location of this record in number of bytes from beginning of file. This is the only field where you enter a value to select a record.
Type The type of operation recorded in this journal record entry. See the Journal File Operations table for possible types.
In transaction Whether or not the update occurred in a transaction.
Process ID Process ID number for the process issuing the command.
ECP system ID ECP system ID number (0 if a local process).
Time stamp Creation time of the journal buffer, in $HOROLOG and human-readable format. This is not the time the Set or Kill operation occurs, so it represents the earliest this particular operation could have happened.
Collation sequence Collation sequence of the global being updated.
Prev address Location of previous record (0 indicates this is the first record).
Next address Location of next record (0 indicates this is the last record).
Cluster sequence # Sequencing for globals in cluster-mounted databases. During cluster failover, journal entries from different nodes are updated in order of this cluster time sequencing.
Mirror Database Name If a mirror journal file, the mirror name for the database on which the operation occurred.
Global Extended reference of global being updated.
New Value For a Set operation, the value assigned to the global.
Old Value For a Set or Kill operation in a transaction, the value that was in the global before the operation.

The following table lists and describes the journal operations displayed in the Op column of a ^JRNDUMP journal file display and the Type field of a ^JRNDUMP journal record listing. For example, in the previous example of a journal file display, S in the Op column represents a JRNSET operation, while in the examples of journal record displays, Set appears in the Type field to indicate a JRNSET operation. Note that the Type column of the journal record display in the management portal (see View Journal Files) differs for some operations from the Type field of the ^JRNDUMP listing; for example, a JRNSET operation is indicated by RemoteSET in the portal and by NSet in ^JRNDUMP output. These differences are shown in the table.

The table also shows the codes that can be specified to filter journal records by operation when using the SELECT^JRNDUMP function.

Journal File Operations
Operation Description Op in file listing Type in ^JRNDUMP record listing Type in Management Portal record listing Numeric SELECT code Alpha SELECT code
JRNSET set a node, local S1 Set

SET

6 s

JRNNSET

set a node, remote

S1 NSet

RemoteSET

10 s

JRNMIRSET

internal mirror operation2

S1

Mirror Set

MirrorSET

19

s

JRNBITSET

set a specified bit position in a node

b1

BitSet

BitSET

14

bs

JRNKILL

kill a node, local

K1

KillNode

KILL

7

k

JRNNKILL

kill a node, remote

K1

NKill

RemoteKILL

11

k

JRNKILLDES

kill a descendant node

k1

KillDesc

KILLdes

8

k

JRNMIRKILL

internal mirror operations2

k1

Mirror Kill

MirrorKILL

20

k

JRNZKILL

kill a node without killing subordinate nodes, local

k1

ZKill

ZKILL

9

zk

JRNNZKILL

kill a node without killing subordinate nodes, remote

k1

NZKill

RemoteZKILL

12

zk

JRNBEGTRANS

begin a transaction

BT

BeginTrans

BeginTrans

4

--

JRNTBEGINLEVEL

begin transaction level

BTL

BeginTrans with Level

BeginTrans with level

16

--

JRNCOMMIT

commit a transaction

CT

CommitTrans

CommitTrans

5

--

JRNTCOMMITLEVEL

commit isolated transaction level

CTL

CommitTrans with Level

CommitTrans with level

18

--

JRNTCOMMITPENDLEVEL

commit pending transaction level

PTL

CommitTrans Pending with Level

CommitTrans Pending with level

17

--

JRNMARK

journal marker

M

JrnMark

Marker

13

--

JRNBIGNET

ECP networking

NN

NetReq

netsyn

15

--

JRNTROLEVEL

roll back a transaction

RB

Rollback

Rollback

21

--

1 T is appended when the operation occurs within a transaction, for example ST for a Set operation within a transaction or kT for a ZKill operation within a transaction.

2 Operation is ignored during journal restore.

Select Journal Records to Dump

The function SELECT^JRNDUMP lets you display any or all of the records in the journal file. InterSystems IRIS dumps selected records from the journal file, starting from the beginning of the file, based on the arguments passed to the function.

The syntax to use the SELECT entry point of the ^JRNDUMP utility is as follows:

SELECT^JRNDUMP(%jfile,%pid,%dir,%glo,%gloall,%operation,%remsysid)
Argument Description
%jfile Journal file name. Default is the current journal file. You must specify the fully qualified path of the journal file.
%pid

Process ID in the journal record. Default is any process.

%dir Directory in the journal record. Default is any directory.
%glo Global reference in the journal record. Default is any global.
%gloall Global indicator whether to list entries related to all global nodes containing the name represented by glo: 0 — Exact match of global reference with the name specified in glo, 1 — Partial match; all records with a global reference that contains the name specified in glo. Default is 0.
%operation Operation type of the journal record. Default is any operation. Use the numeric or alphabetic codes listed in the Journal File Operations table.
%remsysid ECP system ID of journal record. Default is any system.1

1 If %pid is specified, then %remsysid defaults to local system (0); otherwise, it defaults to any system, the same as if it is specified as 0. That is, you cannot select journal entries only from the local system.

You may pass the null string for any argument, in which case the routine uses the defaults.

As with other terminal functions, you can use the Device: prompt to direct the output of SELECT^JRNDUMP to a device other than the terminal, or to a file. (See Introduction to I/O for information on user device selection.) If you direct the output to a file, you are prompted for parameters. You must include R and W when writing to a file; if it is an existing file, include A to append output to the existing content rather than overwriting it; if a new file, you must include N. Enter ? at the Parameters? prompt to display all possible choices.

Note:

If the file you are overwriting is longer than the current output, the excess lines from the original file may not be removed from the updated file.

SELECT^JRNDUMP Examples

The following examples show different ways to select specific journal records.

To select all records in a journal file with the process ID 1203 and send the output to a new file called JRNDUMP.OUT:

%SYS>Do SELECT^JRNDUMP("C:\MyIRIS\mgr\journal\120020507.009","1203")

Device: SYS$LOGIN:JRNDUMP.OUT   
Parameters: "RWN"=>

To select all records in the journal file that contain the global reference ^ABC:

DO SELECT^JRNDUMP("C:\MyIRIS\mgr\journal\20050327.001","","","^ABC",1)

To select only records that have an exact match to the global reference ^ABC:

DO SELECT^JRNDUMP("C:\MyIRIS\mgr\journal\20050327.001","","","^ABC",0)

Note:

Records that are not an exact match, such as ^ABC(1) or ^ABC(100), are not selected.

To select only records for local Set operations of global ^ABC:

DO SELECT^JRNDUMP("C:\MyIRIS\mgr\journal\20050327.001","","","^ABC","","6")

To select only records for local and remote Set operations of global ^ABC:

DO SELECT^JRNDUMP("C:\MyIRIS\mgr\journal\20050327.001","","","^ABC","","s")

Purge Journal Files Using PURGE^JOURNAL

To purge files, use the PURGE^JOURNAL routine or enter 6 at the Option prompt of the ^JOURNAL menu, as shown in the following examples.

Example of running PURGE^JOURNAL directly:

set $namespace="%SYS"
%SYS>Do PURGE^JOURNAL

Example of starting journaling from the ^JOURNAL menu:

%SYS>Do ^JOURNAL
 
 ...
 6) Purge Journal Files (PURGE^JOURNAL)
 ...
Option? 6

1) Purge any journal NOT required for transaction rollback or crash recovery
2) Purge journals based on existing criteria (2 days or 2 backups)
 
Option?

The routine reports on the action taken in response to the option you specify. For example:

Option? 1

The following files have been purged (listed from latest to oldest):

3. c:\intersystems\iris\mgr\journal\20230714.001
2. c:\intersystems\iris\mgr\journal\20230713.001
1. c:\intersystems\iris\mgr\journal\20230710.003

If no files are purged, the following message is displayed:


None purged

Update Journal Settings Using ^JRNOPTS

As an alternative to using the Journal Settings page of the Management Portal, you can update the basic journal configuration settings using the ^JRNOPTS routine or by entering 7 at the Option prompt of the ^JOURNAL menu. To change the setting, type the new value at the prompt and press Enter. For example:

SYS>Do ^JRNOPTS
 
1) Primary Journal Directory: C:\MyIRIS\Mgr\Journal\
2) Alternate Journal Directory: D:\irissys\altjournal\ 
3) Journal File Size Limit (MB) 1024
4) Journal File Prefix: 
5) Journal Purge Options: 2 days OR 2 backups, whichever comes first
6) Compress Journal files: Yes

Entering a question mark (?) displays Help. For example:

Journal File Prefix: ?
  Enter an alphanumeric string ('_' allowed) or . to reset prefix to null

If you change any of the settings, then press Enter at a Change Property? prompt, you are given the option to activate the changes:

Change Property?
Save and activate changes? Yes =>
*** Journal options updated.

If you do not change any settings, you see the following message:

  
*** Nothing changed

Journal Encryption Using ENCRYPT^JOURNAL

For information on option 8) Activate or Deactivate Journal Encryption (ENCRYPT^JOURNAL), see Configure Encryption Startup Settings, which describes details on journal file encryption.

Note:

When both journal encryption and journal compression (see Configuring Journal Settings) are active, journal data is always compressed before being encrypted; encrypted data is never compressed.

Display Journal Status Using Status^JOURNAL

Choosing option 9) Display Journal status displays a concise overview of journal status information including the following:

  • Current journal directory and its remaining space

  • Alternate journal directory (if different) and its remaining space

  • Current journal file, its maximum size, and space used

  • Journaling state, which can be one of the following:

    • Enabled

    • Disabled (stopped)

    • Disabled due to I/O error (suspended)

    • Frozen due to I/O error

    • Journal switch in progress (paused)

    Though suspended and frozen due to I/O error are the same journal state, the system takes different action; when frozen, it discards journal data.

  • If applicable, the process IDs of any process running ^JRNSTART, ^JRNSTOP, or ^JRNSWTCH

For example:

%SYS>Do ^JOURNAL
 
 ...
 9) Display Journal status (Status^JOURNAL)
 ...
Option? 9
 
Current journal directory:  C:\MyIRIS\Mgr\Journal\
Current journal directory free space (KB):  53503904
Alternate journal directory:  C:\MyIRIS\Mgr\
Alternate journal directory free space (KB): 53503904
Current journal file:  C:\MyIRIS\mgr\journal\20231129.001
Current journal file maximum size:    1073741824
Current journal file space used:         1979276
Journaling is enabled.

Manage Transaction Rollback Using Manage^JRNROLL

InterSystems IRIS provides the ^JRNROLL utility to roll back partially completed transactions for records in the journal; use the Manage entry point (Manage^JRNROLL) when transaction rollbacks are pending or in progress at system startup or primary mirror member startup.

To start managing transaction rollback, run Manage^JRNROLL or enter 11 at the Option prompt of the ^JOURNAL menu, as shown in the following example:

%SYS>Do ^JOURNAL
 
 ...
 11) Manage pending or in progress transaction rollback (Manage^JRNROLL)
 ...
Option? 11

Choosing option 11) Manage pending or in progress transaction rollback (Manage^JRNROLL) displays a message similar to the following:

Transaction rollback is pending or in progress 
Do you wish to run Manage^JRNROLL? Yes => Yes 
Rollback operations currently in progress 
     ID   Phase     MB Remaining   Current Open Transaction Count 
     1    scan      307            2 
          Rollback at system startup at 11/29/2023 15:54:35 (578MB) 
          20231129.004 has 2 open transaction(s) starting at offset 11303 
          2 file(s) remaining to process   

1) Restart pending rollback 
2) Interrupt transaction rollback 
3) Redisplay rollback information  
4) Discard pending rollback

Option?  

This option displays the state of transaction rollbacks, including what phase (scanning or rollback) it is in, the amount of data (MBs) remaining to be processed, the number of open transactions it has found, etc.

In addition, it lists suboptions that let you manage the listed transaction rollbacks. For example, you can interrupt the operation, in which case it is queued as a “pending” operation; then you can restart pending rollbacks.

Caution:

Option 4) Discard pending rollback is not reversible; do not use it unless you are certain you want to permanently discard the pending rollback.

Note:

On a mirror, transaction rollback is executed twice: once for nonmirrored databases (at system startup); then for mirrored databases (when a system becomes the primary mirror member). As a result, when starting the primary mirror member, it may be necessary to interrupt the rollback twice, resulting in two pending operations. Restarting the pending operations performs the non-mirror and mirror rollbacks separately.

During rollback, messages are written to the messages log (messages.log) every 10% of the way through (more or less) indicating how much space is left to process and how many open transactions are listed.

When journal files are purged, files required for pending transaction rollback are retained (for example, if they would otherwise have been deleted).

Restore Journal to Mirrored Database Using MirrorCatchup^JRNRESTO

You can restore mirror journal files to mirrored databases by entering 12 at the Option prompt of the ^JOURNAL menu, or by answering yes at the Catch-up mirrored databases? prompt when using Option 4, Restore Globals From Journal (^JRNRESTO). For example:

%SYS>Do ^JOURNAL

 ...
 12) Journal catch-up for mirrored databases (MirrorCatchup^JRNRESTO) 
 ...
Option? Option? 12

Specify the list of mirrored databases you want to catch-up.
Enter database, * for all, ? for a list or to end list? *
Enter database or to end list?
Starting catch-up for the following mirrored database(s):
     sfn #6: c:\intersystems\iris\mgr\mirrordb3\
Catch-up succeeded. 

To catch up mirrored databases, journaling need not be running, but it must have been started at least once to ensure that the current journal directory is available from memory.

Recover from Startup Errors Using ^STURECOV

During the InterSystems IRIS startup procedure if the journal or transaction restore process encounters errors, such as <FILEFULL> or <DATABASE>, the procedure logs the errors in the messages log (messages.log) and starts the system in single-user mode.

InterSystems IRIS provides a utility, ^STURECOV, to help you recover from the errors and start InterSystems IRIS in multiuser mode. The routine has several options which you can use to retry the failed operation and bring the system up, or ignore the errors and bring the system up. The journal restore phase tries to do as much work as possible before it aborts. If a database triggers more than three errors, it aborts the recovery of that database and leaves the database dismounted.

Note:

The ^STURECOV utility does not work on a mirror member on which transaction rollback is pending or in progress because the system does not activate a mirrored database read/write until transaction rollback has been completed. In this case, InterSystems IRIS enables you to run the Manage^JRNROLL routine, which provides a way to force the system to come up and store transaction rollback information which can be used to roll back transactions after the system is up and running. For more information, see Manage Transaction Rollback Using Manage^JRNROLL.

During transaction rollback, the first error in a database causes the rollback process to skip that database in the future. The process does not fully replay transactions that reference that database; it stores them for rollback during the recovery process.

When InterSystems IRIS encounters a problem during the dejournaling phase of startup it generates a series of messages log messages similar to the following:

08/10-11:19:47:024 ( 2240) System Initialized. 
08/10-11:19:47:054 ( 2256) Write daemon started. 
08/10-11:19:48:316 ( 1836) Performing Journal Recovery 
08/10-11:19:49:417 ( 1836) Error in JRNRESTB: <DATABASE>restore+49^JRNRESTB 
     C:\MyIRIS\mgr\journal\20230810.004 addr=977220 
     ^["^^C:\MyIRIS\mgr\jo1666\"]test(4,3,28) 
08/10-11:19:49:427 ( 1836) Error in JRNRESTB: <DATABASE>restore+49^JRNRESTB 
     C:\MyIRIS\mgr\journal\20230810.004 addr=977268 
     ^["^^C:\MyIRIS\mgr\test\"]test(4,3,27) 
08/10-11:19:49:437 ( 1836) Error in JRNRESTB: <DATABASE>restore+49^JRNRESTB 
     C:\MyIRIS\mgr\journal\20230810.004 addr=977316 
     ^["^^C:\MyIRIS\mgr\test\"]test(4,3,26) 
08/10-11:19:49:447 ( 1836) Error in JRNRESTB: <DATABASE>restore+42^JRNRESTB 
     C:\MyIRIS\mgr\journal\20230810.004 addr=977748 
     ^["^^C:\MyIRIS\mgr\test\"]test(4,2,70) 
08/10-11:19:50:459 ( 1836) Too many errors restoring to C:\MyIRIS\mgr\test\. 
 Dismounting and skipping subsequent records 
08/10-11:19:50:539 ( 1836) 4 errors during journal restore, 
see console.log file for details. 
Startup aborted, entering single user mode. 
 

If the errors are from transaction rollback, then the output looks similar to this:

08/11-08:55:08:732 ( 428) System Initialized. 
08/11-08:55:08:752 ( 1512) Write daemon started. 
08/11-08:55:10:444 ( 2224) Performing Journal Recovery 
08/11-08:55:11:165 ( 2224) Performing Transaction Rollback 
08/11-08:55:11:736 ( 2224) Max Journal Size: 1073741824 
08/11-08:55:11:746 ( 2224) START: C:\MyIRIS\mgr\journal\20230811.011 
08/11-08:55:12:487 ( 2224) Journaling selected globals to 
     C:\MyIRIS\mgr\journal\20230811.011 started. 
08/11-08:55:12:487 ( 2224) Rolling back transactions ... 
08/11-08:55:12:798 ( 2224) Error in %ROLLBACK: <DATABASE>set+2^%ROLLBACK 
     C:\MyIRIS\mgr\journal\20230811.010 addr=984744 
     ^["^^C:\MyIRIS\mgr\test\"]test(4,1,80) 
08/11-08:55:12:798 ( 2224) Rollback of transaction for process id #2148 
 aborted at offset 984744 in C:\MyIRIS\mgr\journal\20230811.010. 
08/11-08:55:13:809 ( 2224) C:\MyIRIS\mgr\test\ dismounted - 
      Subsequent records will not be restored 
08/11-08:55:13:809 ( 2224) Rollback of transaction for process id #924 
 aborted at offset 983464 in C:\MyIRIS\mgr\journal\20230811.010. 
08/11-08:55:14:089 ( 2224) STOP: C:\MyIRIS\mgr\journal\20230811.011 
08/11-08:55:14:180 ( 2224) 1 errors during journal rollback, 
see console.log file for details. 
Startup aborted, entering single user mode. 
 

Both output listings end with instructions such as:

Enter IRIS with 
     C:\MyIRIS\bin\irisdb -sC:\MyIRIS\mgr -B 
and D ^STURECOV for help recovering from the errors. 

When InterSystems IRIS cannot start properly, it starts in single-user mode. While in this mode, execute the commands indicated by these instructions to enter InterSystems IRIS (see Administrator Terminal Session).

You are now in the manager’s namespace and can run the startup recovery routine, ^STURECOV:

Do ^STURECOV

The ^STURECOV journal recovery menu appears as follows:


Journal recovery options 
-------------------------------------------------------------- 
1) Display the list of errors from startup 
2) Run the journal restore again 
3) Bring down the system prior to a normal startup 
4) Dismount a database 
5) Mount a database 
6) Database Repair Utility 
7) Check Database Integrity 
8) Reset system so journal is not restored at startup 
9) Display instructions on how to shut down the system 
10) Display Journaling Menu (^JOURNAL)
-------------------------------------------------------------- 
H) Display Help
E) Exit this utility
--------------------------------------------------------------
 
Enter choice (1-10) or [Q]uit/[H]elp?

Only UNIX®/Linux systems contain option 9 on the menu.

Before starting the system in multiuser mode, correct the errors that prevented the journal restore or transaction rollback from completing. You have several options regarding what to do:

  • Option 1 — The journal restore and transaction rollback procedure tries to save the list of errors in the ^%SYS() global. This is not always possible depending on what is wrong with the system. If this information is available, this option displays the errors.

  • Option 2 — This option performs the same journal restore and transaction rollback which was performed when the system was started. The amount of data is small so it should not be necessary to try and restart from where the error occurred.

  • Option 3 — When you are satisfied that the system is ready for use, use this option to bring the instance down prior to restarting it in a normal fashion.

  • Option 4 — This option lets you dismount a database. Generally, use this option if you want to let users back on a system but you want to prevent them from accessing a database which still has problems (^DISMOUNT utility).

  • Option 5 — This option lets you mount a database (^MOUNT utility).

  • Option 6 — This option lets you edit the database structure (^REPAIR utility).

  • Option 7 — This option lets you validate the database structure (^INTEGRIT utility).

  • Option 8 — This updates the system so that it does not attempt journal restore or transaction rollback at startup. This applies only to the next time the startup process is run. Use this in situations where you cannot get journal recovery to complete and you need to allow users back on the system. Consider dismounting the databases which have not been recovered. This operation is not reversible. You can perform journal restore manually using the ^JRNRESTO utility.

  • Option 9 — It is not possible to shut down the system from this utility, but this option displays instructions on how to shut the system down from the UNIX® command line.

  • Option 10 — This option brings up the journaling menu which allows you to browse and restore journal files. There are options which start and stop journaling but these are not generally of interest when resolving problems with journaling at startup.

Take whatever corrective action is necessary to resolve the problem. This may involve using the ^DATABASE routine to extend the maximum size of the database, or it may require freeing space on the file system or using the ^INTEGRIT and ^REPAIR utilities to find and correct database degradation. As you do this work, you can use Option 2 of the ^STURECOV utility to retry the journal replay/transaction rollback as many times as necessary. You can display any errors you encounter, including those from when the system started, using Option 1. When you correct all the problems, and run Option 2 without any errors, use Option 3 to bring the system up in multiuser mode.

If you find that you cannot resolve the problem, but you still want to bring the system up, use Option 8 to clear the information in the InterSystems IRIS image journal (.wij file) that triggers journal restore and transaction rollback at startup. The option also logs the current information in the messages log. Once this completes, use Option 3 to start the system. Use this facility with care, as it is not reversible.

If InterSystems IRIS was unable to store the errors during startup in the ^%SYS() global for ^STURECOV to display, you may get an initial message before the menu that looks like this:


There is no record of any errors during the prior startup 
This could be because there was a problem writing the data 
Do you want to continue ? No => yes 
Enter error type (? for list) [^] => ? 

Supported error types are: 
     JRN - Journal and transaction rollback 

Enter error type (? for list) [^] => JRN 
 

Journaling errors are one type of error that this utility tries to handle and that is the scope of this topic. Other error types are discussed in the appropriate sections of the documentation.

Caution:

Only use the ^STURECOV utility when the system is in single-user mode following an error during startup. Using it while the system is in any other state (for example, up running normally) can cause serious damage to your data as it restores journal information if you ask it to and this information may not be the most current data. The ^STURECOV utility warns you, but it lets you force it to run.

Convert Journal Files Using ^JCONVERT and ^%JREAD

The ^JCONVERT routine is a utility that reads journal files and converts them to a common file in variable record format. The ^%JREAD utility can then read this file and apply journal transactions to databases on a different system. The ^JCONVERT utility exists on older InterSystems database products as well as all versions of InterSystems IRIS. Use these utilities to move journal data between different system versions that do not have compatible journal files.

For example, if you are converting to a new version of InterSystems IRIS and need to minimize downtime, perform the following steps:

  1. Enable journaling on the old system.

  2. Run a backup on the old system; this switches to a new journal file on the old system.

  3. Continue journaling on the old system.

  4. Restore the backup of the old system on the new system and perform any necessary conversions.

  5. Stop the old system and run ^JCONVERT on the journal files created on the old system since the backup.

  6. Apply the transactions from the old system to the new system using the file created from ^JCONVERT as input to ^%JREAD on the new system.

The ^JCONVERT utility uses the same process as the journal restore utility to select and filter the journal files for processing. You can include a range of journal files as input and create one output file. See Restore Globals From Journal Files Using ^JRNRESTO for details on selecting and filtering journal files.

The converted file is in variable record format. The default character encoding is UTF8, which is compatible with the current ^%JREAD utility on all platforms, and can be moved among platforms with binary FTP. If you answer NO at the Use UTF8 character translation? prompt, no character encoding is applied.

Globals in the journal file are stored with a specific directory reference appended to the global reference. You can choose either to include the directory reference in the converted file, or exclude it. If you include it, you can always filter it out or change it later during the ^%JREAD procedure.

The directory reference determines where ^%JREAD sets the global on the target system. If you do not include the directory reference, ^%JREAD makes all sets in the current directory. If you do include the directory reference, the utility makes sets in the same directory as on the source system unless translated by a ^%ZJREAD program you supply. If the target system is on a different operating system or the databases reside in different directories on the target system, you must supply a ^%ZJREAD routine to translate the directory reference.

The ^%JREAD routine reads a common journal file format and applies the journal transactions to the databases on the target system. During the import of records, if a ^%ZJREAD routine exists, the utility calls it for each journal transaction allowing you to manipulate the journal records. You can reference the following variables in your ^%ZJREAD routine:

      type    - Transaction type
      gref    - Global reference
      value   - Global value
      %ZJREAD - 1:Apply transaction, 0:Do not apply transaction

If you decide not to apply a transaction, set the variable %ZJREAD to 0 (zero) to skip the record. You can also modify the other variables. For example, you can change the directory specification by modifying gref.

The following is an example ^%ZJREAD routine. It looks for transactions that contain updates to %SYS(“JOURNAL”, and prevents them from being applied. You can copy this and modify it to suit your needs:

%ZJREAD;
 /*The following variables are defined; you can modify them
   before the transaction gets applied
 
        type - Transaction type
        gref - Global reference
        value - Global value
        %ZJREAD - 1:Apply transaction, 0:Do not apply transaction
 */
 If gref["SYS(""JOURNAL""" Set %ZJREAD=0
 Quit

Sample Run of ^JCONVERT

The following is a sample run of the ^JCONVERT utility:

%SYS>Do ^JCONVERT
 
Journal Conversion Utility  [ IRIS Format --> Common Format ]


The converted file will be in variable record format.
The default character translation UTF8 is compatible with current ^%JREAD 
on all platforms and can be moved among platforms with binary FTP.
If you answer NO, no character translation will be applied.
 
Use UTF8 character translation? <Yes>

 
Globals in the journal file are stored with a specific directory reference
appended to the global reference. You can choose either to include
the directory reference in the converted file, or exclude it. Note that
if you include it, you can always filter it out or change it later during
the %JREAD procedure.  The directory reference determines where ^%JREAD sets
the global on the target system.  If the directory reference is not included,
all sets are made to the current directory.  If the directory reference is
included, sets will be made to the same directory as on the source system
unless translated by a ^%ZJREAD program you supply.  If the target system
is on a different operating system or the databases reside in different
directories on the target system, the ^%ZJREAD program must be used to
translate the directory reference.
 
Include the directory reference? <Yes>

 
Enter common journal file name:  common.jrn
 
Common journal file: common.jrn
Record separator: Variable
Directory reference: Yes

 
Use current journal filter (ZJRNFILT)? no
Use journal marker filter (MARKER^ZJRNFILT)? no
Process all journaled globals in all directories?   enter Yes or No, please
Process all journaled globals in all directories? yes
Specify range of files to process (names in YYYYMMDD.NNN format)
 
from:     <20231201.001> [?] => 20231202.001
 
through:  <20231204.001> [?] =>
 
Prompt for name of the next file to process? No => No
 

Provide or confirm the following configuration settings:
 
Journal File Prefix: =>
 
Files to dejournal will be looked for in:
     C:\MyIRIS\mgr\journal\
     C:\MyIRIS\mgr\
in addition to any directories you are going to specify below, UNLESS
you enter a minus sign ('-' without quotes) at the prompt below,
in which case ONLY directories given subsequently will be searched
 
Directory to search: <return when done>
Here is a list of directories in the order they will be searched for files:
     C:\MyIRIS\mgr\journal\
     C:\MyIRIS\mgr\

You may tailor the response to errors by choosing between the alternative
actions described below.  Otherwise you will be asked to select an action
at the time an error actually occurs.
 
     Either Continue despite database-related problems (e.g., a target
     database is not journaled, cannot be mounted, etc.), skipping affected
     updates
 
     or     Abort if an update would have to be skipped due to a
     database-related problem (e.g., a target database is not journaled,
     cannot be mounted, etc.)
 
     Either Abort if an update would have to be skipped due to a
     journal-related problem (e.g., journal corruption, some cases of missing
     journal files, etc.)
 
     or     Continue despite journal-related problems (e.g., journal
     corruption, some missing journal files, etc.), skipping affected updates
 
     Either Apply sorted updates to databases before aborting
 
     or     Discard sorted, not-yet-applied updates before aborting (faster)
 
Would you like to specify error actions now? No => yes
 
 
     1.  Continue despite database-related problems (e.g., a target database
     is not journaled, cannot be mounted, etc.), skipping affected updates
 
     2.  Abort if an update would have to be skipped due to a database-related
     problem (e.g., a target database is not journaled, cannot be mounted,
     etc.)
 
Select option [1 or 2]:  1
 
     1.  Abort if an update would have to be skipped due to a journal-related
     problem (e.g., journal corruption, some cases of missing journal files,
     etc.)
 
     2.  Continue despite journal-related problems (e.g., journal corruption,
     some missing journal files, etc.), skipping affected updates
 
Select option [1 or 2]:  2
 
     1.  Apply sorted updates to databases before aborting
 
     2.  Discard sorted, not-yet-applied updates before aborting (faster)
 
Select option [1 or 2]:  2
Based on your selection, this restore will
 
** Continue despite database-related problems (e.g., a target database is not
journaled, cannot be mounted, etc.), skipping affected updates
 
** Continue despite journal-related problems (e.g., journal corruption, some
missing journal files, etc.), skipping affected updates
 
** Discard sorted, not-yet-applied updates before aborting (faster)
 
 
 
C:\MyIRIS\mgr\journal\20231202.001
  13.98%  14.93%  15.95%  17.14%  18.25%  19.27%  20.49%  21.63%  22.65%  23.84%
  24.99%  25.97%  27.10%  28.25%  29.31%  30.50%  31.72%  32.84%  33.84%  34.84%
  35.84%  36.85%  37.91%  38.99%  40.10%  41.08%  42.03%  42.97%  43.93%  44.94%
  45.95%  47.05%  48.11%  49.07%  50.04%  51.02%  52.03%  53.07%  54.14%  55.25%
  56.21%  57.17%  58.15%  59.14%  60.18%  61.24%  62.33%  63.28%  64.20%  65.15%
  66.10%  67.11%  68.13%  69.05%  69.94%  70.83%  71.61%  72.41%  73.09%  73.85%
  74.59%  75.32%  76.06%  76.75%  77.73%  78.70%  79.65%  80.59%  81.53%  82.46%
  83.40%  84.33%  85.27%  86.05%  86.59%  87.13%  87.67%  88.23%  88.78%  89.34%
  89.89%  90.61%  93.28%  94.38%  97.12%  98.21%  99.93%100.00%
***Journal file finished at 11:31:36
 
 
C:\MyIRIS\mgr\journal\20231203.001
  14.01%  14.96%  15.98%  17.18%  18.29%  19.31%  20.53%  21.67%  22.69%  23.88%
  25.03%  26.01%  27.15%  28.30%  29.36%  30.55%  31.78%  32.90%  33.90%  34.90%
  35.91%  36.92%  37.99%  39.06%  40.17%  41.16%  42.11%  43.05%  44.01%  45.03%
  46.04%  47.14%  48.20%  49.17%  50.14%  51.11%  52.13%  53.17%  54.25%  55.36%
  56.33%  57.29%  58.27%  59.26%  60.30%  61.36%  62.46%  63.40%  64.33%  65.28%
  66.23%  67.24%  68.26%  69.19%  70.08%  70.97%  71.76%  72.56%  73.25%  74.01%
  74.75%  75.47%  76.22%  76.91%  77.89%  78.87%  79.83%  80.77%  81.70%  82.64%
  83.58%  84.52%  85.46%  86.24%  86.78%  87.32%  87.87%  88.42%  88.98%  89.53%
  90.09%  90.81%  93.49%  94.59%  97.33%  98.42%100.00%
***Journal file finished at 11:31:37
 
 
C:\MyIRIS\mgr\journal\20231204.001
  13.97%  14.92%  15.93%  17.12%  18.24%  19.25%  20.47%  21.61%  22.62%  23.82%
  24.96%  25.94%  27.07%  28.22%  29.28%  30.46%  31.69%  32.80%  33.80%  34.80%
  35.80%  36.81%  37.87%  38.94%  40.05%  41.04%  41.98%  42.92%  43.88%  44.89%
  45.90%  47.00%  48.06%  49.02%  49.98%  50.96%  51.97%  53.01%  54.08%  55.19%
  56.15%  57.11%  58.08%  59.07%  60.12%  61.17%  62.26%  63.20%  64.13%  65.07%
  66.02%  67.03%  68.05%  68.97%  69.86%  70.75%  71.53%  72.33%  73.01%  73.77%
  74.51%  75.23%  75.98%  76.67%  77.64%  78.61%  79.56%  80.50%  81.43%  82.37%
  83.30%  84.24%  85.17%  85.95%  86.49%  87.03%  87.57%  88.13%  88.68%  89.23%
  89.79%  90.51%  93.18%  94.27%  97.01%  98.10%  99.81%100.00%
***Journal file finished at 11:31:38
 
[journal operation completed]
Converted 26364 journal records 

Set Journal Markers Using ^JRNMARK

To set a journal marker in a journal file, use the following routine:

SET rc=$$ADD^JRNMARK(id,text)

Argument Description
id Marker ID (for example, -1 for backup)
text Marker text of any string up to 256 characters (for example, “timestamp” for backup)
rc Journal location of the marker (journal offset and journal file name, delimited by a comma) or, if the operation failed, a negative error code followed by a comma and a message describing the error. Note that a journal offset must be a positive number.

Manipulate Journal Files Using ^JRNUTIL

InterSystems provides several functions in the ^JRNUTIL routine. You can use these functions for writing site-specific routines to manipulate journal records and files.

The following table lists the functions available in the routine.

Functions Available in ^JRNUTIL
Journaling Task Function Syntax
Close a journal file $$CLOSEJRN^JRNUTIL(jrnfile)
Delete a journal file $$DELFILE^JRNUTIL(jrnfile)
Read a record from a journal file into a local array $$GETREC^JRNUTIL(addr,jrnode)
Switch to a different journal file directory $$JRNSWCH^JRNUTIL(newdir)
Open a journal file $$OPENJRN^JRNUTIL(jrnfile)
Use an opened journal file $$USEJRN^JRNUTIL(jrnfile)
Important:

The DELFILE^JRNUTIL function does not check for open transactions before deleting the journal file.

The following table describes the arguments used in the utility.

Argument Description
addr Address of the journal record.
jrnfile Name of journal file.
newdir New journal file directory.
jrnode Local variable passed by reference to return journal record information.

Manage Journaling at the Process Level Using DISABLE^%NOJRN

If journaling is enabled system-wide, you can stop journaling for Set and Kill operations on globals within a particular process by issuing a call to the ^%NOJRN utility from within an application or from programmer mode as follows:

%SYS>DO DISABLE^%NOJRN

Journaling remains disabled until one of the following events occurs:

  • The process halts.

  • The process issues the following call to reactivate journaling:

    %SYS>DO ENABLE^%NOJRN
    
Note:

Disabling journaling using DISABLE^%NOJRN does not affect mirrored databases.

You must have at least read access to the %Admin_Manage resource to use DISABLE^%NOJRN.

Managing Journal Archive Files Using ^ARCHIVE

You can use the ^ARCHIVE routine, available within the %SYS namespace, to manage journal archive files. This routine provides the following menu:

%SYS>do ^ARCHIVE
 
 
1) Copy a file to an archive target
2) Retrieve an archived file
3) Delete an archived file
4) Manage archive targets
 
Option?

For an API, see the class SYS.ArchiveManagerOpens in a new tab.

See Also

FeedbackOpens in a new tab