Skip to main content

Purging Production Data

For each production running in a given namespace, InterSystems IRIS may write entries to the event log, message warehouse, business process log, business rule log, and I/O archive log for the namespace. Since the entries can accumulate over time and consume large amounts of disk space, InterSystems IRIS enables you to purge outdated entries if you have appropriate permissions.

You can do so manually; that is, you can purge production data on an ad hoc basis. You can also schedule regular purges. Typically, you perform manual purges on systems that you are using for development and testing, and you set up scheduled purges for live systems.

Purging generates journaling. If you purge a large volume of data, the resultant journaling can consume a large amount of disk space. To conserve disk space, you can purge smaller amounts of data and review the storage impacts before purging additional data.

For more information about permissions, see Controlling Access to Management Portal Functions.

First-time Purges

Purging generates journaling. If you purge a large volume of data, the resultant journaling can consume a large amount of disk space. To conserve disk space, you can adopt the following approach the first time you purge management data:

  1. Switch to the namespace where you want to purge data.

  2. Navigate to the Interoperability > Manage > Purge Management Data page.

  3. Set the purge parameters so that a relatively small amount of data is purged.

    For example, you can set Do not purge most recent to a relatively large number. For more information, see Settings for Purging Data.

    Caution:

    Purges are irreversible and can lead to unintentionally orphaned data or the loss of unresolved requests. Consequently, InterSystems recommends that you carefully review the description of each setting before proceeding.

  4. Click Start Purge.

  5. Gradually decrease the Do not purge most recent value and purge additional data until you have purged a sufficient amount of data.

Purging Data Manually

The Purge Management Data page enables you to purge entries in the event log, message warehouse, business process log, business rule log, and I/O archive log all at one time for a given namespace. The page displays information about the entries in a table with the following columns:

  • Record Type — Indicates the type of production data associated with the row. Each row contains one type of artifact that running productions produce on an ongoing basis: Event Log, Messages, Business Processes, Business Rule Log, I/O Log, or Managed Alerts.

  • Count — Shows the total number of entries of a given Record Type stored for the production. You can use the Count value to decide whether it is worthwhile to purge the entries and if so, how many days’ worth of records to keep.

  • Deleted — After you click Start Purge and the system completes the purge process, shows the total number of entries of a given Record Type that were purged.

Additionally, the Purge Criteria area displays the default settings that your system administrator configured for manual purges.

To purge production data manually, do the following:

  1. Switch to the namespace where you want to purge data.

  2. Navigate to the Interoperability > Manage > Purge Management Data page.

  3. If you have appropriate permissions, modify the settings in the Purge Criteria area as needed.

    For more information, see Settings for Purging Data.

    Caution:

    Purges are irreversible and can lead to unintentionally orphaned data or the loss of unresolved requests. Consequently, InterSystems recommends that you carefully review the description of each setting before proceeding.

  4. Click Start Purge.

    The system immediately purges the persistent store using the settings in the Purge Criteria area. The page uses a background job to perform purges, and reports the results of the last-run purge, including a status code, or a notice if the background job is running or has failed to run. After the purge, the Deleted column displays the number of records that were purged.

    The Start Purge button is disabled while a purge is being executed in a given namespace.

  5. Purging using the Management Portal only purges 500 chunks of bitmaps at a time. If you are purging a large number of messages this will leave unpurged bitmaps which can take up space. To remove these you can run another purge via the Terminal. Run the purge using the following settings:

    set pDaysToKeep=7
    set pKeepIntegrity=0
    set pBodiesToo=1
    set pBitmapChunkLimit=10000000000
    
    write ##class(Ens.Util.MessagePurge).Purge(pDeletedCount, pDaysToKeep, pKeepIntegrity, pBodiesToo, pBitmapChunkLimit, pExtendedOptions)
    
    zwrite pDeletedCount
    

Purging Data Automatically

The Task Scheduler Wizard enables you to schedule purges for the following types of production data separately or all at one time for a given namespace:

  • Events

  • Messages

  • Business processes

  • Rule logs

  • I/O logs

  • Host monitor data

  • Managed alerts

To purge data automatically at regular intervals, do the following:

  1. Navigate to System Operation > Task Manager, and then select New Task.

  2. Fill in the following fields:

    • Task name — Specify a name for the purge task.

    • Namespace to run task in — Select the namespace where you want to purge data.

    • Task type — Select Ens.Util.Tasks.PurgeOpens in a new tab.

    Various settings for purging data appear.

  3. Modify the setting for purging data as needed.

    For more information, see Settings for Purging Data.

    Caution:

    Purges are irreversible and can lead to unintentionally orphaned data or the loss of unresolved requests. Consequently, InterSystems recommends that you carefully review the description of each setting before proceeding.

  4. Specify other options as needed.

    For more information, see Using the Task Manager.

  5. Click Finish.

Settings for Purging Data

InterSystems IRIS provides several settings that enable you to configure data purges if you have appropriate permissions:

Setting in Purge Management Data and Purge Data Settings Pages Setting in Task Scheduler Wizard Page Default Description
Include message bodies BodiesToo Disabled

Indicates whether InterSystems IRIS purges both message headers and message bodies during a purge operation.

If you select the check box, InterSystems IRIS purges message headers and their corresponding message bodies. If you clear the check box, InterSystems IRIS purges only message headers and retains any corresponding message bodies.

The system verifies that body classes exist and are persistent before purging them.

For more information, see Considerations When Configuring the Include message bodies or BodiesToo Setting

Purge only completed sessions KeepIntegrity Enabled

Indicates whether InterSystems IRIS skips messages that are part of incomplete sessions during the purge process.

If you select the check box, when InterSystems IRIS encounters a message that meets the age criterion for purging, but is in an incomplete session, the system does not purge the message header or body. An incomplete session corresponds to any session that includes a message with a status other than Complete, Error, Aborted, or Discarded.

For more information, see Considerations When Configuring the Purge only completed sessions or KeepIntegrity Setting

Description  

"Include message bodies" is OFF because some Productions may use message objects that are part of a larger environment and not transitory.

"Purge only completed sessions" is ON to preserve messages not yet completely processed.

Provides information about the settings.

InterSystems IRIS includes a default description that you can optionally modify.

Do not purge most recent NumberOfDaysToKeep

7

Specifies how many days’ worth of records to keep. The count of days includes today.

If you set the value to 0 (zero), InterSystems IRIS does not keep any records and purges all the entries that exist at the time of the purge operation. If you set the value to 1 , InterSystems IRIS retains only the messages generated on the current day, according to local server time.

  TypesToPurge Events

Specifies the types of records to purge.

Considerations When Configuring the Include message bodies or BodiesToo Setting

If InterSystems IRIS purges only message headers, the system may accumulate large quantities of message bodies. You cannot delete the retained message bodies from the Management Portal. You can delete them only programmatically. Consequently, InterSystems recommends that you consider your disk space and workflow when you configure the Include message bodies or BodiesToo setting.

Additionally, when InterSystems IRIS purges a message body, it does not necessarily delete all the object-valued properties of the message body. The system deletes only objects that have a serial or child relationship to the message body. You must delete other objects manually by defining a delete trigger or implementing the %OnDelete() method in the message body class, as appropriate. For more information about object-valued properties, see Defining and Using Object-Valued Properties.

Considerations When Configuring the Purge only completed sessions or KeepIntegrity Setting

If you enable the Purge only completed sessions or KeepIntegrity setting, InterSystems IRIS executes a query that reviews all the messages (including business process instances) in each relevant session to identify any incomplete sessions. Consequently, enabling this setting can increase the amount of time required to complete a purge operation.

Preserving session-level integrity supports long-running business processes. InterSystems recommends that you consider whether you need to support long-running business processes and whether your system contains insignificant old messages in incomplete sessions when you configure the Purge only completed sessions or KeepIntegrity setting.

Caution:

Purge operations can include messages associated with long-running system processes, such as workflow processes. If you disable this setting, carefully review the Do not purge most recent value to ensure that you do not purge critical system data.

FeedbackOpens in a new tab