Purging Production Data

Introduction

For each production running in a given namespace, InterSystems IRIS may write entries to the event log, message warehouse, business process log, business rule log, and I/O archive log for the namespace. Since the entries can accumulate over time and consume large amounts of disk space, InterSystems IRIS enables you to purge outdated entries if you have appropriate permissions (see Controlling Access to Management Portal Functions).

You can do so manually; that is, you can purge production data on an ad hoc basis. You can also schedule regular purges. Typically, you perform manual purges on systems that you are using for development and testing, and you set up scheduled purges for live systems.

Purging generates journaling. If you purge a large volume of data, the resultant journaling can consume a large amount of disk space. To conserve disk space, you can purge smaller amounts of data and review the storage impacts before purging additional data.

First-time Purges

Purging generates journaling. If you purge a large volume of data, the resultant journaling can consume a large amount of disk space. To conserve disk space, you can adopt the following approach the first time you purge management data:

1. Switch to the namespace where you want to purge data.

2. Navigate to the Interoperability > Manage > Purge Management Data page.

3. Set the purge parameters so that a relatively small amount of data is purged.

   For example, you can set Do not purge most recent to a relatively large number. For more information, see Settings for Purging Data.

   Caution:

   Purges are irreversible and can lead to unintentionally orphaned data or the loss of unresolved requests. Consequently, InterSystems recommends that you carefully review the description of each setting before proceeding.

4. Click Start Purge.

5. Gradually decrease the Do not purge most recent value and purge additional data until you have purged a sufficient amount of data.

Purging Data Manually

The Purge Management Data page enables you to purge entries in the event log, message warehouse, business process log, business rule log, and I/O archive log all at one time for a given namespace. The page displays information about the entries in a table with the following columns:

• Record Type — Indicates the type of production data associated with the row. Each row contains one type of artifact that running productions produce on an ongoing basis: Event Log, Messages, Business Processes, Business Rule Log, I/O Log, or Managed Alerts.

• Count — Shows the total number of entries of a given Record Type stored for the production. You can use the Count value to decide whether it is worthwhile to purge the entries and if so, how many days’ worth of records to keep.

• Deleted — After you click Start Purge and the system completes the purge process, shows the total number of entries of a given Record Type that were purged.

Additionally, the Purge Criteria area displays the default settings that your system administrator configured for manual purges.

To purge production data manually, do the following:

1. Switch to the namespace where you want to purge data.

2. Navigate to the Interoperability > Manage > Purge Management Data page.

3. If you have appropriate permissions, modify the settings in the Purge Criteria area as needed.

   For more information, see Settings for Purging Data.

   Caution:

   Purges are irreversible and can lead to unintentionally orphaned data or the loss of unresolved requests. Consequently, InterSystems recommends that you carefully review the description of each setting before proceeding.

4. Click Start Purge.

   The system immediately purges the persistent store using the settings in the Purge Criteria area. The page uses a background job to perform purges, and reports the results of the last-run purge, including a status code, or a notice if the background job is running or has failed to run. After the purge, the Deleted column displays the number of records that were purged.

   The Start Purge button is disabled while a purge is being executed in a given namespace.

5. Purging using the Management Portal only purges 500 chunks of bitmaps at a time. If you are purging a large number of messages this will leave unpurged bitmaps which can take up space. To remove these you can run another purge via the Terminal. Run the purge using the following settings:

   set pDaysToKeep=7
   set pKeepIntegrity=0
   set pBodiesToo=1
   set pBitmapChunkLimit=10000000000
   
   write ##class(Ens.Util.MessagePurge).Purge(pDeletedCount, pDaysToKeep, pKeepIntegrity, pBodiesToo, pBitmapChunkLimit, pExtendedOptions)
   
   zwrite pDeletedCount
   

   

Purging Data on a Schedule

The Task Scheduler Wizard enables you to schedule purges for the following types of production data separately or all at one time for a given namespace:

• Events

• Messages

• Business processes

• Rule logs

• I/O logs

• Host monitor data

• Managed alerts

To purge data automatically at regular intervals, do the following:

1. Navigate to System Operation > Task Manager, and then select New Task.

2. Fill in the following fields:

   • Task name — Specify a name for the purge task.

   • Namespace to run task in — Select the namespace where you want to purge data.

   • Task type — Select Ens.Util.Tasks.PurgeOpens in a new tab.

   Various settings for purging data appear.

3. Modify the setting for purging data as needed.

   For more information, see Settings for Purging Data.

   Caution:

   Purges are irreversible and can lead to unintentionally orphaned data or the loss of unresolved requests. Consequently, InterSystems recommends that you carefully review the description of each setting before proceeding.

4. Specify other options as needed.

   For more information, see Using the Task Manager.

5. Click Finish.

Settings Applicable to Data Purges

This section describes the settings that affect data purges. These settings are all specific to the namespace in which they are set. Users with appropriate permissions can modify these.