Handling Batches of Records Efficiently

The RecordMap feature imports a single record at a time, but if you are importing or exporting a large number of records, you can gain substantial efficiency improvement by using RecordMap Batch. The RecordMap Batch feature handles homogeneous records and processes all of the records in a batch at one time. The batch can optionally be preceded by a header record and followed by a trailer record.

To create a RecordMap batch, you implement a class which inherits from %PersistentOpens in a new tab and EnsLib.RecordMap.BatchOpens in a new tab. The Batch class contains methods that handle parsing and writing out any headers and trailers associated with a specific batch. You must provide code that parses and writes your headers. For simple headers and trailers, you can use the EnsLib.RecordMap.SimpleBatchOpens in a new tab class, which inherits from the Batch class and provides code for handling simple headers and trailers. You can extend either of these two batch implementations if you need to process more complex header and trailer data.

Batch processing follows the approach used for other Ensemble message formats like HL7 and X12. This is particularly relevant for the built-in business operations which handle RecordMap batch objects: these business operations accept either Batch objects or RecordMap objects which extend EnsLib.RecordMap.BaseOpens in a new tab, or a request of type BatchRolloverRequest. When records in a particular batch are received, the Batch is opened and the batch header is written to a temporary file, followed by any objects within that batch received by the operation. If the request in synchronous, the classname, Id, and the count of previously written records for the batch will be returned in a EnsLib.RecordMap.BatchResponseOpens in a new tab. Receipt of the batch object (which may be the default batch) will trigger the batch trailer to be written to the temporary file, and this file will then be sent to the desired destination by the adapter for the business operation. If a Batch object is received by itself, then the entire Batch will be written out to a temporary file which will then be transferred to the desired location.

Batch operations also support a default batch option whereby records which do not already belong to a batch are added to a default batch. Output of this batch can be triggered by either sending the batch object to the operation, or sending a BatchRolloverRequest to the operation. The business operation can also be configured to use schedule- or count-based rollover for the default batch. These options are configured on the business operation, and can be used simultaneously.

The options for services primarily concern the way the Visual Trace displays messages within a batch.

The RecordMap Batch operation creates temporary files in the process of generating the final output file. You can control the location of these temporary files by specifying the IntermediateFilePath setting of the RecordMap Batch operation. If the namespace’s database is being mirrored, it is important that all mirror members have access to the temporary file in order to successfully failover during a RecordMap Batch operation. See the Caché High Availability Guide for information on mirroring.

The ENSDEMO namespace contains the RecordMapBatch example that demonstrates how to use RecordMap Batch to import and export batches of records.