Developing Ensemble Productions
Handling Batches of Records Efficiently
The RecordMap feature imports a single record at a time, but if you are importing or exporting a large number of records, you can gain substantial efficiency improvement by using RecordMap Batch. The RecordMap Batch feature handles homogeneous records and processes all of the records in a batch at one time. The batch can optionally be preceded by a header record and followed by a trailer record.
Batch processing follows the approach used for other Ensemble message formats like HL7 and X12. This is particularly relevant for the built-in business operations which handle RecordMap batch objects: these business operations accept either Batch objects or RecordMap objects which extend EnsLib.RecordMap.Base
, or a request of type BatchRolloverRequest. When records in a particular batch are received, the Batch is opened and the batch header is written to a temporary file, followed by any objects within that batch received by the operation. If the request in synchronous, the classname, Id, and the count of previously written records for the batch will be returned in a EnsLib.RecordMap.BatchResponse
. Receipt of the batch object (which may be the default batch) will trigger the batch trailer to be written to the temporary file, and this file will then be sent to the desired destination by the adapter for the business operation. If a Batch object is received by itself, then the entire Batch will be written out to a temporary file which will then be transferred to the desired location.
Batch operations also support a default batch option whereby records which do not already belong to a batch are added to a default batch. Output of this batch can be triggered by either sending the batch object to the operation, or sending a BatchRolloverRequest to the operation. The business operation can also be configured to use schedule- or count-based rollover for the default batch. These options are configured on the business operation, and can be used simultaneously.
The options for services primarily concern the way the Visual Trace displays messages within a batch.
The RecordMap Batch operation creates temporary files in the process of generating the final output file. You can control the location of these temporary files by specifying the IntermediateFilePath
setting of the RecordMap Batch operation. If the namespace’s database is being mirrored, it is important that all mirror members have access to the temporary file in order to successfully failover during a RecordMap Batch operation. See the Caché High Availability Guide
for information on mirroring.
The ENSDEMO namespace contains the RecordMapBatch example that demonstrates how to use RecordMap Batch to import and export batches of records.
© 1997-2019 InterSystems Corporation, Cambridge, MA