Using the Complex Record Mapper
With the Record Mapper, you may need to handle messages whose format consists of multiple heterogeneous records. Typically, these complex records have a header record followed by a pattern of records and terminated by a trailer record. These records can either be fixed-fields records or delimited records and can be optional and repeat. Typically, the records have leading data that identifies the kind of record.
Complex Record Mapper Overview
Complex record maps can describe structured records that can contain:
-
An optional header record.
-
Sequence of elements where each element can be a record defined by a RecordMap or a sequence. A sequence can contain a sequence of records and other sequences.
-
An optional trailer record.
The records within a sequence can either be delimited records or fixed-width records. Although it is possible to mix delimited and fixed-width records within a complex record map, typically all of the records are delimited or all are fixed-width.
The following delimited sample data can be described by a complex record map. The data consists of a header identifying a semester in a college and information about students and classes that each student takes.
SEM|194;2012;Fall;20
STU|12345;Adams;John;Michael;2;john.michael.adams@example.com;617-999-9999
CLS|18.034;1;Differential Equations;4
CLS|21W.759;1;Writing Science Fiction;4
STU|12346;Adams;Jane;Michelle;3;jane.michelle.adams@example.com;
CLS|21L.285;1;Modern Fiction;3
CLS|7.03;1;Genetics;4
STU|12347;Jones;Robert;Alfred;1;bobby.jones@example.com;
CLS|18.02;1;Calculus;4
The complex record map that describes this data consists of:
-
Header record identified by the leading data “SEM|”.
-
A sequence of students, where each student consists of:
-
Student record identified by the leading data “STU|”.
-
A repeating class record identified by the leading data “CLS|”.
-
The sequence of students defines the repeating structure of the complex record but does not correspond to a record in the data.
A complex record map defines both a file structure and an object structure. The complex record map file service parses a file using the file structure defined by the complex record map and then stores the data in an object defined by the object structure. The complex record map file operation performs the reverse—it takes the data in the object and writes it out to a file using the file structure defined by the complex record map.
Creating and Editing a Complex Record Map
You can create a new complex record map or edit an existing one by using the Complex Record Mapper in the Management Portal or by importing an XML definition and editing one in an IDE.
The complex map has a top-level sequence in which you can define the fields. If your complex record consists of a single sequence of records, where the sequence does not repeat, you can enter the record maps of the records directly in the complex record map. But, if the entire sequence can repeat between the header and trailer, then you should enter a repeating sequence record as the only element of the top-level sequence.
Getting Started
To access the Complex Record Mapper from the Management Portal, click Interoperability, Build, and Complex Record Maps. From here, you have the following commands:
-
Open—Opens an existing complex record map.
-
New—Creates a new complex record map.
-
Save—Saves your complex record map structure as a class in the namespace in which you are working in the package specified in the Complex RecordMap Name.
-
Generate—Generates the complex record map parser code and the related persistent message complex record class object.
To generate an object manually use the Generate()Opens in a new tab class method in EnsLib.RecordMap.ComplexGeneratorOpens in a new tab.
-
Delete—Deletes the current complex record map. You can optionally delete the related persistent message complex record class and all stored instances of the classes.
The Save operation only writes the current complex record map to disk. In contrast, the Generate operation generates the parser code and the persistent object structure for the underlying objects. Generating complex record maps will discard manual modification to generated parser code. To help guide this principle, generated classes (including header, footer, and batch) are clearly marked with “DO NOT EDIT” comments.
Editing the Complex Record Map Properties
Whether you are entering properties for a new complex record map, starting from the wizard-generated map, or editing an existing map, the process is the same. For the complex record itself, enter or update values in the following fields:
Name of the complex record map. You should qualify the complex record map name with the package name. If you do not provide a package name and specify an unqualified complex record map name, the complex record map class is saved in the User package by default.
Name of the class to represent the complex record. By default, the Complex Record Mapper sets the target class name to a qualified name equal to the complex record map name followed by “.Batch”, but you can change the target class name. You should qualify the target class name with the package name. If you do not provide a package name and specify an unqualified target class name, the target class is saved in the User package by default.
Character encoding for imported data records. The specified for the complex record map should be the same as the encoding for all record maps included in the complex record map. If they are not the same, the character encoding for the complex record map overrides the character encoding for the record maps.
Text that documents the purpose and use of the complex record map.
If you create a new complex record map, the Complex Record Mapper creates a definition that consists of the following elements.
-
Complex map name and type—enter the name and class for the complex map.
-
Header—if your complex record has a header, enter the name and the class of the record map that describes the header.
-
Trailer—if your complex record has a trailer, enter the name and the class of the record map that describes the trailer.
Editing the Complex Record Map Records and Sequences
A complex record map consists of the following:
-
An optional header record.
-
Sequence of elements where each element can be:
-
Record defined by a RecordMap. It can have the following properties in the complex record map:
-
Required. A value 0 means the record is optional and a value 1 means it is required.
-
Repeating with a minimum and maximum number of occurrences.
-
-
A nested sequence of elements.
-
-
An optional trailer record.
Each record is defined by a record map. A sequence is defined in the complex record map definition. It describes the structure of the data in the message but does not itself correspond to any fields in the data.
The header and trailer records are each defined by a record map. Although it is optional to include a header or trailer record in the complex record map definition, if the definition contains a header record, then the data must contain a header record, and if the definition contains a trailer record, then the data must contain a trailer record. Header and trailer records cannot repeat.
Every sequence must contain at least one record or sequence.
When you are editing a record, you can click the Make Sequence button to replace the record with a sequence. When you are editing a sequence, you can click the Make Record button to replace the sequence with a record.
You can specify the following properties for a record:
-
Record name.
-
RecordMap that defines the record format. The RecordMap specifies the Leading data that identifies the record, whether the record has fixed columns or is delimited, the separators, and the record terminator. For details on defining a RecordMap, see Using the Record Mapper.
-
Whether the record is required.
-
Whether the record can be repeated. If the record can be repeated, you can also specify:
-
Minimum number of repetitions
-
Maximum number of repetitions
-
-
Annotation that documents the purpose and use of the record in the complex record map.
You can specify the following properties for a sequence:
-
Sequence Name
-
Whether the sequence is required.
-
Whether the sequence can be repeated. If the sequence can be repeated, you can also specify:
-
Minimum number of repetitions
-
Maximum number of repetitions
-
-
Annotation that documents the purpose and use of the sequence in the complex record map.
Complex Record Map Class Structure
There are two classes that describe a complex record map in a similar manner to the two classes that describe a record map. The two classes that describe a complex record map are:
-
Complex record map that describes the external structure of the complex record and implements the complex record parser and writer.
-
Generated complex record class that defines the structure of the object containing the data. This object allows you to reference the data in data transformations and in routing rule conditions.
A complex record map business service reads and parses the incoming data and creates a message, which is an instance of the generated record class. A business process can read, modify or generate an instance of the generated complex record class. Finally, a complex record map business operation uses the data in the instance to write the outgoing data using the complex record map as a formatting template. Both the complex record map class and the generated complex record class have hierarchical structures that describe the data. The complex record map class and the generated complex record class have parallel structures. This is different from the RecordMap class, where the generated record class can have a different hierarchal structure.
When you create a new complex record map and then save it in the Management Portal, this action defines a class for that extends the EnsLib.RecordMap.ComplexMapOpens in a new tab and Ens.RequestOpens in a new tab classes. In order to define the generated record class, you must click Generate in the Management Portal, which calls the Generate()Opens in a new tab method in the EnsLib.ComplexGenerator class. Just compiling the ComplexMap class definition does not create the code for the generated record class. You must use the Management Portal or call the ComplexGenerator.Generate() method from the Terminal or from code. The generated class extends the RecordMap.ComplexBatchOpens in a new tab and Ens.RequestOpens in a new tab classes.
The ComplexMap class defines the complex record structure in an XData definition that defines the ComplexBatch with records specified by RecordReference elements and sequences defined by RecordSequence elements. If the RECORDMAPGENERATED parameter of the existing class is 0, then the target class is not modified by the complex record map framework—all changes are then the responsibility of the production developer.
The ComplexBatch class has properties that correspond to the following top-level elements in the complex map definitions:
-
Header record, if specified. This property has its type set to the generated record class for the specified record map.
-
Record that has its type set to the generated record class for the specified record map or, if the record can be repeated, its type is set to an array of the generated record class.
-
Sequence that has its type set to the class defined for the sequence or, if the sequence can be repeated, its type is set to an array of this class.
-
Trailer record, if specified. This property has its type set to the generated record class for the specified record map.
A class is defined for each sequence. The sequence class extends the ComplexSequenceOpens in a new tab and %XML.AdaptorOpens in a new tab classes. The sequence class is defined within the package and namespace defined for the ComplexBatch class. All sequence classes are defined in this level of the namespace even if they are contained within other sequences.
Each sequence has properties that correspond to the records and sequences that it contains.
Using a Complex Record Map in a Production
To create a production that uses complex records, you do the following:
-
Create the individual record maps for each part of the complex record, including the header and trailer. See Using the Record Mapper for a description of how to create the individual record maps. Note that if you intend to use a sample file, you should create a sample file that contains only the part of the complex record that you are defining in the individual record map. The sample file should not contain a complete complex record.
-
Use the Complex Record Mapper to define the structure of the complex record.
-
Create a production and add one or more of the built-in complex record services and operations.
-
If your production is simply passing a complex record from one application to another, you may be able to use a simple routing engine process. But, if your production is converting one complex record into a different complex record, you will create a Data Transformation in the routing engine. If both the input and output complex records have the same structure, you can create a simple data transformation that connects the source and target fields. For example, you could use a simple data transformation to convert a complex record containing delimited records to a complex record containing fixed column records. But if the input complex record does not have the same structure as the output complex record, you must add code either within the data transformation or in a Business Process Language (BPL) process.