Staging Activity (2.13)
Introduction
A staging activity loads data from one or more data schemas , loading data for each schema to its own generated staging table. For each schema, you can control which fields are copied, as well as how the load operation is performed. Within a staging activity, all schemas must belong to a single data source.
Options
For each data schema used in a staging activity, the following options apply:
-
Extraction Strategy —Indicates how data is extracted from this schema. This option is controlled by the schema definition and cannot be changed within a recipe.
-
Fields —Specifies which fields from the data schema are loaded into the staging table. You can select any subset of available fields. Fields that are not selected are not included in the generated staging table.
-
Primary Key Fields —Indicates the primary key fields of the schema. This option is controlled by the schema definition and cannot be changed within a recipe.
-
Skip Loading Old Files —Loads only files that have not previously been loaded by this staging activity. Only files loaded by successfully completed recipe runs are considered previously loaded. Requires the File Timestamp or File List extraction strategies to be enabled for the schema.
-
Stage on Target Table —Enables you to copy data to the table name of your choice. The table is generated for you. If you select this, you are prompted for a full table name.
-
Staging Table —Specifies the table into which this activity loads data. This table name is generated and cannot be changed.
-
Use SQLQuickload —Enables the system to perform a bulk load operation, which can improve performance when loading large volumes of data. If you select this, also specify the following options:
-
Action On Dropped Records —Specifies the action to take when the bulk load encounters more than the specified number of dropped records. The Use SQLQuickload option if that record cannot be inserted (for example, because required fields are missing or values are in an invalid format). For Action On Dropped Records , select one of the following, to specify how the system should behave if the maximum number of dropped records is reached:
-
Abort —The recipe is automatically aborted, without any workflow intervention.
-
WorkflowStrict —The recipe is stopped and a workflow task is generated. The workflow task provides the options Abort and Retry .
-
Workflow —The recipe is stopped and a workflow task is generated. The workflow task provides the options Abort , Retry, and Ignore and Proceed.
-
Proceed —The recipe continues.
-
-
Max Dropped Records —Specifies the maximum number of dropped records before the specified action is triggered.
-
