Cloud Data Integration: Log error at field level
Posted by: Informatica Cloud
Load error log table with column level errors data when a row contains errors in multiple columns.
Overview
Many customers maintain common error tables to log errors encountered when processing data. Often the errors are tracked at column level and when a particular row of data has issues in multiple columns, then multiple rows have to be inserted into the error table. This can be achieved using IDMC transformations (CDQ rule specification/Expression, Union, Joiner). Scenario: Consider that when you are processing input data, if there are NULL values in the Input columns, then, they have to be populated as errors in a separate flat file or table. For example, if there are NULL values for Email and Phone_Number for a particular customer, then the output row to the error table should contain the Customer Name and an error message that says Invalid Email or Invalid Phone_Number. Source rows:
Name |
Address |
City |
State |
Phone No |
Country |
|
Alan |
Joshua |
New York |
NY |
New York |
USA |
- |
John |
Gates |
2012336654 |
NY |
0 |
USA |
- |
Michael |
Parkaway |
New York |
NY |
0 |
USA |
MikeTracy@gmail.com |
Custom Error Table entries :
MT Name |
Mapping Name |
Source Name |
Source_Key Value |
Error Description |
mt_test |
m_test |
STG_CUSTOMERS |
Alan |
Email is NULL |
mt_test |
m_test |
STG_CUSTOMERS |
John |
Email is NULL |
mt_test |
m_test |
STG_CUSTOMERS |
John |
Phone Number is NULL |
mt_test |
m_test |
STG_CUSTOMERS |
Michael |
Phone Number is NULL |
Features
- Column Level Error Logging