Cloud Data Integration: Log error at field level
Posted by: Informatica Cloud
Load error log table with column level errors data when a row contains errors in multiple columns.
Overview
Many customers maintain common error tables to log errors encountered when processing data. Often the errors are tracked at column level and when a particular row of data has issues in multiple columns, then multiple rows have to be inserted into the error table. This can be achieved using IDMC transformations (CDQ rule specification/Expression, Union, Joiner). Scenario: Consider that when you are processing input data, if there are NULL values in the Input columns, then, they have to be populated as errors in a separate flat file or table. For example, if there are NULL values for Email and Phone_Number for a particular customer, then the output row to the error table should contain the Customer Name and an error message that says Invalid Email or Invalid Phone_Number. Source rows:
Name | Address | City | State | Phone No | Country | |
Alan | Joshua | New York | NY | New York | USA | - |
John | Gates | 2012336654 | NY | 0 | USA | - |
Michael | Parkaway | New York | NY | 0 | USA | MikeTracy@gmail.com |
Custom Error Table entries :
MT Name | Mapping Name | Source Name | Source_Key Value | Error Description |
mt_test | m_test | STG_CUSTOMERS | Alan | Email is NULL |
mt_test | m_test | STG_CUSTOMERS | John | Email is NULL |
mt_test | m_test | STG_CUSTOMERS | John | Phone Number is NULL |
mt_test | m_test | STG_CUSTOMERS | Michael | Phone Number is NULL |
Features
- Column Level Error Logging