Cloud Data Integration: Log error at field level

Cloud Data Integration: Log error at field level

Posted by: Informatica Cloud

Load error log table with column level errors data when a row contains errors in multiple columns.

Overview

Many customers maintain common error tables to log errors encountered when processing data. Often the errors are tracked at column level and when a particular row of data has issues in multiple columns, then multiple rows have to be inserted into the error table. This can be achieved using IDMC transformations (CDQ rule specification/Expression, Union, Joiner). Scenario: Consider that when you are processing input data, if there are NULL values in the Input columns, then, they have to be populated as errors in a separate flat file or table. For example, if there are NULL values for Email and Phone_Number for a particular customer, then the output row to the error table should contain the Customer Name and an error message that says Invalid Email or Invalid Phone_Number. Source rows:

 Name

 Address

 City

 State

 Phone No

 Country

 Email

 Alan

 Joshua

 New York

 NY

 New York

 USA

 -

 John

 Gates

 2012336654

 NY

 0

 USA

 -

 Michael

 Parkaway

 New York

 NY

 0

 USA

 MikeTracy@gmail.com

 

Custom Error Table entries :

 MT Name

 Mapping Name

 Source Name

 Source_Key Value

 Error Description

 mt_test

 m_test

 STG_CUSTOMERS

 Alan

 Email is NULL

 mt_test

 m_test

 STG_CUSTOMERS

 John

 Email is NULL

 mt_test

 m_test

 STG_CUSTOMERS

 John

 Phone Number is NULL

 mt_test

 m_test

 STG_CUSTOMERS

 Michael

 Phone Number is NULL

 

Features

  • Column Level Error Logging

Resources