B2B DT Streamer:Processing Large Files

B2B DT Streamer:Processing Large Files

Posted by: Informatica Professional Services

Template to resolve scenarios with large source file requiring the need to use the Streamer component of DT. 

Overview

While working with Informatica B2B, there are scenarios where the source file is very large (+hundred Mb's)  and one would need to use the Streamer component of DT.

For some of these cases the Usage of Streamer would not be trivial for different reasons like breaking the file into Header body Trailer needs more logic then one can apply with the StreamerMarkers, where the breaking is dynamic etc.

For such cases this example gives you an idea of how it can be resolved.

The idea is to treat the buffers coming into the Streamer (by size, not logical) as a pass through buffers into a Parser component that will handle all the logic for breaking the buffers into logical components and aggregating buffers if needed and/or appending left over.

The reason to use a Parser is because it has the strongest capabilities when it comes to handling text buffers.

Download this example which can be used as a template for your project. 

Features

  • DT 4.4 or higher

Resources

READ ME