Performance Tuning: Database Partitioning

Performance Tuning: Database Partitioning

Posted by: Nagaraj Gayakwad

A workflow example that demonstrates various portioning options for database sources and targets.

Overview

You can use pipeline partitioning to improve session performance. Increasing the number of partitions or partition points increases the number of threads. Therefore, increasing the number of partitions or partition points also increases the load on the nodes in the Integration Service. If the Integration Service node or nodes contain ampleCPU bandwidth, processing rows of data in a session concurrently can increase session performance.For your database sources or targets, you can define the following partition types in the Workflow Manager: Database partitioning : The PowerCenter Integration Service queries the IBM DB2 or Oracle system for table partition information. It reads partitioned data from the corresponding nodes in the database. Use database partitioning with Oracle or IBM DB2 source instances on a multi-node table space. Use database partitioning with DB2 targets. Hash partitioning : Use hash partitioning when you want the PowerCenter Integration Service to distribute rows to the partitions by group. For example, you need to sort items by item ID, but you do not know how any items have a particular ID number.You can use the following types of hash partitioning: Hash auto-keys : The PowerCenter Integration Service uses all grouped or sorted ports as a compound partition key. You may need to use hash auto-keys partitioning at Rank, Sorter, and unsorted Aggregator transformations. Hash user keys : The PowerCenter Integration Service uses a hash function to group rows of data among partitions. You define the number of ports to generate the partition key. Key range : You specify one or more ports to form a compound partition key. The PowerCenter Integration Service passes data to each partition depending on the ranges you specify for each port. Use key range partitioning where the sources or targets in the pipeline are partitioned by key range. Pass-through : The PowerCenter Integration Service passes all rows at one partition point to the next partition point without redistributing them. Choose pass-through partitioning where you want to create an additional pipeline stage to improve performance, but do not want to change the distribution of data across partitions. Round-robin : The PowerCenter Integration Service distributes listings of data to one or more partitions. Use round-robin partitioning so that each partition processes rows based on the number and size of the listing s. In the workflow example given here, you will find sessions that explain each partitioning option.

Features

  • Informatica PowerCenter 9.1 or above

Resources

Support