Partitioning and collecting data
Use the Partitioning section in DataStage® stages or connectors that have Input tabs to specify details about how the stage or connector partitions or collects data on the current link before it processes the data or writes it to a data target.
You can also use the Partitioning section to sort data that is arriving on the input link before the data is processed or written to the data target. The availability of sorting depends on the partitioning or collecting method that is chosen. It is not available with the Auto methods. The Partitioning section provides basic sorting facilities. For a more complex sort operation, use the Sort stage.
Note: Partitioning is not currently available for the Transformer stage.
The Partitioning section contains the following controls and fields:
- Partitioning
- Choose the partitioning type from the list.
- Collecting
- Choose the collecting type from the list.
- Sorting
- Use these controls to specify how to sort the data. Data is always sorted within data
partitions. If the stage is partitioning incoming data, the data is sorted after the partitioning.
If the stage is collecting incoming data, the data is sorted before the collection.
- Sort
- Select Perform sort to sort data that comes in on the link.
- Stable
- Select Stable if you want to preserve previously sorted data sets. Stable is set by default.
- Unique
- Select Unique if you want to retain only one record per sorting key value. If multiple records have identical sorting key values, all but one is discarded. If stable sort is also set, the first record with the sorting key value is the record that is retained.