0 / 0
External Filter stage in DataStage

External Filter stage

The External Filter stage allows you to specify a UNIX command that acts as a filter on the data you are processing.

The External Filter stage is a processing stage. It can have a single input link and a single output link. For example, you can use the stage to grep a data set for a certain string, or pattern, and discard records that do not contain a match. This technique can be used to quickly filter data.

Whitespace is stripped from the start and end of the data before the command is executed. To avoid this behavior, use an explicitly wrapped command that sets format options on the schema.

The stage editor has three tabs:

  • Stage . Use this tab to specify general information about the stage.
  • Input. Use this tab to specify details about the input link carrying the data to be filtered.
  • Output. Use this tab to specify details about the filtered data being output from the stage.
Generative AI search and answer
These answers are generated by a large language model in watsonx.ai based on content from the product documentation. Learn more