Rest stage: Stage tab (DataStage)

Specify the details of your REST API calls on the Stage tab.

Specify the type of request to make and the URL. If you select Use expression, you can specify an expression to be evaluated and the last statement will return the real URL string value.

Authentication

Use an API key, basic username and password, or bearer token as the API provider requires.

Request

Query parameters

Specify the query parameters for the HTTP request. Query parameters are key and value pairs. You can set constant values or use expressions to set dynamic values.

Custom headers

Specify the headers for the HTTP request. Headers are key and value pairs. You can set constant values or use expressions to set dynamic values.

Custom cookies

An HTTP cookie is a token or packet of status information that the HTTP agent and the target server can exchange to maintain a session. The state information consists of key-value pairs for attributes such as version, domain, a path that specifies the subset of URLs on the origin server to which the cookie applies, and the maximum time for which the cookie is valid.

To set a constant value for the cookie, specify the value in the Default value field. To set a dynamic value, specify an expression in the Derivation field.

Body

Define the format for the request body. If you select form-data or x-www-form-urlencoded, the REST stage evaluates the expression defined in Derivation and if the result is null uses the value in Default value.

If you select Raw or Binary, choose a content type and charset. If Auto is selected, the content type will be detected automatically, but accuracy is not guaranteed. Specify an expression under text or data to use an input column.

Response

Define whether to save the response body as text or in a file.

If you select Text the response body is output in the RESPONSE_BODY and RESPONSE_BODIES variables with its original data or deseralized object when the response body content type is JSON. The Text option requires an output link, and you must specify a derivation for each output column.

If you select File, you can provide a file path as a string for a constant value or a as an expression for a dynamic value. The response body is written to the file and the file path is saved in the RESPONSE_BODY and RESPONSE_BODIES variables.

Settings

Specify security settings, timeout and retry behavior, and configure proxy servers.

Control

For APIs that support pagination, the REST Stage enables easy management of paginated data. You can set up the stage to handle paginated responses, ensuring complete data retrieval. Specify a dynamic URL that changes with each iteration. The Expression builder provides three URL templates for a typical pagination flow.

Variables

Use variables to hold values that can be used in expressions, scripts, and other parts of your RESTful API workflow. Variables can have derivation expressions that incorporate parameters, columns, and other variables. A number of built-in variables are provided to store response information and keep track of iterations. A number of built-in functions are provided to help with formatting.