Generic S3 connector (DataStage)

Use the Generic S3 connector to connect to object stores that support S3 API and perform various read and write functions.

Prerequisite

Configure the read process.

Table 1. Reading data from Generic S3
Read mode	Procedure
Read a single file	Specify the bucket name that contains the file, and then specify the name of the file to read.
Read binary data	Specify the bucket name that contains the file, and then specify the name of the file to read.
Read binary data from multiple files using wildcards	Specify a wildcard character in the file name for binary data. For example, File name: `test.*.gz` If you use this option, you can read multiple binary files one after another, and each file will be read as a record. If you select Read a file to a row, you must provide two column names in the Output tab of the source stage: The first column must be a string data type. This column is for the file name. The second column must be a binary data type. This column is for the file. The binary column's precision value must be greater than or equal to the maximum file size.
Read multiple files using regex expression	Specify the bucket name that contains the files. You can use a java regex expression for the file name. Examples: `^csv_write_datatypes_h.[0-9]$` `csv_write_datatypes_h.[^12]`
Read multiple files using wildcards	Specify an asterisk () to match zero or more characters. For example, specify `.txt` to match all files with the .txt extension. Specify a question mark (?) to match one character. Examples: `csv_write_datatypes.` `?_abc_test`