Writing partitioned data in file-storage connectors DataStage

Connectors for file storage support certain file formats and have different execution mode behavior for writing partitioned data.

The supported file formats and execution mode behavior work as follows for these file-storage connectors:

• Amazon S3 (without Use DataStage properties selected)
• Box
• Dropbox
• Google Cloud Storage
• IBM Cloud® Object Storage
• Microsoft Azure Blob Storage
• Microsoft Azure File Storage

File formats

The Partitioned property is supported only for CSV and delimited files.

Execution mode behavior

When you select the Partitioned property, the connector will always write in parallel execution mode instead of the default sequential mode.

In parallel execution mode, a folder is created with the name that is specified in the File name property and the files under it are created in this format: part-r-00000, part-r-00001, where the file name starts with part and ends with the node number.