Connecting to Amazon S3 in Watson Query

You can use Watson Query to access data that is stored in an Amazon S3 data source.

For more information, see Data sources in object storage in Watson Query.

Before you begin

To access data that is stored in Cloud Object Storage, you must create a connection to the data source where the files are located. Watson Query integrates with object storage connections and supports PARQUET (or PARQUETFILE), Optimized Row Columnar (ORC), comma-separated values (CSV), tab-separated values (TSV), and JSON data formats. All other file formats are not supported.

Although text-based files that use valid file formats are supported, they cannot contain column headers. You must remove the column headers from files that you want to store in Cloud Object Storage before you virtualize data.

About this task

For more information, see Getting started with Cloud Object Storage.

Authenticating for Watson Query object storage connections is standard for all supported platforms (Ceph, IBM Cloud Object Storage, Amazon S3).

Procedure

To get started, create an Amazon S3 connection.

  1. On the navigation menu, click Data > Data virtualization. The service menu opens to the Data sources page by default.

  2. Click Add connection > New connection to see a list of data sources that can be added to Watson Query.

  3. Select the Amazon S3 data source connection.

  4. Enter the connection name and description.

  5. To configure an Amazon S3 connection, you must provide values for the following parameters.
    • Bucket
    • Endpoint URL
    • Access key
    • Secret key

    To find the values for the Access key and Secret key, see Understanding and getting your AWS credentials.

  6. Click Create to add the connection to the data source environment.

Results

You can now use your Amazon S3 database as a data source in Watson Query.