Adding data to Data Refinery

After you’ve created a project and you’ve created connections or you’ve added data assets to the project, you can add data to Data Refinery and start prepping that data for analysis.

You can add data to Data Refinery in one of several ways:

  • Select Refine from the menu of a data asset in the project
  • Preview a data asset in the project and then choose to refine it
  • Navigate to Data Refinery first and then add data to it

Access Data Refinery from within a project. Click Add to project > Data Refinery flow.

If you already have a Data Refinery flow, you can go to the project’s Assets tab and click New Data Refinery flow in the Data Refinery flows section.

Add data

To add data after you navigate to Data Refinery:

  1. Select the data you want to work with from Data assets or from Connections.

    From Data assets:

    • Select a data file (the selection includes data files that have already been shaped with Data Refinery)
    • Select a connected data asset

    From Connections:

    • Select a connection and file
    • Select a connection, folder, and file
    • Select a connection, schema, and table or view

    Data Refinery supports Avro, CSV, JSON, Parquet, TSV (read only), or delimited text files.

    Data connections marked with a key icon (the key symbol for private connections) are locked. If you are authorized to access the data source you are asked to enter your personal credentials the first time you select it. This is a one-time step that permanently unlocks the connection for you. After you have unlocked the connection, the key icon is no longer displayed. See Adding connections to projects.

  2. Click Add to load the data into Data Refinery.

Tip: If your data doesn’t display in tabular form, specify the format of your data source. Go to the Data tab. Scroll down to the SOURCE FILE information at the bottom of the page. Click the “Specify data format” icon.

Next steps