Preparing data for IBM Manta Data Lineage

Last updated: Nov 07, 2024

To add data to the lineage repository, you need to select Cloud Object Storage instance,create data source definition, and create metadata import.

Required permission

You must have the following user permission:

Manage data lineage

Prerequisites

The data lineage capability is not available by default. You must install the IBM Knowledge Catalog service with the IBM Manta Data Lineage service enabled. For more information how to enable data lineage, see Enable data lineage.

You need a project to store the imported metadata for the data assets. For more information, see Creating a project.

Data lineage setup

Select Cloud Object Storage instance to store data lineage metadata. You can select your storage instance only once. You can't change it later. Make sure that object storage is configured to allow users to create catalogs and projects. See Setting up IBM Cloud Object Storage for use with Cloud Pak for Data as a Service.

To define storage:

Go to the Configurations and settings or Data lineage page and click Data lineage setup.
Select your storage from the list and save your changes.

Preparing data to populate lineage repository

Before viewing lineage, you need to populate your data lineage repository the following way:

Create a data source definition and a connection.

A data source definition is an asset that functions as a unique stable identifier for the location of a data source such as a relational database. Data source definitions use endpoints to identify the data source. For most data source types, an endpoint is the combination of the hostname or IP address, the port number, and the database name or instance identifier. For more information and a procedure, see Creating a data source definition from the Data source definition list.

A connection is used to connect to the external data source. See, Adding platform connections. To view a list of supported connectors for data lineage, see Supported data sources for data lineage.

The connection assignment to a data source definition is done automatically. When creating connection first and, then, a data source definition, the assignment might take a longer time.

Navigate to your project and create metadata import. For more information, see Creating a metadata import asset and importing metadata.
After successful metadata import job, go to Data > Data lineage > View lineage tab to check if your data is visible in the repository tree.

Learn more

Parent topic: Data lineage