Importing metadata | IBM Cloud Pak for Data as a Service

Importing metadata

Last updated: Nov 26, 2024

Importing metadata

You can capture and import technical metadata and lineage information for the data in your organization. This data can be on a wide variety of data sources. When you import metadata, assets are created.

Required services: IBM Knowledge Catalog; Manta Data Lineage (to import lineage metadata); Cloud Object Storage (to import lineage metadata)
Required permissions: To create, manage, and run a metadata import, you must have the Admin or the Editor role in the project.; To import metadata into a catalog, you must also have the Admin or the Editor role in the catalog to which you want to import.; To configure Cloud Object Storage to store lineage metadata, you must have the Manage data lineage permission.
Supported connections: You can import assets from the data sources that are listed in Supported data sources for curation and data quality.

Types of metadata

You can import these types of metadata:

Technical metadata: Technical metadata provides the information that is required to create an asset in a project or catalog. Technical metadata provides asset details, relationships, and the preview of the contents of the asset. For data assets, the technical metadata also allows for data profiling, data quality analysis, and provides access for people to work with the data.
Lineage metadata: Lineage metadata provides the lineage information for the data lineage graph. Data lineage shows where your data comes from, how it changes, and where it moves over time.

Types of assets

You can create the following types of assets by importing metadata:

Data assets: Data tables or files from a connection. If you want to run metadata enrichment or data quality rules on the imported assets, you import them to a project.
Cobol copybooks: The data structure of a COBOL program. You can import Cobol copybooks into projects and catalogs. Such assets cannot be downloaded, profiled, enriched through metadata enrichment, or used in Data Refinery.
Transformation script assets: The data transformations that change the format, structure, or values of data and that usually are part of ETL (extract, transform, and load) processes.

Configuring data lineage

If you want to import lineage metadata, you must enable and configure data lineage.

Enable the data lineage feature in the IBM Knowledge Catalog service features settings. For exact steps, see Setting up the IBM Knowledge Catalog service: Enable data lineage.
Select a Cloud Object Storage instance to store lineage data:
1. From the Cloud Pak for Data navigation menu, go to Administration > Configurations and settings and open Data lineage setup.
2. Select an instance of Cloud Object Storage where you want to store lineage data. For more information about IBM Cloud Object Storage, see IBM Cloud Object Storage on Cloud Pak for Data as a Service.

Next steps

Plan your curation process.

Learn more

Parent topic: Curating data