Masking data with Masking flow
Masking flow allows data administrators to produce masked copies of data for data scientists, business analysts, and application testers. Data is protected with data protection rules that apply automatically to all data imported to the catalog.
Masking flow also introduces advanced masking options for data protection rules, such as enhanced format preservation, one-way hash tokenization, reversible encryption, ability to maintain relationships, and to increase utility of masked data. Data protection rules with advanced masking work only in projects.
Required services Watson Knowledge Catalog
Data format Relational: Tables in relational data sources
Data size Any size
Before creating masking flows, the data admin must complete these prerequisite tasks.
After the prerequisite tasks are completed, both data admins and data users can do one of the following tasks:
- Create a new project and add data assets to be masked in the project.
- Choose an existing project with data assets.
After completing one of the tasks, click New asset > Masking flow.
User roles in Masking flow
As a data administrator (or data engineer), you have a strong knowledge of data assets and data requirements of the data users. You are responsible for preparing data for masking and configuring data user access to masked data. See the tasks that data admins must complete.
As a data user, such as data scientists, business analysts, testers, and developers, you rely on the data admin to curate and provide protected data that you need to do your work. See the tasks that data users can do.
Supported data sources
Masking flow supports the following relational and non-relational data sources:
- Apache Hive
- Db2 LUW
- Db2 Warehouse
- SQL Server
Prerequisite tasks for data admins
You must be an IBM Cloud account administrator.
At the time that Masking flow is installed, there is at least one admin account set up in your organization. This admin can give other users admin access.
Prepare for privatizing data by completing the following tasks:
Add data assets to catalogs by automatically importing data assets with metadata. You create connections to your data in the metadata catalog. When importing the data assets, select the catalog that is created in the previous step as the import target. See Publishing assets from a project into a catalog.
Setting up data protection rules. Data protection rules apply to all governed catalogs and are enforced by Masking flow when you create masked copies of data by using masking flows. Advanced data masking options are only enabled for data classes.
- Managing user access by adding users to an IBM Cloud Account and setting up Cloud Pak for Data as a Service for your organization.
- Adding data users to catalogs by managing access to a catalog.
Avoiding unintentional data leakage
Moving assets from catalogs to projects
By default, data protection rules are not enforced for the asset owner, the user who added the asset to the catalog. This means for the asset owner, catalog previews are not protected for the data assets that they own.
- When you move an asset from a catalog to a project, the asset in the project is a copy of the catalog asset. Project assets are not linked to data protection rules.
- If the person moving the asset is the asset owner, the asset preview is unmasked for all users in the project.
- If the person moving the asset is not the asset owner, the asset preview is masked for all users in the project.
Because data protection rules aren't enforced for asset owners, when asset owners run a masking flow, the data copy loaded to a target database is not masked. Data is only masked when data users run the masking flow.
Best practice to avoid unintentional data leakage
Consider the following best practices to avoid data leakage:
The project used by the admin to import metadata to the catalog should not be used for masking flows. If you want to use the same project for metadata imports and masking flows, ensure that all users in the project have permissions to see unmasked data.
Data admins should not move data from catalogs to projects for creating masking flows. Data admins should add data users as viewers to the catalog, and then only data users should move data from the catalog to the project. They can optionally add other users to the project.
Prerequisite tasks for data users
Data users must already be a member of the platform or have the level of permission for the data scientist role.
Required permissions You must have an IBM Cloud account and be entitled to Watson Knowledge Catalog Lite plan.
- Data admins can give you Editor or Viewer access to catalogs.
- Data admins or other data users can also give you access to individual projects that they create.
Prepare masked data copies by completing the following tasks: