Data governance tutorial: Consume your data

To preview this task, watch the video beginning at 01:12.

Data assets in catalogs are much more than pointers to data. They contain information about the format and meaning of the data and statistics about the data values. Follow these steps to understand the value of data assets:

1. From the Cloud Pak for Data navigation menu , choose Catalogs > View all catalogs.

2. Open the Mortgage Approval Catalog.

3. The featured assets section shows Recently added assets, assets that Watson recommends which are suggested assets from AI and machine learning based on your past usage and popularity, and Highly rated assets that catalog collaborators rated and reviewed.

4. Click Hide featured assets to close that section.

5. Search for mortgage.

6. Click MORTGAGE_APPLICANTS_TRUST to view that catalog asset. The Overview tab and the side panel provide basic information about the asset such as the description, a rating, tags, where the asset is located, business terms, data classes, and related items.

7. Click the Profile tab. The profile information helps you understand the content, the quality, and usability of the data.

8. Scroll to the right to locate the ZIP_CODE column.

9. The data class that was automatically assigned to the ZIP_CODE column is Commercial and Government Entity. Note that the automatically assigned data class may vary. Since the values are zip codes, you can easily reclassify this column. Click the drop-down list to see other possible data classes and their confidence levels. Select US Zip Code.

10. Click the Asset tab to see a preview of the data.

11. Return to the Overview tab to see more metadata about the columns. In the list of columns, search for the EMPLOYMENT_STATUS column to see the metadata including the assigned business terms.

To preview this task, watch the video beginning at 02:49.

You can make assets more valuable by adding information to them. For example, you can add your opinion of the asset, update asset properties, and create relationships to link assets. Follow these steps to enrich assets and create relationships:

1. For the MORTGAGE_APPLICANTS_TRUST catalog asset, click the Review tab. Rate and comment on this asset so that others can find the asset easily.

   1. Select 5 stars for the rating.

   2. For the review, copy and paste the following text:

      This contains high quality customer data from the mortgage system.
      

   3. Click Submit.

2. Click the Overview tab.

3. Click the Edit icon next to the asset name to edit the asset name.

   1. Change the name to:

      MORTGAGE_APPLICANTS_TRUST_PROTECT
      

   2. Click Apply.

4. In the Description section in the right side panel, click the Add .

   Note:

   If this asset has an existing description, you will see an Edit icon instead of an Add icon.

   1. Copy and paste the following description:

      Mortgage applicants from the Mortgage System
      

   2. Click Apply.

5. Because this asset relates to mortgage loans, next to Business terms, click the Add icon or the Edit icon.

   1. In the Search field, type loan.

      Note: It is not necessary to press Enter after typing the search term. You will see a list of results immediately after typing the search term.

   2. Select Loan.

   3. Click Save.

6. Because this asset contains personal information, next to Classifications, click Add icon or the Edit icon.

   1. Select Personally Identifiable Information.

   2. Click Save.

7. Because this asset is related to other mortgage assets, next to Related items, click Add related items > Add related assets.

   1. Select Is related to, and click Next.

   2. Select the CREDIT_SCORE and MORTGAGE_APPLICATION assets, and click Add.

8. Click MORTGAGE_APPLICATION to view that related asset.

To preview this task, watch the video beginning at 04:09.

The data analysts team needs the mortgage applicants data in the mortgage analysis project to refine, visualize, analyze, and use as training data for models. Follow these steps to add the enriched data to a project:

1. Click Mortgage Approval Catalog in the navigation trail.
   

2. At the end of the MORTGAGE_APPLICANTS_TRUST_PROTECT catalog asset row, click the Overflow menu, and choose Add to project.

   1. In the Target drop down list, select the Data governance project.

   2. Click Add.

3. When the notification displays, click Go to project. If you miss the notification, then:

   1. Click the Cloud Pak for Data navigation menu , choose Projects > View all projects.

   2. Click the Data governance project.

4. In the project, click the Assets tab to see the MORTGAGE_APPLICANTS_TRUST_PROTECT data asset.

To preview this task, watch the video beginning at 04:39.

You need to cleanse and refine the mortgage applicants data to get it ready for your analytical tools and models. A quick and easy way to determine how it needs to be shaped is to visualize the data in Data Refinery. The visualization is based on the first 5,000 rows of the data. Follow these steps to visualize the data:

1. Click the MORTGAGE_APPLICANTS_TRUST_PROTECT data asset to preview the data.

2. Click Prepare data to open the data asset in Data Refinery, and wait for the data to be read and processed.

3. In the About this asset panel, click the X to close the panel.

4. In the Steps panel, click the X to close the panel.

5. Click the Visualizations tab.

6. For the Column to visualize, select EMPLOYMENT_STATUS.

7. Click Visualize data. The tool selects a pie chart as the best chart type for this column, which shows the distribution of applicants by employment status. Notice the suggested chart types that are indicated by a blue dot next to bar, word cloud, and sunburst.

8. For the Chart type, select the Bubble chart type. The Bubble chart is one easy way to quickly visualize the distribution of values in a particular data set.

9. From the Chart type drop-down, select the Relationship chart type.

10. This chart type requires two columns. Select these columns:

    1. For the first column, select EMPLOYMENT_STATUS.

    2. Click Add another column.

    3. For the second Column, select EDUCATION.

11. With the Relationship chart, you can select endpoints to see the relationships. For example, you can see applicants employment status by level of education.

To preview this task, watch the video beginning at 05:59.

You can't process applicants without a social security number, so you need to review the data and remove any applicants without social security numbers. To prepare the MORTGAGE_APPLICANTS_TRUST_PROTECT data, you will:

• View the frequency of values in the Social_Security_Number column.
• Filter the applicants with missing values from the Social_Security_Number column.

Follow these steps to prepare the data:

1. In the Data Refinery, click the Profile tab.

2. Scroll to the right to locate the Social_Security_Number column. Notice several missing values.

3. Click the Data tab to filter out these records. In the status bar at the bottom of the screen, Data Refinery indicates that the FULL DATA SET is 1101 rows.

4. If the Steps panel is not visible, click Steps to open the panel.

5. Click New step.

   1. In the Cleanse section, select Filter.

   2. In the Column field, select the Social_Security_Number column.

   3. In the Operator field, select Is not empty.

   4. Click Apply. Notice in the status bar at the bottom of the screen, Data Refinery now indicates that the FULL DATA SET is 1000 rows because the rows with missing Social Security Numbers are filtered out. Notice that a new step displays in the Steps panel showing the Filter operation.

6. Click the Profile tab.

7. Scroll to the right to locate the Social_Security_Number column. Notice that the missing values are gone.

8. From the toolbar, click the Save icon.

9. From the toolbar, click the Export icon, and choose Export current data to CSV.
   

   1. Save the MORTGAGE_APPLICANTS_TRUST_PROTECT_shaped.csv to a local folder.

   2. Navigate to that folder, and open the CSV file, which contains 1000 rows and no applicants are missing the social security number.

10. Return to Cloud Pak for Data, and click the Data governance project in the navigation trail.
    

11. Click All assets, and locate the new Data Refinery flow asset with the name MORTGAGE_APPLICANTS_TRUST_PROTECT_flow.