0 / 0
AI governance tutorial: Build and deploy a model
AI governance tutorial: Build and deploy a model

AI governance tutorial: Build and deploy a model

This tutorial is the first in a series of two tutorials. Take this tutorial to build, deploy, and track a model with the AI governance use case of the data fabric trial. Your goal is to train a model to predict which applicants qualify for mortgages and then deploy the model for evaluation. You must also set up tracking for the model to document the model history and generate an explanation for its performance.

Quick start: If you did not already create the sample project for this tutorial, access the AI governance sample project in the gallery.

The following animated image provides a quick preview of what you’ll accomplish by the end of the second tutorial where you will use Watson OpenScale to configure and evaluate monitors for the deployed model to ensure that the model is accurate and treating all applicants fairly. Click the image to view a larger image.

Animated image

The story for the tutorial is that Golden Bank wants to expand its business by offering low-rate mortgage renewals for online applications. Online applications expand the bank’s customer reach and reduce the bank’s application processing costs. As a data scientist at Golden Bank, you must create a mortgage approval model that avoids unanticipated risk and treats all applicants fairly. You will run a Jupyter Notebook to build a model and automatically capture metadata that tracks the model in an AI Factsheet.

In this tutorial, you will complete these tasks:

  1. Set up tracking for your model.
  2. Create a model entry in the model inventory.
  3. Run the notebook to create the model.
  4. View the model's factsheet and associate it with a model entry.
  5. Deploy the model.

If you need help with this tutorial, you can ask a question or find an answer in the Cloud Pak for Data Community discussion forum.

Tip: For the optimal experience completing this tutorial, open Cloud Pak for Data in one browser window, and keep this tutorial page open in another browser window to switch easily between the two applications. Consider arranging the two browser windows side-by-side to make it easier to follow along.

Side-by-side tutorial and UI

Preview the tutorial

Watch Video Watch this video to preview the steps in this tutorial. There might be slight differences in the user interface shown in the video. The video is intended to be a companion to the written tutorial.

This video provides a visual method as an alternative to following the written steps in this documentation.

Tip: Start the video, then as you scroll through the tutorial, the video moves to picture-in-picture mode. Close the video table of contents for the best experience with picture-in-picture. You can use picture-in-picture mode so you can follow the video as you complete the tasks in this tutorial. Click the timestamps for each task to follow along.

The following animated image shows how to use the video picture-in-picture and table of contents features:

How to use picture-in-picture and chapters

Prerequisites

Sign up for Cloud Pak for Data as a Service

You must sign up for Cloud Pak for Data as a Service and provision the necessary services for the AI governance use case.

  • If you have an existing Cloud Pak for Data as a Service account, then you can get started with this tutorial. If you have a Lite plan account, only one user per account can run this tutorial.
  • If you don't have a Cloud Pak for Data as a Service account yet, then sign up for a data fabric trial.

Verify the necessary provisioned services

To preview this task, watch the video beginning at 01:06.

Important: Model inventory is available in the Dallas region only. If necessary, switch to the Dallas region before continuing.

Follow these steps to verify or provision the necessary services:

  1. In Cloud Pak for Data, verify that you are in the Dallas region. If not, click the region drop down, and then select Dallas.
    Change region

  2. From the Cloud Pak for Data navigation menu Navigation menu, choose Services > Service instances.

  3. Use the Product drop-down list to determine whether there is an existing Watson Studio service instance.

  4. If you need to create a Watson Studio service instance, click Add service.

  5. Select Watson Studio.

    1. For the region, select Dallas.

    2. Select the Lite plan.

    3. Click Create.

  6. Repeat these steps to verify or provision the following additional services:

    • Watson Machine Learning
    • Watson Knowledge Catalog
    • Cloud Object Storage

Checkpoint for Provisioned services Check your progress

The following image shows the provisioned service instances:

Provisioned services

Create the sample project

To preview this task, watch the video beginning at 01:48.

If you did not already create the sample project for this tutorial, follow these steps:

  1. Access the AI governance sample project in the gallery.

  2. Click Create project.

  3. If prompted to associate the project to a Cloud Object Storage instance, select a Cloud Object Storage instance from the list.

  4. Click Create.

  5. Wait for the project import to complete, and then click View new project to verify that the project and assets were created successfully.

    Note: If this occasion is your first time accessing a project, you see a guided tour asking if you want a tour of projects. For now, click Maybe later.
  6. Click the Assets tab to view the assets in the sample project.

Note: You might see a guided tour showing the tutorials that are included with this use case. The links in the guided tour will open these tutorial instructions.

Checkpoint for Sample project Check your progress

The following image shows the sample project. You are now ready to start the tutorial.

Sample project

Tip: If you encounter a guided tour while completing this tutorial in the Cloud Pak for Data user interface, click Maybe later.

Task 1: Set up tracking for your model

To preview this task, watch the video beginning at 02:46.

You track models by adding model entries to a catalog. With the Watson Knowledge Catalog Lite plan, you can create two catalogs. Follow these steps to see whether you already have a catalog and, if not, to create a catalog to store the model entry:

  1. From the Cloud Pak for Data navigation menu Navigation menu, choose Catalogs > View all catalogs.

  2. If you see a catalog on the Catalogs page, then skip to Task 2: Create the model entry. Otherwise, follow these steps to create a new catalog:

  3. Click Create Catalog.

  4. For the Name, copy and paste the catalog name exactly as shown with no leading or trailing spaces:

    Mortgage Approval Catalog
    
  5. If prompted to associate the catalog to a Cloud Object Storage instance, select a Cloud Object Storage from the list.

  6. Select Enforce data protection rules, confirm the selection, and accept the defaults for the other fields.

  7. Click Create.

Checkpoint Check your progress

The following image shows your catalog. You are now ready to create the model entry that is stored in the catalog.

Mortgage Approval Catalog

Task 2: Create the model entry in the model inventory

To preview this task, watch the video beginning at 3:17.

For this type of project, it is best to create the model entry when a project commences. A model entry can reference multiple machine learning models that you can use to solve business problems. Then, data engineers and model evaluators can add models to the model entry and track the model as it progresses through its lifecycle. Follow these steps to create the model entry:

Tip: If this occasion is your first time accessing the Model inventory, you see a guided tour asking if you want to set up model governance. For now, click Maybe later.
  1. From the Cloud Pak for Data navigation menu Navigation menu, choose Catalogs > Model inventory.

  2. Click New model entry.

  3. For the Model entry name, copy and paste the name exactly as shown with no leading or trailing spaces:

    Mortgage Approval Model Entry
    
  4. For the Description, copy and paste the following text:

    This model entry is for the Mortgage approval use case at Golden Bank
    
  5. If you have more than one catalog, then you see a Catalog field. Select Mortgage Approval Catalog or a different existing catalog.

  6. Click Save.

Checkpoint for Mortgage Approval Model Entry Check your progress

The following image shows your model entry. The model entry is now ready for data engineers and model evaluators to add models and track models as they progress through their lifecycle. The next task is to run the notebook to create the model.

Mortgage Approval Model Entry

Task 3: Run the notebook to create the model

To preview this task, watch the video beginning at 04:00.

Now you are ready to run the first notebook included in the sample project. The notebook includes the code to:

  • Set up AI Factsheets used to track the lifecycle of the model.
  • Load the training data.
  • Specify the target, categorical, and numerical columns along with the thresholds used to build the model.
  • Build data pipelines.
  • Build machine learning models.
  • View the model results.
  • Save the model.

Follow these steps to run the notebook included in the sample project. Take some time to read through the comments in the notebook, which explain the code in each cell.

  1. From the Cloud Pak for Data navigation menu Navigation menu, choose Projects > View all projects.

  2. Click the AI governance project name.

  3. Click the Assets tab, and then navigate to Notebooks.
    Left navigation

  4. Open the 1-model-training-with-factsheets notebook.

  5. Since the notebook is in read-only mode, click the Edit Edit icon icon to place the notebook in edit mode.

  6. When you import a project from the Gallery, the first cell of the notebook contains the project access token. If this notebook does not contain a first cell with a project access token, you need to generate the token. From the More menu, select Insert project token. This action inserts a new cell as the first cell in the notebook containing the project token.

  7. Under the Provide your IBM Cloud API key section, you need to pass your credentials to the Watson Machine Learning API using an API key. If you don't already have a saved API key, then follow these steps to create an API key.
    To preview this task, watch the video beginning at 04:55.

    1. Access the IBM Cloud console API keys page.

    2. Click Create an IBM Cloud API key.

    3. Type a name and description.

    4. Click Create.

    5. Copy the API key.

    6. Download the API key for future use.

    7. Return to the notebook, and paste your API key in the ibmcloud_api_key field.

  8. To run all the cells in the notebook, click Cell > Run All. Alternatively, you can run the notebook cell by cell if you want to explore each cell and its output.

  9. The notebook takes 1 - 3 minutes to complete. You can monitor the progress cell by cell, noticing the asterisk "In [*]" changing to a number, for example, "In [1]".

  10. If you encounter any errors during the notebook run, try these tips:

    • Click Kernel > Restart & Clear Output to restart the kernel, and then run the notebook again.
    • Verify that you created the model entry by copying and pasting the specified artifact name exactly with no leading or trailing spaces.

Checkpoint for Notebook run complete Check your progress

The following image shows the notebook when the run is complete. The notebook saved the model in the project, so you are now ready to view and add it to the model inventory.

Notebook run complete

Task 4: View the model's factsheet and associate it with a model entry

To preview this task, watch the video beginning at 06:24.

After running all the cells in notebook, follow these steps to view the model's factsheet in the project and then associate that model with a model entry in the model inventory:

  1. Click the AI governance project name in the navigation trail.
    Navigation trail

  2. Click the Assets tab, and then navigate to Models.

  3. Click the Mortgage Approval Prediction Model asset name that you previously created to open the model.

  4. Review the AI Factsheet for your model. AI Factsheets capture model metadata across the model development lifecycle, facilitating subsequent enterprise validation or external regulation. AI Factsheets enables model validators and approvers to get an accurate, always up-to-date view of the model lifecycle details.
    In the last task, you ran a notebook containing the AI Factsheets Python client code in the notebook that captured training metadata. Scroll to the Training metrics and Training tags sections to review the captured training metadata.
    Checkpoint The following image shows the AI Factsheet for the model:

    Model's AI Factsheet

  5. Scroll up on the model page, and click Track this model.

    1. Select Select an existing model entry.

    2. From the list of model entries, select Mortgage Approval Model Entry.

    3. Click Track.

  6. Back on the model page, click Open in model inventory.

  7. On the model entry page, click the Asset tab.

  8. Under Model tracking, you can see that AI Factsheets track models through their lifecycle. This model is still in the Develop stage as it has not been deployed yet.

Checkpoint for Model entry in Develop phase Check your progress

The following image shows the model entry with the model in the Develop phase. Now that you reviewed metadata such as the training data source, training metrics, and input schema that was captured in the AI Factsheet, you are ready to deploy the model.

Model entry in Develop phase

Task 5: Deploy the model

Before you can deploy the model, you need to promote the model to a new deployment space. Deployment spaces help you to organize supporting resources such as input data and environments; deploy models or functions to generate predictions or solutions; and view or edit deployment details.

Promote the model to a deployment space

To preview this task, watch the video beginning at 07:48.

Follow these steps to promote the model to a new deployment space:

  1. From the model entry, under the Develop phase, click Mortgage Approval Prediction Model.

  2. Click Open in project to open the model in the AI governance project.

  3. On the model page, click Promote to deployment space.

  4. For the Target space, select Create a new deployment space.

    1. For the deployment space name, copy and paste the name exactly as shown with no leading or trailing spaces:

      Golden Bank Preproduction Space
      
    2. Select a storage service from the list.

    3. Select your provisioned machine learning service from the list.

    4. Click Create.

    5. Click Close.

  5. For the Target space, ensure that Golden Bank Preproduction Space is selected.

  6. Check the Go to model in the space after promoting it option.

  7. Click Promote.

Checkpoint for Model in deployment space Check your progress

The following image shows the model in the deployment space. You are now ready to create a model deployment.

Model in deployment space

Create an online deployment for the model

To preview this task, watch the video beginning at 08:43.

Follow these steps to create an online deployment for your model:

  1. When the deployment space opens, click New deployment.

    1. For the Deployment type, select Online.

    2. For the Name, copy and paste the deployment name exactly as shown with no leading or trailing spaces:

      Mortgage Approval Model Deployment
      
    3. For the Serving name, you can specify a descriptive name to use in place of the deployment ID that will help you to identify this deployment quickly. Copy and paste the serving name with no leading or trailing spaces. The name is validated to be unique per region. If this serving name already exists, then add a number (or any unique character) to the end of the serving name.

      mortgage_approval_service
      
    4. Click Create.

  2. The model deployment may take several minutes to complete. When the model is deployed successfully, return to the model inventory; From the navigation menu Navigation menu, choose Catalogs > Model inventory.

  3. For the Mortgage Approval Model Entry, click View details.

  4. Click the Asset tab. Under Model tracking, you can see that the model is now in the Deploy stage.

Checkpoint for Model entry in Deploy phase Check your progress

The following image shows the model entry with the model in the Deploy phase. Your model is now ready for you to evaluate in Watson OpenScale.

Model entry in Deploy phase

As a data scientist at Golden Bank, you created a mortgage approval model by running a Jupyter Notebook that built the model and automatically captured metadata to track the model in an AI Factsheet. You then promoted the model to a deployment space, and deployed the model.

Next steps

You are now ready to validate and monitor your deployed machine learning model to ensure it is working accurately and fairly. For this task, you will use Watson OpenScale. See the Test and validate the model tutorial.

Learn more

Parent topic: Data fabric tutorials