0 / 0
AI governance use case
AI governance use case

AI governance use case

To manage data and model assets across the AI lifecycle, your enterprise needs integrated systems and processes. Cloud Pak for Data as a Service provides the processes and technologies to enable your enterprise to develop, deploy, maintain, and govern machine learning (ML) and AI models in production.

Watch this video to see the data fabric use case for implementing a AI governance solution in Cloud Pak for Data.

This video provides a visual method as an alternative to following the written steps in this documentation.

Challenges

Establishing AI governance solutions for enterprises involves tackling these challenges:

Accessing data
Organizations need to provide easy access to consolidated and governed data for data science teams who use the data to build trustworthy AI and ML applications.

Operationalizing models
Organizations need to implement repeatable processes to build and deploy models and adopt the models in production environments.

Ensuring model governance and compliance
Organizations need to monitor models in production to ensure that the models are explainable, valid, and accurate, and that they are not introducing bias or drifting away from the intended goals.

You can solve these challenges by implementing an AI governance lifecycle with data fabric on Cloud Pak for Data as a Service.


Example: Golden Bank's challenges

Follow the story of Golden Bank as it implements an AI governance process to expand its business by offering low-rate mortgage renewals for online applications. Data scientists at Golden Bank need to create a mortgage approval model that avoids unanticipated risk and treats all applicants fairly.

Process

To implement AI governance for your enterprise, your organization can follow this process:

  1. Share the data
  2. Build and train models
  3. Deploy models
  4. Monitor models
  5. Track models
  6. Automate the AI lifecycle

The Watson Studio, Watson Machine Learning, Watson OpenScale, and Watson Knowledge Catalog services in Cloud Pak for Data as a Service provide the tools and processes that your organization needs to implement a AI governance solution.

Image showing the flow of the AI governance use case

1. Share the data

The catalog serves as a feature store where your data scientist teams can find high-quality data assets with the features that they need. They can add data assets from a catalog into a project, where they collaborate to prepare, analyze, and model the data.

What you can use What you can do Best to use when
Catalogs Use catalogs in Watson Knowledge Catalog as a feature store to organize your assets to share among the collaborators in your organization.

Take advantage of AI-powered semantic search and recommendations to help users find what they need.
Your users need to easily understand, collaborate, enrich, and access the high-quality data.

You want to increase visibility of data and collaboration between business users.

You need users to view, access, manipulate, and analyze data without understanding its physical format or location, and without having to move or copy it.

You want users to enhance assets by rating and reviewing them.


Example: Golden Bank's catalog

The governance team leader creates a catalog, "Mortgage Approval Catalog" and adds the data stewards and data scientists as catalog collaborators. The data stewards publish the data assets that they created into the catalog. The data scientists find the data assets, curated by the data stewards, in the catalog and copy those assets to a project. In their project, the data scientists can refine the data to prepare it for training a model.


2. Build and train models

To get predictive insights based on your data, data scientists, business analysts, and machine learning engineers can build and train models. Data scientists use Cloud Pak for Data as a Service services to build the AI models, ensuring that the right algorithms and optimizations are used to make predictions that help to solve business problems.

What you can use What you can do Best to use when
AutoAI Use AutoAI in Watson Studio to automatically select algorithms, engineer features, generate pipeline candidates, and train model pipeline candidates.

Then, evaluate the ranked pipelines and save the best as models.

Deploy the trained models to a space, or export the model training pipeline that you like from AutoAI into a notebook to refine it.
You want an advanced and automated way to build a good set of training pipelines and models quickly.

You want to be able to export the generated pipelines to refine them.
Notebooks and scripts Use notebooks and scripts in Watson Studio to write your own feature engineering model training, and evaluation code in Python, or R based on training data sets that are available in the project, or connections to data sources such as databases, data lakes, or object storage.

Use your favorite algorithms and libraries.
You want to use Python or R coding skills to have full control over the code that is used to create, train, and evaluate the models.
SPSS Modeler flows Use SPSS Modeler flows in Watson Studio to create your own model training, evaluation, and scoring flows based on training data sets that are available in the project, or connections to data sources such as databases, data lakes, or object storage. You want a simple way to explore data and define model training, evaluation, and scoring flows.
RStudio Analyze data and build and test models by working with R in RStudio. You want to use a development environment to work in R.
Decision Optimization Prepare data, import models, solve problems and compare scenarios, visualize data, find solutions, produce reports, and save models to deploy with Watson Machine Learning. You need to evaluate millions of possibilities to find the best solution to a prescriptive analytics problem.
Federated learning Train a common model using distributed data. You need to train a model without moving, combining, or sharing data that is distributed across multiple locations.


Example: Golden Bank's model building and training

Data scientists at Golden Bank create a model, "Mortgage Approval Model" that avoids unanticipated risk and treats all applicants fairly. They want to track the history and performance of the model from the beginning, so they add a model entry to the "Mortgage Approval Catalog". They run a notebook to build the model and predict which applicants qualify for mortgages. The details of the model training are automatically captured as metadata in the model entry.


3. Deploy models

When operations team members deploy your AI models, the models become available for applications to use for scoring and predictions to help drive actions.

What you can use What you can do Best to use when
Spaces user interface (UI) Use the Spaces UI in Watson Machine Learning to deploy models and other assets from projects to spaces. When you prefer to use a UI.
Command-line tool (cpdctl) Use the cpdctl command-line tool in Watson Machine Learning to manage the lifecycle of models, including the configuration settings, and to automate an end-to-end flow that includes training the model, saving it, creating a deployment space, and deploying the model. You want to deploy and manage models to test or production environments from a command-line.


Example: Golden Bank's model deployment

The operations team members at Golden Bank promote the "Mortgage Approval Model" from the project to a deployment space and then creates an online model deployment.


4. Monitor deployed models

After models are deployed, it is important to govern and monitor them to make sure that they are explainable and transparent. Data scientists need to be able to explain how the models arrive at certain predictions so that they can determine whether the predictions have any implicit or explicit bias. In addition, it's a best practice to watch for model performance and data consistency issues during the lifecycle of the model.

What you can use What you can do Best to use when
Watson OpenScale Monitor model fairness issues across multiple features.

Monitor model performance and data consistency over time.

Explain how the model arrived at certain predictions with weight factors.

Maintain and report on model governance and lifecycle across your organization.
When you have features that are protected or that might contribute to prediction fairness.

You need to trace model performance and data consistencies over time.

You need to know why the model gives certain predictions.


Example: Golden Bank's model monitoring

Data scientists at Golden Bank use Watson OpenScale to monitor the deployed "Mortgage Approval Model" to ensure that it is accurate and treating all Golden Bank mortgage applicants fairly. They run a notebook to set up monitors for the model and then tweak the configuration by using the Watson OpenScale user interface. Using metrics from the Watson OpenScale quality monitor and fairness monitor, the data scientists determine how well the model predicts outcomes and if it produces any biased outcomes. They also gain an understanding of how the model comes to decisions so that the decisions can be explained to the mortgage applicants.


5. Track models

In addition to monitoring your models for fairness and explainability, your team needs to track the production models to ensure that they are performing well.

What you can use What you can do Best to use when
Factsheets In the model inventory in a catalog in Watson Knowledge Catalog, view lifecycle status for all of the registered assets and drill down to detailed factsheets for models or deployments registered to the model entry.

View general model details, training information and metrics, and input and output schema.

View general deployment details, evaluation details, quality metrics, fairness details, and drift details.
You want to make sure that your model is performing as expected.

You want to determine whether you need to make adjustments.


Example: Golden Bank's model tracking

Business analysts at Golden Bank request a "Mortgage Approval Model". They can then track the model through all stages of the AI lifecycle as data scientists build and train the model and ModelOps engineers deploy and evaluate it. Factsheets document details about the model history and generate metrics that show its performance.


6. Automate the ML lifecycle

Your team can automate and simplify the MLOps and AI lifecycle with Watson Pipelines.

What you can use What you can do Best to use when
Watson Pipelines Use pipelines to create repeatable and scheduled flows that automate notebook, Data Refinery, and machine learning pipelines, from data ingestion to model training, testing, and deployment. You want to automate any or all of the steps in an MLOps flow.


Example: Golden Bank's automated ML lifecycle

The data scientists at Golden Bank can also use pipelines to automate their complete AI governance lifecycle and processes to simplify the mortgage approval process.


Tutorials for AI governance

Tutorial Description Expertise for tutorial
Build and deploy a model Train a model, promote it to a deployment space, and deploy the model. Run a notebook.
Test and validate the model Evaluate a model for accuracy, fairness, and explainability. Run a notebook, and view results in user interface.

Learn more

Parent topic: Data fabric solution overview