IBM offers an end-to-end model risk management solution with IBM Watson OpenScale and IBM OpenPages with Watson. IBM OpenPages MRG offers model risk governance to store and manage a comprehensive model inventory. IBM Watson OpenScale evaluates outcomes from AI Models across its lifecycle and validates models. What is IBM OpenPages Model Risk Governance (MRG)? For more information, see the IBM OpenPages with Watson product page and the IBM OpenPages online help.

Setup options

You can use one of the following options to set up your initial environment:

• Automated setup

  Takes just a few minutes to create a workable system with sample data on a publically hosted lite system

• Notebook setup

  Controls some of the setup parameters, which can be used later for managing your own models

• Manual setup

  Gives you the most control over resources and options

Automated setup

The auto setup option can be run when you start the Watson OpenScale service for the first time.

Prerequisites

To perform model evaluation, you must have an IBM Cloud instance and you must provision an IBM Watson OpenScale instance.

Steps

1. Launch Watson OpenScale.

   1. From the IBM Cloud Dashboard, click Services.
   2. Click Watson OpenScale.
   3. Click Launch Application.

2. When prompted about running automatic setup, click Auto setup.

Set up your environment by using a Python Notebook

To get started before you use the model risk management features, set up the following services on IBM Cloud:

• Watson OpenScale, which provides MRM features and metrics
• IBM Watson Machine Learning (two separate instances, one for pre-prod and one for prod), which provides the engine for creating predictive models. This tutorial shows how to use IBM Watson Machine Learning as an engine, but you can also use any other supported machine learning engine)
• Watson Studio, which runs Notebooks and secures assets. (This tutorial shows how to use Watson Studio to create the provided sample models, but you can also use any other IDE to build models.)
• [Optional] Cloud Object Storage, which gives you a place to store model assets, such as training dat1. However, for the tutorial, you can use an internal database. You might want to set up Cloud Object Storage for your own work.

For your instance of IBM OpenPages, you must have login credentials to use on an already provisioned system. It is here that you manage models and view the metrics that are generated by model evaluation.

Notebook file

In addition to the preceding services, you must also have the OpenScale and OpenPages model risk management on IBM Cloud.ipynb Notebook file, which you can download from the https://github.com/IBM/watson-openscale-samples GitHub repository.

Create an IBMid and IBM Cloud account

In case you don’t have an IBM Cloud account yet, you must create one.

1. Point your web browser to the following URL: https://cloud.ibm.com/registration
2. Follow instructions to create an IBMid and IBM Cloud Account.

Add services to your IBM Cloud account

As soon as you have an IBM Cloud account, you can use the dashboard to add the necessary services. For each service, you can choose the Lite or Free plan. You must have instances for the following services: Watson OpenScale, Watson Studio, and IBM Watson Machine Learning (two Instances).

1. From the navigation menu, click Resource.
2. Click Create resource.
3. Search for each of the necessary services by entering keywords, such as OpenScale, studio, or machine learning.

You might not be able to add two Lite plan instances of IBM Watson Machine Learning to your account. To create the second instance, use the IBM Watson Machine Learning Standard plan. Or, create a second IBM Cloud Account that is linked to a separate email address to create the second Watson Machine Learning instance (Link to register the second account: https://cloud.ibm.com/registration).

Add a Cloud Object Storage instance

Use Cloud Object Storage to store training dat1. After you create an instance Cloud Object Storage, Watson Studio, IBM Watson Machine Learning, and Watson OpenScale will be able to access the buckets that are created as part of the model creation process.

1. Use your primary IBMid to log in to your IBM Cloud account.
2. From the IBM Cloud Dashboard, click Add resource, then click Storage.
3. Click the Object Storage tile, select the Lite plan, then click Create.

Manual setup

You can manually set up your model risk management features by completing the following steps. To successfully complete the steps, you must have detailed information about your machine learning provider, the database, and the data that is used for monitoring.

Steps

1. Provision prerequisite IBM Cloud services. You must set up two instances of IBM Watson Machine Learning.
2. Set up Watson Studio projects. You must set up projects for both pre-production and production models.
3. Configure Watson OpenScale.

4. Set up model governance by entering the IBM OpenPages credentials and associating an OpenPages model to your Watson OpenScale model.

   1. From the Insights dashboard, click Configure > Integrations > Edit to add credentials.
   2. From the model configuration, click Configure > Model governance > Edit to associate an OpenPages model.
5. Next steps: Continue to set up the monitors and data logging.

Next steps

• Model risk management and model governance
• Model governance