Installing a Python module to set up Watson OpenScale

To automate the provisioning and configuration of the required IBM Cloud services and see an IBM Watson OpenScale application, including sample data, you can install a Python module.

About this module

  • The module provides an alternative way for technical users to see an instance of Watson OpenScale running without needing to provision and configure the services yourself, as described in the Getting started tutorial.
  • The Python module runs through the process of checking the services that you have and creating the ones that are necessary, including Watson OpenScale. After the module runs successfully, from the IBM Cloud dashboard you can start Watson OpenScale to see how it monitors a model.

Watson OpenScale Python Client

The Watson OpenScale Python Client is a Python library that works directly with the Watson OpenScale service. For development and automation purposes, you can use the Python client to directly configure the data mart database, bind your machine learning engine, and select and monitor deployments. For examples that use the Python client in this way, see the Watson OpenScale sample Notebooks.

 

Before you begin

  1. Create an IBM Cloud API key and download it. You need to enter the API key in a later step.

  2. Install any release of Python 3.

Python 3 includes the pip package management system.

  1. Install the ibm-watson-openscale-cli package by running the following command:

     pip install -U ibm-watson-openscale-cli
    

    If more than one version of pip is installed on your system, you might need to run pip3 instead of pip, as in, pip3 install -U ibm-watson-openscale-cli.

  2. If you have an existing Machine Learning service instance, check the IBM Cloud dashboard to ensure that the service is managed by Cloud Identity and Access Management (IAM), not Cloud Foundry.

Important: The module checks for an instance of Machine Learning. If you have an instance, the module uses it. But if your instance is managed by Cloud Foundry, you must first migrate it to an IAM resource group before you run the module.

Running the module

Run the following command:

ibm-watson-openscale-cli -a <Your API key>

Viewing results in the Watson OpenScale model monitor

To view insights into the fairness and accuracy of the model, details of data that is monitored, and explainability for an individual transaction, open the Watson OpenScale dashboard.

View insights

From the Watson OpenScale dashboard, click the Insights tab, which shows an overview of metrics for deployed models: Insights

  • At a glance, the Insights page shows any issues with fairness and accuracy, as determined by the thresholds that are configured.

  • Each deployment is shown as a tile. If you are following any one of the tutorials, in your dashboard, you see a deployment that is called GermanCreditRiskModel. The following sample shows a dashboard with many deployed and monitored models:

    Insight dashboard with many deployments, each shown as a tile

View monitoring data

  1. From the Insights page, click the GermanCreditRiskModel tile to view details about the monitored data.
  2. Slide the marker across the chart to view a day and time period that show data and then click the View details link.

    • For example, the following screen shows data for a specific date and time. The dates and times vary, depending on when you run the module.

    • For information about interpreting the time series chart, see Getting insights.

    Monitor data is shown

  3. To see details about AGE data monitoring, ensure that AGE is selected from the drop-down menu.
  • Notice that in the following screen capture, no bias exists.

  • For information about interpreting the chart of the data points at a specific hour, see Visualizing data for a specific hour.

    View details are shown

View explainability

To understand the factors that contribute when bias is present for a given time period, from the visualization screen that is shown in the previous section, select the View transactions button.

Transaction IDs for the past hour are listed for those transactions that have bias. For the model used in this module, no bias exists for requests that are available. Therefore, no transactions are shown for the time period in the following screen capture.

Transaction list with no transactions

For more information, see Explaining transactions.

Related information