Installing a Python module to set up Watson OpenScale
To automate the provisioning and configuration of the required IBM Cloud services and see an IBM Watson OpenScale application, including sample data, you can install a Python module.
About this module
- The module provides an alternative way for technical users to see an instance of Watson OpenScale running without needing to provision and configure the services yourself, as described in the Getting started tutorial.
- The Python module runs through the process of checking the services that you have and creating the ones that are necessary, including Watson OpenScale. After the module runs successfully, from the IBM Cloud dashboard you can start Watson OpenScale to see how it monitors a model.
Watson OpenScale Python Client
The Watson OpenScale Python Client is a Python library that works directly with the Watson OpenScale service. For development and automation purposes, you can use the Python client to directly configure the data mart database, bind your machine learning engine, and select and monitor deployments. For examples that use the Python client in this way, see the Watson OpenScale sample Notebooks.
Before you begin
Create an IBM Cloud API key and download it. You need to enter the API key in a later step.
Python 3 includes the pip package management system.
ibm-watson-openscale-clipackage by running the following command:
pip install -U ibm-watson-openscale-cli
If more than one version of pip is installed on your system, you might need to run
pip, as in,
pip3 install -U ibm-watson-openscale-cli.
If you have an existing Machine Learning service instance, check the IBM Cloud dashboard to ensure that the service is managed by Cloud Identity and Access Management (IAM), not Cloud Foundry.
Important: The module checks for an instance of Machine Learning. If you have an instance, the module uses it. But if your instance is managed by Cloud Foundry, you must first migrate it to an IAM resource group before you run the module.
Running the module
Run the following command:
ibm-watson-openscale-cli -a <Your API key>
Viewing results in the Watson OpenScale model monitor
To view insights into the fairness and accuracy of the model, details of data that is monitored, and explainability for an individual transaction, open the Watson OpenScale dashboard.
- To understand the scenario for the sample data, read Use case and the value of Watson OpenScale.
From the Watson OpenScale dashboard, click the Insights tab, which shows an overview of metrics for deployed models:
At a glance, the Insights page shows any issues with fairness and accuracy, as determined by the thresholds that are configured.
Each deployment is shown as a tile. If you are following any one of the tutorials, in your dashboard, you see a deployment that is called
GermanCreditRiskModel. The following sample shows a dashboard with many deployed and monitored models:
View monitoring data
- From the Insights page, click the
GermanCreditRiskModeltile to view details about the monitored data.
Slide the marker across the chart to view a day and time period that show data and then click the View details link.
For example, the following screen shows data for a specific date and time. The dates and times vary, depending on when you run the module.
For information about interpreting the time series chart, see Getting insights.
- To see details about
AGEdata monitoring, ensure that
AGEis selected from the drop-down menu.
Notice that in the following screen capture, no bias exists.
For information about interpreting the chart of the data points at a specific hour, see Visualizing data for a specific hour.
To understand the factors that contribute when bias is present for a given time period, from the visualization screen that is shown in the previous section, select the View transactions button.
Transaction IDs for the past hour are listed for those transactions that have bias. For the model used in this module, no bias exists for requests that are available. Therefore, no transactions are shown for the time period in the following screen capture.
For more information, see Explaining transactions.