Updating notebooks from V1 to V2 Python SDK

The Watson OpenScale Python SDK version 2 update provides you with a more consistent and standard way of configuring monitors, handling datasets, and subscribing to machine learning engines. Because of the change from version 1 to version 2, there are several adjustments that must be made to notebooks so that they run in the new environment, including a new package name, ibm_watson_openscale. The following guide should help you update your notebooks. As the Watson OpenScale samples are updated, they are highlighted for use with version 2.

What is happening?

The Watson OpenScale Python SDK is being updated. The previous version of the SDK had separate microservices and APIs for each of the monitors. The update creates a single API that can be used for each of the pre-defined monitors, as well as the monitors you define.

Of particular note is the dataset API. The new method for interacting with a database is through the dataset UUID, which is connected to a particular dataset.

Why do I care?

The changes make it easier for you to define your own custom monitors while using the same classes and methods as pre-defined monitors, such as drift, quality, or explainability.

What do I need to do?

The following resources can help you with these updates:

Comparing V1 and V2 Watson OpenScale Python SDK

Because of the changes between version 1 and version 2 of the Python SDK, many of your notebooks don’t work with the new version. To continue to use your notebooks, you must update the code. The following code samples are meant to help you understand how to do make these updates.

How to authenticate

Previously, you might have authenticated by supplying either an isntance GUID and API key or your username and password as part of your service credentials. Now, you need to use service credentials and an authenticator.

For more information, see the API authenticator.

How to create subscriptions

Previously, you might have performed machine learning binding to connect to a machine learning provider. Now, you need to create a subscription to a machine learning provider.

The old V1 way:


subscription = ai_client.data_mart.subscriptions.add(WatsonMachineLearningAsset(
    model_uid,
    problem_type=ProblemType.BINARY_CLASSIFICATION,
    input_data_type=InputDataType.STRUCTURED,
    label_column=label_column_name,
    prediction_column=prediction_column_name',
    probability_column=probability_column_name,
    transaction_id_column=transaction_id_column_name,
    feature_columns = feature_columns_list,
    categorical_columns = categorical_columns_list
))

The new V2 way:

added_subscription_info = client.subscriptions.add(
        data_mart_id='997b1474-00d2-4g05-ac02-287ebfc603b5',
        service_provider_id='997b1474-00d2-4g05-ac02-287ebfc603b5',
        asset=Asset(...),
        deployment=AssetDeploymentRequest(...),
        asset_properties=AssetPropertiesRequest(...),
     )

For more information, see the Subscriptions class.

How to work with monitors

Previously, each Watson OpenScale monitor had its own API and set of controls to configure and run analysis. Now, use a standard API and specify which monitor.

The old V1 way:

subscription.fairness_monitoring.enable(
            features=[
                Feature("Sex", majority=['male'], minority=['female'], threshold=0.95),
                Feature("Age", majority=[[26,75]], minority=[[18,25]], threshold=0.95)
            ],
            favourable_classes=['No Risk'],
            unfavourable_classes=['Risk'],
            min_records=200,
            training_data=pd_data
        )

The new V2 way:

target = Target(
    target_type=TargetTypes.SUBSCRIPTION,
    target_id=subscription_id

)
parameters = {
    "features": [
        {"feature": "Sex",
         "majority": ['male'],
         "minority": ['female']
         },
        {"feature": "Age",
         "majority": [[26, 75]],
         "minority": [[18, 25]]
         }
    ],
    "favourable_class": ["No Risk"],
    "unfavourable_class": ["Risk"],
    "min_records": 4
}
thresholds = [
    {
        "metric_id": "fairness_value",
        "specific_values": [
            {
                "applies_to": [
                    {
                        "type": "tag",
                        "value": "Sex",
                        "key": "feature"
                    }
                ],
                "value": 0.95
            }
        ],
        "type": "lower_limit",
        "value": 0.95
    },
    {
        "metric_id": "fairness_value",
        "specific_values": [
            {
                "applies_to": [
                    {
                        "type": "tag",
                        "value": "Age",
                        "key": "feature"
                    }
                ],
                "value": 0.95
            }
        ],
        "type": "lower_limit",
        "value": 0.95
    }
]
fairness_monitor_details = wos_client.monitor_instances.create(
    data_mart_id=data_mart_id,
    background_mode=False,
    monitor_definition_id=wos_client.monitor_definitions.MONITORS.FAIRNESS.ID,
    target=target,
    parameters=parameters,
    thresholds=thresholds).result

For more information, see the ServiceProviders class.

How to read and write data

The old V1 way:

subscription.payload_logging.get_table_content(format='pandas')

The new V2 way:

pl_dataset_id = wos_client.data_sets.list(
    target_id=subscription_id,
    target_type="subscription",
    type="payload_logging"
).result.data_sets[0].metadata.id

wos_client.data_sets.show_records(data_set_id=pl_dataset_id)

or

records = wos_client.data_sets.get_list_of_records(data_set_id=pl_dataset_id, format=RecordsFormatTypes.LIST)

For more information, see the DataSets class.

Installing and running the tool

Use the following command to install the new tool:

pip install ibm-watson-openscale-cli-tool

Use the following command to run the tool:

ibm-watson-openscale-cli -a $CLOUD_API_KEY --env ypprod --mrm --history 7

Next steps

The preceding examples are just some of the ways that the Watson OpenScale Python SDK page has changed. Before running notebooks, it’s best to check over your code. The previous version of the SDK, which uses the ibm_ai_openscale package, is still available for reference here.