Using custom components in a scikit-learn model

This topic demonstrates how to use custom transformers and estimators in a scikit-learn model that you deploy in IBM Watson Machine Learning as an online deployment.

 

Steps

  1. Package components
  2. Store your custom package and model

After the trained model is stored with the required custom package and runtime details, you can deploy models that use custom components as online deployments the same way you do for models that don’t use custom components.

 

Step 1: Package components

To use custom components in your model, you need to package your custom components in a Python distribution package.

See: Requirements for using custom components in your models

Sample

Here is a sample custom package for a scikit-learn model: linalgnorm-0.1.zip external link

 

Step 2: Store your custom package and model

After the model is trained, you must take extra steps when storing a model that uses custom components in the Watson Machine Learning repository:

  1. Store your custom package
  2. Create and store a runtime resource object
  3. Reference your stored runtime resource object when storing the model

See: Storing your custom package

 

Example

This example demonstrates using the Watson Machine Learning Python client external link to store a scikit-learn model that uses a custom transformer.

  1. Store your custom package in the Watson Machine Learning repository:

    pkg_meta = {
    client.runtimes.LibraryMetaNames.NAME        : "linalg_norm_skl",
    client.runtimes.LibraryMetaNames.DESCRIPTION : "A custom linalg transformer",
    client.runtimes.LibraryMetaNames.FILEPATH    : "linalgnorm-0.2.zip",
    client.runtimes.LibraryMetaNames.VERSION     : "0.2",
    client.runtimes.LibraryMetaNames.PLATFORM    : { "name": "python", "versions": ["3.6"] }
}
custom_package_details = client.runtimes.store_library( pkg_meta )
custom_package_uid = client.runtimes.get_library_uid( custom_package_details )

    Notes:

    • For client.runtimes.LibraryMetaNames.NAME, specify the value passed in the name parameter of the setup() function in the setup.py file.
    • For client.runtimes.LibraryMetaNames.FILEPATH, specify the .zip file name of your custom package.
    • You need the identifier of the stored package, custom_package_uid, for the next step. For the sake of example, assume the stored package identifier is 98d438a5-aec0-4c34-8e44-95f3d85e0315.

  2. Create a runtime resource object that references your stored custom package, and then store the runtime resource object in the Watson Machine Learning repository:

    runtime_meta = {
    client.runtimes.ConfigurationMetaNames.NAME: "K_linalg_gnfuv1",
    client.runtimes.ConfigurationMetaNames.DESCRIPTION: "skl linalg gnfuv model",
    client.runtimes.ConfigurationMetaNames.PLATFORM: {
        "name": "python",
        "version": "3.6"
    },
    client.runtimes.ConfigurationMetaNames.LIBRARIES_UIDS: ["98d438a5-aec0-4c34-8e44-95f3d85e0315"]
}
runtime_details    = client.runtimes.store( runtime_meta )
custom_runtime_uid = client.runtimes.get_uid( runtime_details )

    Notes:

    • For the sake of example, assume 98d438a5-aec0-4c34-8e44-95f3d85e0315 is the stored package identifier custom_package_uid from the previous step.
    • You need the identifier of the runtime resource object, custom_runtime_uid, for the next step. For the sake of example, assume the stored runtime resource identifier is f610d429-5037-4702-87ac-a40fd028bd96.

  3. Store your trained model in the Watson Machine Learning repository, referencing your stored runtime resource in meta data:

    model_meta = {
    client.repository.ModelMetaNames.NAME        : "linalg_norm gnfuv1 model",
    client.repository.ModelMetaNames.DESCRIPTION : "cust MNIST with pyfunc",
    client.repository.ModelMetaNames.RUNTIME_UID : "f610d429-5037-4702-87ac-a40fd028bd96"
}
model_details = client.repository.store_model( model=model_path, meta_props=model_meta )

    Notes:

    • For the sake of example, assume f610d429-5037-4702-87ac-a40fd028bd96 is the stored runtime resource identifier custom_runtime_uid from the previous step.

 

See also

For a full example, see this sample notebook: CustLib-SKL-Model-Save-Deploy-Score.ipynb external link