Deploying a model

The basic IBM Watson Machine Learning model-development workflow is: design, train, store, deploy. After you build and train a model, you deploy it, and put it into production, so you can pass data to the model and get scoring data, also known as predictions, back. This topic lists the methods for deploying a model in your Watson Machine Learning service.

Although it is possible to score a model without deploying it, a model has to be deployed before it can be scored from the Watson Machine Learning APIs. Also, a model can only have a single deployment. For a trial account, a user can have only one deployed model at a time.

Deployment types

There are three types of deployment, each suited to a specific use case.

Table 1. Deployment types
Deployment type Use this deployment type when you want to...
Online web service Access your deployment through an API endpoint, using the Watson Machine Learning Python client external link, CLI, or REST API external link (manually or in an application) to analyze data.
Batch Perform bulk analysis of data in IBM Cloud Object Storage external link or IBM Db2 Warehouse on Cloud external link
Core ML Download a model to use in a Core ML application.

For a list of deployment types supported by framework, see: Supported machine learning frameworks

 

Interfaces

You can deploy your model using different interfaces, depending on your scenario and preference.

Table 2. Deployment interfaces
Interface Scenario
project If you are working in a project already, using the Deployments tab of your project is quick and simple.
Watson Machine Learning Python client If you are working in a local Python development environment or in a Python notebook, or if you are developing a Python app, you can deploy models using the Python client.
Watson Machine Learning REST API For ultimate flexibility, you can deploy a model by using the REST API. For example, you can deploy a model from your local enviornment without having to install the Watson Machine Learning CLI. And you can deploy a model from an app in any language without having to import a Watson Machine Learning client library.
Watson Machine Learning CLI If you are working in a local environment, creating automation, or simply prefer a command line interface, you can deploy models from the command line.

 

project

After you train and store your model in a Watson Machine Learning repository that is associated with a project, you can deploy your model in that Watson Machine Learning service from the ACTIONS menu beside the model in the Assets tab of the project:

Deploying from project

Options available to you when you use Watson Machine Learning to deploy and test your model will vary depending on the kind of model you are deploying. The following steps are for a Spark ML model.

To create a Web service deployment:

  1. Click Add Deployment to create a deployment.
  2. Enter a name and description for the deployment.
  3. Choose Web service as the deployment type and click Save.
  4. Click the deployment name to view and test the deployment.
  5. On the Test page, enter test data and click Predict to view predictions.
  6. Click Implementation to view details of the deployment such as the endpoint URL and authentication details. This information is required to interact with the model programmatically or embed it in an app.

Passing payload data to Web service deployments

You can pass payload data to a deployed model. How you structure payload data depends on the type of model you are building. For example, your payload data for an AutoAI model would be structured like this:

{"input_data":[{
        "fields": ["PassengerId","Pclass","Name","Sex","Age","SibSp","Parch","Ticket","Fare","Cabin","Embarked"],
        "values": [[1,3,"Braund, Mr. Owen Harris",0,22,1,0,"A/5 21171",7.25,null,"S"]]
}]}

But, the same data for a Model Builder model would be structured like this:

{
    "fields": ["PassengerId","Pclass","Name","Sex","Age","SibSp","Parch","Ticket","Fare","Cabin","Embarked"],
    "values": [[1,3,"Braund, Mr. Owen Harris",0,22,1,0,"A/5 21171",7.25,,"S"]]
}

For guidance on how to structure your payload data, view the code samples on the Implementation tab for your deployment.

Creating a Batch prediction deployment

A batch deployment processes a series of inputs and returns corresponding predictions.

Prerequisites

Before you begin, depending on the type of data and model, you must have the following resources:

  1. Click Add Deployment to create a deployment.
  2. Enter a name and description for the deployment.
  3. Choose Batch prediction as the deployment type.
  4. Select a runtime environment from the list.
  5. Specify repositories for the test data that will be passed to the deployment and for storing the resulting predictions and click Save. On IBM Cloud, you can store your test data as an asset in your project. You can also store test data in a Cloud Object Storage bucket and specify a second bucket for storing the results. For information on working with Cloud Object Storage buckets, see Creating a bucket
  6. Click the deployment name to view the deployment details and schedule the batch job.
  7. Click Implementation to view details of the deployment such as the endpoint URL and authentication details. This information is required to interact with the model programmatically or embed it in an app.

Watson Machine Learning Python client

How you interact with a Watson Machine Learning model depends on the type of model you are deploying. The Implementation tab of your model’s deployment page will have a code sample for deploying your specific model type.

V3 Python client

Most of the Watson Machine Learning models use a Python client based on Watson Machine Learning Python client library reference, version 3. For details on the deployment syntax, refer to Watson Machine Learning Python client external link.

For an example of deploying a model using the v3 Python client, see Watson Machine Learning Python client tutorial.

V4 Python client

AutoAI and Decision Optimization are built using the Watson Machine Learning Python client library reference, version 4, which has a different syntax than the version 3 APIs used for Model Builder and other Watson Machine Learning models. To interact programmatically with an AutoAI deployment, refer to the deployment syntax in Watson Machine Learning Python client library external link.

For an example of deploying a model using the v4 Python client, see Python Client Example

Example Python code

The Implementation tab of your model’s deployment page will have a code sample for deploying your model.

Python client v3 This code sample shows how to deploy a model created using:

  • Model Builder
  • NeuNetS
  • SPPS Modeler
  • Spark ML Modeler
  • Neural Network Modeler
  • Experiment Builder
import urllib3, requests, json

# NOTE: generate iam_token and retrieve ml_instance_id based on provided documentation	
header = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + iam_token, 'ML-Instance-ID': ml_instance_id}

# NOTE: manually define and pass the array(s) of values to be scored in the next line
payload_scoring = {"fields": ["GENDER", "AGE", "MARITAL_STATUS", "PROFESSION"], "values": [array_of_values_to_be_scored, another_array_of_values_to_be_scored]}

response_scoring = requests.post('https://us-south.ml.cloud.ibm.com/v3/wml_instances/9a6ae85e-5f7f-43ba-9ede-f6ea9fc832ae/deployments/b5a2d435-bbb4-42b3-b5ff-a4609a21cb1c/online', json=payload_scoring, headers=header)
print("Scoring response")
print(json.loads(response_scoring.text))

Python client v4 This Python code sample shows you how to deploy a model built using AutoAI or Decision Optimization.

import urllib3, requests, json

# NOTE: generate iam_token and retrieve ml_instance_id based on provided documentation	
header = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + iam_token, 'ML-Instance-ID': ml_instance_id}

# NOTE: manually define and pass the array(s) of values to be scored in the next line
payload_scoring = {"input_data": [{"fields": [array_of_feature_columns], "values": [array_of_values_to_be_scored, another_array_of_values_to_be_scored]}]}

response_scoring = requests.post('https://wml-svt.ml.test.cloud.ibm.com/v4/deployments/393a19e7-1934-4d61-85a5-f5086f3b44d9/predictions', json=payload_scoring, headers=header)
print("Scoring response")
print(json.loads(response_scoring.text))

 

Watson Machine Learning REST API

To deploy a model using the REST API, get the endpoint URL from the Implementation tab of your model’s deployment page. That page will also have a code sample to help you structure the payload for your deployment. For example:

# TODO: manually define and pass values to be scored below
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header "Authorization: Bearer  $IAM_TOKEN" --header "ML-Instance-ID: $ML_INSTANCE_ID" -d '{"input_data": [{"fields": [$ARRAY_OF_FEATURE_COLUMNS],"values": [$ARRAY_OF_VALUES_TO_BE_SCORED, $ANOTHER_ARRAY_OF_VALUES_TO_BE_SCORED]}]}' https://wml-svt.ml.test.cloud.ibm.com/v4/deployments/393a19e7-1934-4d61-85a5-f5086f3b44d9/predictions

 

Watson Machine Learning CLI

After you train and store you model in a Watson Machine Learning repository, you can deploy the model in that Watson Machine Learning service using the bx ml deploy command:

Deploying from CLI

See: Setting up the CLI