Deploying a model

The last stage of the model development work flow is to deploy a model. This means you put the model into production, so that you can pass data to the model and get a score, or prediction back. In this way, you can evaluate the effectiveness of your model and start to use it to make business decisions.

Model lifecycle

The basic model lifecycle follows this sequence:

  1. Design a model. In this phase you consider what questions you are trying to answer and prepare the data.
  2. Train the model. Use the data you have prepared to train the model so it will be able to process new data and make recommendations based on the training.
  3. Store the model. When you feel the model is ready to test, save the model. You can do this from a Watson Studio tool such as AutoAI or NeuNets, or you can save a model programmatically.
  4. Deploy a model. When you deploy the model, you are making it available to process new data and return confidence scores about how well the new data aligns with the rules the model established during processing. For example, if you are trying to determine whether a customer with a particular profile is likely to buy a red or a blue car, you submit all of the profile data for the customer and get back a score, or prediction, of their likely purchase based on all of the related data used to train the model.

Note: After deploying a model, you can also make the model available for wider use in applications.

Watson Machine Learning deployments

Watson Machine Learning supports all phases of the model development lifecycle, giving you the tools to design, train, and save your model. When it comes to deploying, Watson Machine Learning provides these benefits:

  • One-click deployment for models saved to Watson Studio.
  • No-code, form-based scoring for AutoAI.
  • Code samples to help you structure your input data.
  • Options for generating real-time predictions or for setting up and scheduling batch predictions.

Prerequisite

Before you can deploy, you must either save a model using the Watson Studio interface, or save a model programmatically.

Deployment types

There are three types of deployment, each suited to a specific use case.

Table 1. Deployment types
Deployment type Use this deployment type when you want to...
Online web service Access your deployment through an API endpoint, using the Watson Machine Learning Python client external link, CLI, or REST API external link (manually or in an application) to analyze data.
Batch Perform bulk analysis of data in IBM Cloud Object Storage external link or IBM Db2 Warehouse on Cloud external link
Virtual (Core ML) Download a model to use in a Core ML application.

For a list of deployment types supported by framework, see: Supported machine learning frameworks

 

How to deploy a model

You can deploy your model using different methods, depending on your scenario and preference.

Table 2. Deployment method
Deployment method Scenario
Watson Studio project If you are working in a Watson Studio project already, using the Deployments tab of your project is quick and simple. After you create a deployment, you can either fill out a form to submit data for scoring, submit JSON-formatted code, or set up input and output data sources to process data in a batch. You can also download your deployment for integration in an app.
Watson Machine Learning Python client If you are working in a local Python development environment or in a Python notebook, or if you are developing a Python app, you can deploy models using the Python client.
Watson Machine Learning REST API For ultimate flexibility, you can deploy a model by using the REST API. For example, you can deploy a model from your local enviornment without having to install the Watson Machine Learning CLI. And you can deploy a model from an app in any language without having to import a Watson Machine Learning client library.
Watson Machine Learning CLI If you are working in a local environment, creating automation, or simply prefer a command line interface, you can deploy models from the command line.

 

Watson Studio project

After you train and store your model in a Watson Studio repository that is associated with a Watson Studio project, you can deploy your model in that Watson Machine Learning service from the ACTIONS menu beside the model in the Assets tab of the project:

Deploying from project

Options available to you when you use Watson Machine Learning to deploy and test your model will vary depending on the kind of model you are deploying. The following steps are for a Spark ML model.

Creating and testing a Web service deployment

Deployments in Watson Studio include a no-code way to quickly test a deployment and generate a prediction. For example, if you are testing an AutoAI deployment, the Test tab provides a form that you can fill out to generate a prediction. When no form is available, you can submit test data as JSON code.

Prediction from test data

To create and test a deployment:

  1. Click Add Deployment to create a deployment.
  2. Enter a name and description for the deployment.
  3. Choose Web service as the deployment type and click Save.
  4. Click the deployment name to view and test the deployment.
  5. On the Test page, enter test data and click Predict to view predictions.
  6. Click Implementation to view details of the deployment such as the endpoint URL and authentication details. This information is required to interact with the model programmatically or embed it in an app.

Passing payload data to Web service deployments

You can pass payload data to a deployed model. How you structure payload data depends on the type of model you are building. Your payload data must match the data schema for the model. For example, your payload data for an AutoAI model that returns information about a cruise ship passenger would be structured like this, with the fields defined and then corresponding values passed for the fields.

{"input_data":[{
        "fields": ["PassengerId","Pclass","Name","Sex","Age","SibSp","Parch","Ticket","Fare","Cabin","Embarked"],
        "values": [[1,3,"Braund, Mr. Owen Harris",0,22,1,0,"A/5 21171",7.25,null,"S"]]
}]}

Downloading a virtual deployment for use with Core ML

After creating a deployment in a project, click the Deployment details page to find the Download link. Use this link to download the model for integration in apps.

For guidance on how to structure your payload data, view the code samples on the Implementation tab for your deployment.

Creating a Batch prediction deployment

A batch deployment processes a series of inputs and returns corresponding predictions.

Prerequisites

Before you begin, depending on the type of data and model, you must have the following resources:

  1. Click Add Deployment to create a deployment.
  2. Enter a name and description for the deployment.
  3. Choose Batch prediction as the deployment type.
  4. Select a runtime environment from the list.
  5. Specify repositories for the test data that will be passed to the deployment and for storing the resulting predictions and click Save. On IBM Cloud, you can store your test data as an asset in your project. You can also store test data in a Cloud Object Storage bucket and specify a second bucket for storing the results. For information on working with Cloud Object Storage buckets, see Creating a bucket
  6. Click the deployment name to view the deployment details and schedule the batch job.
  7. Click Implementation to view details of the deployment such as the endpoint URL and authentication details. This information is required to interact with the model programmatically or embed it in an app.

Watson Machine Learning Python client

How you interact with a Watson Machine Learning model depends on the type of model you are deploying. The Implementation tab of your model’s deployment page will have a code sample for deploying your specific model type.

V3 Python client

Most of the Watson Machine Learning models use a Python client based on Watson Machine Learning Python client library reference, version 3. For details on the deployment syntax, refer to Watson Machine Learning Python client external link.

For an example of deploying a model using the v3 Python client, see Watson Machine Learning Python client tutorial.

V4 Python client

AutoAI and Decision Optimization are built using the Watson Machine Learning Python client library reference, version 4, which has a different syntax than the version 3 APIs used for Model Builder and other Watson Machine Learning models. To interact programmatically with an AutoAI deployment, refer to the deployment syntax in Watson Machine Learning Python client library external link.

For an example of deploying a model using the v4 Python client, see Python Client Example

Example Python code

The Implementation tab of your model’s deployment page will have a code sample for deploying your model.

Python client v3 This code sample shows how to deploy a model created using:

  • Model Builder
  • NeuNetS
  • SPPS Modeler
  • Spark ML Modeler
  • Neural Network Modeler
  • Experiment Builder
import urllib3, requests, json

# NOTE: generate iam_token and retrieve ml_instance_id based on provided documentation	
header = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + iam_token, 'ML-Instance-ID': ml_instance_id}

# NOTE: manually define and pass the array(s) of values to be scored in the next line
payload_scoring = {"fields": ["GENDER", "AGE", "MARITAL_STATUS", "PROFESSION"], "values": [array_of_values_to_be_scored, another_array_of_values_to_be_scored]}

response_scoring = requests.post('https://us-south.ml.cloud.ibm.com/v3/wml_instances/9a6ae85e-5f7f-43ba-9ede-f6ea9fc832ae/deployments/b5a2d435-bbb4-42b3-b5ff-a4609a21cb1c/online', json=payload_scoring, headers=header)
print("Scoring response")
print(json.loads(response_scoring.text))

Python client v4 This Python code sample shows you how to deploy a model built using AutoAI or Decision Optimization.

import urllib3, requests, json

# NOTE: generate iam_token and retrieve ml_instance_id based on provided documentation	
header = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + iam_token, 'ML-Instance-ID': ml_instance_id}

# NOTE: manually define and pass the array(s) of values to be scored in the next line
payload_scoring = {"input_data": [{"fields": [array_of_feature_columns], "values": [array_of_values_to_be_scored, another_array_of_values_to_be_scored]}]}

response_scoring = requests.post('https://wml-svt.ml.test.cloud.ibm.com/v4/deployments/393a19e7-1934-4d61-85a5-f5086f3b44d9/predictions', json=payload_scoring, headers=header)
print("Scoring response")
print(json.loads(response_scoring.text))

 

Watson Machine Learning REST API

To deploy a model using the REST API, get the endpoint URL from the Implementation tab of your model’s deployment page. That page will also have a code sample to help you structure the payload for your deployment. For example:

# TODO: manually define and pass values to be scored below
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header "Authorization: Bearer  $IAM_TOKEN" --header "ML-Instance-ID: $ML_INSTANCE_ID" -d '{"input_data": [{"fields": [$ARRAY_OF_FEATURE_COLUMNS],"values": [$ARRAY_OF_VALUES_TO_BE_SCORED, $ANOTHER_ARRAY_OF_VALUES_TO_BE_SCORED]}]}' https://wml-svt.ml.test.cloud.ibm.com/v4/deployments/393a19e7-1934-4d61-85a5-f5086f3b44d9/predictions

 

Watson Machine Learning CLI

After you train and store you model in a Watson Machine Learning repository, you can deploy the model in that Watson Machine Learning service using the bx ml deploy command:

Deploying from CLI

See: Setting up the CLI