Importing trained TensorFlow models into Watson Machine Learning

If you have a TensorFlow model that you trained outside of IBM Watson Machine Learning, this topic describes how to import that model into your Watson Machine Learning service.

Restrictions and requirements

You must use tf.saved_model.builder.SavedModelBuilder to save/serialize a TensorFlow model.
The saved(serialized) model file must be in the top level of the .tar.gz file that gets saved/uploaded to the Watson Machine Learning repository using client.repository.store_model() API.
tf.estimator is not supported
The only supported deployment types for TensorFlow models are: web service and batch
See also: Supported frameworks

Example

The following notebook demonstrates importing a TensorFlow model:

Importing a saved TensorFlow model into Watson Machine Learning

Step 0 for interface options 1 and 2: Build, train, and save a model

The following Python code snippet demonstrates:

Building and training a text classifier
Saving the trained model in a directory called “message-classification-model-dir”
Saving the model files in a tar.gz file called “message-classification-model.tar.gz”

import tensorflow as tf

# Build the graph
# Input: X, labels: y
X = tf.placeholder( tf.float32, shape = ( None, num_inputs ) )
y = tf.placeholder( tf.float32, shape = ( None, num_output_classes ) )
# Layer 1
w1 = tf.Variable( tf.truncated_normal( shape=[ num_inputs, num_layer1_nodes ] ) )
b1 = tf.Variable( tf.zeros( shape=[ num_layer1_nodes ] ) )
layer1_output =  tf.nn.relu( tf.matmul( X, w1 ) +  b1 ) 
# Output
w2 = tf.Variable( tf.truncated_normal( shape=[ num_layer1_nodes, num_output_classes ] ) )
b2 = tf.Variable( tf.zeros( shape=[ num_output_classes ] ) )
output = tf.nn.softmax( tf.matmul( layer1_output, w2 ) + b2 )

# Train the model
loss = tf.losses.softmax_cross_entropy( y, output )
optimizer = tf.train.AdamOptimizer( learning_rate ).minimize( loss )
for epoch in range( num_epochs ):    
    session.run( optimizer, feed_dict={ X : X_train, y : y_train } )

# Save the trained model
builder = tf.saved_model.builder.SavedModelBuilder( "message-classification-model-dir" )
builder.add_meta_graph_and_variables(
      session, [ tf.saved_model.tag_constants.SERVING ],
      signature_def_map={ "classify_message" : classification_signature, },
      main_op=tf.tables_initializer() )
builder.save()
!tar -zcvf message-classification-model.tar.gz -C message-classification-model-dir .

Where:

vocab_size is the number of words in the dictionary
X_train is the tokenized, padded training input strings
y_train is the binary-encoded labels

For the full code example, see: the sample notebook external link

Interface option 1: Watson Machine Learning Python client

Step 1: Store the model in your Watson Machine Learning repository

You can store the model in your Watson Machine Learning repository using the Watson Machine Learning Python client store_model method external link .

Format options:

Trained model saved in a directory:

metadata = {
    client.repository.ModelMetaNames.NAME: "TensorFlow model (directory)",
    client.repository.ModelMetaNames.FRAMEWORK_NAME: "tensorflow",
    client.repository.ModelMetaNames.FRAMEWORK_VERSION: "1.13"
}
model_details_dir = client.repository.store_model( model="message-classification-model-dir", meta_props=metadata )

Trained model saved in a tar.gz file:

metadata = {
    client.repository.ModelMetaNames.NAME: "TensorFlow model (tar.gz)",
    client.repository.ModelMetaNames.FRAMEWORK_NAME: "tensorflow",
    client.repository.ModelMetaNames.FRAMEWORK_VERSION: "1.13"
}
model_details_targz = client.repository.store_model( model="message-classification-model.tar.gz", meta_props=metadata )

Where:

<your-credentials> contains credentials for your Watson Machine Learning service (see: Looking up credentials)

Step 2: Deploy the stored model in your Watson Machine Learning

The following example demonstrates deploying the stored model as a web service, which is the default deployment type:

model_id_dir = model_details_dir["metadata"]["guid"]
deployment_details_dir = client.deployments.create( artifact_uid=model_id_dir, name="TensorFlow deployment (directory)" )

See: Deployments.create

Interface option 2: Watson Machine Learning CLI

Prerequisite: Set up the CLI environment.

Step 1: Store the model in your Watson Machine Learning repository

Example command and corresponding output

>ibmcloud ml store <model-filename> <manifest-filename>
Starting to store ...
OK
Model store successful. Model-ID is '145bca56-134f-7e89-3c12-0d3a7859d21f'.

Where:

<model-filename> is the path and name of the tar.gz file
<manifest-filename> is the path and name of a manifest fest containing metadata about the model being stored

Sample manifest file contents

name: My TensorFlow model
framework:
  name: tensorflow
  version: '1.13'

See: store CLI command external link

Step 2: Deploy the stored model in your Watson Machine Learning

The following example demonstrates deploying the stored model as a web service, which is the default deployment type:

Example command and corresponding output

>ibmcloud ml deploy <model-id> "My TensorFlow model deployment"
Deploying the model with MODEL-ID '145bca56-134f-7e89-3c12-0d3a7859d21f'...
DeploymentId       316a89e2-1234-6472-1390-c5432d16bf73
Scoring endpoint   https://us-south.ml.cloud.ibm.com/v3/wml_instances/5da31...
Name               My TensorFlow model deployment
Type               tensorflow-1.13
Runtime            None Provided
Status             DEPLOY_SUCCESS
Created at         2019-01-14T19:47:51.735Z
OK
Deploy model successful

Where:

<model-id> was returned in the output from the store command

See: deploy CLI command external link