Importing trained TensorFlow models into Watson Machine Learning
If you have a TensorFlow model that you trained outside of IBM Watson Machine Learning, this topic describes how to import that model into your Watson Machine Learning service.
Restrictions and requirements
- You must use
tf.saved_model.builder.SavedModelBuilder
to save/serialize a TensorFlow model. - The saved(serialized) model file must be in the top level of the .tar.gz file that gets saved/uploaded to the Watson Machine Learning repository using
client.repository.store_model()
API. tf.estimator
is not supported- The only supported deployment types for TensorFlow models are: web service and batch
- See also: Supported frameworks
Example
The following notebook demonstrates importing a TensorFlow model:
Interface options
Step 0 for interface options 1 and 2: Build, train, and save a model
The following Python code snippet demonstrates:
- Building and training a text classifier
- Saving the trained model in a directory called “message-classification-model-dir”
- Saving the model files in a tar.gz file called “message-classification-model.tar.gz”
import tensorflow as tf
# Build the graph
# Input: X, labels: y
X = tf.placeholder( tf.float32, shape = ( None, num_inputs ) )
y = tf.placeholder( tf.float32, shape = ( None, num_output_classes ) )
# Layer 1
w1 = tf.Variable( tf.truncated_normal( shape=[ num_inputs, num_layer1_nodes ] ) )
b1 = tf.Variable( tf.zeros( shape=[ num_layer1_nodes ] ) )
layer1_output = tf.nn.relu( tf.matmul( X, w1 ) + b1 )
# Output
w2 = tf.Variable( tf.truncated_normal( shape=[ num_layer1_nodes, num_output_classes ] ) )
b2 = tf.Variable( tf.zeros( shape=[ num_output_classes ] ) )
output = tf.nn.softmax( tf.matmul( layer1_output, w2 ) + b2 )
# Train the model
loss = tf.losses.softmax_cross_entropy( y, output )
optimizer = tf.train.AdamOptimizer( learning_rate ).minimize( loss )
for epoch in range( num_epochs ):
session.run( optimizer, feed_dict={ X : X_train, y : y_train } )
# Save the trained model
builder = tf.saved_model.builder.SavedModelBuilder( "message-classification-model-dir" )
builder.add_meta_graph_and_variables(
session, [ tf.saved_model.tag_constants.SERVING ],
signature_def_map={ "classify_message" : classification_signature, },
main_op=tf.tables_initializer() )
builder.save()
!tar -zcvf message-classification-model.tar.gz -C message-classification-model-dir .
Where:
vocab_size
is the number of words in the dictionaryX_train
is the tokenized, padded training input stringsy_train
is the binary-encoded labels
For the full code example, see: the sample notebook
Interface option 1: Watson Machine Learning Python client
Step 1: Store the model in your Watson Machine Learning repository
You can store the model in your Watson Machine Learning repository using the Watson Machine Learning Python client store_model
method .
Format options:
-
Trained model saved in a directory:
metadata = { client.repository.ModelMetaNames.NAME: "TensorFlow model (directory)", client.repository.ModelMetaNames.FRAMEWORK_NAME: "tensorflow", client.repository.ModelMetaNames.FRAMEWORK_VERSION: "1.13" } model_details_dir = client.repository.store_model( model="message-classification-model-dir", meta_props=metadata )
-
Trained model saved in a tar.gz file:
metadata = { client.repository.ModelMetaNames.NAME: "TensorFlow model (tar.gz)", client.repository.ModelMetaNames.FRAMEWORK_NAME: "tensorflow", client.repository.ModelMetaNames.FRAMEWORK_VERSION: "1.13" } model_details_targz = client.repository.store_model( model="message-classification-model.tar.gz", meta_props=metadata )
Where:
- <your-credentials> contains credentials for your Watson Machine Learning service (see: Looking up credentials)
Step 2: Deploy the stored model in your Watson Machine Learning
The following example demonstrates deploying the stored model as a web service, which is the default deployment type:
model_id_dir = model_details_dir["metadata"]["guid"]
deployment_details_dir = client.deployments.create( artifact_uid=model_id_dir, name="TensorFlow deployment (directory)" )
See: Deployments.create
Interface option 2: Watson Machine Learning CLI
Prerequisite: Set up the CLI environment.
Step 1: Store the model in your Watson Machine Learning repository
Example command and corresponding output
>ibmcloud ml store <model-filename> <manifest-filename>
Starting to store ...
OK
Model store successful. Model-ID is '145bca56-134f-7e89-3c12-0d3a7859d21f'.
Where:
- <model-filename> is the path and name of the tar.gz file
- <manifest-filename> is the path and name of a manifest fest containing metadata about the model being stored
Sample manifest file contents
name: My TensorFlow model
framework:
name: tensorflow
version: '1.13'
See: store
CLI command
Step 2: Deploy the stored model in your Watson Machine Learning
The following example demonstrates deploying the stored model as a web service, which is the default deployment type:
Example command and corresponding output
>ibmcloud ml deploy <model-id> "My TensorFlow model deployment"
Deploying the model with MODEL-ID '145bca56-134f-7e89-3c12-0d3a7859d21f'...
DeploymentId 316a89e2-1234-6472-1390-c5432d16bf73
Scoring endpoint https://us-south.ml.cloud.ibm.com/v3/wml_instances/5da31...
Name My TensorFlow model deployment
Type tensorflow-1.13
Runtime None Provided
Status DEPLOY_SUCCESS
Created at 2019-01-14T19:47:51.735Z
OK
Deploy model successful
Where:
- <model-id> was returned in the output from the
store
command
See: deploy
CLI command