Implementation details: Deploying Python functions in Watson Machine Learning

You can deploy Python functions in Watson Machine Learning the same way that you can deploy models. This topic demonstrates how to leverage the most powerful aspects of deployed functions, such as preprocessing data before passing it to models, and performing error handling within the deployed function instead of in your application logic.

 

Overview topic

Learn the basics of deploying functions in the introductory topic:

 

Sections in this topic

Jump to detailed information and sample code for specific implementation details:

 

Importing libraries and installing packages

Importing libraries

You can import libraries required in your deployed function by using an import statement.

Example Python code

This example demonstrates using the regular expression library, re, to replace the word “Hello” in the payload with the word “Goodbye”:

def my_deployable_function():

    def score( payload ):
    
        return re.sub( "Hello", "Goodbye", payload["values"][0] )

    return score

Testing the example code fails:

Error message

Adding the import statement resolves the error:

def my_deployable_function():
    
    def score( payload ):
    
        import re
        return re.sub( "Hello", "Goodbye", payload["values"][0] )
    
    return score

Test output:

Error resolved

 

Installing packages

Sometimes the package a library requires is not installed. You can install required packages from inside your deployed function.

Example Python code

This example demonstrates importing the Python imaging library:

def my_deployable_function():

    def score( payload ):
    
        from PIL import Image
        
        return payload["values"][0]

    return score

Testing this example locally might succeed if the imaging library happens to be installed locally:

Testing locally succeeds

However, deploying this function and then testing the deployment could still fail, because the deployment environment is different from the development environment:

Package missing in deployment

Using the subprocess library to call the pip install command in the outer function resolves the problem:

def my_deployable_function():
    
    import subprocess
    subprocess.check_output( "pip install Pillow --user", stderr=subprocess.STDOUT, shell=True )

    def score( payload ):
    
        from PIL import Image
        
        return payload["values"][0]
    
    return score

Test output:

Error resolved

 

Testing locally and testing in deployment

As the previous PIL example shows, even when a given package is installed in your development environment (in your notebook, for example) you should not assume that the package is installed in the environment where your function is deployed (in your Watson Machine Learning service.)

Test your function locally and also deploy your function and test the deployment.

 

Preprocessing payload data

AI models expect an input payload that is in a certain format. Data coming from web apps or elsewhere must often be preprocessed before it can be sent in a payload to a model. You can preprocess the raw data in your web app interface or in your web server code; but it can be easiest and most effective to do that preprocessing in a deployed function.

Example scenario

The Node.js MNIST sample app and the Python Flask MNIST sample app both have the same web interface: an HTML canvas object in which digits can be hand drawn using a mouse.

Hand-drawn digit

  • Raw data: The RGBA image data coming from the HTML canvas object is a 1 by ( n * n * 4 ) array of 8-bit unsigned integers, where n is the height (and width), in pixels, of a square bounding box containing a hand-drawn digit.

  • Required model payload format: The sample TensorFlow model built in several of the MNIST tutorials expects input that is a 1 by 784 array of floating-point numbers ranging from 0 to 1.

Example Python code

This sample code demonstrates reshaping and normalizing canvas image data received by the deployed function so the processed data can be passed on to one of the TensorFlow model deployments:

def my_deployable_function():
    
    import subprocess
    subprocess.check_output( "pip install PIL --user", stderr=subprocess.STDOUT, shell=True )        
    
    def getRGBAArr( canvas_data ):
        import numpy as np
        dimension = canvas_data["height"]
        rgba_data = canvas_data["data"]
        rgba_arr  = np.asarray( rgba_data ).astype('uint8')
        return rgba_arr.reshape( dimension, dimension, 4 )

    def getNormAlphaList( img ):
        import numpy as np
        alpha_arr       = np.array( img.split()[-1] )
        norm_alpha_arr  = alpha_arr / 255
        norm_alpha_list = norm_alpha_arr.reshape( 1, 784 ).tolist()
        return norm_alpha_list
    
    def score( function_payload ):

        from PIL import Image
        canvas_data   = function_payload["values"][0]           # Read the payload received by the function
        rgba_arr      = getRGBAArr( canvas_data )               # Create an array object with the required shape
        img           = Image.fromarray( rgba_arr, 'RGBA' )     # Create an image object that can be resized
        sm_img        = img.resize( ( 28, 28 ), Image.LANCZOS ) # Resize the image to 28 x 28 pixels
        alpha_list    = getNormAlphaList( sm_img )              # Create a 1 x 784 array of values between 0 and 1
        model_payload = { "values" : alpha_list }               # Create a payload to be sent to the model
            
        # Send model_payload to the model deployment here ...
        # model_result is what is returned ...
        #
        return model_result


    return score

 

Postprocessing results

Raw results returned from deployed models can sometimes have many details that are important for understanding the result and for troubleshooting. However, those details might not be as relevant to an AI application, so handling those details in your application code is a burden without benefit. In your deployed function, you can postprocess results returned from models and then return simpler results to your application.

Example scenario

The sample TensorFlow model built in several of the MNIST tutorials identifies hand-written digits.

Example input:

Digit: 5

Example model output:

{
  "fields": [
    "prediction"
  ],
  "values": [
    [
      5
    ]
  ]
}

Example Python code

This example shows how a deployed function can prostprocess the results returned from a model deployment, and then return simpler output to your application (a single digit class, instead of the whole JSON-formated result):

def my_deployable_function( parms=ai_parms ):
    
    from watson_machine_learning_client import WatsonMachineLearningAPIClient
    client = WatsonMachineLearningAPIClient( parms["wml_credentials"] )
    
    def score( payload ):
    
        model_result = client.deployments.score( parms["model_endpoint_url"], payload )
        
        return model_result["values"][0]
    
    return score

Example function output:

5

 

Error handling

Implementing error handling in the deployed function can be be easier and more effective than trying to catch all errors in your application logic.

Example 1: Error handling in the score function

This example demonstrates handling an invalid import statement, and returning a clean error that would be easy for an app to handle:

def my_deployable_function():

    def score( payload ):
    
        try:
            
            from PILzzz import Image
            
            return { "result" : payload["values"][0] }
        
        except Exception as e:
            
            return { "error" : repr( e ) }

    return score

Test output:

General exception

Example 2: Simple error handling in the outer function

The body of the outer function of your closure is run once: when the function is deployed. So, although you need to make sure errors are returned cleanly to your apps from the score function, your error-handling strategy in the body of the outer function can be simpler. You can allow errors in the body of the outer function to be caught by the Watson Machine Learning client at deploy time.

This example demonstrates how errors in the outer function appear when you deploy the function:

def my_deployable_function():

    import subprocess
    subprocess.check_output( "pip install Pillzzz --user", stderr=subprocess.STDOUT, shell=True )

    def score( payload ):
    
        return { "result" : payload["values"][0] }

    return score

Output from deploying:

Outer function error

 

Example 3: More detailed error handling in the outer function

If you need to collect more detailed error information, you can handle specific exceptions and then throw the more detailed error information in a general exception for the Watson Machine Learning client to catch.

This example demonstrates capturing more information about an error:

def my_deployable_function():

    try:
        import subprocess
        subprocess.check_output( "pip install Pillzzz --user", stderr=subprocess.STDOUT, shell=True )
    
    except subprocess.CalledProcessError as e:        
        install_err = "subprocess.CalledProcessError:\n\n" + "cmd:\n" + e.cmd + "\n\noutput:\n" + e.output.decode()
        raise Exception( "Installing failed:\n" + install_err )
    
    def score( payload ):
    
        return { "result" : payload["values"][0] }

    return score

Output from deploying:

Outer function error with more details