Python Flask tutorial: Build a web app that recognizes hand-drawn digits

This tutorial guides you through building a Python Flask app that uses a model trained with the MNIST data set to recognize digits that are hand drawn on an HTML canvas.

 

Prerequisites

  1. Build and deploy the function in this tutorial:

    MNIST function deployment tutorial

  2. Look up the endpoint URL of the model deployment and the function deployment from the previous prerequisite.

    See: Looking up an online deployment endpoint URL

  3. Collect these details from your Watson Machine Learning credentials:

    • apikey
    • instance_id
    • url
    • username
    • password

    See: Looking up credentials

 

Steps overview

This tutorial presents the basic steps for integrating a Watson Machine Learning model deployment or function deployment into a Node.js web app:

  1. Download the sample Python Flask app
  2. Update the code to use your deployments
  3. Run the app locally
  4. Push the app to IBM Cloud
  5. Remove hard-coded credentials

 

Step 1: Download the sample Python Flask app

Download the sample code here: Python-Flask-MNIST-sample-app.zip external link

Then, unzip the sample code. For the remainder of this tutorial, the term working directory refers to the directory containing the file server.py, and the examples here will show D:/Python-Flask-MNIST-sample-app/app as the working directory.

Files in Python-Flask-MNIST-sample-app.zip

Table 1. Files in Python-Flask-MNIST-sample-app.zip
File Description
app/.cfignore
app/.dockerignore
When you push the app to IBM Cloud, any files and directories listed in .cfignore and .dockerignore do not get uploaded.
app/Dockerfile A text file containing the commands to create an image for the Python app on IBM Cloud.
app/manifest.yml Lists details about your app which are needed when you push the app to IBM Cloud.
app/Procfile A text file containing the command to run the app on IBM Cloud.
app/requirements.txt Lists packages and versions required by your app when you push the app to IBM Cloud.
app/runtimes.txt Specifies the version of Python required to run on IBM Cloud.
app/server.py The Python Flask server code that drives the functionality of your web app (for example: sending payload data to your model deployment or function deployment using the Watson Machine Learning Python client.)
app/setup.py A Python file that sets up your app and required packages on IBM Cloud.
app/static/index.html The HTML web page of your web app. Also includes Javascript for ajax calls to your Python Flask server.
app/static/css/styles.css Controls the style of objects in the HTML page (for example: font, color, size.)
app/static/javascript/javascript.js Implements callbacks of interface elements in the HTML web page (for example: button clicks.)

Preprocessing payload data

The function submit_drawing() in the file app/static/javascript/javascript.js illustrates an important comparison:

  • Sending canvas data to the Python Flask server, which preprocesses the canvas data and then sends the data to the deployed model
  • Sending canvas data to a deployed function, which preprocesses the canvas data and then sends the data to the deployed model
  • Preprocessing the canvas data in the web app front end and then sending that to a deployed model
    if( deployment_type.match( /webserver/ )  )
    {
      // If having the web server do the preprocessing, we don't need to
      // do any more preprocessing here in the app
      var img_data = { "height" : centered_img.height, "data" : [ Array.from( centered_img.data ) ] };
      ...
    }
    else if( deployment_type.match( /function/ )  )
    {
      // If using the function deployment, we don't need to do any more
      // preprocessing here in the app because the function handles that
      var payload = { "values" : [ { "height" : centered_img.height, "data" : [ Array.from( centered_img.data ) ] } ] };
      ...
    }
    else
    {
      // To send a payload to the model deployment, we need to do
      // more preprocessing here in the app
      ...
      var tiny_img = resize_28x28( centered_img, box_X, box_Y );
      ...
      var gray_img = greyscale( tiny_img );    
      var arr      = create_array_28x28( gray_img );                    
      var payload  = { "values" : [ arr ] };
    

This illustrates how using deployed functions can simplify app development. The best place to perform payload data preprocessing depends on multiple factors. However, on a team where one person specializes in app development and another person specializes in building AI models, it can be easier and more effective to have the data preprocessing done by the person familiar with the models.

 

Step 2: Update the code to use your deployments

Update code in the file server.py in a text editor.

2.1 Credentials

Paste in the Watson Machine Learning credentials you collected in the prerequisites for this tutorial:

#
# 1.  Fill in wml_credentials.
#
var wml_credentials = {
    "apikey"      : "",
    "instance_id" : "",
    "password"    : "",
    "url"         : "",
    "username"    : ""
};

2.2 Deployment endpoint URL

Paste in the deployment endpoint URLs you collected in the prerequisites for this tutorial:

  • If you want to send payload data directly to a model deployment, paste in the endpoint URL for the deployed model
  • If you want to send payload data to a function deployment, paste in the endpoint URL for the deployed function
    #
    # 2.  Fill in one or both of these:
    #     - model_deployment_endpoint_url
    #    - function_deployment_endpoint_url
    #
    model_deployment_endpoint_url    = "";
    function_deployment_endpoint_url = "";
    

 

Step 3: Run the app locally

Note: Running the app locally is not strictly required to be able to deploy the app to IBM Cloud. It’s a good idea to test that the app runs locally, and it’s easier to troubleshoot problems locally. However, you could skip step 3 and move on to deploying the sample app to IBM Cloud in step 4 (if you don’t want to install Python on your computer, for example.)

3.1 Install Python locally

Download Python here: Python download external link

Notes:

  • Make sure to have the installer add Python to your environment variables
  • Mac users, also install pip by issuing this command:
    sudo easy_install pip
    
  • Mac users, also add your user base binary directory to your path:
    1. Find the user base binary directory by running this command:
        python -m site --user-base
      
    1. Add your user base binary directory, with /bin appended, to the file /etc/paths

    See Full instructions external link

3.2 Install required packages

Install dependencies by issuing the following command from the working directory:

pip install -r requirements.txt

Mac users, run this command instead:

pip install --user -r requirements.txt

(This command uses the file requirements.txt to know which packages are needed.)

3.3 Start the local server

From a command prompt in the working directory, start the server by issuing the following command:

python server.py

View your app in a web browser at: http://localhost:8000

3.4 Test the web app locally

Test the web app by drawing a digit on the canvas, and then submitting the drawn digit to the model deployment, the function deployment, or to the Python server to process and then pass on to the model deployment:

Web app in a browser

After you click Analyze, you can see the web app creating the payload displayed in Generated payload:

  1. A rectangular bounding box is created around the digit (the red box is for illustration only, and is not part of the processing.)
  2. A square bounding box is created, with the digit centered in the box.

    If "Function deployment" is checked, processing stops here and this canvas data is sent to the function deployment.

  3. The content in the square box is resized to 28 pixels by 28 pixels.
  4. The image is converted to gray scale by setting RGB values to zero, leaving only the Alpha value remaining.
  5. The Alpha values are collected in a 1 by 784 array.
  6. Values in the array are normalized (divided by 255) to values between 0 and 1.

    If "Model deployment" is checked, this array is set to the model deployment.

Web app in a browser

Results returned from the deployment are displayed in Returned classification:

Web app in a browser

As you interact with the web app in the browser, you can see status information printed in the command window:

Python server console messages

 

Step 4: Push the app to IBM Cloud

4.1 Install the IBM Cloud CLI

Download the IBM Cloud CLI here: IBM Cloud CLI download external link

4.2 Create a Python app in IBM Cloud

Create a Python Cloud Foundry app here: Create a Python Cloud Foundry app external link

  1. Give the app a unique name (for the rest of this example, the sample name will be: “fabulous-flask-app”)
  2. Accept the defaults in the other fields of the form
  3. Choose the 128 MB plan
  4. Click Create

4.3 Update local code for deployment to IBM Cloud

In <working-dir>/app/manifest.yml replace “app-name” with the name you gave your new app in the previous step:

Before

applications:
 - name: app-name
   memory: 128M

After

applications:
 - name: fabulous-flask-app
   memory: 128M

In <working-dir>/app/setup.py replace “app-name” with the name you gave your new app in the previous step:

Before

setup(
    name='app-name',
...

After

setup(
    name='fabulous-flask-app',
...

4.4 Push the app to IBM Cloud

Issue the following commands from the command prompt in your working directory:

Log in to your IBM Cloud account:

ibmcloud login

Target the CloudFoundry API endpoint:

ibmcloud target --cf

Push your app to IBM Cloud:

ibmcloud app push

Now, you can access the app on IBM Cloud at your app URL:

Web app on IBM Cloud

 

Step 5: Remove hard-coded credentials

In general, hard coding your Watson Machine Learning service credentials in your app code is not a secure thing to do.

In IBM Cloud, you can Connect you apps and services (also called “binding a service to an application”), and then use the Cloud Foundry environment variable, VCAP_SERVICES, in your app to access service credentials.

5.1 Connect your Python app to your Watson Machine Learning service

  1. Log in to IBM Cloud dashboard external link and then click View all in the Resource summary.

  2. Expand the Cloud Foundry Apps section, and then click on your Python app to go to the details page for that app.

  3. In the Connections section, click Create connection.

  4. From the list of compatible services, click Connect beside the Watson Machine Learning service.

  5. In the modal that appears, accept the defaults and then click Connect.

  6. When prompted to restage your app, click Restage.

Now, in the Connections tab of your app details page in IBM Cloud, you can see your Watson Machine Learning service instance listed.

5.2 Update server.py to use VCAP_SERVICES

Comment out the credentials you hard coded in substep 2.1, and set wml_credentials using the Cloud Foundry environment, which was updated when you Connected your Watson Machine Learning service to your Python app in the previous substep:

#
# 1.  Fill in wml_credentials.
#
#var wml_credentials = {
#    "apikey"      : "",
#    "instance_id" : "",
#    "password"    : "",
#    "url"         : "",
#    "username"    : ""
#};

vcap = json.loads( os.getenv( "VCAP_SERVICES" ) )  # <-- Add this line
wml_credentials = vcap["pm-20"][0]["credentials"]  # <-- Add this line

client = WatsonMachineLearningAPIClient( wml_credentials )

From the command prompt in your working directory, push your app to IBM Cloud again:

ibmcloud app push

Now, the app uses the credentials in the Cloud Foundry environment to authenticate with Watson Machine Learning.

See also: