Flow editor tutorial: Build a neural network to recognize handwritten digits using the MNIST data set

This tutorial guides you through using the MNIST computer vision data set to train a deep learning model to recognize handwritten digits. In this tutorial, you will design a convolutional neural network (CNN) with one convolutional layer using the flow editor in IBM Watson Studio, and then train, deploy, and test the model using the experiment builder in Watson Studio.

 

Prerequisite

 

Steps overview

This tutorial presents the basic steps for designing and training a deep learning model in Watson Studio:

  1. Set up data files in IBM Cloud Object Storage
  2. Design the neural network
  3. Train the model
  4. Monitor training progress and results
  5. Deploy the trained model
  6. Test the deployed model

This tutorial does not demonstrate distributed deep learning, or using the Watson Machine Learning hyperparameter optimization feature.

 

Step 1: Set up training data files in Cloud Object Storage

Training a deep learning model using Watson Machine Learning relies on using Cloud Object Storage for reading input (such as training data) as well as for storing results (such as log files.)

  1. Download four sample data files to your local computer from here: MNIST data in pickle format external link

    • mnist-keras-train.pkl
    • mnist-keras-validate.pkl
    • mnist-keras-test.pkl
    • mnist-keras-test-payload.json

  2. Open the Cloud Object Storage GUI:

    1. In Watson Studio, from the Services drop-down menu, choose "Data Services".
    2. Select "Manage in IBM Cloud" from the ACTIONS menu beside the service instance of Cloud Object Storage that is associated with your deep learning project. (This opens the Cloud Object Storage GUI.)

  3. Perform these steps in the Cloud Object Storage GUI:

    1. Create two buckets: one for storing training data, and one for storing training results.
      See: Creating a Cloud Object Storage bucket

    2. Upload the three sample .pkl files to the training data bucket. (The sample .json file is not used until step 6.)
      See: Uploading data to Cloud Object Storage

 

Step 2: Design the neural network in flow editor

Using the graphical flow editor in Watson Studio, you can assemble your machine learning model or neural network design by dragging and dropping nodes from a palette. In this tutorial, you don't have to design your neural network from scratch. Instead, this tutorial demonstrates how you can create a neural network design based on a sample in the flow editor user interface.

  1. From the Assets page of your project in Watson Studio, in the Modeler flows section, click New flow.

  2. In the page that opens, click From example.

  3. Select the card labeled "NEURAL NETWORK DESIGN Single Convolution layer on MNIST" by clicking it, then click Create.

    The sample neural network design is loaded into the flow editor:

    Sample neural network design
  4. Update the first node to specify how to read training, test, and validation data:

    1. Double-click the first node, labeled Image Data (this opens the node details side panel.)
    2. In the details side panel, in the DATA section, create a connection to the Cloud Object Storage instance you associated with this project:
      1. click Create a new connection.
      2. Specify a name for the connection.
      3. Click the Create a connection button.
    3. In the Data Connections drop-down menu, select the Cloud Object Storage connection that you just created.
    4. In the Buckets drop-down menu, select the training data bucket you created before.
    5. In the Train data file drop-down menu, select "mnist-keras-train.pkl".
    6. In the Test data file drop-down menu, select "mnist-keras-test.pkl".
    7. In the Validation data file drop-down menu, select "mnist-keras-validate.pkl".
  5. Store the neural network design as a training definition:

    1. Click the publish icon ( Publish icon ).
    2. Specify a name for the training definition.
    3. In the Specify WML instance drop-down menu, select the Watson Machine Learning service instance that is associated with the project.
    4. Click the Publish button.

 

Note: It's not needed for this tutorial, but note that you can download the model-building code that the flow editor generates by clicking the Download button ( Download icon ) and then choosing a framework.

Download options

 

Step 3: Train the model using experiment builder

In this tutorial, the model is trained using experiment builder in Watson Studio.

  1. From the Assets page of your project in Watson Studio, click New experiment.

  2. Specify a name for the experiment.

  3. In the Machine Learning Service drop-down, select the Watson Machine Learning service instance that is associated with the project.

  4. Configure Cloud Object Storage for the experiment:

    1. In the area for Cloud Object Storage click Select.
    2. In the Cloud Object Storage connection drop-down list, select the Cloud Object Storage connection you created before.
    3. For the training data bucket, select the training data bucket you created before.
    4. For the training results bucket, select the results bucket you created before.
    5. Click Create
  5. Add a training definition.

    1. Click Add training definition.
    2. Click the Existing training definitions tab.
    3. From the drop-down menu, select the training definition you published from the flow editor.
    4. Select "1/2 x NVIDIA Tesla K80 (1 GPU)" for the compute plan.
    5. Select "none" for the hyperparameter optimization method.

  6. Click Create and run.

 

Step 4: Monitor training progress and results

  • You can monitor the progress of a training run in the Training Runs tab of experiment builder.

  • When the training run is complete, click on the training definition to view details of its training results, including logs and other output.

 

Step 5: Deploy the trained model

You can use your trained model to classify new images only after the model has been deployed.

  1. In the Training Runs tab of experiment builder, under the ACTIONS menu, select "Save model". Then give the model a name and click Save. This stores the model in the Watson Machine Learning repository.

  2. In the Assets page of your project in Watson Studio, click the new model in the Models section.

  3. Click the Deployments tab and then click Add Deployment.

  4. Choose "Web Service" as the deployment type, specify a name for the deployment, and then click Save.

  5. Click the new deployment to view the model details page.

 

Step 6: Test the deployed model

You can quickly test your deployed model from the deployment details page.

  1. On your local computer, open the sample file, mnist-keras-test-payload.json (that you downloaded in step 1) in a text editor.
  2. In the Test area of the deployment details page in Watson Studio, paste the contents of mnist-keras-test-payload.json:

    {
      "values" :
        [[[[  0],
            [  0],
            [  0],
    ...
            [  0],
            [  0],
            [  0]]]]
    }
    

  3. Click Predict.

    The results will look something like this:

    {
      "fields": [
        "prediction"
      ],
      "values": [
        [
          0,
          0,
          0,
          0,
          0,
          0,
          0,
          1,
          0,
          0
        ]
      ]
    }
    

    This output displays an array of confidence scores for digit classes "0" through "9". The scores indicate how well the input matches each class. The class with the highest score is the digit class "7", which is the correct result.