Core ML tutorial: Build an iOS app that recognizes hand-drawn digits

This tutorial guides you through downloading a Core ML (.mlmodel) file for a IBM Watson Machine Learning model trained in IBM Watson Studio, and then integrating that .mlmodel file into a sample iOS app built using the Apple integrated development environment, Xcode 9.

Software requirement

• To build the iOS app, you must have the Apple integrated development environment, Xcode 9, installed on your local computer: Xcode

Steps overview

This tutorial presents the basic steps for integrating a Watson Machine Learning model into an iOS app using Core ML:

1. Build the sample model in Watson Studio
2. Download the Core ML (.mlmodel) file
3. Download the sample iOS app
4. Add the .mlmodel file to the sample app
5. Build and test the app

Step 1: Build, train, and deploy the sample model in Watson Studio

Work through the flow editor MNIST tutorial to create, train, and deploy a neural network model in Watson Studio:

Flow editor MNIST tutorial

Step 2: Download the Core ML (.mlmodel) file from Watson Studio to your local computer

1. On the Assets page of your project, find your trained model in the Models section. From the ACTIONS menu for that model, select "Deploy". (This takes you to the Deployments tab of the model details page.)

2. In the Deployments tab of the model details page, click Add Deployment.

3. Fill in deployment details:

   • Specify a name for the deployment.
   • Select "Virtual" as the Deployment type. (If "Virtual" is not listed as a deployment type option, the model is not using a supported framework.)
   • Select "Core ML" as the Format.
   • Check the box to specify custom parameters, and then paste this JSON into the text box:

     { "image_input_names" : "input1", "image_scale" : 0.003921568627451 }
     

     You can read about these parameter here: Conversion parameters for models using Keras

4. Click Save.

5. In the Implementation tab of the deployment details page that opens, click Download Core ML model.

6. The download is a compressed file. Uncompress the downloaded file to get the Core ML model (.mlmodel) file, which is always named "model.mlmodel" by default.

Step 3: Download the sample iOS app from GitHub to your local computer

Clone or download this GitHub repo to your local computer: go-digits sample iOS app

Step 4: Add the .mlmodel file to the sample app in Xcode

1. Rename the .mlmodel file to: "mnistCNN.mlmodel".

   Explanation:

   • The .mlmodel file you download from Watson Studio is always named simply "model.mlmodel" by default.
   • When you add a .mlmodel file to your Xcode project, Xcode generates a class for the model with the same base name as the .mlmodel file.
   • In the sample app code, in QuizQuestionViewController.swift, you can see the code is expecting the generated class's initializer to be named mnistCNN:

     let model = mnistCNN()
     

2. Open the sample project Go-Digits.xcodeproj in Xcode.

3. Add the renamed file, "mnistCNN.mlmodel", to your project in Xcode.

   See: Integrating a Core ML model into your app

Points of interest in the sample code file QuizQuestionViewController.swift:

• You can see where the model, named "model", is used to analyze an image of the user input, named "input1":

  let output = try? model.prediction(input1: pixelBuffer!)
  

  Recall that in step 2, the custom parameter image_input_names was set to "input1".

• Also, you can see how the sample code is looping through the confidence scores (prob)returned for each digit class, "0" to "9", looking for the digit class that the user input best matches:

  let output1 = output!.output1
  var prob: Double = output1[0].doubleValue
  var digit:String = "0"
  for index in 0...output1.count-2 {
      if output1[index+1].doubleValue >= prob {
          prob = output1[index + 1].doubleValue
              digit = String(index+1)
          }
      }
  

  *Remember from the flow editor tutorial that the output from the model is an array of values corresponding to how well the input matches each of the digit classes "0" to "9". This was the sample output for a test input image of the digit "7":

  {
    "fields": [
      "prediction"
    ],
    "values": [
      [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        1,
        0,
        0
      ]
    ]
  }
  

Step 5: Build and test the app in Xcode

Xcode has facilities for building and testing the app in a simulator, and even for testing the app on your phone wirelessly.

See: Run your app from Xcode

Sample app screens