Core ML tutorial: Build an iOS app that recognizes hand-drawn digits
This tutorial guides you through downloading a Core ML (.mlmodel) file for a IBM Watson Machine Learning model trained in IBM Watson Studio, and then integrating that .mlmodel file into a sample iOS app built using the Apple integrated development environment, Xcode 9.
Attention: This sample runs with a deprecated service instance.
This sample is designed to work with a deprecated V1 machine learning service instance. It will fail with a V2 service instance, provisioned after September 1, 2020.
During the migration period, you can still run tutorials and examples associated with a legacy v1 Watson Machine Learning service instance. However, note the following requirements and restrictions:
- You must use a v1 service instance and associated credentials to run a v1 sample or example. Follow the authentication steps to authenticate with deprecated samples.
- Lite users can use existing v1 service credentials, but cannot create new credentials.
- Standard and Professional users can use existing v1 service credentials and can also create new v1 service credentials. The Credentials page was removed from the IBM Cloud Services catalog, so follow the steps in Generating legacy Watson Machine Learning credentials to create new credentials using the IBM Cloud CLI.
Software requirement
- To build the iOS app, you must have the Apple integrated development environment, Xcode 9, installed on your local computer: Xcode
Steps overview
This tutorial presents the basic steps for integrating a Watson Machine Learning model into an iOS app using Core ML:
- Build the sample model in Watson Studio
- Download the Core ML (.mlmodel) file
- Download the sample iOS app
- Add the .mlmodel file to the sample app
- Build and test the app
Step 1: Build, train, and deploy the sample model in Watson Studio
Work through the flow editor MNIST tutorial to create, train, and deploy a neural network model in Watson Studio:
Step 2: Download the Core ML (.mlmodel) file from Watson Studio to your local computer
On the Assets page of your project, find your trained model in the Models section. From the ACTIONS menu for that model, select "Deploy". (This takes you to the Deployments tab of the model details page.)
In the Deployments tab of the model details page, click Add Deployment.
Fill in deployment details:
- Specify a name for the deployment.
- Select "Virtual" as the Deployment type. (If "Virtual" is not listed as a deployment type option, the model is not using a supported framework.)
- Select "Core ML" as the Format.
- Check the box to specify custom parameters, and then paste this JSON into the text box:
You can read about these parameter here: Conversion parameters for models using Keras{ "image_input_names" : "input1", "image_scale" : 0.003921568627451 }
-
Click Save.
-
In the Implementation tab of the deployment details page that opens, click Download Core ML model.
-
The download is a compressed file. Uncompress the downloaded file to get the Core ML model (.mlmodel) file, which is always named "model.mlmodel" by default.
Step 3: Download the sample iOS app from GitHub to your local computer
Clone or download this GitHub repo to your local computer: go-digits sample iOS app
Step 4: Add the .mlmodel file to the sample app in Xcode
-
Rename the .mlmodel file to: "mnistCNN.mlmodel".
Explanation:
- The .mlmodel file you download from Watson Studio is always named simply "model.mlmodel" by default.
- When you add a .mlmodel file to your Xcode project, Xcode generates a class for the model with the same base name as the .mlmodel file.
- In the sample app code, in QuizQuestionViewController.swift, you can see the code is expecting the generated class's initializer to be named
mnistCNN
:let model = mnistCNN()
-
Open the sample project
Go-Digits.xcodeproj
in Xcode. -
Add the renamed file, "mnistCNN.mlmodel", to your project in Xcode.
Points of interest in the sample code file QuizQuestionViewController.swift:
-
You can see where the model, named "model", is used to analyze an image of the user input, named "input1":
Recall that in step 2, the custom parameterlet output = try? model.prediction(input1: pixelBuffer!)
image_input_names
was set to "input1". -
Also, you can see how the sample code is looping through the confidence scores (
prob
)returned for each digit class, "0" to "9", looking for the digit class that the user input best matches:let output1 = output!.output1 var prob: Double = output1[0].doubleValue var digit:String = "0" for index in 0...output1.count-2 { if output1[index+1].doubleValue >= prob { prob = output1[index + 1].doubleValue digit = String(index+1) } }
*Remember from the flow editor tutorial that the output from the model is an array of values corresponding to how well the input matches each of the digit classes "0" to "9". This was the sample output for a test input image of the digit "7":
{ "fields": [ "prediction" ], "values": [ [ 0, 0, 0, 0, 0, 0, 0, 1, 0, 0 ] ] }
Step 5: Build and test the app in Xcode
Xcode has facilities for building and testing the app in a simulator, and even for testing the app on your phone wirelessly.