You can deploy a Decision Optimization model, create and monitor jobs and get solutions using the Watson Machine Learning REST API.
Before you begin
- Log in to IBM Cloud.
- Create your API key. Copy or download it from the API key successfully created open window (you cannot access it again once you close this pane).
- Create or select a Machine Learning service. Copy the service instance name, GUID and CRN from the information pane for your instance in the Resource List>Services view on IBM Cloud. (Expand the list of services in the Resource List window. Click anywhere in the row next to your Machine Learning Service name, but not on the name itself. This opens the information pane in the same window.)
- Create or select a Cloud Object Storage. Copy the Cloud Object Storage instance name and CRN from the information pane from the information pane for your instance in the Resource List>Storage view on IBM Cloud.
- From the https://dataplatform.cloud.ibm.com user interface, create a deployment space. Then view it and copy your Space ID from the settings tab. For more information see Creating a deployment space.
- With the REST API. See Creating a deployment space using the REST API.
About this task
For Windows users, use ^ instead of \ for the multi-line separator and double quotation marks " throughout these code examples. Windows users also need to use indentation of at least one character space in the header lines.
For clarity, some code examples in this procedure have been placed in a json file to make the commands more readable and easier to use.
Once you have created a deployment using the REST API, you can also view it and send jobs to it from the Deployment spaces page in the https://dataplatform.cloud.ibm.com user interface.
Procedure
-
Generate an IAM token using your IBM Cloud API key as follows.
curl "https://iam.bluemix.net/identity/token" \ -d "apikey=YOUR_API_KEY_HERE&grant_type=urn%3Aibm%3Aparams%3Aoauth%3Agrant-type%3Aapikey" \ -H "Content-Type: application/x-www-form-urlencoded" \ -H "Authorization: Basic Yng6Yng="
Output example:{ "access_token": "****** obtained IAM token ******************************", "refresh_token": "**************************************", "token_type": "Bearer", "expires_in": 3600, "expiration": 1554117649, "scope": "ibm openid" }
Use the obtained token (access_token value) prepended by the word
Bearer
in theAuthorization
header, and theMachine Learning service GUID
in theML-Instance-ID
header, in all API calls. - Optional: If you have not obtained your
SPACE-ID from the user interface as described previously, you can create a space using the
REST API as follows. Use the previously obtained IAM token prepended by the word
bearer
in theAuthorization
header in all API calls.curl --location --request POST \ "https://api.dataplatform.cloud.ibm.com/v2/spaces" \ -H "Authorization: Bearer TOKEN-HERE" \ -H "ML-Instance-ID: MACHINE-LEARNING-SERVICE-GUID-HERE" \ -H "Content-Type: application/json" \ --data-raw "{ "name": "SPACE-NAME-HERE", "description": "optional description here", "storage": { "resource_crn": "COS-CRN-ID-HERE" }, "compute": [{ "name": "MACHINE-LEARNING-SERVICE-NAME-HERE", "crn": "MACHINE-LEARNING-SERVICE-CRN-ID-HERE" }] }"
For Windows users, put the--data-raw
command on one line and replace all"
with\"
inside this command as follows:
Alternatively put the data in a separate file.curl --location --request POST ^ "https://api.dataplatform.cloud.ibm.com/v2/spaces" ^ -H "Authorization: Bearer TOKEN-HERE" ^ -H "ML-Instance-ID: MACHINE-LEARNING-SERVICE-GUID-HERE" ^ -H "Content-Type: application/json" ^ --data-raw "{\"name\": "SPACE-NAME-HERE",\"description\": \"optional description here\",\"storage\": {\"resource_crn\": \"COS-CRN-ID-HERE\" },\"compute\": [{\"name\": "MACHINE-LEARNING-SERVICE-NAME-HERE\",\"crn\": \"MACHINE-LEARNING-SERVICE-CRN-ID-HERE\" }]}"
A SPACE-ID is returned inid
field of themetadata
section.Output example:
{ "entity": { "compute": [ { "crn": "MACHINE-LEARNING-SERVICE-CRN", "guid": "MACHINE-LEARNING-SERVICE-GUID", "name": "MACHINE-LEARNING-SERVICE-NAME", "type": "machine_learning" } ], "description": "string", "members": [ { "id": "XXXXXXX", "role": "admin", "state": "active", "type": "user" } ], "name": "name", "scope": { "bss_account_id": "account_id" }, "status": { "state": "active" } }, "metadata": { "created_at": "2020-07-17T08:36:57.611Z", "creator_id": "XXXXXXX", "id": "SPACE-ID", "url": "/v2/spaces/SPACE-ID" } }
You must wait until your deployment space status is
"active"
before continuing. You can poll to check for this as follows.curl --location --request GET "https://api.dataplatform.cloud.ibm.com/v2/spaces/SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json"
- Create a new Decision
Optimization model
All API requests require a version parameter that takes a date in the format
version=YYYY-MM-DD
. This code example posts a model that uses the filecreate_model.json
. The URL will vary according to the chosen region/location for your machine learning service. See Endpoint URLs.
The create_model.json file contains the following code:curl --location --request POST \ "https://us-south.ml.cloud.ibm.com/ml/v4/models?version=2020-08-01" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json" \ -d @create_model.json
{ "name": "ModelName", "description": "ModelDescription", "type": "do-docplex_22.1", "software_spec": { "name": "do_22.1" }, "custom": { "decision_optimization": { "oaas.docplex.python": "3.10" } }, "space_id": "SPACE-ID-HERE" }
The Python version is stated explicitly here in a
custom
block. This is optional. Without it your model will use the default version which is currently Python 3.10. As the default version will evolve over time, stating the Python version explicitly enables you to easily change it later or to keep using an older supported version when the default version is updated. Currently supported versions are 3.10.If you want to be able to run jobs for this model from the user interface, instead of only using the REST API , you must define the schema for the input and output data. If you do not define the schema when you create the model, you can only run jobs using the REST API and not from the user interface.
You can also use the schema specified for input and output in your optimization model:
{ "name": "Diet-Model-schema", "description": "Diet", "type": "do-docplex_22.1", "schemas": { "input": [ { "id": "diet_food_nutrients", "fields": [ { "name": "Food", "type": "string" }, { "name": "Calories", "type": "double" }, { "name": "Calcium", "type": "double" }, { "name": "Iron", "type": "double" }, { "name": "Vit_A", "type": "double" }, { "name": "Dietary_Fiber", "type": "double" }, { "name": "Carbohydrates", "type": "double" }, { "name": "Protein", "type": "double" } ] }, { "id": "diet_food", "fields": [ { "name": "name", "type": "string" }, { "name": "unit_cost", "type": "double" }, { "name": "qmin", "type": "double" }, { "name": "qmax", "type": "double" } ] }, { "id": "diet_nutrients", "fields": [ { "name": "name", "type": "string" }, { "name": "qmin", "type": "double" }, { "name": "qmax", "type": "double" } ] } ], "output": [ { "id": "solution", "fields": [ { "name": "name", "type": "string" }, { "name": "value", "type": "double" } ] } ] }, "software_spec": { "name": "do_22.1" }, "space_id": "SPACE-ID-HERE" }
When you post a model you provide information about its model type and the software specification to be used.Model types can be, for example:do-opl_22.1
for OPL modelsdo-cplex_22.1
for CPLEX modelsdo-cpo_22.1
for CP modelsdo-docplex_22.1
for Python models
Version 20.1 can also be used for these model types.
For the software specification, you can use the default specifications using their names
do_22.1
ordo_20.1
. See also Extend software specification notebook which shows you how to extend the Decision Optimization software specification (runtimes with additional Python libraries for docplex models).A MODEL-ID is returned in
id
field in themetadata
.Output example:{ "entity": { "software_spec": { "id": "SOFTWARE-SPEC-ID" }, "type": "do-docplex_20.1" }, "metadata": { "created_at": "2020-07-17T08:37:22.992Z", "description": "ModelDescription", "id": "MODEL-ID", "modified_at": "2020-07-17T08:37:22.992Z", "name": "ModelName", "owner": "***********", "space_id": "SPACE-ID" } }
-
Upload a Decision
Optimization model formulation ready for
deployment. First compress your model into a (
tar.gz, .zip or .jar
) file and upload it to be deployed by the Watson Machine Learning service.This code example uploads a model called diet.zip that contains a Python model and no common data:curl --location --request PUT \ "https://us-south.ml.cloud.ibm.com/ml/v4/models/MODEL-ID-HERE/content?version=2020-08-01&space_id=SPACE-ID-HERE&content_format=native" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/gzip" \ --data-binary "@diet.zip"
You can download this example and other models from the DO-samples. Select the relevant product and version subfolder. - Deploy your model Create a reference to your model. Use the SPACE-ID, the MODEL-ID obtained when you created your model ready for deployment and the hardware specification. For example:
curl --location --request POST "https://us-south.ml.cloud.ibm.com/ml/v4/deployments?version=2020-08-01" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json" \ -d @deploy_model.json
The deploy_model.json file contains the following code:
{ "name": "Test-Diet-deploy", "space_id": "SPACE-ID-HERE", "asset": { "id": "MODEL-ID-HERE" }, "hardware_spec": { "name": "S" }, "batch": {} }
The DEPLOYMENT-ID is returned inid
field in themetadata
. Output example:{ "entity": { "asset": { "id": "MODEL-ID" }, "custom": {}, "description": "", "hardware_spec": { "id": "HARDWARE-SPEC-ID", "name": "S", "num_nodes": 1 }, "name": "Test-Diet-deploy", "space_id": "SPACE-ID", "status": { "state": "ready" } }, "metadata": { "created_at": "2020-07-17T09:10:50.661Z", "description": "", "id": "DEPLOYMENT-ID", "modified_at": "2020-07-17T09:10:50.661Z", "name": "test-Diet-deploy", "owner": "**************", "space_id": "SPACE-ID" } }
-
Once deployed, you can monitor your model's deployment state. Use the
DEPLOYMENT-ID.
For example:
curl --location --request GET "https://us-south.ml.cloud.ibm.com/ml/v4/deployments/DEPLOYMENT-ID-HERE?version=2020-08-01&space_id=SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json"
Output example:
{ "entity": { "asset": { "id": "MODEL-ID" }, "custom": {}, "description": "", "hardware_spec": { "id": "HARDWARE-SPEC-ID", "name": "S", "num_nodes": 1 }, "name": "Test-Diet-deploy", "space_id": "SPACE-ID", "status": { "state": "ready" } }, "metadata": { "created_at": "2020-11-04T16:22:14.442Z", "description": "", "id": "DEPLOYMENT-ID", "modified_at": "2020-11-04T16:22:14.442Z", "name": "test-Diet-deploy", "owner": "**************", "space_id": "SPACE-ID" } }
- You can then Submit jobs for your deployed model defining the input data and the
output (results of the optimization solve) and the log file. For example, the following shows the contents of a file called
myjob.json
. It contains (inline) input data, some solve parameters, and specifies that the output will be a .csv file. For examples of other types of input data references, see Model input and output data adaptation.
This code example posts a job that uses this file{ "name":"test-job-diet", "space_id": "SPACE-ID-HERE", "deployment": { "id": "DEPLOYMENT-ID-HERE" }, "decision_optimization" : { "solve_parameters" : { "oaas.logAttachmentName":"log.txt", "oaas.logTailEnabled":"true" }, "input_data": [ { "id":"diet_food.csv", "fields" : ["name","unit_cost","qmin","qmax"], "values" : [ ["Roasted Chicken", 0.84, 0, 10], ["Spaghetti W/ Sauce", 0.78, 0, 10], ["Tomato,Red,Ripe,Raw", 0.27, 0, 10], ["Apple,Raw,W/Skin", 0.24, 0, 10], ["Grapes", 0.32, 0, 10], ["Chocolate Chip Cookies", 0.03, 0, 10], ["Lowfat Milk", 0.23, 0, 10], ["Raisin Brn", 0.34, 0, 10], ["Hotdog", 0.31, 0, 10] ] }, { "id":"diet_food_nutrients.csv", "fields" : ["Food","Calories","Calcium","Iron","Vit_A","Dietary_Fiber","Carbohydrates","Protein"], "values" : [ ["Spaghetti W/ Sauce", 358.2, 80.2, 2.3, 3055.2, 11.6, 58.3, 8.2], ["Roasted Chicken", 277.4, 21.9, 1.8, 77.4, 0, 0, 42.2], ["Tomato,Red,Ripe,Raw", 25.8, 6.2, 0.6, 766.3, 1.4, 5.7, 1], ["Apple,Raw,W/Skin", 81.4, 9.7, 0.2, 73.1, 3.7, 21, 0.3], ["Grapes", 15.1, 3.4, 0.1, 24, 0.2, 4.1, 0.2], ["Chocolate Chip Cookies", 78.1, 6.2, 0.4, 101.8, 0, 9.3, 0.9], ["Lowfat Milk", 121.2, 296.7, 0.1, 500.2, 0, 11.7, 8.1], ["Raisin Brn", 115.1, 12.9, 16.8, 1250.2, 4, 27.9, 4], ["Hotdog", 242.1, 23.5, 2.3, 0, 0, 18, 10.4 ] ] }, { "id":"diet_nutrients.csv", "fields" : ["name","qmin","qmax"], "values" : [ ["Calories", 2000, 2500], ["Calcium", 800, 1600], ["Iron", 10, 30], ["Vit_A", 5000, 50000], ["Dietary_Fiber", 25, 100], ["Carbohydrates", 0, 300], ["Protein", 50, 100] ] } ], "output_data": [ { "id":".*\\.csv" } ] } }
myjob.json
.curl --location --request POST "https://us-south.ml.cloud.ibm.com/ml/v4/deployment_jobs?version=2020-08-01&space_id=SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json" \ -H "cache-control: no-cache" \ -d @myjob.json
A JOB-ID is returned. Output example: (the job is queued){ "entity": { "decision_optimization": { "input_data": [{ "id": "diet_food.csv", "fields": ["name", "unit_cost", "qmin", "qmax"], "values": [["Roasted Chicken", 0.84, 0, 10], ["Spaghetti W/ Sauce", 0.78, 0, 10], ["Tomato,Red,Ripe,Raw", 0.27, 0, 10], ["Apple,Raw,W/Skin", 0.24, 0, 10], ["Grapes", 0.32, 0, 10], ["Chocolate Chip Cookies", 0.03, 0, 10], ["Lowfat Milk", 0.23, 0, 10], ["Raisin Brn", 0.34, 0, 10], ["Hotdog", 0.31, 0, 10]] }, { "id": "diet_food_nutrients.csv", "fields": ["Food", "Calories", "Calcium", "Iron", "Vit_A", "Dietary_Fiber", "Carbohydrates", "Protein"], "values": [["Spaghetti W/ Sauce", 358.2, 80.2, 2.3, 3055.2, 11.6, 58.3, 8.2], ["Roasted Chicken", 277.4, 21.9, 1.8, 77.4, 0, 0, 42.2], ["Tomato,Red,Ripe,Raw", 25.8, 6.2, 0.6, 766.3, 1.4, 5.7, 1], ["Apple,Raw,W/Skin", 81.4, 9.7, 0.2, 73.1, 3.7, 21, 0.3], ["Grapes", 15.1, 3.4, 0.1, 24, 0.2, 4.1, 0.2], ["Chocolate Chip Cookies", 78.1, 6.2, 0.4, 101.8, 0, 9.3, 0.9], ["Lowfat Milk", 121.2, 296.7, 0.1, 500.2, 0, 11.7, 8.1], ["Raisin Brn", 115.1, 12.9, 16.8, 1250.2, 4, 27.9, 4], ["Hotdog", 242.1, 23.5, 2.3, 0, 0, 18, 10.4]] }, { "id": "diet_nutrients.csv", "fields": ["name", "qmin", "qmax"], "values": [["Calories", 2000, 2500], ["Calcium", 800, 1600], ["Iron", 10, 30], ["Vit_A", 5000, 50000], ["Dietary_Fiber", 25, 100], ["Carbohydrates", 0, 300], ["Protein", 50, 100]] }], "output_data": [ { "id": ".*\\.csv" } ], "solve_parameters": { "oaas.logAttachmentName": "log.txt", "oaas.logTailEnabled": "true" }, "status": { "state": "queued" } }, "deployment": { "id": "DEPLOYMENT-ID" }, "platform_job": { "job_id": "", "run_id": "" } }, "metadata": { "created_at": "2020-07-17T10:42:42.783Z", "id": "JOB-ID", "name": "test-job-diet", "space_id": "SPACE-ID" } }
- You can also monitor job states. Use the JOB-ID For example:
curl --location --request GET \ "https://us-south.ml.cloud.ibm.com/ml/v4/deployment_jobs/JOB-ID-HERE?version=2020-08-01&space_id=SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json"
Output example: (job has completed){ "entity": { "decision_optimization": { "input_data": [{ "id": "diet_food.csv", "fields": ["name", "unit_cost", "qmin", "qmax"], "values": [["Roasted Chicken", 0.84, 0, 10], ["Spaghetti W/ Sauce", 0.78, 0, 10], ["Tomato,Red,Ripe,Raw", 0.27, 0, 10], ["Apple,Raw,W/Skin", 0.24, 0, 10], ["Grapes", 0.32, 0, 10], ["Chocolate Chip Cookies", 0.03, 0, 10], ["Lowfat Milk", 0.23, 0, 10], ["Raisin Brn", 0.34, 0, 10], ["Hotdog", 0.31, 0, 10]] }, { "id": "diet_food_nutrients.csv", "fields": ["Food", "Calories", "Calcium", "Iron", "Vit_A", "Dietary_Fiber", "Carbohydrates", "Protein"], "values": [["Spaghetti W/ Sauce", 358.2, 80.2, 2.3, 3055.2, 11.6, 58.3, 8.2], ["Roasted Chicken", 277.4, 21.9, 1.8, 77.4, 0, 0, 42.2], ["Tomato,Red,Ripe,Raw", 25.8, 6.2, 0.6, 766.3, 1.4, 5.7, 1], ["Apple,Raw,W/Skin", 81.4, 9.7, 0.2, 73.1, 3.7, 21, 0.3], ["Grapes", 15.1, 3.4, 0.1, 24, 0.2, 4.1, 0.2], ["Chocolate Chip Cookies", 78.1, 6.2, 0.4, 101.8, 0, 9.3, 0.9], ["Lowfat Milk", 121.2, 296.7, 0.1, 500.2, 0, 11.7, 8.1], ["Raisin Brn", 115.1, 12.9, 16.8, 1250.2, 4, 27.9, 4], ["Hotdog", 242.1, 23.5, 2.3, 0, 0, 18, 10.4]] }, { "id": "diet_nutrients.csv", "fields": ["name", "qmin", "qmax"], "values": [["Calories", 2000, 2500], ["Calcium", 800, 1600], ["Iron", 10, 30], ["Vit_A", 5000, 50000], ["Dietary_Fiber", 25, 100], ["Carbohydrates", 0, 300], ["Protein", 50, 100]] }], "output_data": [{ "fields": ["Name", "Value"], "id": "kpis.csv", "values": [["Total Calories", 2000], ["Total Calcium", 800.0000000000001], ["Total Iron", 11.278317739831891], ["Total Vit_A", 8518.432542485823], ["Total Dietary_Fiber", 25], ["Total Carbohydrates", 256.80576358904455], ["Total Protein", 51.17372234135308], ["Minimal cost", 2.690409171696264]] }, { "fields": ["name", "value"], "id": "solution.csv", "values": [["Spaghetti W/ Sauce", 2.1551724137931036], ["Chocolate Chip Cookies", 10], ["Lowfat Milk", 1.8311671008899097], ["Hotdog", 0.9296975991385925]] }], "output_data_references": [], "solve_parameters": { "oaas.logAttachmentName": "log.txt", "oaas.logTailEnabled": "true" }, "solve_state": { "details": { "KPI.Minimal cost": "2.690409171696264", "KPI.Total Calcium": "800.0000000000001", "KPI.Total Calories": "2000.0", "KPI.Total Carbohydrates": "256.80576358904455", "KPI.Total Dietary_Fiber": "25.0", "KPI.Total Iron": "11.278317739831891", "KPI.Total Protein": "51.17372234135308", "KPI.Total Vit_A": "8518.432542485823", "MODEL_DETAIL_BOOLEAN_VARS": "0", "MODEL_DETAIL_CONSTRAINTS": "7", "MODEL_DETAIL_CONTINUOUS_VARS": "9", "MODEL_DETAIL_INTEGER_VARS": "0", "MODEL_DETAIL_KPIS": "[\"Total Calories\", \"Total Calcium\", \"Total Iron\", \"Total Vit_A\", \"Total Dietary_Fiber\", \"Total Carbohydrates\", \"Total Protein\", \"Minimal cost\"]", "MODEL_DETAIL_NONZEROS": "57", "MODEL_DETAIL_TYPE": "LP", "PROGRESS_CURRENT_OBJECTIVE": "2.6904091716962637" }, "latest_engine_activity": [ "[2020-07-21T16:37:36Z, INFO] Model: diet", "[2020-07-21T16:37:36Z, INFO] - number of variables: 9", "[2020-07-21T16:37:36Z, INFO] - binary=0, integer=0, continuous=9", "[2020-07-21T16:37:36Z, INFO] - number of constraints: 7", "[2020-07-21T16:37:36Z, INFO] - linear=7", "[2020-07-21T16:37:36Z, INFO] - parameters: defaults", "[2020-07-21T16:37:36Z, INFO] - problem type is: LP", "[2020-07-21T16:37:36Z, INFO] Warning: Model: \"diet\" is not a MIP problem, progress listeners are disabled", "[2020-07-21T16:37:36Z, INFO] objective: 2.690", "[2020-07-21T16:37:36Z, INFO] \"Spaghetti W/ Sauce\"=2.155", "[2020-07-21T16:37:36Z, INFO] \"Chocolate Chip Cookies\"=10.000", "[2020-07-21T16:37:36Z, INFO] \"Lowfat Milk\"=1.831", "[2020-07-21T16:37:36Z, INFO] \"Hotdog\"=0.930", "[2020-07-21T16:37:36Z, INFO] solution.csv" ], "solve_status": "optimal_solution" }, "status": { "completed_at": "2020-07-21T16:37:36.989Z", "running_at": "2020-07-21T16:37:35.622Z", "state": "completed" } }, "deployment": { "id": "DEPLOYMENT-ID" } }, "metadata": { "created_at": "2020-07-21T16:37:09.130Z", "id": "JOB-ID", "modified_at": "2020-07-21T16:37:37.268Z", "name": "test-job-diet", "space_id": "SPACE-ID" } }
- Optional: You can delete jobs as follows:
curl --location --request DELETE "https://us-south.ml.cloud.ibm.com/ml/v4/deployment_jobs/JOB-ID-HERE?version=2020-08-01&space_id=SPACE-ID-HERE&hard_delete=true" \ -H "Authorization: bearer TOKEN-HERE"
If you delete a job using the API, it will still be displayed in the user interface. - Optional: You can delete deployments as follows:
curl --location --request DELETE "https://us-south.ml.cloud.ibm.com/ml/v4/deployments/DEPLOYMENT-ID-HERE?version=2020-08-01&space_id=SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE"
If you delete a deployment that contains jobs using the API, the jobs will still be displayed in the deployment space in the user interface.
Results
Once your model has been deployed and job executed, the solution results are provided either inline or in the file and location that you specified, for example using an S3 reference. You can post new jobs using the deployment-ID without having to redeploy your model.