This notebook contains steps and code to demonstrate how to manage and clean up Watson Machine Learning instance. This notebook contains steps and code to work with ibm-watson-machine-learning library available in PyPI repository. This notebook introduces commands for listing artifacts, getting artifacts details and deleting them.
Some familiarity with Python is helpful. This notebook uses Python 3.8.
Before you use the sample code in this notebook, you must perform the following setup tasks:
Authenticate the Watson Machine Learning service on IBM Cloud. You need to provide platform api_key
and instance location
.
You can use IBM Cloud CLI to retrieve platform API Key and instance location.
API Key can be generated in the following way:
ibmcloud login
ibmcloud iam api-key-create API_KEY_NAME
In result, get the value of api_key
from the output.
Location of your WML instance can be retrieved in the following way:
ibmcloud login --apikey API_KEY -a https://cloud.ibm.com
ibmcloud resource service-instance WML_INSTANCE_NAME
In result, get the value of location
from the output.
Tip: Your Cloud API key
can be generated by going to the Users section of the Cloud console. From that page, click your name, scroll down to the API Keys section, and click Create an IBM Cloud API key. Give your key a name and click Create, then copy the created key and paste it below. You can also get a service specific url by going to the Endpoint URLs section of the Watson Machine Learning docs. You can check your instance location in your Watson Machine Learning (WML) Service instance details.
You can also get service specific apikey by going to the Service IDs section of the Cloud Console. From that page, click Create, then copy the created key and paste it below.
Action: Enter your api_key
and location
in the following cell.
api_key = 'PASTE YOUR PLATFORM API KEY HERE'
location = 'PASTE YOUR INSTANCE LOCATION HERE'
wml_credentials = {
"apikey": api_key,
"url": 'https://' + location + '.ml.cloud.ibm.com'
}
!pip install -U ibm-watson-machine-learning
from ibm_watson_machine_learning import APIClient
client = APIClient(wml_credentials)
First of all, you need to create a space that will be used for your work with Watson Machine Learning. If you do not have space already created, you can use Deployment Spaces Dashboard to create one.
space_id
and paste it belowTip: You can also use SDK to prepare the space for your work. More information here. ##TODO
Action: Assign space ID below
space_id = 'PASTE YOUR SPACE ID HERE'
You can use list
method to print all existing spaces.
client.spaces.list(limit=10)
To be able to interact with all resources available in Watson Machine Learning, you need to set space which you will be using.
client.set.default_space(space_id)
List existing pipelines. If you want to list only part of pipelines use client.pipelines.list(limit=n_pipelines)
.
client.pipelines.list(limit=10)
Get pipelines details. If you want to get only part of pipelines details use client.pipelines.get_details(limit=n_pipelines)
.
You can get each pipeline details by calling client.pipelines.get_details()
and providing pipeline id from listed pipelines.
pipelines_details = client.pipelines.get_details(limit=10)
print(pipelines_details)
Delete all pipelines. You can delete one pipeline by calling client.pipelines.delete()
and providing pipeline id from listed pipelines.
for pipeline in pipelines_details['resources']:
client.pipelines.delete(pipeline['metadata']['id'])
List existing model definitions. If you want to list only part of model definitions use client.model_definitions.list(limit=n_model_definitions)
.
client.model_definitions.list(limit=10)
Get model definiton details by copying model definition uid from above cell and running client.model_definitions.get_details(model_definition_guid)
.
model_definition_guid = "PUT_YOUR_MODEL_DEFINITION_GUID"
model_definitions_details = client.model_definitions.get_details(model_definition_guid)
print(model_definitions_details)
Delete model definitions by calling client.model_definitions.delete(model_definition_guid)
.
client.model_definitions.delete(model_definition_guid)
List existing models. If you want to list only part of models use client.repository.list_models(limit=n_models)
.
client.repository.list_models(limit=10)
Get model details by copying model uid from above cell and running client.repository.get_details(model_guid)
.
model_guid = "PUT_YOUR_MODEL_GUID"
model_details = client.repository.get_details(model_guid)
print(model_details)
To download selected model from repository use:
client.repository.download(model_guid, <path_to_model>)
# To obtain serialized model first decompress it
!tar xzvf <path_to_model>
client.repository.download(model_guid)
Instead of downloading model can be also loaded directly to runtime using:
model = client.repository.load(model_guid)
# Loaded model can be used to perform prediction locally
# If loaded model was a scikit-learn pipeline we can use 'predict' method
model.predict(<test_data>)
client.repository.load(model_guid)
Delete model from repository by calling client.repository.delete(model_guid)
.
client.repository.delete(model_guid)
List existing functions. If you want to list only part of functions use client.repository.list_functions(limit=n_functions)
.
client.repository.list_functions(limit=10)
Get function details by copying function uid from above cell and running client.repository.get_details(function_guid)
.
function_guid = "PUT_YOUR_FUNCTION_GUID"
function_details = client.repository.get_details(function_guid)
print(function_details)
Delete function from repository by calling client.repository.delete(function_guid)
.
client.repository.delete(function_guid)
List existing experiments. If you want to list only part of experiments use client.pipelines.list(limit=n_experiments)
.
client.experiments.list(limit=10)
Get experiments details. If you want to get only part of experiments details use client.experiments.get_details(limit=n_experiments)
.
You can get each experiment details by calling client.experiments.get_details()
and providing experiment id from listed experiments.
experiments_details = client.experiments.get_details()
print(experiments_details)
Delete all experiments. You can delete one experiment by calling client.experiments.delete()
and providing experiment id from listed experiments.
for experiment in experiments_details['resources']:
client.experiments.delete(experiment['metadata']['id'])
List existing trainings. If you want to list only part of trainings use client.training.list(limit=n_trainings)
.
client.training.list(limit=10)
Get trainings details. If you want to get only part of trainings details use client.training.get_details(limit=n_trainings)
.
You can get each training details by calling client.training.get_details()
and providing training id from listed trainings.
trainings_details = client.training.get_details(limit=10)
print(trainings_details)
Delete all trainings. You can delete one training by calling client.training.cancel()
and providing training id from listed trainings.
Note The client.training.cancel()
method has hard_delete
parameter. Please change it to:
Default value is False
.
for training in trainings_details['resources']:
client.training.cancel(training['metadata']['id'])
List existing deployments. If you want to list only part of deployments use client.deployments.list(limit=n_deployments)
.
client.deployments.list(limit=10)
Get deployments details. If you want to get only part of deployments details use client.deployments.get_details(limit=n_deployments)
.
You can get each deployment details by calling client.deployments.get_details()
and providing deployment id from listed deployments.
deployments_details = client.deployments.get_details()
print(deployments_details)
Delete all deployments. You can delete one deployment by calling client.deployments.delete()
and providing deployment id from listed deployments.
for deployment in deployments_details['resources']:
client.deployments.delete(deployment['metadata']['id'])
You successfully completed this notebook! You learned how to use ibm-watson-machine-learning client for Watson Machine Learning instance management and clean up. Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.
Szymon Kucharczyk, Software Engineer at IBM.
Copyright © 2020, 2021, 2022 IBM. This notebook and its source code are released under the terms of the MIT License.