Deploying AI assets
Using IBM Watson Machine Learning, you can deploy machine learning models, scripts, and functions, and prompt templates for generative AI models. After you create deployments, you can test and manage them, and prepare your assets to deploy into pre-production and production environments to generate predictions and insights.
Service The administrator must install the Watson Machine Learning service on Cloud Pak for Data platform to use its capabilities.
Deployments overview
You can use Watson Machine Learning to create an online, batch, or application deployment for machine learning, foundation model, and Decison Optimization assets.
The following graphic shows the typical activities to deploy AI assets:
Types of deployments
The most common types of deployments are as follows:
-
Online deployment: Create an online deployment to process input data in real-time. To test the online deployment in real-time, you can submit new customer data to the deployment endpoint to get a prediction in real-time.
-
Batch deployment: Create a batch deployment to process a large batch of input data from a data source and write the output to a selected destination. You can configure the batch deployment job and run the job on a schedule or on demand.
-
Application deployment: Create an application deployment to deploy your application assets, such as R Shiny applications.
Types of deployable assets
The type of asset that you deploy dictates the type of deployment that you can create. For example, Python functions, scripts, and models, such as AutoAI or Decision Optimization models support online and batch deployments. However, you can create online deployments only for models that are imported from a file. The different types of deployable assets are as follows:
-
Foundation model assets: You can deploy foundation model assets such as tuned model, prompt template assets, or custom foundation models with watsonx.ai.
-
Machine Learning assets: You can deploy machine learning Machine Learning assets such as Python functions, R Shiny applications, NLP models, scripts, and more with Watson Machine Learning.
-
Decision Optimization models: You can deploy Decision Optimization model with Watson Machine Learning.
Ways to deploy assets
You can deploy and manage your assets in the following ways:
-
Deploy and manage assets in a deployment space by using a no-code approach.
-
Deploy and manage assets programmatically by using a custom-code approach.
Deploying and managing assets in deployment spaces
Create a deployment space to collaborate with stakeholders and deploy and manage assets in a deployment space.
To manage your assets within a deployment space, you must promote your assets from a project to your deployment space. You can also import or export assets from your deployment space. You can manage assets and deployments in your deployment space by updating details, scaling the number of copies, monitoring performance, or deleting the deployment to free up resources.
To use you deployed asset in applications for making predictions, retrieve the endpoint URL for online and batch deployments. The model endpoint provides access to an interface to invoke and manage model deployments.
Use the deployments dashboard to get an aggregate view of your deployments and monitor deployment activity. You can use the dashboard to monitor the status of your batch deployment jobs, such as active runs and finished runs based on job schedule that you defined when you created the job. You can also get information about the number of successful and failed online deployments.
Deploying and managing assets programmatically
You can deploy and manage assets programmatically by using Python client or Watson Machine Learning API. .
Managing frameworks and software specifications for deployments
Software specifications and frameworks contain bundles of packages with corresponding versions of the packages.
You can use predefined software specifications or create custom software specifications by adding new packages to existing frameworks, create new packages, or updating package versions in software specifications.
Managing runtime environments for deployments
Runtime envionments provide the necessary functions that are required to run your deployment.
You can use predefined runtime environments or create custom runtime environments to include more components, depending on your use case. To create a custom runtime environment for your deployment, you must
create a Dockerfile and add a base image. Further, you can add the docker commands to build the runtime environment for your deployment.
Learn more
Parent topic: Deploying and managing AI assets