Deployable asset types for online deployments

You can create online deployments for these types of assets:

• Functions
• Models
  • AutoAI models
  • Decision Optimization models
  • Federated learning models
  • Models that are imported from a file
  • Models that are created with supported frameworks
  • Tuned foundation models
  • SPSS Modeler models
• Prompt templates
• Scripts

Ways to create an online deployment

You can create an online deployment one of these ways:

• Use a no-code approach to Create an online deployment from a deployment space.
• Use code to Create an online deployment programmatically by using notebooks.

After you create an online deployment, an endpoint URL is generated. You can use the endpoint URL to test the deployment or to include the deployment in your applications.

Before you begin

You must set up your task credentials by generating an API key. For more information, see Managing task credentials.

Creating an online deployment from a deployment space

Follow these steps to create your online deployment from a deployment space:

1. Promote or add the asset that you want to deploy to a space. For more information, see Assets in a deployment space.

2. From the Assets page, click Deploy from the action menu.

3. From the deployment details page, click New deployment.

4. Choose Online as the deployment type.

5. Provide a name and an optional description for the deployment.

6. Use the Serving name field to specify a name for your deployment instead of deployment ID.

   Note:

   • The serving name must be distinctive within the namespace.
   • The serving name must contain only these characters: [a-z,0-9,_] and must be a maximum 36 characters long.
   • The serving name works only as part of the prediction URL. Sometime, you may have to use the deployment ID.

7. Optional: Select a hardware specification if you're deploying Python functions, Tensorflow models, or models with custom software specifications. For example, if you are scaling a deployment, you might want to increase the hardware resources.

   Restriction:

   You cannot create or select custom hardware specifications from the user interface in a deployment space. To learn more about ways to create and select a hardware specification, see Managing hardware specifications for deployments.

8. Optional: Select a software specification that is compatible with your asset. You can select software specifications for functions only.

9. Click Create to create the deployment.

{"input_data":[{
        "fields": [<field1>, <field2>, ...],
        "values": [[<value1>, <value2>, ...]]
}]}

{"input_data":[{
        "fields": ["PassengerId","Pclass","Name","Sex","Age","SibSp","Parch","Ticket","Fare","Cabin","Embarked"],
        "values": [[1,3,"Braund, Mr. Owen Harris",0,22,1,0,"A/5 21171",7.25,null,"S"]]
}]}

{
    "parameters": {
        "prompt_variables": {
            "input": "input text goes here."
        }
    }
}

Creating online deployment programmatically by using notebooks

You can create an online deployment programmatically by using:

• watsonx.ai Runtime REST API.
• watsonx Python client library.

To access sample notebooks that use demonstrate how to create and manage deployments by using watsonx.ai Runtime Python client, see Machine learning samples and examples.

model_details = client.repository.get_details("<model_id>")  # retrieves details and includes schema
columns_in_schema = []
for i in range(0, len(model_details['entity']['schemas']['input'][0].get('fields'))):
    columns_in_schema.append(model_details['entity']['schemas']['input'][0].get('fields')[i]['name'])

X = X[columns_in_schema] # where X is a pandas dataframe that contains values to be scored
#(...)
scoring_values = X.values.tolist()
array_of_input_fields = X.columns.tolist()
payload_scoring = {"input_data": [{"fields": [array_of_input_fields],"values": scoring_values}]}

Learn more

To learn how to create a batch deployment, see Creating a batch deployment.

To learn how to manage deployment jobs, and update, scale, or delete an online deployment, see Managing assets.

To learn more about deployment endpoints, see Endpoint URLs.

Parent topic: Managing predictive deployments