Deploy a prompt template so you can add it to a business workflow or so you can evaluate the prompt template to measure performance.
Promote a prompt template to a deployment space
To promote a prompt template to a deployment space, your prompt template must contain at least one variable.
You cannot promote prompt templates that do not contain any variables to your deployment space.
To deploy prompt templates with no variables, you must export the Project that contains the prompt template asset and import the Project to your deployment space. For more information, see Importing a prompt template to a deployment space.
Note these additional considerations for deploying a prompt template:
- You can only create deployments of prompt templates that reference model deployments in the same project or space where you created the model deployment. If you promote a template that references a deployed model to a space, a copy of the model asset is automatically added to the space. You must deploy the model in the space before you deploy the prompt template.
- If you export a prompt template that references a deployed model, the associated model asset is automatically exported with the prompt template. When you import the template into a new container, the model is also imported. The model must be deployed in the new container before you deploy the prompt template.
Before you begin
-
Save a prompt template that contains at least one variable as a project asset. See Building reusable prompts.
-
You must set up your task credentials by generating an API key. For more information, see Managing task credentials.
Procedure
To deploy a prompt template, complete the following steps:
- Open the project containing the prompt template.
- Click Promote to space for the template.
-
In the Target deployment space field, choose a deployment space or create a new space. Note the following:
The deployment space must be associated with a machine learning instance that is in the same account as the project where the prompt template was created.
If you don't have a deployment space, choose Create a new deployment space, and then follow the steps in Creating deployment spaces.
If you plan to evaluate the prompt template in the space, the recommended Deployment stage type for the space is Production. For more information on evaluating, see Evaluating a prompt template in a deployment space.
Note: The deployment space stage cannot be changed after the space is created. -
Tip: Select View deployment in deployment space after creating. Otherwise, you need to take more steps to find your deployed asset.
-
From the Assets tab of the deployment space, click Deploy. You create an online deployment, which means you can send data to the endpoint and receive a response in real-time.
-
Optional: In the Deployment serving name field, add a unique label for the deployment.
The serving name is used in the URL for the API endpoint that identifies your deployment. Adding a name is helpful because the human-readable name that you add replaces a long, system-generated unique ID that is assigned otherwise.
The serving name also abstracts the deployment from its service instance details. Applications refer to this name, which allows for the underlying service instance to be changed without impacting users.
The name can have up to 36 characters. The supported characters are [a-z,0-9,_].
The name must be unique across the IBM Cloud region. You might be prompted to change the serving name if the name you choose is already in use.
Importing a prompt template to a deployment space
To deploy prompt templates with no variables, you must export the Project that contains the prompt template asset and import the Project to your deployment space.
Before you begin
Export your Project that contains the prompt template asset as a ZIP file.
Procedure
To import a prompt template asset to a deployment space, follow these steps:
-
From your deployment space toolbar, select the Export/Import space
dropdown menu and click Import space. -
Upload the ZIP file that contains your prompt template asset exported from your Project and click Done.
Testing the deployed prompt template
After the deployment successfully completes, click the deployment name to view the deployment.
- API reference tab includes the API endpoints and code snippets that you need to add this prompt template to an application.
- Test tab supports testing the prompt template. Enter test data as text, streamed text, or in a JSON file.
If the watsonx.governance service is enabled, you also see these tabs:
- Evaluate provides the tools for evaluating the prompt template in the space. Click Activate to choose the dimensions to evaluate. For details, see Evaluating prompt templates.
- AI Factsheets displays all of the metadata that is collected for the prompt template. Use these details for tracking the prompt template for governance and compliance goals. See Tracking prompt templates.
For more information, see Deployment spaces.
Learn more
-
Follow the tutorial Quick start: Evaluate and track a prompt template to evaluate and track a sample prompt template
-
See the blog post IBM watsonx.governance for management and performance assurance of AI models — A prescriptive approach for tips on how to operationalize business applications and workflows with AI inference using the deployed prompt template in a notebook
Parent topic: Deploying and managing assets