Before you begin

1. You must have an existing target deployment space or create a new one where you want to deploy your AI service asset.
2. Build an agentic AI solution in the Agent Lab from your project. For more information, see Agent Lab (beta).
3. You must set up your task credentials by generating an API key. For more information, see Managing task credentials.

Deploying AI services directly

Follow these steps to create an online deployment for an AI service from the Agent Lab tool:

1. To deploy your work from the Agent Lab as an AI service, click Deploy.
2. Enter your deployment details, choose your deployment space, and click Create.

This procedure automatically creates an online deployment for your AI service asset in your project or deployment space. To create a batch deployment for your AI service asset, you must follow the process to manually create a batch deployment from your deployment space. For more information, see Deploying AI service assets.

Deploying AI services with a deployment notebook in Agent Lab

To customize the programming logic of your agentic AI application, you can use the Agent Lab to save your work in a deployment notebook. When you save your work in a deployment notebook, watsonx.ai automatically generates a deployment notebook which captures the logic of your agentic AI application in an AI service.

The deployment notebook contains auto-generated code to promote your AI service asset to a deployment space and create a deployment for the asset. You can edit the deployment notebook for customization, such as creating a batch deployment to deploy an AI service asset instead of an online deployment for your use case.

To save your work in a deployment notebook that contains an AI service from the watsonx.ai Prompt lab, follow these steps:

1. Work with the Agent Lab to build an agentic AI solution with Agents.
2. Click the Save icon and select Save as from the dropdrown menu.
3. In the Save you work dialog box, select Deployment notebook.
   Note: The deployment notebook contains the code to test, promote, and deploy an AI service. To deploy your application, you must save your work in a deployment notebook. You cannot use a standard notebook to deploy an AI service asset.
4. In the Define details section, enter a name and an optional description for your deployment notebook.
5. Click Save.

When you save your work in a deployment notebook, watsonx.ai automatically generates a notebook which contains the code to test, promote, and deploy an AI service. To create an online deployment for your AI service, run the cells in the deployment notebook.

You can customize the auto-generated deployment notebook for your agentic AI applications by clicking the Edit icon.

Inferencing AI services deployed from Agent Lab

The AI service deployed by the notebook can be consumed using a REST API. The following is an example cURL request to call your deployment:

curl --location '${PUBLIC_ENDPOINT}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ${IAM_TOKEN}' \
--data '{ \
  "messages": [{$MESSAGES}]
}'

where

• PUBLIC_ENDPOINT is the public endpoint of your deployment.

• IAM_TOKEN is the authentication token to access IBM Cloud services. The access token that you use must be associated with the same account as the project that is referenced in the notebook.

• MESSAGES is an array of the chat history text entries with the following schema:

  {
   "role": type, // "user" or "assistant"
   "content": content // The text content of the message
  }
  

Before you begin

1. You must have an existing target deployment space or create a new one where you want to deploy your AI service asset.
2. You must create a vector index (in-memory vector store or vector database) to chat with documents. For more information, see Chatting with documents and images.
3. Build a generative AI solution that uses RAG from your project.
4. You must set up your task credentials by generating an API key. For more information, see Managing task credentials.

Deploying an AI service directly

Follow these steps to create an online deployment for an AI service from the watsonx.ai Prompt lab:

1. To deploy your work from the Prompt Lab as an AI service, click Deploy.
2. Enter your deployment details, choose your deployment space, and click Create.

This procedure automatically creates an online deployment for your AI service asset in your project or deployment space. To create a batch deployment for your AI service asset, you must follow the process to manually create a batch deployment from your deployment space. For more information, see Deploying AI service assets.

Deploying AI services with a deployment notebook in Prompt Lab

To customize the programming logic of your generative AI application, you can use the Prompt Lab to save your work in a deployment notebook. When you save your work in a deployment notebook, watsonx.ai automatically generates a deployment notebook which captures the logic of your generative AI application in an AI service.

The deployment notebook contains auto-generated code to promote your AI service asset to a deployment space and create a deployment for the asset. You can edit the deployment notebook for customization, such as creating a batch deployment to deploy an AI service asset instead of an online deployment for your use case.

To save your work in a deployment notebook that contains an AI service from the watsonx.ai Prompt lab, follow these steps:

1. Work with the Prompt Lab to create a generative AI solution.
2. Click the Save icon and select Save as from the dropdrown menu.
3. In the Save you work dialog box, select Deployment notebook.
   Note: The deployment notebook contains the code to test, promote, and deploy an AI service. For deploying your application, you must save your work in a deployment notebook. You cannot use a standard notebook to deploy an AI service asset.
4. In the Define details section, enter a name and an optional description for your deployment notebook.
5. Click Save.

When you save your work in a deployment notebook, watsonx.ai automatically generates a notebook which contains the code to test, promote, and deploy an AI service. To create an online deployment for your AI service, run the cells in the deployment notebook.

You can customize the auto-generated deployment notebook for your generative AI applications by clicking the Edit icon .

Inferencing AI services deployed from Prompt Lab

The AI service deployed by the notebook can be consumed using a REST API. The following is an example cURL request to call your deployment:

curl --location '${PUBLIC_ENDPOINT}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ${IAM_TOKEN}' \
--data '{ \
  "input_data": [{
      "fields": ["Search", "Access token"],
      "values": [
        [${MESSAGES}],
        [${IAM_TOKEN}]]
    }]
}'

where

• PUBLIC_ENDPOINT is the public endpoint of your deployment.

• IAM_TOKEN is the authentication token to access IBM Cloud services. The access token that you use must be associated with the same account as the project that is referenced in the notebook.

• MESSAGES is an array of the chat history text entries with the following schema:

  {
   "role": type, // "user" or "assistant"
   "content": content // The text content of the message
  }
  

Before you begin

1. You must have an existing target deployment space or create a new one where you want to deploy your AI service asset.
2. You must create a vector index (in-memory vector store or vector database) to chat with documents. For more information, see Chatting with documents and images.
3. Build a generative AI solution that uses RAG with the AutoAI experiment builder from your project. For more information, see Building an AutoAI experiment that uses RAG.
4. You must set up your task credentials by generating an API key. For more information, see Managing task credentials.

Deploying AI services directly

Follow these steps to create an online deployment for an AI service from the AutoAI experiment builder tool:

1. To deploy your work from the AutoAI experiment builder, choose the best performing pipeline for deployment and click Save as.
2. Choose Retrieval and generation as the objective, and select the AI service asset type.
3. Enable the option to promote and deploy the AI service to a deployment space.
4. Choose your deployment space and click Create and save.

This procedure automatically creates an online deployment for your AI service asset in your project or deployment space. To create a batch deployment for your AI service asset, you must follow the process to manually create a batch deployment from your deployment space. For more information, see Deploying AI service assets.