Prompt Lab

Chat mode

You chat with the foundation model.

You start the chat by submitting a query or request for the foundation model to answer. Each subsequent turn in the conversation builds on information that was exchanged previously.

Alternatively, you can click a quick start sample to submit to the model. Quick start samples are sent to the Llama foundation model. If you want to work with a different foundation model, add your own prompt text.

Before starting a chat, review and adjust the model choice and parameter settings. To support long dialog exchanges, the Max tokens parameter is set to a high default value. You might want to add a stop sequence to prevent the model from generating wordy outputs, for example.

Predefined text called a system prompt is included at the start of the chat to establish ground rules for the conversation. To review and customize the text, click Edit system prompt .

Some foundation models recommend specific templates that identify different segments of the prompt, such as the prompt instruction and user input. Chat mode adjusts the syntax of your prompt input to conform to each foundation model's recommended format. You can click the text icon to see the full prompt text that will be submitted to the foundation model.

The following features are omitted from chat mode:

• The token usage count is not shown in chat mode.

  Keep in mind that the chat history is sent with each new prompt that you submit which contributes to the overall token count.

  You can check the token count yourself by using the API. Click the text icon to open and copy the full prompt text, and then use the Text tokenization method to count the tokens.

• You cannot define prompt variables in chat mode. As a consequence, you cannot govern saved chat prompt templates.

Watch this video showing Chat mode in the Prompt Lab.

Structured mode

You add parts of your prompt into the appropriate fields:

• Instruction: Add an instruction if it makes sense for your use case. An instruction is an imperative statement, such as Summarize the following article.
• Examples: Add one or more pairs of examples that contain the input and the corresponding output that you want. Providing a few example input-and-output pairs in your prompt is called few-shot prompting. If you need a specific prefix to the input or the output, you can replace the default labels, "Input:" or "Output:", with the labels you want to use. A space is added between the example label and the example text.
• Test your input: In the Try area, enter the final input of your prompt.

Structured mode is designed to help new users create effective prompts. Text from the fields is sent to the model in a template format.

Freeform mode

You add your prompt in plain text. Your prompt text is sent to the model exactly as you typed it.

• Quotation marks in your text are escaped with a backslash (\").
• Newline characters are represented by \n.
• Apostrophes are escaped (it'\''s) so that they can be handled properly in the cURL command.

Model choices

In the Prompt Lab, you can submit your prompt to any of the models that are supported by watsonx.ai. You can choose recently-used models from the drop-down list. Or you can click View all foundation models to view all the supported models, filter them by task, and read high-level information about the models.

If you tuned a foundation model by using the Tuning Studio and deployed the tuned model, your tuned model is also available for prompting from the Prompt Lab.

Model parameters

To control how the model generates output in response to your prompt, you can specify decoding parameters and stopping criteria. For more information, see Model parameters for prompting.

Prompt variables

To add flexibility to your prompts, you can define prompt variables. A prompt variable is a placeholder keyword that you include in the static text of your prompt at creation time and replace with text dynamically at run time. For more information, see Building reusable prompts.

View full prompt text

You might want to see the full prompt text that will be submitted to the foundation model in the following situations:

• When prompt variables are in use, to see resolved variable values in context.
• In chat mode, where the recommended prompt formats for different foundation models are applied automatically.
• In structured mode, where you add parts of the prompt into separate fields.

AI guardrails

When you set the AI guardrails switcher to On, harmful language is automatically removed from the input prompt text and from the output that is generated by the model. Specifically, any sentence in the input or output that contains harmful language is replaced with a message that says that potentially harmful text was removed.

For more information, see Removing harmful content.

View code

When you click the View code icon , a cURL command is displayed that you can call from outside the Prompt Lab to submit the current prompt and parameters to the selected model and get a generated response.

In the command, there is a placeholder for an IBM Cloud IAM token. For information about generating the access token, see Generating an IBM Cloud IAM token.

Programmatic alternative to the Prompt Lab

The Prompt Lab graphical interface is a great place to experiment and iterate with your prompts. However, you can also prompt foundation models in watsonx.ai programmatically by using the Python library or REST API. For details, see Coding generative AI solutions.

Samples

A collection of sample prompts are available in the Prompt Lab. The samples demonstrate effective prompt text and model parameters for different tasks, including classification, extraction, content generation, question answering, and summarization.

When you click a sample, the prompt text loads in the editor, an appropriate model is selected, and optimal parameters are configured automatically.

History

As you experiment with different prompt text, model choices, and parameters, the details are captured in the session history each time you submit your prompt. To load a previous prompt, click the entry in the history and then click Restore.

Saved

From the Saved prompt templates menu, you can load any prompts that you saved to the current project as a prompt template asset.

When watsonx.governance is provisioned, if your prompt template includes at least one prompt variable, you can evaluate the effectiveness of model responses. For more information, see Evaluating prompt templates in projects.