Chat mode

You can chat with the foundation model to see how the model handles dialog or question-answering tasks.

Start the chat by submitting a query or request for the foundation model to answer. Alternatively, you can click a quick start sample to submit to the model. Quick start samples are sent to the Llama foundation model. If you want to work with a different foundation model, add your own prompt text.

Each subsequent turn in the conversation builds on information that was exchanged previously.

Note: You cannot make changes while a chat is in progress. Click the Clear chat icon to stop and make changes.

Before you start a chat, review and adjust the model choice and parameter settings. To support long dialog exchanges, the Max tokens parameter is set to a high default value. You might want to add a stop sequence to prevent the model from generating wordy outputs, for example.

Structured mode

Structured mode is designed to help new users create effective prompts. Text from the fields is sent to the model in a template format.

You add parts of your prompt into the appropriate fields:

• Instruction: Add an instruction if it makes sense for your use case. An instruction is an imperative statement, such as Summarize the following article.

• Examples: Add one or more pairs of examples that contain the input and the corresponding output that you want. Providing a few example input-and-output pairs in your prompt is called few-shot prompting.

  If you need a specific prefix to the input or the output, you can replace the default labels, "Input:" or "Output:", with the labels you want to use. For example, you might replace the default labels with custom labels that were used in training data when a foundation model was prompt-tuned.

  A space is added between the example label and the example text.

• Test your input: In the Try area, enter the final input of your prompt.

Freeform mode

You add your prompt in plain text. Your prompt text is sent to the model exactly as you typed it.

Freeform mode is a good choice when you want to submit structured input and know how to format the prompt.

Model choices

In the Prompt Lab, you can submit your prompt to any of the models that are supported by watsonx.ai. You can choose recently-used models from the drop-down list. Or you can click View all foundation models to view all the supported models, filter them by task, and read high-level information about the models.

If you tuned a foundation model by using the Tuning Studio and deployed the tuned model or you deployed a custom foundation model, the tuned or custom model is also available for prompting from the Prompt Lab.

Model parameters

To control how the model generates output in response to your prompt, you can specify decoding parameters and stopping criteria. For more information, see Model parameters for prompting.

Prompt variables

To add flexibility to your prompts, you can define prompt variables. A prompt variable is a placeholder keyword that you include in the static text of your prompt at creation time and replace with text dynamically at run time. For more information, see Building reusable prompts.

View full prompt text

You might want to see the full prompt text that will be submitted to the foundation model in the following situations:

• When prompt variables are in use, to see resolved variable values in context.
• In chat mode, where the recommended prompt formats for different foundation models are applied automatically.
• In structured mode, where you add parts of the prompt into separate fields.

AI guardrails

When you set the AI guardrails switcher to On, harmful language is automatically removed from the input prompt text and from the output that is generated by the model. Specifically, any sentence in the input or output that contains harmful language is replaced with a message that says that potentially harmful text was removed.

For more information, see Removing harmful content.

View code

When you click the View code icon , a cURL command is displayed that you can call from outside the Prompt Lab to submit the current prompt and parameters to the selected model and get a generated response.

The command includes a placeholder for an IBM Cloud IAM token. For information about generating the access token, see Generating an IBM Cloud IAM token.

Programmatic alternative to the Prompt Lab

The Prompt Lab graphical interface is a great place to experiment and iterate with your prompts. However, you can also prompt foundation models in watsonx.ai programmatically by using the Python library or REST API. For details, see Coding generative AI solutions.

Samples

A collection of sample prompts are available in the Prompt Lab. The samples demonstrate effective prompt text and model parameters for different tasks, including classification, extraction, content generation, question answering, and summarization.

When you click a sample, the prompt text loads in the editor, an appropriate model is selected, and optimal parameters are configured automatically.

History

As you experiment with different prompt text, model choices, and parameters, the details are captured in the session history each time you submit your prompt. To load a previous prompt, click the entry in the history and then click Restore.

Saved

From the Saved prompt templates menu, you can load any prompts that you saved to the current project as a prompt template asset.

When watsonx.governance is provisioned, if your prompt template includes at least one prompt variable, you can evaluate the effectiveness of model responses. For more information, see Evaluating prompt templates in projects.