Explore the foundation model taxonomy

You can learn more about the capabilities of the granite-7b-lab foundation model by exploring the foundation model's taxonomy.

To access the taxonomy, complete the following steps:

1. From the watsonx.ai Prompt Lab in chat mode, click the the Model field, and then choose View all foundation models.

2. Find and click the tile for the granite-7b-lab foundation model.

3. Click to open the Training taxonomy tab of the model card.

   

4. Explore the knowledge and skills in the taxonomy. Each end node of the taxonomy tree represents a skill that the model was trained to do. You can click a skill to see the seed examples that were used to generate the synthetic data that was used to train the model.

Prompting the granite-7b-lab foundation model

To get the best results from the granite-7b-lab foundation model, start by using recommended prompt structures when you craft foundation model input for different tasks. As you work with the model, you can tweak the model parameters and prompt format to find the best settings for your use case.

The granite-7b-lab foundation model is optimized for the following uses:

• Classification
• Chat
• Returning factual answers with the RAG pattern

<|system|>
You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
<|user|>
For each feedback, specify whether the content is Positive or Negative. Your response should only include the answer. Do not provide any further explanation.
Here are some examples, complete the last one:
Feedback:
Carol, the service rep was so helpful. She answered all of my questions and explained things beautifully.
Class:
Positive
   
Feedback:
The service representative did not listen to a word I said. It was a waste of my time.
Class:
Negative
   
Feedback:
Carlo was so helpful and pleasant. He was able to solve a problem that I've been having with my software for weeks now.
Class:
<|assistant|>

Conversing with granite-7b-lab

To get the best results when chatting with the granite-7b-lab foundation model, first follow these recommendations and then experiment to get the results that you want.

The following table lists the recommended model parameters for prompting the granite-7b-lab foundation model for a conversational task.

For more information about the model parameters, see Model parameters for prompting.

To prompt the granite-7b-lab foundation model for a chat task, try these steps:

1. From the Prompt Lab in chat mode, choose the granite-7b-lab foundation model.

   Chat mode has default prompt parameter values that are optimized for conversational exchanges, including a higher Max tokens value.

2. From the Model parameters panel, apply the recommended model parameter values from Table 2.

3. In chat mode, you can submit user input without formatting the input text.

   Chat mode applies system prompt text for you. You can click the text icon to see how your text is formatted.

4. To submit the same prompt in freeform mode, you must set up the system prompt.

   Add instructions. For example, the following instruction text was used to train the model, and therefore is familiar to the model.

   You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.

   To copy the recommended system prompt text, click the Copy to clipboard icon from the following code snippet.

   You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
   

   Start a conversation.

   The optimal structure for a prompt that is used for a chat task follows. The prompt includes syntax that identifies the following segments of the prompt:

   • <|system|>: Identifies the instruction, which is also known as the system prompt for the foundation model.
   • <|user|>: The query text to be answered.
   • <|assistant|>: A cue at the end of the prompt that indicates that a generated answer is expected.

   When you submit prompts from the Prompt Lab in freeform mode, use the expected prompt format.

   <|system|>
   You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
   <|user|>
   {PROMPT}
   <|assistant|>
   
   

   For example:

   <|system|>
   You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
   <|user|>
   What is the capital of Massachusetts?
   <|assistant|>
   
   

5. Ask follow-up questions to keep the conversation going.

   The optimal structure for a prompt that is used for a chat with multiple dialog turns follows. If you submit prompts from the Prompt Lab in freeform mode, use this prompt format.

   <|system|>
   You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
   <|user|>
   {ROUND1_PROMPT}
   <|assistant|>
   {MODEL_RESPONSE}
   <|user|>
   {ROUND2_PROMPT}
   <|assistant|>
   
   

   For example:

   <|system|>
   You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
   <|user|>
   What is the capital of Massachusetts?
   <|assistant|>
   The capital of Massachusetts is Boston. Boston is not only the largest city in Massachusetts but also one of the oldest cities in the United States, with a rich history and vibrant culture. It is home to numerous universities, hospitals, and technology companies, making it a significant center for education, biotechnology, finance, and tourism. If you have any specific questions about Boston or Massachusetts, feel free to ask!
   <|user|>
   What US cities are older?
   <|assistant|>
   There are several cities in the United States that are older than Boston and even older than the country itself! Some of these historical gems include:
   
   1. St. Augustine, Florida: Established in 1565, St. Augustine is the oldest continuously inhabited European-established settlement within the borders of the contiguous United States.
   2. Jamestown, Virginia: Founded in 1607, Jamestown was the first permanent English settlement in North America.
   3. Plymouth, Massachusetts: Established in 1620, Plymouth ...
   

Returning factual answers with the RAG pattern

To guide the granite-7b-lab foundation model to return factual answers, use the retrieval-augmented generation pattern. Retrieval-augmented generation grounds the input that you submit to the model with factual information about the topic to be discussed. For more information, see Retrieval-augmented generation (RAG).

When you want to return factual answers from the granite-7b-lab foundation model, follow these recommendations.

The following table lists the recommended model parameters for prompting the granite-7b-lab foundation model for a retrieval-augmented generation task.

For more information about the model parameters, see Model parameters for prompting.

To prompt the granite-7b-lab foundation model for a retrieval-augmented generation task, try these steps:

1. Find reliable resources with factual information about the topic that you want the model to discuss and that you have permission to use. Copy an excerpt of the document or documents to a text editor or other tool where you can access it later.

   For example, the resource might be product information from your own company website or product documentation.

2. From the Prompt Lab, open freeform mode so that you can structure your prompts. Choose the granite-7b-lab foundation model.

3. From the Model parameters panel, set the recommended model parameters from Table 3.

4. In your prompt, clearly define the system prompt, user input, and where the model's output should go.

   For example, the following prompt structure can help the granite-7b-lab foundation model to return relevant information.

   <|system|>
   You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
   <|user|>
   You are an AI language model designed to function as a specialized Retrieval Augmented Generation (RAG) assistant. When generating responses, prioritize correctness, i.e., ensure that your response is grounded in context and user query. Always make sure that your response is relevant to the question.
   Answer Length: {ANSWER_LENGTH}
   [Document]
   {DOCUMENT1_TITLE}
   {DOCUMENT1_CONTENT}
   [End]
   [Document]
   {DOCUMENT2_TITLE}
   {DOCUMENT2_CONTENT}
   [End]
   [Document]
   {DOCUMENT3_TITLE}
   {DOCUMENT3_CONTENT}
   [End]
   {QUERY}
   <|assistant|>
   
   

   Note: The start and end of the document content is denoted by the special tags [Document] and [End]. Use a similar syntax if you want to add special tags that identify content types or subsection headers in your prompts. When the granite-7b-lab foundation model was created, it was trained to handle the following special tags:

   • <|system|>: Identifies the instruction, which is also known as the system prompt for the foundation model.
   • <|user|>: The query text to be answered.
   • <|assistant|>: A cue at the end of the prompt that indicates that a generated answer is expected.

   Do not use the same <|tagname|> syntax for your custom tags or you might confuse the model.

5. If you do copy this prompt template, after you paste it into the Prompt Lab editor, replace the placeholder variables.

   Table 4: RAG template placeholder variables

   Placeholder variable	Description	Examples
   {ANSWER_LENGTH}	Optional. Defines the expected response length for the answer.	Options include (from shortest to longest answers): single word, concise, narrative
   {DOCUMENTn_TITLE}	Title of the document from which the excerpt with factual information is taken. You can include content from more than one document.	Product Brochure
   {DOCUMENTn_CONTENT}	Text excerpt with the factual information that you want the model to be able to discuss knowledgeably.	Text from a marketing brochure, product documentation, company website, or other trusted resource.
   {QUERY}	Question to be answered factually.	A question about the topic that is discussed in the document.

    

   Tip: Alternatively, you can define and use a prompt variable for the document so that the prompt can be reused and the content can be replaced dynamically each time. For more information, see Building reusable prompts.

<|system|>
You are an AI language model developed by IBM Research. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.
<|user|>
You are an AI language model designed to function as a specialized Retrieval Augmented Generation (RAG) assistant. When generating responses, prioritize correctness, i.e., ensure that your response is grounded in context and user query. Always make sure that your response is relevant to the question.
Answer length: concise
[Document]
How prompt-tuning works
Foundation models are sensitive to the input that you give them. Your input, or how you prompt the model, can introduce context that the model will use to tailor its generated output. Prompt engineering to find the right prompt often works well. However, it can be time-consuming, error-prone, and its effectiveness can be restricted by the context window length that is allowed by the underlying model.
Prompt-tuning a model in the Tuning Studio applies machine learning to the task of prompt engineering. Instead of adding words to the input itself, prompt-tuning is a method for finding a sequence of values that, when added as a prefix to the input text, improve the model's ability to generate the output you want. This sequence of values is called a prompt vector.
Normally, words in the prompt are vectorized by the model. Vectorization is the process of converting text to tokens, and then to numbers defined by the model's tokenizer to identify the tokens. Lastly, the token IDs are encoded, meaning they are converted into a vector representation, which is the input format that is expected by the embedding layer of the model. Prompt-tuning bypasses the model's text-vectorization process and instead crafts a prompt vector directly. This changeable prompt vector is concatenated to the vectorized input text and the two are passed as one input to the embedding layer of the model. Values from this crafted prompt vector affect the word embedding weights that are set by the model and influence the words that the model chooses to add to the output.
To find the best values for the prompt vector, you run a tuning experiment. You demonstrate the type of output that you want for a corresponding input by providing the model with input and output example pairs in training data. With each training run of the experiment, the generated output is compared to the training data output. Based on what it learns from differences between the two, the experiment adjusts the values in the prompt vector. After many runs through the training data, the model finds the prompt vector that works best.
[End]
How does prompt tuning work?
<|assistant|>

Learn more

To learn more about the granite-7b-lab foundation model, read the following resources:

• Model card

Parent topic: IBM foundation models