Quick start: Prompt a foundation model with the retrieval-augmented generation pattern

You need a project to store the sample notebook.

Watch a video to see how to create a sandbox project and associate a service. Then follow the steps to verify that you have an existing project or create a sandbox project.

Follow the steps to verify that you have an existing project or create a project.

1. From the watsonx home screen, scroll to the Projects section. If you see any projects listed, then skip to Associate the Watson Machine Learning service. If you don't see any projects, then follow these steps to create a project.

2. Click Create a sandbox project. When the project is created, you will see the sandbox in the Projects section.

3. Open an existing project or the new sandbox project.

The sample notebook uses a small knowledge base and a simple search component to demonstrate the basic pattern. The scenario used in this notebook is for a company that sells seeds for planting in a garden. The website for an online seed catalog has many articles to help customers plan their garden and ultimately select which seeds to purchase. The new widge is being added to the website to answer customer questions on the contents of the articles.

Watch this video to see how to add a sample notebook to a project, and then follow the steps to add the notebook to your project.

1. Access the Simple introduction to retrieval-augmented generation with watsonx.ai in the Resource hub.

2. Click Add to project.

3. Select your project from the list, and click Add.

4. Type the notebook name and description (optional).

5. Select a runtime environment for this notebook.

6. Click Create. Wait for the notebook editor to load.

7. From the menu, click Kernel > Restart & Clear Output, then confirm by clicking Restart and Clear All Outputs to clear the output from the last saved run.

For more information on associated services, see Adding associated services.

To preview this task, watch the video beginning at 00:57.

Before you can run the notebook, you need to set up the environment. Follow these steps to verify the notebook prerequisites:

1. Scroll to the For IBM watsonx on IBM Cloud section in the notebook to see the two prerequisites to run the notebook.

2. Under the Create an IBM Cloud API key section, you need to pass your credentials to the Watson Machine Learning API using an API key. If you don't already have a saved API key, then follow these steps to create an API key.
   

   1. Access the IBM Cloud console API keys page.

   2. Click Create an IBM Cloud API key. If you have any existing API keys, the button may be labelled Create.

   3. Type a name and description.

   4. Click Create.

   5. Copy the API key.

   6. Download the API key for future use.

3. Review the Associate an instance of the Watson Machine Learning service with the current project section. You completed this prerequisite in Task 1.

4. Scroll to the Run the cell to provide the IBM Cloud API key section:

   1. Click the Run icon to run the cell.

   2. Paste the API key, and press Enter.

5. Under Run the cell to set the credentials for IBM watsonx on IBM Cloud, click the Run icon to run the cell and set the credentials.

To preview this task, watch the video beginning at 01:03.

The sample notebook includes information about the retrieval-augmented generation and how you can adapt the notebook for your specific use case. Follow these steps to run the notebook and review the output:

1. Scroll to the Step 2: Create a Knowledge Base section in the notebook:

   1. Click the Run icon for each of the three cells in that section.

   2. Review the output for the three cells in the section. The code in these cells sets up the knowledge base as a collection of two articles. These articles were written as samples for watsonx.ai, they are not real articles published anywhere else. The authors and publication dates are fictional.

2. Scroll to the Step 3: Build a simple search component section in the notebook:

   1. Click the Run icon for each of the two cells in that section.

   2. Review the output for the two cells in the section. The code in these cells builds a simple search component. Many articles that discuss retrieval-augmented generation assume the retrieval component uses a vector database. However, to perform the general retrieval-augmented generation pattern, any search-and-retrieve method that can reliably return relevant content from the knowledge base will do. In this notebook, the search component is a trivial search function that returns the index of one or the other of the two articles in the knowledge base, based on a simple regular expression match.

3. Scroll to the Step 4: Craft prompt text section in the notebook:

   1. Click the Run icon for each of the two cells in that section.

   2. Review the output for the two cells in the section. The code in these cells crafts the prompt text. There is no one, best prompt for any given task. However, models that have been instruction-tuned, such as bigscience/mt0-xxl-13b, google/flan-t5-xxl-11b, or google/flan-ul2-20b, can generally perform this task with a sample prompt. Conservative decoding methods tend towards succinct answers. In the prompt, notice two string placeholders (marked with %s) that will be replaced at generation time:

      • The first placeholder will be replaced with the text of the relevant article from the knowledge base
      • The second placeholder will be replaced with the question to be answered

4. Scroll to the Step 5: Generate output using the foundation models Python library section in the notebook:

   1. Click the Run icon for each of the three cells in that section.

   2. Review the output for the three cells in the section. The code in these cells generates output by using the Python library. You can prompt foundation models in watsonx.ai programmatically using the Python library. For more information about the library, see the following topics:

   • Introduction to the foundation models Python library
   • Foundation models Python library reference

5. Scroll to the Step 6: Pull everything together to perform retrieval-augmented generation section in the notebook:

   1. Click the Run icon for each of the two cells in that section. This code pulls everything together to perform retrieval-augmented generation.

   2. Review the output for the first cell in the section. The code in this cell sets up the user input elements.

   3. For the second cell in the section, type a question related to tomatoes or cucumbers to see the answer and the source. For example, Do I use mulch with tomatoes?.

   4. Review the answer to your question.