Last updated: Feb 21, 2025
After your AutoAI RAG experiment completes, view the details of a pattern to understand the composition and performance.
From the pipeline leaderboard, you can view a table with the following information for each of the patterns the experiment generates:
| Column | Description |
|---|---|
| Rank | Rank of placement according to best performance for the optimized metric |
| Name | Pattern name |
| Model name | Name of the embedding model used to vectorize and index the documents |
| Optimize metric | Performance result for the optimized metric selected for the experiment. A RAG pattern can be optimized for Answer faithfulness and Answer correctness |
| Chunk size | Amount of data retrieved from the indexed documents |
| Retrieval method | Method (Window or Simple) used to retrieve indexed documents |
| Vector store distance metric | Metric (Cosine or Euclidean) used to measure how relevant the stored vectors are to the input vectors |
Click on a pattern name to review the following configuration details for a pattern. Some of the configuration settings are editable when you create the experiment. For details, see Customizing RAG experiment settings.
| Column | Description |
|---|---|
| Pattern | Pattern name |
| Score type | How the performance metric is calculated and presented. - Mean: The average score of the metric. - CI High: The upper bound of the confidence interval (CI) for the mean score, showing the highest value that the metric is likely to achieve. - CI Low: The lower bound of the confidence interval (CI) for the mean score, showing the lowest value that the metric is likely to achieve. |
| Answer correctness | Correctness of the generated response including both the relevance of the retrieved context and the quality of the generated response. |
| Answer faithfulness | Accuracy of the generated response to the retrieved text. |
| Context correctness | Relevancy of the generated response to the input. |
| Vector store datasource type | Datasource that connects to the vector store. |
| Vector store distance metric | Metric used to measure how relevant the stored vectors are to the input vector. |
| Vector store index name | Index used as a data structure for storing and retrieving vector stores. |
| Vector store operation | Vector store operation for retrieving data. For example, upsert for inserting or updating rows in the database table. |
| Vector store schema fields | Schema for storing the index of vectored documents. |
| Vector store schema ID | Unique identifier of the vector store schema. |
| Vector store schema name | Name of the schema used for structuring the database table |
| Vector store schema type | Determines the kind of data that can be stored in the schema and the operations that can be carried out on the data. The schema type struct is used for storing structured data. |
| Chunk overlap | Number of chunks that share common words or phrases. |
| Chunk size | Amount of data retrieved from the index documents. |
| Chunk augmentation | How relevant text is retrieved for each chunk of the input after splitting the input into multiple chunks. |
| Embedding model | Embedding model used to vectorize and index the document collection. |
| Truncate input tokens | Maximum number of tokens accepted as input. |
| Truncate strategy | Strategy used to determine how to process retrieved documents to optimize the model performance. |
| Retrieval method | Method used to retrieve indexed documents: Window or Simple. |
| Number of chunks | Number of chunks retrieved from the indexed documents. |
| Context template text | Structured template for the generated response. |
| Foundation model | Foundation model used in the experiment. Click the preview icon for details about the model. |
| Decoding method | Process that a model uses to choose the tokens in the generated output. |
| Maximum new tokens | Maximum number of new tokens that can be generated in addition to the tokens retrieved from indexed documents. |
| Minimum new tokens | Minimum number of new tokens that can be generated. |
| Prompt template text | Structure with guidelines for the model to use when retrieving text from indexed documents. |
| Word to token ratio | Number of words in a text divided by the number of generated tokens. |
| Sample Q&A | Questions and correct answers provided as test data to measure the performance of the pattern. |
Parent topic: Automating a RAG pattern with AutoAI