The context relevance metric measures how relevant the context that your model retrieves is with the question that is specified in the prompt.
Metric details
Context relevance is a retrieval quality metric for generative AI quality evaluations that measures the quality of how retrieval systems rank relevant contexts.
Scope
The context relevance metric evaluates generative AI assets only.
- Types of AI assets: Prompt templates
- Generative AI tasks: Retrieval Augmented Generation (RAG)
- Supported languages: English
Scores and values
The context relevance metric score indicates the relevance of the retrieved contexts to the question in the prompt. Higher scores indicate that the context is more relevant to the question in the prompt. When multiple context variables exist, scores are generated when the metric is calculated with fine-tuned models only.
Range of values: 0.0-1.0
Settings
- Thresholds:
- Lower limit: 0
- Upper limit: 1
Parent topic: Evaluation metrics