The density evaluation metric measures how extractive the summary in the foundation model output is from the model input by calculating the average of extractive fragments that closely resemble verbatim extractions from the original text.
Metric details
Density is a content analysis metric for generative Ai quality evaluations that evaluates your model output against your model input or context.
Scope
The density metric evaluates generative AI assets only.
- Types of AI assets: Prompt templates
- Generative AI tasks:
- Retrieval Augmented Generation (RAG)
- Text summarization
- Supported languages: English
Scores and values
The density metric score indicates how abstractive the model output is and how closely extractive fragments resemble verbatim extractions from the original text. Lower scores indicate that the model output is more abstractive and on average the extractive fragments do not closely resemble verbatim extractions from the original text.
Range of values: 0.0-1.0
Settings
- Thresholds:
- Lower limit: 0
- Upper limit: 1
Parent topic: Evaluation metrics