The unsuccessful requests metric measures the ratio of questions that are answered unsuccessfully out of the total number of questions.
Metric details
Copy link to section
Unsuccessful requests is an answer quality metric for generative AI quality evaluations that can help measure the quality of model answers. Answer quality metrics are calculated with LLM-as-a-judge models. Watsonx.governance does not calculate
the unsuccessful requests metric with fine-tuned models.
Scope
Copy link to section
The unsuccessful requests metric evaluates generative AI assets only.
Types of AI assets: Prompt templates
Generative AI tasks:
Retrieval Augmented Generation (RAG)
Question answering
Supported languages: English
Scores and values
Copy link to section
The unsuccessful requests metric score indicates how successfully models provide answers to questions. Higher scores indicate that the model can not provide answers to the question.