0 / 0
Exact match evaluation metric
Last updated: Feb 13, 2025
Exact match evaluation metric

The exact match evaluation metric compares model prediction strings to reference strings to measure how often the strings match.

Metric details

Exact match is a generative AI quality evaluation metric that measures how well generative AI assets perform tasks.

Scope

The exact match metric evaluates generative AI assets only.

  • Types of AI assets: Prompt templates
  • Generative AI tasks:
    • Question answering
    • Entity extraction
    • Retrieval augmented generation (RAG)
  • Supported languages: English

Scores and values

The exact match metric score indicates the proportion of perfect matches between model prediction strings and reference strings. Higher scores indicate that model prediction strings match reference strings more often.

  • Range of values: 0.0-1.0
  • Best possible score: 1.0
  • Ratios:
    • At 0: No match with reference
    • Over 0: At least one exact match

Settings

  • Thresholds:
    • Lower limit: 0.8
    • Upper limit: 1
  • Parameters:
    • Regexes to ignore: Regex expressions of characters to ignore when calculating the exact matches.
    • Ignore case: If True, turns everything to lowercase so that capitalization differences are ignored.
    • Ignore punctuation: If True, removes punctuation before comparing strings.
    • Ignore numbers: If True, removes all digits before comparing strings.

Parent topic: Evaluation metrics