0 / 0
Prediction drift evaluation metric
Last updated: Mar 05, 2025
Prediction drift evaluation metric

The prediction drift metric measures the change in distribution of the LLM predicted classes.

Metric details

Prediction drift is a drift v2 evaluation metric that evaluates data distribution changes.

Scope

The prediction drift metric evaluates generative AI assets only.

Types of AI assets: Prompt templates

  • Generative AI tasks:
    • Text classification
  • Supported languages: English

Scores and values

The prediction drift score indicates the change in distribution of the LLM predicted classes.

  • Range of values: 0.0-1.0
  • Best possible score: 0.0
  • Ratios:
    • At 0: No change is detected.
    • Over 0: Increasing change is detected.

Do the math

The following Jensen Shannon distance formula is used to calculate prediction drift:

Jensen Shannon distance formula is displayed

Jensen Shannon Distance is the normalized form of Kullback-Leibler (KL) Divergence that measures how much one probability distribution differs from the second probabillity distribution. Jensen Shannon Distance is a symmetrical score and always has a finite value.

KL Divergence is displayed is the KL Divergence.

Parent topic: Evaluation metrics