About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Prediction drift evaluation metric
Last updated: Mar 05, 2025
The prediction drift metric measures the change in distribution of the LLM predicted classes.
Metric details
Prediction drift is a drift v2 evaluation metric that evaluates data distribution changes.
Scope
The prediction drift metric evaluates generative AI assets only.
Types of AI assets: Prompt templates
- Generative AI tasks:
- Text classification
- Supported languages: English
Scores and values
The prediction drift score indicates the change in distribution of the LLM predicted classes.
- Range of values: 0.0-1.0
- Best possible score: 0.0
- Ratios:
- At 0: No change is detected.
- Over 0: Increasing change is detected.
Do the math
The following Jensen Shannon distance formula is used to calculate prediction drift:
Jensen Shannon Distance is the normalized form of Kullback-Leibler (KL) Divergence that measures how much one probability distribution differs from the second probabillity distribution. Jensen Shannon Distance is a symmetrical score and always has a finite value.
is the KL Divergence.
Parent topic: Evaluation metrics
Was the topic helpful?
0/1000