The prediction drift metric measures the change in distribution of the LLM predicted classes.
Metric details
Copy link to section
Prediction drift is a drift v2 evaluation metric that evaluates data distribution changes.
Scope
Copy link to section
The prediction drift metric evaluates generative AI assets only.
Types of AI assets: Prompt templates
Generative AI tasks:
Text classification
Supported languages: English
Scores and values
Copy link to section
The prediction drift score indicates the change in distribution of the LLM predicted classes.
Range of values: 0.0-1.0
Best possible score: 0.0
Ratios:
At 0: No change is detected.
Over 0: Increasing change is detected.
Do the math
Copy link to section
The following Jensen Shannon distance formula is used to calculate prediction drift:
Jensen Shannon Distance is the normalized form of Kullback-Leibler (KL) Divergence that measures how much one probability distribution differs from the second probabillity distribution. Jensen Shannon Distance is a symmetrical score and always
has a finite value.
About cookies on this siteOur websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising.For more information, please review your cookie preferences options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.