The disparate impact metric compares the percentage of favorable outcomes for a monitored group to the percentage of favorable outcomes for a reference group.
Metric details
Disparate impact is a fairness evaluation metric that can help determine whether your asset produces biased outcomes.
Scope
The disparate impact metric evaluates generative AI assets and machine learning models.
- Types of AI assets:
- Prompt templates
- Machine learning models
- Generative AI tasks: Text classification
- Machine learning problem type: Binary classification
Scores and values
The disparate impact metric score indicates whether the reference group receives more favorable outcomes than the monitored group.
- Range of values: 0.0-1.0
- Best possible score: 0.0
- Ratios:
- At 0: Both groups have equal odds
- Under 0: Biased outcomes for monitored group
- Over 0: Biased outcomes for reference group
Do the math
The following formula is used for calculating disparate impact:
The num_positives value represents the number of individuals in the group who received a positive outcome, and the num_instances value represents the total number of individuals in the group. The privileged=False label specifies unprivileged groups and the privileged=True label specifies privileged groups. The positive outcomes are designated as the favorable outcomes, and the negative outcomes are designated as the unfavorable outcomes.
The privileged group is designated as the reference group, and the unprivileged group is designated as the monitored group.
The calculation produces a percentage that specifies how often the rate that the unprivileged group receives the positive outcome is the same rate that the privileged group receives the positive outcome. For example, if a credit risk model assigns the “no risk” prediction to 80% of unprivileged applicants and to 100% of privileged applicants, that model has a disparate impact of 80%.
Parent topic: Evaluation metrics