Decision bias risk for AI
Description
When one group is unfairly advantaged over another due to decisions of the model.
Why is decision bias a concern for foundation models?
Bias can harm persons who are affected by the decisions of the model. Business entities might face fines, reputational harms, disruption to operations, and other legal consequences.
Unfairly Advantaged Groups
The 2018 Gender Shades study demonstrated that machine learning algorithms can discriminate based on classes like race and gender. Researchers evaluated commercial gender classification systems that are sold by companies like Microsoft, IBM, and Amazon and showed that darker-skinned females are the most misclassified group (with error rates of up to 35%). In comparison, the error rates for lighter-skinned were no more than 1%.
Parent topic: AI risk atlas
We provide examples covered by the press to help explain many of the foundation models' risks. Many of these events covered by the press are either still evolving or have been resolved, and referencing them can help the reader understand the potential risks and work towards mitigations. Highlighting these examples are for illustrative purposes only.