Last updated: Feb 07, 2025
Description
Testing is unrepresentative when the test inputs are mismatched with the inputs that are expected during deployment.
Why is unrepresentative risk testing a concern for foundation models?
If the model is evaluated in a use, context, or setting that is not the same as the one expected for deployment, the evaluations might not accurately reflect the risks of the model.
Parent topic: AI risk atlas
We provide examples covered by the press to help explain many of the foundation models' risks. Many of these events covered by the press are either still evolving or have been resolved, and referencing them can help the reader understand the potential risks and work towards mitigations. Highlighting these examples are for illustrative purposes only.