0 / 0
Downstream retraining risk for AI

Downstream retraining risk for AI

Risks associated with input
Training and tuning phase
Value alignment
New to generative AI


Using undesirable (for example, inaccurate, inappropriate, user’s content) output from downstream applications for retraining purposes.

Why is downstream retraining a concern for foundation models?

Repurposing downstream output for retraining a model without implementing proper human vetting increases the chances of undesirable outputs being incorporated into the training or tuning data of the model. This, in turn, can generate even more undesirable output. Improper model behavior can result in business entities facing legal consequences or reputational harms. Failing to comply with data transfer laws might result in fines and other legal consequences.

Background image for risks associated with input

Model collapse due to training using AI-generated content

As stated in the source article, a group of researchers from the UK and Canada investigated the problem of using AI-generated content for training instead of human-generated content. They found that the large language models behind the technology might potentially be trained on other AI-generated content. As generated data continues to spread in droves across the internet it can result ina phenomenon they coined as "model collapse."

Parent topic: AI risk atlas

We provide examples covered by the press to help explain many of the foundation models' risks. Many of these events covered by the press are either still evolving or have been resolved, and referencing them can help the reader understand the potential risks and work towards mitigations. Highlighting these examples are for illustrative purposes only.

Generative AI search and answer
These answers are generated by a large language model in watsonx.ai based on content from the product documentation. Learn more