Prompt priming risk for AI
Description
Because generative models tend to produce output like the input provided, the model can be prompted to reveal specific kinds of information. For example, adding personal information in the prompt increases its likelihood of generating similar kinds of personal information in its output. If personal data was included as part of the model’s training, there is a possibility it could be revealed.
Why is prompt priming a concern for foundation models?
Depending on the content revealed, business entities could face fines, reputational harm, and other legal consequences.
Parent topic: AI risk atlas