An AI assistant is often valuable because it can read material on your behalf. That same ability creates a difficult security question: when does the material it reads start influencing how it behaves?
The boundary between information and authority
Consider an assistant reviewing a retrieved document. The document might contain legitimate business content. It could also include text that asks the assistant to ignore its task, reveal information, or change its answer. The document is an information source; it should not acquire the authority of the application or the user simply because it appears in context.
The problem extends beyond a single prompt. Retrieval, browsing, messages, and tool responses can all bring lower-trust material into a workflow. Security review should trace those entry points and examine what privileges sit on the other side.
Review the entire application
Model instructions are one layer. The surrounding application determines available data, tool access, output handling, and approval requirements. If an assistant cannot access a sensitive resource, an instruction in a document cannot grant it that access. This makes permission design a central part of the defense.
- Identify which sources are controlled by your organization and which are external.
- Keep authoritative instructions distinct from retrieved content.
- Limit data and tool permissions to the task.
- Validate consequential actions outside the model’s generated response.
- Test adversarial scenarios across retrieval and tool boundaries.
A question for the next architecture review
What would happen if an attacker controlled one document, one search result, or one tool response in this workflow? Follow the answer through to the data, decisions, and actions that could be affected.