Governance
Prompt injection
Definition
Prompt injection is an attack in which instructions hidden inside content an AI system reads — a document, an email, a web page — are followed by the model as if they came from its operator.
For an agent, everything it ingests is potential instruction. A sentence buried in a submitted PDF can tell a claims agent to approve, exfiltrate, or ignore a control, and the model has no native way to distinguish that text from a legitimate directive. Injection is not a bug in one model; it is a property of systems that both read untrusted content and hold real authority. As agents are given more tools and more reach, the attack surface grows with them, because every new source of text the model reads is a new place an instruction can hide.
Why filters are not a fix
Input scanning catches known phrasings and misses the rest, because the attack surface is natural language itself — there is no finite list of dangerous sentences to block. An attacker only needs one phrasing the filter has not seen, and the phrasing can be trivially rewritten to evade a static rule. The durable mitigation is architectural rather than linguistic: limit what the agent can do with what it reads, and require a second, non-model check before consequential actions, so that even a successful injection cannot translate into a real-world effect without passing a control the model cannot talk its way past.
The session is the real prize
An agent operating inside an authenticated browser session carries the user's permissions — their email access, their internal tools, their approved workflows. Injected instructions inherit those permissions, which is what turns a content problem into an access problem with real financial or regulatory consequence. This is why browser-operating agents deserve more scrutiny than a simple chat assistant: the blast radius of a successful injection scales with whatever the underlying session is authorized to touch, and that authorization is usually far broader than the task the agent was assigned.
Designing for a hostile document
The practical response is to assume every document, email, and web page an agent reads may contain an attack, and to design accordingly rather than reactively. That means separating the channel that carries instructions from the channel that carries content, so the model is structurally less likely to treat retrieved text as a command. It also means testing agents against adversarial documents before launch, the same way a security team would red-team an external-facing application, because the first attacker to try this against your system in production should not be the first time anyone tested it.
In practice
- Treat all retrieved and uploaded content as untrusted input, never as instruction
- Gate irreversible actions behind deterministic checks the model cannot talk past
- Scope agent credentials to the minimum systems the task requires
- Red-team agents against adversarial documents before launch, not after an incident
