Prompt Injection Is Not a Filtering Problem

The security literature and the vendor literature disagree about prompt injection, and the disagreement is not a detail.

Vendor guidance describes layered defence: filter the input, detect at runtime, govern the whole thing. The research literature reports that adaptive attacks bypass essentially every published defence, including ones that score well on the benchmarks used to publish them. Both statements are in print. Only one of them describes what happens when someone is actually trying.

Why filtering keeps losing

The instinct is to treat injection like SQL injection, which was solved by separating code from data. The comparison is the problem: that separation is what a language model does not have.

The system prompt, the retrieved document, the user's message and the tool response arrive as one sequence of text. Labels can be applied and the model will usually respect them, but there is no enforcement, only a tendency. Filtering therefore has to identify hostile intent in natural language, which is an unbounded space. Every published filter has fallen to rewording, encoding, or splitting an instruction across turns.

Filtering removes unskilled attempts, and that has some value. It is not a control a design can rest on.

The benchmark problem

This is the part that most changes how the reported numbers should be read.

Defences are typically evaluated against a fixed suite of known attacks. A fixed suite measures whether those attacks still work. It does not measure whether the defence holds, because a real attacker sees the defence and writes against it.

	Static benchmark	Adaptive attacker
Attack set	Fixed, published in advance	Written against your specific defence
What it measures	Regression against known attacks	Whether the defence holds
Typical reported result	Low attack success rate	Much of the surface recovered
Correct reading	A floor on vulnerability	The number that matters

Recent work makes the mechanism explicit: defences trained to resist injection can be undone by attacks constructed with knowledge of that training. Meta's SecAlign and the architecture-aware attack work are worth reading together, because the second is largely a response to the class the first belongs to.

A defence with a 3% attack success rate on a static suite and no adaptive evaluation has told you almost nothing about deployment.

Indirect injection is the part that gets missed

Most threat models treat the user as untrusted and retrieved content as data. For agent systems this is backwards.

If an agent reads web pages, documents, tickets or emails, anyone who can write to those surfaces can write instructions into its context, without ever touching your application. A line in a document that a human skims past is read by the model with the same weight as your system prompt.

The practical consequence: your trust boundary is not your API. It is the union of every source your retrieval layer can reach. Ask who can write to the knowledge base, and whether that set has ever been reviewed as a security question rather than a permissions convenience. In most systems it has not.

What actually helps

Everything that works shares a shape: it does not try to detect the attack, it limits what a successful one can do.

Constrain the action space. An agent that can call four tools has a smaller failure surface than one that can call forty, and the four can be reasoned about. This costs capability, and that cost is the price of the property.

Scope permissions per task, not per agent. Agents typically run under one identity holding the union of every permission any task might need. Least privilege is satisfied on paper while a request that should touch one record can reach everything.

Put confirmation at side effects. Sending a message, writing to a system of record, moving money, deleting something. These are few and individually judgeable. Confirming internal reasoning steps produces approval fatigue and no safety, the same reason checkpoints belong at externally visible actions rather than at reasoning.

Assume the failure will be quiet. A successful injection produces well-formed, confident output, and nothing in the pipeline is positioned to report it.

What does not work

A second model as a guardrail. It reads the same untrusted content and is injectable by the same means. It catches unsubtle attempts and gives disproportionate confidence.

Instructing the model to ignore instructions in retrieved content. This is a request, not a control. It raises the effort required and does not bound it.

Treating a passed audit as evidence. Controls written for systems with a code-data boundary can be fully satisfied by a system that is trivially manipulable, which is why frameworks pass while the gap stays open.

You may not need any of this

Worth saying plainly, because the topic invites over-engineering.

If your model has no tools, no retrieval and no ability to act, injection gets you a rude answer. The concern scales with the action space, not with the presence of a model. A summariser that reads text and returns text does not need a threat model built for an agent with database write access.

The mitigations above cost capability and latency. Applying them to a system that cannot do anything harmful is expenditure for no return, and it crowds out the work on systems that can.

What we do not know

There is no known way to bound injection risk rather than reduce it. A real solution requires reliably distinguishing instruction from data inside a single text channel, and that problem is open. Anyone offering a solution rather than a mitigation is describing something the field does not have.

We also do not know how long a finding survives. Because behaviour is probabilistic and version-dependent, a defence validated against one model version may not hold on the next, and nobody has characterised that decay rate. Our own position is that adversarial testing should be repeated far more often than is typical, and we cannot tell you what the right interval is.

Sources: Meta SecAlign · Architecture-aware attacks on fine-tuning defences · Review of vulnerabilities and defence mechanisms