Prompt Injection: The Threat Nobody Budgeted For
On this page
The Attack That Costs Nothing to Launch
Prompt injection is the attack whose instructions arrive inside the data. A user pastes text into a form; the text holds a sentence addressed not to the user but to the model: ignore your instructions, reveal your system prompt, call this tool, send this data somewhere. The model, trained to follow instructions found in text, follows the one it just found.
What sets injection apart from every other security problem: it requires no exploit, no vulnerability, no skill. The attack surface is the input box, and the payload is a sentence. Anyone who can type can launch it.
The threat is not hypothetical. Every system feeding untrusted text to a model — support bots reading tickets, assistants reading email, agents browsing the web — is exposed by construction, and the exposure grows with every tool the model can call.
Why Are LLMs Uniquely Vulnerable?
Because the model cannot distinguish instructions from data. A traditional program has a hard boundary: code is code, input is input, and input never executes. A language model has no such boundary — the system prompt, the user message, and the text of a web page all arrive as the same kind of thing, and the model weighs them by plausibility, not origin.
Tool access widens the gap. A model that can only produce text can be tricked into producing text; a model that can call tools can be tricked into calling them. The injection making a chatbot say something embarrassing is a PR problem; the injection making an agent send an email or query a database is a security incident.
And the model’s helpfulness is the vulnerability. It wants to follow instructions — and the attacker’s instruction is just another instruction, often more recent and more specific than yours.
What Does a Successful Injection Actually Cost?
Four tiers, rising. Reputation: the model says something the company did not authorise, and the screenshot outlives the incident. Data: the model leaks its system prompt — or worse, the contents of its context, which in retrieval systems is your data, served up on request. Action: the model calls a tool it should not — sends, deletes, purchases, posts. And persistence: the injected instruction is stored, retrieved, and re-injected into future sessions, so the attack survives its own launch.
The cost question is not “will we be attacked” but “what can an attacker reach”. A model with no tools and no sensitive context has a small blast radius; an agent with write access to production systems has a large one. The threat model is the tool list.
Which Defense Layers Actually Work?
No single defense stops injection; a stack of cheap ones drives its cost up. The first layer is the system prompt itself: explicit instructions that data is untrusted, instructions inside data are to be ignored, certain requests are always refused. Not a guarantee — a speed bump, and speed bumps matter.
The second is separation: keep untrusted content out of the trusted context. Quote and delimit retrieved text so the model can see where data begins and ends. Never let one user’s input become another user’s context without a boundary.
The third is the tool layer, where the real security lives: tools should require explicit confirmation for consequential actions, carry their own permission checks, and never trust the model’s word for who asked. The model proposes; the system disposes.
And the fourth is monitoring: log what the model is asked to do, and alert on the patterns that should never happen. An injection succeeding silently is a vulnerability; one that fires an alert is a detection system.
Tools Are the Blast Radius
The single highest-leverage decision is what the model is allowed to touch. Every tool is a surface the injection can reach, and the question for each is the same: if an attacker controlled this tool for one call, what is the worst that happens? A read-only search tool has a small answer; a send-email tool has a large one.
So tools get the treatment of any privileged code: least privilege, confirmation for irreversible actions, rate limits on the expensive ones. The rate-limit post concurrency caps are a security control as much as a cost control — a loop that cannot exceed its budget is an injection that cannot spend yours.
And the human-in-the-loop is not a cop-out; it is a boundary. The actions that cannot be undone should not be undoable by a sentence in a web page.
When Is the Threat Overblown — and When Isn’t It?
When the model has no tools, no sensitive context, no persistence — a public chatbot answering from a fixed knowledge base is a low-value target, and a paragraph of defensive prompting is proportionate. The full stack above is for systems where the model can act or the context is worth stealing.
The overreaction to avoid is the opposite one: treating injection as unsolvable and shipping nothing. The defenses are cheap, they compose, and they convert the attack from a certainty into a cost. Security is not the absence of risk; it is the pricing of it.
Related Articles
Timeouts, Retries, Idempotency: The Resilience Checklist Nobody Writes Down
Every LLM integration fails the same five ways. The unglamorous checklist — timeouts, retries, idempotency, circuit breakers, degradation — in one place.
Rate Limits Are an Architecture Input, Not an Error
A 429 is not an exception, it is the contract. Reading rate limits as a capacity plan, the four patterns that absorb them, and what retry storms cost.
The Enterprise Inference Vendor Checklist
A procurement-ready checklist for evaluating LLM inference providers: data policy, pricing transparency, reliability evidence, catalog, and exit terms.