The Hidden Cost of Agents

Every cost model written about on this blog assumes a shape: a request goes in, an answer comes out, and the bill is a function of the two. Agents break that shape. An agent is not a request; it is a program that issues requests — planning calls, tool calls, reflection calls, retries — and the number of calls is decided at runtime by the model itself. The per-token price did not change. The thing being priced did.

The consequence: agent costs are not predicted by the pricing page. They are predicted by the loop — how many calls one user action causes, how long each call runs, how often the loop fails and restarts. Teams that budgeted agents like chat features discover the difference in the first invoice.

This post is the cost anatomy of that loop: where the tokens go, which parts are waste, and how to budget a workload that decides its own length.

Four destinations, in rough order of surprise. The conversation itself: every agent call re-sends the accumulated context — system prompt, tool definitions, history so far — and the context grows with every step. A dozen-step loop is not a dozen small requests; it is a dozen requests whose average size is the whole conversation so far.

Tool definitions ride along too. Every call carries the full schema of every tool the agent might use, whether or not this step uses any. Large tool sets are a standing tax on every single call.

Then the failed attempts: the malformed tool call, the parse error, the retry. Each failure is a full request that produced nothing the user will ever see.

And finally the reflection: the model checking its own work, summarising, planning the next step — tokens spent on process, not on answer. None of these are line items on the pricing page. All of them are on the invoice.

Because the loop carries no natural length. A chat answer ends when the model stops talking; an agent ends when it decides the task is done — and that decision is a judgment call made by the same model being paid by the step. Under ambiguity, the loop runs long. Under failure, it runs again.

The multiplication shows in the arithmetic: one user action causing a dozen model calls, each carrying a growing context, costs not one answer but a small campaign. The rate-limit post made the same observation from the capacity side — chatty architectures multiply requests — and the cost side is the same observation in dollars.

Retries are the quiet multiplier. A tool call failing validation is retried with the error message appended, which makes the retry longer than the original. A loop retrying generously converts one failure into several paid attempts, and the user experience does not improve in proportion.

Tool calling is where agents earn their keep — and where the bill hides. Every call carries the tool schemas; every tool result is appended to the context; every step pays for the privilege of the next. A tool-heavy agent can spend more tokens describing what it is about to do than doing it.

Two disciplines blunt the tax. Keep the tool set minimal — an agent with twenty tools pays for twenty schemas on every call, and most steps use one. Keep tool results small: return the field the agent needs, not the whole record. A search tool returning the top result with a snippet is a different cost profile from one returning the top fifty with full text, and the agent rarely needs the difference.

The deeper fix is architectural: not every step needs the frontier model. The planning step, the reflection step, and the final answer may deserve different strengths — the same right-sizing argument from the hard-coding post, applied inside a single loop.

Budget the loop, not the call. Measure steps per completed task, tokens per step, and cost per completed task — the framework from the cost-per-task post, with the task redefined as the whole agent run. That number is the only one that survives contact with the invoice.

Then set the guardrails from the budgets post at the loop level: a step cap, a token cap per step, and a spend alert on the agent feature specifically. An agent that cannot exceed a step budget is a product decision; an agent that can is a liability with a nice demo.

And log the failure taxonomy. Steps that fail, retry, and succeed are different from steps that fail and are abandoned, and both differ from steps that succeed slowly. The invoice cannot tell these apart; the logs must.

Honestly: when the task could be done in one call, and the multi-step version is not worth the multiple — a narrower set than the demos suggest. If a single well-prompted call with retrieval answers the question, an agent is a more expensive way to get the same answer; the fine-tune-prompt-or-RAG post makes the same point about levers.

Agents earn their cost when the work is genuinely sequential: the answer depends on the result of the previous step, and no amount of prompt engineering collapses the sequence. Research requiring reading then reasoning, operations requiring calling then reacting — these are loops by nature.

And the honest test is the eval: the same task set through the agent and through the best single-call alternative, compared on cost per completed task. An agent that does not win on quality by enough to justify its multiple is a feature, not a strategy.

Teams running agents profitably share three habits. They cap the loop — steps, tokens, spend — before launch, not after the first invoice. They right-size the models inside the loop, because a reflection step does not need the strength of the final answer. And they treat the failure taxonomy as a product metric, reviewed with the same seriousness as latency.

None of this makes agents cheap. It makes them predictable — and predictability is what turns an agent from a cost surprise into a cost line. The surprise is the expensive part.