Why Hard-Coding One Model Is Now Your Biggest AI Cost Line

Somewhere in the codebase there is a string that looks like model="...", and it is setting the price of everything the product does. Not because the model is wrong — it is probably a good model — but because the choice was made once, early, and never revisited while the workload grew up around it.

This is the routing problem, and it has quietly become the largest controllable line in most AI budgets. Not the model prices themselves. Not the infrastructure. The fact that a request needing a cheap classification and a request needing multi-step reasoning pay the same rate, because one string says so.

This post is about the category, not about the product. One answer to it is built — Vidman AI Adaptive — and it will be shown alongside the others, because a post arguing only for its own product is a pitch, and those are available anywhere.

Three things, and only one is visible on the invoice.

The visible one: every request bills at that model’s rate. A frontier model charges frontier prices for work that never needed it; a budget model saves money on easy work and then quietly fails on hard work.

The second: the model’s capability ceiling becomes the product’s ceiling. Features the pinned model cannot do well — long documents, tool use, vision — get designed away in planning meetings rather than evaluated, because the model id is treated as fixed infrastructure.

The third is the one nobody notices: the pinned model freezes in time. The catalog around it moves constantly — new models arrive, prices fall — and the gap between what was chosen and what would be chosen today widens every month nobody looks.

Because defaulting to the strongest model is the safe choice — and safe choices are how budgets die. No engineer got fired for routing a support-ticket classifier through a frontier model, and the per-request cost looks trivial in isolation. It stops looking trivial multiplied by every request, every day, forever.

Real products are mixed workloads. Trivial prompts — intent classification, entity extraction, yes-or-no guardrails — sit beside genuinely hard problems justifying strong models. When everything rides one model, either the easy work is over-served or the hard work is under-served. Usually the former, because the visible failure mode of under-serving is angry users, while the failure mode of over-serving is a finance conversation months later.

The uncomfortable part: teams usually know this. What they lack is not awareness but a mechanism. Something has to decide, per request, which tier of model the request deserves. That something is the routing layer — and building it is where most teams stall.

Strip away the vocabulary and routing is a classifier in front of the catalog: look at the request, predict what it needs, send it there. Implementations differ enormously in sophistication.

Rule-based routing is the simplest: keyword checks, length thresholds, explicit task types passed by your own code. It works, it is transparent, and it encodes whatever is already known about the traffic. Its ceiling: rules are only as good as the taxonomy, and traffic drifts out from under taxonomies.

Cascade routing sends everything to a cheap model first and escalates on low confidence or failed validation. Cheap to build, surprisingly effective — but it pays a latency tax on every escalated request (the mechanics are in the latency post) and can thrash when the cheap model is confidently wrong.

Learned routing uses a model or a trained classifier to predict difficulty and pick the target before any token is generated. This is the approach that scales to a large catalog, and it is the approach the adaptive endpoint takes. It is also the hardest to build well yourself, because the predictor must keep pace with every model in the catalog as they all change.

Over-serving costs directly and measurably: frontier rates paid for commodity work. It never shows up as a line item called waste. It shows up as a bill that feels structural, because it is baked into architecture.

In the under-serving direction, the cost hides. A budget model mangling a hard request does not just deliver a bad answer — it triggers retries, human escalations, and downstream corrections, each costing more than the saving. Teams compensate by bolting on validation layers and retry loops, ending up paying frontier-grade complexity on top of budget-model pricing.

The worst outcome is the one that looks like prudence: pinning a mid-tier model for everything. It over-serves the easy half and under-serves the hard half simultaneously — how a reasonable-looking model choice ends up expensive and mediocre at the same time.

Build it. Traffic splitting cleanly into a few known task types makes rule-based routing a week of work and entirely respectable. Own the rules, own the fallbacks, revisit them quarterly. For many products this is the right answer.

Assemble it. Open-source gateways and proxy layers give multi-provider routing primitives — model fallbacks, load balancing, per-model keys — with the difficulty logic supplied on top. More moving parts than building rules, fewer than building a learned router from scratch.

Or rent the decision. An adaptive endpoint like Vidman AI Adaptive takes the whole problem: one model id, the platform picks the serving model per request, billing at the rate of whatever actually answered, failing over internally instead of returning errors. The trade is transparency — trusting the platform’s judgment per request — against the engineering not done. Whether that trade makes sense depends on how mixed the workload is and how much team time should go to model operations. The model library shows the catalog such an endpoint draws from.

Routing and fine-tuning answer different questions; the interesting architecture uses both. Fine-tuning changes what a model is; routing changes which model a request meets. A tuned model embodying the domain is exactly the kind of specialist deserving its own tier in a routing plan — not a reason to abandon the plan.

The trap is treating a fine-tune as a reason to pin everything to it. The tuned model is presumably excellent at the task class it was trained for; the requests outside that class — the guardrail check, the off-topic question, the formatting job — gain nothing from it and still bill at whatever serving tier it sits on. Specialists are for special work.

The composition that works well: tune where there is data and a durable task, route where the workload is mixed, and revisit the boundary whenever the catalog or the traffic shifts. Neither mechanism replaces the other.

A product doing exactly one thing has nothing to route. A single-task pipeline gains nothing from per-request model selection, and a pinned model with a well-tuned prompt beats any general mechanism on both cost and predictability.

A frozen model needed for regulatory or evaluation reasons — an audit trail that must name the exact weights producing every output — makes adaptive routing genuinely wrong. Pin, log, and let nobody make it clever.

And a latency budget tight enough that the routing decision itself matters: measure before adopting anything. Routing adds a decision step — small, but small is not zero — and its cost in the path should be known before the architecture changes.

The platform’s endpoint is one model id away: bios-adaptive on the same OpenAI-compatible API everything else uses (the id renames to vidman-adaptive when the platform’s naming catches up). Swap the id, watch the response metadata carry which model actually answered, and measure — the cost per completed task, the quality bar, the latency — before deciding anything.

The comparison that matters is against the current pinned model on real traffic, which is exactly the shadow-eval pattern from the migration post. A week of shadow traffic answers the routing question with data instead of opinion.

And the decision is reversible: the model id is a string, not a marriage. Try it, measure it, and let the numbers decide — the same discipline this blog applies to every purchase.