What Serverless LLM Inference Actually Costs at Enterprise Volume

Hidden inside every inference bill there is a crossover. Below a certain sustained load, paying per token is the obviously correct choice; above it, a dedicated endpoint quietly becomes cheaper, and every day you do not notice is a day of overspend. Most writing on this subject explains what serverless inference is and stops before the part that matters. This piece is about the line itself: how to find it for your workload, why the naive calculation puts it in the wrong place, and what actually changes when you cross it.

We sell both shapes of capacity, so we have no interest in pretending one is universally better. What we see across teams is simpler: almost nobody has done the arithmetic, and the ones who have usually did it with the wrong inputs.

Serverless bills per token: published rates per million input and per million output tokens, multiplied by what is actually sent and received. Nothing more. Traffic stops, bill stops. No reserved capacity, no minimum commitment, no charge for a machine sitting idle at three in the morning.

The rate card is the whole story, which is what makes it easy to reason about. The model library lists every model with per-token rates and context window, and the same prices apply at ten requests a day or ten thousand. The economics are linear by design: cost tracks usage exactly — zero waste at the bottom, zero discount at the top.

That linearity is the feature at low and spiky volume. It is also, eventually, the problem.

A dedicated endpoint flips the shape: the GPU is billed by the hour, and in return it is entirely yours. The meter runs whether the GPU is answering requests or waiting for them, but the throughput ceiling is yours alone, latency stays steadier because nobody else shares the hardware, and the cost per token falls toward zero as utilisation climbs.

The deployments documentation covers both shapes as they exist on Vidman AI: serverless by default, dedicated clusters for sustained load. Most providers offer some version of the same split under different names.

The pricing consequence is what matters here. Serverless is a line through the origin — zero usage, zero cost. Dedicated is a flat horizontal line — the same cost at any usage. Where the two lines cross is the only number in this entire discussion, and it is different for every workload.

The whole formula is one sentence: dedicated wins when the tokens the GPU could push per hour, multiplied by the serverless per-token price, exceed the GPU’s hourly rate. In symbols — tokens-per-hour T, serverless price per token p, hourly cost h — the crossover is where T · p equals h. Below it, serverless. Above it, dedicated.

Three variables, and two of them sit on your rate card. The one that does not — the throughput a GPU actually sustains on your model, your context lengths, your output lengths — is the one everyone guesses, and guessing it is how the crossover lands in the wrong place. Throughput is not a spec-sheet number: it depends on prompt lengths, completion lengths, and how aggressively requests can be batched.

The honest way to find it: measure your own traffic pattern for a week, then run the arithmetic with real rates rather than marketing ones. The pricing calculator exists for exactly this — pick a model, dial in volume, and the same work is priced across providers, so the serverless side of the line is grounded in published numbers rather than a pitch deck.

Mistake one is averaging. Teams take the daily token count, divide by the hours in a day, and compare that average rate against a GPU. But traffic is never average: a support product peaks through business hours and idles overnight; a batch pipeline sits dead quiet all day and saturates everything at midnight. A dedicated GPU sized for the peak wastes money all night; one sized for the average falls over at noon. The average is precisely the wrong input.

The second mistake is forgetting the shape of the bill you already have. Retries, failed generations you still paid for, oversized prompts — all inflate the token count before it ever reaches a pricing formula. Usage padded with waste produces a crossover for a bloated workload. Clean the workload first; the usual suspects are catalogued in the invoice teardown.

The third mistake is treating latency as free. Shared capacity has scheduling variance, and cold starts exist on every serverless platform whether or not the marketing page mentions them. A hard latency floor pins part of your traffic to dedicated capacity regardless of what the token math says — and the honest calculation prices that traffic separately.

At small volume the decision is a rounding error and should not consume a week. At enterprise volume it compounds daily, and three things change.

First, the blend matters more than the total. Enterprise workloads are rarely one workload: interactive chat beside document pipelines beside classification jobs. Each has its own crossover, and the right answer is usually a split — bursty interactive traffic on serverless, sustained batch windows on dedicated — rather than one shape for everything.

Second, commitments enter the picture. Sustained volume is the leverage behind reserved or committed capacity, and a team that has never measured its baseline cannot negotiate one. Measurement is not just for the comparison; it is the prerequisite for any pricing conversation.

Third, the decision stops being static. Model prices move, traffic grows, and the crossover moves with both. A calculation done once and filed away will be wrong by the next quarter.

Sometimes — and the honest answer matters more than the reassuring one. Shared capacity means shared scheduling: when many tenants burst at once, individual requests wait their turn, and the tail of the latency distribution stretches. Platforms that batch requests for efficiency — most of them, since batching is what makes per-token economics work — add a small, deliberate delay to let batches form. You are trading a little time for a lot of utilisation, whether you know it or not.

Cold starts are the other tax. A model that has not served traffic recently must be loaded before it can answer, and the load time lands on whoever asks first. Providers differ widely in how aggressively they keep models warm, but no shared platform escapes the physics.

Whether any of this matters depends entirely on the product. An overnight document pipeline will never notice scheduling variance; a chat interface with a hard responsiveness floor notices immediately. None of this is a reason to avoid serverless — it is a reason to price the split honestly, which is exactly what the crossover calculation is for.

Pull a week of real usage from the logs: requests per hour across the day, average prompt and completion lengths, and the split between interactive and batch work. Cost the serverless side from published per-token rates — the pricing calculator does this directly from your volume, and the model library has the raw figures — then get a real hourly quote for a dedicated endpoint sized to your peak, not your average.

If the dedicated quote only beats serverless when the GPU runs hot around the clock, you do not have a dedicated workload yet; you have a spiky one, and spiky workloads are what per-token billing exists for. If the baseline load alone clears the crossover before the peaks are even counted, the dedicated endpoint pays for itself and the peaks are gravy.

The answer is rarely all one thing. Price the split, not just the extremes.

If GPU capacity is already owned under a long-term commitment, the marginal cost of one more token on it is close to zero, and no per-token platform can compete with hardware already paid for. Use us for the burst above the baseline instead of pretending the baseline does not exist.

If the traffic is a single, sustained, predictable stream that never sleeps, a dedicated endpoint — ours or anyone else’s — beats per-token pricing on pure arithmetic. We will happily sell you that instead; the deployments docs cover the dedicated path.

And if the traffic has not been measured yet, the wrong choice is deciding at all. Measure first. The crossover is a measurement, not a preference.