API vs self-hosting break-even

Start with the two cost curves

A hosted API usually has a low fixed cost and a cost that grows with tokens. Self-hosting usually has a higher fixed cost and a lower marginal cost when hardware is well used. That creates the classic break-even picture.

The simple version looks like this:

hosted API monthly cost =
  monthly input tokens * input price
  + monthly output tokens * output price

self-hosted monthly cost =
  hardware + cloud infrastructure + engineering + monitoring + risk buffer

The danger is treating this as pure arithmetic. If the self-hosted model is worse, slower, less reliable, or harder to operate, a lower token price may still lose.

Sample numbers

Suppose 2 million requests per month, 1,200 input tokens and 400 output tokens each, API prices of $3 / 1M input and $15 / 1M output, and self-host fixed cost of $6,000/month with $0.40 / 1M tokens variable:

• Hosted API: input $7,200 + output $12,000 = $19,200 / month
• Self-hosted: $6,000 fixed + (3.2B tokens × $0.40 / 1M) = $6,000 + $1,280 = $7,280 / month

At this volume the API path is more expensive on paper, but the crossover depends on your real fixed cost (engineering, replicas, idle headroom) and whether the open model passes evals. Break-even request volume for these prices: about 670k requests/month (where the two lines cross).

Here is the arithmetic part on its own. Set your traffic and prices and watch where the two curves cross. The hosted line starts at zero and climbs with every token. The self-hosted line starts high (the fixed cost) and climbs slowly. Below the crossover the API is cheaper, above it self-hosting is, assuming the quality and operational caveats in the rest of this lesson hold.

Cached input pricing on APIs

Hosted providers increasingly discount repeated input prefixes. That changes break-even math when your system prompt, tool definitions, or RAG context are stable across requests. Structure prompts as static prefix + volatile suffix to maximize cache hits (see Context budgeting).

Illustrative 2025-style tiers (check current pages before budgeting):

If 80% of your 1,200 input tokens hit cache at half price, effective input cost drops materially and the API line in the chart flattens. Re-run break-even with your measured cache-hit rate, not list input price.

Quality can erase the savings

Hosted models and open models may not be interchangeable. If a cheaper self-hosted model needs longer prompts, more retries, a larger fallback rate, or more human review, the real cost rises. If it answers poorly, the cost model is a distraction.

Run task evals before price evals. Compare accuracy, refusal behavior, schema validity, latency, token count, and escalation rate. Then price the passing systems.

Volume helps only if traffic is usable

High monthly token volume does not guarantee self-hosting wins. The traffic must fit the hardware well. Smooth, predictable traffic can keep GPUs busy. Spiky traffic either creates queues or forces you to pay for idle headroom.

Look at hourly and daily shape, not just monthly totals. A workload with huge weekday bursts and quiet nights can be worse for self-hosting than a smaller workload that runs steadily.

Know the non-cost reasons

Sometimes self-hosting is not mainly about price. You may need data residency, custom models, offline operation, strict logging control, lower variance in latency, or a deployment boundary that a hosted API cannot provide.

Those reasons are valid. Just write them down separately from the cost case. Mixing them together makes the decision hard to audit later.

Use a staged decision

The strongest path is usually staged. Start with a hosted API to prove the product. Add tracing and token accounting early. When a feature gets enough stable traffic, run an open-model eval. If quality passes, shadow self-hosted inference. If shadow numbers pass, route a small slice. Expand only after latency, quality, and cost hold up under real traffic.

This avoids the worst version of self-hosting: buying complexity before the product has a stable workload.

Quick check