Break-even duration
7mo
from acquisition
Open reference · v0.8.1
Price the decision, expose the uncertainty.
A local inference system only wins under a workload, a utilization rate, and a performance profile. Explore the crossover point against hosted API spend — with every assumption in view.
01 · model a scenario
Calculator
Start from a curated scenario or build your own. Results recompute locally, are shareable as a URL, and can be saved in this browser.
Assumptions
Live result
Recomputed
locally · no data leaves this page
Net position at horizon
$1,667,534
after $420,000 all-in acquisition
Effective cost / M tokens
$1.79
API equivalent $6.53
Request throughput
535.7 / hr
decode-bound · 10,713,600 input tokens / hr
Break-even utilization
16%
you assumed 60%
Throughput evidence
medium
taken at face value
estimatedAt 60% utilization DGX B300 costs less than Anthropic Claude Opus 5 after 7mo, and is $1,667,534 ahead by the end of the 3-year horizon — if your deployment reaches the modeled throughput, which is estimated evidence at medium confidence.
Read this result with 3 caveats
This request's 20,000 input and 5,000 output tokens are not in the ratio the "base" profile's rates were split at, 8,192 to 1,024.
Request rate is the lower of decode ÷ output tokens and prefill ÷ input tokens, and the two limits meet only at the ratio the rates were split at. Here decode runs out first: 1,240 tokens/s ÷ 5,000 output tokens admits 893 requests an hour at full utilization, where prefill's 9,920 ÷ 20,000 admits 1,786, so 50% of the prefill capacity goes unused. A deployment built for this request would move GPU time from prefill to decode, and at the runs' cost per token it would serve between those two figures; the result is costed at the lower one, so on ratio alone it understates. At 25,000 tokens the request is also longer than the runs' 9,216, and every token of a longer request costs more in both phases, which the calculator does not model and which overstates.
Rates measured at this request's shape can replace the profile's under Throughput evidence.
Aggregate memory fits, but the topology is below official guidance.
Moonshot AI recommends at least 64 accelerators for Kimi K3; this scenario uses 8 × B300. A capacity check is not a statement that the deployment works or performs.
This scenario credits Kimi K3 with more value than Artificial Analysis Intelligence Index v4.3 supports.
Artificial Analysis Intelligence Index v4.3 puts Kimi K3 at 44 and Claude Opus 5 at 51 on the same scale, a ratio of 0.86. Quality parity is set to 1.00×, so the comparison treats one unit of Kimi K3 output as worth more than that scale suggests. Cost per token is only meaningful between models that can do the same job: a composite index is a blunt instrument, but a gap is worth pricing deliberately rather than by default.
Set quality parity to what the two models are worth for your workload — the index is a starting point, not an answer.
Cumulative cost curve
- Self-hosted
- API
showing the whole horizon · drag across the chart to zoom showing month 0 to 36 of 36
At month 36, self-hosted cost is $630,645 and API equivalent is $2,298,180. Break-even lands after 7mo, 20% of the way into the modeled horizon.
What the same workload costs on every current tier
What would have to be true
| Utilization must stay above | 16% | assumed 60% |
|---|---|---|
| Achievable throughput must stay above | 27% | assumed 100% |
| Quality parity must stay above | 0.27× | assumed 1.00× |
What moves the answer
| Driver | Break-even | In cash | Self-host $/M | API value $/M |
|---|---|---|---|---|
| Utilization assumed 60% | ||||
| 25% | 1y 8mo | $4.30 | $6.53 | |
| 50% | 9mo | $2.15 | $6.53 | |
| 75% | 6mo | $1.43 | $6.53 | |
| 100% | 4mo | $1.08 | $6.53 | |
| Achievable throughput assumed 100% | ||||
| 50% | 1y 4mo | $3.58 | $6.53 | |
| 70% | 11mo | $2.56 | $6.53 | |
| 85% | 9mo | $2.11 | $6.53 | |
| 100% | 7mo | $1.79 | $6.53 | |
| Quality parity assumed 1.00× | ||||
| 0.75× | 10mo | 7mo | $1.79 | $4.90 |
| 1.00× | 7mo | $1.79 | $6.53 | |
| 1.25× | 6mo | 7mo | $1.79 | $8.16 |
Where the memory goes
| Weights | 1,561 GB |
|---|---|
| Runtime overhead | 312 GB |
| KV cache | 55 GB |
| Free | 376 GB |
| Sequences that fit | 377 |
| Context that fits | 308,489 |
Sequences and context that fit are counted with one recurrent state per sequence, where vLLM keeps two or three per request by default and SGLang admits one request per five state slots, so on either default they are the most that could fit.
Is the cited rate reachable here
| Requests in flight the rate needs | 2 |
|---|---|
| The fewest it could need | 2 |
| Requests this system can hold | 377 |
| Cache reads alone | 1.4 TB/s |
| Memory bandwidth available | 64 TB/s |
A decode step reads the weights its requests need, then some of every in-flight request's cache. The weight read is shared by every request in the step, so a high token rate comes from spreading it across many requests at once, or from emitting several tokens a step. The first two rows are that arithmetic under two sets of assumptions about how much a step reads; neither describes a particular serving stack.
This system holds the requests the estimate needs, so the rate and this request size are consistent. Requests this system can hold are counted with one recurrent state each, where vLLM and SGLang both keep more by default, so the system may hold fewer. See how this is derived.
Hourly economics
| API-equivalent value / hour | $87.45 |
|---|---|
| Raw API spend / hour | $87.45 |
| Electricity / hour | $2.26 |
| Support / hour | $5.75 |
| Net savings / hour | $79.43 |
02 · inspect the inputs
Catalog
Every value is versioned, dated, and tagged with the kind of evidence behind it.
21hardware systems
34open models
68API price tiers
37benchmark observations
194sources
Browse the full catalog with specifications, prices, evidence badges and source links — researched to 2026-09-13.
Open the catalog →03 · read the frame
Methodology
Reproducible formulas, explicit exclusions, and a clear quality caveat.
What is calculated
Per-request input, cache-hit, cache-write, and output tokens are priced separately. Throughput is constrained by both decode and prefill; facility electricity applies IT power × PUE. Support is an annual ratio of all-in acquisition cost. Break-even exists only when the hourly API-equivalent value, the API bill for the same tokens times quality parity, exceeds self-hosted variable cost.
Memory, including KV cache
Weights fitting is necessary, not sufficient. The memory check covers weights plus a runtime overhead ratio and the KV cache, which grows with context length and with how many requests are in flight — so a model can pass the capacity check and still be unable to hold more than a couple of concurrent sequences. KV cost is summed over layer groups rather than assumed uniform, because current flagships mix attention types: a sliding-window layer's cache stops growing at the window and a linear-attention layer's state never grows at all. One consequence is worth stating plainly — a compressed latent cache is a single key/value head, so ordinary tensor parallelism gives every accelerator a full copy of it. The weights divide across the system and the cache does not. Models with no published cache profile say so rather than guessing.
Evidence, ranked
Values carry the kind of evidence behind them, strongest first. Measured means a run happened and its result was recorded: MLPerf's closed division, a third-party harness, or logged traffic. It does not mean independent — that is a property of the source, and the Evidence page names the publisher beside every claim and grades each source's independence in the registry above them. One of the sources graded this way is this repository owner's own recorded agent traffic. Vendor-reported is a real measurement the vendor published for its own product — usually on a stack it tuned, so it is an upper bound rather than a promise. Official claim is a published specification or list price: a design fact, not a run of a workload. Estimated is derived here from other published values, and assumption is a planning number this repository chose. Where better public evidence exists, it replaces a vendor's own figure.
Achievable throughput
Every cost per token here scales inversely with the throughput you actually reach, and published throughput figures are tuned figures. The common worry is that open-source serving stacks lag a vendor's own; the best same-hardware evidence does not support it — in MLPerf Inference v6.0 a tuned vLLM submission led its 8× B200 category outright, and retuning that same engine moved its own result by about a fifth. What the data does show is a gap between a tuned deployment and an untuned one. The achievable-throughput control models that gap explicitly, and is kept separate from utilization, which is only how busy the hardware is kept.
What the capability score is, and is not
Scores shown beside models are the Artificial Analysis Intelligence Index, cited with a link to the model's own page there. They are that organisation's measurement, not this repository's, and they carry real limits. A composite collapses nine dissimilar evaluations — agentic tool use, terminal coding, PhD-level science questions, long-context reasoning, hallucination rate — into one number, so two models can tie on the headline and differ sharply at the thing you actually need. The index is also a moving target: it has been revised roughly fourteen times since early 2024, and a revision can change which benchmarks count, how they are weighted, or even which model grades the open-ended answers, so figures from different versions are not strictly comparable. And a score belongs to one specific configuration — the same Claude Sonnet 4.6 checkpoint scores 37 or 48 depending only on whether adaptive reasoning is on — while a self-hosted deployment can differ in quantization, reasoning effort and context length from whatever was benchmarked.
Colour bands compare each score to the best in this catalog rather than to the index's nominal ceiling of 100, because today's frontier sits in the low sixties. The score is here for one reason: every headline on this page is a cost ratio, and a cost ratio between two models of very different capability is a category error wearing the clothes of a finding.
Quality-parity caveat
The quality-parity multiplier scales the value of successful local output against the API comparator. It does not claim the models are equivalent, and it never changes physical throughput.
Exclusions and semantics
Financing, taxes, staffing, downtime, resale, opportunity cost, and production support beyond
the configured ratio are excluded. Tokenizers also differ per provider and even per model
generation — Anthropic documents that its newest generation produces roughly 30% more tokens
for the same text than Sonnet 4.6 and earlier — so equal token counts across API tiers do not
describe equal text volumes; affected tiers carry a note. Superseded catalog entries stay
available so older shared scenarios keep resolving, but are hidden until you ask for them. The
research cutoff is the catalog's last-updated boundary; changing the URL scenario changes
assumptions, not the evidence. The repository's docs/architecture.md and docs/PROJECT.md record the full scope.
Independent work this relies on
Two outside efforts do work this repository depends on and has not reproduced. Artificial Analysis produces every capability score shown beside a model or a tier; each one links to the page it was read from, and what a composite index can and cannot tell you is set out above.
Sebastian Raschka publishes per-release architecture notes and an LLM architecture gallery that read new releases down to their attention geometry. Both uses of it here are worth naming. The first is corroboration: his read of GLM-5.3-Flash found the same 34 linear-attention layers against 11 sparse-attention layers this catalog derived from the release's own configuration file, from the same release and with no contact between the two.
The second is disagreement, which was more useful. His published cache-per-token figures match
this catalog exactly on Muse Glimmer 30B at 52 KiB and Qwen3.6 27B at 64 KiB, and differ on
Gemma 4 31B: 840 KiB against 880 KiB here. The entire 40 KiB gap is the ten global layers,
which his figure sizes at half what this catalog does. Two different readings of Gemma 4's
configuration land on that half — using the model-wide head dimension where the global layers
declare their own larger one, or treating its attention_k_eq_v flag as storing keys
and values in one tensor — and this catalog carried the second until 2026-08-21. Which of the two
is behind the published figure cannot be told from the figure, so the entry says so rather than
picking one. Both traps are now written out in the entry itself.
For the explanations behind what this page only models, his KV cache walkthrough and visual guide to attention variants cover the mechanisms the memory page here turns into numbers. These links open in a new tab.